Commit Graph

282 Commits

Author SHA1 Message Date
Derek Collison
fb8525b713 Merge branch 'main' into dev
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 15:55:00 -07:00
Derek Collison
0a86bf4a9a Should reset to false, not true when done
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 14:57:17 -07:00
Derek Collison
e018705a08 Fixed deadlock when checkAndSync was being called as part of storing message.
We violated the locking pattern, so we now make sure we do this in a separate Go routine and put checks to only run it once.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 12:12:36 -07:00
Tomasz Pietrek
d105e68c96 Add consumer api action for create and update
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-08-07 08:28:21 +02:00
Neil
b22cdf18fe Add support for re-encrypting streams with new key (#4296)
This adds a new `prev_key` field to the configuration file to allow
transitioning from one encryption key to another.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-27 10:10:08 +01:00
Derek Collison
9a8f846dbb Merge branch 'main' into dev 2023-07-26 22:22:34 -07:00
Neil Twigg
3df08c3f89 Add support for re-encrypting streams with new key
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-26 14:04:28 +01:00
Waldemar Quevedo
bbfeb2a887 Fix typo on internal function
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-07-22 20:40:26 -07:00
Tom Anderson
83a43838dc Adding option to disable jetstream ascii art
This option is not available in the config, and is only accessable to embeded servers where when using custom loggers can look pretty terrible
2023-06-21 08:04:34 -07:00
Derek Collison
cee68344b7 Merge branch 'main' into dev 2023-06-10 17:03:36 -07:00
Derek Collison
a5de25f213 Only enable JetStream account updates in clustered mode.
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-10 15:21:55 -07:00
Derek Collison
30d9dfd305 Merge branch 'main' into dev 2023-06-03 18:17:28 -07:00
Derek Collison
4c1b93d023 Make sure to put the keyfile back if we did not recover the stream.
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-03 11:21:58 -07:00
Derek Collison
d5ae96f54d When a server was killed on restart before an encrypted stream was recovered the keyfile was removed and could cause the stream to not be recoverable.
We only needed to delete the key file when converting ciphers and right before we add the stream itself.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-03 11:21:47 -07:00
Derek Collison
fbfa593b3d Merge branch 'main' into dev 2023-05-17 16:48:47 -07:00
Derek Collison
44a5875968 Avoimd deadlock with usage lock for an account during checkAndSyncUsage().
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 16:05:46 -07:00
Derek Collison
4220502541 Merge branch 'main' into dev 2023-05-15 15:44:38 -07:00
Derek Collison
832df1cdba Protect against out of bounds access on usage updates.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-15 14:38:26 -07:00
Derek Collison
68f6b59fc7 Merge branch 'main' into dev 2023-05-03 19:51:24 -07:00
Derek Collison
21239022bd Protect against usage drift for any unforseen reason and if detected correct.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-03 17:09:06 -07:00
Derek Collison
eb1eb3c49e Merge branch 'main' into dev 2023-05-01 16:29:35 -07:00
Derek Collison
f098c253aa Make sure we adjust accounting reservations when deleting a stream with any issues.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-01 15:54:37 -07:00
Derek Collison
0321eb6484 Merge branch 'main' into dev 2023-04-29 19:52:57 -07:00
Derek Collison
b27ce6de80 Add in a few more places to check on jetstream shutting down.
Add in a helper method.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-29 11:27:18 -07:00
Derek Collison
db972048ce Detect when we are shutting down or if a consumer is already closed when removing a stream.
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-29 11:18:10 -07:00
Derek Collison
4ebdb69daf Merge branch 'main' into dev 2023-04-26 11:34:37 -07:00
cui fliter
f1f5a59e9b fix some comments
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-04-25 11:28:59 +08:00
Ivan Kozlovic
105237cba8 [ADDED] Multiple routes and ability to have per-account routes
New configuration fields:
```
cluster {
   ...
   pool_size: 5
   accounts: ["A", "B"]
}
```

The configuration `pool_size` in the example above means that this
server will create 5 routes to a remote server, assuming that that
server has the same `pool_size` setting.

Accounts (which are not part of the `accounts[]` configuration)
are assigned a specific route in this pool, and this will be the
same route on all servers in the cluster.

Accounts that are defined in the `accounts` field will each have
a dedicated route connection. This will allow suppression of the
account name in some of the route protocols, reducing bytes transmitted
which may increase performance.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-04-03 09:32:25 -06:00
Derek Collison
b9e7b58f5c Merge branch 'main' into dev 2023-04-02 18:58:54 -07:00
Derek Collison
e6447c982a Protect against concurrent creation of streams and consumers.
Also make sure we have exited monotoring routines when doing resets for both streams and consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 14:29:52 -07:00
Derek Collison
7fe05aaea2 Merge branch 'main' into dev 2023-03-23 17:28:44 -07:00
Neil Twigg
8d5519356e Shut down RAFT groups when disabling JetStream
Signed-off-by: Neil Twigg <neil@nats.io>
2023-03-23 16:54:01 +00:00
Derek Collison
7bd7cda021 Merge branch 'main' into dev 2023-02-28 15:17:24 -08:00
Derek Collison
adbb50fc21 Fixed dios capacity to 4 due to testing under heavy load.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-28 11:45:31 -08:00
Derek Collison
bee149b458 Only need server's rlock here.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-28 11:17:29 -08:00
Derek Collison
7ce23c46ce Merge branch 'main' into dev 2023-02-21 08:34:08 -08:00
Neil Twigg
68961ffedd Refactor ipQueue to use generics, reduce allocations 2023-02-21 14:50:09 +00:00
Derek Collison
469116deae Merge branch 'main' into dev 2023-01-26 09:39:32 -08:00
Derek Collison
f62d929018 Consumer must match replica of parent stream if interest based policy.
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-23 20:16:42 -08:00
Tomasz Pietrek
6e9d05988c Try dirty approach
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-12-28 13:06:17 +01:00
Derek Collison
62ce2e46ab Remove "" and replace with _EMPTY_
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-27 13:59:05 -08:00
Derek Collison
051b986b58 Merge branch 'main' into dev 2022-11-14 15:06:41 -08:00
Derek Collison
b6ef2c8910 Auto cleanup dangling messages from interest policy streams on server start.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-14 15:02:22 -08:00
Todd Beets
571a375f2e add unique_tag to JetStreamConfig for varz/jsz visibility 2022-11-11 13:36:13 -08:00
Ivan Kozlovic
abcfe2e7ac Add the pending msgs/bytes on 409 Shutdown
This is related to PR #3572 and PR #3576

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-27 13:59:21 -06:00
Derek Collison
15dc72db50 Removed migration of ephemerals, added proper signaling for pul consumers pending requests.
Signed-off-by: Derek Collison <derek@nats.io>
2022-10-25 14:35:20 -07:00
Ivan Kozlovic
bec51ed52b [FIXED] JetStream: User given named ephemeral lost after migration
If an ephemeral was given a name by the user, if the consumer leader
was then shutdown, the ephemeral would be migrated using a server
generated new name instead of keeping the user given name.

Also, in some cases the migration would not even occur. This was
likely due to the fact that RAFT node(s) were shutdown prior to
the ephemeral migration code was invoked.

Resolves #3550

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-14 15:20:45 -06:00
Derek Collison
91edd1a8d0 With snapshots both streams are present on restart so sources or mirrors that have a subject change from the origin would not recover.
We now suppress that if we know we are recovering an existing stream.

Signed-off-by: Derek Collison <derek@nats.io>
2022-09-29 09:17:15 -06:00
Derek Collison
52b5cd12bb Allow meta layer to snapshot on a clean shutdown.
Signed-off-by: Derek Collison <derek@nats.io>
2022-09-29 09:17:12 -06:00
Ivan Kozlovic
8c1c6951dc JetStream: R1 durables were incorrectly migrated on shutdown
This could happen for stream with R>1 but with a durable that
has an override of R=1.

Fixed a test to make sure assets have an elected leader.

Also fixed a gateway test that would cause a data race.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-09-07 19:49:16 -06:00