494 Commits

Author SHA1 Message Date
Pierre Mdawar
c46d8093bc Fix updating a non unique consumer on workqueue stream not returning an error 2023-10-12 12:18:24 +03:00
Waldemar Quevedo
4e414f1f05 Skip processing consumer assignments after JS has shutdown (#4625)
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-10-04 13:17:22 -07:00
Derek Collison
00839280fb [IMPROVED] Reduce contention for high connections in a JetStream enabled account with high API usage. (#4613)
Several strategies are used which are listed below.

1. Checking a RaftNode to see if it is the leader now uses atomics.
2. Checking if we are the JetStream meta leader from the server now uses
an atomic.
3. Accessing the JetStream context no longer requires a server lock,
uses atomic.Pointer.
4. Filestore syncBlocks would hold msgBlock locks during sync, now does
not.

Signed-off-by: Derek Collison <derek@nats.io>
2023-10-01 08:17:15 -07:00
Derek Collison
dba03dbc2f Optimizations to reduce contention for high connections in a JetStream enabled account with high API usage.
Several strategies which are listed below.

1. Checking a RaftNode to see if it is the leader now uses atomics.
2. Checking if we are the JetStream meta leader from the server now uses an atomic.
3. Accessing the JetStream context no longer requires a server lock, uses atomic.Pointer.
4. Filestore syncBlocks would hold msgBlock locks during sync, now does not.

Signed-off-by: Derek Collison <derek@nats.io>
2023-09-30 14:52:15 -07:00
Tomasz Pietrek
1f4b986125 Fix consumer info if consumer was closed
Co-authored-by: Derek Collison <derek@nats.io>
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
Signed-off-by: Derek Collison <derek@nats.io>
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-09-29 21:40:55 +02:00
Neil Twigg
212d92ca7e Add more pprof labels to consumers, sources, mirrors
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-29 19:12:47 +01:00
Derek Collison
121adb4719 Bump start interval for cleanup check
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-24 15:44:15 -07:00
Tomasz Pietrek
ac2669a022 Fix consumer limits
Test didn't catch this error, as by default, old JS client
sets ack policy to none. If policy is different, it will fail
to create consumer with defaults.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-09-20 17:37:05 +02:00
Derek Collison
71b8a33456 Update to not pop directly, just bail when we detect leadership change
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-18 13:27:27 -07:00
Derek Collison
850c89e175 When scaling a consumer down make sure to pop the loopAndForwardProposals go routine
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-18 12:26:25 -07:00
Waldemar Quevedo
0e63608716 Fix for data race when changing retention policy
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2023-09-17 18:15:55 -07:00
Derek Collison
f2e7ed91cb Fix for panic in consumer, needed to recheck if consumer was closed
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-15 08:40:21 -07:00
Derek Collison
22f40eafa0 Add in jitter in case there are many that all try to cleanuo at the same time
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-14 11:24:32 -07:00
Derek Collison
392f25b6da Fix for data race and adjustment to do a backoff on making sure consumers are cleaned up.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-14 11:21:11 -07:00
Derek Collison
ad380d48f2 Merge branch 'main' into dev
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-01 11:19:33 -07:00
Derek Collison
c679f9d7f6 Added in detail info when failing to load in a message for a consumer.
E.g. `Unexpected partial cache error looking up message for consumer '$G > TEST > dlc'`

Signed-off-by: Derek Collison <derek@nats.io>
2023-09-01 09:06:29 -07:00
Derek Collison
3a39786972 When we fail to deliver a message for a consumer, either through didNotDeliver() or LoadMsg() failure re-adjust delivered count and waitingRequest accounting.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-01 08:48:28 -07:00
Neil Twigg
487f58f16e Consumers inherit limits for max_ack_pending and inactive_threshold from stream
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-01 10:54:11 +01:00
Derek Collison
346c22788e Merge branch 'main' into dev 2023-08-24 16:20:46 -07:00
Derek Collison
48bf7ba151 When a consumer reached a max delivered condition, we did not properly synchronize the state such that on a restore or leader switch the ack pending could jump and be higher than max ack pending and block the consumer.
This propagates a delivered update and we updated the store state engine to do the right thing when the condition is reached.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-24 16:00:27 -07:00
Derek Collison
bcf5da04e3 Merge branch 'main' into dev 2023-08-22 06:50:36 -07:00
Derek Collison
e5d208bf33 When moving streams, we could check too soon and be in a gap where the replica peer has not registered a catchup request.
This would cause us to think the replica was caughtup incorrectly and drop our leadership, which would cancel any cacthup requests.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 20:07:48 -07:00
Derek Collison
fb8525b713 Merge branch 'main' into dev
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 15:55:00 -07:00
Derek Collison
43314fd439 Fix for a bug that would allow old leaders of pull based durables to delete a consumer from an inactivity threshold.
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 14:53:09 -07:00
Neil Twigg
c437157c1f Recover in consumer assignment when asset already existed
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-17 23:22:10 +01:00
Neil Twigg
c0636d117f Tweak consumer replica scaling, add unit test for orphaned consumer subjects
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-17 15:27:29 +01:00
Tomasz Pietrek
d105e68c96 Add consumer api action for create and update
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-08-07 08:28:21 +02:00
Derek Collison
9d9d760af2 Merge branch 'main' into dev 2023-07-21 11:42:02 -07:00
Derek Collison
9280a552b8 Don't error to server logs if message deleted
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 14:07:35 -07:00
Derek Collison
a1f03513d8 Merge branch 'main' into dev 2023-06-09 09:29:13 -07:00
Derek Collison
9eeffbcf56 Fix performance issues with checkAckFloor.
Bail early if new consumer, meaning stream sequence floor is 0.
Decide which linear space to scan.
Do no work if no pending and we just need to adjust which we do at the end.

Also realized some tests were named wrong and were not being run, or were in wrong file.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-08 18:45:03 -07:00
Derek Collison
f342f6a758 Merge branch 'main' into dev 2023-06-05 14:13:18 -07:00
Derek Collison
4ac45ff6f3 When consumers were R1 and the same name was reused, server restarts could try to cleanup old ones and effect the new ones.
These changes allow consumer name reuse more effectively during server restarts.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-05 12:48:18 -07:00
Derek Collison
8278a7a0b2 Merge branch 'main' into dev 2023-06-02 12:44:02 -07:00
Derek Collison
27bbfb7a85 Only check ack floor if we are interest policy based.
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-02 11:04:00 -07:00
R.I.Pienaar
c24547eb4e Record the stream and consumer info timestamps (#4133)
This records the server time when info for streams and consumers are
created so that tools such as the nats cli can calculate time deltas for
last ack, last delivered and so forth in the context of the server
clock.

This will help aleviate problems with client devices experiencing clock
jitter that can show up in user interfaces as negative seconds since
last ack etc
2023-06-02 08:53:28 +03:00
Tomasz Pietrek
261f39bb7d Make sure that o.subjf is nil or checked if empty
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-06-01 08:29:09 +02:00
Tomasz Pietrek
a463dfe5c4 Improve consumer error handling
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-06-01 08:29:03 +02:00
Tomasz Pietrek
9a792482e9 Improve consumer with multiple filters
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-06-01 08:28:19 +02:00
Derek Collison
4175e4ee9c Merge branch 'main' into dev 2023-05-06 09:55:34 -07:00
Derek Collison
80db7a22ab Optimizations for large single hub account leafnode fleets.
Added a leafnode lock to allow better traversal without copying of large leafnodes in a single hub account.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-05 13:14:49 -07:00
R.I.Pienaar
fb1d86d506 Record the stream and consumer info timestamps
This records the server time when info for streams and
consumers are created so that tools such as the nats cli
can calculate time deltas for last ack, last delivered and
so forth in the context of the server clock.

This will help aleviate problems with client devices experiencing
clock jitter that can show up in user interfaces as negative
seconds since last ack etc

Signed-off-by: R.I.Pienaar <rip@devco.net>
2023-05-05 18:53:09 +02:00
Tomasz Pietrek
7c1c4ea5fb Optimize consumer messages sequences for multiple subjects
If consumer with multiple subjects encountered a sequnece
of messages from the same subject, it tried to load messages
from other subjects in some cases.
This checks for that scenario and optimizes it by early returning.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-05-04 16:02:19 +02:00
Derek Collison
0321eb6484 Merge branch 'main' into dev 2023-04-29 19:52:57 -07:00
Derek Collison
db972048ce Detect when we are shutting down or if a consumer is already closed when removing a stream.
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-29 11:18:10 -07:00
Derek Collison
fac5658966 If we fail to create a consumer, make sure to clean up any raft nodes in meta layer and to shutdown the consumer if created but we encountered an error.
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-29 08:15:33 -07:00
Derek Collison
d573b78aee Merge branch 'main' into dev 2023-04-26 18:42:31 -07:00
Derek Collison
7f06d6f5a7 When Jsz() was asked for consumer details, would report incorrect data if not a consumer leader.
This is due to the way state is maintained for leaders vs followers for consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-26 15:03:15 -07:00
Derek Collison
1de9a1cf3b Merge branch 'main' into dev 2023-04-21 14:09:35 -07:00
Derek Collison
da9a17fd68 Spelling
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-21 12:40:19 -07:00