Commit Graph

97 Commits

Author SHA1 Message Date
Derek Collison
c9f615933a Honor JetStream server settings of 0.
Signed-off-by: Derek Collison <derek@nats.io>
2021-10-12 14:16:47 -07:00
Derek Collison
cbbab295ec Merge pull request #2616 from nats-io/fix_2611
[FIXED] #2611
2021-10-12 08:46:42 -07:00
Derek Collison
8951bc54f4 Fix for stream purge where purge would remove all pending messages for a filtered consumer.
Signed-off-by: Derek Collison <derek@nats.io>
2021-10-12 07:26:13 -07:00
Derek Collison
c789ad9211 Simplified a bit
Signed-off-by: Derek Collison <derek@nats.io>
2021-10-12 06:12:56 -07:00
Ivan Kozlovic
3b9c358a81 Test that reproduces the issue reported in #2611
Issue seem to be from the fact that the filter subject is == to
the stream's subjects list and so setInitialPendingAndStart()
considers that the consumer is not filtered.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-10-11 13:32:04 -06:00
Derek Collison
58e5c7c681 Allow consumers to request only headers to be delivered
Signed-off-by: Derek Collison <derek@nats.io>
2021-10-05 18:54:52 -07:00
Ivan Kozlovic
d34b8fdd7d [FIXED] JetStream: data race on shutdown
Replaced use of eventsEnabled() with EventsEnabled() that will
check under server lock. Also found another reference when
creating templates.

Resolves #2588

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-09-30 11:17:12 -06:00
Ivan Kozlovic
a3be36fcd4 Moving 2 new tests to jetstream test file
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-09-22 17:02:29 -06:00
Derek Collison
9534372113 Fix for #2551
When a mirror would be processed before the origin stream we would not recover the consumers due to failure on looking up source's subjects.
This change processes all streams first then does all consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2021-09-21 08:53:12 -07:00
R.I.Pienaar
3be0b23a3e Merge pull request #2546 from ripienaar/negative_dupe_window_protection
protect against negative dupe window via negative max age
2021-09-20 18:09:19 +02:00
R.I.Pienaar
34b5a11e20 protect against negative dupe window via negative max age
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-09-20 17:34:48 +02:00
Derek Collison
cecb6246f2 Test tweaks
Signed-off-by: Derek Collison <derek@nats.io>
2021-09-19 13:28:37 -07:00
Derek Collison
40a4d40337 Make large batch requests expire more efficiently.
Signed-off-by: Derek Collison <derek@nats.io>
2021-09-16 15:00:18 -07:00
Ivan Kozlovic
108d4060ff [CHANGED] JetStream: flow control requires heartbeats
The server will now reject the creation of a push consumer with
flow control if no heartbeat is set.

Resolves #2520

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-09-15 09:50:15 -06:00
Derek Collison
e18a278196 Added ability to check if no message exists as a test to store the message.
Signed-off-by: Derek Collison <derek@nats.io>
2021-09-09 11:41:23 -07:00
Derek Collison
dadc3b9fae Fixed a bug when an interest retention stream with noack consumers is in clustered mode.
We were not properly propagating the ack state and proper cleanup of the stream messages.

Signed-off-by: Derek Collison <derek@nats.io>
2021-09-08 15:02:09 -07:00
Ivan Kozlovic
ba36aa452b Fix some timing and bump server version
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-09-01 18:03:14 -06:00
Ivan Kozlovic
44c57a5702 [FIXED] Pull requests: don't send 408 when request expires
When expiring requests, the server would send 408 if interest was
still present, which can happen for pull subscribe implementations
that maintain interest for the duration of the pull subscription.

Let's keep the 408 for when a request is "force expired", that
is, a request was removed from the queue because it queue was
full but interest is still found.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-09-01 16:57:13 -06:00
Derek Collison
752fd295a5 Consumer num pending fixes for multiple matches and merging.
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-24 07:52:29 -07:00
Derek Collison
70d28bd221 Use append in case pubAck is larger than stack []byte
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-20 15:53:37 -07:00
Derek Collison
6871d1240b When we expired all messages on a restart we did not properly setup lmb.
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-17 13:45:50 -07:00
Matthias Hanel
2ec87a56ba fixed unit tests where SequencePair comparisons fail due to time last_active
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-08-16 21:05:12 -04:00
Derek Collison
b095870c2a Fixed a leak of internal clients for JetStream consumers.
We were not properly unregistering from the account on cleanup.

Signed-off-by: Derek Collison <derek@nats.io>
2021-08-14 15:09:42 -07:00
Derek Collison
b6c0412d2a More fixes for some flapping tests
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-14 13:35:04 -07:00
Derek Collison
969cf60def Add in push bound status for consumer info.
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-14 08:35:40 -07:00
Derek Collison
c4a78d91b7 Remove PushActive
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-13 15:11:31 -07:00
Derek Collison
5ec5020a8b Add in DeliverGroup to consumer as optional queue group.
Changed sublist register for notifications to be queue bound.

Signed-off-by: Derek Collison <derek@nats.io>
2021-08-13 15:07:56 -07:00
Derek Collison
d252f0f700 Make sure PushActive is nil for pull consumers
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-12 10:15:55 -07:00
Derek Collison
c781256dda Add in deliver status for consumer info
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-11 15:41:33 -07:00
Derek Collison
da3c89efda Add domain to PubAck1
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-11 13:25:24 -07:00
Derek Collison
7dd399e355 Merge branch 'main' into max-per-default
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-10 07:33:58 -07:00
Derek Collison
cb6b397573 Fix for #2423
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-10 06:46:48 -07:00
Derek Collison
633763a202 Fix for #2420
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-10 06:36:08 -07:00
Derek Collison
bf966d84f3 Number of initial pending was off by one when DeliverLastPerSubject was utilized on a consumer.
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-04 14:20:52 -07:00
Derek Collison
154bc40718 Fix for reentrant read lock on a stream that once anyone else wanted the write lock would deadlock.
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-03 15:46:40 -07:00
Derek Collison
067ef19ce9 Fix for #2392. We were not resetting the pending timer properly during stream purge.
No we just let it fire as normal and it will be canceled or not as needed.

Signed-off-by: Derek Collison <derek@nats.io>
2021-08-01 17:28:59 -07:00
Derek Collison
170b0570d4 Off by one bug (surprise) that would skip first new message on deliver last per subject.
Signed-off-by: Derek Collison <derek@nats.io>
2021-07-30 15:19:01 -07:00
Derek Collison
959a0da1f0 Merge branch 'main' into deliver-last-per-subject
Signed-off-by: Derek Collison <derek@nats.io>
2021-07-29 05:42:45 -07:00
Derek Collison
9b0158daf9 Allow delivery policy of DeliverLastPerSubject, which is helpful for scoped watchers for K/V.
Signed-off-by: Derek Collison <derek@nats.io>
2021-07-28 12:49:02 -07:00
Derek Collison
98970ac03d Add descriptions to JetStream streams and consumers. 2021-07-27 14:05:25 -07:00
Derek Collison
225df04662 Fix for a consumer's num pending being stuck at 1.
We were trying to protect the sgap uint64 from wrapping, but in some cases the consumers is eager and can get a message before we sgap++.
Instead of slowing things down and sycnhronizing ++ then --, we allow it to wrap temporarily and have and adjustedPending() func that will set to zero for reporting.

Signed-off-by: Derek Collison <derek@nats.io>
2021-07-10 12:37:08 -07:00
Ivan Kozlovic
d5f5a74438 Fixed some tests
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-07-06 12:20:03 -06:00
Derek Collison
57cfba0964 Make requirement for interest on creation of ephemerals soft vs hard.
Will start the cleanup timer if no interest but allow creation.

Signed-off-by: Derek Collison <derek@nats.io>
2021-07-06 10:48:07 -07:00
Derek Collison
9b73fae5bd We should not sit in place here, no measurable effect on latency in tests
Signed-off-by: Derek Collison <derek@nats.io>
2021-07-01 14:29:09 -07:00
Derek Collison
c2c146c9f2 Fix for #2329.
When we created a filestore we would figure out if we should track by subject based on stream config.
This would cause bad results when a stream was updated to multiple subjects or wildcards.
This change tightens when and what we track but turns it on all the time now.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-30 19:10:31 -07:00
Derek Collison
99fed910f0 Improvements to large numbers of JetStream R1 consumers per stream.
1. We were holding open FDs longer than we should for consumers causing issues with open FD limits. We now do not hold them open and cap updates a bit better.

2. When doing a stream delete, consumer delete was repeating alot of work that was not necessary, causing longer delays. This has been optimized a bit, still more improvements to be made.

3. We cover all JS under a single export, but that was also trapping GetNext for pull based consumers, and since this was a no-op (is handled at user account level) we were creating alot of garbage service import responses and reverse map entries that had to be garbage collected. We have a fix in to avoind this but still looking for a better one.

4. Still had some lingering references to all exports vs single JS export.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-29 05:45:55 -07:00
Derek Collison
bb84ef7d91 Added ability to match based on last expected sequence per subject.
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-28 10:57:50 -07:00
Derek Collison
545230bd9e FIXED Clustered streams can become broken with seq mismatch state on low level store failures.
We can account for high level failure properly but were not properly accounting for low level store failures.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-25 13:46:40 -07:00
Derek Collison
08197de9e0 FIXED max consumers was not enforced when set on stream
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-25 11:45:36 -07:00
Derek Collison
c0e47966ab Added in Stream get last message by subject.
This is to aid in K/V overlay for simple Get ops vs creating a full blown consumer.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-24 13:21:39 -07:00