Commit Graph

111 Commits

Author SHA1 Message Date
Derek Collison
43eff407b8 Add in explicit subscription for import responses when bound to a leafnode.
When we want to track service import response interest across a leafnode we need to send sub and unsub for all response _R_ subjects versus using a wildcard.

Signed-off-by: Derek Collison <derek@nats.io>
2022-01-12 18:24:03 -08:00
Derek Collison
32c3c9ecfb Track interest properly across accounts for pull consumers
Signed-off-by: Derek Collison <derek@nats.io>
2022-01-12 12:16:53 -08:00
Derek Collison
279f31ecb5 Add in ability to have ephemeral pull based consumers
Signed-off-by: Derek Collison <derek@nats.io>
2022-01-10 20:42:39 -08:00
Derek Collison
e12c8cda92 Add in ability to limit aspects of a pull request, specifically batch size and expiration.
Signed-off-by: Derek Collison <derek@nats.io>
2022-01-10 17:29:04 -08:00
Derek Collison
5592d923c4 Updated pull consumers.
Cleaned up code, made more consistent, utilize loopAndGather.
Allow pull consumers to have AckAll as well as AckExplicit.

Signed-off-by: Derek Collison <derek@nats.io>
2022-01-10 16:59:01 -08:00
Derek Collison
c4198d603c Added test to show cross account interest for push consumers works
Signed-off-by: Derek Collison <derek@nats.io>
2021-12-21 19:30:35 -08:00
Matthias Hanel
3e8b66286d Js leaf deny (#2693)
Along a leaf node connection, unless the system account is shared AND the JetStream domain name is identical, the default JetStream traffic (without a domain set) will be denied.

As a consequence, all clients that wants to access a domain that is not the one in the server they are connected to, a domain name must be specified.
Affected from this change are setups where: a leaf node had no local JetStream OR the server the leaf node connected to had no local JetStream. 
One of the two accounts that are connected via a leaf node remote, must have no JetStream enabled.
The side that does not have JetStream enabled, will loose JetStream access and it's clients must set `nats.Domain` manually.

For workarounds on how to restore the old behavior, look at:
https://github.com/nats-io/nats-server/pull/2693#issuecomment-996212582

New config values added:
`default_js_domain` is a mapping from account to domain, settable when JetStream is not enabled in an account.
`extension_hint` are hints for non clustered server to start in clustered mode (and be usable to extend)
`js_domain` is a way to set the JetStream domain to use for mqtt.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-12-16 16:53:20 -05:00
Derek Collison
ca12a11be3 There were situations where invalid subjects could be assigned to streams.
This will patch them on the fly during recovery. Specifically subjects with leading or trailing spaces and mirror streams with any subjects at all.

Signed-off-by: Derek Collison <derek@nats.io>
2021-12-01 14:00:23 -07:00
Ivan Kozlovic
1cf8b40304 Merge pull request #2719 from nats-io/js_mem_corruption
[FIXED] Corrupted headers receiving from consumer with meta-only
2021-12-01 13:42:47 -07:00
Ivan Kozlovic
9f30bf00e0 [FIXED] Corrupted headers receiving from consumer with meta-only
When a consumer is configured with "meta-only" option, and the
stream was backed by a memory store, a memory corruption could
happen causing the application to receive corrupted headers.

Also replaced most of usage of `append(a[:0:0], a...)` to make
copies. This was based on this wiki:
https://github.com/go101/go101/wiki/How-to-efficiently-clone-a-slice%3F

But since Go 1.15, it is actually faster to call make+copy instead.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-12-01 10:50:15 -07:00
R.I.Pienaar
c025d25899 prevent stream update to add subjects to mirrors
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-12-01 18:12:49 +01:00
R.I.Pienaar
4f1bfa969f ensure streams have only valid interest subjects
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-12-01 17:03:28 +01:00
Derek Collison
72ad68fada [FIXED] Bug in memstore that when setting max msgs per subject to 1 would not work properly.
Signed-off-by: Derek Collison <derek@nats.io>
2021-11-19 09:13:43 -08:00
Derek Collison
14469ccfc8 Fix for #2662.
Upon server restart a server would set the check expiration to the configured amount vs delta of next to expire.

Signed-off-by: Derek Collison <derek@nats.io>
2021-11-01 18:04:37 -07:00
Derek Collison
c9f615933a Honor JetStream server settings of 0.
Signed-off-by: Derek Collison <derek@nats.io>
2021-10-12 14:16:47 -07:00
Derek Collison
cbbab295ec Merge pull request #2616 from nats-io/fix_2611
[FIXED] #2611
2021-10-12 08:46:42 -07:00
Derek Collison
8951bc54f4 Fix for stream purge where purge would remove all pending messages for a filtered consumer.
Signed-off-by: Derek Collison <derek@nats.io>
2021-10-12 07:26:13 -07:00
Derek Collison
c789ad9211 Simplified a bit
Signed-off-by: Derek Collison <derek@nats.io>
2021-10-12 06:12:56 -07:00
Ivan Kozlovic
3b9c358a81 Test that reproduces the issue reported in #2611
Issue seem to be from the fact that the filter subject is == to
the stream's subjects list and so setInitialPendingAndStart()
considers that the consumer is not filtered.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-10-11 13:32:04 -06:00
Derek Collison
58e5c7c681 Allow consumers to request only headers to be delivered
Signed-off-by: Derek Collison <derek@nats.io>
2021-10-05 18:54:52 -07:00
Ivan Kozlovic
d34b8fdd7d [FIXED] JetStream: data race on shutdown
Replaced use of eventsEnabled() with EventsEnabled() that will
check under server lock. Also found another reference when
creating templates.

Resolves #2588

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-09-30 11:17:12 -06:00
Ivan Kozlovic
a3be36fcd4 Moving 2 new tests to jetstream test file
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-09-22 17:02:29 -06:00
Derek Collison
9534372113 Fix for #2551
When a mirror would be processed before the origin stream we would not recover the consumers due to failure on looking up source's subjects.
This change processes all streams first then does all consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2021-09-21 08:53:12 -07:00
R.I.Pienaar
3be0b23a3e Merge pull request #2546 from ripienaar/negative_dupe_window_protection
protect against negative dupe window via negative max age
2021-09-20 18:09:19 +02:00
R.I.Pienaar
34b5a11e20 protect against negative dupe window via negative max age
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-09-20 17:34:48 +02:00
Derek Collison
cecb6246f2 Test tweaks
Signed-off-by: Derek Collison <derek@nats.io>
2021-09-19 13:28:37 -07:00
Derek Collison
40a4d40337 Make large batch requests expire more efficiently.
Signed-off-by: Derek Collison <derek@nats.io>
2021-09-16 15:00:18 -07:00
Ivan Kozlovic
108d4060ff [CHANGED] JetStream: flow control requires heartbeats
The server will now reject the creation of a push consumer with
flow control if no heartbeat is set.

Resolves #2520

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-09-15 09:50:15 -06:00
Derek Collison
e18a278196 Added ability to check if no message exists as a test to store the message.
Signed-off-by: Derek Collison <derek@nats.io>
2021-09-09 11:41:23 -07:00
Derek Collison
dadc3b9fae Fixed a bug when an interest retention stream with noack consumers is in clustered mode.
We were not properly propagating the ack state and proper cleanup of the stream messages.

Signed-off-by: Derek Collison <derek@nats.io>
2021-09-08 15:02:09 -07:00
Ivan Kozlovic
ba36aa452b Fix some timing and bump server version
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-09-01 18:03:14 -06:00
Ivan Kozlovic
44c57a5702 [FIXED] Pull requests: don't send 408 when request expires
When expiring requests, the server would send 408 if interest was
still present, which can happen for pull subscribe implementations
that maintain interest for the duration of the pull subscription.

Let's keep the 408 for when a request is "force expired", that
is, a request was removed from the queue because it queue was
full but interest is still found.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-09-01 16:57:13 -06:00
Derek Collison
752fd295a5 Consumer num pending fixes for multiple matches and merging.
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-24 07:52:29 -07:00
Derek Collison
70d28bd221 Use append in case pubAck is larger than stack []byte
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-20 15:53:37 -07:00
Derek Collison
6871d1240b When we expired all messages on a restart we did not properly setup lmb.
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-17 13:45:50 -07:00
Matthias Hanel
2ec87a56ba fixed unit tests where SequencePair comparisons fail due to time last_active
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-08-16 21:05:12 -04:00
Derek Collison
b095870c2a Fixed a leak of internal clients for JetStream consumers.
We were not properly unregistering from the account on cleanup.

Signed-off-by: Derek Collison <derek@nats.io>
2021-08-14 15:09:42 -07:00
Derek Collison
b6c0412d2a More fixes for some flapping tests
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-14 13:35:04 -07:00
Derek Collison
969cf60def Add in push bound status for consumer info.
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-14 08:35:40 -07:00
Derek Collison
c4a78d91b7 Remove PushActive
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-13 15:11:31 -07:00
Derek Collison
5ec5020a8b Add in DeliverGroup to consumer as optional queue group.
Changed sublist register for notifications to be queue bound.

Signed-off-by: Derek Collison <derek@nats.io>
2021-08-13 15:07:56 -07:00
Derek Collison
d252f0f700 Make sure PushActive is nil for pull consumers
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-12 10:15:55 -07:00
Derek Collison
c781256dda Add in deliver status for consumer info
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-11 15:41:33 -07:00
Derek Collison
da3c89efda Add domain to PubAck1
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-11 13:25:24 -07:00
Derek Collison
7dd399e355 Merge branch 'main' into max-per-default
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-10 07:33:58 -07:00
Derek Collison
cb6b397573 Fix for #2423
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-10 06:46:48 -07:00
Derek Collison
633763a202 Fix for #2420
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-10 06:36:08 -07:00
Derek Collison
bf966d84f3 Number of initial pending was off by one when DeliverLastPerSubject was utilized on a consumer.
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-04 14:20:52 -07:00
Derek Collison
154bc40718 Fix for reentrant read lock on a stream that once anyone else wanted the write lock would deadlock.
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-03 15:46:40 -07:00
Derek Collison
067ef19ce9 Fix for #2392. We were not resetting the pending timer properly during stream purge.
No we just let it fire as normal and it will be canceled or not as needed.

Signed-off-by: Derek Collison <derek@nats.io>
2021-08-01 17:28:59 -07:00