Commit Graph

172 Commits

Author SHA1 Message Date
Derek Collison
cbb97a4f4a Fix for Go client behavior change
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-27 10:52:03 -07:00
Derek Collison
545230bd9e FIXED Clustered streams can become broken with seq mismatch state on low level store failures.
We can account for high level failure properly but were not properly accounting for low level store failures.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-25 13:46:40 -07:00
Derek Collison
08197de9e0 FIXED max consumers was not enforced when set on stream
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-25 11:45:36 -07:00
Derek Collison
2e145196b8 Fix for extended purge by sequence.
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-22 07:38:30 -07:00
Derek Collison
5fcd9c5564 Add test for sequence only
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-20 10:42:35 -07:00
Derek Collison
89d930fd0f Updates and fixes to PurgeEx
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-20 10:34:27 -07:00
Derek Collison
9398c3ca28 Allow for more advanced purge operations that filter by subject, specify the sequence or number of messages to keep.
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-19 07:04:44 -07:00
Derek Collison
08cdb2d2ea Make filtered consumers in large mixed streams more efficient.
Allow wider scoped filtered subjects.

We introduce a per subject information tracking to filestore to optimize for large mux'd streams and more efficient filtered consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-15 04:44:05 -07:00
Derek Collison
ceebc3ae07 When checking limits we would check total ask against the server limits if limits were not set.
We were also dynamically setting account limits based on a single server limit.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-12 10:27:43 -07:00
Derek Collison
4912c8b8f7 Make sure leader ready
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-10 21:29:03 -07:00
Derek Collison
326a377b3e When leader changes happened to streams that mirrored or sourced other streams we could continue to try to create consumers.
This could get excessive on a stream that has sourced many upstream origin streams and had several leader changes.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-10 19:47:46 -07:00
Derek Collison
1270977322 When receiving a response across a gateway that has headers and a globally routed subject (_GR_) we were dropping header information.
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-10 14:29:33 -07:00
Derek Collison
cc63915f54 Added active servers to statsz.
This is generally useful but will also help with cli actions since we know from first response how many total responses are expected.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-09 19:14:37 -07:00
Derek Collison
e238512285 Handling of rewrites for subjects to a globally routed subject was not properly handling c.pa.deliver or reply rewrite.
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-09 15:34:47 -07:00
Derek Collison
30fae4f960 Changes to leafnodes to support multiple domains where the hub is JetStream enabled but the hub account is not, and the leafnode is.
We were incorrectly shutting things down via deny clauses when detecting the remote side/hub had JetStream capabilities.
This change moves that logic to the remote side and is signalled off the connect message which let's the remote side know
if the local leafnode account has JetStream enabled.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-07 08:39:11 -07:00
R.I.Pienaar
0d391b02eb richer api errors proposal
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-05-26 08:04:50 +02:00
Derek Collison
0d6b022780 Fix for varz JetStream usage showing total account usage vs server usage.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-24 14:05:42 -07:00
Derek Collison
8888ab51f4 Fix for #2243. We were not allowing replicated acks processing for workqueues properly, only interest retention.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-24 09:53:31 -07:00
Derek Collison
8f2457fd30 Merge branch 'master' into msgid 2021-05-24 09:22:35 -07:00
Derek Collison
308355a2fd Fix for #2242.
When we had a duplicate detected in R>1 mode we set the skip sequence indicator but were not using that when dealing with underlying store.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-24 08:21:41 -07:00
Derek Collison
a52697170c Fix for corrupting a message when extending an existing header.
We had a report of corrupt message payloads when going across leafnodes between streams that were sourced from one another.
We were incorrectly using the underlying buffer when a header already existed.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-23 11:28:48 -07:00
Derek Collison
7936faf00e Merge pull request #2230 from nats-io/mapped
Mapped cross account test
2021-05-20 11:32:33 -07:00
Derek Collison
a14f07f86c Fix flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-19 19:09:49 -07:00
Derek Collison
9ccc843382 Removing peers should wait for RemovePeer entry replication.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-19 18:58:19 -07:00
Derek Collison
6e6edd0b47 Mapped cross account test
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-19 15:58:27 -07:00
Derek Collison
bd01f202af Under double import scenarios we could map to the wrong subject.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-19 05:59:24 -07:00
Derek Collison
060f2ec127 Fix stream source lookup and add in optional External to StreamSourceInfo.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-12 12:32:56 -07:00
Derek Collison
908b5ec304 Fix for #2205
When a response was needed from a leafnode cluster back to a hub, we had rules to disallow.
That rule was a bit dated and since we have cluster origin for leafnode clusters and that
is checked before the message is actually sent we could remove the old rule.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-12 06:49:11 -07:00
Derek Collison
494ddeec5b Fix for #2202
When using multiple source streams from either different accounts or domains, the stream name could be the same and would cause bad behavior.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-11 13:12:10 -07:00
Derek Collison
bbb94467fb Fix for failing test, we needed to account for consumers having filtered subjects configured but not really being filtered.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-10 17:01:31 -07:00
Derek Collison
09c4f5841c Bump version and skip test for now, will fix in followup PR.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-10 10:44:07 -07:00
Derek Collison
f74bb53c83 Remove debug
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 12:21:35 -07:00
Derek Collison
95a9304ae7 Tweak timing
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 12:07:27 -07:00
Derek Collison
35a60289d9 Fixed mirror/source streams from work queues.
Fixed deadlock with no-ack consumers on interest retention streams.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 06:01:52 -07:00
Ivan Kozlovic
a9a49cc2d5 MQTT make session streams domain aware
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-06 20:02:00 -06:00
Derek Collison
2002ea1ee5 Another test fix
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
ea5cddd590 Moved the JetStream logic for solicited leafnodes to after we receive first info.
We needed access to the other side's JetStream status.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
8499376575 Add in support for JetStream domains.
This allows a domain to be set in the JetStream server block that sets a domain name.
Once set this signals that any leafnode connections should operate as separate JetStream domains.
Each domain <NAME> is accessible via "$JS.<NAME>.API.>", even when connected to the same domain.
Also for mixed mode you can set a jetstream block now that defines a domain but specifies "enabled: false".

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
c8d2132ee5 Expand test to target mirrors and sources in different JS domains.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Derek Collison
0bd92e85da Add in formal support for multiple JetStream domains across leafnodes.
This CL adds in support for multiple JetStream domains using mapped subjects.
Mapping subjects aligns well with the JetStream context APIPrefix in clients.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Derek Collison
bd9a782701 Fixed issue when limiting bytes with an unlimited account that would always fail.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-05 07:04:24 -07:00
Derek Collison
d43c818bf8 Make sure when we extend the JetStream domain with a single server leafnode we can place things there.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 18:18:16 -07:00
Derek Collison
8bf99224c5 This adds ability to have a single node server with a system leafnode expand an existing JetStream cluster domain.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 16:20:32 -07:00
Derek Collison
ba31bb6165 When detecting a jetStream domain that is extended to a leafnode or leafnode cluster
we want to auto-suppress JetStream traffic on normal accounts.

We also now track remote accounts so that client info headers can be remapped.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 15:23:12 -07:00
Ivan Kozlovic
552cc737f1 [FIXED] MQTT: asset placement in origin cluster
In a setup with shared system account and a cluster of leaf nodes,
the JS requests did not contain the origin cluster, which caused
assets to possibly be created in the HUB. With this change, the
assets will be created in the origin cluster.

Also, removed use of acc.JetStreamEnabled() but instead fail
start of the server if mqtt is enabled in standalone mode and JS
is not enabled. If JS is enabled, we will get proper error if
account has no JS enabled.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-28 19:28:00 -06:00
Matthias Hanel
ae4374fcd0 fix unit test crashing when leader was nil caused by quorum reformation
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-22 20:35:00 -04:00
Derek Collison
50fabe261d Check for overlapping subjects on stream update.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 15:38:38 -07:00
Derek Collison
9f1bc5882c Make sure to return no consumer if we are still waiting on a leader to be elected.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 12:13:10 -07:00
Derek Collison
a301d3a892 Check direct consumers directly, stream state back to previous behavior
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 09:40:04 -07:00
Derek Collison
b5521053e6 Make sure to clean up ephemerals across a Gateway.
Also report direct consumers in num consumers in stream info.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 08:47:55 -07:00