Commit Graph

198 Commits

Author SHA1 Message Date
Derek Collison
6e6edd0b47 Mapped cross account test
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-19 15:58:27 -07:00
Derek Collison
bd01f202af Under double import scenarios we could map to the wrong subject.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-19 05:59:24 -07:00
Derek Collison
060f2ec127 Fix stream source lookup and add in optional External to StreamSourceInfo.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-12 12:32:56 -07:00
Derek Collison
908b5ec304 Fix for #2205
When a response was needed from a leafnode cluster back to a hub, we had rules to disallow.
That rule was a bit dated and since we have cluster origin for leafnode clusters and that
is checked before the message is actually sent we could remove the old rule.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-12 06:49:11 -07:00
Derek Collison
494ddeec5b Fix for #2202
When using multiple source streams from either different accounts or domains, the stream name could be the same and would cause bad behavior.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-11 13:12:10 -07:00
Derek Collison
bbb94467fb Fix for failing test, we needed to account for consumers having filtered subjects configured but not really being filtered.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-10 17:01:31 -07:00
Derek Collison
09c4f5841c Bump version and skip test for now, will fix in followup PR.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-10 10:44:07 -07:00
Derek Collison
f74bb53c83 Remove debug
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 12:21:35 -07:00
Derek Collison
95a9304ae7 Tweak timing
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 12:07:27 -07:00
Derek Collison
35a60289d9 Fixed mirror/source streams from work queues.
Fixed deadlock with no-ack consumers on interest retention streams.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 06:01:52 -07:00
Ivan Kozlovic
a9a49cc2d5 MQTT make session streams domain aware
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-06 20:02:00 -06:00
Derek Collison
2002ea1ee5 Another test fix
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
ea5cddd590 Moved the JetStream logic for solicited leafnodes to after we receive first info.
We needed access to the other side's JetStream status.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
8499376575 Add in support for JetStream domains.
This allows a domain to be set in the JetStream server block that sets a domain name.
Once set this signals that any leafnode connections should operate as separate JetStream domains.
Each domain <NAME> is accessible via "$JS.<NAME>.API.>", even when connected to the same domain.
Also for mixed mode you can set a jetstream block now that defines a domain but specifies "enabled: false".

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
c8d2132ee5 Expand test to target mirrors and sources in different JS domains.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Derek Collison
0bd92e85da Add in formal support for multiple JetStream domains across leafnodes.
This CL adds in support for multiple JetStream domains using mapped subjects.
Mapping subjects aligns well with the JetStream context APIPrefix in clients.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Derek Collison
bd9a782701 Fixed issue when limiting bytes with an unlimited account that would always fail.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-05 07:04:24 -07:00
Derek Collison
d43c818bf8 Make sure when we extend the JetStream domain with a single server leafnode we can place things there.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 18:18:16 -07:00
Derek Collison
8bf99224c5 This adds ability to have a single node server with a system leafnode expand an existing JetStream cluster domain.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 16:20:32 -07:00
Derek Collison
ba31bb6165 When detecting a jetStream domain that is extended to a leafnode or leafnode cluster
we want to auto-suppress JetStream traffic on normal accounts.

We also now track remote accounts so that client info headers can be remapped.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 15:23:12 -07:00
Ivan Kozlovic
552cc737f1 [FIXED] MQTT: asset placement in origin cluster
In a setup with shared system account and a cluster of leaf nodes,
the JS requests did not contain the origin cluster, which caused
assets to possibly be created in the HUB. With this change, the
assets will be created in the origin cluster.

Also, removed use of acc.JetStreamEnabled() but instead fail
start of the server if mqtt is enabled in standalone mode and JS
is not enabled. If JS is enabled, we will get proper error if
account has no JS enabled.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-28 19:28:00 -06:00
Matthias Hanel
ae4374fcd0 fix unit test crashing when leader was nil caused by quorum reformation
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-22 20:35:00 -04:00
Derek Collison
50fabe261d Check for overlapping subjects on stream update.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 15:38:38 -07:00
Derek Collison
9f1bc5882c Make sure to return no consumer if we are still waiting on a leader to be elected.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 12:13:10 -07:00
Derek Collison
a301d3a892 Check direct consumers directly, stream state back to previous behavior
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 09:40:04 -07:00
Derek Collison
b5521053e6 Make sure to clean up ephemerals across a Gateway.
Also report direct consumers in num consumers in stream info.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 08:47:55 -07:00
Derek Collison
a3f66508dd Merge pull request #2138 from nats-io/ports
Use lower ports to not collide with -1s
2021-04-20 20:15:24 -07:00
Derek Collison
e31361e6a2 Use lower ports to not collide with -1s
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-20 20:10:29 -07:00
Derek Collison
518ff9be14 Concurrent multiple durable subscribers would cause unpredictable behaviors.
Upgraded to current Go client.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-20 19:50:24 -07:00
Derek Collison
5707c1132c Move to dynamic ports
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-20 17:08:47 -07:00
Derek Collison
0a3e54c88a Fix subscription leaks for mirrors and sources for streams as well as flow control subjects for consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-20 12:13:07 -07:00
Derek Collison
fe3537d429 Check err
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-20 07:08:10 -07:00
Derek Collison
2da264c1ec Randomize cluster and gateway ports
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-19 20:24:25 -07:00
Derek Collison
68ddd519d2 Process upstream missing messages for mirrors better.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-19 20:15:21 -07:00
Derek Collison
c41c89ad75 Help with flapping tests
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-18 18:35:29 -07:00
Derek Collison
d84af39f16 Make sure source consumers work properly after expiration/purge
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-16 17:28:04 -07:00
Derek Collison
2117d7750a Use higher timeout for deleting consumer
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-16 14:18:04 -07:00
Derek Collison
d7641b9d38 Move test to norace
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-16 14:00:11 -07:00
Ivan Kozlovic
6e1205b660 Cleanup some tests + GetTLSConnectionState() race fix
Missing defers

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-15 11:37:43 -06:00
Derek Collison
f3f04e6442 Fix flapper
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-15 06:34:12 -07:00
Derek Collison
cf34514f9f Do not limit expansion of new peers
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-14 18:47:11 -07:00
Jaime Piña
1ccb5572ba Fix calling FailNow in parent test
Currently, TestJetStreamClusterMirrorExpirationAndMissingSequences calls
FailNow from the parent test when a subtest fails. This causes an error.

This change calls FailNow to fail the subtest from the subtest itself.
2021-04-14 14:45:11 -07:00
Derek Collison
6ca6db70b2 Add in more extensive test on extending hubs with leafnode clusters
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-13 19:55:41 -07:00
Derek Collison
f6a82a7c98 When messages were no longer available in an upstream stream a mirror could wedge and not resolve.
This fixes that scenario by detecting the situation and inserting skip msgs to catch up.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-13 11:46:03 -07:00
Derek Collison
90989d57d6 Change to report total deleted by default for stream info.
Allow deleted details if requested.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-12 18:10:11 -07:00
Derek Collison
755ef74855 When a cluser of leafnodes connects to a cluster or supercluster hub and they share the system account make the leafnode servers observers.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-12 17:00:55 -07:00
Derek Collison
f893345b40 Pull based consumers when deleted were not removing messages from an interest based stream.
Fix for #2097.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-12 11:16:31 -07:00
Derek Collison
66b0c5c216 Merge pull request #2100 from nats-io/csz
Only adjust cluster size when we have heard from all peers.
2021-04-12 08:52:07 -07:00
Derek Collison
0cee993e3b When checking cluster size we need to make sure we have heard from all peers before making adjustments.
Also check back periodically.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-10 15:55:51 -07:00
Derek Collison
833279388b Add mixedmode test back in
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-10 10:10:37 -07:00