Commit Graph

4453 Commits

Author SHA1 Message Date
Derek Collison
f74bb53c83 Remove debug
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 12:21:35 -07:00
Derek Collison
70a2521f95 For interest or workqueue streams with ephemerals we need to not reduce replication to 1.
We need the consumer state on the stream leader.
Also if we can't find the store yet for a consumer fallback to calculate needsAck.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 12:07:27 -07:00
Derek Collison
95a9304ae7 Tweak timing
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 12:07:27 -07:00
Derek Collison
57395bba02 Fixed bug that could cause raft group to spin trying to catchup.
When a raft group was trying to catch up a consumer but the log is empty and we do have a snapshot but the requested sequence was the first sequence.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 09:13:18 -07:00
Derek Collison
7be6555334 Merge pull request #2187 from nats-io/interest_deadlock
[FIXED] Mirror/Source streams from work queues and a deadlock on interest streams.
2021-05-07 08:52:26 -07:00
Derek Collison
ceb4a86d57 Check for nil chan
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 08:36:45 -07:00
Derek Collison
7715a5b30a 2.2.3-beta.7
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 06:47:34 -07:00
Derek Collison
b6ba5cb4a5 Merge pull request #2188 from nats-io/perm_subjects
Detect if perm subjects have queue group qualifiers.
2021-05-07 06:46:33 -07:00
Derek Collison
32db711723 Detect if perm subjects have queue group qualifiers.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 06:27:43 -07:00
R.I.Pienaar
7f895f45ea Merge pull request #2186 from ripienaar/domain_tweaks
small tweaks to domains
2021-05-07 15:17:41 +02:00
Derek Collison
35a60289d9 Fixed mirror/source streams from work queues.
Fixed deadlock with no-ack consumers on interest retention streams.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-07 06:01:52 -07:00
R.I.Pienaar
cc9b6735a5 remove . from domain names
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-05-07 14:46:22 +02:00
R.I.Pienaar
0bc6ac2e14 small tweaks to domains
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-05-07 11:03:23 +02:00
Derek Collison
6ecfda8d9c 2.2.3-beta.6
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 20:13:11 -07:00
Derek Collison
13f0f67465 Merge pull request #2185 from nats-io/bad_domain
Check for bad domain names
2021-05-06 20:12:15 -07:00
Derek Collison
c2fcc114a5 Update based on PR feedback, moved to validateOptions
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 20:10:44 -07:00
Derek Collison
5a8791b1ef Check for bad domain names
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 19:41:43 -07:00
Ivan Kozlovic
c7e043b264 Merge pull request #2171 from nats-io/islands
Add in formal support for multiple JetStream domains across leafnodes.
2021-05-06 20:30:48 -06:00
Ivan Kozlovic
a9a49cc2d5 MQTT make session streams domain aware
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-06 20:02:00 -06:00
Derek Collison
2002ea1ee5 Another test fix
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
50d5875aa3 Fix test
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
0c74a850b5 Make storedir consistent with mix of flags etc 2021-05-06 18:46:32 -06:00
Derek Collison
ea5cddd590 Moved the JetStream logic for solicited leafnodes to after we receive first info.
We needed access to the other side's JetStream status.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Ivan Kozlovic
f5eb8bef89 Fixed some tests to manually close account resolver
Those tests don't really start the server, so the account resolver's
internal expiration routine would be left running.
Doing an explicit close solves this issue.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-06 18:46:32 -06:00
Derek Collison
8499376575 Add in support for JetStream domains.
This allows a domain to be set in the JetStream server block that sets a domain name.
Once set this signals that any leafnode connections should operate as separate JetStream domains.
Each domain <NAME> is accessible via "$JS.<NAME>.API.>", even when connected to the same domain.
Also for mixed mode you can set a jetstream block now that defines a domain but specifies "enabled: false".

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
41ec9359fc Update client to released version
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
bfd0e00271 Fix data race
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Derek Collison
c8d2132ee5 Expand test to target mirrors and sources in different JS domains.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Derek Collison
df664e780e Rework auto insertion of deny exports and imports for leafnodes.
This shifts to runtime vs setup time.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Derek Collison
c87eed8792 Create helper function to check on account jetstream status.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Derek Collison
0bd92e85da Add in formal support for multiple JetStream domains across leafnodes.
This CL adds in support for multiple JetStream domains using mapped subjects.
Mapping subjects aligns well with the JetStream context APIPrefix in clients.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Ivan Kozlovic
9eb12b6e1c Merge pull request #2178 from nats-io/mqtt_updates
[FIXED] MQTT fixes and improvements
2021-05-05 16:39:50 -06:00
Ivan Kozlovic
f1730593c0 Force server name to be set if mqtt{} defined
This will solve the issue of naming the durable per server for
the "retained messages" stream in situation where a cluster
of servers would not have JetStream defined but connect to another
cluster that has it. All the servers within the cluster without
JetStream would cause the durable's delivery subject to be updated
to the last server starting the durable.

Updated the check for mqtt requiring JetStream if running in
standalone mode to check that no leafnode configuration is present.

Replaced use of fmt.Errorf() when the string was static with
errors created with errors.New(). Updated tests that were checking
for errors to use those errors instead of repeating the string.

Added test that has a hub cluster with JS enabled and a remote server
that has mqtt{} without JetStream and ensure that this works.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-05 13:17:53 -06:00
Matthias Hanel
0bfa7f6793 Merge pull request #2181 from nats-io/jwt-decorated
[fixed] decorated jwt parsing issue by using same functionality of jwt
2021-05-05 14:36:54 -04:00
Derek Collison
55c87226b3 Merge pull request #2183 from nats-io/js-fs-updates
[FIXED] Stream byte limit setting failed on dynamic account, filestore not honoring blksize.
2021-05-05 11:22:04 -07:00
Derek Collison
9bfab00194 Add in comments to help with usage
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-05 08:58:14 -07:00
Derek Collison
a33c64959d [FIXED] Under certain conditions with messages being auto-deleted we would not honor blk size properly.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-05 08:27:22 -07:00
Derek Collison
bd9a782701 Fixed issue when limiting bytes with an unlimited account that would always fail.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-05 07:04:24 -07:00
Derek Collison
7022b59b50 2.2.3-beta.5
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-05 05:41:27 -07:00
Derek Collison
60dd52fcba Merge pull request #2180 from nats-io/fs_updates
[IMPROVED] Stability when expireMsgs and writeMsg and Compact/Purge concurrent.
2021-05-04 20:43:41 -07:00
Derek Collison
71ba4b1bf6 Improved stability when expireMsgs and writeMsg and Compact/Purge all concurrent.
We had issues of instability and incorrect behavior during concurrent operations.
This CL optimizes expiring msgs to be more efficient and hold the lock until completion.
Compact will also now hold the top level lock through completion.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-04 20:12:38 -07:00
Matthias Hanel
61bf08fd98 [fixed] decorated jwt parsing issue by using same functionality of jwt
fixes #2069

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-05-04 23:04:51 -04:00
Ivan Kozlovic
2881e4a1f0 [FIXED] MQTT fixes and improvements
Some issues that have been fixed would manifest by timeouts on
connect, unexpected memory usage on high publish message rate.

Some details:
- Replies were not always GW routed properly because we were looking
at the wrong connection's rsubs
- GW routed replies would not be found because they were tracked
in the subscription's client object, which may not be the same used
to send the reply
- Increased the mqtt timeout to wait for JS replies since in some
tests it was sometimes taking more than the original 2 seconds
- Incoming gateway messages destined for an MQTT internal subscription
may have been rejected as a no interest if the account had service imports
- Don't use time.After(), instead create explicit timer so it can
be stopped when not timing out.
- Unnecessary copy of a slice since we were converting to a string anyway.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-04 20:48:14 -06:00
Ivan Kozlovic
0fd60f65b4 Merge pull request #2179 from nats-io/mqtt_clustered_retained_msg
[FIXED] MQTT retained message in cluster mode may not be delivered
2021-05-04 20:37:19 -06:00
Ivan Kozlovic
e2f93baa9d [FIXED] MQTT retained message in cluster mode may not be delivered
In cluster mode, a sub connects to server 1, another on server 2.
A publisher connects to server 2 and publishes a retained message.
If both subs restart they would properly receive the retained message.

However, if the publisher sens an empty message that "removes" the
retained message for this topic, and then consumer that connects to
server 1 restarts, it would not receive the retained message as it
should.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-04 14:32:53 -06:00
Derek Collison
4533a70667 Bump version
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-04 06:38:56 -07:00
Derek Collison
dda3f1bd44 Merge pull request #2173 from nats-io/raft_tweaks
Under heavy load a leader change could warn about not processing entry responses.
2021-05-04 06:38:23 -07:00
Derek Collison
db402cc444 Under heavy load and a leader change we could warn about not processing responses.
This also adjust the min election timeout to 2 seconds vs just 1 for very large networks.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-03 19:40:40 -07:00
Derek Collison
afc926901b Bump version
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-03 19:11:43 -07:00
Derek Collison
850307ae4a Merge pull request #2172 from nats-io/varz_subs
[FIXED] Getting varz from the http endpoint saw subscriptions always double for each fetch.
2021-05-03 18:57:57 -07:00