Commit Graph

2868 Commits

Author SHA1 Message Date
Derek Collison
6ecfda8d9c 2.2.3-beta.6
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 20:13:11 -07:00
Derek Collison
c2fcc114a5 Update based on PR feedback, moved to validateOptions
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 20:10:44 -07:00
Derek Collison
5a8791b1ef Check for bad domain names
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 19:41:43 -07:00
Ivan Kozlovic
a9a49cc2d5 MQTT make session streams domain aware
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-06 20:02:00 -06:00
Derek Collison
2002ea1ee5 Another test fix
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
50d5875aa3 Fix test
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
0c74a850b5 Make storedir consistent with mix of flags etc 2021-05-06 18:46:32 -06:00
Derek Collison
ea5cddd590 Moved the JetStream logic for solicited leafnodes to after we receive first info.
We needed access to the other side's JetStream status.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Ivan Kozlovic
f5eb8bef89 Fixed some tests to manually close account resolver
Those tests don't really start the server, so the account resolver's
internal expiration routine would be left running.
Doing an explicit close solves this issue.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-06 18:46:32 -06:00
Derek Collison
8499376575 Add in support for JetStream domains.
This allows a domain to be set in the JetStream server block that sets a domain name.
Once set this signals that any leafnode connections should operate as separate JetStream domains.
Each domain <NAME> is accessible via "$JS.<NAME>.API.>", even when connected to the same domain.
Also for mixed mode you can set a jetstream block now that defines a domain but specifies "enabled: false".

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:46:32 -06:00
Derek Collison
bfd0e00271 Fix data race
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Derek Collison
c8d2132ee5 Expand test to target mirrors and sources in different JS domains.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Derek Collison
df664e780e Rework auto insertion of deny exports and imports for leafnodes.
This shifts to runtime vs setup time.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Derek Collison
c87eed8792 Create helper function to check on account jetstream status.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Derek Collison
0bd92e85da Add in formal support for multiple JetStream domains across leafnodes.
This CL adds in support for multiple JetStream domains using mapped subjects.
Mapping subjects aligns well with the JetStream context APIPrefix in clients.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-06 18:45:27 -06:00
Ivan Kozlovic
9eb12b6e1c Merge pull request #2178 from nats-io/mqtt_updates
[FIXED] MQTT fixes and improvements
2021-05-05 16:39:50 -06:00
Ivan Kozlovic
f1730593c0 Force server name to be set if mqtt{} defined
This will solve the issue of naming the durable per server for
the "retained messages" stream in situation where a cluster
of servers would not have JetStream defined but connect to another
cluster that has it. All the servers within the cluster without
JetStream would cause the durable's delivery subject to be updated
to the last server starting the durable.

Updated the check for mqtt requiring JetStream if running in
standalone mode to check that no leafnode configuration is present.

Replaced use of fmt.Errorf() when the string was static with
errors created with errors.New(). Updated tests that were checking
for errors to use those errors instead of repeating the string.

Added test that has a hub cluster with JS enabled and a remote server
that has mqtt{} without JetStream and ensure that this works.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-05 13:17:53 -06:00
Matthias Hanel
0bfa7f6793 Merge pull request #2181 from nats-io/jwt-decorated
[fixed] decorated jwt parsing issue by using same functionality of jwt
2021-05-05 14:36:54 -04:00
Derek Collison
9bfab00194 Add in comments to help with usage
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-05 08:58:14 -07:00
Derek Collison
a33c64959d [FIXED] Under certain conditions with messages being auto-deleted we would not honor blk size properly.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-05 08:27:22 -07:00
Derek Collison
bd9a782701 Fixed issue when limiting bytes with an unlimited account that would always fail.
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-05 07:04:24 -07:00
Derek Collison
7022b59b50 2.2.3-beta.5
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-05 05:41:27 -07:00
Derek Collison
60dd52fcba Merge pull request #2180 from nats-io/fs_updates
[IMPROVED] Stability when expireMsgs and writeMsg and Compact/Purge concurrent.
2021-05-04 20:43:41 -07:00
Derek Collison
71ba4b1bf6 Improved stability when expireMsgs and writeMsg and Compact/Purge all concurrent.
We had issues of instability and incorrect behavior during concurrent operations.
This CL optimizes expiring msgs to be more efficient and hold the lock until completion.
Compact will also now hold the top level lock through completion.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-04 20:12:38 -07:00
Matthias Hanel
61bf08fd98 [fixed] decorated jwt parsing issue by using same functionality of jwt
fixes #2069

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-05-04 23:04:51 -04:00
Ivan Kozlovic
2881e4a1f0 [FIXED] MQTT fixes and improvements
Some issues that have been fixed would manifest by timeouts on
connect, unexpected memory usage on high publish message rate.

Some details:
- Replies were not always GW routed properly because we were looking
at the wrong connection's rsubs
- GW routed replies would not be found because they were tracked
in the subscription's client object, which may not be the same used
to send the reply
- Increased the mqtt timeout to wait for JS replies since in some
tests it was sometimes taking more than the original 2 seconds
- Incoming gateway messages destined for an MQTT internal subscription
may have been rejected as a no interest if the account had service imports
- Don't use time.After(), instead create explicit timer so it can
be stopped when not timing out.
- Unnecessary copy of a slice since we were converting to a string anyway.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-04 20:48:14 -06:00
Ivan Kozlovic
0fd60f65b4 Merge pull request #2179 from nats-io/mqtt_clustered_retained_msg
[FIXED] MQTT retained message in cluster mode may not be delivered
2021-05-04 20:37:19 -06:00
Ivan Kozlovic
e2f93baa9d [FIXED] MQTT retained message in cluster mode may not be delivered
In cluster mode, a sub connects to server 1, another on server 2.
A publisher connects to server 2 and publishes a retained message.
If both subs restart they would properly receive the retained message.

However, if the publisher sens an empty message that "removes" the
retained message for this topic, and then consumer that connects to
server 1 restarts, it would not receive the retained message as it
should.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-04 14:32:53 -06:00
Derek Collison
4533a70667 Bump version
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-04 06:38:56 -07:00
Derek Collison
db402cc444 Under heavy load and a leader change we could warn about not processing responses.
This also adjust the min election timeout to 2 seconds vs just 1 for very large networks.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-03 19:40:40 -07:00
Derek Collison
afc926901b Bump version
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-03 19:11:43 -07:00
Derek Collison
b3f9166b4f [FIXED] Getting varz from the http endoint saw Subscriptions always double for each fetch.
Resolves part of #2170

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-03 18:43:07 -07:00
Derek Collison
670f44f1e8 Bump version
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-01 09:34:44 -07:00
Derek Collison
d10b054f43 Update comment per feedback
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 18:20:29 -07:00
Derek Collison
d43c818bf8 Make sure when we extend the JetStream domain with a single server leafnode we can place things there.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 18:18:16 -07:00
Derek Collison
8bf99224c5 This adds ability to have a single node server with a system leafnode expand an existing JetStream cluster domain.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 16:20:32 -07:00
Derek Collison
a9c591533c Move to Info vs Warn
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 15:39:09 -07:00
Derek Collison
ba31bb6165 When detecting a jetStream domain that is extended to a leafnode or leafnode cluster
we want to auto-suppress JetStream traffic on normal accounts.

We also now track remote accounts so that client info headers can be remapped.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 15:23:12 -07:00
R.I.Pienaar
c7caa576f5 Merge pull request #2165 from ripienaar/custom_auth_username
allow custom authenticators to set a username
2021-04-30 14:37:50 +02:00
R.I.Pienaar
683887ca97 allow custom authenticators to set a username
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-04-30 14:22:19 +02:00
Derek Collison
be5445e916 Bump version
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-29 16:05:03 -07:00
Ivan Kozlovic
ffccc2e1bd [FIXED] TLS: default ciphers not set when tls enabled from command line
If running the server with command lines:
```
nats-server --tlsverify --tlscert "cert.pem" --tlskey "key.pem"
```
the default ciphers would not be set, however, they would using this
equivalent config:
```
tls: {
   verify: true
   cert_file: "cert.pem"
   key_file: "key.pem"
}
```

Reported by @DavidSimner

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-29 12:50:23 -06:00
Ivan Kozlovic
53a14eb5d1 Merge pull request #2163 from nats-io/msg_loop
[FIXED] Message loop with cluster, leaf nodes and queue subs
2021-04-29 10:02:17 -06:00
Derek Collison
cb773f984b Merge pull request #2166 from nats-io/lngwr
[FIXED] JetStream pull based message delivery could drop responses.
2021-04-29 08:14:45 -07:00
Ivan Kozlovic
72f29cc514 Merge pull request #2164 from nats-io/mqtt_cluster_placement
[FIXED] MQTT: asset placement in origin cluster
2021-04-29 08:50:54 -06:00
Derek Collison
96072d66f0 Internal JetStream callbacks for message delivery across GWs could lose responses.
For internal callbacks we can not rely on the GW routed reply tracking
since internal clients can change who receives vs sends response.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-29 07:27:44 -07:00
Ivan Kozlovic
01ff1b76c1 Fixed comments
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-28 19:32:16 -06:00
Ivan Kozlovic
552cc737f1 [FIXED] MQTT: asset placement in origin cluster
In a setup with shared system account and a cluster of leaf nodes,
the JS requests did not contain the origin cluster, which caused
assets to possibly be created in the HUB. With this change, the
assets will be created in the origin cluster.

Also, removed use of acc.JetStreamEnabled() but instead fail
start of the server if mqtt is enabled in standalone mode and JS
is not enabled. If JS is enabled, we will get proper error if
account has no JS enabled.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-28 19:28:00 -06:00
Ivan Kozlovic
e2e3de9977 [FIXED] Message loop with cluster, leaf nodes and queue subs
In a setup with a cluster of servers to which 2 different leaf nodes
attach to, and queue subs are attached to one of the leaf, if the
leaf server is restarted and reconnects to another server in the
cluster, there was a risk for an infinite message loop between
some servers in the "hub" cluster.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-28 17:11:51 -06:00
Matthias Hanel
a67704e245 [fixed] crash when using nats-resolver without system account (#2162)
* [fixed] crash when using nats-resolver without system account

Fixes #2160
Will raise an error instead

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-26 20:50:56 -04:00