Commit Graph

4412 Commits

Author SHA1 Message Date
Ivan Kozlovic
f1730593c0 Force server name to be set if mqtt{} defined
This will solve the issue of naming the durable per server for
the "retained messages" stream in situation where a cluster
of servers would not have JetStream defined but connect to another
cluster that has it. All the servers within the cluster without
JetStream would cause the durable's delivery subject to be updated
to the last server starting the durable.

Updated the check for mqtt requiring JetStream if running in
standalone mode to check that no leafnode configuration is present.

Replaced use of fmt.Errorf() when the string was static with
errors created with errors.New(). Updated tests that were checking
for errors to use those errors instead of repeating the string.

Added test that has a hub cluster with JS enabled and a remote server
that has mqtt{} without JetStream and ensure that this works.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-05 13:17:53 -06:00
Ivan Kozlovic
2881e4a1f0 [FIXED] MQTT fixes and improvements
Some issues that have been fixed would manifest by timeouts on
connect, unexpected memory usage on high publish message rate.

Some details:
- Replies were not always GW routed properly because we were looking
at the wrong connection's rsubs
- GW routed replies would not be found because they were tracked
in the subscription's client object, which may not be the same used
to send the reply
- Increased the mqtt timeout to wait for JS replies since in some
tests it was sometimes taking more than the original 2 seconds
- Incoming gateway messages destined for an MQTT internal subscription
may have been rejected as a no interest if the account had service imports
- Don't use time.After(), instead create explicit timer so it can
be stopped when not timing out.
- Unnecessary copy of a slice since we were converting to a string anyway.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-04 20:48:14 -06:00
Ivan Kozlovic
0fd60f65b4 Merge pull request #2179 from nats-io/mqtt_clustered_retained_msg
[FIXED] MQTT retained message in cluster mode may not be delivered
2021-05-04 20:37:19 -06:00
Ivan Kozlovic
e2f93baa9d [FIXED] MQTT retained message in cluster mode may not be delivered
In cluster mode, a sub connects to server 1, another on server 2.
A publisher connects to server 2 and publishes a retained message.
If both subs restart they would properly receive the retained message.

However, if the publisher sens an empty message that "removes" the
retained message for this topic, and then consumer that connects to
server 1 restarts, it would not receive the retained message as it
should.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-05-04 14:32:53 -06:00
Derek Collison
4533a70667 Bump version
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-04 06:38:56 -07:00
Derek Collison
dda3f1bd44 Merge pull request #2173 from nats-io/raft_tweaks
Under heavy load a leader change could warn about not processing entry responses.
2021-05-04 06:38:23 -07:00
Derek Collison
db402cc444 Under heavy load and a leader change we could warn about not processing responses.
This also adjust the min election timeout to 2 seconds vs just 1 for very large networks.

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-03 19:40:40 -07:00
Derek Collison
afc926901b Bump version
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-03 19:11:43 -07:00
Derek Collison
850307ae4a Merge pull request #2172 from nats-io/varz_subs
[FIXED] Getting varz from the http endpoint saw subscriptions always double for each fetch.
2021-05-03 18:57:57 -07:00
Derek Collison
b3f9166b4f [FIXED] Getting varz from the http endoint saw Subscriptions always double for each fetch.
Resolves part of #2170

Signed-off-by: Derek Collison <derek@nats.io>
2021-05-03 18:43:07 -07:00
Derek Collison
670f44f1e8 Bump version
Signed-off-by: Derek Collison <derek@nats.io>
2021-05-01 09:34:44 -07:00
Derek Collison
48a1d06575 Merge pull request #2169 from nats-io/lnacc
JetStream extended over a leafnode fixes.
2021-05-01 09:34:12 -07:00
Derek Collison
d10b054f43 Update comment per feedback
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 18:20:29 -07:00
Derek Collison
d43c818bf8 Make sure when we extend the JetStream domain with a single server leafnode we can place things there.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 18:18:16 -07:00
Derek Collison
8bf99224c5 This adds ability to have a single node server with a system leafnode expand an existing JetStream cluster domain.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 16:20:32 -07:00
Derek Collison
a9c591533c Move to Info vs Warn
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 15:39:09 -07:00
Derek Collison
ba31bb6165 When detecting a jetStream domain that is extended to a leafnode or leafnode cluster
we want to auto-suppress JetStream traffic on normal accounts.

We also now track remote accounts so that client info headers can be remapped.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-30 15:23:12 -07:00
R.I.Pienaar
c7caa576f5 Merge pull request #2165 from ripienaar/custom_auth_username
allow custom authenticators to set a username
2021-04-30 14:37:50 +02:00
R.I.Pienaar
683887ca97 allow custom authenticators to set a username
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-04-30 14:22:19 +02:00
Derek Collison
be5445e916 Bump version
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-29 16:05:03 -07:00
Ivan Kozlovic
2fd8962191 Merge pull request #2167 from nats-io/fix_tls_cmdline_ciphers
[FIXED] TLS: default ciphers not set when tls enabled from command line
2021-04-29 15:51:10 -06:00
Ivan Kozlovic
ffccc2e1bd [FIXED] TLS: default ciphers not set when tls enabled from command line
If running the server with command lines:
```
nats-server --tlsverify --tlscert "cert.pem" --tlskey "key.pem"
```
the default ciphers would not be set, however, they would using this
equivalent config:
```
tls: {
   verify: true
   cert_file: "cert.pem"
   key_file: "key.pem"
}
```

Reported by @DavidSimner

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-29 12:50:23 -06:00
Ivan Kozlovic
53a14eb5d1 Merge pull request #2163 from nats-io/msg_loop
[FIXED] Message loop with cluster, leaf nodes and queue subs
2021-04-29 10:02:17 -06:00
Derek Collison
cb773f984b Merge pull request #2166 from nats-io/lngwr
[FIXED] JetStream pull based message delivery could drop responses.
2021-04-29 08:14:45 -07:00
Ivan Kozlovic
72f29cc514 Merge pull request #2164 from nats-io/mqtt_cluster_placement
[FIXED] MQTT: asset placement in origin cluster
2021-04-29 08:50:54 -06:00
Derek Collison
96072d66f0 Internal JetStream callbacks for message delivery across GWs could lose responses.
For internal callbacks we can not rely on the GW routed reply tracking
since internal clients can change who receives vs sends response.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-29 07:27:44 -07:00
Ivan Kozlovic
01ff1b76c1 Fixed comments
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-28 19:32:16 -06:00
Ivan Kozlovic
552cc737f1 [FIXED] MQTT: asset placement in origin cluster
In a setup with shared system account and a cluster of leaf nodes,
the JS requests did not contain the origin cluster, which caused
assets to possibly be created in the HUB. With this change, the
assets will be created in the origin cluster.

Also, removed use of acc.JetStreamEnabled() but instead fail
start of the server if mqtt is enabled in standalone mode and JS
is not enabled. If JS is enabled, we will get proper error if
account has no JS enabled.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-28 19:28:00 -06:00
Ivan Kozlovic
e2e3de9977 [FIXED] Message loop with cluster, leaf nodes and queue subs
In a setup with a cluster of servers to which 2 different leaf nodes
attach to, and queue subs are attached to one of the leaf, if the
leaf server is restarted and reconnects to another server in the
cluster, there was a risk for an infinite message loop between
some servers in the "hub" cluster.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-28 17:11:51 -06:00
Matthias Hanel
a67704e245 [fixed] crash when using nats-resolver without system account (#2162)
* [fixed] crash when using nats-resolver without system account

Fixes #2160
Will raise an error instead

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-26 20:50:56 -04:00
Matthias Hanel
4430a55eed [added] leaf deny exports/imports to varz monitoring (#2159)
* [added] leaf deny exports/imports to varz monitoring

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-26 16:34:09 -04:00
Ivan Kozlovic
0ba3aaf7bb Merge pull request #2156 from nats-io/leaf-shuffle
[CHANGED] Randomize Leafnode remote URLs and add option to disable
2021-04-26 12:26:54 -06:00
Jaime Piña
4d04f281fc Randomize leafnode route URLs and add option to disable 2021-04-23 14:59:15 -07:00
Matthias Hanel
a8346465e5 Merge pull request #2157 from nats-io/nil-leader
fix unit test crashing when leader was nil caused by quorum reformation
2021-04-23 14:26:00 -04:00
Ivan Kozlovic
8d4102c404 Merge pull request #2093 from shkim-will/flush_client_only
[CHANGED] Flush in place only if producer and consumer are client connections
2021-04-23 09:18:56 -06:00
Matthias Hanel
ae4374fcd0 fix unit test crashing when leader was nil caused by quorum reformation
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-22 20:35:00 -04:00
Ivan Kozlovic
a5f3aabb13 Merge pull request #2155 from nats-io/release_2_2_2
Release v2.2.2
v2.2.2
2021-04-22 11:48:41 -06:00
Ivan Kozlovic
38dcc79b3b Release v2.2.2
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-22 11:14:09 -06:00
Ivan Kozlovic
796eb8b12f Merge pull request #2152 from nats-io/close-tempfiles
close temp files, fix path print
2021-04-22 11:12:53 -06:00
scottf
486df98373 close tempfiles, fix path print 2021-04-22 12:47:21 -04:00
Ivan Kozlovic
96546040a3 Merge pull request #2151 from nats-io/fix_2126
[IMPROVED] MQTT error message when client connects with websocket
2021-04-22 09:27:43 -06:00
Derek Collison
b0292e40d2 Merge pull request #2149 from nats-io/pending_race
Do not persist or snapshot consumer state after a restore.
2021-04-22 08:22:14 -07:00
Ivan Kozlovic
d0fd62c83b [IMPROVED] MQTT error message when client connects with websocket
Websocket is currently not supported for MQTT clients. When a
client tries to connect with websocket protocol to the MQTT port,
the error message: `mid:9 - not connected` would be logged, which
is not really telling.

The server will now guess if the connection was websocket and report
a more appropriate error message, such as:
```
invalid connection, websocket currently not supported
```

Resolves #2126

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-22 09:03:47 -06:00
Derek Collison
2ac05785c3 Do not persist or snapshot consumer state after a restore.
This can lead to a data race and is not needed after being applied.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 18:50:38 -07:00
Matthias Hanel
eb3af67031 Delaying stream/consumer error response when there is no contact to meta
this gives a stream/consumer leader time to respond first

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-21 20:24:43 -04:00
Derek Collison
f2d3f5fb81 Merge pull request #2147 from nats-io/race
Fix race
2021-04-21 16:26:42 -07:00
Derek Collison
c9c70dea33 Fix race
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 16:17:58 -07:00
Derek Collison
3418847881 Merge pull request #2146 from nats-io/chblock
Make sure to not have the raft layer block on apply channel on exit.
2021-04-21 15:58:50 -07:00
Derek Collison
2ddb95867e Merge pull request #2145 from nats-io/overlap
Check for overlapping subjects on stream update.
2021-04-21 15:55:31 -07:00
Derek Collison
0678e649d3 Make sure to not have the raft layer block on apply channel on exit.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 15:52:54 -07:00