Commit Graph

655 Commits

Author SHA1 Message Date
Neil Twigg
1434ee7da8 More specific check for service imports vs any internal sub
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-12 19:52:31 +01:00
Neil Twigg
fbbbe28278 Evaluate service imports when publishing advisories
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-12 18:18:57 +01:00
Neil Twigg
cf715845d1 Additional check for TLS required in WebSocket INFO
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-21 19:06:49 +01:00
Neil Twigg
afdc2478ca WebSocket-specific INFO
This fixes #4252 by ensuring that `tls_available`, `tls_required`, `host` and `port`
are populated based on the WebSocket listener rather than standard listeners.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-20 11:47:09 +01:00
Artem Seleznev
27a8b96ee3 different panic fixes
Signed-off-by: Artem Seleznev <seleznyov.artyom@gmail.com>
2023-06-02 13:19:22 +03:00
Derek Collison
0c13f174c0 Fixed cap mistake in comment
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 15:07:00 -07:00
Derek Collison
90d1063674 Fix for #4149 to allow proper user filtering on connz for other user types.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 14:19:37 -07:00
Derek Collison
76f4358349 [IMPROVED] Optimizations for large single hub account leafnode fleets. (#4135)
Added a leafnode lock to allow better traversal without copying of large
leafnodes in a single hub account.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-06 09:53:08 -07:00
Derek Collison
80db7a22ab Optimizations for large single hub account leafnode fleets.
Added a leafnode lock to allow better traversal without copying of large leafnodes in a single hub account.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-05 13:14:49 -07:00
Waldemar Quevedo
b886fed2fb Stop using UTC for time for flushClients
In #1943 it was adopted to use `UTC()` in some timestamps,
but an unintended side effect from this is that it strips 
the monotonic time, so it can be prone to clock skews when
subtracting time in other areas of the code.
e5646b23de
2023-05-04 15:50:45 -07:00
Ivan Kozlovic
840c264f45 Cleanup use of s.opts and fixed some lock (deadlock/inversion) issues
One should not access s.opts directly but instead use s.getOpts().
Also, server lock needs to be released when performing an account
lookup (since this may result in server lock being acquired).
A function was calling s.LookupAccount under the client lock, which
technically creates a lock inversion situation.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-03 14:09:02 -06:00
Neil Twigg
2206f9e468 Re-add coalescing to outbound queues
Originally I thought there was a race condition happening here,
but it turns out it is safe after all and the race condition I
was seeing was due to other problems in the WebSocket code.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-04-25 12:15:11 +01:00
Neil Twigg
2ece00b08f Buffer re-use in WebSocket code, fix race conditions
Signed-off-by: Neil Twigg <neil@nats.io>
2023-04-21 15:33:48 +01:00
Neil Twigg
bf286744dd Remove coalescing as it races with the writev syscall
Signed-off-by: Neil Twigg <neil@nats.io>
2023-04-20 23:29:36 +01:00
Derek Collison
722a48d726 Don't require lock for check no echo status
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-08 20:05:33 -07:00
Derek Collison
2da50512e2 Optimize non-inline direct gets to not use simple go routines
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 07:50:57 -07:00
Sandy Kellagher
5ae83b7983 Alternative fix for issue 4014, where we always send pings on ROUTER connections, updating c.lastIn timestamp on receiving client data or ping 2023-04-05 13:03:46 +01:00
Neil Twigg
f2bffec366 Refactor outbound queues, remove dynamic sizing, add buffer reuse
Also try to reduce flakiness of `TestClusterQueueSubs` and `TestCrossAccountServiceResponseTypes`
2023-03-15 09:37:40 +00:00
Waldemar Quevedo
beb179ec15 Check if connection name was already set when storing it
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-02-19 07:58:56 -08:00
peaaceChoi
038037381b Fix some typos in code comment 2023-01-12 10:31:32 +09:00
Sandy Kellagher
7907950a54 Fix for #3682: do not delay PINGs for GATEWAY or solicited LEAF connections, to ensure failover of leaf node connections 2022-12-07 16:19:33 +00:00
Derek Collison
fe57c687f6 Make sure header keys do not have additional prefixes.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-17 06:52:26 -08:00
Ivan Kozlovic
6113c52ae1 [FIXED] Solicited route may not retry to reconnect
Originally, only solicited routes were retried in case of a disconnect,
but that was before gossip protocol was introduced. Since then, two
servers that connect to each other due to gossip should retry to
reconnect if the connection breaks, even if the route is not explicit.
However, server will retry only once or more accurately, ConnectRetries+1.

This PR solves the issue that the reconnect attempt was not initiated
for a "solicited route" that was not explicit.

Maybe related to #3571

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-24 10:39:23 -06:00
Ivan Kozlovic
dde94987ce [FIXED] MQTT: Subjects mapping were not handled
A simple configuration like this:
```
...
mappings = {
  foo: bar
}

mqtt {
   port: 1883
}
```
would cause an MQTT subscription on "bar" to not receive messages
published on "foo".

In otherwords, the subject transformation was not done when parsing
a PUBLISH packet.

This PR also handles the case of service imports where transformation
occurs after the initial publish parsing.

Resolves #3547

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-13 16:00:05 -06:00
Ivan Kozlovic
3358247e6b Added warning if internal sub callback takes too long
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-10 14:39:37 -06:00
Ivan Kozlovic
c3adf78702 [FIXED] Stack overflow when account imports its own export
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-10 13:20:26 -06:00
Ivan Kozlovic
8d9c57ad44 [IMPROVED] Fan-out performance
There was an observed degradation (around 5%) for large fan out in
v2.9.0 compared to earlier release. This is because we added
accounting of the in/out messages for the account, which result
in 4 atomic operations, 2 for in and 2 for out, however, it means
that for a fan-out of say 100 matching subscriptions, it is now
2 + 2 * 100 = 202.

This PR rework how the stats accounting is done which removes
the regression and even boost a bit the numbers since we are
doing the server stats update as an aggregate too.

There are still degradation for queues and no-sub at all that
need to be looked at.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-09-27 19:43:32 -06:00
Caleb Lloyd
3babdda3bb [FIXED] Format protocol error []byte with %q
Protocol errors print arguments that contain arbitrary []byte
and are possibly not formattable strings; use %q to escape

Signed-off-by: Caleb Lloyd <caleb@synadia.com>
2022-09-26 13:52:56 -04:00
Ivan Kozlovic
a8318d1f62 [FIXED] Service import/export cycles causing stack overflow
There was a way to detect a cycle but I believe it needs to be
a stack of "si" not just the one before invoking processServiceImport.

Changes in #3393 would solve issue reported with test TestAccountImportCycle,
but would not address the new reported issue represented by new test
TestLeafNodeSvcImportExportCycle. This current approach seems to solve
all known cases.

Resolves #3397
Replaces #3393
2022-08-26 14:40:02 -06:00
Derek Collison
5690059dac Reserve a system queue group
Signed-off-by: Derek Collison <derek@nats.io>
2022-07-06 13:16:13 -07:00
Derek Collison
e6479dafd2 Close leafnode connection when same cluster name detected
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-30 15:34:22 -07:00
Derek Collison
abc5905aa9 Merge pull request #3221 from nats-io/direct
Made direct get from a stream part of the $JS.API hierarchy vs separate.
2022-06-28 09:59:44 -07:00
Matthias Hanel
aba1da090b [ADD] account specific in/out msgs/bytes stats to CONNS (#3187)
* [ADD] account specific in/out msgs/bytes stats to CONNS

This subject $SYS.ACCOUNT.%s.SERVER.CONNS will now respond with account
specific datastats for Received and sent messages as well as number of slow
consumers for the account.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-06-28 18:59:29 +02:00
Derek Collison
b8ef9b19a0 Made direct get from a stream part of the $JS.API hierarchy vs separate.
Also for direct get and for pull requests, if we are not on a client connection check how long we have been away from the readloop.
If need be execute in a separate go routine.

Signed-off-by: Derek Collison <derek@nats.io>
2022-06-28 08:53:48 -07:00
Derek Collison
830d659aec Do not default to sharing all client info for JetStream, only server and cluster info.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-26 07:11:02 -07:00
Derek Collison
af43dd3c74 When internal system messages were destined for a queue subscriber across a route the reply subject would be empty but not nil which caused mangling of the RMSG proto.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-11 14:12:05 -07:00
R.I.Pienaar
52a1c542f5 export the correct subject transformer subject
While the TransformSubject function was doing the right
thing it did not match first and so would panic for subjects
that do not match the mapping.

The map function does the right thing so this is a more
appropriate function to export.

This undoes the exporting of unsafe TransformSubject and
exports the safer Match instead.

Signed-off-by: R.I.Pienaar <rip@devco.net>
2022-06-02 18:26:12 +02:00
R.I.Pienaar
dc9d6776f8 Export the subject transformer
This exports the one key function of the subject transformer
allowing external tools to be written to test mappings are
valid and see how they would interact without the hassle of
configuring a serrver

The APIs are specifically marked as being unsupported and
having kept the transform struct itself unexported one can
not cast from the interface to the real implementation

Signed-off-by: R.I.Pienaar <rip@devco.net>
2022-05-27 10:33:59 +02:00
Ivan Kozlovic
c4adf0ffed [FIXED] Accounts Export/Import isolation with overlap subjects
I tracked down this issue to have been introduced with PR #2369,
but the code also touched PR #1891 and PR #3088.

I added a test as described in issue #3108 but did not need
JetStream to demonstrate the issue. With the proposed fix, all
tests that were added in aforementioned PRs still pass, including
the new test.

Resolves #3108

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-05-09 12:59:12 -06:00
Derek Collison
806877ebaa Merge pull request #3090 from nats-io/qsub-deny
Combined canSubscribe and canQueueSubscribe
2022-04-29 09:20:57 -07:00
Derek Collison
c20b52251b Combined canSubscribe and canQueueSubscribe for consistency in specialized deny clause handling.
Signed-off-by: Derek Collison <derek@nats.io>
2022-04-29 09:18:45 -07:00
Derek Collison
e0f5fcffb8 Fix for subject transforms and JetStream delivery subjects.
Signed-off-by: Derek Collison <derek@nats.io>
2022-04-28 15:50:28 -07:00
Matthias Hanel
254c970876 Fix subject renaming for leaf connections and queue subs (#3062)
* [fix] on queue sub, a consumers  delivery subject, was not changed

to the original publish subject the stream received
the code added is a copy of what regular subs do

* [fixed] subject renaming for leaf node connections as well

also updated multi server test to test for queue and non queue scenarios

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-04-20 19:23:21 -04:00
Matthias Hanel
0f113aa3d5 [FIXED] subject renaming with hand crafted reply subject (#3026)
do so by rejecting jsackprefix in reply subjects

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-04-11 22:32:02 -04:00
Matthias Hanel
02d25cc640 [FIXED] Consumer deliver subject incorrect when imported and crossing gateway (#3025)
followup to #3017

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-04-11 20:27:25 -04:00
Derek Collison
cd7f16f28a Tweak timing for test to prevent flapping
Signed-off-by: Derek Collison <derek@nats.io>
2022-04-09 11:13:49 -07:00
Derek Collison
331c2faaa6 When using a stream import for a push consumer's messages, if the message crossed a route we dropped the delivered subject.
Signed-off-by: Derek Collison <derek@nats.io>
2022-04-09 06:42:22 -07:00
Ivan Kozlovic
7fa2676353 Fixed comment typos and some rewording
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-04-07 09:22:51 -06:00
Ivan Kozlovic
9e6f965913 [ADDED] LeafNode min_version new option
If set, a server configured to accept leafnode connections will
reject a remote server whose version is below that value. Note
that servers prior to v2.8.0 are not sending their version
in the CONNECT protocol, which means that anything below 2.8.0
would be rejected.

Configuration example:
```
leafnodes {
    port: 7422
    min_version: 2.8.0
}
```
The option is a string and can have the "v" prefix:
```
min_version: "v2.9.1"
```
Note that although suffix such as `-beta` would be accepted,
only the major, minor and update are used for the version comparison.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-04-06 18:40:33 -06:00
Ivan Kozlovic
366d217f44 Some changes based on review
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-04-01 17:55:33 -06:00