Commit Graph

7516 Commits

Author SHA1 Message Date
Ivan Kozlovic
ab281cc7e6 Updates based on PR feedback
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-16 10:18:11 -06:00
Ivan Kozlovic
67498af2dc [ADDED] LeafNode: Support for s2 compression
This is similar to PR #4115 but for LeafNodes.
Compression mode can be set on both side (the accept and in remotes).
```
leafnodes {
   port: 7422
   compression: s2_best
   remotes [
       {
         url: "nats://host2:74222"
         compression: s2_better
       }
   ]
}
```
Possible modes are similar than for routes (described in PR #4115),
except that when not defined we default to `s2_auto`.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-15 17:42:39 -06:00
Jean-Noël Moyne
e07ccf9cc1 [ADDED] Ability to drop partial wildcard tokens in some subject transforms (#4152)
- [X] Tests added
- [X] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [X] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

### Changes proposed in this pull request:

There is currently a blanket requirement that subject transforms
destinations MUST use ALL of the partial wildcards defined in the source
of the transform. This is because the subject transformed defined for
imports must be 'reversible' and therefore the destination transform
must use all of the partial wildcard tokens defined in the source of the
transform.

This reversing of a transform is only used for transforms used by
imports, where in any case it doesn't make sense to use any transform
other than Wildcard.

This PR:
- relaxes this requirement to only apply when the transform is used by
an import, adding the ability to drop a wildcard token in transforms
other than as part of an import.
- Improves transform reverse to support both legacy style wildcards $X
and the new transform function {{Wildcard(X)}}- Improves reversible
transform checking to only allow the use of wildcards in the
destination.

---------

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-05-14 14:02:19 -07:00
Derek Collison
3ff9aed192 Merge branch 'main' into dev 2023-05-12 21:04:51 -07:00
Derek Collison
a982bbcb73 [FIXED] Allow sorting by rtt for connz. (#4157)
Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4150
2023-05-12 20:47:17 -07:00
Derek Collison
421775a32a Fix to allow sorting by rtt for connz.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 20:22:07 -07:00
Derek Collison
c31e710d9e [FIXED] Allow user filtering on connz for other user types like nkeys etc. (#4156)
Signed-off-by: Derek Collison <derek@nats.io>
 
Resolves #4149
2023-05-12 15:38:46 -07:00
Derek Collison
7f17e07d66 Filter by user at the end for closed connections
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 15:24:42 -07:00
Derek Collison
0c13f174c0 Fixed cap mistake in comment
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 15:07:00 -07:00
Derek Collison
c5eb46cb06 Make sure closed clients captures all user types and works with user filtering as well
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 15:05:40 -07:00
Derek Collison
90d1063674 Fix for #4149 to allow proper user filtering on connz for other user types.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 14:19:37 -07:00
Derek Collison
4c26cbb3de Merge branch 'main' into dev 2023-05-12 12:38:20 -07:00
Derek Collison
fc64c6119d Use monotonic time for measuring time internally (#4154)
- [x] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [x] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [x] Build is green in Travis CI
- [x] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)
2023-05-12 12:37:16 -07:00
Waldemar Quevedo
286a1632ca Use monotonic time for measuring time internally
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-05-12 08:27:46 -07:00
Neil
11e364d9c0 Tweak outbound queue memory test to only log if growth exceeds 10% (#4120)
This makes the `TestNoRaceClientOutboundQueueMemory` test useful and
stops unnecessary logging.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-05-12 12:34:24 +01:00
Derek Collison
bdb0ba9ae5 [FIXED] Can't scale up some older streams (#4146)
For some older R1 streams created by previous servers we could have no
cluster for the stream assignment group which would prevent scale up
with newer servers.

This will inherit cluster if detected as absent from the placement tags
or client cluster designation.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-10 21:42:59 -07:00
Derek Collison
5e029d08d5 For older R1 streams created by previous servers we could have no cluster for the stream assignment group which would prevent scale up with newer servers.
This will inherit cluster if detected from placement tags or client cluster designation.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-10 17:59:28 -07:00
Derek Collison
2f2498ab7e Bump to 2.9.17-beta.7
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-10 15:32:45 -07:00
Derek Collison
990ac56557 Merge branch 'main' into dev 2023-05-10 15:31:54 -07:00
Derek Collison
81bf92b2c6 [IMPROVED] Leadership transfer (#4145)
When doing leadership transfer stepdown as soon as we know we have sent
the EntryLeaderTransfer entry.

Delaying could allow something to be sent from the old leader which
would cause the new leader to bail on being a candidate even though it
would have gotten all the votes.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-10 15:30:38 -07:00
Derek Collison
a17357c6ae When doing leadership transfer stepdown as soon as we know we have sent the EntryLeaderTransfer entry.
Delaying could allow something to be sent from the old leader which would cause the new leader to bail on being a candidate even though it would have gotten all the votes.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-10 12:27:33 -07:00
Derek Collison
72485608d0 [IMPROVED] Leader transfer process (#4143)
When doing a leader transfer clear vote state on leader and when
non-chosen peers receive the update.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-10 08:24:15 -07:00
Derek Collison
717afae9ef When doing a leader transfer clear vote state on leader and when non-chosen peers receive the update
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-10 07:49:22 -07:00
Derek Collison
c5c5a34fec Bump to 2.9.17-beta.6
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-09 20:12:22 -07:00
Derek Collison
2f2440f270 Merge branch 'main' into dev 2023-05-09 20:11:53 -07:00
Derek Collison
b951cd155d Improvements on raft leader handoff. (#4142)
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-09 18:22:37 -07:00
Derek Collison
b9af0d0294 Only do no-leader stepdown on transfer after a delay if we are still the leader
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-09 17:19:14 -07:00
Derek Collison
b44beb4b54 Make sure to update peer set and remove old peers after new leader takes over
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-09 15:15:02 -07:00
Derek Collison
6e6ce3a6f6 Backport outbound queues test changes (#4120) to main (#4139)
This backports the changes to the outbound queues test to the `main`
branch.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-05-09 07:41:23 -07:00
Neil Twigg
d7ae2cbb5f Backport #4120 to main
Signed-off-by: Neil Twigg <neil@nats.io>
2023-05-09 11:24:35 +01:00
Derek Collison
16e9d69588 Fixed issue with route s2_auto when compression is actually off (#4137)
This is a fix for PR https://github.com/nats-io/nats-server/pull/4115.
If a server has an s2_auto configuration, the compression level needs to
be updated based on the RTT, however, this should not happen if a
particular route is actually not using compression, either because it is
a connection to an older server or the other side has explicitly
configure compression to be "off".

Extended a test that would have caught this issue.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-08 08:20:59 -07:00
Ivan Kozlovic
1ce12ba195 Fixed issue with route s2_auto when compression is actually off
This is a fix for PR https://github.com/nats-io/nats-server/pull/4001.
If a server has an s2_auto configuration, the compression level
needs to be updated based on the RTT, however, this should not
happen if a particular route is actually not using compression,
either because it is a connection to an older server or the other
side has explicitly configure compression to be "off".

Extended a test that would have caught this issue.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-08 09:06:16 -06:00
Derek Collison
650181a2ae Bump GH Actions versions (Go; node12 deprecation) (#4136)
Bump various GitHub Actions versions to move away from those using the
deprecated node.js 12 runtime, and so remove the warnings.

For the coveralls action, I switched to just `@v2`, per their example
docs and as we do for most actions, instead of major.minor.patch.

For Golang, update one 1.16 to 1.19, and quote the remaining unquoted
instances, even though they had `.x` in there so had to parse as
strings, because we're just switching to always-quoting for consistency.
2023-05-08 07:08:56 -07:00
Phil Pennock
97df36e7eb Bump GH Actions versions (Go; node12 deprecation)
Bump various GitHub Actions versions to move away from those using the
deprecated node.js 12 runtime, and so remove the warnings.

For the coveralls action, I switched to just `@v2`, per their example docs and
as we do for most actions, instead of major.minor.patch.

For Golang, update one 1.16 to 1.19, and quote the remaining unquoted
instances, even though they had `.x` in there so had to parse as strings,
because we're just switching to always-quoting for consistency.
2023-05-07 22:01:34 -04:00
Derek Collison
168fb22df5 Fix one more test that did not set ack policy to explicit
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-06 22:05:31 -07:00
Derek Collison
08938034a5 Fix test that did not set ack policy to explicit
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-06 19:54:42 -07:00
Derek Collison
ed3f8be0c5 Bump version 2.10.0-beta.36
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-06 18:49:13 -07:00
Derek Collison
18244ea8cb Fix test that did not set ack policy to explicit
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-06 15:10:46 -07:00
Derek Collison
caa262513d Fix test that did not set ack policy which is needed
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-06 14:15:44 -07:00
Derek Collison
dbff40f2b6 Adopt same update from main
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-06 09:56:01 -07:00
Derek Collison
4175e4ee9c Merge branch 'main' into dev 2023-05-06 09:55:34 -07:00
Derek Collison
76f4358349 [IMPROVED] Optimizations for large single hub account leafnode fleets. (#4135)
Added a leafnode lock to allow better traversal without copying of large
leafnodes in a single hub account.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-06 09:53:08 -07:00
Derek Collison
80db7a22ab Optimizations for large single hub account leafnode fleets.
Added a leafnode lock to allow better traversal without copying of large leafnodes in a single hub account.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-05 13:14:49 -07:00
Waldemar Quevedo
40ea58fc51 Stop using UTC for time in flushClients (#4132)
In #1943 it was adopted to use `UTC()` in some timestamps, but an
unintended side effect from this is that it strips the monotonic time
(e5646b23de),
so it can be prone to clock skews when subtracting time in other areas
of the code.
2023-05-04 17:35:50 -07:00
Waldemar Quevedo
b886fed2fb Stop using UTC for time for flushClients
In #1943 it was adopted to use `UTC()` in some timestamps,
but an unintended side effect from this is that it strips 
the monotonic time, so it can be prone to clock skews when
subtracting time in other areas of the code.
e5646b23de
2023-05-04 15:50:45 -07:00
Tomasz Pietrek
69fb3db0f5 Optimize consumer messages sequences for multiple subjects (#4129)
If consumer with multiple subjects encountered a sequnece of messages in
a row from the same subject, it tried to load messages from other
subjects in some cases.
This checks for that scenario and optimizes it by early returning.

I added a temporary instrumentation to check for how many times fetching
new messages is called, and it seems that it cuts those calls according
to assumptions. Though it being internal, it's really hard to show that
in test.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-05-04 20:13:13 +02:00
Tomasz Pietrek
7c1c4ea5fb Optimize consumer messages sequences for multiple subjects
If consumer with multiple subjects encountered a sequnece
of messages from the same subject, it tried to load messages
from other subjects in some cases.
This checks for that scenario and optimizes it by early returning.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-05-04 16:02:19 +02:00
Derek Collison
9fa724cd7b Merge branch 'main' into dev 2023-05-03 21:00:35 -07:00
Derek Collison
da8aeac91b Fix flapper
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-03 21:00:17 -07:00
Derek Collison
68f6b59fc7 Merge branch 'main' into dev 2023-05-03 19:51:24 -07:00