Commit Graph

5121 Commits

Author SHA1 Message Date
Derek Collison
75e8df13d7 Allows cluster filtering in account subject mapping (#4175)
- [X] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [ ] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

### Changes proposed in this pull request:

Allows (exposes) the existing cluster filtering option of account level
subject mapping, thereby allowing you to define different mapping
destination per cluster (e.g. to insert the cluster name as a subject
token).

Example:
```
mappings {
	"foo":[
		{destination:"foo-east", cluster: "east", weight:100},
		{destination:"foo-west", cluster: "west", weight:100},
	]
}
```
2023-05-17 17:07:24 -07:00
Derek Collison
fbfa593b3d Merge branch 'main' into dev 2023-05-17 16:48:47 -07:00
Derek Collison
7dfe5e528e Bump to 2.9.17-RC.3
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 16:46:10 -07:00
Derek Collison
94457e2d55 [IMPROVED] Reset logic for streams (#4177)
When we detect conditions to reset streams, make sure we properly clean
up old NRG nodes etc.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 16:45:00 -07:00
Derek Collison
a8d7d3886e Make sure to delete the stream assignment node here
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 16:19:39 -07:00
Derek Collison
44a5875968 Avoimd deadlock with usage lock for an account during checkAndSyncUsage().
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 16:05:46 -07:00
Jean-Noël Moyne
2b843ba9e3 Allows cluster filtering in account subject mapping
Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-05-17 13:34:35 -07:00
Derek Collison
f3553791b1 Updates to stream reset logic.
1. When catching up do not try forever and if needed reset cluster state.
2. In checking if a stream is healthy check for node drift.
3. When restarting a stream make sure the current node is stopped.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 13:14:33 -07:00
Derek Collison
5db57fb053 Bump to 2.9.17-RC.2
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-16 14:02:29 -07:00
Derek Collison
7760aa5107 Merge branch 'main' into dev 2023-05-16 14:01:57 -07:00
Ivan Kozlovic
06bc0fe8b8 [ADDED] LeafNode: Support for s2 compression (#4167)
This is similar to PR #4115 but for LeafNodes.
Compression mode can be set on both side (the accept and in remotes).
```
leafnodes {
   port: 7422
   compression: s2_best
   remotes [
       {
         url: "nats://host2:74222"
         compression: s2_better
       }
   ]
}
```
Possible modes are similar than for routes (described in PR #4115),
except that when not defined we default to `s2_auto`.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-16 14:34:25 -06:00
Derek Collison
a06e1c9b43 Make sure to also stop nodes when dealing with consumer after stream restart
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-16 13:16:47 -07:00
Ivan Kozlovic
e9e334d9fc Fix flapper
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-16 14:15:22 -06:00
Derek Collison
3752a6c500 Make sure to stop the node on a consumer restart if still running
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-16 12:49:46 -07:00
Ivan Kozlovic
cf474d6333 Revert changes related to leafnode PING interval
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-16 13:49:00 -06:00
Derek Collison
734895ae47 Fix test flapper
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-16 12:20:18 -07:00
Derek Collison
b0340ce598 Make sure to wait properly until we believe we are caught up to enable direct gets.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-16 11:02:06 -07:00
Ivan Kozlovic
ab281cc7e6 Updates based on PR feedback
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-16 10:18:11 -06:00
Savion
cd192f0e03 CHANGED - typo err 2023-05-16 16:41:52 +08:00
Ivan Kozlovic
67498af2dc [ADDED] LeafNode: Support for s2 compression
This is similar to PR #4115 but for LeafNodes.
Compression mode can be set on both side (the accept and in remotes).
```
leafnodes {
   port: 7422
   compression: s2_best
   remotes [
       {
         url: "nats://host2:74222"
         compression: s2_better
       }
   ]
}
```
Possible modes are similar than for routes (described in PR #4115),
except that when not defined we default to `s2_auto`.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-15 17:42:39 -06:00
Derek Collison
bca7b4ea44 Bump to 2.9.17-RC.1
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-15 15:45:48 -07:00
Derek Collison
4220502541 Merge branch 'main' into dev 2023-05-15 15:44:38 -07:00
Derek Collison
9434110c05 [FIXED] Additional fix for #3734. (#4166)
When the first block was truncated and missing any index info we would
not properly rebuild the state.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #3734
2023-05-15 15:40:12 -07:00
Waldemar Quevedo
ee38f8bbc5 monitor: change account detail info back to utc when served (#4163)
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-05-15 15:33:57 -07:00
Derek Collison
3602ff5146 Additional fix for #3734.
When the first block was truncated and missing any index info we would not properly rebuild the state.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-15 15:30:55 -07:00
Derek Collison
832df1cdba Protect against out of bounds access on usage updates.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-15 14:38:26 -07:00
Derek Collison
fe71ef524c [FIXED] Service imports reporting for Accountz() when mapping to local subjects. (#4158)
Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4144
2023-05-15 14:04:57 -07:00
Derek Collison
ea75beaeb1 [FIXED] Track all remote servers in a NATS system with different domains. (#4159)
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-15 13:47:06 -07:00
Jean-Noël Moyne
e07ccf9cc1 [ADDED] Ability to drop partial wildcard tokens in some subject transforms (#4152)
- [X] Tests added
- [X] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [X] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

### Changes proposed in this pull request:

There is currently a blanket requirement that subject transforms
destinations MUST use ALL of the partial wildcards defined in the source
of the transform. This is because the subject transformed defined for
imports must be 'reversible' and therefore the destination transform
must use all of the partial wildcard tokens defined in the source of the
transform.

This reversing of a transform is only used for transforms used by
imports, where in any case it doesn't make sense to use any transform
other than Wildcard.

This PR:
- relaxes this requirement to only apply when the transform is used by
an import, adding the ability to drop a wildcard token in transforms
other than as part of an import.
- Improves transform reverse to support both legacy style wildcards $X
and the new transform function {{Wildcard(X)}}- Improves reversible
transform checking to only allow the use of wildcards in the
destination.

---------

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-05-14 14:02:19 -07:00
Waldemar Quevedo
3c4ed549a5 resolver: improve signaling for missing account lookups (#4151)
When using the nats account resolver and a JWT is not found, the client could
often get an i/o timeout error due to not receiving a timely response
before the account resolver fetch request times out. Now instead
of waiting for the fetch request to timeout, a resolver without JWTs
will notify as well that it could not find a matching JWT, waiting for a
response from all active servers.

Also included in this PR is some cleanup to the logs emitted by the
resolver.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-05-14 11:10:25 -07:00
Derek Collison
75d274a636 If a NATS system has multiple domains make sure to process those during a remote update before bailing.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-13 18:36:42 -07:00
Derek Collison
d293af1da6 Fix to service imports reporting for Accountz() when import subject is mapped into different local subject.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-13 12:57:05 -07:00
Derek Collison
3ff9aed192 Merge branch 'main' into dev 2023-05-12 21:04:51 -07:00
Derek Collison
421775a32a Fix to allow sorting by rtt for connz.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 20:22:07 -07:00
Derek Collison
7f17e07d66 Filter by user at the end for closed connections
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 15:24:42 -07:00
Derek Collison
0c13f174c0 Fixed cap mistake in comment
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 15:07:00 -07:00
Derek Collison
c5eb46cb06 Make sure closed clients captures all user types and works with user filtering as well
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 15:05:40 -07:00
Derek Collison
90d1063674 Fix for #4149 to allow proper user filtering on connz for other user types.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 14:19:37 -07:00
Derek Collison
4c26cbb3de Merge branch 'main' into dev 2023-05-12 12:38:20 -07:00
Waldemar Quevedo
286a1632ca Use monotonic time for measuring time internally
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-05-12 08:27:46 -07:00
Derek Collison
5e029d08d5 For older R1 streams created by previous servers we could have no cluster for the stream assignment group which would prevent scale up with newer servers.
This will inherit cluster if detected from placement tags or client cluster designation.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-10 17:59:28 -07:00
Derek Collison
2f2498ab7e Bump to 2.9.17-beta.7
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-10 15:32:45 -07:00
Derek Collison
990ac56557 Merge branch 'main' into dev 2023-05-10 15:31:54 -07:00
Derek Collison
a17357c6ae When doing leadership transfer stepdown as soon as we know we have sent the EntryLeaderTransfer entry.
Delaying could allow something to be sent from the old leader which would cause the new leader to bail on being a candidate even though it would have gotten all the votes.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-10 12:27:33 -07:00
Derek Collison
717afae9ef When doing a leader transfer clear vote state on leader and when non-chosen peers receive the update
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-10 07:49:22 -07:00
Derek Collison
c5c5a34fec Bump to 2.9.17-beta.6
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-09 20:12:22 -07:00
Derek Collison
2f2440f270 Merge branch 'main' into dev 2023-05-09 20:11:53 -07:00
Derek Collison
b9af0d0294 Only do no-leader stepdown on transfer after a delay if we are still the leader
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-09 17:19:14 -07:00
Derek Collison
b44beb4b54 Make sure to update peer set and remove old peers after new leader takes over
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-09 15:15:02 -07:00
Neil Twigg
d7ae2cbb5f Backport #4120 to main
Signed-off-by: Neil Twigg <neil@nats.io>
2023-05-09 11:24:35 +01:00