Commit Graph

7571 Commits

Author SHA1 Message Date
Jean-Noël Moyne
0bd3fa4a13 Harmonize subject mapping error variable names
Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-06-01 14:15:27 -07:00
Jean-Noël Moyne
08987bd173 1: Improves error reporting for weighted mappings:
As it was, any error in a weighted mapping would return a very unhelpfull error message.

e.g. `nats-server: mappingtest.cfg:38:39: interface conversion: interface {} is []interface {}, not string`

This was because the line `err := &configErr{tk, fmt.Sprintf("Error adding mapping for %q to %q : %v", subj, v.(string), err)}` would panic on the `v.(string)` since in weighted mapping that interface{} is actually a map[string]interface{} (since there's can be more than one mapping in weighted mappings).

Now returns the actual error:

e.g. `nats-server: mappingtest.cfg:40:3: Error adding mapping for "bla" : invalid mapping destination: wildcard index out of range in {{wildcard(1)}}`

2: improves subject transform checking and catches if the destination is using a mapping function and there are no partial wildcards in the source.
Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-05-31 16:28:31 -07:00
Derek Collison
24d4bd6039 [FIXED] LeafNode: data race during validation and create leafnode (#4194)
The issue really was that the test was sharing remote options. The way
options are used is not ideal since we reference the user provided
options (but it is relied upon now in many tests and possibly users
setups). The other side of the issue was that when no local account is
specified in a "remote" specification, we set it to the global account,
but that was done when creating the leafnode object (when soliciting),
which in the case of the test could race with the second server doing
the validation.

In this PR we move the setting to global account during the validation,
but also fixed the tests to not share the remote options configuration
slice between the two servers.

Resolves #4191

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-24 17:42:37 -07:00
Ivan Kozlovic
86a319a50e [FIXED] LeafNode: data race during validation and create leafnode
The issue really was that the test was sharing remote options. The
way options are used is not ideal since we reference the user provided
options (but it is relied upon now in many tests and possibly users
setups). The other side of the issue was that when no local account
is specified in a "remote" specification, we set it to the global
account, but that was done when creating the leafnode object (when
soliciting), which in the case of the test could race with the
second server doing the validation.

In this PR we move the setting to global account during the validation,
but also fixed the tests to not share the remote options configuration
slice between the two servers.

Resolves #4191

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-24 17:52:54 -06:00
Derek Collison
0db9d20383 When in async mode make sure to not flush or create new files after close. (#4193)
Some async tests were showing artifacts being left after a message block
was being closed.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-24 14:27:04 -07:00
Derek Collison
aac9135e0f When in async mode make sure to not flush or create new files after being closed.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-24 12:03:41 -07:00
Derek Collison
c9d6900600 Fixed cluster permissions configuration reload (#4183)
This is a rework of incorrect changes made in PR #4001. Changes did not
work for changes to export permissions. Test was modified to add export
changes.

This affects only the `dev` branch.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-19 13:11:29 -07:00
Ivan Kozlovic
7cf00c8ef7 Updates based on PR code review
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-19 12:16:24 -06:00
Ivan Kozlovic
607b0ca7f3 Fixed cluster permissions configuration reload
This is a rework of incorrect changes made in PR #4001.
This affects only the `dev` branch.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-18 19:02:03 -06:00
Derek Collison
646ae9eddf Merge branch 'main' into dev 2023-05-18 09:18:34 -07:00
Derek Collison
25d9762ce2 [IMPROVED] Make health checks more consistent with stream health checks. (#4180)
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-18 09:18:12 -07:00
Derek Collison
7e3f3f4908 Make health checks more consistent with stream health checks.
Check for closed state on leader change for consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-18 08:18:53 -07:00
Derek Collison
ee87df250c Merge branch 'main' into dev 2023-05-17 19:27:58 -07:00
Derek Collison
f63d63fbce [IMPROVED] Stepdown on catchup request for something newer than our state (#4179)
When we receive a catchup request for an item beyond our current state,
we should stepdown.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 19:25:05 -07:00
Derek Collison
4fbc0ee563 Update to Go 1.19.9 (#4178) 2023-05-17 18:01:58 -07:00
Byron Ruth
3a152a0e40 Update to Go 1.19.9
Signed-off-by: Byron Ruth <byron@nats.io>
2023-05-17 20:57:10 -04:00
Derek Collison
8e825001d2 When we receive a catchup request for an item beyond our current state, we should stepdown.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 17:30:35 -07:00
Derek Collison
75e8df13d7 Allows cluster filtering in account subject mapping (#4175)
- [X] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [ ] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

### Changes proposed in this pull request:

Allows (exposes) the existing cluster filtering option of account level
subject mapping, thereby allowing you to define different mapping
destination per cluster (e.g. to insert the cluster name as a subject
token).

Example:
```
mappings {
	"foo":[
		{destination:"foo-east", cluster: "east", weight:100},
		{destination:"foo-west", cluster: "west", weight:100},
	]
}
```
2023-05-17 17:07:24 -07:00
Derek Collison
fbfa593b3d Merge branch 'main' into dev 2023-05-17 16:48:47 -07:00
Derek Collison
7dfe5e528e Bump to 2.9.17-RC.3
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 16:46:10 -07:00
Derek Collison
93eaf8c814 Add workflow for stale issues (#4161)
This adds a workflow to mark issues and PRs stale after the configured
period of time, followed by closing the issue/PR after a subsequent
period of time if there was no additional activity.

The `debug-only` option is so currently, so even when merged, it will do
a dry-run and not perform any actions. Once we inspect the initial logs
of the effect of an initial run (impacting existing issues), we can
adjust accordingly and then follow-up with making it active.

For the debug logs to be enabled, we do need to add a repository secret
named `ACTIONS_STEP_DEBUG` with a value set to `true` per [this
instruction](https://github.com/marketplace/actions/close-stale-issues#debugging).
2023-05-17 16:45:17 -07:00
Derek Collison
94457e2d55 [IMPROVED] Reset logic for streams (#4177)
When we detect conditions to reset streams, make sure we properly clean
up old NRG nodes etc.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 16:45:00 -07:00
Derek Collison
b856bba285 [FIXED] Avoid deadlock with usage lock for an account during checkAndSyncUsage() (#4176)
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 16:44:44 -07:00
Derek Collison
a8d7d3886e Make sure to delete the stream assignment node here
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 16:19:39 -07:00
Derek Collison
44a5875968 Avoimd deadlock with usage lock for an account during checkAndSyncUsage().
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 16:05:46 -07:00
Jean-Noël Moyne
2b843ba9e3 Allows cluster filtering in account subject mapping
Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-05-17 13:34:35 -07:00
Derek Collison
f3553791b1 Updates to stream reset logic.
1. When catching up do not try forever and if needed reset cluster state.
2. In checking if a stream is healthy check for node drift.
3. When restarting a stream make sure the current node is stopped.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-17 13:14:33 -07:00
Derek Collison
5db57fb053 Bump to 2.9.17-RC.2
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-16 14:02:29 -07:00
Derek Collison
7760aa5107 Merge branch 'main' into dev 2023-05-16 14:01:57 -07:00
Derek Collison
ac68a19530 [IMPROVED] Restart consumer behavior during healthz() checks. (#4172)
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-16 13:58:47 -07:00
Ivan Kozlovic
06bc0fe8b8 [ADDED] LeafNode: Support for s2 compression (#4167)
This is similar to PR #4115 but for LeafNodes.
Compression mode can be set on both side (the accept and in remotes).
```
leafnodes {
   port: 7422
   compression: s2_best
   remotes [
       {
         url: "nats://host2:74222"
         compression: s2_better
       }
   ]
}
```
Possible modes are similar than for routes (described in PR #4115),
except that when not defined we default to `s2_auto`.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-16 14:34:25 -06:00
Derek Collison
a06e1c9b43 Make sure to also stop nodes when dealing with consumer after stream restart
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-16 13:16:47 -07:00
Ivan Kozlovic
e9e334d9fc Fix flapper
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-16 14:15:22 -06:00
Derek Collison
3752a6c500 Make sure to stop the node on a consumer restart if still running
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-16 12:49:46 -07:00
Ivan Kozlovic
cf474d6333 Revert changes related to leafnode PING interval
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-16 13:49:00 -06:00
Derek Collison
734895ae47 Fix test flapper
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-16 12:20:18 -07:00
Derek Collison
87f17fcff4 [FIXED] Avoid stale KV reads on server restart for replicated KV stores. (#4171)
Make sure to wait properly until we believe we are caught up to enable
direct gets on followers.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4162
2023-05-16 11:29:37 -07:00
Derek Collison
b0340ce598 Make sure to wait properly until we believe we are caught up to enable direct gets.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-16 11:02:06 -07:00
Ivan Kozlovic
ab281cc7e6 Updates based on PR feedback
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-16 10:18:11 -06:00
Derek Collison
4feb7b95a3 CHANGED - typo err (#4169)
Resolves # typo err

### Changes proposed in this pull request:

 - jetstream_api.go typo err
 - accounts.go typo err
 
/cc @nats-io/core
2023-05-16 07:20:19 -07:00
Savion
cd192f0e03 CHANGED - typo err 2023-05-16 16:41:52 +08:00
Ivan Kozlovic
67498af2dc [ADDED] LeafNode: Support for s2 compression
This is similar to PR #4115 but for LeafNodes.
Compression mode can be set on both side (the accept and in remotes).
```
leafnodes {
   port: 7422
   compression: s2_best
   remotes [
       {
         url: "nats://host2:74222"
         compression: s2_better
       }
   ]
}
```
Possible modes are similar than for routes (described in PR #4115),
except that when not defined we default to `s2_auto`.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-15 17:42:39 -06:00
Derek Collison
bca7b4ea44 Bump to 2.9.17-RC.1
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-15 15:45:48 -07:00
Derek Collison
4220502541 Merge branch 'main' into dev 2023-05-15 15:44:38 -07:00
Derek Collison
9434110c05 [FIXED] Additional fix for #3734. (#4166)
When the first block was truncated and missing any index info we would
not properly rebuild the state.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #3734
2023-05-15 15:40:12 -07:00
Waldemar Quevedo
ee38f8bbc5 monitor: change account detail info back to utc when served (#4163)
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-05-15 15:33:57 -07:00
Derek Collison
3602ff5146 Additional fix for #3734.
When the first block was truncated and missing any index info we would not properly rebuild the state.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-15 15:30:55 -07:00
Derek Collison
584ea85d75 [FIXED] Protect against out of bounds access on usage updates. (#4164)
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-15 14:58:05 -07:00
Derek Collison
832df1cdba Protect against out of bounds access on usage updates.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-15 14:38:26 -07:00
Derek Collison
fe71ef524c [FIXED] Service imports reporting for Accountz() when mapping to local subjects. (#4158)
Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4144
2023-05-15 14:04:57 -07:00