Commit Graph

8034 Commits

Author SHA1 Message Date
Waldemar Quevedo
dc09bb764a Bump to v2.9.22-beta.1 (#4419) 2023-08-22 13:49:37 -07:00
Waldemar Quevedo
2b2fbf7359 Bump to v2.9.22-beta.1
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-22 13:37:12 -07:00
Waldemar Quevedo
9d11988465 Fix JSON compatibility in conf format (#4418)
Fixes a side effect from #4394 which was breaking support for JSON
configs.
2023-08-22 13:08:41 -07:00
Waldemar Quevedo
62242a7215 Fix JSON compatibility in conf format
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-22 12:47:51 -07:00
Waldemar Quevedo
baa2805de9 Fix discarding explicit routes while removing duplicate ones (#4414)
In the new clustering logic for v2.10, sometimes the `TestStressChainedSolicitWorks` 
test would flake because a node would end up with only implicit routes. In this change, 
we stamp that one of the remotes is configured so that the nodes at least have one explicit
configured remote node.
2023-08-22 08:50:35 -07:00
Waldemar Quevedo
6d0fdf5b6c Update LastActivity on connect for routes (#4415)
Sometimes LastActivity can be left uninitialized while creating the
cluster.
2023-08-22 08:49:28 -07:00
Derek Collison
84536761a9 Bump to 2.9.22-beta
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-22 08:27:44 -07:00
Waldemar Quevedo
bdb874a6a8 Update LastActivity on connect for routes
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-22 07:10:30 -07:00
Derek Collison
bcf5da04e3 Merge branch 'main' into dev 2023-08-22 06:50:36 -07:00
Derek Collison
90f5371a4c [FIXED] R1 stream move would sometimes lose all msgs. (#4413)
When moving streams, we could check too soon and be in a gap where the
replica peer has not registered a catchup request but had made contact
via the NRG layer.

This would cause us to think the replica was caught up, incorrectly, and
drop our leadership, which would cancel any catchup requests.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-22 06:49:57 -07:00
Derek Collison
e5d208bf33 When moving streams, we could check too soon and be in a gap where the replica peer has not registered a catchup request.
This would cause us to think the replica was caughtup incorrectly and drop our leadership, which would cancel any cacthup requests.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 20:07:48 -07:00
Derek Collison
e088583cd3 Bump to 2.10.0-beta.50
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 15:59:53 -07:00
Derek Collison
dd4cdfd2fd Specify latest go 1.19 version
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 15:59:30 -07:00
Derek Collison
f0e2765b44 Fixes for merge conflicts from main
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 15:55:31 -07:00
Derek Collison
fb8525b713 Merge branch 'main' into dev
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 15:55:00 -07:00
Derek Collison
2fc3f45ea1 [FIXED] Durable pull consumers could get cleaned up incorrectly on leader change. (#4412)
Fix for a bug that would allow old leaders of pull based durables to
delete a consumer from an inactivity threshold timer inadvertently.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 15:35:44 -07:00
Derek Collison
6e3ae20650 [FIXED] Fixed deadlock when checkAndSync was being called as part of storing message (#4411)
We violated the locking pattern, so we now make sure we do this in a
separate Go routine and put checks to only run it once.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 15:28:58 -07:00
Waldemar Quevedo
673f654fbe Fix discarding explicit routes while removing duplicate ones
In the new clustering logic sometimes the TestStressChainedSolicitWorks test
would fail because the a node would end up with only implicit routes.
In this change, we stamp that one of the remotes is configured so that the nodes
at least have one explicit configured remote node.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-21 15:16:08 -07:00
Derek Collison
0a86bf4a9a Should reset to false, not true when done
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 14:57:17 -07:00
Derek Collison
43314fd439 Fix for a bug that would allow old leaders of pull based durables to delete a consumer from an inactivity threshold.
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 14:53:09 -07:00
Neil
3377f04b00 Send shutdown event on LDM (#4405)
If we send an event when entering lame duck mode, other nodes will mark
the server as offline immediately, therefore R1 assets will not be
placed onto that node. This is not a problem with R3 or higher because
an LDM server operates as a Raft observer only and therefore cannot take
the leadership role from an election, but R1 assets can in theory be
placed onto any node that is not marked as offline.

A final shutdown event will still be sent when the server actually shuts
down so there is no change there.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-21 22:29:30 +01:00
Neil Twigg
d720a6931c Use own subject for LDM event
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-21 22:03:26 +01:00
Neil Twigg
7cc5838a6d Send shutdown event on LDM so that R1 assets do not get assigned to the LDM node
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-21 21:29:01 +01:00
Derek Collison
10f73e888e Remove 1.18 compile build, support 1.19 and above
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 12:12:43 -07:00
Derek Collison
e018705a08 Fixed deadlock when checkAndSync was being called as part of storing message.
We violated the locking pattern, so we now make sure we do this in a separate Go routine and put checks to only run it once.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 12:12:36 -07:00
Waldemar Quevedo
0712628098 Standardize issue forms (#4408) 2023-08-21 06:16:38 -07:00
Neil
4886f1fec4 [IMPROVED] StreamInfo reflecting subject transforms with just a filter and no transformation for Sources (#4403)
- [X] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [x] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

Adds sfs to sourceInfo such that transforms with just a subject filter
(and no transformation, meaning that the transform pointer in streamInfo
is nil) can still be reflected in SourceInfo, which is important since
the filtering is still happening, just no transformation as well.
2023-08-21 09:44:54 +01:00
Jean-Noël Moyne
62f62d4071 Adds sfs to sourceInfo
Adds sfs to SourceInfo such that transforms with just a subject filter (and no transformation, meaning that the transform pointer in streamInfo is nil) can still be reflected in SourceInfo, which is important since the filtering is still happening, just no transformation as well.

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-08-19 12:26:42 -07:00
Byron Ruth
a43075cabc Address typos
Signed-off-by: Byron Ruth <byron@nats.io>
2023-08-19 06:33:19 -04:00
Byron Ruth
ca79ac9a73 Update issue forms
Signed-off-by: Byron Ruth <byron@nats.io>
2023-08-18 21:39:57 -04:00
Neil
ff688ab8ec Tweak consumer replica scaling (#4404)
This should hopefully catch some consumer scaling situations more
reliably, including cases where the consumer filter subjects no longer
match those of the stream after being scaled down to R1 or after a
cluster restart. I've also added a test to test whether filtered
consumers will scale properly even when the stream subject orphans them.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-18 09:36:29 +01:00
Neil Twigg
c437157c1f Recover in consumer assignment when asset already existed
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-17 23:22:10 +01:00
Neil Twigg
3c85490dc0 Backport test helper tweak
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-17 15:28:48 +01:00
Neil Twigg
c0636d117f Tweak consumer replica scaling, add unit test for orphaned consumer subjects
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-17 15:27:29 +01:00
Neil
3f28de8e83 Don't set block profile rate (#4402)
We rarely benefit from block profiles and in many cases a mutex profile
will tell us what we need to know. Additionally, setting the block
profile rate to `1` has the special meaning of capturing every single
blocking event, which can have a fairly significant negative impact on
publish performance.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-16 19:11:49 +01:00
Neil Twigg
19397a5683 Don't set block profile rate
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-16 17:00:07 +01:00
Neil
7670cf581c Fix stream config update of source transforms (#4400)
- [x] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

Fixes potential out of range access during some stream source transform
configuration updates and tiny clean up
Fixes stream sourcing message header parsing for multi-subject transform
in sources
2023-08-16 09:39:12 +01:00
Jean-Noël Moyne
0cc43acb84 Fix Nats-Stream-Source header parsing when using multi-filter transforms
Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-08-15 19:22:09 -07:00
Jean-Noël Moyne
c2d3ef1021 Fix potential out of range for stream source transform update.
Clean up un-needed if statement as it's ok to call NewSubjectTransform with an empty destination (ie no transformation) it will return nil

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-08-15 16:35:19 -07:00
Neil
c2d1e6d051 Add some jitter to leafnode remotes reconnect (#4398)
This adds a jitter delay based on the reconnect delay for when a remote
reconnects.
2023-08-15 17:44:40 +01:00
Waldemar Quevedo
740e5ddc37 Add some jitter to leafnode remotes reconnect
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-15 07:36:37 -07:00
Waldemar Quevedo
1e87c3d820 config: allow empty configs, but prevent bad configs (#4394)
Allows again empty configs and improves support for detecting some types
of invalid configs:

- Adds reporting the line with the bad key position that makes the
config invalid.

```
nats-server: config is invalid (foo.conf:1:2)
nats-server: error parsing include file 'included.conf', config is invalid (included.conf:2:2)
```

- Fixes a few tests with trailing braces which were being handled as
keys and ignored before.
2023-08-15 06:11:17 -07:00
Neil
8717b050e9 [IMPROVED] $SYS.REQ.SERVER.PING.PROFILEZ always honored (#4393) 2023-08-14 22:03:46 +01:00
Jean-Noël Moyne
61a0555336 Call SetBlockProfileRate even it the profiling port is not set
Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-08-14 10:58:20 -07:00
R.I.Pienaar
3cc3037c9b Missing json tag (#4395)
Minor update to server info structures
2023-08-14 20:42:58 +03:00
R.I.Pienaar
1d916ef9c7 Adds a missing json encoding tag
Signed-off-by: R.I.Pienaar <rip@devco.net>
2023-08-14 17:41:02 +03:00
Waldemar Quevedo
3a20f66535 config: parsed empty config only show warnings
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-13 23:59:50 -07:00
Waldemar Quevedo
412dee67f1 config: allow empty configs, but prevent bad configs
- Adds reporting the line with the bad key position
  that makes the config invalid.

- Fixes a few tests with trailing braces which were
  being handled as keys and ignored before.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-13 23:13:32 -07:00
Jean-Noël Moyne
40b8aa434b Remove part of the test that expects an error since now you can always get the profilez through the system account request
Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-08-13 18:00:08 -07:00
Jean-Noël Moyne
b839c53abc [ADDED] Full StreamSource (filters, transforms) functionality to stream mirror (#4354)
- [X] Tests added
- [X] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [x] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

Follow up to #4276 extending to Mirror the full StreamSource
functionality.

---------

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-08-12 15:17:48 -07:00