Commit Graph

7405 Commits

Author SHA1 Message Date
Waldemar Quevedo
2b2fbf7359 Bump to v2.9.22-beta.1
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-22 13:37:12 -07:00
Waldemar Quevedo
9d11988465 Fix JSON compatibility in conf format (#4418)
Fixes a side effect from #4394 which was breaking support for JSON
configs.
2023-08-22 13:08:41 -07:00
Waldemar Quevedo
62242a7215 Fix JSON compatibility in conf format
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-22 12:47:51 -07:00
Derek Collison
84536761a9 Bump to 2.9.22-beta
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-22 08:27:44 -07:00
Derek Collison
90f5371a4c [FIXED] R1 stream move would sometimes lose all msgs. (#4413)
When moving streams, we could check too soon and be in a gap where the
replica peer has not registered a catchup request but had made contact
via the NRG layer.

This would cause us to think the replica was caught up, incorrectly, and
drop our leadership, which would cancel any catchup requests.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-22 06:49:57 -07:00
Derek Collison
e5d208bf33 When moving streams, we could check too soon and be in a gap where the replica peer has not registered a catchup request.
This would cause us to think the replica was caughtup incorrectly and drop our leadership, which would cancel any cacthup requests.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 20:07:48 -07:00
Derek Collison
2fc3f45ea1 [FIXED] Durable pull consumers could get cleaned up incorrectly on leader change. (#4412)
Fix for a bug that would allow old leaders of pull based durables to
delete a consumer from an inactivity threshold timer inadvertently.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 15:35:44 -07:00
Derek Collison
6e3ae20650 [FIXED] Fixed deadlock when checkAndSync was being called as part of storing message (#4411)
We violated the locking pattern, so we now make sure we do this in a
separate Go routine and put checks to only run it once.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 15:28:58 -07:00
Derek Collison
0a86bf4a9a Should reset to false, not true when done
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 14:57:17 -07:00
Derek Collison
43314fd439 Fix for a bug that would allow old leaders of pull based durables to delete a consumer from an inactivity threshold.
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 14:53:09 -07:00
Neil
3377f04b00 Send shutdown event on LDM (#4405)
If we send an event when entering lame duck mode, other nodes will mark
the server as offline immediately, therefore R1 assets will not be
placed onto that node. This is not a problem with R3 or higher because
an LDM server operates as a Raft observer only and therefore cannot take
the leadership role from an election, but R1 assets can in theory be
placed onto any node that is not marked as offline.

A final shutdown event will still be sent when the server actually shuts
down so there is no change there.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-21 22:29:30 +01:00
Neil Twigg
d720a6931c Use own subject for LDM event
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-21 22:03:26 +01:00
Neil Twigg
7cc5838a6d Send shutdown event on LDM so that R1 assets do not get assigned to the LDM node
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-21 21:29:01 +01:00
Derek Collison
10f73e888e Remove 1.18 compile build, support 1.19 and above
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 12:12:43 -07:00
Derek Collison
e018705a08 Fixed deadlock when checkAndSync was being called as part of storing message.
We violated the locking pattern, so we now make sure we do this in a separate Go routine and put checks to only run it once.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-21 12:12:36 -07:00
Waldemar Quevedo
0712628098 Standardize issue forms (#4408) 2023-08-21 06:16:38 -07:00
Byron Ruth
a43075cabc Address typos
Signed-off-by: Byron Ruth <byron@nats.io>
2023-08-19 06:33:19 -04:00
Byron Ruth
ca79ac9a73 Update issue forms
Signed-off-by: Byron Ruth <byron@nats.io>
2023-08-18 21:39:57 -04:00
Neil
ff688ab8ec Tweak consumer replica scaling (#4404)
This should hopefully catch some consumer scaling situations more
reliably, including cases where the consumer filter subjects no longer
match those of the stream after being scaled down to R1 or after a
cluster restart. I've also added a test to test whether filtered
consumers will scale properly even when the stream subject orphans them.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-18 09:36:29 +01:00
Neil Twigg
c437157c1f Recover in consumer assignment when asset already existed
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-17 23:22:10 +01:00
Neil Twigg
3c85490dc0 Backport test helper tweak
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-17 15:28:48 +01:00
Neil Twigg
c0636d117f Tweak consumer replica scaling, add unit test for orphaned consumer subjects
Signed-off-by: Neil Twigg <neil@nats.io>
2023-08-17 15:27:29 +01:00
Waldemar Quevedo
1e87c3d820 config: allow empty configs, but prevent bad configs (#4394)
Allows again empty configs and improves support for detecting some types
of invalid configs:

- Adds reporting the line with the bad key position that makes the
config invalid.

```
nats-server: config is invalid (foo.conf:1:2)
nats-server: error parsing include file 'included.conf', config is invalid (included.conf:2:2)
```

- Fixes a few tests with trailing braces which were being handled as
keys and ignored before.
2023-08-15 06:11:17 -07:00
Waldemar Quevedo
3a20f66535 config: parsed empty config only show warnings
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-13 23:59:50 -07:00
Waldemar Quevedo
412dee67f1 config: allow empty configs, but prevent bad configs
- Adds reporting the line with the bad key position
  that makes the config invalid.

- Fixes a few tests with trailing braces which were
  being handled as keys and ignored before.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-13 23:13:32 -07:00
Derek Collison
f2c7a9d37f Fix for flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-04 12:02:59 -07:00
Derek Collison
b2e7725aed Release v2.9.21 (#4368) 2023-08-04 07:46:27 -07:00
Byron Ruth
c1d1f11a18 Release v2.9.21
Signed-off-by: Byron Ruth <byron@nats.io>
2023-08-04 10:11:06 -04:00
Derek Collison
8c6055babc Bump to 2.9.21-RC.6
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-03 13:25:14 -07:00
Derek Collison
087e14782d [IMPROVED] Also reset clseq to avoid possible immediate sequence mismatch (#4366)
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-03 13:24:24 -07:00
Derek Collison
cbe85c826a Also reset clseq to avoid immediate sequence mismatch
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-03 12:40:17 -07:00
Derek Collison
d522f4656c Bump to 2.9.21-RC.5
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-03 11:17:51 -07:00
Derek Collison
34199ab6a8 [IMPROVED] When taking over make sure to sync and reset clfs for clustered streams. (#4365)
If the failed state of clfs drifts between leaders and followers,
replicas could discard and skip messages possibly incorrectly. This will
force sync if we have a non-zero clfs state when a leader takes over.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-03 11:17:03 -07:00
Derek Collison
66a8e81d49 Bump Go to 1.19.12 (#4364) 2023-08-03 10:45:39 -07:00
Derek Collison
081140ee67 When taking over make sure to sync and reset clfs for clustered streams.
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-03 10:41:10 -07:00
Byron Ruth
af52adb1ee Bump Go to 1.19.12
Signed-off-by: Byron Ruth <byron@nats.io>
2023-08-03 11:24:58 -04:00
Derek Collison
9de5e3e64d OCSP backports and adds (#4362)
This PR backports the OCSP Peer feature option (as in 2.10 train) and
includes two fixes for the existing OCSP Staple feature.

OCSP Staple: 

1. Fixed and clarified how NATS Server determines its own Issuer CA when
obtaining and validating an OCSP Response for subsequent staple
2. Eliminated problematic assumption that all node peers are issued by
same CA when NATS Server validates ROUTE and GATEWAY peer nodes
3. Added OCSP Response effectivity checks on ROUTE and GATEWAY
peer-presented staple

Note for #3: Allowed host clock skew between node peers set at
30-seconds. If the OCSP Response contains an empty assertion for
NextUpdate, NATS Server will default to 1-hour validity (after
ThisUpdate). It is recommended that CA OCSP Responder should assert
NextUpdate.
2023-08-02 18:10:24 -07:00
Todd Beets
ac43a8d4eb Enhance OCSP peer validation for GATEWAY and ROUTE connections. Nodes no longer required to have same CA issuer. OCSP response effectivity now checked using default clock skew and default validity period if not asserted by responder. 2023-08-02 16:09:21 -07:00
Todd Beets
1f0b70d5fc Fixed local issuer determination for OCSP Staple, issue #3773 2023-08-02 11:52:36 -07:00
Todd Beets
209fcd70eb OCSP Peer Feature 2023-08-02 11:25:48 -07:00
Derek Collison
5577d18c67 Fix some system service imports going missing after reload (#4360)
On reload some of the imports from the system account where going
missing on reload, this adds them back after a reload:

```
$SYS.REQ.SERVER.PING.CONNZ
$SYS.REQ.ACCOUNT.PING.STATZ
$SYS.REQ.ACCOUNT.PING.CONNZ
```
2023-08-02 10:14:07 -07:00
Waldemar Quevedo
23b5cb959c review fixes
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-02 08:51:08 -07:00
Waldemar Quevedo
2b252469ca fix: add missing default service imports on reload
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-01 23:34:07 -07:00
Waldemar Quevedo
13cb62e0bf Add test checking subscriptions before/after reload
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2023-08-01 22:51:42 -07:00
Waldemar Quevedo
7c9a91fc91 config: make parsing configurations without usable values invalid (backport) (#4358)
This makes configuration files that are empty, or read and processed by
the parser but with no detected values now return an error.

Fixes #4343 
Backport from dev branch
(https://github.com/nats-io/nats-server/pull/4347)
2023-08-01 22:33:30 -07:00
Waldemar Quevedo
ba596f7de1 config: make parsing configurations without usable values invalid
This makes configuration files that are empty, or read and processed
by the parser but with no detected values now return an error.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-01 21:55:09 -07:00
Derek Collison
aa6ac2d665 Bump to 2.9.21-RC.4
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 21:48:02 -07:00
Derek Collison
09ab23c929 [IMPROVED] Publish performance with lots of no interest subjects (#4359)
Do not hold onto no interest subjects from a client in the unlocked cache.
If sending lots of different subjects all with no interest performance could be affected.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4341
2023-08-01 21:43:23 -07:00
Derek Collison
787b0d922f Do not hold onto no interest subjects from a client in the unlocked cache.
If sending lots of different subjects all with no interest performance could be affected.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 21:20:58 -07:00
Derek Collison
09e78a3349 [FIXED] Accounting drifts (#4357)
Three issues were found and resolved.

    1. Some purge replays after recovery  could execute full purge.
    2. Callback was registered without lock, which  could lead to skew.
3. Cluster reset could stop stream store and recreate it, which could
lead to double accounting.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 20:07:14 -07:00