Commit Graph

7370 Commits

Author SHA1 Message Date
Byron Ruth
af52adb1ee Bump Go to 1.19.12
Signed-off-by: Byron Ruth <byron@nats.io>
2023-08-03 11:24:58 -04:00
Derek Collison
9de5e3e64d OCSP backports and adds (#4362)
This PR backports the OCSP Peer feature option (as in 2.10 train) and
includes two fixes for the existing OCSP Staple feature.

OCSP Staple: 

1. Fixed and clarified how NATS Server determines its own Issuer CA when
obtaining and validating an OCSP Response for subsequent staple
2. Eliminated problematic assumption that all node peers are issued by
same CA when NATS Server validates ROUTE and GATEWAY peer nodes
3. Added OCSP Response effectivity checks on ROUTE and GATEWAY
peer-presented staple

Note for #3: Allowed host clock skew between node peers set at
30-seconds. If the OCSP Response contains an empty assertion for
NextUpdate, NATS Server will default to 1-hour validity (after
ThisUpdate). It is recommended that CA OCSP Responder should assert
NextUpdate.
2023-08-02 18:10:24 -07:00
Todd Beets
ac43a8d4eb Enhance OCSP peer validation for GATEWAY and ROUTE connections. Nodes no longer required to have same CA issuer. OCSP response effectivity now checked using default clock skew and default validity period if not asserted by responder. 2023-08-02 16:09:21 -07:00
Todd Beets
1f0b70d5fc Fixed local issuer determination for OCSP Staple, issue #3773 2023-08-02 11:52:36 -07:00
Todd Beets
209fcd70eb OCSP Peer Feature 2023-08-02 11:25:48 -07:00
Derek Collison
5577d18c67 Fix some system service imports going missing after reload (#4360)
On reload some of the imports from the system account where going
missing on reload, this adds them back after a reload:

```
$SYS.REQ.SERVER.PING.CONNZ
$SYS.REQ.ACCOUNT.PING.STATZ
$SYS.REQ.ACCOUNT.PING.CONNZ
```
2023-08-02 10:14:07 -07:00
Waldemar Quevedo
23b5cb959c review fixes
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-02 08:51:08 -07:00
Waldemar Quevedo
2b252469ca fix: add missing default service imports on reload
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-01 23:34:07 -07:00
Waldemar Quevedo
13cb62e0bf Add test checking subscriptions before/after reload
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2023-08-01 22:51:42 -07:00
Waldemar Quevedo
7c9a91fc91 config: make parsing configurations without usable values invalid (backport) (#4358)
This makes configuration files that are empty, or read and processed by
the parser but with no detected values now return an error.

Fixes #4343 
Backport from dev branch
(https://github.com/nats-io/nats-server/pull/4347)
2023-08-01 22:33:30 -07:00
Waldemar Quevedo
ba596f7de1 config: make parsing configurations without usable values invalid
This makes configuration files that are empty, or read and processed
by the parser but with no detected values now return an error.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-01 21:55:09 -07:00
Derek Collison
aa6ac2d665 Bump to 2.9.21-RC.4
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 21:48:02 -07:00
Derek Collison
09ab23c929 [IMPROVED] Publish performance with lots of no interest subjects (#4359)
Do not hold onto no interest subjects from a client in the unlocked cache.
If sending lots of different subjects all with no interest performance could be affected.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4341
2023-08-01 21:43:23 -07:00
Derek Collison
787b0d922f Do not hold onto no interest subjects from a client in the unlocked cache.
If sending lots of different subjects all with no interest performance could be affected.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 21:20:58 -07:00
Derek Collison
09e78a3349 [FIXED] Accounting drifts (#4357)
Three issues were found and resolved.

    1. Some purge replays after recovery  could execute full purge.
    2. Callback was registered without lock, which  could lead to skew.
3. Cluster reset could stop stream store and recreate it, which could
lead to double accounting.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 20:07:14 -07:00
Derek Collison
5c8db89506 Make sure we do not drift on accounting.
Three issues were found and resolved.

1. Purge replays after recovery could execute full purge.
2. Callback was registered without lock, which could lead to skew.
3. Cluster reset could stop stream store and recreate it, which could lead to double accounting.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 18:35:20 -07:00
Derek Collison
97827be97d Call up to upper layers on Stop() for accounting purposes
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 17:36:40 -07:00
Derek Collison
b8d1ac9475 Allow long form resolver config to be of type MEM
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 17:34:14 -07:00
Derek Collison
039f63e260 Bump to 2.9.21-RC.3
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 13:41:30 -07:00
Derek Collison
5b7c029c64 [FIXED] Do not panic when re-calculating first sequence for SimpleState when fseq moves ahead of old first. (#4346)
When a lazy simple state has an outdated first that needs to be updated,
if fseq had moved past it would panic.
This was not common but with latest fix prior in can become more common,
hence why it showed up.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 13:41:01 -07:00
Derek Collison
4c14be9fbb When a lazy simple state has an outdated first that needs to be updated, if fseq had moved past it would panic.
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 13:09:37 -07:00
Derek Collison
7feebb4f64 Bump to 2.9.21-RC.2
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 09:55:45 -07:00
Derek Collison
c0fe4972f0 [FIXED] Stream recovery with corrupt msg block with sequence gaps. (#4344)
This is a fix for a bad msg blk detected in the field that had sequence
holes.

The stream had max msgs per subject of one and only one subject but had
lots of messages. The stream did not recover correctly, and upon further
inspection determined that a msg blk had holes, which should not be
possible.

We now detect the holes and deal with the situation appropriately.
Heavily tested on the data dump from the field.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 05:10:35 -07:00
Derek Collison
92430513d3 Add in const for msg record hash size
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 04:35:33 -07:00
Derek Collison
21e3d7a512 This is a fix for a bad msg blk detected in the field that had sequence holes.
The stream had max msgs per subject of one and only one subject but had lots of messages.
The stream did not recover correctly, and upon further inspection determined that a msg blk had holes, which should not be possible.

We now detect the holes and deal with the situation appropriately.
Heavily tested on the data dump from the field.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-27 21:43:15 -07:00
Derek Collison
3a2835c3be Cherry-pick "Fix not validating single token filtered consumer" (#4338)
This cherry-picks #3776 into `main` for the next patch release.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-26 22:26:58 -07:00
Derek Collison
865f12f396 Report correct consumer count in paged list response (#4339)
Previously the Total in paged responses would always equal the size of
the first response this would stall paged clients after the first page.

Now correctly sets the total so paging continues, improves the test to
verify these aspects of the report
2023-07-26 22:19:25 -07:00
R.I.Pienaar
60e67ff9a5 Report correct consumer count in paged list response
Previously the Total in paged responses would always equal the
size of the first response this would stall paged clients after
the first page.

Now correctly sets the total so paging continues, improves the
test to verify these aspects of the report

Signed-off-by: R.I.Pienaar <rip@devco.net>
2023-07-27 07:52:24 +03:00
Tomasz Pietrek
4b72e37f27 Fix not validating single token filtered consumer
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-26 16:21:00 +01:00
Waldemar Quevedo
45beab1f17 Fix typo from internal function (#4337)
minor internal typo fix
2023-07-25 06:56:23 -07:00
Derek Collison
0421c65c88 Bump to 2.9.21-RC.1
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-23 14:58:52 -07:00
Derek Collison
4ee3408dfd [IMPROVED] Update dependencies (#4333)
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-23 14:55:07 -07:00
Derek Collison
2e48b54d14 Update dependencies
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-23 11:43:51 -07:00
Waldemar Quevedo
bbfeb2a887 Fix typo on internal function
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-07-22 20:40:26 -07:00
Derek Collison
ac24110ac4 Include logtime_utc option into v2.9 series (#4331)
Includes option to log in UTC with commits from following PRs:

https://github.com/nats-io/nats-server/pull/3833
https://github.com/nats-io/nats-server/pull/3920
2023-07-21 18:37:05 -07:00
Byron Ruth
905f040d86 Change logtime_utc default to false
This ensures that the existing, default behavior of the server emitting the local timezone is not changed.
2023-07-21 16:57:12 -07:00
Neil Twigg
e88517c90e Add logtime_utc option
backport to main: https://github.com/nats-io/nats-server/pull/3833

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-07-21 16:56:13 -07:00
Derek Collison
ba517e4bfb [FIXED] Leaking memory on usage of getHash() (#4329)
If we created lots of hashes, beyond server names, like for consumer or
stream NRG group names etc, these maps would grow and not release
memory. Performance hit is ~300ns per call, and we can use string intern
trick if need be at a future date since it is GC friendly.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4289
2023-07-20 16:04:18 -07:00
Derek Collison
da60f2ab32 [IMPROVED] Don't error to server logs if message was deleted for consumer (#4328)
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 15:20:11 -07:00
Derek Collison
b68aed90d3 If we created lots of hashes, beyond server names, like for consumer or stream NRG groups etc, the maps would grow and not release memory.
In the benchmark on my machine, this added ~300ns per call, but I think that is ok for now vs the memory usage.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 15:11:28 -07:00
Derek Collison
9280a552b8 Don't error to server logs if message deleted
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 14:07:35 -07:00
Derek Collison
6c9fb6a938 [FIXED] Server reload with highly active accounts with service imports could cause panic or dataloss (#4327)
When service imports were reloaded on active accounts with lots of
traffic the server could panic or lose data.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 13:19:17 -07:00
Derek Collison
7477ce8257 When service imports were reloaded on active accounts with lots of traffic the server could panic or lose data.
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 12:20:50 -07:00
Derek Collison
8e49fcc6df Undo branch build restrictions, quote go version
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 11:17:19 -07:00
Derek Collison
0347f272c8 Don't require TLS for in-process connection (#4323)
This should fix a bug where in-process connections expect TLS over the
`net.Pipe` if TLS is configured.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-20 11:01:09 -07:00
Neil Twigg
b7d6b7e7a9 Add unit test to prove connection works
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-20 11:01:32 +01:00
Caleb Lloyd
7993547bee Adjust in-process server info tls_required to tls_available
Signed-off-by: Caleb Lloyd <caleb@synadia.com>
2023-07-20 10:44:03 +01:00
Neil Twigg
ed9fafc796 Don't require TLS for in-process connection
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-20 10:43:58 +01:00
Derek Collison
80fb29f9e3 Add test case for concurrent expected last subject sequence (#4319)
Resolves: #4320
2023-07-18 12:37:57 -07:00
Derek Collison
244dda809c Fix bug that would race around check for last sequence per subject
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-18 11:29:59 -07:00