Commit Graph

7880 Commits

Author SHA1 Message Date
Derek Collison
3c57adcfe5 Bump to 2.10.0-beta.49
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-04 10:16:09 -07:00
Derek Collison
8079495903 Merge branch 'main' into dev
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-04 10:15:35 -07:00
Derek Collison
b2e7725aed Release v2.9.21 (#4368) 2023-08-04 07:46:27 -07:00
Byron Ruth
c1d1f11a18 Release v2.9.21
Signed-off-by: Byron Ruth <byron@nats.io>
2023-08-04 10:11:06 -04:00
Derek Collison
8c6055babc Bump to 2.9.21-RC.6
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-03 13:25:14 -07:00
Derek Collison
087e14782d [IMPROVED] Also reset clseq to avoid possible immediate sequence mismatch (#4366)
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-03 13:24:24 -07:00
Derek Collison
cbe85c826a Also reset clseq to avoid immediate sequence mismatch
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-03 12:40:17 -07:00
Derek Collison
d522f4656c Bump to 2.9.21-RC.5
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-03 11:17:51 -07:00
Derek Collison
34199ab6a8 [IMPROVED] When taking over make sure to sync and reset clfs for clustered streams. (#4365)
If the failed state of clfs drifts between leaders and followers,
replicas could discard and skip messages possibly incorrectly. This will
force sync if we have a non-zero clfs state when a leader takes over.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-03 11:17:03 -07:00
Derek Collison
66a8e81d49 Bump Go to 1.19.12 (#4364) 2023-08-03 10:45:39 -07:00
Derek Collison
081140ee67 When taking over make sure to sync and reset clfs for clustered streams.
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-03 10:41:10 -07:00
Byron Ruth
af52adb1ee Bump Go to 1.19.12
Signed-off-by: Byron Ruth <byron@nats.io>
2023-08-03 11:24:58 -04:00
Derek Collison
9de5e3e64d OCSP backports and adds (#4362)
This PR backports the OCSP Peer feature option (as in 2.10 train) and
includes two fixes for the existing OCSP Staple feature.

OCSP Staple: 

1. Fixed and clarified how NATS Server determines its own Issuer CA when
obtaining and validating an OCSP Response for subsequent staple
2. Eliminated problematic assumption that all node peers are issued by
same CA when NATS Server validates ROUTE and GATEWAY peer nodes
3. Added OCSP Response effectivity checks on ROUTE and GATEWAY
peer-presented staple

Note for #3: Allowed host clock skew between node peers set at
30-seconds. If the OCSP Response contains an empty assertion for
NextUpdate, NATS Server will default to 1-hour validity (after
ThisUpdate). It is recommended that CA OCSP Responder should assert
NextUpdate.
2023-08-02 18:10:24 -07:00
Todd Beets
ac43a8d4eb Enhance OCSP peer validation for GATEWAY and ROUTE connections. Nodes no longer required to have same CA issuer. OCSP response effectivity now checked using default clock skew and default validity period if not asserted by responder. 2023-08-02 16:09:21 -07:00
Todd Beets
1f0b70d5fc Fixed local issuer determination for OCSP Staple, issue #3773 2023-08-02 11:52:36 -07:00
Todd Beets
209fcd70eb OCSP Peer Feature 2023-08-02 11:25:48 -07:00
Derek Collison
5577d18c67 Fix some system service imports going missing after reload (#4360)
On reload some of the imports from the system account where going
missing on reload, this adds them back after a reload:

```
$SYS.REQ.SERVER.PING.CONNZ
$SYS.REQ.ACCOUNT.PING.STATZ
$SYS.REQ.ACCOUNT.PING.CONNZ
```
2023-08-02 10:14:07 -07:00
Waldemar Quevedo
23b5cb959c review fixes
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-02 08:51:08 -07:00
Waldemar Quevedo
2b252469ca fix: add missing default service imports on reload
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-01 23:34:07 -07:00
Waldemar Quevedo
13cb62e0bf Add test checking subscriptions before/after reload
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2023-08-01 22:51:42 -07:00
Waldemar Quevedo
7c9a91fc91 config: make parsing configurations without usable values invalid (backport) (#4358)
This makes configuration files that are empty, or read and processed by
the parser but with no detected values now return an error.

Fixes #4343 
Backport from dev branch
(https://github.com/nats-io/nats-server/pull/4347)
2023-08-01 22:33:30 -07:00
Waldemar Quevedo
ba596f7de1 config: make parsing configurations without usable values invalid
This makes configuration files that are empty, or read and processed
by the parser but with no detected values now return an error.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-01 21:55:09 -07:00
Derek Collison
aa6ac2d665 Bump to 2.9.21-RC.4
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 21:48:02 -07:00
Derek Collison
42752ec551 Merge branch 'main' into dev
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 21:46:54 -07:00
Derek Collison
09ab23c929 [IMPROVED] Publish performance with lots of no interest subjects (#4359)
Do not hold onto no interest subjects from a client in the unlocked cache.
If sending lots of different subjects all with no interest performance could be affected.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4341
2023-08-01 21:43:23 -07:00
Derek Collison
787b0d922f Do not hold onto no interest subjects from a client in the unlocked cache.
If sending lots of different subjects all with no interest performance could be affected.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 21:20:58 -07:00
Derek Collison
09e78a3349 [FIXED] Accounting drifts (#4357)
Three issues were found and resolved.

    1. Some purge replays after recovery  could execute full purge.
    2. Callback was registered without lock, which  could lead to skew.
3. Cluster reset could stop stream store and recreate it, which could
lead to double accounting.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 20:07:14 -07:00
Derek Collison
5c8db89506 Make sure we do not drift on accounting.
Three issues were found and resolved.

1. Purge replays after recovery could execute full purge.
2. Callback was registered without lock, which could lead to skew.
3. Cluster reset could stop stream store and recreate it, which could lead to double accounting.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 18:35:20 -07:00
Derek Collison
97827be97d Call up to upper layers on Stop() for accounting purposes
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 17:36:40 -07:00
Derek Collison
b8d1ac9475 Allow long form resolver config to be of type MEM
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 17:34:14 -07:00
Todd Beets
971c61692a Fixed local issuer determination for OCSP Staple, issue #3773 (#4355)
Resolves problems of [issue
#3773](https://github.com/nats-io/nats-server/issues/3773).

With this fix, NATS Server will locally determine it's own certificate's
issuer from either the configured server certificate (bundle of leaf
cert plus optional intermediate CA certs) or from the configured server
CA trust store, as follows:

1. The operator may provide the server's certificate issuer in the
second position of the server's certificate configuration (typically
`cert_file` but may be `cert_store` on the Windows platform). If a
candidate issuer is found here it is PKI validated as the actual issuer
of the server's cert else a hard error.

2. If not found in [1], NATS Server will seek to create at least one
verified chain with its configured trust store (typically `ca_file` but
could by the system trust store if not configured). It will derive the
issuer from the first verified chain. If no verified chain can be formed
it is a hard error.
2023-08-01 16:13:25 -07:00
Derek Collison
33d1f852b2 jnm/fix_error_code_overlap (#4353)
### Changes proposed in this pull request:

Fix overlap in error codes
2023-08-01 12:26:31 -07:00
Jean-Noël Moyne
73d294adba Re numbers the errors for stream source multi-transform after the PR merge since a previously approved PR grabbed one of those error codes
Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-08-01 11:20:40 -07:00
Jean-Noël Moyne
449b27535e [ADDED] Support for multi-filter in stream sources (#4276)
- [X] Tests added
- [X] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [X] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

### Changes proposed in this pull request:

Adds support for multi-filter (and associated transform destinations) to
stream sources

---------

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-08-01 10:50:11 -07:00
Todd Beets
e51a42963a OCSP Peer Verification (#4258)
New security feature [ADR-38: OCSP Peer
Verification](https://github.com/nats-io/nats-architecture-and-design/pull/226/files#diff-575a9545de9d498a48d2889972b0cb57dbadebde3b4328b65ab02bb43f557935)
providing fine-grain certificate status check via OCSP verification; for
inbound NATS, MQTT, WebSocket, and Leaf client connections (mTLS) as
well as outbound Leaf connections to another NATS System.
2023-08-01 09:17:27 -07:00
Neil
d780da6d0c Prevent configuring first_seq on mirrors (#4345)
This prevents configurations where mirrors are mixed with `first_seq`s
of greater than zero.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-31 13:26:45 +01:00
Derek Collison
3b2231a9d5 Bump to 2.10.0-beta.48
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-30 13:28:19 -07:00
Derek Collison
fd83dfe61f [FIXED] Seqset encode bug that could cause bad stream state snapshots (#4348)
This would cause decode to spin and lock up the server.

This fixes the encode bug (a delete and clear node bug in avl/seqset).
Also addressed receiving a bad snapshot, etc.
2023-07-30 13:26:37 -07:00
Derek Collison
d27c44e6cd Fix another test for more efficient deleteBlocks
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-30 12:02:49 -07:00
Derek Collison
cb9f8c0bf4 Fix to test for more efficient deleteBlocks
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-30 11:53:18 -07:00
Derek Collison
2696320207 When we encounter a bad snapshot, reset our raft state if we own it and return proper error.
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-30 11:15:10 -07:00
Derek Collison
54c5414c3d Detect mal-formed stream state snapshots and return appropriate error
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-30 11:06:06 -07:00
Derek Collison
c6ea1667a7 Fixed a bug in deleteBlocks() that would not nil out our accumulated seqset.
Added in another pedantic test on stream state encoding and decoding.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-30 11:04:36 -07:00
Derek Collison
a778921b8c Fixed a bug that when sequences were deleted and we cleaned up empty nodes we would not redo heights and balances.
This caused a rotate operation to possibly return nil and replace our root with nil when non empty.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-30 11:01:32 -07:00
Waldemar Quevedo
bd2147efe7 config: make parsing configurations without usable values invalid (#4347)
This makes configuration files that are empty, or read and processed by
the parser but with no detected values now return an error.
2023-07-28 20:52:18 -07:00
Waldemar Quevedo
bab7a0e444 config: make parsing configurations without usable values invalid
This makes configuration files that are empty, or read and processed
by the parser but with no detected values now return an error.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-07-28 15:46:17 -07:00
Derek Collison
dfb7fac861 Merge branch 'main' into dev
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 13:56:35 -07:00
Derek Collison
039f63e260 Bump to 2.9.21-RC.3
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 13:41:30 -07:00
Derek Collison
5b7c029c64 [FIXED] Do not panic when re-calculating first sequence for SimpleState when fseq moves ahead of old first. (#4346)
When a lazy simple state has an outdated first that needs to be updated,
if fseq had moved past it would panic.
This was not common but with latest fix prior in can become more common,
hence why it showed up.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 13:41:01 -07:00
Derek Collison
4c14be9fbb When a lazy simple state has an outdated first that needs to be updated, if fseq had moved past it would panic.
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 13:09:37 -07:00