Commit Graph

7360 Commits

Author SHA1 Message Date
Waldemar Quevedo
ba596f7de1 config: make parsing configurations without usable values invalid
This makes configuration files that are empty, or read and processed
by the parser but with no detected values now return an error.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-08-01 21:55:09 -07:00
Derek Collison
aa6ac2d665 Bump to 2.9.21-RC.4
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 21:48:02 -07:00
Derek Collison
09ab23c929 [IMPROVED] Publish performance with lots of no interest subjects (#4359)
Do not hold onto no interest subjects from a client in the unlocked cache.
If sending lots of different subjects all with no interest performance could be affected.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4341
2023-08-01 21:43:23 -07:00
Derek Collison
787b0d922f Do not hold onto no interest subjects from a client in the unlocked cache.
If sending lots of different subjects all with no interest performance could be affected.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 21:20:58 -07:00
Derek Collison
09e78a3349 [FIXED] Accounting drifts (#4357)
Three issues were found and resolved.

    1. Some purge replays after recovery  could execute full purge.
    2. Callback was registered without lock, which  could lead to skew.
3. Cluster reset could stop stream store and recreate it, which could
lead to double accounting.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 20:07:14 -07:00
Derek Collison
5c8db89506 Make sure we do not drift on accounting.
Three issues were found and resolved.

1. Purge replays after recovery could execute full purge.
2. Callback was registered without lock, which could lead to skew.
3. Cluster reset could stop stream store and recreate it, which could lead to double accounting.

Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 18:35:20 -07:00
Derek Collison
97827be97d Call up to upper layers on Stop() for accounting purposes
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 17:36:40 -07:00
Derek Collison
b8d1ac9475 Allow long form resolver config to be of type MEM
Signed-off-by: Derek Collison <derek@nats.io>
2023-08-01 17:34:14 -07:00
Derek Collison
039f63e260 Bump to 2.9.21-RC.3
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 13:41:30 -07:00
Derek Collison
5b7c029c64 [FIXED] Do not panic when re-calculating first sequence for SimpleState when fseq moves ahead of old first. (#4346)
When a lazy simple state has an outdated first that needs to be updated,
if fseq had moved past it would panic.
This was not common but with latest fix prior in can become more common,
hence why it showed up.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 13:41:01 -07:00
Derek Collison
4c14be9fbb When a lazy simple state has an outdated first that needs to be updated, if fseq had moved past it would panic.
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 13:09:37 -07:00
Derek Collison
7feebb4f64 Bump to 2.9.21-RC.2
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 09:55:45 -07:00
Derek Collison
c0fe4972f0 [FIXED] Stream recovery with corrupt msg block with sequence gaps. (#4344)
This is a fix for a bad msg blk detected in the field that had sequence
holes.

The stream had max msgs per subject of one and only one subject but had
lots of messages. The stream did not recover correctly, and upon further
inspection determined that a msg blk had holes, which should not be
possible.

We now detect the holes and deal with the situation appropriately.
Heavily tested on the data dump from the field.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 05:10:35 -07:00
Derek Collison
92430513d3 Add in const for msg record hash size
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-28 04:35:33 -07:00
Derek Collison
21e3d7a512 This is a fix for a bad msg blk detected in the field that had sequence holes.
The stream had max msgs per subject of one and only one subject but had lots of messages.
The stream did not recover correctly, and upon further inspection determined that a msg blk had holes, which should not be possible.

We now detect the holes and deal with the situation appropriately.
Heavily tested on the data dump from the field.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-27 21:43:15 -07:00
Derek Collison
3a2835c3be Cherry-pick "Fix not validating single token filtered consumer" (#4338)
This cherry-picks #3776 into `main` for the next patch release.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-26 22:26:58 -07:00
Derek Collison
865f12f396 Report correct consumer count in paged list response (#4339)
Previously the Total in paged responses would always equal the size of
the first response this would stall paged clients after the first page.

Now correctly sets the total so paging continues, improves the test to
verify these aspects of the report
2023-07-26 22:19:25 -07:00
R.I.Pienaar
60e67ff9a5 Report correct consumer count in paged list response
Previously the Total in paged responses would always equal the
size of the first response this would stall paged clients after
the first page.

Now correctly sets the total so paging continues, improves the
test to verify these aspects of the report

Signed-off-by: R.I.Pienaar <rip@devco.net>
2023-07-27 07:52:24 +03:00
Tomasz Pietrek
4b72e37f27 Fix not validating single token filtered consumer
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-26 16:21:00 +01:00
Waldemar Quevedo
45beab1f17 Fix typo from internal function (#4337)
minor internal typo fix
2023-07-25 06:56:23 -07:00
Derek Collison
0421c65c88 Bump to 2.9.21-RC.1
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-23 14:58:52 -07:00
Derek Collison
4ee3408dfd [IMPROVED] Update dependencies (#4333)
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-23 14:55:07 -07:00
Derek Collison
2e48b54d14 Update dependencies
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-23 11:43:51 -07:00
Waldemar Quevedo
bbfeb2a887 Fix typo on internal function
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-07-22 20:40:26 -07:00
Derek Collison
ac24110ac4 Include logtime_utc option into v2.9 series (#4331)
Includes option to log in UTC with commits from following PRs:

https://github.com/nats-io/nats-server/pull/3833
https://github.com/nats-io/nats-server/pull/3920
2023-07-21 18:37:05 -07:00
Byron Ruth
905f040d86 Change logtime_utc default to false
This ensures that the existing, default behavior of the server emitting the local timezone is not changed.
2023-07-21 16:57:12 -07:00
Neil Twigg
e88517c90e Add logtime_utc option
backport to main: https://github.com/nats-io/nats-server/pull/3833

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-07-21 16:56:13 -07:00
Derek Collison
ba517e4bfb [FIXED] Leaking memory on usage of getHash() (#4329)
If we created lots of hashes, beyond server names, like for consumer or
stream NRG group names etc, these maps would grow and not release
memory. Performance hit is ~300ns per call, and we can use string intern
trick if need be at a future date since it is GC friendly.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4289
2023-07-20 16:04:18 -07:00
Derek Collison
da60f2ab32 [IMPROVED] Don't error to server logs if message was deleted for consumer (#4328)
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 15:20:11 -07:00
Derek Collison
b68aed90d3 If we created lots of hashes, beyond server names, like for consumer or stream NRG groups etc, the maps would grow and not release memory.
In the benchmark on my machine, this added ~300ns per call, but I think that is ok for now vs the memory usage.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 15:11:28 -07:00
Derek Collison
9280a552b8 Don't error to server logs if message deleted
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 14:07:35 -07:00
Derek Collison
6c9fb6a938 [FIXED] Server reload with highly active accounts with service imports could cause panic or dataloss (#4327)
When service imports were reloaded on active accounts with lots of
traffic the server could panic or lose data.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 13:19:17 -07:00
Derek Collison
7477ce8257 When service imports were reloaded on active accounts with lots of traffic the server could panic or lose data.
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 12:20:50 -07:00
Derek Collison
8e49fcc6df Undo branch build restrictions, quote go version
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-20 11:17:19 -07:00
Derek Collison
0347f272c8 Don't require TLS for in-process connection (#4323)
This should fix a bug where in-process connections expect TLS over the
`net.Pipe` if TLS is configured.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-20 11:01:09 -07:00
Neil Twigg
b7d6b7e7a9 Add unit test to prove connection works
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-20 11:01:32 +01:00
Caleb Lloyd
7993547bee Adjust in-process server info tls_required to tls_available
Signed-off-by: Caleb Lloyd <caleb@synadia.com>
2023-07-20 10:44:03 +01:00
Neil Twigg
ed9fafc796 Don't require TLS for in-process connection
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-20 10:43:58 +01:00
Derek Collison
80fb29f9e3 Add test case for concurrent expected last subject sequence (#4319)
Resolves: #4320
2023-07-18 12:37:57 -07:00
Derek Collison
244dda809c Fix bug that would race around check for last sequence per subject
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-18 11:29:59 -07:00
Derek Collison
360f807cef Moved to end for merge with other branches, minor changes
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-18 11:29:26 -07:00
Byron Ruth
e7bf1b3132 Add test case for concurrent expected last subject sequence
Signed-off-by: Byron Ruth <byron@nats.io>
2023-07-18 11:10:51 -04:00
Derek Collison
75ad503ddc Limit branch builds to main and dev (#4312) 2023-07-17 10:39:34 -07:00
Byron Ruth
94c1988ed8 Limit branch builds to main and dev
Signed-off-by: Byron Ruth <byron@nats.io>
2023-07-15 07:01:24 -04:00
Waldemar Quevedo
c1750b8a54 De-flake a few tests (#4311)
This should hopefully make a small handful of tests less flakey.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-14 09:57:23 -07:00
Neil Twigg
2527e11304 Increase threshold in TestNoRaceJetStreamSlowFilteredInititalPendingAndFirstMsg
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-14 17:05:26 +01:00
Neil Twigg
6cdb0c7838 Wait for current in TestJetStreamClusterStreamUpdateSyncBug so in-flight state changes aren't lost
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-14 17:03:21 +01:00
Neil Twigg
abb47c9554 Only send one message in TestJetStreamRedeliverCount to prevent out-of-order redeliveries
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-14 16:56:01 +01:00
Neil Twigg
144abc0dd9 Make TestJetStreamAccountImportJSAdvisoriesAsService insensitive to event order
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-14 16:51:51 +01:00
Neil Twigg
b58204cff7 Use checkFor in TestJetStreamMsgHeaders
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-14 16:44:24 +01:00