Commit Graph

5616 Commits

Author SHA1 Message Date
Tomasz Pietrek
ac2669a022 Fix consumer limits
Test didn't catch this error, as by default, old JS client
sets ack policy to none. If policy is different, it will fail
to create consumer with defaults.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-09-20 17:37:05 +02:00
Jean-Noël Moyne
9fc2603263 Removes the single subject transform dest field from StreamSource
Co-authored-by: Jean-Noël Moyne <jnmoyne@gmail.com>
Co-authored-by: Neil Twigg <neil@nats.io>

Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-20 15:28:45 +01:00
Jean-Noël Moyne
40ce0a9d7e Use filter_subject when calling extended consumer create API
The server consumer creation code is picky and does indeed not accept a request send to the ExT subject if that request specifies the subject filter in the array (even if there is only one entry in the array).

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
Signed-off-by: Neil Twigg <neil@nats.io>

Co-authored-by: Jean-Noël Moyne <jnmoyne@gmail.com>
Co-authored-by: Neil Twigg <neil@nats.io>
2023-09-20 10:51:19 +01:00
Neil Twigg
ad63d702c4 Use new consumer create subject when single subject filter specified in SubjectFilters
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-20 10:51:19 +01:00
Ivan Kozlovic
579ee3b828 [FIXED] LeafNode: TLS Handshake when remote does not have a tls{} block
If a leafnode remote configuration does not have a tls{} block but
connect to a hub that requires TLS, the handshake between the two
servers will fail. A simple workaround is to add in the remote
configuration an empty tls{} block.

This issue was introduced in v2.10.0 due to some refactoring in
order to support compression.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-09-19 19:50:50 -06:00
Byron Ruth
6ba8272034 Bump 2.10.1
Signed-off-by: Byron Ruth <byron@nats.io>
2023-09-19 16:04:35 -04:00
Byron Ruth
f677536a88 Relase v2.10.0
Signed-off-by: Byron Ruth <byron@nats.io>
2023-09-19 15:26:16 -04:00
Neil Twigg
01872d2aa8 Fix empty string constants
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-19 19:07:17 +01:00
Neil Twigg
8b60131e92 Fix race condition in clientHasMovedToDifferentAccount
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-19 18:52:34 +01:00
Neil
ecbfac862c Ignore subject_transform_dest in stream sourcing (#4558)
This is a safer (less lines of code touched) alternative to #4557 for
now, which simply ignores the `subject_transform_dest` field in the API
and the stream assignments. We'll still look to merge the other PR to
clean up but will do so post-release when we have more time to test it.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-19 18:10:01 +01:00
Neil Twigg
887b92bfe2 Fix data race in setStreamAssignment by ensuring JS lock held
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-19 17:42:25 +01:00
Neil Twigg
dff12e465e Ignore subject_transform_dest in stream sourcing
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-19 11:35:26 +01:00
Derek Collison
3dc0627811 Bump to 2.10.0-RC.6
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-18 13:53:25 -07:00
Waldemar Quevedo
2d23e9b348 Fix to stop forwarding proposals in consumers after scaling down a stream (#4556)
Sometimes when scaling down a stream, a raft node could continue
forwarding proposals after already being closed, in the debug logs this
can be confirmed by many entries logging 'Direct proposal ignored, not
leader (state: CLOSED)'.
2023-09-18 13:51:44 -07:00
Derek Collison
71b8a33456 Update to not pop directly, just bail when we detect leadership change
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-18 13:27:27 -07:00
Waldemar Quevedo
ea775a80e8 Skip TestJetStreamClusterRestartThenScaleStreamReplicas for now
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2023-09-18 12:46:53 -07:00
Derek Collison
850c89e175 When scaling a consumer down make sure to pop the loopAndForwardProposals go routine
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-18 12:26:25 -07:00
Waldemar Quevedo
27245891f2 Add test for scaling replica with pull consumers
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2023-09-18 12:26:05 -07:00
Derek Collison
22514a033f Add logfile_max_num feature (#4548)
### Changes proposed in this pull request:

NATS Server 2.9 has `logfile_size_limit` option which allows the
operator to set an optional byte limit on the NATS Server log file which
when met causes a "rotation" such that the current log file is renamed
(original file name appended with a time stamp to nanosecond accuracy)
and a new log file is instantiated.

This PR is a new `logfile_max_num` companion option (alias
`log_max_num`) which allows the operator to designate that the server
should prune the **total number of log files** -- the currently active
log file plus backups -- to the maximum setting.

A max value of `0` (the implicit default) or a negative number has
meaning of unlimited log files (no maximum) as this is an opt-in
feature.

A max value of `1` is effectively a truncate-only logging pattern as any
backup made at rotation will subsequently be purged.

A max value of `2` will maintain the active log file plus the latest
backup. And so on...

> The currently active log file is never purged. Only backups are
purged.

When enabled, backup log deletion is evaluated inline after each
successful rotation event. To be considered for log deletion, backup log
files MUST adhere to the file naming format used in log rotation as well
as agree with the current `logfile` name and location. Any other files
or sub-directories in the log directory will be ignored. E.g. if an
operator makes a manual copy of the log file to `logfile.bak` that file
will not be evaluated as a backup.

### Typical use case:

This feature is useful in a constrained hosting environment for NATS
Server, for example an embedded, edge-compute, or IoT device scenario,
in which _more featureful_ platform or operating system log management
features do not exist or the complexity is not required.
2023-09-18 09:02:41 -07:00
Derek Collison
8f0e65fe0d Bump to 2.10.0-RC.5
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-17 21:38:34 -07:00
Derek Collison
216df811ff Various fixes and improvements to tombstone and buffer gaps. (#4553)
We fixed a few bugs in tombstone handling, and formalized support for
holes in the underlying buffers. Due to customer data from the field we
also now use holes during compaction.

Signed-off-by: Derek Collison <derek@nats.io>
2023-09-17 21:36:07 -07:00
Derek Collison
acffa0668a Various fixes and improvements to tombstone and buffer gaps.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-17 19:49:01 -07:00
Waldemar Quevedo
0e63608716 Fix for data race when changing retention policy
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2023-09-17 18:15:55 -07:00
Waldemar Quevedo
32021f66f1 Use write lock in memstore.LoadNextMsg
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2023-09-17 17:24:53 -07:00
Derek Collison
4e9cd9aa36 Protect access to c.acc
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-17 10:01:24 -07:00
Neil Twigg
1f9ddf2bbd Add Raft goroutine labels, tweak logging
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-16 11:15:06 +01:00
Todd Beets
349e718d39 Changes for max log files option (active plus backups); remove redundant lexical sort of backups; adjust test 2023-09-15 22:08:09 -07:00
Todd Beets
46147cf0ea Add logfile_max_archives feature and test. 2023-09-15 16:21:51 -07:00
Derek Collison
9781025b40 Fix for data race accessing consumer assignment
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-15 16:21:12 -07:00
Derek Collison
a5344c099f AuthCallout request should include TLS data when client is NATS WS client (#4544)
Make sure the client handshake flag is set when TLS handshake is made as
part of WebSocket connection/upgrade (notionally HTTPS) rather than as
part of the NATS protocol TLS initiation chain. AuthCallout tests the
flag when building the data for the AuthCallout service request.

Added AuthCallout unit test for NATS WS client auth that requires the
TLS data.
2023-09-15 15:56:12 -07:00
Todd Beets
aed99441c6 Use preferred value tests (equal, not equal) rather than booleans for better fail logs 2023-09-15 14:41:41 -07:00
Todd Beets
7b0a12d7da Add *tls.Conn safe type check as some black box unit tests override the natural underlying type for test purposes which would otherwise cause a panic 2023-09-15 13:52:41 -07:00
Todd Beets
40cf145ee6 Map both 127.0.0.1 and 127.0.1.1 to localhost for HTTPS server host validate 2023-09-15 13:13:24 -07:00
Todd Beets
75d2ddb26b AuthCallout request should include TLS data when client is NATS WS client 2023-09-15 12:36:34 -07:00
Derek Collison
0af378cf85 Bump to 2.10.0-RC.4
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-15 08:54:27 -07:00
Derek Collison
097e4097d1 Allow longer times due to travis slowdowns
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-15 08:52:50 -07:00
Derek Collison
f2e7ed91cb Fix for panic in consumer, needed to recheck if consumer was closed
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-15 08:40:21 -07:00
Waldemar Quevedo
76cbef79cc Bump to 2.10.0-RC.3
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2023-09-14 12:11:09 -07:00
Derek Collison
22f40eafa0 Add in jitter in case there are many that all try to cleanuo at the same time
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-14 11:24:32 -07:00
Derek Collison
392f25b6da Fix for data race and adjustment to do a backoff on making sure consumers are cleaned up.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-14 11:21:11 -07:00
Waldemar Quevedo
db0faf4538 flake: Fix TestJetStreamConsumerAckFloorFill
Can sometimes fail the first time checking for the ack floor
but fine after checking again.

Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2023-09-14 09:31:38 -07:00
Neil Twigg
f38faafbc9 Bump to 2.10.0-RC.2
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-14 16:35:36 +01:00
Neil
46361e86a3 Fix leaking timers in stream sources (#4532)
Repeated calls to `scheduleSetSourceConsumerRetry` could end up creating
multiple timers for the same source, which would eventually schedule
even more timers, which would result in runaway CPU usage. This PR
instead bounds to one timer per source per stream.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-14 16:32:36 +01:00
Neil Twigg
904f4c388e De-flake TestJetStreamClusterAccountPurge by waiting for account to exist
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-14 11:40:30 +01:00
Neil Twigg
6f3f544841 Fix leaking timers in stream sources
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-14 10:30:24 +01:00
Derek Collison
787f6acf31 Fix for a call into fs.recalculateFirstForSubj() from fs.recalculateFirstForSubj() that did not lock the mb properly.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-13 15:35:34 -07:00
Neil Twigg
505190266a Bump to 2.10.0-RC.1
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-13 17:22:30 +01:00
Neil
59bc094e86 [FIXED] Increased AckWait in TestMQTTQoS2RetriesPublish, TestMQTTQoS2RetriesPubRel (#4518)
TestMQTTQoS2RetriesPublish to 100ms, and in TestMQTTQoS2RetriesPubRel to
50ms.

A lesser value caused another PUBLISH to be fired while the test was
still processing the final QoS2 flow. Reduced the number of retries we
wait for to make the test a little quicker.
2023-09-13 11:49:17 +01:00
Piotr Piotrowski
e08442fbfc flake: Fixes TestServerOperatorModeUserInfoExpiration
Signed-off-by: Piotr Piotrowski <piotr@synadia.com>
2023-09-13 11:57:53 +02:00
Derek Collison
58b5fc4abf Fix for a flapper
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-12 17:19:30 -07:00