Commit Graph

313 Commits

Author SHA1 Message Date
Derek Collison
168fb22df5 Fix one more test that did not set ack policy to explicit
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-06 22:05:31 -07:00
Derek Collison
08938034a5 Fix test that did not set ack policy to explicit
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-06 19:54:42 -07:00
Derek Collison
18244ea8cb Fix test that did not set ack policy to explicit
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-06 15:10:46 -07:00
Derek Collison
caa262513d Fix test that did not set ack policy which is needed
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-06 14:15:44 -07:00
Derek Collison
4ebdb69daf Merge branch 'main' into dev 2023-04-26 11:34:37 -07:00
Derek Collison
47c6bfded4 Update server/jetstream_test.go
Fix spelling

Co-authored-by: Tomasz Pietrek <tomasz@nats.io>
2023-04-24 22:29:05 -07:00
Derek Collison
cae91b8cad In single server mode healthz could mistake a snapshot staging directory during a restore as an account.
If the restore took a long time, stalled, or was aborted, would cause healthz to fail.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-24 22:14:04 -07:00
Derek Collison
3f27b67791 Merge branch 'main' into dev 2023-04-15 10:47:29 -07:00
Derek Collison
034975e767 Fix for a regression in behavior, needed to make sure when we went back to 1 entry for a subject we cleared firstNeedsUpdate.
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-15 10:00:44 -07:00
Tomasz Pietrek
82220d9103 Merge branch 'main' into dev
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-04-14 22:38:45 +02:00
Tomasz Pietrek
a66c67baa5 Fix stream sourcing & mirroring overlap errors
When adding or updating sources/mirrors, server was checking if the stream with
a given name exists to check for subject overlaps, among other things.
However, if sourced/mirrored stream was `External`, checks should
not be executed, as not only stream would never be found,
but also, if `External` stream had the same name as the sourcing stream,
the check would be wrongly performed against itself.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-04-14 21:00:11 +02:00
Derek Collison
a319d24345 Merge branch 'main' into dev 2023-04-13 21:03:05 -07:00
Tomasz Pietrek
692f384f2d Fix consumer reply subject escaping
If the Consumer had a name containing `%`, it could result in
reply subject failing to format with `fmt.Sprintf`, as the `%`
was not properly escaped with `%%`.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-04-12 09:22:08 +02:00
Derek Collison
4525bdea0d [IMPROVE] Improvements and fixes to stream source consumer creation on leadership change or on stream config source updates (#4009)
- [X] Tests added
- [X] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [x] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)
2023-04-04 20:09:15 -07:00
Jean-Noël Moyne
0420c6e317 - Improve and fix stream source consumer sequence number finding on leadership change or stream config update to work with the new stream sourcing
- Updates source index name format and adds two fields (source's filter and transform) to the `"Nats-Stream-Source"` message header
- Backwards compatibility for streams containing previous (2.9) message headers for seamless upgrade
- Update TestJetStreamSourceBasics as it could flap sometimes

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-03-31 15:49:10 -07:00
Tomasz Pietrek
93c49c354e Fix bug in needAck
needAck has reverse logic for checking single subject-single filter
scenario.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-03-31 23:55:50 +02:00
Ivan Kozlovic
db599ac875 Merge branch 'main' into dev 2023-03-30 15:45:51 -06:00
Ivan Kozlovic
a4df4f8727 Fixed some tests
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-03-30 15:02:59 -06:00
Derek Collison
9f69d8734f Merge branch 'main' into dev 2023-03-04 08:42:10 -06:00
Tomasz Pietrek
df282a221c Fix Pull Consumer not sending request timeout
Server did check for timeouts in `processWaiting`,
but that needs to be also checked in `nextWaiting` in case of
tight timings, as `nextWaiting` can remove Pull Request based on
timeouts too.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-03-03 14:49:04 +01:00
Derek Collison
7bd7cda021 Merge branch 'main' into dev 2023-02-28 15:17:24 -08:00
Derek Collison
6bda358fa3 Fix tests that made assumptions about single server processing.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-28 13:24:18 -08:00
Derek Collison
b19fe508c4 Do not block routes/gws on internal stream and consumer info requests
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-28 11:17:29 -08:00
Derek Collison
cac712b1d1 Merge branch 'main' into dev 2023-02-20 18:30:26 -08:00
Derek Collison
3c64d07691 Warn of consumer state update failures.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-20 17:28:11 -08:00
Derek Collison
b6149c51f0 Make sure to clean up redelivered state on purge.
Make sure to update ack floors on messages being expired out from underneath of us.

Signed-off-by: Derek Collison <derek@nats.io>
2023-02-20 11:16:12 -08:00
Derek Collison
6c9a9fb45e Fixed bug that would lose ack pending state during partial stream purge.
General code cleanup to be more correct.

Signed-off-by: Derek Collison <derek@nats.io>
2023-02-19 14:21:53 -08:00
Derek Collison
6920f24217 Merge branch 'main' into dev 2023-02-17 10:45:07 -08:00
Derek Collison
11b0f214d0 Do not re-calculate NumPending on consumer info calls.
We noticed this was being called alot in user environments.
When the consumer was filtered with a wilcard and the stream had a high cardinality of subjects and was falling behind this could take a substantial amount of time.

Signed-off-by: Derek Collison <derek@nats.io>
2023-02-16 16:30:14 -08:00
Derek Collison
0cb01f9e7a Make sure we update storage accounting on extended version purge for filestore.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-16 13:18:40 +04:00
Tomasz Pietrek
45deb3e8ef Add error for both consumer filter fields specified
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-02-13 16:13:35 +01:00
Tomasz Pietrek
b390163908 Make JetStream errors naming consistent
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-02-13 14:08:52 +01:00
Tomasz Pietrek
af338d0d59 Add multiple subject filters 2023-02-13 09:38:40 +01:00
Derek Collison
3fe85e6cc4 Merge branch 'main' into dev 2023-02-06 21:57:49 +04:00
Derek Collison
b611e37e95 For updating a consumer filter subject make sure locking ordere correct and that our sublist is present.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-06 21:34:48 +04:00
Derek Collison
2c4ec09cc7 Merge branch 'main' into dev 2023-02-02 11:01:50 -07:00
Tomasz Pietrek
46af979871 Fix current consumers not getting messages after purge
Until now, purge updated all consumers sequences
even if purge subject was only a subset of given consumer filter.
Because of that, even messages from not purged subjects were not fetched
or properly accounted for existing consumers.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-02-02 16:27:32 +01:00
Derek Collison
7607b17844 Fix for test
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-30 20:14:39 -08:00
Derek Collison
2d6b316efd Merge branch 'main' into dev 2023-01-30 20:07:26 -08:00
Tomasz Pietrek
836848ca64 Fix Consumer not getting messages after filter update
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-01-30 20:47:17 +01:00
Tomasz Pietrek
9ab52d4ba9 Add metadata to StreamConfig and ConsumerConfig
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-01-27 07:27:25 +01:00
Tomasz Pietrek
69aa55a0e7 Merge pull request #3822 from nats-io/jarema/add-batch-completed
Add batch completed status to Pull Consumers
2023-01-26 23:43:00 +01:00
Tomasz Pietrek
34be180875 Add batch completed status to Pull Consumers
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-01-26 23:23:49 +01:00
Jean-Noël Moyne
ddce3ce037 Rename StreamConfig's InputSubjectTransform (and therefore StreamSource's SubjectTransform) 2023-01-26 12:26:34 -08:00
Jean-Noël Moyne
4fbe13ec1b Change test to use require_NoError instead of explicitly checking err and calling Fatalf throughout all the tests in the file
Improvements to readability
2023-01-26 00:31:17 -08:00
Jean-Noël Moyne
a953e84437 Add stream input subject transform functionality
Extract subject transformation code out of accounts.go
Stream sources can now have a subject mapping transform
You can source the same stream more than once
Remove limitation that the subject filter for a source, mirror or consumer must have an overlap with the sourced/mirrored's stream or the stream's subjects
2023-01-25 12:25:51 -08:00
Tomasz Pietrek
86da656fff Fix not validating single token filtered consumer
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-01-11 15:48:38 +01:00
Derek Collison
d9cb1e6286 Fix for #3734
When a msg blk was not writen correctly, but the idx file was, max bytes for a stream would no longer be honored since the deletion of any messages in that empty block were not being handled properly.

Signed-off-by: Derek Collison <derek@nats.io>
2022-12-31 18:50:46 -08:00
Neil
1d53b47b33 Merge pull request #3754 from nats-io/neil/golangcilint
Switch to `golangci-lint`
2022-12-30 20:12:03 +00:00
Neil Twigg
14d0ba1c65 Fix some lint errors after move to golangci-lint 2022-12-30 20:00:08 +00:00