Commit Graph

4363 Commits

Author SHA1 Message Date
Raymond
4d8964e57b Added stream created timestamp to stream detail 2022-11-17 13:59:58 +01:00
Ivan Kozlovic
deb9d252a3 Bump version to 2.9.7-RC2
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-16 17:23:15 -07:00
Ivan Kozlovic
74a16b0097 Merge pull request #3640 from nats-io/fix_3639
[FIXED] JetStream: WorkQueue not preventing overlapping consumers
2022-11-16 17:22:35 -07:00
Ivan Kozlovic
49faba9e33 [FIXED] JetStream: WorkQueue not preventing overlapping consumers
A stream with a WorkQueue retention policy is supposed to allow
more than one consumer if they user filtered subjects, but those
subjects should not overlap.

There was an issue that if a new consumer had a filter subject
"wider" than an existing one, the error was not detected and
the new consumer was incorrectly accepted.

Resolves #3639

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-16 17:09:30 -07:00
Derek Collison
e4b5b7ec64 Make mirror consumers use filtered version of consumer create like source consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-16 15:12:14 -08:00
Derek Collison
6cb7f68ef7 Updates based on feedback w/ Ivan
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-16 11:53:46 -08:00
Derek Collison
aa57adbcd0 Use default here
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-16 11:31:49 -08:00
Derek Collison
f696c6559e Adjust test for jitter on ephemerals.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-15 17:51:01 -08:00
Derek Collison
08c94096db Allow any type of activity to prolong auto cleanup of a consumer.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-15 17:25:18 -08:00
Colin Sullivan
9f6db71937 Fix concurrent map write
Signed-off-by: Colin Sullivan <colin@synadia.com>
2022-11-15 16:26:35 -07:00
Derek Collison
d98d0278a2 Bump to 2.9.7-RC1
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-15 12:13:10 -08:00
Derek Collison
9f241f3322 Offload signaling to consumers when number is large.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-15 11:25:07 -08:00
Ivan Kozlovic
0f79b913ec Merge pull request #3631 from nats-io/fix_3626
[FIXED] JetStream: possible panic on stream info when leader not elected
2022-11-15 12:24:10 -07:00
Ivan Kozlovic
6ffa6d1e4b [FIXED] JetStream: possible panic on stream info when leader not elected
It is possible that a stream info request would be handled at a
time where the raft group would not yet be set/created, causing
a panic.

Resolves #3626 (at least the panic reports there)

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-15 11:56:41 -07:00
Derek Collison
b92ea86b80 When reading state, even on consumer init, should lock.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-15 07:40:44 -08:00
Ivan Kozlovic
9e36918f41 [FIXED] JetStream: Account removal leaks subscriptions
Resolve #3623

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-14 16:40:21 -07:00
Derek Collison
b6ef2c8910 Auto cleanup dangling messages from interest policy streams on server start.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-14 15:02:22 -08:00
Derek Collison
36ef788112 When determing whether we need an ack, no need to copy since under consumer lock.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-14 11:47:31 -08:00
Derek Collison
b76b6a1f68 Merge pull request #3620 from nats-io/fix-3612
[FIXED] Fix logic bug that would prevent some stream messages from being deleted.
2022-11-14 08:40:13 -08:00
Waldemar Quevedo
39185c11a6 Set max payload in gofuzz test (#3622)
* Set max payload in gofuzz test

This was causing the test to overrun mem limits due to max payload size
check being skipped in the parser when processing a `pub` protocol line.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2022-11-14 08:28:19 -08:00
Derek Collison
47dd97e389 Fix logic bug that would prevent some messages from being deleted on an interest based stream.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-13 17:32:38 -08:00
Derek Collison
3ac6052b32 Updated pae threshold and reporting modulo to not spam logs as much.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-11 16:08:58 -08:00
Ivan Kozlovic
da287b0aea [FIXED] Weighted subject mappings updates not applied
Suppose an account is updated to have the following weighted mapping:
```
foo -> bar 40%
```
The server automatically adds foo -> foo at 60%. Sending messages
to "foo" will result in the expected distribution of 60% messages
going to "foo" and 40% going to bar.

However, if a successive update is pushed to the server(s):
```
foo -> bar 40%
foo -> baz 60%
```
The subject mapping should now be as described, that is, no more
mapping from "foo" to "foo" and 40% to bar and 60% to baz, however,
what was happening is that the server would always use the original
mapping.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-11 15:45:32 -07:00
Derek Collison
4dab6ce92c Fix test timing
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-09 19:44:22 -08:00
Ivan Kozlovic
304744ce08 Merge pull request #3615 from nats-io/js_acc_max_streams_consumers
[FIXED] JetStream: Account max streams/consumers not always honoured
2022-11-09 18:02:51 -07:00
Ivan Kozlovic
1b892837cb [FIXED] JetStream: Account max streams/consumers not always honoured
This could happen during concurrent requests where the assignments
are not yet fully processed.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-09 17:29:20 -07:00
Derek Collison
e008e015b3 Make sure to enforce HA asset limits during peer processing as well as assignment.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-09 16:24:54 -08:00
Ivan Kozlovic
3ec42d5b85 Updates to PR #3611
- Save the TLS name only if not already set
- Use the passed URLs slice instead of using s.getOpts().Routes
- Enhanced the test
- Fixed an unrelated DATA RACE report

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-08 09:36:08 -07:00
Ivan Kozlovic
2d181e1c27 [FIXED] Routing: TLS connections to discovered server may fail
The server was not setting "server name" in the TLS configuration
for route connections, which may lead to failed (re)connect if
the certificate does not allow for the IP and the URL did not
have the hostname, which would happen with gossip protocol.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-07 17:26:17 -07:00
Derek Collison
c6031382a1 Fix for #3499
When we deleted a consumer from an interest policy stream we would make sure to clean up any unacked messages.
However we only based start from the ack floor for the consumer and did not take into account the first sequence of the stream.

Signed-off-by: Derek Collison <derek@nats.io>
2022-11-05 13:56:45 -07:00
Ivan Kozlovic
ef00281bc2 Bump to v2.9.7-beta as per release process
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-04 10:46:45 -06:00
Ivan Kozlovic
00a7b4af1e Release v2.9.6
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-04 09:44:54 -06:00
Ivan Kozlovic
ca237bdfa0 [FIXED] JetStream: Stream scale down while it has no quorum
If a stream R2 had one of its server network-partitioned and at
that time the stream was edited to be scaled down to an R1 it
would cause the stream to no longer have quorum even when the
network partition is resolved.

Signed-off-by: Derek Collison <derek@nats.io>
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-04 09:08:31 -06:00
Ivan Kozlovic
c16ccd34c3 [FIXED] JetStream: Sources with OptStartTime gets redelivered
If start by time is before what we remember during recovery use that instead

Resolves #3559

Signed-off-by: Derek Collison <derek@nats.io>
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-03 16:09:06 -06:00
Derek Collison
c9fd776889 Merge pull request #3605 from nats-io/fix-3606
[FIXED] Notifying waiting pull consumers on consumer or parent stream deletion.
2022-11-03 13:05:47 -07:00
Derek Collison
72ff2edb5f Fix for #3603.
Signed-off-by: Derek Collison <derek@nats.io>
2022-11-03 12:46:41 -07:00
Ivan Kozlovic
91c84c03c2 [FIXED] LeafNode: possible duplicate messages in complex setup
This is specific to setup described [here](https://github.com/nats-io/nats-server/issues/3191#issuecomment-1296974382)
and does not require JetStream to be reproduced. The added test
reproduces the above setup but without JetStream enabled in
the accounts.

Each cluster has a leafnode for a given account to the other
cluster. The accounts import/export a subject. When a consumer
is connected to cluster "B" and the producer is on cluster "A"
there was a duplicate message. Due to shadow subscription caused
by the import/export rules, an additional subscription was
sent across the leafnode.

Resolves #3191

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-03 12:34:01 -06:00
Derek Collison
56919ebc97 On stream proposal failures we could accidentally warn on high stream lag.
We were not taking the clfs into account.

Signed-off-by: Derek Collison <derek@nats.io>
2022-11-02 14:40:31 -07:00
Ivan Kozlovic
ab4470ccdc [FIXED] JetStream: possible panic on some rare cases
Very difficult to reproduce. Had to run TestJetStreamSuperClusterMoveCancel
in covermode=atomic on a slow machine to hit the condition where
the monitorConsumer go routine is started by RAFT node is nil,
which caused the warning message to produce the panic (since n is nil)

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-02 10:02:09 -06:00
Ivan Kozlovic
2321eafb0c Bump version to 2.9.6-beta as per release process
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-01 16:23:50 -06:00
Ivan Kozlovic
c16f60add5 Release v2.9.5
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-01 15:15:06 -06:00
Ivan Kozlovic
55e651c118 [FIXED] JetStream: processing of snapshot with expired messages
The issue that a "first sequence mismatch" during processing of
a snapshot was causing the state to be reset and caused a lot
of catchup from the follower. An attempt to fix that in PR #3567
caused an issue that was addressed in PR #3589. However, this was
then causing the follower to sometime never able to catchup or
took a very long time.
This PR - we believe - addresses the original and subsequent issues.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-11-01 12:58:45 -06:00
Derek Collison
1fa5e73177 Honor MaxMsgsPerSubject when a stream config is updated, including enforcing a lower limit.
Signed-off-by: Derek Collison <derek@nats.io>
2022-10-31 17:25:20 -07:00
Ivan Kozlovic
fe588dc9ea Fixing a flapper
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-28 13:45:58 -06:00
Александр Петров
8eb430bfff add test 2022-10-28 20:16:40 +05:00
Александр Петров
19f63bee65 less optimistic cache hit rate 2022-10-28 11:36:46 +05:00
Ivan Kozlovic
3f0f3d7416 Bump version to 2.9.5-beta.1
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-27 19:15:01 -06:00
Derek Collison
121bf6ebb5 Move to past check for nil
Signed-off-by: Derek Collison <derek@nats.io>
2022-10-27 17:30:07 -07:00
Ivan Kozlovic
3a5f465115 Bump to 2.9.5-beta as per release process
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-27 17:15:00 -06:00
Ivan Kozlovic
f101e815fe Release v2.9.4
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-27 16:10:39 -06:00