Commit Graph

4452 Commits

Author SHA1 Message Date
Derek Collison
598e27243f Fix possible deadlock on dataloss
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-30 19:10:19 -08:00
Waldemar Quevedo
13372508e2 Fix for isGroupLeaderless when JS not available (due to shutdown)
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-01-30 15:29:42 -08:00
Derek Collison
52a78c0352 Small optimizations.
1. Only snapshot with minSnap time window like consumers and meta. Make it consistent for all to 5s.
2. Only snapshot at the end of processing all entries pending vs inside the loop.
3. Use fast state when calculating sync request, do not need deleted details there.

Signed-off-by: Derek Collison <derek@nats.io>
2023-01-29 10:58:00 -08:00
Derek Collison
e1a2da8d85 Small optimizations
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-28 11:45:47 -08:00
Derek Collison
bf49f23bb1 Only hold on to so many pending in memory, will fetch from WAL
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-28 11:34:55 -08:00
Derek Collison
64096e5ef5 Fix for flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-27 13:19:44 -08:00
Derek Collison
f4e6481ce7 Allow report cycles between source streams if subjects truly form a cycle.
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-27 13:03:24 -08:00
Derek Collison
7682eaa389 Merge pull request #3804 from madper/dragonflybsd
Add support for dragonfly bsd.
2023-01-27 08:35:42 -08:00
Chengjun Xie
6cbca6b783 Add support for dragonfly bsd.
Simply copy the pse implementation for openbsd.
2023-01-27 10:43:30 +08:00
Derek Collison
8a40326a70 Bump to 2.9.12-RC.1
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-26 10:07:30 -08:00
Derek Collison
c7a75c5a6d Merge pull request #3817 from nats-io/force-consumer-replicas
[FIXED] Force consumer replicas to match for interest policy streams
2023-01-26 09:39:15 -08:00
Derek Collison
3d78459ad1 Fixup for bad merge
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-26 09:09:30 -08:00
Neil Twigg
83932b4be6 Don't mark a clustered stream as unhealthy if making forward progress, add TestJetStreamClusterCurrentVsHealth 2023-01-26 16:57:34 +00:00
Derek Collison
d0a7a8169a Merge branch 'main' into force-consumer-replicas 2023-01-26 08:35:49 -08:00
Derek Collison
461aad17a5 Merge pull request #3820 from nats-io/issue-3791
[FIXED] Select consumer peer(s) from active peers only.
2023-01-26 08:27:11 -08:00
Derek Collison
e15eb22ca6 When we create a consumer with less replicas then the stream, make sure to select from online peers.
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-25 20:08:04 -08:00
Derek Collison
a5cbd0b029 Fixed a bug that would not properly process updates on a stream on restart.
During restart if the stream existed but was also in a meta-snapshot delivered by the leader we would not process the update properly.

Signed-off-by: Derek Collison <derek@nats.io>
2023-01-25 18:16:33 -08:00
Derek Collison
4a3c27a251 Fix MQTT test for consumer replica override.
This was ill-advised by me, not understanding that the messages stream for MQTT was interested policy based.
Interest policy based streams require consumers to match the replica count.

Signed-off-by: Derek Collison <derek@nats.io>
2023-01-25 17:58:57 -08:00
Derek Collison
bcd53ba53a Merge pull request #3808 from nats-io/jarema/fix-not-assigning-sig-sub
Assign signal subscription to consumer when created
2023-01-24 07:12:53 -08:00
Tomasz Pietrek
c89883b6aa Fix assigning signal subscription to consumer
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-01-24 12:26:22 +01:00
Derek Collison
f62d929018 Consumer must match replica of parent stream if interest based policy.
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-23 20:16:42 -08:00
Derek Collison
f4ee6530a0 When updating a stream to Direct Gets we were not spinning up subscription endpoint properly.
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-23 16:51:07 -08:00
Ivan Kozlovic
79ca0c1787 Move test to "norace_test.go"
The test TestJetStreamClusterConsumerListPaging was in the
jetstream_cluster_3_test.go and because of `-race` flag would
take more than 440 seconds (7+ minutes) as seen here:

https://app.travis-ci.com/github/nats-io/nats-server/jobs/593984385#L335

Without the `-race` flag, this test takes ~17 seconds.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-01-23 17:05:18 -07:00
Derek Collison
68b4570226 Fix for filtered state for all subjects when the first sequence(s) are deleted.
Discovered doing the optimizations for interior deletes.

Signed-off-by: Derek Collison <derek@nats.io>
2023-01-21 16:01:44 -08:00
Derek Collison
2aeb5e2c5a Update snapshots to numCores and maxProcs after maxrocs.Set()
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-20 11:30:43 -08:00
Neil Twigg
1baa1fbda8 Use highwayhash for last stream, consumer and cluster snapshots 2023-01-12 16:16:14 +00:00
peaaceChoi
038037381b Fix some typos in code comment 2023-01-12 10:31:32 +09:00
Byron Ruth
f3c0890a97 Bump to 2.9.12-beta
Signed-off-by: Byron Ruth <byron@nats.io>
2023-01-06 12:40:19 -05:00
Byron Ruth
3491f43199 Release v2.9.11
Signed-off-by: Byron Ruth <byron@nats.io>
2023-01-06 12:24:36 -05:00
Neil Twigg
d853b0eb89 Deduplicate *subscription in Sublist 2023-01-06 13:19:20 +00:00
Derek Collison
f5d939ec24 Added test for #3636
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-05 10:56:52 -08:00
Derek Collison
9a69172236 Bump to 2.9.11-RC.1
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-05 10:33:47 -08:00
Derek Collison
ad53d455f8 When migrating leaders off a server when the leafnode is not connected, also ensure leaders can not return until reconnected.
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-05 08:02:50 -08:00
Derek Collison
6c5f0a669d Ensure we add in new consumers from a meta snapshot from the leader.
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-04 22:18:31 -08:00
Derek Collison
d9cb1e6286 Fix for #3734
When a msg blk was not writen correctly, but the idx file was, max bytes for a stream would no longer be honored since the deletion of any messages in that empty block were not being handled properly.

Signed-off-by: Derek Collison <derek@nats.io>
2022-12-31 18:50:46 -08:00
Neil
1d53b47b33 Merge pull request #3754 from nats-io/neil/golangcilint
Switch to `golangci-lint`
2022-12-30 20:12:03 +00:00
Neil Twigg
14d0ba1c65 Fix some lint errors after move to golangci-lint 2022-12-30 20:00:08 +00:00
Derek Collison
713f632fa7 If a stream's meta was not properly written but the file existed, we could re-add the stream but a subsequent restart would lose the stream again.
Also added in healthz for single server systems to make sure all stream directories resulted in recovered streams.

Signed-off-by: Derek Collison <derek@nats.io>
2022-12-29 20:08:56 -08:00
Waldemar Quevedo
861f98128a Remove individual import subs from leafnodes
When a leafnode connection is bound to an account where there was already
a wildcard response import subscription to handle the requests (e.g. `_R_.foo.>`),
this would have created message duplicates due to an extra subscription
being created that also matched the wildcard (e.g. `_R_.foo.bar`).

To avoid this condition, we now skip creating the latter extra subscription
for leafnode connections.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2022-12-29 15:30:17 -08:00
Byron Ruth
1477b675ff Add back existing HealthzOptions.JSEnabled field
This fixes a backwards incompat change for library usage as well as
using the healthz NATS API which depends on the JSON payload.

Signed-off-by: Byron Ruth <byron@nats.io>
2022-12-26 08:45:22 -05:00
Byron Ruth
f6b8d9f422 Bump version to 2.9.11-beta
Signed-off-by: Byron Ruth <b@devel.io>
2022-12-20 10:56:55 -05:00
Byron Ruth
b6a19e5199 Prepare for v2.9.10 release
Signed-off-by: Byron Ruth <b@devel.io>
2022-12-20 10:33:43 -05:00
Derek Collison
c2285ab166 Merge pull request #3715 from nats-io/two-stream-same-sub
fix and test for overlapping stream subscription on update
2022-12-16 13:27:25 -08:00
Todd Beets
c463b398db Validate no overlapping stream subscriptions on update config (non-clustered jetstream) 2022-12-16 12:58:59 -08:00
Derek Collison
51341c6367 Bump to v2.9.10-beta.1
Signed-off-by: Derek Collison <derek@nats.io>
2022-12-15 10:09:13 -08:00
Derek Collison
a85318bc76 Under the right circumstances Compact could leave a danling last msg block leading to error opening msg block file [""]: open : no such file or directory
When the compaction would not uplevel to a normal purge, but after completion all msg blocks were empty the mb.lmb was not cleared or reset properly.

Signed-off-by: Derek Collison <derek@nats.io>
2022-12-15 06:59:54 -08:00
Derek Collison
786298469c Simplify locking for consumer info requests.
Signed-off-by: Derek Collison <derek@nats.io>
2022-12-14 18:52:15 -08:00
Todd Beets
47c87eb71c fix and test for clustered mem store asset no-quorum if leader restarted 2022-12-14 16:16:08 -08:00
Derek Collison
6ff03574fb Merge pull request #3710 from nats-io/sparse-latency-cp
Improve performance and latency with large number of sparse consumers.
2022-12-14 12:12:03 -08:00
Derek Collison
1a55eb5a7a Fix for condition after restart where first seq was wrong and reported zero timestamp and expiration stopped working.
Signed-off-by: Derek Collison <derek@nats.io>
2022-12-14 07:19:05 -08:00