Commit Graph

4639 Commits

Author SHA1 Message Date
Derek Collison
ebe4f8957f Spelling based on review feedback
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 21:08:59 -07:00
Derek Collison
dcbefd5cc4 We can receive these on push consumers, so error if we do
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 21:07:08 -07:00
Derek Collison
07b34f707f Make sure to never process next message requests inline
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 20:50:01 -07:00
Derek Collison
b0c3cf0dbd Only apply consumer entries if not recovering
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 17:22:50 -07:00
Derek Collison
7d3ec51d79 Fix for flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 14:46:59 -07:00
Derek Collison
59175c491f Fix for a datarace
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 14:46:57 -07:00
Derek Collison
b806a8e7e7 Do not opt-out of normal processing for leadership transfers, but make sure they are only processed if explicitly new
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 14:46:55 -07:00
Derek Collison
9dd727034a Make sure to not stop raft layer when we detect we are already running the monitor
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 14:46:47 -07:00
Derek Collison
6b01a21965 No inline jetstream msg processing, always queue inbound
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 08:27:52 -07:00
Derek Collison
94278e731a More tweaks to test due to slow network proxy being more accurate
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 19:57:34 -07:00
Derek Collison
66e7b6f4bd Bump to 2.9.16-RC.4
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 19:49:53 -07:00
Derek Collison
5afcb6c13c Fix for flapping test, network proxy more accurate now so rtt needed to be tweaked
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 19:06:42 -07:00
Derek Collison
b2b3ed9a03 Cherry-pick #3733 (Refactor outbound queues, remove dynamic sizing, add buffer reuse) (#3965)
This brings #3733 forward from `dev` into `main`, to go into the next
release.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-04-02 18:56:28 -07:00
Derek Collison
ff3f102cdd Fix for datarace in healthcheck
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 16:30:13 -07:00
Derek Collison
d5ac4d283a Fix for flapping test, can return invalid sequence as well
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 16:18:23 -07:00
Derek Collison
64b22011dc Better use of LoadAndStore based on review feedback
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 14:50:22 -07:00
Derek Collison
1fb1efd748 Make sure to remove any inflight entries when done
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 14:41:49 -07:00
Derek Collison
e6447c982a Protect against concurrent creation of streams and consumers.
Also make sure we have exited monotoring routines when doing resets for both streams and consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 14:29:52 -07:00
Derek Collison
f3cab83ccf Bump to 2.9.16-RC.3
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 04:24:21 -07:00
Derek Collison
58ca525b3b Process replicated ack regardless of store update. Delay but still stepdown
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:53:16 -07:00
Derek Collison
a8bd2793d5 Fix concurrent map bug on preAcks.
Use monitor check for streams like consumers.
Make sure to stop raft layer if exiting monitorConsumer early.
Allow consumers to force a snapshot on leadership change.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:53:11 -07:00
Derek Collison
874b2b2e02 Hold the lock while checking health since we could update catchup state.
Do not stepdown right away when executing leadership transfer, wait for the commit.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:53:08 -07:00
Derek Collison
b5358fa4b3 Wait for shutdown and sleep to let state build up
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:53:05 -07:00
Derek Collison
b752b8b30d Snapshot on clean shutdown if needed or interest based retention
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:53:03 -07:00
Derek Collison
e54019f87f All should be lowercase
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:53:01 -07:00
Derek Collison
872a9e7927 Add in monitor status similar to consumer
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:52:59 -07:00
Derek Collison
df4982948c Gate remove calls, disqualify delivered and ack updates quicker
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:52:56 -07:00
Derek Collison
4b8229ee42 Do not hold js lock for health check, use healthy not current for meta
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:52:54 -07:00
Derek Collison
e2839e9ec1 Fix for flapper
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:52:43 -07:00
Derek Collison
ad5bb366a0 Updates to preacks when multiple consumers are present but mutually exlusive (filtered).
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-31 10:43:28 -07:00
Derek Collison
c194047caf Bump to 2.9.16-RC.2
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-30 21:23:51 -07:00
Derek Collison
5e85889790 [IMPROVED] Improvements to preAcks. (#4006)
Better handling of multiple consumers so as to not delete messages too
early.
Better cleanup handling.

Signed-off-by: Derek Collison <derek@nats.io>
2023-03-30 21:08:34 -07:00
Derek Collison
8c0a45edf9 Make sure to lock on clearing if not removing.
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-30 20:42:28 -07:00
Derek Collison
937ef0d2a6 Improvements to preAcks.
Better handling of multiple consumers so as to not delete too early.

Signed-off-by: Derek Collison <derek@nats.io>
2023-03-30 20:29:15 -07:00
Ivan Kozlovic
a4df4f8727 Fixed some tests
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-03-30 15:02:59 -06:00
Derek Collison
4646f4af5d Do not allow any JetStream leaders to be placed on a lameduck server
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 20:15:41 -07:00
Derek Collison
873ab0f6b9 Fix for flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 18:55:41 -07:00
Derek Collison
fbc90adf93 Bump to 2.9.16-RC.1
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 17:21:57 -07:00
Derek Collison
02702e4620 [IMPROVEMENT] General stability and bug fixes. (#3999)
This PR has general improvements and fixes to filestore, raft, and the
clustering layer.

Summary

1. Additional support for preAck handling for interest based streams
when replicated acks arrive before the message itself.
2. Better handling when checking state to determine whether to remove an
interest based message.
3. Improved StepDown() and leadership transfer handling after restarts.
4. Improved voting logic for high load systems.
5. Various improvements and fixes for filestore Compact(), which is used
heavily in the raft layer when updating snapshots and the raft wal.

Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 17:09:44 -07:00
Derek Collison
c546828359 Moved log running test to NoRace suite
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 16:56:04 -07:00
Derek Collison
ade0e9d295 Snapshot meta for this function to use in case it gets removed out from underneath of us.
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 16:51:17 -07:00
Derek Collison
9a714e7d7d Update based on review feedback
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 15:47:54 -07:00
Derek Collison
152b25c314 Update server/stream.go
Pre-allocate

Co-authored-by: Neil <neil@nats.io>
2023-03-29 15:29:51 -07:00
Derek Collison
c77872b519 Update server/jetstream_cluster.go
Pre-allocate

Co-authored-by: Neil <neil@nats.io>
2023-03-29 15:29:38 -07:00
Derek Collison
2b89fea9b0 Double check here if the jetstream cluster was shutdown when we released the lock
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 14:46:49 -07:00
Derek Collison
e274693490 On bad or corrupt message load during commit, reset WAL vs mark write error
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 14:07:14 -07:00
Derek Collison
6c3e64b83b Always make sure cluster and meta raft node available when needed
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 13:56:04 -07:00
Derek Collison
ddfa5cdfec Additional protection for bad state when rebuilding a message block
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 12:44:11 -07:00
Derek Collison
a9a4df859f Fix for flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 12:44:08 -07:00
Derek Collison
35d1a7747a Snapshots of no length can hold state as well
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 12:44:04 -07:00