Commit Graph

4865 Commits

Author SHA1 Message Date
Derek Collison
01a2c0472d Merge branch 'main' into dev 2023-04-03 15:33:12 -07:00
Derek Collison
7d3ec51d79 Fix for flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 14:46:59 -07:00
Derek Collison
59175c491f Fix for a datarace
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 14:46:57 -07:00
Derek Collison
b806a8e7e7 Do not opt-out of normal processing for leadership transfers, but make sure they are only processed if explicitly new
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 14:46:55 -07:00
Derek Collison
9dd727034a Make sure to not stop raft layer when we detect we are already running the monitor
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 14:46:47 -07:00
Derek Collison
6b01a21965 No inline jetstream msg processing, always queue inbound
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 08:27:52 -07:00
Derek Collison
70f6caa060 Merge branch 'main' into dev 2023-04-02 19:58:20 -07:00
Derek Collison
94278e731a More tweaks to test due to slow network proxy being more accurate
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 19:57:34 -07:00
Derek Collison
66e7b6f4bd Bump to 2.9.16-RC.4
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 19:49:53 -07:00
Derek Collison
cf462afa9e Merge branch 'main' into dev 2023-04-02 19:06:58 -07:00
Derek Collison
5afcb6c13c Fix for flapping test, network proxy more accurate now so rtt needed to be tweaked
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 19:06:42 -07:00
Derek Collison
b9e7b58f5c Merge branch 'main' into dev 2023-04-02 18:58:54 -07:00
Derek Collison
b2b3ed9a03 Cherry-pick #3733 (Refactor outbound queues, remove dynamic sizing, add buffer reuse) (#3965)
This brings #3733 forward from `dev` into `main`, to go into the next
release.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-04-02 18:56:28 -07:00
Derek Collison
ff3f102cdd Fix for datarace in healthcheck
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 16:30:13 -07:00
Derek Collison
d5ac4d283a Fix for flapping test, can return invalid sequence as well
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 16:18:23 -07:00
Derek Collison
64b22011dc Better use of LoadAndStore based on review feedback
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 14:50:22 -07:00
Derek Collison
1fb1efd748 Make sure to remove any inflight entries when done
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 14:41:49 -07:00
Derek Collison
e6447c982a Protect against concurrent creation of streams and consumers.
Also make sure we have exited monotoring routines when doing resets for both streams and consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 14:29:52 -07:00
Derek Collison
f3cab83ccf Bump to 2.9.16-RC.3
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 04:24:21 -07:00
Derek Collison
48a5d270b2 Merge branch 'main' into dev 2023-04-02 04:23:52 -07:00
Derek Collison
299d9b28a9 Add original message timestamp to republished message headers (#3933)
Adds original message timestamp to republished message as
"Nats-Time-Stamp" header

 - [ ] Link to issue, e.g. `Resolves #NNN`
 - [ ] Documentation added (if applicable)
 - [x] Tests added
- [x] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [x] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [ ] Build is green in Travis CI
- [x] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

Resolves #

### Changes proposed in this pull request:

- Add original message timestamp to republished message as header
"Nats-Time-Stamp"
 -
 -
2023-04-02 04:05:53 -07:00
Derek Collison
58ca525b3b Process replicated ack regardless of store update. Delay but still stepdown
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:53:16 -07:00
Derek Collison
a8bd2793d5 Fix concurrent map bug on preAcks.
Use monitor check for streams like consumers.
Make sure to stop raft layer if exiting monitorConsumer early.
Allow consumers to force a snapshot on leadership change.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:53:11 -07:00
Derek Collison
874b2b2e02 Hold the lock while checking health since we could update catchup state.
Do not stepdown right away when executing leadership transfer, wait for the commit.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:53:08 -07:00
Derek Collison
b5358fa4b3 Wait for shutdown and sleep to let state build up
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:53:05 -07:00
Derek Collison
b752b8b30d Snapshot on clean shutdown if needed or interest based retention
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:53:03 -07:00
Derek Collison
e54019f87f All should be lowercase
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:53:01 -07:00
Derek Collison
872a9e7927 Add in monitor status similar to consumer
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:52:59 -07:00
Derek Collison
df4982948c Gate remove calls, disqualify delivered and ack updates quicker
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:52:56 -07:00
Derek Collison
4b8229ee42 Do not hold js lock for health check, use healthy not current for meta
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:52:54 -07:00
Derek Collison
e2839e9ec1 Fix for flapper
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:52:43 -07:00
Derek Collison
ac96d75c9c Fix bug in needAck (#4008)
needAck has reverse logic for checking single subject-single filter
    scenario.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-03-31 15:56:33 -07:00
Tomasz Pietrek
93c49c354e Fix bug in needAck
needAck has reverse logic for checking single subject-single filter
scenario.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-03-31 23:55:50 +02:00
Derek Collison
d057889b13 Merge branch 'main' into dev 2023-03-31 11:36:35 -07:00
Derek Collison
ad5bb366a0 Updates to preacks when multiple consumers are present but mutually exlusive (filtered).
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-31 10:43:28 -07:00
Derek Collison
c194047caf Bump to 2.9.16-RC.2
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-30 21:23:51 -07:00
Derek Collison
6ced966776 Merge branch 'main' into dev 2023-03-30 21:09:45 -07:00
Derek Collison
5e85889790 [IMPROVED] Improvements to preAcks. (#4006)
Better handling of multiple consumers so as to not delete messages too
early.
Better cleanup handling.

Signed-off-by: Derek Collison <derek@nats.io>
2023-03-30 21:08:34 -07:00
Derek Collison
8c0a45edf9 Make sure to lock on clearing if not removing.
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-30 20:42:28 -07:00
Derek Collison
937ef0d2a6 Improvements to preAcks.
Better handling of multiple consumers so as to not delete too early.

Signed-off-by: Derek Collison <derek@nats.io>
2023-03-30 20:29:15 -07:00
Ivan Kozlovic
db599ac875 Merge branch 'main' into dev 2023-03-30 15:45:51 -06:00
Ivan Kozlovic
a4df4f8727 Fixed some tests
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-03-30 15:02:59 -06:00
Derek Collison
3e2e2bfad9 Merge branch 'main' into dev 2023-03-29 20:34:33 -07:00
Derek Collison
4646f4af5d Do not allow any JetStream leaders to be placed on a lameduck server
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 20:15:41 -07:00
Derek Collison
873ab0f6b9 Fix for flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 18:55:41 -07:00
Derek Collison
fbc90adf93 Bump to 2.9.16-RC.1
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 17:21:57 -07:00
Derek Collison
cba0976926 Bump to 2.10.0-beta.30
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 17:20:53 -07:00
Derek Collison
625ffa8685 Merge branch 'main' into dev 2023-03-29 17:19:45 -07:00
Derek Collison
02702e4620 [IMPROVEMENT] General stability and bug fixes. (#3999)
This PR has general improvements and fixes to filestore, raft, and the
clustering layer.

Summary

1. Additional support for preAck handling for interest based streams
when replicated acks arrive before the message itself.
2. Better handling when checking state to determine whether to remove an
interest based message.
3. Improved StepDown() and leadership transfer handling after restarts.
4. Improved voting logic for high load systems.
5. Various improvements and fixes for filestore Compact(), which is used
heavily in the raft layer when updating snapshots and the raft wal.

Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 17:09:44 -07:00
Derek Collison
c546828359 Moved log running test to NoRace suite
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-29 16:56:04 -07:00