Commit Graph

6766 Commits

Author SHA1 Message Date
Derek Collison
adebb650d4 Merge pull request #3900 from nats-io/kv-accounting
[FIXED] KV accounting errors under heavy concurrent usage
2023-02-22 14:42:19 -07:00
Derek Collison
d03d8e9d93 When having a max msgs per subject (e.g. KV) under heavy concurrent usage could skew the accounting for the underlying filestore.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-22 12:50:43 -08:00
Derek Collison
591f2b754d Merge pull request #3898 from nats-io/neil/consumercompact
Install snapshot and compact when WAL grows, even when no state changes occur
2023-02-22 13:18:01 -07:00
Neil Twigg
cfea34c80c Install snapshot and compact when WAL grows, even when no state changes occur 2023-02-22 20:00:57 +00:00
Derek Collison
b19077c2e9 Bump to 2.9.15-RC.1
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-21 08:35:30 -08:00
Derek Collison
baf2e47c06 Merge pull request #3894 from nats-io/neil/ipqueue
Refactor `ipQueue` to use generics
2023-02-21 09:31:26 -07:00
Neil Twigg
68961ffedd Refactor ipQueue to use generics, reduce allocations 2023-02-21 14:50:09 +00:00
Derek Collison
18b5aca499 Merge pull request #3892 from nats-io/consumer-fixes
[FIXED] Consumer fixes and improvements on state management.
2023-02-20 19:17:39 -07:00
Derek Collison
3c64d07691 Warn of consumer state update failures.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-20 17:28:11 -08:00
Derek Collison
53cf6635e2 Merge pull request #3893 from nats-io/raft-fixes
[FIXED] Spurious "Error storing entry to WAL" log messages
2023-02-20 18:23:40 -07:00
Derek Collison
2a9f085d55 Merge branch 'main' into raft-fixes 2023-02-20 15:43:46 -07:00
Derek Collison
e028b7230a Need to compact wal on snapshot to pindex+1
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-20 14:37:37 -08:00
Neil
7e67ec5b1f Merge pull request #3891 from nats-io/neil/pindex
Off-by-one on RAFT pindex
2023-02-20 22:35:57 +00:00
Derek Collison
ad8aa7c0c5 Merge pull request #3887 from MauriceVanVeen/main
[FIXED] Using partition with multiple wildcard token positions
2023-02-20 13:59:49 -07:00
Derek Collison
d2179e0939 Make sure to also cleanup pending if below our stream ack floor
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-20 11:56:37 -08:00
Derek Collison
b6149c51f0 Make sure to clean up redelivered state on purge.
Make sure to update ack floors on messages being expired out from underneath of us.

Signed-off-by: Derek Collison <derek@nats.io>
2023-02-20 11:16:12 -08:00
Neil Twigg
20980dc324 Off-by-one on RAFT pindex 2023-02-20 18:16:10 +00:00
Maurice van Veen
05695d304c Fixed a bug where partition was used with multiple wildcard token position 2023-02-20 10:27:29 +01:00
Derek Collison
6c9a9fb45e Fixed bug that would lose ack pending state during partial stream purge.
General code cleanup to be more correct.

Signed-off-by: Derek Collison <derek@nats.io>
2023-02-19 14:21:53 -08:00
Derek Collison
5c6b3b620a Merge pull request #3885 from nats-io/jarema/improve-interest-test
Improve test for consumer with inactivity threshold
2023-02-19 10:36:36 -07:00
Tomasz Pietrek
337a9f2cbd Improve test for consumer with inactivity threshold
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-02-19 17:57:09 +01:00
Derek Collison
8fc34596f1 Merge pull request #3886 from nats-io/process-connect-fix-main
Check if connection name was already set when storing it
2023-02-19 09:45:17 -07:00
Waldemar Quevedo
beb179ec15 Check if connection name was already set when storing it
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-02-19 07:58:56 -08:00
Derek Collison
b3f0dbdde4 Merge pull request #3884 from nats-io/fix-consumer-cleanup
[FIXED] Named push consumers with inactive thresholds deleted when still active.
2023-02-19 08:51:37 -07:00
Derek Collison
06fd81d096 Fixed a bug where a named consumer under interest policy was spinning up inactive threshold timers in all replicas not just the leader.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-19 06:08:43 -08:00
Derek Collison
9de8c08537 Merge pull request #3882 from nats-io/replica-warn
[FIXED] Do not warn if consumer replicas configured to 0
2023-02-18 13:02:03 -07:00
Derek Collison
e270e9538f Do not warn if consumer replicas condigured to 0
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-18 11:50:26 -08:00
Derek Collison
35526b0433 Merge pull request #3881 from nats-io/bad-consumer-delete
[FIXED] Do not delete consumers on restarts with non-fatal update errors.
2023-02-18 12:31:02 -07:00
Derek Collison
6a62ac4560 Fix for merge conflict
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-18 11:12:15 -08:00
Derek Collison
6a4c61e1a3 Merge branch 'main' into bad-consumer-delete 2023-02-18 11:09:56 -08:00
Derek Collison
7f5bef4b0f Merge pull request #3880 from nats-io/p-consumer-create
[IMPROVED] Parallel consumer
2023-02-18 10:47:12 -07:00
Derek Collison
01fa89a0b4 Fix for deleting consumers on restarts and non-fatal update errors.
If there was a spurious error on restart, or possibly on an update, we could delete a consumer which was the incorrect behavior.

Signed-off-by: Derek Collison <derek@nats.io>
2023-02-18 09:46:52 -08:00
Derek Collison
efa3bcc49d Parallel consumer creation could drop responses (create and info) and could also run monitorConsumer twice.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-18 05:16:05 -08:00
Derek Collison
689d521c82 Merge pull request #3878 from nats-io/test-fix
Fix to flapping test to make sure we do not quickly blow away all consumer state.
2023-02-17 15:53:36 -07:00
Derek Collison
2d794d09e1 Fix to flapping test to make sure we do not quickly blow away all consumer state.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-17 14:23:34 -08:00
Derek Collison
52e5995a08 Merge pull request #3877 from nats-io/consumer-info-optimization
[IMPROVED] ConsumerInfo request processing.
2023-02-17 11:30:47 -07:00
Derek Collison
11b0f214d0 Do not re-calculate NumPending on consumer info calls.
We noticed this was being called alot in user environments.
When the consumer was filtered with a wilcard and the stream had a high cardinality of subjects and was falling behind this could take a substantial amount of time.

Signed-off-by: Derek Collison <derek@nats.io>
2023-02-16 16:30:14 -08:00
Derek Collison
b91143ad14 Merge pull request #3876 from nats-io/purge-accounting
[FIXED] Extended Purge accounting was not updating account usage.
2023-02-16 17:25:22 -07:00
Derek Collison
0cb01f9e7a Make sure we update storage accounting on extended version purge for filestore.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-16 13:18:40 +04:00
Derek Collison
b3b9e888f3 Merge pull request #3873 from nats-io/diskio-test
[FIXED] Adjusted test to correspond to new limit of 1024.
2023-02-15 20:45:15 -07:00
Derek Collison
98e6b2059d Merge pull request #3872 from nats-io/js-parallel-test-race
Fix TestJetStreamParallelConsumerCreation race
2023-02-15 20:26:20 -07:00
Derek Collison
32b5ec16dd Fixed test to correspond to new limit of 1024.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-16 07:16:19 +04:00
Waldemar Quevedo
4452f64d73 Fix TestJetStreamParallelConsumerCreation race
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-02-15 17:23:48 -08:00
Derek Collison
345496f331 Merge pull request #3867 from nats-io/improvements
Improvements to Filestore
2023-02-14 05:35:40 -07:00
Derek Collison
3bc0af70d0 Only update per subject information if we know we have an update.
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-13 20:12:35 +02:00
Derek Collison
e2f9fb73a9 Increase limit for internal blocking IO vs num cores.
Will help in slow IO situations and lots of expire cache calls.

Signed-off-by: Derek Collison <derek@nats.io>
2023-02-13 18:44:25 +02:00
Derek Collison
1e3c2810f4 Improve expireMsgs minAge calculation for when lots of messages to expire in each callback.
This happens when under extreme load as shown in the skipped test.

Signed-off-by: Derek Collison <derek@nats.io>
2023-02-13 18:39:39 +02:00
Neil
f3fa39c321 Merge pull request #3862 from nats-io/neil/accountstackoverflow
Fix stack overflow panic in reverse entry check when inbox ends with wildcard
2023-02-13 10:54:50 +00:00
Neil Twigg
ba0b1f8bd3 Fix stack overflow panic in reverse entry check when inbox ends with wildcard 2023-02-13 09:45:50 +00:00
Derek Collison
7afddb3aac Merge pull request #3860 from nats-io/compact-subjects-fix
Make sure we adjust per subject info when doing a Compact().
2023-02-10 08:16:57 -07:00