Commit Graph

7033 Commits

Author SHA1 Message Date
Waldemar Quevedo
3602b3882a Fix raft log debug reloading (#4047)
Fixes being able to reload debug logging for running raft groups.
2023-04-13 16:21:53 -07:00
Waldemar Quevedo
a4833d0889 Fix raft log debug reloading
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-04-13 14:57:04 -07:00
Neil
96fe933846 Limit filestore block size to 2MB when encryption is enabled (#4046)
Some write operations in the filestore require re-encrypting the entire
block, so having large blocks can make these operations slow and hurt
performance.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-04-13 17:24:30 +01:00
Neil Twigg
fc9b9004eb Limit filestore block size to 2MB when encryption is enabled
Some write operations in the filestore require re-encrypting the
entire block, so having large blocks can make these operations
slow and hurt performance.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-04-13 16:56:21 +01:00
Derek Collison
69b427bc94 [FIXES/IMPROVED] Raft and filestore logic (#4045)
When failing to send a snapshot to a follower, in addition to stepping
down reset our state.
For a filestore when expiring messages on recovery make sure that the
first message is not a deleted message.
Fix a bad unlock of wrong type which could cause a crash during
shutdown.
Add in raft specific test framework.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-13 07:53:13 -07:00
Derek Collison
e6eaed425f Make sure we have a correct next first after expiring on startup
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-12 14:57:13 -07:00
Derek Collison
2f4677d29e Delay a bit longer if we are not the actual leader, helpful for very large stream reports to avoid possible dupes
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-12 12:36:47 -07:00
Derek Collison
808a2e8c90 On failure to send snapshot to follower, also reset, and on reset make sure to reset term
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-12 11:48:22 -07:00
Derek Collison
a92bb9fe61 Fix bad unlock which could cause crash
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-12 11:48:22 -07:00
Derek Collison
f41e5d86e1 Add in raft test helper
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-12 11:48:22 -07:00
Derek Collison
340fcc90bc Basic raft tests
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-12 11:48:22 -07:00
Derek Collison
4fcc2ff418 Fix FirstSeq not being updated with filestore when purging subject (#4042)
Resolves https://github.com/nats-io/nats-server/issues/4041

Keeps `firstSeqNeedsUpdate` set if set previously, to ensure the
FirstSeq gets updated.
2023-04-12 11:43:50 -07:00
Tomasz Pietrek
2a03d9dff7 Fix consumer reply subject escaping (#4040)
If the Consumer had a name containing `%`, it could result in the reply
subject failing to format with `fmt.Sprintf`, as the `%` was not
properly escaped with `%%`.

Resolves #4038 

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-04-12 13:27:52 +02:00
Maurice van Veen
1e2bba4c7b Fix FirstSeq not being updated with filestore when purging subject 2023-04-12 10:46:16 +02:00
Tomasz Pietrek
692f384f2d Fix consumer reply subject escaping
If the Consumer had a name containing `%`, it could result in
reply subject failing to format with `fmt.Sprintf`, as the `%`
was not properly escaped with `%%`.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-04-12 09:22:08 +02:00
Derek Collison
31a2269710 Bump to 2.9.16-RC.7
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-09 19:32:58 -07:00
Derek Collison
30b50870a1 [IMPROVED] Memory allocations and pressure in the raft layer. (#4036)
Improved some hot spots on the memory subsystem with sync.Pools.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-09 19:21:23 -07:00
Derek Collison
623f2d5289 Fix flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-09 08:13:20 -07:00
Derek Collison
80a57a3d51 Remove peers from string intern map
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-09 08:01:36 -07:00
Derek Collison
6fa55540a7 Better us of entryPool
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-09 07:48:31 -07:00
Derek Collison
aee73a9c77 Fix flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-08 21:58:54 -07:00
Derek Collison
313dd424a3 Optimize to not allocate converting strings to []byte
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-08 20:46:05 -07:00
Derek Collison
ad08ee3898 Remove unused function
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-08 20:28:44 -07:00
Derek Collison
722a48d726 Don't require lock for check no echo status
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-08 20:05:33 -07:00
Derek Collison
c6b2a97ef4 Use entry pool
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-08 19:58:43 -07:00
Derek Collison
fa73b572e6 Optimize with Pool and avoid allocations on []byte cast of strings
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-08 19:58:09 -07:00
Derek Collison
3b9cf1e381 Needed to do more in separate go routine to avoid deadlock
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-08 18:43:58 -07:00
Derek Collison
b78ec39b1f Fix data race and simplify logic
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-08 15:04:30 -07:00
Derek Collison
35bb7c1737 Pool CommittedEntries as well with a ReturnToPool() that will also recycle the Entry. Needs to integrate with upper layers
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-08 11:34:10 -07:00
Derek Collison
ffc49b8f86 Fix flapping test and data race in test
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-08 08:13:31 -07:00
Derek Collison
3be25fdedb Do not put an appendEntryResponse back in the pool if catching up until complete
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-07 10:30:06 -07:00
Derek Collison
4edb08280f Use trick to intern strings with GC for subjString
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-07 09:14:57 -07:00
Derek Collison
2ff6f18ccd Use sync.Map for peers vs internal storage for appendEntryResponses
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-07 08:16:42 -07:00
Derek Collison
1caa56a34f Use pools for appendEntries
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-07 07:38:19 -07:00
Derek Collison
83f08999a7 Fix data race
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-07 07:30:03 -07:00
Derek Collison
d02d59534f Fix data race
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-07 07:18:30 -07:00
Derek Collison
bb2da435c2 Only cast and copy if really needed
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-07 06:53:57 -07:00
Derek Collison
3afdb99f75 Use pools for appendEntryResponses. Also use interior space for peer name from the wire
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-07 06:43:51 -07:00
Derek Collison
84c0d48815 [FIXED] Make sure on reverse match to compensate for wider target subjects. (#4032)
If we have a wider subject, meaning more tokens, but our subs end with a
wildcard token make sure that matches properly.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 16:03:52 -07:00
Derek Collison
dbde8aba36 Make sure on reverse match to compensate for wider target subjects
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 14:27:04 -07:00
Derek Collison
02122a233d [IMPROVED] Finer grained locking for /healthz with large number of JetStream assets. (#4031)
For checking the health of jetstream, do not hold the lock as we
traverse the streams and consumers.

 Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 12:37:06 -07:00
Derek Collison
c16915bff4 For checking the health of jetstream, do not hold the lock as we traverse the streams and consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 11:56:55 -07:00
Derek Collison
33451e5d56 Fix for API data race. (#4030)
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 10:45:29 -07:00
Derek Collison
f02f379468 Move stepdowns of streams and consumers to not be inline. (#4029)
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 10:45:16 -07:00
Derek Collison
a60f623842 Fix flapping test since claims not processed inline now
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 10:39:30 -07:00
Derek Collison
dee5229f9b Fix data race
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 10:37:31 -07:00
Derek Collison
0d2269b1e9 Move stepdowns of streams and consumers to not be inline
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 10:27:36 -07:00
Derek Collison
7547093b24 _EMPTY_
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 08:38:30 -07:00
Derek Collison
1e80ea68b3 [IMPROVED] Optimize non-inline direct gets to not use simple go routines. (#4028)
This was causing regression in certain KV performance tests.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 08:35:37 -07:00
Derek Collison
2da50512e2 Optimize non-inline direct gets to not use simple go routines
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 07:50:57 -07:00