Commit Graph

3037 Commits

Author SHA1 Message Date
Derek Collison
fd9e9480da Merge pull request #2333 from nats-io/srace
Check not really needed here, so removed
2021-07-01 06:19:46 -07:00
Derek Collison
ad4685c84f Fix for crash in test run
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-30 19:30:37 -07:00
Derek Collison
c2c146c9f2 Fix for #2329.
When we created a filestore we would figure out if we should track by subject based on stream config.
This would cause bad results when a stream was updated to multiple subjects or wildcards.
This change tightens when and what we track but turns it on all the time now.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-30 19:10:31 -07:00
Derek Collison
617f061084 Check not really needed here, so removed
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-30 15:08:56 -07:00
Ivan Kozlovic
13334c227d Release v2.3.1
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-06-29 13:51:32 -06:00
Derek Collison
35f6be2056 If dirty flag set always write state out
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-29 12:25:28 -07:00
Derek Collison
a78f155733 2.3.1-RC.1
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-29 08:36:00 -07:00
Derek Collison
26e63a14a6 Merge pull request #2324 from nats-io/large-r1
Improvements to large numbers of JetStream R1 consumers per stream.
2021-06-29 08:35:17 -07:00
Derek Collison
3c16f776dc Protect account prand, use faster Uint64 variant
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-29 06:13:41 -07:00
Derek Collison
99fed910f0 Improvements to large numbers of JetStream R1 consumers per stream.
1. We were holding open FDs longer than we should for consumers causing issues with open FD limits. We now do not hold them open and cap updates a bit better.

2. When doing a stream delete, consumer delete was repeating alot of work that was not necessary, causing longer delays. This has been optimized a bit, still more improvements to be made.

3. We cover all JS under a single export, but that was also trapping GetNext for pull based consumers, and since this was a no-op (is handled at user account level) we were creating alot of garbage service import responses and reverse map entries that had to be garbage collected. We have a fix in to avoind this but still looking for a better one.

4. Still had some lingering references to all exports vs single JS export.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-29 05:45:55 -07:00
Derek Collison
bb84ef7d91 Added ability to match based on last expected sequence per subject.
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-28 10:57:50 -07:00
R.I.Pienaar
0d71d35e43 do not log at Error level for some store failures
Some of these are quite generic errors that can happen a lot
in normal circumstances so no need to be too noisy about them

Fixes one missed old style Api Error

Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-06-28 10:18:16 +02:00
Derek Collison
cbb97a4f4a Fix for Go client behavior change
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-27 10:52:03 -07:00
Derek Collison
63479ff8fd Bump threshold
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-27 08:33:46 -07:00
Derek Collison
146c249d55 2.3.1-beta.1
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-25 13:57:16 -07:00
Derek Collison
545230bd9e FIXED Clustered streams can become broken with seq mismatch state on low level store failures.
We can account for high level failure properly but were not properly accounting for low level store failures.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-25 13:46:40 -07:00
Derek Collison
08197de9e0 FIXED max consumers was not enforced when set on stream
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-25 11:45:36 -07:00
Derek Collison
b596eb0a40 Bump to 2.3.1
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-24 14:04:20 -07:00
Derek Collison
c0e47966ab Added in Stream get last message by subject.
This is to aid in K/V overlay for simple Get ops vs creating a full blown consumer.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-24 13:21:39 -07:00
Derek Collison
a27f198b83 Skip for now, covermode blows up memory and latency thresholds
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-23 13:50:14 -07:00
Ivan Kozlovic
f9cc49a610 Release v2.3.0
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-06-23 12:50:19 -06:00
Derek Collison
225c8b4a85 Bump threshold
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-22 17:44:19 -07:00
Derek Collison
1fcb7cc695 Bump to 2.3.0-RC.1
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-22 16:14:24 -07:00
Derek Collison
87f44ad333 Merge pull request #2306 from nats-io/fspm
Improvements to filtered purge and general memory usage for filestore.
2021-06-22 16:13:38 -07:00
Derek Collison
6bbc29281c Make sure to return tmp bufs to pool when we can
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-22 16:11:21 -07:00
Derek Collison
b3753aba1b Improvements to filtered purge and general memory use for filestore.
We optimized the filtered purge to skip msgBlks that are not in play.
Also optimized msgBlock buffer usage by using two sync.Pools to enhance reuse.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-22 15:47:26 -07:00
Ivan Kozlovic
d7933631a9 [FIXED] Failed route TLS handshake would leave failed conn's lock, locked
This is a regression introduced in v2.2.6.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-06-22 14:05:43 -06:00
Ivan Kozlovic
23ec1daab5 Bump to beta 16 and ran go mod tidy
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-06-22 11:42:43 -06:00
Ivan Kozlovic
216c3d01ed Merge pull request #2304 from nats-io/clamp_max_subs_errors
[CHANGED] Reduce print for an account subs limit to every 2 sec
2021-06-22 11:39:57 -06:00
Derek Collison
2da8718a96 2.3.0-beta.14
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-22 10:36:07 -07:00
Derek Collison
7739eae45a Merge pull request #2302 from nats-io/js-encryption
JetStream Encryption at Rest
2021-06-22 10:35:02 -07:00
Ivan Kozlovic
1d3cddfa7c [CHANGED] Reduce print for an account subs limit to every 2 sec
We could make it for all limits by having a map of error types
instead of applying just to max subs.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-06-22 11:00:41 -06:00
Ivan Kozlovic
364d9600a6 Merge pull request #2301 from nats-io/fix_2282
[FIXED] Clients disconnected on reload when only $SYS account configured
2021-06-22 09:29:56 -06:00
Derek Collison
3185658f42 Merge pull request #2299 from ripienaar/slow_purge
test to show slow purges
2021-06-22 07:43:52 -07:00
Derek Collison
2e145196b8 Fix for extended purge by sequence.
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-22 07:38:30 -07:00
R.I.Pienaar
c6b85fd101 update for review
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-06-22 08:47:08 +02:00
Derek Collison
bf6335dff9 Add in ability to have encrypted JetStream filestores.
This supports XChaChaPoly1305 for Seal and Open and ChaCha20 for our message blocks which use highway hashes and sequence numbers for authenticity.
We support snapshot and restore as well.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-21 19:28:10 -07:00
Ivan Kozlovic
a39b549513 [FIXED] Clients disconnected on reload when only $SYS account configured
This was introduced by PR#1900 by detecting that a single system
account was configured. A "fake" user name was created and added
to the list of users and set as a no_auth_user.
However, on config reload, a new fake user was created, which
would cause existing connected clients to be disconnected.

Resolves #2282

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-06-21 14:03:42 -06:00
Ivan Kozlovic
189336417f [FIXED] Websocket compression/decompression issue with continuation frames
For compression, continuation frames had the compress bit set, which is
wrong since only the first frame should.
For decompression, continuation frames were decompressed individually
instead of assembling the full payload and then decompressing.

Resolves #2287

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-06-21 11:09:19 -06:00
R.I.Pienaar
c9bf329a99 test to show slow purges
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-06-21 17:01:49 +02:00
Derek Collison
6129562b63 2.3.0-beta.12
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-20 11:00:42 -07:00
Derek Collison
5fcd9c5564 Add test for sequence only
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-20 10:42:35 -07:00
Derek Collison
89d930fd0f Updates and fixes to PurgeEx
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-20 10:34:27 -07:00
Derek Collison
9398c3ca28 Allow for more advanced purge operations that filter by subject, specify the sequence or number of messages to keep.
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-19 07:04:44 -07:00
R.I.Pienaar
53ae604403 further tagged error confusion cleanups
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-06-18 21:35:26 +02:00
R.I.Pienaar
a0fcf0bb65 further tagged error confusion cleanups
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-06-18 20:11:09 +02:00
Derek Collison
e7fd6961d6 Fix data race condition
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-18 07:47:19 -07:00
Derek Collison
36f2dbed8b Merge pull request #2292 from dtest11/js_config_check
remove config !nil check,beacuse the if branch is always not nil
2021-06-18 07:36:30 -07:00
Derek Collison
c75d97a0d3 Fix for #2290
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-18 07:03:51 -07:00
dtest11
a268905cd5 remove config !nil check,beacuse the if branch is always not nil 2021-06-18 18:02:45 +08:00