Commit Graph

6273 Commits

Author SHA1 Message Date
Ivan Kozlovic
86d20aa5e6 [FIXED] Parsing of strings starting with numbers and K/G/etc.. suffix
If a configuration variable starts with numbers and has a character
that such as K/k/G/g/etc.. it would assume that it was a number
(expressed in Kb, Gb, etc..).

This PR checks that if the special characters are not the suffix,
that is, the variable does not end after those characters, then
the parsing will treat the whole thing as a string.

Resolves #3431

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-09-02 18:15:04 -06:00
Ivan Kozlovic
c4b5ca7cff Merge pull request #3433 from nats-io/js_offline_nodes
[FIXED] JetStream: Some nodes may never be reported as offline
2022-09-01 13:28:29 -06:00
Ivan Kozlovic
88ece75765 [FIXED] JetStream: Some nodes may never be reported as offline
In some rare situations, it is possible that nodes are added
to the cluster but are not properly tracked and not shown as
offline when they exit the cluster.

Relates to #3258

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-09-01 12:48:12 -06:00
Ivan Kozlovic
ef2099df80 Merge pull request #3430 from mprimi/js-kv-benchmarks
KeyValue Put/Get/Update benchmark
2022-09-01 10:37:35 -06:00
Marco Primi
1e040a5569 KeyValue Put/Get/Update benchmark 2022-08-31 17:12:29 -07:00
Derek Collison
4b5e0a3f87 Merge pull request #3429 from nats-io/stream-overlap-subjects
[FIXED] Do not allow overlap between multiple subjects for a stream.
2022-08-31 16:19:27 -07:00
Ivan Kozlovic
ca54468f3d Merge pull request #3425 from mprimi/js-publish-and-consume-benchmarks
Publish and consume JS benchmarks
2022-08-31 17:12:06 -06:00
Ivan Kozlovic
cc22510669 Merge pull request #3428 from nats-io/fix_3420
[FIXED] JetStream: expired accounts were counted toward limits
2022-08-31 17:06:30 -06:00
Derek Collison
452e3c3955 Make sure update covered
Signed-off-by: Derek Collison <derek@nats.io>
2022-08-31 16:05:32 -07:00
Derek Collison
b649ae8bc3 We detected overlap between different streams but not same stream with multiple subjects.
Signed-off-by: Derek Collison <derek@nats.io>
2022-08-31 15:51:39 -07:00
Ivan Kozlovic
c8af84d3d1 [FIXED] JetStream: expired accounts were counted toward limits
Fix from @JulienVdG in PR #3421. Adding simply a test and make
sure that tests run.

Resolves #3420

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-08-31 16:40:11 -06:00
Marco Primi
2c3879585c JetStream publish and consume benchmarks 2022-08-31 14:52:45 -07:00
Marco Primi
e0821cfa3d Compose cluster URL without trailing comma
Related to: https://github.com/nats-io/nats.go/issues/1056
2022-08-31 14:52:45 -07:00
Marco Primi
f1883561ee Use testing.TB interface instead of *T
Using interface allows reusing helper function in benchmarks
2022-08-31 14:52:45 -07:00
Marco Primi
28d38c0213 Move bitset test utility
From Chaos helpers (excluded from build by default) to JetStream helpers
2022-08-31 14:52:45 -07:00
Derek Collison
62f91c1dd2 Bump to 2.9.0-RC.16
Signed-off-by: Derek Collison <derek@nats.io>
2022-08-31 14:07:47 -07:00
Ivan Kozlovic
721410b23c Merge pull request #3427 from nats-io/gw_rsubs_changes
[FIXED] JetStream: possible routing issues through gateways
2022-08-31 14:52:13 -06:00
Ivan Kozlovic
8d1fb4bc92 [FIXED] JetStream: possible routing issues through gateways
Internally jetstream may subscribe to some subject and then send
a request with a reply subject matching that subscription.
Due to interest propagation through a super cluster, it is possible
that the reply comes back to a node that is not yet aware of
the subscription interest which would cause the reply to be dropped.

Some code detects that the subscription is recent and "map" the
reply subject so that it can be routed back to the origin server.
However, this was done with the use of the connection object that
created the subscription, but at the time of the send, a different
internal "*client" object may be used which would then cause
the code to not be aware of the recent subscription and not do
the mapping.

This code was changed to scope at the account level instead of
connection.

A recent change in PR #3412 is no longer needed and was reverted
in favor of changes in this PR.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-08-31 14:18:28 -06:00
Ivan Kozlovic
fdbee9bdaa Merge pull request #3424 from nats-io/js_catchup_default_max
Bump back the defaultMaxTotalCatchupOutBytes to 128MB
2022-08-31 10:18:30 -06:00
Ivan Kozlovic
5573933034 Bump back the defaultMaxTotalCatchupOutBytes to 128MB
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-08-31 09:19:28 -06:00
Derek Collison
4b9db05a0c Merge pull request #3423 from nats-io/sc
[FIXED] Subjects collide would not properly return false.
2022-08-31 06:58:13 -07:00
Derek Collison
119f1492ec [FIXED] Subjects collide would not properly return false in some conditions.
Signed-off-by: Derek Collison <derek@nats.io>
2022-08-31 06:38:43 -07:00
Derek Collison
cb3b88d4e4 Bump to 2.9.0-RC.15
Signed-off-by: Derek Collison <derek@nats.io>
2022-08-30 16:24:11 -07:00
Derek Collison
1eb1372686 Merge pull request #3419 from nats-io/migrate-v2
Allow optional compression in raft and stream catchup, move improvements for streams and consumers
2022-08-30 16:23:35 -07:00
Derek Collison
98bf861a7a Updates to stream and consumer move logic.
Signed-off-by: Derek Collison <derek@nats.io>
2022-08-30 16:11:35 -07:00
Derek Collison
56e177c329 Allow stream msgs to be compressed within the raft layer and during upper layer catchups.
Signed-off-by: Derek Collison <derek@nats.io>
2022-08-30 16:10:57 -07:00
Ivan Kozlovic
8b697db99e Merge pull request #3418 from nats-io/js_config_max_catchup
[ADDED] JetStream: Ability to configure the per server max catchup bytes
2022-08-30 15:06:19 -06:00
Ivan Kozlovic
ed4bb282f6 Rename config param to max_outstanding_catchup
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-08-30 14:51:34 -06:00
Ivan Kozlovic
9a6a2c31ee [ADDED] JetStream: Ability to configure the per server max catchup bytes
The original value was hardcoded to 128MB and 32MB per stream. The
per-server limit is lowered to 32MB but is configurable with
a new configuration parameter:
```
jetstream {
   max_catchup: 8MB
}
```

The per-stream limit was also lowered from 32MB/128,000msgs to
8MB/32,000 messages.

Tests have shown no difference in performance for fast links.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-08-30 13:46:13 -06:00
Derek Collison
a9790aa0fa Merge pull request #3409 from nats-io/cc-secure
[IMPROVED] Secure consumer create
2022-08-30 09:48:06 -07:00
Derek Collison
aa94a0bc0f New consumer create that allows elevation of stream and consumer names, and optional filter subject to the request subject.
Similar to changes in direct get allows proper security if needed for filter subject selection.

Signed-off-by: Derek Collison <derek@nats.io>
2022-08-30 09:29:38 -07:00
Ivan Kozlovic
8d33702986 Merge pull request #3411 from nats-io/js_replicas_change_stream_info
[FIXED] Stream info numbers may be 0 after cluster restart
2022-08-30 10:02:28 -06:00
Ivan Kozlovic
e609d12061 [FIXED] Stream info numbers may be 0 after cluster restart
This would happen after multiple replicas changes.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-08-30 08:49:39 -06:00
Derek Collison
d738d6fceb Bump to 2.9.0-RC.14
Signed-off-by: Derek Collison <derek@nats.io>
2022-08-30 05:49:55 -07:00
Derek Collison
840310bf25 Merge pull request #3416 from nats-io/fss-double-bug
When writing a msg after the fss state was expired we would count the msg twice.
2022-08-30 05:48:48 -07:00
Derek Collison
5f0ecef6f3 When writing a msg after the fss state was expired we would count the msg twice.
Signed-off-by: Derek Collison <derek@nats.io>
2022-08-30 05:38:16 -07:00
Derek Collison
ce8608a893 Bump to 2.9.0-RC.12
Signed-off-by: Derek Collison <derek@nats.io>
2022-08-29 17:45:53 -07:00
Derek Collison
6be29120e9 Merge pull request #3413 from nats-io/fss-bug
FSS state could skew on expire on recover with no msgs left.
2022-08-29 17:45:30 -07:00
Derek Collison
e837a255cf FSS state could skew on expire on recover with no msgs left.
Also added in sanity check on server start.

Signed-off-by: Derek Collison <derek@nats.io>
2022-08-29 17:34:28 -07:00
Ivan Kozlovic
eb08080e1b Merge pull request #3412 from nats-io/fix_sys_client_gw
[FIXED] Gateway: system request/replies may not work properly
2022-08-29 14:29:28 -06:00
Ivan Kozlovic
9d1e773e8f [FIXED] Gateway: system request/replies may not work properly
When a subscription is recently made, gateway code ensures that if
there is a reply subject, the reply is "mapped" or rewritten to allow
the reply to come back to the origin cluster, regardless of subscription
interest propagation.

The issue was that this uses a map with a `*client` as the key
but the pointer for SYSTEM clients would not always be the same,
which meant that the rewrite would not happen, causing possible
"loss" of replies.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-08-29 14:05:51 -06:00
Derek Collison
dc46f18b3a Merge pull request #3408 from nats-io/pdp/fix-ca-expiry
Fix testdata CA expiration
2022-08-27 13:33:39 -07:00
Phil Pennock
06778cbd6e Regenerate test cert data 2022-08-27 16:23:22 -04:00
Phil Pennock
444e56b425 Set cert lifetime on CA cert too
This was an oversight when automating: we use the `req -x509` approach to
create the CA cert, which didn't pick up the defaults set for other stuff and
so defaulted to 30 days.

Abstract out CERT_DURATION and use for both the cert defaults and the CA's own
lifetime.
2022-08-27 16:21:24 -04:00
Ivan Kozlovic
d73ca7d468 Merge pull request #3407 from nats-io/svc_imp_exp_cycle
[FIXED] Service import/export cycles causing stack overflow
2022-08-26 16:31:04 -06:00
Ivan Kozlovic
a8318d1f62 [FIXED] Service import/export cycles causing stack overflow
There was a way to detect a cycle but I believe it needs to be
a stack of "si" not just the one before invoking processServiceImport.

Changes in #3393 would solve issue reported with test TestAccountImportCycle,
but would not address the new reported issue represented by new test
TestLeafNodeSvcImportExportCycle. This current approach seems to solve
all known cases.

Resolves #3397
Replaces #3393
2022-08-26 14:40:02 -06:00
Derek Collison
97bba60bb5 Merge pull request #3406 from nats-io/max-procs
Don't fail if auto setting of maxprocs fails.
2022-08-25 15:25:20 -07:00
Derek Collison
4edf957570 Don't fail if auto setting of maxprocs fails.
Signed-off-by: Derek Collison <derek@nats.io>
2022-08-25 15:21:13 -07:00
Ivan Kozlovic
431ed98694 Bump to v2.9.0-RC.11
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-08-25 14:46:44 -06:00
Ivan Kozlovic
0400ae5d06 Merge pull request #3404 from nats-io/js_revert_cluster_reset
Revert a change made in PR #3392
2022-08-25 14:46:02 -06:00