Commit Graph

5248 Commits

Author SHA1 Message Date
Derek Collison
87c544946b Fix test for new routing
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-11 18:35:45 -07:00
Derek Collison
bd2753d9f6 Adapt to new server structure for routes
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-11 14:45:29 -07:00
Derek Collison
11e5e049cf Merge branch 'main' into dev 2023-07-11 14:36:10 -07:00
Derek Collison
5390272074 Bump to 2.9.20-beta
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-11 14:29:17 -07:00
Derek Collison
353d543c16 When a queue subscriber was updated multiple times over a leafnode connection we added in more shadow subscriptions which could become zombies when the connection went away.
In a case where a leafnode server had multiple queue subscribers on the same queue group, the hub server would add in multiple shadow subs. These subs would not be properly cleaned up and could lead to stale connections being associated with them.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-10 21:03:47 -07:00
Jean-Noël Moyne
69e137c3d2 [FIXED] Stream config idempotency (#4292)
- [X] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [x] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

Fixes a behavior where idempotency of re-defining the same stream more
than once (with the same attributes) was broken due to the DeepEqual
failing due to the StreamSource struct received from the client app not
having a value for the `iname` structure field (as it's internal) but
the StreamSource struct return from `mset.config()` would have it set.

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-07-07 11:32:09 -07:00
Derek Collison
20ce582b82 Bump to 2.10.0-beta.44
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-05 09:18:29 -07:00
Derek Collison
4d7cd26956 Add in support for segmented binary stream snapshots.
Streams with many interior deletes was causing issues due to the fact that the interior deletes were represented as a sorted []uint64.
This approach introduces 3 sub types of delete blocks, avl bitmask tree, a run length encoding, and the legacy format above.
We also take into account large interior deletes such that on receiving a snapshot we can skip things we already know about.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-03 08:41:33 -07:00
Derek Collison
eb2aa352ec Updated Leafnode Test (#4283)
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-30 11:06:11 -07:00
Derek Collison
b207984a84 Made cloud clusters 5 nodes each
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-30 10:12:39 -07:00
Neil Twigg
0d1c666071 Add TestRouteNoLeakOnSlowConsumer and TestRouteNoLeakOnAuthTimeout
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-30 13:11:42 +01:00
Derek Collison
e31bf5c66d Added reverse flow to the mc leafnode test
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-29 16:10:31 -07:00
Derek Collison
cf393140ab Merge branch 'main' into dev 2023-06-28 17:48:53 -07:00
Derek Collison
a2b9ee9123 Shorten stream size for travis
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-28 15:56:41 -07:00
Derek Collison
1bb1a3cae1 Do not health check streams that are actively being restored.
Could leave them in a bad state.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-28 15:27:45 -07:00
Derek Collison
c16d361ead Merge branch 'main' into dev 2023-06-27 20:41:57 -07:00
Derek Collison
9805101953 Use an account protected method to check for service imports to avoid data race when reloading accounts.
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-27 19:52:43 -07:00
Derek Collison
779fb0dc13 Merge branch 'main' into dev 2023-06-23 15:12:42 -07:00
Derek Collison
855e1bb14e Allow more tolerance for travis
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-23 14:24:00 -07:00
Derek Collison
2b2e22ed52 When creating a consumer on a stream with a very large number of msg blks, calculating numPending could be slow.
This aims to optimize a bit, more work to be done on streams with a very large (> 200k) number of msg blks.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-23 14:11:56 -07:00
Todd Beets
f854e95c11 Cert Store (aka wincert) 2023-06-22 12:25:54 -07:00
Derek Collison
39a36200b5 Merge branch 'main' into dev 2023-06-21 14:37:11 -07:00
Derek Collison
031535b89d Add object store benchmark (#4220)
- [ ] Link to issue, e.g. `Resolves #NNN`
 - [ ] Documentation added (if applicable)
 - [x] Tests added
- [x] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [x] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [x] Build is green in Travis CI
- [x] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

### Changes proposed in this pull request:
 - New benchmark for NATS JetStream Object Store
2023-06-21 14:36:35 -07:00
reubenninan
13e09f2db4 Add benchmarks for object store
Signed-off-by: reubenninan <reuben@nats.io>
2023-06-21 16:05:30 -04:00
Derek Collison
02afb5878c Merge branch 'main' into dev 2023-06-21 12:13:38 -07:00
Neil Twigg
cf715845d1 Additional check for TLS required in WebSocket INFO
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-21 19:06:49 +01:00
Tom Anderson
83a43838dc Adding option to disable jetstream ascii art
This option is not available in the config, and is only accessable to embeded servers where when using custom loggers can look pretty terrible
2023-06-21 08:04:34 -07:00
Ivan Kozlovic
afb5086f17 [CHANGED] LeafNode: remotes from same server binding to same hub account
Previously, the server would reject a second remote leafnode connection
from the same server if it was binding to the same account on the hub
even if the remote was using different local accounts.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-06-20 18:44:52 -06:00
Neil Twigg
d2615b76f2 Annotate CPU and goroutine profiles with account/stream/consumer info
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-20 19:02:40 +01:00
Neil Twigg
165c41fc0c WebSocket-specific INFO
This fixes #4252 by ensuring that `tls_available`, `tls_required`, `host` and `port`
are populated based on the WebSocket listener rather than standard listeners.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-20 09:05:00 -07:00
Byron Ruth
9592b760b9 Release v2.9.19
Signed-off-by: Byron Ruth <byron@nats.io>
2023-06-20 10:55:31 -04:00
Neil Twigg
afdc2478ca WebSocket-specific INFO
This fixes #4252 by ensuring that `tls_available`, `tls_required`, `host` and `port`
are populated based on the WebSocket listener rather than standard listeners.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-20 11:47:09 +01:00
Derek Collison
f9830f4633 Merge branch 'main' into dev 2023-06-19 16:30:05 -07:00
Neil Twigg
e879a9fa0c Test MaxMsgs and MaxMsgsPer in combination
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-19 09:45:17 +01:00
Derek Collison
42827596d8 Merge branch 'main' into dev 2023-06-16 13:56:49 -07:00
Derek Collison
e8094c9f33 Make utility funcs helpers
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-16 12:07:21 -07:00
Derek Collison
b3f913237c Added in another DQ test across leafnodes.
This test has multiple leafnode connections to different accounts and to a shared account to make sure behavior is correct.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-16 11:54:09 -07:00
Derek Collison
3501ca3c1f Merge branch 'main' into dev 2023-06-15 17:49:19 -07:00
Derek Collison
087a28a13e When creating replicated mirrors where the source stream had a very large starting sequence number, the server would use excessive CPU and Memory.
This is due to the mirroring functionality trying to skip messages when it detects a gap. In a replicated stream this puts excessive stress on the raft system.
This step is not needed at all if the mirror stream has no messages, we can simply jump ahead.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-15 17:20:15 -07:00
Todd Beets
832dcc9246 Support TLS 1.2 RSA signature with PKCS#1 v1.5 padding (#4244)
Cert Store (aka wincert) feature wasn't properly handling TLS 1.2
handshake with TLS 1.2 clients that do not support RSA signature with
PSS padding.

With this update, Cert Store will perform either PKCS#1 v1.5 or PSS
padding for RSA signature depending on what type is negotiated by the
TLS 1.2 client.

Issue surfaces with the NATS .NET v1 client which supports TLS 1.2 only
(.NET 4.6.2 dependency) only when the client application was hosted on
Windows 10 Enterprise LTSC 2019 (equivalent also to Windows 10 1809 and
Windows Server 2019). If the same client was executed on a more modern
Windows 10 release, RSA signature with PSS padding was negotiated and
succeeded normally.

The Go NATS client as well as any client operating at TLS 1.3 level
would not exhibit the issue as TLS 1.3 requires PSS.

Fix tested good on Windows 10 Enterprise LTSC 2019 host and in confirmed
fixed in user's Windows environment where the issue was originally
detected.
2023-06-14 08:27:17 -07:00
Ivan Kozlovic
f2d009b244 fix test
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-06-13 17:22:21 -06:00
Byron Ruth
bbf24a6d98 Bump v2.9.19-beta.1
Signed-off-by: Byron Ruth <byron@nats.io>
2023-06-13 15:48:05 -04:00
Byron Ruth
af805b57a4 Release v2.9.18
Signed-off-by: Byron Ruth <byron@nats.io>
2023-06-13 15:19:20 -04:00
Ivan Kozlovic
1ac99fd5db [CHANGED] MQTT: Support for topics with . character.
The `.` character will be transformed to `//` in NATS subject. For
instance an MQTT message published on `spBv1.0/plant1` would
be received by a NATS subscriber as `spBv1//0.plant1`.

Conversely, a NATS message published on `spBv1//0.plant1` would
be received by an MQTT subscriber as `spBv1.0/plant1`.

Resolves #1879
Resolves #3482

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-06-13 13:06:41 -06:00
Todd Beets
056301ff9c Support TLS 1.2 RSA signature with PKCS#1 v1.5 padding 2023-06-13 11:34:13 -07:00
Neil Twigg
afe7f485ea Take the account session lock when deleting from map
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-13 17:47:23 +01:00
Neil Twigg
3b07f4342e Remove unnecessary return, refactor permission check so that it doesn't hold locks longer than needed
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-13 14:34:05 +01:00
Neil Twigg
3fef0edd76 No longer need to manually delete last message on replace
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-13 10:38:30 +01:00
Neil Twigg
a5c0711488 Fix checking retained permissions on config reload
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-13 10:38:30 +01:00
Neil Twigg
8db804ead9 Don't keep MQTT retained message content in memory
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-13 10:38:30 +01:00