Commit Graph

7302 Commits

Author SHA1 Message Date
Derek Collison
ff10e7befe Bump to Go 1.19.11 (#4304) 2023-07-13 11:03:11 -07:00
Byron Ruth
4e6a09400a Bump to Go 1.19.11
Signed-off-by: Byron Ruth <byron@nats.io>
2023-07-13 14:00:38 -04:00
Derek Collison
77189b09c7 [FIXED] JetStream republished messages not being received by MQTT QoS0 subscribers (#4303)
Only discard messages from MQTT QoS0 from internal JetStream clients if
really a QoS1 JetStream publish, not just a JetStream client.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4291
2023-07-13 07:26:26 -07:00
Derek Collison
1f39d744dd Only discard messages from MQTT QoS0 from internal jetstream clients if really a QoS1 jetstream publish.
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-12 16:00:59 -07:00
Derek Collison
0c8552cd34 Evaluate service imports when publishing advisories (#4302)
This should fix #4275 by allowing advisories to be copied into other
accounts via service imports.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-12 12:24:00 -07:00
Neil Twigg
1434ee7da8 More specific check for service imports vs any internal sub
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-12 19:52:31 +01:00
Neil Twigg
fbbbe28278 Evaluate service imports when publishing advisories
Signed-off-by: Neil Twigg <neil@nats.io>
2023-07-12 18:18:57 +01:00
Derek Collison
9cfe8b8f75 [FIXED] Restore of encrypted filestore with no main key could cause dataloss (#4301)
When restoring a filestore with no key generator but it was encrypted,
fail to restore.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-12 08:47:42 -07:00
Piotr Piotrowski
e3b132ae34 Test for gathering JS advisories from multiple accounts on a single account
Signed-off-by: Piotr Piotrowski <piotr@synadia.com>
2023-07-12 13:26:18 +01:00
Derek Collison
9e9a9a082b When restoring a filestore with no key generator but it was encrypted, fail to restore.
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-11 16:27:50 -07:00
Derek Collison
5390272074 Bump to 2.9.20-beta
Signed-off-by: Derek Collison <derek@nats.io>
2023-07-11 14:29:17 -07:00
Derek Collison
97dbd6b28e [FIXED] Duplicate queue subs over leafnodes with hub stream import/export with possible dataloss (#4299)
When a queue subscriber was updated multiple times over a leafnode
connection we added in more shadow subscriptions which could become
zombies when the connection went away.

In a case where a leafnode server had multiple queue subscribers on the
same queue group, the hub server would add in multiple shadow subs.
These subs would not be properly cleaned up and could lead to stale
connections being associated with them.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-11 06:52:26 -07:00
Derek Collison
353d543c16 When a queue subscriber was updated multiple times over a leafnode connection we added in more shadow subscriptions which could become zombies when the connection went away.
In a case where a leafnode server had multiple queue subscribers on the same queue group, the hub server would add in multiple shadow subs. These subs would not be properly cleaned up and could lead to stale connections being associated with them.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-10 21:03:47 -07:00
Derek Collison
eb2aa352ec Updated Leafnode Test (#4283)
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-30 11:06:11 -07:00
Derek Collison
b207984a84 Made cloud clusters 5 nodes each
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-30 10:12:39 -07:00
Neil
57015472a9 Add TestRouteNoLeakOnSlowConsumer and TestRouteNoLeakOnAuthTimeout (#4153)
These tests are to help verify that routes aren't leaking when they go
down for write deadlines or auth failures.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-30 15:34:07 +01:00
Neil Twigg
0d1c666071 Add TestRouteNoLeakOnSlowConsumer and TestRouteNoLeakOnAuthTimeout
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-30 13:11:42 +01:00
Derek Collison
e31bf5c66d Added reverse flow to the mc leafnode test
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-29 16:10:31 -07:00
Derek Collison
503de4593d [FIXED] Do not health check streams that are actively being restored. (#4277)
Could leave them in a bad state.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4270
2023-06-28 17:48:09 -07:00
Derek Collison
a2b9ee9123 Shorten stream size for travis
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-28 15:56:41 -07:00
Derek Collison
1bb1a3cae1 Do not health check streams that are actively being restored.
Could leave them in a bad state.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-28 15:27:45 -07:00
Derek Collison
68a17a4846 Use an account protected method to check for service imports for Gateways. (#4274)
This avoids a potential data race when reloading accounts.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-27 20:37:26 -07:00
Derek Collison
9805101953 Use an account protected method to check for service imports to avoid data race when reloading accounts.
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-27 19:52:43 -07:00
Phil Pennock
83db31240c Adjust nightly name if on a branch (#4272)
If we're on a branch other than main or dev, then when building
"nightly", build it with the name of the branch instead. Overrideable
via an env var.

It's a bit ugly because of limitations of goreleaser templating, we
can't haul the expression out to be defined once, but it works.

Can build a custom Docker image with:

goreleaser release --snapshot -p 2 -f .goreleaser-nightly.yml --clean
2023-06-27 18:48:36 -04:00
Phil Pennock
dc39a9123a Adjust nightly name if on a branch
If we're on a branch other than main or dev, then when building "nightly",
build it with the name of the branch instead.  Overrideable via an env var.

It's a bit ugly because of limitations of goreleaser templating, we can't haul
the expression out to be defined once, but it works.

Can build a custom Docker image with:

    goreleaser release --snapshot -p 2 -f .goreleaser-nightly.yml --clean
2023-06-27 17:00:05 -04:00
Derek Collison
70d8069cdf [IMPROVED] Consumer create time when stream has large number of blks. (#4269)
When creating a consumer on a stream with a very large number of msg
blks, calculating numPending could be slow.

This aims to optimize a bit, more work to be done on streams with a very
large (> 200k) number of msg blks.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-23 15:07:58 -07:00
Derek Collison
855e1bb14e Allow more tolerance for travis
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-23 14:24:00 -07:00
Derek Collison
2b2e22ed52 When creating a consumer on a stream with a very large number of msg blks, calculating numPending could be slow.
This aims to optimize a bit, more work to be done on streams with a very large (> 200k) number of msg blks.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-23 14:11:56 -07:00
Derek Collison
3183f6ed95 Cert Store (aka wincert) (#4268)
For NATS Server on Windows, provide option for TLS certificate and
handshake signature to be provided by the Windows Certificate Store
instead of PEM files.
2023-06-22 17:20:35 -07:00
Todd Beets
f854e95c11 Cert Store (aka wincert) 2023-06-22 12:25:54 -07:00
Derek Collison
031535b89d Add object store benchmark (#4220)
- [ ] Link to issue, e.g. `Resolves #NNN`
 - [ ] Documentation added (if applicable)
 - [x] Tests added
- [x] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [x] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [x] Build is green in Travis CI
- [x] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

### Changes proposed in this pull request:
 - New benchmark for NATS JetStream Object Store
2023-06-21 14:36:35 -07:00
reubenninan
13e09f2db4 Add benchmarks for object store
Signed-off-by: reubenninan <reuben@nats.io>
2023-06-21 16:05:30 -04:00
Derek Collison
a8fc352a20 Additional check for TLS required in WebSocket INFO (#4264)
This should stop us reporting `tls_required` when we aren't reporting
`tls_available`.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-21 12:11:02 -07:00
Neil Twigg
cf715845d1 Additional check for TLS required in WebSocket INFO
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-21 19:06:49 +01:00
Derek Collison
f4f3cce6cc Release v2.9.19 (#4257) 2023-06-20 08:39:14 -07:00
Byron Ruth
9592b760b9 Release v2.9.19
Signed-off-by: Byron Ruth <byron@nats.io>
2023-06-20 10:55:31 -04:00
Derek Collison
04a79b9b1e WebSocket-specific INFO (#4255)
This fixes #4252 by ensuring that `tls_available`, `tls_required`,
`host` and `port` are populated based on the WebSocket listener rather
than standard listeners.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-20 06:33:00 -07:00
Neil Twigg
afdc2478ca WebSocket-specific INFO
This fixes #4252 by ensuring that `tls_available`, `tls_required`, `host` and `port`
are populated based on the WebSocket listener rather than standard listeners.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-20 11:47:09 +01:00
Derek Collison
f7896b4969 Test MaxMsgs and MaxMsgsPer in combination (#4253)
This unit test is modelled around issue #4247 and proves that the
`MaxMsgs` and `MaxMsgsPer` limits are correctly enforced together with
`DiscardNew` and `DiscardNewPer`.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-19 08:58:42 -07:00
Neil Twigg
e879a9fa0c Test MaxMsgs and MaxMsgsPer in combination
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-19 09:45:17 +01:00
Derek Collison
f5b06100d3 Added in another DQ test across leafnodes. (#4250)
This test has multiple leafnode connections to different accounts and to
a shared account to make sure behavior is correct.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-16 13:55:45 -07:00
Derek Collison
e8094c9f33 Make utility funcs helpers
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-16 12:07:21 -07:00
Derek Collison
b3f913237c Added in another DQ test across leafnodes.
This test has multiple leafnode connections to different accounts and to a shared account to make sure behavior is correct.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-16 11:54:09 -07:00
Derek Collison
4a1b281412 [IMPROVED] High CPU and Memory usage on replicated mirrors with very high starting sequence. (#4249)
When creating replicated mirrors where the source stream had a very
large starting sequence number, the server would use excessive CPU and
Memory.

This is due to the mirroring functionality trying to skip messages when
it detects a gap. In a replicated stream this puts excessive stress on
the raft system.

This step is not needed at all if the mirror stream has no messages, we
can simply jump ahead.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-15 17:48:32 -07:00
Derek Collison
087a28a13e When creating replicated mirrors where the source stream had a very large starting sequence number, the server would use excessive CPU and Memory.
This is due to the mirroring functionality trying to skip messages when it detects a gap. In a replicated stream this puts excessive stress on the raft system.
This step is not needed at all if the mirror stream has no messages, we can simply jump ahead.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-15 17:20:15 -07:00
Derek Collison
367d857612 Bump v2.9.19-beta.1 (#4245) 2023-06-13 13:50:13 -07:00
Byron Ruth
bbf24a6d98 Bump v2.9.19-beta.1
Signed-off-by: Byron Ruth <byron@nats.io>
2023-06-13 15:48:05 -04:00
Derek Collison
166eeb243c Release v2.9.18 (#4242) 2023-06-13 12:41:10 -07:00
Byron Ruth
af805b57a4 Release v2.9.18
Signed-off-by: Byron Ruth <byron@nats.io>
2023-06-13 15:19:20 -04:00
Derek Collison
876cb6d837 [UPDATED] go mod tidy to update go.sum (#4240)
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-12 17:31:16 -07:00