Commit Graph

7659 Commits

Author SHA1 Message Date
Neil Twigg
8db804ead9 Don't keep MQTT retained message content in memory
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-13 10:38:30 +01:00
Derek Collison
5438a4822a Merge branch 'main' into dev 2023-06-12 17:31:51 -07:00
Derek Collison
876cb6d837 [UPDATED] go mod tidy to update go.sum (#4240)
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-12 17:31:16 -07:00
Derek Collison
232294c3af go mod tidy to update go.sum
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-12 17:16:35 -07:00
Derek Collison
38f60f07b3 Merge branch 'main' into dev 2023-06-12 14:12:27 -07:00
Derek Collison
378d5a944e Bump client version to v1.27.0 (#4239)
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-06-12 14:11:24 -07:00
Tomasz Pietrek
13bf12ce64 Bump client version to v1.27.0
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-06-12 22:06:08 +02:00
Derek Collison
1d00ea4fa0 Bump to 2.9.18-beta.3
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-11 13:11:32 -07:00
Derek Collison
ce6fffe1a8 Merge branch 'main' into dev 2023-06-11 13:10:52 -07:00
Derek Collison
ab2ac20fe1 [UPDATED] Dependencies (#4236)
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-11 13:09:03 -07:00
Derek Collison
0980384c97 Update dependencies
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-11 12:25:44 -07:00
Derek Collison
8a8c37231f Merge branch 'main' into dev 2023-06-10 20:56:42 -07:00
Derek Collison
860c481f0f [IMPROVED] Optimize statsz locking and sending in standalone mode. (#4235)
If we know we are in stand alone mode only send out statsz updates if we
know we have external interest.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves: #4234
2023-06-10 20:55:43 -07:00
Derek Collison
11963e51fe Optimize statsz locking and only send if we know we have external interest.
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-10 20:25:05 -07:00
Derek Collison
cee68344b7 Merge branch 'main' into dev 2023-06-10 17:03:36 -07:00
Derek Collison
aae218fe77 [IMPROVED] Only enable JetStream account updates in clustered mode. (#4233)
If we know we are in stand alone mode we do not need to run the updates
for JetStream account resources updates.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4227 (Partial)
2023-06-10 17:03:11 -07:00
Derek Collison
1ce0eb7fdb Merge branch 'main' into dev 2023-06-10 15:23:33 -07:00
Derek Collison
783e9491b1 [IMPROVED] Last msg lookup (KV Get) when subject is a literal subject (#4232)
When messages were very small and the key space was very large the
performance of last message gets in the store layer (both file and
memory) would degrade.

If the subject is literal we can optimize and avoid sequence scans that
are needed when multiple subject states need to be considered.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4221
2023-06-10 15:23:11 -07:00
Derek Collison
a5de25f213 Only enable JetStream account updates in clustered mode.
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-10 15:21:55 -07:00
Derek Collison
f81bc80676 Improve last msg lookup (KV Get) when subject is a literal subject.
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-10 13:16:13 -07:00
Derek Collison
d956a5c3a7 Connect JS benchmarks to stream leader (#4219)
- [ ] Link to issue, e.g. `Resolves #NNN`
 - [ ] Documentation added (if applicable)
 - [ ] Tests added
- [x] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [x] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [x] Build is green in Travis CI
- [x] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

### Changes proposed in this pull request:

- Refactors BenchmarkJetStream(Consume|Publish) and BenchmarkJetStreamKV
benchmarks to connect to stream leader for benchmark cases where cluster
size is greater than 1.
 - BenchmarkJetStreamKV now only targets a single bucket (B=1).
 
CC: @mprimi
2023-06-10 12:27:23 -07:00
Derek Collison
60d006bbeb Merge branch 'main' into dev 2023-06-09 18:25:17 -07:00
Derek Collison
975f004a74 Fixed issues with leafnode compression negotiation (#4230)
When a server would send an asynchronous INFO to a remote server it
would incorrectly contain compression information that could cause
issues with one side thinking that the connection should be compressed
while the other side was not.

It also caused the authentication timer to be incorrectly set which
would cause a disconnect.

Signed-off-by: Ivan Kozlovic <ijkozlovic@gmail.com>
2023-06-09 18:20:50 -07:00
Derek Collison
8c513ad2c2 [FIXED] DQ weighted subscribers across separate leafnode connections. (#4231)
Fix for properly distributed queue requests over multiple leafnode
connections.
When a leafnode server joins two accounts in a supercluster, we want to
make sure that each connection properly takes into account the weighted
number of subscribers in each account.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-09 17:50:17 -07:00
Derek Collison
2765e534eb Fix test and update copyright
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-09 15:09:15 -07:00
Derek Collison
ce2dcd3394 Fix for properly distributed queue requests over multiple leafnode connections.
When a leafnode server joins two accounts in a supercluster, we want to make sure that each connection properly takes into account the weighted number of subscribers in each account.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-09 14:43:59 -07:00
Derek Collison
13b6e2613c Reduce messages in chaos tests (#4229)
It doesn't really appear as though, for what these tests are trying to
prove, that an excessively large number of messages is required. Instead
let's drop the count a little in the hope that they run a bit faster.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-09 13:59:35 -07:00
Ivan Kozlovic
7ff0ea449a Fixed issues with leafnode compression negotiation
When a server would send an asynchronous INFO to a remote server
it would incorrectly contain compression information that could
cause issues with one side thinking that the connection should
be compressed while the other side was not.

It also caused the authentication timer to be incorrectly set
which would cause a disconnect.

Signed-off-by: Ivan Kozlovic <ijkozlovic@gmail.com>
2023-06-09 13:20:44 -06:00
Derek Collison
81154c40f5 Bump to 2.9.18-beta.2
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-09 09:29:48 -07:00
Derek Collison
a1f03513d8 Merge branch 'main' into dev 2023-06-09 09:29:13 -07:00
Neil Twigg
7de3568f39 Reduce messages in chaos tests
It doesn't really appear as though, for what these tests are trying to
prove, that an excessively large number of messages is required. Instead
let's drop the count a little in the hope that they run a bit faster.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-09 17:07:53 +01:00
Neil
9d2ae42956 [FIXED] Performance issues with checkAckFloor with large first stream sequence. (#4226)
Bail early if new consumer, meaning stream sequence floor is 0.
Decide which linear space to scan.
Do no work if no pending and we just need to adjust which we do at the
end.

Also realized some tests were named wrong and were not being run, or
were in wrong file.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-09 09:27:52 +01:00
Derek Collison
9eeffbcf56 Fix performance issues with checkAckFloor.
Bail early if new consumer, meaning stream sequence floor is 0.
Decide which linear space to scan.
Do no work if no pending and we just need to adjust which we do at the end.

Also realized some tests were named wrong and were not being run, or were in wrong file.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-08 18:45:03 -07:00
Derek Collison
40619659d5 [FIXED] Check for invalid stream name in sources (#4222)
- [X] Link to issue, e.g. `Resolves #4141`
 - [X] Tests added
 - [X] Branch rebased on top of current dev
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [x] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

Resolves #4141

### Changes proposed in this pull request:

Adds a check that the stream name of a stream source is valid and
associated new error if it isn't.
2023-06-08 15:14:25 -07:00
Jean-Noël Moyne
a979fdb4ba Fix and improve description for the two new errors
Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-06-08 11:01:54 -07:00
Derek Collison
19eba1b8c8 Merge branch 'main' into dev 2023-06-08 09:34:41 -07:00
Derek Collison
d3bde2c6e1 Send peer state when adding peers (#4224)
Currently `UpdateKnownPeers` doesn't send a peer state when a single
peer add operation is taking place, but it seems like this can
potentially race when there are lots of changes to the replica count
happening in rapid succession. Sending the peer state in all cases seems
to fix this issue and, so far in my testing, fixes the failground stream
update replicas test.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-08 09:31:49 -07:00
Jean-Noël Moyne
7ff114162c Adds the same check for valid stream name for Mirror
Fix test using invalid stream names

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-06-08 07:49:47 -07:00
Jean-Noël Moyne
bd6c15d24e Adds a check that the stream name of a stream source is valid and associated new error if it isn't.
Adresses https://github.com/nats-io/nats-server/issues/4141

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-06-08 07:49:46 -07:00
Neil Twigg
6d9955d212 Send peer state when adding peers
Signed-off-by: Neil Twigg <neil@nats.io>
2023-06-08 15:25:18 +01:00
Derek Collison
c85db15a2c Update Go to 1.19.10 (#4223) 2023-06-08 04:39:20 -07:00
Derek Collison
b5c0170527 Turn off leaf compression to stabilize test for now
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-08 04:37:07 -07:00
Byron Ruth
3ca99adcca Update Go to 1.19.10
Signed-off-by: Byron Ruth <byron@nats.io>
2023-06-08 07:10:27 -04:00
Derek Collison
fd082ee8a5 Merge branch 'main' into dev 2023-06-07 14:31:53 -07:00
Derek Collison
779978d817 Extended replay leafnode test to confirm mirror functionality
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-07 14:01:43 -07:00
reubenninan
1d57260179 connects JS benchmarks to stream leader always
Refactors BenchmarkJetStream(Consume|Publish) and BenchmarkJetStreamKV benchmarks to connect to stream leader for benchmark cases where cluster size is greater than 1.

Also, BenchmarkJetStreamKV now only targets a single bucket (B=1).

Signed-off-by: Reuben Ninan <reuben@nats.io>
2023-06-06 19:35:39 -04:00
Derek Collison
822ad00d50 Bump to 2.9.18-beta.1
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-05 14:14:35 -07:00
Derek Collison
f342f6a758 Merge branch 'main' into dev 2023-06-05 14:13:18 -07:00
Derek Collison
2e2ac33920 [IMPROVED] When R1 consumers were recreated with the same name when they became inactive. (#4216)
When consumers were R1 and the same name was reused, server restarts
could try to cleanup old ones and effect the new ones. These changes
allow consumer name reuse more effectively during server restarts.

Signed-off-by: Derek Collison <derek@nats.io>
2023-06-05 14:04:53 -07:00
Derek Collison
df5df3ce99 Panic fixes (#4214)
- [ ] Link to issue, e.g. `Resolves #NNN`
 - [ ] Documentation added (if applicable)
 - [ ] Tests added
- [ ] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [ ] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [x] Build is green in Travis CI
- [x] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

Resolves panics in the code.

### Changes proposed in this pull request:

 - This PR fixes some of the panics in the code
2023-06-05 13:02:05 -07:00