Commit Graph

8143 Commits

Author SHA1 Message Date
Tomasz Pietrek
d07e8eb210 Fix TestNoRaceJetStreamInterestStreamCheckInterestRaceBug
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-09-07 19:14:31 +02:00
Neil
6332512649 Use write lock for memory store filtered state (#4498)
Calls to `filteredStateLocked` can mutate the per-subject state by
recalculating the first sequences. Doing so under a read-only lock can
lead to race conditions, so switch the call sites out to write locks
instead.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-07 16:26:12 +01:00
Neil Twigg
e19e97b0a6 Use write lock for memory store filtered state
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-07 12:47:57 +01:00
Derek Collison
11f0ea99a4 Merge branch 'main' into dev
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-06 13:33:47 -07:00
Waldemar Quevedo
fd50e79da7 Revert "Enables 0s deduplication window duration when the stream has sources" (#4495)
This reverts commit db96238ad9 which was
causing `TestNoRaceJetStreamSuperClusterSources` to fail sometimes with
duplicate messages around shutdown.
2023-09-06 12:53:40 -07:00
Waldemar Quevedo
e1574eca3e Revert "Enables 0s deduplication window duration when the stream has sources (#4476)"
This reverts commit db96238ad9.
2023-09-06 11:51:38 -07:00
Byron Ruth
f650e1ad07 Pin Go version in Travis config (#4494) 2023-09-06 13:29:15 -04:00
Byron Ruth
d29df72f83 Pin Go version in Travis config
Signed-off-by: Byron Ruth <byron@nats.io>
2023-09-06 13:20:10 -04:00
Neil
628a54447c flake: Fixes TestAccountReqMonitoring (#4493)
Also fixes some tests using default JS dir instead of TempDir
2023-09-06 13:53:09 +01:00
Neil
1336a78e62 Fix race condition in hashing stream snapshots (#4492)
The `streamSnapshot` function was not holding the filestore lock while
resetting and using the hasher. This created a data race and also
resulted in flakes in `TestFileStoreSnapshot`.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-06 13:52:49 +01:00
Waldemar Quevedo
579968b60b flake: Fixes TestAccountReqMonitoring
Also fixes some tests using default JS dir instead of TempDir

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-09-06 03:43:11 -07:00
Neil Twigg
25564087f9 Fix race condition in hashing stream snapshots
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-06 11:32:43 +01:00
Neil
7781407618 v2.10: Use Go 1.21 in nightlies, update rand usage (#4490)
- Use Go 1.21 in nightlies
- Both rand.Seed and rand.Read are both deprecated, remove its use to
fix staticcheck errors

```go
server/client.go:95:2: SA1019: rand.Seed has been deprecated since Go 1.20 and an alternative 
  has been available since Go 1.0: As of Go 1.20 there is no reason to call Seed with a random value. 
  Programs that call Seed with a known value to get a specific sequence of results should use 
  New(NewSource(seed)) to obtain a local random generator. (staticcheck)

server/jetstream_test.go:20399:2: SA1019: rand.Read has been deprecated since Go 1.20 
  because it shouldn't be used: For almost all use cases, crypto/rand.Read is more appropriate. (staticcheck)
```
2023-09-06 10:22:25 +01:00
Waldemar Quevedo
69bc8d9626 flakes: Fix TestAccountSubjectMapping (#4491) 2023-09-05 23:55:37 -07:00
Waldemar Quevedo
0c68b891fe flakes: Fix TestAccountSubjectMapping
When it fails it hangs the build.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-09-05 21:17:22 -07:00
Waldemar Quevedo
d144eaaed8 Replace rand.Rand with crypto/rand.Read in Go +1.20
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-09-05 16:57:49 -07:00
Waldemar Quevedo
d5d8f29575 Remove rand.Seed use, not needed in Go +1.20
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-09-05 16:55:04 -07:00
Waldemar Quevedo
eae0e34cf6 [FIXED] MQTT: panic in an error log (#4489)
### Changes proposed in this pull request:

Fixes a panic in error handling/logging
2023-09-05 15:32:58 -07:00
Lev Brouk
759715a2ba [FIXED] MQTT: panic in an error log 2023-09-05 14:39:28 -07:00
Neil
45df4c6f2f De-flake TestJetStreamClusterConsumerMaxDeliveryNumAckPendingBug (#4488)
This test flakes quite often on the final consumer info check because of
one consumer reporting `PushBound` and the other not. By clearing both,
this should clear up the disagreement.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-05 17:46:59 +01:00
Neil Twigg
c539fb7e9a De-flake TestJetStreamClusterConsumerMaxDeliveryNumAckPendingBug by ignoring PushBound
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-05 15:11:21 +01:00
Waldemar Quevedo
8390a4714c v2.9.x: Change CI and nightlies to use Go 1.20 (#4487)
Includes fix for test `TestGatewayTLSMixedIPAndDNS ` that was breaking
in Go 1.20.
2023-09-04 13:33:22 -07:00
Waldemar Quevedo
7a2020a347 Remove Go 1.19 build
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-09-04 12:55:06 -07:00
Waldemar Quevedo
d8bdbc34c4 Fix resetting TLS name from solicited remotes
In +Go 1.20, the x509.HostnameError changed to be wrapped in a
tls.CertificateVerificationError so sometimes the name would not
be reset.

This would make the `TestGatewayTLSMixedIPAndDNS` test flake sometimes.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-09-04 11:50:55 -07:00
Sylvain Rabot
a6f7f5476e Bump v2.9 CI and nightlies to Go 1.20
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-09-04 11:43:22 -07:00
Derek Collison
002446a51d Bump to 2.10.0-beta.55
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-04 11:34:12 -07:00
Derek Collison
bf7b4b81c3 Added ability to control sync intervals and sync always (#4483)
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-04 11:32:10 -07:00
Derek Collison
e7e8a330d4 Allow sync intervals to be set and the ability to have all data writes synchronous.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-04 11:05:13 -07:00
Derek Collison
60e41aa8dc [FIXED] MQTT: more generic names for outgoing stream, etc. (#4484)
- [x] Branch rebased on top of current main (`git pull --rebase origin
dev`)
- [x] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [ ] Build is green in Travis CI

### Changes proposed in this pull request:

Removed `qos2` reference from the MQTT outgoing stream name and
subjects.

Rationale: As I was reviewing the documentation and thinking how to move
forward with MQTT, I am inclined to use the `$MQTT_out` stream and
session-specific consumers (as opposed to subscription-specific) for all
QOS-related packet delivery and PI persistence. I think I will be able
to repurpose it "as is", so proposing the generic name change.

Since these (stream, consumer, subscriptions) are newly introduced in
https://github.com/nats-io/nats-server/pull/4349 and have not yet been
released, this is a safe change.
2023-09-04 10:18:07 -07:00
Derek Collison
72dede6f84 [ADDED] README-MQTT.md: MQTT implementation notes (#4485)
### Changes proposed in this pull request:

- Adds MQTT implementation notes
2023-09-04 10:17:53 -07:00
Neil
bbbd2f939f Use TempDir in more tests (#4486)
This uses `TempDir` in more tests. Some were trying to manage the
directory manually, others were just not managing a directory at all.
There may be more that need doing.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-04 17:27:48 +01:00
Neil Twigg
8de83bc2ef Use TempDir in more tests
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-04 16:54:36 +01:00
Derek Collison
0e78f18bb2 Filestore tweaks and improvements (#4481)
Optimized for startup restore time.

Signed-off-by: Derek Collison <derek@nats.io>
2023-09-04 08:23:15 -07:00
Lev Brouk
885f88a9cd [ADDED] README-MQTT.md: MQTT implementation notes 2023-09-04 06:46:16 -07:00
Lev Brouk
b24941e6c6 [FIXED] MQTT: more generic names for outgoing stream, etc. 2023-09-04 05:32:58 -07:00
Derek Collison
1768f9c770 Always check for orphaned fss and idx files
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-03 14:31:55 -07:00
Derek Collison
ba5d9089b1 Tweak for flapper
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-03 14:31:28 -07:00
Derek Collison
2d2bb77f6e Optimize for restore time.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-03 14:14:00 -07:00
Derek Collison
4a5b76b0e8 Print out restore time for streams to nearest millisecond.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-03 13:28:18 -07:00
Derek Collison
e11ddb8bfe Merge branch 'main' into dev
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-02 14:22:57 -07:00
Derek Collison
8b35c01637 [FIXED] Fix for a bug that would make normal streams use the wrong block size. (#4478)
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-02 14:14:50 -07:00
Derek Collison
34ae2bf4cb Fix for a bug that would make normal streams use the wrong block size.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-02 13:56:34 -07:00
Derek Collison
1bb4a71a4d Merge branch 'main' into dev
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-02 12:15:40 -07:00
Derek Collison
b63318c0c9 Bump to 2.9.22-RC.5
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-02 12:00:39 -07:00
Derek Collison
f61ad322c9 [FIXED] Interface conversion bug for ipQueues in monitor which would cause panics. (#4477)
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-02 12:00:07 -07:00
Derek Collison
2c81224262 Fixed interface conversion for ipQueue in monitor which caused panics.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-02 11:43:08 -07:00
Derek Collison
7f13ecc87b Refine core and TLS benchmarks (#4475)
- [ ] Link to issue, e.g. `Resolves #NNN`
 - [ ] Documentation added (if applicable)
 - [x] Tests added
- [x] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [ ] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [ ] Build is green in Travis CI
- [x] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

Related: https://github.com/nats-io/nats-server/pull/4399

### Changes proposed in this pull request:

 - Avoid binding to hardcoded port, as it may conflict if already in use
- Drop TLS from Core Request/Response benchmark, was not showing much
difference, not enough data going through to become a dominant cost
- Add a different core benchmark that exercises TLS by pushing through
lots of data
2023-09-01 13:58:27 -07:00
Marco Primi
e61e40e3fe Add benchmark for TLS content encryption overhead 2023-09-01 12:58:52 -07:00
Marco Primi
4eedfe8d0c Simplify core request/response benchmark
- Remove TLS, impact is negligible for the amount of data pushed
through
 - Rename benchmark
2023-09-01 12:58:52 -07:00
Marco Primi
bbf42c1f57 Use dynamic port number in benchmark 2023-09-01 12:58:52 -07:00