Commit Graph

5564 Commits

Author SHA1 Message Date
Derek Collison
244ff4489c Fix for panic from a bug in selecting a block and an index when num blocks > 32 and we used new binary search in NumPending().
The reason would be that we were not accounting for gaps as mb.first.seq can move. The behavior should always return a valid index and mb if seq is inclusive of range from first to last.
The panic could orphan held locks for filestore, consumer and possibly stream.

Signed-off-by: Derek Collison <derek@nats.io>
2023-09-12 15:20:09 -07:00
Derek Collison
f18e06bd57 [FIXED] Move test fix - TestJetStreamSuperClusterMovingStreamsAndConsumers/R3 (#4519)
The default timeout for JetStream API calls is 10s, so in the case where
we determine that we are the leader, but the stream info endpoint has
not registered with the server we are connected to, the stream info call
could fail and we would exhaust the whole checkFor since we would stay
in one call for 10s.

Fix is to override and make multiple attempts possible for the checkFor
loops.

Signed-off-by: Derek Collison <derek@nats.io>
2023-09-12 11:57:26 -07:00
Derek Collison
edbaa57e87 Fixes for move test.
The default timeout for JetStream API calls is 10s, so in the case where we determine that we are the leader, but the stream info endpoint has not registered with the server we are connected to, the stream info call could fail and we would exhaust the whole checkFor since we would stay in one call for 10s. Fix is to override and make multiple attempts possible.

Signed-off-by: Derek Collison <derek@nats.io>
2023-09-12 11:38:35 -07:00
Lev
1acc800778 [FIXED] Increased MQTT test R/W timeout from 4 to 5s (TestMQTTSubPropagation) (#4517)
Tracing the connect (ack?) read times in `TestMQTTSubPropagation` showed
that they come in the 2-3s range during normal execution, and it appears
that they occasionally exceed the 4s timeout.

I am not sure exactly why MQTT CONNECT takes such a long time, but as
the name of the test suggests, perhaps it has to do with session
propagation in a cluster.
2023-09-12 11:31:16 -07:00
Derek Collison
292e42d528 Better fix for flapping test - TestJetStreamClusterStaleReadsOnRestart (#4516)
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-12 08:32:16 -07:00
Derek Collison
0308cbb372 Fix for spelling
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-12 07:52:17 -07:00
Derek Collison
3f80348a16 Fix for data race in accessing rg.node
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-12 07:43:14 -07:00
Derek Collison
8e2c30a75b Better fix for flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-12 06:54:36 -07:00
Neil Twigg
3becc840d9 Allow more time in TestFileStoreNumPendingLargeNumBlks, improve logging on failure
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-12 11:02:43 +01:00
Derek Collison
eed5fd8a73 Allow time for the 3rd replica to be current
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-11 21:54:58 -07:00
Derek Collison
e72bb957fd [FIXED] Data race accessing mirror tr (#4512)
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-11 21:31:47 -07:00
Derek Collison
349158a349 Fix for datarace accessing mirror tr
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-11 21:12:15 -07:00
Derek Collison
970dfab52f Need a flush to make sure INFO processed
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-11 20:39:50 -07:00
Derek Collison
9531611feb Add in utility to detect and delete any NRG orphans.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-11 19:15:12 -07:00
Derek Collison
7d041da3c8 Fix for datarace on clfs
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-10 11:07:27 -07:00
Derek Collison
5def0a99b4 _EMPTY_ replacements
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-10 10:22:22 -07:00
Waldemar Quevedo
93072c333b Allow more time in TestNoRaceSeqSetEncodeLarge (#4505)
In CI this is typically < 700uS, but quite often > 500uS.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-09 10:46:36 -07:00
Waldemar Quevedo
eb06efff15 [FIXED] Skip TestMQTTQoS2InflightMsgsDeliveredAfterUnsubscribe, see c… (#4506)
`TestMQTTQoS2InflightMsgsDeliveredAfterUnsubscribe` was flaky on
TravisCI.

At this point it was there to illustrate a design issue with MQTT 3.1.1
QoS support, and much of it is disabled since it'd be failing anyway.
Disabling the test until it can be _fully_ applicable/enabled.
2023-09-09 10:46:18 -07:00
Derek Collison
e4867455c2 Fix TestNoRaceJetStreamSparseConsumers (#4503)
This is the same fix as in
https://github.com/nats-io/nats-server/pull/4500

This means it adds `StallWait` to `PublishAsync`, as that's seems to be
the only
reason why this test fails.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-09-08 10:01:24 -07:00
Lev Brouk
64c34c4b5d [FIXED] Skip TestMQTTQoS2InflightMsgsDeliveredAfterUnsubscribe, see comments 2023-09-08 09:42:51 -07:00
Neil Twigg
e382b52ad8 Allow more time in TestNoRaceSeqSetEncodeLarge
In CI this is typically < 700uS, but quite often > 500uS.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-08 16:36:28 +01:00
Neil Twigg
ede9a6cdfe Improve TestJetStreamAccountImportJSAdvisoriesAsStream
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-08 15:15:18 +01:00
Tomasz Pietrek
7b54a1e6a1 Fix TestNoRaceJetStreamSparseConsumers
This is the same fix as in https://github.com/nats-io/nats-server/pull/4500

This means it adds `StallWait` to `PublishAsync`, as that's seems to be the only
reason why this test fails.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-09-08 11:34:16 +02:00
Todd Beets
5519308047 Re-order auth.go imports following server convention. 2023-09-07 15:20:35 -07:00
Todd Beets
006f7a77f9 Don't log error in legitimate nkey auth callout config case (4479). 2023-09-07 11:21:55 -07:00
Tomasz Pietrek
d07e8eb210 Fix TestNoRaceJetStreamInterestStreamCheckInterestRaceBug
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-09-07 19:14:31 +02:00
Neil Twigg
e19e97b0a6 Use write lock for memory store filtered state
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-07 12:47:57 +01:00
Derek Collison
11f0ea99a4 Merge branch 'main' into dev
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-06 13:33:47 -07:00
Waldemar Quevedo
e1574eca3e Revert "Enables 0s deduplication window duration when the stream has sources (#4476)"
This reverts commit db96238ad9.
2023-09-06 11:51:38 -07:00
Neil
628a54447c flake: Fixes TestAccountReqMonitoring (#4493)
Also fixes some tests using default JS dir instead of TempDir
2023-09-06 13:53:09 +01:00
Waldemar Quevedo
579968b60b flake: Fixes TestAccountReqMonitoring
Also fixes some tests using default JS dir instead of TempDir

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-09-06 03:43:11 -07:00
Neil Twigg
25564087f9 Fix race condition in hashing stream snapshots
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-06 11:32:43 +01:00
Neil
7781407618 v2.10: Use Go 1.21 in nightlies, update rand usage (#4490)
- Use Go 1.21 in nightlies
- Both rand.Seed and rand.Read are both deprecated, remove its use to
fix staticcheck errors

```go
server/client.go:95:2: SA1019: rand.Seed has been deprecated since Go 1.20 and an alternative 
  has been available since Go 1.0: As of Go 1.20 there is no reason to call Seed with a random value. 
  Programs that call Seed with a known value to get a specific sequence of results should use 
  New(NewSource(seed)) to obtain a local random generator. (staticcheck)

server/jetstream_test.go:20399:2: SA1019: rand.Read has been deprecated since Go 1.20 
  because it shouldn't be used: For almost all use cases, crypto/rand.Read is more appropriate. (staticcheck)
```
2023-09-06 10:22:25 +01:00
Waldemar Quevedo
d144eaaed8 Replace rand.Rand with crypto/rand.Read in Go +1.20
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-09-05 16:57:49 -07:00
Waldemar Quevedo
d5d8f29575 Remove rand.Seed use, not needed in Go +1.20
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-09-05 16:55:04 -07:00
Lev Brouk
759715a2ba [FIXED] MQTT: panic in an error log 2023-09-05 14:39:28 -07:00
Neil Twigg
c539fb7e9a De-flake TestJetStreamClusterConsumerMaxDeliveryNumAckPendingBug by ignoring PushBound
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-05 15:11:21 +01:00
Waldemar Quevedo
d8bdbc34c4 Fix resetting TLS name from solicited remotes
In +Go 1.20, the x509.HostnameError changed to be wrapped in a
tls.CertificateVerificationError so sometimes the name would not
be reset.

This would make the `TestGatewayTLSMixedIPAndDNS` test flake sometimes.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-09-04 11:50:55 -07:00
Derek Collison
002446a51d Bump to 2.10.0-beta.55
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-04 11:34:12 -07:00
Derek Collison
e7e8a330d4 Allow sync intervals to be set and the ability to have all data writes synchronous.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-04 11:05:13 -07:00
Derek Collison
60e41aa8dc [FIXED] MQTT: more generic names for outgoing stream, etc. (#4484)
- [x] Branch rebased on top of current main (`git pull --rebase origin
dev`)
- [x] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [ ] Build is green in Travis CI

### Changes proposed in this pull request:

Removed `qos2` reference from the MQTT outgoing stream name and
subjects.

Rationale: As I was reviewing the documentation and thinking how to move
forward with MQTT, I am inclined to use the `$MQTT_out` stream and
session-specific consumers (as opposed to subscription-specific) for all
QOS-related packet delivery and PI persistence. I think I will be able
to repurpose it "as is", so proposing the generic name change.

Since these (stream, consumer, subscriptions) are newly introduced in
https://github.com/nats-io/nats-server/pull/4349 and have not yet been
released, this is a safe change.
2023-09-04 10:18:07 -07:00
Derek Collison
72dede6f84 [ADDED] README-MQTT.md: MQTT implementation notes (#4485)
### Changes proposed in this pull request:

- Adds MQTT implementation notes
2023-09-04 10:17:53 -07:00
Neil
bbbd2f939f Use TempDir in more tests (#4486)
This uses `TempDir` in more tests. Some were trying to manage the
directory manually, others were just not managing a directory at all.
There may be more that need doing.

Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-04 17:27:48 +01:00
Neil Twigg
8de83bc2ef Use TempDir in more tests
Signed-off-by: Neil Twigg <neil@nats.io>
2023-09-04 16:54:36 +01:00
Lev Brouk
885f88a9cd [ADDED] README-MQTT.md: MQTT implementation notes 2023-09-04 06:46:16 -07:00
Lev Brouk
b24941e6c6 [FIXED] MQTT: more generic names for outgoing stream, etc. 2023-09-04 05:32:58 -07:00
Derek Collison
1768f9c770 Always check for orphaned fss and idx files
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-03 14:31:55 -07:00
Derek Collison
ba5d9089b1 Tweak for flapper
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-03 14:31:28 -07:00
Derek Collison
2d2bb77f6e Optimize for restore time.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-03 14:14:00 -07:00
Derek Collison
4a5b76b0e8 Print out restore time for streams to nearest millisecond.
Signed-off-by: Derek Collison <derek@nats.io>
2023-09-03 13:28:18 -07:00