Commit Graph

6039 Commits

Author SHA1 Message Date
Ivan Kozlovic
427e52dba2 Removed export GODEBUG="x509sha1=1" from runTestsOnTravis.sh
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-07-28 09:19:57 -06:00
Phil Pennock
e38419e09e testdata: regenerate X509 certs with new script 2022-07-28 02:52:56 -04:00
Phil Pennock
3f760760de test data: x509 cert reissuance script
We had some certs which used SHA1 still; this top-level script is a bit rough,
but works to recreate all the top-level certs, using a common CA.

RSA 2048 bits with SHA256.

I've put the IPv6 localhost IP into the certs too, wherever the IPv4 localhost
IP was present, as this might improve test-runner resilience in some
situations.  Seems like a good idea, can't think of a reason not to.
2022-07-28 02:52:36 -04:00
Ivan Kozlovic
725c1949e6 Bump version to 2.9.0-beta.14
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-07-27 19:06:12 -06:00
Ivan Kozlovic
258bd4240b Merge pull request #3297 from nats-io/js_fix_panic
Fixed a panic when consumer is closed
2022-07-27 18:50:53 -06:00
Ivan Kozlovic
5786d2d9d6 Changed "return" to "continue"
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-07-27 18:23:54 -06:00
Ivan Kozlovic
38727417df Moving super-cluster tests from cluster tests file to supercluster file
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-07-27 17:14:19 -06:00
Ivan Kozlovic
88203dd5d5 Fixed a panic when consumer is closed
Panic was:
```
=== RUN   TestJetStreamClusterDelete
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xcec8fb]
goroutine 1761 [running]:
github.com/nats-io/nats-server/v2/server.(*stream).config(0x0)
	/home/travis/gopath/src/github.com/nats-io/nats-server/server/stream.go:1192 +0x5b
github.com/nats-io/nats-server/v2/server.(*consumer).replica(0xc000101400)
	/home/travis/gopath/src/github.com/nats-io/nats-server/server/jetstream_cluster.go:3580 +0xea
github.com/nats-io/nats-server/v2/server.(*jetStream).monitorConsumer(0xc0001d2790, 0xc000101400, 0xc0004df0e0)
	/home/travis/gopath/src/github.com/nats-io/nats-server/server/jetstream_cluster.go:3733 +0xe06
github.com/nats-io/nats-server/v2/server.(*jetStream).processClusterCreateConsumer.func1()
	/home/travis/gopath/src/github.com/nats-io/nats-server/server/jetstream_cluster.go:3445 +0x4d
created by github.com/nats-io/nats-server/v2/server.(*Server).startGoRoutine
	/home/travis/gopath/src/github.com/nats-io/nats-server/server/server.go:3057 +0x85
FAIL	github.com/nats-io/nats-server/v2/server	9.911s
```

Seem to have been introduced in #3282

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-07-27 16:51:10 -06:00
Ivan Kozlovic
23c475e0d2 Merge pull request #3296 from nats-io/js_fix_subj_string_race
Fixed data race between UpdateConfif() and subjString
2022-07-27 16:19:56 -06:00
Ivan Kozlovic
d0ee9a1252 Add comment that this is to silence the race detector.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-07-27 15:43:38 -06:00
Ivan Kozlovic
880e854637 Fixed data race between UpdateConfif() and subjString
A message block is checking the filestore's cfg.Subjects to see
if it can "intern" the subject or not. The problem is that this
is done under the message block's lock, but not the filestore.
However, during a stream configuration update, the filestore's
cfg field is switched to a new one, causing the datarace.

By making sure we do the switch under all message blocks lock,
we remove the data race (that could be reproduce by running th
test TestJetStreamClusterMoveCancel with -count=10).

We investigating the use of a string interning library but it
showed a little performance degradation that this approach does
not suffer from.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-07-27 14:39:34 -06:00
Derek Collison
15382d9796 Bump to 2.9.0-beta.12
Signed-off-by: Derek Collison <derek@nats.io>
2022-07-26 18:57:41 -07:00
Matthias Hanel
3358205de3 add implementation for consumer replica change (#3293)
* add implementation for consumer replica change

fixes #3262

also check peer list on every update

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-07-27 03:56:28 +02:00
Derek Collison
5f12c244ab Bump to 2.9.0-beta.11
Signed-off-by: Derek Collison <derek@nats.io>
2022-07-26 17:25:28 -07:00
Matthias Hanel
04ffed48b0 fix peer tracking by removing peers before scaledown (#3289)
in doRemovePeerAsLeader the leader also records the removed peer in the removed set

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-07-26 22:01:03 +02:00
Matthias Hanel
6212087feb fix race by locking arround o.isLeader (#3291)
Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-07-26 21:49:04 +02:00
Ivan Kozlovic
fe370955c8 Merge pull request #3288 from nats-io/debug_test_failure
[FIXED] JetStream: Some scaling up issues
2022-07-26 08:57:17 -06:00
Ivan Kozlovic
cb97cbea4a Merge pull request #3287 from nats-io/js_streams_limit
[FIXED] JetStream/Cluster: Stream names/infos would return bad response
2022-07-25 18:45:01 -06:00
Ivan Kozlovic
1a6c5f1c90 [FIXED] JetStream: Some scaling up issues
- Send snapshot only if leader
- When processing snapshot, start with a smaller inactivity interval
  that will double up to 10sec or use 10sec directly once we get a
  message. Reason for that is that it is possible that the request
  for snapshot is sent while the leader has not yet setup the subscription
  that receives the requests (or subscription has not fully reached the
  cluster).
- Don't remember snapfile on err.
- Do not consider current if we have not had any activity.
- Stabilize stream scale up under active heavy publishing.
- Due to the publish pressure move the check for followers direct subs spinning up til after we stop publishing.

Signed-off-by: Derek Collison <derek@nats.io>
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-07-25 18:44:18 -06:00
Ivan Kozlovic
ebeca00e20 [FIXED] JetStream/Cluster: Stream names/infos would return bad response
If there are more stream names that the current limit of 1024, getting
the list of names would return them all instead of using pagination.

For "stream infos", the Total amount returned would be the API limit
instead of the actual number of streams.

Resolves https://github.com/nats-io/natscli/issues/541

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-07-25 14:41:05 -06:00
Derek Collison
9446170af4 Merge pull request #3286 from nats-io/update_deps
Update dependencies and suppress staticcheck failure
2022-07-25 11:58:57 -07:00
Derek Collison
6e0d3eaa9d Update compress dependency
Signed-off-by: Derek Collison <derek@nats.io>
2022-07-25 11:31:12 -07:00
Derek Collison
e87a30fdb7 Bump to 2.9.0-beta.10
Signed-off-by: Derek Collison <derek@nats.io>
2022-07-23 19:21:58 -07:00
Derek Collison
f377c41898 Merge pull request #3284 from nats-io/expire_on_recover_reset
[FIXED] Losing stream sequence on multiple restarts and leader changes.
2022-07-23 19:20:30 -07:00
Derek Collison
52f7765322 When msgs were expired on restart recovery we could lose track on subsequent restart of starting sequence with no additional activity.
Signed-off-by: Derek Collison <derek@nats.io>
2022-07-23 17:15:16 -07:00
Ivan Kozlovic
07554e970a Bump version to 2.9.0-beta.9
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-07-22 11:56:15 -06:00
Matthias Hanel
5a720d4977 down scale consumer before downscale of stream (#3282)
Now monitorStream waits with scaling down the stream until all
monitorConsumer have scaled down their respective consumer

Also update consumer assignment for later use in monitorConsumer
Same for stream assignment in monitorStream

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-07-22 19:54:13 +02:00
Ivan Kozlovic
3676823e07 Merge pull request #3283 from nats-io/short_idx_write
[FIXED] Short index write could lead to loss of stream sequence for an empty stream
2022-07-22 11:29:50 -06:00
Derek Collison
3a10456e68 Short index write could lead to loss of stream sequence for empty stream
Signed-off-by: Derek Collison <derek@nats.io>
2022-07-22 06:37:19 -07:00
Ivan Kozlovic
a3e62f000c Merge pull request #3265 from abegaj/fix-max-deliver-update
[FIXED] Maximum Deliveries update has no effect
2022-07-21 16:06:41 -06:00
Ardit Begaj
2871dcb235 increase maxDeliver 2022-07-21 23:24:56 +02:00
Ardit Begaj
c04ab08f2a improve tests based on feedback 2022-07-21 23:21:58 +02:00
Matthias Hanel
918ce307af fix and improve unit test (#3281)
reduce number of messages sent
try move+cancel for every server

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-07-21 22:18:59 +02:00
Ivan Kozlovic
a02a617c05 Merge pull request #3280 from nats-io/fix_3273
[IMPROVED] JetStream: stream already exists error description
2022-07-21 10:53:47 -06:00
Ivan Kozlovic
1da5ecfb96 [IMPROVED] JetStream: stream already exists error description
The `JSStreamNameExistErr` will now include in the description that
the stream exists with a different configuration, because that is
the error clients would get when trying to add a stream with a
different configuration (otherwise this is a no-op and client
don't get an error).

Since that error was used in case of restore, a new error is added
but uses the same description prefix "stream name already in use"
but adds ", cannot restore" to indicate that this is a restore
failure because the stream already exists.

Resolves #3273

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-07-21 10:20:07 -06:00
Derek Collison
231fdea6c4 Merge pull request #3279 from nats-io/issue_3278
[FIXED] Issue #3278 - Make sure to protect against mset == nil
2022-07-21 09:05:52 -07:00
Derek Collison
f2abdaeb43 Make sure to protect against mset == nil
Signed-off-by: Derek Collison <derek@nats.io>
2022-07-21 06:53:26 -07:00
Matthias Hanel
51b6d5233f Fix raft issue where pindex of follower was off by 1 (#3277)
introduced by 57395bba02

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-07-21 00:51:26 +02:00
Ivan Kozlovic
45b1490362 Merge pull request #2363 from neilalexander/neilalexander/wasmjs
Stubs for WebAssembly
2022-07-20 13:26:56 -06:00
Ardit Begaj
94d22642d1 move test to appropriate file 2022-07-20 19:49:13 +02:00
Matthias Hanel
62bf8ce7e9 fix jwt unit test by ensuring activation issue time is bigger than expiration (#3275)
Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-07-19 21:38:58 +02:00
Ardit Begaj
3f26dfee25 added cluster test 2022-07-19 20:01:36 +02:00
Todd Beets
302f85dcc2 Merge pull request #3271 from nats-io/repub-fix-srcignored
Fix and test for RePub src ignored at runtime.
2022-07-18 15:02:09 -07:00
Todd Beets
6de64667bc follow convention (_EMPTY_, fwcs). Add special case short-circuit to skip unnecessary tokenization and transform. 2022-07-18 14:20:10 -07:00
Matthias Hanel
e753e897cf Bump version to 2.9.0-beta.8 (#3274)
Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-07-18 19:40:31 +02:00
Matthias Hanel
89b5e872ac Move and cancel fixes (#3270)
The Move/Cancel/Downscale mechanism did not take into account that
the consumer's replica count can be set independently.

This also alters peer selection to have the ability to skip 
unique tag prefix check for server that will be replaced.
Say you have 3 az, and want to add another server to az:1, 
in order to replace a server that is the same zone.
Without this change, uniqueTagPrefix check would filter 
the server to replace with and cause a failure.

The cancel move response could not be received due to 
the wrong account name.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-07-18 18:42:03 +02:00
Todd Beets
7a47631051 optimize for > or empty src (any subject) 2022-07-17 13:45:05 -07:00
Todd Beets
ae9f872ef5 Fix static code check violation on two unit tests and normalize unit test values to ADR-28. 2022-07-15 21:25:29 -07:00
Ivan Kozlovic
2c7d04f075 Merge pull request #3269 from nats-io/js_reject_stream_mirror_updates
[FIXED] JetStream: stream mirror updates not rejected in standalone
2022-07-15 15:21:22 -06:00
Todd Beets
b145952e78 Fix and test for RePub src ignored at runtime. 2022-07-15 13:50:35 -07:00