gogrlx/nats-server
1
0
Fork 0
mirror of https://github.com/gogrlx/nats-server.git synced 2026-04-15 02:30:40 -07:00
Code Issues Packages Projects Releases Wiki Activity
6,080 Commits 30 Branches 48 Tags
b73afbdcb19b4e5d339f52c07af426bbb65553da
Commit Graph

4059 Commits

Author SHA1 Message Date
Ivan Kozlovic
b73afbdcb1 [FIXED] JetStream: reject stream update with changes to RePublish 
The update was not rejected, yet the republish update was not
taking place.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-08-03 16:37:39 -06:00
Derek Collison
f7387f7762 Bump to 2.9.0-beta.19 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-08-03 13:00:07 -07:00
Derek Collison
748890adb1 Auto-set and upgrade AllowDirect when MaxMsgsPerSubject is set. 
Also allow mirrors to inherit properly.

Signed-off-by: Derek Collison <derek@nats.io>
 2022-08-03 12:36:52 -07:00
Derek Collison
d7847c97c1 Bump to 2.9.0-beta.18 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-08-03 08:07:12 -07:00
Derek Collison
f45454b5a4 Merge pull request #3325 from nats-io/dg-perms 
Allow direct get by subject to be all subject based.
 2022-08-03 08:04:41 -07:00
Ivan Kozlovic
22b91238f3 Merge pull request #3323 from nats-io/js_raft_move_warn_to_debug 
Downgrade a RAFT warning to debug
 2022-08-03 08:52:52 -06:00
Matthias Hanel
ed2cb280cc Move to crypto/rand for nonce generation (#3324) 
Signed-off-by: Matthias Hanel <mh@synadia.com>
 2022-08-03 06:52:26 +02:00
Derek Collison
c82c49451c Allow direct get by subject to be all subject based. 
This avoids marshalling or unmarshalling but also allows subject based permissioning.

Signed-off-by: Derek Collison <derek@nats.io>
 2022-08-02 18:19:33 -07:00
Ivan Kozlovic
37c923c28e Downgrade a RAFT warning to debug 
This is related to PR #3307.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-08-02 18:06:39 -06:00
Derek Collison
6450301cfc Improved speed of storing new messages when lots of messages are present in KV mode. 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-08-01 14:02:19 -07:00
Derek Collison
903a06a5b4 Bump to 2.9.0-beta.17 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-31 11:44:15 -07:00
Ivan Kozlovic
b6b746095b Merge pull request #3315 from nats-io/fix_3313 
[FIXED] MQTT: Possible panic when clients misbehave
 2022-07-31 12:43:28 -06:00
Ivan Kozlovic
6460519cf5 [FIXED] MQTT: Possible panic when clients misbehave 
If a client with a given client ID is connected and while connected
another client tries to reuse the same client ID, the spec says that
the old client be closed and the new one accepted.
However, the server protects from this flapping happening all the time
by rejecting new clients that try to connect at a very fast pace.

However, the server was closing a misbehaving client after a second
delay (to prevent immediate reconnect if the client library does that)
but was not blocking the read loop and the compounding issue was that
if that misbehaving client is REALLY misbehaving and not waiting for
the CONNACK to send more protocols (for instance SUB) the server would
panic because the client was not fully configured.

To prevent that, the server will now "block" this misbehaving client
in its readLoop before closing the connection, preventing processing
of possible protocols that follow the CONNECT.

Resolves #3313

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-07-31 12:20:38 -06:00
Derek Collison
717969510d Make sure to reset block encryption counter when clearing block but holding state for tracking sequences. 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-31 07:59:19 -07:00
Derek Collison
8dc1e4b6de When compact would reclaim head of block space, we needed to update block key for counter for new writes. 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-30 13:05:41 -07:00
Derek Collison
e4ef3e4b09 Bump to 2.9.0-beta.16 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-29 16:16:14 -07:00
Derek Collison
5e98263de8 General stability improvements 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-29 16:02:31 -07:00
Byron Ruth
095cfef9eb Add check and test to prevent updating consumer MaxWaiting 
Signed-off-by: Byron Ruth <b@devel.io>
 2022-07-29 15:05:00 -04:00
Derek Collison
8aee7d5e51 Bump to 2.9.0-beta.15 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-29 09:30:07 -07:00
Derek Collison
43871300a3 Merge pull request #3308 from nats-io/encrypt_updates 
Encrypt meta and raft states.
 2022-07-29 09:25:55 -07:00
Derek Collison
259d1a1b9e Merge pull request #3307 from nats-io/raft_improvements 
Improvements to raft layer with snapshots on catchup.
 2022-07-29 09:12:24 -07:00
Derek Collison
27d87a68a4 Improvements to raft layer with snapshots on catchup. 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-29 09:01:03 -07:00
Derek Collison
50a25881e2 Encrypt meta and raft states. 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-29 08:10:57 -07:00
Ivan Kozlovic
746b44a020 Fixed TLS test so it passes on macOS 
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-07-28 17:58:25 -06:00
Ivan Kozlovic
f19908979f Revert direct changes to main 
This reverts commit cf784c19f0.
This reverts commit c269a1ca09.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-07-28 17:25:37 -06:00
jnmoyne
cf784c19f0 - Adds new subject mapping functions: 
{{SplitFromLeft(wildcard index, position)}}
{{SplitFromRight(wildcard index, position)}}
{{SliceFromLeft(wildcard index, slice size)}}
{{SliceFromRight(wildcard index, slice size)}}
{{Split(wildcard index, deliminator)}}

Examples:
	shouldMatch("*", "{{splitfromleft(1,3)}}", "12345", "123.45")
	shouldMatch("*", "{{SplitFromRight(1,3)}}", "12345", "12.345")
	shouldMatch("*", "{{SliceFromLeft(1,3)}}", "1234567890", "123.456.789.0")
	shouldMatch("*", "{{SliceFromRight(1,3)}}", "1234567890", "1.234.567.890")
	shouldMatch("*", "{{split(1,-)}}", "-abc-def--ghi-", "abc.def.ghi")
	shouldMatch("*.*", "{{split(2,-)}}.{{splitfromleft(1,2)}}", "foo.-abc-def--ghij-", "abc.def.ghij.fo.o")

- Subject mapping functions can now be all lower case or Pascal case (or a combination): e.g. splitfromleft, SplitFromLeft, splitFromleft, etc...
 2022-07-28 12:53:14 -07:00
jnmoyne
c269a1ca09 New expendable implementation of subject mapping destinations to transform processing 2022-07-28 12:30:22 -07:00
Derek Collison
e120bb86a9 Update tests to check last seq 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-28 07:23:39 -07:00
Ivan Kozlovic
725c1949e6 Bump version to 2.9.0-beta.14 
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-07-27 19:06:12 -06:00
Ivan Kozlovic
5786d2d9d6 Changed "return" to "continue" 
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-07-27 18:23:54 -06:00
Ivan Kozlovic
38727417df Moving super-cluster tests from cluster tests file to supercluster file 
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-07-27 17:14:19 -06:00
Ivan Kozlovic
88203dd5d5 Fixed a panic when consumer is closed 
Panic was:
```
=== RUN   TestJetStreamClusterDelete
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xcec8fb]
goroutine 1761 [running]:
github.com/nats-io/nats-server/v2/server.(*stream).config(0x0)
	/home/travis/gopath/src/github.com/nats-io/nats-server/server/stream.go:1192 +0x5b
github.com/nats-io/nats-server/v2/server.(*consumer).replica(0xc000101400)
	/home/travis/gopath/src/github.com/nats-io/nats-server/server/jetstream_cluster.go:3580 +0xea
github.com/nats-io/nats-server/v2/server.(*jetStream).monitorConsumer(0xc0001d2790, 0xc000101400, 0xc0004df0e0)
	/home/travis/gopath/src/github.com/nats-io/nats-server/server/jetstream_cluster.go:3733 +0xe06
github.com/nats-io/nats-server/v2/server.(*jetStream).processClusterCreateConsumer.func1()
	/home/travis/gopath/src/github.com/nats-io/nats-server/server/jetstream_cluster.go:3445 +0x4d
created by github.com/nats-io/nats-server/v2/server.(*Server).startGoRoutine
	/home/travis/gopath/src/github.com/nats-io/nats-server/server/server.go:3057 +0x85
FAIL	github.com/nats-io/nats-server/v2/server	9.911s
```

Seem to have been introduced in #3282

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-07-27 16:51:10 -06:00
Ivan Kozlovic
d0ee9a1252 Add comment that this is to silence the race detector. 
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-07-27 15:43:38 -06:00
Ivan Kozlovic
880e854637 Fixed data race between UpdateConfif() and subjString 
A message block is checking the filestore's cfg.Subjects to see
if it can "intern" the subject or not. The problem is that this
is done under the message block's lock, but not the filestore.
However, during a stream configuration update, the filestore's
cfg field is switched to a new one, causing the datarace.

By making sure we do the switch under all message blocks lock,
we remove the data race (that could be reproduce by running th
test TestJetStreamClusterMoveCancel with -count=10).

We investigating the use of a string interning library but it
showed a little performance degradation that this approach does
not suffer from.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-07-27 14:39:34 -06:00
Derek Collison
15382d9796 Bump to 2.9.0-beta.12 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-26 18:57:41 -07:00
Matthias Hanel
3358205de3 add implementation for consumer replica change (#3293) 
* add implementation for consumer replica change

fixes #3262

also check peer list on every update

Signed-off-by: Matthias Hanel <mh@synadia.com>
 2022-07-27 03:56:28 +02:00
Derek Collison
5f12c244ab Bump to 2.9.0-beta.11 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-26 17:25:28 -07:00
Matthias Hanel
04ffed48b0 fix peer tracking by removing peers before scaledown (#3289) 
in doRemovePeerAsLeader the leader also records the removed peer in the removed set

Signed-off-by: Matthias Hanel <mh@synadia.com>
 2022-07-26 22:01:03 +02:00
Matthias Hanel
6212087feb fix race by locking arround o.isLeader (#3291) 
Signed-off-by: Matthias Hanel <mh@synadia.com>
 2022-07-26 21:49:04 +02:00
Ivan Kozlovic
fe370955c8 Merge pull request #3288 from nats-io/debug_test_failure 
[FIXED] JetStream: Some scaling up issues
 2022-07-26 08:57:17 -06:00
Ivan Kozlovic
cb97cbea4a Merge pull request #3287 from nats-io/js_streams_limit 
[FIXED] JetStream/Cluster: Stream names/infos would return bad response
 2022-07-25 18:45:01 -06:00
Ivan Kozlovic
1a6c5f1c90 [FIXED] JetStream: Some scaling up issues 
- Send snapshot only if leader
- When processing snapshot, start with a smaller inactivity interval
  that will double up to 10sec or use 10sec directly once we get a
  message. Reason for that is that it is possible that the request
  for snapshot is sent while the leader has not yet setup the subscription
  that receives the requests (or subscription has not fully reached the
  cluster).
- Don't remember snapfile on err.
- Do not consider current if we have not had any activity.
- Stabilize stream scale up under active heavy publishing.
- Due to the publish pressure move the check for followers direct subs spinning up til after we stop publishing.

Signed-off-by: Derek Collison <derek@nats.io>
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-07-25 18:44:18 -06:00
Ivan Kozlovic
ebeca00e20 [FIXED] JetStream/Cluster: Stream names/infos would return bad response 
If there are more stream names that the current limit of 1024, getting
the list of names would return them all instead of using pagination.

For "stream infos", the Total amount returned would be the API limit
instead of the actual number of streams.

Resolves https://github.com/nats-io/natscli/issues/541

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-07-25 14:41:05 -06:00
Derek Collison
6e0d3eaa9d Update compress dependency 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-25 11:31:12 -07:00
Derek Collison
e87a30fdb7 Bump to 2.9.0-beta.10 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-23 19:21:58 -07:00
Derek Collison
52f7765322 When msgs were expired on restart recovery we could lose track on subsequent restart of starting sequence with no additional activity. 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-23 17:15:16 -07:00
Ivan Kozlovic
07554e970a Bump version to 2.9.0-beta.9 
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
 2022-07-22 11:56:15 -06:00
Matthias Hanel
5a720d4977 down scale consumer before downscale of stream (#3282) 
Now monitorStream waits with scaling down the stream until all
monitorConsumer have scaled down their respective consumer

Also update consumer assignment for later use in monitorConsumer
Same for stream assignment in monitorStream

Signed-off-by: Matthias Hanel <mh@synadia.com>
 2022-07-22 19:54:13 +02:00
Derek Collison
3a10456e68 Short index write could lead to loss of stream sequence for empty stream 
Signed-off-by: Derek Collison <derek@nats.io>
 2022-07-22 06:37:19 -07:00
Ivan Kozlovic
a3e62f000c Merge pull request #3265 from abegaj/fix-max-deliver-update 
[FIXED] Maximum Deliveries update has no effect
 2022-07-21 16:06:41 -06:00