Commit Graph

3927 Commits

Author SHA1 Message Date
Derek Collison
abc5905aa9 Merge pull request #3221 from nats-io/direct
Made direct get from a stream part of the $JS.API hierarchy vs separate.
2022-06-28 09:59:44 -07:00
Matthias Hanel
aba1da090b [ADD] account specific in/out msgs/bytes stats to CONNS (#3187)
* [ADD] account specific in/out msgs/bytes stats to CONNS

This subject $SYS.ACCOUNT.%s.SERVER.CONNS will now respond with account
specific datastats for Received and sent messages as well as number of slow
consumers for the account.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-06-28 18:59:29 +02:00
Matthias Hanel
aabaf6f106 [fixed] reload related races (#3222)
account.rm had races caused by reload copying rm from one account to
another

mset.store was used outsisde the lock

in rare cases the stasz message was not received in time.
Trigger automatically now

sometimes a statsz message received before reload cause issues.
try receiving a second time

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-06-28 18:36:13 +02:00
Derek Collison
b8ef9b19a0 Made direct get from a stream part of the $JS.API hierarchy vs separate.
Also for direct get and for pull requests, if we are not on a client connection check how long we have been away from the readloop.
If need be execute in a separate go routine.

Signed-off-by: Derek Collison <derek@nats.io>
2022-06-28 08:53:48 -07:00
Derek Collison
b0c4ec69ab Merge pull request #3216 from nats-io/update_consumer_filtered_subj
Allow consumer filter subjects to be updated
2022-06-28 08:42:08 -07:00
Matthias Hanel
3421c49310 [Add] ability for operator to move streams (#3217)
Also added:
ability to reload tags
special tag (!jetstream) to remove peer from peer placement
$JS.API.SERVER.STREAM.MOVE subject to initiate move away from a server

This changes a detail about regular stream move as well.
Before differing cluster names where used to start/stop a transfer.
Now only the peer list and it's size relative to configured replica matter.
Once a transfer is considered completed, excess peers will be dropped
from the beginning of the list.
This allows transfers within the cluster as well.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-06-28 02:36:32 +02:00
Derek Collison
e02016db26 Fix race
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-27 16:05:03 -07:00
Derek Collison
92cd7821de Convert server mutex to RW.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-27 16:05:03 -07:00
Derek Collison
830d659aec Do not default to sharing all client info for JetStream, only server and cluster info.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-26 07:11:02 -07:00
Derek Collison
9154fca7f1 Allow consumer filter subjects to be updated
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-24 12:38:01 -07:00
Derek Collison
4de1766f40 Fix TLS tests and allow older test certs for now
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-24 09:43:42 -07:00
Derek Collison
cc197771ec Allow compile and staticheck to pass.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-24 09:17:12 -07:00
Derek Collison
4d6b562450 Bump version to 2.8.5-beta.6
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-21 09:12:22 -07:00
Derek Collison
314abd6028 Merge pull request #3205 from nats-io/issue_3199
[FIXED] Path separators in consumer or stream names prevented restoring a stream
2022-06-21 09:04:44 -07:00
Derek Collison
b4238259b3 Change test to not have path separators
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-20 12:16:46 -07:00
Derek Collison
1ade8fc881 When stream or consumer names contained path separators it prevented backup and restore.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-20 11:59:18 -07:00
Derek Collison
37f73ab229 Allow users directives for leafnodes to not block reloads.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-20 10:39:37 -07:00
Derek Collison
60b52b84c3 If no max bytes set do better randomization of placement.
If the stream is replicated take into consideration number of ha assets on the server already.

Signed-off-by: Derek Collison <derek@nats.io>
2022-06-15 17:12:45 -07:00
Derek Collison
bcffdb954f Bump version to 2.8.5-beta.5
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-15 08:09:01 -07:00
Derek Collison
d24ae4723f Support reload
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-15 07:58:09 -07:00
Derek Collison
9400733606 Allow for MQTT QoS-1 consumers to be auto cleanup after inactive threshold of time.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-14 17:37:45 -07:00
Derek Collison
677d53bd11 Merge pull request #3190 from nats-io/inactive_durables
Add in support for inactivity thresholds for durable consumers.
2022-06-14 10:55:10 -07:00
Derek Collison
4c8110c3ff Add in support for inactivity thresholds for durable consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-14 06:51:00 -07:00
Derek Collison
1adb3ae3bf Fix for data race - issue 3188
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-13 18:02:57 -07:00
Derek Collison
a830015703 Bump to 2.8.5-beta.4
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-12 09:40:41 -07:00
Derek Collison
aba38501f3 Merge pull request #3184 from nats-io/fss_opt
[IMPROVED] Performance of wildcard filtered consumer with stream with many subjects.
2022-06-12 09:37:56 -07:00
Derek Collison
af43dd3c74 When internal system messages were destined for a queue subscriber across a route the reply subject would be empty but not nil which caused mangling of the RMSG proto.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-11 14:12:05 -07:00
Derek Collison
148877b2f0 In the presence of many subjects in a stream and a wildcard filter subject, fall back to linear scan if too many.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-11 10:12:40 -07:00
Derek Collison
0794eafa6f Bump to 2.8.5-beta.2
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-07 07:49:03 -07:00
Derek Collison
fb51162e37 Make error on make bytes exceeded on a pull request a 409
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-07 06:59:08 -07:00
Derek Collison
301eb11725 Merge pull request #3168 from nats-io/no_fds_imp
[IMPROVED] Loaded server and low on resources like FDs.
2022-06-06 06:01:20 -07:00
Derek Collison
b64f3095ce Republish on the republish subject, place original in a header like direct get
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-05 16:21:39 -07:00
Derek Collison
f15934d3fb Bump version to 2.8.5-beta.1
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-05 15:27:36 -07:00
Derek Collison
dfc74dd4c1 Make sure only stream leader does a republish
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-05 15:27:07 -07:00
Derek Collison
fddc31adb5 Merge pull request #3158 from nats-io/kv-direct-get
[IMPROVED] Fast and Direct access to stream messages.
2022-06-05 07:04:18 -07:00
Derek Collison
e1c8f9fb55 This improves when a server is under load or low on resources like FDs and a user is trying to delete a stream with lots of consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-04 16:49:17 -07:00
R.I.Pienaar
52a1c542f5 export the correct subject transformer subject
While the TransformSubject function was doing the right
thing it did not match first and so would panic for subjects
that do not match the mapping.

The map function does the right thing so this is a more
appropriate function to export.

This undoes the exporting of unsafe TransformSubject and
exports the safer Match instead.

Signed-off-by: R.I.Pienaar <rip@devco.net>
2022-06-02 18:26:12 +02:00
Derek Collison
405de254b6 Mark meta recovering state and use to suppress api responses and api audits during restarts.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-01 20:06:27 -07:00
Derek Collison
d8e5bcec55 Merge pull request #3159 from nats-io/kv-fs-race-bug
[FIXED] When stores and load for last for subject where concurrent and compet…
2022-06-01 12:22:39 -07:00
Derek Collison
0979bce543 Updates based on group feedback.
1. Do not use original subject since this could use Request() and we want to use muxing.
2  Place original subject and timestamp into headers.

Signed-off-by: Derek Collison <derek@nats.io>
2022-05-31 19:15:52 -07:00
Derek Collison
1fe1f458d1 Make sure we do not resend responses on restart for certain clustered consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-31 18:57:05 -07:00
Derek Collison
6cc14ff84d When stores and load for last for subject where concurrent and competiting for the same msg, we could fail to retrieve a newly placed message.
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-31 18:04:05 -07:00
Derek Collison
c8a730ce55 Stream get for KV was going through API layer, but with popularity needed a more peformant and lighter weight and direct approach.
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-30 16:34:54 -07:00
Derek Collison
e08f6d863d Allow for republish to be headers only
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-30 12:05:17 -07:00
Derek Collison
5592315e89 Suppress consumer create and R1 stream update advisories on server restart.
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-30 09:58:35 -07:00
Derek Collison
daa4b97eeb Don't do advisories or API stats for a direct get msg from a stream.
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-30 09:32:07 -07:00
R.I.Pienaar
0bc1d96f65 Merge pull request #3154 from ripienaar/subject_transformer
Export the subject transformer
2022-05-27 22:58:42 +02:00
R.I.Pienaar
dc9d6776f8 Export the subject transformer
This exports the one key function of the subject transformer
allowing external tools to be written to test mappings are
valid and see how they would interact without the hassle of
configuring a serrver

The APIs are specifically marked as being unsupported and
having kept the transform struct itself unexported one can
not cast from the interface to the real implementation

Signed-off-by: R.I.Pienaar <rip@devco.net>
2022-05-27 10:33:59 +02:00
Ivan Kozlovic
72d45a046d Bump version to v2.8.5-beta
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-05-26 17:05:51 -06:00
Ivan Kozlovic
de52c0ba22 Release v2.8.4
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-05-26 10:15:52 -06:00