Commit Graph

3914 Commits

Author SHA1 Message Date
Derek Collison
314abd6028 Merge pull request #3205 from nats-io/issue_3199
[FIXED] Path separators in consumer or stream names prevented restoring a stream
2022-06-21 09:04:44 -07:00
Derek Collison
b4238259b3 Change test to not have path separators
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-20 12:16:46 -07:00
Derek Collison
1ade8fc881 When stream or consumer names contained path separators it prevented backup and restore.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-20 11:59:18 -07:00
Derek Collison
37f73ab229 Allow users directives for leafnodes to not block reloads.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-20 10:39:37 -07:00
Derek Collison
60b52b84c3 If no max bytes set do better randomization of placement.
If the stream is replicated take into consideration number of ha assets on the server already.

Signed-off-by: Derek Collison <derek@nats.io>
2022-06-15 17:12:45 -07:00
Derek Collison
bcffdb954f Bump version to 2.8.5-beta.5
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-15 08:09:01 -07:00
Derek Collison
d24ae4723f Support reload
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-15 07:58:09 -07:00
Derek Collison
9400733606 Allow for MQTT QoS-1 consumers to be auto cleanup after inactive threshold of time.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-14 17:37:45 -07:00
Derek Collison
677d53bd11 Merge pull request #3190 from nats-io/inactive_durables
Add in support for inactivity thresholds for durable consumers.
2022-06-14 10:55:10 -07:00
Derek Collison
4c8110c3ff Add in support for inactivity thresholds for durable consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-14 06:51:00 -07:00
Derek Collison
1adb3ae3bf Fix for data race - issue 3188
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-13 18:02:57 -07:00
Derek Collison
a830015703 Bump to 2.8.5-beta.4
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-12 09:40:41 -07:00
Derek Collison
aba38501f3 Merge pull request #3184 from nats-io/fss_opt
[IMPROVED] Performance of wildcard filtered consumer with stream with many subjects.
2022-06-12 09:37:56 -07:00
Derek Collison
af43dd3c74 When internal system messages were destined for a queue subscriber across a route the reply subject would be empty but not nil which caused mangling of the RMSG proto.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-11 14:12:05 -07:00
Derek Collison
148877b2f0 In the presence of many subjects in a stream and a wildcard filter subject, fall back to linear scan if too many.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-11 10:12:40 -07:00
Derek Collison
0794eafa6f Bump to 2.8.5-beta.2
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-07 07:49:03 -07:00
Derek Collison
fb51162e37 Make error on make bytes exceeded on a pull request a 409
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-07 06:59:08 -07:00
Derek Collison
301eb11725 Merge pull request #3168 from nats-io/no_fds_imp
[IMPROVED] Loaded server and low on resources like FDs.
2022-06-06 06:01:20 -07:00
Derek Collison
b64f3095ce Republish on the republish subject, place original in a header like direct get
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-05 16:21:39 -07:00
Derek Collison
f15934d3fb Bump version to 2.8.5-beta.1
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-05 15:27:36 -07:00
Derek Collison
dfc74dd4c1 Make sure only stream leader does a republish
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-05 15:27:07 -07:00
Derek Collison
fddc31adb5 Merge pull request #3158 from nats-io/kv-direct-get
[IMPROVED] Fast and Direct access to stream messages.
2022-06-05 07:04:18 -07:00
Derek Collison
e1c8f9fb55 This improves when a server is under load or low on resources like FDs and a user is trying to delete a stream with lots of consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-04 16:49:17 -07:00
R.I.Pienaar
52a1c542f5 export the correct subject transformer subject
While the TransformSubject function was doing the right
thing it did not match first and so would panic for subjects
that do not match the mapping.

The map function does the right thing so this is a more
appropriate function to export.

This undoes the exporting of unsafe TransformSubject and
exports the safer Match instead.

Signed-off-by: R.I.Pienaar <rip@devco.net>
2022-06-02 18:26:12 +02:00
Derek Collison
405de254b6 Mark meta recovering state and use to suppress api responses and api audits during restarts.
Signed-off-by: Derek Collison <derek@nats.io>
2022-06-01 20:06:27 -07:00
Derek Collison
d8e5bcec55 Merge pull request #3159 from nats-io/kv-fs-race-bug
[FIXED] When stores and load for last for subject where concurrent and compet…
2022-06-01 12:22:39 -07:00
Derek Collison
0979bce543 Updates based on group feedback.
1. Do not use original subject since this could use Request() and we want to use muxing.
2  Place original subject and timestamp into headers.

Signed-off-by: Derek Collison <derek@nats.io>
2022-05-31 19:15:52 -07:00
Derek Collison
1fe1f458d1 Make sure we do not resend responses on restart for certain clustered consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-31 18:57:05 -07:00
Derek Collison
6cc14ff84d When stores and load for last for subject where concurrent and competiting for the same msg, we could fail to retrieve a newly placed message.
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-31 18:04:05 -07:00
Derek Collison
c8a730ce55 Stream get for KV was going through API layer, but with popularity needed a more peformant and lighter weight and direct approach.
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-30 16:34:54 -07:00
Derek Collison
e08f6d863d Allow for republish to be headers only
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-30 12:05:17 -07:00
Derek Collison
5592315e89 Suppress consumer create and R1 stream update advisories on server restart.
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-30 09:58:35 -07:00
Derek Collison
daa4b97eeb Don't do advisories or API stats for a direct get msg from a stream.
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-30 09:32:07 -07:00
R.I.Pienaar
0bc1d96f65 Merge pull request #3154 from ripienaar/subject_transformer
Export the subject transformer
2022-05-27 22:58:42 +02:00
R.I.Pienaar
dc9d6776f8 Export the subject transformer
This exports the one key function of the subject transformer
allowing external tools to be written to test mappings are
valid and see how they would interact without the hassle of
configuring a serrver

The APIs are specifically marked as being unsupported and
having kept the transform struct itself unexported one can
not cast from the interface to the real implementation

Signed-off-by: R.I.Pienaar <rip@devco.net>
2022-05-27 10:33:59 +02:00
Ivan Kozlovic
72d45a046d Bump version to v2.8.5-beta
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-05-26 17:05:51 -06:00
Ivan Kozlovic
de52c0ba22 Release v2.8.4
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-05-26 10:15:52 -06:00
Ivan Kozlovic
a52f12613e Bump version to v2.8.4-beta.2 and fix flapper
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-05-25 13:25:45 -06:00
Ivan Kozlovic
1c984d40f6 Merge pull request #3150 from nats-io/mqtt_sess_diff_domains
[FIXED] MQTT: Sessions with same ID in different domains were considered duplicates
2022-05-25 13:06:32 -06:00
Ivan Kozlovic
b344519176 [FIXED] MQTT: Same session ID in different domains were considered duplicates
There is a mechanism to detect if a connection somewhere in the
cluster is using the session ID of an existing one, and if so,
close one as a duplicate.
However, when different domains are used, they should not be considered
duplicates.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-05-25 11:16:51 -06:00
Derek Collison
72ed48d096 Merge pull request #3149 from nats-io/pull_perf_stable
[FIXED] Spurious pull consumer 408s under load
2022-05-25 09:44:27 -07:00
Derek Collison
d69394efad Fix spurious 408s under load and move processing of acks to their own Go routine.
Signed-off-by: Derek Collison <derek@nats.io>
2022-05-25 09:27:34 -07:00
Derek Collison
46f7f7bfc9 Consumer pending was not correct when stream had max msgs per subject set > 1 and a consumer that filtered out part of the stream was created.
Also make sure to update stream's config on a stream restore in case of changes.

Signed-off-by: Derek Collison <derek@nats.io>
2022-05-24 14:44:15 -07:00
Ivan Kozlovic
7bdd6799b7 Bump version to dev v2.8.4-beta
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-05-23 15:27:40 -06:00
Ivan Kozlovic
f423f07134 Release v2.8.3
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-05-23 14:48:19 -06:00
Ivan Kozlovic
53e3c53d96 [FIXED] JetStream: consumer with deliver new may miss messages
This could happen when a consumer had not sent anything to the
attached NATS subscription and there was a consumer leader
step down or server restart.

Signed-off-by: Derek Collison <derek@nats.io>
2022-05-23 12:01:48 -06:00
Ivan Kozlovic
65094c4ee4 Merge pull request #3142 from nats-io/js_routed_api_reqs
[FIXED] Fast routed JetStream API requests were dropped
2022-05-23 11:54:54 -06:00
Ivan Kozlovic
66b1b51182 [FIXED] MQTT: Errors deleting consumers will now prevent deletion of session
When there was a failure to delete a QoS1 consumer, the session
would still be deleted, which would cause orphaned consumers.

In case of error, the session record will not be deleted, which means
that it is still possible to restart the session and then close
it (with the clean flag).

Relates to #3116

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-05-23 11:28:18 -06:00
Ivan Kozlovic
4bf81420e2 [FIXED] Fast routed JetStream API requests were dropped
If a JS API request is received from a non client connection, it
was processed in its own go routine. To reduce the number of
such go routine, we were limiting the number of outstanding routines
to 4096. However, in some situations, it was possible to issue
many requests at the same time that would then cause those requests
to be dropped.

(an example was an MQTT benchmark tool that would create 5000
sessions, each with one QoS1 R1 consumer (with the use of consumer_replicas=1).
On abrupt exit of the tool, the consumers and their sessions needed
to be deleted. Since would cause fast incoming delete consumer requests
which would cause the original code to drop some of them)

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-05-23 11:15:55 -06:00
Derek Collison
75129609bb Merge pull request #3137 from nats-io/consumer_num_pending
Consumer's num pending can now rely on the stream's store
2022-05-21 11:43:44 -07:00