Commit Graph

3549 Commits

Author SHA1 Message Date
jnmoyne
130b254699 Revert "Initial commit"
This reverts commit b10388c979.
2022-02-21 15:06:40 -08:00
jnmoyne
b10388c979 Initial commit 2022-02-19 23:57:30 -08:00
Derek Collison
58806c1290 Bump to 2.7.3-beta.2
Signed-off-by: Derek Collison <derek@nats.io>
2022-02-17 12:41:30 -08:00
Derek Collison
2942d012f6 Merge pull request #2878 from nats-io/key_file_leak
Cleanup key files when removing message blocks.
2022-02-17 13:26:41 -07:00
Derek Collison
330a40009c Cleanup key files when removing message blocks.
Signed-off-by: Derek Collison <derek@nats.io>
2022-02-17 11:33:41 -08:00
Derek Collison
255a7c3792 Merge pull request #2875 from nats-io/issue_2873
[FIXED] Interest policy and staggered filtered consumers could fail to remove messages.
2022-02-17 12:31:24 -07:00
Derek Collison
4efce40bbd Small improvements to send performance to a full stream.
Cleaned up some locking and if fifo make index updates lazy like writeMsgRecord.

Signed-off-by: Derek Collison <derek@nats.io>
2022-02-17 05:39:27 -08:00
Derek Collison
1c8f7de848 On filtered subjects when consumers were staggered we need to disqualify a filtered consumer if not applicable.
Signed-off-by: Derek Collison <derek@nats.io>
2022-02-16 18:24:27 -08:00
Derek Collison
ca1132a01d Allow stream placement by tags.
Signed-off-by: Derek Collison <derek@nats.io>
2022-02-15 17:07:32 -08:00
Derek Collison
fb15dfd9b7 Allow replica updates during stream update.
Also add in HAAssets count to Jsz.

Signed-off-by: Derek Collison <derek@nats.io>
2022-02-13 19:33:46 -08:00
Derek Collison
5a93b0e9d8 Allow pull requests to specify a heartbeat when idle to detect when a request is invalidated.
Signed-off-by: Derek Collison <derek@nats.io>
2022-02-11 09:51:51 -08:00
R.I.Pienaar
6bb0861eb7 avoid seg fault when stream restore fails
Signed-off-by: R.I.Pienaar <rip@devco.net>
2022-02-11 10:45:09 +01:00
Ivan Kozlovic
55ffde7251 Fixed consumer dlv count and num pending wrong due to redeliveries
Introduced by #2848, so should not have impacted existing releases.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-02-10 17:33:53 -07:00
Derek Collison
9010f3542a Bump to 2.7.3-beta
Signed-off-by: Derek Collison <derek@nats.io>
2022-02-09 17:36:15 -08:00
Derek Collison
6fcd87935c Merge pull request #2859 from nats-io/fss_cleanup
Remove fss files from a snapshot when a block is removed.
2022-02-09 17:33:29 -08:00
Derek Collison
68104d7cf3 During a filestore snapshot we generate the fss files but were not cleaning them up if the block was deleted before a server restart.
https://gist.github.com/nekufa/010185dfb59261f222a0042d3a7d2a1c

Signed-off-by: Derek Collison <derek@nats.io>
2022-02-09 17:12:08 -08:00
Derek Collison
ecfe42630a Merge pull request #2858 from nats-io/add_consumer_with_info
Make sure we snapshot initial consumer info during consumer creation.
2022-02-09 17:05:01 -08:00
Derek Collison
da9046b2e6 Snapshot initial consumer info when needed.
Signed-off-by: Derek Collison <derek@nats.io>
2022-02-09 15:23:53 -08:00
Derek Collison
0cc7302be9 A stream name is tied to its identity and can not be changed on a restore.
Signed-off-by: Derek Collison <derek@nats.io>
2022-02-09 12:38:45 -08:00
Ivan Kozlovic
3dcf0246c6 [FIXED] Adding a consumer could return inaccurate consumer info
The issue is that the consumer info returned by the consumer create
API is gathered after the consumer is added and possibly after
starting to deliver pending messages.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-02-09 09:04:16 -07:00
Derek Collison
c13a84cf44 Fixed a bug that would calculate the first sequence of a filteredPending incorrectly.
Also added in more optimized version to select the first matching message in a message block for LoadNextMsg.

Signed-off-by: Derek Collison <derek@nats.io>
2022-02-08 13:29:38 -08:00
Derek Collison
d50febeeff Improved sparse consumers replay time.
When a stream has multiple subjects and a consumer filters the stream to a small and spread out list of messages the logic would do a linear scan looking for the next message for the filtered consumer.
This CL allows the store layer to utilize the per subject info to improve the times.

Signed-off-by: Derek Collison <derek@nats.io>
2022-02-07 17:26:32 -08:00
Derek Collison
d71bb63e1b Merge pull request #2847 from nats-io/fc_stall
Fixed flow control stall under specific conditions of message size.
2022-02-07 09:28:17 -08:00
Derek Collison
55b7f11c9a Fixed flow control stall under specific conditions of message size.
Signed-off-by: Derek Collison <derek@nats.io>
2022-02-05 20:15:48 -08:00
Ivan Kozlovic
6d92051696 Release v2.7.2
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-02-04 14:49:39 -07:00
Derek Collison
a0a2e32185 Remove dynamic account behaviors.
We used these in tests and for experimenting with sandboxed environments like the demo network.

Signed-off-by: Derek Collison <derek@nats.io>
2022-02-04 13:32:18 -08:00
Derek Collison
5da0453964 Add in NumSubjects to StreamInfo
Signed-off-by: Derek Collison <derek@nats.io>
2022-02-02 08:51:13 -08:00
Derek Collison
6a3cf0f71e Added in ability to get number of subjects from StreamInfo, and optionally details per subject on how many messages each subject has.
This can also be filtered, meaning you can filter out the subjects when asking for details.

Signed-off-by: Derek Collison <derek@nats.io>
2022-02-02 08:51:13 -08:00
Derek Collison
12f5ea3655 When a consumer had not filtered subject and was attached to a interest policy retention stream we could incorrectly drop messages.
Signed-off-by: Derek Collison <derek@nats.io>
2022-02-01 14:21:05 -08:00
Derek Collison
6b5332249b This test was using fetch and failing if the complete batch was not filled.
This has nothing to do with the test, we just want to make sure the leader steps down and there were no low level errors on the fetch.

Signed-off-by: Derek Collison <derek@nats.io>
2022-02-01 13:34:00 -08:00
Derek Collison
8fce45dfa7 Under certain scenarios the pending for a consumer could appear to get stuck.
Under the covers we were calculating pending per msg block incorrectly when a single message existed beyond the requested sequence.

Signed-off-by: Derek Collison <derek@nats.io>
2022-02-01 12:17:08 -08:00
Ivan Kozlovic
30c431a9a3 [FIXED] JetStream: BackOff redeliveries would always use first in list
If the consumer's sequence was not the same than the stream's sequence,
then the redelivery would always use the first duration from the
BackOff list.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-01-31 17:44:08 -07:00
Derek Collison
0d158728d1 Merge pull request #2824 from nats-io/fix-nodeinfo
Store JetStream Config in node info map
2022-01-31 13:58:12 -08:00
Derek Collison
fa814f7cee Fixed behavior for when MaxMsgsPerSubject is set and DiscardNew is also set.
Signed-off-by: Derek Collison <derek@nats.io>
2022-01-31 08:36:37 -08:00
Derek Collison
6b690bd5b6 Bump to 2.7.2-beta
Signed-off-by: Derek Collison <derek@nats.io>
2022-01-30 15:24:07 -08:00
Derek Collison
8815072e34 Fix flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2022-01-30 14:54:24 -08:00
Derek Collison
b38ced51b2 A true no wait pull request was not considering redeliveries.
Signed-off-by: Derek Collison <derek@nats.io>
2022-01-30 11:52:35 -08:00
Derek Collison
275d42628b Fix for #2828. The original design of the consumer and the subsequent store did not allow updates.
Now that we do, we need to store the new config into our storage layer.

Signed-off-by: Derek Collison <derek@nats.io>
2022-01-30 09:45:05 -08:00
Derek Collison
a57bd96def Updating a push consumer to be pull would succeed but cause a panic if used.
This disallows that upgrade. We had a check in place for pull to push, but not the reverse.

Signed-off-by: Derek Collison <derek@nats.io>
2022-01-28 13:11:58 -08:00
Jaime Piña
ae8eedb88e Store JetStream Config in node info map 2022-01-27 14:46:41 -08:00
Ivan Kozlovic
37ff12cc7a Release v2.7.1
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-01-25 13:54:12 -07:00
Ivan Kozlovic
7f81f2d4c6 Merge pull request #2816 from nats-io/revocation-issue-442
Fix jwt based user/activation token revocation and granularity
2022-01-25 13:42:14 -07:00
Matthias Hanel
c5cc81bc1d Fix collectRevocations
Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-01-25 15:02:47 -05:00
Matthias Hanel
274ec6db65 incorporate comments
Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-01-25 14:53:03 -05:00
Matthias Hanel
fa12d096cd Fix jwt based user/activation token revocation and revocation granularity
user and activation token did not honor the jwt value for all * on
connect.

activation token where not re evaluated when the export revoked a key.
In part this is a consistency measure so servers that already have an
account and servers that don't behave the same way.

in jwt activation token revocations are stored per export.
The server stored them per account, thus effectively merging
revocations. Now they are stored per export inside the server too.

fixes nats-io/nsc/issues/442

Signed-off-by: Matthias Hanel <mh@synadia.com>
2022-01-25 13:48:12 -05:00
Derek Collison
6486cd8fc8 Added in /healthz endpoint for health and liveness probes in environments like k8s.
Currently this code returns a 200 and { "status": "ok" } iff all configured ports are open
and if JetStream is configured and we have contact with the metaleader and the cluster and all streams are up to date.

Signed-off-by: Derek Collison <derek@nats.io>
2022-01-24 19:30:10 -08:00
Derek Collison
6be9925127 Update config error
Signed-off-by: Derek Collison <derek@nats.io>
2022-01-24 15:02:41 -08:00
Derek Collison
65b168aa8b Updates based on feedback. MaxDeliver needs to be set properly now to be > len(BackOff) but if larger we will reuse last value in BackOff array.
Signed-off-by: Derek Collison <derek@nats.io>
2022-01-24 15:02:39 -08:00
Derek Collison
bd78b1a99b Formal json version for NAK delay
Signed-off-by: Derek Collison <derek@nats.io>
2022-01-24 15:01:52 -08:00
Derek Collison
d486c24199 Allow a consumer to be configured with BackOffs.
This allows a consumer to have exponential backoffs vs static AckWait and MaxDeliver.
When BackOff is set it will overridde AckWait to BackOff[0] and MaxDeliver will be len(BackOff)+1.

Signed-off-by: Derek Collison <derek@nats.io>
2022-01-24 14:57:36 -08:00