133 Commits

Author SHA1 Message Date
Matthias Hanel
eb3af67031 Delaying stream/consumer error response when there is no contact to meta
this gives a stream/consumer leader time to respond first

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-21 20:24:43 -04:00
Derek Collison
3788822f82 Updates based on feedback
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 12:41:11 -07:00
Derek Collison
9f1bc5882c Make sure to return no consumer if we are still waiting on a leader to be elected.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-21 12:13:10 -07:00
Ivan Kozlovic
56d0d9ec87 Do not propagate service import interest across GW and ROUTES
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-15 11:34:36 -06:00
Derek Collison
90989d57d6 Change to report total deleted by default for stream info.
Allow deleted details if requested.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-12 18:10:11 -07:00
Derek Collison
1ea4a430da If we fail to load an account while processing a stream assignment, send error back to metaleader.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-07 14:23:12 -07:00
Derek Collison
ceca86a79c Fix for #1976
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-31 11:23:31 -07:00
Matthias Hanel
03aee09847 [Added] error when mirror/source stream prefix overlaps with stream subs (#2041)
* [Added] error when mirror/source stream prefix overlaps with stream subs

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-27 14:01:51 -04:00
Derek Collison
e53caee5e8 Enforce server limits even when dynamic limits for accounts in play.
We were not properly enforcing server limits. This commit will allow a server to enforce limits but still remain functional even at the JetStream level.
Also fixed a bug for RAFT replay that could cause instability.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-25 16:06:27 -07:00
Derek Collison
2ed53035ed Reworked flow control for sources and mirrors.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-24 07:07:33 -07:00
Derek Collison
022c5b4ab2 Fix for bad behavior with flow control and multiple sources streams.
Allow chaining of sources and mirrors with filtered consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-22 11:13:51 -07:00
Derek Collison
ee92cc9a5b Properly print when a stream is doing out of band catchup. Print node banner consistently
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-14 07:29:36 -07:00
Derek Collison
cbbe6dc9c5 Make API access determing system not available consistent.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-14 06:18:04 -07:00
Derek Collison
299f44cddf This changes our behaviors for streams and peer removals in several ways.
First we no longer try to auto-remap stream assignments on peer removals from the system.
We also now can always respond to stream info requests if at least a member is running.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-11 06:52:28 -05:00
Derek Collison
f95b6481d2 Make source and mirror consumers direct, meaning they are not mapped by the metaleader.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-10 07:12:48 -05:00
Derek Collison
75b0455e0b Enable opt-in flow control for push based consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-05 08:48:12 -08:00
Derek Collison
207ebd3b3d Changed stream sendq to linked list outq.
Made consumer share streams outq.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 17:19:50 -08:00
Derek Collison
bfb8e3432e Move RAFT comms off internal sendq.
Move route and gateway msgs our of fast path for inbound stream msgs.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 14:45:34 -08:00
Ivan Kozlovic
0f53bf6580 Fixed data race with nodeInfo
Took the approach of storing struct instead of pointer. Of course,
when changing the offline bool from false to true, it means that
we need to call Store again (with same key).

This is based on the assumption that those Load/Store are not too
frequent. Otherwise, we may need to use locking (and keep *nodeInfo)

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-03-03 13:28:45 -07:00
Derek Collison
66cbf75712 Merge branch 'master' into consumer 2021-03-03 10:29:39 -07:00
R.I.Pienaar
236498a142 restore meta peer remove by name rather than id
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-03-03 15:55:50 +01:00
Derek Collison
e4d458c3f8 Make consumer processing of next message not inline if non client connection.
Also fixed a bug with processing ack floors.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 06:52:56 -08:00
Derek Collison
43b9017b74 Merge pull request #1953 from nats-io/api
JetStream API Changes
2021-03-02 19:46:00 -07:00
Matthias Hanel
c50ee2a1c6 [Changed] all times exposed will be computed in UTC (#1943)
This also applies to times that end up in that json.
Where applicable moved time.Now() to where it is used.
Moved calls to .UTC() to where time is created it that time is converted
later anyway.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-02 21:37:42 -05:00
Derek Collison
4583afc2b0 Updated comment
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 18:36:28 -08:00
Derek Collison
c39a2d6bd0 If capturing calls with no dispatch just return
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 17:58:34 -08:00
Derek Collison
4f7fbefc7c In clustered JetStream we need to move API calls out of routes/gateways/leafnodes path.
This moves from explicit imports and subscriptions to one wildcard subscription and a single wildcard export.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 17:54:41 -08:00
Derek Collison
a3c526a0eb Merge pull request #1951 from nats-io/xmirrors
Enable cross account behaviors for mirrors and sources.
2021-03-02 08:30:48 -07:00
Derek Collison
49cd38c064 Enable cross account behaviors for mirrors and sources.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 06:36:57 -08:00
R.I.Pienaar
a9ed69a2df update api types, remove meta peer by id
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-03-02 15:35:02 +01:00
Derek Collison
03954eedc6 Enable cluster server removal API.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 14:14:36 -08:00
Derek Collison
e0d08e1a22 Check for stream updates and disallow changes to mirrors and replicas for now.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 12:04:40 -08:00
Derek Collison
8cfc56f8b0 Enable support for changing config on stream restore.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 10:01:01 -08:00
Derek Collison
09467bd9ae Prep for restore config override
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 05:15:38 -08:00
Matthias Hanel
c73d3b21eb changing NEXT expires field from time to duration in nanoseconds
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-02-26 20:45:44 -05:00
Derek Collison
7c670e4eae Allow sources and mirrors to not be pre-defined.
Will keep trying to establish contact.

Fixed bug where double source headers for sourced messages were causing issues.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-26 07:32:29 -08:00
Derek Collison
c6672260af Merge pull request #1937 from nats-io/wio
[FIXED] Bug where followers would not snapshot/compact WAL.
2021-02-25 21:14:52 -07:00
Derek Collison
e5c8774172 Handle out of space situations, general stability enhancements
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 17:54:29 -08:00
R.I.Pienaar
157d92daf7 ensure sources and mirror info is in clustered stream list
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-02-25 12:57:41 +01:00
Derek Collison
09e3d26fa3 Add in support for stream mirrors and sources.
Add in proper support for stream updates in clustered mode.
Don't send API updates without subjects, caused GW parser errors.
Stream internal loops use their own clients now.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-23 10:57:27 -08:00
Derek Collison
2047b46e54 Updates based on feedback
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-19 11:07:31 -08:00
Derek Collison
fa8a74ceb5 Allow placement directives for metacontroller stepdown to allow placement to new clusters.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-19 10:55:22 -08:00
Ivan Kozlovic
61bd1b8d86 MQTT clustering
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-02-19 08:50:00 -07:00
Derek Collison
89fe3b05df various bug fixes, wal/snapshot stability
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-18 08:41:09 -08:00
Derek Collison
4c6e33c9c6 Restoration of streams would possibly block route and client connections.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-14 18:43:40 -08:00
Derek Collison
d803189eca Changes needed to properly support multi-layer service imports for system services like JS.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-12 17:01:32 -08:00
Derek Collison
fa8a95a06a Improved snapshots and compactions.
Various bug fixes and stability improvements.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-11 11:16:00 -08:00
Derek Collison
08fcc3646e Report any stepdown errors
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-08 15:15:29 -08:00
Derek Collison
ad0f10fb5c Allow metaleader stepdown.
Allow easy way to put authorization on system account and still use $G.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-08 15:01:23 -08:00
Derek Collison
c16f6e193d Move JetStream direct APIs to private.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-07 15:19:22 -08:00