Commit Graph

2425 Commits

Author SHA1 Message Date
Derek Collison
e0d08e1a22 Check for stream updates and disallow changes to mirrors and replicas for now.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 12:04:40 -08:00
Derek Collison
8cfc56f8b0 Enable support for changing config on stream restore.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 10:01:01 -08:00
Derek Collison
b9e1a921ff Use internal wildcard inbox for stream and consumer info requests.
More gateway friendly but suffers from no echo, so added new client based internal send.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 10:01:01 -08:00
Derek Collison
57a6599777 Test updates
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 05:19:45 -08:00
Derek Collison
ef4567f24a Changes for sources and mirrors improvements.
Better handling of messages on restart from a WAL.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 05:18:48 -08:00
Derek Collison
9fc7561c40 Support updating sources via stream update.
Tweaked internal consumers and flow logic for sources and mirrors.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 05:17:17 -08:00
Derek Collison
0d29b0761a Tweaked buffered channels, moved locks for snapshots.
Also placed debug for inline processing of append entries.
This is for removal of that inline.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 05:16:04 -08:00
Derek Collison
09467bd9ae Prep for restore config override
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 05:15:38 -08:00
Derek Collison
f5cbd55b46 Fixed data corruption bug, optimized Compact().
Also trim fs.blks slice appropriately to avoid unbound growth.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 05:14:25 -08:00
Derek Collison
6a03ab9629 Inline flow in clustered mode was flaky, removed that path.
Changed up accounting.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 05:11:59 -08:00
Matthias Hanel
c73d3b21eb changing NEXT expires field from time to duration in nanoseconds
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-02-26 20:45:44 -05:00
Derek Collison
48d9e1cdb0 Bump Version
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-26 07:53:48 -08:00
R.I.Pienaar
89f6bdbe09 Merge pull request #1941 from ripienaar/oos_advisory
extend the out of space advisory
2021-02-26 16:53:07 +01:00
Derek Collison
7c670e4eae Allow sources and mirrors to not be pre-defined.
Will keep trying to establish contact.

Fixed bug where double source headers for sourced messages were causing issues.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-26 07:32:29 -08:00
R.I.Pienaar
a4817bd7b6 extend the out of space advisory
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-02-26 11:10:05 +01:00
Derek Collison
1f43c8b718 Bump version
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 20:15:41 -08:00
Derek Collison
c6672260af Merge pull request #1937 from nats-io/wio
[FIXED] Bug where followers would not snapshot/compact WAL.
2021-02-25 21:14:52 -07:00
Derek Collison
69ffff2278 Skip test for now
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 20:02:06 -08:00
Derek Collison
98f98e214b Properly support memory based WALs
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 19:49:54 -08:00
Derek Collison
7ea5a8a64b Reverse the flag for fip
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 19:02:58 -08:00
Derek Collison
0f69e48511 Bug check err, check for out of space on catchup
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 18:25:16 -08:00
Derek Collison
e5c8774172 Handle out of space situations, general stability enhancements
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 17:54:29 -08:00
Derek Collison
b13ef6b9ec Track write errors. Fixed a few bugs.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 17:53:20 -08:00
Derek Collison
44b08c538a Pass deliver, don't remap
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 17:42:03 -08:00
Derek Collison
752f67acee Explicit on async and sync settings
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 17:08:14 -08:00
Derek Collison
a9ab41f13b Fix lock bug, capture write errors and report better
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 15:01:47 -08:00
R.I.Pienaar
45a42864f6 always store the filter subject
This avoids a situation where a consumer asks for
ORDERS.new today on a stream of ORDERS.new but later
someone makes the same stream ORDERS.* and the new
consumer would then get messages for other purposes
as well

Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-02-25 18:29:21 +01:00
Derek Collison
7ef3e5b629 Remove slow advisory checks
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 04:17:56 -08:00
Derek Collison
a862cc75cc Suppress raft campaigns on restart. Extend election timeout interval.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 04:14:14 -08:00
R.I.Pienaar
157d92daf7 ensure sources and mirror info is in clustered stream list
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-02-25 12:57:41 +01:00
Derek Collison
c7d203db6d Fixed bug where followers WAL would not snapshot/compact.
Tweaked write and sync params.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 20:51:28 -08:00
Derek Collison
b7a5587b0c spelling
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 20:50:00 -08:00
Derek Collison
73ba2d0b2f File writes to term and vote and peerstate were in the direct route path and could cause delays.
This moves the actual writes to a separate Go routine and also allows multiple writes to
be compressed into one write under load. We only want latest.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 20:47:31 -08:00
Derek Collison
2d24571e17 Change default sync to 60s
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 20:44:05 -08:00
Derek Collison
57403b1903 When pulling consumer messages if the consumer and the client were on the same server we needed to process local deliver subject.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 19:28:46 -08:00
Derek Collison
3def02a459 check for '\n' at beginning og key
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 17:46:30 -08:00
Derek Collison
75fbcffedb Make sure it's the exact key
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 14:56:44 -08:00
Derek Collison
36a2cbeff7 Cross account pull consumers would not work correctly due to reply subject rewrite.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 13:33:24 -08:00
Derek Collison
23d7725104 Removed this one due to slow timing now
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 09:16:50 -08:00
Derek Collison
84321f6697 These are slower so need to wait longer
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 08:55:58 -08:00
Derek Collison
78bdc34637 General stability improvements. Fixes to subscription state not cleaning up.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 08:44:34 -08:00
Ivan Kozlovic
1652fe62ef Updates to when do snapshot
Remove panic on runAsLeader when not able to subscribe (which happens
on shutdown)
Gateway name access does not need lock since it is immutable. Will
prevent deadlocks in some situations.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-02-23 19:06:07 -07:00
Derek Collison
bb58d455f6 Revert switching to interest only mode
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-23 18:00:47 -08:00
Derek Collison
ebd522d732 Test cleanup
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-23 17:20:30 -08:00
Derek Collison
20d250c7cf Move storage type to own variable for unlocked access, can't change
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-23 17:20:12 -08:00
Derek Collison
8fe8b835fe Fixes for flapping tests
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-23 14:08:17 -08:00
Derek Collison
09e3d26fa3 Add in support for stream mirrors and sources.
Add in proper support for stream updates in clustered mode.
Don't send API updates without subjects, caused GW parser errors.
Stream internal loops use their own clients now.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-23 10:57:27 -08:00
Derek Collison
dd19f620a9 Does not need receiver
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-23 10:57:18 -08:00
Derek Collison
6d6a6c07ff Don't send empty subjects, always put system account in interest only
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-23 10:57:12 -08:00
Derek Collison
6444038660 Make sure this is GW and leafnode friendly
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-23 10:57:10 -08:00