Commit Graph

106 Commits

Author SHA1 Message Date
Waldemar Quevedo
ed81f84ce5 test: Use lame duck and wait shutdown on JS tests
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-04-14 10:34:11 -07:00
Derek Collison
f6a82a7c98 When messages were no longer available in an upstream stream a mirror could wedge and not resolve.
This fixes that scenario by detecting the situation and inserting skip msgs to catch up.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-13 11:46:03 -07:00
Derek Collison
90989d57d6 Change to report total deleted by default for stream info.
Allow deleted details if requested.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-12 18:10:11 -07:00
Derek Collison
755ef74855 When a cluser of leafnodes connects to a cluster or supercluster hub and they share the system account make the leafnode servers observers.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-12 17:00:55 -07:00
Derek Collison
f893345b40 Pull based consumers when deleted were not removing messages from an interest based stream.
Fix for #2097.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-12 11:16:31 -07:00
Derek Collison
66b0c5c216 Merge pull request #2100 from nats-io/csz
Only adjust cluster size when we have heard from all peers.
2021-04-12 08:52:07 -07:00
Derek Collison
0cee993e3b When checking cluster size we need to make sure we have heard from all peers before making adjustments.
Also check back periodically.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-10 15:55:51 -07:00
Derek Collison
833279388b Add mixedmode test back in
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-10 10:10:37 -07:00
Jaime Piña
27e9628c3a Run gofmt -s to simplify code 2021-04-09 15:18:06 -07:00
Derek Collison
3c051d461c Fix flappers
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-08 18:03:14 -07:00
Derek Collison
deb015ec73 Fix for #2083 to release ack pending when messages expire or hit max redeliveries.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-08 11:47:59 -07:00
Derek Collison
16941c78a0 Merge pull request #2076 from nats-io/account_load
Report an error if we fail to load an account on a stream assignment.
2021-04-07 14:40:27 -07:00
Derek Collison
1ea4a430da If we fail to load an account while processing a stream assignment, send error back to metaleader.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-07 14:23:12 -07:00
Jaime Piña
d929ee1348 Check errors when removing test directories and files
Currently in tests, we have calls to os.Remove and os.RemoveAll where we
don't check the returned error. This hides useful error messages when
tests fail to run, such as "too many open files".

This change checks for more filesystem related errors and calls t.Fatal
if there is an error.
2021-04-07 11:09:47 -07:00
Jaime Piña
e44275b963 Consolidate temporary test files and directories
Currently, temporary test files and directories are written in lots of
different paths within the OS's temp dir. This makes it hard to know
which files are from nats-server and which are unrelated. This in turn
makes it hard to clean up nats-server test files.
2021-04-06 10:42:55 -07:00
Derek Collison
0650b772a9 Fix test, needed prefix
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-30 13:46:03 -07:00
Jaime Piña
6941bb3ade Update Go client in tests 2021-03-30 13:17:34 -07:00
Derek Collison
bb7a8a5f79 Introduced default max ack pending for ack explicit.
Fixed a bug that would introduce performance degradation for durable consumers R>1.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-30 11:47:24 -07:00
Derek Collison
5a48369b4b Make sure to not delete streams on bad updates.
If an update was asssigned but failed at the stream group server we would send back the result which would always delete the stream.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-29 07:35:30 -07:00
Matthias Hanel
03aee09847 [Added] error when mirror/source stream prefix overlaps with stream subs (#2041)
* [Added] error when mirror/source stream prefix overlaps with stream subs

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-27 14:01:51 -04:00
Derek Collison
0f71c260fb Durable consumers with R>1 had performance challenges.
This code changes the way we handle raft based proposals for consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-26 12:53:49 -07:00
Derek Collison
5d5de5925f Introduce a previous leader state in the raft layer to allow quicker responses when leaderless.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-25 17:08:29 -07:00
Derek Collison
e53caee5e8 Enforce server limits even when dynamic limits for accounts in play.
We were not properly enforcing server limits. This commit will allow a server to enforce limits but still remain functional even at the JetStream level.
Also fixed a bug for RAFT replay that could cause instability.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-25 16:06:27 -07:00
Derek Collison
a75e8f8c80 Fix for an issue with multiple restarts that showed stalled and sometimes lost streams.
The issue was when a state was removed from a server and restarted it would catch up properly.
However upon cluster restart the system could exhibit strange behaviors. This was due to on
catchup not properly creating a meta snapshot when one was received, leaving no meta state to recover.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-22 20:06:38 -07:00
Derek Collison
022c5b4ab2 Fix for bad behavior with flow control and multiple sources streams.
Allow chaining of sources and mirrors with filtered consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-22 11:13:51 -07:00
Derek Collison
7284756a8e Update to use Go client
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-21 09:31:17 -07:00
Derek Collison
0f548edcc6 Reduce sliding window for direct consumers and catchup stream windows.
Remove another possible wire blocking operation in raft.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-21 09:24:27 -07:00
Derek Collison
fcd4d0b75f Removed un-needed check in test
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 17:11:58 -07:00
Derek Collison
ced35e5b8c Reworked sources and mirrors on missed data.
Add last delivered sequence to consumer idle heartbeats.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 14:03:46 -07:00
Derek Collison
faa6dc85eb Fix for flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 11:16:40 -07:00
Derek Collison
14a896fee8 Wait longer
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 10:53:12 -07:00
Derek Collison
4c6fd179d6 Can not use sub comparisons for old messages from direct consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 10:30:09 -07:00
Derek Collison
cfe2f448ad Fix for when gateways were dropped and we did not detect interest returning.
Fix for leaked subscription when retrying the source consumers.
Better suppression of old messages from old direct consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 08:44:18 -07:00
Derek Collison
d32f3ce479 Directs may not show up here anymore
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-18 11:29:33 -07:00
Derek Collison
b00b723168 Mirrors were not properly retrying after failures to create their internal consumer.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-18 06:13:26 -07:00
Derek Collison
74cc2b581a When a stream is a mirror or has sources we need to check the upstream streams for filter subject correctness.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-17 06:39:21 -07:00
Derek Collison
4d15658ec2 Merge pull request #1996 from nats-io/updates
JSC Updates
2021-03-14 17:17:08 -07:00
Derek Collison
d4e4c37e94 Test fixes
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-14 06:18:50 -07:00
Derek Collison
6c7d4af2d3 Increase wait time
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-13 18:00:34 -05:00
Derek Collison
76d3a76e14 Skip test since no auto remap atm
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-13 16:53:45 -05:00
Derek Collison
a3a35c0ddb Updated raft processing and dealing with remove peer.
Made sure to not remove us if we were remapped after the peer removal.
Fixed some raft behaviors.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-13 16:28:24 -05:00
Ivan Kozlovic
9e858ff81f Ephemeral cleanup across GWs
Watch for interest loss across GWs so ephemeral consumers are removed
when there is no longer local and GW interest.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-03-13 14:11:34 -07:00
Matthias Hanel
efdb80cc48 More unit test fixes
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-12 01:51:28 -05:00
Matthias Hanel
6a0debbb71 more timeout changes
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-12 00:52:58 -05:00
Matthias Hanel
b316cccfd1 Fixed a quorum formation issue that caused truncation
When a new leader is elected it has to give everyone a chance to reply,
so that we can observe rejections with higher term.

The maximum election timeout is 7.5 seconds.
The new behavior of waiting for the election timeout caused unit tests
to fail. Hence upping the timeout there as well.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-11 19:44:47 -05:00
Ivan Kozlovic
4af86becc7 Merge pull request #1993 from nats-io/remove-peer
Lost quorum changes for streams
2021-03-11 08:20:03 -07:00
Derek Collison
299f44cddf This changes our behaviors for streams and peer removals in several ways.
First we no longer try to auto-remap stream assignments on peer removals from the system.
We also now can always respond to stream info requests if at least a member is running.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-11 06:52:28 -05:00
Matthias Hanel
cab415cf61 Increasing test timeout
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-10 11:52:21 -05:00
Matthias Hanel
32c686d77d Merge pull request #1986 from nats-io/js-test-timeout-3
Increasing test timeout
2021-03-09 20:51:25 -05:00
Matthias Hanel
e554d54b87 Increasing test timeout
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-09 19:44:15 -05:00