Commit Graph

4151 Commits

Author SHA1 Message Date
Derek Collison
e53caee5e8 Enforce server limits even when dynamic limits for accounts in play.
We were not properly enforcing server limits. This commit will allow a server to enforce limits but still remain functional even at the JetStream level.
Also fixed a bug for RAFT replay that could cause instability.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-25 16:06:27 -07:00
Derek Collison
a627db9fc8 Do not request streaminfo from streams that are completely offline.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-24 10:26:09 -07:00
Derek Collison
06803dafbf Tweak seq tracking for flow control, also fixup code
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-24 09:46:54 -07:00
Derek Collison
2ed53035ed Reworked flow control for sources and mirrors.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-24 07:07:33 -07:00
Derek Collison
6f8c0f9193 Fix broken test
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-22 20:31:28 -07:00
Derek Collison
a75e8f8c80 Fix for an issue with multiple restarts that showed stalled and sometimes lost streams.
The issue was when a state was removed from a server and restarted it would catch up properly.
However upon cluster restart the system could exhibit strange behaviors. This was due to on
catchup not properly creating a meta snapshot when one was received, leaving no meta state to recover.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-22 20:06:38 -07:00
Derek Collison
0a48911c00 Merge pull request #2028 from nats-io/RC7
Fix for bad behavior with flow control and multiple sources streams.
2021-03-22 11:20:15 -07:00
Derek Collison
022c5b4ab2 Fix for bad behavior with flow control and multiple sources streams.
Allow chaining of sources and mirrors with filtered consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-22 11:13:51 -07:00
Derek Collison
5c0aecee30 Merge pull request #2027 from nats-io/RC6
Reduce sliding window for direct consumers and catchup stream windows.
2021-03-21 09:32:27 -07:00
Derek Collison
7284756a8e Update to use Go client
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-21 09:31:17 -07:00
Derek Collison
0f548edcc6 Reduce sliding window for direct consumers and catchup stream windows.
Remove another possible wire blocking operation in raft.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-21 09:24:27 -07:00
Derek Collison
a4119e13c1 Fix for new data race detected
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 17:46:59 -07:00
Derek Collison
fcd4d0b75f Removed un-needed check in test
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 17:11:58 -07:00
Derek Collison
0bcda04a9d Send both last consumer and stream sequence for idle hb
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 16:57:13 -07:00
Derek Collison
2d46c112d6 Make sure to not interrupt flow control unless we miss.
Fix deadlock for mirrors.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 15:05:41 -07:00
Derek Collison
25d99931c2 Merge pull request #2026 from nats-io/RC3
Reworked sources and mirrors on missed data.
2021-03-20 14:13:50 -07:00
Derek Collison
ced35e5b8c Reworked sources and mirrors on missed data.
Add last delivered sequence to consumer idle heartbeats.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 14:03:46 -07:00
Derek Collison
ad55fe6b4e Merge pull request #2025 from nats-io/RC2
Can not use sub comparisons for old messages from direct consumers.
2021-03-20 11:17:11 -07:00
Derek Collison
faa6dc85eb Fix for flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 11:16:40 -07:00
Derek Collison
14a896fee8 Wait longer
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 10:53:12 -07:00
Derek Collison
4c6fd179d6 Can not use sub comparisons for old messages from direct consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 10:30:09 -07:00
Derek Collison
ba8d0ef409 Merge pull request #2024 from nats-io/rc1
In operator mode with JetStream we want to load all JS accounts.
2021-03-20 09:30:42 -07:00
Derek Collison
1dde8aed5d Do not send idle heartbeats if not active
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 09:22:06 -07:00
Derek Collison
d9b78b8bc4 Update info status for JetStream when disabling and restarting
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 09:07:47 -07:00
Derek Collison
cfe2f448ad Fix for when gateways were dropped and we did not detect interest returning.
Fix for leaked subscription when retrying the source consumers.
Better suppression of old messages from old direct consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 08:44:18 -07:00
Derek Collison
04a9d51035 Fix for data race
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 07:15:36 -07:00
Derek Collison
61771e88f8 In operator mode with JetStream we want to load accounts that have stable storage.
Also if an account was registered but not JetStream enabled, update it vs error.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 06:53:13 -07:00
Derek Collison
f8b14a078e Merge pull request #2023 from nats-io/jsacc
Make sure the jetstream accounts use the name as the key to the map.
2021-03-18 20:13:59 -07:00
Derek Collison
8eefff2b3b Make sure the jetstream accounts use the name as the key to the map.
This prevents possible double adds under reload or restart scenarios.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-18 17:29:26 -07:00
Derek Collison
ce52ebd9fb Bump to go 1.16
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-18 12:57:20 -07:00
Derek Collison
a2f1977328 Merge pull request #2021 from nats-io/direct
Direct consumer updates.
2021-03-18 11:51:39 -07:00
Derek Collison
4f2b701eb5 Unlock only on return
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-18 11:38:04 -07:00
Derek Collison
d32f3ce479 Directs may not show up here anymore
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-18 11:29:33 -07:00
R.I.Pienaar
52667159e7 Merge pull request #2020 from ripienaar/2019
ensure connect events are always sent
2021-03-18 19:18:02 +01:00
Derek Collison
358ac6a956 Fix for race detector
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-18 11:09:28 -07:00
R.I.Pienaar
14daf39438 ensure connect events are always sent
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-03-18 19:04:36 +01:00
Derek Collison
702e17d48f Direct consumer updates.
1. Reduce maximum sliding window for large N->1.
2. Do not create any store for directs.
3. Relax hard requirement for interest at creation for directs.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-18 10:52:40 -07:00
Derek Collison
4784205c2d Merge pull request #2018 from nats-io/idle_filtered
Fix for filtered consumers and idle heartbeats.
2021-03-18 07:52:51 -07:00
Derek Collison
259cb07284 When a stream was busy but not for a filtered consumer with idle heartbeats they would be suppressed.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-18 07:46:29 -07:00
Derek Collison
aa0c3e12ac Bump version
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-18 06:46:28 -07:00
Derek Collison
b60847c532 Merge pull request #2017 from nats-io/mirror_restart
Mirror consumers were not able to restart after initial failure.
2021-03-18 06:28:04 -07:00
Derek Collison
b00b723168 Mirrors were not properly retrying after failures to create their internal consumer.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-18 06:13:26 -07:00
R.I.Pienaar
11ee71ab8c Merge pull request #2016 from ripienaar/account_failure_logs
improve log line when account registration fails
2021-03-18 13:06:12 +01:00
Derek Collison
b826dd68ae Merge pull request #2015 from nats-io/memp
Updates to memory retention in filestore.
2021-03-18 04:51:14 -07:00
R.I.Pienaar
d6e63cb683 improve log line when account registration fails
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-03-18 12:19:23 +01:00
Derek Collison
566937a880 Updates to memory pressure.
1. Release cache during heavy writes when we move to a new block if no read activity.
2. If we detect a linear scan reading dump the cache on the last seq read for the mb.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-17 18:51:34 -07:00
Derek Collison
601dbf48c1 Protect against the stream nil'ing out the outq for the consumer
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-17 18:51:18 -07:00
R.I.Pienaar
716c6d60b5 Merge pull request #2014 from ripienaar/banner_update
update banner url
2021-03-17 16:50:42 +01:00
R.I.Pienaar
ac70abdcf3 update banner url
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-03-17 16:39:24 +01:00
Derek Collison
20b09480e0 Merge pull request #2013 from nats-io/fsubs
When a stream is a mirror or has sources we need to check the upstream streams.
2021-03-17 06:59:31 -07:00