Commit Graph

60 Commits

Author SHA1 Message Date
Derek Collison
6c2ccaa30b go vet error
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-11 11:24:35 -08:00
Derek Collison
fa8a95a06a Improved snapshots and compactions.
Various bug fixes and stability improvements.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-11 11:16:00 -08:00
Derek Collison
ad0f10fb5c Allow metaleader stepdown.
Allow easy way to put authorization on system account and still use $G.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-08 15:01:23 -08:00
Derek Collison
c99608fa65 Fix for some test flappers
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-08 07:00:52 -08:00
Derek Collison
594defa11b Fix for crash in test
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-08 06:32:05 -08:00
Derek Collison
15355d783b For larger superclusters we send out our server information after a short delay on startup.
For determing leaderless make sure the raft node has been running for long enough.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-08 06:26:33 -08:00
Derek Collison
c16f6e193d Move JetStream direct APIs to private.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-07 15:19:22 -08:00
Derek Collison
42e3210265 Shutdown JetStream on failed stream msg store in clustered mode
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-07 11:52:52 -08:00
Derek Collison
716da693e4 Track peers differently, react to removal entries
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-06 20:11:30 -08:00
Derek Collison
f500527d12 Make peer reassigment supercluster aware.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-05 07:21:00 -08:00
Derek Collison
493425e8b2 Noticed stream snapshots were being processed on recovery causing high cpu usage and memory usage.
Also made consumer Created() UTC.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-04 12:29:07 -08:00
Derek Collison
74b1e1f72c Change the way we decide to compact on purge op replay
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-04 09:45:51 -08:00
Derek Collison
483b07a794 Make sure old purge did what it was supposed to
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-04 07:49:30 -08:00
Derek Collison
c49e3247bb Purge operations would be replayed on restart regardless if they had already been processed.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-04 07:04:50 -08:00
Ivan Kozlovic
6eeb9d2361 Fixed some issues with JS consumers
- Should check if there is interest at top of deleteNotActive timer
callback.
- Use RLock/RUnlock for no interest check
- Remove some checks in cluster mode regarding internal stream/consumers

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-02-03 20:12:06 -07:00
Derek Collison
01ad02ccae Merge pull request #1883 from nats-io/supercluster
First pass at supercluster enablement.
2021-02-03 19:56:36 -07:00
Derek Collison
a1e0f7dc1a First pass at supercluster enablement.
This allows metacontrollers to span superclusters. Also includes placement directives for streams. By default they select the request origin cluster.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-03 17:28:13 -08:00
Derek Collison
625e6859a3 Only respond for delete and purge if leader and not recovering.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-03 16:36:32 -08:00
Derek Collison
1c9258f91c Handle msg delete and stream purge for R=1
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-03 16:30:03 -08:00
Derek Collison
a8982c040f Suppress lost quorum processing if to close to raft node creation time.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-02 06:27:07 -08:00
Derek Collison
f3703a4b85 Make sure audit events have the proper subject regardless of where processed.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-02 05:03:20 -08:00
Derek Collison
e5c1d65fff Added in JS disable per server on reload. Also removing peerw from a stream and leader stepdown for streams and consumers.
Various bug fixes, stability improvments.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-01 19:39:08 -08:00
Derek Collison
095a83bc2a Make sure to suppress duplicate create/delete audit events
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-31 09:16:16 -08:00
Derek Collison
9b20d5c888 Fixed bug on raft inline cacthup when apply channel was full.
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-30 13:22:27 -08:00
Derek Collison
36c4de874a Merge pull request #1868 from nats-io/js-breaks-imports-fix
[FIXED] Fix for #1864
2021-01-29 19:15:32 -07:00
Derek Collison
302ea010b7 Fix for data race
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-29 18:12:57 -08:00
Derek Collison
e763d90496 Merge pull request #1861 from nats-io/js-list-request-fix
js: consumer list request fix
2021-01-29 19:00:34 -07:00
Derek Collison
457ca3b9cf Suppress additional advisories on server restart and leadership changes.
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-29 15:08:22 -08:00
Derek Collison
3c49f087a0 Merge pull request #1859 from nats-io/jsc_ai
Extended AccountInfo to track API calls and errors.
2021-01-29 10:44:42 -07:00
Derek Collison
d2a92221fb Duplicate leader elect and lost advisories to the system account as well.
Also suppress lost quorums to at most once every 10 secs.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-29 08:51:20 -08:00
Derek Collison
0a3124e27d Track API calls per account. Track success and errors.
These tracking data are ephemeral per server. so on restart they reset.
That should be ok since these will most likely be used more for rates.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-28 17:16:50 -08:00
Waldemar Quevedo
fccdb8caa2 js: consumer list request fix
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-01-28 16:17:01 -08:00
Derek Collison
67dc580a39 Removed stream state for advisories
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-28 09:52:56 -08:00
Derek Collison
8b79114168 Add in advisories for leader elected and quorum lost advisories.
Note that quorum lost only fires if the old leader steps down.
If the leader itself fails and that causes the loss of quorum currently no advisory is sent.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-28 08:37:54 -08:00
Derek Collison
6d27307453 Fix for broken stream restore functionality
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-27 17:44:42 -08:00
Derek Collison
132a4e7f7d Allow memory store for clustering
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-27 15:54:39 -08:00
Derek Collison
a9b8948abe Add in tracking for quorum in raft and do auto stepdown.
Also added in API responses when no leader is present for meta, streams and consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-27 13:34:00 -08:00
Derek Collison
c7c86c7929 Attempt to fix flapper
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-25 10:25:47 -08:00
Derek Collison
7b1e84c086 Fixed raft bug that would cause entries to be missed on restart with leader HB trigger.
Also added in creation times to stream and consumer assignments to make them consistent.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-25 08:47:37 -08:00
Derek Collison
a72ddedb55 Fix for issue with stream info and R=1 and fix for a flapper
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-24 19:48:33 -08:00
Derek Collison
9c858d197a Added ability to properly restore consumers from a snapshot.
This made us add forwarding proposals functionality in the raft layer.
More general cleanup and bug fixes as well.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-24 19:30:34 -08:00
Derek Collison
cad0db2aec Cleanup the consumer assignments when consumers become inactive.
This involved extending our raft implementation to forward proposals to the current leader.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-23 13:44:10 -08:00
Derek Collison
cb2433c2fc Fix for cluster info with R=1
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-23 10:04:43 -08:00
Derek Collison
277483b386 Change over last to active and a duration
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-22 12:34:21 -08:00
Derek Collison
227901a56b More cleanup and stabilization for consumers and failing when sending messages.
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-22 10:09:30 -08:00
Derek Collison
6f2b50a374 Added support for clustered account info and limit enforcement
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-21 18:47:21 -08:00
Derek Collison
3db95f48e3 spelling
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-20 12:46:20 -08:00
Derek Collison
ff54c9dc9c Reworked snapshot and restore.
Underestimated the effort to get stream restore working properly in cluster mode.
Some good bug fixes and stability improvments.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-20 11:58:31 -08:00
Derek Collison
2e9545d587 Make snapshot API cluster aware
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-20 11:58:31 -08:00
Derek Collison
a1730f1b31 Report on RAFT group information.
This adds in optional reporting to stream and consumer info when running in clsutered mode.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-20 11:58:31 -08:00