Commit Graph

111 Commits

Author SHA1 Message Date
Waldemar Quevedo
93fd0f60f5 reload: Allow re-enabling JS after it was disabled
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-03-12 17:19:34 -08:00
Waldemar Quevedo
d338e9ae30 Disable JS on restart if on invalid state
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-03-10 01:40:32 -08:00
Matthias Hanel
bea37b089c fixing crash when disabling already shut down client
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-08 19:57:18 -05:00
Ivan Kozlovic
57af977548 Stabilized stream sources under restart
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-03-08 16:41:02 -07:00
Derek Collison
5fe9ba5e94 Switched to fast state
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-06 09:00:15 -08:00
Derek Collison
0b3c686430 Fixes for data races and some locking.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-05 17:19:51 -08:00
Derek Collison
ff98984358 Reworked internal consumers used for sources and mirrors.
Now use ephemerals and heartbeats and flowcontrol.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-05 14:32:57 -08:00
Derek Collison
d7201a110b Better handling on out of disk.
Suppress some stream and consumer bad results since they delete the asset.
Allow rehup to re-enable JetStream.
Various bug fixes and improvements.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 20:12:10 -08:00
Ivan Kozlovic
e7e756034a Switch Gateway JS accounts to interest-only mode + some other fixes
- Fixed the close of a TLS connection which starting Go 1.16
set the deadline to 5 seconds.

- Fixed an issue with setHeader that was causing these error messages
```
=== RUN   TestServiceImportReplyMatchCycleMultiHops
nats: message could not decode headers on connection [4] for subscription on "foo"
--- PASS: TestServiceImportReplyMatchCycleMultiHops (0.04s)
```

- Fixed names of tests in norace_test.go since they must start with
TestNoRace in order to make sure that we execute them in Travis:
```
go test -v -run=TestNoRace --failfast -p=1 ./...
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-03-03 19:15:28 -07:00
Derek Collison
e8029094e3 Cleanup export and import processing
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 06:35:52 -08:00
Derek Collison
00a49ce329 Test for existing service import
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 20:38:16 -08:00
Derek Collison
4f7fbefc7c In clustered JetStream we need to move API calls out of routes/gateways/leafnodes path.
This moves from explicit imports and subscriptions to one wildcard subscription and a single wildcard export.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 17:54:41 -08:00
Derek Collison
d4a8baf030 Stabilize test, wait a bit for migration
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 05:28:55 -08:00
Derek Collison
22749fb1e2 Fix data race, stabilize usage test
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-01 16:47:19 -08:00
Derek Collison
c8e2ad6248 Make sure trailing usage updates accounted for eventually
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-01 14:23:06 -08:00
Derek Collison
84d042c4e2 Make sending account usage updates more efficient
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-01 04:42:50 -08:00
Derek Collison
df77724aa4 Make ephemeral consumers R=1 and provide optimistic migration on peer removal or server shutdown.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 16:50:25 -08:00
R.I.Pienaar
a4817bd7b6 extend the out of space advisory
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-02-26 11:10:05 +01:00
Derek Collison
e5c8774172 Handle out of space situations, general stability enhancements
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-25 17:54:29 -08:00
Derek Collison
78bdc34637 General stability improvements. Fixes to subscription state not cleaning up.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 08:44:34 -08:00
Derek Collison
bb58d455f6 Revert switching to interest only mode
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-23 18:00:47 -08:00
Derek Collison
09e3d26fa3 Add in support for stream mirrors and sources.
Add in proper support for stream updates in clustered mode.
Don't send API updates without subjects, caused GW parser errors.
Stream internal loops use their own clients now.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-23 10:57:27 -08:00
Matthias Hanel
2e4a892c8d [fixes] crash in checkJetStreamExports when there is no system account
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-02-23 00:42:31 -05:00
Derek Collison
d803189eca Changes needed to properly support multi-layer service imports for system services like JS.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-12 17:01:32 -08:00
Derek Collison
579737a5e1 General fixes, stability improvements
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-11 18:13:24 -08:00
Derek Collison
c16f6e193d Move JetStream direct APIs to private.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-07 15:19:22 -08:00
Derek Collison
c9e1cbd571 Modified jetstream varz information
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-02 08:35:35 -08:00
Derek Collison
e5c1d65fff Added in JS disable per server on reload. Also removing peerw from a stream and leader stepdown for streams and consumers.
Various bug fixes, stability improvments.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-01 19:39:08 -08:00
Derek Collison
095a83bc2a Make sure to suppress duplicate create/delete audit events
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-31 09:16:16 -08:00
Derek Collison
8bd290c77a Fix for #1864.
When trying to make sure we properly created all subs for service imports we would check the internal client to see if we should process.
With JS enabled on the server we would place system imports that would break that check and orphan other service imports.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-29 17:51:14 -08:00
Derek Collison
c889321a83 Change to API.Total and API.Errors
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-29 09:42:20 -08:00
Derek Collison
0a3124e27d Track API calls per account. Track success and errors.
These tracking data are ephemeral per server. so on restart they reset.
That should be ok since these will most likely be used more for rates.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-28 17:16:50 -08:00
Derek Collison
6f2b50a374 Added support for clustered account info and limit enforcement
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-21 18:47:21 -08:00
Derek Collison
55d750733b Fix based on feedback from Ivan
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-20 11:58:31 -08:00
Derek Collison
78747b2414 Stability improvements around startup and restore.
We were incorrectly starting clustering before enabling accounts and restoring state.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-17 13:44:49 -08:00
Derek Collison
b606dceb59 Stabilize restart/catchup for raft.
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-16 05:47:48 -08:00
Derek Collison
4b84decc7f Fix for race
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-14 12:59:12 -08:00
Derek Collison
b68d7066c4 Remove alpha banner
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-14 06:05:55 -08:00
Derek Collison
4bfe9d4c24 Fixes to PR.
Add nats to default storage directory
Fix race in raft, change leader notice
Fix test crash on failure

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-14 05:56:05 -08:00
Derek Collison
f0cdf89c61 JetStream Clustering WIP
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-14 01:14:52 -08:00
Derek Collison
bfb726e8e9 Make sure to clear JS resources on reload
Signed-off-by: Derek Collison <derek@nats.io>
2020-11-30 17:18:33 -08:00
Derek Collison
7e27042e6e Fix for #1736
When a system account was configured and not the default when we did a reload we would lose the JetStream service exports.

Signed-off-by: Derek Collison <derek@nats.io>
2020-11-30 16:11:50 -08:00
Derek Collison
44a1373f89 JetStream changes.
Made several changes based on feedback.

1. Made PubAckResponse only optionally include an ApiError and not force an API type.
2. Allow FilterSubject to be set on a consumer config and cleared if it matches the only stream subject.
3. Remove LookupStream by subject, and add in filters for stream names API.

Signed-off-by: Derek Collison <derek@nats.io>
2020-11-25 06:50:25 -08:00
Derek Collison
5ff28b6087 Allow stream lookup by subject.
Allow an API endpoint and public API to lookup a stream by subject. The subject needs to be an exact match or a subset. If the subject is considered a filtered subject for the stream that will also be returned.

Signed-off-by: Derek Collison <derek@nats.io>
2020-11-14 13:35:05 -08:00
Derek Collison
164f9fdf2b Updates to consumer store to support delta updates.
In preparation for clustering we need to have the consumer filestore update state with deltas vs original design.

Signed-off-by: Derek Collison <derek@nats.io>
2020-11-10 19:16:55 -08:00
Derek Collison
1f8ec6c964 Separate out disk available for Windows
Signed-off-by: Derek Collison <derek@nats.io>
2020-10-31 12:56:27 -07:00
Derek Collison
fe2b354414 Stability and performance updates.
The original design had a shared filestore write buffer and individual message blocks had a read cache.
This presented some performance and stability issues when lots of reads and deletes were happening to a
message block that was also being written to actively.

This change eliminates the shared write buffer and uses the message block's cache as a write through as
well as read cache and handles partials correctly.

Signed-off-by: Derek Collison <derek@nats.io>
2020-10-28 07:58:47 -07:00
Matthias Hanel
a3c5fd4575 Updating jetstream account settings from jwt
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-09-04 16:07:38 -04:00
Ivan Kozlovic
22833c8d1a Fix sysSubscribe races
Made changes to processSub() to accept subscription properties,
including the icb callback so that it is set prior to add the
subscription to the account's sublist, which prevent races.
Fixed some other racy conditions, notably in addServiceImportSub()

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-08-03 14:59:00 -06:00
R.I.Pienaar
b80e40340e allow account limits to be applied to unlimited streams
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-07-27 19:02:37 +02:00