Commit Graph

2501 Commits

Author SHA1 Message Date
Derek Collison
0b3c686430 Fixes for data races and some locking.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-05 17:19:51 -08:00
Derek Collison
ff98984358 Reworked internal consumers used for sources and mirrors.
Now use ephemerals and heartbeats and flowcontrol.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-05 14:32:57 -08:00
Derek Collison
dd8acb1a99 Fixed a bug where we were not determing clustered state so were straight processing msgs from routes.
Cleaned up lseq and clseq code.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-05 12:00:19 -08:00
Derek Collison
9813ae3f79 Most tooling, Docker and Kubernetes send SIGTERM, so do graceful shutdown on the signal.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-05 09:21:01 -08:00
Derek Collison
507eb25dc1 Fix for not placing checksum back properly on manual check restore
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-05 09:18:20 -08:00
Derek Collison
75b0455e0b Enable opt-in flow control for push based consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-05 08:48:12 -08:00
Derek Collison
955594ce9d Use code 100 instead
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-05 05:57:50 -08:00
Derek Collison
7e2b2a1033 Allow an option to push based consumers to have idle heartbeats delivered.
This allows an endpoint to know the consumer is still alive.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-05 05:48:00 -08:00
Derek Collison
4bfd6485f6 Can't remove based on interest directly
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 19:11:24 -08:00
Derek Collison
7b1b9a7946 Snapshot on peer state change, e.g. removal
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 18:52:57 -08:00
Derek Collison
038a5cadc4 LQ is longer now
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 18:39:32 -08:00
Derek Collison
4cdfb0ab6e Don't need to release locks now with outq. Also borrow once
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 17:58:01 -08:00
Derek Collison
207ebd3b3d Changed stream sendq to linked list outq.
Made consumer share streams outq.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 17:19:50 -08:00
Derek Collison
e70e46ea4a Updates based on PR feedback
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 16:16:28 -08:00
Derek Collison
051d0159ef rc5
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 14:46:59 -08:00
Derek Collison
bfb8e3432e Move RAFT comms off internal sendq.
Move route and gateway msgs our of fast path for inbound stream msgs.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 14:45:34 -08:00
Derek Collison
e523e8f8de Fix based on feedback
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 09:37:09 -08:00
Derek Collison
051a5c4095 Tweaks to flaky test
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 21:11:14 -08:00
Derek Collison
d7201a110b Better handling on out of disk.
Suppress some stream and consumer bad results since they delete the asset.
Allow rehup to re-enable JetStream.
Various bug fixes and improvements.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 20:12:10 -08:00
Derek Collison
18ee2b7a40 Changes to handle short writes. Bug fixes to truncation.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 20:07:39 -08:00
Derek Collison
86d12b903d Fixed leaking info sub
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 20:06:35 -08:00
Ivan Kozlovic
e7e756034a Switch Gateway JS accounts to interest-only mode + some other fixes
- Fixed the close of a TLS connection which starting Go 1.16
set the deadline to 5 seconds.

- Fixed an issue with setHeader that was causing these error messages
```
=== RUN   TestServiceImportReplyMatchCycleMultiHops
nats: message could not decode headers on connection [4] for subscription on "foo"
--- PASS: TestServiceImportReplyMatchCycleMultiHops (0.04s)
```

- Fixed names of tests in norace_test.go since they must start with
TestNoRace in order to make sure that we execute them in Travis:
```
go test -v -run=TestNoRace --failfast -p=1 ./...
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-03-03 19:15:28 -07:00
Derek Collison
f82ba22be2 Merge pull request #1960 from nats-io/update-sys-acc-js
[Fixed] missing export jsAllApi on system account update
2021-03-03 16:06:35 -07:00
Matthias Hanel
2187c0d2ae [Fixed] missing export jsAllApi on system account update
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-03 16:01:22 -05:00
Ivan Kozlovic
0f53bf6580 Fixed data race with nodeInfo
Took the approach of storing struct instead of pointer. Of course,
when changing the offline bool from false to true, it means that
we need to call Store again (with same key).

This is based on the assumption that those Load/Store are not too
frequent. Otherwise, we may need to use locking (and keep *nodeInfo)

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-03-03 13:28:45 -07:00
Derek Collison
66cbf75712 Merge branch 'master' into consumer 2021-03-03 10:29:39 -07:00
Derek Collison
32ffb55a22 Provide feedback to users when trying to pull from a push based consumer.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 09:17:27 -08:00
Derek Collison
2ecf6be3ef Mark raft node as offline when server is removed
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 08:06:45 -08:00
Derek Collison
d170647c04 More time to make sure mirror consumer has hooked up properly
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 07:58:45 -08:00
Derek Collison
5da343eed8 More time for consumer delivered state to propagate
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 07:43:03 -08:00
Derek Collison
b7cf49949e For large redeliver queues linear search was not good, so added index.
Also set to nil when drained to avoind holding onto large underlying arrays.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 07:29:59 -08:00
R.I.Pienaar
236498a142 restore meta peer remove by name rather than id
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-03-03 15:55:50 +01:00
Derek Collison
e4d458c3f8 Make consumer processing of next message not inline if non client connection.
Also fixed a bug with processing ack floors.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 06:52:56 -08:00
Derek Collison
af6b5d856a Better errors, no need to wait on leader twice
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 06:45:27 -08:00
Derek Collison
e8029094e3 Cleanup export and import processing
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 06:35:52 -08:00
Derek Collison
401484299d Flaps with cluster size of 5 too much
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 06:34:07 -08:00
Derek Collison
c6b0ed6069 Flapping test
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 05:06:24 -08:00
Derek Collison
00a49ce329 Test for existing service import
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 20:38:16 -08:00
Derek Collison
e1f7440361 Revert to previous, will redo out of band version
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 20:18:21 -08:00
Derek Collison
43b9017b74 Merge pull request #1953 from nats-io/api
JetStream API Changes
2021-03-02 19:46:00 -07:00
Matthias Hanel
25ef6b0f0d Merge pull request #1952 from nats-io/goland-lint
Fixed linter issues
2021-03-02 21:43:04 -05:00
Derek Collison
b9498b8c0e Need to return
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 18:38:05 -08:00
Matthias Hanel
c50ee2a1c6 [Changed] all times exposed will be computed in UTC (#1943)
This also applies to times that end up in that json.
Where applicable moved time.Now() to where it is used.
Moved calls to .UTC() to where time is created it that time is converted
later anyway.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-02 21:37:42 -05:00
Derek Collison
4583afc2b0 Updated comment
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 18:36:28 -08:00
Derek Collison
d084853aff Actually need to wait to accumulate, make sure we get at least 8.
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 18:33:22 -08:00
Derek Collison
df02e0be71 Consumers would process next message batches inline.
In clustered mode this could block a route or gateway processor.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 18:28:17 -08:00
Derek Collison
04634d3358 One less api call
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 18:28:06 -08:00
Derek Collison
479176d26c Adjustments for change in actual JSApi subs
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 18:08:08 -08:00
Derek Collison
c39a2d6bd0 If capturing calls with no dispatch just return
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 17:58:34 -08:00
Derek Collison
4f7fbefc7c In clustered JetStream we need to move API calls out of routes/gateways/leafnodes path.
This moves from explicit imports and subscriptions to one wildcard subscription and a single wildcard export.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 17:54:41 -08:00