Commit Graph

649 Commits

Author SHA1 Message Date
Ivan Kozlovic
df9d5f5fd9 Accepting route warns if remote server has same name
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-10-08 17:59:33 -06:00
Derek Collison
cdfb7ba03b Remove delivery count check, could flap
Signed-off-by: Derek Collison <derek@nats.io>
2020-10-02 16:30:40 -07:00
Derek Collison
9ad408e0b3 Use closed variable on consumer, don't check consumer sequences
Signed-off-by: Derek Collison <derek@nats.io>
2020-10-02 16:21:23 -07:00
Derek Collison
8a9f6eaf42 Additional fixes to interest retention based streams and offline durables with redelivery.
Signed-off-by: Derek Collison <derek@nats.io>
2020-10-02 12:58:59 -07:00
Derek Collison
a75be04b0a Various fixes for this PR.
- Fix for updating delivery subject and adjusting next delivery sequences.
- When acking explicitly but out of order, need to make sure we set floor correctly.
- Only update ack floors on an ack if the message is present.
- Fix for needAck for explicitAck out of order consumers detecting if message has been acked.
- Fix for race not locking stream when checking interest during stop.
- Fix for filestore determing if a message block still has a message. Added check to first sequence as well as cache.
- Some additions to the original test.

Signed-off-by: Derek Collison <derek@nats.io>
2020-10-01 21:24:40 -07:00
Ivan Kozlovic
eff27e26be AckExplicit removes message for "offline" durable
The test shows the issue.
It seems that Consumer.needAck() for AckExplicit should consider
that an Ack is needed if sseq > o.asflr and there is no pending
ack at all. However making this change would break the test
TestJetStreamInterestRetentionStream.

Also, running the new test with `-count 100` and by adding an
artificial delay in stream.ackMsg() (just before calling
mset.store.RemoveMsg(seq)) causes sometimes delete requests
for the same sequence to be processed twice, which causes
the new test to fail (even with an attempted fix as discussed
above). I think that the attempt to remove the same sequence twice
is messing up the state.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-10-01 12:22:02 -06:00
Ivan Kozlovic
3977ea04f5 Fixed possible panic on consumer.Delete() during server Shutdown()
The panic was caused by the closing of an already closed Go channel.
The Delete() relied on the consumer's mset being nil to consider
the consumer already closed. However, the consumer's mset is set
to nil after invoking sendDeleteAdvisoryLocked() which internally
invokes sendAdvisory() which releases/reacquires the consumer lock.
This left an open door for a race to occur and Delete() to be
invoked twice on the same consumer.

Moving setting the consumer's mset to nil too early would prevent
the sendAdvisory() to actually do its job. We could pass the mset
to sendAvisory(), but a simpler approach is to simply use a "closed"
boolean on the Consumer object that is set to true at the beginning
of the Delete() function.

Resolves #1621

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-29 10:01:47 -06:00
Derek Collison
7f44d075f7 Merge pull request #1601 from ripienaar/skip_ack_on_acknxt
Do not ack to AckNxt
2020-09-28 17:19:35 -07:00
Ivan Kozlovic
5506b96baa AddConsumer causes redelivery of first message
It seems that when updating the delivery subject, we use as the
first sequence the ack floor, but if no message was ever ack'ed
then it causes the first message to be redelivered twice.

Resolves #1619

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-28 12:28:57 -06:00
Ivan Kozlovic
2605ae71ed [FIXED] Prevent LeafNode loop detection on early reconnect
If the soliciting side detects the disconnect and attempts to
reconnect but the accepting side did not yet close the connection,
a "loop detected" error would be reported and the soliciting server
would not try to reconnect for 30 seconds.

Made a change so that the accepting server checks for existing
leafnode connection for the same server and same account, and if
it is found, close the "old" connection so it is replaced by
the "new" one.

Resolves #1606

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-22 16:58:36 -06:00
R.I.Pienaar
cd376aa753 Do not ack to AckNxt
This causes nil messages mid content stream and is confusing,
the test for this is amended to test the content received for
validity - where previously the test assumed any content is
good content but in fact it only received half the content.

Removing this behaviour until we can design this properly

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-09-21 17:28:53 +02:00
Ivan Kozlovic
7ccbaca782 Added an allowed connection type filter for users
Users and NKey users will now have the option to specify a list
of allowed connection types.

This will allow for instance a certain user to be allowed to
connect as a standard NATS client, but not as Websocket, or
vice-versa.

This also fixes the websocket auth override. Indeed, with
the original behavior, the websocket users would have been bound
to $G, which would not work when there are accounts defined, since
when that is the case, no app can connect/bind to $G account.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-16 18:22:44 -06:00
Waldemar Quevedo
f0c144203b Add Go 1.15 version to CI
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2020-09-09 07:21:32 -07:00
Waldemar Quevedo
26f4971827 Fix all certs for Go 1.15 SAN and CN changes
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2020-09-09 04:43:23 -07:00
Derek Collison
959b35a775 Merge pull request #1581 from nats-io/js_interest_only
JetStream fixes for InterestPolicy retention
2020-09-03 12:19:40 -07:00
Derek Collison
bcbf28fb3a Make sure to clear messages from stream when consumer deleted
Signed-off-by: Derek Collison <derek@nats.io>
2020-09-03 11:41:14 -07:00
Derek Collison
2ba9c97f3d Fix test flapper
Signed-off-by: Derek Collison <derek@nats.io>
2020-09-03 07:54:07 -07:00
Waldemar Quevedo
57f2c74d62 Better support for distinguishedNameMatch in TLS Auth
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2020-09-03 00:52:50 -07:00
Derek Collison
3faa2e7637 Fix for #1580
Signed-off-by: Derek Collison <derek@nats.io>
2020-09-02 17:18:28 -07:00
Derek Collison
cdbea93a87 Interest based retention should not store if no consumers present.
Fix for https://github.com/nats-io/jetstream/issues/314

Signed-off-by: Derek Collison <derek@nats.io>
2020-09-02 16:04:38 -07:00
Matthias Hanel
e1350a05f3 Emit latency traces when sampling is set to headers
Latency reports will include the header(s) responsible for the trace
Updated ADR to have it reflect implementation

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-08-21 20:01:11 -04:00
Derek Collison
3a61a0cef8 Fix for https://github.com/nats-io/jetstream/issues/297
Signed-off-by: Derek Collison <derek@nats.io>
2020-08-17 10:35:07 -07:00
Ivan Kozlovic
22833c8d1a Fix sysSubscribe races
Made changes to processSub() to accept subscription properties,
including the icb callback so that it is set prior to add the
subscription to the account's sublist, which prevent races.
Fixed some other racy conditions, notably in addServiceImportSub()

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-08-03 14:59:00 -06:00
Derek Collison
6e1a892740 Merge pull request #1531 from ripienaar/max_consumers
allow max consumers to be set
2020-07-29 09:09:11 -07:00
R.I.Pienaar
b80e40340e allow account limits to be applied to unlimited streams
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-07-27 19:02:37 +02:00
Matthias Hanel
99921725a9 Ensuring that an untrusted account is not held in memory
The check that an account has to be signed by a configured operator is
done after fetch as well. As a consequence an account claim will never
become an Account in memory.
The original check during client or leaf authentication is left in
place.

Adding unit tests.
Modifying existing tests to not rely on an account but it's name instead.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-07-27 11:59:49 -04:00
Derek Collison
154d4303a9 Add in consumer rate limits
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-24 10:11:32 -07:00
R.I.Pienaar
ca0dc72841 allow max consumers to be set
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-07-24 17:30:41 +02:00
Derek Collison
48b2335cbf Fix for stream delete with many consumers - https://github.com/nats-io/jetstream/issues/260
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-22 07:35:38 -07:00
R.I.Pienaar
5037f00b30 ensure unlimited account limits are calculated correctly
Previously unlimited accounts - ones who inherit server values - would
be unable to publish any messags at all

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-07-20 10:46:34 +02:00
Ivan Kozlovic
9b0967a5d1 [FIXED] Handling of gossiped URLs
If some servers in the cluster have the same connect URLs (due
to the use of client advertise), then it would be possible to
have a server sends the connect_urls INFO update to clients with
missing URLs.

Resolves #1515

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-15 17:39:12 -06:00
Derek Collison
177ca51356 More checks on duplicate window config, signal in PubAck if duplicate
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-10 07:02:34 -07:00
Derek Collison
ad21074177 Merge pull request #1508 from nats-io/eo
Provide exactly once semantics
2020-07-09 16:27:56 -07:00
Derek Collison
2b577b5197 Recover dedupe in memory structures on stream recovery
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-09 16:13:20 -07:00
Derek Collison
b74c2eb2c4 Provide exactly once semantics
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-08 16:56:52 -07:00
Ivan Kozlovic
4ea3f9c57e [FIXED] Handling or real duplicate subscription
That is, if the server receives "SUB foo 1" more than once from
the same client, we would register in the client map this subscription
only once, and add to the account's sublist only once, however we
would have updated shadow subscriptions and route/gateway maps for
each SUB protocol, which would result in inability to send unsubscribe
to routes when the client goes away or unsubscribes.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-08 14:19:32 -06:00
Derek Collison
aecdca874f Pending timers could go negative, this is a fix for #1502
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-06 14:59:05 -07:00
Derek Collison
439e090e0d Updates based on feedback
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-30 18:14:30 -07:00
Derek Collison
06ca580334 Update write deadline, client processing and slow proxy
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-30 16:41:01 -07:00
Derek Collison
920617d64a Updates based on feedback
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-26 10:29:53 -07:00
Derek Collison
6c805eebc7 Properly support leadnode clusters.
Leafnodes that formed clusters were partially supported. This adds proper support for origin cluster, subscription suppression and data message no echo for the origin cluster.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-26 09:03:22 -07:00
Derek Collison
120402241a Fix for #1486
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-18 21:04:34 -07:00
Ivan Kozlovic
53a990f426 Fixed flapper
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-18 16:13:17 -06:00
Derek Collison
98f84bdbc8 Make sure to merge with local deny clauses
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-16 11:56:24 -07:00
Derek Collison
ca4f03c1a6 Properly handle leafnode spoke permissions.
When a leafnode would connect with credentials that had permissions the spoke did not have a way of knowing what those were.
This could lead to being disconnected when sending subscriptions or messages to the hub which were not allowed.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-16 08:33:09 -07:00
Derek Collison
2b9e3e5b15 Merge pull request #1476 from nats-io/cluster_name
Cluster names are now required.
2020-06-15 10:07:30 -07:00
Derek Collison
146d8f5dcb Updates based on feedback, sped up some slow tests
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 17:26:43 -07:00
Ivan Kozlovic
b36672a6bc Fixed flapper
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-12 16:51:40 -06:00
Derek Collison
dd61535e5a Cluster names are now required.
Added cluster names as required for prep work for clustered JetStream. System can dynamically pick a cluster name and settle on one even in large clusters.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 15:48:38 -07:00
Ivan Kozlovic
b9bd5c2d35 Fixed flappers
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-09 15:34:52 -06:00