Commit Graph

639 Commits

Author SHA1 Message Date
Ivan Kozlovic
2605ae71ed [FIXED] Prevent LeafNode loop detection on early reconnect
If the soliciting side detects the disconnect and attempts to
reconnect but the accepting side did not yet close the connection,
a "loop detected" error would be reported and the soliciting server
would not try to reconnect for 30 seconds.

Made a change so that the accepting server checks for existing
leafnode connection for the same server and same account, and if
it is found, close the "old" connection so it is replaced by
the "new" one.

Resolves #1606

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-22 16:58:36 -06:00
Ivan Kozlovic
7ccbaca782 Added an allowed connection type filter for users
Users and NKey users will now have the option to specify a list
of allowed connection types.

This will allow for instance a certain user to be allowed to
connect as a standard NATS client, but not as Websocket, or
vice-versa.

This also fixes the websocket auth override. Indeed, with
the original behavior, the websocket users would have been bound
to $G, which would not work when there are accounts defined, since
when that is the case, no app can connect/bind to $G account.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-16 18:22:44 -06:00
Waldemar Quevedo
f0c144203b Add Go 1.15 version to CI
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2020-09-09 07:21:32 -07:00
Waldemar Quevedo
26f4971827 Fix all certs for Go 1.15 SAN and CN changes
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2020-09-09 04:43:23 -07:00
Derek Collison
959b35a775 Merge pull request #1581 from nats-io/js_interest_only
JetStream fixes for InterestPolicy retention
2020-09-03 12:19:40 -07:00
Derek Collison
bcbf28fb3a Make sure to clear messages from stream when consumer deleted
Signed-off-by: Derek Collison <derek@nats.io>
2020-09-03 11:41:14 -07:00
Derek Collison
2ba9c97f3d Fix test flapper
Signed-off-by: Derek Collison <derek@nats.io>
2020-09-03 07:54:07 -07:00
Waldemar Quevedo
57f2c74d62 Better support for distinguishedNameMatch in TLS Auth
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2020-09-03 00:52:50 -07:00
Derek Collison
3faa2e7637 Fix for #1580
Signed-off-by: Derek Collison <derek@nats.io>
2020-09-02 17:18:28 -07:00
Derek Collison
cdbea93a87 Interest based retention should not store if no consumers present.
Fix for https://github.com/nats-io/jetstream/issues/314

Signed-off-by: Derek Collison <derek@nats.io>
2020-09-02 16:04:38 -07:00
Matthias Hanel
e1350a05f3 Emit latency traces when sampling is set to headers
Latency reports will include the header(s) responsible for the trace
Updated ADR to have it reflect implementation

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-08-21 20:01:11 -04:00
Derek Collison
3a61a0cef8 Fix for https://github.com/nats-io/jetstream/issues/297
Signed-off-by: Derek Collison <derek@nats.io>
2020-08-17 10:35:07 -07:00
Ivan Kozlovic
22833c8d1a Fix sysSubscribe races
Made changes to processSub() to accept subscription properties,
including the icb callback so that it is set prior to add the
subscription to the account's sublist, which prevent races.
Fixed some other racy conditions, notably in addServiceImportSub()

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-08-03 14:59:00 -06:00
Derek Collison
6e1a892740 Merge pull request #1531 from ripienaar/max_consumers
allow max consumers to be set
2020-07-29 09:09:11 -07:00
R.I.Pienaar
b80e40340e allow account limits to be applied to unlimited streams
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-07-27 19:02:37 +02:00
Matthias Hanel
99921725a9 Ensuring that an untrusted account is not held in memory
The check that an account has to be signed by a configured operator is
done after fetch as well. As a consequence an account claim will never
become an Account in memory.
The original check during client or leaf authentication is left in
place.

Adding unit tests.
Modifying existing tests to not rely on an account but it's name instead.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-07-27 11:59:49 -04:00
Derek Collison
154d4303a9 Add in consumer rate limits
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-24 10:11:32 -07:00
R.I.Pienaar
ca0dc72841 allow max consumers to be set
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-07-24 17:30:41 +02:00
Derek Collison
48b2335cbf Fix for stream delete with many consumers - https://github.com/nats-io/jetstream/issues/260
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-22 07:35:38 -07:00
R.I.Pienaar
5037f00b30 ensure unlimited account limits are calculated correctly
Previously unlimited accounts - ones who inherit server values - would
be unable to publish any messags at all

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-07-20 10:46:34 +02:00
Ivan Kozlovic
9b0967a5d1 [FIXED] Handling of gossiped URLs
If some servers in the cluster have the same connect URLs (due
to the use of client advertise), then it would be possible to
have a server sends the connect_urls INFO update to clients with
missing URLs.

Resolves #1515

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-15 17:39:12 -06:00
Derek Collison
177ca51356 More checks on duplicate window config, signal in PubAck if duplicate
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-10 07:02:34 -07:00
Derek Collison
ad21074177 Merge pull request #1508 from nats-io/eo
Provide exactly once semantics
2020-07-09 16:27:56 -07:00
Derek Collison
2b577b5197 Recover dedupe in memory structures on stream recovery
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-09 16:13:20 -07:00
Derek Collison
b74c2eb2c4 Provide exactly once semantics
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-08 16:56:52 -07:00
Ivan Kozlovic
4ea3f9c57e [FIXED] Handling or real duplicate subscription
That is, if the server receives "SUB foo 1" more than once from
the same client, we would register in the client map this subscription
only once, and add to the account's sublist only once, however we
would have updated shadow subscriptions and route/gateway maps for
each SUB protocol, which would result in inability to send unsubscribe
to routes when the client goes away or unsubscribes.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-08 14:19:32 -06:00
Derek Collison
aecdca874f Pending timers could go negative, this is a fix for #1502
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-06 14:59:05 -07:00
Derek Collison
439e090e0d Updates based on feedback
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-30 18:14:30 -07:00
Derek Collison
06ca580334 Update write deadline, client processing and slow proxy
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-30 16:41:01 -07:00
Derek Collison
920617d64a Updates based on feedback
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-26 10:29:53 -07:00
Derek Collison
6c805eebc7 Properly support leadnode clusters.
Leafnodes that formed clusters were partially supported. This adds proper support for origin cluster, subscription suppression and data message no echo for the origin cluster.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-26 09:03:22 -07:00
Derek Collison
120402241a Fix for #1486
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-18 21:04:34 -07:00
Ivan Kozlovic
53a990f426 Fixed flapper
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-18 16:13:17 -06:00
Derek Collison
98f84bdbc8 Make sure to merge with local deny clauses
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-16 11:56:24 -07:00
Derek Collison
ca4f03c1a6 Properly handle leafnode spoke permissions.
When a leafnode would connect with credentials that had permissions the spoke did not have a way of knowing what those were.
This could lead to being disconnected when sending subscriptions or messages to the hub which were not allowed.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-16 08:33:09 -07:00
Derek Collison
2b9e3e5b15 Merge pull request #1476 from nats-io/cluster_name
Cluster names are now required.
2020-06-15 10:07:30 -07:00
Derek Collison
146d8f5dcb Updates based on feedback, sped up some slow tests
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 17:26:43 -07:00
Ivan Kozlovic
b36672a6bc Fixed flapper
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-12 16:51:40 -06:00
Derek Collison
dd61535e5a Cluster names are now required.
Added cluster names as required for prep work for clustered JetStream. System can dynamically pick a cluster name and settle on one even in large clusters.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 15:48:38 -07:00
Ivan Kozlovic
b9bd5c2d35 Fixed flappers
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-09 15:34:52 -06:00
Derek Collison
4dee03b587 Allow mixed TLS and non-TLS on same port
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-05 18:04:11 -07:00
Ivan Kozlovic
25bd5ca352 [FIXED] Unsubscribe may not be propagated through a leaf node
There is a race between the time the processing of a subscription
and the init/send of subscriptions when accepting a leaf node
connection that may cause internally a subscription's subject
to be counted many times, which would then prevent the send of
an LS- when the subscription's interest goes away.

Imagine this sequence of events, each side represents a "thread"
of execution:
```
client readLoop                         leaf node readLoop
----------------------------------------------------------
recv SUB foo 1
sub added to account's sublist

                                         recv CONNECT
                                     auth, added to acc.

updateSmap
smap["foo"]++ -> 1
no LS+ because !allSubsSent

                                         init smap
                                    finds sub in acc sl
                                    smap["foo"]++ -> 2
                                        sends LS+ foo
                                    allSubsSent == true

recv UNSUB 1
updateSmap
smap["foo"]-- -> 1
no LS- because count != 0
----------------------------------------------------------
```
Equivalent result but with slightly diffent execution:
```
client readLoop                         leaf node readLoop
----------------------------------------------------------
recv SUB foo 1
sub added to account's sublist

                                         recv CONNECT
                                     auth, added to acc.

                                         init smap
                                    finds sub in acc sl
                                    smap["foo"]++ -> 1
                                        sends LS+ foo
                                    allSubsSent == true

updateSmap
smap["foo"]++ -> 2
no LS+ because count != 1

recv UNSUB 1
updateSmap
smap["foo"]-- -> 1
no LS- because count != 0
----------------------------------------------------------
```

The approach for the fix is delay the creation of the smap
until we actually initialize the map and send the subs on processing
of the CONNECT.
In the meantime, as soon as the LN connection is registered
and available in updateSmap, we check that smap is nil or
not. If nil, we do nothing.

In "init smap" we keep track of the subscriptions that have been
added to smap. This map will be short lived, just enough to
protect against races above.

In updateSmap, when smap is not nil, we need to checki, if we
are adding, that the subscription has not already been handled.
The tempory subscription map will be ultimately emptied/set to
nil with the use of a timer (if not emptied in place when
processing smap updates).

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-05 10:07:15 -06:00
Derek Collison
c969e7e424 Do proper ubsubscribe when shutting off restore endpoint
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-04 08:58:14 -07:00
Derek Collison
164f44ed18 Require reply subjects for restore chunks
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-04 06:56:07 -07:00
Derek Collison
660ea3c807 Snapshot restore now works across leafnodes.
This also introduces the ability to have flow control inbound for restoring a stream.
If the system detects a reply subject it will respond with a nil payload.
For the last EOF message if a reply is present it will respond with a stream info response or error.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-03 20:00:59 -07:00
Ivan Kozlovic
1e149f4041 Merge pull request #1440 from nats-io/jwt2
Update imports for jwt/v2
2020-06-02 11:10:21 -06:00
Derek Collison
afc7fc367b Remove hdrs for now, find better way to deliver in client
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-02 07:10:23 -07:00
R.I.Pienaar
920dd4269a fix argument order in snapshots
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-06-02 13:51:50 +02:00
Derek Collison
07ef71ff98 Avoid parsing large sizes for messages
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-01 16:54:41 -07:00
aricart
e7590f3065 jwt2 testbed 2020-06-01 18:00:13 -04:00