Added cluster names as required for prep work for clustered JetStream. System can dynamically pick a cluster name and settle on one even in large clusters.
Signed-off-by: Derek Collison <derek@nats.io>
This was found due to a recent test that was flapping. The test
was not checking the correct server for leafnode connection, but
that uncovered the following bug:
When a leafnode connection is solicited, the read/write loops are
started. Then, the connection lock is released and several
functions invoked to register the connection with an account and
add to the connection leafs map.
The problem is that the readloop (for instance) could get a read
error and close the connection *before* the above said code
executes, which would lead to a connection incorrectly registered.
This could be fixed either by delaying the start of read/write loops
after the registration is done, or like in this PR, check the
connection close status after registration, and if closed, manually
undoing the registration with account/leafs map.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
The test was not checking that PINGs were sent, however, without
disabling the first short ping and with a very low interval, there
was chance on Travis that the first short ping was sent before
the client had connected, which would break the NATS client protocol
that expects to receive a PONG to initial PING (after CONNECT).
The client library could arguably be updated to accept PING during
the CONNECT phase.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
We added authentication override block for websocket configuration
in PR #1463 and #1465 which somehow introduced a drop in perf as
reported by the bench tests.
This PR refactors a bit to restore the performance numbers.
This change also fixes the override behavior for websocket auth:
- If websocket's NoAuthUser is configured, the websocket's auth
block MUST define Users, and the user be present.
- If there is any override (username/pwd,token,etc..) then the
whole block config will be used when authenticating a websocket
client, which means that if websocket NoAuthUser is empty we
are not falling back to the regular client's NoAuthUser config.
- TLSMap always override the regular client's config. That is,
whatever TLSMap value specified in the websocket's tls{} block
will be used.
The TLSMap configuration was not used for LeafNodes. The behavior
now will be:
- If LeafNode's auth block contains users and TLSMap is true,
the user is looked up based on the cert's info. If not found,
authentication will fail. If found, it will be authenticated
and bound to associated account.
- If no user is specified in LeafNode's auth block and TLSMap
is true, then the cert's info will be used against the global
users map.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Building server's nkeys and users map out of slices form options
has been made a function so it can be used for the server and
websocket (and in future for mqtt)
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Websocket can now override
- Username/password
- Token
- Users
- NKeys
- no_auth_user
- auth_timeout
For TLS, support for verify and verify_and_map. We used to set
tls config's ClientAuth to NoClientCert. It will now depend
if the config requires client certificate verification, which
is needed if TLSMap is enabled.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
The grace period used to be hardcoded at 10 seconds.
This option allows the user to configure the amount of time the
server will wait before initiating the closing of client connections.
Note that the grace period needs to be strictly lower than the overall
lame_duck_duration. The server deducts the grace period from that
overall duration and spreads the closing of connections during
that time.
For instance, if there are 1000 connections and the lame duck
duration is set to 30 seconds and grace period to 10, then
the server will use 30-10 = 20 seconds to spread the closing
of those 1000 connections, so say roughly 50 clients per second.
Resolves#1459.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
There is a race between the time the processing of a subscription
and the init/send of subscriptions when accepting a leaf node
connection that may cause internally a subscription's subject
to be counted many times, which would then prevent the send of
an LS- when the subscription's interest goes away.
Imagine this sequence of events, each side represents a "thread"
of execution:
```
client readLoop leaf node readLoop
----------------------------------------------------------
recv SUB foo 1
sub added to account's sublist
recv CONNECT
auth, added to acc.
updateSmap
smap["foo"]++ -> 1
no LS+ because !allSubsSent
init smap
finds sub in acc sl
smap["foo"]++ -> 2
sends LS+ foo
allSubsSent == true
recv UNSUB 1
updateSmap
smap["foo"]-- -> 1
no LS- because count != 0
----------------------------------------------------------
```
Equivalent result but with slightly diffent execution:
```
client readLoop leaf node readLoop
----------------------------------------------------------
recv SUB foo 1
sub added to account's sublist
recv CONNECT
auth, added to acc.
init smap
finds sub in acc sl
smap["foo"]++ -> 1
sends LS+ foo
allSubsSent == true
updateSmap
smap["foo"]++ -> 2
no LS+ because count != 1
recv UNSUB 1
updateSmap
smap["foo"]-- -> 1
no LS- because count != 0
----------------------------------------------------------
```
The approach for the fix is delay the creation of the smap
until we actually initialize the map and send the subs on processing
of the CONNECT.
In the meantime, as soon as the LN connection is registered
and available in updateSmap, we check that smap is nil or
not. If nil, we do nothing.
In "init smap" we keep track of the subscriptions that have been
added to smap. This map will be short lived, just enough to
protect against races above.
In updateSmap, when smap is not nil, we need to checki, if we
are adding, that the subscription has not already been handled.
The tempory subscription map will be ultimately emptied/set to
nil with the use of a timer (if not emptied in place when
processing smap updates).
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
