This PR backports the OCSP Peer feature option (as in 2.10 train) and
includes two fixes for the existing OCSP Staple feature.
OCSP Staple:
1. Fixed and clarified how NATS Server determines its own Issuer CA when
obtaining and validating an OCSP Response for subsequent staple
2. Eliminated problematic assumption that all node peers are issued by
same CA when NATS Server validates ROUTE and GATEWAY peer nodes
3. Added OCSP Response effectivity checks on ROUTE and GATEWAY
peer-presented staple
Note for #3: Allowed host clock skew between node peers set at
30-seconds. If the OCSP Response contains an empty assertion for
NextUpdate, NATS Server will default to 1-hour validity (after
ThisUpdate). It is recommended that CA OCSP Responder should assert
NextUpdate.
When service imports were reloaded on active accounts with lots of
traffic the server could panic or lose data.
Signed-off-by: Derek Collison <derek@nats.io>
Added a leafnode lock to allow better traversal without copying of large leafnodes in a single hub account.
Signed-off-by: Derek Collison <derek@nats.io>
One should not access s.opts directly but instead use s.getOpts().
Also, server lock needs to be released when performing an account
lookup (since this may result in server lock being acquired).
A function was calling s.LookupAccount under the client lock, which
technically creates a lock inversion situation.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
- Save the TLS name only if not already set
- Use the passed URLs slice instead of using s.getOpts().Routes
- Enhanced the test
- Fixed an unrelated DATA RACE report
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
The server was not setting "server name" in the TLS configuration
for route connections, which may lead to failed (re)connect if
the certificate does not allow for the IP and the URL did not
have the hostname, which would happen with gossip protocol.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
If an ephemeral was given a name by the user, if the consumer leader
was then shutdown, the ephemeral would be migrated using a server
generated new name instead of keeping the user given name.
Also, in some cases the migration would not even occur. This was
likely due to the fact that RAFT node(s) were shutdown prior to
the ephemeral migration code was invoked.
Resolves#3550
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
A request to `$SYS.REQ.SERVER.PING.JSZ` would now return something
like this:
```
...
"meta_cluster": {
"name": "local",
"leader": "A",
"peer": "NUmM6cRx",
"replicas": [
{
"name": "B",
"current": true,
"active": 690369000,
"peer": "b2oh2L6w"
},
{
"name": "Server name unknown at this time (peerID: jZ6RvVRH)",
"current": false,
"offline": true,
"active": 0,
"peer": "jZ6RvVRH"
}
],
"cluster_size": 3
}
```
Note the "peer" field following the "leader" field that contains
the server name. The new field is the node ID, which is a hash of
the server name.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
The original value was hardcoded to 128MB and 32MB per stream. The
per-server limit is lowered to 32MB but is configurable with
a new configuration parameter:
```
jetstream {
max_catchup: 8MB
}
```
The per-stream limit was also lowered from 32MB/128,000msgs to
8MB/32,000 messages.
Tests have shown no difference in performance for fast links.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Add basic peer certificates information in /connz endpoint when
the "auth" option is provided.
Resolves#3317
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
We are phasing out the optimistic-only mode. Servers accepting
inbound gateway connections will switch the accounts to interest-only
mode.
The servers with outbound gateway connection will check interest
and ignore the "optimistic" mode if it is known that the corresponding
inbound is going to switch the account to interest-only. This is
done using a boolean in the gateway INFO protocol.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
- didRemove in applyMetaEntries() could be reset when processing
multiple entries
- change "no race" test names to include JetStream
- separate raft nodes leader stepdown and stop in server
shutdown process
- in InstallSnapshot, call wal.Compact() with lastIndex+1
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Maybe that is the place it could be set and not in NewServer(), but
want to minimize risk of breaking something close to 2.9.0
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
A test TestJetStreamClusterLeafNodeSPOFMigrateLeaders was added at
some point that needed the remotes to stop (re)connecting. It made
use of existing leafNodeEnabled that was used for GW/Leaf interest
propagation races to disable the reconnect, but that may not be
the best approach since it could affect users embedding servers
and adding leafnodes "dynamically".
So this PR introduced a specific boolean specific for that test.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Added http monitoring endpoint /accstatz
It responds with a list of statz for all accounts with local connections
the argument "unused=1" can be provided to get statz for all accounts
This endpoint is also exposed as nats request under:
This monitoring endpoint is exposed via the system account.
$SYS.REQ.ACCOUNT.*.STATZ
Each server will respond with connection statistics for the requested
account. The format of the data section is a list (size 1) identical to the event
$SYS.ACCOUNT.%s.SERVER.CONNS which is sent periodically as well as on
connect/disconnect. Unless requested by options, server without the account,
or server where the account has no local connections, will not respond.
A PING endpoint exists as well. The response format is identical to
$SYS.REQ.ACCOUNT.*.STATZ
(however the data section will contain more than one account, if they exist)
In addition to general filter options the request takes a list of accounts and
an argument to include accounts without local connections (disabled by default)
$SYS.REQ.ACCOUNT.PING.STATZ
Each account has a new system account import where the local subject
$SYS.REQ.ACCOUNT.PING.STATZ essentially responds as if
the importing account name was used for $SYS.REQ.ACCOUNT.*.STATZ
The only difference between requesting ACCOUNT.PING.STATZ from within
the system account and an account is that the later can only retrieve
statz for the account the client requests from.
Also exposed the monitoring /healthz via the system account under
$SYS.REQ.SERVER.*.HEALTHZ
$SYS.REQ.SERVER.PING.HEALTHZ
No dedicated options are available for these.
HEALTHZ also accept general filter options.
Signed-off-by: Matthias Hanel <mh@synadia.com>
The "InProcess" change make readyForConnections() possibly return
much faster than it used to, which could cause tests to fail.
Restore the original behavior, but in case of DontListen option
wait on the startupComplete gate.
Also fixed some missing checks for leafnode connections.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
