This is similar to PR #4115 but for LeafNodes.
Compression mode can be set on both side (the accept and in remotes).
```
leafnodes {
port: 7422
compression: s2_best
remotes [
{
url: "nats://host2:74222"
compression: s2_better
}
]
}
```
Possible modes are similar than for routes (described in PR #4115),
except that when not defined we default to `s2_auto`.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
- [X] Tests added
- [X] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
- [X] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)
### Changes proposed in this pull request:
There is currently a blanket requirement that subject transforms
destinations MUST use ALL of the partial wildcards defined in the source
of the transform. This is because the subject transformed defined for
imports must be 'reversible' and therefore the destination transform
must use all of the partial wildcard tokens defined in the source of the
transform.
This reversing of a transform is only used for transforms used by
imports, where in any case it doesn't make sense to use any transform
other than Wildcard.
This PR:
- relaxes this requirement to only apply when the transform is used by
an import, adding the ability to drop a wildcard token in transforms
other than as part of an import.
- Improves transform reverse to support both legacy style wildcards $X
and the new transform function {{Wildcard(X)}}- Improves reversible
transform checking to only allow the use of wildcards in the
destination.
---------
Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
The new field `compression` in the `cluster{}` block allows to
specify which compression mode to use between servers.
It can be simply specified as a boolean or a string for the
simple modes, or as an object for the "s2_auto" mode where
a list of RTT thresholds can be specified.
By default, if no compression field is specified, the server
will use the s2_auto mode with default RTT thresholds of
10ms, 50ms and 100ms for the "uncompressed", "fast", "better"
and "best" modes.
```
cluster {
..
# Possible values are "disabled", "off", "enabled", "on",
# "accept", "s2_fast", "s2_better", "s2_best" or "s2_auto"
compression: s2_fast
}
```
To specify a different list of thresholds for the s2_auto,
here is how it would look like:
```
cluster {
..
compression: {
mode: s2_auto
# This means that for RTT up to 5ms (included), then
# the compression level will be "uncompressed", then
# from 5ms+ to 15ms, the mode will switch to "s2_fast",
# then from 15ms+ to 50ms, the level will switch to
# "s2_better", and anything above 50ms will result
# in the "s2_best" compression mode.
rtt_thresholds: [5ms, 15ms, 50ms]
}
}
```
Note that the "accept" mode means that a server will accept
compression from a remote and switch to that same compression
mode, but will otherwise not initiate compression. That is,
if 2 servers are configured with "accept", then compression
will actually be "off". If one of the server had say s2_fast
then they would both use this mode.
If a server has compression mode set (other than "off") but
connects to an older server, there will be no compression between
those 2 routes.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
New configuration fields:
```
cluster {
...
pool_size: 5
accounts: ["A", "B"]
}
```
The configuration `pool_size` in the example above means that this
server will create 5 routes to a remote server, assuming that that
server has the same `pool_size` setting.
Accounts (which are not part of the `accounts[]` configuration)
are assigned a specific route in this pool, and this will be the
same route on all servers in the cluster.
Accounts that are defined in the `accounts` field will each have
a dedicated route connection. This will allow suppression of the
account name in some of the route protocols, reducing bytes transmitted
which may increase performance.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
The server would reset its INFO's TLSRequired to the presence
of a TLS configuration without checking for the allow_non_tls
option.
Resolves#3581
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Code change:
- Do not start the processMirrorMsgs and processSourceMsgs go routine
if the server has been detected to be shutdown. This would otherwise
leave some go routine running at the end of some tests.
- Pass the fch and qch to the consumerFileStore's flushLoop otherwise
in some tests this routine could be left running.
Tests changes:
- Added missing defer NATS connection close
- Added missing defer server shutdown
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
This was an oversight when automating: we use the `req -x509` approach to
create the CA cert, which didn't pick up the defaults set for other stuff and
so defaulted to 30 days.
Abstract out CERT_DURATION and use for both the cert defaults and the CA's own
lifetime.
We are phasing out the optimistic-only mode. Servers accepting
inbound gateway connections will switch the accounts to interest-only
mode.
The servers with outbound gateway connection will check interest
and ignore the "optimistic" mode if it is known that the corresponding
inbound is going to switch the account to interest-only. This is
done using a boolean in the gateway INFO protocol.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
We had some certs which used SHA1 still; this top-level script is a bit rough,
but works to recreate all the top-level certs, using a common CA.
RSA 2048 bits with SHA256.
I've put the IPv6 localhost IP into the certs too, wherever the IPv4 localhost
IP was present, as this might improve test-runner resilience in some
situations. Seems like a good idea, can't think of a reason not to.
Added http monitoring endpoint /accstatz
It responds with a list of statz for all accounts with local connections
the argument "unused=1" can be provided to get statz for all accounts
This endpoint is also exposed as nats request under:
This monitoring endpoint is exposed via the system account.
$SYS.REQ.ACCOUNT.*.STATZ
Each server will respond with connection statistics for the requested
account. The format of the data section is a list (size 1) identical to the event
$SYS.ACCOUNT.%s.SERVER.CONNS which is sent periodically as well as on
connect/disconnect. Unless requested by options, server without the account,
or server where the account has no local connections, will not respond.
A PING endpoint exists as well. The response format is identical to
$SYS.REQ.ACCOUNT.*.STATZ
(however the data section will contain more than one account, if they exist)
In addition to general filter options the request takes a list of accounts and
an argument to include accounts without local connections (disabled by default)
$SYS.REQ.ACCOUNT.PING.STATZ
Each account has a new system account import where the local subject
$SYS.REQ.ACCOUNT.PING.STATZ essentially responds as if
the importing account name was used for $SYS.REQ.ACCOUNT.*.STATZ
The only difference between requesting ACCOUNT.PING.STATZ from within
the system account and an account is that the later can only retrieve
statz for the account the client requests from.
Also exposed the monitoring /healthz via the system account under
$SYS.REQ.SERVER.*.HEALTHZ
$SYS.REQ.SERVER.PING.HEALTHZ
No dedicated options are available for these.
HEALTHZ also accept general filter options.
Signed-off-by: Matthias Hanel <mh@synadia.com>
- Remove code coverage from Travis and add it to a GitHub Action
that will be run as a nightly.
- Use tag builds to exclude some tests, such as the "norace" or
JS tests. Since "go test" does not support "negative" regexs, there
is no other way.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
When a configuration reload is done, the account's leaf node connections
were not transfered to the new instance of the account, causing the
interest to not be propagated until a leafnode reconnect or a server
restart.
Resolves#3009
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
When using subscriptions through import/exports, the server with
a leafnode connection would properly send the interest over, but
if the connection is recreated, this would not happen.
In case of JetStream where that happens under the cover, message
flow would stop after the leafnode restart because the subscriptions
would be created on recovery of the JetStream assets but *before*
the LeafNode connection could be established.
Resolves#3024Resolves#3027Resolves#3009
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
If set, a server configured to accept leafnode connections will
reject a remote server whose version is below that value. Note
that servers prior to v2.8.0 are not sending their version
in the CONNECT protocol, which means that anything below 2.8.0
would be rejected.
Configuration example:
```
leafnodes {
port: 7422
min_version: 2.8.0
}
```
The option is a string and can have the "v" prefix:
```
min_version: "v2.9.1"
```
Note that although suffix such as `-beta` would be accepted,
only the major, minor and update are used for the version comparison.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Also had to change all references from `path.` to `filepath.` when
dealing with files, so that it works properly on Windows.
Fixed also lots of tests to defer the shutdown of the server
after the removal of the storage, and fixed some config files
directories to use the single quote `'` to surround the file path,
again to work on Windows.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
The established ordering is client -> Account, so fixed few places
where we had Account -> client.
Added a new file, locksordering.txt with the list of known ordering
for some of the objects.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
When a gateway connection was created (either accepted or initiated)
the timer to fire the first PING was started at that time, which
means that for an outbound connection, if the INFO coming from
the other side was delayed, it was possible for the outbound to
send the PING protocol before the CONNECT, which would cause
the accepting side to close the connection due to a "parse" error
(since the CONNECT for an inbound is supposed to be the very
first protocol).
Also noticed that we were not setting the auth timer like we do
for the other type of connections. If authorization{timeout:<n>}
is not set, the default is 2 seconds.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
For TLS configuration with `verify_and_map` set to true, if a
connection connects and has a certificate with ID that matches
a user, but that user's `allowed_connection_types` is specified
and does not have the connection type in its list, then the
server will panic.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Currently, OCSP responses that aren't signed by the root CA will fail
with a verification error. This change allows intermediates, or
designated responders, to sign responses on behalf of the CA.
- Add Go 1.17
- Fix go fmt from Go 1.17 (build directives)
- Download version of misspell and staticcheck instead of doing
"go get" since current staticcheck would be broken without go.mod
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
The bug occures when latency tracking is on, a requestor and responder are not connected to the same server, and the responder sends two responses for a single request.
Signed-off-by: Derek Collison <derek@nats.io>
* Redact URLs before logging or returning in error
This does not affect strings which failed to parse, and in such a scenario
there's a mix of "which evil" to accept; we can't sanely find what should be
redacted in those cases, so we leave them alone for debugging.
The JWT library returns some errors for Operator URLs, but it rejects URLs
which contain userinfo, so there can't be passwords in those and they're safe.
Fixes#2597
* Test the URL redaction auxiliary functions
* End-to-end tests for secrets in debug/trace
Create internal/testhelper and move DummyLogger there, so it can be used from
the test/ sub-dir too.
Let DummyLogger optionally accumulate all log messages, not just retain the
last-seen message.
Confirm no passwords logged by TestLeafNodeBasicAuthFailover.
Change TestNoPasswordsFromConnectTrace to check all trace messages, not just the
most recent.
Validate existing trace redaction in TestRouteToSelf.
* Test for password in solicited route reconnect debug
