- When detecting duplicate route, it was possible that a server
would lose track of the peer's gateway URL, which would prevent
it from gossiping that URL to inbound gateway connections
- When a server has gateways enabled and has as a remote its
own gateway, the monitoring endpoint `/varz` would include it
but without the "urls" array.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
When a socket is bound for IP stack protocols with port == 0, the kernel picks
a free port in a specific range and binds it; on Linux, the range can be seen
(and modified) with `sysctl net.ipv4.ip_local_port_range` or looking in
`/proc/sys/net/ipv4/ip_local_port_range`. This defaults to 32768:60999.
When binding explicit ports (for tests), don't use a port number from that
range, or there will be flaky tests as periodically that port will already be
in use from another test.
This renumbers all the JS clustering tests I found binding in that range to be
beneath that range; I checked the code to ensure the new port wasn't already in
use.
* Redact URLs before logging or returning in error
This does not affect strings which failed to parse, and in such a scenario
there's a mix of "which evil" to accept; we can't sanely find what should be
redacted in those cases, so we leave them alone for debugging.
The JWT library returns some errors for Operator URLs, but it rejects URLs
which contain userinfo, so there can't be passwords in those and they're safe.
Fixes#2597
* Test the URL redaction auxiliary functions
* End-to-end tests for secrets in debug/trace
Create internal/testhelper and move DummyLogger there, so it can be used from
the test/ sub-dir too.
Let DummyLogger optionally accumulate all log messages, not just retain the
last-seen message.
Confirm no passwords logged by TestLeafNodeBasicAuthFailover.
Change TestNoPasswordsFromConnectTrace to check all trace messages, not just the
most recent.
Validate existing trace redaction in TestRouteToSelf.
* Test for password in solicited route reconnect debug
There was a bug that would erase the sync subject for upper level catchup for streams.
Raft layer repair was ok but if that was compacted it gets kicked up to the upper layers which would fail.
Users would see "Catchup stalled" messages repeatedly and consumers that had their leaders attached to that replica would also stop working.
Changes were put in to repair the corrupt state after the fact as well, regardless of presence of fix.
Signed-off-by: Derek Collison <derek@nats.io>
When servers leave a cluster and their gateway URLs was not in
the remote cluster's configuration, it is possible that their
gateway URL do not disappear from the list of URLs in the `/varz`
monitoring endpoint.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
We were not escaping the top level iterator across message blocks when calculating when to break due to keep > 0.
Signed-off-by: Derek Collison <derek@nats.io>
Call to mset.unsubscribe() need to use the version that uses
locking when invoked from the subscription callback or from the
go routine when the 10secs have elapsed.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
