user and activation token did not honor the jwt value for all * on
connect.
activation token where not re evaluated when the export revoked a key.
In part this is a consistency measure so servers that already have an
account and servers that don't behave the same way.
in jwt activation token revocations are stored per export.
The server stored them per account, thus effectively merging
revocations. Now they are stored per export inside the server too.
fixes nats-io/nsc/issues/442
Signed-off-by: Matthias Hanel <mh@synadia.com>
This is due to a re-entrant RLock(). It works sometimes, but if there
is a go routine requesting the write lock, then the second RLock()
will not be granted which will lead to a deadlock.
In summary: one should never make re-entrant RLock calls.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
When we want to track service import response interest across a leafnode we need to send sub and unsub for all response _R_ subjects versus using a wildcard.
Signed-off-by: Derek Collison <derek@nats.io>
The bug occures when latency tracking is on, a requestor and responder are not connected to the same server, and the responder sends two responses for a single request.
Signed-off-by: Derek Collison <derek@nats.io>
* Redact URLs before logging or returning in error
This does not affect strings which failed to parse, and in such a scenario
there's a mix of "which evil" to accept; we can't sanely find what should be
redacted in those cases, so we leave them alone for debugging.
The JWT library returns some errors for Operator URLs, but it rejects URLs
which contain userinfo, so there can't be passwords in those and they're safe.
Fixes#2597
* Test the URL redaction auxiliary functions
* End-to-end tests for secrets in debug/trace
Create internal/testhelper and move DummyLogger there, so it can be used from
the test/ sub-dir too.
Let DummyLogger optionally accumulate all log messages, not just retain the
last-seen message.
Confirm no passwords logged by TestLeafNodeBasicAuthFailover.
Change TestNoPasswordsFromConnectTrace to check all trace messages, not just the
most recent.
Validate existing trace redaction in TestRouteToSelf.
* Test for password in solicited route reconnect debug
Added in client kind and sub type for clients.
Added in ability to filter connections based on matching subject interest.
Signed-off-by: Derek Collison <derek@nats.io>
When processing service imports we would swap out the accounts during processing.
With the addition of internal subscriptions and internal clients publishing in JetStream we had an issue with the wrong account being used.
This was specific to delyaed pull subscribers trying to unsubscribe due to max of 1 while other JetStream API calls were running concurrently.
1. We were holding open FDs longer than we should for consumers causing issues with open FD limits. We now do not hold them open and cap updates a bit better.
2. When doing a stream delete, consumer delete was repeating alot of work that was not necessary, causing longer delays. This has been optimized a bit, still more improvements to be made.
3. We cover all JS under a single export, but that was also trapping GetNext for pull based consumers, and since this was a no-op (is handled at user account level) we were creating alot of garbage service import responses and reverse map entries that had to be garbage collected. We have a fix in to avoind this but still looking for a better one.
4. Still had some lingering references to all exports vs single JS export.
Signed-off-by: Derek Collison <derek@nats.io>
* [fixing] leafnode missing retry and service export interest propagation
A missing account on initial connect attempt caused the leaf node
connection to never be established.
An account service import subscription was not propagated along leaf
node connections.
Signed-off-by: Matthias Hanel <mh@synadia.com>
* [fixed] hanging leaf node connection when account can't be found
as a result of the issue, the leaf node connection never got created,
even after the account can be found.
Also tracing account id and name (when available)
Signed-off-by: Matthias Hanel <mh@synadia.com>
fixed by moving setting of the mappings into a common function that is
also called when the jwt is updated
Signed-off-by: Matthias Hanel <mh@synadia.com>
This CL adds in support for multiple JetStream domains using mapped subjects.
Mapping subjects aligns well with the JetStream context APIPrefix in clients.
Signed-off-by: Derek Collison <derek@nats.io>
Some issues that have been fixed would manifest by timeouts on
connect, unexpected memory usage on high publish message rate.
Some details:
- Replies were not always GW routed properly because we were looking
at the wrong connection's rsubs
- GW routed replies would not be found because they were tracked
in the subscription's client object, which may not be the same used
to send the reply
- Increased the mqtt timeout to wait for JS replies since in some
tests it was sometimes taking more than the original 2 seconds
- Incoming gateway messages destined for an MQTT internal subscription
may have been rejected as a no interest if the account had service imports
- Don't use time.After(), instead create explicit timer so it can
be stopped when not timing out.
- Unnecessary copy of a slice since we were converting to a string anyway.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
This also applies to times that end up in that json.
Where applicable moved time.Now() to where it is used.
Moved calls to .UTC() to where time is created it that time is converted
later anyway.
Signed-off-by: Matthias Hanel <mh@synadia.com>
This moves from explicit imports and subscriptions to one wildcard subscription and a single wildcard export.
Signed-off-by: Derek Collison <derek@nats.io>
This change does 4 things:
Refactor to only have one function to validate imports.
Have this function support the jwt field account_token_position.
For completeness make this value configurable as well.
unit tests.
Signed-off-by: Matthias Hanel <mh@synadia.com>
When trying to make sure we properly created all subs for service imports we would check the internal client to see if we should process.
With JS enabled on the server we would place system imports that would break that check and orphan other service imports.
Signed-off-by: Derek Collison <derek@nats.io>
This will cause the server to not trust accounts/user signed by an
identity key
The boot strapping system account will assume the account is issued by
the operator.
If this is not desirable, the system account can be provided right away
as resolver_preload.
[fixes] crash when the system account uses signing keys and an update changes that key set.
Signed-off-by: Matthias Hanel <mh@synadia.com>
Since we were creating subs on the fly, sub.im would always be nil.
We were passing a client because it was needed in sendRouteSubOrUnSubProtos().
This PR simply fills the buffer with each account's subscriptions.
There is also no need to have subs sent from different go routine
based on some threshold. Routes are no longer subject to max pending.
Some code has been made into a function so that they can be shared
by sendSubsToRoute() and sendRouteSubOrUnSubProtos(). The function
is simply adding to given buffer the RS+/- protocol.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
