In some cases, the reply of a request message is prefixed when
going over a gateway so that if it comes back to a different
server than when the request originates, it can be routed back.
For system accounts, this routed reply subject was not tracked
so the server would reply to the inbox and may reach a server
that had not yet processed (through the route) the interest
on that inbox. If the reply came with the GW routed info, that
server would know to route it to the original server.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
There was a test to prevent an errorneous loop detection when a
remote would reconnect (due to a stale connection) while the accepting
side did not detect the bad connection yet.
However, this test was racy because the test was done prior to add
the connections to the map.
In the case of a misconfiguration where the remote creates 2 different
remote connections that end-up binding to the same account in the
accepting side, then it was possible that this would not be detected.
And when it was, the remote side would be unaware since the disconnect/
reconnect attempts would not show up if not running in debug mode.
This change makes sure that the detection is no longer racy and returns
an error to the remote so at least the log/console of the remote will
show the "duplicate connection" error messages.
Resolves#1730
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Suppose a cluster of 2 servers, let's call them leaf1 and leaf2.
These servers are routed and have a leaf connection to another
server, let's call it srv1.
They share the same cluster name.
If a queue subscriber runs on srv1 and a queue subscriber on the
same subject/group name runs on leaf1, if a requestor runs on
leaf2, the request should reach only one of the 2 queue subs.
The defect was that sometimes both queue subs would receive the
message.
The added test checks that only one reply is ever received and
that the local "leaf" cluster is preferred.
Resolves#1722
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Connections normally suppress sending PINGs if there was some
activity. We now force GATEWAY connections to send PINGs at the
configured interval or 15 seconds, whichever is the smallest.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Addresses stack overflow issue wally was seeing with configs
that mix and match streams and services between each other.
Signed-off-by: Derek Collison <derek@nats.io>
The mappings enable traffic shaping functionality.
Also added a subject transform which can match any subject and transform to a new one, even re-ordering partial wildcards.
Signed-off-by: Derek Collison <derek@nats.io>
When creating shadow subscriptions for import streams, we were
not invoking code for gateway subscription accounting, which means
that when the account (for leafnodes) was switched to interest
only, those shadow subscriptions were not sent.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Users and NKey users will now have the option to specify a list
of allowed connection types.
This will allow for instance a certain user to be allowed to
connect as a standard NATS client, but not as Websocket, or
vice-versa.
This also fixes the websocket auth override. Indeed, with
the original behavior, the websocket users would have been bound
to $G, which would not work when there are accounts defined, since
when that is the case, no app can connect/bind to $G account.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
We previously simply called DialTimeout() on a route's url when
soliciting. If it resolved to the IP of the host, it would create
a route to self, which server detects, but then would not try again
with other IPs that would have allowed to form a cluster with
other servers running on the other IPs.
This PR keeps track of local IPs + cluster port and exclude them
from the list of IPs returned by LookupHost API. This even prevent
solicitation of routes to self. Only non-local IPs will be tried.
Resolves#1586
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Latency reports will include the header(s) responsible for the trace
Updated ADR to have it reflect implementation
Signed-off-by: Matthias Hanel <mh@synadia.com>
because times stored are hh:mm:ss it is possible to end up with start > end where end is actually the next day.
jwt.go line 189
Also, ranges are based on the servers location, not the clients.
Signed-off-by: Matthias Hanel <mh@synadia.com>
Made changes to processSub() to accept subscription properties,
including the icb callback so that it is set prior to add the
subscription to the account's sublist, which prevent races.
Fixed some other racy conditions, notably in addServiceImportSub()
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
This change allows the removal of the connection and update of
the server state to be done "in place" but still delay the flushing
of and close of tcp connection to the writeLoop. With ref counting
we ensure that the reconnect happens after the flushing but not
before the state has been updated.
Had to fix some places where we may have called closeConnection()
from under the server lock since it now would deadlock for sure.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
The call sendProtoNow() should not normally be used (only when
setting up a connection when the writeloop is not yet started and
server needs to send something before being able to start the
writeLoop.
Instead, code should use enqueueProto(). For this particular
case though, use queueOutbound() directly and add to the
producer's pcd map.
Also fixed other places where we were using queueOutbound() +
flushSignal() which is what enqueueProto is doing.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
That is, if the server receives "SUB foo 1" more than once from
the same client, we would register in the client map this subscription
only once, and add to the account's sublist only once, however we
would have updated shadow subscriptions and route/gateway maps for
each SUB protocol, which would result in inability to send unsubscribe
to routes when the client goes away or unsubscribes.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
If an IPv6 address contains some "%" characters, this was causing
the connection name in log statement to mess up the Sprintf formatting.
The solution is to escape those "%" characters.
Resolves#1505
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Leafnodes that formed clusters were partially supported. This adds proper support for origin cluster, subscription suppression and data message no echo for the origin cluster.
Signed-off-by: Derek Collison <derek@nats.io>
When a leafnode would connect with credentials that had permissions the spoke did not have a way of knowing what those were.
This could lead to being disconnected when sending subscriptions or messages to the hub which were not allowed.
Signed-off-by: Derek Collison <derek@nats.io>
