If the connection is marked as closed while sending the INFO, the
connection would not be removed from the internal map, which would
cause it to be shown in the monitoring list of opened connections.
Resolves#1384
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
This gives the close reason directly in the log without having to
get that information from the monitoring endpoint. Here is an
example of a route closed due to the remote side not replying to
PINGs:
```
[INF] 127.0.0.1:53839 - rid:2 - Router connection closed: Stale Connection
```
Without this change, the log statement would have been:
```
[INF] 127.0.0.1:53839 - rid:2 - Router connection closed
```
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Added a test that demonstrates the issue and a proposed fix.
Also decrement c.out.pb if closing due to max pending limit.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
On connect message resend reset timer with setFirstPingTimer, so RTT can
be obtained quicker.
Disable short first ping in default server options for client_test.
In log_test prevent immediate scheduling by setting ping interval.
Signed-off-by: Matthias Hanel <mh@synadia.com>
When the server logs information related to a connection, it uses
the connection IP and remote port as a prefix. When it was an IPv6
address, the square brackets would be missing.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
This related to PR #1233.
The computation of the time to stall a fast producer was bogus. Fixed
that and added a unit test for the function computing this stalled
duration.
Also, in PR #1233, I had removed Gosched() when a call to flushOutbound()
realizes that the flag is already set. It was forgetting that readLoop
in some cases will call flushOutbound() in place. So there is still
value in unlock/gosched/lock again in that function.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
- All writes will now be done by the writeLoop, unless when the
writeLoop has not been started yet (likely in connection init).
- Slow consumers for non CLIENT connections will be reported but
not failed. The idea is that routes, gateway, etc.. connections
should stay connected as much as possible. However if a flush
operation times out and no data at all has been written, the
connection will be closed (regardless of type).
- Slow consumers due to max pending is only for CLIENT connections.
This allows sending of SUBs through routes, etc.. to not have
to be chunked.
- The backpressure to CLIENT connections is increased (up to 1sec)
based on the sub's connection pending bytes level.
- Connection is flushed on close from the writeLoop as to not block
the "fast path".
Some tests have been fixed and adapted since now closeConnection()
is not flushing/closing/removing connection in place.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Updated all tests that use "async" clients.
- start the writeLoop (this is in preparation for changes in the
server that will not do send-in-place for some protocols, such
as PING, etc..)
- Added missing defers in several tests
- fixed an issue in client.go where test was wrong possibly causing
a panic.
- Had to skip a test for now since it would fail without server code
change.
The next step will be ensure that all protocols are sent through
the writeLoop and that the data is properly flushed on close (important
for -ERR for instance).
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
This could happen if the remote server is running but not dequeueing
from the socket. TLS connection Close() may send/read and so we
need to protect with a deadline.
For non client/leaf connection, do not call flushOutbound().
Set the write deadline regardless of handshakeComplete flag, and
set it to a low value.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
- Add atomic in client to skip check in processInboundClientMsg()
if value is 0. Avoids getting the lock in fast path if not needed.
- Have a timer per client instead of the global server list that
was expiring: noticed a lot of contention there when running
some perf/profiling tests. The timer is also not reset for
every timestamp that is not yet expired since this too affects
performance. Instead fires are regular interval and cleared
when map is empty after a cycle.
- Move processing of gw map rely on its own function (in inbound msg).
I have verified that this is inlined same way as when code was
directly in processInboundClientMsg.
- Use string(subj[]) for prefix detection: I have verified that
it is actually faster.
- Builds the RMSG with appends to local buffer in handleGatewayReply()
instead of using fmt.Sprintf().
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
- New prefix that includes origin server for the request
- Mapping done if request is service import or requestor has
recent subscription
- Subscription considered recent if less than 250ms
- Destination server strip GW prefix before giving to client
and restore when getting a reply on that subject
- Mapping removed aftert 250ms
- Server rejects client publish on "$GNR." (the new prefix)
- Cluster and server hash are now 8 chars long and from base 62
alphabets
- Mapped replies need to be sent to leafnode servers due to race
(cluster B sends RS+ on GW inbound then RMSG on outbound, the
RS+ may be processed later and cluster A may have given message
to LN before RS+ on reply subject. So LN needs to accept the
mapped reply but will strip to give to client and reassemble
before sending it back)
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
As soon as server has processed a client CONNECT, it was possible
that if Connz() or other was requested, the server will send a
PING to compute the RTT. This would cause clients that expect
the first PONG as part of synchronous CONNECT logic to fail.
Make sure that we delay the first RTT ping to after sending the
first PONG, or if client does not send PING as part of the CONNECT,
after 2 seconds have elapsed since the tcp connection was accepted.
Resolves#1174
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
- Ensure that defaults are set when values are 0
- Fixed some tests
- Added some helpers in jwt tests to reduce copy/paste
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Changed some of client.outbound fields to int64.
Moved fields around to minimize size of struct (checked with
unsafe.Sizeof())
Checked benchmark results before/after
Added test
Resolves#1118
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Take into account tracking of response maps that are created and do proper cleanup.
Also fixes#1089 which was discovered while working on this.
Signed-off-by: Derek Collison <derek@nats.io>
Issue a warning in readLoop if execution of code after connection
Read() until end of for loop reaches a certain threshold.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
This adds a RemoteAddress() method to the ClientAuthentication
interface which would facilitate CustomClientAuthentication
mechanisms considering the remote IP address of the client in
their decisions to allow clients or not
Resolves#832
Signed-off-by: R.I.Pienaar <rip@devco.net>
Add in trusted keys options and binary stamp
User JWT and Account fetch with AccountResolver
Account and User expiration
Account Imports/Exports w/ updates
Import activation expiration
Signed-off-by: Derek Collison <derek@nats.io>
Some tests consume too much memory when running with -race which
can cause some failures on Travis.
Moreover, those tests may not be meaningful if they are running
slow, which -race causes.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
I noticed that when running the test suite, there would be a file
server/log1.txt left. This file is created by one of the config
reload test. Running this test individually was doing the proper
cleanup. I noticed that the Signal test that was checking
that files could be rotated was causing this side effect.
It turns out that none of the config reload tests were disabling
the signal handler (NoSigs=true), and since the go routine would
be left running, running the TestSignalToReOpenLogFile() test
would interact with an already finished test.
I put a thread dump in handleSignals() to track all tests that
were causing this function to start the go routine because NoSigs
was not set to true. I fixed all those tests. At this time, there
are only 2 tests that need to start the signal handler.
I have also fixed the code so that the signal handler routine select
on a server quitCh that is closed on shutdown so that this go routine
exit and is waiting on using the grWG wait group.
The issue was that a subject such as `foo.bar,*,>` would be
inserted to the cache as is, but when trying to remove from the
cache, calling matchLiteral() with the above subject in the cache
against the same subject would return false. This is because
matchLiteral would treat those characters as wildcards token.
Note that the sublist itself splits subjects on the `.` separator
and seem not bothered by such subject (would have `foo` and `bar,*,>`
tokens). Also, note that IsValidSubject() and IsValidLiteralSubject()
properly checked that the characters `*` and `>` are treated
as wildcards only if they are tokens on their own.
Resolves#558
