Commit Graph

469 Commits

Author SHA1 Message Date
Phil Pennock
cceeb94166 WIP: add best guess prominent socket stats logging 2020-10-13 19:53:45 -04:00
Phil Pennock
331f6ca3aa WIP: theoretically, populate expvar maps 2020-10-13 19:53:35 -04:00
Phil Pennock
65be9706b3 WIP: socket stats
At this point, we're collecting for gateways, we have the general framework in
place, and we're populating unpublished expvars.
2020-10-13 18:26:28 -04:00
Ivan Kozlovic
12d84c646c Merge pull request #1535 from harrisa1/improveLogging
[CHANGED] add client provided info into server side client logs when available
2020-09-23 14:53:06 -06:00
Andrew Harris
40cacf5e5f updates from PR 2020-09-23 14:02:03 -04:00
Andrew Harris
b5197cfc47 updates from PR 2020-09-22 09:05:50 -04:00
Ivan Kozlovic
e84f1cf52c Added test with unknown JWT allowed connection type
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-17 18:17:19 -06:00
Ivan Kozlovic
7ccbaca782 Added an allowed connection type filter for users
Users and NKey users will now have the option to specify a list
of allowed connection types.

This will allow for instance a certain user to be allowed to
connect as a standard NATS client, but not as Websocket, or
vice-versa.

This also fixes the websocket auth override. Indeed, with
the original behavior, the websocket users would have been bound
to $G, which would not work when there are accounts defined, since
when that is the case, no app can connect/bind to $G account.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-16 18:22:44 -06:00
Ivan Kozlovic
2ad2bed170 [ADDED] Support for route hostname resolution
We previously simply called DialTimeout() on a route's url when
soliciting. If it resolved to the IP of the host, it would create
a route to self, which server detects, but then would not try again
with other IPs that would have allowed to form a cluster with
other servers running on the other IPs.

This PR keeps track of local IPs + cluster port and exclude them
from the list of IPs returned by LookupHost API. This even prevent
solicitation of routes to self. Only non-local IPs will be tried.

Resolves #1586

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-08 13:40:17 -06:00
Matthias Hanel
cc14e54179 Discard client provided jwt when not in operator mode
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-08-28 12:31:02 -04:00
Matthias Hanel
9d1526cbb8 Adding user jwt payload and subscriber limits
Addresses part of #1552

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-08-24 18:16:25 -04:00
Matthias Hanel
e1350a05f3 Emit latency traces when sampling is set to headers
Latency reports will include the header(s) responsible for the trace
Updated ADR to have it reflect implementation

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-08-21 20:01:11 -04:00
Matthias Hanel
6c61464915 [ADDED] Checks for CIDR blocks and connect time ranges specified in jwt (#1567)
because times stored are hh:mm:ss it is possible to end up with start > end where end is actually the next day.
jwt.go line 189

Also, ranges are based on the servers location, not the clients.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-08-20 13:28:16 -06:00
Ivan Kozlovic
22833c8d1a Fix sysSubscribe races
Made changes to processSub() to accept subscription properties,
including the icb callback so that it is set prior to add the
subscription to the account's sublist, which prevent races.
Fixed some other racy conditions, notably in addServiceImportSub()

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-08-03 14:59:00 -06:00
Ivan Kozlovic
c620175353 Rework closeConnection()
This change allows the removal of the connection and update of
the server state to be done "in place" but still delay the flushing
of and close of tcp connection to the writeLoop. With ref counting
we ensure that the reconnect happens after the flushing but not
before the state has been updated.

Had to fix some places where we may have called closeConnection()
from under the server lock since it now would deadlock for sure.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-31 15:30:17 -06:00
Matthias Hanel
b759282407 Removing gateway as this condition won't be happening
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-07-31 16:26:33 -04:00
Matthias Hanel
e089fae7fb Deliver to all non remote queue subscriber instead of kind == CLIENT
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-07-31 15:18:26 -04:00
Andrew Harris
2113cbddf3 missed the comment that was automatically changed 2020-07-24 16:55:50 -04:00
Andrew Harris
cb000a889c sometimes IDEs are stupid 2020-07-24 16:54:40 -04:00
Andrew Harris
17930baa18 add client provided info into server side client logs when available 2020-07-24 15:40:39 -04:00
Ivan Kozlovic
dbf962f958 [FIXED] Queue subscriptions not able to receive system events.
The issue was caused by the kind of connections being checked.

Resolves #1527

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-23 11:38:43 -06:00
Ivan Kozlovic
4d495104de Fixed no_responders use of sendProtoNow()
The call sendProtoNow() should not normally be used (only when
setting up a connection when the writeloop is not yet started and
server needs to send something before being able to start the
writeLoop.

Instead, code should use enqueueProto(). For this particular
case though, use queueOutbound() directly and add to the
producer's pcd map.

Also fixed other places where we were using queueOutbound() +
flushSignal() which is what enqueueProto is doing.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-09 17:55:14 -06:00
Ivan Kozlovic
bfe4eb68b2 Merge pull request #1506 from nats-io/fix_1505
[FIXED] Connection name in log statement for some IPv6 addresses
2020-07-08 14:59:25 -06:00
Ivan Kozlovic
4ea3f9c57e [FIXED] Handling or real duplicate subscription
That is, if the server receives "SUB foo 1" more than once from
the same client, we would register in the client map this subscription
only once, and add to the account's sublist only once, however we
would have updated shadow subscriptions and route/gateway maps for
each SUB protocol, which would result in inability to send unsubscribe
to routes when the client goes away or unsubscribes.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-08 14:19:32 -06:00
Ivan Kozlovic
2c8bf520d1 [FIXED] Connection name in log statement for some IPv6 addresses
If an IPv6 address contains some "%" characters, this was causing
the connection name in log statement to mess up the Sprintf formatting.
The solution is to escape those "%" characters.

Resolves #1505

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-08 10:17:14 -06:00
Derek Collison
ff8dfa8d35 Reordering for inbound processing
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-01 07:33:34 -07:00
Derek Collison
06ca580334 Update write deadline, client processing and slow proxy
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-30 16:41:01 -07:00
Derek Collison
2a799e7f5f Track destination leafnode status
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-26 11:18:11 -07:00
Derek Collison
6c805eebc7 Properly support leadnode clusters.
Leafnodes that formed clusters were partially supported. This adds proper support for origin cluster, subscription suppression and data message no echo for the origin cluster.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-26 09:03:22 -07:00
Derek Collison
120402241a Fix for #1486
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-18 21:04:34 -07:00
Pavel Khlebovich
d5eb9ff6f2 Support Cookie JWT auth via WebSocket 2020-06-18 19:27:42 +03:00
Derek Collison
fa744fdcda Proper no response
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-17 15:47:55 -07:00
Derek Collison
6c69c6ae48 Off by one, and need to send in place
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-17 15:32:26 -07:00
Derek Collison
3729552a37 Always check publish perms for leafnode delivery
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-16 13:34:53 -07:00
Derek Collison
ca4f03c1a6 Properly handle leafnode spoke permissions.
When a leafnode would connect with credentials that had permissions the spoke did not have a way of knowing what those were.
This could lead to being disconnected when sending subscriptions or messages to the hub which were not allowed.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-16 08:33:09 -07:00
Derek Collison
e1c397342c Merge branch 'master' into no_responders 2020-06-15 10:23:01 -07:00
Derek Collison
ddb4219f7a Allow support for a empty response message when no responders are present.
This will also set a response status of 503 with the new header support.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-15 10:10:21 -07:00
Derek Collison
dd61535e5a Cluster names are now required.
Added cluster names as required for prep work for clustered JetStream. System can dynamically pick a cluster name and settle on one even in large clusters.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 15:48:38 -07:00
Derek Collison
4dee03b587 Allow mixed TLS and non-TLS on same port
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-05 18:04:11 -07:00
Ivan Kozlovic
25bd5ca352 [FIXED] Unsubscribe may not be propagated through a leaf node
There is a race between the time the processing of a subscription
and the init/send of subscriptions when accepting a leaf node
connection that may cause internally a subscription's subject
to be counted many times, which would then prevent the send of
an LS- when the subscription's interest goes away.

Imagine this sequence of events, each side represents a "thread"
of execution:
```
client readLoop                         leaf node readLoop
----------------------------------------------------------
recv SUB foo 1
sub added to account's sublist

                                         recv CONNECT
                                     auth, added to acc.

updateSmap
smap["foo"]++ -> 1
no LS+ because !allSubsSent

                                         init smap
                                    finds sub in acc sl
                                    smap["foo"]++ -> 2
                                        sends LS+ foo
                                    allSubsSent == true

recv UNSUB 1
updateSmap
smap["foo"]-- -> 1
no LS- because count != 0
----------------------------------------------------------
```
Equivalent result but with slightly diffent execution:
```
client readLoop                         leaf node readLoop
----------------------------------------------------------
recv SUB foo 1
sub added to account's sublist

                                         recv CONNECT
                                     auth, added to acc.

                                         init smap
                                    finds sub in acc sl
                                    smap["foo"]++ -> 1
                                        sends LS+ foo
                                    allSubsSent == true

updateSmap
smap["foo"]++ -> 2
no LS+ because count != 1

recv UNSUB 1
updateSmap
smap["foo"]-- -> 1
no LS- because count != 0
----------------------------------------------------------
```

The approach for the fix is delay the creation of the smap
until we actually initialize the map and send the subs on processing
of the CONNECT.
In the meantime, as soon as the LN connection is registered
and available in updateSmap, we check that smap is nil or
not. If nil, we do nothing.

In "init smap" we keep track of the subscriptions that have been
added to smap. This map will be short lived, just enough to
protect against races above.

In updateSmap, when smap is not nil, we need to checki, if we
are adding, that the subscription has not already been handled.
The tempory subscription map will be ultimately emptied/set to
nil with the use of a timer (if not emptied in place when
processing smap updates).

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-05 10:07:15 -06:00
Derek Collison
c969e7e424 Do proper ubsubscribe when shutting off restore endpoint
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-04 08:58:14 -07:00
Ivan Kozlovic
5fe60d3084 Removed double connection close trace
In master, this this what happens when a connection is closed
and server runs with `-D`
```
[95023] 2020/06/03 14:55:28.395532 [DBG] 127.0.0.1:54077 - cid:2 - Client connection created
[95023] 2020/06/03 14:55:29.164118 [DBG] 127.0.0.1:54077 - cid:2 - Client connection closed: Client Closed
[95023] 2020/06/03 14:55:29.164191 [DBG] 127.0.0.1:54077 - cid:2 - Client connection closed
```
Notice the trace of connection closed with the reason, and then
the bare connection closed statement.

This double trace was introduced by mistake during the JS branch
work (dd116fcfd4 (diff-853eb184ac73cf9597d7833f6b89e9c9R3547))

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-03 15:00:34 -06:00
Ivan Kozlovic
1e149f4041 Merge pull request #1440 from nats-io/jwt2
Update imports for jwt/v2
2020-06-02 11:10:21 -06:00
Derek Collison
b5dfb984e9 Fixes for race detections under GHA
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-01 18:34:18 -07:00
aricart
e7590f3065 jwt2 testbed 2020-06-01 18:00:13 -04:00
Ivan Kozlovic
e9805a3109 [FIXED] Possible removal of interest on queue subs with leaf nodes
Server was incorrectly processing a queue subscription removal
as both a plain sub and queue sub, which may have resulted in
drop of interest even when some queue subs remained.

Resolves #1421

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-28 10:21:51 -06:00
Derek Collison
fa59cff105 Add in snapshot and restore JSApi
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-27 20:01:30 -07:00
Ivan Kozlovic
dc0f688cbf [FIXED] LameDuckMode sends INFO to clients
Also send an INFO to routes so that the remotes can remove the
LDM's server client URLs and notify their own clients of this
change.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-21 12:15:20 -06:00
Ivan Kozlovic
9715848a8e [ADDED] Websocket support
Websocket support can be enabled with a new websocket
configuration block:

```
websocket {
    # Specify a host and port to listen for websocket connections
    # listen: "host:port"

    # It can also be configured with individual parameters,
    # namely host and port.
    # host: "hostname"
    # port: 4443

    # This will optionally specify what host:port for websocket
    # connections to be advertised in the cluster
    # advertise: "host:port"

    # TLS configuration is required
    tls {
      cert_file: "/path/to/cert.pem"
      key_file: "/path/to/key.pem"
    }

    # If same_origin is true, then the Origin header of the
    # client request must match the request's Host.
    # same_origin: true

    # This list specifies the only accepted values for
    # the client's request Origin header. The scheme,
    # host and port must match. By convention, the
    # absence of port for an http:// scheme will be 80,
    # and for https:// will be 443.
    # allowed_origins [
    #    "http://www.example.com"
    #    "https://www.other-example.com"
    # ]

    # This enables support for compressed websocket frames
    # in the server. For compression to be used, both server
    # and client have to support it.
    # compression: true

    # This is the total time allowed for the server to
    # read the client request and write the response back
    # to the client. This include the time needed for the
    # TLS handshake.
    # handshake_timeout: "2s"
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-20 11:14:39 -06:00
Derek Collison
c9f78d6f79 Fixes post rebasing with master
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 16:38:19 -07:00