Commit Graph

390 Commits

Author SHA1 Message Date
Phil Pennock
331f6ca3aa WIP: theoretically, populate expvar maps 2020-10-13 19:53:35 -04:00
Ivan Kozlovic
df9d5f5fd9 Accepting route warns if remote server has same name
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-10-08 17:59:33 -06:00
Matthias Hanel
08e37e0d94 Updated jwt library and check (account/token) issuer prior to jwt Validate
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-09-29 21:03:40 -04:00
Matthias Hanel
634ce9f7c8 [Adding] Accountz monitoring endpoint and INFO monitoring req subject
Returned imports/exports are formated like jwt exports imports, even if
they originating account is from config.

Fixes #1604

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-09-23 16:22:48 -04:00
Ivan Kozlovic
7ccbaca782 Added an allowed connection type filter for users
Users and NKey users will now have the option to specify a list
of allowed connection types.

This will allow for instance a certain user to be allowed to
connect as a standard NATS client, but not as Websocket, or
vice-versa.

This also fixes the websocket auth override. Indeed, with
the original behavior, the websocket users would have been bound
to $G, which would not work when there are accounts defined, since
when that is the case, no app can connect/bind to $G account.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-16 18:22:44 -06:00
Ivan Kozlovic
540160116e Merge pull request #1587 from nats-io/js-jwt
Updating jetstream account settings from jwt
2020-09-10 12:52:25 -06:00
Ivan Kozlovic
2ad2bed170 [ADDED] Support for route hostname resolution
We previously simply called DialTimeout() on a route's url when
soliciting. If it resolved to the IP of the host, it would create
a route to self, which server detects, but then would not try again
with other IPs that would have allowed to form a cluster with
other servers running on the other IPs.

This PR keeps track of local IPs + cluster port and exclude them
from the list of IPs returned by LookupHost API. This even prevent
solicitation of routes to self. Only non-local IPs will be tried.

Resolves #1586

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-08 13:40:17 -06:00
Matthias Hanel
a3c5fd4575 Updating jetstream account settings from jwt
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-09-04 16:07:38 -04:00
Matthias Hanel
3954030abf [FIXED] Error when importing an account results in an error, retry later (#1578)
* [FIXED] Error when importing an account results in an error

When the account that could not be imported is updated, update the
original account as well.

Fixes #1582

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-09-03 13:57:06 -06:00
Ivan Kozlovic
7cc0113076 Merge pull request #1565 from nats-io/fix_panic_on_conn_close_early
[FIXED] Possible panic for TLS connections that are aborted early
2020-08-18 16:50:45 -06:00
Ivan Kozlovic
92970316ee [FIXED] Possible panic for TLS connections that are aborted early
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-08-18 16:38:35 -06:00
Matthias Hanel
48c87c1447 Nats based resolver & avoiding nats account server in smaller deployments (#1550)
* Adding nats based resolver and bootstrap system account

These resolver operate on an exclusive  directory
Two types:
full: managing all jwt in the directory
    Will synchronize with other full resolver
    nats-account-server will also run such a resolver
cache: lru cache managing only a subset of all jwt in the directory
    Will lookup jwt from full resolver
    Can overwrite expiration with a ttl for the file

Both:
    track expiration of jwt and clean up
    Support reload
    Notify the server of changed jwt

Bootstrapping system account allows users signed with the system account
jwt to connect, without the server knowing the jwt.
This allows uploading jwt (including system account) using nats by
publishing to $SYS.ACCOUNT.<name>.CLAIMS.UPDATE
Sending a request, server will respond with the result of the operation.

Receive all jwt stored in one server by sending a
request to $SYS.ACCOUNT.CLAIMS.PACK
One server will respond with a message per stored jwt.
The end of the responses is indicated by an empty message.

The content of dirstore.go and dirstore_test.go was moved from
nats-account-server

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-08-18 13:58:41 -06:00
Phil Pennock
3c680eceb9 Inhibit Go's default TCP keepalive settings for NATS (#1562)
Inhibit Go's default TCP keepalive settings for NATS

Go 1.13 changed the semantics of the tuning parameters for TCP keepalives, including the default value.  This affects all TCP listeners.  The NATS protocol has its own L7 keepalive system (PING/PONG) and the Go defaults are not a good fit for some valid deployment scenarios, while Go doesn't directly expose a working API for tuning these.

Rather than add a configuration knob and pull in another dependency (with portability issues) just disable TCP keepalives for all listeners used for speaking the NATS protocol.

Change the tests so we test the same logic.  Do not change HTTP monitoring, profiling, or the websocket API listeners.

Change KeepAlive on client connections too.
2020-08-14 13:37:59 -04:00
Ivan Kozlovic
c620175353 Rework closeConnection()
This change allows the removal of the connection and update of
the server state to be done "in place" but still delay the flushing
of and close of tcp connection to the writeLoop. With ref counting
we ensure that the reconnect happens after the flushing but not
before the state has been updated.

Had to fix some places where we may have called closeConnection()
from under the server lock since it now would deadlock for sure.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-31 15:30:17 -06:00
Ivan Kozlovic
96ccf91566 [FIXED] Possible deadlock with solicited leafnodes when cluster conflict
We cannot call c.closeConnection() under the server lock because
closeConnection() can invoke server lock in some cases.

Created a test that should run without `-race` to reproduce the deadlock
(which it does) but sometimes would fail because cluster would not be
formed. This unconvered an issue with conflict resolution which
test TestRouteClusterNameConflictBetweenStaticAndDynamic() can reproduce
easily. The issue was that we were not updating a dynamic name with
the remote if the remote was non dynamic.

Resolves #1543

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-30 18:45:36 -06:00
Matthias Hanel
00faefec06 Reduce usage of tmpAccounts to only location where it is needed imports
On import handle it with priority as in non recursive situations, it
won't be present.
2020-07-27 17:38:39 -04:00
Matthias Hanel
37692d2cf9 [Fixed] Skip fetch when a non config based account resolver is used
Resolves #1532

Instead of the fetched account we create a dummy account that is
expired. Any client connecting will trigger a fetch of the actual
account jwt.

This also avoids one fetch, thus the unit test was changed to reflect
this.
Unlike other resolver the memory resolver does not depend on external
systems. It is purely based on server configuration. Therefore, fetch
can be done and not finding an account means there is a configuration issue.
2020-07-27 17:36:55 -04:00
Matthias Hanel
99921725a9 Ensuring that an untrusted account is not held in memory
The check that an account has to be signed by a configured operator is
done after fetch as well. As a consequence an account claim will never
become an Account in memory.
The original check during client or leaf authentication is left in
place.

Adding unit tests.
Modifying existing tests to not rely on an account but it's name instead.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-07-27 11:59:49 -04:00
Ivan Kozlovic
b9764db478 Renamed gossipURLs type and moved its declaration to util.go
Also made the add/remove/getAsStringSlice receiver for this type.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-16 11:22:58 -06:00
Ivan Kozlovic
9b0967a5d1 [FIXED] Handling of gossiped URLs
If some servers in the cluster have the same connect URLs (due
to the use of client advertise), then it would be possible to
have a server sends the connect_urls INFO update to clients with
missing URLs.

Resolves #1515

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-15 17:39:12 -06:00
Ivan Kozlovic
9288283d90 Fixed accept loops that could leave connections opened
This was discovered with the test TestLeafNodeWithGatewaysServerRestart
that was sometimes failing. Investigation showed that when cluster B
was shutdown, one of the server on A that had a connection from B
that just broke tried to reconnect (as part of reconnect retries of
implicit gateways) to a server in B that was in the process of shuting down.
The connection had been accepted but createGateway not called because
the server's running boolean had been set to false as part of the shutdown.
However, the connection was not closed so the server on A had a valid
connection to a dead server from cluster B. When the B cluster (now single
server) was restarted and a LeafNode connection connected to it, then
the gateway from B to A was created, that server on A did not create outbound
connection to that B server because it already had one (the zombie one).

So this PR strengthens the starting of accept loops and also make sure
that if a connection (all type of connections) is not accepted because
the server is shuting down, that connection is properly closed.

Since all accept loops had almost same code, made a generic function
that accept functions to call specific create connection functions.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-06 17:03:19 -06:00
Derek Collison
6c805eebc7 Properly support leadnode clusters.
Leafnodes that formed clusters were partially supported. This adds proper support for origin cluster, subscription suppression and data message no echo for the origin cluster.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-26 09:03:22 -07:00
Matthias Hanel
d086a39b64 Add filtering by name and cluster to PING events
On cluster name change, reset internalSendLoop so it picks up the
changed name.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-06-16 18:26:35 -04:00
Derek Collison
2b9e3e5b15 Merge pull request #1476 from nats-io/cluster_name
Cluster names are now required.
2020-06-15 10:07:30 -07:00
Derek Collison
1e52a1007b More updates based on feedback
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-13 08:00:57 -07:00
Derek Collison
146d8f5dcb Updates based on feedback, sped up some slow tests
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 17:26:43 -07:00
Derek Collison
dd61535e5a Cluster names are now required.
Added cluster names as required for prep work for clustered JetStream. System can dynamically pick a cluster name and settle on one even in large clusters.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 15:48:38 -07:00
Ivan Kozlovic
67d2638859 [ADDED] Print the config file being used in startup banner
Resolves #1451

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-12 12:21:50 -06:00
Ivan Kozlovic
d2a8282a0d [FIXED] LeafNode TLSMap and websocket auth override
We added authentication override block for websocket configuration
in PR #1463 and #1465 which somehow introduced a drop in perf as
reported by the bench tests.
This PR refactors a bit to restore the performance numbers.

This change also fixes the override behavior for websocket auth:
- If websocket's NoAuthUser is configured, the websocket's auth
  block MUST define Users, and the user be present.
- If there is any override (username/pwd,token,etc..) then the
  whole block config will be used when authenticating a websocket
  client, which means that if websocket NoAuthUser is empty we
  are not falling back to the regular client's NoAuthUser config.
- TLSMap always override the regular client's config. That is,
  whatever TLSMap value specified in the websocket's tls{} block
  will be used.

The TLSMap configuration was not used for LeafNodes. The behavior
now will be:
- If LeafNode's auth block contains users and TLSMap is true,
  the user is looked up based on the cert's info. If not found,
  authentication will fail. If found, it will be authenticated
  and bound to associated account.
- If no user is specified in LeafNode's auth block and TLSMap
  is true, then the cert's info will be used against the global
  users map.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-11 17:06:54 -06:00
Ivan Kozlovic
d18b013676 Merge pull request #1463 from nats-io/websocket_auth_block
Added dedicated auth block for websocket
2020-06-09 16:09:41 -06:00
Ivan Kozlovic
01b14c2abe Added dedicated auth block for websocket
Websocket can now override
- Username/password
- Token
- Users
- NKeys
- no_auth_user
- auth_timeout

For TLS, support for verify and verify_and_map. We used to set
tls config's ClientAuth to NoClientCert. It will now depend
if the config requires client certificate verification, which
is needed if TLSMap is enabled.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-09 11:29:52 -06:00
Derek Collison
bfe52fdb77 Merge pull request #1458 from nats-io/mixed
Allow mixed TLS and non-TLS on same port.
2020-06-08 11:12:45 -07:00
Ivan Kozlovic
cd6d71deaa [ADDED] lame_duck_grace_period option
The grace period used to be hardcoded at 10 seconds.
This option allows the user to configure the amount of time the
server will wait before initiating the closing of client connections.

Note that the grace period needs to be strictly lower than the overall
lame_duck_duration. The server deducts the grace period from that
overall duration and spreads the closing of connections during
that time.
For instance, if there are 1000 connections and the lame duck
duration is set to 30 seconds and grace period to 10, then
the server will use 30-10 = 20 seconds to spread the closing
of those 1000 connections, so say roughly 50 clients per second.

Resolves #1459.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-08 11:43:25 -06:00
Derek Collison
3a4e835311 Key off TLS first byte
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-06 13:41:34 -07:00
Derek Collison
75395ed862 Read 4 bytes, wait TLS timeout
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-05 18:38:20 -07:00
Derek Collison
4dee03b587 Allow mixed TLS and non-TLS on same port
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-05 18:04:11 -07:00
Ivan Kozlovic
98ea70a590 LameDuckMode takes into account websocket accept loop
This is related to #1408.
Make sure that we close the websocket "accept loop" if configured
before proceeding with the lame duck mode.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-02 17:49:38 -06:00
aricart
e7590f3065 jwt2 testbed 2020-06-01 18:00:13 -04:00
Derek Collison
05e38ae527 Merge branch 'master' into sys-acc 2020-06-01 11:53:14 -07:00
Derek Collison
eca04c6fce First pass header support for JetStream
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-30 10:04:23 -07:00
Derek Collison
2bd7553c71 System Account on by default.
Most of the changes are to turn it off for tests that were watching subscriptions and such.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-29 17:56:45 -07:00
Ivan Kozlovic
762bc84daf Merge pull request #1408 from nats-io/ldm_send_info
[FIXED] LameDuckMode sends INFO to clients
2020-05-26 11:37:59 -06:00
Derek Collison
710ef00383 Don't allow JetStream on system account. Warn when accounts configured but no JS
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 12:17:18 -07:00
Ivan Kozlovic
8678a61e3e Move the send of INFO after client listener has been shutdown
This will ensure that there is no race where clients are accepted
after the LDM INFO notification.

Also add to the test to make sure that we don't send INFO when
routes are disconnected due to internal closing of connections
during the shutdown process.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-24 11:38:49 -06:00
Ivan Kozlovic
dc0f688cbf [FIXED] LameDuckMode sends INFO to clients
Also send an INFO to routes so that the remotes can remove the
LDM's server client URLs and notify their own clients of this
change.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-21 12:15:20 -06:00
Ivan Kozlovic
9715848a8e [ADDED] Websocket support
Websocket support can be enabled with a new websocket
configuration block:

```
websocket {
    # Specify a host and port to listen for websocket connections
    # listen: "host:port"

    # It can also be configured with individual parameters,
    # namely host and port.
    # host: "hostname"
    # port: 4443

    # This will optionally specify what host:port for websocket
    # connections to be advertised in the cluster
    # advertise: "host:port"

    # TLS configuration is required
    tls {
      cert_file: "/path/to/cert.pem"
      key_file: "/path/to/key.pem"
    }

    # If same_origin is true, then the Origin header of the
    # client request must match the request's Host.
    # same_origin: true

    # This list specifies the only accepted values for
    # the client's request Origin header. The scheme,
    # host and port must match. By convention, the
    # absence of port for an http:// scheme will be 80,
    # and for https:// will be 443.
    # allowed_origins [
    #    "http://www.example.com"
    #    "https://www.other-example.com"
    # ]

    # This enables support for compressed websocket frames
    # in the server. For compression to be used, both server
    # and client have to support it.
    # compression: true

    # This is the total time allowed for the server to
    # read the client request and write the response back
    # to the client. This include the time needed for the
    # TLS handshake.
    # handshake_timeout: "2s"
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-20 11:14:39 -06:00
Derek Collison
050db531fc Bumping version
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 16:44:42 -07:00
Derek Collison
c9f78d6f79 Fixes post rebasing with master
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 16:38:19 -07:00
Derek Collison
d8b475c4b0 Better way to retrieve header support
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
55c77d1e4e Added support for delivery of HMSG and support for older clients
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00