Commit Graph

278 Commits

Author SHA1 Message Date
Ivan Kozlovic
61bd1b8d86 MQTT clustering
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-02-19 08:50:00 -07:00
Derek Collison
aeef06b677 Merge pull request #1900 from nats-io/stepdown
Allow meta leader stepdown.
2021-02-08 17:01:48 -07:00
Derek Collison
ad0f10fb5c Allow metaleader stepdown.
Allow easy way to put authorization on system account and still use $G.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-08 15:01:23 -08:00
Matthias Hanel
0cae6ab4e7 [added] support for jwt based account mappings (#1897)
support for jwt based account mappings

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-02-08 17:25:14 -05:00
Matthias Hanel
3799b90011 [Adding] support for account_token_position (#1874)
This change does 4 things:
Refactor to only have one function to validate imports.
Have this function support the jwt field account_token_position.
For completeness make this value configurable as well.
unit tests.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-02-01 19:51:36 -05:00
Ivan Kozlovic
a0fb39fb7e Merge pull request #1862 from nats-io/varz-jwt
[added] field to varz output containing the operator jwt
2021-01-29 11:53:20 -07:00
Matthias Hanel
f487429d9e incorporated comments
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-01-29 13:25:02 -05:00
Ivan Kozlovic
9587bf8cd4 Changed option to make masking the default and option to disable it
This will allow a better experience if there is a load balancer
in between and expects websocket frames to be masked.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-01-29 11:22:22 -07:00
Matthias Hanel
2a34f0daee [added] field to varz output containing the operator jwt/claim
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-01-29 12:32:40 -05:00
Ivan Kozlovic
2b8c6e0124 Support for Websocket Leafnode connections
Added two options in the remote leaf node configuration

- compress, for websocket only at the moment
- ws_masking, to force remote leafnode connections to mask websocket
frames (default is no masking since it is communication between
server to server)

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-01-28 13:13:11 -07:00
Waldemar Quevedo
dbbf4fdf96 Add --server_name flag to set server name
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-01-26 12:16:31 -08:00
Matthias Hanel
9081646109 [added] support for tags and filter ping monitoring requests by tags (#1832)
fixes #1588

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-01-21 21:16:09 -05:00
Derek Collison
f0cdf89c61 JetStream Clustering WIP
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-14 01:14:52 -08:00
Ivan Kozlovic
1dba6418ed [ADDED] MQTT Support
This PR introduces native support for MQTT clients. It requires use
of accounts with JetStream enabled. Since as of now clustering is
not available, MQTT will be limited to single instance.

Only QoS 0 and 1 are supported at the moment. MQTT clients can
exchange messages with NATS clients and vice-versa.

Since JetStream is required, accounts with JetStream enabled must
exist in order for an MQTT client to connect to the NATS Server.
The administrator can limit the users that can use MQTT with the
allowed_connection_types option in the user section. For instance:
```
accounts {
  mqtt {
    users [
      {user: all, password: pwd, allowed_connection_types: ["STANDARD", "WEBSOCKET", "MQTT"]}
      {user: mqtt_only, password: pwd, allowed_connection_types: "MQTT"}
    ]
    jetstream: enabled
  }
}
```
The "mqtt_only" can only be used for MQTT connections, which the user
"all" accepts standard, websocket and MQTT clients.

Here is what a configuration to enable MQTT looks like:
```
mqtt {
  # Specify a host and port to listen for websocket connections
  #
  # listen: "host:port"

  # It can also be configured with individual parameters,
  # namely host and port.
  #
  # host: "hostname"
  port: 1883

  # TLS configuration section
  #
  # tls {
  #  cert_file: "/path/to/cert.pem"
  #  key_file: "/path/to/key.pem"
  #  ca_file: "/path/to/ca.pem"
  #
  #  # Time allowed for the TLS handshake to complete
  #  timeout: 2.0
  #
  #  # Takes the user name from the certificate
  #  #
  #  # verify_an_map: true
  #}

  # Authentication override. Here are possible options.
  #
  # authorization {
  #   # Simple username/password
  #   #
  #   user: "some_user_name"
  #   password: "some_password"
  #
  #   # Token. The server will check the MQTT's password in the connect
  #   # protocol against this token.
  #   #
  #   # token: "some_token"
  #
  #   # Time allowed for the client to send the MQTT connect protocol
  #   # after the TCP connection is established.
  #   #
  #   timeout: 2.0
  #}

  # If an MQTT client connects and does not provide a username/password and
  # this option is set, the server will use this client (and therefore account).
  #
  # no_auth_user: "some_user_name"

  # This is the time after which the server will redeliver a QoS 1 message
  # sent to a subscription that has not acknowledged (PUBACK) the message.
  # The default is 30 seconds.
  #
  # ack_wait: "1m"

  # This limits the number of QoS1 messages sent to a session without receiving
  # acknowledgement (PUBACK) from that session. MQTT specification defines
  # a packet identifier as an unsigned int 16, which means that the maximum
  # value is 65535. The default value is 1024.
  #
  # max_ack_pending: 100
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-11-30 20:08:44 -07:00
Matthias Hanel
8fd1b66f66 Renaming to verify_cert_and_check_known_urls
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-11-20 16:56:44 -05:00
Matthias Hanel
eda80ff7b5 changing the option name verify_and_implicit_allow to verify_and_accept_known_urls
This follows the suggestion by phil. I added the and to be similar to verify_and_map.
I fixed a minor issue where the implicit verify could be overwriting an
explicitly configured one.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-11-20 14:27:24 -05:00
Matthias Hanel
f24aec4af7 Incorporating comments and fixed an issue where code was not invoked
had to change failing tests to use insecure as to not fail due to the
outgoing connection being not trusted.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-11-20 13:57:34 -05:00
Matthias Hanel
74642e024e [Added] verify_and_implicit_allow to tie subject alt name to url in cfg
Only works for gateways and routes. When true the subject alt DNS name
must match one url in the corresponding configuration

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-11-20 13:57:34 -05:00
Matthias Hanel
3b4da21e07 Rename reject_unknown to reject_unknown_cluster
this is supposed to make it clearer what is being rejected

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-11-17 17:54:45 -05:00
Derek Collison
511cdaa5ac Do not report bad latency on auto-unsubscribe triggers
Signed-off-by: Derek Collison <derek@nats.io>
2020-10-22 05:41:33 -07:00
Matthias Hanel
2144f01f21 Adding support and an option for removal of jwt
To enable in full mode configure allow_delete: true
When enabled the file will be renamed to allow for manual restore.

In cache mode it will be enabled by default.
When enabled files will be deleted.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-10-20 16:45:11 -04:00
Derek Collison
4af434e586 Allow mapping destinations to be parsed as singles and not force array syntax
Signed-off-by: Derek Collison <derek@nats.io>
2020-10-17 08:21:02 -07:00
Derek Collison
5f5ceb4668 Allow cluster filters for mappings, changed accountz for mappings
Signed-off-by: Derek Collison <derek@nats.io>
2020-10-16 16:53:14 -07:00
Derek Collison
6faf07d583 Account subject mappings and full wildcard support for exports/imports
The mappings enable traffic shaping functionality.
Also added a subject transform which can match any subject and transform to a new one, even re-ordering partial wildcards.

Signed-off-by: Derek Collison <derek@nats.io>
2020-10-15 10:26:25 -07:00
Matthias Hanel
60c8cbd1f2 Fixed crash on non existing directory
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-10-13 01:30:03 -04:00
Matthias Hanel
8d8f6480d7 Incorporating comments
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-10-12 19:21:32 -04:00
Matthias Hanel
bb63fd5f40 Adding list/delete/update operations for jwt stored by nats-resolver
Update already existed scoped by account, this exposes update without account.
List returns a list of all stored accounts.
Delete deletes accounts.
Fix a crash on startup with non existing directory.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-10-12 18:07:07 -04:00
Ivan Kozlovic
9b6385d6e7 [CHANGED] Default TLS and Auth timeout
The default TLS timeout has been bumped to 2 seconds.
The default Auth timeout is now 2 seconds, but if TLS config
is present, it is equal to TLS timeout (possibly default value) + 1s.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-10-08 16:15:30 -06:00
Ivan Kozlovic
7ccbaca782 Added an allowed connection type filter for users
Users and NKey users will now have the option to specify a list
of allowed connection types.

This will allow for instance a certain user to be allowed to
connect as a standard NATS client, but not as Websocket, or
vice-versa.

This also fixes the websocket auth override. Indeed, with
the original behavior, the websocket users would have been bound
to $G, which would not work when there are accounts defined, since
when that is the case, no app can connect/bind to $G account.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-16 18:22:44 -06:00
Ivan Kozlovic
2ad2bed170 [ADDED] Support for route hostname resolution
We previously simply called DialTimeout() on a route's url when
soliciting. If it resolved to the IP of the host, it would create
a route to self, which server detects, but then would not try again
with other IPs that would have allowed to form a cluster with
other servers running on the other IPs.

This PR keeps track of local IPs + cluster port and exclude them
from the list of IPs returned by LookupHost API. This even prevent
solicitation of routes to self. Only non-local IPs will be tried.

Resolves #1586

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-08 13:40:17 -06:00
Waldemar Quevedo
3161cb1693 Add more config checks for leafnode remotes
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2020-09-04 00:34:40 -07:00
Matthias Hanel
e1350a05f3 Emit latency traces when sampling is set to headers
Latency reports will include the header(s) responsible for the trace
Updated ADR to have it reflect implementation

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-08-21 20:01:11 -04:00
Matthias Hanel
48c87c1447 Nats based resolver & avoiding nats account server in smaller deployments (#1550)
* Adding nats based resolver and bootstrap system account

These resolver operate on an exclusive  directory
Two types:
full: managing all jwt in the directory
    Will synchronize with other full resolver
    nats-account-server will also run such a resolver
cache: lru cache managing only a subset of all jwt in the directory
    Will lookup jwt from full resolver
    Can overwrite expiration with a ttl for the file

Both:
    track expiration of jwt and clean up
    Support reload
    Notify the server of changed jwt

Bootstrapping system account allows users signed with the system account
jwt to connect, without the server knowing the jwt.
This allows uploading jwt (including system account) using nats by
publishing to $SYS.ACCOUNT.<name>.CLAIMS.UPDATE
Sending a request, server will respond with the result of the operation.

Receive all jwt stored in one server by sending a
request to $SYS.ACCOUNT.CLAIMS.PACK
One server will respond with a message per stored jwt.
The end of the responses is indicated by an empty message.

The content of dirstore.go and dirstore_test.go was moved from
nats-account-server

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-08-18 13:58:41 -06:00
Ivan Kozlovic
20a67a5be8 Websocket: add option to disable TLS
The new option Websocket.NoTLS would have to be set to true
to disable the server check that enforces TLS configuration.

Resolves #1529

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-29 17:33:02 -06:00
Derek Collison
9514576b72 Merge pull request #1526 from nats-io/no-header-support-option
added 'no_header_support' config option mapping to NoHeaderSupport
2020-07-22 12:13:30 -05:00
Alberto Ricart
2047c791ea added option so clients can test if a server disables header support 2020-07-22 10:11:44 -05:00
Ivan Kozlovic
0ce84a83c5 Added -cluster_name command line option
This is handy for client libraries that start the server as
external executable and pass command line arguments. Without
specifying the cluster name, routes can take time to establish
and cause some tests to fail.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-21 15:50:24 -06:00
Pavel Khlebovich
d5eb9ff6f2 Support Cookie JWT auth via WebSocket 2020-06-18 19:27:42 +03:00
Derek Collison
146d8f5dcb Updates based on feedback, sped up some slow tests
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 17:26:43 -07:00
Derek Collison
dd61535e5a Cluster names are now required.
Added cluster names as required for prep work for clustered JetStream. System can dynamically pick a cluster name and settle on one even in large clusters.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 15:48:38 -07:00
Ivan Kozlovic
d18b013676 Merge pull request #1463 from nats-io/websocket_auth_block
Added dedicated auth block for websocket
2020-06-09 16:09:41 -06:00
Ivan Kozlovic
01b14c2abe Added dedicated auth block for websocket
Websocket can now override
- Username/password
- Token
- Users
- NKeys
- no_auth_user
- auth_timeout

For TLS, support for verify and verify_and_map. We used to set
tls config's ClientAuth to NoClientCert. It will now depend
if the config requires client certificate verification, which
is needed if TLSMap is enabled.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-09 11:29:52 -06:00
Derek Collison
bfe52fdb77 Merge pull request #1458 from nats-io/mixed
Allow mixed TLS and non-TLS on same port.
2020-06-08 11:12:45 -07:00
Ivan Kozlovic
cd6d71deaa [ADDED] lame_duck_grace_period option
The grace period used to be hardcoded at 10 seconds.
This option allows the user to configure the amount of time the
server will wait before initiating the closing of client connections.

Note that the grace period needs to be strictly lower than the overall
lame_duck_duration. The server deducts the grace period from that
overall duration and spreads the closing of connections during
that time.
For instance, if there are 1000 connections and the lame duck
duration is set to 30 seconds and grace period to 10, then
the server will use 30-10 = 20 seconds to spread the closing
of those 1000 connections, so say roughly 50 clients per second.

Resolves #1459.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-08 11:43:25 -06:00
Derek Collison
4dee03b587 Allow mixed TLS and non-TLS on same port
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-05 18:04:11 -07:00
Matthias Hanel
0a3e89c64a Incorporating comments
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-06-02 18:38:17 -04:00
Matthias Hanel
cf6fcda75c Added default_permissions to accounts and account jwt
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-06-02 16:06:01 -04:00
aricart
e7590f3065 jwt2 testbed 2020-06-01 18:00:13 -04:00
Derek Collison
2bd7553c71 System Account on by default.
Most of the changes are to turn it off for tests that were watching subscriptions and such.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-29 17:56:45 -07:00
R.I.Pienaar
b7c79f66e4 improve consistency between top level and account js options
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-27 16:22:47 +02:00