449 Commits

Author SHA1 Message Date
Waldemar Quevedo
46eccd7658 WIP: NATS Server + OCSP Support
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
Signed-off-by: Jaime Piña <jaime@synadia.com>
2021-04-29 17:59:34 -07:00
Matthias Hanel
a67704e245 [fixed] crash when using nats-resolver without system account (#2162)
* [fixed] crash when using nats-resolver without system account

Fixes #2160
Will raise an error instead

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-26 20:50:56 -04:00
Matthias Hanel
4d112af6c1 Fix account resolver lock ordering on shutdown
This is benign as the inversion was between start and stop.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-20 20:08:14 -04:00
Jaime Piña
e12181cb83 Return not ready for connection reason
Currently, we use ReadyForConnections in server tests to wait for the
server to be ready. However, when this fails we don't get a clue about
why it failed.

This change adds a new unexported method called readyForConnections that
returns an error describing which check failed. The exported
ReadyForConnections version works exactly as before. The unexported
version gets used in internal tests only.
2021-04-20 11:45:08 -07:00
Derek Collison
27d8b939b5 Updated based on comments that the one fix was actually a misconfiguration.
Signed-off-by: Derek Collison <derek@nats.io>
2021-04-09 16:49:24 -07:00
Derek Collison
e438d2f5fa Mixed mode improvements.
1. When in mixed mode and only running the global account we now will check the account for JS.
2. Added code to decrease the cluster set size if we guessed wrong in mixed mode setup.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-09 14:58:35 -07:00
Matthias Hanel
f7a772f097 Ensure that leafNodeFinishConnectProcess is only executed once.
incorporate review comments

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-09 16:53:06 -04:00
Matthias Hanel
5d1f36dd17 [Fixed] leaf node subscription permission negotiation.
On connect all subscription where sent by the soliciting leaf node.
If creds contains sub deny permissions, the leaf node would be
disconnected.
This waits for the permissions to be exchanged and checks permissions
before sending subscriptions.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-09 16:53:06 -04:00
Ivan Kozlovic
c7f8296a85 Merge pull request #2065 from alexpantyukhin/extract_server_update
extract_server_update
2021-04-08 09:01:51 -06:00
alexpantyukhin
e16bebb8df extract update remote subscription. 2021-04-08 16:37:12 +04:00
Matthias Hanel
2664e964a8 [fixed] issue with concurrent account fetch when account was incomplete (#2067)
* [fixed] issue with concurrent account fetch when account was incomplete

This happened when a dummy (expired/incomplete) account was created during
a route operation. The dummy was to avoid fetching the account, which would
cause a lock inversion.
When a non route request required the account, we'd download it as it is
set to expired.
A concurrent request would result in ErrAccountResolverSameClaims which
the caller did not handle.
Fix is to remove ErrAccountResolverSameClaims.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-04-06 12:43:10 -04:00
Jaime Piña
6095e8f46d Guard account client map 2021-03-26 10:30:16 -07:00
Derek Collison
6cd6c380f1 Make standalone mode ignore leaf connections
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-25 16:13:23 -07:00
Derek Collison
e53caee5e8 Enforce server limits even when dynamic limits for accounts in play.
We were not properly enforcing server limits. This commit will allow a server to enforce limits but still remain functional even at the JetStream level.
Also fixed a bug for RAFT replay that could cause instability.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-25 16:06:27 -07:00
Derek Collison
61771e88f8 In operator mode with JetStream we want to load accounts that have stable storage.
Also if an account was registered but not JetStream enabled, update it vs error.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-20 06:53:13 -07:00
Derek Collison
ee92cc9a5b Properly print when a stream is doing out of band catchup. Print node banner consistently
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-14 07:29:36 -07:00
Derek Collison
bfb8e3432e Move RAFT comms off internal sendq.
Move route and gateway msgs our of fast path for inbound stream msgs.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 14:45:34 -08:00
Ivan Kozlovic
0f53bf6580 Fixed data race with nodeInfo
Took the approach of storing struct instead of pointer. Of course,
when changing the offline bool from false to true, it means that
we need to call Store again (with same key).

This is based on the assumption that those Load/Store are not too
frequent. Otherwise, we may need to use locking (and keep *nodeInfo)

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-03-03 13:28:45 -07:00
Matthias Hanel
25ef6b0f0d Merge pull request #1952 from nats-io/goland-lint
Fixed linter issues
2021-03-02 21:43:04 -05:00
Matthias Hanel
c50ee2a1c6 [Changed] all times exposed will be computed in UTC (#1943)
This also applies to times that end up in that json.
Where applicable moved time.Now() to where it is used.
Moved calls to .UTC() to where time is created it that time is converted
later anyway.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-02 21:37:42 -05:00
Matthias Hanel
4f2db7d187 Fixed linter issues
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-02 20:21:44 -05:00
Derek Collison
df77724aa4 Make ephemeral consumers R=1 and provide optimistic migration on peer removal or server shutdown.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 16:50:25 -08:00
Derek Collison
78bdc34637 General stability improvements. Fixes to subscription state not cleaning up.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 08:44:34 -08:00
Ivan Kozlovic
61bd1b8d86 MQTT clustering
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-02-19 08:50:00 -07:00
Derek Collison
30cc739797 Increase internal sendq due to JSC
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-16 13:19:13 -08:00
Derek Collison
579737a5e1 General fixes, stability improvements
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-11 18:13:24 -08:00
Derek Collison
ad0f10fb5c Allow metaleader stepdown.
Allow easy way to put authorization on system account and still use $G.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-08 15:01:23 -08:00
Derek Collison
42e3210265 Shutdown JetStream on failed stream msg store in clustered mode
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-07 11:52:52 -08:00
Derek Collison
1c79d96de8 user single node info struct
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-06 20:10:29 -08:00
Matthias Hanel
7b7543d298 [added] jsz nats and http monitoring endpoint for jetstream (#1881)
The new endpoints are /jsz on http and "$SYS.REQ.SERVER.PING.JSZ" and "$SYS.REQ.SERVER.%s.JSZ".
$SYS.REQ.ACCOUNT.%s.JSZ will only return info for the particular account

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-02-05 18:46:04 -05:00
Derek Collison
a1e0f7dc1a First pass at supercluster enablement.
This allows metacontrollers to span superclusters. Also includes placement directives for streams. By default they select the request origin cluster.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-03 17:28:13 -08:00
Derek Collison
e5c1d65fff Added in JS disable per server on reload. Also removing peerw from a stream and leader stepdown for streams and consumers.
Various bug fixes, stability improvments.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-01 19:39:08 -08:00
Derek Collison
8bd290c77a Fix for #1864.
When trying to make sure we properly created all subs for service imports we would check the internal client to see if we should process.
With JS enabled on the server we would place system imports that would break that check and orphan other service imports.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-29 17:51:14 -08:00
Ivan Kozlovic
2b8c6e0124 Support for Websocket Leafnode connections
Added two options in the remote leaf node configuration

- compress, for websocket only at the moment
- ws_masking, to force remote leafnode connections to mask websocket
frames (default is no masking since it is communication between
server to server)

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-01-28 13:13:11 -07:00
Ivan Kozlovic
131be1cb33 Make TLS client/server handshake helpers function
This reduces code duplication

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-01-28 13:13:11 -07:00
Matthias Hanel
dea9effa8d [added] support for StrictSigningKeyUsage and updated jwt library (#1845)
This will cause the server to not trust accounts/user signed by an
identity key

The boot strapping system account will assume the account is issued by
the operator.
If this is not desirable, the system account can be provided right away
as resolver_preload.

[fixes] crash when the system account uses signing keys and an update changes that key set.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-01-26 17:49:58 -05:00
Derek Collison
df0228f076 Change up for banner
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-26 14:06:50 -08:00
Derek Collison
d278996272 LDM trigger to move raft leaders
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-25 16:52:19 -08:00
Derek Collison
7b1e84c086 Fixed raft bug that would cause entries to be missed on restart with leader HB trigger.
Also added in creation times to stream and consumer assignments to make them consistent.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-25 08:47:37 -08:00
Matthias Hanel
d35cd2996d [added] jwt/issuerkey/nametag/tags to monitoring and event endpoints (#1830)
Also added a trace on jwt authentication

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-01-21 21:16:34 -05:00
Derek Collison
a1730f1b31 Report on RAFT group information.
This adds in optional reporting to stream and consumer info when running in clsutered mode.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-20 11:58:31 -08:00
Ivan Kozlovic
42dcdd2eb2 Simplify sendSubsToRoute()
Since we were creating subs on the fly, sub.im would always be nil.
We were passing a client because it was needed in sendRouteSubOrUnSubProtos().

This PR simply fills the buffer with each account's subscriptions.
There is also no need to have subs sent from different go routine
based on some threshold. Routes are no longer subject to max pending.

Some code has been made into a function so that they can be shared
by sendSubsToRoute() and sendRouteSubOrUnSubProtos(). The function
is simply adding to given buffer the RS+/- protocol.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-01-19 14:01:43 -07:00
Derek Collison
367d000314 Merge pull request #1815 from nats-io/jscfix
Routes send subscriptions by utilizing random clients from an account.
2021-01-15 18:26:38 -07:00
Derek Collison
754e31a3bc Routes send subscriptions by utilizing random clients from an account.
There was a bug where the client chosen under the $SYS account could have a different account.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-15 17:02:38 -08:00
Ivan Kozlovic
ef38abe75b Fixed gateway reply mapping following changes in JetStream clustering
Those changes are required to maintain backward compatibility.
Since the replies are "_G_.<gateway name hash>.<server ID hash>"
and the hash were 6 characters long, changing to 8 the hash function
would break things.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-01-15 17:32:04 -07:00
Derek Collison
f0cdf89c61 JetStream Clustering WIP
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-14 01:14:52 -08:00
Ivan Kozlovic
fc1521636c [FIXED] Config reload for gateways/leaf remote TLS configurations
Presence of TLS config in any remote gateway or leafnode would
cause the config reload to fail (because TLS config internal
content may change which fails the DeepEqual check).

This PR excludes the TLS configs in such case to check for
changes in gateways and leafnodes.

Although GW and LN config reload is technically supported, this
PR updates the internal remotes' TLS configuration so that
changes/updates to TLS certificates would take effect after
a configuration reload.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-12-11 16:56:25 -07:00
Ivan Kozlovic
67425d23c8 Add c.isMqtt() and c.isWebsocket()
This hides the check on "c.mqtt != nil" or "c.ws != nil".
Added some tests.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-12-02 15:52:06 -07:00
Ivan Kozlovic
41fac39f8e Split createClient() into versions for normal, WS and MQTT clients.
This duplicate quite a bit of code, but reduces the conditionals
in the createClient() function.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-12-02 13:50:50 -07:00
Ivan Kozlovic
1dba6418ed [ADDED] MQTT Support
This PR introduces native support for MQTT clients. It requires use
of accounts with JetStream enabled. Since as of now clustering is
not available, MQTT will be limited to single instance.

Only QoS 0 and 1 are supported at the moment. MQTT clients can
exchange messages with NATS clients and vice-versa.

Since JetStream is required, accounts with JetStream enabled must
exist in order for an MQTT client to connect to the NATS Server.
The administrator can limit the users that can use MQTT with the
allowed_connection_types option in the user section. For instance:
```
accounts {
  mqtt {
    users [
      {user: all, password: pwd, allowed_connection_types: ["STANDARD", "WEBSOCKET", "MQTT"]}
      {user: mqtt_only, password: pwd, allowed_connection_types: "MQTT"}
    ]
    jetstream: enabled
  }
}
```
The "mqtt_only" can only be used for MQTT connections, which the user
"all" accepts standard, websocket and MQTT clients.

Here is what a configuration to enable MQTT looks like:
```
mqtt {
  # Specify a host and port to listen for websocket connections
  #
  # listen: "host:port"

  # It can also be configured with individual parameters,
  # namely host and port.
  #
  # host: "hostname"
  port: 1883

  # TLS configuration section
  #
  # tls {
  #  cert_file: "/path/to/cert.pem"
  #  key_file: "/path/to/key.pem"
  #  ca_file: "/path/to/ca.pem"
  #
  #  # Time allowed for the TLS handshake to complete
  #  timeout: 2.0
  #
  #  # Takes the user name from the certificate
  #  #
  #  # verify_an_map: true
  #}

  # Authentication override. Here are possible options.
  #
  # authorization {
  #   # Simple username/password
  #   #
  #   user: "some_user_name"
  #   password: "some_password"
  #
  #   # Token. The server will check the MQTT's password in the connect
  #   # protocol against this token.
  #   #
  #   # token: "some_token"
  #
  #   # Time allowed for the client to send the MQTT connect protocol
  #   # after the TCP connection is established.
  #   #
  #   timeout: 2.0
  #}

  # If an MQTT client connects and does not provide a username/password and
  # this option is set, the server will use this client (and therefore account).
  #
  # no_auth_user: "some_user_name"

  # This is the time after which the server will redeliver a QoS 1 message
  # sent to a subscription that has not acknowledged (PUBACK) the message.
  # The default is 30 seconds.
  #
  # ack_wait: "1m"

  # This limits the number of QoS1 messages sent to a session without receiving
  # acknowledgement (PUBACK) from that session. MQTT specification defines
  # a packet identifier as an unsigned int 16, which means that the maximum
  # value is 65535. The default value is 1024.
  #
  # max_ack_pending: 100
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-11-30 20:08:44 -07:00