Commit Graph

151 Commits

Author SHA1 Message Date
Derek Collison
944dd248c4 Fix for tests
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-14 17:39:51 -07:00
Derek Collison
10167b1bcf Added in ability for normal accounts to access scoped connz info.
Added in client kind and sub type for clients.
Added in ability to filter connections based on matching subject interest.

Signed-off-by: Derek Collison <derek@nats.io>
2021-08-13 10:19:12 -07:00
Derek Collison
925a6fe6b2 Fix for #2388. Leafnodes with no JS can seamlessly access a HUB with JS.
This is the reverse of the early work to have LNs extend a non-JS cluster.
Also have mixed mode tests as well.

Signed-off-by: Derek Collison <derek@nats.io>
2021-08-01 14:57:47 -07:00
Matthias Hanel
a40ea298e5 [fixed] jetstream unique server name requirement across domains (#2378)
* [fixed] jetstream unique server name requirement across domains

including domain in server info
adding check for cluster name in duplicate leaf node connection check

This does not address non unique domains in the same domain, say within
super cluster.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-07-27 18:42:19 -04:00
Derek Collison
f13fa767c2 Remove the swapping of accounts during processing of service imports.
When processing service imports we would swap out the accounts during processing.
With the addition of internal subscriptions and internal clients publishing in JetStream we had an issue with the wrong account being used.
This was specific to delyaed pull subscribers trying to unsubscribe due to max of 1 while other JetStream API calls were running concurrently.
2021-07-26 07:57:10 -07:00
Matthias Hanel
c68ffe5ad5 [adding] kind and client_type to account connect/disconnect events (#2351)
* [adding] kind and client_type to client info. specifically account connect/disconnect events

Kind is Client/Leafnode but can take the value of Router/Gateway/JetStream/Account/System in the future.
When kind is Client, then client_type is set to mqtt/websocket/nats
This fixes #2291

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-07-07 17:43:50 -04:00
Matthias Hanel
fcb3382f4b [fixed] CONNS system endpoint which had issues with header and empty msg
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-07-06 16:46:36 -04:00
Matthias Hanel
a72ca8a9bf [fixed] header handling in system services
On export/import a header was inserted which broke parsing of the
message.
Fixed unit test broken by .beta in version

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-07-01 19:00:52 -04:00
Derek Collison
637973a1c7 fomatting
Signed-off-by: Derek Collison <derek@nats.io>
2021-06-10 10:04:41 -07:00
Derek Collison
4965a5c494 Merge branch 'master' into active_servers 2021-06-10 09:57:55 -07:00
Matthias Hanel
2caf2303f2 [adding] jetstream info to statsz (#2269)
* [adding] jetstream info to statsz

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-06-10 11:54:56 -04:00
Derek Collison
cc63915f54 Added active servers to statsz.
This is generally useful but will also help with cli actions since we know from first response how many total responses are expected.

Signed-off-by: Derek Collison <derek@nats.io>
2021-06-09 19:14:37 -07:00
Matthias Hanel
230128ed01 [added] filter system requests by JS domain
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-05-07 18:58:40 -04:00
Derek Collison
e438d2f5fa Mixed mode improvements.
1. When in mixed mode and only running the global account we now will check the account for JS.
2. Added code to decrease the cluster set size if we guessed wrong in mixed mode setup.

Signed-off-by: Derek Collison <derek@nats.io>
2021-04-09 14:58:35 -07:00
Derek Collison
c8d812d805 Release server lock when getting usage
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-27 17:38:57 -07:00
Derek Collison
282def10fd Fix data race
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-25 17:07:34 -07:00
Waldemar Quevedo
775afd01e7 Fix for JS reload and exports
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-03-14 15:16:56 -07:00
Derek Collison
bfb8e3432e Move RAFT comms off internal sendq.
Move route and gateway msgs our of fast path for inbound stream msgs.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-04 14:45:34 -08:00
Ivan Kozlovic
0f53bf6580 Fixed data race with nodeInfo
Took the approach of storing struct instead of pointer. Of course,
when changing the offline bool from false to true, it means that
we need to call Store again (with same key).

This is based on the assumption that those Load/Store are not too
frequent. Otherwise, we may need to use locking (and keep *nodeInfo)

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-03-03 13:28:45 -07:00
Derek Collison
2ecf6be3ef Mark raft node as offline when server is removed
Signed-off-by: Derek Collison <derek@nats.io>
2021-03-03 08:06:45 -08:00
Matthias Hanel
c50ee2a1c6 [Changed] all times exposed will be computed in UTC (#1943)
This also applies to times that end up in that json.
Where applicable moved time.Now() to where it is used.
Moved calls to .UTC() to where time is created it that time is converted
later anyway.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-03-02 21:37:42 -05:00
Derek Collison
b9e1a921ff Use internal wildcard inbox for stream and consumer info requests.
More gateway friendly but suffers from no echo, so added new client based internal send.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-28 10:01:01 -08:00
Derek Collison
78bdc34637 General stability improvements. Fixes to subscription state not cleaning up.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-24 08:44:34 -08:00
Derek Collison
6444038660 Make sure this is GW and leafnode friendly
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-23 10:57:10 -08:00
Ivan Kozlovic
61bd1b8d86 MQTT clustering
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-02-19 08:50:00 -07:00
Derek Collison
d803189eca Changes needed to properly support multi-layer service imports for system services like JS.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-12 17:01:32 -08:00
R.I.Pienaar
b36abc0f31 skip responses from non leaders in jsz
Adds an error type and a check to avoid sending them
to clients

Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-02-09 17:34:29 +01:00
Derek Collison
7ae8d66c6d Do ramp down on server stats on startup
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-08 06:46:57 -08:00
Derek Collison
15355d783b For larger superclusters we send out our server information after a short delay on startup.
For determing leaderless make sure the raft node has been running for long enough.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-08 06:26:33 -08:00
Derek Collison
f209c60123 Send our own serverinfo when we receive an update from a new server
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-07 06:53:07 -08:00
Derek Collison
6d32c307ef Remove pretty indent for json.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-06 20:09:44 -08:00
Derek Collison
1622c2b60e Send serverInfo on remote shutdown and remove mapping entries
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-05 17:21:55 -08:00
Matthias Hanel
7b7543d298 [added] jsz nats and http monitoring endpoint for jetstream (#1881)
The new endpoints are /jsz on http and "$SYS.REQ.SERVER.PING.JSZ" and "$SYS.REQ.SERVER.%s.JSZ".
$SYS.REQ.ACCOUNT.%s.JSZ will only return info for the particular account

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-02-05 18:46:04 -05:00
Derek Collison
a1e0f7dc1a First pass at supercluster enablement.
This allows metacontrollers to span superclusters. Also includes placement directives for streams. By default they select the request origin cluster.

Signed-off-by: Derek Collison <derek@nats.io>
2021-02-03 17:28:13 -08:00
Derek Collison
457ca3b9cf Suppress additional advisories on server restart and leadership changes.
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-29 15:08:22 -08:00
Derek Collison
a9b8948abe Add in tracking for quorum in raft and do auto stepdown.
Also added in API responses when no leader is present for meta, streams and consumers.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-27 13:34:00 -08:00
Derek Collison
9c858d197a Added ability to properly restore consumers from a snapshot.
This made us add forwarding proposals functionality in the raft layer.
More general cleanup and bug fixes as well.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-24 19:30:34 -08:00
Derek Collison
da698ca888 Merge branch 'master' into jsc_limits 2021-01-22 12:13:28 -07:00
Derek Collison
6f2b50a374 Added support for clustered account info and limit enforcement
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-21 18:47:21 -08:00
Matthias Hanel
d35cd2996d [added] jwt/issuerkey/nametag/tags to monitoring and event endpoints (#1830)
Also added a trace on jwt authentication

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-01-21 21:16:34 -05:00
Matthias Hanel
9081646109 [added] support for tags and filter ping monitoring requests by tags (#1832)
fixes #1588

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-01-21 21:16:09 -05:00
Derek Collison
9949904abc Fix data race
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-20 11:58:31 -08:00
Derek Collison
7c0b6faf2c We were having issues with the account being changed for the internal system client.
This changes when we are sending internal messages through the shared internal sendq but to a different account.
We will now use an internal client that is only accessible to the send loop.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-20 11:58:31 -08:00
Derek Collison
a1730f1b31 Report on RAFT group information.
This adds in optional reporting to stream and consumer info when running in clsutered mode.

Signed-off-by: Derek Collison <derek@nats.io>
2021-01-20 11:58:31 -08:00
Derek Collison
e4bf3767f2 Only send if we deleted properly
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-17 13:48:40 -08:00
Derek Collison
5479a8e867 Fix for segfault
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-17 13:19:11 -08:00
Ivan Kozlovic
ef38abe75b Fixed gateway reply mapping following changes in JetStream clustering
Those changes are required to maintain backward compatibility.
Since the replies are "_G_.<gateway name hash>.<server ID hash>"
and the hash were 6 characters long, changing to 8 the hash function
would break things.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-01-15 17:32:04 -07:00
Derek Collison
f0cdf89c61 JetStream Clustering WIP
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-14 01:14:52 -08:00
Ivan Kozlovic
4fc04d3f55 Revert changes to processSub()
Based on how the MQTT callback operates, it is safe to finish setup
of the MQTT subscriptions after processSub() returns. So I have
reverted the changes to processSub() which will minimize changes
to non-MQTT related code.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-12-01 15:38:47 -07:00
Ivan Kozlovic
1dba6418ed [ADDED] MQTT Support
This PR introduces native support for MQTT clients. It requires use
of accounts with JetStream enabled. Since as of now clustering is
not available, MQTT will be limited to single instance.

Only QoS 0 and 1 are supported at the moment. MQTT clients can
exchange messages with NATS clients and vice-versa.

Since JetStream is required, accounts with JetStream enabled must
exist in order for an MQTT client to connect to the NATS Server.
The administrator can limit the users that can use MQTT with the
allowed_connection_types option in the user section. For instance:
```
accounts {
  mqtt {
    users [
      {user: all, password: pwd, allowed_connection_types: ["STANDARD", "WEBSOCKET", "MQTT"]}
      {user: mqtt_only, password: pwd, allowed_connection_types: "MQTT"}
    ]
    jetstream: enabled
  }
}
```
The "mqtt_only" can only be used for MQTT connections, which the user
"all" accepts standard, websocket and MQTT clients.

Here is what a configuration to enable MQTT looks like:
```
mqtt {
  # Specify a host and port to listen for websocket connections
  #
  # listen: "host:port"

  # It can also be configured with individual parameters,
  # namely host and port.
  #
  # host: "hostname"
  port: 1883

  # TLS configuration section
  #
  # tls {
  #  cert_file: "/path/to/cert.pem"
  #  key_file: "/path/to/key.pem"
  #  ca_file: "/path/to/ca.pem"
  #
  #  # Time allowed for the TLS handshake to complete
  #  timeout: 2.0
  #
  #  # Takes the user name from the certificate
  #  #
  #  # verify_an_map: true
  #}

  # Authentication override. Here are possible options.
  #
  # authorization {
  #   # Simple username/password
  #   #
  #   user: "some_user_name"
  #   password: "some_password"
  #
  #   # Token. The server will check the MQTT's password in the connect
  #   # protocol against this token.
  #   #
  #   # token: "some_token"
  #
  #   # Time allowed for the client to send the MQTT connect protocol
  #   # after the TCP connection is established.
  #   #
  #   timeout: 2.0
  #}

  # If an MQTT client connects and does not provide a username/password and
  # this option is set, the server will use this client (and therefore account).
  #
  # no_auth_user: "some_user_name"

  # This is the time after which the server will redeliver a QoS 1 message
  # sent to a subscription that has not acknowledged (PUBACK) the message.
  # The default is 30 seconds.
  #
  # ack_wait: "1m"

  # This limits the number of QoS1 messages sent to a session without receiving
  # acknowledgement (PUBACK) from that session. MQTT specification defines
  # a packet identifier as an unsigned int 16, which means that the maximum
  # value is 65535. The default value is 1024.
  #
  # max_ack_pending: 100
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-11-30 20:08:44 -07:00