Commit Graph

193 Commits

Author SHA1 Message Date
Derek Collison
4f7fbefc7c In clustered JetStream we need to move API calls out of routes/gateways/leafnodes path.
This moves from explicit imports and subscriptions to one wildcard subscription and a single wildcard export.

Signed-off-by: Derek Collison <derek@nats.io>
2021-03-02 17:54:41 -08:00
R.I.Pienaar
369f604589 track per server stats rather than cluster stats
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-02-11 16:20:14 +01:00
R.I.Pienaar
b36abc0f31 skip responses from non leaders in jsz
Adds an error type and a check to avoid sending them
to clients

Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-02-09 17:34:29 +01:00
Matthias Hanel
c67f1526e4 fix jsz when server is not started in cluster mode and fix landing page (#1901)
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-02-08 21:10:20 -05:00
Matthias Hanel
10154c5388 [added] system_account to varz/accounts and is_system to accountz (#1898)
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-02-08 15:58:53 -05:00
Derek Collison
b25055bbf6 Added leader only response for jsz.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-08 10:40:33 -08:00
Derek Collison
c16f6e193d Move JetStream direct APIs to private.
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-07 15:19:22 -08:00
Matthias Hanel
7b7543d298 [added] jsz nats and http monitoring endpoint for jetstream (#1881)
The new endpoints are /jsz on http and "$SYS.REQ.SERVER.PING.JSZ" and "$SYS.REQ.SERVER.%s.JSZ".
$SYS.REQ.ACCOUNT.%s.JSZ will only return info for the particular account

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-02-05 18:46:04 -05:00
Derek Collison
c9e1cbd571 Modified jetstream varz information
Signed-off-by: Derek Collison <derek@nats.io>
2021-02-02 08:35:35 -08:00
Matthias Hanel
f487429d9e incorporated comments
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-01-29 13:25:02 -05:00
Matthias Hanel
2a34f0daee [added] field to varz output containing the operator jwt/claim
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-01-29 12:32:40 -05:00
Matthias Hanel
c9e0eb6c3a [added] cluster/gateway/leafnode tls required/verify/timeout to varz (#1854)
Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-01-28 14:08:58 -05:00
Matthias Hanel
d35cd2996d [added] jwt/issuerkey/nametag/tags to monitoring and event endpoints (#1830)
Also added a trace on jwt authentication

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-01-21 21:16:34 -05:00
Matthias Hanel
9081646109 [added] support for tags and filter ping monitoring requests by tags (#1832)
fixes #1588

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-01-21 21:16:09 -05:00
Derek Collison
37cf7584bd Merge branch 'master' into jsc 2021-01-14 02:52:35 -07:00
Derek Collison
f0cdf89c61 JetStream Clustering WIP
Signed-off-by: Derek Collison <derek@nats.io>
2021-01-14 01:14:52 -08:00
Ivan Kozlovic
f50c655e75 [FIXED] Monitoring endpoint connz?auth=true show incorrect user
Only the user (from username/password connection method) was reported
in this monitoring endpoint. Will now report proper nkey, public key,
etc..

Resolves #1799

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-01-11 12:59:05 -07:00
Matthias Hanel
c6daffbfcc [Added] ability to use jwt latency sampling properties headers/share
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-12-16 14:34:09 -05:00
Ivan Kozlovic
1dba6418ed [ADDED] MQTT Support
This PR introduces native support for MQTT clients. It requires use
of accounts with JetStream enabled. Since as of now clustering is
not available, MQTT will be limited to single instance.

Only QoS 0 and 1 are supported at the moment. MQTT clients can
exchange messages with NATS clients and vice-versa.

Since JetStream is required, accounts with JetStream enabled must
exist in order for an MQTT client to connect to the NATS Server.
The administrator can limit the users that can use MQTT with the
allowed_connection_types option in the user section. For instance:
```
accounts {
  mqtt {
    users [
      {user: all, password: pwd, allowed_connection_types: ["STANDARD", "WEBSOCKET", "MQTT"]}
      {user: mqtt_only, password: pwd, allowed_connection_types: "MQTT"}
    ]
    jetstream: enabled
  }
}
```
The "mqtt_only" can only be used for MQTT connections, which the user
"all" accepts standard, websocket and MQTT clients.

Here is what a configuration to enable MQTT looks like:
```
mqtt {
  # Specify a host and port to listen for websocket connections
  #
  # listen: "host:port"

  # It can also be configured with individual parameters,
  # namely host and port.
  #
  # host: "hostname"
  port: 1883

  # TLS configuration section
  #
  # tls {
  #  cert_file: "/path/to/cert.pem"
  #  key_file: "/path/to/key.pem"
  #  ca_file: "/path/to/ca.pem"
  #
  #  # Time allowed for the TLS handshake to complete
  #  timeout: 2.0
  #
  #  # Takes the user name from the certificate
  #  #
  #  # verify_an_map: true
  #}

  # Authentication override. Here are possible options.
  #
  # authorization {
  #   # Simple username/password
  #   #
  #   user: "some_user_name"
  #   password: "some_password"
  #
  #   # Token. The server will check the MQTT's password in the connect
  #   # protocol against this token.
  #   #
  #   # token: "some_token"
  #
  #   # Time allowed for the client to send the MQTT connect protocol
  #   # after the TCP connection is established.
  #   #
  #   timeout: 2.0
  #}

  # If an MQTT client connects and does not provide a username/password and
  # this option is set, the server will use this client (and therefore account).
  #
  # no_auth_user: "some_user_name"

  # This is the time after which the server will redeliver a QoS 1 message
  # sent to a subscription that has not acknowledged (PUBACK) the message.
  # The default is 30 seconds.
  #
  # ack_wait: "1m"

  # This limits the number of QoS1 messages sent to a session without receiving
  # acknowledgement (PUBACK) from that session. MQTT specification defines
  # a packet identifier as an unsigned int 16, which means that the maximum
  # value is 65535. The default value is 1024.
  #
  # max_ack_pending: 100
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-11-30 20:08:44 -07:00
Ivan Kozlovic
f155c75da7 [FIXED] LeafNode reject duplicate remote
There was a test to prevent an errorneous loop detection when a
remote would reconnect (due to a stale connection) while the accepting
side did not detect the bad connection yet.

However, this test was racy because the test was done prior to add
the connections to the map.

In the case of a misconfiguration where the remote creates 2 different
remote connections that end-up binding to the same account in the
accepting side, then it was possible that this would not be detected.
And when it was, the remote side would be unaware since the disconnect/
reconnect attempts would not show up if not running in debug mode.

This change makes sure that the detection is no longer racy and returns
an error to the remote so at least the log/console of the remote will
show the "duplicate connection" error messages.

Resolves #1730

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-11-23 13:28:18 -07:00
Matthias Hanel
3b4da21e07 Rename reject_unknown to reject_unknown_cluster
this is supposed to make it clearer what is being rejected

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-11-17 17:54:45 -05:00
Matthias Hanel
c907245bfc Adding more debugging information to accountz
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-10-21 18:29:22 -04:00
Matthias Hanel
243361ac9a Incorporating readability suggestions
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-10-19 17:33:02 -04:00
Matthias Hanel
5a5dbb9dcd Fix nil deref when account is in bad/incomplete state
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-10-19 16:38:42 -04:00
Derek Collison
923335e52d Merge pull request #1644 from nats-io/map
Traffic shaping and full wildcard support for streams and services exports/imports
2020-10-17 16:19:15 -07:00
Derek Collison
5f5ceb4668 Allow cluster filters for mappings, changed accountz for mappings
Signed-off-by: Derek Collison <derek@nats.io>
2020-10-16 16:53:14 -07:00
Derek Collison
d0201d8b15 Added support for mappings in Accountz
Signed-off-by: Derek Collison <derek@nats.io>
2020-10-15 19:14:58 -07:00
Derek Collison
6faf07d583 Account subject mappings and full wildcard support for exports/imports
The mappings enable traffic shaping functionality.
Also added a subject transform which can match any subject and transform to a new one, even re-ordering partial wildcards.

Signed-off-by: Derek Collison <derek@nats.io>
2020-10-15 10:26:25 -07:00
R.I.Pienaar
f2106ab9e4 fix npe for offsets larger than items
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-10-15 11:27:31 +02:00
Matthias Hanel
371861ec59 Holding lock when accessing leaf node account for filtering
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-09-24 18:57:21 -04:00
Matthias Hanel
d501a811b8 [Added] filtering by account to leafz and exposing this as per acc subj
On the monitoring endpoint /leafz specify ?acc=<account id>

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-09-24 17:23:36 -04:00
Matthias Hanel
7a8a7a7234 Incorporating review comments 2020-09-23 18:29:44 -04:00
Matthias Hanel
634ce9f7c8 [Adding] Accountz monitoring endpoint and INFO monitoring req subject
Returned imports/exports are formated like jwt exports imports, even if
they originating account is from config.

Fixes #1604

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-09-23 16:22:48 -04:00
David Simner
a4b13c0f8b Change nats.io resources from HTTP to HTTPS 2020-09-15 15:23:04 +02:00
bfoxstudio
ae2411c013 fix absolute url in monitoring menu 2020-08-10 19:40:36 +03:00
Derek Collison
6c805eebc7 Properly support leadnode clusters.
Leafnodes that formed clusters were partially supported. This adds proper support for origin cluster, subscription suppression and data message no echo for the origin cluster.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-26 09:03:22 -07:00
Matthias Hanel
d086a39b64 Add filtering by name and cluster to PING events
On cluster name change, reset internalSendLoop so it picks up the
changed name.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-06-16 18:26:35 -04:00
Derek Collison
b91c8879ad Consolidate subDetail
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-05 08:37:18 -07:00
Derek Collison
c1ffd48638 Add account details to subsz.
Also allow ability to filter based on account.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-05 05:53:01 -07:00
Ivan Kozlovic
9715848a8e [ADDED] Websocket support
Websocket support can be enabled with a new websocket
configuration block:

```
websocket {
    # Specify a host and port to listen for websocket connections
    # listen: "host:port"

    # It can also be configured with individual parameters,
    # namely host and port.
    # host: "hostname"
    # port: 4443

    # This will optionally specify what host:port for websocket
    # connections to be advertised in the cluster
    # advertise: "host:port"

    # TLS configuration is required
    tls {
      cert_file: "/path/to/cert.pem"
      key_file: "/path/to/key.pem"
    }

    # If same_origin is true, then the Origin header of the
    # client request must match the request's Host.
    # same_origin: true

    # This list specifies the only accepted values for
    # the client's request Origin header. The scheme,
    # host and port must match. By convention, the
    # absence of port for an http:// scheme will be 80,
    # and for https:// will be 443.
    # allowed_origins [
    #    "http://www.example.com"
    #    "https://www.other-example.com"
    # ]

    # This enables support for compressed websocket frames
    # in the server. For compression to be used, both server
    # and client have to support it.
    # compression: true

    # This is the total time allowed for the server to
    # read the client request and write the response back
    # to the client. This include the time needed for the
    # TLS handshake.
    # handshake_timeout: "2s"
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-20 11:14:39 -06:00
Derek Collison
d51566881e First pass at headers awareness for server
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
ea5e5bd364 Services rewrite #2
This contains a rewrite to the services layer for exporting and importing. The code this merges to already had a first significant rewrite that moved from special interest processing to plain subscriptions.

This code changes the prior version's dealing with reverse mapping which was based mostly on thresholds and manual pruning, with some sporadic timer usage. This version uses the jetstream branch's code that understands interest and failed deliveries. So this code is much more tuned to reacting to interest changes. It also removes thresholds and goes only by interest changes or expirations based around a new service export property, response thresholds. This allows a service provider to provide semantics on how long a response should take at a maximum.

This commit also introduces formal support for service export streamed and chunked response types send an empty message to signify EOF.

This commit also includes additions to the service latency tracking such that errors are now sent, not only successful interactions. We have added a Status field and an optional Error fields to ServiceLatency.

We support the following Status codes, these are directly from HTTP.

400 Bad Request (request did not have a reply subject)
408 Request Timeout (when system detects request interest went away, old request style to make dependable)..
503 Service Unavailable (no service responders running)
504 Service Timeout (The new response threshold expired)

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
R.I.Pienaar
fc6d8826f5 show basic jetstream info in varz and server info 2020-05-19 14:16:03 -07:00
Guilherme Santos
25858cba0b Implement basePath for monitoring endpoints 2020-05-13 23:29:11 +02:00
Ivan Kozlovic
81fabde729 Merge pull request #1362 from nats-io/monitoring_as_systemevents
[ADDED] Making monitoring endpoints available via system services.
2020-05-07 11:37:50 -06:00
Matthias Hanel
136feb9bc6 [FIXEd] subsz monitoring endpoint did not account for accounts.
Fixes  #1371 and #1357 by adding up stats and collecting subscriptions
from all accounts.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-05-06 15:48:51 -04:00
Matthias Hanel
14c716052d Making monitoring endpoints available via system services.
Available via $SYS.REQ.SERVER.%s.%s and $SYS.REQ.SERVER.PING.%s
Last token is the endpoint name.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-05-04 13:31:50 -04:00
Ivan Kozlovic
e63fc5f195 Merge pull request #1318 from nats-io/monitoring
[ADDED] Option to include subscription details in monitoring responses
2020-03-30 15:51:51 -06:00
Matthias Hanel
30ba333663 Adding an option to include subscription details in monitoring responses.
Applies to routez and connz and closed connections.
Enable by specifying subs=detail

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-03-23 12:25:51 -04:00
Matthias Hanel
b0ded55b43 Make sure server lock is held while accessing server.gacc
Fixes #1314 by:
There was a data race with a write during reloadAuthorization.
Locking was added to all places where it was missing.
In situations were it appeared feasible, access was moved into existing
lock/unlock.
Where it was added, the lock order was already established.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-03-17 14:23:23 -04:00