Commit Graph

259 Commits

Author SHA1 Message Date
Derek Collison
511cdaa5ac Do not report bad latency on auto-unsubscribe triggers
Signed-off-by: Derek Collison <derek@nats.io>
2020-10-22 05:41:33 -07:00
Matthias Hanel
2144f01f21 Adding support and an option for removal of jwt
To enable in full mode configure allow_delete: true
When enabled the file will be renamed to allow for manual restore.

In cache mode it will be enabled by default.
When enabled files will be deleted.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-10-20 16:45:11 -04:00
Derek Collison
4af434e586 Allow mapping destinations to be parsed as singles and not force array syntax
Signed-off-by: Derek Collison <derek@nats.io>
2020-10-17 08:21:02 -07:00
Derek Collison
5f5ceb4668 Allow cluster filters for mappings, changed accountz for mappings
Signed-off-by: Derek Collison <derek@nats.io>
2020-10-16 16:53:14 -07:00
Derek Collison
6faf07d583 Account subject mappings and full wildcard support for exports/imports
The mappings enable traffic shaping functionality.
Also added a subject transform which can match any subject and transform to a new one, even re-ordering partial wildcards.

Signed-off-by: Derek Collison <derek@nats.io>
2020-10-15 10:26:25 -07:00
Matthias Hanel
60c8cbd1f2 Fixed crash on non existing directory
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-10-13 01:30:03 -04:00
Matthias Hanel
8d8f6480d7 Incorporating comments
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-10-12 19:21:32 -04:00
Matthias Hanel
bb63fd5f40 Adding list/delete/update operations for jwt stored by nats-resolver
Update already existed scoped by account, this exposes update without account.
List returns a list of all stored accounts.
Delete deletes accounts.
Fix a crash on startup with non existing directory.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-10-12 18:07:07 -04:00
Ivan Kozlovic
9b6385d6e7 [CHANGED] Default TLS and Auth timeout
The default TLS timeout has been bumped to 2 seconds.
The default Auth timeout is now 2 seconds, but if TLS config
is present, it is equal to TLS timeout (possibly default value) + 1s.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-10-08 16:15:30 -06:00
Ivan Kozlovic
7ccbaca782 Added an allowed connection type filter for users
Users and NKey users will now have the option to specify a list
of allowed connection types.

This will allow for instance a certain user to be allowed to
connect as a standard NATS client, but not as Websocket, or
vice-versa.

This also fixes the websocket auth override. Indeed, with
the original behavior, the websocket users would have been bound
to $G, which would not work when there are accounts defined, since
when that is the case, no app can connect/bind to $G account.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-16 18:22:44 -06:00
Ivan Kozlovic
2ad2bed170 [ADDED] Support for route hostname resolution
We previously simply called DialTimeout() on a route's url when
soliciting. If it resolved to the IP of the host, it would create
a route to self, which server detects, but then would not try again
with other IPs that would have allowed to form a cluster with
other servers running on the other IPs.

This PR keeps track of local IPs + cluster port and exclude them
from the list of IPs returned by LookupHost API. This even prevent
solicitation of routes to self. Only non-local IPs will be tried.

Resolves #1586

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-09-08 13:40:17 -06:00
Waldemar Quevedo
3161cb1693 Add more config checks for leafnode remotes
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2020-09-04 00:34:40 -07:00
Matthias Hanel
e1350a05f3 Emit latency traces when sampling is set to headers
Latency reports will include the header(s) responsible for the trace
Updated ADR to have it reflect implementation

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-08-21 20:01:11 -04:00
Matthias Hanel
48c87c1447 Nats based resolver & avoiding nats account server in smaller deployments (#1550)
* Adding nats based resolver and bootstrap system account

These resolver operate on an exclusive  directory
Two types:
full: managing all jwt in the directory
    Will synchronize with other full resolver
    nats-account-server will also run such a resolver
cache: lru cache managing only a subset of all jwt in the directory
    Will lookup jwt from full resolver
    Can overwrite expiration with a ttl for the file

Both:
    track expiration of jwt and clean up
    Support reload
    Notify the server of changed jwt

Bootstrapping system account allows users signed with the system account
jwt to connect, without the server knowing the jwt.
This allows uploading jwt (including system account) using nats by
publishing to $SYS.ACCOUNT.<name>.CLAIMS.UPDATE
Sending a request, server will respond with the result of the operation.

Receive all jwt stored in one server by sending a
request to $SYS.ACCOUNT.CLAIMS.PACK
One server will respond with a message per stored jwt.
The end of the responses is indicated by an empty message.

The content of dirstore.go and dirstore_test.go was moved from
nats-account-server

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-08-18 13:58:41 -06:00
Ivan Kozlovic
20a67a5be8 Websocket: add option to disable TLS
The new option Websocket.NoTLS would have to be set to true
to disable the server check that enforces TLS configuration.

Resolves #1529

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-29 17:33:02 -06:00
Derek Collison
9514576b72 Merge pull request #1526 from nats-io/no-header-support-option
added 'no_header_support' config option mapping to NoHeaderSupport
2020-07-22 12:13:30 -05:00
Alberto Ricart
2047c791ea added option so clients can test if a server disables header support 2020-07-22 10:11:44 -05:00
Ivan Kozlovic
0ce84a83c5 Added -cluster_name command line option
This is handy for client libraries that start the server as
external executable and pass command line arguments. Without
specifying the cluster name, routes can take time to establish
and cause some tests to fail.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-07-21 15:50:24 -06:00
Pavel Khlebovich
d5eb9ff6f2 Support Cookie JWT auth via WebSocket 2020-06-18 19:27:42 +03:00
Derek Collison
146d8f5dcb Updates based on feedback, sped up some slow tests
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 17:26:43 -07:00
Derek Collison
dd61535e5a Cluster names are now required.
Added cluster names as required for prep work for clustered JetStream. System can dynamically pick a cluster name and settle on one even in large clusters.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 15:48:38 -07:00
Ivan Kozlovic
d18b013676 Merge pull request #1463 from nats-io/websocket_auth_block
Added dedicated auth block for websocket
2020-06-09 16:09:41 -06:00
Ivan Kozlovic
01b14c2abe Added dedicated auth block for websocket
Websocket can now override
- Username/password
- Token
- Users
- NKeys
- no_auth_user
- auth_timeout

For TLS, support for verify and verify_and_map. We used to set
tls config's ClientAuth to NoClientCert. It will now depend
if the config requires client certificate verification, which
is needed if TLSMap is enabled.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-09 11:29:52 -06:00
Derek Collison
bfe52fdb77 Merge pull request #1458 from nats-io/mixed
Allow mixed TLS and non-TLS on same port.
2020-06-08 11:12:45 -07:00
Ivan Kozlovic
cd6d71deaa [ADDED] lame_duck_grace_period option
The grace period used to be hardcoded at 10 seconds.
This option allows the user to configure the amount of time the
server will wait before initiating the closing of client connections.

Note that the grace period needs to be strictly lower than the overall
lame_duck_duration. The server deducts the grace period from that
overall duration and spreads the closing of connections during
that time.
For instance, if there are 1000 connections and the lame duck
duration is set to 30 seconds and grace period to 10, then
the server will use 30-10 = 20 seconds to spread the closing
of those 1000 connections, so say roughly 50 clients per second.

Resolves #1459.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-08 11:43:25 -06:00
Derek Collison
4dee03b587 Allow mixed TLS and non-TLS on same port
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-05 18:04:11 -07:00
Matthias Hanel
0a3e89c64a Incorporating comments
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-06-02 18:38:17 -04:00
Matthias Hanel
cf6fcda75c Added default_permissions to accounts and account jwt
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-06-02 16:06:01 -04:00
aricart
e7590f3065 jwt2 testbed 2020-06-01 18:00:13 -04:00
Derek Collison
2bd7553c71 System Account on by default.
Most of the changes are to turn it off for tests that were watching subscriptions and such.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-29 17:56:45 -07:00
R.I.Pienaar
b7c79f66e4 improve consistency between top level and account js options
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-27 16:22:47 +02:00
Derek Collison
710ef00383 Don't allow JetStream on system account. Warn when accounts configured but no JS
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 12:17:18 -07:00
Ivan Kozlovic
9715848a8e [ADDED] Websocket support
Websocket support can be enabled with a new websocket
configuration block:

```
websocket {
    # Specify a host and port to listen for websocket connections
    # listen: "host:port"

    # It can also be configured with individual parameters,
    # namely host and port.
    # host: "hostname"
    # port: 4443

    # This will optionally specify what host:port for websocket
    # connections to be advertised in the cluster
    # advertise: "host:port"

    # TLS configuration is required
    tls {
      cert_file: "/path/to/cert.pem"
      key_file: "/path/to/key.pem"
    }

    # If same_origin is true, then the Origin header of the
    # client request must match the request's Host.
    # same_origin: true

    # This list specifies the only accepted values for
    # the client's request Origin header. The scheme,
    # host and port must match. By convention, the
    # absence of port for an http:// scheme will be 80,
    # and for https:// will be 443.
    # allowed_origins [
    #    "http://www.example.com"
    #    "https://www.other-example.com"
    # ]

    # This enables support for compressed websocket frames
    # in the server. For compression to be used, both server
    # and client have to support it.
    # compression: true

    # This is the total time allowed for the server to
    # read the client request and write the response back
    # to the client. This include the time needed for the
    # TLS handshake.
    # handshake_timeout: "2s"
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-20 11:14:39 -06:00
Derek Collison
d51566881e First pass at headers awareness for server
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
a7f1bca534 Additional service latency upgrades.
We now share more information about the responder and the requestor. The requestor information by default is not shared, but can be when declaring the import.

Also fixed bug for error handling on old request style requests that would always result on a 408 response.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
ea5e5bd364 Services rewrite #2
This contains a rewrite to the services layer for exporting and importing. The code this merges to already had a first significant rewrite that moved from special interest processing to plain subscriptions.

This code changes the prior version's dealing with reverse mapping which was based mostly on thresholds and manual pruning, with some sporadic timer usage. This version uses the jetstream branch's code that understands interest and failed deliveries. So this code is much more tuned to reacting to interest changes. It also removes thresholds and goes only by interest changes or expirations based around a new service export property, response thresholds. This allows a service provider to provide semantics on how long a response should take at a maximum.

This commit also introduces formal support for service export streamed and chunked response types send an empty message to signify EOF.

This commit also includes additions to the service latency tracking such that errors are now sent, not only successful interactions. We have added a Status field and an optional Error fields to ServiceLatency.

We support the following Status codes, these are directly from HTTP.

400 Bad Request (request did not have a reply subject)
408 Request Timeout (when system detects request interest went away, old request style to make dependable)..
503 Service Unavailable (no service responders running)
504 Service Timeout (The new response threshold expired)

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
b5daac9638 Properly parse and use jetstream server config
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:21:27 -07:00
Derek Collison
ec0bc1dbec First pass account configuration for jetstream
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:21:27 -07:00
Derek Collison
3ab76a6dcd Write performance tweaks
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
R.I.Pienaar
da5425b3da add some more jetstream options
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:16:03 -07:00
R.I.Pienaar
ef77156890 allow jetstream to be enabled in the config file 2020-05-19 14:16:03 -07:00
Derek Collison
47c28b2fb0 JetStream major refactor for name changes.
MsgSet -> Stream
Observable -> Consumer

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
bbdb61a4e1 Add ability to set storage directory
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:15:11 -07:00
Heine Andersen
543b304b3f add long version to toggle on jetstream 2020-05-19 14:15:11 -07:00
Derek Collison
dd116fcfd4 JetStream first pass basics.
This is the first checkin for JetStream. Has some rudimentary basics working.

TODO
1. Push vs pull mode for observables. (work queues)
2. Disk/File store, memory only for now.
3. clustering code - design shaping up well.
4. Finalize account import semantics.
5. Lots of other little things.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:06:29 -07:00
Matthias Hanel
aa1a768cce Move system account check to validateTrustedOperators
As a consequence of this change, certain unit tests had to actually
start the server and move to a memory resolver to keep the test simpler.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-05-19 15:39:18 -04:00
Matthias Hanel
e509ec59a1 Raise error when system_account in config and operator jwt do not match
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-05-18 23:35:43 -04:00
Matthias Hanel
a2744858bc Looking up system_account from operator jwt
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-05-18 19:09:46 -04:00
Guilherme Santos
25858cba0b Implement basePath for monitoring endpoints 2020-05-13 23:29:11 +02:00
Matthias Hanel
04b81abdde [FIXED] default_permissions apply to nkey users as well
Fixes 1390

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-05-12 17:13:25 -04:00