Commit Graph

369 Commits

Author SHA1 Message Date
Derek Collison
6c805eebc7 Properly support leadnode clusters.
Leafnodes that formed clusters were partially supported. This adds proper support for origin cluster, subscription suppression and data message no echo for the origin cluster.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-26 09:03:22 -07:00
Matthias Hanel
d086a39b64 Add filtering by name and cluster to PING events
On cluster name change, reset internalSendLoop so it picks up the
changed name.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-06-16 18:26:35 -04:00
Derek Collison
2b9e3e5b15 Merge pull request #1476 from nats-io/cluster_name
Cluster names are now required.
2020-06-15 10:07:30 -07:00
Derek Collison
1e52a1007b More updates based on feedback
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-13 08:00:57 -07:00
Derek Collison
146d8f5dcb Updates based on feedback, sped up some slow tests
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 17:26:43 -07:00
Derek Collison
dd61535e5a Cluster names are now required.
Added cluster names as required for prep work for clustered JetStream. System can dynamically pick a cluster name and settle on one even in large clusters.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-12 15:48:38 -07:00
Ivan Kozlovic
67d2638859 [ADDED] Print the config file being used in startup banner
Resolves #1451

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-12 12:21:50 -06:00
Ivan Kozlovic
d2a8282a0d [FIXED] LeafNode TLSMap and websocket auth override
We added authentication override block for websocket configuration
in PR #1463 and #1465 which somehow introduced a drop in perf as
reported by the bench tests.
This PR refactors a bit to restore the performance numbers.

This change also fixes the override behavior for websocket auth:
- If websocket's NoAuthUser is configured, the websocket's auth
  block MUST define Users, and the user be present.
- If there is any override (username/pwd,token,etc..) then the
  whole block config will be used when authenticating a websocket
  client, which means that if websocket NoAuthUser is empty we
  are not falling back to the regular client's NoAuthUser config.
- TLSMap always override the regular client's config. That is,
  whatever TLSMap value specified in the websocket's tls{} block
  will be used.

The TLSMap configuration was not used for LeafNodes. The behavior
now will be:
- If LeafNode's auth block contains users and TLSMap is true,
  the user is looked up based on the cert's info. If not found,
  authentication will fail. If found, it will be authenticated
  and bound to associated account.
- If no user is specified in LeafNode's auth block and TLSMap
  is true, then the cert's info will be used against the global
  users map.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-11 17:06:54 -06:00
Ivan Kozlovic
d18b013676 Merge pull request #1463 from nats-io/websocket_auth_block
Added dedicated auth block for websocket
2020-06-09 16:09:41 -06:00
Ivan Kozlovic
01b14c2abe Added dedicated auth block for websocket
Websocket can now override
- Username/password
- Token
- Users
- NKeys
- no_auth_user
- auth_timeout

For TLS, support for verify and verify_and_map. We used to set
tls config's ClientAuth to NoClientCert. It will now depend
if the config requires client certificate verification, which
is needed if TLSMap is enabled.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-09 11:29:52 -06:00
Derek Collison
bfe52fdb77 Merge pull request #1458 from nats-io/mixed
Allow mixed TLS and non-TLS on same port.
2020-06-08 11:12:45 -07:00
Ivan Kozlovic
cd6d71deaa [ADDED] lame_duck_grace_period option
The grace period used to be hardcoded at 10 seconds.
This option allows the user to configure the amount of time the
server will wait before initiating the closing of client connections.

Note that the grace period needs to be strictly lower than the overall
lame_duck_duration. The server deducts the grace period from that
overall duration and spreads the closing of connections during
that time.
For instance, if there are 1000 connections and the lame duck
duration is set to 30 seconds and grace period to 10, then
the server will use 30-10 = 20 seconds to spread the closing
of those 1000 connections, so say roughly 50 clients per second.

Resolves #1459.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-08 11:43:25 -06:00
Derek Collison
3a4e835311 Key off TLS first byte
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-06 13:41:34 -07:00
Derek Collison
75395ed862 Read 4 bytes, wait TLS timeout
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-05 18:38:20 -07:00
Derek Collison
4dee03b587 Allow mixed TLS and non-TLS on same port
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-05 18:04:11 -07:00
Ivan Kozlovic
98ea70a590 LameDuckMode takes into account websocket accept loop
This is related to #1408.
Make sure that we close the websocket "accept loop" if configured
before proceeding with the lame duck mode.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-06-02 17:49:38 -06:00
aricart
e7590f3065 jwt2 testbed 2020-06-01 18:00:13 -04:00
Derek Collison
05e38ae527 Merge branch 'master' into sys-acc 2020-06-01 11:53:14 -07:00
Derek Collison
eca04c6fce First pass header support for JetStream
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-30 10:04:23 -07:00
Derek Collison
2bd7553c71 System Account on by default.
Most of the changes are to turn it off for tests that were watching subscriptions and such.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-29 17:56:45 -07:00
Ivan Kozlovic
762bc84daf Merge pull request #1408 from nats-io/ldm_send_info
[FIXED] LameDuckMode sends INFO to clients
2020-05-26 11:37:59 -06:00
Derek Collison
710ef00383 Don't allow JetStream on system account. Warn when accounts configured but no JS
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 12:17:18 -07:00
Ivan Kozlovic
8678a61e3e Move the send of INFO after client listener has been shutdown
This will ensure that there is no race where clients are accepted
after the LDM INFO notification.

Also add to the test to make sure that we don't send INFO when
routes are disconnected due to internal closing of connections
during the shutdown process.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-24 11:38:49 -06:00
Ivan Kozlovic
dc0f688cbf [FIXED] LameDuckMode sends INFO to clients
Also send an INFO to routes so that the remotes can remove the
LDM's server client URLs and notify their own clients of this
change.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-21 12:15:20 -06:00
Ivan Kozlovic
9715848a8e [ADDED] Websocket support
Websocket support can be enabled with a new websocket
configuration block:

```
websocket {
    # Specify a host and port to listen for websocket connections
    # listen: "host:port"

    # It can also be configured with individual parameters,
    # namely host and port.
    # host: "hostname"
    # port: 4443

    # This will optionally specify what host:port for websocket
    # connections to be advertised in the cluster
    # advertise: "host:port"

    # TLS configuration is required
    tls {
      cert_file: "/path/to/cert.pem"
      key_file: "/path/to/key.pem"
    }

    # If same_origin is true, then the Origin header of the
    # client request must match the request's Host.
    # same_origin: true

    # This list specifies the only accepted values for
    # the client's request Origin header. The scheme,
    # host and port must match. By convention, the
    # absence of port for an http:// scheme will be 80,
    # and for https:// will be 443.
    # allowed_origins [
    #    "http://www.example.com"
    #    "https://www.other-example.com"
    # ]

    # This enables support for compressed websocket frames
    # in the server. For compression to be used, both server
    # and client have to support it.
    # compression: true

    # This is the total time allowed for the server to
    # read the client request and write the response back
    # to the client. This include the time needed for the
    # TLS handshake.
    # handshake_timeout: "2s"
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-20 11:14:39 -06:00
Derek Collison
050db531fc Bumping version
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 16:44:42 -07:00
Derek Collison
c9f78d6f79 Fixes post rebasing with master
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 16:38:19 -07:00
Derek Collison
d8b475c4b0 Better way to retrieve header support
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
55c77d1e4e Added support for delivery of HMSG and support for older clients
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
d51566881e First pass at headers awareness for server
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
17aca11002 Small changes to event ids, good approach though with separate lock on account
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
ea5e5bd364 Services rewrite #2
This contains a rewrite to the services layer for exporting and importing. The code this merges to already had a first significant rewrite that moved from special interest processing to plain subscriptions.

This code changes the prior version's dealing with reverse mapping which was based mostly on thresholds and manual pruning, with some sporadic timer usage. This version uses the jetstream branch's code that understands interest and failed deliveries. So this code is much more tuned to reacting to interest changes. It also removes thresholds and goes only by interest changes or expirations based around a new service export property, response thresholds. This allows a service provider to provide semantics on how long a response should take at a maximum.

This commit also introduces formal support for service export streamed and chunked response types send an empty message to signify EOF.

This commit also includes additions to the service latency tracking such that errors are now sent, not only successful interactions. We have added a Status field and an optional Error fields to ServiceLatency.

We support the following Status codes, these are directly from HTTP.

400 Bad Request (request did not have a reply subject)
408 Request Timeout (when system detects request interest went away, old request style to make dependable)..
503 Service Unavailable (no service responders running)
504 Service Timeout (The new response threshold expired)

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
R.I.Pienaar
78fdeb661d move events nuid to the server struct 2020-05-19 14:24:31 -07:00
R.I.Pienaar
3182db4c3a move to events having Type not Schema 2020-05-19 14:22:53 -07:00
Derek Collison
b5daac9638 Properly parse and use jetstream server config
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:21:27 -07:00
Derek Collison
ec0bc1dbec First pass account configuration for jetstream
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:21:27 -07:00
Derek Collison
df774e44b0 Rework how service imports are handled to avoid performance hits
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
R.I.Pienaar
fc6d8826f5 show basic jetstream info in varz and server info 2020-05-19 14:16:03 -07:00
R.I.Pienaar
da5425b3da add some more jetstream options
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:16:03 -07:00
Derek Collison
d5aa55a103 Audit advisories, work based off of #1253 from ripienaar
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
bbdb61a4e1 Add ability to set storage directory
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:15:11 -07:00
Derek Collison
c650b1bca3 API fix and rebase fix
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:15:11 -07:00
Derek Collison
ec2161ad53 State recovery and server shutdown
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:12:28 -07:00
Derek Collison
75908f80a4 API cleanup
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:07:02 -07:00
Derek Collison
b7b98df4ee Server limits and account reservations
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:07:02 -07:00
Derek Collison
dd116fcfd4 JetStream first pass basics.
This is the first checkin for JetStream. Has some rudimentary basics working.

TODO
1. Push vs pull mode for observables. (work queues)
2. Disk/File store, memory only for now.
3. clustering code - design shaping up well.
4. Finalize account import semantics.
5. Lots of other little things.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:06:29 -07:00
Ivan Kozlovic
54e014070f Merge pull request #1392 from guilherme-santos/master
[ADDED] base path for monitoring endpoints
2020-05-13 16:28:57 -06:00
Guilherme Santos
25858cba0b Implement basePath for monitoring endpoints 2020-05-13 23:29:11 +02:00
Matthias Hanel
d486f6ab9b Move reset of internal client to after the account sublist was moved.
This does not avoid the race condition, but makes it less likely to
trigger in unit tests.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-05-13 15:52:29 -04:00
Matthias Hanel
11c0669ae2 [FIXES] Unnecessary account reloads and pointer to old accounts
Fixes #1372 by updating s.sys.account pointer.

This issue also showed that accounts are unnecessarily reloaded.
This happened because account imports were not copied and thus,
deepEqual detected a difference were none was.
This was addressed by making the copy less shallow.

Furthermore did deepEqual detects a difference when it compared
slices that were appended to while processing a map.
This was fixed by sorting before comparison.

Noticed that Account.clients stored an unnecessary pointer.
Removed duplicated code in systemAccount.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-05-11 21:51:41 -04:00