Commit Graph

427 Commits

Author SHA1 Message Date
Ivan Kozlovic
1e149f4041 Merge pull request #1440 from nats-io/jwt2
Update imports for jwt/v2
2020-06-02 11:10:21 -06:00
Derek Collison
b5dfb984e9 Fixes for race detections under GHA
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-01 18:34:18 -07:00
aricart
e7590f3065 jwt2 testbed 2020-06-01 18:00:13 -04:00
Ivan Kozlovic
e9805a3109 [FIXED] Possible removal of interest on queue subs with leaf nodes
Server was incorrectly processing a queue subscription removal
as both a plain sub and queue sub, which may have resulted in
drop of interest even when some queue subs remained.

Resolves #1421

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-28 10:21:51 -06:00
Derek Collison
fa59cff105 Add in snapshot and restore JSApi
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-27 20:01:30 -07:00
Ivan Kozlovic
dc0f688cbf [FIXED] LameDuckMode sends INFO to clients
Also send an INFO to routes so that the remotes can remove the
LDM's server client URLs and notify their own clients of this
change.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-21 12:15:20 -06:00
Ivan Kozlovic
9715848a8e [ADDED] Websocket support
Websocket support can be enabled with a new websocket
configuration block:

```
websocket {
    # Specify a host and port to listen for websocket connections
    # listen: "host:port"

    # It can also be configured with individual parameters,
    # namely host and port.
    # host: "hostname"
    # port: 4443

    # This will optionally specify what host:port for websocket
    # connections to be advertised in the cluster
    # advertise: "host:port"

    # TLS configuration is required
    tls {
      cert_file: "/path/to/cert.pem"
      key_file: "/path/to/key.pem"
    }

    # If same_origin is true, then the Origin header of the
    # client request must match the request's Host.
    # same_origin: true

    # This list specifies the only accepted values for
    # the client's request Origin header. The scheme,
    # host and port must match. By convention, the
    # absence of port for an http:// scheme will be 80,
    # and for https:// will be 443.
    # allowed_origins [
    #    "http://www.example.com"
    #    "https://www.other-example.com"
    # ]

    # This enables support for compressed websocket frames
    # in the server. For compression to be used, both server
    # and client have to support it.
    # compression: true

    # This is the total time allowed for the server to
    # read the client request and write the response back
    # to the client. This include the time needed for the
    # TLS handshake.
    # handshake_timeout: "2s"
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-20 11:14:39 -06:00
Derek Collison
c9f78d6f79 Fixes post rebasing with master
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 16:38:19 -07:00
Derek Collison
99d1e56aac Don't send updates to leafnodes before all subs on init
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:56 -07:00
Derek Collison
019c105ca7 Updates based on feedback, more tests, few bug fixes
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
f5ceab339a Server support for headers between routes
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
55c77d1e4e Added support for delivery of HMSG and support for older clients
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
d61f1f5d92 Add in support for client header bool in CONNECT and tests
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
d51566881e First pass at headers awareness for server
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
9a702c2bc7 Removed duplicate init
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
cadd39a01c Major rewrite for NATS JetStream API
API made more consistent. Noun followed by verb.
Name arguments in request subejcts are always at the end now.
Remove enabled call, just use account info.
Getting a message directly from a stream is treated like an admin API and requires JSON request.
Deleting a message directly as well.
StreamList and ConsumerList now include details and support paging.
Streams and Consumers now contain a created field in their info.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
911e7ef35d Add additional fields to client info for latency
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
a7f1bca534 Additional service latency upgrades.
We now share more information about the responder and the requestor. The requestor information by default is not shared, but can be when declaring the import.

Also fixed bug for error handling on old request style requests that would always result on a 408 response.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
b8c04c1abf Make all start times UTC for latency tracking
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
ea5e5bd364 Services rewrite #2
This contains a rewrite to the services layer for exporting and importing. The code this merges to already had a first significant rewrite that moved from special interest processing to plain subscriptions.

This code changes the prior version's dealing with reverse mapping which was based mostly on thresholds and manual pruning, with some sporadic timer usage. This version uses the jetstream branch's code that understands interest and failed deliveries. So this code is much more tuned to reacting to interest changes. It also removes thresholds and goes only by interest changes or expirations based around a new service export property, response thresholds. This allows a service provider to provide semantics on how long a response should take at a maximum.

This commit also introduces formal support for service export streamed and chunked response types send an empty message to signify EOF.

This commit also includes additions to the service latency tracking such that errors are now sent, not only successful interactions. We have added a Status field and an optional Error fields to ServiceLatency.

We support the following Status codes, these are directly from HTTP.

400 Bad Request (request did not have a reply subject)
408 Request Timeout (when system detects request interest went away, old request style to make dependable)..
503 Service Unavailable (no service responders running)
504 Service Timeout (The new response threshold expired)

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
df774e44b0 Rework how service imports are handled to avoid performance hits
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
3d80ff71a4 Fix for queue subscribers as consumers
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
R.I.Pienaar
10a419ca3d support jetstream client kind in a few places 2020-05-19 14:16:03 -07:00
Derek Collison
8d1f3cc7c2 Allow JetStream consumers to work across multi-server hops
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
72cda966f4 Changed API subjects to be authorization friendly
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
34ce97bb8c Added support for wildcards for service imports
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
2237915c97 Added in template support
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
d5676e502c Add ability to use remote servers
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:15:11 -07:00
Derek Collison
0c2d539b06 Remote request API
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:13:22 -07:00
Derek Collison
98bd8234b6 Rework of delivery sequence vs store sequence and ack reply subjects
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:07:02 -07:00
Derek Collison
0fb7ee32bc Auto-expiration of ephemeral push based observables
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:07:02 -07:00
Derek Collison
dd116fcfd4 JetStream first pass basics.
This is the first checkin for JetStream. Has some rudimentary basics working.

TODO
1. Push vs pull mode for observables. (work queues)
2. Disk/File store, memory only for now.
3. clustering code - design shaping up well.
4. Finalize account import semantics.
5. Lots of other little things.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:06:29 -07:00
Ivan Kozlovic
2ec00d86ed Replaced %v with %s so String() is not needed
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-04-15 17:52:44 -06:00
Ivan Kozlovic
be00ea96cf [IMPROVED] Added close reason in the connection close log statement
This gives the close reason directly in the log without having to
get that information from the monitoring endpoint. Here is an
example of a route closed due to the remote side not replying to
PINGs:

```
[INF] 127.0.0.1:53839 - rid:2 - Router connection closed: Stale Connection
```

Without this change, the log statement would have been:
```
[INF] 127.0.0.1:53839 - rid:2 - Router connection closed
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-04-15 15:36:54 -06:00
Ivan Kozlovic
7218da1cbc Merge pull request #1338 from nats-io/reduce_loop_errors
LeafNode: delay connect even when loop detected by accepting side
2020-04-10 18:04:11 -06:00
Ivan Kozlovic
b200368e52 LeafNode: delay connect even when loop detected by accepting side
If the loop is detected by a server accepting the leafnode connection,
an error is sent back and connection is closed.
This change ensures that the server checks an -ERR for "Loop detected"
and then set the connect delay, so that it does not try to reconnect
right away.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-04-10 16:44:16 -06:00
Derek Collison
090abc939d Fix for stream imports and leafnodes, #1332
Signed-off-by: Derek Collison <derek@nats.io>
2020-04-10 10:36:20 -07:00
Ivan Kozlovic
34eb5bda31 [ADDED] Deny import/export options for LeafNode remote configuration
This will allow a leafnode remote connection to prevent unwanted
messages to be received, or prevent local messages to be sent
to the remote server.

Configuration will be something like:
```
leafnodes {
  remotes: [
    {
      url: "nats://localhost:6222"
      deny_imports: ["foo.*", "bar"]
      deny_exports: ["baz.*", "bat"]
    }
  ]
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-04-09 18:55:44 -06:00
Ivan Kozlovic
76e8e1c9b0 [ADDED] Leafnode remote's Hub option
This allows a node that creates a remote LeafNode connection to
act as it was the hub (of the hub and spoke topology). This is
related to subscription interest propagation. Normally, a spoke
(the one creating the remote LN connection) will forward only
its local subscriptions and when receiving subscription interest
would not try to forward to local cluster and/or gateways.
If a remote has the Hub boolean set to true, even though the
node is the one creating the remote LN connection, it will behave
as if it was accepting that connection.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-04-07 13:42:55 -06:00
Matthias Hanel
4aede98bcc Turn if into assignment and minimize changes by capturing s.gacc.
Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-03-17 14:56:34 -04:00
Matthias Hanel
b0ded55b43 Make sure server lock is held while accessing server.gacc
Fixes #1314 by:
There was a data race with a write during reloadAuthorization.
Locking was added to all places where it was missing.
In situations were it appeared feasible, access was moved into existing
lock/unlock.
Where it was added, the lock order was already established.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-03-17 14:23:23 -04:00
Ivan Kozlovic
cbc0e5848a Merge pull request #1300 from nats-io/reload
[FIXED] trace/debug/sys_log reload will affect existing clients
2020-03-09 09:48:24 -06:00
Matthias Hanel
6a1c3fc29b Moving inbound tracing to the caller (client.parse)
Tracing for outgoing operations is always done while
holding the client lock.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-03-04 17:31:18 -05:00
Matthias Hanel
fe373ac597 Incorporating comments.
c -> client
defer in oneliner
argument order

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-03-04 15:48:19 -05:00
Matthias Hanel
f5bd07b36c [FIXED] trace/debug/sys_log reload will affect existing clients
Fixed #1296, by altering client state on reload

Detect a trace level change on reload and update all clients.
To avoid data races, read client.trace while holding the lock,
pass the value into functionis that trace while not holding the lock.
Delete unused client.debug.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-03-04 13:54:15 -05:00
Ivan Kozlovic
6f3418687b Capture original length of the first slice and updated test
Changed test to make the previous code in flushOutbound fail.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-03-03 19:12:42 -07:00
Ivan Kozlovic
fd8539f15f [FIXED] Incorrect buffer reuse in case of partial connection write
Added a test that demonstrates the issue and a proposed fix.

Also decrement c.out.pb if closing due to max pending limit.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-03-03 16:50:03 -07:00
Matthias Hanel
a8e6af30a3 On client connect, send first ping after ping interval.
On connect message resend reset timer with setFirstPingTimer, so RTT can
be obtained quicker.

Disable short first ping in default server options for client_test.
In log_test prevent immediate scheduling by setting ping interval.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-03-02 20:10:15 -05:00
Ivan Kozlovic
34e4c64e50 Merge pull request #1295 from nats-io/sys-trace
Adding option to enable tracing the system account. (default: false)
2020-03-02 16:48:55 -07:00
Ivan Kozlovic
fcb6dfe9a7 Merge pull request #1286 from nats-io/error-suggestion
Suggestion to improve tracing in nats-server through wrapping errors
2020-03-02 16:44:24 -07:00