Commit Graph

581 Commits

Author SHA1 Message Date
Derek Collison
eca04c6fce First pass header support for JetStream
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-30 10:04:23 -07:00
Derek Collison
0a206b4c64 Snapshot performance tweaks
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-29 08:07:31 -07:00
Derek Collison
10e49ca1c4 Fix more flappers
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-28 14:19:11 -07:00
Derek Collison
4ca05d9719 Fix gap test from flapping
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-28 13:43:15 -07:00
Derek Collison
625129f20a Fix flapper test where no messages to receive at end
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-28 13:23:00 -07:00
Ivan Kozlovic
b0e43b6aa9 Fix flappers
- TestResponsePermissions: ensure subscription for service is
registered by server before sending requests.
- TestReloadDoesNotWipeAccountsWithOperatorMode: wait for subject
propagation.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-28 13:41:02 -06:00
Ivan Kozlovic
5d949bf1ea Merge pull request #1424 from nats-io/fix_1421
[FIXED] Possible removal of interest on queue subs with leaf nodes
2020-05-28 11:16:28 -06:00
Ivan Kozlovic
e9805a3109 [FIXED] Possible removal of interest on queue subs with leaf nodes
Server was incorrectly processing a queue subscription removal
as both a plain sub and queue sub, which may have resulted in
drop of interest even when some queue subs remained.

Resolves #1421

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-28 10:21:51 -06:00
Derek Collison
d3ac95a5e6 Add in a terminate delivery for https://github.com/nats-io/jetstream/issues/189
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-28 08:32:34 -07:00
Derek Collison
18822ab866 Merge pull request #1422 from nats-io/snapshot
JetStream Snapshots
2020-05-28 06:59:15 -07:00
Derek Collison
bc0fedbaba Udpates based on PR feedback
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-28 06:08:35 -07:00
Derek Collison
fa59cff105 Add in snapshot and restore JSApi
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-27 20:01:30 -07:00
Waldemar Quevedo
625dd18974 Add support for SPIFFE x.509 SVIDs for auth
This can be enabled by using `verify_and_map`.

```
tls {
 cert_file: "server.pem"
 key_file: "server.key"
 ca_file: "ca.pem"
 timeout: 5
 verify_and_map: true
}

authorization {
  users = [
    {
      user = "spiffe://localhost/my-nats-service/user-a"
    },
    {
      user = "spiffe://localhost/my-nats-service/user-b",
      permissions = { subscribe = { deny = ">" }}
    },
  ]
}
```

Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2020-05-27 13:10:42 -07:00
Derek Collison
8727315eb9 Updated snapshots, added restore, generic hashes
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-26 19:53:16 -07:00
Derek Collison
4c91b69c4f Merge pull request #1413 from nats-io/fix_flappers
Fix flappers
2020-05-26 09:01:04 -07:00
Derek Collison
710ef00383 Don't allow JetStream on system account. Warn when accounts configured but no JS
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 12:17:18 -07:00
Derek Collison
3caf6265d4 Properly recover ephemeral consumers after restart
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 11:06:55 -07:00
Derek Collison
54aa40b352 Wait a bit longer to get subs
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 09:30:36 -07:00
Derek Collison
e27f94eea2 Flush the sub interest
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 09:25:06 -07:00
Derek Collison
ceb7e723c9 Don't let bad rtt estimate fail tests
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 09:20:36 -07:00
Ivan Kozlovic
e5d6bf0c29 Wait for sub propagation on some NewRouteServiceImport
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-25 06:58:23 -07:00
Derek Collison
9dae2cd80f Fixed flapper, will fix bug in other PR
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 06:58:23 -07:00
Derek Collison
79ea95fe44 Fix flapper, wait for sub to propagate
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 06:58:23 -07:00
Derek Collison
b26d389d5e Use old request style, no pause between new sends
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 06:58:23 -07:00
Ivan Kozlovic
46b45b3148 Ensure route INFO is processed before starting queue test
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-25 06:58:23 -07:00
Ivan Kozlovic
7d575e3af9 Remove a test that offers no value but keeps failing
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-25 06:58:23 -07:00
Derek Collison
a693b677c6 Give a bit of room for slow proxy
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 06:58:23 -07:00
Derek Collison
413884d87f Update start time for readloop started, check RTT on flapper test
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 06:58:23 -07:00
Derek Collison
a695d7aeb7 Ignore if we do not have minimum measurements
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-25 06:58:23 -07:00
Ivan Kozlovic
e976e63099 Fixing some flappers
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-25 06:58:23 -07:00
Ivan Kozlovic
5dba3cdd75 [FIXED] Race condition during implicit Gateway reconnection
Say server in cluster A accepts a connection from a server in
cluster B.
The gateway is implicit, in that A does not have a configured
remote gateway to B.
Then the server in B is shutdown, which A detects and initiate
a single reconnect attempt (since it is implicit and if the
reconnect retries is not set).
While this happens, a new server in B is restarted and connects
to A. If that happens before the initial reconnect attempt
failed, A will register that new inbound and do not attempt to
solicit because it has already a remote entry for gateway B.
At this point when the reconnect to old server B fails, then
the remote GW entry is removed, and A will not create an outbound
connection to the new B server.

We fix that by checking if there is a registered inbound when
we get to the point of removing the remote on a failed implicit
reconnect. If there is one, we try the reconnection.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-22 13:01:17 -06:00
Ivan Kozlovic
9715848a8e [ADDED] Websocket support
Websocket support can be enabled with a new websocket
configuration block:

```
websocket {
    # Specify a host and port to listen for websocket connections
    # listen: "host:port"

    # It can also be configured with individual parameters,
    # namely host and port.
    # host: "hostname"
    # port: 4443

    # This will optionally specify what host:port for websocket
    # connections to be advertised in the cluster
    # advertise: "host:port"

    # TLS configuration is required
    tls {
      cert_file: "/path/to/cert.pem"
      key_file: "/path/to/key.pem"
    }

    # If same_origin is true, then the Origin header of the
    # client request must match the request's Host.
    # same_origin: true

    # This list specifies the only accepted values for
    # the client's request Origin header. The scheme,
    # host and port must match. By convention, the
    # absence of port for an http:// scheme will be 80,
    # and for https:// will be 443.
    # allowed_origins [
    #    "http://www.example.com"
    #    "https://www.other-example.com"
    # ]

    # This enables support for compressed websocket frames
    # in the server. For compression to be used, both server
    # and client have to support it.
    # compression: true

    # This is the total time allowed for the server to
    # read the client request and write the response back
    # to the client. This include the time needed for the
    # TLS handshake.
    # handshake_timeout: "2s"
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-20 11:14:39 -06:00
Derek Collison
915e3cd74e Header support for Leafnodes
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:56 -07:00
Derek Collison
dae309ee30 Fix flapper, wait for no clients
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
7b6352c991 Add timestamps to stream state for first and last, addresses #1396
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
5b369ad7dc Do not allow streams to overlap with our public API - https://github.com/nats-io/jetstream/issues/137
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
3b33eb3c13 Fix flapper
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
f5ceab339a Server support for headers between routes
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
7a1ec0ad27 adds API type hints to responses
Also standardise some names of response structures and create
reusable types that can be included into others to construct
our standard API requests and responses.

Fixes some json tags.

Updates the design document of the JSON responses to reflect
the implementation that was done

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
Derek Collison
3a30ef34b7 Make file state cleanup consistent for all JetStream tests
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
bad21bcb42 Make tests use same timings
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
e9b9788fbe Various bug fixes, fixes for flappers
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
585fa8cc89 Test fails under race flag, so scale back
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:31:56 -07:00
Derek Collison
37b8988643 Return 404 codes when resources are not found
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
d36dff7d2f Move list to names, and add list in for detailed info for streams and consumers
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
5256781ba0 Addressed issues raised in https://github.com/nats-io/nats-server/pull/1369
API still only turned on for account info in disabled accounts. Issues with advisories. Plan is still to have all endpoints on in all accounts.
Stream list and Consumer list return names only, page limit increased to 1024.
Stream, Consumer and Template names limited to 256 for now.
Subject API for stream messages, delete and get, not have STREAM.MSG.
Subject API for Durable is now CONSUMER.DURABLE.
Subject API for Templates now STREAM.TEMPLATE.
All subject APIs for list reverted back, so STREAM.LIST, CONSUMER.LIST etc.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
cadd39a01c Major rewrite for NATS JetStream API
API made more consistent. Noun followed by verb.
Name arguments in request subejcts are always at the end now.
Remove enabled call, just use account info.
Getting a message directly from a stream is treated like an admin API and requires JSON request.
Deleting a message directly as well.
StreamList and ConsumerList now include details and support paging.
Streams and Consumers now contain a created field in their info.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
R.I.Pienaar
63845b8577 add type hints to service latency, use time.Time for timestamp
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:26:46 -07:00
Derek Collison
5ffb500857 Be more forgiving on empty api requests, force json request for delete msg api
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
cd5ec0f2e1 Move enabled to Enabled vs JetStream in response
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00