Commit Graph

44 Commits

Author SHA1 Message Date
Andrew Harris
17930baa18 add client provided info into server side client logs when available 2020-07-24 15:40:39 -04:00
Derek Collison
c969e7e424 Do proper ubsubscribe when shutting off restore endpoint
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-04 08:58:14 -07:00
Derek Collison
164f44ed18 Require reply subjects for restore chunks
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-04 06:56:07 -07:00
Derek Collison
012d517ba1 Better error handling and reporting for failures
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-04 06:17:50 -07:00
Derek Collison
660ea3c807 Snapshot restore now works across leafnodes.
This also introduces the ability to have flow control inbound for restoring a stream.
If the system detects a reply subject it will respond with a nil payload.
For the last EOF message if a reply is present it will respond with a stream info response or error.

Signed-off-by: Derek Collison <derek@nats.io>
2020-06-03 20:00:59 -07:00
R.I.Pienaar
920dd4269a fix argument order in snapshots
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-06-02 13:51:50 +02:00
R.I.Pienaar
c57b86128d publish audit advisories to the correct subject
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-06-02 12:40:03 +02:00
R.I.Pienaar
3fc5c9284a send stream advisories using a helper 2020-06-02 08:48:11 +02:00
Derek Collison
5271be49ee Fix race for snapshots
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-01 09:21:22 -07:00
Derek Collison
e584d4efee Merge pull request #1435 from nats-io/js-hdrs
First pass header support for JetStream
2020-05-31 06:01:01 -07:00
Derek Collison
8e407f8db4 Do snapshot setup in go routine as well for checkMsgs
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-30 11:28:43 -07:00
Derek Collison
eca04c6fce First pass header support for JetStream
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-30 10:04:23 -07:00
Derek Collison
e12907ffa6 Allow snapshots to optionally check all message checksums
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-29 09:57:33 -07:00
Derek Collison
0a206b4c64 Snapshot performance tweaks
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-29 08:07:31 -07:00
Derek Collison
d3ac95a5e6 Add in a terminate delivery for https://github.com/nats-io/jetstream/issues/189
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-28 08:32:34 -07:00
Derek Collison
bc0fedbaba Udpates based on PR feedback
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-28 06:08:35 -07:00
Derek Collison
fa59cff105 Add in snapshot and restore JSApi
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-27 20:01:30 -07:00
R.I.Pienaar
c407a3b9d7 create advisories for stream and consumer add/delete/modify
We now publish advisories when streams and consumers are added,
deleted and modified

Also rework how TypedEvents are created to be easier to use

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-22 15:20:16 +02:00
R.I.Pienaar
859a36e7ac ensures that the streams value is never null
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
85da492096 initialize api response structs properly
when go json marshal a unset []string it puts
null in the body and not [] which then makes
it an invalid list of strings.

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
ce26c65174 fix the type hint for msg_get
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
45a4d929da rename ApiPagedResponse 2020-05-19 14:33:06 -07:00
R.I.Pienaar
70670fabd5 consistency fix on stream template list response
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
54f45850f9 improve some error reporting around bad JSON and bad requests
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
7a1ec0ad27 adds API type hints to responses
Also standardise some names of response structures and create
reusable types that can be included into others to construct
our standard API requests and responses.

Fixes some json tags.

Updates the design document of the JSON responses to reflect
the implementation that was done

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
5fa50392f8 minor fix for json keys on consumer lists
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:27:45 -07:00
Derek Collison
37b8988643 Return 404 codes when resources are not found
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
d36dff7d2f Move list to names, and add list in for detailed info for streams and consumers
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
40900ca1a8 Fixes for Template subject API
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
5256781ba0 Addressed issues raised in https://github.com/nats-io/nats-server/pull/1369
API still only turned on for account info in disabled accounts. Issues with advisories. Plan is still to have all endpoints on in all accounts.
Stream list and Consumer list return names only, page limit increased to 1024.
Stream, Consumer and Template names limited to 256 for now.
Subject API for stream messages, delete and get, not have STREAM.MSG.
Subject API for Durable is now CONSUMER.DURABLE.
Subject API for Templates now STREAM.TEMPLATE.
All subject APIs for list reverted back, so STREAM.LIST, CONSUMER.LIST etc.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
cadd39a01c Major rewrite for NATS JetStream API
API made more consistent. Noun followed by verb.
Name arguments in request subejcts are always at the end now.
Remove enabled call, just use account info.
Getting a message directly from a stream is treated like an admin API and requires JSON request.
Deleting a message directly as well.
StreamList and ConsumerList now include details and support paging.
Streams and Consumers now contain a created field in their info.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
R.I.Pienaar
63845b8577 add type hints to service latency, use time.Time for timestamp
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:26:46 -07:00
Derek Collison
5ffb500857 Be more forgiving on empty api requests, force json request for delete msg api
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
cd5ec0f2e1 Move enabled to Enabled vs JetStream in response
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
513bff051b Redesign message-based API
Removed usage of +OK and -ERR. All responses are valid json objects now and optionally can include an ApiError which will have Code and Description.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
R.I.Pienaar
3182db4c3a move to events having Type not Schema 2020-05-19 14:22:53 -07:00
R.I.Pienaar
c64e39ea61 send audit advisories for all api errors 2020-05-19 14:21:27 -07:00
Derek Collison
323b3d0b11 Beginning basics for stream and consumer snapshotting
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
c9d3bdd5ab Added Stream Update abilities for certain properties
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
8d1f3cc7c2 Allow JetStream consumers to work across multi-server hops
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
R.I.Pienaar
7ee7c19fc7 consistency update in api subjects 2020-05-19 14:16:03 -07:00
R.I.Pienaar
60f1407923 consistent timestamps in events
Use the same key name for time in all events
Use the same format in all events

RFC3339 is the standard for this stuff and
for sure it should all be in UTC rather than
local time
2020-05-19 14:16:03 -07:00
Derek Collison
d9dbe9e9e5 Split out host and port on audit advisories
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
d5aa55a103 Audit advisories, work based off of #1253 from ripienaar
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00