Commit Graph

2767 Commits

Author SHA1 Message Date
Derek Collison
55c77d1e4e Added support for delivery of HMSG and support for older clients
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
d61f1f5d92 Add in support for client header bool in CONNECT and tests
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
d51566881e First pass at headers awareness for server
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
3ab203d164 typed messages adr
[ci skip]

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
859a36e7ac ensures that the streams value is never null
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
85da492096 initialize api response structs properly
when go json marshal a unset []string it puts
null in the body and not [] which then makes
it an invalid list of strings.

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
ce26c65174 fix the type hint for msg_get
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
45a4d929da rename ApiPagedResponse 2020-05-19 14:33:06 -07:00
R.I.Pienaar
70670fabd5 consistency fix on stream template list response
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
54f45850f9 improve some error reporting around bad JSON and bad requests
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
R.I.Pienaar
7a1ec0ad27 adds API type hints to responses
Also standardise some names of response structures and create
reusable types that can be included into others to construct
our standard API requests and responses.

Fixes some json tags.

Updates the design document of the JSON responses to reflect
the implementation that was done

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:33:06 -07:00
Derek Collison
9ffaf44d71 Make sure to not turn on cache when it was disable
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
3a30ef34b7 Make file state cleanup consistent for all JetStream tests
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
bad21bcb42 Make tests use same timings
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
e9b9788fbe Various bug fixes, fixes for flappers
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
e4c15d8680 Fix for data race
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
585fa8cc89 Test fails under race flag, so scale back
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:31:56 -07:00
Derek Collison
03aacecb81 Changed cache back to simple map.
We were using a sync.Map. This did provide a benefit with massive contention from lots of Go routines. However this is only about 2x in the crazy extremes now and with a normal map and read locks we can assist the RemoveBatch which was a cause for performance issues.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:31:56 -07:00
Derek Collison
d3a447e6e9 Be more efficient when cleaning up all consumers for a stream
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
acc8da8b6e Improve RemoveBatch by disabling cache
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
9a702c2bc7 Removed duplicate init
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
R.I.Pienaar
5fa50392f8 minor fix for json keys on consumer lists
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:27:45 -07:00
Derek Collison
37b8988643 Return 404 codes when resources are not found
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
d36dff7d2f Move list to names, and add list in for detailed info for streams and consumers
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
40900ca1a8 Fixes for Template subject API
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
5256781ba0 Addressed issues raised in https://github.com/nats-io/nats-server/pull/1369
API still only turned on for account info in disabled accounts. Issues with advisories. Plan is still to have all endpoints on in all accounts.
Stream list and Consumer list return names only, page limit increased to 1024.
Stream, Consumer and Template names limited to 256 for now.
Subject API for stream messages, delete and get, not have STREAM.MSG.
Subject API for Durable is now CONSUMER.DURABLE.
Subject API for Templates now STREAM.TEMPLATE.
All subject APIs for list reverted back, so STREAM.LIST, CONSUMER.LIST etc.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
R.I.Pienaar
e67c1d9d69 expand on empty requests and non json requests
[skip ci]

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:27:45 -07:00
R.I.Pienaar
8bf06597d4 clarify error schemas can be mostly empty 2020-05-19 14:27:45 -07:00
R.I.Pienaar
830d14bb40 move to description for error fields 2020-05-19 14:27:45 -07:00
R.I.Pienaar
ad80d2aee1 initial Architecture Decision Record 2020-05-19 14:27:45 -07:00
Derek Collison
cadd39a01c Major rewrite for NATS JetStream API
API made more consistent. Noun followed by verb.
Name arguments in request subejcts are always at the end now.
Remove enabled call, just use account info.
Getting a message directly from a stream is treated like an admin API and requires JSON request.
Deleting a message directly as well.
StreamList and ConsumerList now include details and support paging.
Streams and Consumers now contain a created field in their info.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
17aca11002 Small changes to event ids, good approach though with separate lock on account
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
R.I.Pienaar
63845b8577 add type hints to service latency, use time.Time for timestamp
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:26:46 -07:00
Derek Collison
5ffb500857 Be more forgiving on empty api requests, force json request for delete msg api
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
cd5ec0f2e1 Move enabled to Enabled vs JetStream in response
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
513bff051b Redesign message-based API
Removed usage of +OK and -ERR. All responses are valid json objects now and optionally can include an ApiError which will have Code and Description.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
7f41b5a6ae Added support for DiscardPolicy on streams
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
911e7ef35d Add additional fields to client info for latency
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
36d5421318 Don't do sample ack unless msg was on pending
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
50443ead89 Change to consumer info, removed state, made pending and redelivery counts
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
ae8bc2d22b Move ack to +ACK, +OK still works
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
7d7a9e8f23 Make sure messages are not redelivered if ack'd late (pull-based this can happen)
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
0bf6b2afbc Allow addition of streams and consumers (durables) with same config to be acceptable
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
a5dbc20e94 Fix for test failure
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
a7f1bca534 Additional service latency upgrades.
We now share more information about the responder and the requestor. The requestor information by default is not shared, but can be when declaring the import.

Also fixed bug for error handling on old request style requests that would always result on a 408 response.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
b8c04c1abf Make all start times UTC for latency tracking
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
ea5e5bd364 Services rewrite #2
This contains a rewrite to the services layer for exporting and importing. The code this merges to already had a first significant rewrite that moved from special interest processing to plain subscriptions.

This code changes the prior version's dealing with reverse mapping which was based mostly on thresholds and manual pruning, with some sporadic timer usage. This version uses the jetstream branch's code that understands interest and failed deliveries. So this code is much more tuned to reacting to interest changes. It also removes thresholds and goes only by interest changes or expirations based around a new service export property, response thresholds. This allows a service provider to provide semantics on how long a response should take at a maximum.

This commit also introduces formal support for service export streamed and chunked response types send an empty message to signify EOF.

This commit also includes additions to the service latency tracking such that errors are now sent, not only successful interactions. We have added a Status field and an optional Error fields to ServiceLatency.

We support the following Status codes, these are directly from HTTP.

400 Bad Request (request did not have a reply subject)
408 Request Timeout (when system detects request interest went away, old request style to make dependable)..
503 Service Unavailable (no service responders running)
504 Service Timeout (The new response threshold expired)

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
016331c270 Fix race condition
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:24:31 -07:00
R.I.Pienaar
78fdeb661d move events nuid to the server struct 2020-05-19 14:24:31 -07:00
R.I.Pienaar
3182db4c3a move to events having Type not Schema 2020-05-19 14:22:53 -07:00