Commit Graph

42 Commits

Author SHA1 Message Date
Derek Collison
a75be04b0a Various fixes for this PR.
- Fix for updating delivery subject and adjusting next delivery sequences.
- When acking explicitly but out of order, need to make sure we set floor correctly.
- Only update ack floors on an ack if the message is present.
- Fix for needAck for explicitAck out of order consumers detecting if message has been acked.
- Fix for race not locking stream when checking interest during stop.
- Fix for filestore determing if a message block still has a message. Added check to first sequence as well as cache.
- Some additions to the original test.

Signed-off-by: Derek Collison <derek@nats.io>
2020-10-01 21:24:40 -07:00
Derek Collison
3faa2e7637 Fix for #1580
Signed-off-by: Derek Collison <derek@nats.io>
2020-09-02 17:18:28 -07:00
Derek Collison
cdbea93a87 Interest based retention should not store if no consumers present.
Fix for https://github.com/nats-io/jetstream/issues/314

Signed-off-by: Derek Collison <derek@nats.io>
2020-09-02 16:04:38 -07:00
Ivan Kozlovic
22833c8d1a Fix sysSubscribe races
Made changes to processSub() to accept subscription properties,
including the icb callback so that it is set prior to add the
subscription to the account's sublist, which prevent races.
Fixed some other racy conditions, notably in addServiceImportSub()

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-08-03 14:59:00 -06:00
Derek Collison
48b2335cbf Fix for stream delete with many consumers - https://github.com/nats-io/jetstream/issues/260
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-22 07:35:38 -07:00
Derek Collison
177ca51356 More checks on duplicate window config, signal in PubAck if duplicate
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-10 07:02:34 -07:00
Derek Collison
2b577b5197 Recover dedupe in memory structures on stream recovery
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-09 16:13:20 -07:00
Derek Collison
ba9dd37595 Rename json field
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-09 08:41:32 -07:00
Derek Collison
e750af84c0 Redo for fast exit when key not found
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-08 17:42:16 -07:00
Derek Collison
b74c2eb2c4 Provide exactly once semantics
Signed-off-by: Derek Collison <derek@nats.io>
2020-07-08 16:56:52 -07:00
Derek Collison
8e9462dea4 Make arg order same for Snapshot
Signed-off-by: Derek Collison <derek@nats.io>
2020-06-02 06:24:46 -07:00
R.I.Pienaar
920dd4269a fix argument order in snapshots
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-06-02 13:51:50 +02:00
Derek Collison
e584d4efee Merge pull request #1435 from nats-io/js-hdrs
First pass header support for JetStream
2020-05-31 06:01:01 -07:00
Derek Collison
dbde2479c2 Add in headers to consumer delivered messages
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-30 15:03:54 -07:00
Derek Collison
eca04c6fce First pass header support for JetStream
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-30 10:04:23 -07:00
Derek Collison
e12907ffa6 Allow snapshots to optionally check all message checksums
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-29 09:57:33 -07:00
Derek Collison
bc0fedbaba Udpates based on PR feedback
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-28 06:08:35 -07:00
Derek Collison
fa59cff105 Add in snapshot and restore JSApi
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-27 20:01:30 -07:00
Derek Collison
8727315eb9 Updated snapshots, added restore, generic hashes
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-26 19:53:16 -07:00
R.I.Pienaar
09dca63efe removes the configuration from advisories
On reflection I think this is a bad idea,
it's enough to know there was a change we
do not need to expose this to potential 3rd
parties

Also since advisories are versioned but
configuration is not, this is awkward.

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-22 16:08:44 +02:00
R.I.Pienaar
c407a3b9d7 create advisories for stream and consumer add/delete/modify
We now publish advisories when streams and consumers are added,
deleted and modified

Also rework how TypedEvents are created to be easier to use

Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-22 15:20:16 +02:00
Derek Collison
7b6352c991 Add timestamps to stream state for first and last, addresses #1396
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
5b369ad7dc Do not allow streams to overlap with our public API - https://github.com/nats-io/jetstream/issues/137
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
d3a447e6e9 Be more efficient when cleaning up all consumers for a stream
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
5256781ba0 Addressed issues raised in https://github.com/nats-io/nats-server/pull/1369
API still only turned on for account info in disabled accounts. Issues with advisories. Plan is still to have all endpoints on in all accounts.
Stream list and Consumer list return names only, page limit increased to 1024.
Stream, Consumer and Template names limited to 256 for now.
Subject API for stream messages, delete and get, not have STREAM.MSG.
Subject API for Durable is now CONSUMER.DURABLE.
Subject API for Templates now STREAM.TEMPLATE.
All subject APIs for list reverted back, so STREAM.LIST, CONSUMER.LIST etc.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
cadd39a01c Major rewrite for NATS JetStream API
API made more consistent. Noun followed by verb.
Name arguments in request subejcts are always at the end now.
Remove enabled call, just use account info.
Getting a message directly from a stream is treated like an admin API and requires JSON request.
Deleting a message directly as well.
StreamList and ConsumerList now include details and support paging.
Streams and Consumers now contain a created field in their info.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
7f41b5a6ae Added support for DiscardPolicy on streams
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
ae8bc2d22b Move ack to +ACK, +OK still works
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
0bf6b2afbc Allow addition of streams and consumers (durables) with same config to be acceptable
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
R.I.Pienaar
fb37c121d1 expose the arrival timestamp in the message metadata
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:21:27 -07:00
Derek Collison
0a36706958 PubAck details that provide stream name and sequence assigned
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:21:27 -07:00
Derek Collison
2d94caee87 Fix for https://github.com/nats-io/jetstream/issues/144
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
4774d41f9d Fixes for filestore consistency on server restarts under heavy load with purge operations
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
323b3d0b11 Beginning basics for stream and consumer snapshotting
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
3ab76a6dcd Write performance tweaks
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
623fa5ccec Impl for GetSeqFromTime for FileStore was missing
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
c9d3bdd5ab Added Stream Update abilities for certain properties
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
685efc36df Allow JS to work over leafnodes for streams
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
cd052a42d6 Fix for non-js msg failed delivery stopping normal subscribers
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
4046babb77 Make other API calls for next, ack ,and byseq more aligned with others
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
72cda966f4 Changed API subjects to be authorization friendly
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
47c28b2fb0 JetStream major refactor for name changes.
MsgSet -> Stream
Observable -> Consumer

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00