Commit Graph

1594 Commits

Author SHA1 Message Date
Derek Collison
ec0bc1dbec First pass account configuration for jetstream
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:21:27 -07:00
Derek Collison
1394a7118d Breaking change to upgrade ConsumerConfig for consumer creation.
This is a breaking change and will not be able to restore consumer's from a filestore when upgraded.
We are getting close to settling on the API an once that happens we will not be introducnig any
breaking changes.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:20:02 -07:00
Derek Collison
ffae57e20e Fold replay original logic into main loop, fixed pull bug on replay original
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:20:02 -07:00
Derek Collison
d2ff4311d4 Rebase with master, updates to go.mod and vendor, bumped version
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:20:02 -07:00
Derek Collison
2d94caee87 Fix for https://github.com/nats-io/jetstream/issues/144
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
4a30f593bd Fix for https://github.com/nats-io/jetstream/issues/143
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
0b970130b0 Allow filter subjects to have wildcards. Fix for https://github.com/nats-io/jetstream/issues/136
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
3c1dcd2401 Remove pull based consumers that are no longer valid or have registered interest
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
R.I.Pienaar
871edd996a add the stream name in consumer info 2020-05-19 14:18:34 -07:00
Derek Collison
faea8f930c small fix for race
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
4774d41f9d Fixes for filestore consistency on server restarts under heavy load with purge operations
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
ade58bea87 Do not deadlock when triggering lots of didNotDeliver notices. Don't hold lock on chan send
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
323b3d0b11 Beginning basics for stream and consumer snapshotting
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
f2ece0515c Increased purge performance for very large streams
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
df774e44b0 Rework how service imports are handled to avoid performance hits
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:18:34 -07:00
Derek Collison
3ab76a6dcd Write performance tweaks
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
3d80ff71a4 Fix for queue subscribers as consumers
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
5e9f487b61 Fixes for test and locking, bumped version
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
4a46a92cea Redesigned file storage engine.
The old engine was a simple design to get us started but had a single lock and held that lock around I/O.
This design allows message blocks to have the own RW locks and operate independently and to also not hold
locks while doing most I/O.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
bb539d74cf Should not ask match to acquire lock already held
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
R.I.Pienaar
10a419ca3d support jetstream client kind in a few places 2020-05-19 14:16:03 -07:00
R.I.Pienaar
fc6d8826f5 show basic jetstream info in varz and server info 2020-05-19 14:16:03 -07:00
Ivan Kozlovic
6cd07c7cf0 Fixed improper field pass to Fatalf()
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-05-19 14:16:03 -07:00
Derek Collison
623fa5ccec Impl for GetSeqFromTime for FileStore was missing
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
c9d3bdd5ab Added Stream Update abilities for certain properties
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
51cfe266db Version 2.2.0-beta.2
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
7d5fddee7e Fix for test
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
20acaadf46 Version 2.2.0-beta.1
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
8d1f3cc7c2 Allow JetStream consumers to work across multi-server hops
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
d0652539f5 Fix for filestore age timer bug
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
685efc36df Allow JS to work over leafnodes for streams
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
R.I.Pienaar
da5425b3da add some more jetstream options
Signed-off-by: R.I.Pienaar <rip@devco.net>
2020-05-19 14:16:03 -07:00
R.I.Pienaar
ef77156890 allow jetstream to be enabled in the config file 2020-05-19 14:16:03 -07:00
R.I.Pienaar
dc207be04f handles 0 length tokens correctly
Without this nats-req '$JS.TEMPLATE..DELETE' '' causes a panic
2020-05-19 14:16:03 -07:00
R.I.Pienaar
7ee7c19fc7 consistency update in api subjects 2020-05-19 14:16:03 -07:00
R.I.Pienaar
60f1407923 consistent timestamps in events
Use the same key name for time in all events
Use the same format in all events

RFC3339 is the standard for this stuff and
for sure it should all be in UTC rather than
local time
2020-05-19 14:16:03 -07:00
Derek Collison
d9dbe9e9e5 Split out host and port on audit advisories
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
d5aa55a103 Audit advisories, work based off of #1253 from ripienaar
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
313dcd21ba Removed condition not needed
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
5c99355fa5 Separate out consumer API for Durable vs Ephemeral
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
cd052a42d6 Fix for non-js msg failed delivery stopping normal subscribers
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
e703d62bd5 Tweak ACK reply subject
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
f0b5fd2882 Add in API support for templates, proto error helper
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
4046babb77 Make other API calls for next, ack ,and byseq more aligned with others
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
R.I.Pienaar
88760724e1 also validate that subject matches request 2020-05-19 14:16:03 -07:00
R.I.Pienaar
25e535eab8 add stream name to create subject
I anticipate it would be desirable to allow a microservice running
in an account to create only the stream it needs when it starts up
if there is no stream but no others, so a ACL would be written to
allow that.

Thus adding the T pattern to stream create too.
2020-05-19 14:16:03 -07:00
Derek Collison
72cda966f4 Changed API subjects to be authorization friendly
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
34ce97bb8c Added support for wildcards for service imports
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
R.I.Pienaar
24939d51db use "advisory" instead of "notification" 2020-05-19 14:16:03 -07:00
R.I.Pienaar
200ebbd47e proposed schemas for events
This is a proposal to add schema, timestamps and unique ids to
events.  For now just JS ones but I propose to do these for all
events the server sends that might be end user consumed.

These exist to help the user identify a message even if it was
sent to a 3rd party system, the schema will be translatable to
a well known url like https://nats.io/schames/jetstream/event/v1/obs_ack.json
or to another wll known identifier like a NATS subject name

We'd publish JSON schema documents at these well known locations
describing each key and so forth
2020-05-19 14:16:03 -07:00