gogrlx/nats-server
1
0
Fork 0
mirror of https://github.com/gogrlx/nats-server.git synced 2026-04-02 03:38:42 -07:00
Code Issues Packages Projects Releases Wiki Activity
4,095 Commits 30 Branches 48 Tags
go-mod-split
Commit Graph

48 Commits

Author SHA1 Message Date
Waldemar Quevedo
86a64fbc46 Updates to JS consumer errors 
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
 2021-03-09 09:46:28 -08:00
Derek Collison
d31fda5dac Added code to constrain size of WAL under most scenarios. 
Signed-off-by: Derek Collison <derek@nats.io>
 2021-03-06 08:38:56 -08:00
Derek Collison
e5c8774172 Handle out of space situations, general stability enhancements 
Signed-off-by: Derek Collison <derek@nats.io>
 2021-02-25 17:54:29 -08:00
Derek Collison
c16f6e193d Move JetStream direct APIs to private. 
Signed-off-by: Derek Collison <derek@nats.io>
 2021-02-07 15:19:22 -08:00
Derek Collison
b358773ddf Force filestore to flush in place by default. 
Track lost data and truncate message blocks when detecting failures or write errors.

Signed-off-by: Derek Collison <derek@nats.io>
 2021-02-06 20:04:47 -08:00
Derek Collison
9c858d197a Added ability to properly restore consumers from a snapshot. 
This made us add forwarding proposals functionality in the raft layer.
More general cleanup and bug fixes as well.

Signed-off-by: Derek Collison <derek@nats.io>
 2021-01-24 19:30:34 -08:00
Derek Collison
ff54c9dc9c Reworked snapshot and restore. 
Underestimated the effort to get stream restore working properly in cluster mode.
Some good bug fixes and stability improvments.

Signed-off-by: Derek Collison <derek@nats.io>
 2021-01-20 11:58:31 -08:00
Derek Collison
f0cdf89c61 JetStream Clustering WIP 
Signed-off-by: Derek Collison <derek@nats.io>
 2021-01-14 01:14:52 -08:00
Derek Collison
7564768027 Added Compact to store interface for WAL functionality 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-12-03 16:18:58 -08:00
Derek Collison
28cb4e8c34 Fix bug when removing the same message from a stream. 
We would release locks and call into upper layers when removing a message. The upper layers may call back into the lower layers to get more information, such as the subject.
This fix has the storage updates optionally supply the subject for filtered consumers and fixes the bug of double deletes.

Signed-off-by: Derek Collison <derek@nats.io>
 2020-11-13 17:05:24 -08:00
Derek Collison
164f9fdf2b Updates to consumer store to support delta updates. 
In preparation for clustering we need to have the consumer filestore update state with deltas vs original design.

Signed-off-by: Derek Collison <derek@nats.io>
 2020-11-10 19:16:55 -08:00
Derek Collison
fe2b354414 Stability and performance updates. 
The original design had a shared filestore write buffer and individual message blocks had a read cache.
This presented some performance and stability issues when lots of reads and deletes were happening to a
message block that was also being written to actively.

This change eliminates the shared write buffer and uses the message block's cache as a write through as
well as read cache and handles partials correctly.

Signed-off-by: Derek Collison <derek@nats.io>
 2020-10-28 07:58:47 -07:00
Derek Collison
df4ee081a5 Track number of stream pending for each consumer. 
This will track the stream pending state for each consumer.
This code does account for filtered consumers.

Signed-off-by: Derek Collison <derek@nats.io>
 2020-10-27 19:30:42 -07:00
Derek Collison
ad247d1853 Add store.SkipMsg() and update no interest retention streams 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-10-22 19:35:28 -07:00
Derek Collison
e584d4efee Merge pull request #1435 from nats-io/js-hdrs 
First pass header support for JetStream
 2020-05-31 06:01:01 -07:00
Derek Collison
eca04c6fce First pass header support for JetStream 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-30 10:04:23 -07:00
Derek Collison
e12907ffa6 Allow snapshots to optionally check all message checksums 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-29 09:57:33 -07:00
Derek Collison
fa59cff105 Add in snapshot and restore JSApi 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-27 20:01:30 -07:00
Derek Collison
8727315eb9 Updated snapshots, added restore, generic hashes 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-26 19:53:16 -07:00
Derek Collison
7b6352c991 Add timestamps to stream state for first and last, addresses #1396 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:33:06 -07:00
Derek Collison
7f41b5a6ae Added support for DiscardPolicy on streams 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:26:46 -07:00
R.I.Pienaar
fb37c121d1 expose the arrival timestamp in the message metadata 
Signed-off-by: R.I.Pienaar <rip@devco.net>
 2020-05-19 14:21:27 -07:00
R.I.Pienaar
efc68f13bb ensure deliver policy can be marshalled 
Signed-off-by: R.I.Pienaar <rip@devco.net>
 2020-05-19 14:21:27 -07:00
Derek Collison
4774d41f9d Fixes for filestore consistency on server restarts under heavy load with purge operations 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:18:34 -07:00
Derek Collison
323b3d0b11 Beginning basics for stream and consumer snapshotting 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:18:34 -07:00
Derek Collison
c9d3bdd5ab Added Stream Update abilities for certain properties 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:16:03 -07:00
Derek Collison
47c28b2fb0 JetStream major refactor for name changes. 
MsgSet -> Stream
Observable -> Consumer

Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:16:03 -07:00
Derek Collison
2237915c97 Added in template support 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:16:03 -07:00
Derek Collison
f1a81454b5 Added in error for Delete and Stop for store 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:16:03 -07:00
Derek Collison
e3be6f45c4 Add in delete for observables for filestore impl 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:16:03 -07:00
R.I.Pienaar
423e017724 update readme to point to new CLI and updated guide 2020-05-19 14:15:11 -07:00
R.I.Pienaar
a01698bcf7 also track delivery counts and msg set 2020-05-19 14:15:11 -07:00
R.I.Pienaar
92f00b31b1 correctly name pending and redelivery 2020-05-19 14:15:11 -07:00
R.I.Pienaar
e2a1b18de6 report observable config and observables count on message sets 2020-05-19 14:15:11 -07:00
Derek Collison
cfc31f7a15 Allow direct lookup of message from msgset 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:15:11 -07:00
Derek Collison
b3b2feebc5 Include config in info for msgset 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:15:11 -07:00
Derek Collison
e127039622 More API impls, tests for DeleteMsg 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:13:22 -07:00
Derek Collison
79b85ae385 Optimize for concurrent pub/sub 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:13:22 -07:00
Derek Collison
68860b39c3 metadata and checksum storage 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:12:28 -07:00
Derek Collison
81704d285b Plugin filestore to jetstream tests 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:12:28 -07:00
Derek Collison
c3a5df135f Observable state storage 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:12:28 -07:00
Derek Collison
613d65c58c Erased msg recovery, bug fix on linux for idx file, LoadMsg vs Lookup 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:12:28 -07:00
Derek Collison
e8f465eaf3 Filestore read perf, secure delete for stores 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:12:28 -07:00
Derek Collison
39aba7de92 filestore first pass 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:12:28 -07:00
Derek Collison
da7838368d Tracking usage and limits 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:07:02 -07:00
Derek Collison
7e00a975b1 Add purge and interest retention tests 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:07:02 -07:00
Derek Collison
0a92d8e87d AckWait and redelivery 
Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:07:02 -07:00
Derek Collison
dd116fcfd4 JetStream first pass basics. 
This is the first checkin for JetStream. Has some rudimentary basics working.

TODO
1. Push vs pull mode for observables. (work queues)
2. Disk/File store, memory only for now.
3. clustering code - design shaping up well.
4. Finalize account import semantics.
5. Lots of other little things.

Signed-off-by: Derek Collison <derek@nats.io>
 2020-05-19 14:06:29 -07:00