Commit Graph

6 Commits

Author SHA1 Message Date
Derek Collison
4d7cd26956 Add in support for segmented binary stream snapshots.
Streams with many interior deletes was causing issues due to the fact that the interior deletes were represented as a sorted []uint64.
This approach introduces 3 sub types of delete blocks, avl bitmask tree, a run length encoding, and the legacy format above.
We also take into account large interior deletes such that on receiving a snapshot we can skip things we already know about.

Signed-off-by: Derek Collison <derek@nats.io>
2023-07-03 08:41:33 -07:00
Derek Collison
a45c7106b8 Only set minimum when removing first item
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-18 22:50:57 -07:00
Derek Collison
aa66c87d53 Make sure to set node count to 1
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-18 22:34:13 -07:00
Derek Collison
a744e1d5c9 Added ability to set initial minimum value for seqset when known.
We know the minimum value when creating a new filestore msgBlk.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-18 22:23:11 -07:00
Neil Twigg
1a24e955d0 Add size to preamble, check capacity instead of length when encoding
* If we don't encode the `size`, it is lost during an encoding-decoding round-trip
* If we don't check capacity, we might reallocate needlessly instead of just growing the slice

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-17 20:17:12 -07:00
Derek Collison
1f6aa94405 SequenceSet is an AVL tree with variable bitmask nodes to contain large delete maps for streams.
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-17 20:17:03 -07:00