mirror of
https://github.com/gogrlx/nats-server.git
synced 2026-04-02 03:38:42 -07:00
adef8281a2
Historically we kept indexing information, either by sequence or by subject, as a per msg block operation. These were the "*.idx" and "*.fss" indexing files. When streams became very large this could have an impact on recovery time. Also, for encryption the fast path for determining if the indexing was current would require loading and decrypting the complete block. This design moves to a more traditional WAL and snapshot approach. The snapshots for the complete stream, including sumary information, global per subject information maps (PSIM) and per msg block details including summary and dmap, are processed asynchronously. The snapshot includes the msh block and has for the last record considered in the snapshot. On recovery the snapshot is read and processed and any additional records past the point of the snapshot itself are processed. To this end, any removal of a message has to be expressed as a delete tombstone that is always added the the fs.lmb file. These are processed on recovery and our indexing layer knows to skip them. Changing to this method drastically improves startup and recovery times, and has simplified the code. Some normal performance benefits have been seen as well. Signed-off-by: Derek Collison <derek@nats.io>
Tests
Tests that run on Travis have been split into jobs that run in their own VM in parallel. This reduces the overall running time but also is allowing recycling of a job when we get a flapper as opposed to have to recycle the whole test suite.
JetStream Tests
For JetStream tests, we need to observe a naming convention so that no tests are omitted when running on Travis.
The script runTestsOnTravis.sh will run a given job based on the definition found in ".travis.yml".
As for the naming convention:
- All JetStream tests name should start with
TestJetStream - Cluster tests should go into
jetstream_cluster_test.goand start withTestJetStreamCluster - Super-cluster tests should go into
jetstream_super_cluster_test.goand start withTestJetStreamSuperCluster
Not following this convention means that some tests may not be executed on Travis.