This extends the previous work in #3733 with the following:
1. Remove buffer coalescing, as this could result in a race condition
during the `writev` syscall in rare circumstances
2. Add a third buffer size, to ensure that we aren't allocating more
than we need to without coalescing
3. Refactor buffer handling in the WebSocket code to reduce allocations
and ensure owned buffers aren't incorrectly being pooled resulting in
further race conditions
Fixesnats-io/nats.ws#194.
Signed-off-by: Neil Twigg <neil@nats.io>
A stream could have a complicated state with interior deletes.
This is a simpler way to determine if we need to consider a snapshot
that involves much less time and CPU and memory.
Signed-off-by: Derek Collison <derek@nats.io>
This is a simpler way to determine if we need to consider a snapshot that involves much less time and CPU and memory.
Signed-off-by: Derek Collison <derek@nats.io>
Reset our WAL on edge conditions instead of trying to recover.
Also if we are timing out and trying to become a candidate but are doing
a catchup check if we are stalled.
Signed-off-by: Derek Collison <derek@nats.io>
Also if we are timing out and trying to become a candidate but are doing a catchup check if we are stalled.
Signed-off-by: Derek Collison <derek@nats.io>
When tracking information per subject went from >1 back to only 1 we
needed to make sure we cleared firstNeedsUpdate.
Thanks to @scottf for finding it.
Signed-off-by: Derek Collison <derek@nats.io>
If we see another leader with same term we should step down, otherwise
we could spin with catchups and WAL truncates.
Signed-off-by: Derek Collison <derek@nats.io>
When adding or updating sources/mirrors, server was checking if the
stream with a given name exists to check for subject overlaps, among
other things.
However, if sourced/mirrored stream was `External`, checks should not be
executed, as not only stream would never be found, but also, if
`External` stream had the same name as the sourcing stream, the check
would be wrongly performed against itself.
cc @jnmoyne
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
When adding or updating sources/mirrors, server was checking if the stream with
a given name exists to check for subject overlaps, among other things.
However, if sourced/mirrored stream was `External`, checks should
not be executed, as not only stream would never be found,
but also, if `External` stream had the same name as the sourcing stream,
the check would be wrongly performed against itself.
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
Fixes this test [TestJetStreamClusterDeleteAndRestoreAndRestart] which
would flap since it would not snapshot since hash was same but had
entries that would erase stream data.
Signed-off-by: Derek Collison <derek@nats.io>
Fixes this test [TestJetStreamClusterDeleteAndRestoreAndRestart] which would flap since it would not snapshot since hash was same but had entries that would erase stream data.
Signed-off-by: Derek Collison <derek@nats.io>
Make sure to process consumer entries on recovery in case state was not
committed.
And sync other consumers when taking over as leader but no need to
process snapshots when we are in fact the leader.
Do not let the consumer redelivered count go down on re-applying state
on restarts.
Signed-off-by: Derek Collison <derek@nats.io>
Also sync other consumers when taking over as leader but no need to process snapshots when we are in fact the leader.
Signed-off-by: Derek Collison <derek@nats.io>
Improve performance on storing msgs when multiple subjects exists with
multiple messages and we have store limits that are being hit.
This will not fix any performance degradation per se when per subject
limits are being hit constantly across a vast array of keys. That will
be addressed more so in 2.10.
Resolves#4043
Signed-off-by: Derek Collison <derek@nats.io>
Some write operations in the filestore require re-encrypting the entire
block, so having large blocks can make these operations slow and hurt
performance.
Signed-off-by: Neil Twigg <neil@nats.io>
