Commit Graph

4309 Commits

Author SHA1 Message Date
Derek Collison
24081ae78e Bump to 2.9.4-RC.2
Signed-off-by: Derek Collison <derek@nats.io>
2022-10-27 09:19:22 -07:00
Derek Collison
d89992990b Merge pull request #3582 from nats-io/fix
Make sure to check for ss == nil
2022-10-27 11:07:11 -05:00
Derek Collison
8f6d55a878 Make sure to check for ss == nil
Signed-off-by: Derek Collison <derek@nats.io>
2022-10-27 08:52:42 -07:00
Ivan Kozlovic
b3e0431959 [FIXED] allow_non_tls is lost after server reload
The server would reset its INFO's TLSRequired to the presence
of a TLS configuration without checking for the allow_non_tls
option.

Resolves #3581

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-27 09:49:44 -06:00
Derek Collison
8cc87c988f Bump to 2.9.4-RC.1
Signed-off-by: Derek Collison <derek@nats.io>
2022-10-26 17:52:28 -07:00
Derek Collison
6128b83507 On abnormal server exit, for streams or KV with max msgs per subject set we could recover more than N msgs per subject.
This fix allows for recover of correct state on restart when index files are missing or not current.

Signed-off-by: Derek Collison <derek@nats.io>
2022-10-26 16:00:57 -07:00
Tomasz Pietrek
f0219e1d95 Merge pull request #3572 from nats-io/jarema/add-pending-info-to-request-timeout
Added pending messages/bytes info to request statuses and errors
2022-10-26 21:20:20 +02:00
Tomasz Pietrek
ef764598ee Add pending messages/bytes info to request errors and statuses
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-10-26 20:02:11 +02:00
Derek Collison
2241ad089e Make local error since non-fatal for now.
Signed-off-by: Derek Collison <derek@nats.io>
2022-10-25 16:56:10 -07:00
Derek Collison
aa52c2fecf Added warning for high message lag into a clustered stream.
Signed-off-by: Derek Collison <derek@nats.io>
2022-10-25 16:11:35 -07:00
Derek Collison
db13766f18 Merge pull request #3576 from nats-io/signal-pull-consumers
Removed ephemeral consumer migration.
2022-10-25 17:35:35 -05:00
Derek Collison
f0afa49b9f Make sure to stop raft nodes on all monitor exits.
Signed-off-by: Derek Collison <derek@nats.io>
2022-10-25 14:48:28 -07:00
Derek Collison
ff2cd1d7f9 Fixed test and bug that would override consumer replicas.
Signed-off-by: Derek Collison <derek@nats.io>
2022-10-25 14:35:20 -07:00
Derek Collison
15dc72db50 Removed migration of ephemerals, added proper signaling for pul consumers pending requests.
Signed-off-by: Derek Collison <derek@nats.io>
2022-10-25 14:35:20 -07:00
Ivan Kozlovic
39f31b0dbe [FIXED] JetStream: InactivityThreshold updates not always working
This is based of @neilalexander PR #3558.

It ensures that the timer is reset/canceled on configuration
update (by the leader only).

Fixed also the issue with a super-cluster where the delete timer
would always be reset at every gateway interval check.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-25 09:54:01 -06:00
Ivan Kozlovic
7ca85e0e80 [FIXED] JetStream: Update of an R1 consumer would not get a response
The update was accepted but the server would not respond to the
client/CLI.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-25 09:04:35 -06:00
Ivan Kozlovic
6113c52ae1 [FIXED] Solicited route may not retry to reconnect
Originally, only solicited routes were retried in case of a disconnect,
but that was before gossip protocol was introduced. Since then, two
servers that connect to each other due to gossip should retry to
reconnect if the connection breaks, even if the route is not explicit.
However, server will retry only once or more accurately, ConnectRetries+1.

This PR solves the issue that the reconnect attempt was not initiated
for a "solicited route" that was not explicit.

Maybe related to #3571

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-24 10:39:23 -06:00
Derek Collison
d97abf0b61 On a write error we rebuild, so update accounting after flush write attempt.
If we had to flush while loading in subject info we would fail there and not properly rebuild state as we did on a failure for flushPending.

Signed-off-by: Derek Collison <derek@nats.io>
2022-10-18 11:39:42 -07:00
Ivan Kozlovic
f8aa3ac11d [FIXED] JetStream: "first sequence mismatch" error on catchup with message expiration
When a server was restarted and expired messages, but the leader had a snapshot that
still had the old messages we would reset complete follower stream state, this fix
just skips over the expired as we prepare the request to the leader.

Resolves #3516

Signed-off-by: Derek Collison <derek@nats.io>
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-17 17:02:08 -06:00
Derek Collison
37e876c7de Merge pull request #3560 from fantashley/nil-fss-panic
fix: Prevent nil map assignment panic if disk is full
2022-10-17 15:22:58 -07:00
Ivan Kozlovic
5df93f9c81 Fix a test that would sometimes fail
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-17 11:35:47 -06:00
Ivan Kozlovic
90e9c89594 Added specific tests for using non system extended setup similar to NGS
Signed-off-by: Derek Collison derek@nats.io
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-17 10:42:03 -06:00
Ivan Kozlovic
bec51ed52b [FIXED] JetStream: User given named ephemeral lost after migration
If an ephemeral was given a name by the user, if the consumer leader
was then shutdown, the ephemeral would be migrated using a server
generated new name instead of keeping the user given name.

Also, in some cases the migration would not even occur. This was
likely due to the fact that RAFT node(s) were shutdown prior to
the ephemeral migration code was invoked.

Resolves #3550

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-14 15:20:45 -06:00
Ashley Nelson
c909847571 Prevent nil map assignment panic if disk is full
If the server is unable to write to disk, there is a failure mode that
results in the fss map being nil when writing message block data to it,
resulting in a panic. This fix checks whether an error is returned by
the function that loads the fss map, and if so, returns an error to
prevent future assignment.
2022-10-14 19:35:39 +00:00
Neil Alexander
ff23c217ea Add missing RUnlock in needAck 2022-10-14 11:54:06 +01:00
Ivan Kozlovic
dde94987ce [FIXED] MQTT: Subjects mapping were not handled
A simple configuration like this:
```
...
mappings = {
  foo: bar
}

mqtt {
   port: 1883
}
```
would cause an MQTT subscription on "bar" to not receive messages
published on "foo".

In otherwords, the subject transformation was not done when parsing
a PUBLISH packet.

This PR also handles the case of service imports where transformation
occurs after the initial publish parsing.

Resolves #3547

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-13 16:00:05 -06:00
Ivan Kozlovic
04c4cce4e9 Merge pull request #3543 from nats-io/fix_flappers_update_cov_go
Fix some flappers and update Go version for nightly code coverage
2022-10-10 17:28:22 -06:00
Ivan Kozlovic
a98af54d92 Fix some flappers and update Go version for nightly code coverage
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-10 16:42:43 -06:00
Ivan Kozlovic
ccfc818e8b Bump version to v2.9.4-beta as per release instructions
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-10 15:44:44 -06:00
Ivan Kozlovic
096b82271e Release v2.9.3
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-10 14:52:49 -06:00
Ivan Kozlovic
3358247e6b Added warning if internal sub callback takes too long
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-10 14:39:37 -06:00
Ivan Kozlovic
9bd11580e3 [FIXED] JetStream: User-defined ephemeral Name not used in cluster mode
If the user sends a CONSUMER.CREATE request with a configuration that
specifies the name that the user wants for the ephemeral consumer,
this would not work on cluster mode, that is, the server would still
pick a name instead of using the provided one.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-10 13:48:38 -06:00
Ivan Kozlovic
e42260cc80 Merge pull request #3539 from nats-io/js_handle_bad_replicas_count
[FIXED] JetStream: return error on negative replicas count
2022-10-10 13:44:34 -06:00
Ivan Kozlovic
c3adf78702 [FIXED] Stack overflow when account imports its own export
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-10 13:20:26 -06:00
Ivan Kozlovic
3c7aa554f7 [FIXED] JetStream: return error on negative replicas count
If a stream is created or updated with a negative replicas count,
and error is now returned. Same for consumers.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-10 12:32:41 -06:00
Derek Collison
c88784dcad Test to make sure a consumer that is deleted while a server is down recovers correctly.
Signed-off-by: Derek Collison <derek@nats.io>
2022-10-07 09:24:54 -07:00
Ivan Kozlovic
57a3594355 [FIXED] JetStream: Purge with additional options may leave some messages
While going over message blocks, when some blocks were removed there
was a risk that a block is ignored, leaving some messages around.

Resolves #3528

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-06 13:21:20 -06:00
Ivan Kozlovic
37ca972251 Bump version to 2.9.3-beta.1
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-05 07:14:01 -06:00
Ivan Kozlovic
3472f6aec2 [FIXED] JetStream: unresponsiveness while creating raft group
Originally, createRaftGroup() would not hold the jetstream's lock
for the whole duration. But some race reports made us change
this function to keep the lock for the whole duration. A test
called TestJetStreamClusterRaceOnRAFTCreate() was demonstrating
the race between "consumer info" request handling and createRaftGroup
code. Since then, the race has been fixed, so this PR restores
the more fine-grained locking inside createRaftGroup.

Resolves #3516

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-10-04 17:27:36 -06:00
Ivan Kozlovic
9f4513c9f7 Bump version to v2.9.3-beta as part of release process
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-09-30 10:27:58 -06:00
Ivan Kozlovic
34851bfeae Release v2.9.2
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-09-29 15:21:53 -06:00
Ivan Kozlovic
1039df088c [UPDATED] Dependencies
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-09-29 15:00:44 -06:00
Ivan Kozlovic
46aec649e4 [FIXED] JetStream: redeliveries for LastPerSubject delivery policy
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-09-29 14:00:18 -06:00
Ivan Kozlovic
0841786d6c Bump to 2.9.2-beta.1
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-09-29 09:26:09 -06:00
Derek Collison
91edd1a8d0 With snapshots both streams are present on restart so sources or mirrors that have a subject change from the origin would not recover.
We now suppress that if we know we are recovering an existing stream.

Signed-off-by: Derek Collison <derek@nats.io>
2022-09-29 09:17:15 -06:00
Derek Collison
52b5cd12bb Allow meta layer to snapshot on a clean shutdown.
Signed-off-by: Derek Collison <derek@nats.io>
2022-09-29 09:17:12 -06:00
Derek Collison
fef702a688 [FIXED] bug in consumer names paging, did not honor limits and returned duplicate results.
Signed-off-by: Derek Collison <derek@nats.io>
2022-09-29 06:14:00 -07:00
Ivan Kozlovic
8d9c57ad44 [IMPROVED] Fan-out performance
There was an observed degradation (around 5%) for large fan out in
v2.9.0 compared to earlier release. This is because we added
accounting of the in/out messages for the account, which result
in 4 atomic operations, 2 for in and 2 for out, however, it means
that for a fan-out of say 100 matching subscriptions, it is now
2 + 2 * 100 = 202.

This PR rework how the stats accounting is done which removes
the regression and even boost a bit the numbers since we are
doing the server stats update as an aggregate too.

There are still degradation for queues and no-sub at all that
need to be looked at.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-09-27 19:43:32 -06:00
Ivan Kozlovic
8247ecbf20 Merge pull request #3502 from nats-io/fix_3493
[FIXED] JetStream: Scale down of consumer to R1 would not get a response
2022-09-27 15:33:38 -06:00
Ivan Kozlovic
e151cfcd57 [FIXED] JetStream: Scale down of consumer to R1 would not get a response
Updating a consumer configuration from say R3 to R1 would work
but no response was received by the client sending the request.

Resolves #3493

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-09-27 10:02:31 -06:00