gogrlx/nats-server
1
0
Fork 0
mirror of https://github.com/gogrlx/nats-server.git synced 2026-04-02 03:38:42 -07:00
Code Issues Packages Projects Releases Wiki Activity
8,406 Commits 30 Branches 48 Tags
main
Commit Graph

8406 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Derek Collison
2d2bb77f6e Optimize for restore time. 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-03 14:14:00 -07:00
Derek Collison
4a5b76b0e8 Print out restore time for streams to nearest millisecond. 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-03 13:28:18 -07:00
Derek Collison
e11ddb8bfe Merge branch 'main' into dev 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-02 14:22:57 -07:00
Derek Collison
8b35c01637 [FIXED] Fix for a bug that would make normal streams use the wrong block size. (#4478) 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-02 14:14:50 -07:00
Derek Collison
34ae2bf4cb Fix for a bug that would make normal streams use the wrong block size. 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-02 13:56:34 -07:00
Derek Collison
1bb4a71a4d Merge branch 'main' into dev 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-02 12:15:40 -07:00
Derek Collison
b63318c0c9 Bump to 2.9.22-RC.5 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-02 12:00:39 -07:00
Derek Collison
f61ad322c9 [FIXED] Interface conversion bug for ipQueues in monitor which would cause panics. (#4477) 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-02 12:00:07 -07:00
Derek Collison
2c81224262 Fixed interface conversion for ipQueue in monitor which caused panics. 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-02 11:43:08 -07:00
Derek Collison
7f13ecc87b Refine core and TLS benchmarks (#4475) 
- [ ] Link to issue, e.g. `Resolves #NNN`
 - [ ] Documentation added (if applicable)
 - [x] Tests added
- [x] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [ ] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [ ] Build is green in Travis CI
- [x] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

Related: https://github.com/nats-io/nats-server/pull/4399

### Changes proposed in this pull request:

 - Avoid binding to hardcoded port, as it may conflict if already in use
- Drop TLS from Core Request/Response benchmark, was not showing much
difference, not enough data going through to become a dominant cost
- Add a different core benchmark that exercises TLS by pushing through
lots of data
 2023-09-01 13:58:27 -07:00
Marco Primi
e61e40e3fe Add benchmark for TLS content encryption overhead 2023-09-01 12:58:52 -07:00
Marco Primi
4eedfe8d0c Simplify core request/response benchmark 
- Remove TLS, impact is negligible for the amount of data pushed
through
 - Rename benchmark
 2023-09-01 12:58:52 -07:00
Marco Primi
bbf42c1f57 Use dynamic port number in benchmark 2023-09-01 12:58:52 -07:00
Jean-Noël Moyne
db96238ad9 Enables 0s deduplication window duration when the stream has sources (#4476) 
- [X] Link to issue, e.g. `Resolves #NNN`
- [X] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [X] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

Resolves #4459

Allows the user to set the deduplication window duration to 0s when the
stream has sources defined. Remember that if the stream in question is
also listening on subjects as well as sourcing the deduplication window
is the same for sourced and listened messages.

---------

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
 2023-09-01 12:47:14 -07:00
Derek Collison
ad380d48f2 Merge branch 'main' into dev 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-01 11:19:33 -07:00
Derek Collison
c9e0de3358 Update to dependencies (#4474) 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-01 10:12:10 -07:00
Derek Collison
c3648d27bd Update to dependencies 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-01 09:54:16 -07:00
Derek Collison
aca74d0b27 Bump to 2.9.22-RC.4 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-01 09:44:53 -07:00
Waldemar Quevedo
ed8b50d943 Fix monitoring server connz idle time sorting (#4463) 
- Changed how [`byIdle`](887a4ae692/server/monitor_sort_opts.go (L97))
struct compares the idle times (was subtracting the start time from the last activity time).

 - Added tests for `byIdle`.

- Changed and simplified the test
[`TestConnzSortedByIdle`](8a9f441c40/server/monitor_test.go (L1185))
(No need for the clients to publish and subscribe if we are manually
changing the client's `last` time, and we only need to test the sorting
order). Also the test was not catching the problem because the clients
`start` time was the same, now every client has a different start time.

Resolves #4462
 2023-09-01 09:44:19 -07:00
Derek Collison
f6aaea195e [FIXED] We should update accounting before clearing ebit (#4473) 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-01 09:43:11 -07:00
Derek Collison
4422a95a8e We should update accounting before clearing ebit 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-01 09:31:12 -07:00
Derek Collison
a2373d9162 [IMPROVED] Consumer failing to deliver re-adjusts delivered count and any waiting request. (#4472) 
When we fail to deliver a message for a consumer, either through
didNotDeliver() or LoadMsg() failure re-adjust delivered count and
waitingRequest accounting.

Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-01 09:30:22 -07:00
Derek Collison
c679f9d7f6 Added in detail info when failing to load in a message for a consumer. 
E.g. `Unexpected partial cache error looking up message for consumer '$G > TEST > dlc'`

Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-01 09:06:29 -07:00
Derek Collison
3a39786972 When we fail to deliver a message for a consumer, either through didNotDeliver() or LoadMsg() failure re-adjust delivered count and waitingRequest accounting. 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-09-01 08:48:28 -07:00
Neil
752d35015c Consumers inherit defaults/limits for max_ack_pending and inactive_threshold from stream (#4105) 
Closes #3727 by adding the ability to set `limit_max_ack_pending` and
`limit_inactive_threshold` values at the stream level, which consumers
will automatically inherit if they don't specify their own values and
will limit to if the stream limits are reduced.

Signed-off-by: Neil Twigg <neil@nats.io>
 2023-09-01 15:36:01 +01:00
Pierre Mdawar
d24d51292f Fix monitoring server connz idle time sorting 2023-09-01 14:32:08 +03:00
Neil Twigg
487f58f16e Consumers inherit limits for max_ack_pending and inactive_threshold from stream 
Signed-off-by: Neil Twigg <neil@nats.io>
 2023-09-01 10:54:11 +01:00
Derek Collison
0fadaf211f Fix for a filestore data race on hash during snapshots (#4470) 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 20:38:32 -07:00
Derek Collison
4df5f515ca Fix for filestore data race on hash during snapshots 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 19:38:09 -07:00
Derek Collison
cb8b94a9e9 Fixes to /healthz response (v2.10) (#4467) 
Follow up from #4437 content-type fix for v2.9.22, some fixes to the
response from `/healthz` for dev:

- In #[3326](https://github.com/nats-io/nats-server/pull/4097) it was
changed to return 500 status when before we used to return 503 so this
changes it back.
- Also as part of #3326 we started to return `status_code` in the
healthz response (e.g `{"status":"ok","status_code":200}`) so this
removes it for http responses just relying on the http header.
 2023-08-31 19:26:33 -07:00
Derek Collison
83fab5c9a7 [FIXED] Unlock panic on start when filestore needs to remove msgs for enforcement. (#4469) 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 19:26:03 -07:00
Derek Collison
0ec42f85f0 Fix for merge issue that duplicated the index increment, causing snapshot tests to fail 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 18:51:34 -07:00
Derek Collison
411ac175fc Fixed: MQTT: more consistent name for PUBREL durable (#4466) 
Resolves: no ticket

### Changes proposed in this pull request:

- rename PUBREL durable consumer from `<idhash>_pubrel` to
`$MQTT_PUBREL_<idhash>` for consistency with other durable consumer
names.
 2023-08-31 17:15:00 -07:00
Derek Collison
60fa2d8781 Only have removeMsg release lock if it really has a callback. 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 16:56:40 -07:00
Derek Collison
9ff3261af2 On startup make sure to hold lock for enforcing limits due to removeMsg() needing to remove msgs possibly. 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 16:56:35 -07:00
Waldemar Quevedo
1f2d56a554 Fixes to http healthz monitoring response 
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
 2023-08-31 16:05:09 -07:00
Derek Collison
b1a59a35e2 Bump to 2.10.0-beta.54 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 15:52:58 -07:00
Derek Collison
2bfa14d9bd Fix from main merge 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 15:52:36 -07:00
Derek Collison
49c30b6d2f Merge branch 'main' into dev 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 15:52:00 -07:00
Derek Collison
45e6812d70 [FIXED] Sending too fast to have replicas be caught up enough to register directs. (#4468) 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 15:43:14 -07:00
Derek Collison
afb052651a Sending too fast to have replicas be caught up enough to register direct subs 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 15:16:19 -07:00
Derek Collison
d7ea3b94d9 [FIXED] Check for checksum violations for all records and before any sequence processing. (#4465) 
Also small bug fix for leaking fds under certain scenarios during
corruption.

Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 15:08:04 -07:00
Derek Collison
a45281d51f Added check to test 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 14:00:14 -07:00
Pierre Mdawar
6d6d3cfa55 Fix Content-Type header in /healthz when status is not 200 OK (#4437) 
- Added a new internal function `handleResponse` that accepts the HTTP 
  status code and sets it after setting the headers
- Added tests for the `/healthz` endpoint for the `ok`, `error` and `unavailable` statuses
- Changed the IETF API health check URL to 
https://datatracker.ietf.org/doc/html/draft-inadarei-api-health-check

Resolves #4436
 2023-08-31 13:55:20 -07:00
Derek Collison
c110ceea94 Check for checksum violations for all records and before sequence processing. 
Also fix for bitrot test and a small bug fix for a leaking fd.

Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 13:53:28 -07:00
Derek Collison
7c8f402264 Fix data race when updating account (#4435) 
Fixes race that would make the `TestJetStreamJWTMove` test fail
sometimes:

[0]:
f1bf4127c5/server/accounts.go (L3535)
[1]:
f1bf4127c5/server/server.go (L1902)

 ```
=== FAIL: server TestJetStreamJWTMove/non-tiered/R1 (4.79s)
==================
WARNING: DATA RACE
Write at 0x00c0014631f8 by goroutine 22900:

github.com/nats-io/nats-server/v2/server.(*Server).updateAccountClaimsWithRefresh()
      /go/server/accounts.go:3535 +0x53dc

github.com/nats-io/nats-server/v2/server.(*Server).UpdateAccountClaims()
      /go/server/accounts.go:3074 +0x45

github.com/nats-io/nats-server/v2/server.(*Server).updateAccountWithClaimJWT()
      /go/server/server.go:1937 +0x3e5
  github.com/nats-io/nats-server/v2/server.(*Server).updateAccount()
      /go/server/server.go:1910 +0x1f1
  github.com/nats-io/nats-server/v2/server.(*Server).lookupAccount()
      /go/server/server.go:1875 +0x176
  github.com/nats-io/nats-server/v2/server.(*Server).LookupAccount()
      /go/server/server.go:1895 +0x2e4
  github.com/nats-io/nats-server/v2/server.(*Server).getRequestInfo()
      /go/server/jetstream_api.go:936 +0x2b4

github.com/nats-io/nats-server/v2/server.(*Server).jsStreamCreateRequest()
      /go/server/jetstream_api.go:1285 +0xca

github.com/nats-io/nats-server/v2/server.(*Server).jsStreamCreateRequest-fm()
      <autogenerated>:1 +0xcc

github.com/nats-io/nats-server/v2/server.(*Server).processJSAPIRoutedRequests()
      /go/server/jetstream_api.go:799 +0x60c

github.com/nats-io/nats-server/v2/server.(*Server).processJSAPIRoutedRequests-fm()
      <autogenerated>:1 +0x39

github.com/nats-io/nats-server/v2/server.(*Server).startGoRoutine.func1()
      /go/server/server.go:3604 +0x27d
 
Previous read at 0x00c0014631f8 by goroutine 22995:
  github.com/nats-io/nats-server/v2/server.(*Server).updateAccount()
      /go/server/server.go:1902 +0x6d
  github.com/nats-io/nats-server/v2/server.(*Server).lookupAccount()
      /go/server/server.go:1875 +0x176
  github.com/nats-io/nats-server/v2/server.(*Server).LookupAccount()
      /go/server/server.go:1895 +0x4e

github.com/nats-io/nats-server/v2/server.(*Server).updateInterestForAccountOnGateway()
      /go/server/leafnode.go:2030 +0x3a

github.com/nats-io/nats-server/v2/server.(*client).processGatewayRSub.func1()
      /go/server/gateway.go:1966 +0xc4
  runtime.deferreturn()
      /usr/local/go/src/runtime/panic.go:476 +0x32
  github.com/nats-io/nats-server/v2/server.(*client).parse()
      /go/server/parser.go:664 +0x40b7
  github.com/nats-io/nats-server/v2/server.(*client).readLoop()
      /go/server/client.go:1373 +0x1c98

github.com/nats-io/nats-server/v2/server.(*Server).createGateway.func1()
      /go/server/gateway.go:858 +0x37

github.com/nats-io/nats-server/v2/server.(*Server).startGoRoutine.func1()
      /go/server/server.go:3604 +0x27d
 
Goroutine 22900 (running) created at:
  github.com/nats-io/nats-server/v2/server.(*Server).startGoRoutine()
      /go/server/server.go:3600 +0x2f2

github.com/nats-io/nats-server/v2/server.(*Server).setJetStreamExportSubs()
      /go/server/jetstream_api.go:820 +0x178
  github.com/nats-io/nats-server/v2/server.(*Server).enableJetStream()
      /go/server/jetstream.go:425 +0xcf1
  github.com/nats-io/nats-server/v2/server.(*Server).EnableJetStream()
      /go/server/jetstream.go:217 +0x6f7
  github.com/nats-io/nats-server/v2/server.(*Server).Start()
      /go/server/server.go:2218 +0x1924
  github.com/nats-io/nats-server/v2/server.RunServer()
      /go/server/server_test.go:95 +0x30e
  github.com/nats-io/nats-server/v2/server.RunServerWithConfig()
      /go/server/server_test.go:117 +0x44

github.com/nats-io/nats-server/v2/server.createJetStreamSuperClusterWithTemplateAndModHook()
      /go/server/jetstream_helpers_test.go:449 +0x1331
  github.com/nats-io/nats-server/v2/server.TestJetStreamJWTMove.func1()
      /go/server/jetstream_jwt_test.go:303 +0x204
github.com/nats-io/nats-server/v2/server.TestJetStreamJWTMove.func3.2()
      /go/server/jetstream_jwt_test.go:409 +0x50
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1576 +0x216
  testing.(*T).Run.func1()
      /usr/local/go/src/testing/testing.go:1629 +0x47
 
Goroutine 22995 (running) created at:
  github.com/nats-io/nats-server/v2/server.(*Server).startGoRoutine()
      /go/server/server.go:3600 +0x2f2
  github.com/nats-io/nats-server/v2/server.(*Server).createGateway()
      /go/server/gateway.go:858 +0xf04
  github.com/nats-io/nats-server/v2/server.(*Server).solicitGateway()
      /go/server/gateway.go:707 +0x12e7

github.com/nats-io/nats-server/v2/server.(*Server).solicitGateways.func1()
      /go/server/gateway.go:643 +0x44

github.com/nats-io/nats-server/v2/server.(*Server).startGoRoutine.func1()
      /go/server/server.go:3604 +0x27d
==================
    testing.go:1446: race detected during execution of test
        --- FAIL: TestJetStreamJWTMove/non-tiered/R1 (4.79s)
 
=== FAIL: server TestJetStreamJWTMove/non-tiered (11.03s)
    testing.go:1446: race detected during execution of test
    --- FAIL: TestJetStreamJWTMove/non-tiered (11.03s)
 
=== FAIL: server TestJetStreamJWTMove (23.30s)
    testing.go:1446: race detected during execution of test
```
 2023-08-31 13:46:17 -07:00
Lev Brouk
8de48339d3 Fixed: MQTT: more consistent name for PUBREL durable 2023-08-31 12:46:13 -07:00
Waldemar Quevedo
76c3942609 Fix leaf connection missing LS+ sometimes (#4464) 
`TestNoRaceLeafNodeSmapUpdate` could occasionally fail with missing
`LS+` commands due not capturing all the inflight SUB commands as they
were being processed outside the client lock.
 2023-08-31 11:18:00 -07:00
Ivan Kozlovic
9a9e84ea5c Fix leaf connection missing LS+ sometimes 
Signed-off-by: Waldemar Quevedo <wally@nats.io>
 2023-08-31 10:06:02 -07:00
Derek Collison
2834142bdd Revert lock guard 
Signed-off-by: Derek Collison <derek@nats.io>
 2023-08-31 08:59:15 -07:00