Commit Graph

308 Commits

Author SHA1 Message Date
Ivan Kozlovic
06bc0fe8b8 [ADDED] LeafNode: Support for s2 compression (#4167)
This is similar to PR #4115 but for LeafNodes.
Compression mode can be set on both side (the accept and in remotes).
```
leafnodes {
   port: 7422
   compression: s2_best
   remotes [
       {
         url: "nats://host2:74222"
         compression: s2_better
       }
   ]
}
```
Possible modes are similar than for routes (described in PR #4115),
except that when not defined we default to `s2_auto`.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-16 14:34:25 -06:00
Ivan Kozlovic
67498af2dc [ADDED] LeafNode: Support for s2 compression
This is similar to PR #4115 but for LeafNodes.
Compression mode can be set on both side (the accept and in remotes).
```
leafnodes {
   port: 7422
   compression: s2_best
   remotes [
       {
         url: "nats://host2:74222"
         compression: s2_better
       }
   ]
}
```
Possible modes are similar than for routes (described in PR #4115),
except that when not defined we default to `s2_auto`.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-15 17:42:39 -06:00
Derek Collison
4220502541 Merge branch 'main' into dev 2023-05-15 15:44:38 -07:00
Waldemar Quevedo
ee38f8bbc5 monitor: change account detail info back to utc when served (#4163)
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-05-15 15:33:57 -07:00
Derek Collison
d293af1da6 Fix to service imports reporting for Accountz() when import subject is mapped into different local subject.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-13 12:57:05 -07:00
Derek Collison
3ff9aed192 Merge branch 'main' into dev 2023-05-12 21:04:51 -07:00
Derek Collison
421775a32a Fix to allow sorting by rtt for connz.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 20:22:07 -07:00
Derek Collison
7f17e07d66 Filter by user at the end for closed connections
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 15:24:42 -07:00
Derek Collison
c5eb46cb06 Make sure closed clients captures all user types and works with user filtering as well
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 15:05:40 -07:00
Derek Collison
90d1063674 Fix for #4149 to allow proper user filtering on connz for other user types.
Signed-off-by: Derek Collison <derek@nats.io>
2023-05-12 14:19:37 -07:00
Derek Collison
4c26cbb3de Merge branch 'main' into dev 2023-05-12 12:38:20 -07:00
Waldemar Quevedo
286a1632ca Use monotonic time for measuring time internally
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-05-12 08:27:46 -07:00
Derek Collison
0321eb6484 Merge branch 'main' into dev 2023-04-29 19:52:57 -07:00
Derek Collison
d107ba3549 Under certain scenarios we have witnessed healthz() that never retrun healthy due to a stream or consumer being missing or stopped.
This will now allow the healthy call to attempt to restart those assets.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-28 17:11:08 -07:00
Ivan Kozlovic
5b8c9ee364 Changes based on code review
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-04-28 14:34:32 -06:00
Ivan Kozlovic
d6fe9d4c2d [ADDED] Support for route S2 compression
The new field `compression` in the `cluster{}` block allows to
specify which compression mode to use between servers.

It can be simply specified as a boolean or a string for the
simple modes, or as an object for the "s2_auto" mode where
a list of RTT thresholds can be specified.

By default, if no compression field is specified, the server
will use the s2_auto mode with default RTT thresholds of
10ms, 50ms and 100ms for the "uncompressed", "fast", "better"
and "best" modes.

```
cluster {
..
  # Possible values are "disabled", "off", "enabled", "on",
  # "accept", "s2_fast", "s2_better", "s2_best" or "s2_auto"
  compression: s2_fast
}
```

To specify a different list of thresholds for the s2_auto,
here is how it would look like:
```
cluster {
..
  compression: {
    mode: s2_auto
    # This means that for RTT up to 5ms (included), then
    # the compression level will be "uncompressed", then
    # from 5ms+ to 15ms, the mode will switch to "s2_fast",
    # then from 15ms+ to 50ms, the level will switch to
    # "s2_better", and anything above 50ms will result
    # in the "s2_best" compression mode.
    rtt_thresholds: [5ms, 15ms, 50ms]
  }
}
```

Note that the "accept" mode means that a server will accept
compression from a remote and switch to that same compression
mode, but will otherwise not initiate compression. That is,
if 2 servers are configured with "accept", then compression
will actually be "off". If one of the server had say s2_fast
then they would both use this mode.

If a server has compression mode set (other than "off") but
connects to an older server, there will be no compression between
those 2 routes.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-04-27 17:59:25 -06:00
Derek Collison
4ebdb69daf Merge branch 'main' into dev 2023-04-26 11:34:37 -07:00
Derek Collison
cae91b8cad In single server mode healthz could mistake a snapshot staging directory during a restore as an account.
If the restore took a long time, stalled, or was aborted, would cause healthz to fail.

Signed-off-by: Derek Collison <derek@nats.io>
2023-04-24 22:14:04 -07:00
Derek Collison
8375ab5cde Merge branch 'main' into dev 2023-04-14 16:44:25 -07:00
Waldemar Quevedo
d12152c48f Add server name / remote server name to routez
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-04-14 12:47:00 -07:00
Derek Collison
ce0d8514be Merge branch 'main' into dev 2023-04-07 05:32:05 -07:00
Derek Collison
c16915bff4 For checking the health of jetstream, do not hold the lock as we traverse the streams and consumers.
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-06 11:56:55 -07:00
Derek Collison
1ae51b23a9 [ADDED] Multiple routes and ability to have per-account routes (#4001)
New configuration fields:
```
cluster {
   ...
   pool_size: 5
   accounts: ["A", "B"]
}
```

The configuration `pool_size` in the example above means that this
server will create 5 routes to a remote server, assuming that that
server has the same `pool_size` setting.

Accounts (which are not part of the `accounts[]` configuration)
are assigned a specific route in this pool, and this will be the
same route on all servers in the cluster.

Accounts that are defined in the `accounts` field will each have
a dedicated route connection. This will allow suppression of the
account name in some of the route protocols, reducing bytes transmitted
which may increase performance.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-04-03 15:33:46 -07:00
Derek Collison
01a2c0472d Merge branch 'main' into dev 2023-04-03 15:33:12 -07:00
Derek Collison
59175c491f Fix for a datarace
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-03 14:46:57 -07:00
Ivan Kozlovic
105237cba8 [ADDED] Multiple routes and ability to have per-account routes
New configuration fields:
```
cluster {
   ...
   pool_size: 5
   accounts: ["A", "B"]
}
```

The configuration `pool_size` in the example above means that this
server will create 5 routes to a remote server, assuming that that
server has the same `pool_size` setting.

Accounts (which are not part of the `accounts[]` configuration)
are assigned a specific route in this pool, and this will be the
same route on all servers in the cluster.

Accounts that are defined in the `accounts` field will each have
a dedicated route connection. This will allow suppression of the
account name in some of the route protocols, reducing bytes transmitted
which may increase performance.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-04-03 09:32:25 -06:00
Derek Collison
b9e7b58f5c Merge branch 'main' into dev 2023-04-02 18:58:54 -07:00
Derek Collison
ff3f102cdd Fix for datarace in healthcheck
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 16:30:13 -07:00
Derek Collison
48a5d270b2 Merge branch 'main' into dev 2023-04-02 04:23:52 -07:00
Derek Collison
4b8229ee42 Do not hold js lock for health check, use healthy not current for meta
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-02 03:52:54 -07:00
Derek Collison
6dd0f5a660 Merge branch 'main' into dev 2023-03-19 11:55:45 -07:00
Derek Collison
027f2e42c8 Remove snapshot of cores and maxprocs
Signed-off-by: Derek Collison <derek@nats.io>
2023-03-17 15:09:50 -07:00
Derek Collison
6507a913b3 Merge branch 'main' into dev 2023-03-01 05:05:41 -08:00
Jeremy Saenz
26f241cb62 Updated LEAFZ names to use remoteServer name/id and added is_spoke 2023-02-28 18:09:24 -08:00
Derek Collison
7bd7cda021 Merge branch 'main' into dev 2023-02-28 15:17:24 -08:00
Jeremy Saenz
9d4a603aaf Update LEAFZ to include leafnode server/connection name 2023-02-28 14:20:18 -08:00
Waldemar Quevedo
891064318f Add raft query parameter to /jsz to include raft group info
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-02-27 05:54:44 -08:00
Waldemar Quevedo
74b703549d Add raft query parameter to /jsz to include raft group info
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-02-27 05:42:11 -08:00
Derek Collison
7ce23c46ce Merge branch 'main' into dev 2023-02-21 08:34:08 -08:00
Neil Twigg
68961ffedd Refactor ipQueue to use generics, reduce allocations 2023-02-21 14:50:09 +00:00
Derek Collison
469116deae Merge branch 'main' into dev 2023-01-26 09:39:32 -08:00
Neil Twigg
83932b4be6 Don't mark a clustered stream as unhealthy if making forward progress, add TestJetStreamClusterCurrentVsHealth 2023-01-26 16:57:34 +00:00
Derek Collison
3c4e47d540 Merge branch 'main' into dev 2023-01-20 13:29:55 -08:00
Derek Collison
2aeb5e2c5a Update snapshots to numCores and maxProcs after maxrocs.Set()
Signed-off-by: Derek Collison <derek@nats.io>
2023-01-20 11:30:43 -08:00
Neil Twigg
68953678bb Add profilez server endpoint for retrieving pprof profiles 2023-01-11 16:09:09 +00:00
Derek Collison
713f632fa7 If a stream's meta was not properly written but the file existed, we could re-add the stream but a subsequent restart would lose the stream again.
Also added in healthz for single server systems to make sure all stream directories resulted in recovered streams.

Signed-off-by: Derek Collison <derek@nats.io>
2022-12-29 20:08:56 -08:00
Byron Ruth
1477b675ff Add back existing HealthzOptions.JSEnabled field
This fixes a backwards incompat change for library usage as well as
using the healthz NATS API which depends on the JSON payload.

Signed-off-by: Byron Ruth <byron@nats.io>
2022-12-26 08:45:22 -05:00
Byron Ruth
566d1adfa7 Fix /healthz?js-enabled=true behavior
When js-enabled is set to true, the condition was only checked if
the `getJetStream()` call returned `nil`. However, if it non-nil,
all remaining checks were executed, including assessing the health
of the assets (streams and consumers).

This change addresses two issues:

- Switch to use `js.isEnabled()` which will check whether the value
  is nil OR `js.disabled = true` which can occur if the subsystem
  is temporarily disabled (insufficient resources).
- Correctly exit the check after the assertion and before meta and
  asset checks are performed.

In addition, the option has been renamed to `js-enabled-only` to align
with the `js-server-only` naming. The previous `js-enabled` name still
works, but is mapped to this new option. A warning is emitted noting
the previous option is deprecated.

Fix #3703

Signed-off-by: Byron Ruth <b@devel.io>
2022-12-10 07:34:32 -05:00
Raymond
4d8964e57b Added stream created timestamp to stream detail 2022-11-17 13:59:58 +01:00
Ivan Kozlovic
170ff49837 [ADDED] JetStream: peer (the hash of server name) in statsz/jsz
A request to `$SYS.REQ.SERVER.PING.JSZ` would now return something
like this:
```
...
    "meta_cluster": {
      "name": "local",
      "leader": "A",
      "peer": "NUmM6cRx",
      "replicas": [
        {
          "name": "B",
          "current": true,
          "active": 690369000,
          "peer": "b2oh2L6w"
        },
        {
          "name": "Server name unknown at this time (peerID: jZ6RvVRH)",
          "current": false,
          "offline": true,
          "active": 0,
          "peer": "jZ6RvVRH"
        }
      ],
      "cluster_size": 3
    }
```
Note the "peer" field following the "leader" field that contains
the server name. The new field is the node ID, which is a hash of
the server name.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2022-09-16 15:31:37 -06:00