Commit Graph

315 Commits

Author SHA1 Message Date
Derek Collison
f342f6a758 Merge branch 'main' into dev 2023-06-05 14:13:18 -07:00
Derek Collison
df5df3ce99 Panic fixes (#4214)
- [ ] Link to issue, e.g. `Resolves #NNN`
 - [ ] Documentation added (if applicable)
 - [ ] Tests added
- [ ] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [ ] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [x] Build is green in Travis CI
- [x] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

Resolves panics in the code.

### Changes proposed in this pull request:

 - This PR fixes some of the panics in the code
2023-06-05 13:02:05 -07:00
Derek Collison
df901dc1aa Merge branch 'main' into dev 2023-06-02 16:45:07 -07:00
Derek Collison
1bce79750e When we were optimizing for single cluster but large number of leafnodes we inadvertently broke a daisy chained scenarion where a server was a spoke and a hub with a single hub cluster.
Signed-off-by: Derek Collison <derek@nats.io>
2023-06-02 15:16:36 -07:00
Artem Seleznev
27a8b96ee3 different panic fixes
Signed-off-by: Artem Seleznev <seleznyov.artyom@gmail.com>
2023-06-02 13:19:22 +03:00
Jean-Noël Moyne
2b843ba9e3 Allows cluster filtering in account subject mapping
Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-05-17 13:34:35 -07:00
Derek Collison
7760aa5107 Merge branch 'main' into dev 2023-05-16 14:01:57 -07:00
Savion
cd192f0e03 CHANGED - typo err 2023-05-16 16:41:52 +08:00
Derek Collison
4220502541 Merge branch 'main' into dev 2023-05-15 15:44:38 -07:00
Jean-Noël Moyne
e07ccf9cc1 [ADDED] Ability to drop partial wildcard tokens in some subject transforms (#4152)
- [X] Tests added
- [X] Branch rebased on top of current main (`git pull --rebase origin
main`)
- [X] Changes squashed to a single commit (described
[here](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 - [X] Build is green in Travis CI
- [X] You have certified that the contribution is your original work and
that you license the work to the project under the [Apache 2
license](https://github.com/nats-io/nats-server/blob/main/LICENSE)

### Changes proposed in this pull request:

There is currently a blanket requirement that subject transforms
destinations MUST use ALL of the partial wildcards defined in the source
of the transform. This is because the subject transformed defined for
imports must be 'reversible' and therefore the destination transform
must use all of the partial wildcard tokens defined in the source of the
transform.

This reversing of a transform is only used for transforms used by
imports, where in any case it doesn't make sense to use any transform
other than Wildcard.

This PR:
- relaxes this requirement to only apply when the transform is used by
an import, adding the ability to drop a wildcard token in transforms
other than as part of an import.
- Improves transform reverse to support both legacy style wildcards $X
and the new transform function {{Wildcard(X)}}- Improves reversible
transform checking to only allow the use of wildcards in the
destination.

---------

Signed-off-by: Jean-Noël Moyne <jnmoyne@gmail.com>
2023-05-14 14:02:19 -07:00
Waldemar Quevedo
3c4ed549a5 resolver: improve signaling for missing account lookups (#4151)
When using the nats account resolver and a JWT is not found, the client could
often get an i/o timeout error due to not receiving a timely response
before the account resolver fetch request times out. Now instead
of waiting for the fetch request to timeout, a resolver without JWTs
will notify as well that it could not find a matching JWT, waiting for a
response from all active servers.

Also included in this PR is some cleanup to the logs emitted by the
resolver.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-05-14 11:10:25 -07:00
Derek Collison
4c26cbb3de Merge branch 'main' into dev 2023-05-12 12:38:20 -07:00
Waldemar Quevedo
286a1632ca Use monotonic time for measuring time internally
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2023-05-12 08:27:46 -07:00
Derek Collison
4175e4ee9c Merge branch 'main' into dev 2023-05-06 09:55:34 -07:00
Derek Collison
80db7a22ab Optimizations for large single hub account leafnode fleets.
Added a leafnode lock to allow better traversal without copying of large leafnodes in a single hub account.

Signed-off-by: Derek Collison <derek@nats.io>
2023-05-05 13:14:49 -07:00
Ivan Kozlovic
95e4f2dfe1 Fixed accounts configuration reload
Issues could manifest with subscription interest not properly
propagated.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-05-03 14:35:06 -06:00
Derek Collison
e158c46884 Merge branch 'main' into dev 2023-04-30 17:37:47 -07:00
Derek Collison
c15cc0054a When a fleet of leafnodes are isolated (not routed but using same cluster) we could do better at optimizing how we update the other leafnodes.
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-30 17:08:16 -07:00
Ivan Kozlovic
8d2683a062 Fixed data race
Reverts changes made in PR#4001: 105237cba8 (diff-1322a81c43dfdd05284ae128c43d9ea51c1a3b677587686561ef6de47024e14aR1340)

Since a fix was made here: b78ec39b1f
the changes made in PR need to be reverted. The test
TestRoutePoolAndPerAccountWithServiceLatencyNoDataRace now passes.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-04-28 10:18:14 -06:00
Derek Collison
4ebdb69daf Merge branch 'main' into dev 2023-04-26 11:34:37 -07:00
cui fliter
f1f5a59e9b fix some comments
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-04-25 11:28:59 +08:00
Derek Collison
09afcee9d9 Merge branch 'main' into dev 2023-04-17 08:43:08 -07:00
Derek Collison
9a3e0b783c Fix for a data race when setting up service import subscriptions.
Signed-off-by: Derek Collison <derek@nats.io>
2023-04-17 06:40:09 -07:00
Derek Collison
ff8701b724 Merge branch 'main' into dev 2023-04-06 08:37:11 -07:00
Julien Viard de Galbert
1b1610f2da Adding hard_delete option on resolver configuration
Signed-off-by: Julien Viard de Galbert <jviarddegalbert@scaleway.com>
2023-04-05 10:05:48 +02:00
Ivan Kozlovic
fe5d6bede4 Fixed accounts configuration reload
Issues could manifest with subscription interest not properly
propagated.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-04-03 09:32:28 -06:00
Ivan Kozlovic
105237cba8 [ADDED] Multiple routes and ability to have per-account routes
New configuration fields:
```
cluster {
   ...
   pool_size: 5
   accounts: ["A", "B"]
}
```

The configuration `pool_size` in the example above means that this
server will create 5 routes to a remote server, assuming that that
server has the same `pool_size` setting.

Accounts (which are not part of the `accounts[]` configuration)
are assigned a specific route in this pool, and this will be the
same route on all servers in the cluster.

Accounts that are defined in the `accounts` field will each have
a dedicated route connection. This will allow suppression of the
account name in some of the route protocols, reducing bytes transmitted
which may increase performance.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2023-04-03 09:32:25 -06:00
Alberto Ricart
0033f3309b [FIX] referenced JWT constant for AnyAccount (#3988) 2023-03-24 15:23:34 -05:00
Alberto Ricart
30afb48c2a [FEAT] enable callout service to generate users for any account 2023-03-23 16:01:18 -05:00
Derek Collison
7bd7cda021 Merge branch 'main' into dev 2023-02-28 15:17:24 -08:00
Tomasz Pietrek
10c2c387b7 Fix panic if serviceExport is nil
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-02-28 15:36:37 +01:00
Tomasz Pietrek
8732022d26 Fix JWT claims update if headers are passed in request
Claims update message requires only payload to be passed,
but passing headers should not fail the request.
This change ensures we extract payload from raw message
before decoding it.
Before this change, passing claims update with headers
would return cryptic `expected x chunks` error.

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2023-02-28 11:09:18 +01:00
Derek Collison
b40621b508 Fixups for large merge with main
Signed-off-by: Derek Collison <derek@nats.io>
2023-02-20 18:41:03 -08:00
Derek Collison
cac712b1d1 Merge branch 'main' into dev 2023-02-20 18:30:26 -08:00
Maurice van Veen
05695d304c Fixed a bug where partition was used with multiple wildcard token position 2023-02-20 10:27:29 +01:00
Derek Collison
5eb7d9b99f Merge branch 'main' into dev 2023-02-16 07:45:44 +04:00
Neil Twigg
ba0b1f8bd3 Fix stack overflow panic in reverse entry check when inbox ends with wildcard 2023-02-13 09:45:50 +00:00
Derek Collison
f7615acb7d Merge branch 'main' into dev 2023-02-04 22:23:50 -08:00
Derek Collison
9c02be2409 Various fixes for snapshots.
Due to bug, in rare circumstances could write an empty snapshot for aplied == 0. This would cause a spinning at the raft layer.

1. Allow Truncate() to also properly do a reset of the store when terms were only mismatch.
2. During testing fixed memstore truncate and also made sure per subject info was also cleaned up.
3. Then added fix to detect a bad snapshot on initialization and remove.
4. Do not allow snapshots for applied == 0.

Signed-off-by: Derek Collison <derek@nats.io>
2023-02-04 13:46:06 -08:00
Jean-Noël Moyne
ccc492bdce Improve checking that subject filters and destination transforms are valid subjects.
Improve error messages when bad filters/transforms are encountered.
Cover all cases of updating an existing stream's subject transform or sources
2023-01-27 17:54:37 -08:00
R.I.Pienaar
348c84a469 restore public APIs and fix breaking interface renames
Signed-off-by: R.I.Pienaar <rip@devco.net>
2023-01-27 12:36:15 +01:00
Jean-Noël Moyne
a953e84437 Add stream input subject transform functionality
Extract subject transformation code out of accounts.go
Stream sources can now have a subject mapping transform
You can source the same stream more than once
Remove limitation that the subject filter for a source, mirror or consumer must have an overlap with the sourced/mirrored's stream or the stream's subjects
2023-01-25 12:25:51 -08:00
Derek Collison
acad660540 Make sure connection events during auth callouts correct.
Fixed one extraneous account update for $G. We sent for the addition before switching but suppressed the change back to 0.
We now suppress all for $G as was designed.

Signed-off-by: Derek Collison <derek@nats.io>
2023-01-20 18:42:36 -08:00
Derek Collison
3c889478bd Merge pull request #3719 from nats-io/auth_callout
Authorization Callouts
2023-01-03 15:34:10 -08:00
Derek Collison
ff79afef39 Merge branch 'main' into dev 2022-12-30 12:23:23 -08:00
Neil Twigg
14d0ba1c65 Fix some lint errors after move to golangci-lint 2022-12-30 20:00:08 +00:00
Derek Collison
43c26a49f8 Merge branch 'main' into dev 2022-12-29 15:40:00 -08:00
Waldemar Quevedo
861f98128a Remove individual import subs from leafnodes
When a leafnode connection is bound to an account where there was already
a wildcard response import subscription to handle the requests (e.g. `_R_.foo.>`),
this would have created message duplicates due to an extra subscription
being created that also matched the wildcard (e.g. `_R_.foo.bar`).

To avoid this condition, we now skip creating the latter extra subscription
for leafnode connections.

Signed-off-by: Waldemar Quevedo <wally@nats.io>
2022-12-29 15:30:17 -08:00
Derek Collison
2daf90493b Authentication and Authorization callouts for server configuration mode.
This adds the ability to augment or override the NATS auth system.

A server will send a signed request to $SYS.REQ.USER.AUTH on the specified account. The request will contain client information, all client options sent to the server, and optionally TLS information and client certificates.
The external auth service will respond with an empty message if not authorized, or a signed User JWT that the user will bind to.

The response can change the account the client will be bound to.

Signed-off-by: Derek Collison <derek@nats.io>
2022-12-28 10:32:45 -08:00
Waldemar Quevedo
6df92f40dc Fix to reenable JetStream for account when reenabled
Signed-off-by: Waldemar Quevedo <wally@nats.io>
2022-12-06 15:53:03 -08:00