Extract subject transformation code out of accounts.go
Stream sources can now have a subject mapping transform
You can source the same stream more than once
Remove limitation that the subject filter for a source, mirror or consumer must have an overlap with the sourced/mirrored's stream or the stream's subjects
When a leafnode connection is bound to an account where there was already
a wildcard response import subscription to handle the requests (e.g. `_R_.foo.>`),
this would have created message duplicates due to an extra subscription
being created that also matched the wildcard (e.g. `_R_.foo.bar`).
To avoid this condition, we now skip creating the latter extra subscription
for leafnode connections.
Signed-off-by: Waldemar Quevedo <wally@nats.io>
Code change:
- Do not start the processMirrorMsgs and processSourceMsgs go routine
if the server has been detected to be shutdown. This would otherwise
leave some go routine running at the end of some tests.
- Pass the fch and qch to the consumerFileStore's flushLoop otherwise
in some tests this routine could be left running.
Tests changes:
- Added missing defer NATS connection close
- Added missing defer server shutdown
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
There was a way to detect a cycle but I believe it needs to be
a stack of "si" not just the one before invoking processServiceImport.
Changes in #3393 would solve issue reported with test TestAccountImportCycle,
but would not address the new reported issue represented by new test
TestLeafNodeSvcImportExportCycle. This current approach seems to solve
all known cases.
Resolves#3397
Replaces #3393
past processing the import response, c.pa was not reset to the
appropriate state, which lead to an unintended recursion
Signed-off-by: Matthias Hanel <mh@synadia.com>
* 1: Allows spaces to be used inside {{}} subject mapping functions:
2: Rework and improve mapping destinations validation and error handling with much more helpful error messages, e.g.:
* Error adding mapping for "foo.*.*" to "bar.{{wildcard(1)}}" : invalid mapping destination: not using all of the token wildcard(s) in bar.{{wildcard(1)}}
* Error adding mapping for "myservice.request.*" to "myservice.request.{{ partition(10) }}.{{wildcard(1)}}" : invalid mapping destination: not enough arguments passed to the function in {{ partition(10) }}
* Error adding mapping for "myservice.request.*" to "myservice.request.{{ partition(10,2) }}.{{wildcard(1)}}" : invalid mapping destination: wildcard index out of range in {{ partition(10,2) }}: [2]
* Error adding mapping for "myservice.request.*" to "myservice.request.{{ partition(10,1) }}.{{wildcard()}}" : invalid mapping destination: not enough arguments passed to the function in {{wildcard()}}
* Error adding mapping for "myservice.request.*" to "myservice.request.{{ xxxpartition(10,1) }}.{{wildcard(1)}}" : invalid mapping destination: unknown function in {{ xxxpartition(10,1) }}
* Error adding mapping for "myservice.request.*" to "myservice. request.{{ xxxpartition(10,1) }}.{{wildcard(1)}}" : invalid mapping destination: invalid subject
* implement PR comments
While the TransformSubject function was doing the right
thing it did not match first and so would panic for subjects
that do not match the mapping.
The map function does the right thing so this is a more
appropriate function to export.
This undoes the exporting of unsafe TransformSubject and
exports the safer Match instead.
Signed-off-by: R.I.Pienaar <rip@devco.net>
I tracked down this issue to have been introduced with PR #2369,
but the code also touched PR #1891 and PR #3088.
I added a test as described in issue #3108 but did not need
JetStream to demonstrate the issue. With the proposed fix, all
tests that were added in aforementioned PRs still pass, including
the new test.
Resolves#3108
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
fixes#2361
The subject used was not a subset of the import. Nor the other way
around. Instead it is an overlap that needs to be dynamically computed.
Signed-off-by: Matthias Hanel <mh@synadia.com>
Currently, we use ReadyForConnections in server tests to wait for the
server to be ready. However, when this fails we don't get a clue about
why it failed.
This change adds a new unexported method called readyForConnections that
returns an error describing which check failed. The exported
ReadyForConnections version works exactly as before. The unexported
version gets used in internal tests only.
Currently in tests, we have calls to os.Remove and os.RemoveAll where we
don't check the returned error. This hides useful error messages when
tests fail to run, such as "too many open files".
This change checks for more filesystem related errors and calls t.Fatal
if there is an error.
This change does 4 things:
Refactor to only have one function to validate imports.
Have this function support the jwt field account_token_position.
For completeness make this value configurable as well.
unit tests.
Signed-off-by: Matthias Hanel <mh@synadia.com>
Imports in JWT where extended to contain a new filed LocalSubject.
This Change pulls the new JWT library version in.
It was needed as prefix did not exist in the JWT library and the
original field could not be used. The field To has been deprecated.
When LocalSubject is set, service imports can be configured the same way
they are in config. Meaning, no reversal due to the type.
This change also ensures that wildcard references in transforms are only
set in To/LocalSubject. Before, for services, $1 would have to be set in Subject.
Signed-off-by: Matthias Hanel <mh@synadia.com>
