Commit Graph

59 Commits

Author SHA1 Message Date
Derek Collison
9ffaf44d71 Make sure to not turn on cache when it was disable
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
e4c15d8680 Fix for data race
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
03aacecb81 Changed cache back to simple map.
We were using a sync.Map. This did provide a benefit with massive contention from lots of Go routines. However this is only about 2x in the crazy extremes now and with a normal map and read locks we can assist the RemoveBatch which was a cause for performance issues.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:31:56 -07:00
Derek Collison
acc8da8b6e Improve RemoveBatch by disabling cache
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
cadd39a01c Major rewrite for NATS JetStream API
API made more consistent. Noun followed by verb.
Name arguments in request subejcts are always at the end now.
Remove enabled call, just use account info.
Getting a message directly from a stream is treated like an admin API and requires JSON request.
Deleting a message directly as well.
StreamList and ConsumerList now include details and support paging.
Streams and Consumers now contain a created field in their info.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
ea5e5bd364 Services rewrite #2
This contains a rewrite to the services layer for exporting and importing. The code this merges to already had a first significant rewrite that moved from special interest processing to plain subscriptions.

This code changes the prior version's dealing with reverse mapping which was based mostly on thresholds and manual pruning, with some sporadic timer usage. This version uses the jetstream branch's code that understands interest and failed deliveries. So this code is much more tuned to reacting to interest changes. It also removes thresholds and goes only by interest changes or expirations based around a new service export property, response thresholds. This allows a service provider to provide semantics on how long a response should take at a maximum.

This commit also introduces formal support for service export streamed and chunked response types send an empty message to signify EOF.

This commit also includes additions to the service latency tracking such that errors are now sent, not only successful interactions. We have added a Status field and an optional Error fields to ServiceLatency.

We support the following Status codes, these are directly from HTTP.

400 Bad Request (request did not have a reply subject)
408 Request Timeout (when system detects request interest went away, old request style to make dependable)..
503 Service Unavailable (no service responders running)
504 Service Timeout (The new response threshold expired)

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:26:46 -07:00
Derek Collison
bb539d74cf Should not ask match to acquire lock already held
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
R.I.Pienaar
dc207be04f handles 0 length tokens correctly
Without this nats-req '$JS.TEMPLATE..DELETE' '' causes a panic
2020-05-19 14:16:03 -07:00
Derek Collison
72cda966f4 Changed API subjects to be authorization friendly
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
119c48ee53 Do not allow overlapping message sets for now
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
58d98d42cb Allow notifications for literal subjects.
This allows us to register for notifications on subjects to determine when interest comes and goes.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:15:11 -07:00
Derek Collison
d5676e502c Add ability to use remote servers
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:15:11 -07:00
Matthias Hanel
136feb9bc6 [FIXEd] subsz monitoring endpoint did not account for accounts.
Fixes  #1371 and #1357 by adding up stats and collecting subscriptions
from all accounts.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2020-05-06 15:48:51 -04:00
Ivan Kozlovic
bd28a015b1 [FIXED] Sublist isSubsetMatch to handle empty tokens
If a subject has empty tokens, returns false.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-01-14 18:28:14 -07:00
Derek Collison
52430c304a System level services for debugging.
This is the first pass at introducing exported services to the system account for generally debugging of blackbox systems.
The first service reports number of subscribers for a given subject. The payload of the request is the subject, and optional queue group, and can contain wildcards.

Signed-off-by: Derek Collison <derek@nats.io>
2019-09-17 09:37:35 -07:00
Derek Collison
7989118c3f First pass latency tracking for exported services
Signed-off-by: Derek Collison <derek@nats.io>
2019-08-30 10:52:48 -07:00
antmanler
ca773bf07b Make sure Sublist.All collect all subscriptions 2019-07-10 21:41:44 +08:00
Derek Collison
8168aa1f81 Allow sublist cache do be disabled globally
Signed-off-by: Derek Collison <derek@nats.io>
2019-07-02 07:34:02 -07:00
Derek Collison
3c3e97f729 Fold under cache conditional
Signed-off-by: Derek Collison <derek@nats.io>
2019-07-02 06:06:53 -07:00
Derek Collison
acc1031705 Protect stats when no cache is present
Signed-off-by: Derek Collison <derek@nats.io>
2019-07-02 05:47:39 -07:00
Derek Collison
6584a9a828 lint updates
Signed-off-by: Derek Collison <derek@nats.io>
2019-05-06 15:41:38 -07:00
Derek Collison
4ccfef004c Update for Ivan's suggestion on just checking s.cache since we have read lock
Signed-off-by: Derek Collison <derek@nats.io>
2019-04-22 18:35:24 -07:00
Derek Collison
da2dab92d1 Allow disabling of shared cache with new constuctor. Also share empty results.
Signed-off-by: Derek Collison <derek@nats.io>
2019-04-22 17:53:14 -07:00
Derek Collison
bacb73a403 First pass at leaf nodes. Basic functionality working, including gateways.
What is not completed:
1. TLS
2. config to bind local account.
3. Info updates for solicitor to track topology changes like a client.
4. CONNECT sent after INFO for nonce authroization.
5. Authorization
6. Services and Streams tests.
7. config file parsing.

Signed-off-by: Derek Collison <derek@nats.io>
2019-03-25 08:54:47 -07:00
Derek Collison
574fd62e01 Allow servers to send and receive messages directly
Signed-off-by: Derek Collison <derek@nats.io>
2018-11-29 12:15:08 -08:00
Derek Collison
15bdfbb4aa Fix for #793
Signed-off-by: Derek Collison <derek@nats.io>
2018-11-07 19:50:45 -08:00
Derek Collison
1ce1a434b0 Fix for #792
Allow deny clauses for subscriptions to still allow wildcard subscriptions but do not deliver the messages themselves.

Signed-off-by: Derek Collison <derek@nats.io>
2018-11-06 15:00:21 -08:00
Derek Collison
ea5a6d9589 Updates for comments, some golint fixes
Signed-off-by: Derek Collison <derek@nats.io>
2018-10-31 20:28:44 -07:00
Derek Collison
47963303f8 First pass at new cluster design
Signed-off-by: Derek Collison <derek@nats.io>
2018-10-24 21:29:29 -07:00
Derek Collison
620e1d7364 Basic account mapping via import and export
Signed-off-by: Derek Collison <derek@nats.io>
2018-09-29 13:04:19 +02:00
Ivan Kozlovic
2ce3aba434 Bump version and fix megacheck report
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2018-09-05 09:33:01 -06:00
Derek Collison
5b7cabd25e Avoid race between sublist and cache on store-s 2018-08-29 11:34:55 -06:00
Derek Collison
ab9e4c7d8a Fix for wildcard cache addition
Signed-off-by: Derek Collison <derek@nats.io>
2018-08-27 12:28:32 -06:00
Derek Collison
d21ac8de35 Use sync.Map for cache, fast version of literal test
Signed-off-by: Derek Collison <derek@nats.io>
2018-08-25 17:42:36 -07:00
Derek Collison
543d403337 Optimize sublist cache, add tests for cache contention
Signed-off-by: Derek Collison <derek@nats.io>
2018-08-25 14:33:15 -07:00
Derek Collison
cc07d500dd new subs collector
Signed-off-by: Derek Collison <derek@nats.io>
2018-06-04 17:45:05 -07:00
Derek Collison
50bb4b9a1b delivery last activity update 2018-06-04 17:45:05 -07:00
Derek Collison
8502fb1edc Add fast slice for large psubs for Match
Signed-off-by: Derek Collison <derek@nats.io>
2018-06-04 17:45:05 -07:00
Derek Collison
b9c73e96ee Changed sublist to avoid quadratic time in removal with large N
Signed-off-by: Derek Collison <derek@nats.io>
2018-06-04 17:45:05 -07:00
Derek Collison
00901acc78 Update license to Apache 2 2018-03-15 22:31:07 -07:00
Ivan Kozlovic
3bfb2d7cb4 Variant of copy and add some tests
Use the append([]*subscription(nil), qr...) notation instead of
make()+copy().
2018-03-15 14:54:10 -06:00
Ivan Kozlovic
eb28cf0eda Do the copy only of the array of subscriptions that we add to the results
Instead of making a copy of the whole results, make sure that
we don't pass a sublist array to the result but its copy.
2018-03-10 08:35:15 -07:00
Ivan Kozlovic
668229fc1d [FIXED] RACE between sublist remove and go through match results
This would manifest for instance when server tries to send messages
to queue subscribers and a subscription is unsubsribed at the same
time.

Resolves #640
2018-03-09 19:05:33 -07:00
Ivan Kozlovic
8ec1f77ac8 [FIXED] Sublist Insert and Remove with wildcard characters in literals
This is similar to #561 where `*` and `>` characters appear in tokens
as literals, not wilcards.
Both Insert() and Remove() were checking that the first character
was `*` or `>` and consider it a wildcard node. This is wrong. Any
token that is more than 1 character long must be treated as a literal.
Only for token of size one should we check if the character is `*`
or `>`.
Added a test case for Insert and Remove with subject like `foo.*-`
or `foo.>-`.
2017-08-31 18:48:45 -06:00
Ivan Kozlovic
0cc49ece4a Improve matchLiteral further and add some more tests 2017-08-17 13:00:09 -06:00
Ivan Kozlovic
42b27d2710 Improve matchLiteral performance and simplify if conditions 2017-08-16 17:43:33 -06:00
Ivan Kozlovic
515779ae0b [FIXED] Use of * and > in subjects as literals
The issue was that a subject such as `foo.bar,*,>` would be
inserted to the cache as is, but when trying to remove from the
cache, calling matchLiteral() with the above subject in the cache
against the same subject would return false. This is because
matchLiteral would treat those characters as wildcards token.

Note that the sublist itself splits subjects on the `.` separator
and seem not bothered by such subject (would have `foo` and `bar,*,>`
tokens). Also, note that IsValidSubject() and IsValidLiteralSubject()
properly checked that the characters `*` and `>` are treated
as wildcards only if they are tokens on their own.

Resolves #558
2017-08-16 12:12:18 -06:00
Derek Collison
f7ba3d175e Correct invocation of misspell with fixes 2017-04-21 09:21:33 -07:00
Derek Collison
b11ba5b118 Merge pull request #456 from wallyqs/gometalinter-checks
Fixes to a couple of `gometalinter` checks
2017-03-24 09:35:09 -07:00
Waldemar Quevedo
8926fa0ff1 Fix unconvert checks in server 2017-03-23 20:41:38 -07:00