Commit Graph

40 Commits

Author SHA1 Message Date
Ivan Kozlovic
e0487b95cc [FIXED] Return no match result if subject contains empty token
A subject such as `foo..bar` is invalid, but if it is published
from a connection that has disabled pedantic, then the message
is matched against subscriptions and will be delivered.

This change causes Sublist.Match() to return no result.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-01-28 17:43:22 -07:00
Ivan Kozlovic
23f8e3d5b9 Fixed sublist notification
The insert notification was done based on the creation of a node
during an insert, which was wrong since the node may have already
existed and still the subscription could be all new. For instance,
suppose that there is a subscription on "foo.bar".
We register an notification interest for "foo", which does not
notify, which is normal. Then we create a subscription on "foo".
During the insert, "foo" node already exists so notification would
not be sent, but it should.
Fixed also removed by having removeFromNode() returning a boolean
to indicate if the subscription was the last in that node.
However, it seems that we again check for interest in
chkForRemoveNotification(), so not sure if that is required.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-01-13 20:59:36 -07:00
Ivan Kozlovic
1dba6418ed [ADDED] MQTT Support
This PR introduces native support for MQTT clients. It requires use
of accounts with JetStream enabled. Since as of now clustering is
not available, MQTT will be limited to single instance.

Only QoS 0 and 1 are supported at the moment. MQTT clients can
exchange messages with NATS clients and vice-versa.

Since JetStream is required, accounts with JetStream enabled must
exist in order for an MQTT client to connect to the NATS Server.
The administrator can limit the users that can use MQTT with the
allowed_connection_types option in the user section. For instance:
```
accounts {
  mqtt {
    users [
      {user: all, password: pwd, allowed_connection_types: ["STANDARD", "WEBSOCKET", "MQTT"]}
      {user: mqtt_only, password: pwd, allowed_connection_types: "MQTT"}
    ]
    jetstream: enabled
  }
}
```
The "mqtt_only" can only be used for MQTT connections, which the user
"all" accepts standard, websocket and MQTT clients.

Here is what a configuration to enable MQTT looks like:
```
mqtt {
  # Specify a host and port to listen for websocket connections
  #
  # listen: "host:port"

  # It can also be configured with individual parameters,
  # namely host and port.
  #
  # host: "hostname"
  port: 1883

  # TLS configuration section
  #
  # tls {
  #  cert_file: "/path/to/cert.pem"
  #  key_file: "/path/to/key.pem"
  #  ca_file: "/path/to/ca.pem"
  #
  #  # Time allowed for the TLS handshake to complete
  #  timeout: 2.0
  #
  #  # Takes the user name from the certificate
  #  #
  #  # verify_an_map: true
  #}

  # Authentication override. Here are possible options.
  #
  # authorization {
  #   # Simple username/password
  #   #
  #   user: "some_user_name"
  #   password: "some_password"
  #
  #   # Token. The server will check the MQTT's password in the connect
  #   # protocol against this token.
  #   #
  #   # token: "some_token"
  #
  #   # Time allowed for the client to send the MQTT connect protocol
  #   # after the TCP connection is established.
  #   #
  #   timeout: 2.0
  #}

  # If an MQTT client connects and does not provide a username/password and
  # this option is set, the server will use this client (and therefore account).
  #
  # no_auth_user: "some_user_name"

  # This is the time after which the server will redeliver a QoS 1 message
  # sent to a subscription that has not acknowledged (PUBACK) the message.
  # The default is 30 seconds.
  #
  # ack_wait: "1m"

  # This limits the number of QoS1 messages sent to a session without receiving
  # acknowledgement (PUBACK) from that session. MQTT specification defines
  # a packet identifier as an unsigned int 16, which means that the maximum
  # value is 65535. The default value is 1024.
  #
  # max_ack_pending: 100
}
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-11-30 20:08:44 -07:00
Derek Collison
e6797efde7 Make interest notifications explicit match only
Signed-off-by: Derek Collison <derek@nats.io>
2020-11-12 07:10:47 -08:00
Derek Collison
9ffaf44d71 Make sure to not turn on cache when it was disable
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:33:06 -07:00
Derek Collison
03aacecb81 Changed cache back to simple map.
We were using a sync.Map. This did provide a benefit with massive contention from lots of Go routines. However this is only about 2x in the crazy extremes now and with a normal map and read locks we can assist the RemoveBatch which was a cause for performance issues.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:31:56 -07:00
Derek Collison
cadd39a01c Major rewrite for NATS JetStream API
API made more consistent. Noun followed by verb.
Name arguments in request subejcts are always at the end now.
Remove enabled call, just use account info.
Getting a message directly from a stream is treated like an admin API and requires JSON request.
Deleting a message directly as well.
StreamList and ConsumerList now include details and support paging.
Streams and Consumers now contain a created field in their info.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:27:45 -07:00
Derek Collison
72cda966f4 Changed API subjects to be authorization friendly
Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:16:03 -07:00
Derek Collison
58d98d42cb Allow notifications for literal subjects.
This allows us to register for notifications on subjects to determine when interest comes and goes.

Signed-off-by: Derek Collison <derek@nats.io>
2020-05-19 14:15:11 -07:00
Ivan Kozlovic
7ab9c76f2b Fixed benchmark tests to be able to run with Go 1.13+
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-02-26 12:12:18 -07:00
Ivan Kozlovic
bd28a015b1 [FIXED] Sublist isSubsetMatch to handle empty tokens
If a subject has empty tokens, returns false.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2020-01-14 18:28:14 -07:00
R.I.Pienaar
bcf96fa1de Allows a descriptive server_name to be set
This adds a new config option server_name that
when set will be exposed in varz, events and more
as a descriptive name for the server.

If unset though the server_name will default to the pk

Signed-off-by: R.I.Pienaar <rip@devco.net>
2019-10-17 18:51:19 +02:00
antmanler
24ab33b879 Add a simple test for Sublist.All 2019-07-10 22:32:15 +08:00
Derek Collison
8168aa1f81 Allow sublist cache do be disabled globally
Signed-off-by: Derek Collison <derek@nats.io>
2019-07-02 07:34:02 -07:00
Derek Collison
acc1031705 Protect stats when no cache is present
Signed-off-by: Derek Collison <derek@nats.io>
2019-07-02 05:47:39 -07:00
Derek Collison
da2dab92d1 Allow disabling of shared cache with new constuctor. Also share empty results.
Signed-off-by: Derek Collison <derek@nats.io>
2019-04-22 17:53:14 -07:00
Ivan Kozlovic
dba4cfa791 Initialized the sublist for benchmark tests only when needed
This saves a lot of memory and init time when running non benchmark
tests in race mode.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2018-12-01 12:12:45 -07:00
Derek Collison
574fd62e01 Allow servers to send and receive messages directly
Signed-off-by: Derek Collison <derek@nats.io>
2018-11-29 12:15:08 -08:00
Derek Collison
a2e310ffc1 Vendor jwt, fixes for nkey, jwt changes
Signed-off-by: Derek Collison <derek@nats.io>
2018-11-21 19:22:04 -08:00
Derek Collison
47963303f8 First pass at new cluster design
Signed-off-by: Derek Collison <derek@nats.io>
2018-10-24 21:29:29 -07:00
Derek Collison
620e1d7364 Basic account mapping via import and export
Signed-off-by: Derek Collison <derek@nats.io>
2018-09-29 13:04:19 +02:00
Ivan Kozlovic
156224a85b Fix sublist cache flapping test
Use `checkFor` to ensure that cache size is ok. Repeated test
1000 times and no failure now.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2018-08-27 15:01:38 -06:00
Derek Collison
2c4b7e7d54 Let cache sweeper run
Signed-off-by: Derek Collison <derek@nats.io>
2018-08-27 12:56:28 -06:00
Derek Collison
ad3a1506d3 Move test into cache test, make sure it fails
Signed-off-by: Derek Collison <derek@nats.io>
2018-08-27 12:49:29 -06:00
Derek Collison
ab9e4c7d8a Fix for wildcard cache addition
Signed-off-by: Derek Collison <derek@nats.io>
2018-08-27 12:28:32 -06:00
Derek Collison
d21ac8de35 Use sync.Map for cache, fast version of literal test
Signed-off-by: Derek Collison <derek@nats.io>
2018-08-25 17:42:36 -07:00
Derek Collison
543d403337 Optimize sublist cache, add tests for cache contention
Signed-off-by: Derek Collison <derek@nats.io>
2018-08-25 14:33:15 -07:00
Derek Collison
8502fb1edc Add fast slice for large psubs for Match
Signed-off-by: Derek Collison <derek@nats.io>
2018-06-04 17:45:05 -07:00
Derek Collison
b9c73e96ee Changed sublist to avoid quadratic time in removal with large N
Signed-off-by: Derek Collison <derek@nats.io>
2018-06-04 17:45:05 -07:00
Derek Collison
00901acc78 Update license to Apache 2 2018-03-15 22:31:07 -07:00
Ivan Kozlovic
3bfb2d7cb4 Variant of copy and add some tests
Use the append([]*subscription(nil), qr...) notation instead of
make()+copy().
2018-03-15 14:54:10 -06:00
Ivan Kozlovic
668229fc1d [FIXED] RACE between sublist remove and go through match results
This would manifest for instance when server tries to send messages
to queue subscribers and a subscription is unsubsribed at the same
time.

Resolves #640
2018-03-09 19:05:33 -07:00
Ivan Kozlovic
8ec1f77ac8 [FIXED] Sublist Insert and Remove with wildcard characters in literals
This is similar to #561 where `*` and `>` characters appear in tokens
as literals, not wilcards.
Both Insert() and Remove() were checking that the first character
was `*` or `>` and consider it a wildcard node. This is wrong. Any
token that is more than 1 character long must be treated as a literal.
Only for token of size one should we check if the character is `*`
or `>`.
Added a test case for Insert and Remove with subject like `foo.*-`
or `foo.>-`.
2017-08-31 18:48:45 -06:00
Ivan Kozlovic
0cc49ece4a Improve matchLiteral further and add some more tests 2017-08-17 13:00:09 -06:00
Ivan Kozlovic
30eeac27e9 add matchLiteral benchmark 2017-08-16 16:28:24 -06:00
Ivan Kozlovic
515779ae0b [FIXED] Use of * and > in subjects as literals
The issue was that a subject such as `foo.bar,*,>` would be
inserted to the cache as is, but when trying to remove from the
cache, calling matchLiteral() with the above subject in the cache
against the same subject would return false. This is because
matchLiteral would treat those characters as wildcards token.

Note that the sublist itself splits subjects on the `.` separator
and seem not bothered by such subject (would have `foo` and `bar,*,>`
tokens). Also, note that IsValidSubject() and IsValidLiteralSubject()
properly checked that the characters `*` and `>` are treated
as wildcards only if they are tokens on their own.

Resolves #558
2017-08-16 12:12:18 -06:00
Ivan Kozlovic
20926a6176 Added megacheck
This tool combines staticcheck, gosimple and unused.
Fixed reports from unused.
2017-08-11 17:28:18 -06:00
Derek Collison
f7ba3d175e Correct invocation of misspell with fixes 2017-04-21 09:21:33 -07:00
Derek Collison
96d9ce5048 Queue subscriber performance
Reworked sublist to sort out normal subscribers from queue subscribers into
a result set that can be cached and easily iterated over.
2016-04-03 13:04:06 -07:00
Derek Collison
df02bc0bcf Removed sublist, hash and hashmap, no longer needed. 2016-04-02 12:52:48 -07:00