Suppress some stream and consumer bad results since they delete the asset.
Allow rehup to re-enable JetStream.
Various bug fixes and improvements.
Signed-off-by: Derek Collison <derek@nats.io>
This also applies to times that end up in that json.
Where applicable moved time.Now() to where it is used.
Moved calls to .UTC() to where time is created it that time is converted
later anyway.
Signed-off-by: Matthias Hanel <mh@synadia.com>
Presence of TLS config in any remote gateway or leafnode would
cause the config reload to fail (because TLS config internal
content may change which fails the DeepEqual check).
This PR excludes the TLS configs in such case to check for
changes in gateways and leafnodes.
Although GW and LN config reload is technically supported, this
PR updates the internal remotes' TLS configuration so that
changes/updates to TLS certificates would take effect after
a configuration reload.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
This PR introduces native support for MQTT clients. It requires use
of accounts with JetStream enabled. Since as of now clustering is
not available, MQTT will be limited to single instance.
Only QoS 0 and 1 are supported at the moment. MQTT clients can
exchange messages with NATS clients and vice-versa.
Since JetStream is required, accounts with JetStream enabled must
exist in order for an MQTT client to connect to the NATS Server.
The administrator can limit the users that can use MQTT with the
allowed_connection_types option in the user section. For instance:
```
accounts {
mqtt {
users [
{user: all, password: pwd, allowed_connection_types: ["STANDARD", "WEBSOCKET", "MQTT"]}
{user: mqtt_only, password: pwd, allowed_connection_types: "MQTT"}
]
jetstream: enabled
}
}
```
The "mqtt_only" can only be used for MQTT connections, which the user
"all" accepts standard, websocket and MQTT clients.
Here is what a configuration to enable MQTT looks like:
```
mqtt {
# Specify a host and port to listen for websocket connections
#
# listen: "host:port"
# It can also be configured with individual parameters,
# namely host and port.
#
# host: "hostname"
port: 1883
# TLS configuration section
#
# tls {
# cert_file: "/path/to/cert.pem"
# key_file: "/path/to/key.pem"
# ca_file: "/path/to/ca.pem"
#
# # Time allowed for the TLS handshake to complete
# timeout: 2.0
#
# # Takes the user name from the certificate
# #
# # verify_an_map: true
#}
# Authentication override. Here are possible options.
#
# authorization {
# # Simple username/password
# #
# user: "some_user_name"
# password: "some_password"
#
# # Token. The server will check the MQTT's password in the connect
# # protocol against this token.
# #
# # token: "some_token"
#
# # Time allowed for the client to send the MQTT connect protocol
# # after the TCP connection is established.
# #
# timeout: 2.0
#}
# If an MQTT client connects and does not provide a username/password and
# this option is set, the server will use this client (and therefore account).
#
# no_auth_user: "some_user_name"
# This is the time after which the server will redeliver a QoS 1 message
# sent to a subscription that has not acknowledged (PUBACK) the message.
# The default is 30 seconds.
#
# ack_wait: "1m"
# This limits the number of QoS1 messages sent to a session without receiving
# acknowledgement (PUBACK) from that session. MQTT specification defines
# a packet identifier as an unsigned int 16, which means that the maximum
# value is 65535. The default value is 1024.
#
# max_ack_pending: 100
}
```
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Users and NKey users will now have the option to specify a list
of allowed connection types.
This will allow for instance a certain user to be allowed to
connect as a standard NATS client, but not as Websocket, or
vice-versa.
This also fixes the websocket auth override. Indeed, with
the original behavior, the websocket users would have been bound
to $G, which would not work when there are accounts defined, since
when that is the case, no app can connect/bind to $G account.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
* Adding nats based resolver and bootstrap system account
These resolver operate on an exclusive directory
Two types:
full: managing all jwt in the directory
Will synchronize with other full resolver
nats-account-server will also run such a resolver
cache: lru cache managing only a subset of all jwt in the directory
Will lookup jwt from full resolver
Can overwrite expiration with a ttl for the file
Both:
track expiration of jwt and clean up
Support reload
Notify the server of changed jwt
Bootstrapping system account allows users signed with the system account
jwt to connect, without the server knowing the jwt.
This allows uploading jwt (including system account) using nats by
publishing to $SYS.ACCOUNT.<name>.CLAIMS.UPDATE
Sending a request, server will respond with the result of the operation.
Receive all jwt stored in one server by sending a
request to $SYS.ACCOUNT.CLAIMS.PACK
One server will respond with a message per stored jwt.
The end of the responses is indicated by an empty message.
The content of dirstore.go and dirstore_test.go was moved from
nats-account-server
Signed-off-by: Matthias Hanel <mh@synadia.com>
Websocket support can be enabled with a new websocket
configuration block:
```
websocket {
# Specify a host and port to listen for websocket connections
# listen: "host:port"
# It can also be configured with individual parameters,
# namely host and port.
# host: "hostname"
# port: 4443
# This will optionally specify what host:port for websocket
# connections to be advertised in the cluster
# advertise: "host:port"
# TLS configuration is required
tls {
cert_file: "/path/to/cert.pem"
key_file: "/path/to/key.pem"
}
# If same_origin is true, then the Origin header of the
# client request must match the request's Host.
# same_origin: true
# This list specifies the only accepted values for
# the client's request Origin header. The scheme,
# host and port must match. By convention, the
# absence of port for an http:// scheme will be 80,
# and for https:// will be 443.
# allowed_origins [
# "http://www.example.com"
# "https://www.other-example.com"
# ]
# This enables support for compressed websocket frames
# in the server. For compression to be used, both server
# and client have to support it.
# compression: true
# This is the total time allowed for the server to
# read the client request and write the response back
# to the client. This include the time needed for the
# TLS handshake.
# handshake_timeout: "2s"
}
```
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
API made more consistent. Noun followed by verb.
Name arguments in request subejcts are always at the end now.
Remove enabled call, just use account info.
Getting a message directly from a stream is treated like an admin API and requires JSON request.
Deleting a message directly as well.
StreamList and ConsumerList now include details and support paging.
Streams and Consumers now contain a created field in their info.
Signed-off-by: Derek Collison <derek@nats.io>
This contains a rewrite to the services layer for exporting and importing. The code this merges to already had a first significant rewrite that moved from special interest processing to plain subscriptions.
This code changes the prior version's dealing with reverse mapping which was based mostly on thresholds and manual pruning, with some sporadic timer usage. This version uses the jetstream branch's code that understands interest and failed deliveries. So this code is much more tuned to reacting to interest changes. It also removes thresholds and goes only by interest changes or expirations based around a new service export property, response thresholds. This allows a service provider to provide semantics on how long a response should take at a maximum.
This commit also introduces formal support for service export streamed and chunked response types send an empty message to signify EOF.
This commit also includes additions to the service latency tracking such that errors are now sent, not only successful interactions. We have added a Status field and an optional Error fields to ServiceLatency.
We support the following Status codes, these are directly from HTTP.
400 Bad Request (request did not have a reply subject)
408 Request Timeout (when system detects request interest went away, old request style to make dependable)..
503 Service Unavailable (no service responders running)
504 Service Timeout (The new response threshold expired)
Signed-off-by: Derek Collison <derek@nats.io>
Fixes#1372 by updating s.sys.account pointer.
This issue also showed that accounts are unnecessarily reloaded.
This happened because account imports were not copied and thus,
deepEqual detected a difference were none was.
This was addressed by making the copy less shallow.
Furthermore did deepEqual detects a difference when it compared
slices that were appended to while processing a map.
This was fixed by sorting before comparison.
Noticed that Account.clients stored an unnecessary pointer.
Removed duplicated code in systemAccount.
Signed-off-by: Matthias Hanel <mh@synadia.com>
Fixes#1314 by:
There was a data race with a write during reloadAuthorization.
Locking was added to all places where it was missing.
In situations were it appeared feasible, access was moved into existing
lock/unlock.
Where it was added, the lock order was already established.
Signed-off-by: Matthias Hanel <mh@synadia.com>
Fixed#1296, by altering client state on reload
Detect a trace level change on reload and update all clients.
To avoid data races, read client.trace while holding the lock,
pass the value into functionis that trace while not holding the lock.
Delete unused client.debug.
Signed-off-by: Matthias Hanel <mh@synadia.com>
A new config section allows to specify specific TLS parameters for
the account resolver:
```
resolver_tls {
cert_file: ...
key_file: ...
ca_file: ...
}
```
Resolves#1271
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
On config reload, the URL account resolver was recreated and a
Fetch() with empty account was done. Move the empty fetch test
in NewServer() instead.
Added a test that shows that fetch is no longer invoked on reload
but server reports failure on startup.
Resolves#1229
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
- All writes will now be done by the writeLoop, unless when the
writeLoop has not been started yet (likely in connection init).
- Slow consumers for non CLIENT connections will be reported but
not failed. The idea is that routes, gateway, etc.. connections
should stay connected as much as possible. However if a flush
operation times out and no data at all has been written, the
connection will be closed (regardless of type).
- Slow consumers due to max pending is only for CLIENT connections.
This allows sending of SUBs through routes, etc.. to not have
to be chunked.
- The backpressure to CLIENT connections is increased (up to 1sec)
based on the sub's connection pending bytes level.
- Connection is flushed on close from the writeLoop as to not block
the "fast path".
Some tests have been fixed and adapted since now closeConnection()
is not flushing/closing/removing connection in place.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Ensure that lookupAccount does not hold server lock during
updateAccount and fetchAccount.
Updating the account cannot have the server lock because it is
possible that during updateAccountClaims(), clients are being
removed, which would try to get the server lock (deep down in
closeConnection/s.removeClient).
Added a test that would have show the deadlock prior to changes
in this PR.
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
When tls is on routes it can cause reloadAuthorization to be called.
We were assuming configured accounts, but did not copy the remote map.
This copies the remote map when transferring for configured accounts
and also handles operator mode. In operator mode we leave the accounts
in place, and if we have a memory resolver we will remove accounts that
are not longer defined or have bad claims.
Signed-off-by: Derek Collison <derek@nats.io>
