taigrr/nats.docs
1
0
Fork 0
mirror of https://github.com/taigrr/nats.docs synced 2025-01-18 04:03:23 -08:00
Code Issues Projects Releases Wiki Activity

review changes

Browse Source
This commit is contained in:
ainsley 2019-09-04 16:36:14 -05:00
parent 4ef08006bc
commit df3253de42
8 changed files with 58 additions and 42 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
nats_streaming/configuring/cfgfile.md
View File

@ -1,10 +1,10 @@
# Configuration file
# Configuration File
You can use a configuration file to configure the options specific to the NATS Streaming server.
You can use a configuration file to configure the options specific to the NATS Streaming Server.
Use the `-sc` or `-stan_config` command line parameter to specify the file to use.
For the embedded NATS Server, you can use another configuration file and pass it to the Streaming server using `-c` or `--config` command line parameters.
For the embedded NATS Server, you can use another configuration file and pass it to the Streaming Server using `-c` or `--config` command line parameters.
Since most options do not overlap, it is possible to combine all options into a single file and specify this file using either the `-sc` or `-c` command line parameter.
@ -34,20 +34,20 @@ streaming: {
}
```
However, if you want to avoid any possible conflict, simply use two different configuration files!
However, if you want to avoid any possible conflict, simply use two different configuration files.
Note the order in which options are applied during the start of a NATS Streaming server:
1. Start with some reasonable default options.
2. If a configuration file is specified, override those options
with all options defined in the file. This include options that are defined
with all options defined in the file. This includes options that are defined
but have no value specified. In this case, the zero value for the type of the
option will be used.
3. Any command line parameter override all of the previous set options.
In general the configuration parameters are the same as the command line arguments. Below is the list of NATS Streaming parameters:
| Parameter | Meaning | Possible values | Usage example |
| Parameter | Meaning | Possible Values | Usage Example |
|:----|:----|:----|:----|
| cluster_id | Cluster name | String, underscore possible | `cluster_id: "my_cluster_name"` |
| discover_prefix | Subject prefix for server discovery by clients | NATS Subject | `discover_prefix: "_STAN.Discovery"` |
@ -57,34 +57,34 @@ In general the configuration parameters are the same as the command line argumen
| sv | Enable trace logging | `true` or `false` | `sv: true` |
| nats_server_url | If specified, connects to an external NATS Server, otherwise starts an embedded one | NATS URL | `nats_server_url: "nats://localhost:4222"` |
| secure | If true, creates a TLS connection to the server but without the need to use TLS configuration (no NATS Server certificate verification) | `true` or `false` | `secure: true` |
| tls | TLS Configuration | Map: `tls: { ... }` | **See details below** |
| store_limits | Store Limits | Map: `store_limits: { ... }` | **See details below** |
| file_options | File Store specific options | Map: `file_options: { ... }` | **See details below** |
| sql_options | SQL Store specific options | Map: `sql_options: { ... }` | **See details below** |
| tls | TLS Configuration | Map: `tls: { ... }` | [**See details below**](#tls-configuration) |
| store_limits | Store Limits | Map: `store_limits: { ... }` | [**See details below**](#store-limits-configuration) |
| file_options | File Store specific options | Map: `file_options: { ... }` | [**See details below**](#file-options-configuration) |
| sql_options | SQL Store specific options | Map: `sql_options: { ... }` | [**See details below**](#sql-options-configuration) |
| hb_interval | Interval at which the server sends an heartbeat to a client | Duration | `hb_interval: "10s"` |
| hb_timeout | How long the server waits for a heartbeat response from the client before considering it a failed heartbeat | Duration | `hb_timeout: "10s"` |
| hb_fail_count | Count of failed heartbeats before server closes the client connection. The actual total wait is: (fail count + 1) * (hb interval + hb timeout) | Number | `hb_fail_count: 2` |
| ft_group | In Fault Tolerance mode, you can start a group of streaming servers with only one server being active while others are running in standby mode. This is the name of this FT group | String | `ft_group: "my_ft_group"` |
| partitioning | If set to true, a list of channels must be defined in store_limits/channels section. This section then serves two purposes, overriding limits for a given channel or adding it to the partition | `true` or `false` | `partitioning: true` |
| cluster | Cluster Configuration | Map: `cluster: { ... }` | **See details below** |
| cluster | Cluster Configuration | Map: `cluster: { ... }` | [**See details below**](#cluster-configuration) |
| encrypt | Specify if server should encrypt messages (only the payload) when storing them | `true` or `false` | `encrypt: true` |
| encryption_cipher | Cipher to use for encryption. Currently support AES and CHAHA (ChaChaPoly). Defaults to AES | `AES` or `CHACHA` | `encryption_cipher: "AES"` |
| encryption_key | Encryption key. It is recommended to specify the key through the `NATS_STREAMING_ENCRYPTION_KEY` environment variable instead | String | `encryption_key: "mykey"` |
TLS Configuration:
## TLS Configuration
Note that the Streaming server uses a connection to a NATS Server, and so the NATS Streaming TLS Configuration
Note that the Streaming Server uses a connection to a NATS Server, and so the NATS Streaming TLS Configuration
is in fact a client-side TLS configuration.
| Parameter | Meaning | Possible values | Usage example |
| Parameter | Meaning | Possible Values | Usage Example |
|:----|:----|:----|:----|
| client_cert | Client key for the streaming server | File path | `client_cert: "/path/to/client/cert_file"` |
| client_key | Client certificate for the streaming server | File path | `client_key: "/path/to/client/key_file"` |
| client_ca | Client certificate CA for the streaming server | File path | `client_ca: "/path/to/client/ca_file"` |
Store Limits Configuration:
## Store Limits Configuration
| Parameter | Meaning | Possible values | Usage example |
| Parameter | Meaning | Possible Values | Usage Example |
|:----|:----|:----|:----|
| max_channels | Maximum number of channels, 0 means unlimited | Number >= 0 | `max_channels: 100` |
| max_subs | Maximum number of subscriptions per channel, 0 means unlimited | Number >= 0 | `max_subs: 100` |
@ -92,7 +92,9 @@ Store Limits Configuration:
| max_bytes | Total size of messages per channel, 0 means unlimited | Number >= 0 | `max_bytes: 1GB` |
| max_age | How long messages can stay in the log | Duration | `max_age: "24h"` |
| max_inactivity | How long without any subscription and any new message before a channel can be automatically deleted | Duration | `max_inactivity: "24h"` |
| channels | A map of channel names with specific limits | Map: `channels: { ... }` | **See details below** |
| channels | A map of channel names with specific limits | Map: `channels: { ... }` | [**See details below**](#channels) |
## Channels
The `channels` section is a map with the key being the channel name. For instance:
```
@ -104,7 +106,7 @@ The `channels` section is a map with the key being the channel name. For instanc
```
For a given channel, the possible parameters are:
| Parameter | Meaning | Possible values | Usage example |
| Parameter | Meaning | Possible Values | Usage Example |
|:----|:----|:----|:----|
| max_subs | Maximum number of subscriptions per channel, 0 means unlimited | Number >= 0 | `max_subs: 100` |
| max_msgs | Maximum number of messages per channel, 0 means unlimited | Number >= 0 | `max_msgs: 10000` |
@ -112,9 +114,9 @@ For a given channel, the possible parameters are:
| max_age | How long messages can stay in the log | Duration | `max_age: "24h"` |
| max_inactivity | How long without any subscription and any new message before a channel can be automatically deleted | Duration | `max_inactivity: "24h"` |
File Options Configuration:
## File Options Configuration
| Parameter | Meaning | Possible values | Usage example |
| Parameter | Meaning | Possible Values | Usage Example |
|:----|:----|:----|:----|
| compact | Enable/disable file compaction. Only some of the files (`clients.dat` and `subs.dat`) are subject to compaction | `true` or `false` | `compact: true` |
| compact_fragmentation | Compaction threshold (in percentage) | Number >= 0 | `compact_fragmentation: 50` |
@ -133,9 +135,9 @@ File Options Configuration:
| read_buffer_size | Size of buffers used to read ahead from message stores. This can significantly speed up sending messages to consumers after messages have been published. Default is 2MB. Set to 0 to disable | Bytes | `read_buffer_size: 2MB` |
| auto_sync | Interval at which the store should be automatically flushed and sync'ed on disk. Default is every minute. Set to <=0 to disable | Duration | `auto_sync: "2m"` |
Cluster Configuration:
## Cluster Configuration
| Parameter | Meaning | Possible values | Usage example |
| Parameter | Meaning | Possible Values | Usage Example |
|:----|:----|:----|:----|
| node_id | ID of the node within the cluster if there is no stored ID | String (no whitespace) | `node_id: "node-a"` |
| bootstrap | Bootstrap the cluster if there is no existing state by electing self as leader | `true` or `false` | `bootstrap: true` |
@ -151,9 +153,9 @@ Cluster Configuration:
| raft_lease_timeout | Specifies how long a leader waits without being able to contact a quorum of nodes before stepping down as leader | Duration | `raft_lease_timeout: "1s"` |
| raft_commit_timeout | Specifies the time without an Apply() operation before sending an heartbeat to ensure timely commit. Due to random staggering, may be delayed as much as 2x this value | Duration | `raft_commit_timeout: "100ms"` |
SQL Options Configuration:
## SQL Options Configuration
| Parameter | Meaning | Possible values | Usage example |
| Parameter | Meaning | Possible Values | Usage Example |
|:----|:----|:----|:----|
| driver | Name of the SQL driver to use | `mysql` or `postgres` | `driver: "mysql"` |
| source | How to connect to the database. This is driver specific | String | `source: "ivan:pwd@/nss_db"` |

7
nats_streaming/configuring/cmdline.md
View File

@ -1,4 +1,4 @@
# Command line arguments
# Command Line Arguments
The NATS Streaming Server accepts command line arguments to control its behavior. There is a set of parameters specific to the NATS Streaming Server and some to the embedded NATS Server.
@ -72,8 +72,7 @@ Streaming Server SQL Store Options:
--sql_max_open_conns <int> Maximum number of opened connections to the database
Streaming Server TLS Options:
-secure <bool> Use a TLS connection to the NATS server without
verification; weaker than specifying certificates.
-secure <bool> Use a TLS connection to the NATS server without verification; weaker than specifying certificates.
-tls_client_key <string> Client key for the streaming server
-tls_client_cert <string> Client certificate for the streaming server
-tls_client_cacert <string> Client certificate CA for the streaming server
@ -121,5 +120,5 @@ NATS Clustering Options:
Common Options:
-h, --help Show this message
-v, --version Show version
--help_tls TLS help.
--help_tls TLS help
```

10
nats_streaming/configuring/configuring.md
View File

@ -1,3 +1,9 @@
## Configuration and Administration
# Configuration and Administration
NATS Streaming provides a rich set of commands and parameters to configure all aspects of the server.
NATS Streaming provides a rich set of commands and parameters to configure all aspects of the server.
* [Command Line Arguments](cmdline.md)
* [Configuration File](cfgfile.md)
* [Store Limits](storelimits.md)
* [Persistence](persistence.md)
* [Securing](tls.md)

4
nats_streaming/configuring/filestore.md
View File

@ -23,14 +23,14 @@ Finally, the number of stored messages for a given channel can also be limited w
## Options
As described in the [Configuring](nats_streaming/configuring/configuring.md) section, there are several options that you can use to configure a file store.
As described in the [Configuring](../configuring/cfgfile.html#configuration-file) section, there are several options that you can use to configure a file store.
Regardless of channel limits, you can configure message logs to be split in individual files (called file slices). You can configure
those slices by number of messages it can contain (`--file_slice_max_msgs`), the size of the file - including the corresponding index file
(`--file_slice_max_bytes`), or the period of time that a file slice should cover - starting at the time the first message is stored in
that slice (`--file_slice_max_age`). The default file store options are defined such that only the slice size is configured to 64MB.
Note: If you don't configure any slice limit but you do configure channel limits, then the server will automatically
>**Note**: If you don't configure any slice limit but you do configure channel limits, then the server will automatically
set some limits for file slices.
When messages accumulate in a channel, and limits are reached, older messages are removed. When the first file slice

2
nats_streaming/configuring/sqlstore.md
View File

@ -19,7 +19,7 @@ Here is an example of creating an user `nss` with password `password` for the My
mysql -u root -e "CREATE USER 'nss'@'localhost' IDENTIFIED BY 'password'; GRANT ALL PRIVILEGES ON *.* TO 'nss'@'localhost'; CREATE DATABASE nss_db;"
```
The above has gives all permissions to user `nss`. Once this user is created, we can then create the tables using this user
The above gives all permissions to user `nss`. Once this user is created, we can then create the tables using this user
and selecting the `nss_db` database. We then execute all the SQL statements creating the tables from the sql file that
is provided in this repo:

10
nats_streaming/configuring/storelimits.md
View File

@ -3,7 +3,7 @@
The `store_limits` section in the configuration file (or the command line parameters
`-mc`, `-mm`, etc..) allow you to configure the global limits.
These limits somewhat offer some upper bound on the size of the storage. By multiplying
These limits offer some upper bounds on the size of the storage. By multiplying
the limits per channel with the maximum number of channels, you will get a total limit.
It is not the case, though, if you override limits of some channels. Indeed, it is possible
@ -72,7 +72,7 @@ of channels. ***This is true only for channels without wildcards.***
Channels limits can override global limits by being either higher, lower or even set to
unlimited.
***An unlimited value applies to the specified limit, not to the whole channel***
***An unlimited value applies to the specified limit, not to the whole channel.***
That is, in the configuration above, `baz` has the maximum number of messages set
to 0, which means ignored or unlimited. Yet, other limits such as max bytes, max age
@ -81,7 +81,7 @@ the store will not check the number of messages but still check the other limits
For a truly unlimited channel *all* limits need to be set to 0.
## Limits inheritance
## Limits Inheritance
When starting the server from the command line, global limits that are not specified
(configuration file or command line parameters) are inherited from default limits
@ -100,9 +100,9 @@ limit to indicate that the limit was inherited from the default store limits.
For channels that have been configured, their name is displayed and only the
limits being specifically set are displayed to minimize the output.
### Wildcards
## Wildcards
Wildcards are allowed for channels configuration. Limits for `foo.>`
Wildcards are allowed for channel configuration. Limits for `foo.>`
will apply to any channel that starts with `foo` (but has at least one more token).
If `foo.bar.>` is specified, it will inherit from `foo.>` and from global limits.

4
nats_streaming/configuring/tls.md
View File

@ -1,13 +1,13 @@
# Securing NATS Streaming Server
## Authenticating users
## Authenticating Users
To enable user authentication from the command line, you can use the same mechanism as the NATS Server (`nats-server`). You pass in the `—user <user>` and `—pass <pass>` commands, or `--auth` parameters, and the NATS streaming server will automatically use these credentials. Or you can use a configuration file with a single user or token.
When using a configuration file with multi-user authorization, you must use the `—user` and `—pass` parameters with the NATS streaming server, matching a user in the configuration file, in order to specify which user the NATS streaming server should authenticate with to it's embedded NATS server.
For example, if you pass the NATS streaming server a file with a several users, you must run the streaming server as a user such as "Joe" who is defined in the configuration file.
For example, if you pass the NATS Streaming server a file with a several users, you must run the streaming server as a user such as "Joe" who is defined in the configuration file.
## Using TLS

13
nats_streaming/gettingstarted/changes.md
View File

@ -1,5 +1,14 @@
# Important Changes
* [Version `0.10.0`](#version-0100)
* [Version `0.9.0`](#version-090)
* [Version `0.8.0-beta`](#version-080-beta)
* [Version `0.6.0`](#version-060)
* [Version `0.5.0`](#version-050)
* [Version `0.4.0`](#version-040)
## Version `0.10.0`
The server needs to persist more state for a client connection. Therefore, the Store interface has been changed:
@ -54,7 +63,7 @@ The store implementation should ignore the call if the given sequence number is
The Store interface has been heavily modified. Some of the responsibilities have been moved into the server
resulting on deletion of some Store APIs and removal of `UserData` fields in `Client` and `ChannelStore` (renamed `Channel`) objects.
NOTE: Although the interface has changed, the file format of the FileStore implementation has not, which means
>**NOTE:** Although the interface has changed, the file format of the FileStore implementation has not, which means
that there is backward/forward compatibility between this and previous releases.
The Store interface was updated:
@ -64,7 +73,7 @@ The Store interface was updated:
* `RecoveredChannel` has a pointer to a `Channel` (formely `ChannelStore`) and an array of pointers to `RecoveredSubscription` objects.
* `RecoveredSubscription` replaces `RecoveredSubState`.
* `Client` no longer stores a `UserData` field.
* `Channel` (formely `ChannelStore`) no longer stores a `UserData` field.
* `Channel` (formerly `ChannelStore`) no longer stores a `UserData` field.
* `CreateChannel()` no longer accepts a `userData interface{}` parameter. It returns a `*Channel` and an `error`. If the channel
already exists, the error `ErrAlreadyExists` is returned.
* `LookupChannel()`, `HasChannel()`, `GetChannels()`, `GetChannelsCount()`, `GetClient()`, `GetClients`, `GetClientsCount()` and `MsgsState()` APIs