From df3253de4250bbda4c7149cf5ce9b879a862cf4e Mon Sep 17 00:00:00 2001 From: ainsley Date: Wed, 4 Sep 2019 16:36:14 -0500 Subject: [PATCH] review changes --- nats_streaming/configuring/cfgfile.md | 50 ++++++++++++----------- nats_streaming/configuring/cmdline.md | 7 ++-- nats_streaming/configuring/configuring.md | 10 ++++- nats_streaming/configuring/filestore.md | 4 +- nats_streaming/configuring/sqlstore.md | 2 +- nats_streaming/configuring/storelimits.md | 10 ++--- nats_streaming/configuring/tls.md | 4 +- nats_streaming/gettingstarted/changes.md | 13 +++++- 8 files changed, 58 insertions(+), 42 deletions(-) diff --git a/nats_streaming/configuring/cfgfile.md b/nats_streaming/configuring/cfgfile.md index 8057fa1..63d09b7 100644 --- a/nats_streaming/configuring/cfgfile.md +++ b/nats_streaming/configuring/cfgfile.md @@ -1,10 +1,10 @@ -# Configuration file +# Configuration File -You can use a configuration file to configure the options specific to the NATS Streaming server. +You can use a configuration file to configure the options specific to the NATS Streaming Server. Use the `-sc` or `-stan_config` command line parameter to specify the file to use. -For the embedded NATS Server, you can use another configuration file and pass it to the Streaming server using `-c` or `--config` command line parameters. +For the embedded NATS Server, you can use another configuration file and pass it to the Streaming Server using `-c` or `--config` command line parameters. Since most options do not overlap, it is possible to combine all options into a single file and specify this file using either the `-sc` or `-c` command line parameter. @@ -34,20 +34,20 @@ streaming: { } ``` -However, if you want to avoid any possible conflict, simply use two different configuration files! +However, if you want to avoid any possible conflict, simply use two different configuration files. Note the order in which options are applied during the start of a NATS Streaming server: 1. Start with some reasonable default options. 2. If a configuration file is specified, override those options - with all options defined in the file. This include options that are defined + with all options defined in the file. This includes options that are defined but have no value specified. In this case, the zero value for the type of the option will be used. 3. Any command line parameter override all of the previous set options. In general the configuration parameters are the same as the command line arguments. Below is the list of NATS Streaming parameters: -| Parameter | Meaning | Possible values | Usage example | +| Parameter | Meaning | Possible Values | Usage Example | |:----|:----|:----|:----| | cluster_id | Cluster name | String, underscore possible | `cluster_id: "my_cluster_name"` | | discover_prefix | Subject prefix for server discovery by clients | NATS Subject | `discover_prefix: "_STAN.Discovery"` | @@ -57,34 +57,34 @@ In general the configuration parameters are the same as the command line argumen | sv | Enable trace logging | `true` or `false` | `sv: true` | | nats_server_url | If specified, connects to an external NATS Server, otherwise starts an embedded one | NATS URL | `nats_server_url: "nats://localhost:4222"` | | secure | If true, creates a TLS connection to the server but without the need to use TLS configuration (no NATS Server certificate verification) | `true` or `false` | `secure: true` | -| tls | TLS Configuration | Map: `tls: { ... }` | **See details below** | -| store_limits | Store Limits | Map: `store_limits: { ... }` | **See details below** | -| file_options | File Store specific options | Map: `file_options: { ... }` | **See details below** | -| sql_options | SQL Store specific options | Map: `sql_options: { ... }` | **See details below** | +| tls | TLS Configuration | Map: `tls: { ... }` | [**See details below**](#tls-configuration) | +| store_limits | Store Limits | Map: `store_limits: { ... }` | [**See details below**](#store-limits-configuration) | +| file_options | File Store specific options | Map: `file_options: { ... }` | [**See details below**](#file-options-configuration) | +| sql_options | SQL Store specific options | Map: `sql_options: { ... }` | [**See details below**](#sql-options-configuration) | | hb_interval | Interval at which the server sends an heartbeat to a client | Duration | `hb_interval: "10s"` | | hb_timeout | How long the server waits for a heartbeat response from the client before considering it a failed heartbeat | Duration | `hb_timeout: "10s"` | | hb_fail_count | Count of failed heartbeats before server closes the client connection. The actual total wait is: (fail count + 1) * (hb interval + hb timeout) | Number | `hb_fail_count: 2` | | ft_group | In Fault Tolerance mode, you can start a group of streaming servers with only one server being active while others are running in standby mode. This is the name of this FT group | String | `ft_group: "my_ft_group"` | | partitioning | If set to true, a list of channels must be defined in store_limits/channels section. This section then serves two purposes, overriding limits for a given channel or adding it to the partition | `true` or `false` | `partitioning: true` | -| cluster | Cluster Configuration | Map: `cluster: { ... }` | **See details below** | +| cluster | Cluster Configuration | Map: `cluster: { ... }` | [**See details below**](#cluster-configuration) | | encrypt | Specify if server should encrypt messages (only the payload) when storing them | `true` or `false` | `encrypt: true` | | encryption_cipher | Cipher to use for encryption. Currently support AES and CHAHA (ChaChaPoly). Defaults to AES | `AES` or `CHACHA` | `encryption_cipher: "AES"` | | encryption_key | Encryption key. It is recommended to specify the key through the `NATS_STREAMING_ENCRYPTION_KEY` environment variable instead | String | `encryption_key: "mykey"` | -TLS Configuration: +## TLS Configuration -Note that the Streaming server uses a connection to a NATS Server, and so the NATS Streaming TLS Configuration +Note that the Streaming Server uses a connection to a NATS Server, and so the NATS Streaming TLS Configuration is in fact a client-side TLS configuration. -| Parameter | Meaning | Possible values | Usage example | +| Parameter | Meaning | Possible Values | Usage Example | |:----|:----|:----|:----| | client_cert | Client key for the streaming server | File path | `client_cert: "/path/to/client/cert_file"` | | client_key | Client certificate for the streaming server | File path | `client_key: "/path/to/client/key_file"` | | client_ca | Client certificate CA for the streaming server | File path | `client_ca: "/path/to/client/ca_file"` | -Store Limits Configuration: +## Store Limits Configuration -| Parameter | Meaning | Possible values | Usage example | +| Parameter | Meaning | Possible Values | Usage Example | |:----|:----|:----|:----| | max_channels | Maximum number of channels, 0 means unlimited | Number >= 0 | `max_channels: 100` | | max_subs | Maximum number of subscriptions per channel, 0 means unlimited | Number >= 0 | `max_subs: 100` | @@ -92,7 +92,9 @@ Store Limits Configuration: | max_bytes | Total size of messages per channel, 0 means unlimited | Number >= 0 | `max_bytes: 1GB` | | max_age | How long messages can stay in the log | Duration | `max_age: "24h"` | | max_inactivity | How long without any subscription and any new message before a channel can be automatically deleted | Duration | `max_inactivity: "24h"` | -| channels | A map of channel names with specific limits | Map: `channels: { ... }` | **See details below** | +| channels | A map of channel names with specific limits | Map: `channels: { ... }` | [**See details below**](#channels) | + +## Channels The `channels` section is a map with the key being the channel name. For instance: ``` @@ -104,7 +106,7 @@ The `channels` section is a map with the key being the channel name. For instanc ``` For a given channel, the possible parameters are: -| Parameter | Meaning | Possible values | Usage example | +| Parameter | Meaning | Possible Values | Usage Example | |:----|:----|:----|:----| | max_subs | Maximum number of subscriptions per channel, 0 means unlimited | Number >= 0 | `max_subs: 100` | | max_msgs | Maximum number of messages per channel, 0 means unlimited | Number >= 0 | `max_msgs: 10000` | @@ -112,9 +114,9 @@ For a given channel, the possible parameters are: | max_age | How long messages can stay in the log | Duration | `max_age: "24h"` | | max_inactivity | How long without any subscription and any new message before a channel can be automatically deleted | Duration | `max_inactivity: "24h"` | -File Options Configuration: +## File Options Configuration -| Parameter | Meaning | Possible values | Usage example | +| Parameter | Meaning | Possible Values | Usage Example | |:----|:----|:----|:----| | compact | Enable/disable file compaction. Only some of the files (`clients.dat` and `subs.dat`) are subject to compaction | `true` or `false` | `compact: true` | | compact_fragmentation | Compaction threshold (in percentage) | Number >= 0 | `compact_fragmentation: 50` | @@ -133,9 +135,9 @@ File Options Configuration: | read_buffer_size | Size of buffers used to read ahead from message stores. This can significantly speed up sending messages to consumers after messages have been published. Default is 2MB. Set to 0 to disable | Bytes | `read_buffer_size: 2MB` | | auto_sync | Interval at which the store should be automatically flushed and sync'ed on disk. Default is every minute. Set to <=0 to disable | Duration | `auto_sync: "2m"` | -Cluster Configuration: +## Cluster Configuration -| Parameter | Meaning | Possible values | Usage example | +| Parameter | Meaning | Possible Values | Usage Example | |:----|:----|:----|:----| | node_id | ID of the node within the cluster if there is no stored ID | String (no whitespace) | `node_id: "node-a"` | | bootstrap | Bootstrap the cluster if there is no existing state by electing self as leader | `true` or `false` | `bootstrap: true` | @@ -151,9 +153,9 @@ Cluster Configuration: | raft_lease_timeout | Specifies how long a leader waits without being able to contact a quorum of nodes before stepping down as leader | Duration | `raft_lease_timeout: "1s"` | | raft_commit_timeout | Specifies the time without an Apply() operation before sending an heartbeat to ensure timely commit. Due to random staggering, may be delayed as much as 2x this value | Duration | `raft_commit_timeout: "100ms"` | -SQL Options Configuration: +## SQL Options Configuration -| Parameter | Meaning | Possible values | Usage example | +| Parameter | Meaning | Possible Values | Usage Example | |:----|:----|:----|:----| | driver | Name of the SQL driver to use | `mysql` or `postgres` | `driver: "mysql"` | | source | How to connect to the database. This is driver specific | String | `source: "ivan:pwd@/nss_db"` | diff --git a/nats_streaming/configuring/cmdline.md b/nats_streaming/configuring/cmdline.md index 6068ae5..dd71cf9 100644 --- a/nats_streaming/configuring/cmdline.md +++ b/nats_streaming/configuring/cmdline.md @@ -1,4 +1,4 @@ -# Command line arguments +# Command Line Arguments The NATS Streaming Server accepts command line arguments to control its behavior. There is a set of parameters specific to the NATS Streaming Server and some to the embedded NATS Server. @@ -72,8 +72,7 @@ Streaming Server SQL Store Options: --sql_max_open_conns Maximum number of opened connections to the database Streaming Server TLS Options: - -secure Use a TLS connection to the NATS server without - verification; weaker than specifying certificates. + -secure Use a TLS connection to the NATS server without verification; weaker than specifying certificates. -tls_client_key Client key for the streaming server -tls_client_cert Client certificate for the streaming server -tls_client_cacert Client certificate CA for the streaming server @@ -121,5 +120,5 @@ NATS Clustering Options: Common Options: -h, --help Show this message -v, --version Show version - --help_tls TLS help. + --help_tls TLS help ``` \ No newline at end of file diff --git a/nats_streaming/configuring/configuring.md b/nats_streaming/configuring/configuring.md index 077d717..9c44dcc 100644 --- a/nats_streaming/configuring/configuring.md +++ b/nats_streaming/configuring/configuring.md @@ -1,3 +1,9 @@ -## Configuration and Administration +# Configuration and Administration -NATS Streaming provides a rich set of commands and parameters to configure all aspects of the server. \ No newline at end of file +NATS Streaming provides a rich set of commands and parameters to configure all aspects of the server. + + * [Command Line Arguments](cmdline.md) + * [Configuration File](cfgfile.md) + * [Store Limits](storelimits.md) + * [Persistence](persistence.md) + * [Securing](tls.md) \ No newline at end of file diff --git a/nats_streaming/configuring/filestore.md b/nats_streaming/configuring/filestore.md index f4dd7b1..7bfea60 100644 --- a/nats_streaming/configuring/filestore.md +++ b/nats_streaming/configuring/filestore.md @@ -23,14 +23,14 @@ Finally, the number of stored messages for a given channel can also be limited w ## Options -As described in the [Configuring](nats_streaming/configuring/configuring.md) section, there are several options that you can use to configure a file store. +As described in the [Configuring](../configuring/cfgfile.html#configuration-file) section, there are several options that you can use to configure a file store. Regardless of channel limits, you can configure message logs to be split in individual files (called file slices). You can configure those slices by number of messages it can contain (`--file_slice_max_msgs`), the size of the file - including the corresponding index file (`--file_slice_max_bytes`), or the period of time that a file slice should cover - starting at the time the first message is stored in that slice (`--file_slice_max_age`). The default file store options are defined such that only the slice size is configured to 64MB. -Note: If you don't configure any slice limit but you do configure channel limits, then the server will automatically +>**Note**: If you don't configure any slice limit but you do configure channel limits, then the server will automatically set some limits for file slices. When messages accumulate in a channel, and limits are reached, older messages are removed. When the first file slice diff --git a/nats_streaming/configuring/sqlstore.md b/nats_streaming/configuring/sqlstore.md index ce1dbf1..647872b 100644 --- a/nats_streaming/configuring/sqlstore.md +++ b/nats_streaming/configuring/sqlstore.md @@ -19,7 +19,7 @@ Here is an example of creating an user `nss` with password `password` for the My mysql -u root -e "CREATE USER 'nss'@'localhost' IDENTIFIED BY 'password'; GRANT ALL PRIVILEGES ON *.* TO 'nss'@'localhost'; CREATE DATABASE nss_db;" ``` -The above has gives all permissions to user `nss`. Once this user is created, we can then create the tables using this user +The above gives all permissions to user `nss`. Once this user is created, we can then create the tables using this user and selecting the `nss_db` database. We then execute all the SQL statements creating the tables from the sql file that is provided in this repo: diff --git a/nats_streaming/configuring/storelimits.md b/nats_streaming/configuring/storelimits.md index 27ecc6d..246d93c 100644 --- a/nats_streaming/configuring/storelimits.md +++ b/nats_streaming/configuring/storelimits.md @@ -3,7 +3,7 @@ The `store_limits` section in the configuration file (or the command line parameters `-mc`, `-mm`, etc..) allow you to configure the global limits. -These limits somewhat offer some upper bound on the size of the storage. By multiplying +These limits offer some upper bounds on the size of the storage. By multiplying the limits per channel with the maximum number of channels, you will get a total limit. It is not the case, though, if you override limits of some channels. Indeed, it is possible @@ -72,7 +72,7 @@ of channels. ***This is true only for channels without wildcards.*** Channels limits can override global limits by being either higher, lower or even set to unlimited. -***An unlimited value applies to the specified limit, not to the whole channel*** +***An unlimited value applies to the specified limit, not to the whole channel.*** That is, in the configuration above, `baz` has the maximum number of messages set to 0, which means ignored or unlimited. Yet, other limits such as max bytes, max age @@ -81,7 +81,7 @@ the store will not check the number of messages but still check the other limits For a truly unlimited channel *all* limits need to be set to 0. -## Limits inheritance +## Limits Inheritance When starting the server from the command line, global limits that are not specified (configuration file or command line parameters) are inherited from default limits @@ -100,9 +100,9 @@ limit to indicate that the limit was inherited from the default store limits. For channels that have been configured, their name is displayed and only the limits being specifically set are displayed to minimize the output. -### Wildcards +## Wildcards -Wildcards are allowed for channels configuration. Limits for `foo.>` +Wildcards are allowed for channel configuration. Limits for `foo.>` will apply to any channel that starts with `foo` (but has at least one more token). If `foo.bar.>` is specified, it will inherit from `foo.>` and from global limits. diff --git a/nats_streaming/configuring/tls.md b/nats_streaming/configuring/tls.md index 83e7184..54cdc30 100644 --- a/nats_streaming/configuring/tls.md +++ b/nats_streaming/configuring/tls.md @@ -1,13 +1,13 @@ # Securing NATS Streaming Server -## Authenticating users +## Authenticating Users To enable user authentication from the command line, you can use the same mechanism as the NATS Server (`nats-server`). You pass in the `—user ` and `—pass ` commands, or `--auth` parameters, and the NATS streaming server will automatically use these credentials. Or you can use a configuration file with a single user or token. When using a configuration file with multi-user authorization, you must use the `—user` and `—pass` parameters with the NATS streaming server, matching a user in the configuration file, in order to specify which user the NATS streaming server should authenticate with to it's embedded NATS server. -For example, if you pass the NATS streaming server a file with a several users, you must run the streaming server as a user such as "Joe" who is defined in the configuration file. +For example, if you pass the NATS Streaming server a file with a several users, you must run the streaming server as a user such as "Joe" who is defined in the configuration file. ## Using TLS diff --git a/nats_streaming/gettingstarted/changes.md b/nats_streaming/gettingstarted/changes.md index 9dd4fd5..cc6077c 100644 --- a/nats_streaming/gettingstarted/changes.md +++ b/nats_streaming/gettingstarted/changes.md @@ -1,5 +1,14 @@ # Important Changes + * [Version `0.10.0`](#version-0100) + * [Version `0.9.0`](#version-090) + * [Version `0.8.0-beta`](#version-080-beta) + * [Version `0.6.0`](#version-060) + * [Version `0.5.0`](#version-050) + * [Version `0.4.0`](#version-040) + + + ## Version `0.10.0` The server needs to persist more state for a client connection. Therefore, the Store interface has been changed: @@ -54,7 +63,7 @@ The store implementation should ignore the call if the given sequence number is The Store interface has been heavily modified. Some of the responsibilities have been moved into the server resulting on deletion of some Store APIs and removal of `UserData` fields in `Client` and `ChannelStore` (renamed `Channel`) objects. -NOTE: Although the interface has changed, the file format of the FileStore implementation has not, which means +>**NOTE:** Although the interface has changed, the file format of the FileStore implementation has not, which means that there is backward/forward compatibility between this and previous releases. The Store interface was updated: @@ -64,7 +73,7 @@ The Store interface was updated: * `RecoveredChannel` has a pointer to a `Channel` (formely `ChannelStore`) and an array of pointers to `RecoveredSubscription` objects. * `RecoveredSubscription` replaces `RecoveredSubState`. * `Client` no longer stores a `UserData` field. -* `Channel` (formely `ChannelStore`) no longer stores a `UserData` field. +* `Channel` (formerly `ChannelStore`) no longer stores a `UserData` field. * `CreateChannel()` no longer accepts a `userData interface{}` parameter. It returns a `*Channel` and an `error`. If the channel already exists, the error `ErrAlreadyExists` is returned. * `LookupChannel()`, `HasChannel()`, `GetChannels()`, `GetChannelsCount()`, `GetClient()`, `GetClients`, `GetClientsCount()` and `MsgsState()` APIs