diff --git a/SUMMARY.md b/SUMMARY.md index 8851cd9..09d8cba 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -22,6 +22,7 @@ * [Connecting to the Default Server](developing-with-nats/connecting/default_server.md) * [Connecting to a Specific Server](developing-with-nats/connecting/specific_server.md) * [Connecting to a Cluster](developing-with-nats/connecting/cluster.md) + * [Connection Name](developing-with-nats/connecting/name.md) * [Setting a Connect Timeout](developing-with-nats/connecting/connect_timeout.md) * [Ping/Pong Protocol](developing-with-nats/connecting/pingpong.md) * [Controlling the Client/Server Protocol](developing-with-nats/connecting/protocol.md) @@ -29,8 +30,8 @@ * [Automatic Reconnections](developing-with-nats/reconnect/README.md) * [Disabling Reconnect](developing-with-nats/reconnect/disable.md) * [Set the Number of Reconnect Attempts](developing-with-nats/reconnect/max.md) - * [Pausing Between Reconnect Attempts](developing-with-nats/reconnect/wait.md) * [Avoiding the Thundering Herd](developing-with-nats/reconnect/random.md) + * [Pausing Between Reconnect Attempts](developing-with-nats/reconnect/wait.md) * [Listening for Reconnect Events](developing-with-nats/reconnect/events.md) * [Buffering Messages During Reconnect Attempts](developing-with-nats/reconnect/buffer.md) * [Securing Connections](developing-with-nats/security/README.md) diff --git a/developing-with-nats/connecting/cluster.md b/developing-with-nats/connecting/cluster.md index 9720fd6..b65f558 100644 --- a/developing-with-nats/connecting/cluster.md +++ b/developing-with-nats/connecting/cluster.md @@ -7,13 +7,13 @@ When connecting to a cluster, there are a few things to think about. * The reconnect algorithm \(discussed later\) * Server provided URLs -When a client library first tries to connect it will use the list of URLs provided to the connection options or function. These URLs are checked, usually in order, and the first successful connection is used. +When a client library first tries to connect it will use the list of URLs provided to the connection options or function. These URLs are usually checked in random order as to not have every client connect to the same server. The first successful connection is used. Randomization can be [explicitly disabled](../reconnect/random.md). After a client connects to the server, the server may provide a list of URLs for additional known servers. This allows a client to connect to one server and still have other servers available during reconnect. -To insure the initial connection, your code should include a list of reasonable _front line_ servers. Those servers may know about other members of the cluster, and may tell the client about those members. But you don't have to configure the client to pass every valid member of the cluster in the connect method. +To ensure the initial connection, your code should include a list of reasonable _front line_ or _seed_ servers. Those servers may know about other members of the cluster, and may tell the client about those members. But you don't have to configure the client to pass every valid member of the cluster in the connect method. -By providing the ability to pass multiple connect options NATS can handle the possibility of a machine going down or being unavailable to a client. By adding the ability of the server to feed clients a list of known servers as part of the client-server protocol the mesh created by a cluster can grow and change organically while the clients are running. +By providing the ability to pass multiple connect options, NATS can handle the possibility of a machine going down or being unavailable to a client. By adding the ability of the server to feed clients a list of known servers as part of the client-server protocol the mesh created by a cluster can grow and change organically while the clients are running. _Note, failure behavior is library dependent, please check the documentation for your client library on information about what happens if the connect fails._ diff --git a/developing-with-nats/connecting/connect_timeout.md b/developing-with-nats/connecting/connect_timeout.md index ac08ac6..1c4ed7a 100644 --- a/developing-with-nats/connecting/connect_timeout.md +++ b/developing-with-nats/connecting/connect_timeout.md @@ -1,6 +1,6 @@ # Setting a Connect Timeout -Each library has its own, language preferred way, to pass connection options. One of the most common options is a connection timeout. To set the maximum time to connect to a server to 10 seconds: +Each library has its own, language preferred way, to pass connection options. One of the most common options is a connect timeout. It limits how long it can take to establish a connection to a server. Should multiple URLs be provided, this timeout applies to each cluster member individually. To set the maximum time to connect to a server to 10 seconds: {% tabs %} {% tab title="Go" %} @@ -30,13 +30,29 @@ nc.close(); {% endtab %} {% tab title="JavaScript" %} - +```javascript +let nc = NATS.connect({ + url: "nats://demo.nats.io:4222", + timeout: 10*1000 //10s +}); +nc.on('connect', (c) => { + // Do something with the connection + doSomething(); + // When done close it + nc.close(); +}); +nc.on('error', (err) => { + failed(err); +}); +``` {% endtab %} {% tab title="Python" %} ```python nc = NATS() -await nc.connect(connect_timeout=2) +await nc.connect( + servers=["nats://demo.nats.io:4222"], + connect_timeout=10) # Do something with the connection @@ -49,7 +65,7 @@ await nc.close() # There is currently no connect timeout as part of the Ruby NATS client API, but you can use a timer to mimic it. require 'nats/client' -timer = EM.add_timer(5) do +timer = EM.add_timer(10) do NATS.connect do |nc| # Do something with the connection @@ -65,7 +81,7 @@ EM.cancel_timer(timer) ```typescript let nc = await connect({ url: "nats://demo.nats.io:4222", - timeout: 1000 + timeout: 10*1000 //10s }); ``` {% endtab %} diff --git a/developing-with-nats/connecting/name.md b/developing-with-nats/connecting/name.md index b1dc56c..e41ce42 100644 --- a/developing-with-nats/connecting/name.md +++ b/developing-with-nats/connecting/name.md @@ -1,5 +1,89 @@ # Setting the Connection Name -Connections can be assigned a name which will appear in some of the server monitoring data. This name is not required but can help in debugging and testing. +Connections can be assigned a name which will appear in some of the server monitoring data. This name is not required, but is **highly recommended** as a friendly connection name will help in monitoring, error reporting, debugging, and testing. -!INCLUDE "../../_examples/connect_name.html" \ No newline at end of file +{% tabs %} +{% tab title="Go" %} +```go +nc, err := nats.Connect("demo.nats.io", nats.Name("API Name Option Example")) +if err != nil { + log.Fatal(err) +} +defer nc.Close() + +// Do something with the connection +``` +{% endtab %} + +{% tab title="Java" %} +```java +Options options = new Options.Builder(). + server("nats://demo.nats.io:4222"). + connectionName("API Name Option Example"). // Set Name + build(); +Connection nc = Nats.connect(options); + +// Do something with the connection + +nc.close(); +``` +{% endtab %} + +{% tab title="JavaScript" %} +```javascript +let nc = NATS.connect({ + url: "nats://demo.nats.io:4222", + name: "API Name Option Example" +}); +nc.on('connect', (c) => { + // Do something with the connection + doSomething(); + // When done close it + nc.close(); +}); +nc.on('error', (err) => { + failed(err); +}); +``` +{% endtab %} + +{% tab title="Python" %} +```python +nc = NATS() +await nc.connect( + servers=["nats://demo.nats.io:4222"], + name="API Name Option Example") + +# Do something with the connection + +await nc.close() +``` +{% endtab %} + +{% tab title="Ruby" %} +```ruby +require 'nats/client' + +NATS.start(servers: ["nats://demo.nats.io:4222"], name: "API Name Option Example") do |nc| + # Do something with the connection + + # Close the connection + nc.close +end +``` +{% endtab %} + +{% tab title="TypeScript" %} +```typescript +// will throw an exception if connection fails +let nc = await connect({ + url:"nats://demo.nats.io:4222", + name: "API Name Option Example" +}) +// Do something with the connection + +// Close the connection +nc.close(); +``` +{% endtab %} +{% endtabs %} \ No newline at end of file diff --git a/developing-with-nats/connecting/pingpong.md b/developing-with-nats/connecting/pingpong.md index 531868c..04ea312 100644 --- a/developing-with-nats/connecting/pingpong.md +++ b/developing-with-nats/connecting/pingpong.md @@ -1,18 +1,20 @@ # Ping/Pong Protocol -The client and server use a simple PING/PONG protocol to check that they are both still connected. The client will ping the server on a regular, configured interval so that the server usually doesn't have to initiate the PING/PONG interaction. +The client and server use a simple PING/PONG protocol to check that either of them are still connected to the other. On a regular interval the client will ping the server, which responds with a pong. ![](../../.gitbook/assets/pingpong.svg) -## Set the Ping Interval +Once a configurable maximum of outstanding pings without a single pong reply is hit, the connection is closed as stale. Together these two values define a timeout for the connection which specifies how quickly the client will be notified of a problem. This will also help when there is a remote network partition where the operating system does not detect a socket error. Upon connection close the client will attempt to reconnect. When it knows about other server, these will be tried next. -If you have a connection that is going to be open a long time with few messages traveling on it, setting this PING interval can control how quickly the client will be notified of a problem. However on connections with a lot of traffic, the client will often figure out there is a problem between PINGS, and as a result the default PING interval is often on the order of minutes. To set the interval to 20s: +In the presence of traffic, such as messages or client side pings, the server will not initiate the PING/PONG interaction. + +On connections with a lot of traffic, the client will often figure out there is a problem between PINGS, and as a result the default PING interval is often on the order of minutes. To set the interval to 20s and limit outstanding pings to 5, thus force a closed connection after 100s of inactivity: {% tabs %} {% tab title="Go" %} ```go // Set Ping Interval to 20 seconds -nc, err := nats.Connect("demo.nats.io", nats.Name("API Ping Example"), nats.PingInterval(20*time.Second)) +nc, err := nats.Connect("demo.nats.io", nats.Name("API Ping Example"), nats.PingInterval(20*time.Second), nats.MaxPingsOutstanding(5)) if err != nil { log.Fatal(err) } @@ -27,93 +29,6 @@ defer nc.Close() Options options = new Options.Builder(). server("nats://demo.nats.io:4222"). pingInterval(Duration.ofSeconds(20)). // Set Ping Interval - build(); -Connection nc = Nats.connect(options); - -// Do something with the connection - -nc.close(); -``` -{% endtab %} - -{% tab title="JavaScript" %} -```javascript -let nc = NATS.connect({ - pingInterval: 20*1000, //20s - url: "nats://demo.nats.io:4222" -}); -``` -{% endtab %} - -{% tab title="Python" %} -```python -nc = NATS() - -await nc.connect( - servers=["nats://demo.nats.io:4222"], - # Set Ping Interval to 20 seconds - ping_interval=20, - ) - -# Do something with the connection. -``` -{% endtab %} - -{% tab title="Ruby" %} -```ruby -require 'nats/client' - -NATS.start(ping_interval: 20) do |nc| - nc.on_reconnect do - puts "Got reconnected to #{nc.connected_server}" - end - - nc.on_disconnect do |reason| - puts "Got disconnected! #{reason}" - end - - # Do something with the connection -end -``` -{% endtab %} - -{% tab title="TypeScript" %} -```typescript -// will throw an exception if connection fails -let nc = await connect({ - pingInterval: 20*1000, //20s - url: "nats://demo.nats.io:4222" -}); -nc.close(); -``` -{% endtab %} -{% endtabs %} - -## Limit Outgoing Pings - -The PING/PONG interaction is also used by most of the clients as a way to flush the connection to the server. Clients that cache outgoing messages provide a flush call that will run a PING/PONG. The flush will wait for the PONG to return, telling it that all cached messages have been processed, including the PING. The number of cached PING requests can be limited in most clients to insure that traffic problems are identified early. This configuration for _max outgoing pings_ or similar will usually default to a small number and should only be increased if you are worried about fast flush traffic, perhaps in multiple threads. - -For example, to set the maximum number of outgoing pings to 5: - -{% tabs %} -{% tab title="Go" %} -```go -// Set maximum number of PINGs out without getting a PONG back - // before the connection will be disconnected as a stale connection. - nc, err := nats.Connect("demo.nats.io", nats.Name("API MaxPing Example"), nats.MaxPingsOutstanding(5)) - if err != nil { - log.Fatal(err) - } - defer nc.Close() - - // Do something with the connection -``` -{% endtab %} - -{% tab title="Java" %} -```java -Options options = new Options.Builder(). - server("nats://demo.nats.io:4222"). maxPingsOut(5). // Set max pings in flight build(); Connection nc = Nats.connect(options); @@ -127,6 +42,7 @@ nc.close(); {% tab title="JavaScript" %} ```javascript let nc = NATS.connect({ + pingInterval: 20*1000, //20s maxPingOut: 5, url: "nats://demo.nats.io:4222" }); @@ -139,10 +55,9 @@ nc = NATS() await nc.connect( servers=["nats://demo.nats.io:4222"], - # Set maximum number of PINGs out without getting a PONG back - # before the connection will be disconnected as a stale connection. + # Set Ping Interval to 20 seconds + ping_interval=20, max_outstanding_pings=5, - ping_interval=1, ) # Do something with the connection. @@ -153,7 +68,7 @@ await nc.connect( ```ruby require 'nats/client' -NATS.start(max_outstanding_pings: 5) do |nc| +NATS.start(ping_interval: 20, max_outstanding_pings: 5) do |nc| nc.on_reconnect do puts "Got reconnected to #{nc.connected_server}" end @@ -171,6 +86,7 @@ end ```typescript // will throw an exception if connection fails let nc = await connect({ + pingInterval: 20*1000, //20s maxPingOut: 5, url: "nats://demo.nats.io:4222" }); @@ -178,4 +94,3 @@ nc.close(); ``` {% endtab %} {% endtabs %} - diff --git a/developing-with-nats/connecting/specific_server.md b/developing-with-nats/connecting/specific_server.md index 5d49cda..ee6ea2b 100644 --- a/developing-with-nats/connecting/specific_server.md +++ b/developing-with-nats/connecting/specific_server.md @@ -2,7 +2,7 @@ The NATS client libraries can take a full URL, `nats://demo.nats.io:4222`, to specify a specific server host and port to connect to. -Libraries are removing the requirement for an explicit protocol and may allow `nats://demo.nats.io:4222` or just `demo.nats.io:4222`. Check with your specific client library's documentation to see what URL formats are supported. +Libraries are removing the requirement for an explicit protocol and may allow `demo.nats.io:4222` or just `demo.nats.io`. In the later example the default port 4222 will be used. Check with your specific client library's documentation to see what URL formats are supported. For example, to connect to the demo server with a URL you can use: diff --git a/developing-with-nats/events/events.md b/developing-with-nats/events/events.md index 0ffc499..7de5352 100644 --- a/developing-with-nats/events/events.md +++ b/developing-with-nats/events/events.md @@ -11,6 +11,20 @@ Connection events may include the connection being closed, disconnected or recon ```go // There is not a single listener for connection events in the NATS Go Client. // Instead, you can set individual event handlers using: +nc, err := nats.Connect("demo.nats.io", + nats.DisconnectErrHandler(func(_ *nats.Conn, err error) { + log.Printf("client disconnected: %v", err) + }), + nats.ReconnectHandler(func(_ *nats.Conn) { + log.Printf("client reconnected") + }), + nats.ClosedHandler(func(_ *nats.Conn) { + log.Printf("client closed") + })) +if err != nil { + log.Fatal(err) +} +defer nc.Close() DisconnectHandler(cb ConnHandler) ReconnectHandler(cb ConnHandler) @@ -114,7 +128,7 @@ await nc.connect(**options) {% tab title="Ruby" %} ```ruby -r# There is not a single listener for connection events in the Ruby NATS Client. +# There is not a single listener for connection events in the Ruby NATS Client. # Instead, you can set individual event handlers using: NATS.on_disconnect do @@ -229,7 +243,7 @@ nc.on('serversDiscovered', (urls) => { {% tab title="Ruby" %} ```ruby -r# The Ruby NATS client does not support discovered servers handler right now +# The Ruby NATS client does not support discovered servers handler right now ``` {% endtab %} @@ -354,4 +368,3 @@ nc.on('error', (err) => { ``` {% endtab %} {% endtabs %} - diff --git a/developing-with-nats/events/slow.md b/developing-with-nats/events/slow.md index 2c87d48..8cd8e6e 100644 --- a/developing-with-nats/events/slow.md +++ b/developing-with-nats/events/slow.md @@ -1,8 +1,8 @@ # Slow Consumers -NATS is designed to move messages through the server quickly. As a result, NATS depends on the applications to consider and respond to changing message rates. The server will do a bit of impedance matching, but if a client is too slow the server will eventually cut them off. These cut off connections are called _slow consumers_. +NATS is designed to move messages through the server quickly. As a result, NATS depends on the applications to consider and respond to changing message rates. The server will do a bit of impedance matching, but if a client is too slow the server will eventually cut them off by closing the connection. These cut off connections are called _slow consumers_. -One way some of the libraries deal with bursty message traffic is to cache incoming messages for a subscription. So if an application can handle 10 messages per second and sometimes receives 20 messages per second, the library may hold the extra 10 to give the application time to catch up. To the server, the application will appear to be handling the messages and consider the connection healthy. It is up to the client library to decide what to do when the cache is too big, but most client libraries will drop incoming messages. +One way some of the libraries deal with bursty message traffic is to buffer incoming messages for a subscription. So if an application can handle 10 messages per second and sometimes receives 20 messages per second, the library may hold the extra 10 to give the application time to catch up. To the server, the application will appear to be handling the messages and consider the connection healthy. Most client libraries will notify the application that there is a SlowConsumer error and discard messages. Receiving and dropping messages from the server keeps the connection to the server healthy, but creates an application requirement. There are several common patterns: @@ -101,7 +101,7 @@ await nc.subscribe("updates", cb=cb, pending_bytes_limit=5*1024*1024, pending_ms {% tab title="Ruby" %} ```ruby -# The Ruby NATS client currently does not have option to customize slow consumer limits per sub. +# The Ruby NATS client currently does not have option to specify a subscribers pending limits. ``` {% endtab %} @@ -200,7 +200,7 @@ public class SlowConsumerListener { if len(msgs) == 3: # Head of line blocking on other messages caused - # by single message proccesing taking long... + # by single message processing taking too long... await asyncio.sleep(1) await nc.subscribe("updates", cb=cb, pending_msgs_limit=5) diff --git a/developing-with-nats/receiving/README.md b/developing-with-nats/receiving/README.md index 9582fb5..5bad6b4 100644 --- a/developing-with-nats/receiving/README.md +++ b/developing-with-nats/receiving/README.md @@ -4,7 +4,6 @@ In general, applications can receive messages asynchronously or synchronously. R Some languages, like Go or Java, provide synchronous and asynchronous APIs, while others may only support one type of subscription. -In all cases, the process of subscribing involves having the client library tell the NATS system that an application is interested in a particular subject. - -Under the covers, the client library will assign a unique id to each subscription. This id is used as a closure when the server sends messages to a specific subscription. Each subscription gets a unique id, so if the same connection is used multiple times for the same subject, the server will send multiple copies of the same message. When an application is done with a subscription it unsubscribes which tells the server to stop sending messages. +In all cases, the process of subscribing involves having the client library tell the NATS system that an application is interested in a particular subject. When an application is done with a subscription it unsubscribes telling the server to stop sending messages. +A client will receive a message for each matching subscription ,so if a connection has multiple subscriptions using identical or overlapping subjects (say `foo` and `>`) the same message will be sent to the client multiple times. diff --git a/developing-with-nats/receiving/queues.md b/developing-with-nats/receiving/queues.md index 2f0aa29..774105e 100644 --- a/developing-with-nats/receiving/queues.md +++ b/developing-with-nats/receiving/queues.md @@ -1,8 +1,8 @@ # Queue Subscriptions -Subscribing to a queue group is only slightly different than subscribing to a subject alone. The application simply includes a queue name with the subscription. The effect of including the group is fairly major, since the server will now load balance messages between the members of the queue group, but the code differences are minimal. +Subscribing to a [queue group](../../nats-concepts/queue.md) is only slightly different than subscribing to a subject alone. The application simply includes a queue name with the subscription. The server will load balance between all members of the queue group. In a cluster setup, every member has the same chance of receiving a particular message. -Keep in mind that the queue groups in NATS are dynamic and do not require any server configuration. You can almost think of a regular subscription as a queue group of 1, but it is probably not worth thinking too much about that. +Keep in mind that queue groups in NATS are dynamic and do not require any server configuration. ![](../../.gitbook/assets/queues.svg) @@ -122,37 +122,3 @@ await nc.subscribe('updates', (err, msg) => { {% endtabs %} If you run this example with the publish examples that send to `updates`, you will see that one of the instances gets a message while the others you run won't. But the instance that receives the message will change. - -## Queue Permissions - -Added in NATS Server v2.1.2, Queue Permissions allow you to express authorization for queue groups. As queue groups are integral to implementing horizontally scalable microservices, control of who is allowed to join a specific queue group is important to the overall security model. - -A Queue Permission can be defined with the syntax ` `, where the name of the queue can also use wildcards, for example the following would allow clients to join queue groups v1 and v2.\*, but won't allow plain subscriptions: - -```text -allow = ["foo v1", "foo v2.*"] -``` - -The full wildcard can also be used, for example the following would prevent plain subscriptions on `bar` but allow the client to join any queue: - -```text -allow = ["bar >"] -``` - -Permissions for Queue Subscriptions can be combined with plain subscriptions as well though, for example you could allow plain subscriptions on `foo` but constrain the queues to which a client can join, as well as preventing any service from using a queue subscription with the name `*.prod`: - -```text -users = [ - { - user: "foo", permissions: { - sub: { - # Allow plain subscription foo, but only v1 groups or *.dev queue groups - allow: ["foo", "foo v1", "foo v1.>", "foo *.dev"] - - # Prevent queue subscriptions on prod groups - deny: ["> *.prod"] - } - } -] -``` - diff --git a/developing-with-nats/receiving/unsub_after.md b/developing-with-nats/receiving/unsub_after.md index 7140280..c5d7e58 100644 --- a/developing-with-nats/receiving/unsub_after.md +++ b/developing-with-nats/receiving/unsub_after.md @@ -2,13 +2,9 @@ NATS provides a special form of unsubscribe that is configured with a message count and takes effect when that many messages are sent to a subscriber. This mechanism is very useful if only a single message is expected. -The message count you provide is the total message count for a subscriber. So if you unsubscribe with a count of 1, the server will stop sending messages to that subscription after it has received one message. If the subscriber has already received one or more messages, the unsubscribe will be immediate. This action based on history can be confusing if you try to auto unsubscribe on a long running subscription, but is logical for a new one. +The message count you provide is the total message count for a subscriber. So if you unsubscribe with a count of 1, the server will stop sending messages to that subscription after it has received one message. If the subscriber has already received one or more messages, the unsubscribe will be immediate. This action based on history can be confusing if you try to auto-unsubscribe on a long running subscription, but is logical for a new one. -> Auto unsubscribe is based on the total messages sent to a subscriber, not just the new ones. - -Auto unsubscribe can also result in some tricky edge cases if a server cluster is used. The client will tell the server of the unsubscribe count when the application requests it. But if the client disconnects before the count is reached, it may have to tell another server of the remaining count. This dance between previous server notifications and new notifications on reconnect can result in unplanned behavior. - -Finally, most of the client libraries also track the max message count after an auto unsubscribe request. Which means that the client will stop allowing messages to flow even if the server has miscounted due to reconnects or some other failure in the client library. +Auto-unsubscribe is based on the total messages sent to a subscriber, not just the new ones. Most of the client libraries also track the max message count after an auto-unsubscribe request. On reconnect, this enables clients to resend the unsubscribe with an updated total. The following example shows unsubscribe after a single message: @@ -67,7 +63,7 @@ nc.close(); let nc = NATS.connect({ url: "nats://demo.nats.io:4222" }); -// `max` specifies the number of messages that the server will forward. +// `max` specifies the number of messages that the server will forward // The server will auto-cancel. let opts = {max: 10}; let sub = nc.subscribe(NATS.createInbox(), opts, (msg) => { @@ -131,8 +127,8 @@ end {% tab title="TypeScript" %} ```typescript -// `max` specifies the number of messages that the server will forward. -// The server will auto-cancel. +// `max` specifies the number of messages that the server will forward +// The server will auto-cancel let opts = {max: 10}; let sub = await nc.subscribe(createInbox(), (err, msg) => { t.log(msg.data); diff --git a/developing-with-nats/receiving/wildcards.md b/developing-with-nats/receiving/wildcards.md index 013855a..6bd18ca 100644 --- a/developing-with-nats/receiving/wildcards.md +++ b/developing-with-nats/receiving/wildcards.md @@ -1,8 +1,6 @@ # Wildcard Subscriptions -There is no special code to subscribe with a wildcard subject. Wildcards are a normal part of the subject name. - -However, there is a common technique that may come in to play when you use wildcards. This technique is to use the subject provided with the incoming message to determine what to do with the message. +There is no special code to subscribe with a [wildcard subject](../../nats-concepts/subjects.md#wildcards). Wildcards are a normal part of the subject name. However, it is a common technique to use the subject provided with the incoming message to determine what to do with the message. For example, you can subscribe using `*` and then act based on the actual subject. diff --git a/developing-with-nats/reconnect/README.md b/developing-with-nats/reconnect/README.md index de13b62..98227c2 100644 --- a/developing-with-nats/reconnect/README.md +++ b/developing-with-nats/reconnect/README.md @@ -2,7 +2,4 @@ Most, if not all, of the client libraries will reconnect to the NATS system if they are disconnected for any reason. The reconnect logic can differ by library, so check your client library's documentation. -In general, the client will try to connect to all of the servers it knows about, either through the URLs provided in the `connect` call or the URLs provided by the NATS system itself. The NATS system will inform clients of new endpoints that can be used to reconnect. The library may have several options to help control reconnect behavior. - -The list of servers used during reconnect is library dependent, but generally is constructed from connect function/options and the list of servers provided by the NATS system itself. This feature allows NATS applications and the NATS system itself to self heal and reconfigure itself with no additional configuration or intervention. - +In general, the client will try to re-connect to one of the servers it knows about, either through the URLs provided in the `connect` call or the URLs provided by the NATS system during earlier connects. This feature allows NATS applications and the NATS system itself to self heal and reconfigure itself with no additional configuration or intervention. The library may have several options to help control reconnect behavior, to notify about reconnect state and to inform about a new server. diff --git a/developing-with-nats/reconnect/buffer.md b/developing-with-nats/reconnect/buffer.md index 91c3244..8cbb616 100644 --- a/developing-with-nats/reconnect/buffer.md +++ b/developing-with-nats/reconnect/buffer.md @@ -2,9 +2,9 @@ The NATS client libraries try as much as possible to be fire and forget. One of the features that may be included in the library you are using is the ability to buffer outgoing messages when the connection is down. -During a short reconnect, these client can allow applications to publish messages that, because the server is offline, will be cached in the client. The library will then send those messages on reconnect. When the maximum reconnect buffer is reached, messages will no longer be publishable by the client. +During a short reconnect, the client can allow applications to publish messages that, because the server is offline, will be cached in the client. The library will then send those messages once reconnected. When the maximum reconnect buffer is reached, messages will no longer be publishable by the client and an error will be returned. -Be aware, while the message appears to be sent to the application it is possible that it is never sent because the connection is never remade. Your applications should use patterns like acknowledgements to ensure delivery. +Be aware, while the message appears to be sent to the application it is possible that it is never sent because the connection is never remade. Your applications should use patterns like [acknowledgements](../../nats-concepts/acks.md) to ensure delivery. For clients that support this feature, you are able to configure the size of this buffer with bytes, messages or both. diff --git a/developing-with-nats/reconnect/max.md b/developing-with-nats/reconnect/max.md index 308dca5..6b81001 100644 --- a/developing-with-nats/reconnect/max.md +++ b/developing-with-nats/reconnect/max.md @@ -1,6 +1,6 @@ # Set the Number of Reconnect Attempts -Applications can set the maximum reconnect attempts. Generally, this will limit the actual number of attempts total, but check your library documentation. For example, in Java, if the client knows about 3 servers and the maximum reconnects is set to 2, it will not try all of the servers. On the other hand, if the maximum is set to 6 it will try all of the servers twice before considering the reconnect a failure and closing. +Applications can set the maximum reconnect attempts per server. This includes the server provided to the clients connect call, as well as the server the client discovered through another server. Once re-connect to a server fails the specified amount of times in a row, it will be removed from the connect list. After a successful re-connect to a server, the client will reset that servers failed reconnect attempt count. If a server was removed from the connect list, it can be re-discovered on connect. This effectively resets the connect attempt count as well. If the client runs out of servers to re-connect, it will close the connection and [raise an error](events.md). {% tabs %} {% tab title="Go" %} @@ -71,7 +71,6 @@ end // will throw an exception if connection fails let nc = await connect({ maxReconnectAttempts: 10, - servers: ["nats://demo.nats.io:4222"] }); nc.close(); ``` diff --git a/developing-with-nats/reconnect/random.md b/developing-with-nats/reconnect/random.md index f37ce1b..7d07260 100644 --- a/developing-with-nats/reconnect/random.md +++ b/developing-with-nats/reconnect/random.md @@ -2,7 +2,7 @@ When a server goes down, there is a possible anti-pattern called the _Thundering Herd_ where all of the clients try to reconnect immediately, thus creating a denial of service attack. In order to prevent this, most NATS client libraries randomize the servers they attempt to connect to. This setting has no effect if only a single server is used, but in the case of a cluster, randomization, or shuffling, will ensure that no one server bears the brunt of the client reconnect attempts. -However, if you want to disable the randomization process, so that servers are always checked in the same order, you can do that in most libraries with a connection options: +However, if you want to disable the randomization process for connect and re-connect, so that servers are always checked in the same order, you can do that in most libraries with a connection option: {% tabs %} {% tab title="Go" %} @@ -39,7 +39,7 @@ nc.close(); {% tab title="JavaScript" %} ```javascript let nc = NATS.connect({ - noRandomize: false, + noRandomize: true, servers: ["nats://127.0.0.1:4443", "nats://demo.nats.io:4222" ] @@ -82,7 +82,7 @@ end ```typescript // will throw an exception if connection fails let nc = await connect({ - noRandomize: false, + noRandomize: true, servers: ["nats://127.0.0.1:4443", "nats://demo.nats.io:4222" ] diff --git a/developing-with-nats/reconnect/wait.md b/developing-with-nats/reconnect/wait.md index 3af9a23..50d84dd 100644 --- a/developing-with-nats/reconnect/wait.md +++ b/developing-with-nats/reconnect/wait.md @@ -1,6 +1,8 @@ # Pausing Between Reconnect Attempts -It doesn’t make much sense to try to connect to the same server over and over. To prevent this sort of thrashing, and wasted reconnect attempts, libraries provide a wait setting. This setting will pause the reconnect logic if the same server is being tried multiple times **in a row**. In the previous example, if you have 3 servers and 6 attempts, the Java library would loop over the three servers. If none were connectable, it will then try all three again. However, the Java client doesn’t wait between each attempt, only when trying the same server again, so in that example the library may never wait. If on the other hand, you only provide a single server URL and 6 attempts, the library will wait between each attempt. +It doesn’t make much sense to try to connect to the same server over and over. To prevent this sort of thrashing, and wasted reconnect attempts, especially when using TLS, libraries provide a wait setting. Generally clients make sure that between two reconnect attempts to the **same** server at least a certain amount of time has passed. The concrete implementation depends on the library used. + +This setting not only prevents wasting client resources, it also alleviates a [_thundering herd_](random.md) situation when additional servers are not available. {% tabs %} {% tab title="Go" %} @@ -33,7 +35,6 @@ nc.close(); {% tab title="JavaScript" %} ```javascript let nc = NATS.connect({ - reconnectTimeWait: 10 * 1000, //10s servers: ["nats://demo.nats.io:4222"] }); ``` @@ -70,7 +71,6 @@ end ```typescript // will throw an exception if connection fails let nc = await connect({ - reconnectTimeWait: 10*1000, //10s servers: ["nats://demo.nats.io:4222"] }); nc.close(); diff --git a/developing-with-nats/security/creds.md b/developing-with-nats/security/creds.md index e1ecd27..0c56d28 100644 --- a/developing-with-nats/security/creds.md +++ b/developing-with-nats/security/creds.md @@ -1,6 +1,6 @@ # Authenticating with a Credentials File -The 2.0 version of NATS server introduced the idea of JWT-based authentication. Clients interact with this new scheme using a user JWT and the private key from an NKey pair. To help make connecting with a JWT easier, the client libraries support the concept of a credentials file. This file contains both the private key and the JWT and can be generated with the `nsc` tool. The contents will look like the following and should be protected because it contains a private key. This creds file is unused and only for example purposes. +The 2.0 version of NATS server introduced the idea of decentralized authentication based on [JSON Web Tokens \(JWT\)](https://jwt.io/). Clients interact with this new scheme using a [user JWT](../../nats-server/configuration/securing_nats/auth_intro/jwt_auth.md) and corresponding [NKey](../../nats-server/configuration/securing_nats/auth_intro/nkey_auth.md) private key. To help make connecting with a JWT easier, the client libraries support the concept of a credentials file. This file contains both the private key and the JWT and can be generated with the `nsc` [tool](../../nats-tools/nsc/README.md). The contents will look like the following and should be protected because it contains a private key. This credentials file is unused and only for example purposes. ```text -----BEGIN NATS USER JWT----- diff --git a/developing-with-nats/security/nkey.md b/developing-with-nats/security/nkey.md index c5c87f6..ac8393c 100644 --- a/developing-with-nats/security/nkey.md +++ b/developing-with-nats/security/nkey.md @@ -1,6 +1,6 @@ # Authenticating with an NKey -The 2.0 version of NATS server introduces a new challenge response authentication option. This challenge response is based on a wrapper we call NKeys which uses [Ed25519](https://ed25519.cr.yp.to/) signing. The server can use these keys in several ways for authentication. The simplest is for the server to be configured with a list of known public keys and for the clients to respond to the challenge by signing it with its private key. This challenge-response ensures security by ensuring that the client has the private key, but also protects the private key from the server which never has to actually see it. +The 2.0 version of NATS server introduces a new challenge response authentication option. This challenge response is based on a wrapper we call [NKeys](../../nats-server/configuration/securing_nats/auth_intro/nkey_auth.md). The server can use these keys in several ways for authentication. The simplest is for the server to be configured with a list of known public keys and for the clients to respond to the challenge by signing it with its private key. (A printable private NKey is referred to as seed). This challenge-response ensures security by ensuring that the client has the private key, but also protects the private key from the server, which never has access to it! Handling challenge response may require more than just a setting in the connection options, depending on the client library. diff --git a/developing-with-nats/security/tls.md b/developing-with-nats/security/tls.md index 753962c..4226e56 100644 --- a/developing-with-nats/security/tls.md +++ b/developing-with-nats/security/tls.md @@ -1,19 +1,17 @@ -# Encrypting Connections with TLS +# Encrypting and Authenticating Connections with TLS -While authentication limits which clients can connect, TLS can be used to check the server’s identity and optionally the client’s identity and will encrypt all traffic between the two. The most secure version of TLS with NATS is to use verified client certificates. In this mode, the client can check that it trusts the certificate sent by NATS system but the individual server will also check that it trusts the certificate sent by the client. From an application's perspective connecting to a server that does not verify client certificates may appear identical. Under the covers, disabling TLS verification removes the server side check on the client’s certificate. When started in TLS mode, a `nats-server` will require all clients to connect with TLS. Moreover, if configured to connect with TLS, client libraries will fail to connect to a server without TLS. +While authentication limits which clients can connect, TLS can be used to encrypt traffic between client/server and check the server’s identity. Additionally - in the most secure version of TLS with NATS - the server can be configured to verify the client's identity, thus authenticating it. When started in [TLS mode](../../nats-server/configuration/securing_nats/tls.md), a `nats-server` will require all clients to connect with TLS. Moreover, if configured to connect with TLS, client libraries will fail to connect to a server without TLS. -The [Java examples repository](https://github.com/nats-io/java-nats-examples/tree/master/src/main/resources) contains certificates for starting the server in TLS mode. +## Connecting with TLS and verify client identity + +Using TLS to connect to a server that verifies the client's identity is straightforward. The client has to provide a certificate and private key. The NATS client will use these to prove it's identity to the server. For the client to verify the server's identity, the CA certificate is provided as well. + +The [Java examples repository](https://github.com/nats-io/java-nats-examples/tree/master/src/main/resources) contains certificates and a server config for this example. ```bash -> nats-server -c /src/main/resources/tls.conf - or > nats-server -c /src/main/resources/tls_verify.conf ``` -## Connecting with TLS - -Connecting to a server with TLS is straightforward. Most clients will automatically use TLS when connected to a NATS system using TLS. Setting up a NATS system to use TLS is primarily an exercise in setting up the certificate and trust managers. Clients may also need additional information, for example: - {% tabs %} {% tab title="Go" %} ```go @@ -199,186 +197,4 @@ let nc = await connect({ ## Connecting with the TLS Protocol -Some clients may support the `tls` protocol as well as a manual setting to turn on TLS. However, in that case there is likely some form of default or environmental settings to allow the TLS libraries to find certificate and trust stores. - -{% tabs %} -{% tab title="Go" %} -```go -nc, err := nats.Connect("tls://localhost", nats.RootCAs("resources/certs/ca.pem")) // May need this if server is using self-signed certificate -if err != nil { - log.Fatal(err) -} -defer nc.Close() - -// Do something with the connection -``` -{% endtab %} - -{% tab title="Java" %} -```java -class SSLUtils { - public static String KEYSTORE_PATH = "src/main/resources/keystore.jks"; - public static String TRUSTSTORE_PATH = "src/main/resources/cacerts"; - public static String STORE_PASSWORD = "password"; - public static String KEY_PASSWORD = "password"; - public static String ALGORITHM = "SunX509"; - - public static KeyStore loadKeystore(String path) throws Exception { - KeyStore store = KeyStore.getInstance("JKS"); - BufferedInputStream in = new BufferedInputStream(new FileInputStream(path)); - - try { - store.load(in, STORE_PASSWORD.toCharArray()); - } finally { - if (in != null) { - in.close(); - } - } - - return store; - } - - public static KeyManager[] createTestKeyManagers() throws Exception { - KeyStore store = loadKeystore(KEYSTORE_PATH); - KeyManagerFactory factory = KeyManagerFactory.getInstance(ALGORITHM); - factory.init(store, KEY_PASSWORD.toCharArray()); - return factory.getKeyManagers(); - } - - public static TrustManager[] createTestTrustManagers() throws Exception { - KeyStore store = loadKeystore(TRUSTSTORE_PATH); - TrustManagerFactory factory = TrustManagerFactory.getInstance(ALGORITHM); - factory.init(store); - return factory.getTrustManagers(); - } - - public static SSLContext createSSLContext() throws Exception { - SSLContext ctx = SSLContext.getInstance(Options.DEFAULT_SSL_PROTOCOL); - ctx.init(createTestKeyManagers(), createTestTrustManagers(), new SecureRandom()); - return ctx; - } -} - -public class ConnectTLS { - public static void main(String[] args) { - - try { - SSLContext ctx = SSLUtils.createSSLContext(); - Options options = new Options.Builder(). - server("nats://localhost:4222"). - sslContext(ctx). // Set the SSL context - build(); - Connection nc = Nats.connect(options); - - // Do something with the connection - - nc.close(); - } catch (Exception e) { - e.printStackTrace(); - } - } -} -``` -{% endtab %} - -{% tab title="JavaScript" %} -```javascript -let caCert = fs.readFileSync(caCertPath); -let clientCert = fs.readFileSync(clientCertPath); -let clientKey = fs.readFileSync(clientKeyPath); -let nc = NATS.connect({ - url: url, - tls: { - ca: [caCert], - key: [clientKey], - cert: [clientCert] - } -}); -``` -{% endtab %} - -{% tab title="Python" %} -```python -nc = NATS() - -ssl_ctx = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH) -ssl_ctx.load_verify_locations('ca.pem') -ssl_ctx.load_cert_chain(certfile='client-cert.pem', - keyfile='client-key.pem') -await nc.connect(io_loop=loop, tls=ssl_ctx) - -await nc.connect(servers=["nats://demo.nats.io:4222"], tls=ssl_ctx) - -# Do something with the connection. -``` -{% endtab %} - -{% tab title="Ruby" %} -```ruby -EM.run do - - options = { - :servers => [ - 'nats://localhost:4222', - ], - :tls => { - :private_key_file => './spec/configs/certs/key.pem', - :cert_chain_file => './spec/configs/certs/server.pem' - } - } - - NATS.connect(options) do |nc| - puts "#{Time.now.to_f} - Connected to NATS at #{nc.connected_server}" - - nc.subscribe("hello") do |msg| - puts "#{Time.now.to_f} - Received: #{msg}" - end - - nc.flush do - nc.publish("hello", "world") - end - - EM.add_periodic_timer(0.1) do - next unless nc.connected? - nc.publish("hello", "hello") - end - - # Set default callbacks - nc.on_error do |e| - puts "#{Time.now.to_f } - Error: #{e}" - end - - nc.on_disconnect do |reason| - puts "#{Time.now.to_f} - Disconnected: #{reason}" - end - - nc.on_reconnect do |nc| - puts "#{Time.now.to_f} - Reconnected to NATS server at #{nc.connected_server}" - end - - nc.on_close do - puts "#{Time.now.to_f} - Connection to NATS closed" - EM.stop - end - end -end -``` -{% endtab %} - -{% tab title="TypeScript" %} -```typescript -let caCert = readFileSync(caCertPath); -let clientCert = readFileSync(clientCertPath); -let clientKey = readFileSync(clientKeyPath); -let nc = await connect({ - url: url, - tls: { - ca: [caCert], - key: [clientKey], - cert: [clientCert] - } -}); -``` -{% endtab %} -{% endtabs %} - +Clients (such as Go, Java, Javascript, Ruby and Type Script) support providing a URL containing the `tls` protocol to the NATS connect call. This will turn on TLS without the need for further code changes. However, in that case there is likely some form of default or environmental settings to allow the TLS libraries of your programming language to find certificate and trusted CAs. Unless these settings are taken into accounts or otherwise modified, this way of connecting is very likely to fail. diff --git a/developing-with-nats/security/token.md b/developing-with-nats/security/token.md index be40208..f7fe283 100644 --- a/developing-with-nats/security/token.md +++ b/developing-with-nats/security/token.md @@ -1,6 +1,6 @@ # Authenticating with a Token -Tokens are basically random strings, much like a password, and can provide a simple authentication mechanism in some situations. However, tokens are only as safe as they are secret so other authentication schemes can provide more security in large installations. +Tokens are basically random strings, much like a password, and can provide a simple authentication mechanism in some situations. However, tokens are only as safe as they are secret so other authentication schemes can provide more security in large installations. It is highly recommended to use one of the other NATS authentication mechanisms. For this example, start the server using: @@ -50,7 +50,7 @@ let nc = NATS.connect({url: `nats://127.0.0.1:${port}`, token: "mytoken!"}); ```python nc = NATS() -await nc.connect(servers=["nats://mytoken@demo.nats.io:4222"]) +await nc.connect(servers=["nats://demo.nats.io:4222"], token="mytoken") # Do something with the connection. ``` @@ -58,7 +58,7 @@ await nc.connect(servers=["nats://mytoken@demo.nats.io:4222"]) {% tab title="Ruby" %} ```ruby -NATS.start(token: "deadbeef") do |nc| +NATS.start(token: "mytoken") do |nc| puts "Connected using token" end ``` @@ -105,7 +105,7 @@ nc.close(); {% tab title="JavaScript" %} ```javascript -let url = `nats://mytoken!@127.0.0.1:${port}`; +let url = `nats://mytoken@127.0.0.1:${port}`; let nc = NATS.connect({url: url}); ``` {% endtab %} @@ -122,7 +122,7 @@ await nc.connect(servers=["nats://mytoken@demo.nats.io:4222"]) {% tab title="Ruby" %} ```ruby -NATS.start("deadbeef@127.0.0.1:4222") do |nc| +NATS.start("mytoken@127.0.0.1:4222") do |nc| puts "Connected using token!" end ``` @@ -130,7 +130,7 @@ end {% tab title="TypeScript" %} ```typescript -let url = `nats://:mytoken!@127.0.0.1:${port}`; +let url = `nats://:mytoken@127.0.0.1:${port}`; let nc = await connect({url: url}); ``` {% endtab %} diff --git a/developing-with-nats/security/userpass.md b/developing-with-nats/security/userpass.md index 2f661b5..755a043 100644 --- a/developing-with-nats/security/userpass.md +++ b/developing-with-nats/security/userpass.md @@ -6,7 +6,7 @@ For this example, start the server using: > nats-server --user myname --pass password ``` -You can encrypt passwords to pass to `nats-server` using a simple tool provided by the server: +You can encrypt passwords to pass to `nats-server` using a simple [tool](../../nats-tools/mkpasswd.md) provided by the server: ```bash > go run mkpasswd.go -p diff --git a/developing-with-nats/sending/caches.md b/developing-with-nats/sending/caches.md index 1907590..dca1ab5 100644 --- a/developing-with-nats/sending/caches.md +++ b/developing-with-nats/sending/caches.md @@ -1,6 +1,6 @@ -# Caches, Flush and Ping +# Buffers and Flush -For performance reasons, most if not all, of the client libraries will cache outgoing data so that bigger chunks can be written to the network at one time. This may be as simple as a byte buffer that stores up a few messages before being pushed to the network. +For performance reasons, most if not all, of the client libraries will buffer outgoing data so that bigger chunks can be written to the network at one time. This may be as simple as a byte buffer that stores a few messages before being pushed to the network. These buffers do not hold messages forever, generally they are designed to hold messages in high throughput scenarios, while still providing good latency in low throughput situations. @@ -113,5 +113,6 @@ nc.close(); ## Flush and Ping/Pong -Many of the client libraries use the PING/PONG interaction built into the NATS protocol to insure that flush pushed all of the cached messages to the server. When an application calls flush most libraries will put a PING on the outgoing queue of messages, and wait for the server to send PONG before saying that the flush was successful. +Many of the client libraries use the [PING/PONG interaction](../connecting/pingpong.md) built into the NATS protocol to ensure that flush pushed all of the buffered messages to the server. When an application calls flush, most libraries will put a PING on the outgoing queue of messages, and wait for the server to respond with a PONG before saying that the flush was successful. +Even though the client may use PING/PONG for flush, pings sent this way do not count towards [max outgoing pings](../connecting/pingpong.md). diff --git a/developing-with-nats/tutorials/pubsub.md b/developing-with-nats/tutorials/pubsub.md index 9bd6855..e1d8d9e 100644 --- a/developing-with-nats/tutorials/pubsub.md +++ b/developing-with-nats/tutorials/pubsub.md @@ -1,6 +1,6 @@ # Explore NATS Pub/Sub -NATS is a publish subscribe messaging system. Subscribers listening on a subject receive messages on that subject. If the subscriber is not actively listening on the subject, the message is not received. Subscribers can use the wildcard tokens such as `*` and `>` to match a single token or to match the tail of a subject. +NATS is a [publish subscribe](../../nats-concepts/pubsub.md) messaging system [based on subjects](../../nats-concepts/subjects.md). Subscribers listening on a subject receive messages published on that subject. If the subscriber is not actively listening on the subject, the message is not received. Subscribers can use the wildcard tokens such as `*` and `>` to match a single token or to match the tail of a subject. ![](../../.gitbook/assets/pubsubtut.svg) @@ -82,9 +82,9 @@ or ### 8. Verify message publication and receipt -You should see that the publisher sends the message: _Published \[msg.test\] : 'NATS MESSAGE'_ +You should see that the publisher sends the message and prints: _Published \[msg.test\] : 'NATS MESSAGE'_ -And that the subscriber receives the message: _\[\#1\] Received on \[msg.test\]: 'NATS MESSAGE'_ +And that the subscriber receives the message and prints: _\[\#1\] Received on \[msg.test\]: 'NATS MESSAGE'_ Note that if the receiver does not get the message, check that you are using the same subject name for the publisher and the subscriber. @@ -94,7 +94,7 @@ Note that if the receiver does not get the message, check that you are using the % go run nats-pub/main.go msg.test "NATS MESSAGE 2" ``` -You should see that the subscriber receive message 2. Note that the message count is incremented each time your subscribing client receives a message on that subject: +You should see that the subscriber receives message 2. Note that the message count is incremented each time your subscribing client receives a message on that subject: ### 10. Start another shell or command prompt session @@ -106,7 +106,7 @@ You will use this session to run a second NATS subscriber. % cd $GOPATH/src/github.com/nats-io/nats.go/examples ``` -### 12. Subscribe to the message +### 12. Start a second client subscriber program ```bash % go run nats-sub/main.go msg.test @@ -130,7 +130,7 @@ You will use this session to run a third NATS subscriber. % cd $GOPATH/src/github.com/nats-io/nats.go/examples ``` -### 16. Subscribe to a different message +### 16. Subscribe to a different subject ```bash % go run nats-sub/main.go msg.test.new @@ -140,7 +140,7 @@ All the but last subscriber receives the message. Why? Because that subscriber i ### 17. Update the last subscriber to use a wildcard -NATS supports the use of wildcard characters for message subscribers. You cannot publish a message using a wildcard subject. +NATS supports the use of wildcard characters for message subscribers only. You cannot publish a message using a wildcard subject. Change the last subscriber the listen on msg.\* and run it: diff --git a/developing-with-nats/tutorials/queues.md b/developing-with-nats/tutorials/queues.md index 6a3170b..391d4b0 100644 --- a/developing-with-nats/tutorials/queues.md +++ b/developing-with-nats/tutorials/queues.md @@ -1,10 +1,10 @@ # Explore NATS Queueing -NATS supports a form of load balancing using queue groups. Subscribers register a queue group name. A single subscriber in the group is randomly selected to receive the message. +NATS supports a form of load balancing using [queue groups](../../nats-concepts/queue.md). Subscribers register a queue group name. A single subscriber in the group is randomly selected to receive the message. ## Prerequisites -Go and the NATS server should be installed. +Go, node.js, ruby and the NATS server should be installed. ### 1. Start the NATS server @@ -20,7 +20,7 @@ git clone https://github.com/nats-io/nats.js.git git clone https://github.com/nats-io/nats.rb.git ``` -### 3. Run the Go client subscriber with queue group name +### 3. Run the Go client subscriber, providing a queue group name ```bash cd $GOPATH/src/github.com/nats-io/nats.go/examples @@ -58,9 +58,9 @@ go run nats-pub/main.go foo "Hello NATS!" ### 8. Verify message publication and receipt -You should see that the publisher sends the message: _Published \[foo\] : 'Hello NATS!'_ +You should see that the publisher the message and prints: _Published \[foo\] : 'Hello NATS!'_ -You should see that only one of the my-queue group subscribers receives the message. In addition, the Go client subscriber not in the my-queue group should also receive the message. +You should see that only one of the my-queue group subscribers receives the message and prints it. In addition, the Go client subscriber not in the my-queue group should also receive and print the message. ### 9. Publish another message diff --git a/developing-with-nats/tutorials/reqreply.md b/developing-with-nats/tutorials/reqreply.md index 4510f0e..285128b 100644 --- a/developing-with-nats/tutorials/reqreply.md +++ b/developing-with-nats/tutorials/reqreply.md @@ -1,6 +1,6 @@ # Explore NATS Request/Reply -NATS supports request/reply messaging. In this tutorial you explore how to exchange point-to-point messages using NATS. +NATS supports [request/reply](../../nats-concepts/reqreply.md) messaging. In this tutorial you explore how to exchange point-to-point messages using NATS. ## Prerequisites @@ -25,18 +25,19 @@ You will use these sessions to run the NATS request and reply clients. ### 4. In one terminal, run the reply client listener ```bash -% go run nats-rply/main.go foo "this is my response" +% go run nats-rply/main.go help.please "OK, I CAN HELP!!!" ``` -You should see the message `Receiver is listening`, and that the NATS receiver client is listening on the "help.please" subject. The reply client acts as a receiver, listening for message requests. In NATS, the receiver is a subscriber. +You should see the message: _Listening on \[help.please\]_ + +This means that the NATS receiver client is listening for requests messages on the "help.please" subject. In NATS, the receiver is a subscriber. ### 5. In the other terminal, run the request client ```bash -% go run nats-req/main.go foo "request payload" +% go run nats-req/main.go help.please "I need help!" ``` -The NATS requestor client makes a request by sending the message "some message" on the “help.please” subject. +The NATS requestor client makes a request by sending the message "I need help!" on the “help.please” subject. The NATS receiver client receives the message, formulates the reply \("OK, I CAN HELP!!!"\), and sends it to the inbox of the requester. - diff --git a/nats-concepts/intro.md b/nats-concepts/intro.md index dc202e9..14ac0e0 100644 --- a/nats-concepts/intro.md +++ b/nats-concepts/intro.md @@ -6,5 +6,4 @@ NATS makes it easy for programs to communicate across different environments, la ![](../.gitbook/assets/intro.svg) -NATS core offers an **at most once** quality of service. If a subscriber is not listening on the subject \(no subject match\), or is not active when the message is sent, the message is not received. This is the same level of guarantee that TCP/IP provides. By default, NATS is a fire-and-forget messaging system. If you need higher levels of service, you can use [NATS Streaming](../nats-streaming-concepts/intro.md) or build additional reliability into your client applications with proven and scalable reference designs. - +NATS core offers an **at most once** quality of service. If a subscriber is not listening on the subject \(no subject match\), or is not active when the message is sent, the message is not received. This is the same level of guarantee that TCP/IP provides. By default, NATS is a fire-and-forget messaging system. If you need higher levels of service, you can use [NATS Streaming](../nats-streaming-concepts/intro.md) or build additional reliability into your client applications with proven and scalable reference designs such as [acks](acks.md) and [sequence numbers](seq_num.md). diff --git a/nats-concepts/queue.md b/nats-concepts/queue.md index d08a2c7..6f1222f 100644 --- a/nats-concepts/queue.md +++ b/nats-concepts/queue.md @@ -4,6 +4,8 @@ NATS provides a built-in load balancing feature called distributed queues. Using To create a queue subscription, subscribers register a queue name. All subscribers with the same queue name form the queue group. This requires no configuration. As messages on the registered subject are published, one member of the group is chosen randomly to receive the message. Although queue groups have multiple subscribers, each message is consumed by only one. +Queue group names follow the same naming rules as [subjects](subjects.md). Foremost, they are case sensitive and cannot contain whitespace. Consider structuring queue groups hierarchically using `.`. Some server functionalities can use [wildcard matching](subjects.md#wildcards) on them. + One of the great features of NATS is that queue groups are defined by the application and their queue subscribers, not on the server configuration. Queue subscribers are ideal for scaling services. Scale up is as simple as running another application, scale down is terminating the application with a signal that drains the in flight requests. This flexibility and lack of any configuration changes makes NATS an excellent service communication technology that can work with all platform technologies. diff --git a/nats-concepts/reqreply.md b/nats-concepts/reqreply.md index 38b3e80..eefbbbd 100644 --- a/nats-concepts/reqreply.md +++ b/nats-concepts/reqreply.md @@ -2,7 +2,7 @@ Request-Reply is a common pattern in modern distributed systems. A request is sent and the application either waits on the response with a certain timeout or receives a response asynchronously. The increased complexity of modern systems requires features such as location transparency, scale up and scale down, observability and more. Many technologies need additional components, sidecars and proxies to accomplish the complete feature set. -NATS supports this pattern with its core communication mechanism, publish and subscribe. A request is published on a given subject with a reply subject, and responders listen on that subject and send responses to the reply subject. Reply subjects are usually a subject called an \_INBOX that will be directed back to the requestor dynamically, regardless of location of either party. +NATS supports this pattern with its core communication mechanism, publish and subscribe. A request is published on a given subject with a reply subject, and responders listen on that subject and send responses to the reply subject. Reply subjects are unique subjects called _inbox_ that are dynamically directed back to the requestor, regardless of location of either party. NATS allows multiple responders to run and form dynamic queue groups for transparent scale up. The ability for NATS applications to drain before exiting allows scale down with no requests being dropped. And since NATS is based on publish-subscribe, observability is as simple as running another application that can view requests and responses to measure latency, watch for anomalies, direct scalability and more. diff --git a/nats-concepts/subjects.md b/nats-concepts/subjects.md index 57dfbeb..2af8b1e 100644 --- a/nats-concepts/subjects.md +++ b/nats-concepts/subjects.md @@ -1,6 +1,6 @@ # Subject-Based Messaging -Fundamentally NATS is about publishing and listening for messages. Both of these depend heavily on _Subjects_ which scope messages into streams or topics. At its simplest, a subject is just a string of characters that form a name the publisher and subscriber can use to find each other. +Fundamentally, NATS is about publishing and listening for messages. Both of these depend heavily on _Subjects_ which scope messages into streams or topics. At its simplest, a subject is just a string of characters that form a name the publisher and subscriber can use to find each other. ![](../.gitbook/assets/subjects1.svg) @@ -38,3 +38,6 @@ The second wildcard is `>` which will match one or more tokens, and can only app Subject to your security configuration, wildcards can be used for monitoring by creating something sometimes called a _wire tap_. In the simplest case you can create a subscriber for `>`. This application will receive all messages -- again, subject to security settings -- sent on your NATS cluster. +### Mix Wildcards + +The wildcard `*` can appear multiple times in the same subject. Both types be used as well. For example, `*.*.east.>` will receive `time.us.east.atlanta`. \ No newline at end of file diff --git a/nats-server/configuration/securing_nats/authorization.md b/nats-server/configuration/securing_nats/authorization.md index ea5a5e7..6bc1476 100644 --- a/nats-server/configuration/securing_nats/authorization.md +++ b/nats-server/configuration/securing_nats/authorization.md @@ -2,7 +2,7 @@ The NATS server supports authorization using subject-level permissions on a per-user basis. Permission-based authorization is available with multi-user authentication via the `users` list. -Each permission specifies the subjects the user can publish to and subscribe to. The parser is generous at understanding what the intent is, so both arrays and singletons are processed. For more complex configuration, you can specify a `permission` object which explicitly allows or denies subjects. The specified subjects can specify wildcards. Permissions can make use of [variables](../#variables). +Each permission specifies the subjects the user can publish to and subscribe to. The parser is generous at understanding what the intent is, so both arrays and singletons are processed. For more complex configuration, you can specify a `permission` object which explicitly allows or denies subjects. The specified subjects can specify wildcards. Permissions can make use of [variables](../README.md#variables). You configure authorization by creating a `permissions` entry in the `authorization` object. @@ -125,4 +125,35 @@ authorization: { User `a` has no restrictions. User `b` can listen on `q` for requests and can only publish once to reply subjects. All other subjects will be denied. User `c` can also listen on `q` for requests, but is able to return at most 5 reply messages, and the reply subject can be published at most for `1` minute. +## Queue Permissions +Added in NATS Server v2.1.2, Queue Permissions allow you to express authorization for queue groups. As queue groups are integral to implementing horizontally scalable microservices, control of who is allowed to join a specific queue group is important to the overall security model. + +A Queue Permission can be defined with the syntax ` `, where the name of the queue can also use wildcards, for example the following would allow clients to join queue groups v1 and v2.\*, but won't allow plain subscriptions: + +```text +allow = ["foo v1", "foo v2.*"] +``` + +The full wildcard can also be used, for example the following would prevent plain subscriptions on `bar` but allow the client to join any queue: + +```text +allow = ["bar >"] +``` + +Permissions for Queue Subscriptions can be combined with plain subscriptions as well though, for example you could allow plain subscriptions on `foo` but constrain the queues to which a client can join, as well as preventing any service from using a queue subscription with the name `*.prod`: + +```text +users = [ + { + user: "foo", permissions: { + sub: { + # Allow plain subscription foo, but only v1 groups or *.dev queue groups + allow: ["foo", "foo v1", "foo v1.>", "foo *.dev"] + + # Prevent queue subscriptions on prod groups + deny: ["> *.prod"] + } + } +] +```