From bf9b8a8247061c6651397c29521103ba752beedd Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Mon, 3 Feb 2020 15:39:04 -0500 Subject: [PATCH 01/36] Reworded to describe behaviour of non java clients and merge examples. Specifically removed the reference to flush which is not influenced by this setting. Merged the examples as both values together make more sense. Signed-off-by: Matthias Hanel --- developing-with-nats/connecting/pingpong.md | 105 ++------------------ 1 file changed, 8 insertions(+), 97 deletions(-) diff --git a/developing-with-nats/connecting/pingpong.md b/developing-with-nats/connecting/pingpong.md index 531868c..b2f6166 100644 --- a/developing-with-nats/connecting/pingpong.md +++ b/developing-with-nats/connecting/pingpong.md @@ -1,18 +1,16 @@ # Ping/Pong Protocol -The client and server use a simple PING/PONG protocol to check that they are both still connected. The client will ping the server on a regular, configured interval so that the server usually doesn't have to initiate the PING/PONG interaction. +The client and server use a simple PING/PONG protocol to check that either of them are still connected to the other. On a regular interval he client will ping the server, which responds with a pong. Once a configurable maximum of outstanding pings without a single pong reply is hit, the connection is closed as stale. Together these two values define a timeout for the connection. In the pressence of traffic, such as messages or client side pings, the server will not initiate the PING/PONG interaction. ![](../../.gitbook/assets/pingpong.svg) -## Set the Ping Interval - -If you have a connection that is going to be open a long time with few messages traveling on it, setting this PING interval can control how quickly the client will be notified of a problem. However on connections with a lot of traffic, the client will often figure out there is a problem between PINGS, and as a result the default PING interval is often on the order of minutes. To set the interval to 20s: +If you have a connection that is going to be open a long time with few messages traveling on it, setting the PING interval and/or limit how many are outstanding, can control how quickly the client will be notified of a problem. However on connections with a lot of traffic, the client will often figure out there is a problem between PINGS, and as a result the default PING interval is often on the order of minutes. To set the interval to 20s and limit outstanding pings to 5, thus force a closed connection after 100s of inactivity: {% tabs %} {% tab title="Go" %} ```go // Set Ping Interval to 20 seconds -nc, err := nats.Connect("demo.nats.io", nats.Name("API Ping Example"), nats.PingInterval(20*time.Second)) +nc, err := nats.Connect("demo.nats.io", nats.Name("API Ping Example"), nats.PingInterval(20*time.Second), nats.MaxPingsOutstanding(5)) if err != nil { log.Fatal(err) } @@ -27,93 +25,6 @@ defer nc.Close() Options options = new Options.Builder(). server("nats://demo.nats.io:4222"). pingInterval(Duration.ofSeconds(20)). // Set Ping Interval - build(); -Connection nc = Nats.connect(options); - -// Do something with the connection - -nc.close(); -``` -{% endtab %} - -{% tab title="JavaScript" %} -```javascript -let nc = NATS.connect({ - pingInterval: 20*1000, //20s - url: "nats://demo.nats.io:4222" -}); -``` -{% endtab %} - -{% tab title="Python" %} -```python -nc = NATS() - -await nc.connect( - servers=["nats://demo.nats.io:4222"], - # Set Ping Interval to 20 seconds - ping_interval=20, - ) - -# Do something with the connection. -``` -{% endtab %} - -{% tab title="Ruby" %} -```ruby -require 'nats/client' - -NATS.start(ping_interval: 20) do |nc| - nc.on_reconnect do - puts "Got reconnected to #{nc.connected_server}" - end - - nc.on_disconnect do |reason| - puts "Got disconnected! #{reason}" - end - - # Do something with the connection -end -``` -{% endtab %} - -{% tab title="TypeScript" %} -```typescript -// will throw an exception if connection fails -let nc = await connect({ - pingInterval: 20*1000, //20s - url: "nats://demo.nats.io:4222" -}); -nc.close(); -``` -{% endtab %} -{% endtabs %} - -## Limit Outgoing Pings - -The PING/PONG interaction is also used by most of the clients as a way to flush the connection to the server. Clients that cache outgoing messages provide a flush call that will run a PING/PONG. The flush will wait for the PONG to return, telling it that all cached messages have been processed, including the PING. The number of cached PING requests can be limited in most clients to insure that traffic problems are identified early. This configuration for _max outgoing pings_ or similar will usually default to a small number and should only be increased if you are worried about fast flush traffic, perhaps in multiple threads. - -For example, to set the maximum number of outgoing pings to 5: - -{% tabs %} -{% tab title="Go" %} -```go -// Set maximum number of PINGs out without getting a PONG back - // before the connection will be disconnected as a stale connection. - nc, err := nats.Connect("demo.nats.io", nats.Name("API MaxPing Example"), nats.MaxPingsOutstanding(5)) - if err != nil { - log.Fatal(err) - } - defer nc.Close() - - // Do something with the connection -``` -{% endtab %} - -{% tab title="Java" %} -```java -Options options = new Options.Builder(). - server("nats://demo.nats.io:4222"). maxPingsOut(5). // Set max pings in flight build(); Connection nc = Nats.connect(options); @@ -127,6 +38,7 @@ nc.close(); {% tab title="JavaScript" %} ```javascript let nc = NATS.connect({ + pingInterval: 20*1000, //20s maxPingOut: 5, url: "nats://demo.nats.io:4222" }); @@ -139,10 +51,9 @@ nc = NATS() await nc.connect( servers=["nats://demo.nats.io:4222"], - # Set maximum number of PINGs out without getting a PONG back - # before the connection will be disconnected as a stale connection. + # Set Ping Interval to 20 seconds + ping_interval=20, max_outstanding_pings=5, - ping_interval=1, ) # Do something with the connection. @@ -153,7 +64,7 @@ await nc.connect( ```ruby require 'nats/client' -NATS.start(max_outstanding_pings: 5) do |nc| +NATS.start(ping_interval: 20, max_outstanding_pings: 5) do |nc| nc.on_reconnect do puts "Got reconnected to #{nc.connected_server}" end @@ -171,6 +82,7 @@ end ```typescript // will throw an exception if connection fails let nc = await connect({ + pingInterval: 20*1000, //20s maxPingOut: 5, url: "nats://demo.nats.io:4222" }); @@ -178,4 +90,3 @@ nc.close(); ``` {% endtab %} {% endtabs %} - From ce76c8997245457031c3009c80d1da8d56508ce6 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Mon, 3 Feb 2020 15:45:27 -0500 Subject: [PATCH 02/36] Adding examples for name option. This page is currently not part of the index. Will fix this in a later change. Signed-off-by: Matthias Hanel --- developing-with-nats/connecting/name.md | 86 ++++++++++++++++++++++++- 1 file changed, 85 insertions(+), 1 deletion(-) diff --git a/developing-with-nats/connecting/name.md b/developing-with-nats/connecting/name.md index b1dc56c..7b83f4c 100644 --- a/developing-with-nats/connecting/name.md +++ b/developing-with-nats/connecting/name.md @@ -2,4 +2,88 @@ Connections can be assigned a name which will appear in some of the server monitoring data. This name is not required but can help in debugging and testing. -!INCLUDE "../../_examples/connect_name.html" \ No newline at end of file +{% tabs %} +{% tab title="Go" %} +```go +nc, err := nats.Connect("demo.nats.io", nats.Name("API Name Option Example")) +if err != nil { + log.Fatal(err) +} +defer nc.Close() + +// Do something with the connection +``` +{% endtab %} + +{% tab title="Java" %} +```java +Options options = new Options.Builder(). + server("nats://demo.nats.io:4222"). + connectionName("API Name Option Example"). // Set Name + build(); +Connection nc = Nats.connect(options); + +// Do something with the connection + +nc.close(); +``` +{% endtab %} + +{% tab title="JavaScript" %} +```javascript +let nc = NATS.connect({ + url: "nats://demo.nats.io:4222", + name: "API Name Option Example" +}); +nc.on('connect', (c) => { + // Do something with the connection + doSomething(); + // When done close it + nc.close(); +}); +nc.on('error', (err) => { + failed(err); +}); +``` +{% endtab %} + +{% tab title="Python" %} +```python +nc = NATS() +await nc.connect( + servers=["nats://demo.nats.io:4222"], + name="API Name Option Example") + +# Do something with the connection + +await nc.close() +``` +{% endtab %} + +{% tab title="Ruby" %} +```ruby +require 'nats/client' + +NATS.start(servers: ["nats://demo.nats.io:4222"], name: "API Name Option Example") do |nc| + # Do something with the connection + + # Close the connection + nc.close +end +``` +{% endtab %} + +{% tab title="TypeScript" %} +```typescript +// will throw an exception if connection fails +let nc = await connect({ + url:"nats://demo.nats.io:4222", + name: "API Name Option Example" +}) +// Do something with the connection + +// Close the connection +nc.close(); +``` +{% endtab %} +{% endtabs %} \ No newline at end of file From e7d70aaea743b80ad151c36f1e1ccffb36a3ee08 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Mon, 3 Feb 2020 15:50:06 -0500 Subject: [PATCH 03/36] Mention seed server and connect in randomized order. Signed-off-by: Matthias Hanel --- developing-with-nats/connecting/cluster.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/developing-with-nats/connecting/cluster.md b/developing-with-nats/connecting/cluster.md index 9720fd6..a1947b2 100644 --- a/developing-with-nats/connecting/cluster.md +++ b/developing-with-nats/connecting/cluster.md @@ -7,11 +7,11 @@ When connecting to a cluster, there are a few things to think about. * The reconnect algorithm \(discussed later\) * Server provided URLs -When a client library first tries to connect it will use the list of URLs provided to the connection options or function. These URLs are checked, usually in order, and the first successful connection is used. +When a client library first tries to connect it will use the list of URLs provided to the connection options or function. These URLs are usually checked in random order as to not have every client connect to the same server. The first successful connection is used. Randomization can be [explicitly disabled](../reconnect/random.md). After a client connects to the server, the server may provide a list of URLs for additional known servers. This allows a client to connect to one server and still have other servers available during reconnect. -To insure the initial connection, your code should include a list of reasonable _front line_ servers. Those servers may know about other members of the cluster, and may tell the client about those members. But you don't have to configure the client to pass every valid member of the cluster in the connect method. +To insure the initial connection, your code should include a list of reasonable _front line_ or _seed_ servers. Those servers may know about other members of the cluster, and may tell the client about those members. But you don't have to configure the client to pass every valid member of the cluster in the connect method. By providing the ability to pass multiple connect options NATS can handle the possibility of a machine going down or being unavailable to a client. By adding the ability of the server to feed clients a list of known servers as part of the client-server protocol the mesh created by a cluster can grow and change organically while the clients are running. From 232c52a3f108f6506c1173f1d8bafda3732446b0 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Mon, 3 Feb 2020 15:53:36 -0500 Subject: [PATCH 04/36] Clearer language, added java script example, fixed values in examples Signed-off-by: Matthias Hanel --- .../connecting/connect_timeout.md | 26 +++++++++++++++---- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/developing-with-nats/connecting/connect_timeout.md b/developing-with-nats/connecting/connect_timeout.md index ac08ac6..e6f87f0 100644 --- a/developing-with-nats/connecting/connect_timeout.md +++ b/developing-with-nats/connecting/connect_timeout.md @@ -1,6 +1,6 @@ # Setting a Connect Timeout -Each library has its own, language preferred way, to pass connection options. One of the most common options is a connection timeout. To set the maximum time to connect to a server to 10 seconds: +Each library has its own, language preferred way, to pass connection options. One of the most common options is a connect timeout. It limits how long it can take to establishe a connection to a server. Should multiple urls be provided, this timeout applies to each cluster member individually. To set the maximum time to connect to a server to 10 seconds: {% tabs %} {% tab title="Go" %} @@ -30,13 +30,29 @@ nc.close(); {% endtab %} {% tab title="JavaScript" %} - +```javascript +let nc = NATS.connect({ + url: "nats://demo.nats.io:4222", + timeout: 10*1000 //10s +}); +nc.on('connect', (c) => { + // Do something with the connection + doSomething(); + // When done close it + nc.close(); +}); +nc.on('error', (err) => { + failed(err); +}); +``` {% endtab %} {% tab title="Python" %} ```python nc = NATS() -await nc.connect(connect_timeout=2) +await nc.connect( + servers=["nats://demo.nats.io:4222"], + connect_timeout=10) # Do something with the connection @@ -49,7 +65,7 @@ await nc.close() # There is currently no connect timeout as part of the Ruby NATS client API, but you can use a timer to mimic it. require 'nats/client' -timer = EM.add_timer(5) do +timer = EM.add_timer(10) do NATS.connect do |nc| # Do something with the connection @@ -65,7 +81,7 @@ EM.cancel_timer(timer) ```typescript let nc = await connect({ url: "nats://demo.nats.io:4222", - timeout: 1000 + timeout: 10*1000 //10s }); ``` {% endtab %} From 3380706b5c66ca3307917dced66d63a11e429ca9 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Mon, 3 Feb 2020 19:16:59 -0500 Subject: [PATCH 05/36] Fix description Signed-off-by: Matthias Hanel --- developing-with-nats/connecting/specific_server.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/connecting/specific_server.md b/developing-with-nats/connecting/specific_server.md index 5d49cda..ee6ea2b 100644 --- a/developing-with-nats/connecting/specific_server.md +++ b/developing-with-nats/connecting/specific_server.md @@ -2,7 +2,7 @@ The NATS client libraries can take a full URL, `nats://demo.nats.io:4222`, to specify a specific server host and port to connect to. -Libraries are removing the requirement for an explicit protocol and may allow `nats://demo.nats.io:4222` or just `demo.nats.io:4222`. Check with your specific client library's documentation to see what URL formats are supported. +Libraries are removing the requirement for an explicit protocol and may allow `demo.nats.io:4222` or just `demo.nats.io`. In the later example the default port 4222 will be used. Check with your specific client library's documentation to see what URL formats are supported. For example, to connect to the demo server with a URL you can use: From 0de28c17df0b640442c2949e545ddd247aa4fe1a Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Mon, 3 Feb 2020 20:24:42 -0500 Subject: [PATCH 06/36] Adding a subject example with a mix of wildcards Signed-off-by: Matthias Hanel --- nats-concepts/subjects.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/nats-concepts/subjects.md b/nats-concepts/subjects.md index 57dfbeb..dc8634b 100644 --- a/nats-concepts/subjects.md +++ b/nats-concepts/subjects.md @@ -1,6 +1,6 @@ # Subject-Based Messaging -Fundamentally NATS is about publishing and listening for messages. Both of these depend heavily on _Subjects_ which scope messages into streams or topics. At its simplest, a subject is just a string of characters that form a name the publisher and subscriber can use to find each other. +Fundamentally, NATS is about publishing and listening for messages. Both of these depend heavily on _Subjects_ which scope messages into streams or topics. At its simplest, a subject is just a string of characters that form a name the publisher and subscriber can use to find each other. ![](../.gitbook/assets/subjects1.svg) @@ -38,3 +38,6 @@ The second wildcard is `>` which will match one or more tokens, and can only app Subject to your security configuration, wildcards can be used for monitoring by creating something sometimes called a _wire tap_. In the simplest case you can create a subscriber for `>`. This application will receive all messages -- again, subject to security settings -- sent on your NATS cluster. +### Mix Wildcards + +The wildcard `*` can appear multiple times in the same subject. Both types be used as well. For example, `*.*.*.>` will receive `time.us.east.atlanta` as it is at least 4 tokens long. \ No newline at end of file From ec03a97d3c13be3f856ab91ffae72c0db25c5400 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Mon, 3 Feb 2020 22:17:19 -0500 Subject: [PATCH 07/36] Wording/formatting changes, adding links, match up text and example Signed-off-by: Matthias Hanel --- developing-with-nats/tutorials/pubsub.md | 14 +++++++------- developing-with-nats/tutorials/queues.md | 10 +++++----- developing-with-nats/tutorials/reqreply.md | 11 ++++++----- 3 files changed, 18 insertions(+), 17 deletions(-) diff --git a/developing-with-nats/tutorials/pubsub.md b/developing-with-nats/tutorials/pubsub.md index 9bd6855..e1d8d9e 100644 --- a/developing-with-nats/tutorials/pubsub.md +++ b/developing-with-nats/tutorials/pubsub.md @@ -1,6 +1,6 @@ # Explore NATS Pub/Sub -NATS is a publish subscribe messaging system. Subscribers listening on a subject receive messages on that subject. If the subscriber is not actively listening on the subject, the message is not received. Subscribers can use the wildcard tokens such as `*` and `>` to match a single token or to match the tail of a subject. +NATS is a [publish subscribe](../../nats-concepts/pubsub.md) messaging system [based on subjects](../../nats-concepts/subjects.md). Subscribers listening on a subject receive messages published on that subject. If the subscriber is not actively listening on the subject, the message is not received. Subscribers can use the wildcard tokens such as `*` and `>` to match a single token or to match the tail of a subject. ![](../../.gitbook/assets/pubsubtut.svg) @@ -82,9 +82,9 @@ or ### 8. Verify message publication and receipt -You should see that the publisher sends the message: _Published \[msg.test\] : 'NATS MESSAGE'_ +You should see that the publisher sends the message and prints: _Published \[msg.test\] : 'NATS MESSAGE'_ -And that the subscriber receives the message: _\[\#1\] Received on \[msg.test\]: 'NATS MESSAGE'_ +And that the subscriber receives the message and prints: _\[\#1\] Received on \[msg.test\]: 'NATS MESSAGE'_ Note that if the receiver does not get the message, check that you are using the same subject name for the publisher and the subscriber. @@ -94,7 +94,7 @@ Note that if the receiver does not get the message, check that you are using the % go run nats-pub/main.go msg.test "NATS MESSAGE 2" ``` -You should see that the subscriber receive message 2. Note that the message count is incremented each time your subscribing client receives a message on that subject: +You should see that the subscriber receives message 2. Note that the message count is incremented each time your subscribing client receives a message on that subject: ### 10. Start another shell or command prompt session @@ -106,7 +106,7 @@ You will use this session to run a second NATS subscriber. % cd $GOPATH/src/github.com/nats-io/nats.go/examples ``` -### 12. Subscribe to the message +### 12. Start a second client subscriber program ```bash % go run nats-sub/main.go msg.test @@ -130,7 +130,7 @@ You will use this session to run a third NATS subscriber. % cd $GOPATH/src/github.com/nats-io/nats.go/examples ``` -### 16. Subscribe to a different message +### 16. Subscribe to a different subject ```bash % go run nats-sub/main.go msg.test.new @@ -140,7 +140,7 @@ All the but last subscriber receives the message. Why? Because that subscriber i ### 17. Update the last subscriber to use a wildcard -NATS supports the use of wildcard characters for message subscribers. You cannot publish a message using a wildcard subject. +NATS supports the use of wildcard characters for message subscribers only. You cannot publish a message using a wildcard subject. Change the last subscriber the listen on msg.\* and run it: diff --git a/developing-with-nats/tutorials/queues.md b/developing-with-nats/tutorials/queues.md index 6a3170b..391d4b0 100644 --- a/developing-with-nats/tutorials/queues.md +++ b/developing-with-nats/tutorials/queues.md @@ -1,10 +1,10 @@ # Explore NATS Queueing -NATS supports a form of load balancing using queue groups. Subscribers register a queue group name. A single subscriber in the group is randomly selected to receive the message. +NATS supports a form of load balancing using [queue groups](../../nats-concepts/queue.md). Subscribers register a queue group name. A single subscriber in the group is randomly selected to receive the message. ## Prerequisites -Go and the NATS server should be installed. +Go, node.js, ruby and the NATS server should be installed. ### 1. Start the NATS server @@ -20,7 +20,7 @@ git clone https://github.com/nats-io/nats.js.git git clone https://github.com/nats-io/nats.rb.git ``` -### 3. Run the Go client subscriber with queue group name +### 3. Run the Go client subscriber, providing a queue group name ```bash cd $GOPATH/src/github.com/nats-io/nats.go/examples @@ -58,9 +58,9 @@ go run nats-pub/main.go foo "Hello NATS!" ### 8. Verify message publication and receipt -You should see that the publisher sends the message: _Published \[foo\] : 'Hello NATS!'_ +You should see that the publisher the message and prints: _Published \[foo\] : 'Hello NATS!'_ -You should see that only one of the my-queue group subscribers receives the message. In addition, the Go client subscriber not in the my-queue group should also receive the message. +You should see that only one of the my-queue group subscribers receives the message and prints it. In addition, the Go client subscriber not in the my-queue group should also receive and print the message. ### 9. Publish another message diff --git a/developing-with-nats/tutorials/reqreply.md b/developing-with-nats/tutorials/reqreply.md index 4510f0e..6c387e0 100644 --- a/developing-with-nats/tutorials/reqreply.md +++ b/developing-with-nats/tutorials/reqreply.md @@ -1,6 +1,6 @@ # Explore NATS Request/Reply -NATS supports request/reply messaging. In this tutorial you explore how to exchange point-to-point messages using NATS. +NATS supports [request/reply](../../nats-concepts/reqreply.md) messaging. In this tutorial you explore how to exchange point-to-point messages using NATS. ## Prerequisites @@ -25,18 +25,19 @@ You will use these sessions to run the NATS request and reply clients. ### 4. In one terminal, run the reply client listener ```bash -% go run nats-rply/main.go foo "this is my response" +% go run nats-rply/main.go help.please "OK, I CAN HELP!!!" ``` -You should see the message `Receiver is listening`, and that the NATS receiver client is listening on the "help.please" subject. The reply client acts as a receiver, listening for message requests. In NATS, the receiver is a subscriber. +You should see the message: _Listening on \[help.please\]_ + +This means that the NATS receiver client is listening for requests messages on the "help.please" subject. In NATS, the receiver is a subscriber. ### 5. In the other terminal, run the request client ```bash -% go run nats-req/main.go foo "request payload" +% go run nats-req/main.go help.please "some message" ``` The NATS requestor client makes a request by sending the message "some message" on the “help.please” subject. The NATS receiver client receives the message, formulates the reply \("OK, I CAN HELP!!!"\), and sends it to the inbox of the requester. - From 29deb9206dc459c1b8c52a2706e88fc1c0aff4a9 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Tue, 4 Feb 2020 11:41:21 -0500 Subject: [PATCH 08/36] Adding links, slight rewording, adding matcher example Signed-off-by: Matthias Hanel --- nats-concepts/intro.md | 3 +-- nats-concepts/reqreply.md | 2 +- nats-concepts/subjects.md | 2 +- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/nats-concepts/intro.md b/nats-concepts/intro.md index dc202e9..14ac0e0 100644 --- a/nats-concepts/intro.md +++ b/nats-concepts/intro.md @@ -6,5 +6,4 @@ NATS makes it easy for programs to communicate across different environments, la ![](../.gitbook/assets/intro.svg) -NATS core offers an **at most once** quality of service. If a subscriber is not listening on the subject \(no subject match\), or is not active when the message is sent, the message is not received. This is the same level of guarantee that TCP/IP provides. By default, NATS is a fire-and-forget messaging system. If you need higher levels of service, you can use [NATS Streaming](../nats-streaming-concepts/intro.md) or build additional reliability into your client applications with proven and scalable reference designs. - +NATS core offers an **at most once** quality of service. If a subscriber is not listening on the subject \(no subject match\), or is not active when the message is sent, the message is not received. This is the same level of guarantee that TCP/IP provides. By default, NATS is a fire-and-forget messaging system. If you need higher levels of service, you can use [NATS Streaming](../nats-streaming-concepts/intro.md) or build additional reliability into your client applications with proven and scalable reference designs such as [acks](acks.md) and [sequence numbers](seq_num.md). diff --git a/nats-concepts/reqreply.md b/nats-concepts/reqreply.md index 38b3e80..eefbbbd 100644 --- a/nats-concepts/reqreply.md +++ b/nats-concepts/reqreply.md @@ -2,7 +2,7 @@ Request-Reply is a common pattern in modern distributed systems. A request is sent and the application either waits on the response with a certain timeout or receives a response asynchronously. The increased complexity of modern systems requires features such as location transparency, scale up and scale down, observability and more. Many technologies need additional components, sidecars and proxies to accomplish the complete feature set. -NATS supports this pattern with its core communication mechanism, publish and subscribe. A request is published on a given subject with a reply subject, and responders listen on that subject and send responses to the reply subject. Reply subjects are usually a subject called an \_INBOX that will be directed back to the requestor dynamically, regardless of location of either party. +NATS supports this pattern with its core communication mechanism, publish and subscribe. A request is published on a given subject with a reply subject, and responders listen on that subject and send responses to the reply subject. Reply subjects are unique subjects called _inbox_ that are dynamically directed back to the requestor, regardless of location of either party. NATS allows multiple responders to run and form dynamic queue groups for transparent scale up. The ability for NATS applications to drain before exiting allows scale down with no requests being dropped. And since NATS is based on publish-subscribe, observability is as simple as running another application that can view requests and responses to measure latency, watch for anomalies, direct scalability and more. diff --git a/nats-concepts/subjects.md b/nats-concepts/subjects.md index dc8634b..2af8b1e 100644 --- a/nats-concepts/subjects.md +++ b/nats-concepts/subjects.md @@ -40,4 +40,4 @@ Subject to your security configuration, wildcards can be used for monitoring by ### Mix Wildcards -The wildcard `*` can appear multiple times in the same subject. Both types be used as well. For example, `*.*.*.>` will receive `time.us.east.atlanta` as it is at least 4 tokens long. \ No newline at end of file +The wildcard `*` can appear multiple times in the same subject. Both types be used as well. For example, `*.*.east.>` will receive `time.us.east.atlanta`. \ No newline at end of file From 4a8299c45eacd6ab2dd00f3ea990662dae49993e Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Tue, 4 Feb 2020 14:40:35 -0500 Subject: [PATCH 09/36] Reword and add links Signed-off-by: Matthias Hanel --- developing-with-nats/reconnect/README.md | 5 +---- developing-with-nats/reconnect/buffer.md | 4 ++-- developing-with-nats/reconnect/events.md | 1 + developing-with-nats/reconnect/max.md | 3 +-- developing-with-nats/reconnect/random.md | 6 +++--- developing-with-nats/reconnect/wait.md | 4 +--- 6 files changed, 9 insertions(+), 14 deletions(-) diff --git a/developing-with-nats/reconnect/README.md b/developing-with-nats/reconnect/README.md index de13b62..3f039bd 100644 --- a/developing-with-nats/reconnect/README.md +++ b/developing-with-nats/reconnect/README.md @@ -2,7 +2,4 @@ Most, if not all, of the client libraries will reconnect to the NATS system if they are disconnected for any reason. The reconnect logic can differ by library, so check your client library's documentation. -In general, the client will try to connect to all of the servers it knows about, either through the URLs provided in the `connect` call or the URLs provided by the NATS system itself. The NATS system will inform clients of new endpoints that can be used to reconnect. The library may have several options to help control reconnect behavior. - -The list of servers used during reconnect is library dependent, but generally is constructed from connect function/options and the list of servers provided by the NATS system itself. This feature allows NATS applications and the NATS system itself to self heal and reconfigure itself with no additional configuration or intervention. - +In general, the client will try to re-connect to one of the servers it knows about, either through the URLs provided in the `connect` call or the URLs provided by the NATS system during erlier connects. This feature allows NATS applications and the NATS system itself to self heal and reconfigure itself with no additional configuration or intervention. The library may have several options to help control reconnect behavior, notify about recennect state and inform about new server. \ No newline at end of file diff --git a/developing-with-nats/reconnect/buffer.md b/developing-with-nats/reconnect/buffer.md index 91c3244..93b0e0d 100644 --- a/developing-with-nats/reconnect/buffer.md +++ b/developing-with-nats/reconnect/buffer.md @@ -2,9 +2,9 @@ The NATS client libraries try as much as possible to be fire and forget. One of the features that may be included in the library you are using is the ability to buffer outgoing messages when the connection is down. -During a short reconnect, these client can allow applications to publish messages that, because the server is offline, will be cached in the client. The library will then send those messages on reconnect. When the maximum reconnect buffer is reached, messages will no longer be publishable by the client. +During a short reconnect, these client can allow applications to publish messages that, because the server is offline, will be cached in the client. The library will then send those messages once reconnected. When the maximum reconnect buffer is reached, messages will no longer be publishable by the client and an error will be returned. -Be aware, while the message appears to be sent to the application it is possible that it is never sent because the connection is never remade. Your applications should use patterns like acknowledgements to ensure delivery. +Be aware, while the message appears to be sent to the application it is possible that it is never sent because the connection is never remade. Your applications should use patterns like [acknowledgements](../../nats-concepts/acks.md) to ensure delivery. For clients that support this feature, you are able to configure the size of this buffer with bytes, messages or both. diff --git a/developing-with-nats/reconnect/events.md b/developing-with-nats/reconnect/events.md index 8ea3bef..1671e3e 100644 --- a/developing-with-nats/reconnect/events.md +++ b/developing-with-nats/reconnect/events.md @@ -1,6 +1,7 @@ # Listening for Reconnect Events Because reconnect is primarily under the covers many libraries provide an event listener you can use to be notified of reconnect events. This event can be especially important for applications sending a lot of messages. +- [ ] what do clients commonly do if reconnect is in progress and say a message is published? Block or error? {% tabs %} {% tab title="Go" %} diff --git a/developing-with-nats/reconnect/max.md b/developing-with-nats/reconnect/max.md index 308dca5..ccc095d 100644 --- a/developing-with-nats/reconnect/max.md +++ b/developing-with-nats/reconnect/max.md @@ -1,6 +1,6 @@ # Set the Number of Reconnect Attempts -Applications can set the maximum reconnect attempts. Generally, this will limit the actual number of attempts total, but check your library documentation. For example, in Java, if the client knows about 3 servers and the maximum reconnects is set to 2, it will not try all of the servers. On the other hand, if the maximum is set to 6 it will try all of the servers twice before considering the reconnect a failure and closing. +Applications can set the maximum reconnect attempts per server. This includes server provided to the clients connect call, as well as server the client discovered through other server. Once re-connect to a server failed the specified amount of times in a row, it will be removed from the connect list. After a succesfull re-connect to a server, the client will reset this servers failed reconnect attempt count. If a server was removed from the connect list, it can be re-discovered on connect. This effectively resets the connect attempt count as well. If the client runs out of servers to re-connect, it will close the connection and [raise an error](events.md). {% tabs %} {% tab title="Go" %} @@ -71,7 +71,6 @@ end // will throw an exception if connection fails let nc = await connect({ maxReconnectAttempts: 10, - servers: ["nats://demo.nats.io:4222"] }); nc.close(); ``` diff --git a/developing-with-nats/reconnect/random.md b/developing-with-nats/reconnect/random.md index f37ce1b..7d07260 100644 --- a/developing-with-nats/reconnect/random.md +++ b/developing-with-nats/reconnect/random.md @@ -2,7 +2,7 @@ When a server goes down, there is a possible anti-pattern called the _Thundering Herd_ where all of the clients try to reconnect immediately, thus creating a denial of service attack. In order to prevent this, most NATS client libraries randomize the servers they attempt to connect to. This setting has no effect if only a single server is used, but in the case of a cluster, randomization, or shuffling, will ensure that no one server bears the brunt of the client reconnect attempts. -However, if you want to disable the randomization process, so that servers are always checked in the same order, you can do that in most libraries with a connection options: +However, if you want to disable the randomization process for connect and re-connect, so that servers are always checked in the same order, you can do that in most libraries with a connection option: {% tabs %} {% tab title="Go" %} @@ -39,7 +39,7 @@ nc.close(); {% tab title="JavaScript" %} ```javascript let nc = NATS.connect({ - noRandomize: false, + noRandomize: true, servers: ["nats://127.0.0.1:4443", "nats://demo.nats.io:4222" ] @@ -82,7 +82,7 @@ end ```typescript // will throw an exception if connection fails let nc = await connect({ - noRandomize: false, + noRandomize: true, servers: ["nats://127.0.0.1:4443", "nats://demo.nats.io:4222" ] diff --git a/developing-with-nats/reconnect/wait.md b/developing-with-nats/reconnect/wait.md index 3af9a23..0d59475 100644 --- a/developing-with-nats/reconnect/wait.md +++ b/developing-with-nats/reconnect/wait.md @@ -1,6 +1,6 @@ # Pausing Between Reconnect Attempts -It doesn’t make much sense to try to connect to the same server over and over. To prevent this sort of thrashing, and wasted reconnect attempts, libraries provide a wait setting. This setting will pause the reconnect logic if the same server is being tried multiple times **in a row**. In the previous example, if you have 3 servers and 6 attempts, the Java library would loop over the three servers. If none were connectable, it will then try all three again. However, the Java client doesn’t wait between each attempt, only when trying the same server again, so in that example the library may never wait. If on the other hand, you only provide a single server URL and 6 attempts, the library will wait between each attempt. +It doesn’t make much sense to try to connect to the same server over and over. To prevent this sort of thrashing, and wasted reconnect attempts, libraries provide a wait setting. Generally clients make sure that between two reconnect attempts to the **same** server at least a certain amount of time has passed. The concrete implementation depends on the library used. {% tabs %} {% tab title="Go" %} @@ -33,7 +33,6 @@ nc.close(); {% tab title="JavaScript" %} ```javascript let nc = NATS.connect({ - reconnectTimeWait: 10 * 1000, //10s servers: ["nats://demo.nats.io:4222"] }); ``` @@ -70,7 +69,6 @@ end ```typescript // will throw an exception if connection fails let nc = await connect({ - reconnectTimeWait: 10*1000, //10s servers: ["nats://demo.nats.io:4222"] }); nc.close(); From ca1c9888b9cf29d656525a1cd6ba1937d9928233 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Tue, 4 Feb 2020 18:33:16 -0500 Subject: [PATCH 10/36] Reword text, delete duplicated examples. Signed-off-by: Matthias Hanel --- developing-with-nats/security/tls.md | 200 ++------------------------- 1 file changed, 8 insertions(+), 192 deletions(-) diff --git a/developing-with-nats/security/tls.md b/developing-with-nats/security/tls.md index 753962c..531b766 100644 --- a/developing-with-nats/security/tls.md +++ b/developing-with-nats/security/tls.md @@ -1,19 +1,17 @@ -# Encrypting Connections with TLS +# Encrypting and Authenticating Connections with TLS -While authentication limits which clients can connect, TLS can be used to check the server’s identity and optionally the client’s identity and will encrypt all traffic between the two. The most secure version of TLS with NATS is to use verified client certificates. In this mode, the client can check that it trusts the certificate sent by NATS system but the individual server will also check that it trusts the certificate sent by the client. From an application's perspective connecting to a server that does not verify client certificates may appear identical. Under the covers, disabling TLS verification removes the server side check on the client’s certificate. When started in TLS mode, a `nats-server` will require all clients to connect with TLS. Moreover, if configured to connect with TLS, client libraries will fail to connect to a server without TLS. +While authentication limits which clients can connect, TLS can be used to encrypt traffic between client/server and check the server’s identity. Additionally - in the most secure version of TLS with NATS - the server can be configured to verify the client's identity, thus authenticating it. When started in TLS mode, a `nats-server` will require all clients to connect with TLS. Moreover, if configured to connect with TLS, client libraries will fail to connect to a server without TLS. -The [Java examples repository](https://github.com/nats-io/java-nats-examples/tree/master/src/main/resources) contains certificates for starting the server in TLS mode. +## Connecting with TLS and verify client identity + +Using TLS to connect to a server that verifies the client's identity is straightforward. The client has to provide a certificate and private key. The NATS client will use these to prove it's identity to the server. For the client to verify the server's identity, the CA certificate is provided as well. + +The [Java examples repository](https://github.com/nats-io/java-nats-examples/tree/master/src/main/resources) contains certificates and a server config for this example. ```bash -> nats-server -c /src/main/resources/tls.conf - or > nats-server -c /src/main/resources/tls_verify.conf ``` -## Connecting with TLS - -Connecting to a server with TLS is straightforward. Most clients will automatically use TLS when connected to a NATS system using TLS. Setting up a NATS system to use TLS is primarily an exercise in setting up the certificate and trust managers. Clients may also need additional information, for example: - {% tabs %} {% tab title="Go" %} ```go @@ -199,186 +197,4 @@ let nc = await connect({ ## Connecting with the TLS Protocol -Some clients may support the `tls` protocol as well as a manual setting to turn on TLS. However, in that case there is likely some form of default or environmental settings to allow the TLS libraries to find certificate and trust stores. - -{% tabs %} -{% tab title="Go" %} -```go -nc, err := nats.Connect("tls://localhost", nats.RootCAs("resources/certs/ca.pem")) // May need this if server is using self-signed certificate -if err != nil { - log.Fatal(err) -} -defer nc.Close() - -// Do something with the connection -``` -{% endtab %} - -{% tab title="Java" %} -```java -class SSLUtils { - public static String KEYSTORE_PATH = "src/main/resources/keystore.jks"; - public static String TRUSTSTORE_PATH = "src/main/resources/cacerts"; - public static String STORE_PASSWORD = "password"; - public static String KEY_PASSWORD = "password"; - public static String ALGORITHM = "SunX509"; - - public static KeyStore loadKeystore(String path) throws Exception { - KeyStore store = KeyStore.getInstance("JKS"); - BufferedInputStream in = new BufferedInputStream(new FileInputStream(path)); - - try { - store.load(in, STORE_PASSWORD.toCharArray()); - } finally { - if (in != null) { - in.close(); - } - } - - return store; - } - - public static KeyManager[] createTestKeyManagers() throws Exception { - KeyStore store = loadKeystore(KEYSTORE_PATH); - KeyManagerFactory factory = KeyManagerFactory.getInstance(ALGORITHM); - factory.init(store, KEY_PASSWORD.toCharArray()); - return factory.getKeyManagers(); - } - - public static TrustManager[] createTestTrustManagers() throws Exception { - KeyStore store = loadKeystore(TRUSTSTORE_PATH); - TrustManagerFactory factory = TrustManagerFactory.getInstance(ALGORITHM); - factory.init(store); - return factory.getTrustManagers(); - } - - public static SSLContext createSSLContext() throws Exception { - SSLContext ctx = SSLContext.getInstance(Options.DEFAULT_SSL_PROTOCOL); - ctx.init(createTestKeyManagers(), createTestTrustManagers(), new SecureRandom()); - return ctx; - } -} - -public class ConnectTLS { - public static void main(String[] args) { - - try { - SSLContext ctx = SSLUtils.createSSLContext(); - Options options = new Options.Builder(). - server("nats://localhost:4222"). - sslContext(ctx). // Set the SSL context - build(); - Connection nc = Nats.connect(options); - - // Do something with the connection - - nc.close(); - } catch (Exception e) { - e.printStackTrace(); - } - } -} -``` -{% endtab %} - -{% tab title="JavaScript" %} -```javascript -let caCert = fs.readFileSync(caCertPath); -let clientCert = fs.readFileSync(clientCertPath); -let clientKey = fs.readFileSync(clientKeyPath); -let nc = NATS.connect({ - url: url, - tls: { - ca: [caCert], - key: [clientKey], - cert: [clientCert] - } -}); -``` -{% endtab %} - -{% tab title="Python" %} -```python -nc = NATS() - -ssl_ctx = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH) -ssl_ctx.load_verify_locations('ca.pem') -ssl_ctx.load_cert_chain(certfile='client-cert.pem', - keyfile='client-key.pem') -await nc.connect(io_loop=loop, tls=ssl_ctx) - -await nc.connect(servers=["nats://demo.nats.io:4222"], tls=ssl_ctx) - -# Do something with the connection. -``` -{% endtab %} - -{% tab title="Ruby" %} -```ruby -EM.run do - - options = { - :servers => [ - 'nats://localhost:4222', - ], - :tls => { - :private_key_file => './spec/configs/certs/key.pem', - :cert_chain_file => './spec/configs/certs/server.pem' - } - } - - NATS.connect(options) do |nc| - puts "#{Time.now.to_f} - Connected to NATS at #{nc.connected_server}" - - nc.subscribe("hello") do |msg| - puts "#{Time.now.to_f} - Received: #{msg}" - end - - nc.flush do - nc.publish("hello", "world") - end - - EM.add_periodic_timer(0.1) do - next unless nc.connected? - nc.publish("hello", "hello") - end - - # Set default callbacks - nc.on_error do |e| - puts "#{Time.now.to_f } - Error: #{e}" - end - - nc.on_disconnect do |reason| - puts "#{Time.now.to_f} - Disconnected: #{reason}" - end - - nc.on_reconnect do |nc| - puts "#{Time.now.to_f} - Reconnected to NATS server at #{nc.connected_server}" - end - - nc.on_close do - puts "#{Time.now.to_f} - Connection to NATS closed" - EM.stop - end - end -end -``` -{% endtab %} - -{% tab title="TypeScript" %} -```typescript -let caCert = readFileSync(caCertPath); -let clientCert = readFileSync(clientCertPath); -let clientKey = readFileSync(clientKeyPath); -let nc = await connect({ - url: url, - tls: { - ca: [caCert], - key: [clientKey], - cert: [clientCert] - } -}); -``` -{% endtab %} -{% endtabs %} - +Clients (such as go, java, javascript, ruby and type script) support providing a URL scontaining the `tls` protocol to the NATS connect call. This will turn on TLS without the need for further code changes. However, in that case there is likely some form of default or environmental settings to allow the TLS libraries of your programming language to find certificate and trusted CAs. Unless these settings are taken into accounts or otherwise modified this way of connecting is very likely to fail. \ No newline at end of file From bd776a0ec85b9c282f701b3cb35c2ae7653dd821 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Tue, 4 Feb 2020 19:43:48 -0500 Subject: [PATCH 11/36] add link to ssl Signed-off-by: Matthias Hanel --- developing-with-nats/security/tls.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/security/tls.md b/developing-with-nats/security/tls.md index 531b766..c92a475 100644 --- a/developing-with-nats/security/tls.md +++ b/developing-with-nats/security/tls.md @@ -1,6 +1,6 @@ # Encrypting and Authenticating Connections with TLS -While authentication limits which clients can connect, TLS can be used to encrypt traffic between client/server and check the server’s identity. Additionally - in the most secure version of TLS with NATS - the server can be configured to verify the client's identity, thus authenticating it. When started in TLS mode, a `nats-server` will require all clients to connect with TLS. Moreover, if configured to connect with TLS, client libraries will fail to connect to a server without TLS. +While authentication limits which clients can connect, TLS can be used to encrypt traffic between client/server and check the server’s identity. Additionally - in the most secure version of TLS with NATS - the server can be configured to verify the client's identity, thus authenticating it. When started in [TLS mode](../../nats-server/configuration/securing_nats/tls.md), a `nats-server` will require all clients to connect with TLS. Moreover, if configured to connect with TLS, client libraries will fail to connect to a server without TLS. ## Connecting with TLS and verify client identity From a3f7e14dfd8a1dd64347c9eaf9753c24d745adc1 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Tue, 4 Feb 2020 19:46:28 -0500 Subject: [PATCH 12/36] Fix examples, same token, python was using token in url Signed-off-by: Matthias Hanel --- developing-with-nats/security/token.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/developing-with-nats/security/token.md b/developing-with-nats/security/token.md index be40208..850c141 100644 --- a/developing-with-nats/security/token.md +++ b/developing-with-nats/security/token.md @@ -50,7 +50,7 @@ let nc = NATS.connect({url: `nats://127.0.0.1:${port}`, token: "mytoken!"}); ```python nc = NATS() -await nc.connect(servers=["nats://mytoken@demo.nats.io:4222"]) +await nc.connect(servers=["nats://demo.nats.io:4222"], token="mytoken") # Do something with the connection. ``` @@ -58,7 +58,7 @@ await nc.connect(servers=["nats://mytoken@demo.nats.io:4222"]) {% tab title="Ruby" %} ```ruby -NATS.start(token: "deadbeef") do |nc| +NATS.start(token: "mytoken") do |nc| puts "Connected using token" end ``` @@ -105,7 +105,7 @@ nc.close(); {% tab title="JavaScript" %} ```javascript -let url = `nats://mytoken!@127.0.0.1:${port}`; +let url = `nats://mytoken@127.0.0.1:${port}`; let nc = NATS.connect({url: url}); ``` {% endtab %} @@ -122,7 +122,7 @@ await nc.connect(servers=["nats://mytoken@demo.nats.io:4222"]) {% tab title="Ruby" %} ```ruby -NATS.start("deadbeef@127.0.0.1:4222") do |nc| +NATS.start("mytoken@127.0.0.1:4222") do |nc| puts "Connected using token!" end ``` @@ -130,7 +130,7 @@ end {% tab title="TypeScript" %} ```typescript -let url = `nats://:mytoken!@127.0.0.1:${port}`; +let url = `nats://:mytoken@127.0.0.1:${port}`; let nc = await connect({url: url}); ``` {% endtab %} From 3b8a29f6572cda88b1cd6143576f3ff2aa6b9113 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Wed, 5 Feb 2020 18:07:31 -0500 Subject: [PATCH 13/36] Slight rewording/clarification, adding links. Signed-off-by: Matthias Hanel --- developing-with-nats/security/creds.md | 2 +- developing-with-nats/security/nkey.md | 2 +- developing-with-nats/security/userpass.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/developing-with-nats/security/creds.md b/developing-with-nats/security/creds.md index e1ecd27..cc2896e 100644 --- a/developing-with-nats/security/creds.md +++ b/developing-with-nats/security/creds.md @@ -1,6 +1,6 @@ # Authenticating with a Credentials File -The 2.0 version of NATS server introduced the idea of JWT-based authentication. Clients interact with this new scheme using a user JWT and the private key from an NKey pair. To help make connecting with a JWT easier, the client libraries support the concept of a credentials file. This file contains both the private key and the JWT and can be generated with the `nsc` tool. The contents will look like the following and should be protected because it contains a private key. This creds file is unused and only for example purposes. +The 2.0 version of NATS server introduced the idea of decentralized authentication based on [JSON Web Tokens \(JWT\)](https://jwt.io/). Clients interact with this new scheme using a [user JWT](../../nats-server/configuration/securing_nats/auth_intro/jwt_auth.md) and corresponding [NKey](../../nats-server/configuration/securing_nats/auth_intro/nkey_auth.md) private key. To help make connecting with a JWT easier, the client libraries support the concept of a credentials file. This file contains both the private key and the JWT and can be generated with the `nsc` [tool](../../nats-tools/nsc/README.md). The contents will look like the following and should be protected because it contains a private key. This creds file is unused and only for example purposes. ```text -----BEGIN NATS USER JWT----- diff --git a/developing-with-nats/security/nkey.md b/developing-with-nats/security/nkey.md index c5c87f6..7479d6b 100644 --- a/developing-with-nats/security/nkey.md +++ b/developing-with-nats/security/nkey.md @@ -1,6 +1,6 @@ # Authenticating with an NKey -The 2.0 version of NATS server introduces a new challenge response authentication option. This challenge response is based on a wrapper we call NKeys which uses [Ed25519](https://ed25519.cr.yp.to/) signing. The server can use these keys in several ways for authentication. The simplest is for the server to be configured with a list of known public keys and for the clients to respond to the challenge by signing it with its private key. This challenge-response ensures security by ensuring that the client has the private key, but also protects the private key from the server which never has to actually see it. +The 2.0 version of NATS server introduces a new challenge response authentication option. This challenge response is based on a wrapper we call [NKeys](../../nats-server/configuration/securing_nats/auth_intro/nkey_auth.md). The server can use these keys in several ways for authentication. The simplest is for the server to be configured with a list of known public keys and for the clients to respond to the challenge by signing it with its private key. (A printable private NKey is refered to as seed). This challenge-response ensures security by ensuring that the client has the private key, but also protects the private key from the server which never has to actually see it. Handling challenge response may require more than just a setting in the connection options, depending on the client library. diff --git a/developing-with-nats/security/userpass.md b/developing-with-nats/security/userpass.md index 2f661b5..755a043 100644 --- a/developing-with-nats/security/userpass.md +++ b/developing-with-nats/security/userpass.md @@ -6,7 +6,7 @@ For this example, start the server using: > nats-server --user myname --pass password ``` -You can encrypt passwords to pass to `nats-server` using a simple tool provided by the server: +You can encrypt passwords to pass to `nats-server` using a simple [tool](../../nats-tools/mkpasswd.md) provided by the server: ```bash > go run mkpasswd.go -p From c5b9305ca6e2be1f15040c995d876bedf7c4b1c2 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Wed, 5 Feb 2020 18:36:22 -0500 Subject: [PATCH 14/36] Clarifications. unsub_after was made to fit go client Signed-off-by: Matthias Hanel --- developing-with-nats/receiving/README.md | 5 ++--- developing-with-nats/receiving/unsub_after.md | 6 +----- developing-with-nats/receiving/wildcards.md | 4 +--- 3 files changed, 4 insertions(+), 11 deletions(-) diff --git a/developing-with-nats/receiving/README.md b/developing-with-nats/receiving/README.md index 9582fb5..b4be18f 100644 --- a/developing-with-nats/receiving/README.md +++ b/developing-with-nats/receiving/README.md @@ -4,7 +4,6 @@ In general, applications can receive messages asynchronously or synchronously. R Some languages, like Go or Java, provide synchronous and asynchronous APIs, while others may only support one type of subscription. -In all cases, the process of subscribing involves having the client library tell the NATS system that an application is interested in a particular subject. - -Under the covers, the client library will assign a unique id to each subscription. This id is used as a closure when the server sends messages to a specific subscription. Each subscription gets a unique id, so if the same connection is used multiple times for the same subject, the server will send multiple copies of the same message. When an application is done with a subscription it unsubscribes which tells the server to stop sending messages. +In all cases, the process of subscribing involves having the client library tell the NATS system that an application is interested in a particular subject. When an application is done with a subscription it unsubscribes which tells the server to stop sending messages. +If a connection has multiple subscriptions using identical or overlapping subjects (say `foo` and `>`), the same message will be sent to the client multiple times. diff --git a/developing-with-nats/receiving/unsub_after.md b/developing-with-nats/receiving/unsub_after.md index 7140280..30b4cb9 100644 --- a/developing-with-nats/receiving/unsub_after.md +++ b/developing-with-nats/receiving/unsub_after.md @@ -4,11 +4,7 @@ NATS provides a special form of unsubscribe that is configured with a message co The message count you provide is the total message count for a subscriber. So if you unsubscribe with a count of 1, the server will stop sending messages to that subscription after it has received one message. If the subscriber has already received one or more messages, the unsubscribe will be immediate. This action based on history can be confusing if you try to auto unsubscribe on a long running subscription, but is logical for a new one. -> Auto unsubscribe is based on the total messages sent to a subscriber, not just the new ones. - -Auto unsubscribe can also result in some tricky edge cases if a server cluster is used. The client will tell the server of the unsubscribe count when the application requests it. But if the client disconnects before the count is reached, it may have to tell another server of the remaining count. This dance between previous server notifications and new notifications on reconnect can result in unplanned behavior. - -Finally, most of the client libraries also track the max message count after an auto unsubscribe request. Which means that the client will stop allowing messages to flow even if the server has miscounted due to reconnects or some other failure in the client library. +Auto unsubscribe is based on the total messages sent to a subscriber, not just the new ones. Most of the client libraries also track the max message count after an auto unsubscribe request. On reconnect, this enables clients to resend the unsubscribe with an updated total. The following example shows unsubscribe after a single message: diff --git a/developing-with-nats/receiving/wildcards.md b/developing-with-nats/receiving/wildcards.md index 013855a..e0556fa 100644 --- a/developing-with-nats/receiving/wildcards.md +++ b/developing-with-nats/receiving/wildcards.md @@ -1,8 +1,6 @@ # Wildcard Subscriptions -There is no special code to subscribe with a wildcard subject. Wildcards are a normal part of the subject name. - -However, there is a common technique that may come in to play when you use wildcards. This technique is to use the subject provided with the incoming message to determine what to do with the message. +There is no special code to subscribe with a [wildcard subject](../../nats-concepts/subjects.md#wildcards). Wildcards are a normal part of the subject name. However, it is a common technique to use use the subject provided with the incoming message to determine what to do with the message. For example, you can subscribe using `*` and then act based on the actual subject. From 62a7de85bbcf919a60306a7d399afa8b673aa68b Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Wed, 5 Feb 2020 18:43:12 -0500 Subject: [PATCH 15/36] Move queue permissions, hint at queue name hierarchy, add/fix links Signed-off-by: Matthias Hanel --- developing-with-nats/receiving/queues.md | 38 +------------------ nats-concepts/queue.md | 2 + .../securing_nats/authorization.md | 36 +++++++++++++++++- 3 files changed, 39 insertions(+), 37 deletions(-) diff --git a/developing-with-nats/receiving/queues.md b/developing-with-nats/receiving/queues.md index 2f0aa29..774105e 100644 --- a/developing-with-nats/receiving/queues.md +++ b/developing-with-nats/receiving/queues.md @@ -1,8 +1,8 @@ # Queue Subscriptions -Subscribing to a queue group is only slightly different than subscribing to a subject alone. The application simply includes a queue name with the subscription. The effect of including the group is fairly major, since the server will now load balance messages between the members of the queue group, but the code differences are minimal. +Subscribing to a [queue group](../../nats-concepts/queue.md) is only slightly different than subscribing to a subject alone. The application simply includes a queue name with the subscription. The server will load balance between all members of the queue group. In a cluster setup, every member has the same chance of receiving a particular message. -Keep in mind that the queue groups in NATS are dynamic and do not require any server configuration. You can almost think of a regular subscription as a queue group of 1, but it is probably not worth thinking too much about that. +Keep in mind that queue groups in NATS are dynamic and do not require any server configuration. ![](../../.gitbook/assets/queues.svg) @@ -122,37 +122,3 @@ await nc.subscribe('updates', (err, msg) => { {% endtabs %} If you run this example with the publish examples that send to `updates`, you will see that one of the instances gets a message while the others you run won't. But the instance that receives the message will change. - -## Queue Permissions - -Added in NATS Server v2.1.2, Queue Permissions allow you to express authorization for queue groups. As queue groups are integral to implementing horizontally scalable microservices, control of who is allowed to join a specific queue group is important to the overall security model. - -A Queue Permission can be defined with the syntax ` `, where the name of the queue can also use wildcards, for example the following would allow clients to join queue groups v1 and v2.\*, but won't allow plain subscriptions: - -```text -allow = ["foo v1", "foo v2.*"] -``` - -The full wildcard can also be used, for example the following would prevent plain subscriptions on `bar` but allow the client to join any queue: - -```text -allow = ["bar >"] -``` - -Permissions for Queue Subscriptions can be combined with plain subscriptions as well though, for example you could allow plain subscriptions on `foo` but constrain the queues to which a client can join, as well as preventing any service from using a queue subscription with the name `*.prod`: - -```text -users = [ - { - user: "foo", permissions: { - sub: { - # Allow plain subscription foo, but only v1 groups or *.dev queue groups - allow: ["foo", "foo v1", "foo v1.>", "foo *.dev"] - - # Prevent queue subscriptions on prod groups - deny: ["> *.prod"] - } - } -] -``` - diff --git a/nats-concepts/queue.md b/nats-concepts/queue.md index d08a2c7..6f1222f 100644 --- a/nats-concepts/queue.md +++ b/nats-concepts/queue.md @@ -4,6 +4,8 @@ NATS provides a built-in load balancing feature called distributed queues. Using To create a queue subscription, subscribers register a queue name. All subscribers with the same queue name form the queue group. This requires no configuration. As messages on the registered subject are published, one member of the group is chosen randomly to receive the message. Although queue groups have multiple subscribers, each message is consumed by only one. +Queue group names follow the same naming rules as [subjects](subjects.md). Foremost, they are case sensitive and cannot contain whitespace. Consider structuring queue groups hierarchically using `.`. Some server functionalities can use [wildcard matching](subjects.md#wildcards) on them. + One of the great features of NATS is that queue groups are defined by the application and their queue subscribers, not on the server configuration. Queue subscribers are ideal for scaling services. Scale up is as simple as running another application, scale down is terminating the application with a signal that drains the in flight requests. This flexibility and lack of any configuration changes makes NATS an excellent service communication technology that can work with all platform technologies. diff --git a/nats-server/configuration/securing_nats/authorization.md b/nats-server/configuration/securing_nats/authorization.md index a946bb9..e79f756 100644 --- a/nats-server/configuration/securing_nats/authorization.md +++ b/nats-server/configuration/securing_nats/authorization.md @@ -2,7 +2,7 @@ The NATS server supports authorization using subject-level permissions on a per-user basis. Permission-based authorization is available with multi-user authentication via the `users` list. -Each permission specifies the subjects the user can publish to and subscribe to. The parser is generous at understanding what the intent is, so both arrays and singletons are processed. For more complex configuration, you can specify a `permission` object which explicitly allows or denies subjects. The specified subjects can specify wildcards. Permissions can make use of [variables](../#variables). +Each permission specifies the subjects the user can publish to and subscribe to. The parser is generous at understanding what the intent is, so both arrays and singletons are processed. For more complex configuration, you can specify a `permission` object which explicitly allows or denies subjects. The specified subjects can specify wildcards. Permissions can make use of [variables](../README.md#variables). You configure authorization by creating a `permissions` entry in the `authorization` object. @@ -95,3 +95,37 @@ authorization: { } ``` +## Queue Permissions + +Added in NATS Server v2.1.2, Queue Permissions allow you to express authorization for queue groups. As queue groups are integral to implementing horizontally scalable microservices, control of who is allowed to join a specific queue group is important to the overall security model. + +A Queue Permission can be defined with the syntax ` `, where the name of the queue can also use wildcards, for example the following would allow clients to join queue groups v1 and v2.\*, but won't allow plain subscriptions: + +```text +allow = ["foo v1", "foo v2.*"] +``` + +The full wildcard can also be used, for example the following would prevent plain subscriptions on `bar` but allow the client to join any queue: + +```text +allow = ["bar >"] +``` + +Permissions for Queue Subscriptions can be combined with plain subscriptions as well though, for example you could allow plain subscriptions on `foo` but constrain the queues to which a client can join, as well as preventing any service from using a queue subscription with the name `*.prod`: + +```text +users = [ + { + user: "foo", permissions: { + sub: { + # Allow plain subscription foo, but only v1 groups or *.dev queue groups + allow: ["foo", "foo v1", "foo v1.>", "foo *.dev"] + + # Prevent queue subscriptions on prod groups + deny: ["> *.prod"] + } + } +] +``` + + From 51561bc6a64ac992b220fe87282a45523dba5bd8 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Wed, 5 Feb 2020 19:10:42 -0500 Subject: [PATCH 16/36] renamed chache -> buffer, flush is not subject to max outgoing pings. fixed a typo. Signed-off-by: Matthias Hanel --- developing-with-nats/connecting/pingpong.md | 2 +- developing-with-nats/sending/caches.md | 7 ++++--- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/developing-with-nats/connecting/pingpong.md b/developing-with-nats/connecting/pingpong.md index b2f6166..9dd8f61 100644 --- a/developing-with-nats/connecting/pingpong.md +++ b/developing-with-nats/connecting/pingpong.md @@ -1,6 +1,6 @@ # Ping/Pong Protocol -The client and server use a simple PING/PONG protocol to check that either of them are still connected to the other. On a regular interval he client will ping the server, which responds with a pong. Once a configurable maximum of outstanding pings without a single pong reply is hit, the connection is closed as stale. Together these two values define a timeout for the connection. In the pressence of traffic, such as messages or client side pings, the server will not initiate the PING/PONG interaction. +The client and server use a simple PING/PONG protocol to check that either of them are still connected to the other. On a regular interval he client will ping the server, which responds with a pong. Once a configurable maximum of outstanding pings without a single pong reply is hit, the connection is closed as stale. Together these two values define a timeout for the connection. In the presence of traffic, such as messages or client side pings, the server will not initiate the PING/PONG interaction. ![](../../.gitbook/assets/pingpong.svg) diff --git a/developing-with-nats/sending/caches.md b/developing-with-nats/sending/caches.md index 1907590..309e258 100644 --- a/developing-with-nats/sending/caches.md +++ b/developing-with-nats/sending/caches.md @@ -1,6 +1,6 @@ -# Caches, Flush and Ping +# Buffers and Flush -For performance reasons, most if not all, of the client libraries will cache outgoing data so that bigger chunks can be written to the network at one time. This may be as simple as a byte buffer that stores up a few messages before being pushed to the network. +For performance reasons, most if not all, of the client libraries will buffer outgoing data so that bigger chunks can be written to the network at one time. This may be as simple as a byte buffer that stores a few messages before being pushed to the network. These buffers do not hold messages forever, generally they are designed to hold messages in high throughput scenarios, while still providing good latency in low throughput situations. @@ -113,5 +113,6 @@ nc.close(); ## Flush and Ping/Pong -Many of the client libraries use the PING/PONG interaction built into the NATS protocol to insure that flush pushed all of the cached messages to the server. When an application calls flush most libraries will put a PING on the outgoing queue of messages, and wait for the server to send PONG before saying that the flush was successful. +Many of the client libraries use the [PING/PONG interaction](../connecting/pingpong.md) built into the NATS protocol to insure that flush pushed all of the buffered messages to the server. When an application calls flush, most libraries will put a PING on the outgoing queue of messages, and wait for the server to respond with a PONG before saying that the flush was successful. +Even though the client may use PING/PONG for flush, pings sent this way do not count towards [max outgoing pings](../connecting/pingpong.md). From 6fd732492886181944075c6f05c912f6456aa071 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Wed, 5 Feb 2020 19:30:30 -0500 Subject: [PATCH 17/36] Fixing/adding samples, slight reword Signed-off-by: Matthias Hanel --- developing-with-nats/events/events.md | 19 ++++++++++++++++--- developing-with-nats/events/slow.md | 8 ++++---- 2 files changed, 20 insertions(+), 7 deletions(-) diff --git a/developing-with-nats/events/events.md b/developing-with-nats/events/events.md index 0ffc499..7de5352 100644 --- a/developing-with-nats/events/events.md +++ b/developing-with-nats/events/events.md @@ -11,6 +11,20 @@ Connection events may include the connection being closed, disconnected or recon ```go // There is not a single listener for connection events in the NATS Go Client. // Instead, you can set individual event handlers using: +nc, err := nats.Connect("demo.nats.io", + nats.DisconnectErrHandler(func(_ *nats.Conn, err error) { + log.Printf("client disconnected: %v", err) + }), + nats.ReconnectHandler(func(_ *nats.Conn) { + log.Printf("client reconnected") + }), + nats.ClosedHandler(func(_ *nats.Conn) { + log.Printf("client closed") + })) +if err != nil { + log.Fatal(err) +} +defer nc.Close() DisconnectHandler(cb ConnHandler) ReconnectHandler(cb ConnHandler) @@ -114,7 +128,7 @@ await nc.connect(**options) {% tab title="Ruby" %} ```ruby -r# There is not a single listener for connection events in the Ruby NATS Client. +# There is not a single listener for connection events in the Ruby NATS Client. # Instead, you can set individual event handlers using: NATS.on_disconnect do @@ -229,7 +243,7 @@ nc.on('serversDiscovered', (urls) => { {% tab title="Ruby" %} ```ruby -r# The Ruby NATS client does not support discovered servers handler right now +# The Ruby NATS client does not support discovered servers handler right now ``` {% endtab %} @@ -354,4 +368,3 @@ nc.on('error', (err) => { ``` {% endtab %} {% endtabs %} - diff --git a/developing-with-nats/events/slow.md b/developing-with-nats/events/slow.md index 2c87d48..6b327db 100644 --- a/developing-with-nats/events/slow.md +++ b/developing-with-nats/events/slow.md @@ -1,8 +1,8 @@ # Slow Consumers -NATS is designed to move messages through the server quickly. As a result, NATS depends on the applications to consider and respond to changing message rates. The server will do a bit of impedance matching, but if a client is too slow the server will eventually cut them off. These cut off connections are called _slow consumers_. +NATS is designed to move messages through the server quickly. As a result, NATS depends on the applications to consider and respond to changing message rates. The server will do a bit of impedance matching, but if a client is too slow the server will eventually cut them off by closing the connection. These cut off connections are called _slow consumers_. -One way some of the libraries deal with bursty message traffic is to cache incoming messages for a subscription. So if an application can handle 10 messages per second and sometimes receives 20 messages per second, the library may hold the extra 10 to give the application time to catch up. To the server, the application will appear to be handling the messages and consider the connection healthy. It is up to the client library to decide what to do when the cache is too big, but most client libraries will drop incoming messages. +One way some of the libraries deal with bursty message traffic is to buffer incoming messages for a subscription. So if an application can handle 10 messages per second and sometimes receives 20 messages per second, the library may hold the extra 10 to give the application time to catch up. To the server, the application will appear to be handling the messages and consider the connection healthy. It is up to the client library to decide what to do when the cache is too big, but most client libraries will drop incoming messages. Receiving and dropping messages from the server keeps the connection to the server healthy, but creates an application requirement. There are several common patterns: @@ -101,7 +101,7 @@ await nc.subscribe("updates", cb=cb, pending_bytes_limit=5*1024*1024, pending_ms {% tab title="Ruby" %} ```ruby -# The Ruby NATS client currently does not have option to customize slow consumer limits per sub. +# The Ruby NATS client currently does not have option to customize specify a subscribers pending limits. ``` {% endtab %} @@ -200,7 +200,7 @@ public class SlowConsumerListener { if len(msgs) == 3: # Head of line blocking on other messages caused - # by single message proccesing taking long... + # by single message proccesing taking long... await asyncio.sleep(1) await nc.subscribe("updates", cb=cb, pending_msgs_limit=5) From aa5be434603a01db94c9d8643de9619f5b7f5526 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Wed, 5 Feb 2020 20:14:29 -0500 Subject: [PATCH 18/36] Fixing typos Signed-off-by: Matthias Hanel --- developing-with-nats/connecting/connect_timeout.md | 2 +- developing-with-nats/reconnect/README.md | 2 +- developing-with-nats/reconnect/max.md | 2 +- developing-with-nats/security/creds.md | 2 +- developing-with-nats/security/nkey.md | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/developing-with-nats/connecting/connect_timeout.md b/developing-with-nats/connecting/connect_timeout.md index e6f87f0..88cce4c 100644 --- a/developing-with-nats/connecting/connect_timeout.md +++ b/developing-with-nats/connecting/connect_timeout.md @@ -1,6 +1,6 @@ # Setting a Connect Timeout -Each library has its own, language preferred way, to pass connection options. One of the most common options is a connect timeout. It limits how long it can take to establishe a connection to a server. Should multiple urls be provided, this timeout applies to each cluster member individually. To set the maximum time to connect to a server to 10 seconds: +Each library has its own, language preferred way, to pass connection options. One of the most common options is a connect timeout. It limits how long it can take to established a connection to a server. Should multiple urls be provided, this timeout applies to each cluster member individually. To set the maximum time to connect to a server to 10 seconds: {% tabs %} {% tab title="Go" %} diff --git a/developing-with-nats/reconnect/README.md b/developing-with-nats/reconnect/README.md index 3f039bd..85e9387 100644 --- a/developing-with-nats/reconnect/README.md +++ b/developing-with-nats/reconnect/README.md @@ -2,4 +2,4 @@ Most, if not all, of the client libraries will reconnect to the NATS system if they are disconnected for any reason. The reconnect logic can differ by library, so check your client library's documentation. -In general, the client will try to re-connect to one of the servers it knows about, either through the URLs provided in the `connect` call or the URLs provided by the NATS system during erlier connects. This feature allows NATS applications and the NATS system itself to self heal and reconfigure itself with no additional configuration or intervention. The library may have several options to help control reconnect behavior, notify about recennect state and inform about new server. \ No newline at end of file +In general, the client will try to re-connect to one of the servers it knows about, either through the URLs provided in the `connect` call or the URLs provided by the NATS system during earlier connects. This feature allows NATS applications and the NATS system itself to self heal and reconfigure itself with no additional configuration or intervention. The library may have several options to help control reconnect behavior, notify about reconnect state and inform about new server. \ No newline at end of file diff --git a/developing-with-nats/reconnect/max.md b/developing-with-nats/reconnect/max.md index ccc095d..e3402d1 100644 --- a/developing-with-nats/reconnect/max.md +++ b/developing-with-nats/reconnect/max.md @@ -1,6 +1,6 @@ # Set the Number of Reconnect Attempts -Applications can set the maximum reconnect attempts per server. This includes server provided to the clients connect call, as well as server the client discovered through other server. Once re-connect to a server failed the specified amount of times in a row, it will be removed from the connect list. After a succesfull re-connect to a server, the client will reset this servers failed reconnect attempt count. If a server was removed from the connect list, it can be re-discovered on connect. This effectively resets the connect attempt count as well. If the client runs out of servers to re-connect, it will close the connection and [raise an error](events.md). +Applications can set the maximum reconnect attempts per server. This includes server provided to the clients connect call, as well as server the client discovered through other server. Once re-connect to a server failed the specified amount of times in a row, it will be removed from the connect list. After a successful re-connect to a server, the client will reset this servers failed reconnect attempt count. If a server was removed from the connect list, it can be re-discovered on connect. This effectively resets the connect attempt count as well. If the client runs out of servers to re-connect, it will close the connection and [raise an error](events.md). {% tabs %} {% tab title="Go" %} diff --git a/developing-with-nats/security/creds.md b/developing-with-nats/security/creds.md index cc2896e..0c56d28 100644 --- a/developing-with-nats/security/creds.md +++ b/developing-with-nats/security/creds.md @@ -1,6 +1,6 @@ # Authenticating with a Credentials File -The 2.0 version of NATS server introduced the idea of decentralized authentication based on [JSON Web Tokens \(JWT\)](https://jwt.io/). Clients interact with this new scheme using a [user JWT](../../nats-server/configuration/securing_nats/auth_intro/jwt_auth.md) and corresponding [NKey](../../nats-server/configuration/securing_nats/auth_intro/nkey_auth.md) private key. To help make connecting with a JWT easier, the client libraries support the concept of a credentials file. This file contains both the private key and the JWT and can be generated with the `nsc` [tool](../../nats-tools/nsc/README.md). The contents will look like the following and should be protected because it contains a private key. This creds file is unused and only for example purposes. +The 2.0 version of NATS server introduced the idea of decentralized authentication based on [JSON Web Tokens \(JWT\)](https://jwt.io/). Clients interact with this new scheme using a [user JWT](../../nats-server/configuration/securing_nats/auth_intro/jwt_auth.md) and corresponding [NKey](../../nats-server/configuration/securing_nats/auth_intro/nkey_auth.md) private key. To help make connecting with a JWT easier, the client libraries support the concept of a credentials file. This file contains both the private key and the JWT and can be generated with the `nsc` [tool](../../nats-tools/nsc/README.md). The contents will look like the following and should be protected because it contains a private key. This credentials file is unused and only for example purposes. ```text -----BEGIN NATS USER JWT----- diff --git a/developing-with-nats/security/nkey.md b/developing-with-nats/security/nkey.md index 7479d6b..54dd975 100644 --- a/developing-with-nats/security/nkey.md +++ b/developing-with-nats/security/nkey.md @@ -1,6 +1,6 @@ # Authenticating with an NKey -The 2.0 version of NATS server introduces a new challenge response authentication option. This challenge response is based on a wrapper we call [NKeys](../../nats-server/configuration/securing_nats/auth_intro/nkey_auth.md). The server can use these keys in several ways for authentication. The simplest is for the server to be configured with a list of known public keys and for the clients to respond to the challenge by signing it with its private key. (A printable private NKey is refered to as seed). This challenge-response ensures security by ensuring that the client has the private key, but also protects the private key from the server which never has to actually see it. +The 2.0 version of NATS server introduces a new challenge response authentication option. This challenge response is based on a wrapper we call [NKeys](../../nats-server/configuration/securing_nats/auth_intro/nkey_auth.md). The server can use these keys in several ways for authentication. The simplest is for the server to be configured with a list of known public keys and for the clients to respond to the challenge by signing it with its private key. (A printable private NKey is referred to as seed). This challenge-response ensures security by ensuring that the client has the private key, but also protects the private key from the server which never has to actually see it. Handling challenge response may require more than just a setting in the connection options, depending on the client library. From 24a57f1c0ac79abe8cbe554305b18403eb785c24 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 11:34:45 -0600 Subject: [PATCH 19/36] Update pingpong.md --- developing-with-nats/connecting/pingpong.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/connecting/pingpong.md b/developing-with-nats/connecting/pingpong.md index 9dd8f61..e4a16d4 100644 --- a/developing-with-nats/connecting/pingpong.md +++ b/developing-with-nats/connecting/pingpong.md @@ -1,6 +1,6 @@ # Ping/Pong Protocol -The client and server use a simple PING/PONG protocol to check that either of them are still connected to the other. On a regular interval he client will ping the server, which responds with a pong. Once a configurable maximum of outstanding pings without a single pong reply is hit, the connection is closed as stale. Together these two values define a timeout for the connection. In the presence of traffic, such as messages or client side pings, the server will not initiate the PING/PONG interaction. +The client and server use a simple PING/PONG protocol to check that either of them are still connected to the other. On a regular interval the client will ping the server, which responds with a pong. Once a configurable maximum of outstanding pings without a single pong reply is hit, the connection is closed as stale. Together these two values define a timeout for the connection. In the presence of traffic, such as messages or client side pings, the server will not initiate the PING/PONG interaction. ![](../../.gitbook/assets/pingpong.svg) From 22a54e1d71c430210510723809d6496024b12487 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 11:41:51 -0600 Subject: [PATCH 20/36] Update cluster.md --- developing-with-nats/connecting/cluster.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/developing-with-nats/connecting/cluster.md b/developing-with-nats/connecting/cluster.md index a1947b2..b65f558 100644 --- a/developing-with-nats/connecting/cluster.md +++ b/developing-with-nats/connecting/cluster.md @@ -11,9 +11,9 @@ When a client library first tries to connect it will use the list of URLs provid After a client connects to the server, the server may provide a list of URLs for additional known servers. This allows a client to connect to one server and still have other servers available during reconnect. -To insure the initial connection, your code should include a list of reasonable _front line_ or _seed_ servers. Those servers may know about other members of the cluster, and may tell the client about those members. But you don't have to configure the client to pass every valid member of the cluster in the connect method. +To ensure the initial connection, your code should include a list of reasonable _front line_ or _seed_ servers. Those servers may know about other members of the cluster, and may tell the client about those members. But you don't have to configure the client to pass every valid member of the cluster in the connect method. -By providing the ability to pass multiple connect options NATS can handle the possibility of a machine going down or being unavailable to a client. By adding the ability of the server to feed clients a list of known servers as part of the client-server protocol the mesh created by a cluster can grow and change organically while the clients are running. +By providing the ability to pass multiple connect options, NATS can handle the possibility of a machine going down or being unavailable to a client. By adding the ability of the server to feed clients a list of known servers as part of the client-server protocol the mesh created by a cluster can grow and change organically while the clients are running. _Note, failure behavior is library dependent, please check the documentation for your client library on information about what happens if the connect fails._ From eb8ed79218157abbf9d1d0a593df6c2d9c278a2f Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 11:43:14 -0600 Subject: [PATCH 21/36] Update connect_timeout.md --- developing-with-nats/connecting/connect_timeout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/connecting/connect_timeout.md b/developing-with-nats/connecting/connect_timeout.md index 88cce4c..f4b3a9e 100644 --- a/developing-with-nats/connecting/connect_timeout.md +++ b/developing-with-nats/connecting/connect_timeout.md @@ -1,6 +1,6 @@ # Setting a Connect Timeout -Each library has its own, language preferred way, to pass connection options. One of the most common options is a connect timeout. It limits how long it can take to established a connection to a server. Should multiple urls be provided, this timeout applies to each cluster member individually. To set the maximum time to connect to a server to 10 seconds: +Each library has its own, language preferred way, to pass connection options. One of the most common options is a connect timeout. It limits how long it can take to established a connection to a server. Should multiple URLs be provided, this timeout applies to each cluster member individually. To set the maximum time to connect to a server to 10 seconds: {% tabs %} {% tab title="Go" %} From 3fc1922e98efaf503b112e3cffe122f5dc8574f5 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 11:56:56 -0600 Subject: [PATCH 22/36] Update slow.md --- developing-with-nats/events/slow.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/events/slow.md b/developing-with-nats/events/slow.md index 6b327db..cf291cb 100644 --- a/developing-with-nats/events/slow.md +++ b/developing-with-nats/events/slow.md @@ -200,7 +200,7 @@ public class SlowConsumerListener { if len(msgs) == 3: # Head of line blocking on other messages caused - # by single message proccesing taking long... + # by single message processing taking too long... await asyncio.sleep(1) await nc.subscribe("updates", cb=cb, pending_msgs_limit=5) From d55fabce9b0366f1817aaab8f48652106c49f412 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 11:58:32 -0600 Subject: [PATCH 23/36] Update README.md --- developing-with-nats/receiving/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/receiving/README.md b/developing-with-nats/receiving/README.md index b4be18f..6fbe358 100644 --- a/developing-with-nats/receiving/README.md +++ b/developing-with-nats/receiving/README.md @@ -4,6 +4,6 @@ In general, applications can receive messages asynchronously or synchronously. R Some languages, like Go or Java, provide synchronous and asynchronous APIs, while others may only support one type of subscription. -In all cases, the process of subscribing involves having the client library tell the NATS system that an application is interested in a particular subject. When an application is done with a subscription it unsubscribes which tells the server to stop sending messages. +In all cases, the process of subscribing involves having the client library tell the NATS system that an application is interested in a particular subject. When an application is done with a subscription it unsubscribes telling the server to stop sending messages. If a connection has multiple subscriptions using identical or overlapping subjects (say `foo` and `>`), the same message will be sent to the client multiple times. From f679971b40365557ada05baebd6f975ccc83b8fd Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 12:04:17 -0600 Subject: [PATCH 24/36] Update unsub_after.md --- developing-with-nats/receiving/unsub_after.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/developing-with-nats/receiving/unsub_after.md b/developing-with-nats/receiving/unsub_after.md index 30b4cb9..c5d7e58 100644 --- a/developing-with-nats/receiving/unsub_after.md +++ b/developing-with-nats/receiving/unsub_after.md @@ -2,9 +2,9 @@ NATS provides a special form of unsubscribe that is configured with a message count and takes effect when that many messages are sent to a subscriber. This mechanism is very useful if only a single message is expected. -The message count you provide is the total message count for a subscriber. So if you unsubscribe with a count of 1, the server will stop sending messages to that subscription after it has received one message. If the subscriber has already received one or more messages, the unsubscribe will be immediate. This action based on history can be confusing if you try to auto unsubscribe on a long running subscription, but is logical for a new one. +The message count you provide is the total message count for a subscriber. So if you unsubscribe with a count of 1, the server will stop sending messages to that subscription after it has received one message. If the subscriber has already received one or more messages, the unsubscribe will be immediate. This action based on history can be confusing if you try to auto-unsubscribe on a long running subscription, but is logical for a new one. -Auto unsubscribe is based on the total messages sent to a subscriber, not just the new ones. Most of the client libraries also track the max message count after an auto unsubscribe request. On reconnect, this enables clients to resend the unsubscribe with an updated total. +Auto-unsubscribe is based on the total messages sent to a subscriber, not just the new ones. Most of the client libraries also track the max message count after an auto-unsubscribe request. On reconnect, this enables clients to resend the unsubscribe with an updated total. The following example shows unsubscribe after a single message: @@ -63,7 +63,7 @@ nc.close(); let nc = NATS.connect({ url: "nats://demo.nats.io:4222" }); -// `max` specifies the number of messages that the server will forward. +// `max` specifies the number of messages that the server will forward // The server will auto-cancel. let opts = {max: 10}; let sub = nc.subscribe(NATS.createInbox(), opts, (msg) => { @@ -127,8 +127,8 @@ end {% tab title="TypeScript" %} ```typescript -// `max` specifies the number of messages that the server will forward. -// The server will auto-cancel. +// `max` specifies the number of messages that the server will forward +// The server will auto-cancel let opts = {max: 10}; let sub = await nc.subscribe(createInbox(), (err, msg) => { t.log(msg.data); From 30d8bbc8f8b9a464aa8f47d80306bb365c3c9fc2 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 12:05:20 -0600 Subject: [PATCH 25/36] Update wildcards.md --- developing-with-nats/receiving/wildcards.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/receiving/wildcards.md b/developing-with-nats/receiving/wildcards.md index e0556fa..6bd18ca 100644 --- a/developing-with-nats/receiving/wildcards.md +++ b/developing-with-nats/receiving/wildcards.md @@ -1,6 +1,6 @@ # Wildcard Subscriptions -There is no special code to subscribe with a [wildcard subject](../../nats-concepts/subjects.md#wildcards). Wildcards are a normal part of the subject name. However, it is a common technique to use use the subject provided with the incoming message to determine what to do with the message. +There is no special code to subscribe with a [wildcard subject](../../nats-concepts/subjects.md#wildcards). Wildcards are a normal part of the subject name. However, it is a common technique to use the subject provided with the incoming message to determine what to do with the message. For example, you can subscribe using `*` and then act based on the actual subject. From 5142b5f6ab77152b640fa431969d45ffe6854c28 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 12:07:12 -0600 Subject: [PATCH 26/36] Update README.md --- developing-with-nats/reconnect/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/reconnect/README.md b/developing-with-nats/reconnect/README.md index 85e9387..48eed21 100644 --- a/developing-with-nats/reconnect/README.md +++ b/developing-with-nats/reconnect/README.md @@ -2,4 +2,4 @@ Most, if not all, of the client libraries will reconnect to the NATS system if they are disconnected for any reason. The reconnect logic can differ by library, so check your client library's documentation. -In general, the client will try to re-connect to one of the servers it knows about, either through the URLs provided in the `connect` call or the URLs provided by the NATS system during earlier connects. This feature allows NATS applications and the NATS system itself to self heal and reconfigure itself with no additional configuration or intervention. The library may have several options to help control reconnect behavior, notify about reconnect state and inform about new server. \ No newline at end of file +In general, the client will try to re-connect to one of the servers it knows about, either through the URLs provided in the `connect` call or the URLs provided by the NATS system during earlier connects. This feature allows NATS applications and the NATS system itself to self heal and reconfigure itself with no additional configuration or intervention. The library may have several options to help control reconnect behavior, notify about reconnect state and to inform about a new server. From d0ea1ea702f374be432b9332f7827989e3e75f51 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 12:07:45 -0600 Subject: [PATCH 27/36] Update README.md --- developing-with-nats/reconnect/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/reconnect/README.md b/developing-with-nats/reconnect/README.md index 48eed21..98227c2 100644 --- a/developing-with-nats/reconnect/README.md +++ b/developing-with-nats/reconnect/README.md @@ -2,4 +2,4 @@ Most, if not all, of the client libraries will reconnect to the NATS system if they are disconnected for any reason. The reconnect logic can differ by library, so check your client library's documentation. -In general, the client will try to re-connect to one of the servers it knows about, either through the URLs provided in the `connect` call or the URLs provided by the NATS system during earlier connects. This feature allows NATS applications and the NATS system itself to self heal and reconfigure itself with no additional configuration or intervention. The library may have several options to help control reconnect behavior, notify about reconnect state and to inform about a new server. +In general, the client will try to re-connect to one of the servers it knows about, either through the URLs provided in the `connect` call or the URLs provided by the NATS system during earlier connects. This feature allows NATS applications and the NATS system itself to self heal and reconfigure itself with no additional configuration or intervention. The library may have several options to help control reconnect behavior, to notify about reconnect state and to inform about a new server. From 02d8d46f944a0149be66400e8ef09771f88750f2 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 12:11:50 -0600 Subject: [PATCH 28/36] Update buffer.md --- developing-with-nats/reconnect/buffer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/reconnect/buffer.md b/developing-with-nats/reconnect/buffer.md index 93b0e0d..8cbb616 100644 --- a/developing-with-nats/reconnect/buffer.md +++ b/developing-with-nats/reconnect/buffer.md @@ -2,7 +2,7 @@ The NATS client libraries try as much as possible to be fire and forget. One of the features that may be included in the library you are using is the ability to buffer outgoing messages when the connection is down. -During a short reconnect, these client can allow applications to publish messages that, because the server is offline, will be cached in the client. The library will then send those messages once reconnected. When the maximum reconnect buffer is reached, messages will no longer be publishable by the client and an error will be returned. +During a short reconnect, the client can allow applications to publish messages that, because the server is offline, will be cached in the client. The library will then send those messages once reconnected. When the maximum reconnect buffer is reached, messages will no longer be publishable by the client and an error will be returned. Be aware, while the message appears to be sent to the application it is possible that it is never sent because the connection is never remade. Your applications should use patterns like [acknowledgements](../../nats-concepts/acks.md) to ensure delivery. From 069bf345a40a475207d8042302a7c60571296a18 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 12:15:38 -0600 Subject: [PATCH 29/36] Update max.md --- developing-with-nats/reconnect/max.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/reconnect/max.md b/developing-with-nats/reconnect/max.md index e3402d1..6b81001 100644 --- a/developing-with-nats/reconnect/max.md +++ b/developing-with-nats/reconnect/max.md @@ -1,6 +1,6 @@ # Set the Number of Reconnect Attempts -Applications can set the maximum reconnect attempts per server. This includes server provided to the clients connect call, as well as server the client discovered through other server. Once re-connect to a server failed the specified amount of times in a row, it will be removed from the connect list. After a successful re-connect to a server, the client will reset this servers failed reconnect attempt count. If a server was removed from the connect list, it can be re-discovered on connect. This effectively resets the connect attempt count as well. If the client runs out of servers to re-connect, it will close the connection and [raise an error](events.md). +Applications can set the maximum reconnect attempts per server. This includes the server provided to the clients connect call, as well as the server the client discovered through another server. Once re-connect to a server fails the specified amount of times in a row, it will be removed from the connect list. After a successful re-connect to a server, the client will reset that servers failed reconnect attempt count. If a server was removed from the connect list, it can be re-discovered on connect. This effectively resets the connect attempt count as well. If the client runs out of servers to re-connect, it will close the connection and [raise an error](events.md). {% tabs %} {% tab title="Go" %} From 80d508c98754b2d610bc1a0383357e70058b1a42 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 12:22:23 -0600 Subject: [PATCH 30/36] Update tls.md --- developing-with-nats/security/tls.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/security/tls.md b/developing-with-nats/security/tls.md index c92a475..4226e56 100644 --- a/developing-with-nats/security/tls.md +++ b/developing-with-nats/security/tls.md @@ -197,4 +197,4 @@ let nc = await connect({ ## Connecting with the TLS Protocol -Clients (such as go, java, javascript, ruby and type script) support providing a URL scontaining the `tls` protocol to the NATS connect call. This will turn on TLS without the need for further code changes. However, in that case there is likely some form of default or environmental settings to allow the TLS libraries of your programming language to find certificate and trusted CAs. Unless these settings are taken into accounts or otherwise modified this way of connecting is very likely to fail. \ No newline at end of file +Clients (such as Go, Java, Javascript, Ruby and Type Script) support providing a URL containing the `tls` protocol to the NATS connect call. This will turn on TLS without the need for further code changes. However, in that case there is likely some form of default or environmental settings to allow the TLS libraries of your programming language to find certificate and trusted CAs. Unless these settings are taken into accounts or otherwise modified, this way of connecting is very likely to fail. From be826babb79bdde24f67544fbe2c60436ae37c17 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Thu, 6 Feb 2020 12:25:04 -0600 Subject: [PATCH 31/36] Update caches.md --- developing-with-nats/sending/caches.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developing-with-nats/sending/caches.md b/developing-with-nats/sending/caches.md index 309e258..dca1ab5 100644 --- a/developing-with-nats/sending/caches.md +++ b/developing-with-nats/sending/caches.md @@ -113,6 +113,6 @@ nc.close(); ## Flush and Ping/Pong -Many of the client libraries use the [PING/PONG interaction](../connecting/pingpong.md) built into the NATS protocol to insure that flush pushed all of the buffered messages to the server. When an application calls flush, most libraries will put a PING on the outgoing queue of messages, and wait for the server to respond with a PONG before saying that the flush was successful. +Many of the client libraries use the [PING/PONG interaction](../connecting/pingpong.md) built into the NATS protocol to ensure that flush pushed all of the buffered messages to the server. When an application calls flush, most libraries will put a PING on the outgoing queue of messages, and wait for the server to respond with a PONG before saying that the flush was successful. Even though the client may use PING/PONG for flush, pings sent this way do not count towards [max outgoing pings](../connecting/pingpong.md). From e66580b2e088c1c2ffd7f2f397ef36a8bfbafe27 Mon Sep 17 00:00:00 2001 From: ainsley Date: Fri, 7 Feb 2020 14:57:40 -0600 Subject: [PATCH 32/36] adding `accounts` to nats-server config table --- nats-server/configuration/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/nats-server/configuration/README.md b/nats-server/configuration/README.md index d9210ba..62a9c3f 100644 --- a/nats-server/configuration/README.md +++ b/nats-server/configuration/README.md @@ -107,6 +107,7 @@ authorization: { | Property | Description | | :--- | :--- | | [`authorization`](securing_nats/auth_intro/) | Configuration map for client authentication/authorization | +| [`accounts`](securing_nats/auth_intro/accounts) | Configuration map for accounts | | [`cluster`](clustering/cluster_config.md) | Configuration map for clustering configuration | | `connect_error_reports` | Number of attempts at which a repeated failed route, gateway or leaf node connection is reported. Default is 3600, approx every hour. | | `debug` | If `true` enable debug log messages | From edddbce8876ce7e1da8d050f3f8c5ff0612afc31 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Fri, 7 Feb 2020 14:59:31 -0600 Subject: [PATCH 33/36] Update README.md --- nats-server/configuration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nats-server/configuration/README.md b/nats-server/configuration/README.md index 62a9c3f..da58f0b 100644 --- a/nats-server/configuration/README.md +++ b/nats-server/configuration/README.md @@ -106,8 +106,8 @@ authorization: { | Property | Description | | :--- | :--- | -| [`authorization`](securing_nats/auth_intro/) | Configuration map for client authentication/authorization | | [`accounts`](securing_nats/auth_intro/accounts) | Configuration map for accounts | +| [`authorization`](securing_nats/auth_intro/) | Configuration map for client authentication/authorization | | [`cluster`](clustering/cluster_config.md) | Configuration map for clustering configuration | | `connect_error_reports` | Number of attempts at which a repeated failed route, gateway or leaf node connection is reported. Default is 3600, approx every hour. | | `debug` | If `true` enable debug log messages | From 81cd57f5dc4a3531505d93adde41050bf62ae139 Mon Sep 17 00:00:00 2001 From: Ginger Collison Date: Fri, 7 Feb 2020 15:11:36 -0600 Subject: [PATCH 34/36] Update README.md --- nats-server/configuration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nats-server/configuration/README.md b/nats-server/configuration/README.md index da58f0b..eef5762 100644 --- a/nats-server/configuration/README.md +++ b/nats-server/configuration/README.md @@ -106,7 +106,7 @@ authorization: { | Property | Description | | :--- | :--- | -| [`accounts`](securing_nats/auth_intro/accounts) | Configuration map for accounts | +| [`accounts`](securing_nats/auth_intro/accounts.md) | Configuration map for accounts | | [`authorization`](securing_nats/auth_intro/) | Configuration map for client authentication/authorization | | [`cluster`](clustering/cluster_config.md) | Configuration map for clustering configuration | | `connect_error_reports` | Number of attempts at which a repeated failed route, gateway or leaf node connection is reported. Default is 3600, approx every hour. | From 34220c41145de0b121cf35fc6051250e9335d636 Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Sun, 9 Feb 2020 15:07:03 -0500 Subject: [PATCH 35/36] Incorporating comments from PR Signed-off-by: Matthias Hanel --- SUMMARY.md | 3 ++- developing-with-nats/connecting/name.md | 2 +- developing-with-nats/connecting/pingpong.md | 8 ++++++-- developing-with-nats/events/slow.md | 2 +- developing-with-nats/receiving/README.md | 2 +- developing-with-nats/reconnect/events.md | 1 - developing-with-nats/reconnect/wait.md | 4 +++- developing-with-nats/security/nkey.md | 2 +- developing-with-nats/security/token.md | 2 +- developing-with-nats/tutorials/reqreply.md | 4 ++-- 10 files changed, 18 insertions(+), 12 deletions(-) diff --git a/SUMMARY.md b/SUMMARY.md index 8851cd9..09d8cba 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -22,6 +22,7 @@ * [Connecting to the Default Server](developing-with-nats/connecting/default_server.md) * [Connecting to a Specific Server](developing-with-nats/connecting/specific_server.md) * [Connecting to a Cluster](developing-with-nats/connecting/cluster.md) + * [Connection Name](developing-with-nats/connecting/name.md) * [Setting a Connect Timeout](developing-with-nats/connecting/connect_timeout.md) * [Ping/Pong Protocol](developing-with-nats/connecting/pingpong.md) * [Controlling the Client/Server Protocol](developing-with-nats/connecting/protocol.md) @@ -29,8 +30,8 @@ * [Automatic Reconnections](developing-with-nats/reconnect/README.md) * [Disabling Reconnect](developing-with-nats/reconnect/disable.md) * [Set the Number of Reconnect Attempts](developing-with-nats/reconnect/max.md) - * [Pausing Between Reconnect Attempts](developing-with-nats/reconnect/wait.md) * [Avoiding the Thundering Herd](developing-with-nats/reconnect/random.md) + * [Pausing Between Reconnect Attempts](developing-with-nats/reconnect/wait.md) * [Listening for Reconnect Events](developing-with-nats/reconnect/events.md) * [Buffering Messages During Reconnect Attempts](developing-with-nats/reconnect/buffer.md) * [Securing Connections](developing-with-nats/security/README.md) diff --git a/developing-with-nats/connecting/name.md b/developing-with-nats/connecting/name.md index 7b83f4c..e41ce42 100644 --- a/developing-with-nats/connecting/name.md +++ b/developing-with-nats/connecting/name.md @@ -1,6 +1,6 @@ # Setting the Connection Name -Connections can be assigned a name which will appear in some of the server monitoring data. This name is not required but can help in debugging and testing. +Connections can be assigned a name which will appear in some of the server monitoring data. This name is not required, but is **highly recommended** as a friendly connection name will help in monitoring, error reporting, debugging, and testing. {% tabs %} {% tab title="Go" %} diff --git a/developing-with-nats/connecting/pingpong.md b/developing-with-nats/connecting/pingpong.md index e4a16d4..04ea312 100644 --- a/developing-with-nats/connecting/pingpong.md +++ b/developing-with-nats/connecting/pingpong.md @@ -1,10 +1,14 @@ # Ping/Pong Protocol -The client and server use a simple PING/PONG protocol to check that either of them are still connected to the other. On a regular interval the client will ping the server, which responds with a pong. Once a configurable maximum of outstanding pings without a single pong reply is hit, the connection is closed as stale. Together these two values define a timeout for the connection. In the presence of traffic, such as messages or client side pings, the server will not initiate the PING/PONG interaction. +The client and server use a simple PING/PONG protocol to check that either of them are still connected to the other. On a regular interval the client will ping the server, which responds with a pong. ![](../../.gitbook/assets/pingpong.svg) -If you have a connection that is going to be open a long time with few messages traveling on it, setting the PING interval and/or limit how many are outstanding, can control how quickly the client will be notified of a problem. However on connections with a lot of traffic, the client will often figure out there is a problem between PINGS, and as a result the default PING interval is often on the order of minutes. To set the interval to 20s and limit outstanding pings to 5, thus force a closed connection after 100s of inactivity: +Once a configurable maximum of outstanding pings without a single pong reply is hit, the connection is closed as stale. Together these two values define a timeout for the connection which specifies how quickly the client will be notified of a problem. This will also help when there is a remote network partition where the operating system does not detect a socket error. Upon connection close the client will attempt to reconnect. When it knows about other server, these will be tried next. + +In the presence of traffic, such as messages or client side pings, the server will not initiate the PING/PONG interaction. + +On connections with a lot of traffic, the client will often figure out there is a problem between PINGS, and as a result the default PING interval is often on the order of minutes. To set the interval to 20s and limit outstanding pings to 5, thus force a closed connection after 100s of inactivity: {% tabs %} {% tab title="Go" %} diff --git a/developing-with-nats/events/slow.md b/developing-with-nats/events/slow.md index cf291cb..7dd837a 100644 --- a/developing-with-nats/events/slow.md +++ b/developing-with-nats/events/slow.md @@ -101,7 +101,7 @@ await nc.subscribe("updates", cb=cb, pending_bytes_limit=5*1024*1024, pending_ms {% tab title="Ruby" %} ```ruby -# The Ruby NATS client currently does not have option to customize specify a subscribers pending limits. +# The Ruby NATS client currently does not have option to specify a subscribers pending limits. ``` {% endtab %} diff --git a/developing-with-nats/receiving/README.md b/developing-with-nats/receiving/README.md index 6fbe358..5bad6b4 100644 --- a/developing-with-nats/receiving/README.md +++ b/developing-with-nats/receiving/README.md @@ -6,4 +6,4 @@ Some languages, like Go or Java, provide synchronous and asynchronous APIs, whil In all cases, the process of subscribing involves having the client library tell the NATS system that an application is interested in a particular subject. When an application is done with a subscription it unsubscribes telling the server to stop sending messages. -If a connection has multiple subscriptions using identical or overlapping subjects (say `foo` and `>`), the same message will be sent to the client multiple times. +A client will receive a message for each matching subscription ,so if a connection has multiple subscriptions using identical or overlapping subjects (say `foo` and `>`) the same message will be sent to the client multiple times. diff --git a/developing-with-nats/reconnect/events.md b/developing-with-nats/reconnect/events.md index 1671e3e..8ea3bef 100644 --- a/developing-with-nats/reconnect/events.md +++ b/developing-with-nats/reconnect/events.md @@ -1,7 +1,6 @@ # Listening for Reconnect Events Because reconnect is primarily under the covers many libraries provide an event listener you can use to be notified of reconnect events. This event can be especially important for applications sending a lot of messages. -- [ ] what do clients commonly do if reconnect is in progress and say a message is published? Block or error? {% tabs %} {% tab title="Go" %} diff --git a/developing-with-nats/reconnect/wait.md b/developing-with-nats/reconnect/wait.md index 0d59475..50d84dd 100644 --- a/developing-with-nats/reconnect/wait.md +++ b/developing-with-nats/reconnect/wait.md @@ -1,6 +1,8 @@ # Pausing Between Reconnect Attempts -It doesn’t make much sense to try to connect to the same server over and over. To prevent this sort of thrashing, and wasted reconnect attempts, libraries provide a wait setting. Generally clients make sure that between two reconnect attempts to the **same** server at least a certain amount of time has passed. The concrete implementation depends on the library used. +It doesn’t make much sense to try to connect to the same server over and over. To prevent this sort of thrashing, and wasted reconnect attempts, especially when using TLS, libraries provide a wait setting. Generally clients make sure that between two reconnect attempts to the **same** server at least a certain amount of time has passed. The concrete implementation depends on the library used. + +This setting not only prevents wasting client resources, it also alleviates a [_thundering herd_](random.md) situation when additional servers are not available. {% tabs %} {% tab title="Go" %} diff --git a/developing-with-nats/security/nkey.md b/developing-with-nats/security/nkey.md index 54dd975..ac8393c 100644 --- a/developing-with-nats/security/nkey.md +++ b/developing-with-nats/security/nkey.md @@ -1,6 +1,6 @@ # Authenticating with an NKey -The 2.0 version of NATS server introduces a new challenge response authentication option. This challenge response is based on a wrapper we call [NKeys](../../nats-server/configuration/securing_nats/auth_intro/nkey_auth.md). The server can use these keys in several ways for authentication. The simplest is for the server to be configured with a list of known public keys and for the clients to respond to the challenge by signing it with its private key. (A printable private NKey is referred to as seed). This challenge-response ensures security by ensuring that the client has the private key, but also protects the private key from the server which never has to actually see it. +The 2.0 version of NATS server introduces a new challenge response authentication option. This challenge response is based on a wrapper we call [NKeys](../../nats-server/configuration/securing_nats/auth_intro/nkey_auth.md). The server can use these keys in several ways for authentication. The simplest is for the server to be configured with a list of known public keys and for the clients to respond to the challenge by signing it with its private key. (A printable private NKey is referred to as seed). This challenge-response ensures security by ensuring that the client has the private key, but also protects the private key from the server, which never has access to it! Handling challenge response may require more than just a setting in the connection options, depending on the client library. diff --git a/developing-with-nats/security/token.md b/developing-with-nats/security/token.md index 850c141..f7fe283 100644 --- a/developing-with-nats/security/token.md +++ b/developing-with-nats/security/token.md @@ -1,6 +1,6 @@ # Authenticating with a Token -Tokens are basically random strings, much like a password, and can provide a simple authentication mechanism in some situations. However, tokens are only as safe as they are secret so other authentication schemes can provide more security in large installations. +Tokens are basically random strings, much like a password, and can provide a simple authentication mechanism in some situations. However, tokens are only as safe as they are secret so other authentication schemes can provide more security in large installations. It is highly recommended to use one of the other NATS authentication mechanisms. For this example, start the server using: diff --git a/developing-with-nats/tutorials/reqreply.md b/developing-with-nats/tutorials/reqreply.md index 6c387e0..285128b 100644 --- a/developing-with-nats/tutorials/reqreply.md +++ b/developing-with-nats/tutorials/reqreply.md @@ -35,9 +35,9 @@ This means that the NATS receiver client is listening for requests messages on t ### 5. In the other terminal, run the request client ```bash -% go run nats-req/main.go help.please "some message" +% go run nats-req/main.go help.please "I need help!" ``` -The NATS requestor client makes a request by sending the message "some message" on the “help.please” subject. +The NATS requestor client makes a request by sending the message "I need help!" on the “help.please” subject. The NATS receiver client receives the message, formulates the reply \("OK, I CAN HELP!!!"\), and sends it to the inbox of the requester. From e9dc3ace9edb3ab7cf404b72398a5197575bf96a Mon Sep 17 00:00:00 2001 From: Matthias Hanel Date: Sun, 9 Feb 2020 17:33:35 -0500 Subject: [PATCH 36/36] Clarified behavior on slow consumer error and fixed typo Signed-off-by: Matthias Hanel --- developing-with-nats/connecting/connect_timeout.md | 2 +- developing-with-nats/events/slow.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/developing-with-nats/connecting/connect_timeout.md b/developing-with-nats/connecting/connect_timeout.md index f4b3a9e..1c4ed7a 100644 --- a/developing-with-nats/connecting/connect_timeout.md +++ b/developing-with-nats/connecting/connect_timeout.md @@ -1,6 +1,6 @@ # Setting a Connect Timeout -Each library has its own, language preferred way, to pass connection options. One of the most common options is a connect timeout. It limits how long it can take to established a connection to a server. Should multiple URLs be provided, this timeout applies to each cluster member individually. To set the maximum time to connect to a server to 10 seconds: +Each library has its own, language preferred way, to pass connection options. One of the most common options is a connect timeout. It limits how long it can take to establish a connection to a server. Should multiple URLs be provided, this timeout applies to each cluster member individually. To set the maximum time to connect to a server to 10 seconds: {% tabs %} {% tab title="Go" %} diff --git a/developing-with-nats/events/slow.md b/developing-with-nats/events/slow.md index 7dd837a..8cd8e6e 100644 --- a/developing-with-nats/events/slow.md +++ b/developing-with-nats/events/slow.md @@ -2,7 +2,7 @@ NATS is designed to move messages through the server quickly. As a result, NATS depends on the applications to consider and respond to changing message rates. The server will do a bit of impedance matching, but if a client is too slow the server will eventually cut them off by closing the connection. These cut off connections are called _slow consumers_. -One way some of the libraries deal with bursty message traffic is to buffer incoming messages for a subscription. So if an application can handle 10 messages per second and sometimes receives 20 messages per second, the library may hold the extra 10 to give the application time to catch up. To the server, the application will appear to be handling the messages and consider the connection healthy. It is up to the client library to decide what to do when the cache is too big, but most client libraries will drop incoming messages. +One way some of the libraries deal with bursty message traffic is to buffer incoming messages for a subscription. So if an application can handle 10 messages per second and sometimes receives 20 messages per second, the library may hold the extra 10 to give the application time to catch up. To the server, the application will appear to be handling the messages and consider the connection healthy. Most client libraries will notify the application that there is a SlowConsumer error and discard messages. Receiving and dropping messages from the server keeps the connection to the server healthy, but creates an application requirement. There are several common patterns: