taigrr/nats.docs
1
0
Fork 0
mirror of https://github.com/taigrr/nats.docs synced 2025-01-18 04:03:23 -08:00
Code Issues Projects Releases Wiki Activity

Added custom dialer

Browse Source 
Removed benchmark tutorial (it is in tools now)
This commit is contained in:
Stephen Asbury 2019-05-15 14:13:03 -07:00
parent 42ebd158b8
commit b2d09e1a26
3 changed files with 133 additions and 152 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
SUMMARY.md
View File

@ -94,12 +94,10 @@
* [Explore NATS Pub/Sub](developer/tutorials/pubsub.md) * [Explore NATS Pub/Sub](developer/tutorials/pubsub.md)
* [Explore NATS Request/Reply](developer/tutorials/reqreply.md) * [Explore NATS Request/Reply](developer/tutorials/reqreply.md)
* [Explore NATS Queueing](developer/tutorials/queues.md) * [Explore NATS Queueing](developer/tutorials/queues.md)
* [Benchmarking NATS](developer/tutorials/benchmarking.md) * [Advanced Connect and Custom Dialer in Go](developer/tutorials/custom_dialer.md)
## NATS Protocol ## NATS Protocol
* [Protocol Demo](nats_protocol/nats-protocol-demo.md) * [Protocol Demo](nats_protocol/nats-protocol-demo.md)
* [Client Protocol](nats_protocol/nats-protocol.md) * [Client Protocol](nats_protocol/nats-protocol.md)
* [Developing a Client](nats_protocol/nats-client-dev.md) * [Developing a Client](nats_protocol/nats-client-dev.md)
* [NATS Cluster Protocol](nats_protocol/nats-server-protocol.md) * [NATS Cluster Protocol](nats_protocol/nats-server-protocol.md)

149
developer/tutorials/benchmarking.md
View File

@ -1,149 +0,0 @@
# Benchmarking NATS
NATS is fast and lightweight and places a priority on performance. NATS provides tools for measuring performance. In this tutorial you learn how to benchmark and tune NATS on your systems and environment.
## Prerequisites
Go and the NATS server should be installed.
## 1. Start the NATS server with monitoring enabled
```sh
% gnatsd -m 8222
```
Verify that the NATS server starts successfully, as well as the HTTP monitor:
```sh
[18541] 2016/10/31 13:26:32.037819 [INF] Starting nats-server version 0.9.4
[18541] 2016/10/31 13:26:32.037912 [INF] Starting http monitor on 0.0.0.0:8222
[18541] 2016/10/31 13:26:32.037997 [INF] Listening for client connections on 0.0.0.0:4222
[18541] 2016/10/31 13:26:32.038020 [INF] Server is ready
```
## 2. Installing and running the benchmark utility
The NATS benchmark can be installed and run via Go. Ensure your golang environment is setup.
There are two approaches; you can either install the `nats-bench` utility in the directory specified in your `GOBIN` environment variable:
```sh
% go install $GOPATH/src/github.com/nats-io/go-nats/examples/nats-bench.go
```
... or you can simply run it via `go run`:
```sh
% go run $GOPATH/src/github.com/nats-io/go-nats/examples/nats-bench.go
```
*On windows use the % environment variable syntax, replacing `$GOPATH` with `%GOPATH%`.*
For the purpose of this tutorial, we'll assume that you chose the first option, and that you've added the `GOBIN` environment variable to your `PATH`.
The `nats-bench` utility is straightforward to use. The options are as follows:
```sh
% nats-bench -h
Usage: nats-bench [-s server (nats://localhost:4222)] [--tls] [-np NUM_PUBLISHERS] [-ns NUM_SUBSCRIBERS] [-n NUM_MSGS] [-ms MESSAGE_SIZE] [-csv csvfile] <subject>
```
The options are self-explanatory. Each publisher or subscriber runs in its own go routine with its own NATS connection.
## 3. Run a publisher throughput test
Let's run a test to see how fast a single publisher can publish one million 16 byte messages to the NATS server.
```sh
% nats-bench -np 1 -n 100000 -ms 16 foo
```
The output tells you the number of messages and the number of payload bytes that the client was able to publish per second:
```sh
Starting benchmark [msgs=100000, msgsize=16, pubs=1, subs=0]
Pub stats: 7,055,644 msgs/sec ~ 107.66 MB/sec
```
Now increase the number of messages published:
```sh
% nats-bench -np 1 -n 10000000 -ms 16 foo
Starting benchmark [msgs=10000000, msgsize=16, pubs=1, subs=0]
Pub stats: 7,671,570 msgs/sec ~ 117.06 MB/sec
```
## 4. Run a publish/subscribe throughput test
When using both publishers and subscribers, `nats-bench` reports aggregate, as well as individual publish and subscribe throughput performance.
Let's look at throughput for a single publisher with a single subscriber:
```sh
% nats-bench -np 1 -ns 1 -n 100000 -ms 16 foo
```
Note that the output shows the aggregate throughput as well as the individual publisher and subscriber performance:
```sh
Starting benchmark [msgs=100000, msgsize=16, pubs=1, subs=1]
NATS Pub/Sub stats: 2,009,230 msgs/sec ~ 30.66 MB/sec
Pub stats: 1,076,537 msgs/sec ~ 16.43 MB/sec
Sub stats: 1,004,615 msgs/sec ~ 15.33 MB/sec
```
## 5. Run a 1:N throughput test
When specifying multiple publishers, or multiple subscribers, `nats-bench` will also report statistics for each publisher and subscriber individually, along with min/max/avg and standard deviation.
Let's increase both the number of messages, and the number of subscribers.:
```sh
% nats-bench -np 1 -ns 5 -n 10000000 -ms 16 foo
```
Output:
```sh
Starting benchmark [msgs=10000000, msgsize=16, pubs=1, subs=5]
NATS Pub/Sub stats: 5,730,851 msgs/sec ~ 87.45 MB/sec
Pub stats: 955,279 msgs/sec ~ 14.58 MB/sec
Sub stats: 4,775,709 msgs/sec ~ 72.87 MB/sec
[1] 955,157 msgs/sec ~ 14.57 MB/sec (10000000 msgs)
[2] 955,150 msgs/sec ~ 14.57 MB/sec (10000000 msgs)
[3] 955,157 msgs/sec ~ 14.57 MB/sec (10000000 msgs)
[4] 955,156 msgs/sec ~ 14.57 MB/sec (10000000 msgs)
[5] 955,153 msgs/sec ~ 14.57 MB/sec (10000000 msgs)
min 955,150 | avg 955,154 | max 955,157 | stddev 2 msgs
```
## 6. Run a N:M throughput test
When more than 1 publisher is specified, `nats-bench` evenly distributes the total number of messages (`-n`) across the number of publishers (`-np`).
Now let's increase the number of publishers and examine the output:
```sh
% nats-bench -np 5 -ns 5 -n 10000000 -ms 16 foo
```
The output:
```sh
Starting benchmark [msgs=10000000, msgsize=16, pubs=5, subs=5]
NATS Pub/Sub stats: 6,716,465 msgs/sec ~ 102.49 MB/sec
Pub stats: 1,119,653 msgs/sec ~ 17.08 MB/sec
[1] 226,395 msgs/sec ~ 3.45 MB/sec (2000000 msgs)
[2] 225,955 msgs/sec ~ 3.45 MB/sec (2000000 msgs)
[3] 225,889 msgs/sec ~ 3.45 MB/sec (2000000 msgs)
[4] 224,552 msgs/sec ~ 3.43 MB/sec (2000000 msgs)
[5] 223,933 msgs/sec ~ 3.42 MB/sec (2000000 msgs)
min 223,933 | avg 225,344 | max 226,395 | stddev 937 msgs
Sub stats: 5,597,054 msgs/sec ~ 85.40 MB/sec
[1] 1,119,461 msgs/sec ~ 17.08 MB/sec (10000000 msgs)
[2] 1,119,466 msgs/sec ~ 17.08 MB/sec (10000000 msgs)
[3] 1,119,444 msgs/sec ~ 17.08 MB/sec (10000000 msgs)
[4] 1,119,444 msgs/sec ~ 17.08 MB/sec (10000000 msgs)
[5] 1,119,430 msgs/sec ~ 17.08 MB/sec (10000000 msgs)
min 1,119,430 | avg 1,119,449 | max 1,119,466 | stddev 12 msgs
```

132
developer/tutorials/custom_dialer.md Normal file
View File

@ -0,0 +1,132 @@
# Advanced Connect and Custom Dialer in Go
The Go NATS client features a [CustomDialer](https://godoc.org/github.com/nats-io/go-nats#CustomDialer) option which allows you to customize
the connection logic against the NATS server without having to modify the internals
of the client. For example, let's say that you want to make the client use the `context`
package to use `DialContext` and be able to cancel connecting to NATS altogether with a deadline,
you could then do define a Dialer implementation as follows:
```go
package main
import (
"context"
"log"
"net"
"time"
"github.com/nats-io/go-nats"
)
type customDialer struct {
ctx context.Context
nc *nats.Conn
connectTimeout time.Duration
connectTimeWait time.Duration
}
func (cd *customDialer) Dial(network, address string) (net.Conn, error) {
ctx, cancel := context.WithTimeout(cd.ctx, cd.connectTimeout)
defer cancel()
for {
log.Println("Attempting to connect to", address)
if ctx.Err() != nil {
return nil, ctx.Err()
}
select {
case <-cd.ctx.Done():
return nil, cd.ctx.Err()
default:
d := &net.Dialer{}
if conn, err := d.DialContext(ctx, network, address); err == nil {
log.Println("Connected to NATS successfully")
return conn, nil
} else {
time.Sleep(cd.connectTimeWait)
}
}
}
}
```
With the dialer implementation above, the NATS client will retry a number of times to connect
to the NATS server until the context is no longer valid:
```go
func main() {
// Parent context cancels connecting/reconnecting altogether.
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
var err error
var nc *nats.Conn
cd := &customDialer{
ctx: ctx,
connectTimeout: 10 * time.Second,
connectTimeWait: 1 * time.Second,
}
opts := []nats.Option{
nats.SetCustomDialer(cd),
nats.ReconnectWait(2 * time.Second),
nats.ReconnectHandler(func(c *nats.Conn) {
log.Println("Reconnected to", c.ConnectedUrl())
}),
nats.DisconnectHandler(func(c *nats.Conn) {
log.Println("Disconnected from NATS")
}),
nats.ClosedHandler(func(c *nats.Conn) {
log.Println("NATS connection is closed.")
}),
nats.NoReconnect(),
}
go func() {
nc, err = nats.Connect("127.0.0.1:4222", opts...)
}()
WaitForEstablishedConnection:
for {
if err != nil {
log.Fatal(err)
}
// Wait for context to be canceled either by timeout
// or because of establishing a connection...
select {
case <-ctx.Done():
break WaitForEstablishedConnection
default:
}
if nc == nil || !nc.IsConnected() {
log.Println("Connection not ready")
time.Sleep(200 * time.Millisecond)
continue
}
break WaitForEstablishedConnection
}
if ctx.Err() != nil {
log.Fatal(ctx.Err())
}
for {
if nc.IsClosed() {
break
}
if err := nc.Publish("hello", []byte("world")); err != nil {
log.Println(err)
time.Sleep(1 * time.Second)
continue
}
log.Println("Published message")
time.Sleep(1 * time.Second)
}
// Disconnect and flush pending messages
if err := nc.Drain(); err != nil {
log.Println(err)
}
log.Println("Disconnected")
}
```