taigrr/nats.docs
1
0
Fork 0
mirror of https://github.com/taigrr/nats.docs synced 2025-01-18 04:03:23 -08:00
Code Issues Projects Releases Wiki Activity

adding gists

Browse Source
This commit is contained in:
ainsley 2019-12-20 16:28:34 -06:00
parent 44d8ce8907
commit 0f8d6c61b9
3 changed files with 336 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

202
nats-server/configuration/securing_nats/auth_intro/nkey_auth.md
View File

@ -59,3 +59,205 @@ const nc = NATS.connect({
The client provides a function that it uses to parse the seed \(the private key\) and sign the connection challenge. The client provides a function that it uses to parse the seed \(the private key\) and sign the connection challenge.
## Mixing NKEYS and Trusted Operator setup in NATS v2
Mixing both nkeys static config and trusted operator setup (decentralized auth model) is possible
but needs some preparation in order to be able to do it.
The way this can be done is by **first** preparing a basic trusted operator setup
that could be used in the future, and then base from that configuration to create the
NKEYS static config using the same shared public nkeys for the accounts and then use
clustering routes to bridge the two different auth setups during the transition.
For example, creating the following initial setup using [NSC](/nsc/readme.md):
```sh
nsc add account --name SYS
nsc add user --name sys
nsc add account --name A
nsc add user -a A --name test
nsc add account --name B
nsc add user -a B --name test
```
This will then generate something like the following:
```sh
nsc list accounts
╭─────────────────────────────────────────────────────────────────╮
│ Accounts │
├──────┬──────────────────────────────────────────────────────────┤
│ Name │ Public Key │
├──────┼──────────────────────────────────────────────────────────┤
│ A │ ADFB2JXYTXOJEL6LNAXDREUGRX35BOLZI3B4PFFAC7IRPR3OA4QNKBN2 │
│ B │ ACWOMQA7PZTKJSBTR7BF6TBK3D776734PWHWDKO7HFMQOM5BIOYPSYZZ │
│ SYS │ ABKOWIYVTHNEK5HELPWLAT2CF2CUPELIK4SZH2VCJHLFU22B5U2IIZUO │
╰──────┴──────────────────────────────────────────────────────────╯
nsc list users -a A
╭─────────────────────────────────────────────────────────────────╮
│ Users │
├──────┬──────────────────────────────────────────────────────────┤
│ Name │ Public Key │
├──────┼──────────────────────────────────────────────────────────┤
│ test │ UAPOK2P7EN3UFBL7SBJPQK3M3JMLALYRYKX5XWSVMVYK63ZMBHTOHVJR │
╰──────┴──────────────────────────────────────────────────────────╯
```
We could use this configuration as the initial starting configuration for an nkeys config now,
where all the NKEYS users public nkeys are explicitly listed (centralized auth model).
```hcl
port = 4222
cluster {
port = 6222
# We will bridge two different servers with different auth models via routes
# routes [ nats://127.0.0.1:6223 ]
}
system_account = ABKOWIYVTHNEK5HELPWLAT2CF2CUPELIK4SZH2VCJHLFU22B5U2IIZUO
accounts {
# Account A
ADFB2JXYTXOJEL6LNAXDREUGRX35BOLZI3B4PFFAC7IRPR3OA4QNKBN2 {
nkey: ADFB2JXYTXOJEL6LNAXDREUGRX35BOLZI3B4PFFAC7IRPR3OA4QNKBN2
users = [
{nkey: "UAPOK2P7EN3UFBL7SBJPQK3M3JMLALYRYKX5XWSVMVYK63ZMBHTOHVJR" }
]
}
# Account B
ACWOMQA7PZTKJSBTR7BF6TBK3D776734PWHWDKO7HFMQOM5BIOYPSYZZ {
}
# Account SYS
ABKOWIYVTHNEK5HELPWLAT2CF2CUPELIK4SZH2VCJHLFU22B5U2IIZUO {
}
}
```
By using `nsc` it is possible to create a mem based resolver for the trusted operator setup:
```
nsc generate config --mem-resolver --sys-account SYS
```
An example configuration from the second node with the trusted operator setup could then be:
```hcl
port = 4223
cluster {
port = 6223
routes [ nats://127.0.0.1:6222 ]
}
# debug = true
# trace = true
# Operator
operator = eyJ0eXAiOiJqd3QiLCJhbGciOiJlZDI1NTE5In0.eyJqdGkiOiJQNDJBSkFTVVA0TUdGRU1EQzVCRVVGUkM1MlQ1M05OTzRIWkhRNEdETVk0S0RZTFVRV0JBIiwiaWF0IjoxNTc0Mzc1OTE2LCJpc3MiOiJPQ09KSk5aSUNINkNHUU5LM1NRMzVXTFpXWkpDUkRBTFJIWjZPVzQ0RFpZVVdNVVYzV1BSSEZSRCIsIm5hbWUiOiJLTyIsInN1YiI6Ik9DT0pKTlpJQ0g2Q0dRTkszU1EzNVdMWldaSkNSREFMUkhaNk9XNDREWllVV01VVjNXUFJIRlJEIiwidHlwZSI6Im9wZXJhdG9yIiwibmF0cyI6e319.pppa9-xhWXJLSCCtqj_dqlvXKR7WlVCh0cqoZ6lr8zg3WlWM8U0bNf6FHw_67-wRS7jj0n4PuA0P0MAJdE3pDA
system_account = ABKOWIYVTHNEK5HELPWLAT2CF2CUPELIK4SZH2VCJHLFU22B5U2IIZUO
resolver = MEMORY
resolver_preload = {
# Account "A"
ADFB2JXYTXOJEL6LNAXDREUGRX35BOLZI3B4PFFAC7IRPR3OA4QNKBN2: eyJ0eXAiOiJqd3QiLCJhbGciOiJlZDI1NTE5In0.eyJqdGkiOiIyQjNYNUJPQzQ3M05TU0hFWUVHTzJYRUhaTVNGRVBORFFBREJXWkJLRVdQVlg2TUlZU1JRIiwiaWF0IjoxNTc0Mzc1OTE2LCJpc3MiOiJPQ09KSk5aSUNINkNHUU5LM1NRMzVXTFpXWkpDUkRBTFJIWjZPVzQ0RFpZVVdNVVYzV1BSSEZSRCIsIm5hbWUiOiJBIiwic3ViIjoiQURGQjJKWFlUWE9KRUw2TE5BWERSRVVHUlgzNUJPTFpJM0I0UEZGQUM3SVJQUjNPQTRRTktCTjIiLCJ0eXBlIjoiYWNjb3VudCIsIm5hdHMiOnsiZXhwb3J0cyI6W3sibmFtZSI6InRlc3QiLCJzdWJqZWN0IjoidGVzdCIsInR5cGUiOiJzZXJ2aWNlIiwic2VydmljZV9sYXRlbmN5Ijp7InNhbXBsaW5nIjoxMDAsInJlc3VsdHMiOiJsYXRlbmN5Lm9uLnRlc3QifX1dLCJsaW1pdHMiOnsic3VicyI6LTEsImNvbm4iOi0xLCJsZWFmIjotMSwiaW1wb3J0cyI6LTEsImV4cG9ydHMiOi0xLCJkYXRhIjotMSwicGF5bG9hZCI6LTEsIndpbGRjYXJkcyI6dHJ1ZX19fQ.zZBetgDN6nCFDVpwzF_124BPkc8amGPDnrOmiKUa12xski5zskUI0Y0OeIa1vTo0bkHIKTgM2QDYpmXUQOHnAQ
# Account "B"
ACWOMQA7PZTKJSBTR7BF6TBK3D776734PWHWDKO7HFMQOM5BIOYPSYZZ: eyJ0eXAiOiJqd3QiLCJhbGciOiJlZDI1NTE5In0.eyJqdGkiOiJBTFNFQkZGWDZMR0pQTlVMU1NXTDNTRTNISkk2WUZSWlVKSDNLV0E1VE41WUtWRE5MVTJRIiwiaWF0IjoxNTc0Mzc1OTE2LCJpc3MiOiJPQ09KSk5aSUNINkNHUU5LM1NRMzVXTFpXWkpDUkRBTFJIWjZPVzQ0RFpZVVdNVVYzV1BSSEZSRCIsIm5hbWUiOiJCIiwic3ViIjoiQUNXT01RQTdQWlRLSlNCVFI3QkY2VEJLM0Q3NzY3MzRQV0hXREtPN0hGTVFPTTVCSU9ZUFNZWloiLCJ0eXBlIjoiYWNjb3VudCIsIm5hdHMiOnsiaW1wb3J0cyI6W3sibmFtZSI6InRlc3QiLCJzdWJqZWN0IjoidGVzdCIsImFjY291bnQiOiJBREZCMkpYWVRYT0pFTDZMTkFYRFJFVUdSWDM1Qk9MWkkzQjRQRkZBQzdJUlBSM09BNFFOS0JOMiIsInRvIjoidGVzdCIsInR5cGUiOiJzZXJ2aWNlIn1dLCJsaW1pdHMiOnsic3VicyI6LTEsImNvbm4iOi0xLCJsZWFmIjotMSwiaW1wb3J0cyI6LTEsImV4cG9ydHMiOi0xLCJkYXRhIjotMSwicGF5bG9hZCI6LTEsIndpbGRjYXJkcyI6dHJ1ZX19fQ.AnzziBwt5Tnphc2prONUUOpMpkkAlJHvCPaag0GUtTYPCHKDphcJrwtAHi4v5NOI6npjoes0F0MlrfnHqidDAg
# Account "SYS"
ABKOWIYVTHNEK5HELPWLAT2CF2CUPELIK4SZH2VCJHLFU22B5U2IIZUO: eyJ0eXAiOiJqd3QiLCJhbGciOiJlZDI1NTE5In0.eyJqdGkiOiI1WVUyWkc1UkRTSU1TN1pGVE1MU0NZQUtLVkVFWUpPUlc0TDJPTlY3N1g1TlJZWkFGSkRRIiwiaWF0IjoxNTc0Mzc1OTE2LCJpc3MiOiJPQ09KSk5aSUNINkNHUU5LM1NRMzVXTFpXWkpDUkRBTFJIWjZPVzQ0RFpZVVdNVVYzV1BSSEZSRCIsIm5hbWUiOiJTWVMiLCJzdWIiOiJBQktPV0lZVlRITkVLNUhFTFBXTEFUMkNGMkNVUEVMSUs0U1pIMlZDSkhMRlUyMkI1VTJJSVpVTyIsInR5cGUiOiJhY2NvdW50IiwibmF0cyI6eyJsaW1pdHMiOnsic3VicyI6LTEsImNvbm4iOi0xLCJsZWFmIjotMSwiaW1wb3J0cyI6LTEsImV4cG9ydHMiOi0xLCJkYXRhIjotMSwicGF5bG9hZCI6LTEsIndpbGRjYXJkcyI6dHJ1ZX19fQ.5FrO4sZbWuFgRLuy7c1eQLUq_BQ4PNhIAN5A-sRLkYWmvlc4c_Y4VfTbgl5zhNzCxfvj9SxT7ySgphup2BiRAA
}
```
**These two servers even though they have different auth mechanisms are able to route the accounts because the account public nkeys are the same.**
We have created at least one user, in this case with creds:
```conf
-----BEGIN NATS USER JWT-----
eyJ0eXAiOiJqd3QiLCJhbGciOiJlZDI1NTE5In0.eyJqdGkiOiJNRkM3V1E1N0hKUE9aWUVOTEhVRTZTWFVQTDVKTURWSkxIQzJRTkpYNUVJS0RGR1U1REhRIiwiaWF0IjoxNTc0Mzc1OTE2LCJpc3MiOiJBREZCMkpYWVRYT0pFTDZMTkFYRFJFVUdSWDM1Qk9MWkkzQjRQRkZBQzdJUlBSM09BNFFOS0JOMiIsIm5hbWUiOiJ0ZXN0Iiwic3ViIjoiVUFQT0syUDdFTjNVRkJMN1NCSlBRSzNNM0pNTEFMWVJZS1g1WFdTVk1WWUs2M1pNQkhUT0hWSlIiLCJ0eXBlIjoidXNlciIsIm5hdHMiOnsicHViIjp7ImFsbG93IjpbIl9JTkJPWC5cdTAwM2UiLCJfUl8iLCJfUl8uXHUwMDNlIiwidGVzdCIsInRlc3QuXHUwMDNlIl19LCJzdWIiOnsiYWxsb3ciOlsiX0lOQk9YLlx1MDAzZSIsIl9SXyIsIl9SXy5cdTAwM2UiLCJsYXRlbmN5Lm9uLnRlc3QiLCJ0ZXN0IiwidGVzdC5cdTAwM2UiXX19fQ.MSU2aUIBK1iUsg7h52lLrfEfTwVMF_wB3HDq75ECskxSyyDDMtk9_3957UtQF-3yoGCIhKOkWjzX8C-WXnLADw
------END NATS USER JWT------
************************* IMPORTANT *************************
NKEY Seed printed below can be used to sign and prove identity.
NKEYs are sensitive and should be treated as secrets.
-----BEGIN USER NKEY SEED-----
SUANVBWRHHFMGHNIT6UJHPN2TGVBVIILE7VPVNEQ7DGCJ26ZD2V3KAHT4M
------END USER NKEY SEED------
*************************************************************
```
And this same user is able to connect to either one of the servers (bound to 4222 and 4223 respectively):
Subscriber Service:
```go
package main
import (
"log"
"github.com/nats-io/nats.go"
)
func main() {
opts := make([]nats.Option, 0)
// Extract public nkey from seed
//
// Public: UAPOK2P7EN3UFBL7SBJPQK3M3JMLALYRYKX5XWSVMVYK63ZMBHTOHVJR
// Private: SUANVBWRHHFMGHNIT6UJHPN2TGVBVIILE7VPVNEQ7DGCJ26ZD2V3KAHT4M
//
nkey, err := nats.NkeyOptionFromSeed("path/to/seed.nkey")
if err != nil {
log.Fatal(err)
}
opts = append(opts, nkey)
nc, err := nats.Connect("127.0.0.1:4222", opts...)
if err != nil {
log.Fatal(err)
}
nc.Subscribe("test", func(m *nats.Msg){
log.Printf("[Received] %q, replying... \n", string(m.Data))
m.Respond([]byte("pong from nkeys based server"))
})
select {}
}
```
Requestor:
```go
package main
import (
"log"
"time"
"github.com/nats-io/nats.go"
)
func main() {
nc, err := nats.Connect("127.0.0.1:4223", nats.UserCredentials("path/to/user.creds"))
if err != nil {
log.Fatal(err)
}
for range time.NewTicker(1 * time.Second).C {
resp, err := nc.Request("test", []byte("test"), 1*time.Second)
if err != nil {
log.Println("[Error]", err)
continue
}
log.Println("[Received]", string(resp.Data))
}
}
```

102
nats-server/installation.md
View File

@ -103,3 +103,105 @@ To test your installation \(provided the $GOPATH/bin is set\):
[41634] 2019/05/13 09:42:11.746252 [INF] Server is ready [41634] 2019/05/13 09:42:11.746252 [INF] Server is ready
``` ```
## NATS v2 and Go Modules
If you are having issues when using the recent versions of NATS and Go modules such as:
```
go: github.com/nats-io/go-nats@v1.8.1: parsing go.mod: unexpected module path "github.com/nats-io/nats.go"
go: github.com/nats-io/go-nats-streaming@v0.5.0: parsing go.mod: unexpected module path "github.com/nats-io/stan.go"
```
To fix it:
1) Update your `go.mod` using the latest tags, for example for both NATS and NATS Streaming clients:
```ruby
module github.com/wallyqs/hello-nats-go-mod
go 1.12
require (
github.com/nats-io/nats.go v1.8.1
github.com/nats-io/stan.go v0.5.0
)
```
Or if you want to import the NATS Server v2 to embed it, notice the `/v2` after the nats-server module name. If that is not present, then go modules will not fetch it and would accidentally end up with 1.4.1 version of the server.
```ruby
require (
github.com/nats-io/nats-server/v2 v2.0.0
github.com/nats-io/nats.go v1.8.1
)
```
If embedding both NATS Streaming and NATS Servers:
```ruby
require (
github.com/nats-io/nats-server/v2 v2.0.0 // indirect
github.com/nats-io/nats-streaming-server v0.15.1
)
```
2) Next, update the imports within the repo:
```sh
find ./ -type f -name "*.go" -exec sed -i -e 's/github.com\/nats-io\/go-nats-streaming/github.com\/nats-io\/stan.go/g' {} \;
find ./ -type f -name "*.go" -exec sed -i -e 's/github.com\/nats-io\/go-nats/github.com\/nats-io\/nats.go/g' {} \;
find ./ -type f -name "*.go" -exec sed -i -e 's/github.com\/nats-io\/gnatsd/github.com\/nats-io\/nats-server\/v2/g' {} \;
find ./ -type f -name "*.go" -exec sed -i -e 's/github.com\/nats-io\/nats-server/github.com\/nats-io\/nats-server\/v2/g' {} \;
```
3) (Recommended) Run Go fmt as the rename will affect the proper ordering of the imports
### Gotchas when using `go get`
If using `go get` for the client, then have to be careful with adding an extra slash at the end of the repo for example:
```
GO111MODULE=on go get github.com/nats-io/nats.go/@latest
GO111MODULE=on go get github.com/nats-io/nats.go/@v1.8.1
```
If trying to fetch the latest version of the server with `go get`, then have to add `v2` at the end:
```
GO111MODULE=on go get github.com/nats-io/nats-server/v2@latest
```
Otherwise, `go get` will fetch the `v1.4.1` version of the server, which is also named (`gnatsd`), the previous name for nats-server.
```
GO111MODULE=on go get github.com/nats-io/nats-server@latest
go: finding github.com/nats-io/gnatsd/server latest
go: finding golang.org/x/crypto/bcrypt latest
go: finding golang.org/x/crypto latest
```
In order to use an older tag, you will also have to use the previous name (gnatsd) otherwise it would results in `go mod` parsing errors.
```
# OK
GO111MODULE=on go get github.com/nats-io/go-nats/@v1.7.2
# Not OK
GO111MODULE=on go get github.com/nats-io/nats.go/@v1.7.2
go: finding github.com/nats-io/nats.go v1.7.2
go: downloading github.com/nats-io/nats.go v1.7.2
go: extracting github.com/nats-io/nats.go v1.7.2
go: finding github.com/nats-io/go-nats/encoders/builtin latest
go: finding github.com/nats-io/go-nats/util latest
go: finding github.com/nats-io/go-nats/encoders latest
go: finding github.com/nats-io/go-nats v1.8.1
go: downloading github.com/nats-io/go-nats v1.8.1
go: extracting github.com/nats-io/go-nats v1.8.1
go: github.com/nats-io/go-nats@v1.8.1: parsing go.mod: unexpected module path "github.com/nats-io/nats.go"
go: error loading module requirements
```
For more information you can review the original issue in [GitHub](https://github.com/nats-io/nats.go/issues/478).

32
nats-streaming-concepts/monitoring/README.md
View File

@ -65,3 +65,35 @@ curl 127.0.0.1:8222/streaming/channelsz
} }
``` ```
## Monitoring a NATS Streaming channel with Grafana and Prometheus
Here you can find a few examples of monitoring NATS Streaming channels:
### Pending Messages from Channel Foo
```
sum(nss_chan_subs_pending_count{channel="foo"}) by (client_id)
```
<img width="1580" alt="Pending" src="https://user-images.githubusercontent.com/26195/54960400-b0c52e80-4f19-11e9-9e92-88fba89fd55e.png">
### Messages Per Sec Delivered on Channel Foo
In this case, `3` is the size of the quorum of NATS Streaming Server nodes. In case of a single instance backed by a relational database we would set it to `1`:
```
sum(rate(nss_chan_msgs_total{channel="foo"}[5m])) by (channel) / 3
```
<img width="1579" alt="msgs-per-sec" src="https://user-images.githubusercontent.com/26195/54960588-80ca5b00-4f1a-11e9-92d5-de59c81b6c63.png">
### Msgs/Sec vs Pending on Channel
Example of combining the rate of messages with the pending count to detect whether processing is getting behind:
```
sum(rate(nss_chan_msgs_total{channel="foo"}[5m])) by (channel) / 3
sum(nss_chan_subs_pending_count{channel="foo"}) by (channel) / 3
```
<img width="1468" alt="combination" src="https://user-images.githubusercontent.com/26195/54960992-4235a000-4f1c-11e9-8e55-47515a5d944d.png">