Merge pull request #94 from nats-io/doctl-super-cluster

Create super-cluster-on-digital-ocean.md

nats-on-kubernetes/super-cluster-on-digital-ocean.md

@@ -0,0 +1,102 @@
+# Creating a NATS Super Cluster in Digital Ocean with Helm
+
+Let's create a super cluster using NATS Gateways.  First let's create
+3 different clusters in NYC, Amsterdam, and San Francisco:
+
+```sh
+doctl kubernetes cluster create nats-k8s-nyc1 --count 3 --region nyc1
+doctl kubernetes cluster create nats-k8s-sfo2 --count 3 --region sfo2
+doctl kubernetes cluster create nats-k8s-ams3 --count 3 --region ams3
+```
+
+Next, open up the firewall across the 3 regions to be able to access the client, leafnode and gateways ports:
+
+```sh
+for firewall in `doctl compute firewall list | tail -n 3 | awk '{print $1}'`; do
+  doctl compute firewall add-rules $firewall --inbound-rules protocol:tcp,ports:4222,address:0.0.0.0/0
+  doctl compute firewall add-rules $firewall --inbound-rules protocol:tcp,ports:7422,address:0.0.0.0/0
+  doctl compute firewall add-rules $firewall --inbound-rules protocol:tcp,ports:7522,address:0.0.0.0/0
+done
+```
+
+For this setup, we will create a super cluster using the external IPs
+from the nodes of the 3 clusters.  For a production type of setup, it
+is recommended to use a DNS entry and a A record for each one of the
+servers.
+
+```sh
+for ctx in do-ams3-nats-k8s-ams3 do-nyc1-nats-k8s-nyc1 do-sfo2-nats-k8s-sfo2; do
+  echo "name: $ctx"
+  for externalIP in `kubectl get nodes -o jsonpath='{.items[*].status.addresses[?(@.type=="ExternalIP")].address}'`; do 
+    echo "- nats://$externalIP:7522"; 
+  done
+  echo
+done
+```
+
+The Helm definition would look as follows for the 3 clusters:
+
+```yaml
+# super-cluster.yaml
+nats:
+  externalAccess: true
+  logging:
+    debug: false
+    trace: false
+
+cluster:
+  enabled: true
+
+gateway:
+  enabled: true
+
+  # NOTE: defined via --set gateway.name="$ctx"
+  # name: $ctx
+
+  gateways:
+  - name: do-ams3-nats-k8s-ams3
+    urls:
+    - nats://142.93.251.181:7522
+    - nats://161.35.12.245:7522
+    - nats://161.35.2.153:7522
+
+  - name: do-nyc1-nats-k8s-nyc1
+    urls:
+    - nats://142.93.251.181:7522
+    - nats://161.35.12.245:7522
+    - nats://161.35.2.153:7522
+
+  - name: do-sfo2-nats-k8s-sfo2
+    urls:
+    - nats://142.93.251.181:7522
+    - nats://161.35.12.245:7522
+    - nats://161.35.2.153:7522
+
+natsbox:
+  enabled: true
+```
+
+Let's deploy the super cluster with Helm using the name of cluster as the name of the gateway:
+
+```sh
+for ctx in do-ams3-nats-k8s-ams3 do-nyc1-nats-k8s-nyc1 do-sfo2-nats-k8s-sfo2; do
+  helm --kube-context $ctx install nats nats/nats -f super-cluster.yaml --set gateway.name=$ctx
+done
+```
+
+That's it! It should be now possible to send some messages across regions:
+
+```sh
+# Start subscription in Amsterdam
+nats-box:~# kubectl --context do-ams3-nats-k8s-ams3 exec -it nats-box -- /bin/sh -l
+nats-box:~# nats-sub -s nats hello
+
+# Send messages from San Francisco region
+nats-box:~# kubectl --context do-sfo2-nats-k8s-sfo2 exec -it nats-box -- /bin/sh -l
+nats-box:~# nats-pub -s nats hello 'Hello World!'
+
+# From outside of k8s can use the external IPs
+$ nats-sub -s 142.93.251.181 hello
+$ nats-pub -s 161.35.2.153 hello 'Hello World!'
+```
+