adding JetStream Docs

2025-01-18 04:03:23 -08:00 · 2020-11-24 15:12:44 -06:00
parent 8eb1e6a171
commit 1fd9cab1a5
19 changed files with 2109 additions and 0 deletions
--- a/jetstream/configuration_mgmt/configuration_mgmt.md
+++ b/jetstream/configuration_mgmt/configuration_mgmt.md
@@ -0,0 +1,105 @@
+## Configuration Management
+
+In many cases managing the configuration in your application code is the best model, many teams though wish to pre-create Streams and Consumers. 
+
+We support a number of tools to assist with this:
+
+ * `nats` CLI with configuration files 
+ * [Terraform](https://www.terraform.io/)
+ * [GitHub Actions](https://github.com/features/actions)
+ * [Kubernetes JetStream Controller](https://github.com/nats-io/nack#jetstream-controller) 
+
+### nats Admin CLI
+
+The `nats` CLI can be used to manage Streams and Consumers easily using it's `--config` flag, for example:
+
+### Add a new Stream
+
+This creates a new Stream based on `orders.json`. The `orders.json` file can be extracted from an existing stream using `nats stream info ORDERS -j | jq .config`
+
+```
+$ nats str add ORDERS --config orders.json
+```
+
+### Edit an existing Stream
+
+This edits an existing stream ensuring it complies with the configuration in `orders.json`
+```
+$ nats str edit ORDERS --config orders.json
+```
+
+### Add a New Consumer
+
+This creates a new Consumer based on `orders_new.json`. The `orders_new.json` file can be extracted from an existing stream using `nats con info ORDERS NEW -j | jq .config`
+
+```
+$ nats con add ORDERS NEW --config orders_new.json
+```
+
+### Terraform
+
+Terraform is a Cloud configuration tool from Hashicorp found at [terraform.io](https://www.terraform.io/), we maintain a Provider for Terraform called [terraform-provider-jetstream](https://github.com/nats-io/terraform-provider-jetstream/) that can maintain JetStream using Terraform.
+
+#### Setup
+
+Our provider is not hosted by Hashicorp so installation is a bit more complex than typical. Browse to the [Release Page](https://github.com/nats-io/terraform-provider-jetstream/releases) and download the release for your platform and extract it into your Terraform plugins directory.
+
+```
+$ unzip -l terraform-provider-jetstream_0.0.2_darwin_amd64.zip
+Archive:  terraform-provider-jetstream_0.0.2_darwin_amd64.zip
+  Length      Date    Time    Name
+---------  ---------- -----   ----
+    11357  03-09-2020 10:48   LICENSE
+     1830  03-09-2020 12:53   README.md
+ 24574336  03-09-2020 12:54   terraform-provider-jetstream_v0.0.2
+```
+
+Place the `terraform-provider-jetstream_v0.0.2` file in `~/.terraform.d/plugins/terraform-provider-jetstream_v0.0.2`
+
+In your project you can configure the Provider like this:
+
+```terraform
+provider "jetstream" {
+  servers = "connect.ngs.global"
+  credentials = "ngs_jetstream_admin.creds"
+}
+```
+
+And start using it, here's an example that create the `ORDERS` example. Review the [Project README](https://github.com/nats-io/terraform-provider-jetstream#readme) for full details.
+
+```terraform
+resource "jetstream_stream" "ORDERS" {
+  name     = "ORDERS"
+  subjects = ["ORDERS.*"]
+  storage  = "file"
+  max_age  = 60 * 60 * 24 * 365
+}
+
+resource "jetstream_consumer" "ORDERS_NEW" {
+  stream_id      = jetstream_stream.ORDERS.id
+  durable_name   = "NEW"
+  deliver_all    = true
+  filter_subject = "ORDERS.received"
+  sample_freq    = 100
+}
+
+resource "jetstream_consumer" "ORDERS_DISPATCH" {
+  stream_id      = jetstream_stream.ORDERS.id
+  durable_name   = "DISPATCH"
+  deliver_all    = true
+  filter_subject = "ORDERS.processed"
+  sample_freq    = 100
+}
+
+resource "jetstream_consumer" "ORDERS_MONITOR" {
+  stream_id        = jetstream_stream.ORDERS.id
+  durable_name     = "MONITOR"
+  deliver_last     = true
+  ack_policy       = "none"
+  delivery_subject = "monitor.ORDERS"
+}
+
+output "ORDERS_SUBJECTS" {
+  value = jetstream_stream.ORDERS.subjects
+}
+```
--- a/jetstream/configuration_mgmt/github_actions.md
+++ b/jetstream/configuration_mgmt/github_actions.md
@@ -0,0 +1,55 @@
+### GitHub Actions
+
+We have a pack of GitHub Actions that let you manage an already running JetStream Server, useful for managing releases or standing up test infrastructure.
+
+Full details and examples are in the [jetstream-gh-actions](https://github.com/nats-io/jetstream-gh-action) repository, here's an example.
+
+```yaml
+on: push
+name: orders
+jobs:
+
+  # First we delete the ORDERS stream and consumer if they already exist
+  clean_orders:
+    runs-on: ubuntu-latest
+    steps:
+      - name: orders_stream
+        uses: nats-io/jetstream-gh-action/delete/stream@master
+        with:
+          missing_ok: 1
+          stream: ORDERS
+          server: js.example.net
+
+  # Now we create the Stream and Consumers using the same configuration files the 
+  # nats CLI utility would use as shown above
+  create_orders:
+    runs-on: ubuntu-latest
+    needs: clean_orders
+    steps:
+      - uses: actions/checkout@master
+      - name: orders_stream
+        uses: nats-io/jetstream-gh-action/create/stream@master
+        with:
+          config: ORDERS.json
+          server: js.example.net
+      - name: orders_new_consumer
+        uses: nats-io/jetstream-gh-action/create/consumer@master
+        with:
+          config: ORDERS_NEW.json
+          stream: ORDERS
+          server: js.example.net
+
+  # We publish a message to a specific Subject, perhaps some consumer is 
+  # waiting there for it to kick off tests
+  publish_message:
+    runs-on: ubuntu-latest
+    needs: create_orders
+    steps:
+      - uses: actions/checkout@master
+      - name: orders_new_consumer
+        uses: nats-io/jetstream-gh-action@master
+        with:
+          subject: ORDERS.deployment
+          message: Published new deployment via "${{ github.event_name }}" in "${{ github.repository }}"
+          server: js.example.net
+```
--- a/jetstream/configuration_mgmt/kubernetes_controller.md
+++ b/jetstream/configuration_mgmt/kubernetes_controller.md
@@ -0,0 +1,57 @@
+### Kubernetes JetStream Controller
+
+The JetStream controllers allows you to manage NATS JetStream Streams and Consumers via K8S CRDs. You can find more info on how to deploy and usage [here](https://github.com/nats-io/nack#getting-started).  Below you can find an example on how to create a stream and a couple of consumers:
+
+```yaml
+---
+apiVersion: jetstream.nats.io/v1beta1
+kind: Stream
+metadata:
+  name: mystream
+spec:
+  name: mystream
+  subjects: ["orders.*"]
+  storage: memory
+  maxAge: 1h
+---
+apiVersion: jetstream.nats.io/v1beta1
+kind: Consumer
+metadata:
+  name: my-push-consumer
+spec:
+  streamName: mystream
+  durableName: my-push-consumer
+  deliverSubject: my-push-consumer.orders
+  deliverPolicy: last
+  ackPolicy: none
+  replayPolicy: instant
+---
+apiVersion: jetstream.nats.io/v1beta1
+kind: Consumer
+metadata:
+  name: my-pull-consumer
+spec:
+  streamName: mystream
+  durableName: my-pull-consumer
+  deliverPolicy: all
+  filterSubject: orders.received
+  maxDeliver: 20
+  ackPolicy: explicit
+```
+
+Once the CRDs are installed you can use `kubectl` to manage the streams and consumers as follows:
+
+```sh
+$ kubectl get streams
+NAME       STATE     STREAM NAME   SUBJECTS
+mystream   Created   mystream      [orders.*]
+
+$ kubectl get consumers
+NAME               STATE     STREAM     CONSUMER           ACK POLICY
+my-pull-consumer   Created   mystream   my-pull-consumer   explicit
+my-push-consumer   Created   mystream   my-push-consumer   none
+
+# If you end up in an Errored state, run kubectl describe for more info.
+#     kubectl describe streams mystream
+#     kubectl describe consumers my-pull-consumer
+```