NATS Controllers for Kubernetes (NACK)
The JetStream controllers allows you to manage NATS JetStream resources via Kubernetes CRDs.
NACK supports two controller modes with different capabilities:
| Mode | Streams | Consumers | Key/Value | Object Store | Accounts |
|---|---|---|---|---|---|
| Legacy (default) | ✅ | ✅ | ❌ | ❌ | ❌ |
Control-loop (--control-loop) |
âś… | âś… | âś… | âś… | âś… |
Important: Key/Value stores and Object stores are only supported in control-loop mode. If you create KeyValue or ObjectStore resources without enabling control-loop mode, they will not be reconciled.
Resources managed by NACK controllers are expected to exclusively be managed by NACK, and configuration state will be enforced if mutated by an external client.
Install with Helm:
helm repo add nats https://nats-io.github.io/k8s/helm/charts/
helm repo update
helm upgrade --install nats nats/nats \
--set config.jetstream.enabled=true \
--set config.jetstream.memoryStore.enabled=true \
--set config.cluster.enabled=true --wait
helm upgrade --install nack nats/nack \
--set jetstream.nats.url=nats://nats.default.svc.cluster.local:4222 --wait
Note: The updated controllers will more reliably enforce resource state. If migrating from an older version of NACK, as long as all NATS resources are in-sync with NACK resources no modifications are expected.
The
jetstream-controllerlogs will contain a diff of any changes the controller has made.
helm upgrade nack nats/nack \
--set jetstream.nats.url=nats://nats.default.svc.cluster.local:4222 \
--set jetstream.additionalArgs={--control-loop} --wait
The are several approaches for managing multiple NATS Systems with NACK within one Kubernetes cluster. These options are not mutually exclusive.
You can run multiple NACK controllers on the same Kubernetes cluster. Add --set config.namespaced=true to your install flags or set namespaced: true in your values.yaml. When set, the controller will only reconcile resources within its own namespace.
helm upgrade --install nack nats/nack \
--create-namespace --namespace nats \
--set namespaced=true \
--set jetstream.nats.url=nats://nats.nats.svc.cluster.local:4222 --waitThe Accounts resource acts as a connection config for other resources. You may define multiple accounts for the same, or for distinct, NATS Systems.
---
apiVersion: jetstream.nats.io/v1beta2
kind: Account
metadata:
name: a
spec:
name: a
creds:
secret:
name: account-a-creds
servers:
- nats://nats.nats-a.svc.cluster.local
---
apiVersion: jetstream.nats.io/v1beta2
kind: Account
metadata:
name: b
spec:
name: b
creds:
secret:
name: account-b-creds
servers:
- nats://nats.nats-b.svc.cluster.local
---
apiVersion: jetstream.nats.io/v1beta2
kind: Stream
metadata:
name: foo-a
spec:
name: foo
subjects: ["foo", "foo.>"]
storage: file
replicas: 3
maxAge: 1h
account: a
---
apiVersion: jetstream.nats.io/v1beta2
kind: Stream
metadata:
name: foo-b
spec:
name: foo
subjects: ["foo", "foo.>"]
storage: file
replicas: 3
maxAge: 1h
account: bThe above manifests will define two Account resources, each pulling credentials from a Kubernetes secret. Account a is configured to use the NATS Cluster in namspace nats-a and Account b is configured to use the NATS Cluster in namespace nats-b. The NATS clusters do not need to be in Kubernetes, this is just an example.
This will also create an identical stream, foo, in each cluster. Note: The resource names, foo-a and foo-b, must be distinct to not conflict as Kubernetes resources, but the stream names themselves are both foo.
See more details in the Getting Started with Accounts section.
You may define some connection options within the resource manifests directly. If not running in the newer --control-loop mode, set --crd-connect.
If running with --control-loop, resource-level connection config will always override any global config.
helm upgrade nack nats/nack \
--set jetstream.additionalArgs={--crd-connect} --wait // Not required if running with `--control-loop`apiVersion: jetstream.nats.io/v1beta2
kind: Stream
metadata:
name: bar
spec:
name: bar
subjects: ["bar", "bar.>"]
storage: file
replicas: 3
maxAge: 1h
servers:
- nats://nats.nats.svc.cluster.local:4222Let's create a a stream and a couple of consumers:
---
apiVersion: jetstream.nats.io/v1beta2
kind: Stream
metadata:
name: mystream
spec:
name: mystream
subjects: ["orders.*"]
storage: memory
maxAge: 1h
---
apiVersion: jetstream.nats.io/v1beta2
kind: Consumer
metadata:
name: my-push-consumer
spec:
streamName: mystream
durableName: my-push-consumer
deliverSubject: my-push-consumer.orders
deliverPolicy: last
ackPolicy: none
replayPolicy: instant
---
apiVersion: jetstream.nats.io/v1beta2
kind: Consumer
metadata:
name: my-pull-consumer
spec:
streamName: mystream
durableName: my-pull-consumer
deliverPolicy: all
filterSubject: orders.received
maxDeliver: 20
ackPolicy: explicit
---
# Note: KeyValue requires control-loop mode to be enabled
apiVersion: jetstream.nats.io/v1beta2
kind: KeyValue
metadata:
name: my-key-value
spec:
bucket: my-key-value
history: 20
storage: file
maxBytes: 2048
compression: true
---
# Note: ObjectStore requires control-loop mode to be enabled
apiVersion: jetstream.nats.io/v1beta2
kind: ObjectStore
metadata:
name: my-object-store
spec:
bucket: my-object-store
storage: file
replicas: 1
maxBytes: 536870912 # 512 MB
compression: true# Create a stream.
$ kubectl apply -f https://raw.githubusercontent.com/nats-io/nack/main/deploy/examples/stream.yml
# Check if it was successfully created.
$ kubectl get streams
NAME STATE STREAM NAME SUBJECTS
mystream Ready mystream [orders.*]
# Create a push-based consumer
$ kubectl apply -f https://raw.githubusercontent.com/nats-io/nack/main/deploy/examples/consumer_push.yml
# Create a pull based consumer
$ kubectl apply -f https://raw.githubusercontent.com/nats-io/nack/main/deploy/examples/consumer_pull.yml
# Check if they were successfully created.
$ kubectl get consumers
NAME STATE STREAM CONSUMER ACK POLICY
my-pull-consumer Ready mystream my-pull-consumer explicit
my-push-consumer Ready mystream my-push-consumer none
# If you end up in an Errored state, run kubectl describe for more info.
# kubectl describe streams mystream
# kubectl describe consumers my-pull-consumerNow we're ready to use Streams and Consumers. Let's start off with writing some
data into mystream.
# Run nats-box that includes the NATS management utilities, and exec into it.
$ kubectl exec -it deployment/nats-box -- /bin/sh -l
# Publish a couple of messages from nats-box
nats-box:~$ nats pub orders.received "order 1"
nats-box:~$ nats pub orders.received "order 2"First, we'll read the data using a pull-based consumer.
From the above my-pull-consumer Consumer CRD, we have set the filterSubject
of orders.received. You can double check with the following command:
$ kubectl get consumer my-pull-consumer -o jsonpath={.spec.filterSubject}
orders.receivedSo that's the subject my-pull-consumer will pull messages from.
# Pull first message.
nats-box:~$ nats consumer next mystream my-pull-consumer
--- subject: orders.received / delivered: 1 / stream seq: 1 / consumer seq: 1
order 1
Acknowledged message
# Pull next message.
nats-box:~$ nats consumer next mystream my-pull-consumer
--- subject: orders.received / delivered: 1 / stream seq: 2 / consumer seq: 2
order 2
Acknowledged messageNext, let's read data using a push-based consumer.
From the above my-push-consumer Consumer CRD, we have set the deliverSubject
of my-push-consumer.orders, as you can confirm with the following command:
$ kubectl get consumer my-push-consumer -o jsonpath={.spec.deliverSubject}
my-push-consumer.ordersSo pushed messages will arrive on that subject. This time all messages arrive automatically.
nats-box:~$ nats sub my-push-consumer.orders
17:57:24 Subscribing on my-push-consumer.orders
[#1] Received JetStream message: consumer: mystream > my-push-consumer / subject: orders.received /
delivered: 1 / consumer seq: 1 / stream seq: 1 / ack: false
order 1
[#2] Received JetStream message: consumer: mystream > my-push-consumer / subject: orders.received /
delivered: 1 / consumer seq: 2 / stream seq: 2 / ack: false
order 2You can create an Account resource with the following CRD. The Account resource can be used to specify server and TLS information.
Note The
Accountresource does not create or manage NATS accounts. It functions as a connection and authentication config for the managed resources.
The nsc tool can be used to manage your NATS account configuration on the server-side. You can find more details about NATS decentralized auth in the docs.
---
apiVersion: jetstream.nats.io/v1beta2
kind: Account
metadata:
name: a
spec:
name: a
servers:
- nats://nats:4222
tls:
secret:
name: nack-a-tls
ca: "ca.crt"
cert: "tls.crt"
key: "tls.key"You can then link an Account to a Stream so that the Stream uses the Account information for its creation.
---
apiVersion: jetstream.nats.io/v1beta2
kind: Stream
metadata:
name: foo
spec:
name: foo
subjects: ["foo", "foo.>"]
storage: file
replicas: 1
account: a # <-- Create stream using account A informationThe following is an example of how to get Accounts working with a custom NATS Server URL and TLS certificates.
# Install cert-manager
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.17.0/cert-manager.yaml
# Install TLS certs
cd examples/secure
# Install certificate issuer
kubectl apply -f issuer.yaml
# Install account A cert
kubectl apply -f nack-a-client-tls.yaml
# Install server cert
kubectl apply -f server-tls.yaml
# Install nats-box cert
kubectl apply -f client-tls.yaml
# Install NATS cluster
helm upgrade --install -f nats-helm.yaml nats nats/nats
# Verify pods are healthy
kubectl get pods
# Install JetStream Controller from nack
helm upgrade --install nack nats/nack --set jetstream.enabled=true
# Verify pods are healthy
kubectl get pods
# Create account A resource
kubectl apply -f nack/nats-account-a.yaml
# Create stream using account A
kubectl apply -f nack/nats-stream-foo-a.yaml
# Create consumer using account A
kubectl apply -f nack/nats-consumer-bar-a.yamlAfter Accounts, Streams, and Consumers are created, let's log into the nats-box container to run the management CLI.
# Get container shell
kubectl exec -it deployment/nats-box -- /bin/sh -lThere should now be some Streams available, verify with nats command.
# List streams
nats stream lsYou can now publish messages on a Stream.
# Push message
nats pub foo hiAnd pull messages from a Consumer.
# Pull message
nats consumer next foo bar# First, build the jetstream controller.
make jetstream-controller
# Next, run the controller like this
./jetstream-controller -kubeconfig ~/.kube/config -s nats://localhost:4222
# Pro tip: jetstream-controller uses klog just like kubectl or kube-apiserver.
# This means you can change the verbosity of logs with the -v flag.
#
# For example, this prints raw HTTP requests and responses.
# ./jetstream-controller -v=10
# You'll probably want to start a local Jetstream-enabled NATS server, unless
# you use a public one.
nats-server -DV -jsBuild Docker image
make jetstream-controller-docker ver=1.2.3This is a sidecar that you can use to automatically reload your NATS Server configuration file.
For more information see the Chart repo.
helm repo add nats https://nats-io.github.io/k8s/helm/charts/
helm upgrade --install nats nats/nats
reloader:
enabled: true
image: natsio/nats-server-config-reloader:0.16.1
pullPolicy: IfNotPresent# First, build the config reloader.
make nats-server-config-reloader
# Next, run the reloader like this
./nats-server-config-reloaderBuild Docker image
make nats-server-config-reloader-docker ver=1.2.3For more information see the Chart repo.
helm repo add nats https://nats-io.github.io/k8s/helm/charts/
helm upgrade --install nats nats/nats
bootconfig:
image: natsio/nats-boot-config:0.16.1
pullPolicy: IfNotPresent# First, build the project.
make nats-boot-config
# Next, run the project like this
./nats-boot-configBuild Docker image
make nats-boot-config-docker ver=1.2.3