Doc 5692 2103x

antora.yml

@@ -19,6 +19,7 @@ asciidoc:
     pulsar: 'Apache Pulsar'
     pulsar-short: 'Pulsar'
     pulsar-beam: 'Pulsar Beam'
+    kaap-short: 'KAAP Operator'
     kaap: 'Kubernetes Autoscaling for Apache Pulsar (KAAP)'
     astra-stream: 'Astra Streaming'
     kafka-reg: 'Apache Kafka(R)'
@@ -34,10 +35,11 @@ asciidoc:
     pulsar-heartbeat-repo: 'https://github.com/datastax/pulsar-heartbeat'
     pulsar-sink-repo: 'https://github.com/datastax/pulsar-sink'
     pulsar-ansible-repo: 'https://github.com/datastax/pulsar-ansible'
-    pulsar-helm-chart-repo: 'https://github.com/datastax/pulsar-helm-chart'
+    pulsar-helm-chart-repo: 'https://github.com/datastax/pulsar-helm-chart' # deprecated, use kaap-operator-repo, kaap, and kaap-short instead
     pulsar-openid-connect-repo: 'https://github.com/datastax/pulsar-openid-connect-plugin'
     pulsar-repo: 'https://github.com/datastax/pulsar'
     pulsar-beam-repo: 'https://github.com/kafkaesque-io/pulsar-beam'
+    kaap-operator-repo: 'https://github.com/datastax/kaap'
     bookkeeper-reg: 'Apache BookKeeper(TM)'
     bookkeeper-short: 'BookKeeper'
     zookeeper-reg: 'Apache ZooKeeper(TM)'

modules/ROOT/pages/faqs.adoc

@@ -38,22 +38,24 @@ They include Minikube, K8d, Kind, Google Kubernetes Engine (GKE), Microsoft Azur
 
 There are several public repos, each with a different purpose. See:
 
-* {pulsar-repo}[{pulsar-repo}] : This is the distro repo (a fork of apache/pulsar).
-* {pulsar-admin-console-repo}[{pulsar-admin-console-repo}] : This is the repo for the {pulsar-short} admin console, which allows for the configuration and monitoring of {pulsar-short}.
+* {pulsar-repo}[{pulsar-repo}]: This is the distro repo (a fork of `apache/pulsar`).
+* {pulsar-admin-console-repo}[{pulsar-admin-console-repo}]: This is the repo for the {pulsar-short} admin console, which allows for the configuration and monitoring of {pulsar-short}.
 * {pulsar-heartbeat-repo}[{pulsar-heartbeat-repo}] : This is a monitoring/observability tool for {pulsar-short} that tracks the health of the cluster and can generate alerts in Slack and OpsGenie.
-* {pulsar-helm-chart-repo}[{pulsar-helm-chart-repo}] : This is the Helm chart for deploying the {company} {pulsar-short} Distro in an existing Kubernetes cluster.
-* {pulsar-sink-repo}[{pulsar-sink-repo}] : This is the {company} {pulsar} Connector (`pulsar-sink`) repo.
-* https://github.com/datastax/burnell[https://github.com/datastax/burnell] : This is a utility for {pulsar-short} that provides various functions, such as key initialization for authentication, and JWT token creation API.
+* {kaap-operator-repo}[{kaap-operator-repo}]: Use the {kaap-short} to deploy the {company} {pulsar-short} Distro in an existing Kubernetes cluster.
++
+{company} recommends the {kaap-short} as a replacement for the deprecated {pulsar-helm-chart-repo}[{company} {pulsar-short} Helm chart].
+* {pulsar-sink-repo}[{pulsar-sink-repo}]: This is the {company} {pulsar} Connector (`pulsar-sink`) repo.
 
 == Is there a prerequisite version of Java needed for the {company} {product} installation?
 
 The {company} {product} distribution is designed for Java 11. However, because the product releases Docker images, you do not need to install Java (8 or 11) in advance. Java 11 is bundled in the Docker image.
 
 == What are the install options for {company} {product}?
 
-* Use the Helm chart provided at {pulsar-helm-chart-repo}[{pulsar-helm-chart-repo}] to install {company} {product} in an existing Kubernetes cluster on your laptop or hosted by a cloud provider.
-* Use the tarball provided at {pulsar-repo}/releases[{pulsar-repo}/releases] to install {company} {product} on a server or VM.
-* Use the {company} Ansible scripts provided at {pulsar-ansible-repo}[{pulsar-ansible-repo}] to install {company} {product} on a server or VM with our provided playbooks.
+* **{kaap-short} Helm chart (Recommended)**: Use the {company} {kaap-operator-repo}[{kaap-short} Helm chart], which provides Kubernetes-native autoscaling and simplified management for {pulsar} clusters. For more information, see the xref:kaap-operator::index.adoc[{kaap}] documentation.
+* **{product} Helm chart (Deprecated)**: The Helm chart provided at {pulsar-helm-chart-repo}[{pulsar-helm-chart-repo}] is deprecated and no longer maintained. Use the {kaap-short} instead.
+* **Tarball**: Use the tarball provided at {pulsar-repo}/releases[{pulsar-repo}/releases] to install {company} {product} on a server or VM.
+* **Ansible**: Use the {company} Ansible scripts provided at {pulsar-ansible-repo}[{pulsar-ansible-repo}] to install {company} {product} on a server or VM with our provided playbooks.
 
 == How do I install {company} {product} in my Kubernetes cluster?
 

modules/components/attachments/beam/values.yaml

@@ -0,0 +1,71 @@
+extra:
+  broker: false
+  brokerSts: true
+  pulsarBeam: true
+  proxy: true
+  wsproxy: false
+  bastion: false
+
+zookeeper:
+  replicaCount: 1
+
+bookkeeper:
+  replicaCount: 1
+
+broker:
+  component: "broker" # this has to match brokerSts naming
+  replicaCount: 0
+
+brokerSts:
+  component: "broker" # this has to match broker naming
+  replicaCount: 1
+  functionsWorkerEnabled: true # to overcome the need for functions working
+  ledger: # the default assumes 2 bookies but this chart only deploys 1
+    defaultEnsembleSize: 1
+    defaultAckQuorum: 1
+    defaultWriteQuorum: 1
+
+# this is a legacy value that the beam sub-chart uses
+tokenServer:
+  allowedRoles: superuser,admin,websocket,proxy
+
+proxy:
+  component: "proxy"
+  replicaCount: 1
+  service:
+    ports:
+      - name: "pulsarbeam"
+        port: 8085
+        protocol: TCP
+      - name: "http"
+        port: 8080
+        protocol: TCP
+      - name: "pulsar"
+        port: 6650
+        protocol: TCP
+
+pulsarBeam:
+  component: "pulsarbeam"
+  replicaCount: 1
+
+image:
+  proxy:
+    repository: datastax/lunastreaming
+    pullPolicy: IfNotPresent
+    tag: 2.10_2.4
+  brokerSts:
+    repository: datastax/lunastreaming
+    pullPolicy: IfNotPresent
+    tag: 2.10_2.4
+  zookeeper:
+    repository: datastax/lunastreaming
+    pullPolicy: IfNotPresent
+    tag: 2.10_2.4
+  bookkeeper:
+    repository: datastax/lunastreaming
+    pullPolicy: IfNotPresent
+    tag: 2.10_2.4
+  pulsarBeam:
+    repository: kesque/pulsar-beam
+    pullPolicy: IfNotPresent
+    tag: "1.0.0"

modules/components/attachments/client.conf

@@ -0,0 +1,50 @@
+# Configuration for pulsar-client and pulsar-admin CLI tools
+
+# URL for Pulsar REST API (for admin operations)
+# For TLS:
+# webServiceUrl=https://localhost:8443/
+webServiceUrl=http://127.0.0.1:8080/
+
+# URL for Pulsar Binary Protocol (for produce and consume operations)
+# For TLS:
+# brokerServiceUrl=pulsar+ssl://localhost:6651/
+brokerServiceUrl=pulsar://127.0.0.1:6650/
+
+# Authentication plugin to authenticate with servers
+# e.g. for TLS
+# authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
+authPlugin=
+
+# Parameters passed to authentication plugin.
+# A comma separated list of key:value pairs.
+# Keys depend on the configured authPlugin.
+# e.g. for TLS
+# authParams=tlsCertFile:/path/to/client-cert.pem,tlsKeyFile:/path/to/client-key.pem
+authParams=
+
+# Allow TLS connections to servers whose certificate cannot be
+# be verified to have been signed by a trusted certificate
+# authority.
+tlsAllowInsecureConnection=false
+
+# Whether server hostname must match the common name of the certificate
+# the server is using.
+tlsEnableHostnameVerification=false
+
+# Path for the trusted TLS certificate file.
+# This cert is used to verify that any cert presented by a server
+# is signed by a certificate authority. If this verification
+# fails, then the cert is untrusted and the connection is dropped.
+tlsTrustCertsFilePath=
+
+# Enable TLS with KeyStore type configuration in broker.
+useKeyStoreTls=false
+
+# TLS KeyStore type configuration: JKS, PKCS12
+tlsTrustStoreType=JKS
+
+# TLS TrustStore path
+tlsTrustStorePath=
+
+# TLS TrustStore password
+tlsTrustStorePassword=

modules/components/attachments/sql/values.yaml

@@ -0,0 +1,50 @@
+extra:
+  broker: false
+  brokerSts: true
+  pulsarSQL: true
+  proxy: false
+  wsproxy: false
+  bastion: false
+
+pulsarSQL:
+  server:
+    workers: 1
+  service:
+    type: ClusterIP
+
+zookeeper:
+  replicaCount: 1
+
+bookkeeper:
+  replicaCount: 1
+
+broker:
+  component: "broker" # this has to match brokerSts naming due to a bug
+  replicaCount: 0
+
+brokerSts:
+  component: "broker" # this has to match broker naming due to a bug
+  replicaCount: 1
+  functionsWorkerEnabled: true # to overcome the need for functions working
+  ledger: # the default assumes 2 bookies but this chart only deploys 1
+    defaultEnsembleSize: 1
+    defaultAckQuorum: 1
+    defaultWriteQuorum: 1
+
+image:
+  brokerSts:
+    repository: datastax/lunastreaming-all #because we want connectors too
+    pullPolicy: IfNotPresent
+    tag: 2.10_2.4
+  zookeeper:
+    repository: datastax/lunastreaming
+    pullPolicy: IfNotPresent
+    tag: 2.10_2.4
+  bookkeeper:
+    repository: datastax/lunastreaming
+    pullPolicy: IfNotPresent
+    tag: 2.10_2.4
+  pulsarSQL:
+    repository: datastax/lunastreaming
+    tag: 2.10_2.4
+    pullPolicy: IfNotPresent

modules/components/pages/pulsar-beam.adoc

@@ -1,7 +1,7 @@
 = {pulsar-beam} with {product}
 :navtitle: {pulsar-beam}
 :description: Install a minimal {product} Helm chart that includes {pulsar-beam}
-:helmValuesPath: https://raw.githubusercontent.com/datastaxdevs/luna-streaming-examples/main/beam/values.yaml
+:helmValuesPath: beam
 
 The {pulsar-beam-repo}[{pulsar-beam}] project is an HTTP-based streaming and queueing system for use with {pulsar}.
 

modules/components/pages/pulsar-sql.adoc

@@ -1,7 +1,7 @@
 = Using {pulsar-short} SQL with {product}
 :navtitle: {pulsar-short} SQL
 :description: This guide installs the {product} Helm chart using minimum values for a working {pulsar-short} cluster that includes SQL workers
-:helmValuesPath: https://raw.githubusercontent.com/datastaxdevs/luna-streaming-examples/main/pulsar-sql/values.yaml
+:helmValuesPath: sql
 
 {pulsar-short} SQL allows enterprises to query {pulsar} topic data with SQL.
 This is a powerful feature for an Enterprise, and SQL is a language they're likely familiar with.
@@ -71,20 +71,14 @@ The minimalist Helm chart values use the https://github.com/datastax/release-not
 This example uses the "public" tenant and "default" namespace.
 These are created by default in {pulsar-short}, but you can use whatever tenant and namespace you are comfortable with.
 
-. Download the minimalist {pulsar-short} client.
-This "client.conf" assumes the port forwarding addresses we will perform in the next step.
-+
-[source,shell]
-----
-wget https://raw.githubusercontent.com/datastaxdevs/luna-streaming-examples/main/client.conf
-----
+. xref:components:attachment$client.conf[Download the minimalist {pulsar-short} client].
+This `client.conf` assumes the port forwarding addresses we will perform in the next step.
 
-. Set the client environment variable. Replace the value with the absolute path to the "client.conf" you just downloaded.
+. Set a client configuration environment variable with the absolute path to the downloaded `client.conf` file:
 +
 [source,shell]
 ----
-# the client conf path must be an absolute path to the downloaded conf file
-export PULSAR_CLIENT_CONF=<REPLACE_WITH /path/to/client.conf>
+export PULSAR_CLIENT_CONF=/path/to/client.conf
 ----
 
 . Navigate to the {pulsar-short} home folder and run the following command.

modules/components/partials/install-helm.adoc

@@ -1,3 +1,9 @@
+[IMPORTANT]
+====
+The {pulsar-helm-chart-repo}[{company} {pulsar-short} Helm chart] is deprecated.
+For Kubernetes deployments, use xref:kaap-operator::index.adoc[{kaap}] instead.
+====
+
 . Add the {company} Helm chart repo to your Helm store:
 +
 [source,shell]
@@ -9,13 +15,12 @@ helm repo add datastax-pulsar https://datastax.github.io/pulsar-helm-chart
 This command creates a Helm release named "my-pulsar-cluster" using the {company} Luna Helm chart, within the K8s namespace "datastax-pulsar".
 The minimal cluster creates only the essential components and has no ingress or load balanced services.
 +
-[source,shell,subs="attributes+"]
+[source,shell]
 ----
-VALUES_URL="{helmValuesPath}"
 helm install \
   --namespace datastax-pulsar \
   --create-namespace \
-  --values $VALUES_URL \
+  --values /path/to/values.yaml \
   --version 3.0.4 \
   my-pulsar-cluster \
   datastax-pulsar/pulsar
@@ -24,7 +29,7 @@ helm install \
 . Wait for the broker pod to be in a running state.
 You might see a few restarts as your components start up.
 +
-[source,shell,subs="attributes+"]
+[source,shell]
 ----
 kubectl -n datastax-pulsar wait --for=condition=Ready pod/pulsar-broker-0 --timeout=120s
 ----

modules/install-upgrade/pages/quickstart-helm-installs.adoc

@@ -3,7 +3,15 @@
 You have options for installing *{company} {product}*:
 
 * With the provided *{company} Helm chart* for an existing Kubernetes environment locally or with a cloud provider, as covered in this topic.
++
+[IMPORTANT]
+====
+The {pulsar-helm-chart-repo}[{company} {pulsar-short} Helm chart] is deprecated.
+For Kubernetes deployments, use xref:kaap-operator::index.adoc[{kaap}] instead.
+====
+
 * With the *{company} {product} tarball* for deployment to a single server/VM, or to multiple servers/VMs. See xref:install-upgrade:quickstart-server-installs.adoc[Quick Start for Server/VM installs].
+
 * With the *{company} Ansible scripts* provided at {pulsar-ansible-repo}[{pulsar-ansible-repo}].
 
 The Helm chart and options described below configure an {pulsar} cluster.

modules/operations/pages/scale-cluster.adoc

@@ -6,9 +6,13 @@ include::operations:partial$operator-scaling.adoc[]
 
 == Install {pulsar-short} cluster
 
-For our {pulsar-short} cluster installation, use this {pulsar-helm-chart-repo}[Helm chart].
+[IMPORTANT]
+====
+The {pulsar-helm-chart-repo}[{company} {pulsar-short} Helm chart] is deprecated.
+For Kubernetes deployments, use xref:kaap-operator::index.adoc[{kaap}] instead.
+====
 
-To start the cluster, use the values provided in this {pulsar-helm-chart-repo}/blob/master/examples/dev-values.yaml[YAML file]:
+To start the cluster, use the values provided in the {pulsar-helm-chart-repo}/blob/master/examples/dev-values.yaml[{company} {pulsar-short} values YAML file]:
 
 [source,bash]
 ----