Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .agents/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
## Release Notes Formatting Rules

When adding new server release notes (under `doc-server/docs/releases/server/`):

1. **Frontmatter Invariants**:
- Set `sidebar_position: -<number>` where `<number>` corresponds to the version hierarchy (e.g., `-110` for `1.1.0`, `-100` for `1.0.0`, `-99` for `0.99.0`). This ensures proper sorting in Docusaurus.

2. **Bullet Points**:
- Always use `-` instead of `*` for list items.

3. **Entry Cleaning**:
- Remove GitHub usernames and pull request URLs (e.g. strip `by @username in https://github...` from the end of each line).
- Normalize prefixes: remove scopes like `(infra)` or `(security)` from the tag prefix (e.g. simplify `fix(infra):` to `fix:`).
- Capitalize the first letter of the description.

4. **Categorization & Sorting**:
- Group entries into the standard categories: `Troubleshoot`, `Notifications`, `Tickets`, `Workflow`, `Cloud`, `AI`, `Integrations`.
- Within each category, sort features (`feat:`) first, followed by fixes (`fix:`).

5. **Exclusions**:
- Omit internal developer chores, infrastructure/CI updates, test suites, back-merges, and package dependency bumps.

6. **Index Updates**:
- Link the new release page at the top of the "Recent Releases" list on the release index page (`doc-server/docs/releases/server/index.md`).
12 changes: 11 additions & 1 deletion doc-server/docs/FAQ.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 4
sidebar_position: 100
---

# FAQ
Expand All @@ -17,6 +17,16 @@ sidebar_position: 4
- TensorFlow for recommendations for replicas
- AWS Bedrock using LLAMA (optional) for AI-based recommendations on logs/errors

### What are the hard dependencies for the NudgeBee Server?
- **PostgreSQL**: Hard requirement. Stores cluster configurations, user metadata, alert rules, and workflow states. Queries and services fail without it.
- **RabbitMQ**: Hard requirement. Powers internal message queues between backend services and workers. Consumers will not bootstrap without it.
- **Redis**: Optional. Falls back to in-memory caching if omitted.

### Is the NudgeBee Agent mandatory if I already connected a cloud account?
- **Cloud Account Connection** provides inventory discovery across your AWS/Azure/GCP resources and auto-detects existing Kubernetes clusters.
- **The Agent** is required inside each cluster to collect real-time workload metrics, stream pod logs, capture eBPF network telemetry, and perform automated AI root cause analysis.
- For full AI troubleshooting and cost optimizations on a cluster, installing the Agent is required.

### What are the pre-conditions(software/hardware) for installing NudgeBee on my cluster?

#### For Agent:
Expand Down
73 changes: 54 additions & 19 deletions doc-server/docs/features/Cloud/AWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,25 +73,60 @@ StackSets deploy only to **member** accounts, not the management account itself.

---

## CloudWatch Alarms Permissions

NudgeBee collects existing CloudWatch alarms from your AWS account and can create new alarms based on recommendations.

### Permissions Included in CloudFormation Template

The CloudFormation template includes:

**Read permissions** (via `ReadOnlyAccess` managed policy):
```
cloudwatch:DescribeAlarms
cloudwatch:DescribeAlarmsForMetric
## Least-Privilege IAM Policy (Manual Role Creation)

If you prefer to create a custom IAM role manually instead of using the managed CloudFormation template, attach the following least-privilege policy document to your cross-account role:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "NudgeBeeCloudWatchDiscovery",
"Effect": "Allow",
"Action": [
"cloudwatch:DescribeAlarms",
"cloudwatch:DescribeAlarmsForMetric",
"cloudwatch:GetMetricData",
"cloudwatch:ListMetrics"
],
"Resource": "*"
},
{
"Sid": "NudgeBeeEKSDiscovery",
"Effect": "Allow",
"Action": [
"eks:DescribeCluster",
"eks:ListClusters"
],
"Resource": "*"
},
{
"Sid": "NudgeBeeCostAndUsageDiscovery",
"Effect": "Allow",
"Action": [
"cur:DescribeReportDefinitions",
"ce:GetCostAndUsage",
"ce:GetCostForecast",
"ce:GetDimensionValues"
],
"Resource": "*"
},
{
"Sid": "NudgeBeeCURS3Access",
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::<YOUR_CUR_BUCKET_NAME>",
"arn:aws:s3:::<YOUR_CUR_BUCKET_NAME>/*"
]
}
]
}
```

**Write permissions** (included when the account is connected in **Standard** access mode):
```
cloudwatch:PutMetricAlarm
```

These permissions allow NudgeBee to both monitor existing alarms and create new ones for your resources.

<!-- assets verified -->
45 changes: 44 additions & 1 deletion doc-server/docs/features/optimizations.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,50 @@ NudgeBee's **FinOps AI-Assistant** continuously analyzes your Kubernetes workloa
To automatically apply optimization recommendations without manual approval, configure [Autopilot Auto-Optimize](../autopilot/auto_optimize/). To have NudgeBee raise pull requests with the recommended changes, connect a [GitHub](../integrations/Code%20Repository/GitHub/github-integration.md) or [GitLab](../integrations/Code%20Repository/GitLab/gitlab-integration.md) repository.
:::

![NudgeBee Optimize summary showing potential monthly savings, prioritized right-sizing recommendations, and a per-account cost and health overview](./img/optimize-overview.png)
---

## Right-Sizing Calculation Methodology

NudgeBee uses deterministic statistical analysis over historical Prometheus metrics to calculate safe resource recommendations:

| Recommendation Type | Observation Window | Sizing Metric | Strategy & Risk Profile |
|---|---|---|---|
| **CPU Limits** | Past 14 Days | Max P99 + 20% Headroom | **Low Risk** — Prevents CPU throttling during unexpected traffic spikes. |
| **CPU Requests** | Past 14 Days | P95 Utilization | **Low Risk** — Maximizes bin-packing efficiency across worker nodes. |
| **Memory Limits** | Past 7 Days | Peak Utilization + 15% Buffer | **Zero OOM Tolerance** — Prevents kernel OOM-killer termination of stateful pods. |
| **Memory Requests** | Past 7 Days | Peak Utilization | **Low Risk** — Eliminates idle reserved memory overhead. |
| **Unattached Volumes** | Past 30 Days | 0 Read/Write IOPS | **Zero Impact** — Flags unmounted, detached PVCs/EBS volumes for safe removal. |

### Pricing Engine

Cost figures are calculated using:
- **In-Cluster OpenCost Engine**: Accurately accounts for node instance types, storage classes, and shared namespace allocations.
- **Cloud Provider Pricing APIs**: Real-time integration with AWS Pricing API, GCP Cloud Billing, and Azure Retail Rates.
- **Custom Discount Rates**: Support for enterprise discount agreements (EDP/MCA) and reserved instance commitments.

---

## GitOps & Automated Pull Request Workflow

Instead of applying manual `kubectl` patches, NudgeBee enables infrastructure-as-code teams to review and merge recommendations via GitOps pull requests:

```mermaid
graph LR
O[FinOps Recommendation] --> G[NudgeBee Git Engine]
G --> B[Create Branch & Commit Diff]
B --> PR[Open Pull Request in GitHub / GitLab]
PR --> CI[CI / ArgoCD / Flux Deployment]
```

### How to Raise Automated PRs

1. Connect your repository under **Admin $\rightarrow$ Integrations $\rightarrow$ Code Repositories** ([GitHub](../integrations/Code%20Repository/GitHub/github-integration.md) or [GitLab](../integrations/Code%20Repository/GitLab/gitlab-integration.md)).
Comment thread
blue4209211 marked this conversation as resolved.
2. Navigate to **Optimizations $\rightarrow$ Workload Right-Sizing**.
3. Select the target deployment or StatefulSet.
4. Click **Create Pull Request**.
5. NudgeBee creates a new branch, updates the Helm `values.yaml` or Terraform manifest with the recommended CPU/memory requests and limits, and opens a Pull Request with a clear rationale table for your engineering team to review.

---

### Watch a Walkthrough

Expand Down
24 changes: 24 additions & 0 deletions doc-server/docs/features/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,30 @@ NudgeBee does not store passwords. All authentication is handled through SSO pro
## Authorization
- Currently, NudgeBee supports tenant level authorization with 2 roles(admin, readonly)

## Kubernetes Agent RBAC & Least-Privilege Permissions

For security compliance teams reviewing the in-cluster agent deployment, the NudgeBee agent requests read-only Kubernetes `ClusterRole` permissions by default:

| API Group | Resources | Verb Permissions | Purpose |
|---|---|---|---|
| `""` (Core) | `pods`, `nodes`, `namespaces`, `services`, `endpoints`, `events`, `persistentvolumes`, `persistentvolumeclaims` | `get`, `list`, `watch` | Topology mapping, resource health, and incident triage. |
| `apps` | `deployments`, `statefulsets`, `daemonsets`, `replicasets` | `get`, `list`, `watch` | Workload configuration and right-sizing analysis. |
| `batch` | `jobs`, `cronjobs` | `get`, `list`, `watch` | Batch workload failure detection. |
| `networking.k8s.io` | `ingresses`, `networkpolicies` | `get`, `list`, `watch` | Traffic routing and network topology mapping. |
| `autoscaling` | `horizontalpodautoscalers` | `get`, `list`, `watch` | HPA scaling tracking. |

:::note No Secret Data Access
The agent **never reads Secret data contents** (`get` or `list` on `secrets` is excluded from default ClusterRole bindings).
:::

## Network Security & Firewall Rules

| Traffic Flow | Direction | Protocol / Port | Destination | Purpose |
|---|---|---|---|---|
| **Agent to Server** | Outbound | TCP `443` (WSS / HTTPS) | `app.nudgebee.com` or self-hosted Ingress | Streaming metrics, events, and telemetry. |
| **Agent to Cloud Pricing** | Outbound | TCP `443` (HTTPS) | AWS, GCP, Azure pricing endpoints | OpenCost pricing calculations. |
| **Server to In-Cluster Relay** | Inbound / Internal | TCP `8080` (Internal ClusterIP) | `relay-server.nudgebee.svc` | Real-time agent relay communication. |

## Audit
- User operations are tracked and can be viewed on the UI.

Expand Down
65 changes: 63 additions & 2 deletions doc-server/docs/features/troubleshooting/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,70 @@ NudgeBee's troubleshooting dashboard gives you a real-time view of events, error
- **Attach event playbooks** — Run evidence-collection actions (logs, metrics, custom SQL, kubectl, cloud CLI, SSH, …) automatically on each event so the LLM has the data it needs. See [Playbook Catalog](./playbook-catalog.md).
- **Understand the split between playbooks and workflows** — Playbooks gather evidence for the LLM; [workflows](../workflow-builder/index.md) post-process the resulting event (triage, ticketing, remediation). See [Event Playbooks vs Workflows](./event-playbooks-vs-workflows.md).

:::info
**Prerequisites**: To use troubleshooting features, you need at least one [Kubernetes cluster connected](../../installation/agent/installation/) and an [observability source integrated](../../integrations/Observability/). For AI-powered analysis, an [LLM connection](../../integrations/LLM/) is also needed.
:::info Prerequisites
To use troubleshooting features, you need at least one [Kubernetes cluster connected](../../installation/agent/installation/) and an [observability source integrated](../../integrations/Observability/). For AI-powered analysis, an [LLM connection](../../integrations/LLM/) is also needed.
:::

---

## How NuBi Investigates Incidents

NuBi does not simply summarize alerts — it performs multi-hop reasoning over the **Semantic Knowledge Graph (SKG)** to find the true root cause:

```mermaid
graph LR
E[Kubernetes Event<br/>CrashLoopBackOff] --> T[1. Topology Traversal<br/>Pod → Deployment → Node → Ingress]
T --> L[2. Evidence Gathering<br/>Pod Logs + Exit Code + Events]
L --> G[3. Knowledge Graph Linkage<br/>Correlate ConfigMaps / Secrets / Commits]
G --> R[4. Root Cause Hypothesis<br/>Missing API Secret in Namespace]
R --> A[5. Remediation Plan<br/>kubectl create secret ...]
```

1. **Topology Traversal**: Navigates from the failing entity upstream and downstream across Kubernetes objects, namespaces, and cloud dependencies.
2. **Evidence Collection**: Queries Prometheus metrics for resource spikes, fetches pod stdout/stderr logs around the timestamp, and inspects recent Kubernetes events.
3. **Change & Config Correlation**: Cross-references recent CI/CD deployments, git commits, ConfigMap changes, and Secret references.
4. **Causality Synthesis**: Forms a structured 5-Whys explanation backed by exact timestamps, log snippets, and trace IDs.

---

## Execution Guardrails & Human-in-the-Loop

NudgeBee is designed with strict enterprise safety controls:

- **Default Mode (Human-in-the-Loop)**: NuBi produces diagnostic conclusions and generates exact remediation commands (e.g. `kubectl patch`, rollback commands, or secret creations). NuBi **never runs destructive commands automatically** without explicit human one-click approval in the UI or Slack.
- **Autopilot Mode (Policy-Gated)**: If you enable [Autopilot](../autopilot/autopilot.md), safe non-destructive operations (such as pod restarts, vertical right-sizing, or node drains) can be executed automatically according to rules and schedules you define.
- **Audit Logging**: Every action proposed, approved, or executed is logged with user attribution for compliance.

---

## Sample Incident Walkthrough: CrashLoopBackOff

Here is an end-to-end example of NuBi diagnosing an outage caused by a missing database secret:

```
[Incident Alert] Pod auth-service-78d49fb99-x7k2q is CrashLoopBackOff in namespace production
```

#### 1. Evidence Collected by NuBi
- **K8s Event**: `Warning BackOff Back-off 5m0s restarting failed container auth-app`
- **Pod Container Log**: `FATAL: Failed to read DB_PASSWORD from environment variable: Secret "auth-db-secret" not found`
- **Recent Change**: Helm upgrade `auth-service` release `v2.4.1` deployed 4 minutes ago with a new secret key reference.

#### 2. NuBi's Reasoning Output
> **Root Cause**: The `auth-service` deployment was upgraded to `v2.4.1` which introduced a reference to Secret `auth-db-secret`, but the secret was not created in the `production` namespace prior to deployment.
>
> **Recommended Fix**:
> ```bash
> kubectl create secret generic auth-db-secret \
> --from-literal=DB_PASSWORD="<PROD_PASSWORD>" \
> -n production
> ```

#### 3. Action
Click **Apply Fix** in the UI to create the secret, or click **Create Jira Ticket** to assign it to the on-call team with the full investigation payload attached.

---

### Watch a Walkthrough

<div style={{position: "relative", paddingBottom: "56.25%", height: 0}}><iframe src="https://www.loom.com/embed/46381390d75c40d09a77e9ab0f5b4a98?sid=95ee4109-b754-4584-8cba-a5111db775f4" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style={{position: "absolute", top: 0, left: 0, width: "100%", height: "100%"}}></iframe></div>
Expand All @@ -30,3 +90,4 @@ NudgeBee's troubleshooting dashboard gives you a real-time view of events, error
- **[Alerting](./alerting.md)** — Configure custom alerting rules and attach playbook actions for auto-triage.
- **[Playbook Catalog](./playbook-catalog.md)** — Full reference of every event-playbook action and its parameters, including custom data-collection (proxy DB query, cloud CLI, SSH, kubectl).
- **[Templating & Best Practices](./templating.md)** — Use gonja (Jinja-style) templates in action parameters, with patterns for labels, outputs, conditionals, and `for_each` loops.

53 changes: 36 additions & 17 deletions doc-server/docs/features/workflow-builder/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,28 +191,47 @@ You can make tasks run conditionally or branch your workflow using special task
3. Connect different downstream tasks to each case branch
4. Conditional connections appear as colored, thicker lines with a condition label

### Using Template Expressions
### Using Template Expressions & Variables

Template expressions let you use dynamic values in task parameters. They use the `{{ }}` syntax.
Template expressions allow you to dynamically pass data between workflow steps, evaluate conditional branching, and reference global configurations. They use double curly braces `{{ }}`.

**Common patterns:**
#### Variable Reference Cheatsheet

| Expression | Description |
|-----------|-------------|
| `{{ Task.output.value }}` | Reference an output from a previous task |
| `{{ Configs.key_name }}` | Reference a shared configuration value |
| `{{ variable == "value" }}` | Conditional expression for branching or filtering |
| `{{ event.source == "my-source" }}` | Filter expression for event triggers |
| Expression Syntax | Scope | Description & Example |
|---|---|---|
| `{{ steps.<task_id>.output }}` | Task Output | Full JSON output of a completed prior task. |
| `{{ steps.fetch_logs.output.log_lines }}` | Nested Field | Extract a specific field or array from a previous task's output. |
| `{{ event.source }}` | Event Trigger | Originating event source (e.g. `prometheus`, `kubernetes`, `aws_eventbridge`). |
| `{{ event.payload.cluster }}` | Event Payload | Name of the cluster where the event was detected. |
| `{{ event.payload.namespace }}` | Event Payload | Kubernetes namespace of the affected entity. |
| `{{ Configs.<key_name> }}` | Global Config | Value of a shared secret or global constant (e.g. `{{ Configs.slack_channel_alerts }}`). |
| `{{ input.<param_name> }}` | Manual Input | User-supplied parameter from a manual run modal. |

**Where you can use them:**
- Any text or textarea field in task parameters
- Conditional execution fields
- Event filter expressions
- Output parameter definitions
---

## Cookbook: 3 Ready-to-Use Workflow Blueprints

Below are 3 popular workflow patterns you can import or assemble in minutes:

### 1. Automated Node Drain on Spot Termination Notice
**Trigger**: Event (`source == "aws_eventbridge"`, `detail-type == "EC2 Spot Instance Interruption Warning"`)
- **Step 1 (`Kubectl`)**: Taint node: `kubectl cordon {{ event.payload.instance_id }}`
- **Step 2 (`Kubectl`)**: Safely evict workloads: `kubectl drain {{ event.payload.instance_id }} --ignore-daemonsets --delete-emptydir-data --grace-period=120`
- **Step 3 (`IM Notification`)**: Send Slack message: `⚠️ Node {{ event.payload.instance_id }} drained ahead of spot termination.`

**How to insert them:**
- Type the expression directly using `{{ }}` syntax
- Or drag an output field from the **Previous tasks outputs** panel in the task configuration sidebar
### 2. Slack Alert Enrichment with Pod Logs & RCA
**Trigger**: Event (`source == "prometheus"`, `severity == "critical"`)
- **Step 1 (`Query Logs`)**: Fetch last 100 log lines from the crashing pod: `{{ event.payload.pod_name }}`
- **Step 2 (`LLM Investigation`)**: Feed logs and events into NuBi for automated diagnosis: `Investigate root cause for {{ steps.query_logs.output.logs }}`
- **Step 3 (`IM Notification`)**: Post an interactive Slack card with the pod status, root cause summary, and one-click remediation button.

### 3. Automated Cleanup of Orphaned PVs & Stale Namespaces
**Trigger**: Schedule (Cron: `0 2 * * 0` — every Sunday at 2 AM)
- **Step 1 (`Kubectl`)**: List unattached PersistentVolumes in `Released` state: `kubectl get pv -o json`
- **Step 2 (`Manual Approval`)**: Send summary to SRE team with approval button before deletion.
- **Step 3 (`Kubectl`)**: Delete approved released PVs and reclaim cloud storage costs.

---

## Managing Workflows

Expand Down
Loading
Loading