Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .githooks/pre-commit
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/bash

set -e

echo "Running tests before commit..."

# Change to the repository root
REPO_ROOT="$(git rev-parse --show-toplevel)"
cd "$REPO_ROOT"

# Check if endpoint-exposer tests need to be run (if any endpoint-exposer files changed)
if git diff --cached --name-only | grep -q "^endpoint-exposer/"; then
echo "Endpoint-exposer files changed, running tests..."

if command -v bats &> /dev/null; then
cd endpoint-exposer
bats test/
else
echo "⚠️ BATS not installed, skipping tests"
echo "Install BATS: brew install bats-core (macOS) or see https://bats-core.readthedocs.io"
exit 0
fi

echo "✅ All endpoint-exposer tests passed!"
fi
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,6 @@
*.iml
out
gen

# VSCode project files
.vscode/
190 changes: 190 additions & 0 deletions endpoint-exposer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
# Endpoint Exposer Service

## Overview

The **endpoint-exposer** service is a infrastructure component of Nnullplatform that manages dynamic exposure of application endpoints through public and private domains. It functions as a route orchestrator that translates high-level specifications into native Kubernetes configurations using HttpRoutes.

## Core Responsibilities

### 1. Dynamic Endpoint Management
- Expose application endpoints declaratively
- Configure separate public and private domains for different access levels
- Update route configurations with zero downtime
- Maintain configuration synchronized with desired state

### 2. Route Configuration
- Define route patterns (exact, regex, wildcards)
- Specify allowed HTTP methods (GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS)
- Associate routes with nullplatform scopes for access control
- Control route visibility (public vs. private)

### 3. Kubernetes and Istio Integration
- Generate HTTPRoute resources (Kubernetes Gateway API v1)

### 4. Scope-Based Access Control
- Map endpoints to specific nullplatform scopes

## Key Features

### Route Management
```yaml
routes:
- method: GET
path: /api/users
scope: user-management
visible_on: public
```

- **Path Types**:
- Exact: `/api/users`
- Regex with parameters: `/api/users/{id}`
- Wildcard: `/api/users/*`

- **HTTP Methods**: Supports all standard HTTP methods
- **Visibility**: Public or private routes on separate domains

### Domain Separation

**Public Domain:**
- Endpoints accessible from the internet
- Typically for public APIs
- Connected to `gateway-public` gateway

**Private Domain:**
- Internal organization endpoints
- Requires private network access
- Connected to `gateway-private` gateway

## Architecture

### Workflow

1. **Build Context**
- Extracts service action parameters
- Retrieves Kubernetes namespace information
- Classifies routes by visibility (public/private)

2. **Build HTTPRoutes**
- Generates base HTTPRoute templates per domain
- Queries scopes associated with each route
- Constructs Istio routing rules

3. **Process Routes**
- Sorts routes by specificity (exact > regex > prefix)
- Generates AuthorizationPolicies if authorization is enabled
- Maps scope IDs to backend services

4. **Apply Configuration**
- Applies generated YAML manifests to the cluster
- Manages cleanup of obsolete resources
- Maintains tracking of applied resources

### Technologies

- **Kubernetes**: Orchestration platform (Gateway API v1)
- **Istio**: Service mesh for traffic management and security
- **Bash**: Workflow scripting and automation
- **jq**: JSON processing and manipulation
- **gomplate**: Resource template generation
- **kubectl**: Kubernetes resource management

## File Structure

```
/endpoint-exposer
├── configure # Service configuration script
├── entrypoint/ # Entry points for actions
│ ├── service-action # Service action handler
├── specs/ # Service specifications
│ └── service-specification.json
├── workflows/istio/ # Workflow definitions
│ └── service-action.json
├── scripts/istio/ # Core routing logic
│ ├── build_context
│ ├── build_httproute
│ ├── process_routes
│ ├── build_rule
│ └── build_ingress_with_rule
├── scripts/common/ # Shared utilities
│ ├── apply
│ └── delete
├── templates/istio/ # K8s resource templates
│ └── httproute.yaml.tmpl
├── test/ # BATS test suite
└── container-scope-override/ # Custom deployment support for override scope agent
```

## Configuration

### Environment Variables

- `K8S_NAMESPACE`: Kubernetes namespace for resources (default: `nullplatform`)
- `PUBLIC_GATEWAY_NAME`: Public gateway name (default: `gateway-public`)
- `PRIVATE_GATEWAY_NAME`: Private gateway name (default: `gateway-private`)
- `GATEWAY_NAMESPACE`: Gateway namespace (default: `gateways`)

### Route Configuration Example

```json
{
"routes": [
{
"method": "GET",
"path": "/api/v1/resource/{id}",
"scope": "resource-read",
"visible_on": "public",
},
{
"method": "POST",
"path": "/api/v1/resource",
"scope": "resource-write",
"visible_on": "private",
}
],
"public_domain": "api.example.com",
"private_domain": "internal-api.example.com"
}
```

## Testing

The service uses BATS (Bash Automated Testing System) for testing:

```bash
# Run all tests
./test/run-tests.sh

# Run specific tests
bats test/istio/
```

Tests cover:
- Simple routes
- Public and private routes
- Authorization scenarios
- JWT configurations
- Manifest generation

## Operations

### Create/Update Endpoints

The service responds to Nullplatform actions:
- `create`: Generates and applies initial configuration
- `update`: Modifies existing configuration
- `delete`: Cleans up Kubernetes resources

### Monitoring

Generated resources can be monitored with:

```bash
# View HTTPRoutes
kubectl get httproutes -n <namespace>

# View AuthorizationPolicies
kubectl get authorizationpolicies -n <namespace>

# View gateway status
kubectl get gateway -n gateways
```
172 changes: 172 additions & 0 deletions endpoint-exposer/container-scope-override/deployment/sync_exposer
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
#!/bin/bash

echo "=== DEBUG: Starting sync_exposer script ==="

APPLICATION_NRN=$(jq -r .application.nrn <<< "$CONTEXT")

echo "SERVICE SPECIFICATION SLUG: $SERVICE_SPECIFICATION_SLUG, APPLICATION_NRN: $APPLICATION_NRN"

# Step 1: Get service specification by slug
echo "DEBUG: Fetching service specifications..."
SERVICE_SPECS=$(np service specification list --nrn "$APPLICATION_NRN" --type dependency --format json)
SERVICE_SPEC=$(jq -c --arg slug "$SERVICE_SPECIFICATION_SLUG" '
.results
| map(select(.slug == $slug))
| .[0]
' <<< "$SERVICE_SPECS")

SERVICE_SPEC_ID=$(jq -r .id <<< "$SERVICE_SPEC")

if [[ -z "$SERVICE_SPEC_ID" || "$SERVICE_SPEC_ID" == "null" ]]; then
echo "Error: Could not find service specification with slug '$SERVICE_SPECIFICATION_SLUG'"
exit 1
fi

echo "DEBUG: SERVICE_SPEC_ID=$SERVICE_SPEC_ID"

# Step 2: Get service instance that matches the SERVICE_SPEC_ID
echo "DEBUG: Fetching services for application..."
SERVICES=$(np service list --nrn "$APPLICATION_NRN" --format json)

SERVICE=$(jq -c --arg spec_id "$SERVICE_SPEC_ID" '
.results
| map(select(.specification_id == $spec_id))
| .[0]
' <<< "$SERVICES")

SERVICE_ID=$(jq -r .id <<< "$SERVICE")

if [[ -z "$SERVICE_ID" || "$SERVICE_ID" == "null" ]]; then
echo "Could not find service instance for specification '$SERVICE_SPEC_ID', skipping exposer sync"
exit 0
fi

echo "DEBUG: SERVICE_ID=$SERVICE_ID"

# Step 3: Get service attributes as parameters
echo "DEBUG: Reading service attributes..."
SERVICE_DATA=$(np service read --id "$SERVICE_ID" --format json)
export PARAMETERS=$(jq -c .attributes <<< "$SERVICE_DATA")

echo "DEBUG: PARAMETERS=$PARAMETERS"

# Step 4: Get action specification with slug "update-<service-spec-slug>"
ACTION_SLUG="update-$SERVICE_SPECIFICATION_SLUG"
echo "DEBUG: Fetching action specifications (looking for slug: $ACTION_SLUG)..."
SERVICE_ACTIONS=$(np service specification action specification list --serviceSpecificationId "$SERVICE_SPEC_ID" --format json)

ACTION_SPEC=$(jq -c --arg slug "$ACTION_SLUG" '
.results
| map(select(.slug == $slug))
| .[0]
' <<< "$SERVICE_ACTIONS")

ACTION_SPEC_ID=$(jq -r .id <<< "$ACTION_SPEC")

if [[ -z "$ACTION_SPEC_ID" || "$ACTION_SPEC_ID" == "null" ]]; then
echo "Error: Could not find action specification with slug '$ACTION_SLUG' for service specification '$SERVICE_SPEC_ID'"
exit 1
fi

echo "DEBUG: ACTION_SPEC_ID=$ACTION_SPEC_ID"

# Step 5: Create service action with parameters (with retry for concurrency)
echo "DEBUG: Creating service action..."

MAX_CREATE_RETRIES=10
RETRY_DELAY=5
create_attempt=0
ACTION_ID=""

while [[ -z "$ACTION_ID" || "$ACTION_ID" == "null" ]]; do
((create_attempt++))
echo "DEBUG: Create attempt $create_attempt/$MAX_CREATE_RETRIES"

if [ "$create_attempt" -gt $MAX_CREATE_RETRIES ]; then
echo "Error: Maximum number of create attempts (${MAX_CREATE_RETRIES}) reached. Could not create action."
exit 1
fi

# Add delay before retry (except on first attempt)
if [ "$create_attempt" -gt 1 ]; then
echo "DEBUG: Waiting ${RETRY_DELAY} seconds before retry..."
sleep $RETRY_DELAY
fi

# Try to create the action
ACTION_RESPONSE=$(np service action create --serviceId "$SERVICE_ID" --body "$(jq -n --argjson params "$PARAMETERS" --arg spec_id "$ACTION_SPEC_ID" '{name: "update", parameters: $params, specification_id: $spec_id}')" --format json 2>&1 || true)

# Check if response contains an error about action already in progress
if echo "$ACTION_RESPONSE" | grep -q "already an action with status.*in_progress"; then
echo "DEBUG: Action already in progress detected"

# Try to find the existing in_progress action
echo "DEBUG: Attempting to find existing in_progress action..."
EXISTING_ACTIONS=$(np service action list --serviceId "$SERVICE_ID" --format json)
EXISTING_ACTION=$(echo "$EXISTING_ACTIONS" | jq -c --arg spec_id "$ACTION_SPEC_ID" '
.results
| map(select(.specification_id == $spec_id and .status == "in_progress"))
| .[0]
')

EXISTING_ACTION_ID=$(echo "$EXISTING_ACTION" | jq -r '.id // empty')

if [[ -n "$EXISTING_ACTION_ID" && "$EXISTING_ACTION_ID" != "null" ]]; then
echo "DEBUG: Found existing in_progress action with ID: $EXISTING_ACTION_ID"
ACTION_ID="$EXISTING_ACTION_ID"
echo "Using existing action instead of creating new one"
break
fi

echo "DEBUG: No existing action found, will retry..."
elif echo "$ACTION_RESPONSE" | grep -q '"error"'; then
echo "ERROR: Failed to create action: $ACTION_RESPONSE"
echo "DEBUG: Will retry after delay..."
else
# Success - extract action ID
ACTION_ID=$(echo "$ACTION_RESPONSE" | jq -r '.id // empty')

if [[ -n "$ACTION_ID" && "$ACTION_ID" != "null" ]]; then
echo "DEBUG: ACTION_ID=$ACTION_ID"
echo "Created endpoint exposer update action[id=$ACTION_ID], waiting for its completion"
break
else
echo "DEBUG: Could not extract ACTION_ID from response: $ACTION_RESPONSE"
echo "DEBUG: Will retry after delay..."
fi
fi
done

# Step 6: Wait for action to complete
MAX_ITERATIONS=20
iteration=0

echo "DEBUG: Starting polling loop for action status..."
while true; do
((iteration++))
echo "DEBUG: Iteration $iteration/$MAX_ITERATIONS"

if [ "$iteration" -gt $MAX_ITERATIONS ]; then
echo "Error: Maximum number of iterations (${MAX_ITERATIONS}) reached. Could not update the endpoint exposer."
exit 1
fi

echo "DEBUG: Reading action status..."
ACTION_RESPONSE=$(np service action read --serviceId "$SERVICE_ID" --id "$ACTION_ID" --format json)
ACTION_STATUS=$(jq -r .status <<< "$ACTION_RESPONSE")

echo "Checking endpoint exposer update action[id=$ACTION_ID, status=$ACTION_STATUS]"

if [[ "$ACTION_STATUS" == "success" ]]; then
echo "✅ Endpoint exposer successfully updated"
break
elif [[ "$ACTION_STATUS" == "failed" ]]; then
echo "❌ Could not update endpoint exposer, deployment will be rollbacked"
exit 1
fi

echo "DEBUG: Sleeping for 5 seconds..."
sleep 5
done

echo "=== DEBUG: sync_exposer script completed successfully ==="
Loading