Arda Collateral Intelligence Service

AI-powered collateral double-pledge detection service for the Arda Credit platform.

Overview

arda-collateral-intel is a dedicated Rust microservice that provides intelligent collateral analysis using:

• Knowledge Graph - SurrealDB-based graph of collateral relationships and deal lifecycle
• AI Detection - Vector embeddings and semantic search for similarity detection
• Deal Lifecycle Awareness - Understanding of "in-use" vs "not-in-use" collateral states
• Event-Driven Architecture - Processes events from arda-credit and publishes detection results

See Technical Specification v2.0 for complete architecture details.

Core Features

• Double-Pledge Detection: Identifies when similar collateral is pledged to multiple active deals
• Knowledge Graph: Maintains historical context of collateral usage and relationships
• Lifecycle Intelligence: Distinguishes between currently pledged vs. available collateral
• Document Cross-Reference: Analyzes collateral mentions across security agreements

Prerequisites

• Rust 1.75 or higher
• SurrealDB 2.0 or higher
• Docker and Docker Compose (for local development)
• Make (optional, for convenience commands)

Quick Start

Local Development Setup

This project uses Makefile for convenient development commands. All database operations are managed through docker-compose.

1. Clone the repository

   git clone https://github.com/ardaglobal/arda-collateral-intel.git
   cd arda-collateral-intel

2. Initial setup (starts database and creates .env file)

   make dev-setup

   This will:

   • Start SurrealDB in Docker
   • Copy .env.example to .env (if it doesn't exist)
   • Wait for database to be ready

3. Configure environment variables

   # Edit .env with your settings
   # For local development, the defaults work out of the box
   # Update SURREALDB_PASSWORD if you changed it in docker-compose.yml

4. Build and run the service

   make run
   # Or: cargo run

5. Verify service is running

   curl http://localhost:3010/health
   curl http://localhost:3010/ready

Database Management

# Start SurrealDB
make db-up

# Stop SurrealDB
make db-down

# Clean database (removes all data)
make db-clean

# View database logs
make db-logs

# Start database and service together
make dev-start

# Stop all services
make dev-stop

SurrealDB Cloud (Dev/Staging/Production)

For deployed environments, configure .env to use SurrealDB Cloud:

SURREALDB_URL=wss://proud-galaxy-06d1403t6pqbn8lsiermaos8os.aws-use1.surreal.cloud
SURREALDB_NAMESPACE=dev  # or staging, production
SURREALDB_DATABASE=collateral_intel
SURREALDB_USERNAME=your_username
SURREALDB_PASSWORD=your_password

The schema is automatically applied on service startup, so no manual migration is needed.

Development Commands

Using Make (Recommended)

# Build the project
make build

# Run the service
make run

# Run all tests
make test

# Run tests with output
make test-verbose

# Check formatting
make fmt

# Run linter
make lint

# Run all checks (format, lint, test)
make check

# Setup development environment
make dev-setup

# Start database and service
make dev-start

Using Cargo Directly

# Build the project
cargo build

# Run unit tests
cargo test

# Run integration tests (requires running SurrealDB)
cargo test --test '*' -- --ignored

# Run with logging
RUST_LOG=debug cargo run

# Check code formatting
cargo fmt --check

# Run linter
cargo clippy -- -D warnings

# Build release binary
cargo build --release

Docker

Build Image

docker build -t arda-collateral-intel:latest .

Run Container

docker run -d \
  -p 3010:3010 \
  --env-file .env \
  --name arda-collateral-intel \
  arda-collateral-intel:latest

Health Check

docker exec arda-collateral-intel curl http://localhost:3010/health

Environment Configuration

All configuration is done via environment variables. See .env.example for the complete list.

Required Variables

• SURREALDB_PASSWORD - Password for SurrealDB connection

Optional Variables (with defaults)

• PORT (default: 3010)
• SIMILARITY_THRESHOLD (default: 0.85)
• MAX_CONCURRENT_ANALYSES (default: 10)

API Endpoints

Health & Status

• GET /health - Liveness probe (always returns 200)
• GET /ready - Readiness probe (checks dependencies)

Phase 1 Additional Endpoints (Coming Soon)

• POST /api/v1/events/collateral - Receive collateral events from arda-credit
• POST /api/v1/events/deal - Receive deal events from arda-credit
• GET /api/v1/collateral/{id}/detection-history - Query detection results

Architecture

┌─────────────────┐         Events          ┌─────────────────┐
│   arda-credit   │ ───────────────────────▶│ arda-collateral │
│ (System of      │                         │ -intel          │
│  Record)        │◀─────────────────────── │ (Intelligence   │
│                 │    Detection Results    │  Layer)         │
└─────────────────┘                         └─────────────────┘
        │                                            │
        │ PostgreSQL                                 │ SurrealDB
        ▼                                            ▼
  ┌──────────┐                                 ┌──────────┐
  │ Deals    │                                 │Knowledge │
  │Collateral│                                 │  Graph   │
  └──────────┘                                 └──────────┘

Implementation Phases

Phase 1: Foundation (Current)

• ✅ Service scaffold with Axum
• ✅ Health endpoints
• ✅ SurrealDB schema and connection
• ✅ Knowledge graph manager with CRUD operations
• ✅ Docker Compose for local development
• ⏳ Event webhook endpoints
• ⏳ Basic detection pipeline

Phase 2: Deal Lifecycle Intelligence

• Deal status event processing
• Collateral lifecycle state machine
• Smart detection: only alert if BOTH deals active

Phase 3: Document Cross-Reference

• arda-ingest API client
• Document-based similarity signals
• Enhanced match scoring

Phase 4: Knowledge Graph Enhancement

• Embedding cache optimization
• Similarity index for faster lookups
• Detection pattern analysis

Testing

Unit Tests

# Run all unit tests
cargo test

# Run tests with output
cargo test -- --nocapture

# Run specific test
cargo test test_health_endpoint

Integration Tests

Integration tests require a running SurrealDB instance. They use unique test namespaces to avoid conflicts.

# Start local SurrealDB
make db-up

# Run integration tests
cargo test --test database_integration_test -- --ignored
cargo test --test knowledge_graph_test -- --ignored
cargo test --test semantic_similarity_test -- --ignored

# Or run all ignored tests
cargo test -- --ignored

The integration tests:

• Connect to SurrealDB (defaults to ws://localhost:8000)
• Create unique test namespaces per test
• Apply the full schema
• Test CRUD operations on collateral and deals
• Test graph relationships and queries
• Test lifecycle status management
• Test double-pledge detection scenarios

Semantic Similarity Tests

The semantic_similarity_test.rs suite validates embedding-based similarity matching:

1. High Similarity Detection (>0.9) - Validates detection of highly similar collaterals
2. Medium Similarity Detection (0.7-0.9) - Tests medium similarity threshold handling
3. Low Similarity Filtering (<0.7) - Ensures low similarity items are filtered out
4. Missing Embeddings Handling - Tests graceful handling of collaterals without embeddings
5. min_similarity Parameter - Validates threshold parameter is respected
6. limit Parameter - Tests result pagination/limiting
7. Score Ordering - Verifies results are sorted by similarity descending
8. Cosine Similarity Accuracy - Unit tests for cosine similarity calculation
9. Vector Normalization - Tests embedding vector normalization
10. Cross-Type Matching - Tests matching across different collateral types
11. Combined Matching - Tests semantic + exact match combinations
12. Embedding Cache - Validates cache utilization for performance

# Run only semantic similarity tests
cargo test --test semantic_similarity_test

# Run with database (integration tests)
cargo test --test semantic_similarity_test -- --ignored

Test Configuration

Set these environment variables to customize test database connection:

export SURREALDB_URL=ws://localhost:8000
export SURREALDB_USERNAME=root
export SURREALDB_PASSWORD=secret

cargo test -- --ignored

Deployment

See helm/arda-collateral-intel/ for Kubernetes deployment configuration.

# Lint Helm chart
helm lint helm/arda-collateral-intel/

# Install to Kubernetes
helm install arda-collateral-intel helm/arda-collateral-intel/ \
  --namespace arda \
  --create-namespace

# Upgrade deployment
helm upgrade arda-collateral-intel helm/arda-collateral-intel/ \
  --namespace arda

Contributing

1. Follow the implementation plan in the technical specification
2. Run cargo fmt and cargo clippy before committing
3. Add tests for new functionality
4. Update documentation as needed

References

• Technical Specification v2.0
• CLAUDE.md - AI development context
• Arda Credit API Documentation
• Arda Ingest API Documentation

License

MIT