StratOps-Intel

Enterprise Multi-Tenant Signal Intelligence & Decision Platform
Engineered with GBNF-constrained neural extraction, temporal knowledge graph RAG, pointer-only state orchestration, and vLLM-backed model mesh inference.

💡 Business Perspective & Problem Statement

Modern enterprise strategy teams, hedge funds, and executive leaders face severe information overload from fragmented, unstructured market signals—including SEC XBRL filings, earnings call audio, web news, and hiring trends. Manual competitive intelligence is too slow and costly to inform high-stakes decision-making.

StratOps-Intel bridges this gap by automating multi-modal data ingestion, linking market entities within a temporal knowledge graph, and leveraging autonomous AI agent orchestration to deliver real-time, decision-ready executive briefings.

Core Value Drivers

Eliminates Information Silos: Unifies disparate unstructured data streams (audio, text, tables) into a single queryable intelligence mesh.
Accelerates Decision Velocity: Reduces manual analyst processing time by up to 80% through self-correcting agentic workflows.
Preserves Temporal Context: Tracks how entity relationships, corporate disclosures, and executive signals evolve over time rather than relying on static document summarization.

🎯 Key Business Use Cases

Automated Competitive Intelligence: Continuous automated monitoring of rival firm movements, executive shifts, patent filings, and talent acquisition trends.
Living Executive Briefings: Synthesizing daily or weekly decision support briefs that highlight exact delta updates (Δ delta_summary) so executives only focus on newly surfaced shifts.
Early Risk & Opportunity Detection: Identifying hidden market signals—such as sudden hiring spikes in specialized technical domains cross-referenced with SEC risk factor disclosures.

🔄 End-to-End Business Flow

Multi-Modal Signal Ingestion: Web crawlers, SEC XBRL parsers, speech-to-text audio engines, and job board feeds capture raw payloads into MinIO S3 storage while publishing event triggers to Redis.
Autonomous Cognitive Processing: A stateful 5-node LangGraph agent workflow processes the signal:
- Entity Extractor: Extracts target organizations, key leaders, technologies, and products.
- Correlation Engine: Ingests entities into Neo4j with explicit temporal bounds (valid_from $\rightarrow$ valid_to).
- Trend Analyzer & Anomaly Detector: Executes STL momentum and Isolation Forest models to isolate statistical anomalies.
- Briefing Composer: Formats structured intelligence for downstream LLM synthesis.
LLM Synthesis & Delta Tracking: Localized Qwen models within a BentoML/vLLM mesh synthesize graph context into structured Markdown briefs with delta-update tracking.
Analyst Delivery: Strategic leads review live alert streams on the Strategy Dashboard, map relationships in the Graph Explorer, and inspect raw payloads in the Signal Explorer.

🏗️ Architecture

🗺️ System Overview

🖥️ User Interface & Operations Control

StratOps-Intel features a dark-mode mission control center built on Next.js 14, delivering real-time agent execution telemetry, vector search similarity scores, and temporal graph exploration.

💡 Quick Run (Zero-Dependency Demo Mode): Want to explore the UI immediately without spinning up backend containers?
cd frontend
cp .env.example .env.local  # Ensures NEXT_PUBLIC_DEMO_MODE=true
npm install && npm run dev
Open http://localhost:3000 (or 3100) to launch with pre-populated signal streams and graph context.

1. Strategy Dashboard (`/dashboard`)

System Health Telemetry: Live operational status checks across FastAPI Gateway, PostgreSQL/pgvector, Redis Streams, and Neo4j.
5-Node LangGraph Agent Flow: Real-time visual tracking of the cognitive state machine (EntityExtractor → CorrelationEngine → TrendAnalyzer → AnomalyDetector → BriefingComposer) with live terminal execution logs.
Streamed Event Feed: Server-Sent Events (SSE) stream pushing high-relevance signals (SEC filings, earnings call audio triggers, patent grants) directly to the timeline.

2. Signal Explorer (`/signals`)

Multi-Modal Source Triage: Category filter tabs to segment feeds across Web, SEC Filings, Audio Transcripts, Job Boards, and Patents.
Vector Match Scoring: Real-time cosine similarity relevance badges (cos 94%, cos 89%) generated via dense vector retrieval.
S3 Raw Payload Inspection: One-click Inspect payload drawers allow analysts to drill down into raw JSON documents offloaded to MinIO S3 storage.

📐 Architectural Principles & Design Paradigms

🌊 Event-Driven Microservices (Redis Streams)

Redis Streams provides the event backbone for all inter-service communication. The platform does not use Celery or traditional task queues because:

Consumer groups enable parallel partitioned consumption — each tenant's stream (tenant:{id}:*) is consumed independently by a consumer group, guaranteeing exactly-once processing per consumer without external coordination.
Persistence — Redis Streams append-only log retains events durably with configurable MAXLEN trimming, enabling replay for debugging and reprocessing without external message brokers.
Backpressure — XREADGROUP ... BLOCK with configurable batch sizes provides natural backpressure: consumers pull at their own rate, and the pending entry list (PEL) automatically tracks in-flight messages for at-least-once delivery.
Latency — Sub-millisecond publish latency enables real-time signal propagation from ingestion through intelligence processing to briefing generation.

📍 Pointer-Only State Management (LangGraph)

LangGraph checkpoints persist only lightweight pointers (S3/MinIO URIs, Redis keys, entity IDs) rather than the full multi-modal payloads they reference:

Checkpoint size stays under 5 KB regardless of input volume. A typical ingestion run may produce 1.5 MB of raw HTML and extracted entities, but the LangGraph state dict contains only the S3 URI (s3://stratops-prod/tenants/{tid}/docs/{doc_id}.json) and metadata fields (entity_count, extracted_entity_ids).
Node functions dereference pointers at execution time — the CorrelationEngineNode reads entity IDs from state, queries Neo4j for the actual entity data, and operates on the live graph rather than stale snapshots.
Checkpoint serialization cost drops from ~200 ms (full state JSON) to <10 ms (pointer dict), enabling fast graph resumption and sub-second checkpoint intervals.

⚡ vLLM-Backed Model Serving (BentoML + PagedAttention)

All LLM inference runs on vLLM's LLMEngine via BentoML service wrappers:

PagedAttention — KV-cache is allocated in fixed-size blocks shared across requests, enabling O(1) GPU memory allocation per token and supporting 3-5x higher concurrent request throughput compared to contiguous allocation.
Continuous batching — vLLM dynamically batches incoming requests mid-generation, maintaining GPU utilization at >85% under variable load without requiring explicit batch size tuning.
Guided decoding — Extraction services enforce structured JSON output using GBNF (Grammatical BNF) grammar constraints, guaranteeing parseable output without post-generation regex cleanup.
Adaptive scheduling — BentoML's @batchable decorator with max_batch_size and max_latency_ms parameters controls batch formation per service (e.g., extraction: 32 requests / 50 ms; fallback: 64 requests / 30 ms).

🔐 Deterministic Security & Multi-Tenancy

Tenant isolation is enforced at the database layer, not via application-level filtering:

PostgreSQL Row-Level Security (RLS) — Each API request sets APP_TENANT_ID as a session variable. RLS policies on every table enforce WHERE tenant_id = current_setting('app.tenant_id'), making cross-tenant data leakage structurally impossible.
Table partitioning — PARTITION BY LIST (tenant_id) on high-volume tables (signals, embeddings, documents) enables per-tenant index maintenance and query planning without full-table scans.
Neo4j tenant filtering — All Cypher queries include WHERE n.tenant_id = $tid predicates, enforced by a shared TenantSession wrapper.

🧩 Micro-Batched Graph Mutation

Neo4j write operations are buffered in Redis and flushed via UNWIND ... MERGE Cypher to prevent deadlocks:

Redis buffer — Entity updates are LPUSHed to stratops:tenant:{tid}:graph:pending as JSON payloads. An in-memory buffer mirrors the Redis list for crash recovery.
Deduplication — On flush, updates are deduplicated by (entity_id, rel_type), keeping only the latest write. This prevents redundant MERGE operations on Neo4j.
Batch flush — The GraphWriterWorker flushes on either batch_size threshold (100 updates) or flush_interval_ms (500 ms), whichever comes first. The flush executes UNWIND $batch AS row MERGE (n:Entity {id: row.entity_id}) SET n += row.properties, reducing per-entity round-trips from O(n) to O(1).
Deadlock avoidance — Neo4j acquires write locks per-node; micro-batching with UNWIND presents a single Cypher transaction per flush rather than hundreds of individual transactions, eliminating lock contention under concurrent ingestion.

⚙️ Core Subsystem Specifications

🚪 API Gateway Layer (`backend/api/`)

Framework — FastAPI with lifespan-managed startup (DB pool, Redis connection, Neo4j driver).
Authentication — Dual-mode: JWT Bearer tokens (HMAC-SHA256, access + refresh rotation) and API key authentication (X-API-Key header).
Tenant Context — TenantContextMiddleware extracts X-Tenant-ID header, validates as UUID, stores on request.state.tenant_id. All downstream operations inherit this context.
Rate Limiting — Redis-backed sliding-window rate limiter using atomic Lua scripts (ZREMRANGEBYSCORE + ZCARD + ZADD). Limits are per-tenant or per-IP with configurable ceilings (default: 100 req/min). Fails open on Redis errors.
Cost Tracking — Middleware injects X-Cost-Tier headers for billing attribution per request class (extraction, embedding, summarization).
Health Probes — /health (liveness), /health/ready (readiness: Postgres + Redis + Neo4j connectivity), /health/tenant (RLS validation).

🚌 Event Bus Layer (`backend/streams/`)

Stream Topology — Per-tenant partitioning: stratops:tenant:{id}:events, stratops:tenant:{id}:graph:buffer, stratops:tenant:{id}:briefings.
Key Construction — StreamKeyBuilder generates stream names from tenant ID and event type, ensuring consistent key formatting across producers and consumers.
Consumer Groups — Named consumer groups (e.g., cg:extraction, cg:graph_writer) enable parallel consumption. Each consumer claims messages via XREADGROUP ... > and acknowledges via XACK.
Dead-Letter Queue — Messages that fail processing after retry limits are routed to stream:{type}:dlq for manual inspection rather than being dropped.
Retry Logic — _with_redis_retry wraps all Redis operations with exponential backoff (1s, 2s, 4s, 8s, 16s) and configurable max retries.

📥 Ingestion Engine (`backend/ingestion/`)

Adapter Registry — SourceAdapter abstract base class with concrete implementations: WebMonitorAdapter (Playwright), SECFilingAdapter (EDGAR RSS), EarningsAdapter (transcript parsing), PatentAdapter (patent databases).
Content Fingerprinting — SimHash generates 64-bit fingerprints from raw text. Identical or near-duplicate content (edit distance <3) produces the same fingerprint, enabling deduplication before storage.
Payload Offloading — Raw payloads exceeding 10 KB are offloaded to MinIO at s3://stratops-{env}/tenants/{tid}/raw/{fingerprint}.json. The signal record retains only the S3 URI.
Normalized Emission — Every adapter emits a standardized RawSignal Pydantic model with fields: source_type, raw_content, fingerprint, collected_at, structured_data, metadata.

🕸️ BentoML Model Mesh (`bentoml/services/`)

Service	Model	Quantization	GPU	Max Batch	Timeout	Purpose
`extraction-svc`	Qwen2.5-7B-Instruct	AWQ 4-bit	1× A10G 24GB	32	30s	Structured JSON extraction with GBNF guided decoding
`embedding-svc`	bge-large-en-v1.5	FP16	1× A10G 24GB	64	30s	1024-dim vector generation for pgvector storage
`fallback-svc`	Qwen2.5-3B-Instruct	GGUF	1× A10G 8GB	64	30s	Low-latency fallback on GPU timeout or circuit breaker open
`narrative-svc`	Qwen2.5-14B-Instruct	AWQ 4-bit	1× A10G 24GB	8	60s	Briefing synthesis and multi-document narrative generation

All services expose /metrics (Prometheus) and /health endpoints. GPU memory utilization targets 85-90% via vLLM's gpu_memory_utilization parameter.

🧠 Intelligence Core & Decision Engine (`backend/intelligence/`)

LangGraph Orchestrator — Stateful, graph-based agent orchestration managing deterministic execution pathways across 5 specialized cognitive nodes. Each node reads pointers from state, dereferences external data (Neo4j, S3, pg), operates on live data, and writes updated pointers back to state:
- EntityExtractorNode: Invokes extraction-svc via GBNF-constrained JSON schemas to parse entities, relations, and temporal timestamps (valid_from / valid_to) from raw signals.
- CorrelationEngineNode: Executes GraphRAG path traversal on Neo4j to link newly extracted entities with existing enterprise domain graphs, detecting hidden cross-source relationships.
- TrendAnalyzerNode: Evaluates signal velocity and trajectory using time-series STL decomposition and Z-score metrics to detect strategic inflection points.
- AnomalyDetectorNode: Runs Isolation Forest algorithms against feature embeddings to surface market deviations, competitive posture shifts, and operational anomalies.
- BriefingComposerNode: Synthesizes multi-source correlations and anomalies into structured, delta-updated executive briefings.
Self-Correcting RAG Loops — Evaluates output structured JSON against validation rules; automatically triggers bounded retry loops with feedback prompts on schema or parsing failures.

💾 Multi-Tier Storage & Memory Architecture (`backend/db/`)

StratOps-Intel implements a 5-tier storage and memory system optimized for low latency, long-term retention, tenant isolation, and minimal state overhead:

Execution Memory (LangGraph State Checkpointing):
- Purpose: Ephemeral state passed between agent nodes during pipeline execution.
- Mechanism: Pointer-only checkpointing under 5 KB by storing MinIO object URIs and DB IDs rather than bulky raw text or vector blobs.
Semantic Memory (PostgreSQL 16 + pgvector):
- Purpose: Unstructured signal history, document chunks, and dense vector embeddings.
- Mechanism: Tenant-scoped RLS policies, PARTITION BY LIST (tenant_id) partitioning, partitioned HNSW cosine similarity indexes, and hybrid retrieval (sparse ts_rank_cd + dense cosine via Reciprocal Rank Fusion).
Structural & Temporal Memory (Neo4j 5 Property Graph):
- Purpose: Relational entity-attribute-value triples, organizational hierarchies, and competitive landscapes over time.
- Mechanism: Temporal property graph enforcing valid_from / valid_to relationship bounds for point-in-time Cypher queries; all traversals enforce tenant_id predicates.
Artifact & Raw Storage Memory (MinIO / S3):
- Purpose: Long-term immutable persistence of raw payloads (HTML, SEC filings, transcripts) and final Markdown briefing outputs.
- Mechanism: Multi-tenant buckets accessed via deterministic URI schemes (s3://stratops-{env}/tenants/{tid}/{category}/{id}.json).
Caching & Buffering Layer (Redis 7):
- Purpose: Low-latency ephemeral coordination across services.
- Mechanism: Multi-role engine serving Redis Streams (event bus), semantic cache (embedding similarity), sliding-window rate limiting, and UNWIND ... MERGE graph write buffering.

🤖 Agent Nodes, Specialized Tools & MCP Invocation

StratOps-Intel strictly decouples LLM reasoning (Agent Nodes) from deterministic execution (Specialized Tools):

Cognitive Agent Nodes (backend/intelligence/agents/):
- Stateful LangGraph nodes (EntityExtractorNode, CorrelationEngineNode, TrendAnalyzerNode, AnomalyDetectorNode, BriefingComposerNode) executing LLM decision logic, schema validation, and self-correcting retry loops.
Deterministic Specialized Tools (backend/intelligence/tools/):
- SimHash Tool: 64-bit Hamming distance content deduplication.
- STL Momentum Tool: Time-series seasonal-trend decomposition & Z-score trajectory analysis.
- Isolation Forest Tool: Vector embedding anomaly and outlier detector.
- Neo4j Cypher Micro-Batcher: Transactional graph mutation tool (UNWIND...MERGE).
Model Context Protocol (MCP) & Invocation Architecture:
- In-Process Tools: Algorithmic and database search utilities bound directly to LangGraph nodes via Pydantic tool schemas.
- MCP Protocol Servers: External scrapers (Playwright), document parsers (SEC EDGAR), and custom connectors run as isolated MCP Servers, allowing Agent Nodes (MCP Clients) to discover (tools/list) and execute (tools/call) capabilities over standard JSON-RPC interfaces.

🛡️ Resilience, Observability & Routing

Circuit Breaker — Three-state breaker (CLOSED → OPEN → HALF-OPEN) on every model service call. Configurable failure_threshold (default: 3) and recovery_timeout (default: 30s). On OPEN, requests fast-fail to fallback-svc without waiting for timeout.
A/B Model Router — Two routing modes: deterministic tenant-hash (same tenant always routes to same variant) and weighted random (configurable traffic percentages). Experiments are tracked via X-Experiment-Id / X-Experiment-Variant headers.
OpenTelemetry Tracing — OTLP/gRPC exporter with auto-instrumentation for FastAPI, SQLAlchemy, and httpx. Manual span wrappers for Redis xadd/xreadgroup and Neo4j Cypher executions. TraceContextFilter injects trace_id/span_id into every structlog event for log-to-trace correlation.
Prometheus Metrics — bentoml_inference_duration_seconds (histogram), bentoml_batch_size_distribution, bentoml_gpu_memory_utilization_ratio, bentoml_queue_length (gauge), bentoml_requests_total / bentoml_errors_total (counters).

🔁 End-to-End Data Pipeline Execution Flow

1. RAW INGESTION
   SourceAdapter (Playwright/EDGAR/etc.)
     → SimHash fingerprint deduplication
     → Raw payload offload to MinIO (s3://.../raw/{fingerprint}.json)
     → Emit RawSignal to Redis Stream (stratops:tenant:{id}:events)

2. SIGNAL EXTRACTION
   IngestionWorker (XREADGROUP from events stream)
     → EntityExtractorNode via extraction-svc (vLLM Qwen2.5-7B + GBNF)
     → Structured entities (JSON) + embeddings (bge-large-en-v1.5)
     → Entity updates LPUSH to graph:pending buffer in Redis
     → Extracted entity IDs written to LangGraph state (pointer-only)

3. GRAPH CORRELATION
   GraphWriterWorker (flushes graph:pending buffer)
     → Deduplication by (entity_id, rel_type)
     → UNWIND ... MERGE Cypher to Neo4j (batch write)
     → Neo4j temporal indices updated (valid_from/valid_to)

4. TREND & ANOMALY ANALYSIS
   IntelligenceWorker
     → TrendAnalyzerNode: Z-score + STL decomposition on entity time-series
     → AnomalyDetectorNode: Isolation Forest scoring
     → Results written to pgvector (for hybrid retrieval) and S3 (pointers to state)

5. BRIEFING GENERATION
   BriefingComposerNode
     → Reads entity pointers from LangGraph state
     → Fetches latest entity data from Neo4j
     → LLM synthesis (Qwen2.5-14B) with delta updates
     → Living briefing document stored in S3, delta summary in Redis

6. CLIENT DELIVERY
   Real-time push via SSE or polling
     → API Gateway serves briefing + signal data
     → Frontend (Next.js) renders dashboard with live updates

✨ Features

Feature	Description
Multi-modal Signal Ingestion	Web scraping (Playwright), SEC EDGAR filings, earnings transcripts, patent databases
Neural Structured Extraction	vLLM + Qwen2.5-7B with guided JSON decoding and adaptive batch scheduling
Temporal Knowledge Graph	Neo4j-backed entity relationships with time-windowed Cypher queries
Hybrid Vector Retrieval	pgvector cosine + BM25 sparse ranking via Reciprocal Rank Fusion (RRF)
Pointer-Only State	LangGraph checkpoints under 5 KB — heavy payloads in S3/Redis, state holds only pointers
Circuit-Breaked Fallback	Three-state breaker (CLOSED/OPEN/HALF-OPEN) auto-routes to fallback model on GPU timeout
A/B Model Routing	Deterministic tenant-hash and weighted random routing for model experiments
Distributed Tracing	OpenTelemetry OTLP exporter with FastAPI/SQLAlchemy/Redis/Neo4j span instrumentation
Grafana Observability	Pre-built dashboards: API latency, GPU utilization, stream lag, storage saturation
Multi-Tenant Isolation	PostgreSQL RLS, per-tenant Redis stream partitions, Neo4j tenant_id filtering

🚀 Quickstart

📋 Prerequisites

Python 3.11+, Node.js 20+
Docker Engine 24+ with Docker Compose v2
NVIDIA GPU with 24 GB VRAM (for BentoML model serving)

1. 📦 Clone and Install

git clone https://github.com/aragit/stratops-intel.git
cd stratops-intel

# Backend
python -m venv .venv && source .venv/bin/activate
pip install -r backend/requirements.txt

# Frontend
cd frontend && npm ci && cd ..

2. 🐳 Start Infrastructure

docker compose -f docker/docker-compose.infra.yml up -d
# Starts: PostgreSQL 16, Redis 7, Neo4j 5, Prometheus, Grafana

3. 🗃️ Apply Database Migrations

cd backend && alembic upgrade head && cd ..

4. ⚡ Launch BentoML Model Services (requires GPU)

docker compose -f docker/docker-compose.bentoml.yml up -d
# Starts: extraction-svc:3000, embedding-svc:3001, fallback-svc:3002

5. 🌐 Start the API Gateway

cd backend && uvicorn api.gateway:app --host 0.0.0.0 --port 8000 --reload

6. 🖥️ Start the Frontend

cd frontend && npm run dev
# Dashboard at http://localhost:3000

7. ✅ Verify

curl http://localhost:8000/health/ready
# {"status": "healthy", "postgres": "ok", "redis": "ok", "neo4j": "ok"}

🔧 Environment Variables

Variable	Default	Description
`DATABASE_URL`	`postgresql+asyncpg://localhost/stratops_intel`	Async PostgreSQL DSN with pgvector
`REDIS_URL`	`redis://localhost:6379/0`	Redis connection URL
`NEO4J_URI`	`bolt://localhost:7687`	Neo4j Bolt URI
`NEO4J_USER`	`neo4j`	Neo4j auth user
`NEO4J_PASSWORD`	`changeme`	Neo4j auth password
`JWT_SECRET`	(required)	HMAC-SHA256 signing key (min 32 bytes)
`EXTRACTION_MODEL`	`Qwen/Qwen2.5-7B-Instruct-AWQ`	vLLM extraction model
`FALLBACK_MODEL`	`Qwen/Qwen2.5-3B-Instruct-GGUF`	vLLM fallback model
`EMBEDDING_MODEL_NAME`	`all-MiniLM-L6-v2`	Sentence-Transformer for cache embedding
`SEMANTIC_CACHE_THRESHOLD`	`0.92`	Cosine similarity threshold for cache hit
`RATE_LIMIT_REQUESTS_PER_MINUTE`	`100`	Sliding-window rate limiter ceiling
`OTEL_EXPORTER_OTLP_ENDPOINT`	`localhost:4317`	OpenTelemetry collector gRPC endpoint

📁 Project Structure

stratops-intel/
├── backend/                          # Python backend (FastAPI + LangGraph)
│   ├── api/                          # Gateway, auth, middleware, rate limiter
│   ├── db/                           # PostgreSQL RLS, Alembic, Neo4j client, pgvector
│   ├── streams/                      # Redis Streams producer/consumer base
│   ├── ingestion/                    # Source adapters (web, SEC, earnings, patents)
│   ├── intelligence/                 # LangGraph agents (extractor, correlation, trend, anomaly, composer)
│   ├── workers/                      # Ingestion, intelligence, graph writer workers
│   ├── resilience/                   # Circuit breaker, retry logic
│   ├── experiments/                  # A/B model router
│   ├── observability/                # OpenTelemetry tracing
│   ├── cache/                        # Tenant-isolated semantic cache
│   ├── mcp/                          # FastMCP tool server
│   ├── alerts/                       # Rule engine, Slack/email/webhook router
│   ├── briefings/                    # Delta generator, repository
│   └── tests/                        # Unit, integration, chaos test suites
├── bentoml/                          # BentoML + vLLM model services
│   ├── services/                     # extraction-svc, embedding-svc, fallback-svc
│   ├── composed/                     # Entity pipeline (chained services)
│   └── observability/                # Prometheus metrics exporter
├── frontend/                         # Next.js 14 App Router dashboard
│   ├── app/                          # Dashboard, signals, briefings, graph pages
│   └── lib/                          # API client (TanStack Query)
├── benchmarks/                       # Batch sweep, load test, reports
├── observability/grafana/            # Grafana dashboard JSON
├── docs/                             # ADRs, benchmarks, deployment guide
│   └── adr/                          # Architecture Decision Records
├── docker/                           # Docker Compose files
└── .github/workflows/                # CI, security, release workflows

🧪 Quality Assurance & Test Suite

StratOps-Intel maintains a strict multi-tier testing matrix across both backend microservices and the Next.js frontend application, guaranteeing zero-downtime resiliency, schema validation, and real-time UI fidelity.

Automated Testing Matrix Summary

Tier	Layer	Framework / Tool	Test Count / Coverage	Objective
Backend	Unit & Integration	`pytest` + `pytest-asyncio`	418 Passed	API endpoints, LangGraph cognitive nodes, GBNF schema parsing
Backend	Chaos & Resiliency	`pytest` + `chaos-engineering`	9 Passed	Database failover, Redis buffer overflow, model mesh timeouts
Backend	Linting & Typing	`ruff` + `mypy`	Strict (PEP 8)	Static code analysis and strong typing validation
Frontend	Unit & Component	`vitest` + React Testing Library	19 Passed	React hooks (`useSignalStream`), TenantContext, Markdown diff rendering
Frontend	End-to-End (E2E)	`playwright` (Headless Chromium)	7 Passed	Full user flow automation, route mocking, SSE real-time state
Frontend	Typecheck & Lint	`tsc` + `eslint`	Clean	Next.js 14 App Router type safety and syntax validation

Executing Backend Test Suite

cd backend

# Run fast unit test suite (418+ tests)
pytest tests/unit/ -v --timeout=30

# Run infrastructure chaos & fault injection tests (9 tests)
pytest tests/chaos/ -v --timeout=30

# Run static linting and formatting verification
ruff check .
ruff format --check .

# Run full combined backend test suite quietly
pytest tests/unit/ tests/chaos/ -q --timeout=30

Executing Frontend Test Suite

cd frontend

# Run Vitest unit & component test suite (19 tests)
npm run test

# Run Playwright E2E browser automation suite (7 specs)
npm run test:e2e

# Run TypeScript type safety and lint checks
npm run typecheck && npm run lint

📊 Benchmarks

See docs/BENCHMARKS.md for detailed results.

Metric	Target	Achieved
Batch throughput (size=16)	> 800 tok/s	880 tok/s
Error rate (100 concurrent)	< 0.1%	0.20%
p99 inference latency (batch=8)	< 350 ms	287 ms
GPU VRAM utilization	< 95%	~88%

☸️ Deployment

See docs/DEPLOYMENT.md for Docker Compose and Kubernetes instructions.

📜 Architecture Decision Records

ADR	Decision
ADR 001	BentoML + vLLM Model Mesh over Raw Transformers
ADR 002	Pointer-Only LangGraph State Management (<5 KB checkpoints)

🤝 Contributing

We welcome professional contributions to StratOps-Intel! We appreciate your help in improving the platform's neural extraction capabilities, temporal graph handling, or multi-tenant architecture.

To contribute, please follow these guidelines to ensure a smooth collaboration process.

Our Development Workflow

We use a standard GitHub flow for development:

Fork and Clone: Fork the repository and clone your fork to your local machine.
Check Issues: Browse the issue tracker for open tasks. If you have a new idea, please open a detailed issue discussion first.
Create a Feature Branch: Create a branch for your work using a descriptive name (e.g., feature/gbnf-validation-error-handling or fix/neo4j-temporal-query-deadlock). Do not work directly on main.
Local Development & Environment:
- Ensure you have the required prerequisites installed (Python 3.11+, Docker Compose).
- Install development dependencies: pip install -r backend/requirements-dev.txt.
- Set up pre-commit hooks to automatically run linting and code formatting checks: pre-commit install.
Code Standards: We enforce strict coding standards to maintain code quality.
- Python: Follow PEP 8 styles. We use ruff for linting and black style formatting.
- Typing: All Python code must use strong typing (Mypy).
- GBNF/Pydantic: Ensure any updates to the extraction schemas maintain GBNF compatibility and Pydantic validation.
Writing Tests: Contributions will not be accepted without corresponding tests.
- Write units tests (pytest) for backend components (backend/tests/).
- Write integration tests for API endpoints.
- Ensure all existing tests pass before submitting.
Commit Messages: Write clear, descriptive commit messages. Follow standard conventional commit guidelines.
Submit a Pull Request (PR): Push your branch to GitHub and open a PR against the main branch.
- Fill out the PR template completely.
- Reference the issue your PR addresses.
- Ensure all CI/CD checks pass.

Code of Conduct

All contributors are expected to uphold our Code of Conduct, fostering a welcoming and respectful environment for collaboration.

📚 Citation

If you use StratOps-Intel in your research, commercial derivative works, or publications, please cite this project to acknowledge its contribution to your work.

We recommend using the following BibTeX entry for academic citations:

@software{stratopsintel2024,
  author = {Nicoomanesh, Arash},
  title = {StratOps-Intel: Enterprise AI Platform for Multi-Modal Signal Intelligence},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub Repository},
  howpublished = {\url{https://github.com/aragit/stratops-intel}},
}

For general attribution, please use the following APA style:

Arash Nicoomanesh. (2024). StratOps-Intel: Enterprise AI Platform for Multi-Modal Signal Intelligence [Computer software]. GitHub. https://github.com/aragit/stratops-intel

🚀 Future Work & Architectural Roadmap

While StratOps-Intel delivers a state-of-the-art neuro-symbolic intelligence architecture, future iterations will focus on eliminating operational bottlenecks, reducing node redundancy, and scaling inference throughput.

1. Infrastructure Consolidation & Dynamic Scaling

KEDA Autoscaling: Implement Kubernetes Event-driven Autoscaling (KEDA) based on Redis Stream consumer group lag to automatically scale vLLM worker pods up or down on Ray clusters.
Storage Consolidation: Offload secondary relational graph queries to Apache AGE on PostgreSQL, reserving Neo4j exclusively for multi-hop temporal traversals to streamline infrastructure management.

2. Asynchronous Canonicalization & Entity Disambiguation

Vector-Assisted Entity Linking: Intercept incoming entity candidates in an asynchronous staging table using pgvector cosine similarity and SimHash fingerprinting prior to Neo4j graph insertion.
APOC Master Node Merging: Execute background graph maintenance jobs via Neo4j APOC procedures (apoc.refactor.mergeNodes) to merge duplicate entity aliases into canonical nodes without breaking temporal history.

3. LangGraph DAG Parallelism & Early Short-Circuiting

Parallel Branch Execution: Refactor the sequential 5-node agent pipeline into a Directed Acyclic Graph (DAG) to run TrendAnalyzer and AnomalyDetector concurrently after CorrelationEngine completes.
Speculative Gating: Implement a gatekeeper threshold check following anomaly detection to bypass the costly BriefingComposer node when signals fall below minimum confidence thresholds.

4. Speculative Decoding & Tiered Routing Mesh

Speculative Token Generation: Pair lightweight local draft models (Qwen2.5-3B-Instruct GGUF) with larger target reasoning models (DeepSeek-R1-Distill-32B) to accelerate token throughput while maintaining output quality.
Dynamic Complexity Routing: Integrate LiteLLM semantics-based routing to direct standard entity extraction to 3B/7B quantized endpoints while auto-escalating complex regulatory filings to higher-parameter models.

📄 License

MIT

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
assets		assets
backend		backend
benchmarks		benchmarks
bentoml		bentoml
docker		docker
docs		docs
frontend		frontend
observability/grafana		observability/grafana
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
README.md		README.md

Latest commit

History

Folders and files

Repository files navigation