Semantic N-Body

N-body gravitational diffusion in 128-dimensional semantic space. Takes behavioral anchors (embeddings from activity recognition), simulates gravitational attraction between them, and extracts activity chains from the resulting clusters.

Written in Julia with KernelAbstractions.jl for vendor-neutral GPU compute (AMD/NVIDIA/CPU from the same source code).

How it works

1. Each anchor has a 128-dimensional embedding placed on the unit hypersphere
2. Anchors attract each other via inverse-square gravitational force, weighted per dimension group (temporal, spatial, weather, lighting, activity, rhythm, learned)
3. After 250 iterations of force/velocity/position updates with momentum and damping, nearby anchors form clusters
4. BFS connected components extraction with adaptive distance thresholding produces activity chains

graph LR
    F[Forge] -->|dispatch job_id| N[Nomad]
    N -->|run worker.sif| W[Singularity Worker]
    W <-->|pub/sub<br/>compute/jobs/JOB_ID/*| B[MQTT Broker]
    F -.->|publish params<br/>read result| B

Loading

For why the design works the way it does (physics rationale, dimension weighting, threshold heuristic), see CONCEPTS.md. For the technical structure (components, data flow, invariants), see ARCHITECTURE.md. For local development setup and troubleshooting, see docs/development.md.

Prerequisites

• Julia 1.12.5+
• libmosquitto-dev (for MQTT — only needed if running the worker, not for tests)

Quick start

# Install dependencies
julia --project=. -e 'using Pkg; Pkg.instantiate()'

# Run tests (75 tests)
julia --project=. test/runtests.jl

# Run local integration test (no MQTT needed)
julia --project=. test_local.jl

Project structure

src/
  SemanticNBody.jl    Main module
  types.jl            Data structures (immutable except SimulationState)
  config.jl           Configuration from environment variables
  physics.jl          Force computation, velocity/position updates, diffusion loop
  kernels.jl          KernelAbstractions.jl GPU kernels + backend selection
  chains.jl           Pairwise distances, adaptive threshold, BFS extraction
  serialization.jl    JSON3 struct mappings
  mqtt.jl             Mosquitto.jl MQTT client
  app.jl              Entry point (julia_main, process_job)

test/                 Unit tests (physics, chains, kernels, serialization)
test_data/            Reference test fixtures
deploy/
  nomad/semantic-n-body.hcl   Nomad parameterized batch job definition
.gitea/workflows/build.yml    CI/CD: builds Singularity image and registers Nomad job on push to main

MQTT protocol

The worker is an MQTT client that processes exactly one job and exits (Nomad dispatch pattern).

Topics:

• Subscribe: compute/jobs/{JOB_ID}/params (QoS 1)
• Publish result: compute/jobs/{JOB_ID}/result (QoS 1)
• Publish status: compute/jobs/{JOB_ID}/status (QoS 0)
• Publish logs: compute/jobs/{JOB_ID}/logs (QoS 0)

Job input (JSON on params topic):

{
  "job_id": "daily-2026-02-21",
  "anchors": [
    {
      "id": "morning-routine-00",
      "timestamp": 1762142400,
      "location": "bedroom",
      "embedding": [0.264, 0.0, ...]
    }
  ]
}

Job output (JSON on result topic):

{
  "job_id": "daily-2026-02-21",
  "success": true,
  "result": {
    "anchors": 100,
    "chains": 21,
    "iterations": 250,
    "convergence": 0.000392,
    "chain_assignments": [
      {"chain_id": 1, "anchor_ids": ["morning-routine-00", "morning-routine-01"]}
    ]
  },
  "worker_id": "nomad-abc123",
  "timestamp": "2026-02-21T10:00:00.000Z"
}

Environment variables

Variable	Required	Default	Description
JOB_ID	yes	—	Job identifier
MQTT_BROKER	no	tcp://localhost:1883	Broker URL
MQTT_USER	no	—	MQTT auth username
MQTT_PASSWORD	no	—	MQTT auth password
WORKER_ID	no	worker-{uuid}	Worker identifier
USE_GPU	no	false	Enable GPU backend
SYSIMAGE_PATH	no	/app/sysimage.so	Path to precompiled sysimage

Building

Singularity (production)

singularity build --fakeroot worker.sif singularity.def

CI/CD handles this automatically via the Gitea workflow on push to main.

Deploying

Nomad

The job is a parameterized batch job dispatched by Forge. Register the job definition:

nomad job run deploy/nomad/semantic-n-body.hcl

Dispatch manually for testing:

nomad job dispatch -meta job_id=<uuid> semantic-n-body

Secrets (MQTT_BROKER, MQTT_USER, MQTT_PASSWORD) are injected from Vault at secret/data/nomad/forge.

Performance

On a 12-thread CPU (Apple M-series):

• 100 anchors: ~29ms (after JIT warmup)
• 976 anchors: ~1.2s
• Daily workload (~150 anchors): trivial, runs on a Raspberry Pi 4

With PackageCompiler sysimage, cold startup drops from ~30s to ~1s.

Documentation

Document	Purpose
CONCEPTS.md	Why the algorithm works the way it does — physics rationale, dimension weighting, chain extraction heuristic
ARCHITECTURE.md	System structure, components, data flows, invariants
docs/development.md	Local setup, tests, GPU dev, profiling, troubleshooting
docs/datamodel.md	Type definitions and JSON wire format
docs/api-reference.md	Reference for exported functions
docs/messaging.md	MQTT topic schema, QoS, message lifecycle
docs/subsystems/physics/	CPU diffusion loop
docs/subsystems/gpu-backend/	KernelAbstractions kernels and AMDGPU
docs/subsystems/chains/	Adaptive threshold and BFS extraction
docs/subsystems/worker/	MQTT client and job lifecycle
docs/subsystems/deployment/	Singularity, Nomad, Vault, Gitea CI/CD
docs/subsystems/sysimage/	PackageCompiler sysimage build (and the AMDGPU LLVM workaround)

License

MIT