Skip to content

development.md

Latest commit

 

History

History
80 lines (63 loc) · 4.83 KB

development.md

File metadata and controls

80 lines (63 loc) · 4.83 KB
Raw

Development

Daedalus is a layered Rust workspace. Keep changes inside the layer that owns the behavior.

Layout

  • crates/core: shared primitive types only.
  • crates/transport: generic payload transport contracts.
  • crates/data: portable value/type/descriptor model.
  • crates/registry: declarations and capability snapshots.
  • crates/planner: graph validation, lowerings, adapter selection, and scheduling inputs.
  • crates/runtime: runtime plans, executor, host bridge, streaming, state, and telemetry.
  • crates/engine: application-facing facade over registry/planner/runtime.
  • crates/gpu: backend selection and GPU resource/dispatch support.
  • crates/ffi: language-neutral contracts, host runner, and language SDK integration.
  • examples: runnable examples and plugin fixtures.
  • scripts and testing: local validation and CI support.

Validation Loop

Run the default loop before sending broad changes:

./scripts/repo-clean.sh
cargo fmt --all -- --check
./scripts/check-file-sizes.sh
./scripts/check-workspace-deps.sh
./scripts/check-gpu-async-blocking.sh
cargo test --workspace --all-targets --features "engine,plugins"
cargo clippy --workspace --all-targets --features "engine,plugins" -- -D warnings
cargo doc --workspace --no-deps

Use gpu-mock for deterministic GPU-path tests and gpu-wgpu only on machines with a real backend available.

Dependency Policy

  • Workspace crates should inherit common dependencies from root workspace.dependencies.
  • Library error types should be typed and use thiserror.
  • Runtime instrumentation should use tracing.
  • Avoid adding dependencies to core and transport unless the owning contract truly requires them.
  • Keep backend variants behind stable feature names.
  • Use cargo tree -d --workspace during release review and treat new duplicate dependency roots as review input.

Public API Policy

  • Prefer fallible graph and registry construction in production code: try_connect, try_connect_ports, try_merge, try_on, and related helpers.
  • Prefer typed PortHandles or explicit (node, port) tuples for graph wiring. Plain host-port strings such as "input" are fine at host boundaries; dotted strings such as "node.output" are shorthand and should stay in tests, small demos, or compile-time-fixed graphs.
  • Panic-first helpers are acceptable in tests, small demos, and compile-time-fixed graph construction.
  • Public ids, type keys, package descriptors, and telemetry fields should remain deterministic because planner/runtime goldens and FFI fixtures depend on stable serialization.
  • Host-facing queues should be bounded deliberately; internal finite graph edges default to compatibility-oriented FIFO behavior unless a graph or runtime config selects otherwise.

Observability

  • Initialize a tracing subscriber in binaries and integration tests.
  • Useful targets include daedalus_runtime::executor, daedalus_runtime::executor::queue, daedalus_runtime::host_bridge, daedalus_runtime::stream, daedalus_runtime::config, daedalus_planner::passes, daedalus_gpu::wgpu, daedalus_gpu::dispatch, daedalus_gpu::readback, and daedalus_gpu::poll_driver.
  • Metrics levels are Off, Basic, Timing, Detailed, Hardware, Profile, and Trace.
  • Detailed is the normal level for transport and allocation debugging.
  • Profile adds per-node profile snapshots.
  • Trace records lifecycle-level data movement details.
  • DAEDALUS_NODE_CPU_TIME=1 enables Linux per-node CPU timing.
  • DAEDALUS_NODE_PERF_COUNTERS=1 enables Linux perf counters.

Runtime Defaults

  • Stream workers use DEFAULT_STREAM_IDLE_SLEEP unless configured through EngineConfig, RuntimeSection, or StreamWorkerConfig.
  • Host bridge event recording is enabled by default and retains DEFAULT_HOST_BRIDGE_EVENT_LIMIT events per bridge.
  • Internal edge queues preserve compatibility defaults; streaming, camera, daemon, and interactive workloads should set explicit bounded/latest-only policies.
  • WGPU staging behavior is configured through WgpuStagingPoolConfig or DAEDALUS_WGPU_STAGING_* before backend construction.

Troubleshooting

Symptom First checks
Missing adapter Inspect planner diagnostics and RuntimePlan::explain(). Confirm source and target TypeKey values and registered adapter declarations.
Type mismatch Log producer payload.type_key() and compare it with the consumer adaptation target.
Queue pressure Enable RUST_LOG=daedalus_runtime::executor::queue=trace and inspect the edge policy and pressure reason.
GPU unavailable Reproduce with gpu-mock, then retry gpu-wgpu with RUST_LOG=daedalus_gpu::wgpu=trace.
Missing host output Inspect host_events() and output queue policy.
FFI worker failure Check BackendConfig, protocol version negotiation, worker stderr, and normalized InvokeResponse validation errors.