Skip to content

TESTING.md

Latest commit

 

History

History
134 lines (106 loc) · 6.06 KB

TESTING.md

File metadata and controls

134 lines (106 loc) · 6.06 KB
Raw

Testing

This project uses Unity Test Framework EditMode tests for production-core code and PlayMode smoke tests for scene-facing UI/runtime behavior.

EditMode Tests

Current coverage starts with:

  • VisionFrameResult result-count and pose keypoint behavior;
  • VisionPerformanceStats stage totals;
  • VisionImageTransform stretch, letterbox, normalized mapping, and source/model round trips.
  • NMSProcessor IoU, class-aware suppression, null input, and confidence ordering behavior.
  • IOUTracker and SORTTracker track ID stability, missed-frame lifecycle, reset behavior, and confirmation behavior.
  • VisionProfileValidator model and pipeline profile validation behavior.
  • VisionAdapterRegistry adapter registration, duplicate protection, resolution, and runtime creation behavior.
  • VisionHealthStatus lifecycle transitions for start, degradation, recovery, stop, and failure.
  • Parser selection diagnostics for explicit parser ids, unsupported profile combinations, and missing parser coverage.
  • Production IVisionFrameSource implementations for texture, render texture, Unity camera, and video contract behavior.
  • UnityInferenceRuntimeAdapter and YoloModelAdapter runtime creation, processing, and disposal behavior through fakes.
  • Parser contracts and YOLO golden row fixtures for YOLO11-style raw-grid outputs, YOLO26 end-to-end detection/pose/segmentation rows, COCO label fallback, segmentation contour reconstruction, and empty-mask rejection.
  • Parser registry and raw-output runtime flow through UnityInferenceRuntimeAdapter.
  • Unity Inference raw output provider construction and missing-asset failure behavior.
  • UI Toolkit dashboard receiver and overlay renderer PlayMode smoke behavior.
  • Performance/allocation budget policy for parser, overlay, tracking, and pipeline stages.
  • Setup Wizard scene configuration behavior.
  • Package sample manifest and file layout, including YOLO model-profile samples.
  • Optional module descriptor and package isolation rules.
  • Profile validation summary and dashboard budget status behavior.
  • Profile compatibility summary behavior for parser, runtime, capability, input, and output schemas.
  • Dashboard runtime context binding for source, model, runtime, and last error status.
  • Dashboard runtime controls for visualization toggles, model thresholds, and target FPS wiring.
  • Adapter registry behavior for implemented and unimplemented model families.
  • GPU output handle propagation for future mask/keypoint render paths without mandatory CPU readback.
  • Editor tooling entry points for Control Center and pipeline profile creation.

Run from PowerShell:

& 'D:\Unity\6000.5.1f1\Editor\Unity.exe' `
  -batchmode `
  -nographics `
  -projectPath 'D:\Projects\ProAnimaStudio\pas-UCT' `
  -runTests `
  -testPlatform EditMode `
  -testResults 'D:\Projects\ProAnimaStudio\pas-UCT\Temp\editmode-test-results.xml' `
  -logFile 'D:\Projects\ProAnimaStudio\pas-UCT\Temp\editmode-test.log'

Do not pass -quit with -runTests; Unity Test Runner exits on its own after the run.

On Unity 6000.5.1f1, the log may report that results were saved to both the requested path and Unity's LocalLow default path. If the requested XML is missing, check:

C:\Users\<user>\AppData\LocalLow\DefaultCompany\pas-UCT\TestResults.xml

Expected current baseline:

EditMode: 138 tests, 138 passed, 0 failed
PlayMode: 3 tests, 3 passed, 0 failed

Run PlayMode smoke tests from PowerShell:

& 'D:\Unity\6000.5.1f1\Editor\Unity.exe' `
  -batchmode `
  -projectPath 'D:\Projects\ProAnimaStudio\pas-UCT' `
  -runTests `
  -testPlatform PlayMode `
  -testResults 'D:\Projects\ProAnimaStudio\pas-UCT\Temp\playmode-test-results.xml' `
  -logFile 'D:\Projects\ProAnimaStudio\pas-UCT\Temp\playmode-test.log'

Run the same UPM package validation used by CI:

python scripts/validate_upm_package.py

GitHub Actions

The repository includes a GitHub Actions workflow:

.github/workflows/unity-editmode-tests.yml
.github/workflows/package-validation.yml

It uses GameCI Unity Test Runner to run Unity tests on push and pull requests. The package validation workflow checks the UPM manifest, sample paths, required documentation, assembly definitions, and the policy that model weights are not bundled in the package.

Required repository secrets:

UNITY_LICENSE
UNITY_EMAIL
UNITY_PASSWORD

For organization repositories, configure these secrets in GitHub repository settings before relying on CI.

Production Test Matrix

Current automated tests cover the pure runtime core. A production-ready release should expand coverage into these layers:

Layer Test Type Status
Result API EditMode unit tests Started
Coordinate mapping EditMode unit tests Started
Runtime dashboard geometry EditMode unit tests Started
Plugin/adapters contracts EditMode unit tests Started
Vision pipeline orchestration EditMode unit tests Started
NMS/postprocess math EditMode unit tests Started
Tracking lifecycle EditMode unit tests Started
Model profile validation EditMode unit tests Started
Parser output schemas Golden fixture tests Started
Frame sources EditMode + PlayMode tests Started
WebCam/RenderTexture path PlayMode/manual device tests Planned
Unity Inference runtime smoke PlayMode/batchmode tests Planned
UI overlay mapping PlayMode screenshot tests Planned
Performance/allocation budgets Performance tests Planned
AR/XR sources Platform integration tests Planned

Package Release Gate

Before tagging a release, verify:

  • package.json version matches the release tag.
  • The package installs through the git URL with ?path=/Packages/com.proanima.universal-vision-tracker.
  • Importable samples compile after import into a clean Unity project.
  • EditMode and PlayMode baselines pass.
  • No .onnx, .sentis, .tflite, .pt, or other large model-weight files are bundled in the package unless explicitly approved.
  • VisionModelProfile assets include license and source metadata for model templates.