Skip to content

Gate AIGuard producer lineage handoff #282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hyeokjun32 merged 1 commit into from
May 25, 2026
Merged

Gate AIGuard producer lineage handoff #282

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -471,7 +471,7 @@ When EdgeEnv evidence is attached, Lab keeps the ownership boundary explicit:
If the EdgeEnv report includes `runtime_telemetry_context`, Lab also shows supplemental telemetry context, including baseline/candidate coverage, telemetry-history entries, missing fields, and producer-side replay summaries such as `runtime_telemetry_context.history.telemetry_coverage`.

When `--with-guard` is used with EdgeEnv evidence, Lab preserves deterministic AIGuard evidence in the same Lab-owned report. This can include runtime latency regression, telemetry-context coverage, and telemetry replay-history context.
If AIGuard preserves EdgeEnv/Orchestrator `candidate_context.producer` lineage, Lab shows the device-local producer source, task stage, event count, and supplemental role as traceability evidence.
If AIGuard preserves EdgeEnv/Orchestrator `candidate_context.producer` lineage, Lab shows the device-local producer source, task stage, event count, and supplemental role as traceability evidence. The Runtime Intelligence gates also require `edgeenv_orchestrator_producer_lineage` so this handoff cannot disappear silently from the Lab-owned report.

Markdown and HTML reports include a Runtime Intelligence Risk Summary that connects:

Expand Down
2 changes: 1 addition & 1 deletion docs/ci/runtime_intelligence_gitlab_artifacts.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ The same gate now also checks `source_repositories`, `artifact_roles`,
`producer_contracts`, and device-local producer lineage so the smoke remains a
cross-repo handoff fixture rather than a Lab-only report sample.

The artifact gate is implemented by `scripts/check_runtime_intelligence_artifact_bundle.py`. It checks the generated Markdown / HTML report for the required Runtime Intelligence rows, including Lab ownership, EdgeEnv comparability, telemetry coverage-gap markers, Orchestrator operation feed context, AIGuard runtime operation anomalies, and triggered deployment review rules.
The artifact gate is implemented by `scripts/check_runtime_intelligence_artifact_bundle.py`. It checks the generated Markdown / HTML report for the required Runtime Intelligence rows, including Lab ownership, EdgeEnv comparability, telemetry coverage-gap markers, Orchestrator operation feed context, AIGuard runtime operation anomalies, `edgeenv_orchestrator_producer_lineage`, and triggered deployment review rules.

The CI artifact gate is implemented by `scripts/check_runtime_intelligence_ci_artifacts.py`. It runs in the deployment-risk stage and verifies that the collected optional GitLab artifacts include the manifest gate summary, report gate summary, Runtime Intelligence Risk Summary report, portfolio demo status, and the validated contract markers from the bundle manifest gate. This keeps the final CI gate file-based and deterministic without turning GitLab into a runtime control plane.

Expand Down
3 changes: 2 additions & 1 deletion docs/portfolio/edgeenv_runtime_regression_lab_handoff.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,12 +133,13 @@ Expected Lab behavior:
- The same handoff gate verifies that the referenced `runtime_telemetry_history` artifact exists and preserves EdgeEnv history schema, telemetry coverage, and Runtime history seed ownership markers.
- The same handoff gate verifies that missing telemetry entries remain evidence gaps while preserving Orchestrator producer markers, owner boundary flags, and EdgeEnv mapping hints when Orchestrator context is attached.
- The bundle gate also requires AIGuard coverage evidence raw context to preserve the same Orchestrator mapping hint and producer markers, proving that AIGuard kept EdgeEnv/Orchestrator ownership markers as diagnosis context rather than recomputing coverage or owning deployment policy.
- The same gate requires AIGuard `edgeenv_orchestrator_producer_lineage` evidence to preserve candidate and missing-telemetry device-local producer lineage as traceability evidence.
- The same gate requires AIGuard replay raw context to preserve Orchestrator producer markers and mapping hints for EdgeEnv history `missing_telemetry` entries when present, keeping missing telemetry as replay evidence gap context.
- Additional Lab test fixtures under `tests/fixtures/edgeenv_regression/` mirror EdgeEnv replay examples for candidate telemetry gaps and execution sequence inversion. These fixture smokes verify that replay warnings become Lab-owned report context without making Lab recompute EdgeEnv comparability.
- Markdown/HTML reports include a `Runtime Intelligence Risk Summary` that summarizes EdgeEnv comparability/regression, telemetry replay gaps, Runtime history seed traceability, AIGuard deterministic evidence, and the Lab-owned deployment decision in one reviewer-facing table.
- When EdgeEnv includes preserved Orchestrator feed context, the `Runtime Intelligence Risk Summary` surfaces queue, thermal, throttling, memory, and fallback context as supplemental runtime evidence.
- When `--guard-analysis` is provided, Lab ingests the precomputed AIGuard artifact as evidence without requiring AIGuard to be installed in the Lab environment.
- The committed Runtime Intelligence guard fixture preserves AIGuard's coverage-gap diagnosis, including `runtime_telemetry_field_gap` and EdgeEnv history missing-field runs, as deterministic review context rather than a Lab policy override.
- The committed Runtime Intelligence guard fixture preserves AIGuard's coverage-gap diagnosis, `edgeenv_orchestrator_producer_lineage`, and EdgeEnv history missing-field runs as deterministic review context rather than a Lab policy override.
- Guard evidence details preserve explanatory fields such as `why_it_matters`, evidence-local `suspected_causes`, and `recommendation`.
- Deployment decision is `review_required`.
- Triggered rules include `edgeenv_runtime_regression_review`.
Expand Down
289 changes: 289 additions & 0 deletions examples/runtime_intelligence_chain/aiguard_runtime_operation_guard_analysis.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,294 @@
}
}
},
{
"type": "edgeenv_orchestrator_producer_lineage",
"metric_name": "device_local_producer_context_count",
"observed_value": 2,
"baseline_value": 2,
"threshold": 2,
"delta": 0,
"delta_pct": null,
"increase_factor": null,
"severity": "low",
"status": "passed",
"explanation": "Device-local Orchestrator producer lineage is preserved for 2 of 2 expected operation contexts.",
"why_it_matters": "Device-local producer lineage explains which Orchestrator source created supplemental operation context. Preserving it lets Lab review runtime evidence traceability without making AIGuard or Orchestrator the deployment decision owner.",
"suspected_causes": [],
"recommendation": "Device-local Orchestrator producer lineage is preserved in the EdgeEnv runtime telemetry context.",
"raw_context": {
"edgeenv_regression": {
"baseline_run_id": "edgeenv-smoke-baseline",
"candidate_run_id": "edgeenv-smoke-candidate",
"comparable": true,
"mode": "same-condition",
"regression_detected": false,
"regression_type": "operation-context",
"severity": "medium",
"recommendation": "review_required",
"mean_delta_pct": null,
"p95_delta_pct": null,
"p99_delta_pct": null,
"fps_delta_pct": null,
"memory_peak_delta_pct": null,
"triggered_thresholds": [],
"runtime_telemetry_context_present": true,
"runtime_telemetry_source": "result_artifacts+runtime_telemetry_history",
"runtime_telemetry_history_schema_version": "edgeenv.runtime-telemetry-history.v1",
"history_orchestrator_feed_runs": 2.0,
"history_registered_runs": 3.0,
"history_telemetry_runs": 2.0,
"history_telemetry_seed_runs": 2.0,
"baseline_runtime_telemetry_history_seed_schema_version": "inferedge-runtime-telemetry-history-seed-v1",
"candidate_runtime_telemetry_history_seed_schema_version": "inferedge-runtime-telemetry-history-seed-v1",
"baseline_runtime_telemetry_history_seed_registry_owner": "edgeenv",
"candidate_runtime_telemetry_history_seed_registry_owner": "edgeenv",
"baseline_runtime_telemetry_history_seed_decision_owner": "lab",
"candidate_runtime_telemetry_history_seed_decision_owner": "lab",
"candidate_runtime_telemetry_history_seed_production_monitoring": false,
"candidate_runtime_telemetry_history_seed_missing_telemetry_is_failure": false,
"candidate_runtime_telemetry_history_seed_point_count": 1.0,
"history_missing_telemetry_runs": 1.0,
"telemetry_coverage_source": "history_telemetry_coverage",
"history_telemetry_coverage_missing_field_run_count": 1.0,
"history_telemetry_coverage_missing_field_runs": [
{
"run_id": "edgeenv-smoke-candidate",
"missing_fields": [
"queue_depth"
],
"missing_field_count": 1,
"missing_telemetry_is_failure": false
}
],
"history_telemetry_coverage_run_summaries_present": true,
"baseline_telemetry_present": true,
"candidate_telemetry_present": true,
"baseline_history_entry_present": true,
"candidate_history_entry_present": true,
"baseline_telemetry_coverage_ratio": 1.0,
"candidate_telemetry_coverage_ratio": 0.666667,
"baseline_telemetry_coverage_missing_fields": [],
"candidate_telemetry_coverage_missing_fields": [
"queue_depth"
],
"telemetry_coverage_missing_field_count": 1.0,
"baseline_missing_telemetry_is_failure": false,
"candidate_missing_telemetry_is_failure": false,
"baseline_execution_sequence_id": 1.0,
"candidate_execution_sequence_id": 2.0,
"execution_sequence_order_valid": true,
"baseline_orchestrator_context_present": false,
"candidate_orchestrator_context_present": true,
"orchestrator_source_repository": "InferEdgeOrchestrator",
"orchestrator_artifact_role": "orchestrator-supplemental-operation-context",
"orchestrator_producer_contract": "inferedge-orchestrator-edgeenv-runtime-telemetry-feed-v1",
"orchestrator_candidate_context_telemetry_source": "inferedge_orchestrator_operation_summary",
"orchestrator_edgeenv_mapping_hint": {
"copy_candidate_context_to": "runtime_telemetry_context.candidate",
"operation_context_role": "supplemental",
"coverage_summary_owner": "edgeenv",
"coverage_summary_path": "runtime_telemetry_context.history.telemetry_coverage",
"candidate_context_required_fields": [
"run_id",
"telemetry_source",
"operation",
"resource"
],
"aiguard_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
]
},
"orchestrator_mapping_hint_copy_candidate_context_to": "runtime_telemetry_context.candidate",
"orchestrator_mapping_hint_operation_context_role": "supplemental",
"orchestrator_mapping_hint_coverage_summary_owner": "edgeenv",
"orchestrator_mapping_hint_coverage_summary_path": "runtime_telemetry_context.history.telemetry_coverage",
"orchestrator_mapping_hint_candidate_context_required_fields": [
"run_id",
"telemetry_source",
"operation",
"resource"
],
"orchestrator_mapping_hint_aiguard_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
],
"baseline_max_temperature_c": null,
"candidate_max_temperature_c": 78.5,
"baseline_throttling_detected": null,
"candidate_throttling_detected": true,
"baseline_queue_depth": null,
"candidate_queue_depth": 7.0,
"evidence_gap_count": 1.0,
"evidence_gaps": [],
"history_missing_orchestrator_context_count": 1.0,
"history_missing_orchestrator_context_run_ids": [
"edgeenv-smoke-missing"
],
"history_missing_orchestrator_contexts": [
{
"schema_version": "inferedge-orchestrator-edgeenv-runtime-telemetry-feed-v1",
"role": "orchestrator_operation_context_for_edgeenv",
"source_repository": "InferEdgeOrchestrator",
"artifact_role": "orchestrator-supplemental-operation-context",
"producer_contract": "inferedge-orchestrator-edgeenv-runtime-telemetry-feed-v1",
"source": "orchestration_summary",
"run_id": "edgeenv-smoke-missing",
"not_a_regression_judgement": true,
"not_a_comparability_gate": true,
"decision_owner": "lab",
"regression_owner": "edgeenv",
"candidate_context": {
"run_id": "edgeenv-smoke-missing",
"telemetry_source": "inferedge_orchestrator_operation_summary",
"queue_depth": 4,
"operation": {
"queue_depth": 4,
"deadline_missed_count": 1,
"fallback_count": 0
},
"resource": {
"source": "runtime_health_snapshot",
"gpu_temperature": 72.0,
"throttling_detected": false
}
},
"edgeenv_mapping_hint": {
"copy_candidate_context_to": "runtime_telemetry_context.candidate",
"operation_context_role": "supplemental",
"coverage_summary_owner": "edgeenv",
"coverage_summary_path": "runtime_telemetry_context.history.telemetry_coverage",
"candidate_context_required_fields": [
"run_id",
"telemetry_source",
"operation",
"resource"
],
"aiguard_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
]
}
}
],
"history_missing_orchestrator_source_repository": "InferEdgeOrchestrator",
"history_missing_orchestrator_artifact_role": "orchestrator-supplemental-operation-context",
"history_missing_orchestrator_producer_contract": "inferedge-orchestrator-edgeenv-runtime-telemetry-feed-v1",
"history_missing_orchestrator_candidate_context_telemetry_source": "inferedge_orchestrator_operation_summary",
"history_missing_orchestrator_edgeenv_mapping_hint": {
"copy_candidate_context_to": "runtime_telemetry_context.candidate",
"operation_context_role": "supplemental",
"coverage_summary_owner": "edgeenv",
"coverage_summary_path": "runtime_telemetry_context.history.telemetry_coverage",
"candidate_context_required_fields": [
"run_id",
"telemetry_source",
"operation",
"resource"
],
"aiguard_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
]
},
"history_missing_orchestrator_mapping_hint_aiguard_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
],
"orchestrator_candidate_context_producer": {
"operation_context_role": "supplemental",
"producer_sources": [
"device_local_cli_override",
"orchestration_summary"
],
"device_local_producer_sources": [
"device_local_cli_override"
],
"producer_sources_by_task": {
"vision_agent": [
"device_local_cli_override"
]
},
"producer_stage_by_task": {
"vision_agent": "device_local_starter"
},
"producer_event_count": 4,
"device_local_event_count": 2,
"device_local_task_count": 1
},
"orchestrator_candidate_producer_sources": [
"device_local_cli_override",
"orchestration_summary"
],
"orchestrator_candidate_device_local_producer_sources": [
"device_local_cli_override"
],
"orchestrator_candidate_producer_sources_by_task": {
"vision_agent": [
"device_local_cli_override"
]
},
"orchestrator_candidate_producer_stage_by_task": {
"vision_agent": "device_local_starter"
},
"orchestrator_candidate_producer_event_count": 4.0,
"orchestrator_candidate_device_local_event_count": 2.0,
"orchestrator_candidate_device_local_task_count": 1.0,
"orchestrator_candidate_operation_context_role": "supplemental",
"history_missing_orchestrator_candidate_context_producer": {
"operation_context_role": "supplemental",
"producer_sources": [
"device_local_cli_override",
"orchestration_summary"
],
"device_local_producer_sources": [
"device_local_cli_override"
],
"producer_sources_by_task": {
"vision_agent": [
"device_local_cli_override"
]
},
"producer_stage_by_task": {
"vision_agent": "device_local_starter"
},
"producer_event_count": 4,
"device_local_event_count": 2,
"device_local_task_count": 1
},
"history_missing_orchestrator_candidate_producer_sources": [
"device_local_cli_override",
"orchestration_summary"
],
"history_missing_orchestrator_candidate_device_local_producer_sources": [
"device_local_cli_override"
],
"history_missing_orchestrator_candidate_producer_event_count": 4.0,
"history_missing_orchestrator_candidate_device_local_event_count": 2.0,
"history_missing_orchestrator_candidate_device_local_task_count": 1.0,
"history_missing_orchestrator_candidate_operation_context_role": "supplemental"
},
"producer_lineage": {
"candidate_expected": true,
"candidate_device_local_sources": [
"device_local_cli_override"
],
"candidate_stage_by_task": {
"vision_agent": "device_local_starter"
},
"missing_expected": true,
"missing_device_local_sources": [
"device_local_cli_override"
],
"missing_context_run_ids": [
"edgeenv-smoke-missing"
],
"operation_context_role": "supplemental",
"missing_operation_context_role": "supplemental"
}
}
},
{
"type": "runtime_thermal_instability",
"metric_name": "candidate_max_temperature_c",
Expand Down Expand Up @@ -844,6 +1132,7 @@
],
"recommendations": [
"Inspect telemetry coverage missing fields, rerun telemetry history export if needed, and preserve runtime_telemetry artifacts for both baseline and candidate before relying on trend diagnosis.",
"Device-local Orchestrator producer lineage is preserved in the EdgeEnv runtime telemetry context.",
"Review EdgeEnv telemetry history, power mode, cooling, and sustained run conditions before treating runtime regression as stable.",
"Inspect Orchestrator queue policy, target FPS, drop/fallback behavior, and runtime telemetry before deployment."
],
Expand Down
Loading
Loading