diff --git a/README.md b/README.md index 52d7a45..539b407 100644 --- a/README.md +++ b/README.md @@ -471,7 +471,7 @@ When EdgeEnv evidence is attached, Lab keeps the ownership boundary explicit: If the EdgeEnv report includes `runtime_telemetry_context`, Lab also shows supplemental telemetry context, including baseline/candidate coverage, telemetry-history entries, missing fields, and producer-side replay summaries such as `runtime_telemetry_context.history.telemetry_coverage`. When `--with-guard` is used with EdgeEnv evidence, Lab preserves deterministic AIGuard evidence in the same Lab-owned report. This can include runtime latency regression, telemetry-context coverage, and telemetry replay-history context. -If AIGuard preserves EdgeEnv/Orchestrator `candidate_context.producer` lineage, Lab shows the device-local producer source, task stage, event count, and supplemental role as traceability evidence. +If AIGuard preserves EdgeEnv/Orchestrator `candidate_context.producer` lineage, Lab shows the device-local producer source, task stage, event count, and supplemental role as traceability evidence. The Runtime Intelligence gates also require `edgeenv_orchestrator_producer_lineage` so this handoff cannot disappear silently from the Lab-owned report. Markdown and HTML reports include a Runtime Intelligence Risk Summary that connects: diff --git a/docs/ci/runtime_intelligence_gitlab_artifacts.md b/docs/ci/runtime_intelligence_gitlab_artifacts.md index 3532ea6..6343c55 100644 --- a/docs/ci/runtime_intelligence_gitlab_artifacts.md +++ b/docs/ci/runtime_intelligence_gitlab_artifacts.md @@ -139,7 +139,7 @@ The same gate now also checks `source_repositories`, `artifact_roles`, `producer_contracts`, and device-local producer lineage so the smoke remains a cross-repo handoff fixture rather than a Lab-only report sample. -The artifact gate is implemented by `scripts/check_runtime_intelligence_artifact_bundle.py`. It checks the generated Markdown / HTML report for the required Runtime Intelligence rows, including Lab ownership, EdgeEnv comparability, telemetry coverage-gap markers, Orchestrator operation feed context, AIGuard runtime operation anomalies, and triggered deployment review rules. +The artifact gate is implemented by `scripts/check_runtime_intelligence_artifact_bundle.py`. It checks the generated Markdown / HTML report for the required Runtime Intelligence rows, including Lab ownership, EdgeEnv comparability, telemetry coverage-gap markers, Orchestrator operation feed context, AIGuard runtime operation anomalies, `edgeenv_orchestrator_producer_lineage`, and triggered deployment review rules. The CI artifact gate is implemented by `scripts/check_runtime_intelligence_ci_artifacts.py`. It runs in the deployment-risk stage and verifies that the collected optional GitLab artifacts include the manifest gate summary, report gate summary, Runtime Intelligence Risk Summary report, portfolio demo status, and the validated contract markers from the bundle manifest gate. This keeps the final CI gate file-based and deterministic without turning GitLab into a runtime control plane. diff --git a/docs/portfolio/edgeenv_runtime_regression_lab_handoff.md b/docs/portfolio/edgeenv_runtime_regression_lab_handoff.md index 3599b1a..908563a 100644 --- a/docs/portfolio/edgeenv_runtime_regression_lab_handoff.md +++ b/docs/portfolio/edgeenv_runtime_regression_lab_handoff.md @@ -133,12 +133,13 @@ Expected Lab behavior: - The same handoff gate verifies that the referenced `runtime_telemetry_history` artifact exists and preserves EdgeEnv history schema, telemetry coverage, and Runtime history seed ownership markers. - The same handoff gate verifies that missing telemetry entries remain evidence gaps while preserving Orchestrator producer markers, owner boundary flags, and EdgeEnv mapping hints when Orchestrator context is attached. - The bundle gate also requires AIGuard coverage evidence raw context to preserve the same Orchestrator mapping hint and producer markers, proving that AIGuard kept EdgeEnv/Orchestrator ownership markers as diagnosis context rather than recomputing coverage or owning deployment policy. +- The same gate requires AIGuard `edgeenv_orchestrator_producer_lineage` evidence to preserve candidate and missing-telemetry device-local producer lineage as traceability evidence. - The same gate requires AIGuard replay raw context to preserve Orchestrator producer markers and mapping hints for EdgeEnv history `missing_telemetry` entries when present, keeping missing telemetry as replay evidence gap context. - Additional Lab test fixtures under `tests/fixtures/edgeenv_regression/` mirror EdgeEnv replay examples for candidate telemetry gaps and execution sequence inversion. These fixture smokes verify that replay warnings become Lab-owned report context without making Lab recompute EdgeEnv comparability. - Markdown/HTML reports include a `Runtime Intelligence Risk Summary` that summarizes EdgeEnv comparability/regression, telemetry replay gaps, Runtime history seed traceability, AIGuard deterministic evidence, and the Lab-owned deployment decision in one reviewer-facing table. - When EdgeEnv includes preserved Orchestrator feed context, the `Runtime Intelligence Risk Summary` surfaces queue, thermal, throttling, memory, and fallback context as supplemental runtime evidence. - When `--guard-analysis` is provided, Lab ingests the precomputed AIGuard artifact as evidence without requiring AIGuard to be installed in the Lab environment. -- The committed Runtime Intelligence guard fixture preserves AIGuard's coverage-gap diagnosis, including `runtime_telemetry_field_gap` and EdgeEnv history missing-field runs, as deterministic review context rather than a Lab policy override. +- The committed Runtime Intelligence guard fixture preserves AIGuard's coverage-gap diagnosis, `edgeenv_orchestrator_producer_lineage`, and EdgeEnv history missing-field runs as deterministic review context rather than a Lab policy override. - Guard evidence details preserve explanatory fields such as `why_it_matters`, evidence-local `suspected_causes`, and `recommendation`. - Deployment decision is `review_required`. - Triggered rules include `edgeenv_runtime_regression_review`. diff --git a/examples/runtime_intelligence_chain/aiguard_runtime_operation_guard_analysis.json b/examples/runtime_intelligence_chain/aiguard_runtime_operation_guard_analysis.json index d98fc96..e208850 100644 --- a/examples/runtime_intelligence_chain/aiguard_runtime_operation_guard_analysis.json +++ b/examples/runtime_intelligence_chain/aiguard_runtime_operation_guard_analysis.json @@ -283,6 +283,294 @@ } } }, + { + "type": "edgeenv_orchestrator_producer_lineage", + "metric_name": "device_local_producer_context_count", + "observed_value": 2, + "baseline_value": 2, + "threshold": 2, + "delta": 0, + "delta_pct": null, + "increase_factor": null, + "severity": "low", + "status": "passed", + "explanation": "Device-local Orchestrator producer lineage is preserved for 2 of 2 expected operation contexts.", + "why_it_matters": "Device-local producer lineage explains which Orchestrator source created supplemental operation context. Preserving it lets Lab review runtime evidence traceability without making AIGuard or Orchestrator the deployment decision owner.", + "suspected_causes": [], + "recommendation": "Device-local Orchestrator producer lineage is preserved in the EdgeEnv runtime telemetry context.", + "raw_context": { + "edgeenv_regression": { + "baseline_run_id": "edgeenv-smoke-baseline", + "candidate_run_id": "edgeenv-smoke-candidate", + "comparable": true, + "mode": "same-condition", + "regression_detected": false, + "regression_type": "operation-context", + "severity": "medium", + "recommendation": "review_required", + "mean_delta_pct": null, + "p95_delta_pct": null, + "p99_delta_pct": null, + "fps_delta_pct": null, + "memory_peak_delta_pct": null, + "triggered_thresholds": [], + "runtime_telemetry_context_present": true, + "runtime_telemetry_source": "result_artifacts+runtime_telemetry_history", + "runtime_telemetry_history_schema_version": "edgeenv.runtime-telemetry-history.v1", + "history_orchestrator_feed_runs": 2.0, + "history_registered_runs": 3.0, + "history_telemetry_runs": 2.0, + "history_telemetry_seed_runs": 2.0, + "baseline_runtime_telemetry_history_seed_schema_version": "inferedge-runtime-telemetry-history-seed-v1", + "candidate_runtime_telemetry_history_seed_schema_version": "inferedge-runtime-telemetry-history-seed-v1", + "baseline_runtime_telemetry_history_seed_registry_owner": "edgeenv", + "candidate_runtime_telemetry_history_seed_registry_owner": "edgeenv", + "baseline_runtime_telemetry_history_seed_decision_owner": "lab", + "candidate_runtime_telemetry_history_seed_decision_owner": "lab", + "candidate_runtime_telemetry_history_seed_production_monitoring": false, + "candidate_runtime_telemetry_history_seed_missing_telemetry_is_failure": false, + "candidate_runtime_telemetry_history_seed_point_count": 1.0, + "history_missing_telemetry_runs": 1.0, + "telemetry_coverage_source": "history_telemetry_coverage", + "history_telemetry_coverage_missing_field_run_count": 1.0, + "history_telemetry_coverage_missing_field_runs": [ + { + "run_id": "edgeenv-smoke-candidate", + "missing_fields": [ + "queue_depth" + ], + "missing_field_count": 1, + "missing_telemetry_is_failure": false + } + ], + "history_telemetry_coverage_run_summaries_present": true, + "baseline_telemetry_present": true, + "candidate_telemetry_present": true, + "baseline_history_entry_present": true, + "candidate_history_entry_present": true, + "baseline_telemetry_coverage_ratio": 1.0, + "candidate_telemetry_coverage_ratio": 0.666667, + "baseline_telemetry_coverage_missing_fields": [], + "candidate_telemetry_coverage_missing_fields": [ + "queue_depth" + ], + "telemetry_coverage_missing_field_count": 1.0, + "baseline_missing_telemetry_is_failure": false, + "candidate_missing_telemetry_is_failure": false, + "baseline_execution_sequence_id": 1.0, + "candidate_execution_sequence_id": 2.0, + "execution_sequence_order_valid": true, + "baseline_orchestrator_context_present": false, + "candidate_orchestrator_context_present": true, + "orchestrator_source_repository": "InferEdgeOrchestrator", + "orchestrator_artifact_role": "orchestrator-supplemental-operation-context", + "orchestrator_producer_contract": "inferedge-orchestrator-edgeenv-runtime-telemetry-feed-v1", + "orchestrator_candidate_context_telemetry_source": "inferedge_orchestrator_operation_summary", + "orchestrator_edgeenv_mapping_hint": { + "copy_candidate_context_to": "runtime_telemetry_context.candidate", + "operation_context_role": "supplemental", + "coverage_summary_owner": "edgeenv", + "coverage_summary_path": "runtime_telemetry_context.history.telemetry_coverage", + "candidate_context_required_fields": [ + "run_id", + "telemetry_source", + "operation", + "resource" + ], + "aiguard_evidence_candidates": [ + "runtime_queue_overload", + "runtime_thermal_instability" + ] + }, + "orchestrator_mapping_hint_copy_candidate_context_to": "runtime_telemetry_context.candidate", + "orchestrator_mapping_hint_operation_context_role": "supplemental", + "orchestrator_mapping_hint_coverage_summary_owner": "edgeenv", + "orchestrator_mapping_hint_coverage_summary_path": "runtime_telemetry_context.history.telemetry_coverage", + "orchestrator_mapping_hint_candidate_context_required_fields": [ + "run_id", + "telemetry_source", + "operation", + "resource" + ], + "orchestrator_mapping_hint_aiguard_evidence_candidates": [ + "runtime_queue_overload", + "runtime_thermal_instability" + ], + "baseline_max_temperature_c": null, + "candidate_max_temperature_c": 78.5, + "baseline_throttling_detected": null, + "candidate_throttling_detected": true, + "baseline_queue_depth": null, + "candidate_queue_depth": 7.0, + "evidence_gap_count": 1.0, + "evidence_gaps": [], + "history_missing_orchestrator_context_count": 1.0, + "history_missing_orchestrator_context_run_ids": [ + "edgeenv-smoke-missing" + ], + "history_missing_orchestrator_contexts": [ + { + "schema_version": "inferedge-orchestrator-edgeenv-runtime-telemetry-feed-v1", + "role": "orchestrator_operation_context_for_edgeenv", + "source_repository": "InferEdgeOrchestrator", + "artifact_role": "orchestrator-supplemental-operation-context", + "producer_contract": "inferedge-orchestrator-edgeenv-runtime-telemetry-feed-v1", + "source": "orchestration_summary", + "run_id": "edgeenv-smoke-missing", + "not_a_regression_judgement": true, + "not_a_comparability_gate": true, + "decision_owner": "lab", + "regression_owner": "edgeenv", + "candidate_context": { + "run_id": "edgeenv-smoke-missing", + "telemetry_source": "inferedge_orchestrator_operation_summary", + "queue_depth": 4, + "operation": { + "queue_depth": 4, + "deadline_missed_count": 1, + "fallback_count": 0 + }, + "resource": { + "source": "runtime_health_snapshot", + "gpu_temperature": 72.0, + "throttling_detected": false + } + }, + "edgeenv_mapping_hint": { + "copy_candidate_context_to": "runtime_telemetry_context.candidate", + "operation_context_role": "supplemental", + "coverage_summary_owner": "edgeenv", + "coverage_summary_path": "runtime_telemetry_context.history.telemetry_coverage", + "candidate_context_required_fields": [ + "run_id", + "telemetry_source", + "operation", + "resource" + ], + "aiguard_evidence_candidates": [ + "runtime_queue_overload", + "runtime_thermal_instability" + ] + } + } + ], + "history_missing_orchestrator_source_repository": "InferEdgeOrchestrator", + "history_missing_orchestrator_artifact_role": "orchestrator-supplemental-operation-context", + "history_missing_orchestrator_producer_contract": "inferedge-orchestrator-edgeenv-runtime-telemetry-feed-v1", + "history_missing_orchestrator_candidate_context_telemetry_source": "inferedge_orchestrator_operation_summary", + "history_missing_orchestrator_edgeenv_mapping_hint": { + "copy_candidate_context_to": "runtime_telemetry_context.candidate", + "operation_context_role": "supplemental", + "coverage_summary_owner": "edgeenv", + "coverage_summary_path": "runtime_telemetry_context.history.telemetry_coverage", + "candidate_context_required_fields": [ + "run_id", + "telemetry_source", + "operation", + "resource" + ], + "aiguard_evidence_candidates": [ + "runtime_queue_overload", + "runtime_thermal_instability" + ] + }, + "history_missing_orchestrator_mapping_hint_aiguard_evidence_candidates": [ + "runtime_queue_overload", + "runtime_thermal_instability" + ], + "orchestrator_candidate_context_producer": { + "operation_context_role": "supplemental", + "producer_sources": [ + "device_local_cli_override", + "orchestration_summary" + ], + "device_local_producer_sources": [ + "device_local_cli_override" + ], + "producer_sources_by_task": { + "vision_agent": [ + "device_local_cli_override" + ] + }, + "producer_stage_by_task": { + "vision_agent": "device_local_starter" + }, + "producer_event_count": 4, + "device_local_event_count": 2, + "device_local_task_count": 1 + }, + "orchestrator_candidate_producer_sources": [ + "device_local_cli_override", + "orchestration_summary" + ], + "orchestrator_candidate_device_local_producer_sources": [ + "device_local_cli_override" + ], + "orchestrator_candidate_producer_sources_by_task": { + "vision_agent": [ + "device_local_cli_override" + ] + }, + "orchestrator_candidate_producer_stage_by_task": { + "vision_agent": "device_local_starter" + }, + "orchestrator_candidate_producer_event_count": 4.0, + "orchestrator_candidate_device_local_event_count": 2.0, + "orchestrator_candidate_device_local_task_count": 1.0, + "orchestrator_candidate_operation_context_role": "supplemental", + "history_missing_orchestrator_candidate_context_producer": { + "operation_context_role": "supplemental", + "producer_sources": [ + "device_local_cli_override", + "orchestration_summary" + ], + "device_local_producer_sources": [ + "device_local_cli_override" + ], + "producer_sources_by_task": { + "vision_agent": [ + "device_local_cli_override" + ] + }, + "producer_stage_by_task": { + "vision_agent": "device_local_starter" + }, + "producer_event_count": 4, + "device_local_event_count": 2, + "device_local_task_count": 1 + }, + "history_missing_orchestrator_candidate_producer_sources": [ + "device_local_cli_override", + "orchestration_summary" + ], + "history_missing_orchestrator_candidate_device_local_producer_sources": [ + "device_local_cli_override" + ], + "history_missing_orchestrator_candidate_producer_event_count": 4.0, + "history_missing_orchestrator_candidate_device_local_event_count": 2.0, + "history_missing_orchestrator_candidate_device_local_task_count": 1.0, + "history_missing_orchestrator_candidate_operation_context_role": "supplemental" + }, + "producer_lineage": { + "candidate_expected": true, + "candidate_device_local_sources": [ + "device_local_cli_override" + ], + "candidate_stage_by_task": { + "vision_agent": "device_local_starter" + }, + "missing_expected": true, + "missing_device_local_sources": [ + "device_local_cli_override" + ], + "missing_context_run_ids": [ + "edgeenv-smoke-missing" + ], + "operation_context_role": "supplemental", + "missing_operation_context_role": "supplemental" + } + } + }, { "type": "runtime_thermal_instability", "metric_name": "candidate_max_temperature_c", @@ -844,6 +1132,7 @@ ], "recommendations": [ "Inspect telemetry coverage missing fields, rerun telemetry history export if needed, and preserve runtime_telemetry artifacts for both baseline and candidate before relying on trend diagnosis.", + "Device-local Orchestrator producer lineage is preserved in the EdgeEnv runtime telemetry context.", "Review EdgeEnv telemetry history, power mode, cooling, and sustained run conditions before treating runtime regression as stable.", "Inspect Orchestrator queue policy, target FPS, drop/fallback behavior, and runtime telemetry before deployment." ], diff --git a/scripts/check_runtime_intelligence_artifact_bundle.py b/scripts/check_runtime_intelligence_artifact_bundle.py index e4bf0f6..fb38ca4 100644 --- a/scripts/check_runtime_intelligence_artifact_bundle.py +++ b/scripts/check_runtime_intelligence_artifact_bundle.py @@ -35,6 +35,10 @@ "aiguard_producer_lineage_handoff": ( "| AIGuard producer lineage handoff | sources=device_local_cli_override" ), + "aiguard_producer_lineage_evidence": "edgeenv_orchestrator_producer_lineage", + "aiguard_producer_lineage_recommendation": ( + "Device-local Orchestrator producer lineage is preserved" + ), "aiguard_history_seed_handoff": "| AIGuard history seed handoff | seeds=2.0", "guard_warning_rule": "guard_warning_review", "edgeenv_regression_rule": "edgeenv_runtime_regression_review", @@ -51,6 +55,10 @@ "aiguard_operation_anomalies": "runtime_queue_overload, runtime_thermal_instability", "aiguard_orchestrator_handoff": "AIGuard Orchestrator context handoff", "aiguard_producer_lineage_handoff": "AIGuard producer lineage handoff", + "aiguard_producer_lineage_evidence": "edgeenv_orchestrator_producer_lineage", + "aiguard_producer_lineage_recommendation": ( + "Device-local Orchestrator producer lineage is preserved" + ), "aiguard_device_local_producer_source": "device_local_cli_override", "runtime_history_seed": "Runtime telemetry history seed", "aiguard_history_seed_handoff": "AIGuard history seed handoff", diff --git a/scripts/check_runtime_intelligence_bundle_manifest.py b/scripts/check_runtime_intelligence_bundle_manifest.py index 0a00797..60bc734 100644 --- a/scripts/check_runtime_intelligence_bundle_manifest.py +++ b/scripts/check_runtime_intelligence_bundle_manifest.py @@ -85,6 +85,7 @@ } REQUIRED_GUARD_TYPES = { "runtime_telemetry_context_coverage", + "edgeenv_orchestrator_producer_lineage", "runtime_queue_overload", "runtime_thermal_instability", } @@ -119,6 +120,7 @@ "orchestrator_mapping_hint: operation_context_role=supplemental", "orchestrator_mapping_hint: aiguard_evidence_candidates=runtime_queue_overload,runtime_thermal_instability", "orchestrator_device_local_producer_lineage: candidate_context.producer validated", + "aiguard_evidence: edgeenv_orchestrator_producer_lineage validated", "aiguard_raw_context: telemetry_coverage_source=history_telemetry_coverage", "aiguard_raw_context: orchestrator_mapping_hint preserved", "aiguard_raw_context: orchestrator_producer_markers preserved", @@ -1134,6 +1136,82 @@ def _validate_guard_analysis(guard_analysis: dict[str, Any], errors: list[str]) ) if item.get("type") == "runtime_telemetry_context_coverage": _validate_coverage_gap_evidence(item, index, errors) + if item.get("type") == "edgeenv_orchestrator_producer_lineage": + _validate_producer_lineage_evidence(item, index, errors) + + +def _validate_producer_lineage_evidence( + item: dict[str, Any], + index: int, + errors: list[str], +) -> None: + _record( + item.get("status") == "passed", + errors, + f"AIGuard evidence[{index}] producer lineage status must be passed", + ) + _record( + item.get("observed_value") == 2, + errors, + f"AIGuard evidence[{index}] producer lineage observed_value must be 2", + ) + _record( + item.get("baseline_value") == 2, + errors, + f"AIGuard evidence[{index}] producer lineage baseline_value must be 2", + ) + raw_context = item.get("raw_context") or {} + _record( + isinstance(raw_context.get("edgeenv_regression"), dict), + errors, + f"AIGuard evidence[{index}] raw_context.edgeenv_regression must be an object", + ) + producer_lineage = raw_context.get("producer_lineage") + _record( + isinstance(producer_lineage, dict), + errors, + f"AIGuard evidence[{index}] raw_context.producer_lineage must be an object", + ) + if not isinstance(producer_lineage, dict): + return + _record( + producer_lineage.get("candidate_device_local_sources") + == ["device_local_cli_override"], + errors, + "AIGuard producer lineage candidate_device_local_sources must be " + "['device_local_cli_override']", + ) + _record( + producer_lineage.get("missing_device_local_sources") + == ["device_local_cli_override"], + errors, + "AIGuard producer lineage missing_device_local_sources must be " + "['device_local_cli_override']", + ) + _record( + producer_lineage.get("candidate_stage_by_task") + == {"vision_agent": "device_local_starter"}, + errors, + "AIGuard producer lineage candidate_stage_by_task must preserve " + "vision_agent:device_local_starter", + ) + _record( + producer_lineage.get("missing_context_run_ids") + == ["edgeenv-smoke-missing"], + errors, + "AIGuard producer lineage missing_context_run_ids must include " + "edgeenv-smoke-missing", + ) + _record( + producer_lineage.get("operation_context_role") == "supplemental", + errors, + "AIGuard producer lineage operation_context_role must be supplemental", + ) + _record( + producer_lineage.get("missing_operation_context_role") == "supplemental", + errors, + "AIGuard producer lineage missing_operation_context_role must be supplemental", + ) def _validate_coverage_gap_evidence( diff --git a/scripts/check_runtime_intelligence_ci_artifacts.py b/scripts/check_runtime_intelligence_ci_artifacts.py index 3080427..bc62198 100644 --- a/scripts/check_runtime_intelligence_ci_artifacts.py +++ b/scripts/check_runtime_intelligence_ci_artifacts.py @@ -35,6 +35,7 @@ "orchestrator_mapping_hint: operation_context_role=supplemental", "orchestrator_mapping_hint: aiguard_evidence_candidates=runtime_queue_overload,runtime_thermal_instability", "orchestrator_device_local_producer_lineage: candidate_context.producer validated", + "aiguard_evidence: edgeenv_orchestrator_producer_lineage validated", "aiguard_raw_context: telemetry_coverage_source=history_telemetry_coverage", "aiguard_raw_context: orchestrator_mapping_hint preserved", "aiguard_raw_context: orchestrator_producer_markers preserved", diff --git a/tests/test_runtime_intelligence_bundle_manifest.py b/tests/test_runtime_intelligence_bundle_manifest.py index 0b46f0d..ac3795c 100644 --- a/tests/test_runtime_intelligence_bundle_manifest.py +++ b/tests/test_runtime_intelligence_bundle_manifest.py @@ -675,6 +675,45 @@ def test_runtime_intelligence_bundle_manifest_gate_fails_for_bad_guard_producer_ ) in summary +def test_runtime_intelligence_bundle_manifest_gate_fails_for_bad_guard_lineage_evidence( + tmp_path, +): + manifest = json.loads(MANIFEST.read_text(encoding="utf-8")) + guard_path = ( + REPO_ROOT + / "examples" + / "runtime_intelligence_chain" + / manifest["files"]["aiguard_guard_analysis"] + ) + guard_analysis = json.loads(guard_path.read_text(encoding="utf-8")) + lineage_evidence = next( + item + for item in guard_analysis["evidence"] + if item.get("type") == "edgeenv_orchestrator_producer_lineage" + ) + lineage_evidence["observed_value"] = 1 + lineage_evidence["raw_context"]["producer_lineage"][ + "missing_device_local_sources" + ] = [] + + guard_copy = tmp_path / "aiguard_guard_analysis.json" + guard_copy.write_text(json.dumps(guard_analysis), encoding="utf-8") + manifest["files"]["aiguard_guard_analysis"] = str(guard_copy) + manifest_path = tmp_path / "bundle_manifest.json" + manifest_path.write_text(json.dumps(manifest), encoding="utf-8") + summary_path = tmp_path / "bundle_manifest_gate_summary.md" + + result = manifest_gate(manifest=str(manifest_path), summary_out=str(summary_path)) + + assert result == 2 + summary = summary_path.read_text(encoding="utf-8") + assert "producer lineage observed_value must be 2" in summary + assert ( + "AIGuard producer lineage missing_device_local_sources must be " + "['device_local_cli_override']" + ) in summary + + def test_runtime_intelligence_bundle_manifest_gate_fails_for_bad_guard_missing_context( tmp_path, ): diff --git a/tests/test_runtime_intelligence_ci_template.py b/tests/test_runtime_intelligence_ci_template.py index 3b0c1b6..1046663 100644 --- a/tests/test_runtime_intelligence_ci_template.py +++ b/tests/test_runtime_intelligence_ci_template.py @@ -113,6 +113,7 @@ def test_runtime_intelligence_ci_artifact_gate_passes_for_expected_outputs(tmp_p "- orchestrator_mapping_hint: operation_context_role=supplemental", "- orchestrator_mapping_hint: aiguard_evidence_candidates=runtime_queue_overload,runtime_thermal_instability", "- orchestrator_device_local_producer_lineage: candidate_context.producer validated", + "- aiguard_evidence: edgeenv_orchestrator_producer_lineage validated", "- aiguard_raw_context: telemetry_coverage_source=history_telemetry_coverage", "- aiguard_raw_context: orchestrator_mapping_hint preserved", "- aiguard_raw_context: orchestrator_producer_markers preserved", @@ -355,6 +356,7 @@ def test_runtime_intelligence_ci_artifact_gate_fails_for_failed_deployment_risk( "- orchestrator_mapping_hint: operation_context_role=supplemental", "- orchestrator_mapping_hint: aiguard_evidence_candidates=runtime_queue_overload,runtime_thermal_instability", "- orchestrator_device_local_producer_lineage: candidate_context.producer validated", + "- aiguard_evidence: edgeenv_orchestrator_producer_lineage validated", "- aiguard_raw_context: telemetry_coverage_source=history_telemetry_coverage", "- aiguard_raw_context: orchestrator_mapping_hint preserved", "- aiguard_raw_context: orchestrator_producer_markers preserved", diff --git a/tests/test_runtime_intelligence_evidence_chain_smoke.py b/tests/test_runtime_intelligence_evidence_chain_smoke.py index 0ad8ae6..f3763b7 100644 --- a/tests/test_runtime_intelligence_evidence_chain_smoke.py +++ b/tests/test_runtime_intelligence_evidence_chain_smoke.py @@ -75,6 +75,8 @@ def test_runtime_intelligence_chain_smoke_ingests_precomputed_guard_artifact(): assert bundle["guard_analysis"]["primary_reason"] == ( "Runtime telemetry context has evidence gaps that require review." ) + evidence_types = {item["type"] for item in bundle["guard_analysis"]["evidence"]} + assert "edgeenv_orchestrator_producer_lineage" in evidence_types coverage_evidence = next( item for item in bundle["guard_analysis"]["evidence"] @@ -166,6 +168,23 @@ def test_runtime_intelligence_chain_smoke_ingests_precomputed_guard_artifact(): assert guard_edgeenv_context["history_missing_orchestrator_context_run_ids"] == [ "edgeenv-smoke-missing" ] + producer_lineage_evidence = next( + item + for item in bundle["guard_analysis"]["evidence"] + if item["type"] == "edgeenv_orchestrator_producer_lineage" + ) + assert producer_lineage_evidence["status"] == "passed" + assert producer_lineage_evidence["observed_value"] == 2 + assert producer_lineage_evidence["baseline_value"] == 2 + assert producer_lineage_evidence["raw_context"]["producer_lineage"][ + "candidate_device_local_sources" + ] == ["device_local_cli_override"] + assert producer_lineage_evidence["raw_context"]["producer_lineage"][ + "missing_device_local_sources" + ] == ["device_local_cli_override"] + assert producer_lineage_evidence["raw_context"]["producer_lineage"][ + "missing_context_run_ids" + ] == ["edgeenv-smoke-missing"] assert ( guard_edgeenv_context["history_missing_orchestrator_source_repository"] == "InferEdgeOrchestrator" @@ -242,10 +261,14 @@ def test_compare_cmd_runtime_intelligence_chain_writes_markdown_and_html( assert "AIGuard runtime operation anomalies" in markdown assert "Orchestrator context attached runs" in markdown assert "AIGuard producer lineage handoff" in markdown + assert "edgeenv_orchestrator_producer_lineage" in markdown + assert "Device-local Orchestrator producer lineage is preserved" in markdown assert "device_local_cli_override" in markdown assert "AIGuard history seed handoff" in markdown assert "Runtime Intelligence Risk Summary" in html assert "AIGuard producer lineage handoff" in html + assert "edgeenv_orchestrator_producer_lineage" in html + assert "Device-local Orchestrator producer lineage is preserved" in html assert "device_local_cli_override" in html assert "Runtime telemetry history seed" in html assert "runtime_queue_overload, runtime_thermal_instability" in html