Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 50 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ name: ci
on: [push, pull_request]
jobs:
suite:
# The repo alone, as a buyer first clones it: no listing identity next
# to it, so the 5 identity-pinned tests in tests/test_day3.py skip.
# They are run for real, with zero skips allowed, in with-identity below.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
Expand All @@ -11,4 +14,50 @@ jobs:
# CI run installed bare pytest and died on collection because
# analysis.py imports the pillar packages at module level.
- run: pip install -e ".[pillars,crypto,dev]"
- run: python -m pytest -q
- run: python -m pytest -q -rs

with-identity:
# Added 2026-10-13 (repair plan 1.6). Until then every CI run skipped the
# 5 identity tests, so a green badge said nothing about them. Here the
# real listing-assistants identity is cloned and ANY skip fails the job.
# Python 3.10 is the floor pyproject.toml declares; Windows runs because
# the listing product ships a Windows desk on this core.
strategy:
fail-fast: false
matrix:
include:
- {os: ubuntu-latest, python: "3.10"}
- {os: ubuntu-latest, python: "3.12"}
- {os: ubuntu-latest, python: "3.13"}
- {os: windows-latest, python: "3.12"}
runs-on: ${{ matrix.os }}
timeout-minutes: 30
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: {python-version: "${{ matrix.python }}"}
- run: pip install -e ".[pillars,crypto,dev]"
- name: clone the listing identity (same branch name if it exists, else main)
run: |
BR="${GITHUB_HEAD_REF:-$GITHUB_REF_NAME}"
if git clone --depth 1 -b "$BR" https://github.com/QuietFireAI/listing-assistants.git "$RUNNER_TEMP/listing-assistants" 2>/dev/null; then
echo "listing identity: branch $BR"
else
git clone --depth 1 https://github.com/QuietFireAI/listing-assistants.git "$RUNNER_TEMP/listing-assistants"
echo "listing identity: main (no branch named $BR)"
fi
- name: full suite, no skips allowed
env:
# Keep Windows on its real default text encoding (cp1252), so a
# missing encoding="utf-8" fails here and not on a buyer's machine.
PYTHONUTF8: "0"
run: |
export IDENTITY_DIR="$RUNNER_TEMP/listing-assistants"
python -m pytest -q -rs | tee out.txt
if grep -qi "skipped" out.txt; then
echo "::error::identity tests skipped - the identity was not found or not loaded"
exit 1
fi
6 changes: 6 additions & 0 deletions EVIDENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,5 +133,11 @@ Any new claim added to any README in this stack must be added to this ledger fir
| Loop protection suspends per (context,intent) at threshold; retries never inflate the count | **MEASURED** | test_day3.py; threshold 20 RATIFIED (owner, 2026-07-17) as a deliberate placeholder, revisit when after-action data exists |
| Manners re-injection triggers instrumented (constants + backstop), fade-trackable per run | **MEASURED** | audit events + KPI + after-action section; conduct efficacy itself is UNTESTED |
| Ed25519 authority: verifier cannot forge; tamper/forge reject; boot manifest signable incl. MANNERS.md hash | **MEASURED** | test_day3.py; asymmetric tier, optional cryptography dep |
| A held outbound message can be declined by the same signed key that releases it; a decline drops it unsent and is on the chain; a release-signed key cannot decline and a decline-signed key cannot release (2026-10-12) | **MEASURED** | tests/test_absolute_signal.py decline tests; tests/test_sweep_core_hub_2026_10_13.py (unsigned and registry-refused answers are integrity violations; reason kept to 500 characters) |
| The record says a person was notified only when the notifier reports the message delivered (sent / delivered / shown); a failed or raising notifier, or one that answers with anything odd, is recorded as human.notify_failed; the escalation KPI and the after-action report pair each escalation with its own outcome - the outcome names the escalation entry's hash - so one escalation's delivery never covers another's failure, even when two processes raise on one queue, and an unreported delivery gets no transport time (2026-10-13) | **MEASURED** | tests/test_notification_truth.py (33); before this, a Twilio 401 went on the chain as human.notified |
| A log with an unreadable line does not open, and the error names the first break in the file (line, reason, cut-off last write or not) - an edited line before a cut-off end is named first, so the repair is only suggested when it will accept the file; writers take turns through an OS lock on <log>.lock (on POSIX readers share it; reading needs no write access to the folder), so two processes never append at once and one process never waits on itself; a writer that finds the file's size or time changed since its own last write re-checks the chain and refuses to append if it is damaged, cut back, edited or replaced, and continues from another writer's valid entries; a person repairs a cut-off last write with tools/audit_repair.py, which under the lock replaces the log in one step with every complete entry byte for byte plus the record of the repair, keeping the file's permissions; any other damage, or a swap that fails, changes nothing (2026-10-13) | **MEASURED** | tests/test_audit_log_damage.py (56), including three processes appending 450 entries at once, Windows line endings, a lock held by another process, a read-only folder, and a repair that dies before its swap. Not seen at append time: an edit that keeps both size and modification time, and a deletion or replacement made before this writer ever wrote or saw an entry (verify_chain or a saved anchor finds those) |
| The six pillar packages are pinned to exact commits; their own suites pass at those commits (9, 6, 11, 27, 20, 14) | **MEASURED** | pyproject.toml; each pinned pillar's suite run 2026-10-13 |
| The core suite runs the listing identity tests on every push, on Linux (Python 3.10, 3.12, 3.13) and Windows, and fails on any skip | **MEASURED** | .github/workflows/ci.yml job with-identity; first green run 38437495999 |
| Every mutant the 2026-10-13 sweep puts into the 16 core files is caught by the core suite (931: 919 killed, 1 timeout), except 2 argued equivalent with evidence, 1 that only changes CPU use while waiting for the lock, and 8 on Windows-only lines a Linux sweep cannot run | **MEASURED** | findings/MUTATION_SWEEP_2026-10-13.md; tools/mutation_sweep.py reproduces it. Mutation testing shows the tests notice these changes; it does not prove the code correct |

Not claimed for the runtime: production readiness; real-spoke latency; conduct-level efficacy of manners; licensed legal review of vertical content.
10 changes: 6 additions & 4 deletions PILLAR_TESTING_MANUAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,17 @@ EXPECTED: seven installs, no errors. (PEP 668 systems: add

Run every pillar's OWN suite from its own clone: `python3 -m pytest -q`
EXPECTED, exactly: open-mind 9 | before-turn 6 | pre-response-selfcheck 11 |
agent-open-mind 27 | sleep-marks 20 | splitvantage 12. Any other number =
agent-open-mind 27 | sleep-marks 20 | splitvantage 14 (at the commits pinned
in pyproject.toml; checked 2026-10-13). Any other number =
that pillar regressed; stop, report the raw failure. Do not proceed to
integration on a broken pillar - the set cannot be validated on a bad part.

Then the dispatcher's suite from its clone: `python3 -m pytest tests/ -q`
EXPECTED: 132 passed, 5 skipped. (The 5 identity-dependent tests activate
EXPECTED: 339 passed, 5 skipped. (The 5 identity-dependent tests activate
when IDENTITY_DIR points at a listing-assistants clone, or when one sits at
the conventional sibling path `../listing-assistants` - then 137 passed, as
of 2026-10-12.)
the conventional sibling path `../listing-assistants` - then 344 passed, as
of 2026-10-13. CI runs both: the clone alone, and with the listing identity
on Linux 3.10/3.12/3.13 and Windows, where any skip fails the job.)

## PHASE 2 - Six pillars together on the bare dispatcher

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ git clone https://github.com/QuietFireAI/dispatcher-agents
cd dispatcher-agents
pip install -e ".[pillars,crypto,dev]" # routing core is zero-dep; signing needs crypto (Ed25519)
git clone https://github.com/QuietFireAI/listing-assistants ../listing-assistants
IDENTITY_DIR=../listing-assistants python3 -m pytest tests/ # 137 tests vs the real 52-route track
IDENTITY_DIR=../listing-assistants python3 -m pytest tests/ # 344 tests vs the real 52-route track
IDENTITY_DIR=../listing-assistants python3 demo/run_p11_demo.py # real spokes chain 11 envelopes from 2 signals

# Identity-gated tests target the repo bundled at IDENTITY_DIR (default: a fixture track); point IDENTITY_DIR at listing-assistants for the real 52-route run.
Expand Down
2 changes: 1 addition & 1 deletion SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ pip install -e ".[pillars,crypto,dev]" # pillars pulled from source, never ven
### Verify before claiming anything works

```bash
python -m pytest tests/ # 89 doctrine tests (identity-gated tests activate with IDENTITY_DIR)
python -m pytest tests/ # 339 passed + 5 skipped with no listing-assistants clone beside this repo; 344 passed with one (found at ../listing-assistants, or set IDENTITY_DIR)
python demo/baseline_run.py # must print all six pillar events nonzero
python tools/kpi_gate.py baseline-out/audit-<id>.jsonl \
--taints-expected 1 --selfcheck-bait-expected 1
Expand Down
4 changes: 2 additions & 2 deletions demo/run_p11_demo.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,9 @@ def stub_model_a(prompt: str) -> dict:
def stub_model_b(prompt: str) -> dict:
return {"model": "stub-b", "response": "Fine.", "thinking": ""}

IDENTITY = os.environ.get(
IDENTITY = os.path.abspath(os.environ.get(
"IDENTITY_DIR",
os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "..", "listing-assistants"))
os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "..", "listing-assistants")))


def signal(ctx, payload):
Expand Down
35 changes: 25 additions & 10 deletions dispatcher/after_action.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@
agent memory, never self-reported. One markdown report per run.

Schema fields the runtime cannot yet populate are DECLARED, not faked:
- manners re-injections: no manners events exist in the runtime yet
- human response time: notification is instrumented, response is not
(Corrected 2026-10-13: this list also named manners re-injections, which the
report below does read from the log.)
An unreported deviation is a fabricated report (schema's own words), so
deviations are computed mechanically: any audited reject, hold, dead-letter,
taint, or siding event inside the run window is a deviation, verbatim.
Expand All @@ -14,7 +15,7 @@

import datetime

from .kpi import compute_kpis
from .kpi import compute_kpis, pair_escalations


def _iso(ts: float) -> str:
Expand All @@ -34,10 +35,15 @@ def generate_report(events: list[dict], playbook: str, run_id: str,
"""events: full audit log. steps: the playbook's step list as executed,
each {step, agent, intent, envelope_id} - proof and latency are looked up
in the log, never taken from the caller's word."""
# this run's events only: its client context, or an envelope of its
# steps. (Until 2026-10-13 every escalation.raised and notification on
# the log was added regardless of context, so a report listed other
# runs' escalations as its own.) Notifications are still paired with
# escalations over the whole log (kpi.pair_escalations), so an outcome
# recorded without this context still answers this run's escalation.
run_events = [e for e in events
if e.get("client_context_id") == client_context_id
or e.get("envelope_id") in {s.get("envelope_id") for s in steps}
or e["kind"] in ("escalation.raised", "human.notified")]
or e.get("envelope_id") in {s.get("envelope_id") for s in steps}]
if not run_events:
raise ValueError(f"no audited events for run {run_id!r} - a report "
f"without log evidence would be fabricated")
Expand Down Expand Up @@ -89,17 +95,26 @@ def generate_report(events: list[dict], playbook: str, run_id: str,
lines.append("- none on log")

lines += ["", "## escalations"]
notif = {e.get("queue"): e["ts"] for e in run_events
if e["kind"] == "human.notified"}
# each escalation's own outcome (kpi.pair_escalations), paired over the
# whole log so another escalation's notification never stands in
outcome = {id(r): o for r, o in pair_escalations(events)}
if esc:
for e in esc:
q = e.get("queue")
if q in notif and notif[q] >= e["ts"]:
lines.append(f"- {q}: transport {notif[q]-e['ts']:.4f}s; "
f"human response time: NOT INSTRUMENTED")
else:
o = outcome.get(id(e))
if o is None:
lines.append(f"- {q}: raised, NO human.notified on log - "
f"transport unproven")
elif o["kind"] == "human.notify_failed":
why = o.get("notifier_status") or o.get("error") or "unknown"
lines.append(f"- {q}: raised, NOTIFICATION FAILED ({why}) - "
f"no person was reached")
elif o.get("transport") == "not reported":
lines.append(f"- {q}: raised, the notifier reported nothing - "
f"delivery unconfirmed, no transport time")
else:
lines.append(f"- {q}: transport {o['ts'] - e['ts']:.4f}s; "
f"human response time: NOT INSTRUMENTED")
else:
lines.append("- none")

Expand Down
Loading
Loading