Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,18 @@ jobs:
name: backend (pytest + import smoke)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7

- name: Set up Python 3.11
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: "3.11"
cache: pip

- name: Set up Node 20 for backend source guards
uses: actions/setup-node@v4
- name: Set up Node 24 for backend source guards
uses: actions/setup-node@v7
with:
node-version: "20"
node-version: "24"
cache: npm
cache-dependency-path: frontend2/package-lock.json

Expand Down Expand Up @@ -50,12 +50,12 @@ jobs:
run:
working-directory: frontend2
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7

- name: Set up Node 20
uses: actions/setup-node@v4
- name: Set up Node 24
uses: actions/setup-node@v7
with:
node-version: "20"
node-version: "24"
cache: npm
cache-dependency-path: frontend2/package-lock.json

Expand Down
105 changes: 83 additions & 22 deletions artifacts/portfolio/tiny-stories-engineering-evidence-summary.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,21 @@
"schema_version": "tiny_stories_engineering_evidence_summary.v1",
"project": "Tiny Stories",
"framing": "Productized LLM / applied AI systems engineering, not HCI research.",
"current_live_evidence_anchor": {
"date": "2026-07-16",
"branch": "codex/tiny-stories-target-mode",
"template_id": "tmpl_159a158cc547",
"session_id": "sess_aafa73537725",
"completed_turns": 12,
"failure_count": 0,
"step_judge_passes": 12,
"contract_judge_passes": 12,
"quality_schema_version": "tiny_stories_golden_path_quality.v2",
"quality_status": "pass",
"quality_note": "Mode-aware character evidence passed with 12 responsive turns, 12 shifted turns, and three distinct NPC ids. This deterministic gate is not population-level fun or retention evidence.",
"visibility": "local_live_validation",
"public_link": false
},
"historical_live_evidence_anchor": {
"commit": "4382874 fix: keep opening live for eval gate",
"snapshot": "snapshot/story-brief-opening-live-reliability-2026-06-08",
Expand All @@ -21,56 +36,102 @@
},
"live_gate": {
"status": "pass",
"template_id": "tmpl_e6a132445fe8",
"session_id": "sess_61d85a921fda",
"template_id": "tmpl_159a158cc547",
"session_id": "sess_aafa73537725",
"completed_turns": 12,
"failure_count": 0,
"required_operations": [
{
"stage": "Story Butler",
"operation": "create.story_butler_turn",
"call_count": 1,
"source": "live",
"status": "success",
"latency_ms": 1568,
"input_tokens": 2697,
"latency_ms": 2464,
"latency_ms_min": 2464,
"latency_ms_max": 2464,
"input_tokens": 2392,
"cached_input_tokens": 384,
"output_tokens": 32,
"total_tokens": 2729,
"output_tokens": 39,
"total_tokens": 2431,
"retry_count_max": 0,
"fallback": null
},
{
"stage": "Story Brief",
"operation": "narrative.story_brief",
"call_count": 1,
"source": "live",
"status": "success",
"latency_ms": 2479,
"input_tokens": 1408,
"cached_input_tokens": 1280,
"output_tokens": 156,
"total_tokens": 1564,
"latency_ms": 2063,
"latency_ms_min": 2063,
"latency_ms_max": 2063,
"input_tokens": 1477,
"cached_input_tokens": 1408,
"output_tokens": 148,
"total_tokens": 1625,
"retry_count_max": 0,
"fallback": null
},
{
"stage": "Opening",
"operation": "narrative.opening",
"call_count": 1,
"source": "live",
"status": "success",
"latency_ms": 13100,
"input_tokens": 2630,
"cached_input_tokens": 0,
"output_tokens": 832,
"total_tokens": 3462,
"latency_ms": 10942,
"latency_ms_min": 10942,
"latency_ms_max": 10942,
"input_tokens": 2678,
"cached_input_tokens": 1408,
"output_tokens": 693,
"total_tokens": 3371,
"retry_count_max": 0,
"fallback": null
},
{
"stage": "Play turn",
"stage": "Play turns",
"operation": "narrative.advance_turn",
"call_count": 12,
"source": "live",
"status": "success",
"latency_ms": 14032,
"latency_ms_min": 8845,
"latency_ms_max": 16247,
"input_tokens": 98750,
"cached_input_tokens": 71680,
"output_tokens": 9264,
"total_tokens": 108014,
"retry_count_max": 0,
"fallback": null
}
],
"completion_operations": [
{
"operation": "narrative.ending",
"source": "live",
"status": "success",
"latency_ms": 17471,
"total_tokens": 5639,
"retry_count": 0,
"fallback": null
},
{
"operation": "narrative.highlights",
"source": "live",
"status": "success",
"latency_ms": 14190,
"total_tokens": 6957,
"retry_count": 0,
"fallback": null
},
{
"operation": "narrative.branches",
"source": "live",
"status": "success",
"latency_ms": 7701,
"input_tokens": 6464,
"cached_input_tokens": 4864,
"output_tokens": 420,
"total_tokens": 6884,
"latency_ms": 17252,
"total_tokens": 7777,
"retry_count": 0,
"fallback": null
}
]
Expand Down
65 changes: 50 additions & 15 deletions docs/tiny-stories-engineering-evidence-packet.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,29 @@ observability, and repeatable validation gates.

Framing: Productized LLM / applied AI systems engineering, not HCI research.

Current live evidence anchor:

On 2026-07-16, the strict live golden-path harness completed a fresh isolated
Create -> Story Brief -> Opening -> 12-turn Play -> Ending run against the
configured gateway. Required calls were `live/success`, no required call used
fallback, and the persisted retry count was zero for every successful call.

- Branch under review: `codex/tiny-stories-target-mode`
- Template/session: `tmpl_159a158cc547` / `sess_aafa73537725`
- Result: 12/12 turns and a generated ending; failure count 0
- Reviewer evidence: 12 Step Judge passes and 12 Contract Judge passes
- Deterministic quality packaging v2: `pass`. Consequence clarity, choice
diversity, escalation, mode-aware character response, brief payoff, and
playable options passed. In `story` mode, character quality is measured from
persisted NPC responses and state shifts rather than the active-agenda
contract reserved for `gauntlet`: 12/12 responsive turns, 12/12 shifted
turns, and three distinct NPC ids in this run.
- Player-surface check: the generated ending rendered with no horizontal
overflow and zero browser console warnings/errors. A bounded copy scan found
no technical provider/debug/schema/token language; substring matches inside
ordinary words such as `escaped` and `traceable` were discarded as false
positives.

Historical live evidence anchor:

This is a dated 2026-06-08 live-gate snapshot kept for admissions and
Expand Down Expand Up @@ -101,36 +124,48 @@ The current acceptance gate is live-backed. It is not satisfied by `/health`
alone and does not accept deterministic fallback for required product text
paths.

Live run exercised:
The 2026-07-16 live run exercised:

1. `/health` configured status for `text_llm`, `create_story_butler`,
`story_brief`, `opening`, and `play_turns`;
2. Story Butler guide turn;
3. Story Brief shaping;
4. template/opening generation;
5. Play story fetch with reviewer trace enabled;
6. one Play turn;
7. reviewer-only telemetry query.
6. twelve Play turns through the configured live text path;
7. live ending, highlight, and branch generation;
8. reviewer-only telemetry and judge evidence.

Run result:

- Status: pass
- Template/session: `tmpl_e6a132445fe8` / `sess_61d85a921fda`
- Template/session: `tmpl_159a158cc547` / `sess_aafa73537725`
- Failure count: 0
- Story Butler latency: 6216ms
- Story Brief latency: 2487ms
- template/opening latency: 13108ms
- Play turn latency: 9649ms
- telemetry endpoint latency: 3ms
- Completed turns: 12/12
- Step Judge / Contract Judge: 12 pass / 12 pass
- Story Butler latency: 2464ms
- Story Brief latency: 2063ms
- opening latency: 10942ms
- Play turn latency: 8845-16247ms; median 14032ms
- Required-operation fallback count: 0
- Maximum persisted retry count: 0

Required live telemetry table:

| Stage | Operation | Source | Status | Latency | Input | Cached input | Output | Total | Fallback |
| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- |
| Story Butler | `create.story_butler_turn` | `live` | `success` | 1568ms | 2697 | 384 | 32 | 2729 | none |
| Story Brief | `narrative.story_brief` | `live` | `success` | 2479ms | 1408 | 1280 | 156 | 1564 | none |
| Opening | `narrative.opening` | `live` | `success` | 13100ms | 2630 | 0 | 832 | 3462 | none |
| Play turn | `narrative.advance_turn` | `live` | `success` | 7701ms | 6464 | 4864 | 420 | 6884 | none |
| Stage | Operation | Calls | Source/status | Latency (min/median/max) | Input | Cached input | Output | Total | Retry max | Fallback |
| --- | --- | ---: | --- | --- | ---: | ---: | ---: | ---: | ---: | --- |
| Story Butler | `create.story_butler_turn` | 1 | `live/success` | 2464/2464/2464ms | 2392 | 384 | 39 | 2431 | 0 | none |
| Story Brief | `narrative.story_brief` | 1 | `live/success` | 2063/2063/2063ms | 1477 | 1408 | 148 | 1625 | 0 | none |
| Opening | `narrative.opening` | 1 | `live/success` | 10942/10942/10942ms | 2678 | 1408 | 693 | 3371 | 0 | none |
| Play turns | `narrative.advance_turn` | 12 | `live/success` | 8845/14032/16247ms | 98750 | 71680 | 9264 | 108014 | 0 | none |
| Ending | `narrative.ending` | 1 | `live/success` | 17471/17471/17471ms | 5311 | 256 | 328 | 5639 | 0 | none |
| Highlights | `narrative.highlights` | 1 | `live/success` | 14190/14190/14190ms | 5783 | 640 | 1174 | 6957 | 0 | none |
| Branches | `narrative.branches` | 1 | `live/success` | 17252/17252/17252ms | 7393 | 640 | 384 | 7777 | 0 | none |

These are system-validation observations from one bounded canonical run, not
population-level latency, quality, retention, or market evidence. In particular,
the 14.0s median Play-turn latency is a real provider-bound product risk even though the run
completed without fallback.

The gate rejects these rows for the required operations:

Expand Down
12 changes: 12 additions & 0 deletions frontend2/src/api/contracts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -802,6 +802,7 @@ export type NarrativeStoryBrief = {
display_title?: string | null
display_intro?: string | null
premise_summary: string
player_role?: string | null
genre_tone: string
tension_profile: NarrativeTensionProfile
story_kernel: string
Expand Down Expand Up @@ -839,10 +840,21 @@ export type NarrativeStoryBriefConsistencyCheck = {
should_retry: boolean
}

export type NarrativeStoryBriefGuideContext = {
scene_summary: string
player_role: string
cast_or_factions: string[]
pressure: string
constraints: string[]
tone: string
confirmed_facts: string[]
}

export type NarrativeStoryBriefAdvisorRequest = {
seed: string
language?: NarrativeTemplateLanguage
desired_tension_profile?: NarrativeTensionProfile | null
guide_context?: NarrativeStoryBriefGuideContext | null
}

export type NarrativeStoryBriefAdvisorResponse = {
Expand Down
9 changes: 5 additions & 4 deletions frontend2/src/pages/create/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,10 @@
1. Home `Write a new story` opens `#/create` without a login/name gate.
2. The page is a Story Butler chat, not a SaaS form.
3. The assistant asks for missing scene-plan ingredients and redirects unsafe/out-of-spec prompts while keeping pre-plan chat natural.
4. When enough fields are present, the scene plan is shaped automatically and appears inside the transcript.
5. `Generate and enter story` lives with the scene plan.
6. Generate creates a template/session and routes to Play.
4. When enough fields are present, the compact Story Butler context is sent with the seed so the scene plan preserves confirmed roles, cast, pressure, constraints, and clues instead of rediscovering them from raw transcript text.
5. The scene plan opens with only the playable essentials. Inferred settings, full cast planning, revisions, and handoff mechanics remain available in a closed details section.
6. `Generate and enter story` lives with the scene plan.
7. Generate creates a template/session and routes to Play.

## Story-Shape Settings

Expand All @@ -40,7 +41,7 @@ Examples handled by `shared/lib/story-guide-settings.ts`, with loop orchestratio
- `backstage`, `disappearance`, `public scandal` -> high drama.
- `cozy`, `clues`, `small town` -> cozy mystery.

The inferred read is kept internal during ordinary collection so pre-plan chat stays one-question-at-a-time. It is surfaced inside the final scene plan before Generate. If a user correction changes story-shape settings after a scene plan is ready, the flow must require reshaping before Generate.
The inferred read is kept internal during ordinary collection so pre-plan chat stays one-question-at-a-time. It is available inside the scene plan's closed planning details before Generate. If a user correction changes story-shape settings after a scene plan is ready, the flow must require reshaping before Generate.

## Visibility

Expand Down
Loading