Codex Copilot

Live demo: https://codex-copilot.dk5506934.workers.dev/

A chat assistant that answers questions about a team's internal Engineering Codex — its written engineering standards — grounded in a small seeded corpus of standards documents. Built entirely on Cloudflare: Workers AI for the model, a Worker for coordination, a Durable Object for per-session memory, and a static chat UI served by the same Worker.

Ask it "what is our branch naming convention?", then follow up with "what about for hotfixes?" — the second question has no subject of its own, and it is answered correctly because the conversation is stored in a Durable Object and replayed into the prompt.

---

The four required components

Component	Implementation
1. LLM	Workers AI, @cf/meta/llama-3.3-70b-instruct-fp8-fast, via the AI binding (worker/src/index.ts)
2. Workflow / coordination	A Worker that routes each turn: load memory → retrieve standards → build prompt → call the model → persist state and prompt log (worker/src/index.ts)
3. User input via chat	Static HTML/CSS/JS chat UI served from the same Worker (frontend/)
4. Memory / state	ConversationSession Durable Object, one instance per session ID, holding the transcript and the prompt log (worker/src/durable-object.ts)

Architecture

   Browser (frontend/index.html)
   sessionId in localStorage
            │
            │  POST /chat  { sessionId, message }
            ▼
   ┌─────────────────────────────────────────────────┐
   │  Worker  (worker/src/index.ts)                  │
   │                                                 │
   │  1. stub = CONVERSATION.idFromName(sessionId)   │
   │  2. history  ◄──── Durable Object storage       │
   │  3. retrieve(message, recent user turns)        │
   │       └── keyword scoring over seeded .md docs  │
   │           (worker/src/retrieval.ts)             │
   │  4. messages = [system, context, history, user] │
   │  5. env.AI.run(llama-3.3, messages)  ──────────►│──► Workers AI
   │  6. append user + assistant turns ─────────────►│
   │  7. append full prompt + response to log ──────►│──► Durable Object
   │                                                 │
   └─────────────────────────────────────────────────┘
            │
            ▼
   { answer, sources[] }  →  rendered as a chat bubble with its sources

Each session ID maps to exactly one Durable Object instance via idFromName(sessionId), so every turn of a conversation is handled by the same object and its state is strongly consistent without any external database.

Retrieval

worker/src/retrieval.ts does keyword matching — no vector database:

1. The five seeded documents are imported as text at build time (via the Text rule in wrangler.toml) and split into sections on ## headings.
2. The question is lowercased, tokenized, stripped of stopwords, and crudely stemmed for plurals.
3. Each section scores 1 + log(occurrences) per distinct matching term, plus a bonus of 2 for a term that appears in the section's heading.
4. The previous two user turns are scored in as well at weight 0.3. This is what keeps a bare follow-up like "what about for hotfixes?" retrieving from branch-naming.md rather than drifting to whichever document happens to mention hotfixes most.
5. The top 2 sections are injected into the prompt as the context block.

The system prompt instructs the model to answer only from that block, to say so when the standards don't cover a question, and to cite the source file.

Prompt history

Every call to the model appends an entry to the session's prompt log inside the Durable Object, containing the exact messages array that was sent (system prompt, retrieved context, replayed history, new question), the retrieved document names and scores, the response, and the model latency. It is logged after the call using the same array that was passed to env.AI.run, so the log cannot drift from what the model actually received.

Export it:

# JSON
curl "http://127.0.0.1:8787/prompt-history?sessionId=<id>"

# Markdown, ready to attach to a submission
curl "http://127.0.0.1:8787/prompt-history?sessionId=<id>&format=md" > session-log.md

The UI's "Prompt history" link opens the markdown form for the current session. example-prompt-history.md is a real export from a three-turn session against the deployed model.

HTTP API

Method & path	Purpose
POST /chat	{ sessionId, message } → { sessionId, answer, sources[] }
GET /history?sessionId=	Stored transcript for a session
GET /prompt-history?sessionId=[&format=md]	Prompt log for a session
POST /reset?sessionId=	Clear a session's stored state
GET /docs	Names of the seeded standards
GET /health	Liveness, active model, document count
anything else	The static chat UI

Run it locally

cd worker
npm install
npx wrangler login   # only needed once
npm run dev

Then open http://127.0.0.1:8787.

Note that the AI binding always calls the real Workers AI service, including under wrangler dev — local development still needs a logged-in Cloudflare account, and model calls count against the account's usage. Durable Object storage, by contrast, is local under wrangler dev.

Verify the memory behaviour from the command line:

curl -s -X POST http://127.0.0.1:8787/chat -H 'content-type: application/json' \
  -d '{"sessionId":"demo","message":"What is our branch naming convention?"}'

curl -s -X POST http://127.0.0.1:8787/chat -H 'content-type: application/json' \
  -d '{"sessionId":"demo","message":"What about for hotfixes?"}'

Deploy

cd worker
npm run deploy

wrangler deploy uploads the Worker, applies the v1 Durable Object migration, and publishes frontend/ as the Worker's static assets — one deploy for the whole application.

Layout

codex-copilot/
├── worker/
│   ├── src/
│   │   ├── index.ts           # Worker entry: routing, prompt assembly, AI call
│   │   ├── durable-object.ts  # ConversationSession: transcript + prompt log
│   │   ├── retrieval.ts       # keyword retrieval over the seeded docs
│   │   ├── docs.d.ts          # type declaration for .md text imports
│   │   └── docs/              # the seeded Engineering Codex (5 documents)
│   ├── wrangler.toml
│   ├── tsconfig.json
│   └── package.json
├── frontend/
│   ├── index.html             # chat UI
│   ├── styles.css
│   └── app.js
├── example-prompt-history.md  # a real exported session log
├── prompt-history.md          # prompts used to build this project
└── README.md

The seeded documents (branch-naming.md, pr-review-policy.md, commit-message-format.md, ci-failure-escalation.md, oncall-rotation.md) are fictional standards written for this exercise. Replacing them with a real team's documents requires no code change beyond the import list in retrieval.ts.

Design notes and trade-offs

• Keyword retrieval, not embeddings. With five short documents, keyword scoring is fast, has no index to build or keep in sync, and is easy to reason about when an answer is wrong — you can see exactly why a section scored.
• Durable Object rather than KV for memory. A conversation is a read-modify-write on a single key per turn. A Durable Object gives that serialized, strongly consistent access; KV's eventual consistency would allow a fast follow-up to read a stale transcript.
• Bounded prompt, unbounded transcript. Only the last 12 turns are replayed into the prompt so it cannot grow without limit, but the full transcript stays in storage, so /history and the prompt log remain complete.
• Prompt log lives with the session. Keeping it in the same Durable Object means it is written in the same place as the state it describes; there is no second store to keep consistent. The trade-off is that the log is per-session rather than global — exporting across sessions would need an index.

Future improvements

• Cloudflare Vectorize for semantic retrieval, so "how do I name a branch for an urgent production fix?" matches the hotfix rule without sharing any keywords with it. The retrieve() signature is already the seam for this.
• Streaming responses. env.AI.run supports stream: true; the UI would render tokens as they arrive instead of waiting for the full answer.
• Cloudflare Workflows for multi-step coordination — for example a "check this PR against the Codex" flow that fetches a diff, evaluates it against several standards in parallel, and retries durably on failure.
• Evaluation set. A fixed list of question/expected-source pairs run against retrieve() in CI would catch retrieval regressions when documents change.
• Session expiry. A Durable Object alarm could clear transcripts after a period of inactivity rather than keeping them indefinitely.