Skip to content

feat(backend): generation + audio endpoints, session listing, capability prompt#42

Closed
spencerwozniak wants to merge 3 commits into
zeapsu:mainfrom
spencerwozniak:split/02-backend
Closed

feat(backend): generation + audio endpoints, session listing, capability prompt#42
spencerwozniak wants to merge 3 commits into
zeapsu:mainfrom
spencerwozniak:split/02-backend

Conversation

@spencerwozniak

@spencerwozniak spencerwozniak commented May 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Split out of #37 — all new backend API surface in one reviewable unit. No frontend changes.

  • POST /api/generate/{flashcards,quiz,report} — KB-grounded generation. Pulls excerpts via semantic search (when a topic is provided) or random sample, prompts the configured LLM with response_format=json_object, validates the payload. Empty KB returns 400.
  • POST /api/generate/audio + GET /api/audio/{audio_id}.mp3 — KB-grounded narration script with sentence-level segment timings. Synthesizes to MP3 via OpenAI tts-1 when OPENAI_API_KEY is set; otherwise returns null audio_url and the frontend falls back to browser SpeechSynthesis. Streamed file path is regex-guarded against traversal.
  • GET /api/chat/sessions?limit=<n> — one-SQL session list joining sessions/messages/tomes for the History panel.
  • Chat SYSTEM_PROMPT gains an "App capabilities" section so the model can describe quiz / flashcards / audio / report / history / tomes / KB / upload accurately.

Depends on nothing; safe to merge before the frontend PR (the new endpoints are inert until UI calls them).

Test plan

  • Each /api/generate/* endpoint returns valid JSON with KB content present
  • Audio endpoint with OPENAI_API_KEY returns audio_url and the MP3 streams; without the key, audio_url is null
  • GET /api/chat/sessions returns the expected shape with tome_name, first_user_message, last_message_at
  • Existing chat behavior unchanged

spencerwozniak and others added 3 commits May 22, 2026 15:36
Adds POST /api/generate/{flashcards,quiz,report} on a new router. Each
endpoint pulls excerpts from the knowledge base (semantic search when a
topic is provided, random sample otherwise — optionally scoped to a
tome), prompts the configured LLM with response_format=json_object on
OpenAI-compatible providers, and validates the parsed payload before
responding. An empty KB returns 400 with a hint to upload documents.

Also exposes an llm_text(...) helper used by the audio endpoint for
plain-text completions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
POST /api/generate/audio retrieves KB context (or randomly samples it),
prompts the configured provider for a spoken-style narration script,
and segments it into sentence-level chunks with word-rate-estimated
timings for transcript scroll-lock.

When OPENAI_API_KEY is configured, the script is also synthesized to
MP3 via OpenAI's tts-1 voice (default: alloy) and cached to disk; the
response includes audio_url. Without a key, audio_url is null and the
frontend falls back to the browser's SpeechSynthesis voice.

GET /api/audio/{audio_id}.mp3 streams the cached file with a regex
guard against directory traversal.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Adds KnowledgeStore.list_sessions(): one SQL pass joining sessions,
  messages, and tomes that returns derived title (first user message),
  tome name, message count, and last activity timestamp. Ordered by
  most-recent activity.
- Adds GET /api/chat/sessions?limit=<n> as a thin wrapper for the
  History panel.
- Extends the chat SYSTEM_PROMPT with an "App capabilities you should
  know about" section so the model can describe its own routed
  features (quiz / flashcards / audio / report / history / tomes /
  knowledge base / upload) accurately when the user asks "what can
  you do?", and skips replying when the frontend has already routed
  away.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zeapsu

zeapsu commented May 25, 2026

Copy link
Copy Markdown
Owner

Thanks for splitting this backend API work out of the larger foundations PR. This is much easier to review as a standalone unit.

Overall, the direction looks useful and aligned with the Sage roadmap. The endpoints map nicely onto the planned Sources, Report, Quiz, Flashcards, Audio, Chat, and History surfaces.

I reviewed this in an isolated checkout and ran backend checks.

Checks:

  • backend pytest with isolated HOME: passed
    • 62 passed
  • fake-provider smoke test: passed for:
    • POST /api/generate/flashcards
    • POST /api/generate/quiz
    • POST /api/generate/report
    • POST /api/generate/audio
    • GET /api/chat/sessions
    • invalid audio ID guard

No critical issues jumped out, but I would like a small test follow-up before merging because this adds a meaningful amount of new backend API surface.

Main thing I’d want covered:

  • empty KB returns 400 for generate endpoints
  • successful flashcards/quiz/report/audio response shape with mocked provider
  • invalid audio ID returns 400
  • missing audio file returns 404
  • /api/chat/sessions returns the expected session shape
  • provider malformed JSON returns a controlled 502

A few smaller notes:

  • GET /api/chat/sessions?limit=<n> should bound or validate limit, for example with FastAPI Query(default=100, ge=1, le=200).
  • /api/generate/audio writes MP3s to /tmp/sage_audio. That is probably okay for an initial slice, but Sage is local-first and privacy-sensitive, so we should eventually add a cache lifecycle or use an app-scoped cache directory.
  • Report/audio prompt context may need a total character or token budget so large chunks do not create unexpectedly huge provider calls.

Recommended next step:

Add targeted backend tests for the new routes, then this becomes a much safer merge candidate.

Great Sage likes the split and the direction here. I mostly want the test coverage to match the new API surface before this becomes a dependency for the frontend work.

@zeapsu

zeapsu commented May 27, 2026

Copy link
Copy Markdown
Owner

Thanks for splitting this backend work out. PR #46 has now been merged into main as 53b83e0 and cherry-picked/integrated these backend generation, audio, session-listing, and capability-prompt changes, with a small amount of follow-up test/polish on top. I'm closing this as superseded by #46 so the open PR list reflects the current mainline state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants