[Enhancement] Personalized learning-path recommendation endpoint with validated structured output

[zeemscript]

Summary

Deen Bridge sells structured Islamic courses, but the AI service can't tell a learner what to study next — it has no notion of a user's progress at all. This issue adds a learning-path recommendation endpoint: it accepts a defined contract describing the user's completed courses, quiz performance, and goals (mirroring what dnb-backend can supply), and returns a structured, ordered learning path where every step carries an explicit justification — generated by Gemini under a validated JSON schema and checked by programmatic evaluation cases (persona fixtures with asserted properties, not exact-match outputs).

Current state

• The AI service is stateless about users: ChatRequest in main.py carries only prompt/chat_id/context, and nothing anywhere reads course or progress data.
• The platform's user/course data lives in dnb-backend (see the service table in README.md); stellar.py already documents the cross-repo coordination pattern in its module docstring ("dnb-backend uses the same issuers in src/services/stellar/stellarService.js") — this issue needs the same kind of explicit contract.
• Structured output is unused in the codebase today (chat() passes only temperature/top_p/top_k/max_output_tokens in generation_config), though the pinned google-generativeai==0.8.3 supports JSON mode with response_schema.

What to build

1. Input contract (the API is the deliverable as much as the model call) — POST /learning-path in a new learning.py router, with pydantic request models designed to be fillable by dnb-backend without schema gymnastics:
   • LearnerProfile: level (beginner|intermediate|advanced), goals (list of enum'd goals, e.g. quran_reading, fiqh_basics, arabic, seerah), optional time_per_week_hours, optional free-text notes (length-capped).
   • ProgressRecord[]: course_id, title, category, level, completion_pct, optional quiz_scores.
   • CourseCatalogItem[]: the candidate courses to recommend from (id, title, category, level, prerequisites, description) — passed in by the caller, so the AI service stays stateless and never invents course ids. Document this contract in the endpoint docstring and README so the backend team can implement the caller side against it.
2. Output schema — LearningPath: ordered steps[], each {course_id, title, order, reason (grounded in the profile/progress, non-empty), prerequisites_satisfied: bool, estimated_weeks} plus a path-level summary and the standard scholarly-humility note for religious-content sequencing. Validators: every course_id must exist in the submitted catalog (reject hallucinated ids), no duplicate steps, order contiguous from 1.
3. Generation — Gemini JSON mode (response_mime_type/response_schema translated from the pydantic models), low temperature, with a carefully engineered prompt: respect stated goals, don't recommend completed courses (completion ≥ threshold), respect prerequisite edges from the catalog, match level progression (no advanced-fiqh-first for beginners), and justify each step by reference to the profile. Bounded retry-on-invalid (same pattern as the quiz-generation issue if it lands first — share the retry helper rather than duplicating it).
4. Deterministic guardrails in code, not just prompt — post-validate the model's path: filter out completed courses and prerequisite violations programmatically even if the prompt asked nicely; if the filtered path drops below a minimum length, retry with the violations fed back. The endpoint must be unable to return a path that violates hard constraints, regardless of model behavior.
5. Evaluation cases — tests/eval_learning_path/: at least 5 persona fixtures (e.g. absolute beginner; intermediate learner aiming at Arabic; advanced learner with one gap; learner who completed everything in a category; conflicting goals + tiny catalog). For each, assert properties: no completed course recommended, prerequisites ordered before dependents, first step level ≤ persona level, every reason non-empty and mentioning a profile fact (string containment on goal/category tokens is acceptable). Run offline against recorded model outputs; keep a live opt-in mode for regenerating recordings. Note: the separately scoped evaluation-harness issue owns general chat-answer quality — these are endpoint-specific contract tests, and should be written so that harness can later absorb them.
6. Failure behavior — empty catalog → 422; model output failing hard constraints after retries → 502 with a structured error; oversized inputs (catalog > e.g. 200 items, notes too long) → 422 before any model call.

Acceptance criteria

• POST /learning-path accepts the documented profile/progress/catalog contract and returns a schema-valid LearningPath with per-step reasons.
• Hallucinated course ids, duplicate steps, and non-contiguous ordering are structurally impossible in returned payloads (validator-covered by tests).
• Programmatic post-filters guarantee no completed courses and no prerequisite violations in the final path, proven by a test whose fake model deliberately returns a violating path.
• All five persona evaluation fixtures pass their property assertions offline (no API key in CI).
• Contract is documented (endpoint docstring + README section addressed to dnb-backend implementers), including the statelessness decision.
• Input limits and error statuses (422/502) behave as specified.
• CI (flake8 extended to learning.py + pytest) stays green.

Pointers

• main.py (router inclusion, generation-config style), stellar.py (typed-router pattern and the cross-repo contract precedent in its docstring), requirements.txt, README.md (service table).
• Keep google-generativeai==0.8.3 pinned; JSON mode works on this version. Put the model call behind a seam for the separately scoped SDK migration.
• If the quiz/flashcard structured-output issue is in flight, coordinate on a shared structured_gen retry/validation helper instead of two copies.
• The catalog-passed-in design is deliberate — do not add a database or call dnb-backend from this endpoint in v1; that keeps the issue self-contained and testable.
• CI lints only main.py today — extend .github/workflows/ci.yml. PRs target dev.

Difficulty

High — API contract design across repos, constrained structured generation with hard programmatic guarantees, and property-based evaluation fixtures go well beyond a single prompt-and-parse endpoint.

---

🏆 GrantFox OSS — Official Campaign | FWC26. Apply for this issue through the GrantFox campaign page. The maintainer assigns one contributor before work starts; unassigned PRs may not be reviewed. PRs target the dev branch. Quality bar: CI must stay green.

💬 Questions or need help? Reach the maintainers and other contributors on the DeenBridge Telegram: https://t.me/+nst9lXNj1wc4ZDE0

[mayborn005]

Please assign this issue to me.

[OpensrcLord]

Hello @maintainer, I understand the expected outcome and would love to work on this. I'll provide a solution that satisfies the acceptance criteria.

[theFirstCodeManiac]

Please let me fix this issue for you. I have the required experience.

[grantfox-oss]

🦊 GrantFox — @theFirstCodeManiac has been assigned to this issue as part of the Official Campaign | FWC26 campaign!

Next steps:

1. Open a Pull Request referencing this issue (e.g., Closes #33)
2. Your PR will be reviewed by the Deen-Bridge maintainers

Good luck! Track your progress on GrantFox.