Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 

Repository files navigation

OpenPathshala — Architecture

An open-source, offline-first, multilingual AI workflow toolkit for Indian schools. This repository is the architecture overview: the problem research, the system design, the module boundaries, the model-routing approach, and the privacy/data-governance model. It is a technical document — it deliberately contains no product, pricing, or go-to-market material.

Status: Architecture / vision (v0.1) License: Apache-2.0 (code) · CC BY-SA-4.0 (project-authored content) — see §9


0. What this document is

A modular toolkit that automates the highest-friction, lowest-judgment workflows in Indian schools — assessment, content generation, administration, accessibility — designed to run across the full spectrum of Indian school infrastructure, from no-electricity rural schools to fully-connected urban ones.

This document does two things:

  1. Grounds the problem in current official data (UDISE+ 2024-25, NEP 2020, DPDP Act 2023). The architecture only earns trust if the problem statement is accurate, so numbers are cited and dated.
  2. Specifies a buildable system — module boundaries, a model-routing approach, and a privacy/child-safety design — not hand-waving.

1. Research foundation — the Indian school landscape

1.1 Scale

Metric Value Source/year
Total schools ~14.71 lakh (1,471,473) UDISE+ 2024-25
Total students enrolled ~24.8 crore UDISE+ 2023-24
Total teachers ~1.01 crore (crossed 1 crore for the first time) UDISE+ 2024-25
Government schools ~10.17 lakh UDISE+ 2023-24
Private/aided/other schools ~4.54 lakh UDISE+ 2023-24
Secondary-stage dropout rate ~10.9% UDISE+ 2023-24

This is not one environment. It is at least three, separated by infrastructure, and the design treats that separation as a first-class constraint rather than an afterthought.

1.2 The digital divide — the central design constraint

UDISE+ 2024-25 national infrastructure coverage:

Facility % of schools Functional % Notes
Drinking water 99.3% 99.0% Near-universal
Toilets (girls) 96.8% 95.3% Near-universal
Electricity 93.7% 92.0% ~1.2 lakh schools still lack functional power
Computers 64.7% 58.0% One-third have none; many "have" but non-functional
Internet 63.5% One-third have no connectivity at all
Functional solar panels 10.9% Power resilience is rare

State-level disparity is severe. Internet connectivity ranges from ~100% (Delhi, Andhra Pradesh, Gujarat) down to ~36–46% (Uttar Pradesh, Jharkhand) and historically below 20% in West Bengal and Bihar. North-eastern states lag furthest. A national tool that assumes connectivity excludes the schools that need help most.

A practical, field-tested mitigation: tablet/Chromebook ICT labs charged on a single power strip — devices charge in 3–4 hours and run 7–8 hours, covering a full school day without continuous electricity. This validates an offline-first, intermittently-syncing design over a cloud-streaming one.

1.3 Structural realities the architecture must respect

  • Multilingualism is not optional. Instruction spans 22+ scheduled languages; NEP 2020 mandates mother-tongue / home-language instruction in early grades. DIKSHA supports 36 Indian languages; Bhashini (via Anuvaad) provides translation across 22 languages. Any grading or content tool must handle code-mixed, regional-script and English-medium inputs.
  • Board and curriculum fragmentation. CBSE, CISCE, NIOS and ~30 distinct state boards each define their own syllabi, marking schemes and question patterns. Content generation must be curriculum-pluggable, not hard-coded to one board.
  • NEP 2020 four-stage structure. Foundational (pre-primary + Grades 1–2), Preparatory (3–5), Middle (6–8), Secondary (9–12). Workflows and rubrics differ sharply by stage.
  • Existing open public infrastructure to interoperate with, not replace. DIKSHA is built on the open-source Sunbird / NDEAR building blocks; NCERT content is openly licensed; QR-coded "energized textbooks" already link print to digital. Building on these is the right engineering choice.
  • Teacher load. With ~1 crore teachers for ~24.8 crore students, the binding constraint on quality is teacher time. Every workflow below is chosen because it returns hours to a teacher.
  • Data protection & minors. The DPDP Act, 2023 governs personal data; children's data requires verifiable parental consent and heightened safeguards. Privacy is a hard requirement (§8).

2. The workflow problem catalogue

Each workflow is scored for who it helps, the pain it removes, and which infrastructure tier it can realistically serve.

Tier key: T0 = no electricity/connectivity (paper + intermittent device), T1 = some shared devices / intermittent internet, T2 = connected (1:1 or lab + broadband).

2.1 Assessment & grading

# Workflow Pain removed Tiers
A1 Grade handwritten assignments & exams Hours of manual marking; inconsistent rubrics; slow feedback T0¹/T1/T2
A2 Generate question papers from textbooks (online + scanned) Days of paper-setting; blueprint compliance; difficulty balance T1/T2
A3 Rubric-aligned feedback generation — a mark, plus why + next step Marks without learning; no individualized feedback at scale T1/T2
A4 Foundational literacy & numeracy (FLN) assessment (NIPUN Bharat aligned) No scalable way to assess early-grade reading/math T0¹/T1
A5 Diagnostic mis-conception detection — cluster wrong answers Teacher can't see class-wide gaps from a stack of papers T1/T2
A6 Cohort-level similarity detection (not punitive surveillance) Cannot detect mass copying in large sections T2

¹ T0 grading works asynchronously: photograph on a phone when a device is briefly available, queue, sync, grade in batch, print/return.

2.2 Content & curriculum generation

# Workflow Pain removed Tiers
C1 Blueprint-constrained question-paper generation (marks distribution, Bloom's levels, board pattern) Manual blueprint balancing T1/T2
C2 Differentiated worksheet & practice-set generation No time to make tiered material T1/T2
C3 Lesson-plan drafting mapped to learning outcomes Lesson planning eats prep time T1/T2
C4 Textbook → multilingual explainer (chapter to mother-tongue notes) Mother-tongue material scarce for many languages T1/T2
C5 Answer-key & model-answer generation with step marking Marking schemes are slow to build T1/T2

2.3 Administrative automation

# Workflow Pain removed Tiers
D1 UDISE+ / APAAR data-entry assistance — extract from registers, pre-fill, validate Mandatory data entry is manual and error-prone T0/T1
D2 Attendance digitization from paper registers (photo → structured) Manual tallying; no analytics T0/T1
D3 Report-card generation — marks + comments in the parent's language Hand-writing 40+ report cards per class T1/T2

2.4 Accessibility, inclusion & communication

# Workflow Pain removed Tiers
E1 Read-aloud / text-to-speech of notes in regional languages (CWSN support) Print-disabled & low-literacy learners excluded T1/T2
E2 Image/diagram description for visually-impaired students No alt-text for visual material T2
E4 Textbook-grounded doubt-solving tutor (no open-web speculation) No after-hours academic support T2

The entry-point workflows are A1 + A2 — they remove the most teacher-hours.


3. Design philosophy

Five principles, each derived from §1:

  1. Offline-first, paper-first. Paper is the universal interface in Indian schools. The system must work when capture happens on paper and a phone, and processing happens later. Connectivity is an optimization, never a precondition.
  2. Tiered degradation, not feature gating. The same workflow runs at all three tiers with graceful degradation in model quality/latency — not "premium schools get features, poor schools get nothing."
  3. Teacher-in-the-loop, always. AI proposes; a human disposes. Especially for grading, output is a defensible draft a teacher reviews and overrides — never an unaccountable final mark on a child's record.
  4. Route by judgment required, not by default to the biggest model. Most tasks are extraction/classification (cheap). A minority need reasoning. A small fraction need deep reasoning. Discipline comes from honest routing (§6).
  5. Interoperate with public infrastructure. Speak DIKSHA/Sunbird, Bhashini, NCERT-licensed content and APAAR/UDISE schemas. Don't rebuild the commons; extend it.

4. System architecture (high level)

                        ┌──────────────────────────────────────────────┐
                        │                CAPTURE LAYER                   │
                        │  Web capture (PWA) · scanner/printer bridge ·  │
                        │  register photos · camera scan (deskew)        │
                        └───────────────┬──────────────────────────────┘
                                        │  (queue locally; sync when online)
                        ┌───────────────▼──────────────────────────────┐
                        │              SYNC / GATEWAY                     │
                        │  Offline queue (IndexedDB/SQLite) ⇄ sync        │
                        │  Edge node (BRC/school server) optional         │
                        └───────────────┬──────────────────────────────┘
                                        │
            ┌───────────────────────────▼───────────────────────────────┐
            │                      PROCESSING CORE                        │
            │  ┌────────────┐   ┌──────────────┐   ┌──────────────────┐  │
            │  │ INGEST/OCR  │→ │  AI REASONING │→ │  OUTPUT/RENDER    │  │
            │  │ handwriting │  │  (router →    │  │  papers, marks,   │  │
            │  │ + layout    │  │   cheap/mid/  │  │  feedback, report │  │
            │  │ + language  │  │   frontier /  │  │  cards, TTS,      │  │
            │  │ detect      │  │   open-weights)│  │  translations     │  │
            │  └────────────┘   └──────────────┘   └──────────────────┘  │
            │  ┌──────────────────────────────────────────────────────┐  │
            │  │   KNOWLEDGE / GROUNDING (RAG over curriculum)         │  │
            │  │   NCERT + state-board textbooks, blueprints, rubrics  │  │
            │  └──────────────────────────────────────────────────────┘  │
            └───────────────────────────┬───────────────────────────────┘
                                        │
                        ┌───────────────▼──────────────────────────────┐
                        │           DATA & GOVERNANCE LAYER              │
                        │  PII redaction · consent ledger (DPDP) ·       │
                        │  audit log · role-based access · retention     │
                        └───────────────┬──────────────────────────────┘
                                        │
                        ┌───────────────▼──────────────────────────────┐
                        │      INTEGRATIONS: DIKSHA/Sunbird · Bhashini · │
                        │      UDISE+/APAAR schemas                       │
                        └───────────────────────────────────────────────┘

The processing core is stateless and deployable three ways: (a) cloud for T2, (b) a school/block edge node (a single mini-PC at a Block Resource Centre serving nearby schools) for T1, (c) on-device with small open models for T0/T1 offline windows.


5. Module deep-dives (the two entry-point workflows)

5.1 Module A1 — Grading handwritten assignments & exams

Pipeline:

  1. Capture. Photograph or scan answer scripts (a phone camera is the realistic device). Auto page-boundary detection, deskew, and per-student grouping via a roll-number cover sheet or a printed QR/OMR header.
  2. Pre-process. Image cleanup, orientation, and a language/script detector (Devanagari, Gujarati, Tamil, Latin, code-mixed) to route downstream choices.
  3. Handwriting transcription (HTR). A vision model converts handwriting → text with spatial structure preserved (which answer maps to which question). Swappable: a frontier vision model at T2, an on-device open HTR model (TrOCR-family / Bhashini OCR) at T0/T1, with confidence scores.
  4. Rubric grounding. The relevant marking scheme / model answer is retrieved from the curriculum store and prompt-cached (it is reused across an entire class set — a large caching win).
  5. Grade + justify. A reasoning model scores each answer against the rubric, returning per-criterion marks, a short justification, a confidence, and flags low-confidence items for mandatory human review.
  6. Teacher review UI. Marks render over the original script image; the teacher accepts/overrides with one tap; overrides are logged and become fine-tuning / few-shot signal for that teacher's style.
  7. Output. Marks to the gradebook, feedback to the student in their language, and a class-level misconception cluster (A5) for the teacher.

Why this is safe: the teacher always sees the original script beside the proposed mark; low-confidence and high-stakes items are never auto-finalized; the audit log makes every machine-proposed mark traceable.

Quality guardrails: double-pass on disagreement (escalate to a stronger model when two cheap passes disagree beyond a threshold); numeric-answer verification via a calculator tool rather than the model's arithmetic; explicit "illegible" handling instead of guessing.

5.2 Module A2 — Question-paper generation from textbooks

Inputs: (a) a source — online textbook (NCERT/state, openly licensed) or a scanned copy; (b) a blueprint — marks distribution, question types (MCQ/short/long), Bloom's-taxonomy spread, chapter weightage, board pattern, difficulty mix, language.

Pipeline:

  1. Ingest source. If scanned, OCR with layout reconstruction (tables, diagrams, equations). Chunk by chapter/learning-outcome and embed into the curriculum store.
  2. Blueprint as constraints. The blueprint is a structured constraint object, not prose — the generator must satisfy it (e.g. exactly 5 marks of "apply"-level questions from Chapter 3).
  3. Generate candidate items, each tagged with chapter, outcome, Bloom's level, marks, and a generated answer key with step marking.
  4. Constraint-satisfaction assembly. A solver/validator assembles items into a paper that provably meets the blueprint; if a category is over/under-produced, only that slice is regenerated (cheap, targeted).
  5. Render to print-ready PDF (school header, instructions, marks in margins) and a separate answer key — in the chosen language.
  6. Anti-leakage & freshness. Seeded variation so two sections get equivalent-but-different papers; a similarity check against previously generated papers.

Scanned-textbook nuance: Indian classrooms heavily use photocopied/scanned material. Robust ingest of poor-quality scans (skew, marginalia, bleed-through) is a genuine differentiator and a reusable component on its own.


6. The AI reasoning layer — model routing

The single most important quality decision is which model does which step. Defaulting every task to the largest model is the most common and most expensive mistake.

Step Judgment required Default tier Fallback (offline/T0)
Language/script detect Trivial On-device classifier rules
Handwriting transcription Perceptual, medium Vision: cheap → escalate on low confidence open HTR (TrOCR/Bhashini OCR)
Extraction / structuring Low Cheap tier small open LLM
Worksheet/notice generation Low–medium Cheap / mid tier small open LLM
Grading against rubric Medium (reasoning + fairness) Mid tier open mid LLM + human review
Question-paper generation Medium–high Mid tier
Misconception analysis, ambiguous scripts High Frontier tier (sparingly) escalate-to-cloud-only

Routing logic: a lightweight classifier scores each request; the large majority routes to the cheap tier, a minority to the mid tier, a small fraction to the frontier tier. Reserving frontier models for genuinely hard or low-confidence cases materially reduces total inference work with negligible quality loss, because cheap tasks don't benefit from bigger models.

Three structural efficiency levers, all applicable here:

  • Prompt caching (large discount on cached input): the rubric/marking-scheme, board blueprint and textbook chapter are identical across an entire class set — cache once, reuse for 40+ scripts.
  • Batch processing (large discount): grading and overnight paper-generation are not latency-sensitive — queue and process in batch.
  • Tier routing: reserve frontier models for genuinely hard or low-confidence cases.

On open models (the T0/T1 and sovereignty story): every step has a swappable open-weights fallback so the toolkit is never hard-locked to one vendor. This is correct engineering (resilience, sovereignty, offline) and correct open-source posture — the project must be usable with no paid API at all. Frontier models are the quality ceiling the router targets when available; open models are the floor that guarantees the toolkit always runs.


7. Interoperability with the public commons

The toolkit is designed to extend, not replace, India's public education infrastructure:

  • DIKSHA / Sunbird / NDEAR — the open building blocks for content and identity; energized (QR-coded) textbooks link print to digital.
  • Bhashini / Anuvaad — the national language-AI mission for translation across 22 languages.
  • NCERT — openly-licensed textbooks and resources for grounding generation and RAG.
  • UDISE+ / APAAR — the official school- and student-record schemas for administrative workflows.

Speaking these schemas is both the right engineering choice and the reason the project can be a durable public good rather than a silo.


8. Privacy, child-safety & data governance

This is a hard requirement, not a section to skim — the users are minors and the law is the DPDP Act, 2023.

  • Data minimization & on-device redaction. Student names, faces, roll numbers and identifiers are redacted/tokenized before any data leaves the device or edge node. The model grades anonymized scripts wherever possible (a script ID, not a child's name).
  • Consent ledger. A DPDP-aligned record of verifiable parental/guardian consent per student, with purpose limitation and revocation. No consent → no processing of that child's identifiable data.
  • Teacher-in-the-loop accountability. No machine-proposed mark becomes final on a child's record without human confirmation; every proposal is auditable.
  • Grounded, bounded generation. The doubt-solving tutor and content modules are RAG-grounded strictly in the prescribed curriculum — refusing open-web speculation reduces both hallucination and child-safety surface area.
  • Data residency & sovereignty. Edge/on-device deployment keeps sensitive data in-school; cloud routing is opt-in and region-aware.
  • Retention & deletion. Default short retention for raw images; structured marks retained per school policy; hard-delete honored.

9. Open-source structure & licensing

Repository model: a monorepo of independently-usable packages — the "building blocks" model (mirroring Sunbird/NDEAR) is what lets pieces be adopted and depended upon by others.

openpathshala/
├── packages/
│   ├── capture-pwa/          # offline-capable web capture (camera scan, deskew, queue)
│   ├── ingest-htr/           # handwriting + scanned-doc OCR, layout, language detect
│   ├── core-router/          # model routing, caching, batch, budget caps
│   ├── grade-engine/         # rubric grounding + grading + review API
│   ├── paper-gen/            # blueprint-constrained question-paper generation
│   ├── curriculum-rag/       # NCERT/state textbook ingest + vector store
│   ├── redact/               # DPDP-aligned PII redaction & consent ledger
│   ├── i18n-bhashini/        # multilingual bridge (Bhashini + model)
│   └── integrations/         # DIKSHA/Sunbird, UDISE/APAAR
├── deploy/
│   ├── cloud/                # T2
│   ├── edge-node/            # T1 (single mini-PC for a cluster of schools)
│   └── on-device/            # T0/T1 offline
└── docs/                     # this document + ADRs + deployment guides

Licensing: Apache-2.0 for code (permissive → maximizes adoption and dependency); CC BY-SA-4.0 for any project-authored curricula/rubrics; upstream NCERT licenses (CC BY-NC-ND / CC BY-NC-SA) are respected for sourced content.


Appendix — primary sources

  • UDISE+ 2024-25 & 2023-24 — Ministry of Education, Govt. of India (school counts, infrastructure %, teacher count, dropout). udiseplus.gov.in
  • DIKSHA / CIET-NCERT — open Sunbird/NDEAR architecture, 36 languages, energized textbooks, content licensing. diksha.gov.in
  • Bhashini / Anuvaad — national language-AI mission, 22-language translation.
  • NEP 2020 — four-stage structure; mother-tongue instruction; clauses 2.60 & 23.60 (DIKSHA as national repository).
  • DPDP Act, 2023 — personal-data protection; children's-data safeguards.

All statistics are dated; verify against the latest official releases before publication, as UDISE+ updates annually.

About

Architecture overview for Learn Clue — an open-source, offline-first, multilingual AI workflow toolkit for Indian schools.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors