Changes in Assistant architecture

[vanguard-hack]

Feature Ticket: Assistant Architecture Upgrade — Page Context, Smarter Router, Agentic Fallback, Multimodal Input

Summary

Upgrade the crime analytics assistant's backend architecture. Current flow is a single Router 0 deciding between RAG, Text2ZCQL, and a plain LLaMA-70B (Groq) fallback. This ticket covers giving the LLM full webpage context, improving router accuracy (including hybrid multi-route execution), making the fallback model agentic with tool access, and supporting multimodal input — hardened per a security/architecture review covering ZCQL injection risk, redaction timing, payload bloat, routing efficiency, ReAct loop/timeout limits on Catalyst, and audit-log immutability.

Architecture diagram (Lucid, v3 — security hardened): https://lucid.app/lucidchart/3a95d661-8c82-422e-b1fa-942f7e2a9027/edit

---

Current State

User query -> Router 0 -> [RAG | Text2ZCQL | Fallback LLaMA-70B (Groq)]

• Router 0 has no visibility into what's currently on the officer's screen.
• Fallback model just answers from its own knowledge — no tools, no internet, degrades badly on anything outside training data or outside the internal KB/DB.
• No image/document input support.

---

Target State

Client (Chat UI + standardized page-context metadata, multimodal input)
   -> Request Orchestrator (normalizes text + image + page context + history)
   -> Enhanced Intent Router (confidence-scored, hybrid fan-out capable)
        -> RAG Pipeline -> Vector DB -> Pre-Retrieval Clearance Filter
        -> Text2ZCQL Pipeline -> ZCQL Validator (AST) -> read-only Execution -> Pre-Retrieval Clearance Filter
        -> Fast Vision Pre-Parser (Zia Vision / QuickML quick OCR) -> simple result, or escalate ->
        -> Agentic Fallback Engine (LLaMA-70B Groq on Catalyst AppSail, ReAct loop, capped iterations)
   -> Response Aggregator & Formatter (+ Post-Generation Guardrail) -> back to Client
   -> Immutable, Append-Only Audit & Logging Layer (all paths)

This revision addresses a security/architecture review — see the Security & Data Protection and Pipeline & Routing Changes sections below for the specific issues raised and how they're addressed.

---

1. Full Webpage Context as LLM Input

Goal: The assistant should be aware of what the officer is currently looking at (open FIR, active filters, visible table/report) without them having to repeat it in the query.

Requirements:

• Add a Page Context Extractor on the client side that captures a lightweight, standardized metadata schema on each query — never a raw DOM/state snapshot. Example shape:

  {
    "active_fir_id": "FIR-2026-04829",
    "current_module": "suspect_search",
    "applied_filters": { "district": "Bengaluru East", "date_range": "last_30_days" }
  }

• This snapshot is sent alongside every chat message as a page_context object in the request payload.
• Backend passes page_context into the Router (for routing decisions) and into whichever pipeline handles the query (for grounding — e.g., "summarize this" should resolve to the FIR currently open).
• Do not send full raw HTML/DOM — a generic DOM tree bloats payload size and burns LLM context tokens for no benefit. The schema above should be the contract between client and backend, versioned so both sides can evolve it deliberately.
• Page context should have a TTL/staleness check — discard if older than a few seconds to avoid acting on stale screen state.

---

Security & Data Protection (Critical — added after architecture review)

These three issues were flagged in review and must be designed in from the start, not bolted on later.

A. ZCQL Injection & Unconstrained Mutation

Risk: If Text2ZCQL generates queries directly from raw user/agent input, a prompt injection could attempt data extraction beyond the caller's authorization, or worse, a mutation.

Requirements:

• The execution role the generated ZCQL runs under must be strictly read-only at the database/DataStore permission level — not just "the app doesn't expose write UI," an actual read-only credential/role.
• Every generated ZCQL statement passes through a ZCQL Validator (AST parser) before execution:
  • Reject UPDATE, DELETE, INSERT, and any DDL outright.
  • Reject table-wide scans without a WHERE clause.
  • Reject queries that lack a tenant/precinct-level isolation clause (every query must be scoped to the caller's jurisdiction/permission boundary — this can't be optional).
  • Log and hard-fail (don't silently rewrite) any query that fails validation; return a safe "I can't run that query" message rather than attempting to auto-correct it.
• This validator sits between "Text2ZCQL generation" and "Execution" as its own explicit pipeline stage (see diagram) — not folded into the generation prompt as a soft instruction, since prompt-level instructions are exactly what injection attacks target.

B. Late-Stage Redaction Vulnerability

Risk: If role-based redaction only happens at the Response Aggregator, sensitive data (PII, juvenile FIR records, victim identities) has already been ingested into the LLM's context during generation — the model has "seen" it even if the final output is redacted, which is both a leakage risk (via prompt injection, model errors, or logging) and hard to audit.

Requirements — dual-tier redaction:

1. Pre-Retrieval Clearance Filter (new pipeline stage, applies to both RAG and Text2ZCQL): filter Vector DB / ZCQL results against the caller's badge ID and clearance level before the results are assembled into LLM context. The LLM should never receive data the caller isn't authorized to see in the first place.
2. Post-Generation Guardrail (at the Response Aggregator, existing stage — now explicitly a redaction checkpoint, not just formatting): a second pass over the generated response to catch anything that slipped through — e.g., the model paraphrasing or inferring something it shouldn't have surfaced.

• Both stages must be logged (see Audit section) — specifically what was filtered and why, for compliance review.

C. Payload Bloat from DOM/State Snapshots

Covered above in Section 1 — resolved by using the standardized page_context metadata schema instead of raw DOM extraction.

---

Pipeline & Routing Changes (from review)

Fast-path for simple image tasks

Problem: Routing every image upload straight into the full Agentic Fallback Engine is slow and expensive when the ask is simple (e.g., reading a vehicle plate or an ID card) — the full ReAct loop is overkill.

Fix: Add a Fast Vision Pre-Parser as its own router destination, sitting alongside RAG/Text2ZCQL/Agentic Fallback rather than folded inside the agent:

• Handles plates, ID cards, barcodes, and other narrow extraction tasks via Zia Vision APIs / QuickML quick OCR directly — no agent loop, no multi-tool reasoning, low latency.
• If the Fast Vision Pre-Parser determines the task needs open-ended reasoning (e.g., "what's going on in this photo," "does this damage match the FIR description"), it escalates to the Agentic Fallback Engine, which then uses the Qwen2.5-VL vision tool for the heavier reasoning case.
• Router should default image-only/image-primary requests to the Fast Vision Pre-Parser first; only route directly to the full Agentic Fallback when the query text itself signals complex reasoning is needed regardless of the image.

ReAct loop latency and Catalyst function timeouts

Problem: A multi-turn ReAct loop calling Web Search, Vision, and ZCQL tools via Groq can easily exceed Zoho Catalyst's Basic/Advanced I/O Function execution timeout (typically 15–30s).

Fix:

• Cap ReAct iterations at 3–4 steps — this was 5–6 in the original draft; tightened based on Catalyst's timeout constraints.
• Add step-level timeouts so a single slow tool call (e.g., web search) can't consume the entire budget.
• Host the Agentic Orchestrator on Catalyst AppSail, not Basic/Advanced I/O Functions — AppSail supports longer-lived operations and streaming responses, which this agent loop needs. This is a hosting/infra decision that should be made before implementation starts, since it affects how the orchestrator is deployed and scaled.

Intent collisions between RAG and Text2ZCQL

Problem: Ambiguous queries genuinely need both — e.g., "What is the procedure when handling FIR #4029?" needs SOP knowledge (RAG) and the specific FIR's data (ZCQL). Forcing a single exclusive route produces an incomplete answer either way.

Fix: The Router supports hybrid parallel execution: on detecting a mixed-intent query, fan out to RAG and Text2ZCQL simultaneously, then merge both results at the Response Aggregator rather than picking one path. This should be a distinct router outcome (not just "low confidence, send to fallback") — a mixed-intent query is a different case from an ambiguous/unclear one.

---

Goal: Reduce misrouted queries (e.g., structured data queries going to RAG, or KB questions going to Text2ZCQL) and use the fallback engine only when genuinely needed.

Requirements:

• Move from a single-shot classifier to a confidence-scored router:
  • Router returns a route + confidence score, not just a route.
  • If confidence is below threshold, route to the Agentic Fallback Engine instead of guessing between RAG/Text2ZCQL.
• Support a hybrid fan-out outcome: when a query needs both SOP knowledge and specific record data (e.g., "procedure for FIR #4029"), trigger RAG and Text2ZCQL in parallel and merge at the aggregator, rather than forcing a single exclusive path. See Pipeline & Routing Changes above.
• Treat "image attached" as its own routing signal: default to the Fast Vision Pre-Parser, not straight to Agentic Fallback — see Pipeline & Routing Changes above.
• Router input should now include: user query + page_context + last N turns of conversation history (for follow-up queries like "show me his other cases too").
• Log router decisions (query, chosen route, confidence, page_context snapshot) for offline review — this is how we'll tune the classifier over time.
• Add a lightweight override mechanism: if the query matches a known slash command or a clearly structured pattern (e.g., FIR number regex), skip the classifier and route directly — cheaper and more reliable than relying on the LLM classifier for obvious cases.
• Define and document the confidence threshold(s) with the team — start conservative (favor fallback over wrong routing) and tune based on logged data.

---

3 & 4. Agentic Fallback Engine (with Tools + Internet Access)

Goal: Replace the current plain-answer LLaMA-70B fallback with an agent that can reason, call tools, and fetch live data instead of guessing or refusing.

Requirements:

• Keep LLaMA-70B on Groq as the underlying model (fast inference matters here), but wrap it in an agent loop (ReAct-style: reason → act → observe → repeat) using function/tool calling.
• Host the orchestrator on Catalyst AppSail, not Basic/Advanced I/O Functions — those have a ~15–30s execution timeout that a multi-tool ReAct loop calling Web Search, Vision, and ZCQL tools can easily exceed. AppSail supports longer-lived operations and streaming.
• Cap iterations at 3–4 tool-call cycles (tightened from an initial 5–6 based on the Catalyst timeout constraint) and add step-level timeouts so one slow tool call can't consume the whole budget.
• Tool registry for this agent should include, at minimum:
  • Web Search Tool — internet search + fetch, for anything outside internal KB/DB (e.g., general legal reference, IPC/BNS section lookups, public info)
  • Page Context Tool — lets the agent re-read current screen state mid-reasoning if needed
  • Internal DB / ZCQL Tool — same validated, read-only path as the Text2ZCQL pipeline (ZCQL Validator + read-only role + pre-retrieval clearance filter apply here too — the agent doesn't get a shortcut around those checks just because it's calling the tool itself)
  • Calculator / Date-Utility Tool — for date range math, statistics, etc.
  • Vision Tool: Catalyst QuickML — Qwen2.5-VL (7B Vision-Language model) — for complex, open-ended image reasoning only; simple extraction (plates, IDs, barcodes) should already have been handled by the Fast Vision Pre-Parser before the request reaches this engine. See recommendation and rationale in the Multimodal section below.
  • Zia Vision APIs (object recognition, image moderation, barcode scanning) — for narrow, deterministic extraction tasks the agent can call alongside the VLM (see below)
• Every tool call and its result must be logged to the immutable Audit Layer (see Audit section) — this is a fallback path handling potentially sensitive queries, so traceability matters.
• Define a clear timeout and graceful degradation behavior: if the agent can't resolve within the iteration/time budget, return partial findings with a clear "I couldn't fully verify this — here's what I found" rather than hanging or hallucinating a confident answer.
• Web search tool should be scoped/filtered where possible (e.g., avoid pulling from unreliable sources) — align with existing content-safety practices already used elsewhere in the platform.

ZCQL grammar note: ZCQL has real syntactic constraints versus standard SQL — limits on complex nested joins, specific aggregation functions, and pagination per batch. Whatever few-shot examples or fine-tuning data back the Text2ZCQL generator (used both in the main pipeline and as the agent's DB tool) need to be strictly grounded in actual ZCQL grammar, not generic SQL, or the validator will end up rejecting a lot of generated queries that were written against the wrong dialect.

---

5. Multimodal Input Support

Goal: Officers should be able to attach images (e.g., evidence photos, scanned documents, screenshots) alongside text queries.

Recommended model: Zoho Catalyst QuickML — Qwen2.5-VL (7B Vision-Language)

Since the platform is already Zoho Catalyst, the right choice is the vision-language model natively hosted inside Catalyst QuickML, rather than bolting on an external vendor (OpenAI/Anthropic/Gemini vision APIs, etc.). Reasoning:

• Already available in-platform. QuickML's LLM serving supports Qwen2.5-VL-7B directly — deployable as a chat/completion endpoint with OAuth-based auth, the same auth model used elsewhere in Catalyst. No new vendor contract, no new credential/network path to secure.
• Data stays in Catalyst. For a police crime-analytics platform, keeping evidence photos and scanned documents inside the existing Catalyst boundary (rather than sending them to a third-party API) is a meaningfully simpler compliance story than routing images externally.
• Fits the existing agent design. It can be wired in as just another callable tool in the Agentic Fallback Engine's tool registry — same OAuth pattern Catalyst functions already use to call QuickML endpoints.
• Proven for image reasoning, not just OCR. Catalyst has already used this same model internally for chart/image insight generation (trend/anomaly detection from visual data), so it's a validated path, not a novel integration.

Two complementary pieces — don't conflate them:

1. Qwen2.5-VL (QuickML) — generative, open-ended image understanding: "what does this photo show," "summarize this scanned FIR page," "describe this vehicle damage." This is the one added to the agent's tool registry as the primary vision tool.
2. Zia Vision APIs (already part of Catalyst — object recognition, image moderation, barcode scanning, face analytics) — narrow, deterministic extraction tasks. Useful as additional callable tools for specific sub-tasks (e.g., moderate an uploaded image before processing, extract a barcode from an evidence tag) rather than as a replacement for the VLM's open-ended reasoning.

Requirements:

• Client: add image/file attachment support to the chat input bar.
• Request Orchestrator: accept and normalize multimodal payloads (text + image references) into the unified request format sent downstream.
• Router: image-primary requests default to the Fast Vision Pre-Parser (Zia Vision APIs / QuickML quick OCR) for simple extraction; only escalate to the Agentic Fallback Engine's Qwen2.5-VL tool when open-ended reasoning is genuinely needed. See Pipeline & Routing Changes above — this avoids running the full ReAct loop for a plate read or ID scan.
• Agentic Fallback Engine: deploy Qwen2.5-VL-7B as a QuickML LLM-serving endpoint; register it as a callable tool the agent invokes when complex image reasoning is needed. Register relevant Zia Vision APIs as separate narrow-purpose tools for both the Fast Vision Pre-Parser and (when needed) the agent.
• Define supported file types and size limits up front (e.g., JPG/PNG, max 10MB) and add clear client-side validation/error messaging.
• Confirm QuickML endpoint latency under expected load — 7B vision models are lighter than 70B text models but worth benchmarking, especially for the Fast Vision Pre-Parser's low-latency path.

---

Response Aggregator & Formatter

• All pipelines (RAG, Text2ZCQL, Fast Vision Pre-Parser, Agentic Fallback) should return a common internal response shape (answer text, sources/citations if any, confidence/route metadata) so the aggregator can format consistently for the UI regardless of which path handled the query.
• This is also where the Post-Generation Guardrail runs — the second tier of the dual-tier redaction design (see Security & Data Protection above). This is not just formatting: it's an explicit re-check for anything that shouldn't be in the response, applied after the pre-retrieval filter has already done the primary job of keeping unauthorized data out of the LLM's context in the first place.
• Role-based redaction is applied here as a single choke point for final output, on top of (not instead of) the pre-retrieval filtering in each pipeline.

---

Audit & Logging Layer

Must be immutable and append-only — write-restricted Catalyst DataStore (a role/table that the application can only insert into, never update or delete from) or an external SIEM integration. Since this handles FIR/criminal records, the audit trail itself has to be tamper-evident; a mutable log is not sufficient for compliance review.

Every request, regardless of route, must log:

• User ID/badge number
• Route taken + router confidence score (including hybrid fan-out decisions)
• page_context snapshot at time of query
• ZCQL Validator decisions (query submitted, whether it passed/failed validation and why)
• Pre-retrieval and post-generation redaction actions (what was filtered, at which tier)
• For Fast Vision Pre-Parser and Agentic Fallback: every tool call made, its input/output, and iteration count
• Final response returned
• Timestamp

This applies uniformly across every pipeline — not just the sensitive-lookup commands from the slash-command ticket.

---

Acceptance Criteria

• Every chat request includes a structured, schema-versioned page_context object (not raw DOM) reflecting current screen state
• Router returns route + confidence score; supports a distinct hybrid fan-out outcome for mixed RAG+ZCQL queries
• Router decisions are logged for offline tuning
• Text2ZCQL-generated queries pass through the ZCQL Validator (AST parser) before execution; UPDATE/DELETE/DDL/unscoped scans are hard-rejected, not auto-corrected
• ZCQL execution role is verified read-only at the database/DataStore permission level (not just app-level restriction)
• Pre-Retrieval Clearance Filter runs on RAG and ZCQL results before they enter LLM context, filtered by caller badge ID/clearance
• Post-Generation Guardrail runs at the Response Aggregator as a second redaction pass
• Image-primary requests route to the Fast Vision Pre-Parser by default; only escalate to the full Agentic Fallback Engine when open-ended reasoning is needed
• Agentic Fallback Engine runs on Catalyst AppSail, with iteration cap of 3–4 steps and step-level timeouts
• Agentic Fallback runs a tool-calling loop (not a single-shot LLM call) with at least: web search, page context, validated read-only ZCQL, calculator/date, and vision tools
• Chat input supports image/file attachment; multimodal queries are correctly routed and handled
• All pipelines return a common response shape to the aggregator
• Audit logging is append-only/immutable and captures route, confidence, page context, ZCQL validator decisions, redaction actions at both tiers, tool calls, and final response for every query
• Existing RAG and Text2ZCQL pipelines continue to function unchanged for queries that route to them

---