AgentKap

Autonomous shopping assistant for Kapruka.com — built for the Kapruka Agent Challenge. AgentKap combines a conversational AI concierge, live Kapruka MCP commerce tools, a multi-agent runtime, multilingual voice support, consent-based shopper memory, and an owner admin dashboard.


Live demo	https://agentkap.pasidumihiranga.me/chat
Repository	https://github.com/Pasidu-Mihiranga/AgentKap
Architecture reference	`architecture.html` (full C4-style diagrams and module inventory)
Applicant	PMERY — Pasidu Mihiranga Ilamperuma

What AgentKap does
Screenshots
Feature inventory
Prediction and sales continuity
Architecture overview
Multi-agent runtime
Application routes
Tech stack
Admin portal access
Quick start (local)
Production deployment
Environment variables
Voice and ML sidecar
Security and privacy
Database
Project structure
Scripts
Troubleshooting

What AgentKap does

AgentKap is a full-stack agentic commerce application. A shopper signs in, starts a chat mission, and receives one warm concierge voice while specialist agents coordinate behind the scenes. The assistant:

Runs a structured interview (recipient, occasion, city, budget, dietary needs) before searching Kapruka.
Calls the live Kapruka MCP for product search, detail, delivery checks, order creation, and tracking.
Renders product cards, compare studio, cart actions, and checkout handoff inside the chat.
Starts each new chat with personalised prediction chips built from past chat knowledge, memory, Sri Lankan events, live trends, and Mirofish ranking — so shoppers land on high-intent missions instead of a blank prompt.
Continues active and post-purchase chats with a Conversation Coach that predicts the next best actions (add-ons, alternatives, gift notes, delivery checks) to keep the journey moving and increase sales.
Learns preferences through a Memory Confidence Engine with explicit user consent.
Supports English, Sinhala, Tamil, and Singlish UI plus voice input/output.
Exposes an owner admin dashboard with anonymized cross-user insights and Mirofish-style confidence scoring.

The shopper sees a single assistant. The system runs a supervisor agent, tool router, event bus, task queue, and domain specialists documented in architecture.html.

Screenshots

Chat and product discovery

Multilingual UI (English / Sinhala / Tamil / Singlish) with localized question chips and product cards.

Memory and profile

Consent-based memory capture: preferences are extracted during chat, reviewed, then saved per language.

Cart

Admin owner dashboard

Role-gated `/admin` dashboard: KPIs, predictions, catalog gaps, delivery issues, and anonymized evidence.

Architecture diagrams

Exported architecture views. The canonical source with full SVG diagrams is architecture.html.

Feature inventory

Conversational shopping

Feature	Description
Streaming chat	`/api/chat` uses Vercel AI SDK `streamText` with Kapruka MCP tool calling
Supervisor prompt	Slot-based interview before search; skips filled slots from memory
Question chips	`<question>` JSON blocks rendered as clickable options; bare inline JSON stripped client-side
Product cards	Search results in carousel/fan layouts with add-to-cart and compare
Image search	Upload or capture product photo; OCR-driven Kapruka search
Compare Studio	Side-by-side product comparison with delivery, attributes, and AI verdict
Cart and checkout	Idempotent action queue; Kapruka delivery checkout handoff
Order tracking	`/track` with Kapruka reference lookup
Conversation coach	In-chat next-action chips that predict how to continue shopping and upsell after each turn
Starter missions	New-chat prediction chips ranked from past chats, memory, events, trends, and Mirofish

Memory and learning

Feature	Description
Memory Confidence Engine	Scores importance, confidence, frequency, and decay per memory kind
Consent flow	Review panel before saving; nothing persisted without explicit approval
Language-scoped memory	Separate preference ledgers for `en`, `si`, `ta`, `singlish`
Settings management	View and forget saved memories from `/settings`
Insights page	Shopper-facing summary of saved preferences at `/insights`

Voice and accessibility

Feature	Description
Voice composer	Browser Speech API where supported; ML fallback for Sinhala/Tamil
ASR sidecar	FastAPI `/asr` — Groq Whisper (production) or local faster-whisper
TTS sidecar	`/tts` with MMS-TTS models per language
Voice normalization	`/translate/voice` and question localization for Sinhala/Tamil script
Replay voice	Per-message TTS replay in chat

Multilingual

Feature	Description
i18n	English, Sinhala, Tamil, Singlish UI strings
Localized prompts	System prompt language directive per active locale
Localized question chips	Fallback maps when model emits English options in non-English UI

Admin and observability

Feature	Description
Owner dashboard	`/admin` — KPIs, predictions, recommendations, confidence levels
Mirofish config	Tunable insight categories, evidence thresholds, confidence weights
Mission Control	`/mission-control` — agent fleet and timeline events
Predictions	`/predictions` — named pipelines (purchase forecast, demand, cart abandonment, seasonal, reorder, and more)
Agent event bus	`agent_events` table for goal/step/suggestion/trust alerts
Task queue	Background `warm_starter_cache` and `generate_prediction` work decoupled via `task_queue`

Security

Feature	Description
Supabase Auth	Email/password sign-in; optional Google when Firebase vars are set
Row-level security	User-scoped tables with `auth.uid()` policies
PII masking	Message storage masks emails, phones, NIC, cards before persistence
Sensitive entity map	Encrypted mapping for redacted values
Prompt injection guard	Detects and blocks common injection patterns in chat input
Admin role gate	`checkAdminAccess` + `user_roles` table; service-role reads for aggregates only

DevOps

Feature	Description
GitHub Actions CI	Lint and production build on push/PR
GitHub Actions deploy	SCP to VPS, symlink release, systemd restart, smoke test
VPS stack	Nginx TLS, Node 22 web service, Python ML sidecar, shared ML venv
Nitro node-server	Production preset with `/ml/` reverse proxy

Prediction and sales continuity

AgentKap treats every chat as part of a sales journey, not a one-off Q&A. Two complementary systems use past behaviour, Mirofish scoring, and live commerce signals to start chats well and keep them converting.

New-chat predictions (Starter Suggestion Agent)

When a signed-in shopper opens /chat, the Starter Suggestion Agent produces four explainable mission chips instead of an empty composer:

Pipeline	What it does
Purchase Forecast	Predicts the next likely buy from past categories, recipients, and memory
Price Trend Watch	Surfaces categories worth revisiting when price/value signals matter
Demand Predictor	Bias toward categories trending on Kapruka right now
Cart Abandonment	Helps continue an open cart with a useful add-on or alternative before checkout
Seasonal Engine	Avurudu, Vesak, Christmas, Poya, and other Sri Lankan calendar nudges
Gift Matcher	Matches recipient profile + favourite categories to gift missions
Budget Optimizer	Keeps suggestions inside the shopper's usual budget band
Reorder Predictor	Suggests reorders for consumables based on history

How ranking works

Build candidates from the shopper memory ledger (past chats and saved preferences), Sri Lankan event calendar, live Kapruka trending categories, and evergreen exploration seeds.
Score with Morofish / Mirofish-style intelligence (memory fit, event proximity, trend lift, prediction-kind boost) and apply diversity so chips are not all the same category or recipient.
Optionally rewrite titles/prompts with a small LLM for warmer copy.
Cache results (starter_cache), record impressions/clicks, and persist prediction rows (predictions, horizon next_chat) for the /predictions page and learning loop.
Background task_queue jobs (warm_starter_cache, generate_prediction) keep caches warm so the next chat opens quickly.

Each chip shows why it was suggested (memory, city, budget, event, trend), so the prediction system stays explainable.

In-chat and post-chat continuation (Conversation Coach)

Once a mission is running, the Conversation Coach predicts the next best shopper actions so the assistant can keep selling without sounding pushy:

Fuses journey step, flow state (discovery → search → compare → cart → checkout → post-purchase), on-screen products, cart lines, and memory.
Blends learned click/success patterns, sequence boosts (what usually comes next historically), and Morofish event boosts from the Sri Lankan calendar.
Applies business rules and diversity, then surfaces primary/secondary chips (for example: compare these two, check same-day delivery, add a gift message, suggest an add-on, recover an abandoned cart).
Records telemetry and 2-step sequences so future predictions get better at converting.

Goal: when a chat pauses or ends, the next prediction is ready — either as coach chips in the current thread or as starter missions for the next chat — so AgentKap can continue the relationship and increase basket size and repeat purchases.

Mirofish configuration

Owner-side Mirofish config (src/lib/admin/mirofish.config.ts) tunes insight categories, evidence thresholds, and confidence weights used across admin insights and prediction confidence. Categories include catalog gaps, demand trends, UX friction, delivery issues, pricing feedback, and agent quality — so sales predictions stay grounded in anonymized shopper evidence, not just heuristics.

Shoppers can browse their recent prediction outputs on /predictions; owners see aggregated prediction and insight quality on /admin.

Architecture overview

AgentKap is a TanStack Start full-stack React 19 application with separated shopper and admin interfaces.

Shopper Browser
    -> ChatWindow / routes (client)
    -> /api/chat + createServerFn modules (server)
    -> Supabase (auth, threads, memory, cart, orders, events)
    -> Kapruka MCP (live catalog)
    -> Lovable AI Gateway / Gemini / Groq (LLM)
    -> ML sidecar /ml/ (ASR, TTS, translation, PII)

Primary runtime flow (from architecture.html):

chat.$threadId.tsx loads thread, messages, and memory context into ChatWindow.
/api/chat trims history, compresses memory, builds slot state, masks PII, selects tools.
Supervisor streams via Lovable AI Gateway (Gemini primary, Groq fallback).
Kapruka MCP tools execute search, detail, delivery, order, and track calls.
State persists to Supabase; cart side effects apply idempotently via the action queue.

Open architecture.html in a browser for full SVG diagrams: system context, multi-agent runtime, container boundaries, database groupings, and module inventory.

Multi-agent runtime

The shopper interacts with one concierge. Internally, specialized agents coordinate:

Agent	Role	Key modules
Supervisor	Plans tools, streams one public voice	`src/routes/api/chat.ts`, `chat-prompt-trim.ts`
Journey Orchestrator	Infers shopping goal and step	`journey/orchestrator.ts`
Memory Agent	Extracts, scores, decays preferences	`memory-extract.ts`, `memory-scoring.ts`, `threads.functions.ts`
Catalog Agent	Kapruka search, detail, delivery	`mcp.server.ts`, `catalog.functions.ts`
Trust Engine	Compare enrichment, verification	`compare.functions.ts`, `compare-domain.ts`
Coach Agent	In-chat next-action / upsell chips from flow, patterns, sequences, and Morofish	`coach/*.ts`
Prediction Agent	New-chat starter missions + named prediction pipelines + cache/queue	`starters.functions.ts`, `starter-candidates.ts`, `intelligence.ts`
Cart Agent	Idempotent cart/checkout actions	`action-queue.ts`, `cart-*.functions.ts`
Voice / PII Agent	ASR, TTS, translation, anonymization	`ml-service/app.py`, `src/lib/voice/`, `security/`

Coordination fabric:

agent_events — meaningful agent trace events
task_queue — decoupled background work
action-queue.ts — browser-owned cart side effects applied once per tool call

Application routes

Shopping

Route	Purpose
`/`	Redirects to `/chat`
`/chat`	Chat layout with thread sidebar
`/chat/$threadId`	Active conversation
`/browse`	Product browse
`/cart`	Cart review and checkout prep
`/orders`	Order history
`/track`	Kapruka order tracking

Account

Route	Purpose
`/auth`	Sign in / sign up (email/password)
`/forgot-password`	Password reset email
`/reset-password`	Set new password after recovery link
`/settings`	Profile, memory management
`/insights`	Shopper preference insights

Owner / admin

Route	Purpose
`/admin`	Owner dashboard (role-gated)
`/mission-control`	Agent fleet command deck
`/predictions`	Named prediction pipelines and recent next-chat forecasts

HTTP endpoints

Endpoint	Purpose
`/api/chat`	Streaming AI supervisor
`/api/public/search`	Public product search
`/ml/health`, `/ml/asr`, `/ml/tts`, …	ML sidecar (proxied in production)

Tech stack

Layer	Technology
Frontend	React 19, TanStack Router/Start, Tailwind CSS v4, Radix UI, Framer Motion, GSAP, Embla
AI	Vercel AI SDK, Lovable AI Gateway, Google Gemini, Groq fallback
Commerce	Kapruka MCP (`https://mcp.kapruka.com/mcp`)
Backend	TanStack Start server functions, Nitro node-server preset
Database	Supabase (Postgres, Auth, RLS)
Voice ML	FastAPI, faster-whisper, Groq Whisper, MMS-TTS, ffmpeg
Deploy	GitHub Actions, Ubuntu VPS, Nginx, systemd

Admin portal access

The admin dashboard is at /admin. Access requires a Supabase account with the admin role in public.user_roles.

Demo admin credentials

Field	Value
URL	https://agentkap.pasidumihiranga.me/admin
Sign-in page	https://agentkap.pasidumihiranga.me/auth?next=/admin
Email	`admin@kapruka.com`
Password	`Kapruka123##`

How login works:

Open /auth and sign in with the credentials above (normal Sign in form — no separate admin tab).
After Supabase authentication, the app calls checkAdminAccess.
If user_roles.role = 'admin', you are redirected to /admin.
Non-admin accounts attempting /admin are signed out with an authorization error.

Grant admin role manually (Supabase SQL Editor):

-- Replace with the auth.users UUID for admin@kapruka.com
insert into public.user_roles (user_id, role)
values ('YOUR_USER_UUID', 'admin')
on conflict do nothing;

The admin dashboard aggregates anonymized signals from chats, cart events, analytics, and orders. It does not expose raw emails, phone numbers, or full conversation transcripts by default.

Quick start (local)

Requirements

Tool	Version
Node.js	20 LTS or newer (22 recommended for Supabase Realtime)
npm	10+
Git	any recent
ffmpeg	required for voice (WebM decode)
uv	optional; used by `ml-service`

Install and run

git clone https://github.com/Pasidu-Mihiranga/AgentKap.git
cd AgentKap
npm ci
cp .env.example .env
# Fill in Supabase, AI keys, and optional Groq key — see Environment variables
npm run dev:all

Open http://localhost:8080/chat.

First-run walkthrough

Sign in at /auth (create a shopper account or use admin credentials above).
Click New to start a chat thread.
Ask: Find a birthday cake under LKR 5,000 for delivery to Colombo.
Answer clarifying question chips (recipient, occasion, city, budget).
Review product cards; toggle Compare on 2–4 items.
Add to cart; open /cart to review lines.
Enable the mic for voice input (requires ML sidecar on port 8000 locally).

Production deployment

Live site: https://agentkap.pasidumihiranga.me

Service	Path / port
Nginx TLS	`443` -> Node web `127.0.0.1:3010`
ML proxy	`/ml/` -> `127.0.0.1:8010`
Web systemd	`agentkap-web.service`
ML systemd	`agentkap-ml.service`

Deploy is automated via .github/workflows/deploy.yml on push to main. Manual deploy script: scripts/vps/deploy.sh. Nginx reference config: scripts/vps/nginx-agentkap.conf.

Environment variables

Copy .env.example to .env. Minimum keys for a working chat:

# Supabase
VITE_SUPABASE_URL="https://your-project.supabase.co"
VITE_SUPABASE_PUBLISHABLE_KEY="your-anon-key"
SUPABASE_SERVICE_ROLE_KEY="your-service-role-key"

# AI (at least one)
GOOGLE_GENERATIVE_AI_API_KEY="your-gemini-key"
GROQ_API_KEY="your-groq-key"

# Kapruka MCP
KAPRUKA_MCP_URL="https://mcp.kapruka.com/mcp"

# Voice (local dev)
VITE_ML_SERVICE_URL="http://127.0.0.1:8000"

# Production same-origin ML proxy
# VITE_ML_SERVICE_URL="/ml"

See .env.example for token budgets, ASR backend options, PII encryption key, and Firebase (optional Google login).

Never commit .env or service role keys to git.

Voice and ML sidecar

The Python sidecar in ml-service/ exposes:

Endpoint	Purpose
`GET /health`	Sidecar status, model cache, Groq availability
`POST /asr`	Speech-to-text (Groq Whisper in production)
`POST /tts`	Text-to-speech WAV
`POST /translate/voice`	Voice transcript normalization
`POST /translate/ui`	UI string translation
`POST /pii/analyze`	PII entity detection
`POST /pii/anonymize`	PII redaction

Local:

npm run dev:ml      # sidecar only
npm run dev:all     # web + sidecar

Production: Groq ASR is used for all languages when GROQ_API_KEY is set and AGENTKAP_SINHALA_ASR_BACKEND=groq. Nginx /ml/ proxy timeouts are set to 600s for ASR/TTS.

Full ML documentation: ml-service/README.md.

Security and privacy

Authentication: Supabase email/password; Google optional via Firebase env vars.
Authorization: RLS on user tables; admin reads via service role inside server functions only.
PII: Messages masked before storage; sensitive values encrypted in sensitive_entity_map.
Prompt injection: Input scanned; suspicious patterns get a security notice prepended to the system prompt.
Admin aggregates: Cross-user dashboard data is anonymized; evidence snippets are truncated and redacted.

Database

Schema lives in supabase/migrations/. Key table groups:

Group	Tables
Identity	`profiles`, `preferences`, `addresses`, `user_roles`
Conversation	`conversations`, `messages`, `memories`, `memory_embeddings`
Commerce	`carts`, `cart_items`, `orders`, `order_events`, `product_cache`, `category_cache`
Learning	`analytics_events`, `cart_events`, `starter_cache`, `starter_impressions`, `predictions`, `trends`, `insights`
Agent ops	`agent_events`, `agent_runs`, `task_queue`, `tool_calls`, `mission_steps`
Security	`sensitive_entity_map`, `security_audit_events`

Apply pending locale migrations in Supabase Dashboard if needed:

# Run supabase/apply-locale-migrations.sql in SQL Editor

Generated TypeScript types: src/integrations/supabase/types.ts.

Project structure

src/
  routes/                 TanStack file routes + /api/chat
  components/             chat-window, compare-studio, coach, memory panel
  lib/kapruka/            commerce, memory, coach, compare, cart, orders
  lib/admin/              owner insights + Mirofish config
  lib/voice/              browser speech, normalization, question localization
  lib/security/           PII, prompt injection, encryption
  lib/ai/                 gateway provider factory
  integrations/supabase/  client, server client, auth middleware, types
  i18n/                   language resources
ml-service/               FastAPI voice/translation/PII sidecar
supabase/migrations/      Postgres schema and RLS policies
scripts/vps/              deploy.sh, nginx-agentkap.conf
Photos/                   README screenshots and architecture exports
architecture.html         Full architecture diagrams and module map
.github/workflows/        CI and deploy pipelines

Scripts

npm run dev          # Vite dev server (:8080)
npm run dev:ml       # ML sidecar (:8000)
npm run dev:all      # web + ML together
npm run build        # production build -> .output/
npm run preview      # preview production build
npm run lint         # ESLint
npm run format       # Prettier

Troubleshooting

Symptom	Fix
Chat does not stream	Verify `GOOGLE_GENERATIVE_AI_API_KEY` or gateway key; restart dev server
Unauthorized on chat	Sign in at `/auth` first
Empty Kapruka results	MCP may throttle; wait 30s and use a narrower query
Voice 504 timeout	Ensure ML sidecar is running; production needs Groq key + nginx `/ml/` timeouts
Raw JSON in question chips	Hard refresh; client strips inline question JSON
Memory save fails (`locale` column)	Run `supabase/apply-locale-migrations.sql` in Supabase
Admin access denied	Confirm `user_roles` row with `role = 'admin'` for your user UUID
Deploy ML health timeout	VPS ML sidecar slow on 1GB RAM; deploy script retries up to 180s

Built with the official Kapruka MCP endpoint (https://mcp.kapruka.com/mcp) — no separate API key required for catalog access.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
Photos		Photos
ml-service		ml-service
public		public
scripts/vps		scripts/vps
src		src
supabase		supabase
.env.example		.env.example
.gitignore		.gitignore
.prettierignore		.prettierignore
.prettierrc		.prettierrc
README.md		README.md
apply_migrations.cjs		apply_migrations.cjs
architecture.html		architecture.html
bun.lock		bun.lock
bunfig.toml		bunfig.toml
combined_migrations.sql		combined_migrations.sql
components.json		components.json
eslint.config.js		eslint.config.js
list_models.cjs		list_models.cjs
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json
vite.config.ts		vite.config.ts

Latest commit

History

Folders and files

Repository files navigation