A comprehensive AI-powered meeting intelligence system that converts video/audio meetings into searchable, actionable insights with video clip retrieval.
- Problem: Research papers use visual change detection, but meeting videos are static (talking heads)
- Solution: Audio-aware padding (start-2s to end+2s) prevents mid-word audio cutoff
- Implementation: FFmpeg-based clipping with smart padding heuristics
- Problem: Standard chunking loses context (e.g., "he said it's too high" without "budget" reference)
- Solution: Prepend metadata context
[Time: 00:01:23-00:02:45 | Speakers: Alice, Bob]to embeddings - Result: Queries like "what did they decide about the budget?" now match correctly
- Problem: LLMs hallucinate speaker attribution in long contexts
- Solution: Structured prompts with explicit speaker ID requirements
- Result: Action items correctly attributed to speakers
- Problem: Plain text responses are hard to scan and distinguish important information
- Solution: LLM uses markdown formatting; frontend renders rich text with highlighting
- Features:
- Bold for important terms, decisions, metrics (e.g.,
**$500K budget**) - Headers (# ## ###) for organization and hierarchy
- Bullet lists and numbered lists for clarity
Inline codefor technical terms- Automatic keyword highlighting (decisions, actions, deadlines, etc.)
- Proper spacing and visual hierarchy
- Bold for important terms, decisions, metrics (e.g.,
- ASR: WhisperX (CTranslate2-accelerated Whisper) with word-level alignment
- Speaker Diarization: pyannote.audio (speaker-diarization-3.1)
- Embeddings: sentence-transformers (all-MiniLM-L6-v2, 384-dim)
- Vector Search: FAISS (IndexFlatIP, cosine similarity)
- LLM: Anthropic Claude / OpenAI GPT / OpenRouter (configurable)
- Orchestration: LangChain + LangGraph
- Framework: FastAPI (async, high performance)
- Server: Uvicorn (ASGI)
- Language: Python 3.11+
- Video Processing: FFmpeg (audio extraction + video clipping)
- Vector DB: FAISS (persisted to disk, lazy-loaded)
- Chat Storage: SQLite (persistent chat history)
- Config: Pydantic Settings (
.envdriven) - Real-time: WebSocket (pipeline progress) + SSE (streaming chat)
- UI: React (Single Page Application)
- Styling: Tailwind CSS
- Communication: REST API + SSE streaming + WebSocket
- Components: UploadSection, JobList, JobView, PipelineBar, ChatTab, ClipCard
- Build: Create React App, served by FastAPI static mount
- Package Management: UV (fast, reliable)
- Environment: Virtual environments (.venv)
- Cross-platform: pathlib, subprocess (works on Windows/macOS/Linux)
- Python 3.11+
- FFmpeg (for video processing)
- UV package manager (recommended)
# Clone repository
git clone https://github.com/Pandharimaske/Meeting_Intelligence_Platform.git
cd Meeting_Intelligence_Platform
# Install dependencies (using UV - recommended)
uv sync
# Or using pip
pip install -e .# Copy and edit config
cp .env.example .env
# Edit .env with your API keys (optional)# Option 1: One-click startup (builds frontend + starts server)
bash scripts/start.sh
# Option 2: Manual start
uv run uvicorn backend.routes:app --host 0.0.0.0 --port 8000
# Open in browser
open http://localhost:8000# Upload a video
curl -X POST "http://localhost:8000/api/v1/upload" \
-F "file=@meeting.mp4"
# Ask a question and get AI-powered answer
curl -X POST "http://localhost:8000/api/v1/jobs/{job_id}/chat" \
-H "Content-Type: application/json" \
-d '{"question": "What were the key decisions?"}'
# Full API docs available at: http://localhost:8000/docsClip keywords: show, clip, play, video, segment, watch, see, display, footage, recording, playback, zoom, visual, etc.
Use case: Visual context, speaker emphasis, exact wording verification
UX Benefit: Clips appear inline in chat flow (not in separate panel) → keeps conversation context in one view
Example response to "What are the decisions and action items?"
## 📋 Key Decisions
- **Decision:** Approved **$500K budget** for Q2 marketing [00:15:30]
- **Owner:** Sarah Chen (Marketing Lead)
## ✅ Action Items
1. **Owner:** John Smith - Set up customer feedback survey by **Friday 3/15** [00:18:45]
2. **Owner:** Lisa Wong - Schedule engineering review for timeline impact [00:19:20]
3. **Owner:** Ahmed Patel - Update stakeholders on budget allocation [00:21:05]
## 📝 Important Notes
- Timeline is **critical path** for Q2 launch
- Customer feedback will drive feature prioritization
- Follow-up meeting scheduled for **Monday 9am**
Formatting Features:
- ✅ Bold highlighting for important terms and metrics
- ✅ Headers for section organization (## ### ####)
- ✅ Bullet lists for key points
- ✅ Numbered lists for sequential action items
- ✅ Automatic keyword highlighting of: decisions, actions, deadlines, owners, critical terms
- ✅ Inline code for technical terms and product names
- ✅ Proper spacing for visual hierarchy and readability
Meeting_Intelligence_Platform/
├── backend/
│ ├── routes.py # FastAPI app, REST/WebSocket/SSE endpoints
│ ├── chat_db.py # SQLite chat history persistence
│ └── core/
│ └── settings.py # Pydantic Settings (.env config)
├── processing/
│ ├── audio/
│ │ ├── extractor.py # FFmpeg audio extraction
│ │ └── transcription/
│ │ └── converter.py # WhisperX transcription + diarization
│ ├── text/
│ │ ├── parser.py # SRT/VTT parser
│ │ ├── chunking/
│ │ │ └── chunker.py # Semantic chunking with metadata
│ │ └── diarization/
│ │ └── speaker_diarization.py # Standalone speaker ID
│ ├── vector/
│ │ └── store.py # FAISS vector store (build/search/persist)
│ ├── reports/
│ │ ├── mom_generator.py # Template-based MoM
│ │ └── rag_mom_generator.py # RAG-based MoM + chat engine
│ └── video/
│ └── clipper.py # FFmpeg video clipping
├── frontend/
│ ├── src/
│ │ ├── App.js # Main React app (tabs, chat, video)
│ │ ├── components/
│ │ │ ├── UploadSection.js # Drag-and-drop upload
│ │ │ ├── JobList.js # Sidebar job listing
│ │ │ ├── JobView.js # Transcript/MoM/Chat tabs
│ │ │ └── ProgressOverlay.js # Pipeline progress bar
│ │ ├── hooks/
│ │ │ └── useWebSocket.js # WebSocket progress hook
│ │ └── utils/
│ │ └── api.js # REST + SSE API client
│ └── build/ # Production build (served by FastAPI)
├── scripts/
│ ├── run_server.py # Uvicorn launcher
│ └── start.sh # One-click startup script
├── data/ # Runtime data (not committed)
│ ├── jobs/jobs.json # Job registry
│ ├── jobs/{id}/ # Per-job: chunks, MoM, vectors, transcripts
│ ├── video/ # Uploaded videos
│ ├── audio/ # Extracted audio
│ └── clips/ # Generated video clips
├── docs/ # Documentation
├── pyproject.toml # Dependencies & metadata
└── .env # API keys & configuration
- ✅ End-to-end pipeline: Video → Transcript → MoM → Chat → Clips
- ✅ All research gaps addressed (2/4 fixed, 2 identified for future)
- ✅ Production-ready error handling and logging
- Transcription: ~10x realtime on CPU (WhisperX base model)
- MoM Generation: <30 seconds for 1-hour meeting (RAG per-section)
- Video Clipping: <5 seconds for any segment (FFmpeg stream-copy)
- Search Latency: <100ms for FAISS semantic queries
- Chat Streaming: Token-by-token SSE delivery
- Job Reload: <1 second (persisted to disk)
- ✅ Intuitive web interface
- ✅ Real-time progress updates
- ✅ Click-to-play video segments
- ✅ Timestamped citations throughout
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
# Install in development mode
uv sync --dev
# Run tests
pytest
# Format code
black .
isort .This project is licensed under the MIT License - see the LICENSE file for details.
- Research Papers: AMMGS, AutoMeet, CLIP-It! for foundational concepts
- Open Source: Whisper, pyannote.audio, FAISS, sentence-transformers
- Community: FastAPI, UV, and the broader Python ecosystem
Built with ❤️ for making meetings more productive and searchable.
Previously, refreshing the browser or restarting the server would lose all processing state, requiring users to re-upload and re-process meetings every time.
All jobs are now persisted to disk automatically:
data/jobs/
├── jobs.json # Job database (auto-updated)
├── {job_id}/
│ ├── {video|audio|transcript} # Original file
│ ├── chunks.json # Semantic chunks
│ ├── vector_store/ # FAISS index (persisted)
│ │ ├── index.faiss # Vector index
│ │ ├── chunks.pkl # Chunk metadata
│ │ └── meta.json # Model info
│ ├── transcripts/ # JSON + TXT transcripts
│ └── mom.json # Minutes of Meeting
- Auto-Save After Each Step - Jobs database (jobs.json) is saved after each pipeline step
- Vector Store Persistence - FAISS indexes are serialized to disk when building completes
- On-Demand Loading - When you open a previous job, vector store is loaded from disk automatically
- Zero Manual Action - No cache management needed; works transparently
Before:
- Upload video → wait 5 minutes processing → refresh → LOST! Start over
After:
- Upload video → wait 5 minutes → refresh → jobs still in sidebar
- Click previous job → instantly loads (from cache, no re-processing)
- ✅ First load of job: Full processing (~5-10 min for 1 hour video)
- ✅ Reload after refresh: Instant (<1 second to load from cache)
- ✅ Switching between jobs: Instant (vector store lazy-loaded from disk)
- ✅ Disk usage: ~100-300 MB per hour of video including compressed FAISS index
Built with ❤️ for making meetings more productive and searchable.