MockAI is a multimodal AI-powered web application designed to help candidates practice job interviews and receive structured, personalized performance feedback.
The platform evaluates interview responses across three modalities:
- 🧠 NLP for semantic and content analysis
- 🎙️ Speech analysis for delivery and fluency
- 👤 Facial analysis for emotion and composure
The individual results are combined through a multimodal fusion and scoring pipeline to generate a comprehensive interview performance report.
- Secure authentication
- Candidate dashboard
- Mock interview sessions
- Text-based responses
- Speech-based responses
- Video-based responses
- Dynamic interview questions
- Question difficulty levels
- Per-question evaluation
- Multimodal AI analysis
- Overall performance scoring
- Interview history
- Performance tracking
- Personalized feedback
- Detailed performance reports
- Admin authentication
- User management
- Question bank management
- Question category management
- Interview monitoring
- Performance statistics
- Interview record inspection
- Audit log monitoring
- Administrative analytics
MockAI processes candidate responses through three primary analysis pipelines.
The NLP pipeline evaluates the semantic quality and relevance of candidate answers.
Candidate Answer
↓
Text Processing
↓
Transformer Embeddings
↓
Semantic Similarity
↓
NLP Content Score
The implementation uses transformer-based embeddings and semantic similarity against curated evaluation content.
The speech pipeline evaluates communication and delivery characteristics.
Candidate Audio
↓
Speech Transcription
↓
Transcript + Acoustic Signals
↓
Delivery Analysis
↓
Speech Score
Evaluated signals include:
- Speaking rate
- Words per minute
- Pauses
- Hesitation
- Fluency
- Delivery characteristics
The vision pipeline analyzes facial expressions during video interviews.
Candidate Video
↓
Frame Processing
↓
Face Detection
↓
Facial Emotion Recognition
↓
Emotion & Composure Features
↓
Vision Score
The implementation uses pre-trained facial analysis components rather than claiming custom foundation-model training.
MockAI combines the outputs from NLP, speech, and vision analysis using weighted late fusion.
NLP Score
│
│ 50%
▼
┌─────────────┐
Speech ──►│ Fusion │◄── Vision
30% │ Engine │ 20%
└──────┬──────┘
│
▼
Final Score
│
▼
Feedback Engine
│
▼
Performance Report
| Modality | Weight |
|---|---|
| 🧠 NLP / Content | 50% |
| 🎙️ Speech / Delivery | 30% |
| 👤 Vision / Facial | 20% |
The system also handles unavailable or low-quality modalities through dynamic weight redistribution.
Interview questions are assigned difficulty multipliers.
| Difficulty | Weight |
|---|---|
| Easy | 1.0 |
| Medium | 1.25 |
| Hard | 1.5 |
The final interview score is calculated from the weighted scores of individual questions.
Candidate
│
▼
React Web Application
│
▼
REST API
│
▼
FastAPI Backend
│
┌──────────────────┼──────────────────┐
│ │ │
▼ ▼ ▼
Authentication Interview Admin
Module Module Module
│
┌──────────┼──────────┐
│ │ │
▼ ▼ ▼
Text Speech Video
│ │ │
│ ▼ ▼
│ Speech Facial
│ Analysis Analysis
│ │ │
▼ ▼ ▼
Multimodal Fusion
│
▼
Scoring Engine
│
▼
Feedback Engine
│
▼
Report Generation
│ │
▼ ▼
MongoDB Cloudinary
- React.js
- Tailwind CSS
- JavaScript
- Python
- FastAPI
- REST APIs
- JWT Authentication
- MongoDB Atlas
- Transformer-based NLP
- BERT / DistilBERT architecture
- Sentence Transformers
- Facial emotion recognition
- OpenCV
- FERPlus CNN
- Speech recognition
- Cloudinary
- Git
- GitHub
- Postman
- Docker
- Vercel
- Python Virtual Environment
MockAI/
│
├── frontend/
│ ├── src/
│ │ ├── components/
│ │ ├── pages/
│ │ ├── services/
│ │ └── ...
│ └── ...
│
├── backend/
│ ├── routes/
│ ├── services/
│ ├── models/
│ ├── utils/
│ ├── tests/
│ └── ...
│
├── README.md
└── ...
MockAI implements several application-level security mechanisms:
- JWT-based authentication
- Role-based administrative access
- Protected API routes
- Candidate ownership validation
- User-level data isolation
- Secure database access
- Controlled media storage
- Upload validation
MockAI provides performance insights at both candidate and administrative levels.
- Overall interview score
- Per-question scores
- NLP performance
- Speech performance
- Facial analysis
- Confidence indicators
- Stress indicators
- Historical performance
- Personalized improvement guidance
- User statistics
- Interview participation
- Average performance
- Score distributions
- Interview records
- Question bank statistics
- System activity logs
MockAI includes testing across the major AI and application components.
Tested areas include:
- Speech delivery analysis
- Empty audio handling
- Empty transcript handling
- Extreme speaking-rate handling
- Multimodal fusion
- Candidate ownership isolation
- Background evaluation
- Facial analysis
- NLP evaluation
- End-to-end evaluation
Functional Requirements 36 / 36
Sub-Requirements 108 / 108
Passed 108
Partial 0
Missing 0
A typical multimodal interview follows this flow:
1. Candidate starts an interview
↓
2. Interview questions are presented
↓
3. Candidate responds through text,
speech, or video
↓
4. Response data is processed
↓
5. NLP analyzes answer content
↓
6. Speech analysis evaluates delivery
↓
7. Vision analysis evaluates facial signals
↓
8. Multimodal fusion combines results
↓
9. Scoring engine calculates performance
↓
10. Feedback engine generates guidance
↓
11. Report is generated
↓
12. Results are stored and displayed
MockAI aims to:
- Provide an accessible interview practice platform
- Simulate realistic interview sessions
- Evaluate multiple communication modalities
- Provide structured AI-based feedback
- Help candidates identify performance weaknesses
- Track performance across multiple interviews
- Improve interview preparation through personalized evaluation
The project focuses on English text, speech, and video interview experiences.
The current application uses an integrated backend architecture.
For larger workloads, the AI processing pipeline could be separated into dedicated workers and services.
Users
│
▼
Load Balancer
│
▼
API Gateway
│
Interview Service
│
▼
Task Queue
│
┌─────────────┼─────────────┐
▼ ▼ ▼
NLP Worker Speech Worker Vision Worker
│ │ │
└─────────────┼─────────────┘
▼
Fusion Worker
│
▼
Report Service
│
┌───────┴───────┐
▼ ▼
MongoDB Object Storage
Potential future infrastructure includes:
- Redis
- Message queues
- Background workers
- Load balancing
- Horizontal scaling
- Centralized logging
- Monitoring
- Service isolation
MockAI is a Final Year Project developed for the BS Computer Science program at:
COMSATS University Islamabad, Lahore Campus
- Artificial Intelligence
- Natural Language Processing
- Computer Vision
- Data Science
- Web Development
- Software Engineering
Software Development & System Integration
AI / Machine Learning
Frontend ✅
Backend ✅
Authentication ✅
Candidate Platform ✅
Admin Platform ✅
NLP Evaluation ✅
Speech Evaluation ✅
Facial Analysis ✅
Multimodal Fusion ✅
Scoring Engine ✅
Feedback Engine ✅
Report Generation ✅
MongoDB Integration ✅
Security ✅
Testing ✅
Built to help candidates understand what they answer, how they communicate, and how they present themselves during an interview.