A personal research project exploring multimodal AI for detecting the Say-Do Gap in user research.
🎓 Part of my Computational Product Research learning journey
📅 February 2026
🔬 Methodology validation study
Knowledge graph of 50 synthetic user interviews revealing the Say-Do Gap: 611 nodes (2 user segments, 50 interviews, 559 behavioral cues)
Research shows that 42% of startup failures...
Research shows that 42% of startup failures are attributed to misreading market demand - building products users said they wanted during research, but refused to adopt upon launch.
One of the potential culprits? Social Desirability Bias - users smooth over negative feedback to be polite.
Someone might say: "This feature is easy to use."
But their audio reveals: "It's... [pause 3s]... easy." [frustrated tone]
Traditional text-based research misses this gap entirely.
Can we automatically detect the Say-Do Gap by analyzing audio behavioral cues using multimodal AI?
- Pauses (>2 seconds)
- Vocal hesitation ("um", "uh")
- Frustrated tone
- Confused tone
- Sentiment mismatch
Dataset: 50 synthetic user interviews (AcmeCal - a fictional camping gear rental marketplace)
Bias Injection:
- 40 Admin users (smooth experience)
- 10 End Users (friction-filled experience)
Analysis Pipeline:
- Text → Audio (OpenAI TTS)
- Audio → Behavioral Cues (Gemini 3 Pro)
- Cues → Knowledge Graph (Neo4j)
- Graph → Say-Do Consistency Score
Say-Do Consistency Scores:
| User Segment | Score | Interpretation |
|---|---|---|
| Admin Users | 83.0% | HIGH CONSISTENCY - Trustworthy feedback ✅ |
| End Users | 3.2% | LOW CONSISTENCY - Hidden problems detected |
Bias Gap: 79.8 percentage points
- End Users showed 2X more behavioral friction (18.8 vs 9.3 cues per interview)
- But expressed similar verbal sentiment to Admin users
- The audio revealed what the text concealed
| Timestamp | Quote | Behavioral Cue |
|---|---|---|
| 02:24 | "I can't tell if I selected it or not." | Frustrated tone |
| 03:06 | "Honestly, it's kind of confusing..." | Pause (4s) + vocal markers |
| 03:38 | "This is taking a while." | Frustrated tone |
Yet in verbal summaries, these users described the experience as "generally positive."
┌─────────────────┐
│ Text Scripts │
│ (50 interviews) │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Audio Files │
│ (OpenAI TTS) │
└────────┬────────┘
│
▼
┌─────────────────────────┐
│ Gemini 3 Pro │
│ (Audio-Direct Analysis) │
│ - Extract pauses │
│ - Detect vocal markers │
│ - Analyze tone │
└────────┬────────────────┘
│
▼
┌─────────────────────┐
│ Behavioral Cues │
│ (559 cues extracted)│
└────────┬────────────┘
│
▼
┌─────────────────────┐
│ Neo4j Knowledge │
│ Graph │
│ - 50 Interviews │
│ - 559 Cues │
│ - 2 User Segments │
└────────┬────────────┘
│
▼
┌─────────────────────┐
│ Say-Do Consistency │
│ Score Calculation │
└─────────────────────┘
Intelligence:
- Gemini 3 Pro (
gemini-3.0-pro) - Multimodal audio analysis - OpenAI TTS (
tts-1) - Synthetic audio generation
Infrastructure:
- Neo4j Aura - Knowledge graph storage
- Python 3.11+ - Analysis pipeline
- Google Cloud Run - Deployment (planned)
See the full methodology in action, including:
- Live Neo4j graph queries
- Say-Do Score calculation
- Behavioral cue extraction examples
- Evidence of the 79.8 point bias gap
📄 Read the full methodology paper
Detailed explanation of:
- Theoretical foundation (Social Desirability Bias)
- Experimental design
- Results analysis
- Limitations & future work
This is a methodology validation study with important limitations:
- Synthetic data only - Real human validation needed
- English language only - Cross-cultural generalization unknown
- Controlled TTS voices - Natural speech variation not tested
- Single product domain - B2B/Enterprise contexts may differ
Next step: Seeking research collaborators for academic conferences (e.g. CHI 2027 paper). Validation study across diverse populations and contexts.
Sample behavioral cue extraction:
{
"interview_id": "end_user_script_03",
"behavioral_cues": [
{
"timestamp": "01:38",
"type": "pause",
"duration": "3 seconds",
"context": "Before confirming selection"
},
{
"timestamp": "02:24",
"type": "frustrated",
"quote": "I can't tell if I selected it or not.",
"context": "Attempting to complete task"
},
{
"timestamp": "03:06",
"type": "vocal_marker",
"quote": "Um, uh, this is kind of confusing",
"context": "Navigation confusion"
}
],
"say_do_score": 3.2,
"interpretation": "LOW CONSISTENCY - Hidden problems detected"
}Seeking research collaborators for academic conferences (e.g. CHI 2027 paper).
If you're a UX researcher or product researcher or similar product roles interested in validating this methodology:
What's involved:
- Provide 1-2 real user research sessions (audio recordings)
- Receive CausalTrack analysis report
- Validate findings against your expert judgment
- Co-author credit on CHI submission (if desired)
Target: Researchers or relevant roles across diverse domains
Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)
You are free to:
- Use this methodology for academic research
- Share and adapt the approach
- Cite this work in publications
Under these terms:
- Attribution required - Credit this work appropriately
- Non-commercial - Not for commercial use without permission
For commercial licensing inquiries: kgkazakos@gmail.com
If you use this methodology in your research:
@misc{causaltrack2026,
author = {Kostas Kazakos},
title = {CausalTrack: Audio-Based Behavioral Truth Detection for User Research},
year = {2026},
month = {February},
url = {https://github.com/kgkazakos/causaltrack},
note = {Methodology validation study}
}Built with:
- Gemini 3 Pro by Google DeepMind
- Neo4j graph database
- OpenAI TTS for synthetic audio
Inspired by decades of research on Social Desirability Bias and the Say-Do Gap in behavioral science.
Personal Research Project
Not affiliated with any employer.
📧 Email: kgkazakos@gmail.com
💼 LinkedIn: www.linkedin.com/in/kazakosk/
Last Updated: February 23, 2026
Status: Methodology Validation Phase
Next Milestone: Scaled validation for academic conference