Codswallop

A VSCode extension designed to help students, interns, and apprentices develop genuine programming skills by identifying over-reliance on AI-generated code and reinforcing understanding through targeted comprehension quizzes.

The Problem: Stunted Engineering Skills

The rise of AI coding assistants has created a new challenge in software education. While these tools can accelerate development, they can also lead to vibe coding - the practice of blindly accepting AI-generated code without understanding how or why it works.

This creates a dangerous skills gap:

• Surface-level knowledge: Learners can produce working code but cannot explain, debug, or modify it
• Fragile foundations: Without understanding fundamentals, learners struggle when AI suggestions fail or are incorrect
• Interview unpreparedness: Technical interviews expose comprehension gaps that AI assistants masked during coursework
• Workplace challenges: Junior developers who cannot reason about code become bottlenecks rather than contributors

Codswallop addresses this by detecting potential vibe-coded sections and ensuring learners genuinely understand the code they're committing.

How It Works

Codswallop monitors coding patterns to identify when code may have been added without full comprehension. When suspicious patterns are detected, it generates context-aware quizzes ("vibechecks") using Claude AI to verify understanding. Educators can monitor progress through a real-time dashboard.

Detection Indicators

The extension monitors three key patterns:

Indicator	Threshold	What It Catches
Line Spike	>50 lines/min	Rapid code additions suggesting copy-paste from AI or external sources
High Complexity	>20 cyclomatic	Complex functions that may have been accepted without understanding branching logic
Paste Detection	>10 lines	Direct paste operations of substantial code blocks

When any threshold is exceeded, a clickable CodeLens appears above the code offering a comprehension quiz.

AI-Powered Comprehension Quizzes

Quizzes are generated by Claude AI and tailored to the detection type:

Detection Type	Quiz Focus Areas
Line Spike	Control flow, edge cases, variable scope, function purpose
High Complexity	Branching logic, loop conditions, error handling paths, state management
Paste Detection	Overall logic, data flow, API contracts, integration points

Pass threshold: 75% - Learners must demonstrate genuine understanding to pass.

Real-Time Educator Dashboard

A Next.js dashboard provides educators with:

• Live activity feed: See vibecheck attempts as they happen
• Student leaderboard: Track comprehension scores across the cohort
• Analytics and trends: Identify common struggle areas and improvement over time
• Classroom management: Create classrooms with join codes for easy onboarding

Animated Companion Character

An animated detective-style blob companion ("Vibe") reacts to coding activity:

• Idle: Gently floating with subtle animations
• Watching: Alert eyes tracking code changes
• Detecting: Suspicious squint with magnifying glass when patterns detected
• Questioning: Thoughtful pose during quizzes
• Celebrating: Star eyes and confetti when quizzes are passed
• Concerned: Encouraging expression when more practice is needed
• Sleeping: Zzz floating after extended inactivity

Installation

From VS Code Marketplace

1. Open VS Code
2. Go to Extensions (Ctrl+Shift+X / Cmd+Shift+X)
3. Search for "Codswallop"
4. Click Install

From VSIX Package

cd extension
npm install
npm run package

This creates codswallop-0.1.8.vsix which can be installed via:

• Command Palette: Extensions: Install from VSIX...

Development Mode

cd extension
npm install
npm run compile

Press F5 in VS Code to launch the extension in debug mode.

Command Palette Usage Guide

Access all commands via the Command Palette (Ctrl+Shift+P / Cmd+Shift+P):

Core Commands

Command	Description
Codswallop: Start Vibecheck	Manually trigger a comprehension quiz for the current file. Useful when you want to test your understanding proactively rather than waiting for automatic detection.
Codswallop: Show Detections	Display a list of all active detections in your workspace. Shows which files and lines have triggered vibe detection, allowing you to address them systematically.
Codswallop: Show Vibe Character	Open the animated companion panel. The character provides visual feedback on your coding activity and quiz performance.

Authentication Commands

Command	Description
Codswallop: Login	Authenticate with your educator dashboard account. Required to sync your progress and join classrooms. Opens a browser window for secure OAuth authentication.
Codswallop: Logout	Sign out of your current session. Your local progress will no longer sync to the dashboard until you log in again.
Codswallop: Show Auth Status	Check your current authentication state. Displays whether you're logged in, your username, and connected classroom information.

Classroom Commands

Command	Description
Codswallop: Join Classroom	Enter a 6-character classroom code provided by your educator. Once joined, your vibecheck results sync to the classroom dashboard for progress tracking.
Codswallop: Open Dashboard	Open the educator dashboard in your default browser. Students can view their personal progress; educators can monitor their classrooms.

Configuration Commands

Command	Description
Codswallop: Set Anthropic API Key	Configure your personal Anthropic API key for quiz generation. Only required if not using the hosted service or if you prefer local generation.
Codswallop: Clear Anthropic API Key	Remove your stored API key. Useful when switching between personal and hosted quiz generation.

Typical Workflow

For Students/Interns/Apprentices:

1. Run Codswallop: Login to authenticate
2. Run Codswallop: Join Classroom with the code from your educator
3. Code as normal - vibechecks trigger automatically when patterns are detected
4. Click the CodeLens prompt or run Codswallop: Start Vibecheck to take quizzes
5. Use Codswallop: Show Vibe Character to see your companion's reactions

For Educators:

1. Create a classroom via the dashboard at https://codswallop-m6n9.vercel.app
2. Share the 6-character join code with your students
3. Monitor progress via Codswallop: Open Dashboard

Configuration

Configure via VS Code Settings (Ctrl+, / Cmd+,):

Detection Thresholds

Setting	Default	Description
codswallop.enabled	true	Enable or disable vibe detection
codswallop.thresholds.complexity	20	Cyclomatic complexity threshold - lower values catch simpler functions
codswallop.thresholds.lineSpike	50	Lines per minute threshold - lower values are more sensitive
codswallop.thresholds.pasteLines	10	Pasted lines threshold - lower values catch smaller pastes
codswallop.debounceWindow	60000	Cooldown in milliseconds before re-triggering at the same location

Service Configuration

Setting	Default	Description
codswallop.convexUrl	https://shocking-peacock-740.convex.cloud	Convex deployment URL for data sync
codswallop.dashboardUrl	https://codswallop-m6n9.vercel.app	Dashboard URL for authentication
codswallop.mcpServerUrl	https://codswallop-m6n9.vercel.app/api/mcp	Hosted MCP server URL for quiz generation

Dashboard Setup (For Educators)

cd dashboard
npm install

# Terminal 1: Start Convex backend
npx convex dev

# Terminal 2: Start Next.js frontend
npm run dev

Visit http://localhost:3000 for the local dashboard, or use the hosted version at https://codswallop-m6n9.vercel.app.

Project Structure

codswallop/
├── extension/           # VS Code extension (TypeScript)
│   ├── src/
│   │   ├── extension.ts           # Entry point and command registration
│   │   ├── vibe-detector.ts       # Detection orchestration
│   │   ├── complexity-analyser.ts # Cyclomatic complexity analysis
│   │   ├── document-tracker.ts    # Line spike and paste detection
│   │   ├── vibecheck-panel.ts     # Quiz webview UI
│   │   ├── vibe-panel-provider.ts # Animated character panel
│   │   ├── vibe-svg-character.ts  # SVG character generation
│   │   ├── vibe-state-machine.ts  # Character state management
│   │   ├── convex-client.ts       # Dashboard data sync
│   │   ├── auth-provider.ts       # Authentication handling
│   │   └── mcp-client.ts          # Quiz generation client
│   └── media/                     # Gutter icons
├── dashboard/           # Next.js educator dashboard
│   ├── app/
│   │   ├── dashboard/             # Dashboard pages
│   │   └── extension-auth/        # OAuth callback handler
│   ├── components/                # React components
│   └── convex/                    # Convex functions and schema
├── www/                 # Marketing website
├── docs/                # Architecture documentation
└── thoughts/            # Implementation notes

Supported Languages

• TypeScript
• JavaScript
• TypeScript React (TSX)
• JavaScript React (JSX)
• Python

Requirements

• VS Code 1.85.0 or later
• Node.js 18 or later (for development)
• Anthropic API key (optional - hosted service available)
• Convex account (optional - for self-hosted dashboard)

Common Errors

MCP Connection Error

If you encounter an MCP error when attempting to generate quizzes, this typically indicates the hosted MCP server is temporarily unavailable.

Solution: Use the Command Palette (Ctrl+Shift+P / Cmd+Shift+P) and run Codswallop: Set Anthropic API Key to configure your personal API key. This allows the extension to generate quizzes directly using your own Anthropic account, bypassing the hosted service.

Development

# Extension development
cd extension
npm run watch    # Watch mode compilation

# Dashboard development
cd dashboard
npm run dev      # Development server
npm run build    # Production build

# Marketing site
cd www
npm run dev      # Development server

Why "Codswallop"?

Codswallop (British slang for "nonsense") captures the essence of what we're detecting: code that looks legitimate but represents hollow understanding. The name serves as a gentle reminder that accepting code without comprehension is, well, codswallop.

Licence

See LICENCE file.