SlopBlock

A GitHub App that gates pull request merges behind a diff-grounded quiz for the PR author.

---

This is the official source code for the SlopBlock GitHub App. You can use the hosted version directly from the Marketplace, or clone this repo and self-host your own instance.

What It Does

SlopBlock adds a required status check to pull requests. Before a PR can merge, the author must answer a short multiple-choice quiz about their own diff. Questions are generated by an LLM and grounded entirely in the changes being merged.

• Asks only about the diff.
• Skips trivial and obvious pull requests automatically.
• Uses repository context only to generate better questions.
• Works with any OpenAI-compatible LLM endpoint.
• Exposes a slopblock commit status for branch protection.

How It Works

1. A pull_request webhook fires when a PR is opened or marked ready for review.
2. SlopBlock evaluates whether the diff is trivial using heuristics first, then asks the model for borderline cases.
3. If a quiz is needed, it posts a comment with a link to an authenticated answer UI.
4. The PR author answers one question at a time until the quiz is passed.
5. The slopblock commit status updates to success and the PR can merge.

Bots and fork PRs are skipped by default.

Configuration

All configuration is managed through the SlopBlock settings dashboard. After installing the app on a repository, sign in at your instance's /settings page to configure:

• LLM Provider -- Connect via OpenRouter (one-click OAuth) or enter any OpenAI-compatible base URL and API key manually. Credentials are encrypted at rest (AES-256-GCM).
• Models -- Pick separate models for quiz generation, answer validation, and skip decisions.
• Quiz Behavior -- Min/max questions, retry mode, allowed wrong answers, bot/fork skipping.
• Token Budget -- Max token spend per quiz and fallback behavior when the budget is exceeded.
• Custom Prompts -- System prompt and quiz instruction overrides (paid plans).

Settings can also be overridden per-repository by committing a .github/slopblock.yml file. Database settings take priority over the YAML file.

Stack

• SvelteKit on Vercel (serverless)
• TypeScript
• Prisma + PostgreSQL
• GitHub App auth via @octokit/app
• Any OpenAI-compatible LLM endpoint

Self-Hosting

Prerequisites

• A Vercel account (or any platform that can host SvelteKit)
• A PostgreSQL-compatible database
• A GitHub App (you create this yourself)
• An OpenAI-compatible LLM API key (e.g. OpenRouter, OpenAI, Anthropic)

1. Create a GitHub App

Set these repository permissions:

• Contents: read
• Pull requests: read/write
• Issues: read/write
• Commit statuses: write
• Metadata: read

Subscribe to webhook events:

• Pull request

2. Deploy to Vercel

Clone this repo and create a Vercel project from it.

3. Set Environment Variables

These are the infrastructure variables required by the deployment. LLM provider/model configuration is handled through the settings UI, not environment variables.

Variable	Required	Description
DATABASE_URL	Yes	PostgreSQL connection string (pooled)
DATABASE_URL_UNPOOLED	Yes	PostgreSQL connection string (direct, for migrations)
ENCRYPTION_KEY	Yes	32-byte hex string for AES-256-GCM encryption of stored API keys
GITHUB_APP_ID	Yes	Your GitHub App's ID
GITHUB_APP_PRIVATE_KEY	Yes	Your GitHub App's RSA private key
GITHUB_WEBHOOK_SECRET	Yes	Secret for verifying webhook signatures
GITHUB_CLIENT_ID	Yes	GitHub OAuth Client ID (for settings dashboard login)
GITHUB_CLIENT_SECRET	Yes	GitHub OAuth Client Secret
APP_BASE_URL	No	Public URL of your deployment (falls back to Vercel's auto-detected URL)
SESSION_SECRET	No	Session cookie signing key (falls back to GITHUB_WEBHOOK_SECRET)
LANGFUSE_SECRET_KEY	No	LangFuse tracing secret key (observability, disabled if absent)
LANGFUSE_PUBLIC_KEY	No	LangFuse tracing public key
LANGFUSE_BASE_URL	No	LangFuse endpoint (defaults to https://cloud.langfuse.com)

4. Run Migrations

pnpm install
pnpm run prisma:migrate -- --name init

5. Point Your GitHub App Webhook URL

https://<your-domain>/api/github/webhooks

6. Install and Configure

1. Install your GitHub App on a repository.
2. Go to /settings on your deployment, sign in with GitHub, and configure your LLM provider and models.
3. Open a PR to test. Watch for the slopblock status and bot comment.
4. Once satisfied, add slopblock as a required status check in your branch protection rules.

Local Development

pnpm install
pnpm run prisma:generate
pnpm run prisma:migrate -- --name init
pnpm run dev

Prisma migrations use .env.local for database credentials.

Local Prompt Harness

Build the CLI and run prompts locally against fixture data:

pnpm run build:cli
node dist/cli.cjs skip --api-key ... --base-url https://openrouter.ai/api/v1 --diff fixtures/diff.txt --files fixtures/files.txt
node dist/cli.cjs quiz --api-key ... --base-url https://openrouter.ai/api/v1 --context fixtures/context.json --questions 3

Checks

pnpm run lint
pnpm run fmt:check
pnpm run check:quality
pnpm run check

Known Limits

• Repository context is fetched via the GitHub API, so very large repos may need tighter token budgets.
• Fork PRs are skipped by default because repository secrets are not exposed there safely.
• Model defaults are role-specific: quiz generation uses anthropic/claude-sonnet-4.5, validation uses anthropic/claude-opus-4.1, and skip decisions use anthropic/claude-sonnet-4.5 unless overridden in settings.

Roadmap

• Insights over time: Show longitudinal signals so teams can see where engineers repeatedly trip up, not just whether a single PR passed or failed.
• Extra git context: Use repository history to target questions at the tricky parts of the codebase, including most-touched files, least-touched files, likely bug hotspots, and files that appear to be missing test coverage. Handy candidate commands for upstream support include churn, shortlog, bug-cluster, velocity, and revert/hotfix scans such as those described in The Git Commands I Run Before Reading Any Code.
• Cross-repo system context: Let question generation understand related repositories and shared boundaries, so diffs can be interpreted in the context of the wider system instead of a single repo in isolation.
• Coverage-aware questioning: Prefer deeper scrutiny when changed code sits in weakly tested areas, and surface that risk explicitly.

License

This project is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).

You are free to fork, modify, and self-host this software for non-commercial use, provided you give appropriate credit. Commercial use (selling the software, offering it as a paid service, etc.) is not permitted without separate written permission.