Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .claude/commands/github-review-comments.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ For each unresolved comment, read the full body and categorise it:
2. Check if the suggestion is technically correct for THIS codebase
3. Check if it would break existing functionality
4. Check if existing patterns/conventions contradict the suggestion
5. Check CLAUDE.md rules -- project conventions override reviewer preferences
5. Check AGENTS.md rules -- project conventions override reviewer preferences

---

Expand Down Expand Up @@ -220,7 +220,7 @@ When pushing back:

- Use technical reasoning grounded in the actual codebase
- Reference existing patterns if the suggestion contradicts them
- Reference CLAUDE.md rules when applicable
- Reference AGENTS.md rules when applicable
- Explain what would break or what edge case the reviewer missed
- If the suggestion is valid in principle but wrong for this context, say so

Expand All @@ -231,7 +231,7 @@ When pushing back:
- Always read the actual code before evaluating a comment -- reviewers sometimes misread diffs
- If a comment reveals a genuine bug you missed, fix it without defensiveness
- If multiple comments suggest the same change, implement it once and reference the fix in all replies
- Bot reviewers (CodeRabbit, etc.) sometimes suggest changes that conflict with project conventions -- verify against CLAUDE.md
- Bot reviewers (CodeRabbit, etc.) sometimes suggest changes that conflict with project conventions -- verify against AGENTS.md
- If a new round of review comments appears after your push (from re-review), report that to the user rather than entering an infinite loop

Now begin by determining the PR number from `$ARGUMENTS` or the current branch.
4 changes: 2 additions & 2 deletions .claude/commands/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Protect this session's context: delegate mechanical exploration to cheaper subag

1. Fan out Explore agents (`model: haiku`) for file discovery and naming-convention sweeps; use `model: sonnet` agents when a subsystem needs to be read and summarized. Launch independent explorations in parallel.
2. Read the load-bearing files yourself — the ones the design decision actually hinges on. Don't design from subagent summaries alone.
3. Check the architecture layers in `CLAUDE.md` and read the matching source files — past decisions and gotchas live there.
3. Check the architecture layers in `AGENTS.md` and read the matching source files — past decisions and gotchas live there.
4. Check `git log` for recent related work; the design should extend it, not fight it.

## Phase 2 — Surface the unknowns (blindspot pass + interview)
Expand All @@ -41,7 +41,7 @@ Investigation tells you what the codebase says; this phase finds what the REQUES
- edge cases the codebase makes possible that the request never mentions
- anything with no precedent in this repo — flag it explicitly as unknown-unknown territory
2. **Interview the user** with AskUserQuestion, one question at a time, prioritized by blast radius: architecture-changing answers first, then public API / config surface, then UX. Rules:
- Skip anything the codebase, CLAUDE.md, or an existing issue already answers.
- Skip anything the codebase, AGENTS.md, or an existing issue already answers.
- 2–5 questions is the sweet spot; zero is fine when the request is genuinely unambiguous — say so rather than inventing questions.
- Every question offers concrete options with a recommended default, never an open-ended essay prompt.
3. **Record the answers** in the plan's Decision section as `Settled in interview:` bullets — constraints the executor must not re-litigate.
Expand Down
50 changes: 50 additions & 0 deletions .rtk/filters.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Project-local RTK filters (https://github.com/rtk-ai/rtk#custom-filters).
#
# The `rtk hook claude` PreToolUse hook rewrites a matching command to
# `rtk <cmd>`; rtk runs it and drops the lines listed here before the agent
# sees the output. Filters are line-based: they strip noise, never reformat.
#
# `match_command` is matched against the command with the first word reduced
# to its basename, so `bin/validate_all`, `./bin/validate_all` and
# `validate_all` all hit `^validate_all\b`.
#
# Trust is a SHA-256 of this file: after every edit run `rtk trust --yes`,
# then `rtk verify` to run the inline tests.
schema_version = 1

[filters.rake-build]
description = "bundle exec rake build (CI's 'Verify gem builds' step): drop the packaged-file listing, keep the gem metadata and unpack confirmation"
match_command = "^(?:bundle\\s+exec\\s+)?rake\\s+build\\b"
strip_lines_matching = [
"^\\s*$",
"^/tmp/gem-verify/",
]
on_empty = "rake build: no output left after filtering (check the exit code)"

[[tests.rake-build]]
name = "keeps gem metadata, unpack confirmation and shell echoes; drops blank lines and the packaged-file listing"
input = """gem build pgbus.gemspec --strict
Successfully built RubyGem
Name: pgbus
Version: 0.16.7
File: pgbus-0.16.7.gem
gem unpack pgbus-0.16.7.gem --target /tmp/gem-verify
Unpacked gem: '/tmp/gem-verify/pgbus-0.16.7'
find /tmp/gem-verify -type f | sort

=== Gem contents ===
/tmp/gem-verify/pgbus-0.16.7/lib/pgbus.rb
/tmp/gem-verify/pgbus-0.16.7/lib/pgbus/version.rb
/tmp/gem-verify/pgbus-0.16.7/README.md
rm -rf /tmp/gem-verify pgbus-0.16.7.gem
"""
expected = """gem build pgbus.gemspec --strict
Successfully built RubyGem
Name: pgbus
Version: 0.16.7
File: pgbus-0.16.7.gem
gem unpack pgbus-0.16.7.gem --target /tmp/gem-verify
Unpacked gem: '/tmp/gem-verify/pgbus-0.16.7'
find /tmp/gem-verify -type f | sort
=== Gem contents ===
rm -rf /tmp/gem-verify pgbus-0.16.7.gem"""
200 changes: 200 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
# Pgbus

PostgreSQL-native job processing and event bus for Rails, built on PGMQ.

Project instructions for every agent: Claude Code (`CLAUDE.md` imports this file); Grok, Cursor,
Copilot and Codex read this file directly. Claude-only material (slash commands, rules) lives
under `.claude/`.

## Tech Stack

- **Ruby**: >= 3.3 | **Rails**: >= 7.1
- **Transport**: pgmq-ruby (PGMQ — extension or embedded SQL)
- **Concurrency**: concurrent-ruby
- **Autoloading**: zeitwerk
- **Testing**: RSpec
- **Linting**: RuboCop

## Critical Rules

### Never Do
1. **NO direct PGMQ calls** — always go through `Pgbus::Client`
2. **NO hardcoded queue names** — use `config.queue_name()`
3. **NO raw SQL in dashboard** — use `Web::DataSource`
4. **NO `Marshal.load`** — JSON serialization only
5. **NO unsynchronized shared state** — use Mutex or Concurrent primitives
6. **NO swallowing errors** — log via `Pgbus.logger`, track in `pgbus_failed_events`
7. **NO `Record` suffix on model classes** — see Model Naming below

### Always Do
1. **TDD**: Write tests BEFORE implementation
2. **Worker recycling**: Configure `max_jobs`, `max_memory_mb`, `max_lifetime`
3. **Dead letter routing**: Check `read_ct` > `max_retries`
4. **LISTEN/NOTIFY**: Use `enable_notify_insert` for instant wake-up
5. **Queue prefix**: All queues through `config.queue_name()`
6. **Visibility timeout**: Always pass `vt:` parameter on reads
7. **Performance**: Measure before/after for hot-path changes (`/perf`); see `docs/performance.md`

## Commands

```bash
bundle exec rspec # Run tests
bundle exec rubocop # Lint
bundle exec rake # Both
bundle exec rake bench # Unit benchmarks (serialization, client, executor)
bundle exec rake bench:one[client_bench] # Single benchmark by name
bundle exec rake bench:memory # Detailed memory profiling
bundle exec rake bench:integration # Real DB benchmarks (requires PGBUS_DATABASE_URL)
bundle exec rake bench:streams # SSE streaming benchmarks (requires PGBUS_DATABASE_URL)
bundle exec rake frontend:css # Rebuild app/frontend/pgbus/style.css after adding a Tailwind class to a view
bin/release list # Last releases + next patch/minor/major version
bin/release [minor|major|X.Y.Z] [-n] # Cut a release (patch by default) via rake release; -n = dry run
```

Command output is condensed by rtk (PreToolUse hook). `.rtk/filters.toml` covers this repo's
scripts (`rake build`); every edit to it needs `rtk trust --yes` + `rtk verify`. Write commands in
hook-rewritable shapes: no `for`/subshell wrappers, no `| head` on rtk-handled commands,
`bundle exec rubocop` not `bin/rubocop`.

## Slash Commands

| Command | Purpose |
|---------|---------|
| `/lfg` | Full autonomous workflow: branch → understand → explore → plan → TDD → verify → PR |
| `/plan` | Fable-powered planning → GitHub issue or `docs/plans/` markdown (read-only; execute with `/lfg`) |
| `/github-review-comments` | Process unresolved PR review comments |
| `/review-pr` | Review a PR for pattern compliance |
| `/tdd` | Enforce RED → GREEN → REFACTOR cycle |
| `/security` | Security audit (PGMQ ops, connections, auth, deserialization) |
| `/architect` | Coordinate multi-layer development |
| `/perf` | Benchmark current branch against main (before/after with worktree) |

Commands and agents pin a model tier via frontmatter aliases: `sonnet` for pattern-following implementation (the default), `opus` for orchestration, security, and full PR review, `fable` for read-only planning (`/plan`). Use aliases, not full model IDs, so commands track the latest model in each tier. When spawning subagents for mechanical work (file finding, pattern scans), pass a cheaper model explicitly rather than letting them inherit the session model.

## Architecture

```
Layer 6: Dashboard app/controllers/pgbus/, app/views/pgbus/
Layer 5: CLI lib/pgbus/cli.rb
Layer 4: Process Model lib/pgbus/process/ (supervisor, worker, dispatcher, consumer)
Execution Pools lib/pgbus/execution_pools/ (thread_pool, async_pool)
Layer 3: Event Bus lib/pgbus/event_bus/ (publisher, subscriber, registry, handler)
Layer 2: ActiveJob lib/pgbus/active_job/ (adapter, executor)
Layer 1: Client lib/pgbus/client.rb (PGMQ wrapper)
Layer 0: Config lib/pgbus/configuration.rb, config_loader.rb
```

## Model Naming

ActiveRecord models live in `app/models/pgbus/` and inherit from `Pgbus::ApplicationRecord`.
**Never use a `Record` suffix.** Resolve naming conflicts as follows:

| Model Class | Table | Why not the obvious name |
|---|---|---|
| `Pgbus::BusRecord` | (abstract) | Base class in `lib/pgbus/` — loaded by Zeitwerk gem loader, avoids engine boot-order issues |
| `Pgbus::ApplicationRecord` | (abstract) | Backward-compatible alias for `BusRecord` in `app/models/` |
| `Pgbus::BatchEntry` | `pgbus_batches` | `Pgbus::Batch` is the batch API class |
| `Pgbus::BlockedExecution` | `pgbus_blocked_executions` | — |
| `Pgbus::ProcessEntry` | `pgbus_processes` | `Process` conflicts with Ruby's `Process` module |
| `Pgbus::ProcessedEvent` | `pgbus_processed_events` | — |
| `Pgbus::RecurringExecution` | `pgbus_recurring_executions` | — |
| `Pgbus::RecurringTask` | `pgbus_recurring_tasks` | `Pgbus::Recurring::Task` is a different namespace |
| `Pgbus::Semaphore` | `pgbus_semaphores` | `Pgbus::Concurrency::Semaphore` is a different namespace |

When a model name collides with a service/module name, prefer `Entry` suffix or a descriptive alternative over `Record`.

## Separate Database Support

Pgbus supports running in the primary database or a dedicated database (like SolidQueue).

**Configuration** (`config.connects_to`):
- `nil` (default) — uses the primary Rails database
- `{ database: { writing: :pgbus } }` — uses a separate database

**Generator flags**:
- `rails generate pgbus:install --database=pgbus` — migrations go to `db/pgbus_migrate/`
- `rails generate pgbus:add_recurring --database=pgbus` — recurring migrations also go to `db/pgbus_migrate/`
- `rails generate pgbus:upgrade_pgmq --database=pgbus` — upgrade migrations also go to `db/pgbus_migrate/`
- `rails generate pgbus:tune_fillfactor --database=pgbus` — fillfactor tuning for existing installations
- Without `--database` — migrations go to `db/migrate/` (default)

**database.yml example**:
```yaml
production:
primary:
<<: *default
database: myapp_production
pgbus:
<<: *default
database: myapp_pgbus_production
migrations_paths: db/pgbus_migrate
```

## Key Design Decisions

- Worker recycling via `max_jobs_per_worker`, `max_memory_mb`, `max_worker_lifetime` — fixes solid_queue's memory leak problem
- LISTEN/NOTIFY via PGMQ's `enable_notify_insert` for instant wake-up (polling as fallback only)
- Dead letter queues: after `max_retries` failed reads (tracked by PGMQ's `read_ct`), move to `_dlq` queue
- Idempotent events: `pgbus_processed_events` table with (event_id, handler_class) unique index
- Dashboard via Tailwind CDN + Turbo CDN — zero npm dependency
- PGMQ schema install: extension-first with embedded SQL fallback (`pgmq_schema_mode: :auto | :extension | :embedded`)
- Fillfactor=70 on queue tables: reserves 30% page space to reduce page density during PGMQ's heavy read UPDATE churn
- Proactive table maintenance: dispatcher periodically checks pg_stat_user_tables for bloated tables and vacuums them (inspired by pgque)

## PGMQ Schema Management

PGMQ can be installed via PostgreSQL extension or embedded SQL (no extension required).

**Configuration** (`config.pgmq_schema_mode`):
- `:auto` (default) — tries extension, falls back to embedded SQL
- `:extension` — requires the pgmq PostgreSQL extension
- `:embedded` — uses vendored SQL, no extension needed

**Generators**:
- `rails generate pgbus:install --pgmq-schema-mode=auto` — initial setup
- `rails generate pgbus:upgrade_pgmq` — upgrade PGMQ schema to latest vendored version

**Rake tasks**:
- `rake pgbus:pgmq:status` — show installed vs available PGMQ version
- `rake pgbus:pgmq:versions` — list vendored PGMQ versions

**Key files**: `lib/pgbus/pgmq_schema.rb`, `lib/pgbus/pgmq_schema/pgmq_v*.sql`

## Queue Naming

All PGMQ queues are prefixed: `{queue_prefix}_{name}` (default: `pgbus_default`).
DLQ queues append `_dlq` suffix.

## Screenshots on PRs and issues (always)

The dashboard (`app/controllers/pgbus/`, `app/views/pgbus/*.html.erb`, `app/frontend/pgbus/`) is
a real UI — Tailwind + Turbo + ApexCharts. Any change to a dashboard view, its CSS, or its JS
ships with before/after pictures **on the PR**, attached from the terminal. Never a local path, a
base64 blob, or "screenshot available on request".

```bash
gh pr create --attach './after.png#Queue detail page with retry counts' --title … --body … # picture in hand already
gh pr comment <n> --attach './after.png#Queue detail page with retry counts' --body 'Before/after for the queue detail page.'
gh pr comment <n> --attach ./before.png --attach ./after.png # repeat the flag, up to 50 files
gh issue comment <n> --attach ./repro.mp4 # video renders as a player
```

- Quote the whole argument: the alt text has spaces and bare `<`/`>` would redirect. `<file>#<alt text>`
sets the alt text; without it the filename is used. A body that already
references the file (`![alt](./after.png)`) gets that reference rewritten to the uploaded
asset, so images can sit inline; unreferenced attachments are appended at the end.
- `create`, `edit` and `comment` all take `--attach` (all three landed in gh 2.99). Attach at create time when
the picture already exists; comment when it comes later, as it does after a verification run
against `rake dummy:server`.
- Capture with the tool already in hand: `agent-browser screenshot <file>` or the Playwright MCP
`browser_take_screenshot`. Save under the scratchpad, never in the repo.
- No `--attach` flag means an old `gh`: `brew upgrade gh`.

## More Documentation

See `.claude/` directory:
- `commands/` — Slash command definitions (including `/perf` for benchmarking)
- `rules/` — Coding style, git workflow, testing, agents, performance, security

See `docs/` directory:
- `docs/performance.md` — Hot paths, measuring guide, allocation budgets, CI integration
Loading
Loading