README.md

noot

Test interactive CLIs. Think Stagehand, but for the terminal.

• f.step("Select 'pyright' from dropdown"): Define CLI interactions in plain English.
• f.expect("Linting options must contain 'pyright'"): Define expected CLI states in plain English too.
• assert "pyright" in f.screen().lower(): Or make assertions on CLI state in good ol' Python.
• Record with LLM once, replay locally and in CI/CD.

Installation

pip install noot

Requires tmux and an ANTHROPIC_API_KEY environment variable.

Quick Start

Scaffold a new project:

noot init

Or add noot to an existing project:

from noot import Flow

def test_create_web_project():
    with Flow.spawn('python setup_wizard.py') as f:
        f.expect('Welcome to Project Setup Wizard')

        f.step("Enter project name 'mywebapp' and press enter")

        # `expect` parses assertions from natural language
        f.expect('Web Application project option is available')

        f.step('Press enter to select Web Application')

        # or specify assertions on screen state directly
        assert "author name" in f.screen()

        f.step("Enter author name 'Alice' and press enter")

Run your tests:

pytest tests/test_cli.py

The first run records LLM responses to the cassette file. Subsequent runs replay from the cassette, so no API calls are made.

API

Method	Description
Flow.spawn(cmd)	Context manager. Start a CLI process in a managed terminal session
f.step(instruction)	Execute a natural language instruction (e.g., "Press enter", "Type 'hello'")
f.expect(condition)	Assert the screen matches a natural language condition
f.screen()	Return the current terminal output as a string

Recording Modes

Control recording behavior with the RECORD_MODE environment variable:

RECORD_MODE	Behavior
once	(Default) Record if cassette is missing, replay if it exists.
none	Replay only. Fails if a request isn't cached. Use this in CI.
all	Always re-record, overwriting existing cassettes.

By default you don't have to think about recording and replay:

pytest tests/test_cli.py
# Subsequent runs will use cache

Example - force re-recording:

RECORD_MODE=all pytest tests/test_cli.py

Example - CI mode (fail if cassette is missing):

RECORD_MODE=none pytest tests/test_cli.py

Cassettes are stored in <project_root>/.cassettes/:

• CLI cassettes (LLM responses): .cassettes/cli/
• HTTP cassettes (API recordings): .cassettes/http/

CI/CD

Run tests in CI with RECORD_MODE=none to replay from cached cassettes (no API key needed):

# .github/workflows/test.yml
name: Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install uv
        uses: astral-sh/setup-uv@v5

      - name: Install dependencies
        run: uv sync

      - name: Run tests (replay mode)
        run: uv run pytest tests/ -v -s
        env:
          RECORD_MODE: none

Key points:

• RECORD_MODE: none ensures tests replay from cassettes and fail if any recording is missing
• -v -s flags provide verbose output for easier debugging in CI logs
• No ANTHROPIC_API_KEY needed in replay mode—cassettes contain all recorded responses

Commit your .cassettes/ directory to version control so CI can replay recordings.

Troubleshooting

"ANTHROPIC_API_KEY environment variable required"

• You're running in record mode without an API key. Either set ANTHROPIC_API_KEY or use RECORD_MODE=none to replay from existing cassettes.

"Cache miss in replay mode"

• A test is making an LLM call that wasn't recorded. Run locally with RECORD_MODE=once (or all) to record the missing interaction, then commit the updated cassette.

Cassette not found

• Ensure .cassettes/ is committed to version control and not in .gitignore.

Contributing

Issues and PRs welcome.

License

Apache 2.0