diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 0000000..ec7ce75 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,3 @@ +# Copilot instructions + +See [AGENTS.md](../AGENTS.md) for repository guidance. It applies here too. diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..a474e89 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,109 @@ +# AGENTS.md + +This file provides guidance to AI coding agents (Claude Code, GitHub Copilot, +and others) when working with code in this repository. `CLAUDE.md` and +`.github/copilot-instructions.md` point here so the guidance lives in one place. + +## What this gem is + +`morph-cli` gives the `morph` command, which tars up the scraper in the current +directory, uploads it to a morph.io server, runs it there and streams the +output back to the terminal. The scraper never runs locally, which is the whole +point: developers get the morph.io scraper environment without installing it. + +The audience is developers, so error messages and documentation here can assume +command-line familiarity. + +## Layout + +- `lib/morph-cli.rb` is the substance: finding the files, building the tar, + posting it, and decoding the streamed response. +- `lib/morph-cli/cli.rb` is a thin Thor wrapper handling options, the API key + prompt and turning Faraday exceptions into messages worth reading. +- `bin/morph` runs `execute` when given no arguments, so bare `morph` and + `morph execute` are the same thing. + +## Things that will catch you out + +- **`~/.morph` holds a live API key.** It is YAML with separate + `:development` and `:production` sections, written with mode 0600 and read + back with `YAML.safe_load_file(..., permitted_classes: [Symbol])`. Never echo + its contents, paste them into an issue, or use a real key in a spec or + fixture. Use an obvious placeholder. +- **The upload includes almost everything in the directory, dot-files + included.** `all_paths` prunes directories whose name starts with `.`, so + `.git` and `.bundle` are skipped, but a dot-*file* is not a directory and so + gets packed: a top-level `.env` is uploaded, as is `data.sqlite` and anything + else the person happens to have sitting there. Keep that distinction in mind + before changing `all_paths`, and don't describe the behaviour as "skips + hidden files", because it doesn't. +- **The server streams newline-delimited JSON, not plain text.** Each line has + a `stream` (`stdout`, `internalout` or `stderr`) and `text`, and `log` raises + on any other stream value. Partial chunks are buffered on the newline, so + changes there need to keep handling a chunk that splits a line. +- `create_tar` deliberately returns an open, rewound tempfile: it closes the + tar writer to flush the trailer but leaves the underlying handle usable for + the upload. The comments in that method say so; keep them true if you touch + it. +- The default request timeout is 600 seconds, overridable per environment with + `:timeout` in the config. `--dev` switches the whole thing to + `http://127.0.0.1:3000` for people working on morph.io itself. +- `scraper.rb` in the repository root is a leftover sample morph.io scraper + for a NSW council. It is tracked, but it is not gem code, nothing in `lib/` + loads it, and it is neither linted nor tested. It is there so you can run + `morph` against this directory by hand. Don't treat it as an example of the + house style. + +## Commands + + bundle exec rspec + bundle exec rubocop + bundle exec bundler-audit check --update + bundle exec rake build + +Those are the four CI jobs; the test job runs on Ruby 3.2, 3.3 and 3.4 and the +rest on 3.2. `.ruby-version` pins 3.2.2 locally. + +**`bundle exec rake` on its own fails here** with "Don't know how to build task +'default'". The Rakefile is nothing but `require "bundler/gem_tasks"`, so there +is no default task and no `rake spec`. Run `rspec` directly. + +`spec/spec_helper.rb` sets SimpleCov `minimum_coverage 90`, so the suite fails +on a coverage regression even when every example passes. It also sets +`disable_monkey_patching!`, so use `RSpec.describe` and the `expect` syntax, +and it loads `webmock/rspec` so specs don't reach the network. + +`.rubocop.yml` inherits `.rubocop_todo.yml`, which is an existing backlog of +accepted offences. Fix them or leave them, but don't regenerate that file +wholesale as part of an unrelated change. + +`Gemfile.lock` is gitignored and untracked. `bundle install` changes it +locally, which is expected. Never force-add it. + +## Releasing + +Don't run `bundle exec rake release`. It exists only because of +`bundler/gem_tasks` and would tag and publish from your machine. Releases are +automated: a version bump in `lib/morph-cli/version.rb` merged to `main` +publishes the gem through RubyGems trusted publishing in the `rubygems` +environment. Follow the README's "Releasing a new version" section. + +## Org-level guidance + +Workflow, branch naming, commit sign-off, AI disclosure and review conventions +are org-wide and deliberately not restated here, so there is one copy to keep +current rather than a stale paraphrase in every repository. They live in +`openaustralia/.github`: + +- [CONTRIBUTING.md](https://github.com/openaustralia/.github/blob/main/.github/CONTRIBUTING.md) +- [AGENTS.md](https://github.com/openaustralia/.github/blob/main/AGENTS.md) + +To read them without leaving the terminal, fetch the raw files with any HTTP +client. Don't assume a particular tool is installed, the GitHub CLI included: + + https://raw.githubusercontent.com/openaustralia/.github/main/.github/CONTRIBUTING.md + https://raw.githubusercontent.com/openaustralia/.github/main/AGENTS.md + +The README's Contributing section points people at the same guide. This +repository has no overrides of it. If one is ever agreed, record it here with +the reason, so the difference reads as a decision rather than drift. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..43c994c --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md