From 5a61b5b9049b78407dc794e954ed47ed6367a1ae Mon Sep 17 00:00:00 2001 From: vstet Date: Thu, 24 Sep 2026 20:11:21 -0500 Subject: [PATCH] Document running maintenance scripts as modules Adds a Maintenance scripts section to the README covering populate_api_urls and verify_no_click, and notes in CLAUDE.md that scripts/ must be run with python -m from the repo root. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 5 +++++ README.md | 15 +++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index 7227a97..52068a1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -23,8 +23,13 @@ make format # auto-format code (Ruff) # Populate API URLs with validation python -m scripts.populate_api_urls --validate + +# Verify no_click values (writes data/companies_verified.csv) +python -m scripts.verify_no_click ``` +Scripts in `scripts/` must be run as modules (`python -m scripts.`) from the repo root — `python scripts/.py` breaks their `from config import`. + ## Architecture The tool scans company career pages for matching job titles. Each `run()` call in [main.py](main.py) follows this flow: diff --git a/README.md b/README.md index 4563f21..d86bd3e 100644 --- a/README.md +++ b/README.md @@ -95,6 +95,21 @@ python scheduler.py # start scheduler, wait for first scheduled time python scheduler.py --run-now # run a scan immediately, then follow the schedule ``` +#### Maintenance scripts +Standalone tools in `scripts/` for keeping `companies.csv` up to date. Run them as modules from the project root (`python -m scripts.`). Running `python scripts/.py` directly fails, because the scripts import `config` from the root. + +```bash +# Fill the api_url column after adding companies (same as `make populate`) +python -m scripts.populate_api_urls +python -m scripts.populate_api_urls --validate # also HEAD-check each generated URL +python -m scripts.populate_api_urls --input data/companies.csv --output data/companies.csv + +# Visit each open_positions_url and set the correct no_click value +python -m scripts.verify_no_click # reads data/companies.csv, writes data/companies_verified.csv +python -m scripts.verify_no_click --resume # continue from the last checkpoint +python -m scripts.verify_no_click --no-headless # show browser window +``` + --- ## How It Works