Skip to content

Document running maintenance scripts as modules #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ukrainets merged 2 commits into from
Sep 25, 2026
Merged

Document running maintenance scripts as modules #27

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5a61b5b
Document running maintenance scripts as modules
ukrainets Sep 25, 2026
aa471f7
Merge branch 'main' into docs/maintenance-scripts
ukrainets Sep 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,13 @@ make format # auto-format code (Ruff)

# Populate API URLs with validation
python -m scripts.populate_api_urls --validate

# Verify no_click values (writes data/companies_verified.csv)
python -m scripts.verify_no_click
```

Scripts in `scripts/` must be run as modules (`python -m scripts.<name>`) from the repo root — `python scripts/<name>.py` breaks their `from config import`.

## Architecture

The tool scans company career pages for matching job titles. Each `run()` call in [main.py](main.py) follows this flow:
Expand Down
15 changes: 15 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,21 @@ python scheduler.py # start scheduler, wait for first scheduled time
python scheduler.py --run-now # run a scan immediately, then follow the schedule
```

#### Maintenance scripts
Standalone tools in `scripts/` for keeping `companies.csv` up to date. Run them as modules from the project root (`python -m scripts.<name>`). Running `python scripts/<name>.py` directly fails, because the scripts import `config` from the root.

```bash
# Fill the api_url column after adding companies (same as `make populate`)
python -m scripts.populate_api_urls
python -m scripts.populate_api_urls --validate # also HEAD-check each generated URL
python -m scripts.populate_api_urls --input data/companies.csv --output data/companies.csv

# Visit each open_positions_url and set the correct no_click value
python -m scripts.verify_no_click # reads data/companies.csv, writes data/companies_verified.csv
python -m scripts.verify_no_click --resume # continue from the last checkpoint
python -m scripts.verify_no_click --no-headless # show browser window
```

---

## How It Works
Expand Down
Loading