Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,11 @@ jobs:
print(f'argus {argus.__version__} — config OK, {len(cfg[\"tags\"])} workstreams')
"
# Proves the "works with no API key" promise, writing outside the repo so
# the committed dashboard is never touched by a test run.
# the committed dashboard is never touched by a test run. --force because
# the template ships with the sweep switched off; CI must still exercise it.
- name: Pipeline runs end-to-end with no secrets at all
run: |
python -m argus.run --stage score --db "$RUNNER_TEMP/ci.db"
python -m argus.run --stage render --db "$RUNNER_TEMP/ci.db" --out "$RUNNER_TEMP/index.html"
python -m argus.run --stage score --db "$RUNNER_TEMP/ci.db" --force
python -m argus.run --stage render --db "$RUNNER_TEMP/ci.db" --out "$RUNNER_TEMP/index.html" --force
grep -q "<title>" "$RUNNER_TEMP/index.html"
git diff --exit-code # a test run must leave the tree clean
33 changes: 30 additions & 3 deletions .github/workflows/pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,32 @@ permissions:
id-token: write # required by actions/deploy-pages
issues: write # open a failure alert
jobs:
# The master switch, read before anything expensive happens. A disabled repo
# spends a few seconds here and nothing else — no dependency install, no
# collection, no Pages deploy. A manual dispatch runs regardless, so you can
# always try a sweep without editing config.
gate:
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
enabled: ${{ steps.check.outputs.enabled }}
steps:
- uses: actions/checkout@v4
- id: check
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
echo "enabled=true" >> "$GITHUB_OUTPUT"
echo "manual dispatch — running regardless of the config switch"
elif grep -qiE '^enabled:[[:space:]]*(false|no|off)([[:space:]]*#.*)?$' config.yaml; then
echo "enabled=false" >> "$GITHUB_OUTPUT"
echo "::notice::Scheduled sweep is switched off. Set 'enabled: true' in config.yaml once your sources are configured. Run this workflow manually to try a one-off sweep."
else
echo "enabled=true" >> "$GITHUB_OUTPUT"
fi

run:
needs: gate
if: needs.gate.outputs.enabled == 'true'
runs-on: ubuntu-latest
timeout-minutes: 20 # a wedged collector should fail the run, not hang for 6h
steps:
Expand Down Expand Up @@ -81,10 +106,12 @@ jobs:
uses: ./.github/workflows/pages.yml
permissions: {contents: read, pages: write, id-token: write}

# `always()` so a failed sweep still alerts — that is the entire point.
# `always()` so a failed sweep still alerts — that is the entire point. Gated
# on the switch too, so a deliberately-off repo doesn't spin a runner to
# report that nothing happened.
alert:
needs: run
if: always()
needs: [gate, run]
if: always() && needs.gate.outputs.enabled == 'true'
uses: ./.github/workflows/alert.yml
with: {conclusion: "${{ needs.run.result }}"}
permissions: {contents: read, issues: write}
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,18 @@ publish a tag that disagrees with it.

## [Unreleased]

### Added

- **A master switch: `enabled` in `config.yaml`.** `false` means scheduled
sweeps do nothing — no collection, no scoring, no commit — while a manual run
of the pipeline workflow still works. Checked in a cheap `gate` job before any
dependency install, so a switched-off repository costs seconds rather than a
full sweep. `--force` overrides it locally.
- The template now ships with `enabled: false`. It was sweeping every 8 hours
for the example field, and — worse — whatever it collected became the starting
data of every repository created from it. The committed database and dashboard
are reset to empty for the same reason.

## [1.3.0] — 2026-08-09

### Added
Expand Down
3 changes: 2 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ install:
dry-run: ## collect + per-source counts, no writes
python -m argus.run --dry-run

# ARGS passes flags through, e.g. make run ARGS=--force while `enabled: false`
run: ## full pipeline (keyword mode unless a provider key is set)
python -m argus.run
python -m argus.run $(ARGS)

collect:
python -m argus.run --stage collect
Expand Down
26 changes: 24 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ file. Works with **no API key** (keyword scoring) and **no email** (dashboard
only) — every capability degrades cleanly when its secret is absent, and a run
prints which scorer it actually used.

> **A fresh copy starts switched off.** `config.yaml` ships with
> `enabled: false` so it doesn't sweep for the example field before it's yours.
> Set `enabled: true` once your sources are configured — step 5 below.

> This is a **fork-per-person template**. Each researcher runs their own copy,
> pointed at their own topics. It ships configured for one example field
> (Southeast-Asia power systems) so it runs out of the box — replace that config
Expand All @@ -36,7 +40,11 @@ prints which scorer it actually used.
few-shot examples for your work. This is the system prompt used for LLM scoring.
4. **Pick a scorer** — see the table below. The free keyword scorer needs
nothing; an LLM scorer needs one API key, and several are free.
5. **Turn on Actions and Pages.** Two one-time switches in your fork:
5. **Switch it on.** `config.yaml` ships with `enabled: false`, so a fresh
copy doesn't sweep for the example field before it's yours. Set
`enabled: true` when steps 1-3 are done. (A manual run of the pipeline
workflow always works, switch or no switch, if you want to try one first.)
6. **Turn on Actions and Pages.** Two one-time switches in your fork:
- **Actions** tab → enable workflows (forks start with them disabled).
- **Settings → Pages → Source: GitHub Actions** — needed once before the
dashboard can publish. Skip it and everything still works; the sweep just
Expand Down Expand Up @@ -193,13 +201,27 @@ your scoring provider unless `digest.synthesis_provider` says otherwise.
```bash
make install # pip install -r requirements.txt
make dry-run # collect + per-source counts, no writes
make run # full pipeline
make run # full pipeline (ARGS=--force while `enabled: false`)
make test # offline unit tests — no network, no keys
make journals # find a journal's ISSN by name, or audit your watchlist
make vacuum # reclaim DB space after pruning (occasional; see below)
open docs/index.html
```

### Pausing it

`enabled: false` in `config.yaml` stops scheduled sweeps entirely — no
collection, no scoring, no commits. Useful for a template, and equally for going
on leave or quieting a radar you're mid-way through retuning.

It's checked before anything expensive happens, so a paused repo costs a few
seconds per schedule rather than a full sweep. Running the pipeline workflow
**manually still works** either way, and locally you can force one:

```bash
make run ARGS=--force
```

### Sweep cadence and the commit log

Each sweep appends to a per-source run log, so the SQLite file changes every
Expand Down
27 changes: 27 additions & 0 deletions argus/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,21 @@ def _read(path: str) -> str:
return f.read()


def enabled(cfg: dict) -> bool:
"""Is the sweep switched on? Defaults to True, so an existing config that
predates this switch keeps running.

The template ships with it off. Two reasons: a template shouldn't burn
Actions minutes collecting for an example field nobody is reading, and
whatever it collected would otherwise become the starting data of every
repository created from it.
"""
value = cfg.get("enabled", True)
if isinstance(value, str):
return value.strip().lower() not in ("false", "no", "off", "0", "")
return bool(value)


def _library_due(conn, lib_cfg: dict) -> bool:
"""Library discovery runs at most every `every_days` (default 7) — it's a
weekly shortlist, not something to rebuild every 4h. dry-run (conn=None) always runs."""
Expand Down Expand Up @@ -136,6 +151,16 @@ def score(conn, cfg: dict, kw: dict, mode: str, spec: dict) -> dict:

def run_pipeline(args) -> int:
cfg = _load_yaml(args.config)

# The master switch. Off means a scheduled sweep does nothing at all: no
# collection, no scoring, no commit. `--force` overrides it so you can still
# try a run by hand without editing config.
if not enabled(cfg) and not args.force:
print("sweep disabled — config.yaml has `enabled: false`. "
"Set it to true once your sources are configured, "
"or run with --force for a one-off.")
return 0

sources = _load_yaml(os.path.join(ROOT, "registry", "sources.yaml"))
kw = sources.get("keywords", {})
window_days = cfg.get("window_days", 7)
Expand Down Expand Up @@ -195,6 +220,8 @@ def main(argv=None) -> int:
p.add_argument("--db", default=None)
p.add_argument("--out", default=None, help="dashboard path (default docs/index.html)")
p.add_argument("--dry-run", action="store_true")
p.add_argument("--force", action="store_true",
help="run even when config.yaml has `enabled: false`")
p.add_argument("--vacuum", action="store_true",
help="reclaim space after pruning (rewrites the whole file — occasional, not scheduled)")
p.add_argument("--force-digest", action="store_true")
Expand Down
10 changes: 10 additions & 0 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,16 @@
# * edit registry/sources.yaml (your queries, journals, feeds, keywords)
# * edit `tags` (your workstreams) and prompts/research.md (your profile+rubric)
# * pick a scoring provider below (several are free)
# MASTER SWITCH. `false` means scheduled sweeps do nothing at all — no
# collection, no scoring, no commit. A manual run of the pipeline workflow still
# works, so you can try one without editing this.
#
# It ships OFF in the template on purpose: an unconfigured copy would otherwise
# collect for the example field below, burn Actions minutes, and hand whatever
# it gathered to the next person who uses the template as their starting data.
# Turn it on once steps 1-3 of the README are done and your sources are yours.
enabled: false

title: "ARGUS"

window_days: 7 # collectors ignore items older than this
Expand Down
Binary file modified data/argus.db
Binary file not shown.
Loading
Loading