Skip to content

build: enforce README.md presence on every rule#33

Merged
slashben merged 2 commits into
mainfrom
sub-7177-agents-md-and-readme-check
May 11, 2026
Merged

build: enforce README.md presence on every rule#33
slashben merged 2 commits into
mainfrom
sub-7177-agents-md-and-readme-check

Conversation

@slashben
Copy link
Copy Markdown
Contributor

@slashben slashben commented May 11, 2026

Summary

  • Adds scripts/check_readmes.sh (invoked from gen.sh) — fails the build if any rule directory under pkg/rules/ is missing or has an empty README.md.
  • Adds AGENTS.md documenting the expectation for future contributors and agents (each rule YAML change must accompany a README update).
  • Adds bats tests for the check script.
  • CI installs bats via a pinned action and runs the bats tests before generating the Rules CRD.

Test plan

  • bats tests cover three cases: missing README, present README, empty README
  • gen.sh correctly aborts when run against the current README-less tree (gate is live)
  • gen.sh guard catches a moved/missing scripts/check_readmes.sh
  • CI passes on the first push — verify the bats-core/bats-action@3.0.0 step works

⚠️ Merge ordering

This gate fails builds until per-rule README.md files exist for all 26 rules. The companion PR adding those READMEs (SUB-7177 Task 8) should merge before this one, or merged together such that main is never red.

Refs: SUB-7177

Summary by CodeRabbit

  • Documentation

    • Added contributor guidelines requiring rules to include complete documentation.
  • Tests

    • Introduced automated tests to validate rule documentation requirements.
  • Chores

    • Enhanced the release workflow to enforce rule documentation standards.

Review Change Stack

slashben added 2 commits May 11, 2026 20:00
Adds scripts/check_readmes.sh (called from gen.sh) which fails the
build if any rule directory under pkg/rules/ is missing or has an
empty README.md. AGENTS.md documents the expectation for future
contributors and agents.

Refs SUB-7177.
Signed-off-by: Ben <ben@armosec.io>
- gen.sh: guard against missing scripts/check_readmes.sh path
- AGENTS.md: drop broken reference to internal design doc
- CI: replace unpinned apt-get bats install with pinned action

Refs SUB-7177.
Signed-off-by: Ben <ben@armosec.io>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 11, 2026

📝 Walkthrough

Walkthrough

This PR adds a validation gate to enforce that every rule directory in pkg/rules/ contains a non-empty README.md file. The validation is executed during release builds via gen.sh and tested in CI using a Bats test suite with temporary fixtures.

Changes

README Validation Enforcement for Rule Directories

Layer / File(s) Summary
Contribution Policy
AGENTS.md
Documents the requirement that each rule directory must have a non-empty README.md, updated together with rule YAML changes, enforced at release build time.
Core Validation Script
scripts/check_readmes.sh
Bash script that validates pkg/rules/ (or a provided path), checking that each r[0-9]* directory has a non-empty README.md; exits non-zero on failures.
Bats Test Suite
scripts/check_readmes_test.bats
Three tests verify script behavior: missing README.md causes failure, adding it causes success, and empty README.md causes failure. Uses temporary fixture directories.
Release Build Integration
gen.sh
Integrates validation as a preflight check: locates scripts/check_readmes.sh relative to gen.sh, runs it against pkg/rules, and exits if validation fails.
CI Workflow
.github/workflows/test.yml
Installs bats-core/bats-action@3.0.0 and runs the Bats test suite during the test job.

🎯 2 (Simple) | ⏱️ ~10 minutes

🐰 A validation gate so neat,
Each rule needs a README complete!
Release builds shall enforce,
No empty docs, of course—
Quality rules, a rabbit's treat! 🎉

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the primary change: enforcing README.md presence on every rule directory during the build process.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch sub-7177-agents-md-and-readme-check

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/check_readmes.sh`:
- Line 26: The find in scripts/check_readmes.sh currently restricts iteration to
directories named r[0-9]*, allowing other rule dirs to bypass README checks;
update the find invocation used in the loop (the command ending with -print0
that feeds the while/done) to remove the -name 'r[0-9]*' filter so it iterates
all immediate subdirectories (keep -mindepth 1 -maxdepth 1 -type d -print0), and
adjust any downstream logic that assumed the rNN pattern to operate on the
directory basename generically (e.g., variable used inside the while loop that
validates README files).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e1671545-c9fc-4de3-8f88-7b188ceac21c

📥 Commits

Reviewing files that changed from the base of the PR and between dd7f631 and 37f9448.

📒 Files selected for processing (5)
  • .github/workflows/test.yml
  • AGENTS.md
  • gen.sh
  • scripts/check_readmes.sh
  • scripts/check_readmes_test.bats

Comment thread scripts/check_readmes.sh
echo "check_readmes: empty README.md in $(basename "$rule_dir")" >&2
failed=1
fi
done < <(find "$RULES_DIR" -mindepth 1 -maxdepth 1 -type d -name 'r[0-9]*' -print0)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Policy bypass risk: validation only targets r[0-9]* directory names.

Line 26 scopes checks to a naming pattern, so a rule directory with a different name can skip README enforcement.

Proposed fix
-done < <(find "$RULES_DIR" -mindepth 1 -maxdepth 1 -type d -name 'r[0-9]*' -print0)
+done < <(find "$RULES_DIR" -mindepth 1 -maxdepth 1 -type d -print0)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
done < <(find "$RULES_DIR" -mindepth 1 -maxdepth 1 -type d -name 'r[0-9]*' -print0)
done < <(find "$RULES_DIR" -mindepth 1 -maxdepth 1 -type d -print0)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/check_readmes.sh` at line 26, The find in scripts/check_readmes.sh
currently restricts iteration to directories named r[0-9]*, allowing other rule
dirs to bypass README checks; update the find invocation used in the loop (the
command ending with -print0 that feeds the while/done) to remove the -name
'r[0-9]*' filter so it iterates all immediate subdirectories (keep -mindepth 1
-maxdepth 1 -type d -print0), and adjust any downstream logic that assumed the
rNN pattern to operate on the directory basename generically (e.g., variable
used inside the while loop that validates README files).

@slashben slashben merged commit 2c6a706 into main May 11, 2026
3 of 4 checks passed
@slashben slashben deleted the sub-7177-agents-md-and-readme-check branch May 11, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant