Skip to content

Feature/auditability scoring rubric - #22

Open
pathankhansalman wants to merge 2 commits into
safal207:mainfrom
pathankhansalman:feature/auditability-scoring-rubric
Open

Feature/auditability scoring rubric#22
pathankhansalman wants to merge 2 commits into
safal207:mainfrom
pathankhansalman:feature/auditability-scoring-rubric

Conversation

@pathankhansalman

Copy link
Copy Markdown

Summary

One or two sentences. What changes and why.

Spec impact

  • No change to docs/SPEC.md.
  • Editorial change to docs/SPEC.md (no behavioral change).
  • Normative change to docs/SPEC.md. List affected sections below.

If this is a normative change, describe:

  • What invariant or field is added, removed, or changed.
  • Whether previously valid records may now be rejected, or vice versa.
  • The corresponding CHANGELOG.md entry.

Validation and testing

  • schema/drp.schema.json updated if the machine-checkable shape changed.
  • tools/drp_validator.py updated if a new invariant was introduced.
  • Positive fixture(s) added under fixtures/valid/.
  • Negative fixture(s) added under fixtures/invalid/.
  • Tests added or updated under tests/.
  • python3 -m pytest tests/ passes locally.

Breaking change?

  • No. Strictly additive or editorial.
  • Yes. Explain why it is necessary and how consumers migrate.

Checklist

  • Branch is up to date with main.
  • CHANGELOG.md updated under [Unreleased].
  • VERSION updated if this PR cuts a release.
  • No new runtime dependencies added to the reference validator.

Copy link
Copy Markdown
Owner

This is a strong and thoughtful first research contribution — thank you. The distinction between validation and scoring is especially useful, and the 8-dimension rubric gives DRP a credible path toward measurable auditability.

Before merge, please make these focused changes:

  1. Remove the CONTRIBUTING.md change from this PR. A contributor path has already been merged into main, so this part now duplicates existing content. This PR should stay focused on docs/AUDITABILITY_SCORING.md.

  2. Add explicit N/A handling and normalized scoring. Some dimensions do not apply to every record. For example, a root decision may have no parent links, and a decision that replaces nothing should not be penalized for missing supersession. Please define the final score as earned points / applicable maximum × 100 and explain when a dimension is marked N/A.

  3. Update the PR description. Please replace the template text with a short summary, mark this as a documentation/research change with no protocol semantics change, and include Closes #21.

Small cleanup:

  • fix Breath of alternatives to Breadth of alternatives in the summary table;
  • make clear that automated or NLP checks are proposed future implementations, not guarantees of the current repository.

Once these are addressed, I will re-review quickly. This is valuable work and I would like to get it merged.

safal207 commented Jul 4, 2026

Copy link
Copy Markdown
Owner

Hi! Just checking in on this PR. Your auditability rubric is still valuable to the project, and we’d be happy to review the requested updates when you have time.

Please let us know whether you plan to continue with the PR or whether any of the requested changes need clarification or help. No pressure — we just want to keep the contribution path clear and avoid leaving your work in limbo.

@safal207 safal207 mentioned this pull request Jul 18, 2026
15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants