Skip to content

feat: explain supplier scoring decisions deterministically #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yigitcan-ozturk merged 5 commits into from
Aug 28, 2026
Merged

feat: explain supplier scoring decisions deterministically #9

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5f9a056
feat: add deterministic decision explanations
yigitcan-ozturk Aug 28, 2026
fbcd536
test: cover decision explanation report fields
yigitcan-ozturk Aug 28, 2026
b5fe963
test: correct expected decision margin
yigitcan-ozturk Aug 28, 2026
bd2b2d5
test: cover deterministic decision explanations
yigitcan-ozturk Aug 28, 2026
5302baa
docs: document deterministic decision explanations
yigitcan-ozturk Aug 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 71 additions & 21 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Mixed-currency quotations should first be normalized with [`currency-normalizer`

Supplier quotations are often compared in spreadsheets where assumptions, weights and recommendation logic become difficult to audit. `rfqdiff` keeps the commercial comparison small and inspectable: every score is produced from explicit inputs and explicit weights.

The goal is not to automate procurement judgment. The goal is to make a commercial comparison reproducible enough that a reviewer can understand how the recommendation was produced and trace each loaded quotation back to its input file.
The goal is not to automate procurement judgment. The goal is to make a commercial comparison reproducible enough that a reviewer can understand how the recommendation was produced, see which criteria contributed to the result, and trace each loaded quotation back to its input file.

## Decision boundary

Expand All @@ -25,8 +25,9 @@ It does:
- compare price, lead time and payment terms;
- import quotations from JSON, CSV and Excel (`.xlsx`);
- use explicit default or user-supplied commercial scoring weights;
- produce deterministic supplier scores;
- return machine-readable JSON for downstream decision systems;
- produce deterministic supplier scores and criterion-level score contributions;
- identify the runner-up and score margin;
- return machine-readable decision explanations for downstream systems;
- export ranked comparison reports as CSV or Excel;
- attach source-file provenance to loaded quotations;
- preserve upstream normalization metadata when present in JSON inputs.
Expand All @@ -37,6 +38,7 @@ It intentionally does **not**:
- determine technical compliance;
- score operational supplier risk;
- make contractual acceptance decisions;
- generate opaque AI rationale for a recommendation;
- accept hidden or unsupported scoring criteria;
- treat file hashes as digital signatures or proof of authenticity.

Expand Down Expand Up @@ -134,6 +136,44 @@ Weight profiles are deliberately strict:

The effective profile is returned in the output payload under `weights`, keeping each recommendation auditable.

## Deterministic decision explanations

Every scored supplier contains a `score_breakdown` showing the weighted contribution of each supported criterion to the final score:

```json
{
"name": "Supplier A",
"score": 97.1,
"score_breakdown": {
"price": 47.1,
"lead_time": 30.0,
"payment_terms": 20.0
}
}
```

The result also identifies the runner-up and the score margin between the top two suppliers. `decision_explanation` then exposes the winning supplier's component scores and the supplier leading each raw commercial criterion:

```json
"decision_explanation": {
"winner": "Supplier A",
"runner_up": "Supplier B",
"score_margin": 30.0,
"winner_score_breakdown": {
"price": 47.1,
"lead_time": 30.0,
"payment_terms": 20.0
},
"criterion_leaders": {
"price": "Supplier B",
"lead_time": "Supplier A",
"payment_terms": "Supplier A"
}
}
```

This is a deterministic explanation layer, not generated prose. Reviewers can reproduce every contribution from the quotation values and the effective weights rather than accepting an opaque narrative.

## Quotation provenance

Every quotation loaded from a file receives an `rfqdiff_source` object. It records the input artifact used for that specific supplier row without storing the caller's full local path.
Expand Down Expand Up @@ -172,23 +212,23 @@ The provenance object stays with each supplier through scoring and JSON output.

CSV reports contain one ranked supplier per row with:

- rank;
- recommended flag;
- supplier name;
- currency and price;
- lead time;
- payment terms;
- rank and recommended flag;
- supplier name, currency and price;
- lead time and payment terms;
- final score;
- price, lead-time and payment-term score contributions;
- source provenance fields.

Excel reports contain two worksheets:

- `Comparison` — the ranked supplier table plus source provenance;
- `Summary` — recommended supplier, lowest price, fastest lead time, best payment terms and the effective scoring weights.
- `Comparison` — the ranked supplier table, component scores and source provenance;
- `Summary` — recommended supplier, runner-up, score margin, lowest price, fastest lead time, best payment terms and effective scoring weights.

The report uses the same scored payload as the JSON output, so the human-facing export and downstream integration result remain aligned.

The JSON contract contains:
## Machine-readable contract

A result includes the recommendation, ranked suppliers, decision summary, explanation, effective weights and any quotation provenance:

```json
{
Expand All @@ -200,6 +240,11 @@ The JSON contract contains:
{
"name": "Supplier A",
"score": 97.1,
"score_breakdown": {
"price": 47.1,
"lead_time": 30.0,
"payment_terms": 20.0
},
"rfqdiff_source": {
"file": "quotations.csv",
"format": "csv",
Expand All @@ -208,6 +253,11 @@ The JSON contract contains:
}
}
],
"decision_summary": {
"recommended_supplier": {"name": "Supplier A", "score": 97.1},
"runner_up": {"name": "Supplier B", "score": 67.1},
"score_margin": 30.0
},
"weights": {
"price": 0.5,
"lead_time": 0.3,
Expand All @@ -216,7 +266,7 @@ The JSON contract contains:
}
```

The full supplier objects also include price, lead time, payment terms and any upstream normalization metadata.
The full `decision_summary` also includes lowest price, fastest lead time and best payment terms. The full supplier objects include commercial values and any upstream normalization metadata.

## Public Python API

Expand Down Expand Up @@ -310,6 +360,7 @@ GitHub Actions validates:
- source-file provenance, row/sheet traceability and SHA-256 fingerprints;
- rejection of caller-supplied reserved provenance fields;
- default and configurable scoring profiles;
- criterion-level score breakdowns, runner-up selection and score margins;
- rejection of incomplete or unsupported weight profiles;
- CSV and Excel comparison report exports;
- wheel and source-distribution builds;
Expand All @@ -319,9 +370,10 @@ GitHub Actions validates:

## Engineering principles

- **Explicit scoring** — weights and criteria remain visible.
- **Deterministic results** — identical supported inputs produce identical comparison results.
- **Explicit scoring** — weights, criterion contributions and final scores remain visible.
- **Deterministic explanations** — recommendation rationale is computed from the same scoring model, not generated as opaque prose.
- **Traceable inputs** — loaded quotations retain source artifact fingerprints and tabular locations.
- **Deterministic results** — identical supported inputs produce identical comparison results.
- **Separation of concerns** — FX, technical compliance and supplier risk stay outside quotation scoring.
- **Machine-readable handoff** — downstream tools consume a stable structured result instead of scraping presentation text.
- **Review before authority** — a recommendation supports procurement judgment; it does not replace approval.
Expand All @@ -331,19 +383,17 @@ GitHub Actions validates:
| Tool | Role |
| --- | --- |
| [`currency-normalizer`](https://github.com/yigitcan-ozturk/currency-normalizer) | Normalize quotation values across currencies |
| **[`rfqdiff`](https://github.com/yigitcan-ozturk/rfqdiff)** | Compare and score normalized quotations |
| **[`rfqdiff`](https://github.com/yigitcan-ozturk/rfqdiff)** | Compare and explain normalized commercial quotations |
| [`payment-terms-parser`](https://github.com/yigitcan-ozturk/payment-terms-parser) | Convert payment terms into commercial-risk signals |
| [`vendor-risk-engine`](https://github.com/yigitcan-ozturk/vendor-risk-engine) | Score operational, quality, compliance and dependency risk |
| [`bidlint`](https://github.com/yigitcan-ozturk/bidlint) | Produce evidence-backed technical-compliance findings and scores |
| [`supplier-scorecard`](https://github.com/yigitcan-ozturk/supplier-scorecard) | Combine commercial, risk and technical signals into one supplier recommendation |

## Roadmap

- Richer decision explanations

## Status

Early-stage project, currently at **v0.2**. The current line provides a stable JSON integration contract, an installable Python package and console CLI, JSON/CSV/XLSX quotation ingestion, auditable configurable commercial scoring weights, CSV/XLSX comparison report export, and source-level quotation provenance.
Early-stage project, currently at **v0.2**. The v0.2 development line now provides JSON/CSV/XLSX quotation ingestion, auditable configurable weights, criterion-level scoring explanations, source provenance, CSV/XLSX report export, an installable Python package and console CLI, and a stable machine-readable integration contract.

The original v0.2 product roadmap is implemented. The next milestone is release/pilot hardening rather than adding more scoring criteria to this tool.

## License

Expand Down
65 changes: 65 additions & 0 deletions main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,11 @@ def score_quotes(

scored_quote = quote.copy()
scored_quote["score"] = round(total_score, 1)
scored_quote["score_breakdown"] = {
"price": round(price_score, 1),
"lead_time": round(lead_time_score, 1),
"payment_terms": round(payment_score, 1),
}
scored_quotes.append(scored_quote)

return sorted(
Expand All @@ -323,10 +328,20 @@ def build_result(
) -> dict[str, Any]:
effective_weights = validate_weights(WEIGHTS if weights is None else weights)
winner = scored_quotes[0]
runner_up = scored_quotes[1] if len(scored_quotes) > 1 else None
score_margin = (
round(winner["score"] - runner_up["score"], 1) if runner_up else None
)
cheapest = min(scored_quotes, key=lambda item: item["price"])
fastest = min(scored_quotes, key=lambda item: item["lead_time_weeks"])
best_terms = max(scored_quotes, key=lambda item: item["payment_days"])

runner_up_summary = (
{"name": runner_up["name"], "score": runner_up["score"]}
if runner_up
else None
)

return {
"tool": "rfqdiff",
"version": VERSION,
Expand All @@ -338,6 +353,8 @@ def build_result(
"name": winner["name"],
"score": winner["score"],
},
"runner_up": runner_up_summary,
"score_margin": score_margin,
"lowest_price": {
"name": cheapest["name"],
"price": cheapest["price"],
Expand All @@ -351,6 +368,17 @@ def build_result(
"payment_days": best_terms["payment_days"],
},
},
"decision_explanation": {
"winner": winner["name"],
"runner_up": runner_up["name"] if runner_up else None,
"score_margin": score_margin,
"winner_score_breakdown": winner["score_breakdown"],
"criterion_leaders": {
"price": cheapest["name"],
"lead_time": fastest["name"],
"payment_terms": best_terms["name"],
},
},
"weights": effective_weights,
}

Expand Down Expand Up @@ -387,14 +415,19 @@ def print_report(

payload = build_result(quotes, currency, effective_weights)
summary = payload["decision_summary"]
explanation = payload["decision_explanation"]
winner = summary["recommended_supplier"]
runner_up = summary["runner_up"]
cheapest = summary["lowest_price"]
fastest = summary["fastest_lead_time"]
best_terms = summary["best_payment_terms"]

print("\nDecision summary")
print("-" * 86)
print(f"Recommended supplier : {winner['name']} ({winner['score']}/100)")
if runner_up:
print(f"Runner-up : {runner_up['name']} ({runner_up['score']}/100)")
print(f"Score margin : {summary['score_margin']} points")
print(
f"Lowest price : {cheapest['name']} "
f"({money(cheapest['price'], currency)})"
Expand All @@ -408,6 +441,15 @@ def print_report(
f"({best_terms['payment_days']} days)"
)

breakdown = explanation["winner_score_breakdown"]
print("\nWinner score breakdown")
print("-" * 86)
print(
f"Price {breakdown['price']}/100 contribution | "
f"Lead time {breakdown['lead_time']}/100 contribution | "
f"Payment terms {breakdown['payment_terms']}/100 contribution"
)

print("\nScoring weights")
print(
f"Price {effective_weights['price']:.0%} | "
Expand Down Expand Up @@ -445,6 +487,9 @@ def write_csv_report(payload: dict[str, Any], path: Path) -> None:
"lead_time_weeks",
"payment_days",
"score",
"price_score",
"lead_time_score",
"payment_terms_score",
"source_file",
"source_format",
"source_sha256",
Expand All @@ -455,6 +500,7 @@ def write_csv_report(payload: dict[str, Any], path: Path) -> None:
writer = csv.DictWriter(file, fieldnames=fieldnames)
writer.writeheader()
for rank, quote in enumerate(payload["suppliers"], start=1):
breakdown = quote.get("score_breakdown", {})
row = {
"rank": rank,
"recommended": quote["name"] == payload["recommended_supplier"],
Expand All @@ -464,6 +510,9 @@ def write_csv_report(payload: dict[str, Any], path: Path) -> None:
"lead_time_weeks": quote["lead_time_weeks"],
"payment_days": quote["payment_days"],
"score": quote["score"],
"price_score": breakdown.get("price", ""),
"lead_time_score": breakdown.get("lead_time", ""),
"payment_terms_score": breakdown.get("payment_terms", ""),
}
row.update(_report_source_fields(quote))
writer.writerow(row)
Expand All @@ -490,6 +539,9 @@ def write_xlsx_report(payload: dict[str, Any], path: Path) -> None:
"Lead Time (Weeks)",
"Payment Days",
"Score",
"Price Score",
"Lead Time Score",
"Payment Terms Score",
"Source File",
"Source Format",
"Source SHA-256",
Expand All @@ -499,6 +551,7 @@ def write_xlsx_report(payload: dict[str, Any], path: Path) -> None:
)
for rank, quote in enumerate(payload["suppliers"], start=1):
source = quote.get("rfqdiff_source", {})
breakdown = quote.get("score_breakdown", {})
comparison.append(
[
rank,
Expand All @@ -509,6 +562,9 @@ def write_xlsx_report(payload: dict[str, Any], path: Path) -> None:
quote["lead_time_weeks"],
quote["payment_days"],
quote["score"],
breakdown.get("price", ""),
breakdown.get("lead_time", ""),
breakdown.get("payment_terms", ""),
source.get("file", ""),
source.get("format", ""),
source.get("sha256", ""),
Expand All @@ -529,6 +585,15 @@ def write_xlsx_report(payload: dict[str, Any], path: Path) -> None:
decision["recommended_supplier"]["score"],
]
)
runner_up = decision.get("runner_up")
summary.append(
[
"Runner-up",
runner_up["name"] if runner_up else "",
runner_up["score"] if runner_up else "",
]
)
summary.append(["Score margin", "", decision.get("score_margin", "")])
summary.append(
[
"Lowest price",
Expand Down
Loading
Loading