Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/geval.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ jobs:
- name: Run Geval
run: |
./geval/target/release/geval check \
--contract geval/examples/contract.yaml \
--signals signals.json \
--policy geval/examples/policy.yaml \
--env prod
continue-on-error: true

Expand Down
67 changes: 38 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,27 +58,28 @@ geval init

This creates a **.geval** folder with:

- **signals.json** — sample signals (scores, presence-only). Edit and add yours.
- **policy.yaml** — sample rules. Edit and add yours.
- **README.md** — how to run from here.
- **contract.yaml** — Contract: name, version, combine rule, and list of policy paths.
- **policies/** — Policy files (e.g. security.yaml, quality.yaml). Edit and add rules.
- **signals.json** — Sample signals. Edit and add yours.
- **README.md** — How to run from here.

Then run:

```bash
geval check --signals .geval/signals.json --policy .geval/policy.yaml
geval check --contract .geval/contract.yaml --signals .geval/signals.json
```

Use a different folder: `geval init my-rules`. Overwrite existing template files: `geval init --force`.
Use a different folder: `geval init my-rules`. Overwrite existing files: `geval init --force`.

### Updating

Use the same download commands. Replace your old file with the new one. Check version: `geval --version`.

---

## Use Geval with your own signals and rules
## Use Geval with your own signals and contract

You need **two files**: **your signals** (any kind — scores, flags, presence-only) and **your rules**. Geval doesn't decide; it **orchestrates** and **reconciles** your rules against your signals and returns one outcome. Use `geval init` for a ready-made template, or create the files yourself below.
You need a **contract** (one YAML that references one or more **policy** files) and a **signals** file. Geval evaluates each policy against the same signals, then combines outcomes (e.g. all must pass, or any block blocks). Use `geval init` for a template with a contract and two policies, or create the files yourself below.

**All kinds of signals:** Not every signal needs a score. You can mix: entries with a numeric `value`, and entries with no value (presence-only). Use a rule with `operator: presence` to match “this metric exists.” [Details →](geval/docs/signals-and-rules.md)

Expand All @@ -99,15 +100,25 @@ Example — save as `mydata.json`:

You can add labels like `component` or `system` if you need them. [Full example →](geval/examples/signals.json)

### Step 2: Your rules (rules file)
### Step 2: Your contract and policies

A list of rules in order. Geval applies the first rule, then the next, and stops at the first match. It doesn't interpret — it just evaluates your conditions against your signals.
A **contract** is a YAML file that lists one or more **policy** files and a **combination rule** (how to merge their outcomes). Each **policy** file contains ordered rules: **When** [condition on signals], **then** [pass / block / require_approval].

Each rule says: **When** [something about your signals], **then** [allow / need approval / block].
Example contract — save as `contract.yaml`:

Example — save as `myrules.yaml`:
```yaml
name: my-gate
version: "1.0.0"
combine: all_pass
policies:
- path: policy.yaml
```

Example policy — save as `policy.yaml` (path relative to the contract file):

```yaml
name: quality
version: "1.0.0"
policy:
rules:
- priority: 1
Expand All @@ -118,8 +129,6 @@ policy:
threshold: 0
then:
action: block
reason: "Engagement dropped"

- priority: 2
name: allow_good_accuracy
when:
Expand All @@ -130,38 +139,34 @@ policy:
action: pass
```

**Operators:** `>` greater than, `<` less than, `>=` at least, `<=` at most, `==` equal, `presence` = metric exists (no threshold; use for signals without a score).

**Actions:** `pass` = allow. `block` = don’t allow. `require_approval` = a person must say yes first.
**Combine rules:** `all_pass` = PASS only if every policy passes; `any_block_blocks` = any policy BLOCK → overall BLOCK. **Operators:** `>`, `<`, `>=`, `<=`, `==`, `presence`. **Actions:** `pass`, `block`, `require_approval`.

[Full example →](geval/examples/policy.yaml)
[Full example →](geval/examples/contract.yaml) and [policy →](geval/examples/policy.yaml)

### Step 3: Run Geval

Point Geval at your two files:

```bash
./geval check --signals mydata.json --policy myrules.yaml
./geval check --contract contract.yaml --signals mydata.json
```

(Windows: `.\geval.exe check --signals mydata.json --policy myrules.yaml`)
(Windows: `.\geval.exe check --contract contract.yaml --signals mydata.json`)

### Step 4: Read the outcome

- **PASS** — No rule matched a block or require-approval. You’re good to go.
- **REQUIRE_APPROVAL** — A rule says someone must approve before you go.
- **BLOCK** — A rule says stop. Fix the issue before going.
- **PASS** — Every policy passed (or combined rule says go).
- **REQUIRE_APPROVAL** — At least one policy requires approval.
- **BLOCK** — At least one policy blocks.

To see **which rule** produced that outcome (and which signals it used):
To see **per-policy results** and the combined decision:

```bash
./geval explain --signals mydata.json --policy myrules.yaml
./geval explain --contract contract.yaml --signals mydata.json
```

To check that your rules file is valid (no run needed):
To validate the contract and all referenced policies:

```bash
./geval validate-policy myrules.yaml
./geval validate-contract contract.yaml
```

---
Expand Down Expand Up @@ -215,15 +220,19 @@ Each run is recorded: which rules, which signals, when. So you can always answer
| `geval check` | Orchestrate: run your signals + rules → one outcome (PASS / REQUIRE_APPROVAL / BLOCK) |
| `geval explain` | Show which rule produced the outcome and which signals were used |
| `geval approve` / `geval reject` | Record a person’s approval or rejection |
| `geval validate-policy` | Check your rules file is valid |
| `geval validate-contract` | Validate contract and all referenced policies |

---

## Documentation

| Guide | Description |
|-------|-------------|
| [**Architecture**](geval/docs/architecture.md) | Contract = multiple policies + combine rule; module layout |
| [**Signals and rules**](geval/docs/signals-and-rules.md) | Non-uniform signals (scores, presence-only, mix); how rules use them |
| [**Signal assumptions**](geval/docs/signal-assumptions.md) | What we assume; what input forms we accept (number, string, trace, object) |
| [**Versioning**](geval/docs/versioning.md) | Contract, policy, and signals versioning; nothing unversioned |
| [**Extending**](geval/docs/extending.md) | How to add a combination rule or change behavior; process and conventions |
| [**GitHub Actions**](geval/docs/github-actions.md) | Use Geval in CI |
| [**Examples**](geval/examples/README.md) | Sample data and rules files |
| [**Installation**](geval/docs/installation.md) | Install, PATH, build from source |
Expand Down
14 changes: 14 additions & 0 deletions geval/.geval/decisions/2026-03-18T18:40:04Z.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"artifact_version": "1",
"geval_version": "0.1.2",
"policy_name": "demo-contract",
"policy_version": "1.0.0",
"signals_name": "demo-signals",
"signals_version": "1.0.0",
"policy_hash": "7664d9d21340214b7b6342bb7b90db6ec3c9f546ea4c8dd35a6cb154fa3c655f",
"signals_hash": "344a3d19b3b0241f3a36d817d0a6ce4d3194bd75172353b8207ebcc6c90fc8b2",
"decision": "BLOCK",
"matched_rule": "business_block",
"timestamp": "2026-03-18T18:40:04Z",
"approval": null
}
32 changes: 32 additions & 0 deletions geval/.geval/decisions/2026-03-18T18:53:33Z.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
"artifact_version": "2",
"geval_version": "0.1.2",
"contract_name": "release-gate",
"contract_version": "1.0.0",
"contract_hash": "c969102e3d85f861eef2f28c19d619775275adb1830bafb16276e93b9f032a4d",
"signals_name": "my-signals",
"signals_version": "1.0.0",
"signals_hash": "77b6fe14da32a7a2262580067ef84258acd839643b8c7e758f4348008307519e",
"combine_rule": "all_pass",
"policy_results": [
{
"policy_path": "policies/security.yaml",
"policy_name": "security",
"policy_version": "1.0.0",
"policy_hash": "78cdf683db7d3a416717d1fb9627075f47c6a9154eadda0b237d5d0eb9248347",
"outcome": "PASS"
},
{
"policy_path": "policies/quality.yaml",
"policy_name": "quality",
"policy_version": "1.0.0",
"policy_hash": "8bf934d103cdcbd4a81cdc1bdddf20cb7114c030be70ea1b1ba3ea39188358c9",
"outcome": "BLOCK",
"matched_rule": "block_engagement_drop"
}
],
"combined_decision": "BLOCK",
"combined_matched_rule": "policies/quality.yaml:block_engagement_drop",
"timestamp": "2026-03-18T18:53:33Z",
"approval": null
}
25 changes: 25 additions & 0 deletions geval/.geval/decisions/2026-03-18T18:53:37Z.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"artifact_version": "2",
"geval_version": "0.1.2",
"contract_name": "demo",
"contract_version": "1.0.0",
"contract_hash": "61e788400405fb9aa92868aefe5b4ecfc4c29118232aa7bbb7df1df2d2ef307b",
"signals_name": "demo-signals",
"signals_version": "1.0.0",
"signals_hash": "344a3d19b3b0241f3a36d817d0a6ce4d3194bd75172353b8207ebcc6c90fc8b2",
"combine_rule": "all_pass",
"policy_results": [
{
"policy_path": "policy.yaml",
"policy_name": "demo-contract",
"policy_version": "1.0.0",
"policy_hash": "7664d9d21340214b7b6342bb7b90db6ec3c9f546ea4c8dd35a6cb154fa3c655f",
"outcome": "BLOCK",
"matched_rule": "business_block"
}
],
"combined_decision": "BLOCK",
"combined_matched_rule": "policy.yaml:business_block",
"timestamp": "2026-03-18T18:53:37Z",
"approval": null
}
59 changes: 59 additions & 0 deletions geval/docs/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Geval Architecture

Geval is **contract-centric**: a **contract** is a named, versioned set of **policies** evaluated together with a **combination rule**. Every decision is fully versioned and auditable.

## Core concepts

| Concept | Description |
|--------|-------------|
| **Contract** | YAML file: `name`, `version`, `combine` (rule), and list of policy paths. The unit of evaluation. |
| **Policy** | YAML file: optional `name`/`version`, `environment`, and ordered `rules`. Each rule has `when` (conditions) and `then` (action: pass / block / require_approval). |
| **Signals** | JSON: optional `name`/`version`, and array of signal objects (metric, value, component, etc.). Facts fed into the engine. |
| **Combination rule** | How to merge outcomes from multiple policies: `all_pass` or `any_block_blocks`. |

## Data flow

1. **Load contract** – Parse contract YAML; resolve policy paths relative to the contract file; load each policy.
2. **Load signals** – Parse signals JSON; build an in-memory signal graph (metric → value lookup).
3. **Evaluate each policy** – For each policy, evaluate rules in priority order; first matching rule gives that policy’s outcome (PASS / REQUIRE_APPROVAL / BLOCK).
4. **Combine** – Apply the contract’s combination rule to the list of policy outcomes → single combined decision.
5. **Artifact** – Write `.geval/decisions/<timestamp>.json` with contract identity, per-policy results, combined decision, and hashes.

## Module layout

```
geval/src/
contract/ # Contract = multiple policies + combine rule
model.rs # ContractDef, PolicyRef
combine.rs # CombineRule (all_pass, any_block_blocks), apply_combine_rule
loader.rs # load_contract, load_contract_and_policies, parse_contract_str
runner.rs # run_contract → ContractResult (per-policy + combined)
policy/ # Single policy model and parser
model.rs # Policy, Rule, RuleCondition, RuleConsequence, Action, Operator
parser.rs # parse_policy, parse_policy_str
evaluator/ # Single-policy evaluation
engine.rs # evaluate(policy, graph) → Decision; evaluate_with_trace
signal_graph/ # Build lookup from signals for rule matching
signals/ # Load signals JSON (name, version, signals array)
hashing/ # SHA256 for contract, policy, signals (audit)
artifact/ # Write decision artifact (contract + per-policy + combined)
explanation/ # Human-readable report (explain_contract_result, explain_decision)
approval/ # Approval/rejection artifact (versioned)
cli/ # Commands: check, init, demo, explain, validate-contract, approve, reject
```

## Invariants

- **Nothing unversioned** – Contract, policies, and signals have name/version; artifact records them and hashes.
- **Deterministic** – Same contract + same signals → same combined decision.
- **No remote calls** – All inputs and outputs are local files.

## Adding a new combination rule

1. Add a variant to `CombineRule` in `contract/combine.rs`.
2. Implement the logic in `apply_combine_rule` (match on the new variant).
3. Add `Serialize`/`Deserialize` (and `FromStr`/`Display` if you want CLI/artifact string).
4. Add tests in `contract/combine::tests`.
5. Document in [signals-and-rules.md](signals-and-rules.md) or [versioning.md](versioning.md).

See [extending.md](extending.md) for the full change process.
29 changes: 18 additions & 11 deletions geval/docs/auditing.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# Accountability and Auditing

Geval is designed so auditors can answer:
Geval is designed so **nothing is unversioned**: every decision and every action is auditable. Auditors can answer:

- **Why was this deployed?** – Decision report and matched rule (and optional approval reason).
- **Who approved it?** – `geval approve` writes an artifact with `approved_by` and `reason`.
- **What policy was used?** – Policy is version-controlled; artifact stores `policy_hash` (SHA256).
- **What signals existed?** – Artifact stores `signals_hash` (SHA256); signals themselves are produced by your pipeline and can be archived separately.
- **Who approved it?** – `geval approve` writes an artifact with `approved_by`, `reason`, and artifact `version`.
- **What policy (contract) was used?** – Artifact stores `policy_name`, `policy_version`, and `policy_hash` (SHA256).
- **What signals were used?** – Artifact stores `signals_name`, `signals_version`, and `signals_hash` (SHA256).
- **Which Geval binary?** – Artifact stores `geval_version`.

**Rule of thumb:** When you change policy or signals, bump their `version` so every decision is tied to a specific version. No update without a version update.

## Artifacts

Expand All @@ -14,11 +17,15 @@ Geval is designed so auditors can answer:
Each `geval check` run writes:

- **Path:** `.geval/decisions/<timestamp>.json`
- **Contents:**
- `policy_hash` – SHA256 of the policy used
- `signals_hash` – SHA256 of the signals used
- `decision` – PASS | REQUIRE_APPROVAL | BLOCK
- `matched_rule` – name of the rule that fired (if any)
- **Contents (artifact_version 2, contract-centric):**
- `artifact_version` – schema version (e.g. `"2"`)
- `geval_version` – binary version that produced the decision
- `contract_name`, `contract_version`, `contract_hash` – contract identity and content hash
- `signals_name`, `signals_version`, `signals_hash` – signals identity and content hash
- `combine_rule` – how policy outcomes were merged (e.g. `all_pass`, `any_block_blocks`)
- `policy_results` – array of `{ policy_path, policy_name?, policy_version?, policy_hash, outcome, matched_rule? }` for each policy
- `combined_decision` – final outcome (PASS | REQUIRE_APPROVAL | BLOCK)
- `combined_matched_rule` – first non-PASS policy and rule (if any)
- `timestamp` – ISO8601
- `approval` – optional; set when an approval is recorded for this decision

Expand All @@ -27,12 +34,12 @@ Each `geval check` run writes:
`geval approve` / `geval reject` write:

- **Path:** configurable (e.g. `.geval/approval.json`)
- **Contents:** `approved_by`, `reason`, `timestamp`, `approved` (true/false)
- **Contents:** `version` (artifact format), `approved_by`, `reason`, `timestamp`, `approved` (true/false)

## Reproducibility

- **Deterministic:** Same signals + same policy → same decision.
- **Hashes:** Stored in the decision artifact so you can verify which policy and which signals were used.
- **Versions + hashes:** Decision artifact records policy/signals name and version (human identity) and content hashes (integrity). You can verify exactly which contract and signals version was used.
- **No remote services:** All inputs and outputs are local files; no telemetry or external calls.

## What Geval does not do
Expand Down
Loading
Loading