Skip to content

Wire calibration into the analyzer so confidence is earned (and --check can enforce) #48

Description

@hyperpolymath

crates/oikosbot-analysis/src/calibration.rs provides estimate_operation() returning min/typical/max ResourceRanges with Confidence::Calibrated for known operation kinds (HashLookup, Sort, FileIO, NetworkCall, Allocation, StringOp, MathCompute). Its module doc says it "replaces naive complexity * 0.1 J with pattern-based resource profiles".

It was never wired in. Analyzer::estimate_resources() is still exactly that naive form:

let energy = Energy::joules(complexity as f64 * 0.1);
let duration = Duration::milliseconds(complexity as f64 * 0.5);
let memory = Memory::kilobytes(complexity * 2);

and every result is hardcoded Confidence::Estimated.

Consequences

  1. Every published figure is a guess — energy, carbon, duration, memory, and therefore EcoScore, EconScore, ParetoScore and the health index.
  2. --check can never enforce. Only Measured/Calibrated inputs may block, so the enforcement path is inert. fix(check): refuse loudly instead of passing silently #47 makes that refusal loud rather than silent, but the underlying capability is still missing.

What to do

  • Map detected patterns (patterns.rs: nested-loops, busy-wait, string-concat-in-loop, clone-in-loop, unbuffered-io, large-allocation) onto OperationKinds.
  • Use estimate_operation() for recognised patterns; keep the naive path for unrecognised code and label it Estimated — confidence must be earned per finding, never blanket-assigned.
  • Propagate the real ResourceRange (min/typical/max) rather than collapsing to a point estimate; a range is what makes "Calibrated" meaningful.
  • Add tests proving (a) a recognised pattern yields Calibrated, (b) unrecognised code stays Estimated, (c) --check can now exit 1 on an undocumented, calibrated regression — a falsifier for the gate.

Caution

This changes every resource figure and every downstream score, including for the 17 estate consumers. It needs its own reviewed change with before/after numbers, not a drive-by. Do not relabel existing estimates as Calibrated to unblock --check — that would be overclaiming.

Blocks: real regulator mode.
Related: #42 (Pareto engine), #47 (loud refusal).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew capability or improvement to existing behaviour

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions