Skip to content

enhancement: establish calibrated prediction-confidence and explanation boundaries #28

Description

@CoreyLeath-code

Audit finding

The repository provides a PyTorch/FastAPI seven-class facial-expression classifier and already states that expression does not reliably reveal internal emotional state. Searches found no committed reasoning or confidence_score implementation. Raw scores must not be labeled calibrated confidence or human-emotion certainty without a reproducible protocol.

Evidence inspected

  • README.md: FER-style classification with a research/demo, non-medical boundary.

  • Tests cover model, API, pipeline, and production-context paths.

  • Existing issue Evaluation protocol audit #26 tracks evaluation protocol; this issue complements it.

Proposed enhancement

Separate predicted expression label, model score, calibrated confidence only when supported, model version, input-validation result, and an abstain/low-confidence state. Explanations must remain concise and must not claim mental state, intent, or pixel-level causality.

Acceptance criteria

  • Preserve inference behavior and return calibrated confidence only after reproducible calibration.

  • Test invalid images, low confidence, abstention, schema validation, and model-version metadata.

  • Document calibration split, method, and limitations before publishing metrics.

  • Keep optional context retrieval separate from image-model evidence.

NEEDS HUMAN DECISION

Approve the dataset, subgroup review, and calibration target before enabling a numeric confidence claim.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions