Skip to content

feat(worstcase): add report_individual flag for per-record member_prob#461

Merged
JessUWE merged 2 commits into
mainfrom
452-worstcase-report-individual
Jun 11, 2026
Merged

feat(worstcase): add report_individual flag for per-record member_prob#461
JessUWE merged 2 commits into
mainfrom
452-worstcase-report-individual

Conversation

@shamykyzer

@shamykyzer shamykyzer commented May 25, 2026

Copy link
Copy Markdown
Contributor

closes #452

adds a report_individual flag to WorstCaseAttack (default False)

when set to True, each repetition's metrics dict gets an "individual" key holding per record member_prob and member, matching how LIRAAttack and QMIAAttack already report per record results

this lets MetaAttack fold worstcase into its vulnerability matrix later, that wiring is a separate follow up

@shamykyzer shamykyzer requested a review from jim-smith May 26, 2026 14:40
Base automatically changed from 428-meta-attack to main May 26, 2026 16:49
@shamykyzer shamykyzer requested a review from rpreen May 28, 2026 08:58
@rpreen

rpreen commented May 28, 2026

Copy link
Copy Markdown
Contributor

This still needs clean up? There are conflicts to be resolved; the PR title scopes it to "worstcase" yet introduces a variety of changes to tests for other attacks, the introduction of a constants module, a meta attack example, and changes to the readme, etc. The initial comment indicates that this PR is stacked on top of #441, but that appears to now be merged, so why are there still so many changes unrelated from the description?

@shamykyzer shamykyzer force-pushed the 452-worstcase-report-individual branch from 217474f to e769ea1 Compare May 29, 2026 14:24
@codecov

codecov Bot commented May 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.65%. Comparing base (85c9f0f) to head (5b62d2e).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #461   +/-   ##
=======================================
  Coverage   99.65%   99.65%           
=======================================
  Files          27       27           
  Lines        3436     3439    +3     
=======================================
+ Hits         3424     3427    +3     
  Misses         12       12           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Jessica Ikechukwu <Jessica.Ikechukwu@uwe.ac.uk>
@JessUWE JessUWE merged commit 7176627 into main Jun 11, 2026
4 checks passed
@JessUWE JessUWE deleted the 452-worstcase-report-individual branch June 11, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WorstCaseAttack to expose per-record scores

3 participants