Skip to content

Enhancement: add confidence intervals and Mann-Whitney U test to analysis output #1

Description

@stevjoo

Right now analyze_results.py only computes descriptive statistics (mean, median, stddev, percentiles), but there's no way to tell whether a measured overhead difference is real or just noise. For example, if SELinux shows +3.2% overhead and AppArmor shows +2.8%, we can't say the 0.4% gap is meaningful because it could easily be hypervisor preemption or cache state randomness.

Two things need to be added:

  1. 95% Confidence Intervals on the overhead percentage, using bootstrap resampling. This gives a range like +3.2% (95% CI: +2.8% to +3.6%) instead of just a point estimate. If the CI for two MACs overlaps, we can't claim one is faster. If they don't overlap, we have statistical evidence. Bootstrap is the right method here because the overhead ratio (MAC − baseline) / baseline is a derived quantity that doesn't have a clean analytical CI formula.

  2. Mann-Whitney U test instead of a t-test, because syscall latency distributions are right-skewed (many fast iterations, a few slow outliers from cache misses and preemption), not Gaussian. Mann-Whitney U is non-parametric—it doesn't assume any distribution shape. The test answers: "Is the MAC latency distribution statistically different from baseline?" with a p-value. If p < 0.05, the overhead is statistically significant; if not, we can't claim there's a real difference.

The output should look something like:

Category    MAC        Overhead    95% CI            p-value    Significant?
network     selinux    +3.2%       [+2.8%, +3.6%]    0.0001     YES
network     apparmor   +2.8%       [+2.1%, +3.5%]    0.0012     YES
ptrace      apparmor   +1.2%       [-0.3%, +2.7%]    0.0823     NO

That last row is the important case: the CI crosses zero and p > 0.05, meaning the 1.2% overhead is not statistically distinguishable from noise. Without these tests, we'd wrongly report it as real overhead.

This requires adding SciPy as a dependency (pip install scipy) and modifying analyze_results.py to compute bootstrap confidence intervals and scipy.stats.mannwhitneyu for each MAC-vs-baseline pair, then include ci_lower, ci_upper, and p_value in the summary output and plots.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions