Skip to content

statistical-annotations.md

Latest commit

 

History

History
339 lines (240 loc) · 16.9 KB

statistical-annotations.md

File metadata and controls

339 lines (240 loc) · 16.9 KB
Raw

Statistical Annotations

APA 7th Edition statistical formatting for SSCI/SCI figures (Sections 6.36, 6.40--6.45, 7.28). All function signatures match scripts/ssci_style.py.

1. p-Value Reporting

1.1 In-Figure (Asterisk System)

Symbol Meaning Symbol Meaning
*** p < .001 * p < .05
** p < .01 n.s. Not significant

Optional: dagger for p < .10 (marginal significance; confirm journal policy first).

Every asterisk system requires a probability note: *p < .05. **p < .01. ***p < .001. Consistency rule: same star count = same alpha level across all figures in the manuscript.

The non-significant marker is 'n.s.' (with periods, per APA convention for abbreviated phrases). Both p_to_stars() and add_significance_bracket() emit 'n.s.' consistently after D10 bug 3 fix.

from ssci_style import p_to_stars, add_significance_bracket

p_to_stars(0.003)   # '**'
p_to_stars(0.12)    # 'n.s.'

add_significance_bracket(ax, x1=0, x2=1, y=4.5, p_value=0.003)
# Optional: height=0.02, text_offset=0.01, lw=1.0, color=NEUTRAL['error_bar']

1.2 Figure Text (Exact p-Values)

Situation Format Example
Typical p = .XXX p = .032
Very small p < .001 p < .001

No leading zero (p cannot exceed 1). Report 2--3 decimal places. Never write p = .000.

from ssci_style import format_p_value
format_p_value(0.032)   # '*p* = .032'
format_p_value(0.0004)  # '*p* < .001'

1.3 Inline Statistical Annotations (add_inline_stats)

For multi-statistic clusters placed inside the axes area (the Chetty-style inline block widely used in top SSCI journals), the Skill provides add_inline_stats() in scripts/ssci_style.py. It accepts a dict (or list of tuples) of stat name -> value and auto-formats each entry per APA conventions:

from ssci_style import add_inline_stats

# Dict form (key -> value)
add_inline_stats(ax, {
    'slope': 0.35,
    'r':     0.71,
    'p':     0.0002,
    'n':     1248,
})
# Renders four lines at top-left:
#   slope = 0.35
#   *r* = .71
#   *p* < .001
#   *n* = 1,248

# Mixed form with pre-formatted CI string
add_inline_stats(ax, {
    'HR': 0.42,
    'CI': '[0.31, 0.57]',
    'p':  '< .001',
}, position='top_right', bbox=True)
# bbox=True draws white semi-transparent background for readability over data

Key parameters (full docstring in scripts/ssci_style.py):

Param Type / Default Effect
items dict OR list of (key, val) or (key, val, ci) tuples Auto-formatted per stat
position 'top_left' (default), 'top_right', 'bottom_left', 'bottom_right', or (x, y) axes-fraction tuple Anchor placement
fontsize None (reads _ACTIVE_PRESET['inline_stats_size'], default 8) Override per call
sep '\\n' (default; one stat per line) Use ', ' for single-line layout
bbox False (default) Draw white semi-transparent background
italic_latin True (default) Auto-italicize Latin stat keys via $\\mathit{...}$

Auto-formatting rules (matched to APA 7 / journal-house conventions):

Key class Examples Format
Correlation r 2 decimals, no leading zero: *r* = .42
Probability p 3 decimals, no leading zero, < .001 for very small: *p* = .032
Sample size n, N Comma-separated integer: *n* = 1,248
Effect size d, g, b, B 2 decimals, with leading zero: *d* = 0.45
Ratio AUC, HR, OR, RR 2 decimals, with leading zero: *HR* = 0.42
String any Rendered verbatim (use for pre-formatted CI strings like '[0.18, 0.52]')

Italic / upright rules (from APA 7 Table 6.5):

  • Latin keys auto-italicize: r, p, n, N, t, F, M, SD, d, b, B, z, OR, HR, SE, AUC, R, Mdn, g, U, MS, SS, k, f, df, RR
  • Greek keys stay upright: alpha, beta, chi, eta, omega, mu, sigma, tau, phi, lambda, pi, rho
  • Acronyms (CFI, RMSEA, SRMR, SHAP, BF_10) are upright by APA convention

Position aliases (matplotlib-style upper_* / lower_*) are accepted alongside the top_* / bottom_* names: upper_left is equivalent to top_left, etc.

For more context on when inline statistics improve "publication feel" -- and when the figure note is the right place instead -- see apa-figure-standards.md §13 (In-Figure Statistical Annotations).

2. Effect Sizes

2.1 Common Metrics

Metric Context Metric Context
d Group mean difference eta^2 / eta_p^2 ANOVA effect
g Small-sample corrected d omega^2 Less biased ANOVA
r Bivariate association f^2 Regression (Cohen)
R^2 Variance explained OR / RR Odds / risk ratio

2.2 Formatting

  • Italic for Latin-letter metrics: d, g, r, R^2, f^2
  • Not italic for Greek-letter metrics: eta^2, eta_p^2, omega^2
  • 2 decimal places. Pair with 95% CI when available.
  • Leading zero: omit when metric cannot exceed 1 (r = .45); include otherwise (d = 1.23).

2.3 Benchmarks (Cohen, 1988)

Metric Small Medium Large
d 0.20 0.50 0.80
r .10 .30 .50
eta^2 .01 .06 .14
f^2 .02 .15 .35

Interpret in domain context; these are guidelines, not rigid cutoffs.

2.4 In-Figure Annotation

from ssci_style import annotate_effect_size
annotate_effect_size(ax, d=0.45, ci=(0.12, 0.78))          # "d = 0.45, 95% CI [0.12, 0.78]"
annotate_effect_size(ax, d=0.38, label="g")                 # "g = 0.38"
annotate_effect_size(ax, d=0.52, label="r", x=0.05, y=0.95) # position at top-left
# Parameters: label ('d'|'g'|'r'|...), x/y (axes fraction), fontsize (8)

3. Confidence Intervals

3.1 Format

APA format: 95% CI [lower, upper] -- square brackets, comma-separated, explicit confidence level.

Examples: d = 0.45, 95% CI [0.12, 0.78] / OR = 2.31, 95% CI [1.45, 3.67]

3.2 Error Bars in Figures

The Note must state what error bars represent:

Type When to use Note wording
SE Precision of mean estimate "Error bars represent standard errors."
SD Data variability "Error bars represent standard deviations."
95% CI Inferential (APA preferred) "Error bars represent 95% confidence intervals."

For repeated-measures designs, apply Morey (2008) within-subject correction and state: "Error bars represent within-subject 95% CIs (Morey, 2008)."

4. Statistical Symbol Italic Rules

4.1 Must Be Italic (Latin Letters) -- APA Table 6.5

Core Latin algebraic symbols (italic):

b, B (regression), d (Cohen's), df, f, F, g (Hedges'), k, M, Mdn, MS, n, N, OR, p, r, R, R^2, SD, SE, SS, t, U, z

Note: n.s. (the non-significant marker, formerly ns) is not italic -- it is treated as an abbreviation per the D10 bug 3 fix that aligns p_to_stars() and add_significance_bracket() on the same string with periods.

4.2 Must NOT Be Italic

Greek letters (upright): alpha, beta, chi^2, eta^2, eta_p^2, sigma, mu, omega^2, phi, lambda, pi, rho, tau, epsilon, delta

phi (the phi coefficient for 2x2 chi-square contingency) is upright per APA 7 Table 6.5. Do not confuse phi (Greek, upright) with phi written as a Latin variable (italic phi-like glyph -- not standard).

Abbreviations (upright): CI, ANOVA, SEM, CFA, EFA, CFI, RMSEA, SRMR, TLI, NFI, AIC, BIC, SHAP, BF, ROC, AUC (when written as the acronym rather than the metric -- see §4.5)

Numbers, operators (+, -, =, <, >), and label subscripts are also upright.

4.3 Matplotlib Implementation

ax.text(x, y, r'$\mathit{p}$ < .001', fontsize=8)       # italic Latin
ax.text(x, y, r'$\eta^2$ = .06', fontsize=8)             # upright Greek
ax.text(x, y, r'$\mathit{R}^2$ = .34', fontsize=8)      # italic R, superscript 2
ax.text(x, y, r'$\phi$ = .28', fontsize=8)              # upright Greek phi
ax.text(x, y, r'$\mathit{I}^2$ = 28%', fontsize=8)      # italic I, superscript 2 (heterogeneity)
ax.text(x, y, r'$\tau^2$ = .04', fontsize=8)             # upright Greek tau, superscript 2

annotate_effect_size() handles italic automatically via $\mathit{label}$. add_inline_stats() auto-italicizes Latin keys per APA 7 §6.42 (see §1.3).

After apply_style(), mathtext.fontset='custom' ensures that bare expressions $r$ / $p$ / $N$ render in the active body font's italic (Arial-Italic by default), not the matplotlib default DejaVuSans-Oblique. This is the D10 bug 1 fix.

4.4 Why Add Discipline-Specific Symbols?

The 25+ chart types in this Skill cover survival analysis, meta-analysis, ROC analysis, mediation, Bayesian comparison, and SHAP / interpretable-ML figures. Each of these brings statistical symbols (*AUC*, *HR*, *I*^2, *tau*^2, BF_10, SHAP) that are not in APA 7's basic worked examples (which are dominated by M, SD, t, F, p). The italic / upright classification of these symbols is settled by APA 7's general rule (Latin italic, Greek upright) plus accepted convention in each chart type's home discipline. §4.5 codifies the resolution for each new symbol.

4.5 New Statistical Symbols Reference Table

Symbol Meaning Source chart type Italic? Format example
AUC Area Under ROC Curve ROC + Calibration plot italic (Latin) *AUC* = 0.83, 95% CI [0.78, 0.88]
HR Hazard Ratio Kaplan-Meier, Cox regression italic (Latin) *HR* = 0.42, 95% CI [0.31, 0.57]
RR Relative Risk / Risk Ratio Meta-analysis, epidemiology italic (Latin) *RR* = 1.85, 95% CI [1.23, 2.78]
I^2 Heterogeneity statistic Forest plot (meta-analysis) italic Latin I + upright superscript 2 *I*^2 = 28%
Q Cochran's Q (heterogeneity) Forest plot (meta-analysis) italic (Latin) *Q*(9) = 12.45, *p* = .19
phi Phi coefficient (2x2 chi^2) Mosaic, Marimekko, association upright (Greek) phi = .28
tau Kendall's tau OR meta-analysis Tau rank corr, forest plot upright (Greek) when used as Kendall's tau coefficient; check journal conventions for meta-analysis -- some treat tau as italic Latin equivalent in equations tau = .31 (Kendall) or *tau*^2 = .04 (meta)
tau^2 Between-study variance (meta) Forest plot upright Greek tau + upright superscript 2 tau^2 = 0.04
BF_10 Bayes Factor in favor of H1 Bayesian comparison upright (acronym) BF_10 = 12.4
SHAP Shapley Additive Explanation SHAP summary plot upright (acronym) `Mean
b / B Unstandardized regression Mediation, path, regression italic (Latin) *b* = 0.35, *SE* = 0.04

Notes on the ambiguous cases:

  • tau has two distinct uses: (1) Kendall's tau -- a rank correlation coefficient, conventionally written as the upright Greek tau per APA 7 Greek-upright rule; (2) meta-analysis tau / tau^2 -- the between-study variance in random-effects models. Some methodological journals treat this as an italic Latin equivalent (since it functions as a statistic estimated from data, not a population parameter); APA 7 itself does not address the case. Default to upright Greek unless the target journal's style guide explicitly directs otherwise; the leading meta-analysis statistical packages (metafor R, meta R) render it upright in their output.

  • I^2 has two interpretations in literature: (1) the heterogeneity statistic where I is treated as a Latin-letter statistic so the symbol is italic; (2) some older texts treat I as an acronym for "inconsistency", in which case I is upright. APA 7 favors interpretation (1); this Skill renders *I*^2 (italic I, upright superscript 2).

  • AUC is upright when written as the acronym ("AUC was 0.83") but italic when written as a statistical symbol in an equation ("AUC = 0.83"). The format guidance in §4.5 favors the italic symbol form because that is how it appears in inline figure annotations.

  • HR / RR / OR follow the same dual-use rule: italic when used as a statistical symbol with a value, upright when used as an acronym ("the hazard ratio was 0.42" -- upright HR -- vs. "HR = 0.42" -- italic).

The add_inline_stats() helper handles this automatically: when you pass {'HR': 0.42, 'AUC': 0.83, 'p': 0.0002, 'I2': 28} it renders italic Latin keys, upright Greek keys, and follows the per-statistic formatting rules in §1.3.

5. Path Diagrams / SEM

5.1 Standardized Coefficients

beta = .45*** (Greek beta, upright, with stars on the path arrow)

5.2 Unstandardized Coefficients

b = 1.23, SE = 0.34 (italic Latin letters)

5.3 Indirect Effects (Mediation)

ab = .12, 95% CI [.04, .22] (bootstrap CI for product of paths)

5.4 Model Fit in Note

*Note.* Standardized coefficients shown. Model fit: chi^2(24) = 36.45,
p = .049, CFI = .97, RMSEA = .045, 90% CI [.001, .074], SRMR = .038.
Solid lines = significant paths; dashed lines = non-significant paths.
*p < .05. **p < .01. ***p < .001.

Non-significant paths: dashed lines, NEUTRAL["ns_path"] color (see scripts/ssci_style.py).

For the full set of conventions governing mediation diagram rendering (arrow semantics, c' curvature rule, indirect-effect reporting, panel-label cases), see apa-figure-standards.md §11 Mediation Diagram Conventions.

6. Best Practices

  1. Prefer exact p-values over asterisks alone; include exact values in Note when feasible.
  2. Always report effect sizes alongside significance -- p says "exists"; effect size says "matters."
  3. Multiple comparisons: state correction in Note ("Bonferroni-corrected p values are reported.").
  4. Bayes factors (optional): BF_10 = 12.4. Not italic (abbreviation). APA does not prescribe format.
  5. Survival analysis figures: report HR with 95% CI and log-rank p-value in Note. For Kaplan-Meier curves intended for NEJM / Lancet, include the at-risk table below the x-axis (see chart-types-applied.md §1).
  6. Meta-analysis figures: report heterogeneity statistics (Q, I^2, tau^2) and pooling method (REML / DL / inverse-variance) in Note. Format: Heterogeneity: *Q*(9) = 12.45, *p* = .19, *I*^2 = 28%, *tau*^2 = 0.04, pooling method = REML.
  7. Bayesian comparisons: if reporting BF_10 (favoring H1), also state the prior assumption ("default Cauchy prior, scale = 0.707" or the actual prior used). APA does not prescribe Bayes factor format; the format above is consistent with BayesFactor R package output.
  8. ROC / AUC figures: report AUC with bootstrap 95% CI (typical: 2,000 iterations) and diagonal reference line ("Diagonal reference line represents chance performance.").
  9. SHAP / feature importance figures: report mean absolute SHAP value per feature; state the prediction target and model class in the Note. SHAP itself is an upright acronym, the mean magnitude is a continuous statistic without italic convention.

7. Decimal Place Conventions

Statistic Decimals Leading zero? Example
M, SD 1--2 Yes M = 3.45
r, proportions 2 No r = .45
t, F, chi^2 2 Yes F(2, 97) = 4.56
p 2--3 No p = .032
d, g 2 Yes d = 0.45
R^2, eta^2 2 No R^2 = .34
OR, RR 2 Yes OR = 2.31

Leading-zero rule: omit only when the statistic cannot exceed 1.00 in absolute value.

8. Helper Function Quick Reference

Function Purpose Key parameters
p_to_stars(p) p-value to ***/**/*/n.s. p: float
format_p_value(p) APA string: *p* = .032 p: float, style='apa'
add_significance_bracket(ax, x1, x2, y, p_value) Bracket with stars height, text_offset, lw, color
annotate_effect_size(ax, d, ci=None) Effect-size text on axes label, x, y, fontsize
add_inline_stats(ax, items, position=...) Multi-stat inline cluster position, fontsize, bbox, italic_latin, sep
add_reference_line(ax, value, orientation=...) Dashed gray reference line color, linestyle, linewidth, label, label_position
format_apa_figure_text(figure_number, title, ...) Full APA figure Markdown note_text, specific_notes, p_levels, error_bar_type, model_fit, zwsp

format_apa_figure_text example:

from ssci_style import format_apa_figure_text
text = format_apa_figure_text(
    figure_number=1,
    title="Mean Anxiety Scores by Treatment Condition",
    note_text="N = 354. Error bars represent 95% confidence intervals.",
    specific_notes=["^a n = 178.", "^b n = 176."],
    p_levels=[("*", ".05"), ("**", ".01"), ("***", ".001")],
)

Output:

**Figure 1**

*Mean Anxiety Scores by Treatment Condition*

*Note.* N = 354. Error bars represent 95% confidence intervals.

^a n = 178. ^b n = 176.

*p < .05. **p < .01. ***p < .001.