Skip to content

Bug 2067298 Simple View: Update How to read the results for notes on CLES, CD, Significance #1106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kala-moz wants to merge 1 commit into
base: main
Choose a base branch
from
+117 −6
Open

Bug 2067298 Simple View: Update How to read the results for notes on CLES, CD, Significance #1106

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions src/__tests__/CompareResults/ResultsView.test.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -573,6 +573,16 @@ describe('Results View', () => {
const panel = screen.getByTestId('how-to-read-results');
expect(panel).toBeInTheDocument();

// The advanced stats are explained in a nested list, even though those
// columns are off by default.
expect(within(panel).getByText(/Cliff's Delta/)).toBeInTheDocument();
expect(within(panel).getByText('CLES')).toBeInTheDocument();
expect(
within(panel).getByText(/Common Language Effect Size/),
).toBeInTheDocument();
expect(within(panel).getByText('Significance')).toBeInTheDocument();
expect(within(panel).getByText(/Mann-Whitney U test/)).toBeInTheDocument();

// Dismiss via the panel's close button (scoped: the MWU warning banner
// also has a close button).
const user = userEvent.setup({ advanceTimers: jest.advanceTimersByTime });
Expand Down
86 changes: 82 additions & 4 deletions src/__tests__/CompareResults/__snapshots__/ResultsTable.test.tsx.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1094,9 +1094,48 @@ exports[`Results Table Should match snapshot 1`] = `
<li>
Use
<strong>
Advanced columns
Advanced options
</strong>
to add the expert stats (Cliff's Delta, CLES, and Significance).
to add these expert stats as extra columns:
<ul
class="MuiBox-root css-10m8p6r"
>
<li>
<strong>
Cliff's Delta
</strong>
(header:
<strong>
CD
</strong>
) is the number behind Magnitude, so it takes the Magnitude column's place. It runs from -1 to 1: a score near 0 means little difference, and a score above 0.47 or below -0.47 means a large one. A negative score means New values are usually higher.
</li>
<li>
<strong>
CLES
</strong>
(Common Language Effect Size) is shown as a percentage: the chance that a New value is lower than a Base value. Near 50% means New and Base are about equally likely to be higher; the farther from 50%, the clearer the difference.
</li>
<li>
<strong>
Significance
</strong>
(header:
<strong>
Sig
</strong>
when space is tight) says whether the difference is

<strong>
Real
</strong>
or just
<strong>
Noise
</strong>
, based on a Mann-Whitney U test — “Real” means a p-value under 0.05. It's the same signal as the “Noise” tag on Status, as its own sortable, filterable column.
</li>
</ul>
</li>
</ul>
</div>
Expand Down Expand Up @@ -4129,9 +4168,48 @@ exports[`Results Table for MannWhitneyResultsItem for mann-whitney-u testVersion
<li>
Use
<strong>
Advanced columns
Advanced options
</strong>
to add the expert stats (Cliff's Delta, CLES, and Significance).
to add these expert stats as extra columns:
<ul
class="MuiBox-root css-10m8p6r"
>
<li>
<strong>
Cliff's Delta
</strong>
(header:
<strong>
CD
</strong>
) is the number behind Magnitude, so it takes the Magnitude column's place. It runs from -1 to 1: a score near 0 means little difference, and a score above 0.47 or below -0.47 means a large one. A negative score means New values are usually higher.
</li>
<li>
<strong>
CLES
</strong>
(Common Language Effect Size) is shown as a percentage: the chance that a New value is lower than a Base value. Near 50% means New and Base are about equally likely to be higher; the farther from 50%, the clearer the difference.
</li>
<li>
<strong>
Significance
</strong>
(header:
<strong>
Sig
</strong>
when space is tight) says whether the difference is

<strong>
Real
</strong>
or just
<strong>
Noise
</strong>
, based on a Mann-Whitney U test — “Real” means a p-value under 0.05. It's the same signal as the “Noise” tag on Status, as its own sortable, filterable column.
</li>
</ul>
</li>
</ul>
</div>
Expand Down
27 changes: 25 additions & 2 deletions src/components/CompareResults/HowToReadResults.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,31 @@ function HowToReadResults() {
changes.
</li>
<li>
Use <strong>Advanced columns</strong> to add the expert stats
(Cliff&apos;s Delta, CLES, and Significance).
Use <strong>Advanced options</strong> to add these expert stats as
extra columns:
<Box component='ul' sx={{ m: 0, mt: 0.5, pl: 2.5 }}>
<li>
<strong>Cliff&apos;s Delta</strong> (header: <strong>CD</strong>)
is the number behind Magnitude, so it takes the Magnitude
column&apos;s place. It runs from -1 to 1: a score near 0 means
little difference, and a score above 0.47 or below -0.47 means a
large one. A negative score means New values are usually higher.
</li>
<li>
<strong>CLES</strong> (Common Language Effect Size) is shown as a
percentage: the chance that a New value is lower than a Base
value. Near 50% means New and Base are about equally likely to be
higher; the farther from 50%, the clearer the difference.
</li>
<li>
<strong>Significance</strong> (header: <strong>Sig</strong> when
space is tight) says whether the difference is{' '}
<strong>Real</strong> or just <strong>Noise</strong>, based on a
Mann-Whitney U test — &ldquo;Real&rdquo; means a p-value under
0.05. It&apos;s the same signal as the &ldquo;Noise&rdquo; tag on
Status, as its own sortable, filterable column.
</li>
</Box>
</li>
</Box>
</Alert>
Expand Down