Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,37 @@ jobs:
echo "$OUTPUTS" | jq -e '.results | fromjson | all(has("delta"))'



cards:
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
id-token: write
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
- id: map
continue-on-error: true
uses: ./
with:
packages: |
test/fixtures/*
# The preview of evloghq/evlog#1075, which writes the `event` key the
# card reads. Back to the default once a release carries it.
version: https://pkg.pr.new/@evlog/cli@1075
cards-url: https://evlog-cloud.vercel.app
comment: true
comment-key: cards
annotations: false
- name: The app stored the run and the comment links to it
env:
CARDS_URL: ${{ steps.map.outputs.cards-url }}
run: |
echo "$CARDS_URL"
case "$CARDS_URL" in https://evlog-cloud.vercel.app/r/*) ;; *) exit 1 ;; esac
curl -fsS -o /dev/null -w '%{content_type}\n' "$CARDS_URL/report.svg" | grep -q 'image/svg+xml'

telemetry:
runs-on: ubuntu-latest
steps:
Expand Down
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ The judgment is the CLI's. The action adds no rule of its own, so `npx evlog map
| `summary` | `true` | Write the report to the job summary. |
| `comment` | `true` | `true` posts one comment and keeps it updated; `on-failure` only posts when the gate failed, and updates an existing comment either way; `false` posts nothing. Needs `pull-requests: write`; skipped with a notice when the token cannot write (forks). |
| `comment-key` | `default` | Keeps separate comments when the action runs more than once on a pull request. |
| `cards-url` | | URL of an evlog cloud app that draws the report as image cards in the comment. Needs `id-token: write`. Preview. |
| `telemetry` | `true` | Send action health and scan counters. `false` disables telemetry for both the action and its CLI scans. |
| `token` | `${{ github.token }}` | Token for the check run and the comment. |

Expand All @@ -61,6 +62,7 @@ The judgment is the CLI's. The action adds no rule of its own, so `npx evlog map
| `instrumented`, `partial`, `dark` | Entry point counts, every package together. |
| `passed` | `true` when every package met the gate, whatever `gate` is set to. |
| `results` | JSON, one entry per package: `name`, `framework`, `score`, `status`, `reasons`, `summary`, `delta`, `regressions`, `fixed`. |
| `cards-url` | The run page on the `cards-url` app, empty when no cards were drawn. |

## Recipes

Expand Down Expand Up @@ -126,6 +128,22 @@ The comment is posted with the workflow's token, so it shows as `github-actions`
token: ${{ steps.app.outputs.token }}
```

### Report cards (preview)

```yaml
permissions:
contents: read
checks: write
pull-requests: write
id-token: write
# ...
- uses: evloghq/action@v1
with:
cards-url: https://evlog-cloud.vercel.app
```

The action posts the run to the app, which stores it with the base branch's score history and answers with an image card. The comment opens on it: the gate, the score and how it moved, the base branch's score commit by commit and where the pull request lands, then the entry points the pull request changed, with the `log.set()` fields and `log.audit()` actions each one loses or gains. Below it, the comment keeps the gate alert with links into the diff, the wide event diff as text, and every entry point against every check, both folded. The wide event needs an `@evlog/cli` that writes the `event` key. The OIDC token proves the repository to the app, so there is no secret to create. If the app is down or refuses the run, the comment goes out without cards and a notice says why. `https://evlog-cloud.vercel.app` is a proof of concept deployment and may move.

## Frameworks

Everything `evlog map` scans: Nuxt, Nitro, Next.js App Router, TanStack Start, Hono, Express, and Fastify. Detection reads `package.json`; nothing is installed.
Expand Down
7 changes: 7 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ inputs:
comment-key:
description: Distinguishes comments when the action runs more than once on a pull request.
default: default
cards-url:
description: 'URL of an evlog cloud app that draws the report as image cards in the comment. Empty draws none. Needs `id-token: write`, which proves the repository to the app.'
default: ''
telemetry:
description: Send action and CLI telemetry. No repository names, source, paths or tokens. Also disabled by DO_NOT_TRACK=1 or EVLOG_TELEMETRY=0.
default: 'true'
Expand Down Expand Up @@ -78,6 +81,9 @@ outputs:
results:
description: JSON array with one entry per package (directory, framework, score, delta, summary, regressions, fixed).
value: ${{ steps.run.outputs.results }}
cards-url:
description: The run page on the `cards-url` app, empty when no cards were drawn.
value: ${{ steps.run.outputs.cards-url }}

runs:
using: composite
Expand All @@ -100,4 +106,5 @@ runs:
INPUT_COMMENT_KEY: ${{ inputs.comment-key }}
INPUT_TOKEN: ${{ inputs.token }}
INPUT_TELEMETRY: ${{ inputs.telemetry }}
INPUT_CARDS_URL: ${{ inputs.cards-url }}
GITHUB_ACTION_REF: ${{ github.action_ref }}
99 changes: 99 additions & 0 deletions src/lib/cards.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
/**
* Report cards: the run goes to the evlog cloud app, which stores it and
* renders it as images the comment embeds. The comment's Markdown stays the
* source of truth; the cards draw the same numbers.
*/

const AUDIENCE = 'https://evlog.cloud'

function checkCell(check) {
return check.suppressed ? 'silenced' : check.status
}

/** The sensitivity categories the CLI found, `money` out of `money: path says "checkout"`. */
function sensitivity(route) {
if (route.sensitivity.level === 'none') return []
return [...new Set(route.sensitivity.reasons.map(reason => reason.split(':')[0]))]
}

/** What the app stores for one run: the CLI's JSON per package, trimmed to what a card draws. */
export function runPayload(results, context) {
const { repository, serverUrl, sha, refName, pullRequest, cliVersion, minScore, gate = true } = context
return {
repository,
serverUrl,
sha: pullRequest?.headSha ?? sha,
branch: pullRequest?.base ?? refName,
pullRequest: pullRequest && { number: pullRequest.number, baseSha: pullRequest.baseSha, headSha: pullRequest.headSha },
cliVersion,
minScore,
gate,
packages: results.map(result => ({
name: result.name === '.' ? result.projectName : result.name,
framework: result.framework,
score: result.score,
status: result.status,
reasons: result.reasons,
summary: result.summary,
baseScore: result.baseline?.baselineScore,
regressions: result.baseline?.regressions ?? [],
fixed: result.baseline?.fixed ?? [],
routes: result.routes.map(route => ({
id: route.id,
method: route.method,
path: route.path,
file: route.file,
line: route.handler?.line,
score: route.score,
baseScore: result.baselineRoutes?.[route.id],
sensitivity: sensitivity(route),
checks: Object.fromEntries(Object.entries(route.checks).map(([id, check]) => [id, checkCell(check)])),
messages: Object.fromEntries(Object.entries(route.checks).filter(([, check]) => check.status === 'fail' && check.message).map(([id, check]) => [id, check.message])),
event: route.event,
baseEvent: result.baselineEvents?.[route.id],
})),
})),
}
}

/**
* A GitHub Actions OIDC token for the app, which proves the repository the
* run comes from. Only there with `id-token: write`.
*/
async function oidcToken(env, fetchFn) {
if (!env.ACTIONS_ID_TOKEN_REQUEST_URL || !env.ACTIONS_ID_TOKEN_REQUEST_TOKEN) return undefined
const url = `${env.ACTIONS_ID_TOKEN_REQUEST_URL}&audience=${encodeURIComponent(AUDIENCE)}`
const response = await fetchFn(url, { headers: { authorization: `Bearer ${env.ACTIONS_ID_TOKEN_REQUEST_TOKEN}` } })
if (!response.ok) return undefined
return (await response.json()).value
}

/**
* Post the run. Returns the run page and its cards, or why there are none:
* the cards add to the comment, so an app that is down or refuses the run is
* a notice and the comment goes out without them.
*/
export async function publishCards({ endpoint, payload, env = process.env, fetchFn = fetch }) {
const token = await oidcToken(env, fetchFn)
let response
try {
response = await fetchFn(`${endpoint.replace(/\/$/, '')}/api/runs`, {
method: 'POST',
headers: { 'content-type': 'application/json', ...(token ? { authorization: `Bearer ${token}` } : {}) },
body: JSON.stringify(payload),
signal: AbortSignal.timeout(10_000),
})
} catch (error) {
return { outcome: 'failed', reason: `${endpoint} unreachable: ${error.message}` }
}
if (response.status === 401 && !token) return { outcome: 'skipped', reason: 'the cards need `id-token: write` to prove the repository' }
if (!response.ok) return { outcome: 'failed', reason: `${endpoint} answered ${response.status}` }
return { outcome: 'created', ...(await response.json()) }
}

const attr = text => String(text).replace(/&/g, '&').replace(/"/g, '"')

/** One card as GitHub Markdown: the image, linking to the run page. */
export function picture(card, url) {
return `<a href="${attr(url)}"><img alt="${attr(card.alt)}" src="${attr(card.url)}" width="880"></a>`
}
1 change: 1 addition & 0 deletions src/lib/event.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ export function readEvent(env = process.env) {
serverUrl: env.GITHUB_SERVER_URL ?? 'https://github.com',
workspace: env.GITHUB_WORKSPACE ?? process.cwd(),
sha: env.GITHUB_SHA ?? '',
refName: env.GITHUB_REF_NAME ?? '',
runId: env.GITHUB_RUN_ID ?? '',
outputFile: env.GITHUB_OUTPUT,
summaryFile: env.GITHUB_STEP_SUMMARY,
Expand Down
8 changes: 8 additions & 0 deletions src/lib/inputs.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@ function integer(name, raw, { min, max }) {
return value
}

function url(name, raw) {
const value = raw.trim()
if (value === '') return undefined
if (!/^https?:\/\/[^\s/]+/.test(value)) throw new InputError(`Input \`${name}\` must be an http(s) URL, got "${raw}"`)
return value
}

function commentMode(raw) {
const mode = COMMENT_MODES[raw.trim().toLowerCase() || 'true']
if (!mode) throw new InputError(`Input \`comment\` must be true, false or on-failure, got "${raw}"`)
Expand Down Expand Up @@ -71,6 +78,7 @@ export function readInputs(env = process.env) {
summary: bool('summary', get('SUMMARY') || 'true'),
comment: commentMode(get('COMMENT')),
commentKey: get('COMMENT_KEY').trim() || 'default',
cardsUrl: url('cards-url', get('CARDS_URL')),
token: get('TOKEN'),
}
}
Loading
Loading