diff --git a/README.md b/README.md index 5b8a79f..646271c 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ Research pipeline for forecasting the compendium prediction card of The Internat | [Historical validation and weaknesses](docs/ti26/2026-08-08-known-weaknesses.md) | Raw rating models did not clear the constant baseline; calibrated Glicko cleared the narrowly registered D3b gate. Repeat-event card skill remains unproven. | | [Series scoring](reports/series_score/series_score.md) | Positive historical diagnostic; within-event dependence limits its nominal significance calculation. | | [Group postmortem](reports/card_postmortem.md) and [playoff postmortem](reports/playoff_postmortem.md) | Dependent diagnostics from the same tournament, not independent replications. Group results feed playoff strengths. | -| [External playoff cards](data/ti2026_playoff_cards.yaml) | Owner-supplied comparisons with incomplete generation provenance, not a controlled model benchmark. Original [LLM evidence](predictions-from-llms/) is frozen. | +| [External playoff cards](data/ti2026_playoff_cards.yaml) | Comparisons the owner generated with consumer AI apps (Gemini and ChatGPT, Deep Research), not a controlled model benchmark. Prompts and settings are not recorded; see [data sources](docs/data-sources.md). Original [LLM evidence](predictions-from-llms/) is frozen. | | Simulated category marginals | Conditional on point strengths and assumed rules. Glicko rating-deviation uncertainty is not propagated. Map calibration does not establish card calibration. | Registered gates remain immutable. Diagnostics cannot promote, demote, or change the shipping card. Measurements live in producer artifacts rather than copied headline numbers. diff --git a/docs/audits/2026-09-27-release-candidate-verification.json b/docs/audits/2026-09-27-release-candidate-verification.json index 7710929..24537a8 100644 --- a/docs/audits/2026-09-27-release-candidate-verification.json +++ b/docs/audits/2026-09-27-release-candidate-verification.json @@ -381,7 +381,8 @@ "pip-audit resolved lock markers for this macOS host, so colorama (win32-only) was not queried; the 2026-09-26 supplement records it with no advisories.", "The frozen output oracle re-derives the historical gate refusals and failures as part of its replay; its final comparison is the result.", "Commits after candidate_sha may touch only this receipt and docs/release-checklist.md.", - "Historical receipts remain unchanged and do not attest later edits." + "Historical receipts remain unchanged and do not attest later edits.", + "Superseded on 2026-09-28 for commits after 3b1cb7c: see addendum.rule_revision." ], "recorded_at_utc": "2026-09-27T19:21:31Z", "addendum": { @@ -416,6 +417,11 @@ {"name": "verify-61c63f4a.log", "path": "/private/tmp/ti26-addendum/verify-61c63f4a.log", "sha256": "1293c8e3d01ef576d81e464420b065c4fb0d40b63f340f0c384c8407f43dbdd0"} ], "rule_after_addendum": "Commits after 3b1cb7c may touch only this receipt and docs/release-checklist.md.", + "rule_revision": { + "date": "2026-09-28", + "rule": "After 3b1cb7c, frozen paths must not change: `git diff --name-only 3b1cb7c HEAD -- src tests config data reports predictions-from-llms pyproject.toml uv.lock .github LICENSE .gitignore` stays empty. Documentation may change: README.md, CONTRIBUTING.md, SECURITY.md, CITATION.cff, mlsd-scorecard-ti26.md and docs/. Hosted CI must pass on every published or tagged commit, and the local-link check is re-run and recorded as an addendum before tagging.", + "reason": "The receipt-and-checklist-only rule contradicted the completion plan's release-docs step. This verification re-derives code, tests, config, data, reports, analyses, the lock and CI. Of documentation, it attests only local links, which the pre-tag check re-establishes. The revision widens what may change to documentation only." + }, "recorded_at_utc": "2026-09-27T20:01:35Z" } } diff --git a/docs/data-sources.md b/docs/data-sources.md index 0832f88..af3a012 100644 --- a/docs/data-sources.md +++ b/docs/data-sources.md @@ -4,9 +4,9 @@ | Material | Provenance and treatment | |---|---| -| [Raw snapshots](../data/raw/) | OpenDota responses with chunk hashes and acquisition manifests; Steam news is listed separately below. No blanket dataset redistribution license established. Public release held pending applicable terms or written permission. | -| [Rules evidence](../data/evidence/) and quoted rules | Valve/Steam sources and owner-imported rendered text. Preserve attribution and manifests. Public release held pending an applicable redistribution basis; see the source register below. | -| [External analyses](../predictions-from-llms/) and cards | Vitalii Babenko confirms generating the analyses using Gemini, ChatGPT, and other unspecified services. File-specific service/account details, generation settings, and copied-material status remain unconfirmed. Public release held pending that review. | +| [Raw snapshots](../data/raw/) | OpenDota responses with chunk hashes and acquisition manifests; Steam news is listed separately below. No blanket dataset license is established. An odota maintainer approved publishing them in [odota/core#2989](https://github.com/odota/core/issues/2989#issuecomment-5865765389); see the source register below. | +| [Rules evidence](../data/evidence/) and quoted rules | Valve/Steam sources and owner-imported rendered text. Preserve attribution and manifests. No permission from Valve; published with attribution and a takedown route as accepted risk under the [publication decision](#publication-decision). | +| [External analyses](../predictions-from-llms/) and cards | Vitalii Babenko generated both analyses on personal accounts: one in Google's Gemini app and one in OpenAI's ChatGPT app, each with Deep Research. The owner approved publishing both as they are on 2026-09-28; see the source register below. | | Derived reports | Repository-produced research, still subject to review of underlying data restrictions and incorporated quotations. | | Dependencies | Separate upstream licenses apply. Review dependency notices before distributing bundled dependencies. | @@ -18,23 +18,25 @@ Owner approval records a release decision; it cannot supply missing third-party | Source and retained material | Terms reviewed and attribution | Publication decision / missing evidence | |---|---|---| -| OpenDota API responses in timestamped `data/raw/` snapshots | [Core software MIT license](https://github.com/odota/core) does not establish API dataset rights. Retain OpenDota attribution, request provenance, acquisition manifests, and chunk hashes. Exact dataset attribution conditions remain unresolved. | **Hold.** Permission inquiry sent through the official Discord help channel; response pending. Ask whether underlying Valve restrictions apply to snapshots and player IDs. | -| [Dota rules page](https://www.dota2.com/esports/ti15/tirules): `data/evidence/imports/valve-group-stage-rendered.txt`, corresponding `data/evidence/rules/` captures, [archival text](ti26/2026-08-08-ti2026-rules-fetched.md), and [owner transcription](ti26/2026-08-08-published-format-rules.md) | Rendered-page capture/transcription, not an established Steam API response. Retain source URL, acquisition records, and embedded Valve copyright/trademark notice. [Valve legal page](https://www.valvesoftware.com/en/legal) restricts republication of its site material, but applicability to this Dota page is unconfirmed. | **Hold.** Identify governing Dota-page terms or obtain written permission for the retained verbatim text and duplicates. Steam API terms cannot be assumed to cover it. | -| [Steam announcement](https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1840944183772671): [raw response](../data/raw/steam-news/1840944183772671.json), `data/evidence/imports/valve-ti2026-format-rendered.txt`, and matching `format-rendered.txt` capture | Acquired through `ISteamNews/GetNewsForApp/v2`; metadata attributes author `krAnk0r`. Preserve title, author, source/feed metadata, links, and capture manifests. [Steam API terms](https://steamcommunity.com/dev/apiterms) condition distribution on the application/use and include privacy, branding, disclaimer, and termination obligations. [Subscriber Agreement](https://store.steampowered.com/subscriber_agreement/) also reserves rights. | **Hold.** Confirm whether API terms cover this endpoint and permanent public Git archival, and how their conditions would be met. No unconditional MIT grant established. | -| [Gemini-named analysis](../predictions-from-llms/gemini-3-1-pro.md) | Owner confirms generating the supplied analyses with AI services; filename is not verified model provenance. Retain AI-generation provenance disclosure and the works-cited list. [Google generative-AI terms](https://policies.google.com/terms/generative-ai) are a review starting point; account/product and applicable terms at generation remain unknown. | **Hold.** Record file-specific service/account provenance and whether prompts or outputs contain copied passages; review those sources separately. | -| [GPT-named analysis](../predictions-from-llms/gpt-5-6-xhigh.md) | Owner confirms generating the supplied analyses with AI services. Opaque citation markers lack persisted source URLs. [OpenAI terms](https://openai.com/policies/row-terms-of-use/) distinguish consumer, regional, and business use; output ownership does not grant third-party rights. Applicable account/region/version remains unknown. | **Hold.** Resolve citation sources, file-specific service/account provenance, and copied-material status. Preserve AI-generation provenance disclosure. | +| OpenDota API responses in timestamped `data/raw/` snapshots | [Core software MIT license](https://github.com/odota/core) does not establish API dataset rights. Retain OpenDota attribution, request provenance, acquisition manifests, and chunk hashes. The reply states no attribution conditions; OpenDota is credited, as the request committed. | **Permission granted** on 2026-09-28 by `howardchung` (odota organization member; top contributor to `odota/core`) in [odota/core#2989](https://github.com/odota/core/issues/2989#issuecomment-5865765389): "This sounds fine to me." It covers the request as described there: frozen OpenDota explorer responses, including match and player account IDs, published in a public repository, with OpenDota credited and removal on request. No conditions were stated. The request's question about Valve's terms was not answered. | +| [Dota rules page](https://www.dota2.com/esports/ti15/tirules): `data/evidence/imports/valve-group-stage-rendered.txt`, corresponding `data/evidence/rules/` captures, [archival text](ti26/2026-08-08-ti2026-rules-fetched.md), and [owner transcription](ti26/2026-08-08-published-format-rules.md) | Rendered-page capture/transcription, not an established Steam API response. Retain source URL, acquisition records, and embedded Valve copyright/trademark notice. [Valve legal page](https://www.valvesoftware.com/en/legal) restricts republication of its site material, but applicability to this Dota page is unconfirmed. | **Accepted risk** under the [publication decision](#publication-decision): published with attribution and a takedown route. The governing Dota-page terms were not identified, and no written permission exists for the retained verbatim text and duplicates. Steam API terms cannot be assumed to cover it. | +| [Steam announcement](https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1840944183772671): [raw response](../data/raw/steam-news/1840944183772671.json), `data/evidence/imports/valve-ti2026-format-rendered.txt`, and matching `format-rendered.txt` capture | Acquired through `ISteamNews/GetNewsForApp/v2`; metadata attributes author `krAnk0r`. Preserve title, author, source/feed metadata, links, and capture manifests. [Steam API terms](https://steamcommunity.com/dev/apiterms) condition distribution on the application/use and include privacy, branding, disclaimer, and termination obligations. [Subscriber Agreement](https://store.steampowered.com/subscriber_agreement/) also reserves rights. | **Accepted risk** under the [publication decision](#publication-decision): published with the Valve/Steam credit line and a takedown route. Whether the API terms cover this endpoint and permanent public Git archival remains unconfirmed. No unconditional MIT grant established. | +| [Gemini-named analysis](../predictions-from-llms/gemini-3-1-pro.md) | Generated by the owner in Google's Gemini app with Deep Research, on a personal account. The filename is not verified model provenance. Retain the AI-generation disclosure and the works-cited list. [Google generative-AI terms](https://policies.google.com/terms/generative-ai) are a review starting point; the terms version in force at generation is not recorded. | **Approved as-is** by the owner on 2026-09-28, as risk acceptance under the [publication decision](#publication-decision). | +| [GPT-named analysis](../predictions-from-llms/gpt-5-6-xhigh.md) | Generated by the owner in OpenAI's ChatGPT app with Deep Research, on a personal account. Opaque citation markers lack persisted source URLs. [OpenAI terms](https://openai.com/policies/row-terms-of-use/) distinguish consumer, regional, and business use; output ownership does not grant third-party rights. The region and terms version in force at generation are not recorded. | **Approved as-is** by the owner on 2026-09-28, as risk acceptance under the [publication decision](#publication-decision). The unresolved citation markers remain disclosed. | ### Publication decision -Recorded 2026-09-27 by Vitalii Babenko. If no permission response arrives by 2026-10-11, the owner publishes the full repository history on the terms below. This accepts residual legal risk; it does not establish permission. The owner re-posts the inquiry in a durable public channel before the deadline. A reply before then supersedes this decision. +Recorded 2026-09-27 by Vitalii Babenko. If no permission response arrives by 2026-10-11, the owner publishes the full repository history on the terms below. This accepts residual legal risk; it does not establish permission. The owner re-posted the inquiry publicly as [odota/core#2989](https://github.com/odota/core/issues/2989) on 2026-09-28. A reply before then supersedes this decision. + +Update 2026-09-28: OpenDota granted permission (see the register above), so the deadline no longer gates publication. Valve and Steam material is still published on the terms below as accepted risk. The owner decides when to publish. - **Attribution:** match data comes from [OpenDota](https://www.opendota.com). Rules text and the Steam news response are Valve material from [dota2.com](https://www.dota2.com) and [Steam](https://store.steampowered.com). Dota 2, The International and Steam are trademarks of Valve Corporation. This repository is not affiliated with or endorsed by Valve or OpenDota. - **Takedown:** a rights holder may request removal through the contact in [SECURITY.md](../SECURITY.md). Removal happens in a new commit, per the [release checklist](release-checklist.md). Earlier public history, forks and clones are not retracted. -- **Owner-generated analyses:** publishing full history publishes both files, so there is no per-file hold under this route. Before the deadline, the owner must either approve both files as they are or switch to a fresh-history repository. Their bytes are digest-bound, so any edit needs a new replay attestation, and history keeps the originals. +- **Owner-generated analyses:** the owner approved both as they are on 2026-09-28 (see the register above). Full-history publication includes them. Their bytes are digest-bound, so any edit needs a new replay attestation, and history keeps the originals. ### OpenDota inquiry -Sent by Vitalii Babenko through [OpenDota Discord — help](https://discord.com/channels/144629812982448128/421587769664471069), the contact server linked from the official core README. Delivery was observed in the channel; no permission response has been recorded. +Sent by Vitalii Babenko through [OpenDota Discord — help](https://discord.com/channels/144629812982448128/421587769664471069), the contact server linked from the official core README. Delivery was observed in the channel. On 2026-09-28 the owner re-posted the question as the GitHub issue [odota/core#2989](https://github.com/odota/core/issues/2989), which states the planned publication date. At 2026-09-28T07:52:54Z, `howardchung` replied there: ["This sounds fine to me."](https://github.com/odota/core/issues/2989#issuecomment-5865765389) > I maintain a reproducible Dota 2 research repository. May I publicly redistribute frozen historical API-response snapshots, including match and player account IDs, with source attribution? My original code is MIT-licensed; third-party data would retain its applicable terms. Please identify the terms or permission covering this use, including any underlying Valve restrictions. @@ -42,7 +44,7 @@ Record any reply with its source link, sender's authority, covered materials, co ## Identifiers -Match rows retain player account identifiers, match identifiers, timestamps, and team/roster links. These are pseudonymous, not anonymous. Roster identity depends on account sets; removing them changes reproducibility. Vitalii Babenko approves retaining player IDs for reproducibility, subject to source permission. Do not silently redact frozen evidence. +Match rows retain player account identifiers, match identifiers, timestamps, and team/roster links. These are pseudonymous, not anonymous. Roster identity depends on account sets; removing them changes reproducibility. Vitalii Babenko approves retaining player IDs for reproducibility, subject to source permission. OpenDota's permission covers the player account IDs named in the request; no position from Valve is recorded. Do not silently redact frozen evidence. Local author/committer review found the owner's personal Gmail, work-domain email, and GitHub noreply identity. Vitalii Babenko chose to preserve history and explicitly allowed the work email to remain public after reviewing removal's impact. Rewriting identities changes affected commits and descendants; historical verification resolves manifest-bound original Git revisions, so deleting those objects would break that verification. A `.mailmap` or new Git identity changes presentation/future commits, not stored historical addresses. The configured identity for new commits uses the approved personal Gmail. diff --git a/docs/release-checklist.md b/docs/release-checklist.md index 903c74d..2e314c0 100644 --- a/docs/release-checklist.md +++ b/docs/release-checklist.md @@ -1,6 +1,6 @@ # Release checklist -Owner: Vitalii Babenko. Scope: source-repository retrospective with limited maintenance. MIT is selected for code and original documentation. The existing GitHub repository is the candidate destination; no release version/tag is selected. +Owner: Vitalii Babenko. Scope: source-repository retrospective with limited maintenance. MIT is selected for code and original documentation. The existing GitHub repository is the candidate destination. The release version is `v0.1.0`; the tag is applied after the flip. ## Prepared work @@ -14,18 +14,26 @@ Owner: Vitalii Babenko. Scope: source-repository retrospective with limited main ## Publication blockers -- [ ] Establish redistribution basis for OpenDota snapshots, Valve/Steam material, and external analyses. The [source-permission register](data-sources.md) records source-specific holds and the sent OpenDota inquiry; owner approval cannot replace third-party rights. The owner's [publication decision](data-sources.md#publication-decision) sets a 2026-10-11 deadline. After it, the repository is published with attribution and a takedown route as accepted risk. +- [x] Establish a redistribution basis for each material; see the [source-permission register](data-sources.md). Owner approval cannot replace third-party rights, so each basis is stated separately: + - **OpenDota snapshots:** permission granted by an odota maintainer in [odota/core#2989](https://github.com/odota/core/issues/2989#issuecomment-5865765389) (2026-09-28). + - **External analyses:** approved as they are by the owner (2026-09-28). + - **Valve/Steam material:** no permission; published with attribution and a takedown route as accepted risk under the owner's [publication decision](data-sources.md#publication-decision). The 2026-10-11 deadline no longer gates publication. - [x] Owner approves retained player identifiers for reproducibility, subject to source permission. - [x] Owner reviews author-email exposure and chooses to preserve history, including the work email; new commits use the approved Gmail identity. - [x] Record the owner-confirmed private security email in SECURITY.md. - [x] Run history secret scans and locked-dependency advisory/license review; see the [history scan](audits/2026-09-26-publication-scan.json), [pre-commit scan](audits/2026-09-26-publication-candidate.json), and [supplement](audits/2026-09-26-publication-supplement.json). These receipts cover their recorded inputs; later edits need a new candidate receipt. Rights decisions remain open. -- [x] Verify the final candidate: [receipt](audits/2026-09-27-release-candidate-verification.json) for 0186447. It covers unfiltered tests, Ruff, all bundles, the postmortem replay, the full forecast oracle, a full-history secret scan, dependency advisories and local links (external URLs not checked). Its addendum verifies 3b1cb7c, an audit-record correction. Commits after 3b1cb7c may touch only the receipt and this checklist. -- [ ] Merge the release PR with "Create a merge commit". Squash or rebase would strand receipt-cited SHAs. +- [x] Verify the final candidate: [receipt](audits/2026-09-27-release-candidate-verification.json) for 0186447. It covers unfiltered tests, Ruff, all bundles, the postmortem replay, the full forecast oracle, a full-history secret scan, dependency advisories and local links (external URLs not checked). Its addendum verifies 3b1cb7c, an audit-record correction. Change rule after 3b1cb7c: + - **Frozen:** `git diff --name-only 3b1cb7c HEAD -- src tests config data reports predictions-from-llms pyproject.toml uv.lock .github LICENSE .gitignore` stays empty. + - **Documentation, may change:** README.md, CONTRIBUTING.md, SECURITY.md, CITATION.cff, mlsd-scorecard-ti26.md and docs/. + - **CI:** hosted CI must pass on every published or tagged commit. + - **History:** revised 2026-09-28. The earlier rule allowed only the receipt and this checklist to change, which contradicted the planned release-docs step. The revision widens what may change to documentation only. +- [ ] Before tagging: re-run the local-link check over all tracked markdown and record the result as a receipt addendum. +- [x] Merge the release PR with "Create a merge commit": [#38](https://github.com/vbabenko97/dota-2-analytics/pull/38) merged as 8cd4184, whose tree equals the PR head. Squash or rebase would have stranded receipt-cited SHAs. - [ ] Before the flip: update the repository description and topics; require approval for fork pull-request workflows from outside collaborators. - [ ] After the flip: enable secret scanning with push protection, Dependabot alerts (no update PRs; `uv.lock` is replay-bound), private vulnerability reporting, and a ruleset on `main` requiring Offline CI. - [ ] If a DOI is wanted: enable Zenodo after the flip and before the first GitHub Release. - [x] Verify a committed candidate in a clean full-history checkout; see the [SHA-bound receipt](audits/2026-09-26-committed-candidate-verification.json). This receipt and checklist completion update are local follow-up documentation outside the verified candidate commit. -- [ ] Select the release version/tag after candidate verification and rights clearance. +- [x] Select the release version: the owner chose `v0.1.0` on 2026-09-28, matching `pyproject.toml`. The tag is applied at release, after the flip. - [ ] Explicitly authorize push, visibility change, and publication. No history rewrite, deletion, push, tag, PR, visibility change, or publication is implied. The repository was private during read-only inspection. Redaction or history removal needs a separate impact review and owner decision. diff --git a/docs/release-notes.md b/docs/release-notes.md index 127ade8..785d33a 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -4,6 +4,6 @@ Prepared a research retrospective spanning Swiss forecasts, group evaluation, pl Changes add navigation, contributor/data policies, MIT code licensing, citation metadata, lock provenance for new releases, postmortem replay attestations, and offline CI. -The research conclusion is unchanged: temporal validation and provenance are strong; repeat-event predictive skill remains unproven. Postmortems share an event. External cards have incomplete generation provenance. Simulation conditions on point strengths without propagating rating-deviation uncertainty. +The research conclusion is unchanged: temporal validation and provenance are strong; repeat-event predictive skill remains unproven. Postmortems share an event. The owner generated the external cards with consumer AI apps. Prompts and settings are not recorded in this repository, and one file's citations do not resolve. Simulation conditions on point strengths without propagating rating-deviation uncertainty. Maintenance is retrospective only. See the [release checklist](release-checklist.md) for blockers and [reproduction guide](reproduce.md) for checks. No publication, tag, or redistribution clearance is claimed; the release checklist tracks hosted CI status. diff --git a/docs/superpowers/plans/2026-09-27-public-release-completion.md b/docs/superpowers/plans/2026-09-27-public-release-completion.md index e698a99..f0eecac 100644 --- a/docs/superpowers/plans/2026-09-27-public-release-completion.md +++ b/docs/superpowers/plans/2026-09-27-public-release-completion.md @@ -105,6 +105,16 @@ - **D6:** default applied in SECURITY.md. - **Still open:** D3, D5, D7, and the D6 settings themselves. +**Recorded 2026-09-28:** + +- PR #38 merged with a merge commit. +- **D3:** both analyses approved as they are. Provenance: Google's Gemini app and OpenAI's ChatGPT app, each with Deep Research, on the owner's personal accounts. +- **D5:** `v0.1.0`. +- **Change rule after verification:** replaced by a path-scoped rule that widens what may change to documentation only. Frozen paths are unchanged, and a pre-tag link check is added (see the release checklist). +- **OpenDota inquiry:** re-posted as [odota/core#2989](https://github.com/odota/core/issues/2989). +- **OpenDota reply:** `howardchung`, an odota member, replied "This sounds fine to me." Permission is recorded, and the D1 deadline no longer gates publication. Valve/Steam material stays accepted risk. +- **Still open:** D7 and the D6 settings. + Evidence for D1–D2 (non-authoritative; not legal advice): - The [YASP data dump](https://academictorrents.com/details/5c5deeb6cfe1c944044367d2e7465fd8bd2f4acf), from OpenDota's predecessor, is licensed CC BY-SA 4.0 and asks for attribution. @@ -303,4 +313,4 @@ Run in a fresh detached worktree or a full-history clone **outside `/tmp`**, bec ## Acceptance -The public `main` equals a tree verified by a committed receipt. Hosted CI is green on it. The replay-bound guard is in the suite. Every D-decision is recorded. The tag and the release exist. No frozen or replay-bound byte changed: `git diff 6ba2c0f -- reports data predictions-from-llms` is empty, and the Task 1 guard passes. +On the public `main`, every frozen path (see the release checklist) equals the tree verified by a committed receipt. Hosted CI is green on it. The local-link check was re-run and recorded after the last documentation change before the tag. The replay-bound guard is in the suite. Every D-decision is recorded. The tag and the release exist. No frozen or replay-bound byte changed: `git diff 6ba2c0f -- reports data predictions-from-llms` is empty, and the Task 1 guard passes. diff --git a/mlsd-scorecard-ti26.md b/mlsd-scorecard-ti26.md index 160ab4d..7523a52 100644 --- a/mlsd-scorecard-ti26.md +++ b/mlsd-scorecard-ti26.md @@ -23,6 +23,8 @@ **Top fix:** Close the third-party rights hold before publication ([data sources, publication decision](docs/data-sources.md#publication-decision), deadline 2026-10-11) — OpenDota permission is still pending, and the owner must approve both analyses in `predictions-from-llms/` (or switch to a fresh-history repository) before the deadline, because full-history publication leaves no per-file hold. +**Status (2026-09-28):** the owner approved both analyses as they are. OpenDota granted permission in [odota/core#2989](https://github.com/odota/core/issues/2989#issuecomment-5865765389). Valve/Steam material remains accepted risk with attribution and a takedown route. The grades above are unchanged. + **Takeaway:** Reproducibility makes a forecast inspectable; repeated independent evaluation is still needed to establish predictive skill. Evidence mode: docs and repository, including later TI postmortems, a full git diff of every rubric-relevant path across the two reviewed revisions, one independently re-executed mutation check, and one read-only hosted-CI status query. External LLM cards are frozen inputs, so a live LLM/RAG/agent-system row is inapplicable. Grades are reviewer judgments, not empirical measurements; their arithmetic, reviewed revision, file hashes, checks, and limits are recorded in the [re-review evidence](docs/audits/2026-09-27-release-design-rereview.json), which supersedes the [prior review evidence](docs/audits/2026-09-26-public-release-review.json) (reviewed an earlier revision, before the public-release documentation existed). Findings and the staged remedy are in the [release proposal](docs/public-release-plan.md) and the [completion plan](docs/superpowers/plans/2026-09-27-public-release-completion.md). No implementation or publication approval is implied.