Skip to content

Add CITATION.cff and the 2026 Zenodo DOI - #124

Merged
rocklambros merged 4 commits into
mainfrom
feat/zenodo-citation
Aug 26, 2026
Merged

rocklambros merged 4 commits into
mainfrom
feat/zenodo-citation

Conversation

@stefanoamorelli

@stefanoamorelli stefanoamorelli commented Jul 26, 2026 •

Copy link
Copy Markdown
Collaborator

The 2026 edition is archived on Zenodo as record 22109015. CITATION.cff and the README now carry the concept DOI, 10.5281/zenodo.22109014, which always resolves to the newest edition and is the one to cite; the version DOI for 2026 alone is 10.5281/zenodo.22109015. GitHub renders the same metadata under "Cite this repository".

The author list follows the Acknowledgements page of the published PDF rather than this repository, since that page is the authoritative credit for an edition. Checking one against the other added two entry leads who were missing here, Anitha Dakamarri (LLM05) and Venkata Sai Kishore Modalavalasa (LLM06), corrected Gavin Klondike and Savio Dsouza, and reordered the list to match the page: project leads, then entry leads LLM01 to LLM10. Five authors now carry an ORCID, and the rest are welcome to add theirs.

The release workflow is dropped for now. Depositing from CI means keeping a Zenodo API token in a repository secret, and that token publishes under the identity of whoever minted it, so it is worth settling who owns the record first. The deposit script, its config and its tests all stay, and documentation/ZENODO.md covers running them by hand against the existing record.

@stefanoamorelli stefanoamorelli changed the title Add CITATION.cff and automated Zenodo deposit for the 2026 edition Add CITATION.cff and automated Zenodo deposit Jul 26, 2026
Papers already cite the Top 10 (NVIDIA's garak paper cites the first
version), but with no DOI there is no identifier for Crossref, OpenAIRE
or Google Scholar to resolve against, so finding that use means manual
searching. This adds citation metadata and a workflow that deposits each
edition to Zenodo, which mints the DOI.

CITATION.cff carries the authors of the 2026 entries. GitHub renders it
as "Cite this repository", and the release workflow builds the Zenodo
record from it, so the author list is maintained in one place rather
than two.

.github/zenodo.config.json holds the file list for each deposit.
`{edition}` comes from the release tag, so the 2027 edition needs no
config change. Each key under `deposits` is a separate Zenodo record,
so any future publication from this repo can take its own concept DOI
instead of becoming a version of the Top 10.

.github/scripts/zenodo_release.py builds the bundle and deposits it
through the Zenodo REST API, with `--check` to validate the config and
CITATION.cff, `--dry-run` to print the file list, and `--sandbox` to
target sandbox.zenodo.org. The workflow deposits on release and runs the
first two on pull requests that touch these files. documentation/ZENODO.md
covers setup and the release steps, and the README gains a citation
section and a DOI badge, commented out until the first release supplies
the number.

The 2026 deposit holds 22 files (2.6 MB): the ten entries, the preface,
the AISVS appendix, three published figures and CITATION.cff. It
excludes the 2025 list, 2026/working/ and the diagram tool's scratch
files under report/images/screenshots and report/images/uploads.

Editions are versions of one Zenodo record, each with its own DOI under
a concept DOI that resolves to the newest. Cite the concept DOI. This
follows DataCite's versioning guidance for major revisions of a work.

I did not use Zenodo's GitHub integration. It archives the whole
repository tarball at the tag with no include/exclude option, so the
2026 DOI would ship the 2025 list and the working drafts, and it
supports only one record per repository.

Four values are still missing and marked TODO where they belong: the
real publication date, which stands at the sprint plan's 2026-06-15
target; the contributor list, for reviewers and editors who are not
credited as entry authors; the concept DOI, which Zenodo issues at the
first publish; and author ORCIDs, which are optional.

Publishing also needs the ZENODO_TOKEN secret, and the
ZENODO_CONCEPT_RECID variable once the first publish reports it.
Without the latter, each release mints an unrelated record and the
citation history splits.

Signed-off-by: Stefano Amorelli <stefano@amorelli.tech>
The deposit path had no test coverage: every assertion so far came from
--check and --dry-run, which never issue a request.

test_zenodo_release.py stubs the requests module and runs both paths,
the first deposition and a new version of an existing concept record.
It asserts the call sequence, that inherited files are removed before
the new bundle is uploaded, that draft mode skips the publish action,
and that --sandbox never reaches the production host. It also pins the
metadata shape: creators as 'Family, Given', publication_date as
ISO8601, and no 569JNRXghklprefixed config comments leaking into the payload.

Confirmed cc-by-sa-4.0 against Zenodo's live licence vocabulary. The
/api/licenses path now redirects to /api/vocabularies/licenses.

The tests stub the API, so they do not prove Zenodo accepts the
metadata. That still needs a sandbox run with a real token.

Signed-off-by: Stefano Amorelli <stefano@amorelli.tech>
@stefanoamorelli

Copy link
Copy Markdown
Collaborator Author

cc @rocklambros

@rocklambros

Copy link
Copy Markdown
Collaborator

@stefanoamorelli Thanks for putting this together. The version-DOI-under-a-concept-DOI structure is the right model, and skipping Zenodo's native GitHub integration was the correct call given it would archive the whole tarball including the 2025 list and the working drafts.

Holding this for now. The 2026 content is locked ahead of release, and we do not want a deposit workflow pointed at a moving target. For the first edition we will likely do the Zenodo deposit manually once the document is released, then come back and wire up the automation with the concept record ID already in hand. That removes the risk you flagged of a first automated run splitting the citation history before ZENODO_CONCEPT_RECID is set.

Leaving this open as a draft. I will follow up here once the release date is fixed. If we go the manual route for 2026, CITATION.cff and .github/zenodo.config.json are still what we would build the record from, so the work carries over either way.

@Mayur021

Mayur021 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

One thing worth confirming before the tag. The reason given for rehearsing on sandbox, that Zenodo cannot delete a published record, applies a step earlier too: to what is actually in 2026/final/ on the branch that gets tagged.

Comparing main against release-candidate-1 as they stand right now:

  • main has 12 files in 2026/final/, release-candidate-1 has 15
  • Three exist only on the release branch: Appendix_A_Related_Framework_Mappings.md (74,693 bytes), Appendix_B_LLM_Application_Architecture_and_Threat_Model.md, and references.md (34,564 bytes)
  • 14 of the 15 differ in size, including every entry file

Where I spot-checked, the published v1.0 PDF matches the release branch rather than main. LLM03's mitigation 7 in the PDF reads "permits low-consequence or easily reversible actions to auto-approve, while high-consequence or irreversible ones route to human review", with the store-credit and external-payout example. main still carries "permits low-consequence actions to auto-approve while high-consequence ones route to human review" and the older refund-threshold example.

If that reading is right, tagging main as it stands would mint a version DOI over a tree that does not contain the AISVS appendix this PR lists as in scope, and whose ten entries differ from the document people are downloading today.

Happy to be wrong about which branch is canonical here. The file counts and sizes are checkable either way, and the point only matters because a version DOI freezes whatever is there.

Signed-off-by: Stefano Amorelli <stefano@amorelli.tech>

# Conflicts:
#	README.md
@stefanoamorelli
stefanoamorelli marked this pull request as ready for review August 9, 2026 11:47
@stefanoamorelli

Copy link
Copy Markdown
Collaborator Author

@rocklambros updated following the latest release details.

@Mayur021 this should be addressed now 🙏

@Mayur021

Mayur021 commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Confirmed. main now carries all 17 files in 2026/final/ at the same sizes as release-candidate-1, including Appendix_A_Related_Framework_Mappings.md, references.md and the AISVS appendix. LLM03's mitigation 7 on main now matches the published PDF, store credit and external payout example included. Nothing diverging that I can see.

One thing I checked while I was in there, since it only matters once a record is immutable. Appendix_B_LLM_Application_Architecture_and_Threat_Modeling.md is a single img tag pointing at report/images/OWASP LLM Threat Model 2026.png. That file is present and the bundle excludes only strip report/images/uploads and report/images/screenshots, so it should carry through. Still worth an eyeball on the sandbox run, because that appendix is nothing but the image.

Thanks for turning it around.

@Mayur021

Mayur021 commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

One casualty of the conflict resolution. 6c8d717 dropped the whole ## Citation section from the README, so main...feat/zenodo-citation is now just the three commented-out DOI badge lines. The "cite the concept DOI, which resolves to the newest edition" line went with it.

CITATION.cff still drives the "Cite this repository" sidebar by itself, so the machine-readable half is fine. The part worth restoring is the concept-versus-version instruction, since that is the one thing a citing author gets wrong by default.

@Mayur021

Copy link
Copy Markdown
Contributor

On the open item in the description, adding contributors who are not credited as entry authors. If that pass is still happening before the tag, I would ask to be included.

PR #117, merged into release-candidate-1 on 16 July, is the reversibility wording in LLM03 mitigation 7, the store credit versus external payout example that is in the published PDF and now on main.

Raising it only because the description already says each version keeps the author list it was published with, so this edition's list freezes at the tag. If the intent is entry authors only, that is a fine answer and I will not push it.

The 2026 edition is archived at https://zenodo.org/records/22109015. Both
`CITATION.cff` and the README now carry the concept DOI,
10.5281/zenodo.22109014, which always resolves to the newest edition; the
version DOI for 2026 alone is 10.5281/zenodo.22109015.

The author list follows the Acknowledgements page of the published PDF rather
than this repository. That page is the authoritative credit for an edition, and
checking it added two entry leads who were missing here, Anitha Dakamarri
(LLM05) and Venkata Sai Kishore Modalavalasa (LLM06), corrected Gavin Klondike
and Savio Dsouza, and reordered the list to match: project leads, then entry
leads LLM01 to LLM10. Five authors now carry an ORCID.

The release workflow is removed. Running the deposit from CI means holding a
Zenodo API token in a repository secret, and that token publishes under the
identity of whoever minted it, so it should wait until the project decides who
owns the record. The script, its config and its tests stay, and
`documentation/ZENODO.md` covers running them by hand.

Signed-off-by: Stefano Amorelli <stefano@amorelli.tech>
@stefanoamorelli stefanoamorelli changed the title Add CITATION.cff and automated Zenodo deposit Add CITATION.cff and the 2026 Zenodo DOI Aug 26, 2026
@stefanoamorelli

stefanoamorelli commented Aug 26, 2026 •

Copy link
Copy Markdown
Collaborator Author

@rocklambros I I published the 2026 record on Zenodo by hand, the route you laid out above: https://zenodo.org/records/22109015

The concept DOI is 10.5281/zenodo.22109014, which always resolves to the newest edition and is the one to cite. The version DOI for 2026 alone is 10.5281/zenodo.22109015. The record carries the published PDF alongside a source bundle built from origin/main at 7bbe0f0, so every file in it exists at a public commit.

This PR is updated to match. CITATION.cff and the README now carry the concept DOI, and the author list follows the Acknowledgements page of the published PDF (treated as the single source of truth).

cc @virtualsteve-star

@rocklambros rocklambros left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and verified against the live Zenodo record.

Record 22109015 matches CITATION.cff field for field: version DOI, concept DOI 10.5281/zenodo.22109014, publication date 2026-08-04, version 2026, license cc-by-sa-4.0, upload type publication/report, and all 20 creators in order with the same five ORCIDs. The README badge resolves to a real record.

Checks run:

  • zenodo_release.py --check passes
  • test_zenodo_release.py passes, 22 assertions, API stubbed, no network
  • semgrep p/python, p/security-audit, p/secrets on .github/scripts/: 0 findings
  • no workflow file added, so the script stays inert and no Zenodo token enters repository secrets
  • no 2026/ or 2025/ content touched, so the release content lock holds
  • all four commits carry valid signatures

Dropping the release workflow was the right call. Settling record ownership before a token lands in a repository secret is the sticking point, and documentation/ZENODO.md states that plainly.

Two follow-ups, neither blocking:

  1. deposit() passes the token as an access_token query parameter. Query strings land in server access logs and proxy logs. Zenodo accepts Authorization: Bearer, which keeps the token out of the URL. Worth switching before this ever runs from CI.
  2. The bucket upload attaches the token to a URL taken from the API response (draft['links']['bucket']). The API base is hardcoded, so this rests on trusting Zenodo rather than on an open redirect, but a host check before attaching the credential would close it.

Merging with admin rights. This bypasses the second required review, which I am accepting because the executable additions never run automatically and the citation metadata mirrors a record that is already public.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants