Skip to content

Clean data plus citations - #210

Merged
mccalluc merged 32 commits into
mainfrom
clean-data-plus-citations
Sep 11, 2026
Merged

mccalluc merged 32 commits into
mainfrom
clean-data-plus-citations

Conversation

@mccalluc

@mccalluc mccalluc commented Jul 31, 2026 •

Copy link
Copy Markdown
Collaborator

Questions:

  • Do we need a convention for the keys? I'm not sure that we do... but at the same time these may be visible to the end-user in the in-text short citations. "Author-year" doesn't really work:
    • Either no explicit author, or too many authors
    • There may be no explicit year
    • Often we have a pair of similarly named docs: The titles/years/authors don't distinguish them, but one is a corporate blog post, and one is a mathy pdf... and so I've ended up with keys like blog and paper.
  • Do we need a place for comments about a particular source? Where there extra elements, I've move them into yaml # comments: not going to be visible in the UI.
  • I've been very conservative in filling in the fields. There are some (like year) where we could make a guess, or I could add DOIs for all the arxive ones, but I'm not sure that we gain much by doing that.
  • I ended up using "Formal publication by the deploying organization" on the vast majority, but I wonder if for the bloggy ones, "Authorized organizational attestation" is what you want. What do we want the user to get from this distinction?
  • I was using the bibtext doc types earlier, but then realized I was using misc for almost everything, so I dropped it. It's another case where it feels like there's a choice which invites woodshedding, but won't make any difference to the user.
  • I've pretty much copied author lists as-is. While we could certainly prescribe a format, I think it would be work that doesn't add value for the user.

@mccalluc
mccalluc requested a review from tercer July 31, 2026 18:27
@mccalluc mccalluc mentioned this pull request Jul 31, 2026
mccalluc and others added 5 commits August 3, 2026 17:23
* copy citations from #209

* translate 3 records

* two more

* three more

* finish census

* checkpoint: some covid projects

* five more

* five more

* three more

* three more

* three more

* wikipedia!

* drop sources

* drop other options beside misc

* misc -> citation
@mccalluc

Copy link
Copy Markdown
Collaborator Author

Do we need a place for comments about a particular source? Where there extra elements, I've move them into yaml # comments: not going to be visible in the UI.

James thinks it's ok to add a comment field. Doesn't need to block this PR.

@mccalluc

Copy link
Copy Markdown
Collaborator Author

I ended up using "Formal publication by the deploying organization" on the vast majority, but I wonder if for the bloggy ones, "Authorized organizational attestation" is what you want. What do we want the user to get from this distinction?

This came from Damien, where there's been verbal confirmation.

@tercer tercer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Terrific

* add a check for global uniqueness

* Make the tags globally unique

* finish rename
Base automatically changed from 195-trust-assumptions-and-last-bits to main September 11, 2026 17:12
@mccalluc
mccalluc merged commit 4a2c42e into main Sep 11, 2026
1 check passed
@mccalluc
mccalluc deleted the clean-data-plus-citations branch September 11, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use bibtex for citations

2 participants