Migrate research figures off dibs-web01.vm.duke.edu (closes #33)#38
Merged
Conversation
Same file already exists locally; just swap the dibs-web01 URL for the local path. One down, thirteen to go.
Stdlib-only Python script. Tries the live dibs-web01 host first (in case it's reachable for the runner), falls back to the most recent Wayback Machine snapshot via archive.org's API. Saves files into organized subdirectories under images/ matching the original URL categories (website/, fmri/, vocal/, zebrafish/). I (Claude) can't run this from my sandbox — outbound to all the relevant hosts is firewalled. A maintainer needs to run it from a machine with internet access, then commit the resulting images/ files. After that, a separate commit will update source files to point at the local paths and the dibs-web01 exclude can come out of lychee.toml.
Reverts 768d5bc — the local images/john.png is out of date; the current dibs-web01 version is what should be on the site. Adds john.png to the recovery script's URL list, removes the stale local file, and reverts people.html to point at the dibs-web01 URL (it will get swapped to /images/john.png in the URL-replacement step after the script runs). Also removes the "skip if dest exists" guard from the recovery script so it always fetches fresh — relevant for john.png and useful in general if someone needs to re-run.
13 references updated across 6 files: - location.md: 4 building entrance photos (-> /images/location/) - people.html: john, Trevor, shiyang - research.md: colorFish, pipelineNewpng3, vae_finch, website_VAEGAM_fig - _layouts/home.html: DUSOM_Dept_Neurobio_stack - _posts/2019-7-26-huang-poster.md: raymond_poster - _posts/2018-12-5-incubator-award.md: pipelineNewpng3 The 14th reference (ava_preprint.png in 2019-10-29-vae-preprint.md) is not changed — that image wasn't recoverable from dibs-web01 or Wayback; handled in the next commit.
ava_preprint.png didn't recover from dibs-web01 or Wayback. Removed the broken <img> tag from the post and left an HTML comment noting the gap so it's easy to restore if a copy of the figure surfaces. The post still reads fine without the figure. Removed the dibs-web01 exclude from lychee.toml — no remaining references on the active site, and we want lychee to catch any future regressions to that domain.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #33.
Summary
dibs-web01.vm.duke.eduimage URLs acrosslocation.md,people.html,research.md,_layouts/home.html, and 2 blog posts with local/images/paths.images/(4 inimages/location/, 9 at the top level) — compressed via the Phase 1 pre-commit hooks.raymond_poster.jpgandwebsite_VAEGAM_fig.pngwere still over the 1 MB cap afterjpegoptim --max=85; both added to.image-size-overrideswith justification.scripts/recover_dibs_web01.py— stdlib-only Python that tries the live host then falls back to the Wayback Machine. Used to do the recovery; kept in the repo as documentation / for re-runs if more images surface.dibs-web01\.vm\.duke\.edufromlychee.toml'sexcludelist now that there's nothing left to point at it.Not recovered
ava_preprint.png(referenced in_posts/2019-10-29-vae-preprint.md) wasn't on the live host or in Wayback. Removed the broken<img>tag and left an HTML comment in the post noting the gap — easy to restore if a copy of the figure surfaces later (e.g., from Jack or the bioRxiv PDF).Test plan
bundle exec jekyll buildproduces a clean_site/lychee --config ./lychee.toml --no-progress --root-dir _site --exclude-path _site/blog _sitereports 0 errors locallypre-commit,site-health) pass on the PRFollow-ups worth tracking separately (low priority)
raymond_poster.jpg(currently 2 MB @ 4032×3024 — way bigger than needed for web display) andwebsite_VAEGAM_fig.png(1.7 MB), then remove the override entries.ava_preprint.pngif a copy is locatable.https://claude.ai/code/session_01S5QXfkxZBNSAf2Y1XAD8H7
Generated by Claude Code