diff --git a/_posts/2018-12-5-incubator-award.md b/_posts/2018-12-5-incubator-award.md
index 845ea0f..0484148 100644
--- a/_posts/2018-12-5-incubator-award.md
+++ b/_posts/2018-12-5-incubator-award.md
@@ -5,7 +5,7 @@ post_title: "DIBS incubator award"
author: John Pearson
category: blog
---
-
+
[We won an Incubator Award](https://dibs.duke.edu/news/2018-2019-research-incubator-awards-announced) from the Duke Institute for Brain Sciences for Anne's work in collaboration with [Eva Naumann](https://www.neuro.duke.edu/people/faculty/eva-naumann) on real-time analysis of zebrafish data. See more information on our [research page](https://pearsonlab.github.io/research.html#work-in-progress-real-time-analysis-of-neural-data).
diff --git a/_posts/2019-10-29-vae-preprint.md b/_posts/2019-10-29-vae-preprint.md
index 95bacff..944ccc3 100644
--- a/_posts/2019-10-29-vae-preprint.md
+++ b/_posts/2019-10-29-vae-preprint.md
@@ -5,7 +5,9 @@ post_title: "New preprint on bioRxiv!"
author: Anne Draelos
category: blog
---
-
+
Jack's work on latent space analyses for animal vocalizations is [up on the bioRxiv](https://www.biorxiv.org/content/10.1101/811661v2)! Autoencoded Vocal Analysis, the Python package used to generate, plot, and analyze latent features, is freely available [on github](https://github.com/jackgoffinet/autoencoded-vocal-analysis). For more information on this project, feel free to check out our research page.
diff --git a/_posts/2019-7-26-huang-poster.md b/_posts/2019-7-26-huang-poster.md
index 87390a4..5de72b8 100644
--- a/_posts/2019-7-26-huang-poster.md
+++ b/_posts/2019-7-26-huang-poster.md
@@ -5,7 +5,7 @@ post_title: "Huang Fellows summer poster session"
author: Anne Draelos
category: blog
---
-
+
Congratulations to Raymond Chen for his excellent work this summer, as well as his successful presentation during the Huang Fellows summer poster session. We see great things in his future research endeavors!
diff --git a/images/DUSOM_Dept_Neurobio_stack.jpg b/images/DUSOM_Dept_Neurobio_stack.jpg
new file mode 100644
index 0000000..5798be1
Binary files /dev/null and b/images/DUSOM_Dept_Neurobio_stack.jpg differ
diff --git a/images/Trevor.jpg b/images/Trevor.jpg
new file mode 100644
index 0000000..4cc4baa
Binary files /dev/null and b/images/Trevor.jpg differ
diff --git a/images/colorFish.png b/images/colorFish.png
new file mode 100644
index 0000000..e7d7c3d
Binary files /dev/null and b/images/colorFish.png differ
diff --git a/images/john.png b/images/john.png
index f3d83e7..e8a5c9f 100644
Binary files a/images/john.png and b/images/john.png differ
diff --git a/images/location/admin_ent.jpg b/images/location/admin_ent.jpg
new file mode 100644
index 0000000..a54590c
Binary files /dev/null and b/images/location/admin_ent.jpg differ
diff --git a/images/location/bryan_ent.jpg b/images/location/bryan_ent.jpg
new file mode 100644
index 0000000..0694b5d
Binary files /dev/null and b/images/location/bryan_ent.jpg differ
diff --git a/images/location/bryan_ext.jpg b/images/location/bryan_ext.jpg
new file mode 100644
index 0000000..d2bd673
Binary files /dev/null and b/images/location/bryan_ext.jpg differ
diff --git a/images/location/ctn_ent.jpg b/images/location/ctn_ent.jpg
new file mode 100644
index 0000000..8134427
Binary files /dev/null and b/images/location/ctn_ent.jpg differ
diff --git a/images/pipelineNewpng3.png b/images/pipelineNewpng3.png
new file mode 100644
index 0000000..7c505ca
Binary files /dev/null and b/images/pipelineNewpng3.png differ
diff --git a/images/raymond_poster.jpg b/images/raymond_poster.jpg
new file mode 100644
index 0000000..3d1e096
Binary files /dev/null and b/images/raymond_poster.jpg differ
diff --git a/images/shiyang.jpg b/images/shiyang.jpg
new file mode 100644
index 0000000..9b861dc
Binary files /dev/null and b/images/shiyang.jpg differ
diff --git a/images/vae_finch.png b/images/vae_finch.png
new file mode 100644
index 0000000..1388f8d
Binary files /dev/null and b/images/vae_finch.png differ
diff --git a/images/website_VAEGAM_fig.png b/images/website_VAEGAM_fig.png
new file mode 100644
index 0000000..2578796
Binary files /dev/null and b/images/website_VAEGAM_fig.png differ
diff --git a/location.md b/location.md
index 859a4c8..32f5703 100644
--- a/location.md
+++ b/location.md
@@ -6,16 +6,16 @@ nav: Find Us # what shows up in the navbar at the top (do not define if you don'
We're located in the [Bryan Research Building](https://maps.duke.edu/?focus=68). This is a view of it from across Research Drive:
-
+
If you walk toward the building from either side, you'll find the entrance inside the breezeway in the middle:
-
+
Inside the lobby, turn right and enter the department offices:
-
+
Head toward the back, where you'll find the Center for Theoretical Neurobiology. The lab and John's office are inside.
-
+
diff --git a/lychee.toml b/lychee.toml
index 5c16f0e..c373d1d 100644
--- a/lychee.toml
+++ b/lychee.toml
@@ -33,14 +33,10 @@ accept = [200, 204, 206, 401, 403, 405, 429, 999]
# HTTP-checking them only tells us about the *currently deployed*
# site, not our build. Internal page-to-page links should be relative
# anyway (lychee resolves them against the local _site/ directory).
-# - dibs-web01.vm.duke.edu: known dead/flaky external image host. The
-# migration into the repo is tracked as Phase 5; remove this exclude
-# once that migration lands so any new dead URLs are caught.
# - maxcdn.bootstrapcdn.com: stable enough that transient 5xx responses
# shouldn't fail the build.
exclude = [
"pearsonlab\\.github\\.io",
- "dibs-web01\\.vm\\.duke\\.edu",
"maxcdn\\.bootstrapcdn\\.com",
# localhost references in README.md (jekyll-readme-index copies
# README into _site/ so lychee scans it).
diff --git a/people.html b/people.html
index 3708d17..b7915ac 100644
--- a/people.html
+++ b/people.html
@@ -45,7 +45,7 @@
Principal Investigator:
{% include person.html
- image="https://dibs-web01.vm.duke.edu/pearson/assets/images/website/john.png"
+ image="/images/john.png"
site="http://jmxpearson.com" github="https://github.com/jmxpearson"
email="john.pearson@duke.edu"
linkedin="https://www.linkedin.com/in/jmxpearson" name="John Pearson"
@@ -66,7 +66,7 @@
Graduate Students:
{%
- include person.html image="https://dibs-web01.vm.duke.edu/pearson/assets/images/website/Trevor.jpg"
+ include person.html image="/images/Trevor.jpg"
github="https://github.com/Trevorules" email="trevor.alston@duke.edu"
linkedin="www.linkedin.com/in/trevor-alston" name="Trevor Alston" desc="Trevor is a sixth-year Neurobiology graduate student at Duke University.
He holds two Biomedical Engineering Degrees; he received his BS degree from Rutgers University and his MS degree from Northwestern University.
@@ -95,7 +95,7 @@
Graduate Students:
name="David St-Amand" title="Neurobiology Graduate Student" desc="David is a third-year graduate student in Neurobiology. He graduated from McGill University, where he received his BS in Psychology and his MS in Neuroscience. His research investigates how efficient coding models can help us understand how information is computed in the retina. Outside the lab, he likes to work out and play games.
" %}
{%
- include person.html image="https://dibs-web01.vm.duke.edu/pearson/assets/images/website/shiyang.jpg"
+ include person.html image="/images/shiyang.jpg"
github="https://github.com/WaAaaAterfall" email="shiyang.pan@duke.edu"
linkedin="https://www.linkedin.com/in/shiyangpan"
site="https://waaaaaterfall.github.io"
diff --git a/research.md b/research.md
index 5dec431..e8920be 100644
--- a/research.md
+++ b/research.md
@@ -10,7 +10,7 @@ nav: Research
Together with [Eva Naumann's](https://www.neuro.duke.edu/research/faculty-labs/naumann-lab) lab, we've developed *[improv](https://github.com/project-improv/improv)* ([paper](https://www.biorxiv.org/content/10.1101/2021.02.22.432006v1)), a software platform for designing and orchestrating adaptive experiments. By analyzing data in real time, we can measure, model, and manipulate neural activity in response to new data. We've shown how these tools, in conjunction with holographic photostimulation, could in principle map functional connectivity of large circuits in a few hours ([paper](https://proceedings.nips.cc/paper/2020/file/531d29a813ef9471aad0a5558d449a73-Paper.pdf), [expanded version](https://arxiv.org/abs/2007.13911)). More recently, we've worked on methods for fast dimensionality reduction and modeling of neural populations in real time ([paper](https://arxiv.org/abs/2108.13941)).
-
A whole zebrafish brain activity map, showing the distribution of motion-sensitive neurons, color-coded to show the preferred motion direction.
@@ -18,7 +18,7 @@ Together with [Eva Naumann's](https://www.neuro.duke.edu/research/faculty-labs/n
-
Concept for the closed-loop pipeline. Neural data from the zebrafish are collected in the form of images, preprocessed, and analyzed in real-time. Targets for optical stimulation are then chosen based on the results of this analysis, creating adaptive experiments that test causal hypotheses.
@@ -31,7 +31,7 @@ Vocalization is a complex behavior that underlies vocal communication and vocal
-
a. Syllable VAE: Segmented syllables from adult Zebra Finch song are projected to a low-dimensional space, then reconstructed from that space using a VAE. b. Shotgun VAE: The VAE is trained on 20ms segments of adult Zebra Finch song to model variability on a millisecond duration. Visualized are songs projected into the latent space using these shorter segments.
@@ -53,7 +53,7 @@ In recent work, we've explored the idea of using variational autoencoder (VAE) m
-
A) VAE-GAM Model Schematic: brain volumes with signal of interest are compressed to a lower dimensional representation using encoder network. Sampled latents are then fed through decoder network to yield a base map and separate spatial effect maps. Each effect map is scaled by a potentially non-linear gain modelled using a Gaussian Process. Variance is modeled separately on a per voxel basis. B) Sample Effect Maps for VAE-GAM and GLM: effect maps for a visual stimulation task dataset analysed using the proposed VAE-GAM approach (top) vs. the tranditional (GLM) approach.
diff --git a/scripts/recover_dibs_web01.py b/scripts/recover_dibs_web01.py
new file mode 100755
index 0000000..6c01cba
--- /dev/null
+++ b/scripts/recover_dibs_web01.py
@@ -0,0 +1,149 @@
+#!/usr/bin/env python3
+"""Download images currently hosted on dibs-web01.vm.duke.edu into the repo.
+
+Tracked by issue #33. Run from the repo root:
+
+ python3 scripts/recover_dibs_web01.py
+
+For each URL, tries the live host first, then falls back to the most
+recent Wayback Machine snapshot. Files land under images/ with the
+target path shown below. After this finishes, commit the new images
+and a separate URL-replacement commit will swap the source files to
+the local paths.
+
+Exit code is the number of URLs that couldn't be recovered.
+"""
+
+from __future__ import annotations
+
+import os
+import sys
+import time
+import urllib.request
+import urllib.error
+import urllib.parse
+
+
+# Mapping of dibs-web01 URLs to the local path we want them at.
+# Picked to mirror the categorization in the URL itself.
+URLS: dict[str, str] = {
+ # Building entrances (location.md)
+ "http://dibs-web01.vm.duke.edu/pearson/assets/images/website/admin_ent.jpg":
+ "images/location/admin_ent.jpg",
+ "http://dibs-web01.vm.duke.edu/pearson/assets/images/website/bryan_ent.jpg":
+ "images/location/bryan_ent.jpg",
+ "http://dibs-web01.vm.duke.edu/pearson/assets/images/website/bryan_ext.jpg":
+ "images/location/bryan_ext.jpg",
+ "http://dibs-web01.vm.duke.edu/pearson/assets/images/website/ctn_ent.jpg":
+ "images/location/ctn_ent.jpg",
+ # People photos (people.html)
+ "https://dibs-web01.vm.duke.edu/pearson/assets/images/website/john.png":
+ "images/john.png",
+ "https://dibs-web01.vm.duke.edu/pearson/assets/images/website/Trevor.jpg":
+ "images/Trevor.jpg",
+ "https://dibs-web01.vm.duke.edu/pearson/assets/images/website/shiyang.jpg":
+ "images/shiyang.jpg",
+ # Duke School of Medicine Neurobio Dept logo (_layouts/home.html)
+ "https://dibs-web01.vm.duke.edu/pearson/assets/images/website/DUSOM_Dept_Neurobio_stack.jpg":
+ "images/DUSOM_Dept_Neurobio_stack.jpg",
+ # Research figures (research.md)
+ "https://dibs-web01.vm.duke.edu/pearson/assets/images/fmri/website_VAEGAM_fig.png":
+ "images/research/website_VAEGAM_fig.png",
+ "https://dibs-web01.vm.duke.edu/pearson/assets/images/vocal/vae_finch.png":
+ "images/research/vae_finch.png",
+ "https://dibs-web01.vm.duke.edu/pearson/assets/images/zebrafish/colorFish.png":
+ "images/research/colorFish.png",
+ "https://dibs-web01.vm.duke.edu/pearson/assets/images/zebrafish/pipelineNewpng3.png":
+ "images/research/pipelineNewpng3.png",
+ # Blog post images (_posts/*)
+ "https://dibs-web01.vm.duke.edu/pearson/assets/images/website/ava_preprint.png":
+ "images/blog/ava_preprint.png",
+ "https://dibs-web01.vm.duke.edu/pearson/assets/images/website/raymond_poster.jpg":
+ "images/blog/raymond_poster.jpg",
+}
+
+WAYBACK_API = "https://archive.org/wayback/available?url="
+USER_AGENT = "Mozilla/5.0 (compatible; pearsonlab-recover-script)"
+TIMEOUT = 30
+
+
+def fetch(url: str) -> bytes | None:
+ req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT})
+ try:
+ with urllib.request.urlopen(req, timeout=TIMEOUT) as resp:
+ if resp.status == 200:
+ data = resp.read()
+ if len(data) > 100: # filter out tiny error pages
+ return data
+ except (urllib.error.URLError, urllib.error.HTTPError, TimeoutError):
+ pass
+ return None
+
+
+def wayback_lookup(url: str) -> str | None:
+ """Find the most recent Wayback snapshot URL for a given URL."""
+ import json
+ api_url = WAYBACK_API + urllib.parse.quote(url, safe="")
+ try:
+ req = urllib.request.Request(api_url, headers={"User-Agent": USER_AGENT})
+ with urllib.request.urlopen(req, timeout=TIMEOUT) as resp:
+ payload = json.loads(resp.read())
+ snapshot = payload.get("archived_snapshots", {}).get("closest")
+ if snapshot and snapshot.get("available"):
+ return snapshot["url"]
+ except Exception:
+ pass
+ return None
+
+
+def recover(url: str, dest: str) -> bool:
+ print(f"\n[*] {url}")
+ print(f" -> {dest}")
+ os.makedirs(os.path.dirname(dest), exist_ok=True)
+
+ # Try live host first.
+ data = fetch(url)
+ source = "dibs-web01" if data else None
+
+ # Fall back to Wayback.
+ if not data:
+ snapshot_url = wayback_lookup(url)
+ if snapshot_url:
+ print(f" trying Wayback: {snapshot_url}")
+ data = fetch(snapshot_url)
+ if data:
+ source = "wayback"
+
+ if not data:
+ print(f" FAILED: not reachable on live host or Wayback")
+ return False
+
+ with open(dest, "wb") as f:
+ f.write(data)
+ size_kb = len(data) / 1024
+ print(f" saved ({size_kb:.0f} KB, source: {source})")
+ return True
+
+
+def main() -> int:
+ failed: list[str] = []
+ for url, dest in URLS.items():
+ if not recover(url, dest):
+ failed.append(url)
+ time.sleep(0.5) # be polite
+
+ print(f"\n{'='*60}")
+ print(f"Done. Recovered {len(URLS) - len(failed)}/{len(URLS)} images.")
+ if failed:
+ print(f"\nFailed URLs ({len(failed)}):")
+ for url in failed:
+ print(f" - {url}")
+ print("\nFor these, options are:")
+ print(" - Ask the original author for the file")
+ print(" - Manually paste a screenshot or alternate version into the repo")
+ print(" - Remove the reference from the source HTML/markdown")
+ return len(failed)
+
+
+if __name__ == "__main__":
+ sys.exit(main())
+ 