Model Downloader

Model Downloader

Download published Sirocco spectral grids and curated pre-trained emulator models from HuggingFace into Speculate's local working directories.

---

Overview

Sirocco grids are too large to ship inside the Speculate repository, so they are published as separate datasets under the Sirocco-rt HuggingFace organisation. Pre-trained Speculate emulator models are published separately under the Sirocco-rt/speculate_cv_emulators and Sirocco-rt/speculate_agn_emulators HuggingFace model repositories.

The Model Downloader is the supported local tool for bringing both resources onto your machine. It can decompress LZMA-encoded .spec.xz archives into sirocco_grids/<dataset>/, and it can stage .npz emulator files into Grid-Emulator_Files/ where the Inference Tool, Quick Fit, Benchmark Suite, and local training workflows expect to find them.

The tool is local-only. The hosted HuggingFace Space exposes Quick Fit and auto-caches the small pre-trained Quick Fit models when needed; local users can use this page to persist GP and Quick Fit models explicitly.

Availability

Mode	Available
Local install	Yes
HuggingFace Space	No (locked in the sidebar)

Spectral Grid Workflow

1. Pick a dataset from the dropdown of Sirocco-rt/*grid* repositories.
2. Choose the scope — the entire dataset, or one specific runNNN.spec.xz archive.
3. Preview the lookup table (optional) to confirm the physical parameters of the selected run before committing to a download.
4. Run an action:
   • Download & Decompress — the standard path; downloads archives, stages auxiliary files, and writes extracted .spec files.
   • Download Only — populates the HuggingFace cache without unpacking.
   • Decompress Downloaded Files — expands archives that are already in the cache.

Pre-trained Model Workflow

1. Review the model list discovered from the configured pretrained emulator repositories: Sirocco-rt/speculate_cv_emulators and Sirocco-rt/speculate_agn_emulators.
2. Choose All models, All GP emulators, All Quick Fit models, or one specific model.
3. Click Download Selected Models.

Downloaded model files are copied into Grid-Emulator_Files/. Existing files are skipped by default, so re-running the downloader is safe and quick.

Controls

Control	Effect
Select a dataset	Chooses one dataset under Sirocco-rt whose name contains grid
What would you like to download?	Toggles between full-dataset and single-run grid mode
Select file	Picks a runNNN.spec.xz archive in single-run mode
Lookup table preview	Reads grid_run_lookup_table.parquet over HTTP and shows the parameter row for the selected run
Download & Decompress	End-to-end grid download and extraction into sirocco_grids/<dataset>/
Download Only	Downloads grid archives into the HuggingFace cache; no extraction
Decompress Downloaded Files	Extracts grid archives that are already cached locally
Model download selector	Chooses all models, GP-only models, Quick Fit-only models, or one model file
Download Selected Models	Stages selected .npz models into Grid-Emulator_Files/

Outputs

• sirocco_grids/<dataset>/runNNN.spec — decompressed spectra.
• sirocco_grids/<dataset>/<auxiliary files> — anything bundled with the dataset that is not a per-run archive.
• Grid-Emulator_Files/*_emu_*.npz — GP emulator files for the Inference Tool and Benchmark Suite.
• Grid-Emulator_Files/*_qfnn_*.npz, *_qfnn-ensemble_*.npz, or *_qfgi_*.npz — Quick Fit pre-trained model files.
• HuggingFace cache (~/.cache/huggingface/) — original downloaded artifacts, retained by the Hub client.

Troubleshooting

• No datasets appear. The HuggingFace dataset API query failed or returned nothing. Check connectivity and that the Sirocco-rt organisation is reachable.
• No model files appear. Check that at least one configured model repository is reachable and contains supported .npz files with _emu_, _qfnn_, _qfnn-ensemble_, or _qfgi_ in their names.
• Lookup preview fails but grid download still works. The dataset is missing grid_run_lookup_table.parquet or the parquet HTTP fetch failed. Downloads do not depend on the preview.
• Archives downloaded but no .spec files visible. You used Download Only. Re-run with Download & Decompress or use Decompress Downloaded Files.
• A model download is skipped. A file with the same name already exists in Grid-Emulator_Files/.

References

• speculate_model_downloader.py — full marimo notebook driving the tool.
• Sirocco-rt on HuggingFace — published datasets and model repositories.
• Grid Inspector — next step after a successful grid download.
• Quick Fit and Inference Tool — primary consumers of downloaded models.