Skip to content

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MTV Chilloutzone Website Data

This workspace contains only the website-data work built from:

  • https://www.mtv-chilloutzone.com/
  • archive seed: https://www.mtv-chilloutzone.com/playlistarchive.htm

What Is In Scope

  • local mirror data in mirror/www.mtv-chilloutzone.com/
  • extracted playlist archive data in data/
  • serviceable broadcast-level records in data/serviceable/
  • the scripts used to build and regenerate those outputs

Main Outputs

  • mirror/www.mtv-chilloutzone.com/
    • local mirror of the site data currently kept in scope
  • data/mtv_chilloutzone_archive.csv
    • song-level archive export
  • data/mtv_chilloutzone_book.md
    • readable year/date/edition export
  • data/mtv_chilloutzone_manifest.csv
    • fetched URL manifest
  • data/mtv_chilloutzone_summary.txt
    • archive summary
  • data/serviceable/broadcasts_serviceable.csv
    • one row per broadcast instance
  • data/serviceable/broadcasts_serviceable.jsonl
    • one JSON object per broadcast instance
  • data/serviceable/tracks_serviceable.csv
    • flat serviceable track rows
  • data/serviceable/serviceable.sqlite
    • SQLite database with broadcasts and tracks
  • data/serviceable/serviceable_summary.txt
    • serviceable output summary

Scripts

  • scripts/mtv_archive_pipeline.py
    • crawl, resume, extract, prune
  • scripts/make_records_serviceable.py
    • build serviceable records from the song-level archive CSV

Commands

Build or resume the archive mirror:

python3 scripts/mtv_archive_pipeline.py --resume

Build or resume the broader same-host site mirror:

python3 scripts/mtv_archive_pipeline.py --resume --full-site

Rebuild the archive CSV and book from the current mirror:

python3 scripts/mtv_archive_pipeline.py --extract-only

Prune the local mirror back to the current policy:

python3 scripts/mtv_archive_pipeline.py --prune-mirror --full-site

Build serviceable records:

python3 scripts/make_records_serviceable.py

Show service-builder help:

python3 scripts/make_records_serviceable.py --help

Current Mirror Policy

  • full-site mirroring excludes video/screenshots material by default
  • excluded branch:
    • watchthevideos.htm
    • screenshots.htm
    • intros.htm
    • intros-promos-vids.htm
    • ss-*.htm and their descendant assets

Notes

  • the website is the canonical source
  • the archive CSV preserves the original raw song line
  • same-date same-edition duplicate playlist instances are preserved as separate broadcast instances
  • for the serviceable layer, see SERVICEABLE_RECORDS_GUIDE.md

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages