Skip to content

chore(prod): region-layers mount reuses ${DATA_DIR}, so the region defs page serves the data dir #204

Open

Description

@diffwireauto

Reported by ITblacksheep in Discord: the region definitions page is not working on prod. He suspected a missing /app prefix in the compose mount. It is not that — the container-side path is correct.

Root cause

/opt/corescope/docker-compose.yml line 17 reuses ${DATA_DIR} instead of its own variable:

- ${DATA_DIR:-./data}:/app/data
- ${DATA_DIR:-./frontend/region-layers}:/app/public/region-layers:ro

:-default only applies when the variable is unset. /opt/corescope/.env contains exactly DATA_DIR=./data, so the ./frontend/region-layers default never fires and both lines resolve to the same host directory.

Verified on the live container, not inferred

[box: deadbeef, 2026-10-16]

docker inspect corescope --format '{{range .Mounts}}...'
  /opt/corescope/data -> /app/data                    (rw)
  /opt/corescope/data -> /app/public/region-layers    (ro)
docker exec corescope ls /app/public/region-layers
  caddy  config.json  config.json.bak-20260727-031511  ...

The container is serving the data directory at that path.

The precise user-visible failure

[repo: public/region-def.js:4] the page fetches region-layers/layers.json.

check result
layers.json on the host, correct dir PRESENT
layers.json inside the container No such file or directory
GET /region-layers/layers.json HTTP 200, body is <html lang="en">…

So the fetch succeeds and returns the SPA's index.html. The page then fails parsing JSON. That is why it breaks rather than erroring cleanly.

The data is fine

/opt/corescope/frontend/region-layers/ on the host holds layers.json plus aoh, chm, cvg, day, fwa, oki, states, us-midwest, us-southeast geojson. Nothing is missing or misplaced. Only the mount source is wrong.

Ruled out

  • Stale mount from an older compose. Compose last modified 2026-10-16 17:08:14 UTC; container created 2026-10-16 20:37:30 UTC. The container is 3.5h newer, so the live mounts reflect the current file.
  • An override file. No docker-compose.override.yml. Only .bak copies, which compose does not read.
  • Wrong container-side path. The server runs -public /app/public, so /app/public/region-layers is correct.

The fix

Give line 17 its own source. Hardcoding rather than adding a second variable:

- ./frontend/region-layers:/app/public/region-layers:ro

DATA_DIR is a variable because the data location is genuinely environment-specific. The region layers are repo-relative content that ships with the app, so a variable adds indirection with no benefit — and indirection is what caused this. A hardcoded relative path resolves against the compose file's directory and has no failure mode.

⚠ The larger finding: this file is not under version control

[verified: 2026-10-16]

  • /opt/corescope is not a git clone
  • no repo compose file mentions region-layers — grep -l region-layers docker-compose*.yml returns nothing across all five
  • deploy-prod.yml never deploys a compose file; it references /opt/corescope/docker-compose.yml as a one-time prereq and then runs docker compose up -d against whatever is already on the box
  • the only history is six .bak files with epoch-stamped names

So prod's compose is load-bearing configuration with no version control, no review, and no CI that could catch a defect in it. This bug shipped and nothing could have stopped it. This is the same shape as the config-contract hazard in docs/UPSTREAM.md checklist item 1: the only artifact that disagrees with the code is the config on the box, and CI has never seen it.

Filed separately as a gap; this issue is the fix only. ITblacksheep's own Discord remark — "I need to switch our docker compose to ansible" — is the same conclusion from the other direction.

Cost of applying

A bind-mount change requires recreating the container. Prod's cold start measured ~260s to /api/healthz ready, so roughly four minutes unavailable. A manual docker compose up -d has no auto-rollback — that logic lives only in deploy-prod.yml. Backup taken before the edit.

Verification

  • docker compose config parses before applying
  • docker inspect shows /opt/corescope/frontend/region-layers -> /app/public/region-layers
  • docker exec corescope ls /app/public/region-layers lists the geojson and layers.json
  • GET /region-layers/layers.json returns JSON, not HTML
  • container healthy, no exited: corescope- in logs, counts still climbing

Done when

The region definitions page loads on prod and /region-layers/layers.json serves the real manifest.

Refs #187

Activity

  1. diffwireauto commented on Oct 16, 2026

    @diffwireauto
    ContributorAuthor

    Fixed and verified on prod

    Line 17 of /opt/corescope/docker-compose.yml now reads:

    - ./frontend/region-layers:/app/public/region-layers:ro

    Backup taken first: docker-compose.yml.bak-pre204-1792194232, md5 confirmed identical to the original before editing. The replacement asserted exactly one occurrence of the old string before writing.

    Validated before applying

    docker compose config parsed and resolved the two mounts to different sources:

    source: /opt/corescope/data                      target: /app/data
    source: /opt/corescope/frontend/region-layers    target: /app/public/region-layers
    

    Verified after the recreate

    [box: deadbeef, 2026-10-16]

    docker inspect corescope
      /opt/corescope/frontend/region-layers -> /app/public/region-layers
      /opt/corescope/data                   -> /app/data
    
    docker exec corescope ls /app/public/region-layers
      aoh chm cvg day fwa ind layers.json oki states us-midwest us-southeast  (geojson)
    

    The failing request now works. GET /region-layers/layers.json returns real JSON ("highlight": false, "coverage": "full", "url": [...]) instead of the SPA's index.html.

    Container Up 4 minutes (healthy), exited: corescope- count 0, ingesting 13,471 packets/hour across 1,867 nodes. Ready at ~280s after recreate, in line with the ~260s cold start measured previously.

    One correction worth recording

    ITblacksheep suggested ./frontend/regions-layers — plural. That directory does not exist; the real one is region-layers singular [verified: ls /opt/corescope/frontend/ shows only region-layers]. Docker creates a missing bind-mount source as an empty root-owned directory, so that spelling would have mounted an empty dir: the page would still break, and docker inspect would have looked correct, sending the next person after a phantom. His diagnosis and method were right; only the directory name was off, recalled from memory while driving.

    Unrelated observation

    Prod is on v0.3.3, not the v0.3.2 I last verified. A release was cut by someone else between my checks. Not part of this change and not my doing — noting it so the version in this record is not mistaken for something this issue did.

    The version-control gap called out in the issue body is not addressed here and is still open: /opt/corescope is not a git clone, no repo compose mentions region-layers, and nothing in CI can validate that file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions