Conversation
commit: |
Member
Author
|
@joeberkovitz I am preparing to add tides to the Open Waters app, so I thought I would explore a similar approach you described for floatingtrails.com. This pull request adds a pmtiles build of the database. I would love your feedback on it. |
Adds a packages/tiles workspace that builds a MapLibre-ready PMTiles vector tileset of all stations: - Stations are point features in a single "stations" source-layer. Nested objects (datums, harmonic_constituents, offsets, source, license, epoch) are encoded as JSON-string properties since vector tiles only support scalar values. - Lean properties (id/name/type) at z0-7, full station data at z8-10, merged with tile-join so every tile stays small while no station is ever dropped. - Feature ids are stable 53-bit FNV-1a hashes of station ids for MapLibre setFeatureState. - Built with tippecanoe via the pinned ghcr.io/openwatersio/tippecanoe Docker image; tests decode tiles and round-trip station data against the stations export. - publish.yml attaches the stable-named neaps.pmtiles to releases with the release version embedded in the tileset metadata.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a new packages/tiles workspace that builds and tests a PMTiles vector tileset (neaps.pmtiles) for MapLibre/Mapbox Vector Tile consumers, and updates the release workflow to publish the artifact.
Changes:
- Add a Docker-backed build pipeline that generates lean (z0–7) + full (z8–10) station tiles, then merges them into a single PMTiles file.
- Add Vitest coverage that validates PMTiles metadata, completeness at z0, lean/full property boundaries, and round-trip JSON-encoded station fields.
- Extend the release workflow to build and attach
.pmtilesartifacts to GitHub Releases.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
packages/tiles/test/tiles.test.ts |
Adds end-to-end validation of the generated PMTiles (header/metadata, zoom behavior, and property round-trips). |
packages/tiles/test/pmtiles.ts |
Adds local-file PMTiles reader + MVT decoding utilities for tests. |
packages/tiles/README.md |
Documents tileset usage, structure, and property encoding expectations. |
packages/tiles/package.json |
Declares the new tiles workspace scripts and dev dependencies. |
packages/tiles/features.ts |
Implements stable 53-bit FNV-1a feature IDs and lean/full property shaping (JSON-string encoding for nested objects). |
packages/tiles/docker-compose.yml |
Pins and configures tippecanoe/tile-join containers used by the build. |
packages/tiles/build.ts |
Generates deterministic NDJSON inputs and collision-checks feature IDs before tile generation. |
packages/tiles/build |
Orchestrates the two tippecanoe builds and merges them into dist/neaps.pmtiles with metadata. |
.github/workflows/publish.yml |
Builds PMTiles during publish and uploads .pmtiles artifacts alongside existing .tcd outputs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1
to
+21
| import { openSync, readSync } from "fs"; | ||
| import { PMTiles, type Source, type RangeResponse } from "pmtiles"; | ||
| import { VectorTile } from "@mapbox/vector-tile"; | ||
| import Protobuf from "pbf"; | ||
|
|
||
| /** Byte-range source for reading a local .pmtiles file. */ | ||
| class FileSource implements Source { | ||
| private fd: number; | ||
|
|
||
| constructor(private path: string) { | ||
| this.fd = openSync(path, "r"); | ||
| } | ||
|
|
||
| getKey(): string { | ||
| return this.path; | ||
| } | ||
|
|
||
| async getBytes(offset: number, length: number): Promise<RangeResponse> { | ||
| const buffer = Buffer.alloc(length); | ||
| readSync(this.fd, buffer, 0, length, offset); | ||
| return { |
| | `chart_datum` | string | Key into `datums`, e.g. `MLLW`, `LAT` | | ||
| | `disclaimers` | string | Optional | | ||
| | `datums` | JSON string | `{ "MLLW": 1.01, "MSL": 2.532, ... }` | | ||
| | `harmonic_constituents` | JSON string | `[{ "name": "M2", "amplitude", "phase" },...]` | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a
packages/tilesworkspace that builds a PMTiles vector tileset (neaps.pmtiles, ~15MB) of all 6,085 stations for direct integration with MapLibre GL.stationssource-layer, z0–10: lean properties (id/name/type) at z0–7 for rendering dots, full station data at z8–10. Two tippecanoe runs merged withtile-join; no station is ever dropped at any zoom.datums,harmonic_constituents,offsets,source,license,epoch) are JSON-string properties that clientsJSON.parse. Subordinate stations include their reference station's datums/constituents, so each feature is self-sufficient for prediction.setFeatureStateand consistent across releases.npm run build -w tiles(requires Docker; uses the pinnedghcr.io/openwatersio/tippecanoe:2.79.0image, mirroring thepackages/tcdpattern).publish.ymlattaches a stable-namedneaps.pmtilesto releases soreleases/latest/download/neaps.pmtilesis a permanent URL; the release version is embedded in the tileset metadata description.packages/tcdreferences.Testing
npm test -w tiles: 10 tests open the built file, verify header/metadata (single mergedvector_layersentry spanning z0–10), assert the z0 tile contains every station under 500KB compressed, check the lean/full boundary at z7→z8, and round-trip parsed constituents/datums/offsets against thestationsexport for reference and subordinate stations.type; at z9 a queried feature's parsed JSON yielded 50 constituents and datums matching the source.npm test(31,214 tests) andnpm run lintpass.