Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 98 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@
- **行政区划/GeoJSON 下载**:按内置区域目录或 GeoJSON 层级创建下载计划。
- **多地图源配置**:支持天地图、Mapbox、OSM、Google 样例和自定义瓦片 URL。
- **任务和产物管理**:独立任务页查看历史、进度、失败记录,并下载 ZIP/MBTiles。
- **资源限制和排队**:默认每父任务全部图层合计最多 100 万瓦片,同时运行最多 3 个子任务,其余持久排队;暂停任务继续占用执行槽。
- **完整重试和安全删除**:按任务及运行隔离产物,补齐失败瓦片后发布累计结果,删除时清理任务记录及生成文件。
- **任务区域预览**:独立展示完整区域、切换层级,自动续期预览凭证;支持移动端创建和任务操作。

### 适合谁

Expand All @@ -28,7 +31,7 @@

### 三分钟启动

源码运行:
源码运行需要 Go 1.25 或更高版本。在仓库应用目录启动,确保能读取 `conf.toml`、`static/` 和 `geojson/`:

```powershell
git clone https://github.com/Joe5027/map-tile-fetcher.git
Expand All @@ -43,7 +46,10 @@ go run .
- 用户名:`admin`
- 密码:`adminmap`

生产部署前请复制 `.env.example` 为 `.env`,并修改默认账号和密码。
首次用于生产前,保持登录启用并设置自己的初始账号和密码。源码或二进制启动读取
`conf.toml` 和进程环境变量(如 `AUTH_DEFAULT_USERNAME`、`AUTH_DEFAULT_PASSWORD`),
不会自动加载 `.env`;Docker Compose 和下方 `docker run --env-file .env` 才会注入该文件。
这些账号配置用于初始化用户,修改配置不会重置数据库中已有同名用户的密码。

### Docker 镜像和部署

Expand Down Expand Up @@ -90,6 +96,9 @@ Docker 配置项:
| `APP_DATABASE` | `tiler.db` | SQLite 数据库文件名,保存在 `data/`。 |
| `AUTH_DEFAULT_USERNAME` | `admin` | 默认登录用户名,生产部署必须修改。 |
| `AUTH_DEFAULT_PASSWORD` | `adminmap` | 默认登录密码,生产部署必须修改。 |
| `TASK_MAX_TILES` | `1000000` | 每父任务所有图层累计瓦片预算;区域按包围范围保守估算。 |
| `TASK_MAX_ACTIVE` | `3` | 同时运行的子任务上限,暂停任务继续占槽,其余排队。 |
| `TASK_WORKERS` | `3` | 默认请求线程数;用户可选 1~50,受地图来源并发上限约束。 |
| `TZ` | `Asia/Shanghai` | 容器时区。 |

Compose 会持久化 `data/`、`output/`、`geojson/` 和 `conf.toml`。如果端口被占用,把 `.env` 里的 `HOST_PORT=8081` 改成其他端口,例如 `HOST_PORT=18081`。
Expand All @@ -106,7 +115,22 @@ Compose 会持久化 `data/`、`output/`、`geojson/` 和 `conf.toml`。如果

![artifact download](docs/assets/artifact-download.png)

`我的任务` Tab 独立展示历史任务、运行状态、进度、失败数和产物入口。任务完成后可以下载 ZIP 文件树或 MBTiles;失败记录会持久化,便于调整并发、请求间隔或代理后重试。
`我的任务` Tab 展示排队状态、本次运行进度、累计完整度、未解决失败数和产物入口。
实际线程数取用户设置与来源 `worker_count` 上限的较小值;请求间隔不低于用户设置
和来源最低间隔,页面显示生效参数。

### 重试、删除和历史任务

- **失败重试**:先复制旧成功结果到新运行,再补齐未解决瓦片,重新生成累计 ZIP/MBTiles。
同名任务及重复运行互不覆盖;新产物发布成功前继续提供上一份可用下载,失败或取消不切换入口。
重试仍有失败时会显示缺口,不会标为全部成功。
- **彻底删除**:删除父任务会清理所有子任务、历史运行、产物及关联记录;内置区域文件和仍被
其他任务引用的区域文件受保护。运行、暂停或打包中的任务需先取消并等待结束;清理失败可重试。
- **历史核对**:通过显式核对动作检查本地文件树、ZIP 或 MBTiles,不自动下载瓦片。
旧成功瓦片缺失时,失败重试返回 `409 baseline_missing`,需主动选择“重新创建完整任务”。

升级前备份数据库和输出,并先用隔离副本验证。迁移方式、磁盘空间要求和各项回归证据见
[修复验收记录](docs/repair-acceptance-2026-09-11.md)。

### 配置地图源

Expand All @@ -122,13 +146,24 @@ Compose 会持久化 `data/`、`output/`、`geojson/` 和 `conf.toml`。如果

### 发布和安装

- GitHub Release:[`v0.1.0`](https://github.com/Joe5027/map-tile-fetcher/releases/tag/v0.1.0)
- 发布说明:[`docs/releases/v0.1.0.md`](docs/releases/v0.1.0.md)
- 已发布预览版:[`v0.3.0`](https://github.com/Joe5027/map-tile-fetcher/releases/tag/v0.3.0)(发布说明及下载包)
- 历史发布说明:[`docs/releases/v0.1.0.md`](docs/releases/v0.1.0.md)
- 用户手册:[`docs/user-manual-zh.md`](docs/user-manual-zh.md)
- English manual: [`docs/user-manual.md`](docs/user-manual.md)

本 README 描述当前 `main` 的行为;上述预览版早于本次修复,不包含全部新行为。
需要本次修复时,请使用当前 `main` 源码运行或构建 Docker 镜像。

如果使用二进制发布包,解压后保持 `conf.toml`、`static/`、`geojson/` 和可执行文件在同一目录,再启动程序。

### 开发验证

在应用目录执行 `go test ./...`、`go vet ./...` 和 `node scripts/release_preflight.mjs`。
浏览器检查需要 Node.js、Playwright 及 Chromium;命令示例见下方 [Developer Validation](#developer-validation)。
Go 集成测试贯通真实 API、临时 SQLite、正常工作进程与本地模拟瓦片服务,检查任务控制和最终产物。
发布预检还覆盖渲染安全、390/768/1440px 页面、预览续期和登录失效后的重新登录;
Linux CI 额外执行 `go test -race ./...`。详见 [验证链路](docs/validation-chain.md)。

## English Guide

### What It Does
Expand All @@ -139,6 +174,9 @@ Map Tile Fetcher is a self-hosted Go Web app for downloading authorized map tile
- **GeoJSON/admin-region downloads**: create tasks from built-in region catalogs or GeoJSON files.
- **Configurable map sources**: Tianditu, Mapbox, OSM, Google examples, and custom tile URLs.
- **Task and artifact management**: view task history, progress, failures, retries, and download ZIP/MBTiles artifacts.
- **Resource limits and queues**: by default, each parent task has a combined budget of 1,000,000 tiles across all layers. Up to three children run at once; others remain in a persistent queue. Paused children retain their slots.
- **Cumulative retries and safe deletion**: isolate artifacts by task and run, publish cumulative results after retrying failed tiles, and delete task records together with generated files.
- **Task area previews**: view complete areas on an independent overlay, switch levels, and renew preview credentials automatically; create and manage tasks on mobile screens.

### Who It Is For

Expand All @@ -148,7 +186,7 @@ Map Tile Fetcher is a self-hosted Go Web app for downloading authorized map tile

### Quick Start

Run from source:
Source builds require Go 1.25 or later. Start in the application directory so the app can read `conf.toml`, `static/`, and `geojson/`:

```powershell
git clone https://github.com/Joe5027/map-tile-fetcher.git
Expand All @@ -163,7 +201,11 @@ Development login:
- Username: `admin`
- Password: `adminmap`

Before production deployment, copy `.env.example` to `.env` and change the default credentials.
Before first production use, keep login enabled and set your own initial credentials.
Source and binary runs read `conf.toml` and process environment variables such as
`AUTH_DEFAULT_USERNAME` and `AUTH_DEFAULT_PASSWORD`; they do not load `.env` automatically.
Docker Compose and the `docker run --env-file .env` command below inject that file.
These settings initialize users; changing them does not reset an existing user's password.

### Docker Image And Deployment

Expand Down Expand Up @@ -219,6 +261,9 @@ Docker environment variables:
| `APP_DATABASE` | `tiler.db` | SQLite database file stored under `data/`. |
| `AUTH_DEFAULT_USERNAME` | `admin` | Default login username. Change it for production. |
| `AUTH_DEFAULT_PASSWORD` | `adminmap` | Default login password. Change it for production. |
| `TASK_MAX_TILES` | `1000000` | Combined tile budget per parent across all layers; regions use a conservative bounding-range estimate. |
| `TASK_MAX_ACTIVE` | `3` | Maximum running children. Paused children retain slots; others queue. |
| `TASK_WORKERS` | `3` | Default request concurrency. Users can choose 1-50, subject to the source concurrency cap. |
| `TZ` | `Asia/Shanghai` | Container timezone. |

Docker Compose persists `data/`, `output/`, `geojson/`, and `conf.toml`. If port `8081` is already in use, change `HOST_PORT=8081` in `.env`, for example to `HOST_PORT=18081`.
Expand All @@ -231,6 +276,28 @@ Docker Compose persists `data/`, `output/`, `geojson/`, and `conf.toml`. If port
4. Set concurrency, request interval, artifact format, and schedule.
5. After task creation, switch to the `My Tasks` tab to track progress and download artifacts.

The task view separates queue status, current-run progress, cumulative completeness,
unresolved failures, and published downloads. Effective concurrency is the smaller
of the requested value and the source's `worker_count` cap. The request interval
is at least both the user setting and the source minimum; the UI shows the effective values.

### Retries, Deletion, And Legacy Tasks

- **Retry failures**: copy previously successful tiles into a new run, fetch unresolved tiles,
then rebuild cumulative ZIP/MBTiles output. Identically named tasks and repeated runs do not
overwrite one another. The previous download remains available until publication succeeds;
failure or cancellation does not switch it. Remaining failures are reported as gaps.
- **Delete permanently**: deleting a parent removes its children, historical runs, artifacts,
and related records. Built-in area files and files still referenced by other tasks are protected.
Cancel running, paused, or packing tasks and wait for them to stop before deletion. Failed cleanup can be retried.
- **Reconcile legacy output**: explicitly check existing file trees, ZIPs, or MBTiles without
downloading tiles. When previously successful tiles are missing, retries return
`409 baseline_missing`; use the explicit full-task recreation action to download again.

Back up the database and output before upgrading and validate an isolated copy first.
See the [repair acceptance record](docs/repair-acceptance-2026-09-11.md) for migration,
disk-space requirements, and regression evidence.

### Map Source Configuration

Example map-source configuration lives in `apps/admin-region-tiler/conf.toml`.
Expand All @@ -245,23 +312,44 @@ Real tokens should stay in local `.env`, local config, or your deployment secret

### Release And Install

- GitHub Release: [`v0.1.0`](https://github.com/Joe5027/map-tile-fetcher/releases/tag/v0.1.0)
- Release notes: [`docs/releases/v0.1.0.md`](docs/releases/v0.1.0.md)
- Published preview: [`v0.3.0`](https://github.com/Joe5027/map-tile-fetcher/releases/tag/v0.3.0) (release notes and downloads)
- Historical release notes: [`docs/releases/v0.1.0.md`](docs/releases/v0.1.0.md)
- Chinese manual: [`docs/user-manual-zh.md`](docs/user-manual-zh.md)
- English manual: [`docs/user-manual.md`](docs/user-manual.md)

This README describes the current `main` branch. The preview above predates these repairs
and does not include all current behavior. Run or build a Docker image from current `main`
to use the repairs.

For binary release packages, keep the executable, `conf.toml`, `static/`, and `geojson/` in the same directory before starting the app.

### Developer Validation

In addition to Go, browser checks require Node.js, Playwright, and Chromium.
Install them once if they are not already available:

```powershell
npm install -g playwright
npx playwright install chromium
```

Then run:

```powershell
cd apps\admin-region-tiler
go test ./...
go vet ./...
node --check .\static\script.js
node .\scripts\release_preflight.mjs
```

The release preflight runs Go tests, JavaScript checks, browser UI smoke tests, sensitive-value scanning, and tracked generated-file scanning.
Go integration tests exercise real APIs, temporary SQLite databases, ordinary worker
processes, and a local tile fixture, including task controls and final artifact checks.
The release preflight also runs JavaScript checks, rendering-security contracts,
browser smoke and regressions at 390/768/1440px, preview renewal, re-login after session
expiry, sensitive-value scanning, and tracked generated-file scanning.
Linux CI additionally runs `go test -race ./...` and `go vet ./...`.
See the [validation chain](docs/validation-chain.md) for the full workflow.

## Repository Notes

Expand Down
6 changes: 4 additions & 2 deletions apps/admin-region-tiler/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,10 @@ go run .
- 用户名:`admin`
- 密码:`adminmap`

可信本地环境可设置 `AUTH_ENABLED=false` 免登录。生产部署应保持登录启用,并通过
`.env` 覆盖默认账号。
可信本地环境可设置 `AUTH_ENABLED=false` 免登录。首次用于生产前应保持登录启用,
通过 `conf.toml` 或进程环境变量 `AUTH_DEFAULT_USERNAME`、`AUTH_DEFAULT_PASSWORD`
设置初始账号。源码和二进制启动不会自动加载 `.env`;下方 Docker Compose 或
`docker run --env-file .env` 会注入该文件。修改初始账号配置不会重置已有同名用户的密码。

## Docker 镜像

Expand Down
6 changes: 6 additions & 0 deletions docs/long-term-memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

## Facts

- 2026-09-11 README follow-up: root documentation now covers repair behavior,
limits, legacy handling and validation. Source/binary runs read process
environment variables, not `.env` automatically; initial credentials do not
reset existing users. GitHub's published `v0.3.0` preview predates the repairs;
current behavior refers to `main`. PR #3 was merged as `8b0933b`, its resulting
main CI passed, and `agent/audit-hardening` was removed locally and remotely.
- 2026-09-11 repair delivery: the ten-defect implementation, baseline failure
evidence, current regression results and migration procedures are indexed in
`docs/repair-acceptance-2026-09-11.md`. Use that file before repeating audits.
Expand Down
Loading