Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
ed8d7ab
Frozen app: programs outside the bundle get the machine's library pat…
epheterson Nov 11, 2026
b13ef59
Voices: a voice whose files are here but whose engine is not needs on…
epheterson Nov 11, 2026
cef0ca8
Almanac head tiles: a row of one-line labels carries no empty line ab…
epheterson Nov 11, 2026
3c27c53
Own ZIMs: a released file name is spent even when squash-merged; zimi…
epheterson Nov 11, 2026
f4bf1a6
Games: ten-language strings for the credit line, Snake walls and the …
epheterson Nov 11, 2026
91d5881
Games in kiwix-serve and Kiwix JS: the shelf's rows find their tiles …
epheterson Nov 11, 2026
501f3c6
Tests for the games fixes and the credit line; the Q-ID audit script
epheterson Nov 11, 2026
c0e50a1
Golden Record: every picture credited, and a Kiwix Almanac with only …
epheterson Nov 10, 2026
f5fe9ff
Golden Record: a long credit wraps instead of cutting off
epheterson Nov 10, 2026
233735f
Golden Record: the omitted picture's card says only that; its caption…
epheterson Nov 10, 2026
3e63dcc
Golden Record: one scrolling list, a line of text for what an edition…
epheterson Nov 10, 2026
f305267
Kiwix Almanac: Name zimi_eng_almanac and Title "Zimi Almanac" by default
epheterson Nov 10, 2026
54bd902
Golden Record note: owners are named beside each picture now
epheterson Nov 10, 2026
613cc69
Almanac ZIM outside Zimi: a needs-scripts note, room above Kiwix JS's…
epheterson Nov 11, 2026
cec5c8f
Zimi 1.15.6
epheterson Nov 11, 2026
6eb23ad
Kiwix editions: scripts build the Games and Almanac request files wit…
epheterson Nov 11, 2026
e41fbfb
Own ZIMs rebuilt: zimi_games_2026-11c, zimi_almanac_2026-11d
epheterson Nov 11, 2026
4915720
ZIM descriptions: no 'offline', the long text no longer repeats the s…
epheterson Nov 11, 2026
a096cd1
Own ZIMs rebuilt with the new descriptions
epheterson Nov 11, 2026
a7ea7e5
Credit line centred and with the same margin in the Games and the Alm…
epheterson Nov 11, 2026
5dbaf7b
Almanac ZIM in Zimi's ten languages: all ten strings files packed in,…
epheterson Nov 11, 2026
ba7fc9d
Own ZIMs rebuilt: credit line centred, the Almanac in ten languages
epheterson Nov 11, 2026
6673c73
Changelog: the Almanac in ten languages, the centred credit line, no …
epheterson Nov 11, 2026
729a88c
ZIM LongDescriptions: the Almanac's first sentence, and Zimi Games na…
epheterson Nov 11, 2026
b87cf3f
Own ZIMs rebuilt with the new LongDescriptions
epheterson Nov 11, 2026
b547e9e
Almanac: the clock reads left to right in Hebrew and Arabic, a Latin …
epheterson Nov 11, 2026
8b12f4f
Own Almanac ZIM rebuilt with the clock and label fixes
epheterson Nov 11, 2026
33d7585
Changelog: 1.15.6 in three lines
epheterson Nov 11, 2026
d74bb59
Tests green on the release branch: the Golden Record note's Sie is th…
epheterson Nov 11, 2026
848d249
Games: the credit-line loader no longer replaces the page's own loadS…
epheterson Nov 11, 2026
b44d384
subproc: the bundle's folders are compared by one spelling of a path …
epheterson Nov 11, 2026
f03e634
Own Games ZIM rebuilt with the loadScript fix
epheterson Nov 11, 2026
58446d3
Review fixes: behaviors keeps its subprocess import, Wiki Race ignore…
epheterson Nov 11, 2026
76e573f
Own ZIMs rebuilt with the review fixes
epheterson Nov 11, 2026
41402f9
Review findings: PYTHONPATH keeps the user's own entries, Docker prob…
epheterson Nov 11, 2026
32cba39
Own ZIMs rebuilt
epheterson Nov 11, 2026
bafda95
Tests: UTF-8 for the fake program file, the bare-name test finds a Wi…
epheterson Nov 11, 2026
fe60731
Tests: Windows finds a bare program in the current folder first, so t…
epheterson Nov 11, 2026
37c0013
Tests: the Almanac corners tests wait for the state they measure, not…
epheterson Nov 11, 2026
7f52bb0
Test: when the Moon's hand-off does not finish, the failure carries t…
epheterson Nov 11, 2026
3b3c9c6
Test: the Moon hand-off gesture starts when the page answers promptly…
epheterson Nov 11, 2026
a4f7778
Tests: the pointer events a failed Moon hand-off was given go in its …
epheterson Nov 11, 2026
d1b12f9
Test: the station pan test waits for the frame that holds the slide i…
epheterson Nov 11, 2026
e4ce415
Tests: the Moon hand-off starts once the loading veil has gone and th…
epheterson Nov 11, 2026
5cd9574
Make five flaky browser tests wait on what they test
epheterson Nov 10, 2026
0b0a948
Almanac: the side gutter is the inner column's, so a print cannot lea…
epheterson Nov 10, 2026
dc214cf
3D view: a pinch zooms by the fingers' spread since it began; station…
epheterson Nov 10, 2026
79ab207
Landmarks test waits for Pluto's place, which the probes' table now b…
epheterson Nov 10, 2026
6fb6811
Tests: the pinch test's slide threshold matches a pinch that ends at …
epheterson Nov 11, 2026
2de4354
Own Almanac ZIM rebuilt with the gutter and pinch fixes
epheterson Nov 11, 2026
74274a5
Test: a relaunch that does not pick up the new build says what the pa…
epheterson Nov 11, 2026
cc7286a
Test: when the open window does not reload onto the replaced ZIM, the…
epheterson Nov 11, 2026
6b4d819
Tests: the download batch tests leave the process-wide queue empty
epheterson Nov 11, 2026
11c1560
Tests: the offline tests start with no catalog in memory
epheterson Nov 11, 2026
8a1ad32
Tests: wait for the header's slide to finish instead of 400 ms
epheterson Nov 11, 2026
d2f9190
Tests: the archive duty's offline retry fails its own catalog fetch
epheterson Nov 11, 2026
869f81e
Tests: the relaunch tests wait until the Almanac is kept whole
epheterson Nov 11, 2026
8266436
Tests: the typist's 'more from' is the named source, not whichever ra…
epheterson Nov 11, 2026
f6ce223
Tests: the Discover redraw test sets the page's own follow-up draws a…
epheterson Nov 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/).

## [Unreleased]

## [1.15.6]

### Highlights

**Zimi**

- Voices install on the portable Linux build (#151).
- A voice missing its engine gets a Set up button.

**Games**

- Fixes and Kiwix compatibility, so the Games ZIM can be shared with the community.

**Almanac**

- Ten languages, fixes and Kiwix compatibility, so the Almanac ZIM can be shared with the community.

## [1.15.5]

### Highlights
Expand Down
5 changes: 3 additions & 2 deletions ci/smoke_portable.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,9 @@ echo "zimi config (the portable folder decides the paths)..."
"$Z" config | tee "$STICK/config.txt"
grep -q "portable" "$STICK/config.txt" || { echo "FAIL: config does not show the portable folder"; exit 1; }

echo "zimi doctor --check..."
"$Z" doctor --check
echo "zimi doctor --check (also: the machine's python3, started from the app, is not handed the bundle's libraries, #151)..."
"$Z" doctor --check | tee "$STICK/doctor.txt"
grep -q "would not run properly" "$STICK/doctor.txt" && { echo "FAIL: the app leaks its environment to outside programs"; exit 1; }

echo "zimi service print (ExecStart must be this executable)..."
"$Z" service print | tee "$STICK/unit.txt"
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "zimi"
version = "1.15.5"
version = "1.15.6"
description = "Offline knowledge server for ZIM files — search and read Wikipedia, Stack Overflow and 50+ sources with no internet"
readme = "README.md"
license = {text = "GPL-3.0-or-later"}
Expand Down
99 changes: 99 additions & 0 deletions scripts/build_almanac_kiwix.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
"""Build the Almanac for openZIM/Kiwix: the standard Almanac without the Golden Record pictures that are not public domain.

python scripts/build_almanac_kiwix.py [--out DIR] [--file NAME] [--name ZIM_NAME] [--title TITLE]

openZIM and Kiwix take only free content. The Almanac's Golden Record gallery shows 49 of the record's pictures:
8 NASA photographs (public domain) and 41 that are copyright their owners (golden-record/credits.json). This
edition keeps the 8 and names each of the 41 in the gallery's list (number, title, credit), so the record's
sequence stays whole (almanacstage.stage, community). Everything else is the Almanac ``python -m zimi.ownzims`` builds;
Zimi's own Almanac keeps all 49. Nothing is committed: the file goes to --out.

Its Name and Title are openZIM's, not the ones Zimi's own copy carries: Name ``zimi_mul_almanac`` (openZIM's
{domain}_{lang}_{selection}; ``mul`` for several languages) and Title "Zimi Almanac". The page speaks Zimi's ten languages.
"""

import argparse
import datetime
import os
import re
import sys
import tempfile

ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, ROOT)

DEFAULT_NAME = "zimi_mul_almanac" # openZIM: {domain}_{lang}_{selection}; several languages are "mul"
#: The Language metadata: the ten languages the page speaks, ISO 639-3, in the order the Games use.
LANGUAGES = "eng,zho,hin,spa,ara,fra,por,rus,deu,heb"
DEFAULT_TITLE = "Zimi Almanac"
FILE_FMT = "%s_%s.zim"
_NAME = re.compile(r"^zimi_([a-z]{2,3})_([a-z0-9][a-z0-9.-]*)$")
#: Text for the ZIM's Description or LongDescription instead of zimi.txt's (None keeps zimi.txt's). Change a line, rebuild.
DESCRIPTION = None
LONG_DESCRIPTION = None


def set_fields(folder, **fields):
"""Rewrite lines of the folder's zimi.txt; each field must already be there exactly once. None leaves it alone."""
meta = os.path.join(folder, "zimi.txt")
with open(meta, encoding="utf-8") as f:
text = f.read()
for key, value in fields.items():
if value is None:
continue
text, n = re.subn(r"(?m)^%s:.*$" % key, lambda m: "%s: %s" % (key, value), text)
if n != 1:
raise SystemExit("zimi.txt has no single %s line" % key)
with open(meta, "w", encoding="utf-8", newline="\n") as f:
f.write(text)


def build(out_dir, file_name=None, name=DEFAULT_NAME, title=DEFAULT_TITLE, month=None, description=None, long_description=None):
"""The community Almanac, built from the source as it is now. Returns its path."""
from zimi import almanacstage, creator, ownzims

m = _NAME.match(name)
if not m:
raise SystemExit(
"--name must look like zimi_<lang>_<scope>, e.g. %s" % DEFAULT_NAME
)
month = month or datetime.date.today().strftime("%Y-%m")
os.makedirs(out_dir, exist_ok=True)
path = os.path.join(out_dir, file_name or FILE_FMT % (name, month))
with tempfile.TemporaryDirectory(prefix="zimi-kiwix-") as tmp:
dest = os.path.join(tmp, m.group(2))
os.mkdir(dest)
almanacstage.stage(dest, community=True)
set_fields(dest, Title=title, Language=LANGUAGES, Description=description or DESCRIPTION, LongDescription=long_description or LONG_DESCRIPTION)
# The Name is set outright: creator derives it from the Language metadata's 3-letter code (zimi_eng_...).
real = creator.zim_name
creator.zim_name = lambda scope, language="eng": name
try:
creator.create_folder_zim(dest, out_path=path, register=False, extra_items=ownzims.with_build_year())
finally:
creator.zim_name = real
Comment on lines +70 to +74

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Concurrent builds write incorrect ZIM names

When kiwix build functions overlap another ZIM build in the same Python process, both build functions replace the process-wide creator.zim_name while packaging, so a build can write another build’s Name metadata and out-of-order restores can leave a stale naming function installed; the archives have incorrect names and later builds can inherit them.

Prevent concurrent builds from sharing a temporarily replaced creator.zim_name, or avoid mutating that process-wide function.

Also at scripts/build_games_kiwix.py:41-46.

Prompt for AI agents
In `scripts/build_almanac_kiwix.py` at lines 70-74:

**Concurrent builds write incorrect ZIM names**

When kiwix build functions overlap another ZIM build in the same Python process, both build functions replace the process-wide `creator.zim_name` while packaging, so a build can write another build’s Name metadata and out-of-order restores can leave a stale naming function installed; the archives have incorrect names and later builds can inherit them.

Prevent concurrent builds from sharing a temporarily replaced `creator.zim_name`, or avoid mutating that process-wide function.

Also at `scripts/build_games_kiwix.py:41-46`.

return path


def main(argv=None):
ap = argparse.ArgumentParser()
ap.add_argument("--out", default=os.path.join(ROOT, "dist", "kiwix"))
ap.add_argument("--file", help="the file's name (default <name>_<YYYY-MM>.zim)")
ap.add_argument(
"--name",
default=DEFAULT_NAME,
help="the ZIM's Name metadata (default %(default)s)",
)
ap.add_argument(
"--title", default=DEFAULT_TITLE, help="the ZIM's Title (default %(default)s)"
)
ap.add_argument("--description", help="the ZIM's Description (default zimi.txt's)")
ap.add_argument("--long-description", help="the ZIM's LongDescription (default zimi.txt's)")
a = ap.parse_args(argv)
path = build(a.out, a.file, a.name, a.title, description=a.description, long_description=a.long_description)
print("built %s (%.1f MB)" % (path, os.path.getsize(path) / 1e6))
return 0


if __name__ == "__main__":
sys.exit(main())
65 changes: 65 additions & 0 deletions scripts/build_games_kiwix.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
"""Build Zimi Games for openZIM/Kiwix: the same games, with the ZIM's Name, Title and languages openZIM's catalogue uses.

python scripts/build_games_kiwix.py [--out DIR] [--file NAME] [--name ZIM_NAME] [--title TITLE]

The games folder is copied to a temporary folder whose zimi.txt gets the Title and the ten interface languages; nothing in
the repo changes. A ZIM in several languages is named ``mul`` by Kiwix, so the Name is set outright (``zimi_mul_games``).
build_almanac_kiwix.py holds the sibling for the Almanac, and the DESCRIPTION / LONG_DESCRIPTION switches that both share.
"""

import argparse
import datetime
import os
import shutil
import sys
import tempfile

ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, ROOT)
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))

DEFAULT_NAME = "zimi_mul_games"
DEFAULT_TITLE = "Zimi Games"
FILE_FMT = "%s_%s.zim"
#: Text for the ZIM's Description or LongDescription instead of zimi.txt's (None keeps zimi.txt's). Change a line, rebuild.
DESCRIPTION = None
LONG_DESCRIPTION = None


def build(out_dir, file_name=None, name=DEFAULT_NAME, title=DEFAULT_TITLE, month=None, description=None, long_description=None):
"""The community Games, built from the source as it is now. Returns its path."""
from build_almanac_kiwix import LANGUAGES, set_fields
from zimi import creator, ownzims

month = month or datetime.date.today().strftime("%Y-%m")
os.makedirs(out_dir, exist_ok=True)
path = os.path.join(out_dir, file_name or FILE_FMT % (name, month))
with tempfile.TemporaryDirectory(prefix="zimi-kiwix-") as tmp:
src = os.path.join(tmp, "games")
shutil.copytree(os.path.join(ownzims.SOURCE_ROOT, "games"), src)
set_fields(src, Title=title, Language=LANGUAGES, Description=description or DESCRIPTION, LongDescription=long_description or LONG_DESCRIPTION)
real = creator.zim_name
creator.zim_name = lambda scope, language="eng": name
try:
creator.create_folder_zim(src, out_path=path, register=False, extra_items=ownzims.with_build_year())
finally:
creator.zim_name = real
return path


def main(argv=None):
ap = argparse.ArgumentParser()
ap.add_argument("--out", default=os.path.join(ROOT, "dist", "kiwix"))
ap.add_argument("--file", help="the file's name (default <name>_<YYYY-MM>.zim)")
ap.add_argument("--name", default=DEFAULT_NAME, help="the ZIM's Name metadata (default %(default)s)")
ap.add_argument("--title", default=DEFAULT_TITLE, help="the ZIM's Title (default %(default)s)")
ap.add_argument("--description", help="the ZIM's Description (default zimi.txt's)")
ap.add_argument("--long-description", help="the ZIM's LongDescription (default zimi.txt's)")
a = ap.parse_args(argv)
path = build(a.out, a.file, a.name, a.title, description=a.description, long_description=a.long_description)
print("built %s (%.1f MB)" % (path, os.path.getsize(path) / 1e6))
return 0


if __name__ == "__main__":
sys.exit(main())
99 changes: 99 additions & 0 deletions scripts/verify_games_qids.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
#!/usr/bin/env python3
"""Check ZimiGames' Q-IDs against English Wikipedia.

python3 scripts/verify_games_qids.py

Each row of zimi/builtin/games/games.json that stands for a game with an
encyclopedia article carries `qid` and `wiki` (the English title). The About
link and the "learn more" links resolve through the Q-ID, so a wrong one opens
the wrong article. For every row this asks the English Wikipedia API for the
title's `wikibase_item` (redirects followed) and reports a title that is
missing, a disambiguation page, or whose item is not the Q-ID written down.
Same lookup as verify_almanac_qids.py. Needs the network; nothing at runtime does.
"""

import json
import os
import sys
import urllib.parse
import urllib.request

sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from verify_almanac_qids import BATCH, USER_AGENT, lookup # noqa: E402

GAMES_JSON = os.path.join(
os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
"zimi",
"builtin",
"games",
"games.json",
)


def entries(path=GAMES_JSON):
"""[(q, en title, page)] for every row that has an article."""
with open(path, encoding="utf-8") as f:
rows = json.load(f)
return [
(r["qid"], r["wiki"], r["path"]) for r in rows if r.get("qid") and r.get("wiki")
]


WIKIDATA = "https://www.wikidata.org/w/api.php"


def enwiki_titles(qids):
"""{Q-ID: the English Wikipedia title the item's sitelink names}, 50 at a time."""
out = {}
qids = list(dict.fromkeys(qids))
for i in range(0, len(qids), BATCH):
url = WIKIDATA + "?" + urllib.parse.urlencode(
{
"action": "wbgetentities",
"format": "json",
"props": "sitelinks",
"sitefilter": "enwiki",
"ids": "|".join(qids[i : i + BATCH]),
}
)
req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT})
with urllib.request.urlopen(req, timeout=60) as r:
data = json.load(r)["entities"]
for q, ent in data.items():
link = ent.get("sitelinks", {}).get("enwiki")
if link:
out[q] = link["title"]
return out


def audit(found, res, sitelinks=lambda qids: {}):
"""[(q, title, page, why)] for each entry whose article is not its Q-ID.

A title that is a redirect to another item's article still fits when the
Q-ID's own English sitelink is that very title (Wikidata files a game under
the redirect), so only the rest are reported."""
bad = []
mismatched = [q for q, en, _ in found if not (res[en]["missing"] or res[en]["disambig"]) and res[en]["q"] != q]
own = sitelinks(mismatched) if mismatched else {}
for q, en, page in found:
r = res[en]
if r["missing"]:
bad.append((q, en, page, "missing"))
elif r["disambig"]:
bad.append((q, en, page, "disambiguation page"))
elif r["q"] != q and own.get(q) != en:
bad.append((q, en, page, "is %s" % r["q"]))
return bad


def main():
found = entries()
bad = audit(found, lookup([en for _, en, _ in found]), enwiki_titles)
for q, en, page, why in bad:
print("BAD %-10s %-45s %-28s %s" % (q, en, page, why))
print("%d games, %d bad" % (len(found), len(bad)))
return 1 if bad else 0


if __name__ == "__main__":
sys.exit(main())
13 changes: 13 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -345,3 +345,16 @@ def _no_router_changes(request, monkeypatch):
monkeypatch.setattr(p2p, "is_upnp_enabled", lambda: False)
monkeypatch.setattr(p2p_nat, "add_port_mapping", lambda port: False)
yield


@pytest.fixture(autouse=True)
def _p2p_prefs_path_is_put_back(monkeypatch):
"""A test that points the BT prefs at its own temp file (p2p.set_prefs_path) leaves them there for the next test of the
worker, whose "seeding is on by default" then reads a stranger's file."""
try:
from zimi import p2p
except Exception:
yield
return
monkeypatch.setattr(p2p, "_prefs_path", p2p._prefs_path)
yield
Loading
Loading