Repository navigation
Zimi 1.15.6 #154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Zimi 1.15.6 #154
Changes from all commits
Commits
Show all changes
59 commits
Select commit
Hold shift + click to select a range
ed8d7ab
Frozen app: programs outside the bundle get the machine's library pat…
epheterson b13ef59
Voices: a voice whose files are here but whose engine is not needs on…
epheterson cef0ca8
Almanac head tiles: a row of one-line labels carries no empty line ab…
epheterson 3c27c53
Own ZIMs: a released file name is spent even when squash-merged; zimi…
epheterson f4bf1a6
Games: ten-language strings for the credit line, Snake walls and the …
epheterson 91d5881
Games in kiwix-serve and Kiwix JS: the shelf's rows find their tiles …
epheterson 501f3c6
Tests for the games fixes and the credit line; the Q-ID audit script
epheterson c0e50a1
Golden Record: every picture credited, and a Kiwix Almanac with only …
epheterson f5fe9ff
Golden Record: a long credit wraps instead of cutting off
epheterson 233735f
Golden Record: the omitted picture's card says only that; its caption…
epheterson 3e63dcc
Golden Record: one scrolling list, a line of text for what an edition…
epheterson f305267
Kiwix Almanac: Name zimi_eng_almanac and Title "Zimi Almanac" by default
epheterson 54bd902
Golden Record note: owners are named beside each picture now
epheterson 613cc69
Almanac ZIM outside Zimi: a needs-scripts note, room above Kiwix JS's…
epheterson cec5c8f
Zimi 1.15.6
epheterson 6eb23ad
Kiwix editions: scripts build the Games and Almanac request files wit…
epheterson e41fbfb
Own ZIMs rebuilt: zimi_games_2026-11c, zimi_almanac_2026-11d
epheterson 4915720
ZIM descriptions: no 'offline', the long text no longer repeats the s…
epheterson a096cd1
Own ZIMs rebuilt with the new descriptions
epheterson a7ea7e5
Credit line centred and with the same margin in the Games and the Alm…
epheterson 5dbaf7b
Almanac ZIM in Zimi's ten languages: all ten strings files packed in,…
epheterson ba7fc9d
Own ZIMs rebuilt: credit line centred, the Almanac in ten languages
epheterson 6673c73
Changelog: the Almanac in ten languages, the centred credit line, no …
epheterson 729a88c
ZIM LongDescriptions: the Almanac's first sentence, and Zimi Games na…
epheterson b87cf3f
Own ZIMs rebuilt with the new LongDescriptions
epheterson b547e9e
Almanac: the clock reads left to right in Hebrew and Arabic, a Latin …
epheterson 8b12f4f
Own Almanac ZIM rebuilt with the clock and label fixes
epheterson 33d7585
Changelog: 1.15.6 in three lines
epheterson d74bb59
Tests green on the release branch: the Golden Record note's Sie is th…
epheterson 848d249
Games: the credit-line loader no longer replaces the page's own loadS…
epheterson b44d384
subproc: the bundle's folders are compared by one spelling of a path …
epheterson f03e634
Own Games ZIM rebuilt with the loadScript fix
epheterson 58446d3
Review fixes: behaviors keeps its subprocess import, Wiki Race ignore…
epheterson 76e573f
Own ZIMs rebuilt with the review fixes
epheterson 41402f9
Review findings: PYTHONPATH keeps the user's own entries, Docker prob…
epheterson 32cba39
Own ZIMs rebuilt
epheterson bafda95
Tests: UTF-8 for the fake program file, the bare-name test finds a Wi…
epheterson fe60731
Tests: Windows finds a bare program in the current folder first, so t…
epheterson 37c0013
Tests: the Almanac corners tests wait for the state they measure, not…
epheterson 7f52bb0
Test: when the Moon's hand-off does not finish, the failure carries t…
epheterson 3b3c9c6
Test: the Moon hand-off gesture starts when the page answers promptly…
epheterson a4f7778
Tests: the pointer events a failed Moon hand-off was given go in its …
epheterson d1b12f9
Test: the station pan test waits for the frame that holds the slide i…
epheterson e4ce415
Tests: the Moon hand-off starts once the loading veil has gone and th…
epheterson 5cd9574
Make five flaky browser tests wait on what they test
epheterson 0b0a948
Almanac: the side gutter is the inner column's, so a print cannot lea…
epheterson dc214cf
3D view: a pinch zooms by the fingers' spread since it began; station…
epheterson 79ab207
Landmarks test waits for Pluto's place, which the probes' table now b…
epheterson 6fb6811
Tests: the pinch test's slide threshold matches a pinch that ends at …
epheterson 2de4354
Own Almanac ZIM rebuilt with the gutter and pinch fixes
epheterson 74274a5
Test: a relaunch that does not pick up the new build says what the pa…
epheterson cc7286a
Test: when the open window does not reload onto the replaced ZIM, the…
epheterson 6b4d819
Tests: the download batch tests leave the process-wide queue empty
epheterson 11c1560
Tests: the offline tests start with no catalog in memory
epheterson 8a1ad32
Tests: wait for the header's slide to finish instead of 400 ms
epheterson d2f9190
Tests: the archive duty's offline retry fails its own catalog fetch
epheterson 869f81e
Tests: the relaunch tests wait until the Almanac is kept whole
epheterson 8266436
Tests: the typist's 'more from' is the named source, not whichever ra…
epheterson f6ce223
Tests: the Discover redraw test sets the page's own follow-up draws a…
epheterson File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| """Build the Almanac for openZIM/Kiwix: the standard Almanac without the Golden Record pictures that are not public domain. | ||
|
|
||
| python scripts/build_almanac_kiwix.py [--out DIR] [--file NAME] [--name ZIM_NAME] [--title TITLE] | ||
|
|
||
| openZIM and Kiwix take only free content. The Almanac's Golden Record gallery shows 49 of the record's pictures: | ||
| 8 NASA photographs (public domain) and 41 that are copyright their owners (golden-record/credits.json). This | ||
| edition keeps the 8 and names each of the 41 in the gallery's list (number, title, credit), so the record's | ||
| sequence stays whole (almanacstage.stage, community). Everything else is the Almanac ``python -m zimi.ownzims`` builds; | ||
| Zimi's own Almanac keeps all 49. Nothing is committed: the file goes to --out. | ||
|
|
||
| Its Name and Title are openZIM's, not the ones Zimi's own copy carries: Name ``zimi_mul_almanac`` (openZIM's | ||
| {domain}_{lang}_{selection}; ``mul`` for several languages) and Title "Zimi Almanac". The page speaks Zimi's ten languages. | ||
| """ | ||
|
|
||
| import argparse | ||
| import datetime | ||
| import os | ||
| import re | ||
| import sys | ||
| import tempfile | ||
|
|
||
| ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) | ||
| sys.path.insert(0, ROOT) | ||
|
|
||
| DEFAULT_NAME = "zimi_mul_almanac" # openZIM: {domain}_{lang}_{selection}; several languages are "mul" | ||
| #: The Language metadata: the ten languages the page speaks, ISO 639-3, in the order the Games use. | ||
| LANGUAGES = "eng,zho,hin,spa,ara,fra,por,rus,deu,heb" | ||
| DEFAULT_TITLE = "Zimi Almanac" | ||
| FILE_FMT = "%s_%s.zim" | ||
| _NAME = re.compile(r"^zimi_([a-z]{2,3})_([a-z0-9][a-z0-9.-]*)$") | ||
| #: Text for the ZIM's Description or LongDescription instead of zimi.txt's (None keeps zimi.txt's). Change a line, rebuild. | ||
| DESCRIPTION = None | ||
| LONG_DESCRIPTION = None | ||
|
|
||
|
|
||
| def set_fields(folder, **fields): | ||
| """Rewrite lines of the folder's zimi.txt; each field must already be there exactly once. None leaves it alone.""" | ||
| meta = os.path.join(folder, "zimi.txt") | ||
| with open(meta, encoding="utf-8") as f: | ||
| text = f.read() | ||
| for key, value in fields.items(): | ||
| if value is None: | ||
| continue | ||
| text, n = re.subn(r"(?m)^%s:.*$" % key, lambda m: "%s: %s" % (key, value), text) | ||
| if n != 1: | ||
| raise SystemExit("zimi.txt has no single %s line" % key) | ||
| with open(meta, "w", encoding="utf-8", newline="\n") as f: | ||
| f.write(text) | ||
|
|
||
|
|
||
| def build(out_dir, file_name=None, name=DEFAULT_NAME, title=DEFAULT_TITLE, month=None, description=None, long_description=None): | ||
| """The community Almanac, built from the source as it is now. Returns its path.""" | ||
| from zimi import almanacstage, creator, ownzims | ||
|
|
||
| m = _NAME.match(name) | ||
| if not m: | ||
| raise SystemExit( | ||
| "--name must look like zimi_<lang>_<scope>, e.g. %s" % DEFAULT_NAME | ||
| ) | ||
| month = month or datetime.date.today().strftime("%Y-%m") | ||
| os.makedirs(out_dir, exist_ok=True) | ||
| path = os.path.join(out_dir, file_name or FILE_FMT % (name, month)) | ||
| with tempfile.TemporaryDirectory(prefix="zimi-kiwix-") as tmp: | ||
| dest = os.path.join(tmp, m.group(2)) | ||
| os.mkdir(dest) | ||
| almanacstage.stage(dest, community=True) | ||
| set_fields(dest, Title=title, Language=LANGUAGES, Description=description or DESCRIPTION, LongDescription=long_description or LONG_DESCRIPTION) | ||
| # The Name is set outright: creator derives it from the Language metadata's 3-letter code (zimi_eng_...). | ||
| real = creator.zim_name | ||
| creator.zim_name = lambda scope, language="eng": name | ||
| try: | ||
| creator.create_folder_zim(dest, out_path=path, register=False, extra_items=ownzims.with_build_year()) | ||
| finally: | ||
| creator.zim_name = real | ||
| return path | ||
|
|
||
|
|
||
| def main(argv=None): | ||
| ap = argparse.ArgumentParser() | ||
| ap.add_argument("--out", default=os.path.join(ROOT, "dist", "kiwix")) | ||
| ap.add_argument("--file", help="the file's name (default <name>_<YYYY-MM>.zim)") | ||
| ap.add_argument( | ||
| "--name", | ||
| default=DEFAULT_NAME, | ||
| help="the ZIM's Name metadata (default %(default)s)", | ||
| ) | ||
| ap.add_argument( | ||
| "--title", default=DEFAULT_TITLE, help="the ZIM's Title (default %(default)s)" | ||
| ) | ||
| ap.add_argument("--description", help="the ZIM's Description (default zimi.txt's)") | ||
| ap.add_argument("--long-description", help="the ZIM's LongDescription (default zimi.txt's)") | ||
| a = ap.parse_args(argv) | ||
| path = build(a.out, a.file, a.name, a.title, description=a.description, long_description=a.long_description) | ||
| print("built %s (%.1f MB)" % (path, os.path.getsize(path) / 1e6)) | ||
| return 0 | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main()) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| """Build Zimi Games for openZIM/Kiwix: the same games, with the ZIM's Name, Title and languages openZIM's catalogue uses. | ||
|
|
||
| python scripts/build_games_kiwix.py [--out DIR] [--file NAME] [--name ZIM_NAME] [--title TITLE] | ||
|
|
||
| The games folder is copied to a temporary folder whose zimi.txt gets the Title and the ten interface languages; nothing in | ||
| the repo changes. A ZIM in several languages is named ``mul`` by Kiwix, so the Name is set outright (``zimi_mul_games``). | ||
| build_almanac_kiwix.py holds the sibling for the Almanac, and the DESCRIPTION / LONG_DESCRIPTION switches that both share. | ||
| """ | ||
|
|
||
| import argparse | ||
| import datetime | ||
| import os | ||
| import shutil | ||
| import sys | ||
| import tempfile | ||
|
|
||
| ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) | ||
| sys.path.insert(0, ROOT) | ||
| sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) | ||
|
|
||
| DEFAULT_NAME = "zimi_mul_games" | ||
| DEFAULT_TITLE = "Zimi Games" | ||
| FILE_FMT = "%s_%s.zim" | ||
| #: Text for the ZIM's Description or LongDescription instead of zimi.txt's (None keeps zimi.txt's). Change a line, rebuild. | ||
| DESCRIPTION = None | ||
| LONG_DESCRIPTION = None | ||
|
|
||
|
|
||
| def build(out_dir, file_name=None, name=DEFAULT_NAME, title=DEFAULT_TITLE, month=None, description=None, long_description=None): | ||
| """The community Games, built from the source as it is now. Returns its path.""" | ||
| from build_almanac_kiwix import LANGUAGES, set_fields | ||
| from zimi import creator, ownzims | ||
|
|
||
| month = month or datetime.date.today().strftime("%Y-%m") | ||
| os.makedirs(out_dir, exist_ok=True) | ||
| path = os.path.join(out_dir, file_name or FILE_FMT % (name, month)) | ||
| with tempfile.TemporaryDirectory(prefix="zimi-kiwix-") as tmp: | ||
| src = os.path.join(tmp, "games") | ||
| shutil.copytree(os.path.join(ownzims.SOURCE_ROOT, "games"), src) | ||
| set_fields(src, Title=title, Language=LANGUAGES, Description=description or DESCRIPTION, LongDescription=long_description or LONG_DESCRIPTION) | ||
| real = creator.zim_name | ||
| creator.zim_name = lambda scope, language="eng": name | ||
| try: | ||
| creator.create_folder_zim(src, out_path=path, register=False, extra_items=ownzims.with_build_year()) | ||
| finally: | ||
| creator.zim_name = real | ||
| return path | ||
|
|
||
|
|
||
| def main(argv=None): | ||
| ap = argparse.ArgumentParser() | ||
| ap.add_argument("--out", default=os.path.join(ROOT, "dist", "kiwix")) | ||
| ap.add_argument("--file", help="the file's name (default <name>_<YYYY-MM>.zim)") | ||
| ap.add_argument("--name", default=DEFAULT_NAME, help="the ZIM's Name metadata (default %(default)s)") | ||
| ap.add_argument("--title", default=DEFAULT_TITLE, help="the ZIM's Title (default %(default)s)") | ||
| ap.add_argument("--description", help="the ZIM's Description (default zimi.txt's)") | ||
| ap.add_argument("--long-description", help="the ZIM's LongDescription (default zimi.txt's)") | ||
| a = ap.parse_args(argv) | ||
| path = build(a.out, a.file, a.name, a.title, description=a.description, long_description=a.long_description) | ||
| print("built %s (%.1f MB)" % (path, os.path.getsize(path) / 1e6)) | ||
| return 0 | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main()) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| #!/usr/bin/env python3 | ||
| """Check ZimiGames' Q-IDs against English Wikipedia. | ||
|
|
||
| python3 scripts/verify_games_qids.py | ||
|
|
||
| Each row of zimi/builtin/games/games.json that stands for a game with an | ||
| encyclopedia article carries `qid` and `wiki` (the English title). The About | ||
| link and the "learn more" links resolve through the Q-ID, so a wrong one opens | ||
| the wrong article. For every row this asks the English Wikipedia API for the | ||
| title's `wikibase_item` (redirects followed) and reports a title that is | ||
| missing, a disambiguation page, or whose item is not the Q-ID written down. | ||
| Same lookup as verify_almanac_qids.py. Needs the network; nothing at runtime does. | ||
| """ | ||
|
|
||
| import json | ||
| import os | ||
| import sys | ||
| import urllib.parse | ||
| import urllib.request | ||
|
|
||
| sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) | ||
| from verify_almanac_qids import BATCH, USER_AGENT, lookup # noqa: E402 | ||
|
|
||
| GAMES_JSON = os.path.join( | ||
| os.path.dirname(os.path.dirname(os.path.abspath(__file__))), | ||
| "zimi", | ||
| "builtin", | ||
| "games", | ||
| "games.json", | ||
| ) | ||
|
|
||
|
|
||
| def entries(path=GAMES_JSON): | ||
| """[(q, en title, page)] for every row that has an article.""" | ||
| with open(path, encoding="utf-8") as f: | ||
| rows = json.load(f) | ||
| return [ | ||
| (r["qid"], r["wiki"], r["path"]) for r in rows if r.get("qid") and r.get("wiki") | ||
| ] | ||
|
|
||
|
|
||
| WIKIDATA = "https://www.wikidata.org/w/api.php" | ||
|
|
||
|
|
||
| def enwiki_titles(qids): | ||
| """{Q-ID: the English Wikipedia title the item's sitelink names}, 50 at a time.""" | ||
| out = {} | ||
| qids = list(dict.fromkeys(qids)) | ||
| for i in range(0, len(qids), BATCH): | ||
| url = WIKIDATA + "?" + urllib.parse.urlencode( | ||
| { | ||
| "action": "wbgetentities", | ||
| "format": "json", | ||
| "props": "sitelinks", | ||
| "sitefilter": "enwiki", | ||
| "ids": "|".join(qids[i : i + BATCH]), | ||
| } | ||
| ) | ||
| req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT}) | ||
| with urllib.request.urlopen(req, timeout=60) as r: | ||
| data = json.load(r)["entities"] | ||
| for q, ent in data.items(): | ||
| link = ent.get("sitelinks", {}).get("enwiki") | ||
| if link: | ||
| out[q] = link["title"] | ||
| return out | ||
|
|
||
|
|
||
| def audit(found, res, sitelinks=lambda qids: {}): | ||
| """[(q, title, page, why)] for each entry whose article is not its Q-ID. | ||
|
|
||
| A title that is a redirect to another item's article still fits when the | ||
| Q-ID's own English sitelink is that very title (Wikidata files a game under | ||
| the redirect), so only the rest are reported.""" | ||
| bad = [] | ||
| mismatched = [q for q, en, _ in found if not (res[en]["missing"] or res[en]["disambig"]) and res[en]["q"] != q] | ||
| own = sitelinks(mismatched) if mismatched else {} | ||
| for q, en, page in found: | ||
| r = res[en] | ||
| if r["missing"]: | ||
| bad.append((q, en, page, "missing")) | ||
| elif r["disambig"]: | ||
| bad.append((q, en, page, "disambiguation page")) | ||
| elif r["q"] != q and own.get(q) != en: | ||
| bad.append((q, en, page, "is %s" % r["q"])) | ||
| return bad | ||
|
|
||
|
|
||
| def main(): | ||
| found = entries() | ||
| bad = audit(found, lookup([en for _, en, _ in found]), enwiki_titles) | ||
| for q, en, page, why in bad: | ||
| print("BAD %-10s %-45s %-28s %s" % (q, en, page, why)) | ||
| print("%d games, %d bad" % (len(found), len(bad))) | ||
| return 1 if bad else 0 | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main()) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟡 Medium · Concurrent builds write incorrect ZIM names
When kiwix build functions overlap another ZIM build in the same Python process, both build functions replace the process-wide
creator.zim_namewhile packaging, so a build can write another build’s Name metadata and out-of-order restores can leave a stale naming function installed; the archives have incorrect names and later builds can inherit them.Prevent concurrent builds from sharing a temporarily replaced
creator.zim_name, or avoid mutating that process-wide function.Also at
scripts/build_games_kiwix.py:41-46.Prompt for AI agents