Repository navigation
Delete the KLayout application install and the aarch64 KLayout source build; KLayout's Python module writes the GDS file - #4757
Open
oharboe wants to merge 5 commits into
Conversation
Quality of life for those who don't need or want a GDS file. KLayout is a big application, packaged differently on every distribution, and the flow needs it for the small task of writing out a GDS file at the very end. Without it, `make` stopped there: Error: KLayout not found. Install KLayout or set KLAYOUT_CMD. Now klayout.sh, when KLAYOUT_CMD is empty and the call is the stream-out (def2stream.py with an out_file), writes an empty file and says so: WARNING: klayout not found, mocking klayout by writing empty file <out_file> and the flow runs to the end. Installing KLayout opts out. Nobody mistakes an empty GDS file for a valid one, and the warning says why it is empty. Every other KLayout call, DRC, LVS, the gallery and the viewers, still stops with the error. No Makefile conditionals: the decision is made in klayout.sh, at the one call that writes the GDS file. The GDS merge no longer depends on check-klayout. DRC and LVS were guarded by check-klayout only through the merge they depend on, so they get it directly: `make drc` without KLayout stops with the same error as before. The hint under that error names what still needs KLayout. A hierarchical design works the same way: each block's GDS is written empty, and so is the parent's merge of them. flow/test/test_klayout_sh.py checks the stream-out writes an empty file and warns, DRC and the viewer still stop without KLayout, and an installed KLayout gets the call unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request refactors the flow to make KLayout an optional dependency by utilizing the klayout Python module for GDS/OAS stream generation, falling back to mocking (writing an empty file with a warning) when KLayout is entirely missing. It updates dependency installers, Makefiles, and documentation to reflect this change, and adds corresponding unit tests. The review feedback correctly identifies a critical issue in flow/util/def2stream.py where a subprocess version check could cause an unnecessary hard failure, and points out that the custom environment dictionary is not being forwarded to the spawned subprocesses.
KLayout is a big application, packaged differently on every distribution, and the flow needs it for the small task of writing the GDS file at the end. KLayout's own Python module, `pip install klayout`, does that task: it is a prebuilt wheel with no Qt and nothing to compile, for Linux (x86_64 and aarch64, glibc 2.27 and up) and macOS. def2stream.py now has a command line and is run by PYTHON_EXE. It takes the first KLayout it finds: 1. the klayout Python module, in-process; 2. a KLayout executable, KLAYOUT_CMD or klayout on PATH, run as before (klayout -zz -rd ... -r def2stream.py); 3. neither: the stream file is written empty with the warning klayout.sh gave for it until now. Only an absent KLayout is let through. A KLayout that is there but misconfigured stops the flow: KLAYOUT_CMD naming no file, an executable that fails `-v` (klayout.sh ran it under `set -e` too), or a failing merge. The executable gets the environment the choice was made with. klayout.sh goes back to running a KLayout executable only, for DRC, LVS, the gallery and the viewers, which still need one: the DRC and LVS decks are scripts in KLayout's Ruby-based DSL, which the Python module does not include. The wrapped-macro merge passed `-rd config_file=$(FILL_CONFIG)`, which def2stream.py never read; it is not passed any more. flow/test/test_def2stream.py checks each of the three choices: the module gets the arguments, an executable on PATH gets them as -rd variables with the environment and its exit status is returned, and without either the file is written empty with the warning. It checks that a misconfigured KLayout is an error and writes no file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
The flow writes its GDS file with KLayout's Python module, so the installer no longer installs the KLayout application. The module comes with the other Python packages, from etc/requirements-common_lock.txt: a prebuilt wheel for Python 3.10 and up on Linux (x86_64 and aarch64, glibc 2.27 and up) and macOS. The lock already needs Python 3.10, so this adds no new requirement on the Python version. Deleted: - etc/DependencyInstaller.sh: klayoutVersion, and the KLayout install for each OS: the CentOS 7, CentOS 8 and Rocky Linux 9 RPMs with their version check and reinstall, the Ubuntu .deb with its md5 table per release and the "Unsupported Ubuntu version" exit, the build from source on Ubuntu aarch64 with its Qt, Ruby and libgit2 build dependencies, `klayout` from apt on rodete and the Homebrew cask on macOS. - env.sh: the KLayout.app PATH entry on macOS. - flow/scripts/variables.mk: the lookup of a KLayout built into tools/install/klayout, which nothing installs. KLAYOUT_CMD is the klayout on PATH unless set, and is exported bare. - docs/user/BuildWithPrebuilt.md: the KLayout version note and the LD_LIBRARY_PATH line for a KLayout built from source. DRC, LVS, the gallery and the viewers still need the KLayout application, installed by those who run them; the docs say where to get it. None of them runs in the default flow or in CI's `finish metadata`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe
force-pushed
the
gds-pypi-klayout
branch
from
October 28, 2026 07:45
1ccf595 to
1971407
Compare
Collaborator
Author
|
@maliberty @eder-matheus @vvbandeira TL;DR klayout has been a thorny dependency and it confuses users This is intended to prune as much as possible from ORFS install scripts, etc. while giving each use-case the least chafing experience. We try PyPi klayout first, then local and finally we write an empty .gds file and a warning. The full klayout install is still needed for make lvs/drc and gallery, but I think those are out of the way for most users and PyPi klayout doesn't have ruby, so there's nothing for it, full klayout install is needed. This would be solved by documentation, this is an out of the way use-case you have to install klayout yourself. |
Member
🔍 QoR checkMetrics reflect the PR merge build — i.e. what will land on the target branch. Commit 62 design(s) checked — 0 with regression(s), 0 without a comparable baseline. |
oharboe
changed the title
oharboe
marked this pull request as ready for review
November 2, 2026 17:17
An empty GDS file breaks the principle of least astonishment: make reports success and hands over a file that is not a layout. The GDS merge now has one KLayout, the Python module the installer installs, and without it stops: ERROR: the klayout Python module is not installed, pip install klayout (etc/requirements-common_lock.txt) with exit status 1 and no file written. The fallback to a KLayout executable goes too: def2stream.py no longer looks for KLAYOUT_CMD or klayout on PATH, nor runs itself under `klayout -r`. The module comes with the installer's Python packages, so the merge has one path. DRC, LVS, the gallery and the viewers need the KLayout application, installed by the user, and stop at check-klayout without it, as before. test_def2stream.py checks the error, and that no file is written; the tests of the executable fallback and the empty file are deleted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
This was referenced Nov 2, 2026
oharboe
marked this pull request as draft
November 4, 2026 10:03
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe
marked this pull request as ready for review
November 4, 2026 14:31
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
KLayout's own Python module,
pip install klayout, writes the GDS file. It is a prebuilt wheel, with no Qt and nothing to compile, for Python 3.10 and up on Linux (x86_64 and aarch64, glibc 2.27 and up) and macOS. With it, the installer no longer installs the KLayout application, and the flow writes a real GDS file wherever its Python packages are installed. Without the module the GDS merge stops with an error; it never writes an empty file.lb_32x128,make do-gdsLayoutDiff, 12 cells)Installation, simpler
etc/DependencyInstaller.shloses its KLayout install for every OS, and the module comes in with the other Python packages frometc/requirements-common_lock.txt; on Ubuntu 23.04 and later, into the virtual environment #4787 adds. The lock already needs Python 3.10, so this adds no new requirement on the Python version. Deleted:klayoutVersionand the version-check-and-reinstall logic, written three times (CentOS 7, CentOS 8, Rocky Linux 9) around klayout.org RPMs..debdownload, its md5 table per release, and the "Unsupported Ubuntu version" exit, so KLayout's packaging no longer limits which Ubuntu releases the installer supports.klayoutfrom apt on rodete, and the Homebrew cask on macOS.env.sh: the KLayout.appPATHentry on macOS.flow/scripts/variables.mk: the lookup of a KLayout intools/install/klayout, which nothing installs.docs/user/BuildWithPrebuilt.md: the KLayout version note and theLD_LIBRARY_PATHline for a KLayout built from source.Added:
klayoutinetc/requirements-common.in, pinned at 0.30.12 in the lock, the version the installer installed until now.Without the module: an error
make finishandmake gdswithout the module stop at the GDS merge, exit status 1, no file written:def2stream.pyuses the module only; it no longer runs itself under a KLayout executable (KLAYOUT_CMDorklayoutonPATH), so the merge has one path.What still needs the KLayout application
make drc,make lvs, the gallery and the viewers. The DRC and LVS rule decks are scripts in KLayout's Ruby-based DSL, which the Python module does not include. None of them runs in the defaultmake, nor in CI'sfinish metadata. Those who run them install KLayout themselves; the docs say where. Without it they stop atcheck-klayout, as before.Commits
def2stream.pyhas a command line and is run byPYTHON_EXE, with the Python module in-process, a KLayout executable as a fallback, and klayout.sh: without KLayout, stream out an empty GDS file and warn #4712's empty file without either.klayout.shgoes back to running the KLayout application only.env.sh,variables.mk, the requirements and the docs, as above.def2stream.py: the Python module only, and without it an error instead of an empty file. The executable fallback and the empty file are deleted, with their tests.Tests
flow/test/test_def2stream.pychecks that the module gets the arguments, and that without it the merge exits with status 1, prints the error and writes no file.flow/test/test_klayout_sh.pychecks that DRC and the viewer stop without the KLayout application, and that an installed one gets the call unchanged.make do-gdson sky130hdlb_32x128with the module and with the application wrote the layouts in the table above (commits 1-3; commit 4 calls the module's merge unchanged). Commit 4's error:def2stream.pyrun without the module exits 1 and writes no file.WRAP_LEFS), which only gf12 designs use. It calls the same script with the same arguments as before.🤖 Generated with Claude Code