Repository navigation
Conversation
…nfer command A parameter no module annotates can still be typed from the arguments its callers pass. The new infer runner drives pyrefly over a package tree, collects the checker's argument facts at every call site, and reduces them to the pins every caller agrees on. The lowerer applies a pin only when the subset can express it, and records every accepted inference so pyscriptc infer and compile --debug show what was proved. The bridge's pyrefly lane pins check-unannotated-defs and infer-return-types=checked through a synthetic pyrefly.toml, so an unannotated def reports its inferred return type instead of Any. A second checker pass runs over a pinned copy of each module, so the checker's own return inference is consistent with what lowering applies. A call site the checker could not resolve, or one that disagrees, refuses the pin; the soundness rule is the compiler's own, runtime behavior must match CPython, so an inference is accepted only under the checker's proof. pyscriptc infer reports the per-lever delta: native definitions and fences with and without inference, the pinned parameters, and the accepted records. --overlay writes the pins as a sidecar the user can apply by hand. The report never edits the source.
… server pinned_facts spawned a fresh pyrefly tsp server per file (one server start per file of the tree), and the lowering pass built its per-file bridges the same way, so the re-check and the bridged lowering pass each cost ~5s per file on a real package. The tsp server keeps one snapshot per open document, so a whole tree runs on one process: the runner's build_facts already proves the pattern. - PyreflyBridge.open()/close() hold one shared PyreflyTspFactsAdapter; facts() reuses it when open and keeps the one-shot path otherwise - BridgeProvider.open()/close() start the shared server once per tree; pyscriptc infer wraps the inferred lowering pass in it, so the per-file bridge builds open one document each instead of spawning a server each - pinned_facts runs the whole re-check on one server and skips files no pin lands in: their first-pass facts already hold, so the re-check cannot change them - the infer JSON report gains a pinned_facts_seconds phase timer Results stay byte-identical: same pins, same native/fence counts, same records_detail on a copy of yt-dlp utils/ (520 defs).
The pin pass calls _call_site_types once per pending def; each call re-ran peek_namespace_fields, a pure function of the module AST. One run per pass serves every def. yt-dlp utils/: infer 4m44s to 3m37s, results byte-identical (pins, native_inferred 154, fences_inferred 530, records 13).
The pin pass asked every pending def for the same caller facts: the caller's Name-callee call list (a pure AST walk) and the locals table _call_site_locals derives. Neither names a def, so both are facts of the caller alone, yet the walk ran once per pending def: 248440 walks over yt-dlp's utils/ (30M AST node visits, 55% of the whole run). The pin pass now caches both per caller, keyed by a pin epoch. The epoch bumps whenever a pin lands - a caller parameter got an annotation, or a new signature entered self.sigs - so a table computed before the pin is never reused after it. Within one epoch the state is the same for every def, so the cached table is exactly what the uncached walk would have produced, and the pin-arrival order is preserved. infer on yt-dlp utils/: 3m29.6s -> 39.0s wall (baseline 93.8s -> 5.9s, inferred 92.5s -> 8.2s). The pins, pinned_params, records_detail, native_inferred and fences_inferred are byte-identical to the run before, on utils/ and on extractor/youtube/.
…read The catalog fixpoint re-lowered every unit on every round, but a round re-lowers a unit only when an export-table entry the unit read changed. The lowerer now records every export-table read (module, member, value seen), and the catalog memo reuses the previous round's exports, failed definitions and exception-code counter when every entry still holds the value the unit saw. A unit reads its exception base too, so a shift in the codes an earlier unit consumed dirties it. The coroutine frame scan reads the whole table, so it records a table-wide read and dirties on any table change. The fixpoint comparison still runs over the whole table, so the rounds stop exactly where they stopped before. On yt-dlp utils/ the final fixpoint round drops from 0.88s to 0.0s and round 1 re-lowers 3 of 10 units instead of 10; the full inferred report is byte-identical on yt-dlp utils/ and extractor/youtube.
… reads resolve natively A module-level global that lives in a runtime slot (a list, a dict, a threaded record) keeps one pointer for the process, but the export table only carried sigs, constants and classes. A consumer that read the name fenced with PS2002, and the fence cascaded. The export table now records the slot with its type and the defining module, the consumer installs a local alias slot keyed by the defining module's qualified key, and the runtime registry (already keyed by the module-qualified name) resolves the read natively. The module's own init moves into the synthesized __init__ entry, so the slot initializes when the module body runs at import, in source order. A --call entry bypasses the program entry, so the merge records the module init order and the binary main runs the module bodies first. A star import of a package module binds the slot the same way a named import does. Also: the program entry prefers the synthesized __entry__ over a user main, so the module-level statements (the import inits) run before the user's main is called.
…hes its definition The pin map is keyed by the qualified name of the callee (`pkg.mod.helper`). The lowerer asks for the pins of the name it lowers the definition under, so a pin keyed under another spelling never reaches the definition it proves. The infer runner built the name from the file's own path, so a package directory joined the name twice: `utils/_utils.py` in a `yt_dlp` tree became `yt_dlp.utils.utils._utils`, while the lowerer lowers it as `yt_dlp.utils._utils`. Every pin on the tree missed its definition. The name now follows the tree the lowerer lowers: a package tree namespaces its modules under the package directory, a program tree (no `__init__.py` at the root) uses the plain relative path, and the program entry file stays `__main__`. Tests: `tests/test_infer_module_names.py` (the package spelling, the program spelling, and the names matching the lowerer's unit names).
…annotated parameter lowers `pyscriptc infer` proves a parameter's type at every call site and pins the ones that agree. The pins reached the lowerer only through the `infer` command's own report; a compile could not apply them, so an unannotated parameter kept its PS1001 fence and its definition never entered the export table. `compile --infer` now builds the pins, re-checks a pinned copy of each module so the checker's return and local types agree with what the lowerer applies, and rides the pin map on every per-file bridge. The bridge for a directory compile is a provider (one bridge per unit from one checker server) instead of a single-file bridge, so the pins and the pinned facts reach every unit. Tests: `tests/test_infer_pin_native.py` (the pin is keyed under the callee's qualified name, the pin lifts the parameter fence, the binary calls the helper natively and agrees with CPython, and the same helper keeps its fence without the pins).
Deploying pyscriptc with
|
| Latest commit: |
1801afa
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://5289e043.pyscriptc.pages.dev |
| Branch Preview URL: | https://work-ytnative.pyscriptc.pages.dev |
added 7 commits
November 2, 2026 06:11
The --infer flag and the infer subcommand landed without a docs regeneration, so the generated-docs seam reported the CLI reference as out of date.
The runtime-linked stdlib manifest check that ytrun carries adds 7 self-host fences in stdlibzip.py, so the --infer lane's census pin (3270) is stale on the merged tree. The merged tree measures 3277 fences and the same package sum; the pin moves with a comment that names both deltas.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Native-path work on yt-dlp, stacked on #791 (which is stacked on #771). Includes the
pyscriptc inferpipeline from #784 so the pins can be applied at compile time.Changes:
cb57d55fthe export table carries global slots, so a cross-module read of a typed module global resolves natively. Tests intests/test_global_export.py.d3902d6binfernamed modules from the file's own path, so a pin never reached its definition; it now names them the way the lowerer does. Tests intests/test_infer_module_names.py.931c1069compile --inferapplies the package-wide pins, so an unannotated parameter whose call sites agree lowers natively. Costs one checker pass plus one pinned re-check (about 262 s on the 1,041-file yt-dlp tree). Tests intests/test_infer_pin_native.py.Proof: a two-module program with an unannotated
helpers.parse_countlowers todefine i64 @helpers.parse_count(%ps.str, i64)called directly from__main__.main, with no interpreter in the process, and prints the CPython-matching output. Without--inferthe same tree lowers nothing and refuses.yt-dlp numbers (coverage): native 2,291 to 2,309, 33 PS1001 fences lifted by the pins.
utils/_utils.pystays at 253 PS1001 fences, which is a yt-dlp typing gap.The finding that matters: the island tier's granularity is the program, not the module.
_program_islandinpackage.pyroutes the whole program island-side when any module is fenced, so yt-dlp's executed fraction is 0% native regardless of the 2,309 capable definitions. Closingbinascii,codecsandcollections.abcwould not change that (the 122 definitions they gate are fenced for other reasons too). A per-module or per-definition plane split is the leverage point.