Skip to content

Mirror the proto path to the Python package path to eliminate generated-import workarounds #236

Description

@conradbzura

Description

Move proto/wire.proto to proto/wool/protocol/wire.proto so the proto directory structure mirrors the Python package path, and regenerate with the same -Iproto root. With the mirrored layout, grpc_tools.protoc emits an absolute import in the generated gRPC stub (from wool.protocol import wire_pb2 as wool_dot_protocol_dot_wire__pb2) instead of the flat sibling import (import wire_pb2), which makes both existing workarounds deletable:

  • The sys.path.insert hack at the top of src/wool/protocol/_wire.py.
  • The regex import rewrite in build-hooks/build.py (EditableInstallHook.initialize) that patches ^import (\w+_pb2) to from . import ... in generated *_pb2_grpc.py files.

Required changes:

  1. git mv proto/wire.proto proto/wool/protocol/wire.proto.
  2. pyproject.toml: change output_path from src/wool/protocol to src under [tool.hatch.build.hooks.protobuf] (proto_paths = ["proto"] is unchanged). hatch-protobuf passes output_path straight through as --python_out, and its {proto_path} artifact-tracking template resolves relative to the proto root, so both the actual protoc output and the hook's tracked outputs land at src/wool/protocol/wire_pb2*.py.
  3. Delete the sys.path hack and its comment from src/wool/protocol/_wire.py.
  4. Delete the import-rewrite block from build-hooks/build.py.
  5. Update any documented local-regeneration commands to reference the new proto path.

Motivation

The flat sibling import in generated *_pb2_grpc.py files is a long-standing upstream limitation that will not be fixed: protocolbuffers/protobuf#1491 was closed as working-as-intended in 2017 (the protobuf team's position is that generated Python packages are identified by directory structure, so the proto path is supposed to mirror the Python package path), and the gRPC-side issues grpc/grpc#9575 and grpc/grpc#29459 were closed deferring to it. The project currently carries two independent workarounds for the same problem — a runtime sys.path mutation and a build-time regex rewrite — both of which the sanctioned directory-mirroring layout makes unnecessary. The mirrored layout was verified locally: generation from proto/wool/protocol/wire.proto produces absolute imports, and the generated modules import cleanly in package context with no path manipulation.

Expected Outcome

  • The generated wire_pb2_grpc.py imports wire_pb2 via an absolute from wool.protocol import ... import with no post-processing.
  • src/wool/protocol/_wire.py contains no sys.path manipulation.
  • build-hooks/build.py contains no generated-code import rewriting.
  • Wheel, sdist, and editable installs all import wool.protocol cleanly; the full test suite passes.
  • Note: the descriptor-pool registration key changes from wire.proto to wool/protocol/wire.proto — this is a process-local registry key with no wire-format impact.

Metadata

Metadata

Assignees

No one assigned

    Labels

    buildBuild system or dependency changes

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions