diff --git a/docs/reference/generics.rst b/docs/reference/generics.rst index c39d2bc1..99ff841d 100644 --- a/docs/reference/generics.rst +++ b/docs/reference/generics.rst @@ -19,11 +19,7 @@ Here is a very simple generic class that represents a stack: .. code-block:: python - from typing import TypeVar, Generic - - T = TypeVar('T') - - class Stack(Generic[T]): + class Stack[T]: def __init__(self) -> None: # Create an empty list with items of type T self.items: list[T] = [] @@ -37,6 +33,23 @@ Here is a very simple generic class that represents a stack: def empty(self) -> bool: return not self.items +.. note:: + + The type parameter syntax (e.g., ``class Foo[T]:``) is available in Python 3.12 and newer. + For earlier Python versions, generic classes need to be defined using + ``TypeVar`` and ``Generic``, as shown below. + +For compatibility with older Python versions, the same class may be written as: + +.. code-block:: python + + from typing import TypeVar, Generic + + T = TypeVar('T') + + class Stack(Generic[T]): + ... + The ``Stack`` class can be used to represent a stack of any type: ``Stack[int]``, ``Stack[tuple[int, str]]``, etc. @@ -56,7 +69,7 @@ construction of the instance will be type checked correspondingly. .. code-block:: python - class Box(Generic[T]): + class Box[T]: def __init__(self, content: T) -> None: self.content = content @@ -70,17 +83,14 @@ Defining subclasses of generic classes ************************************** User-defined generic classes and generic classes defined in :py:mod:`typing` -can be used as a base class for another class (generic or non-generic). For example: +can be used as base classes for other classes (generic or non-generic). For example: .. code-block:: python - from typing import Generic, TypeVar, Mapping, Iterator - - KT = TypeVar('KT') - VT = TypeVar('VT') + from typing import Mapping, Iterator # This is a generic subclass of Mapping - class MyMap(Mapping[KT, VT]): + class MyMap[KT, VT](Mapping[KT, VT]): def __getitem__(self, k: KT) -> VT: ... def __iter__(self) -> Iterator[KT]: ... def __len__(self) -> int: ... @@ -96,12 +106,12 @@ can be used as a base class for another class (generic or non-generic). For exam data: StrDict[int, int] # error: "StrDict" expects no type arguments, but 2 given data2: StrDict # OK - # This is a user-defined generic class - class Receiver(Generic[T]): - def accept(self, value: T) -> None: ... + # This is a user-defined generic class + class Receiver[T]: + def accept(self, value: T) -> None: ... - # This is a generic subclass of Receiver - class AdvancedReceiver(Receiver[T]): ... + # This is a generic subclass of Receiver + class AdvancedReceiver[T](Receiver[T]): ... .. note:: @@ -111,36 +121,21 @@ can be used as a base class for another class (generic or non-generic). For exam protocols like :py:class:`~typing.Iterable`, which use :ref:`structural subtyping `. -:py:class:`Generic ` can be omitted from bases if there are -other base classes that include type variables, such as ``Mapping[KT, VT]`` -in the above example. If you include ``Generic[...]`` in bases, then -it should list all type variables present in other bases (or more, -if needed). The order of type variables is defined by the following -rules: - -* If ``Generic[...]`` is present, then the order of variables is - always determined by their order in ``Generic[...]``. -* If there are no ``Generic[...]`` in bases, then all type variables - are collected in the lexicographic order (i.e. by first appearance). - -For example: +When using the new syntax, the order of type parameters is determined +by their order in the class definition's type parameter list. For example: .. code-block:: python - from typing import Generic, TypeVar, Any - - T = TypeVar('T') - S = TypeVar('S') - U = TypeVar('U') + from typing import Any - class One(Generic[T]): ... - class Another(Generic[T]): ... + class One[T]: ... + class Another[T]: ... - class First(One[T], Another[S]): ... - class Second(One[T], Another[S], Generic[S, U, T]): ... + class First[T, S](One[T], Another[S]): ... + class Second[S, U, T](One[T], Another[S]): ... x: First[int, str] # Here T is bound to int, S is bound to str - y: Second[int, str, Any] # Here T is Any, S is int, and U is str + y: Second[int, str, Any] # Here S is int, U is str, and T is Any .. _generic-functions: @@ -152,12 +147,10 @@ where the types of the arguments or return value have some relationship: .. code-block:: python - from typing import TypeVar, Sequence - - T = TypeVar('T') + from typing import Sequence # A generic function! - def first(seq: Sequence[T]) -> T: + def first[T](seq: Sequence[T]) -> T: return seq[0] As with generic classes, the type variable can be replaced with any @@ -181,17 +174,15 @@ functions do not share any typing relationship to each other: .. code-block:: python - from typing import TypeVar, Sequence - - T = TypeVar('T') + from typing import Sequence - def first(seq: Sequence[T]) -> T: + def first[T](seq: Sequence[T]) -> T: return seq[0] - def last(seq: Sequence[T]) -> T: + def last[T](seq: Sequence[T]) -> T: return seq[-1] -Variables should not have a type variable in their type unless the type variable +Variables should not have a type parameter in their type unless the type parameter is bound by a containing generic class, generic function or generic alias. .. _generic-methods-and-generic-self: @@ -205,16 +196,16 @@ the class definition. .. code-block:: python - # T is the type variable bound by this class - class PairedBox(Generic[T]): + # T is the type parameter bound by this class + class PairedBox[T]: def __init__(self, content: T) -> None: self.content = content - # S is a type variable bound only in this method - def first(self, x: list[S]) -> S: + # S is a type parameter bound only in this method + def first[S](self, x: list[S]) -> S: return x[0] - def pair_with_first(self, x: list[S]) -> tuple[S, T]: + def pair_with_first[S](self, x: list[S]) -> tuple[S, T]: return (x[0], self.content) box = PairedBox("asdf") @@ -228,12 +219,8 @@ methods: .. code-block:: python - from typing import TypeVar - - T = TypeVar('T', bound='Shape') - class Shape: - def set_scale(self: T, scale: float) -> T: + def set_scale[T: Shape](self: T, scale: float) -> T: self.scale = scale return self @@ -259,15 +246,13 @@ For class methods, you can also define generic ``cls``, using :py:class:`type`: .. code-block:: python - from typing import Optional, TypeVar, Type - - T = TypeVar('T', bound='Friend') + from typing import Optional class Friend: other: Optional["Friend"] = None @classmethod - def make_pair(cls: type[T]) -> tuple[T, T]: + def make_pair[T: Friend](cls: type[T]) -> tuple[T, T]: a, b = cls(), cls() a.other = b b.other = a @@ -352,10 +337,12 @@ Let us illustrate this by few simple examples: .. code-block:: python - # We'll use these classes in the examples below - class Shape: ... - class Triangle(Shape): ... - class Square(Shape): ... + from typing import Sequence, Union, Callable + + # We'll use these classes in the examples below + class Shape: ... + class Triangle(Shape): ... + class Square(Shape): ... * Most immutable containers, such as :py:class:`~typing.Sequence` and :py:class:`~typing.FrozenSet` are covariant. :py:data:`~typing.Union` is @@ -422,10 +409,13 @@ Let us illustrate this by few simple examples: Another example of an invariant type is :py:class:`~typing.Dict`. Most mutable containers are invariant. -By default, all user-defined generics are invariant. -To declare a given generic class as covariant or contravariant use -type variables defined with special keyword arguments ``covariant`` or -``contravariant``. For example: +The variance of user-defined generics depends on how type parameters +are used. When using Python 3.12 type parameter syntax, type checkers +can infer the variance automatically. + +Explicit variance annotations are mainly needed when using legacy +``TypeVar`` syntax with the ``covariant`` or ``contravariant`` +arguments. For example: .. code-block:: python @@ -458,25 +448,22 @@ It's therefore often useful to be able to limit the types that a type variable can take on, for instance, by restricting it to values that are subtypes of a specific type. -Such a type is called the upper bound of the type variable, and is specified -with the ``bound=...`` keyword argument to :py:class:`~typing.TypeVar`. +Such a type is called the upper bound of the type parameter. In modern Python +(3.12+), this can be expressed directly in the type parameter list using +``[T: Bound]`` syntax. .. code-block:: python - from typing import TypeVar, SupportsAbs + from typing import SupportsAbs - T = TypeVar('T', bound=SupportsAbs[float]) + def largest_in_absolute_value[T: SupportsAbs[float]](*xs: T) -> T: + return max(xs, key=abs) In the definition of a generic function that uses such a type variable ``T``, the type represented by ``T`` is assumed to be a subtype of its upper bound, so the function can use methods of the upper bound on values of type ``T``. -.. code-block:: python - - def largest_in_absolute_value(*xs: T) -> T: - return max(xs, key=abs) # Okay, because T is a subtype of SupportsAbs[float]. - In a call to such a function, the type ``T`` must be replaced by a type that is a subtype of its upper bound. Continuing the example above: @@ -501,6 +488,12 @@ be avoided if an upper bound can be made to work instead, as above. An example is a type variable that can only have values ``str`` and ``bytes``: +.. note:: + + Constrained type variables defined with :py:class:`~typing.TypeVar` have + slightly different semantics from bounded type parameters (see below), so + the legacy ``TypeVar`` syntax is still commonly used for this pattern. + .. code-block:: python from typing import TypeVar @@ -620,6 +613,7 @@ Here's how one could annotate the decorator: from typing import Any, Callable, TypeVar, cast + # Legacy approach using TypeVar; see ParamSpec below for a better alternative. F = TypeVar('F', bound=Callable[..., Any]) # A decorator that preserves the signature. @@ -651,12 +645,9 @@ for a more faithful type annotation: .. code-block:: python - from typing import Callable, ParamSpec, TypeVar - - P = ParamSpec('P') - T = TypeVar('T') + from typing import Callable - def printing_decorator(func: Callable[P, T]) -> Callable[P, T]: + def printing_decorator[**P, T](func: Callable[P, T]) -> Callable[P, T]: def wrapper(*args: P.args, **kwds: P.kwargs) -> T: print("Calling", func) return func(*args, **kwds) @@ -672,13 +663,10 @@ alter the signature of the input function: .. code-block:: python - from typing import Callable, ParamSpec, TypeVar - - P = ParamSpec('P') - T = TypeVar('T') + from typing import Callable # We reuse 'P' in the return type, but replace 'T' with 'str' - def stringify(func: Callable[P, T]) -> Callable[P, str]: + def stringify[**P, T](func: Callable[P, T]) -> Callable[P, str]: def wrapper(*args: P.args, **kwds: P.kwargs) -> str: return str(func(*args, **kwds)) return wrapper @@ -695,12 +683,11 @@ Or insert an argument: .. code-block:: python - from typing import Callable, Concatenate, ParamSpec, TypeVar - - P = ParamSpec('P') - T = TypeVar('T') + from typing import Callable, Concatenate - def printing_decorator(func: Callable[P, T]) -> Callable[Concatenate[str, P], T]: + def printing_decorator[**P, T]( + func: Callable[P, T], + ) -> Callable[Concatenate[str, P], T]: def wrapper(msg: str, /, *args: P.args, **kwds: P.kwargs) -> T: print("Calling", func, "with", msg) return func(*args, **kwds) @@ -720,6 +707,12 @@ Decorator factories Functions that take arguments and return a decorator (also called second-order decorators), are similarly supported via generics: +.. note:: + + The following examples use legacy ``TypeVar`` syntax because + decorator factories using :py:func:`~typing.overload` do not yet + have a concise PEP 695 equivalent. + .. code-block:: python from typing import Any, Callable, TypeVar @@ -776,11 +769,9 @@ Example: .. code-block:: python - from typing import Protocol, TypeVar - - T = TypeVar('T') + from typing import Protocol - class Box(Protocol[T]): + class Box[T](Protocol): content: T def do_stuff(one: Box[str], other: Box[bytes]) -> None: @@ -818,7 +809,23 @@ return type, but the type variable is invariant: class ReadOnlyBox(Protocol[T]): # error: Invariant type variable "T" used in protocol where covariant one is expected def content(self) -> T: ... -This example correctly uses a covariant type variable: +With PEP 695 syntax, type checkers infer the variance automatically +based on how the type parameter is used. Since ``T`` only appears in +covariant position (return type), this is accepted: + +.. code-block:: python + + from typing import Protocol + + class ReadOnlyBox[T](Protocol): # OK -- T inferred as covariant + def content(self) -> T: ... + + ax: ReadOnlyBox[float] = ... + ay: ReadOnlyBox[int] = ... + ax = ay # OK -- ReadOnlyBox is covariant + +With the legacy ``TypeVar`` syntax, covariance must be declared +explicitly using ``covariant=True``: .. code-block:: python @@ -839,9 +846,9 @@ Generic protocols can also be recursive. Example: .. code-block:: python - T = TypeVar('T') + from typing import Protocol - class Linked(Protocol[T]): + class Linked[T](Protocol): val: T def next(self) -> 'Linked[T]': ... @@ -849,7 +856,7 @@ Generic protocols can also be recursive. Example: val: int def next(self) -> 'L': ... - def last(seq: Linked[T]) -> T: ... + def last[T](seq: Linked[T]) -> T: ... result = last(L()) reveal_type(result) # Revealed type is "builtins.int" @@ -868,28 +875,24 @@ variables replaced with ``Any``. Examples (following :pep:`PEP 484: Type aliases .. code-block:: python - from typing import TypeVar, Iterable, Union, Callable - - S = TypeVar('S') + from typing import Iterable, Union, Callable - TInt = tuple[int, S] - UInt = Union[S, int] - CBack = Callable[..., S] + type TInt[S] = tuple[int, S] + type UInt[S] = Union[S, int] + type CBack[S] = Callable[..., S] def response(query: str) -> UInt[str]: # Same as Union[str, int] ... - def activate(cb: CBack[S]) -> S: # Same as Callable[..., S] + def activate[S](cb: CBack[S]) -> S: # Same as Callable[..., S] ... table_entry: TInt # Same as tuple[int, Any] - T = TypeVar('T', int, float, complex) + type Vec[T: (int, float, complex)] = Iterable[tuple[T, T]] - Vec = Iterable[tuple[T, T]] - - def inproduct(v: Vec[T]) -> T: + def inproduct[T: (int, float, complex)](v: Vec[T]) -> T: return sum(x*y for x, y in v) - def dilate(v: Vec[T], scale: T) -> Vec[T]: + def dilate[T: (int, float, complex)](v: Vec[T], scale: T) -> Vec[T]: return ((x * scale, y * scale) for x, y in v) v1: Vec[int] = [] # Same as Iterable[tuple[int, int]] @@ -903,7 +906,7 @@ defeating the purpose of using aliases. Example: .. code-block:: python - from typing import TypeVar, Generic, Optional + from typing import Optional from example1 import AliasType from example2 import Vec @@ -912,9 +915,7 @@ defeating the purpose of using aliases. Example: def fun() -> AliasType: ... - T = TypeVar('T') - - class NewVec(Vec[T]): + class NewVec[T](Vec[T]): ... for i, j in NewVec[int](): @@ -931,7 +932,9 @@ Generic class internals You may wonder what happens at runtime when you index a generic class. Indexing returns a *generic alias* to the original class that returns instances -of the original class on instantiation: +of the original class on instantiation. The following example uses the legacy +syntax for illustration, but the runtime behavior is identical for classes +defined with the new ``class Stack[T]:`` syntax: .. code-block:: python