Porting guide, example, etc.

Author: encukou

peps/pep-0793.rst

@@ -152,6 +152,7 @@ Specification
 
 The export hook
 ---------------
+
 When importing an extension module, Python will now first look for an export hook
 like this:
 
@@ -213,10 +214,11 @@ future.
 The ``name`` will be used *instead of* the ``Py_mod_name`` slot (just like
 ``PyModule_FromDefAndSpec`` ignores ``PyModuleDef.m_name``).
 
-To simplify the implementation, the slots arrays for both
-``PyModule_FromSlotsAndSpec`` and the new export hook will only allow up to one
-``Py_mod_exec`` slot.
-(Arrays in ``PyModuleDef.m_slots`` may have more; this will not change.)
+The slots arrays for both ``PyModule_FromSlotsAndSpec`` and the new export hook
+will only allow up to one ``Py_mod_exec`` slot.
+Arrays in ``PyModuleDef.m_slots`` may have more; this will not change.
+This limitation is easy to work around and multiple ``exec`` slots are rarely
+used [#multiexec]_.
 
 For modules created without a ``PyModuleDef``, the ``Py_mod_create`` function
 will be called with ``NULL`` for the second argument (*def*).
@@ -380,12 +382,117 @@ If an existing module is ported to use the new mechanism, then
 We claim that how a module was defined is an implementation detail of that
 module, so this should not be considered a breaking change.
 
+Similarly, ``PyType_GetModuleByDef`` will not match modules that are not
+defined using a *def*.
+The new ``PyType_GetModuleByToken`` function may be used instead.
+
 The ``Py_mod_create`` function may now be called with ``NULL`` for the second
 argument.
 This could trip people porting from *def* to *slots*, so it needs to be
 mentioned in porting notes.
 
 
+Forward compatibility
+---------------------
+
+If a module defines the new export hook, CPython versions that implement this
+PEP will ignore the traditional ``PyInit_*`` hook.
+
+Extensions that straddle vPython ersions are expected to define both hooks;
+each build of CPython will “pick” the newest one that it supports.
+
+
+.. _pep793-porting-notes:
+
+Porting guide
+-------------
+
+Here is a guide to convert an existing module to the new API, including
+some tricky edge cases.
+It should be moved to a HOWTO in the documentation.
+
+#. Scan your code for uses of ``PyModule_GetDef``. This function will
+   return ``NULL`` for modules that use the new mechanism. Instead:
+
+   * For getting the contents of the module's ``PyModuleDef``, use the C struct
+     directly. Alternatively, get attributes from the module using, for
+     example, ``PyModule_GetNameObject``, the ``__doc__`` attribute, and
+     ``PyModule_GetStateSize``.
+     (Note that Python code can mutate a module's attributes.)
+   * For testing if a module object is “yours”, use ``PyModule_GetToken``
+     instead.
+     Later in this guide, you'll set the token to *be* the existing
+     ``PyModuleDef`` structure.
+
+#. Scan your code for uses of ``PyType_GetModuleByDef``, and replace them by
+   ``PyType_GetModuleByToken``.
+   Later in this guide, you'll set the token to *be* the existing
+   ``PyModuleDef`` structure.
+
+#. Look at the function identified by ``Py_mod_create``, if any.
+   Make sure that it does not use its second argument (``PyModuleDef``),
+   as it will be called with ``NULL``.
+   Instead of the argument, use the existing ``PyModuleDef`` struct directly.
+#. If using multiple ``Py_mod_create`` slots, consolidate them: pick one of
+   the functions, or write a new one, and call the others from it.
+   Remove all but one ``Py_mod_create`` slots.
+#. Make a copy of the existing ``PyModuleDef_Slot`` array pointed to by
+   the ``m_slots`` member of your ``PyModuleDef``. If you don't have an
+   existing slots array, create one like this:
+
+   .. code-block:: c
+
+    static PyModuleDef_Slot module_slots[] = {
+        {0}
+    };
+
+   Give this array a unique name.
+   Further examples will assume that you've named it ``module_slots``.
+
+#. Add slots for all members of the existing ``PyModuleDef`` structure.
+   See :ref:`pep793-api-summary` for a list of the new slots.
+   For example, to add a name and docstring:
+
+   .. code-block:: c
+
+    static PyModuleDef_Slot module_slots[] = {
+        {Py_mod_name, "mymodule"},
+        {Py_mod_doc, (char*)PyDoc_STR("my docstring")},
+        // ... (keep existing slots here)
+        {0}
+    };
+
+#. If you switched from ``PyModule_GetDef`` to ``PyModule_GetToken``,
+   and/or from ``PyType_GetModuleByDef`` to ``PyType_GetModuleByToken``,
+   add a ``Py_mod_token`` slot pointing to the existing ``PyModuleDef`` struct:
+
+   .. code-block:: c
+
+    static PyModuleDef_Slot module_slots[] = {
+        // ... (keep existing slots here)
+        {Py_mod_token, your_module_def},
+        {0}
+    };
+
+
+#. Add a new export hook.
+
+   .. code-block:: c
+
+    PyMODEXPORT_FUNC PyModExport_examplemodule(PyObject);
+
+    PyMODEXPORT_FUNC
+    PyModExport_examplemodule(PyObject *spec)
+    {
+        return module_slots;
+    }
+
+The new export hook will be used on Python 3.15 and above.
+Once your module no longer supports lower versions, delete the ``PyInit_``
+function and any unused data.
+
+
+
 Security Implications
 =====================
 
@@ -395,10 +502,16 @@ None known
 How to Teach This
 =================
 
-In addition to regular reference docs, a guide for porting a module from
-*def* to *slots* will be added to the documentation.
+In addition to regular reference docs, the :ref:`pep793-porting-notes` should
+be added as a new HOWTO.
+
+
+Example
+=======
+
+.. literalinclude:: pep-0793/examplemodule.c
+   :language: c
 
-We'll rewrite the "Extending and Embedding" tutorial to use this.
 
 
 Reference Implementation
@@ -475,6 +588,17 @@ The inittab is used for embedding, where a common/stable ABI is not that
 important. So, it might be OK to leave this to a later change.
 
 
+Footnotes
+=========
+
+.. [#multiexec] A `quick survey <https://github.com/python/peps/pull/4435/files#r2105731314>`_
+   of multiple ``Py_mod_exec`` slots found zero uses in the top 15,000 PyPI
+   projects, and three in the stardard library (including tests for the
+   feature).
+   The easy workaround is consolidating the ``exec`` functions; see
+   :ref:`pep793-porting-notes` for details.
+
+
 Copyright
 =========
 

peps/pep-0793/examplemodule.c

@@ -0,0 +1,64 @@
+/*
+Example module with C-level module-global state, and a simple function to
+update and query it.
+Once compiled and renamed to not include a version tag (for example
+examplemodule.so on Linux), this will run succesfully on both regular
+and free-threaded builds.
+
+Python usage:
+
+import examplemodule
+print(examplemodule.increment_value())  # 0
+print(examplemodule.increment_value())  # 1
+print(examplemodule.increment_value())  # 2
+print(examplemodule.increment_value())  # 3
+
+*/
+
+// Avoid CPython-version-specific ABI (inline functions & macros):
+#define Py_LIMITED_API 0x030f0000  // 3.15
+
+#include <Python.h>
+
+typedef struct {
+    int value;
+} examplemodule_state;
+
+PyObject *
+increment_value(PyObject *module, PyObject *_ignored)
+{
+    examplemodule_state *state = PyModule_GetState(module);
+    int result = ++(state->value);
+    return PyLong_FromLong(result);
+}
+
+PyMethodDef examplemodule_methods[] = {
+    {"increment_value", increment_value, METH_NOARGS},
+    {NULL}
+};
+
+int examplemodule_exec(PyObject *module) {
+    examplemodule_state *state = PyModule_GetState(module);
+    state->value = -1;
+    return 0;
+}
+
+PyDoc_STRVAR(examplemodule_doc, "Example extension.");
+
+static PyModuleDef_Slot examplemodule_slots[] = {
+    {Py_mod_name, "examplemodule"},
+    {Py_mod_doc, (char*)examplemodule_doc},
+    {Py_mod_exec, (void*)examplemodule_exec},
+    {Py_mod_methods, examplemodule_methods},
+    {Py_mod_state_size, (void*)sizeof(examplemodule_state)},
+    {0}
+};
+
+// Avoid "implicit declaration of function" warning:
+PyMODEXPORT_FUNC PyModExport_examplemodule(PyObject *);
+
+PyMODEXPORT_FUNC
+PyModExport_examplemodule(PyObject *spec)
+{
+    return examplemodule_slots;
+}