Skip to main content

pycore_import.h: Internal Import Machinery

pycore_import.h declares the internal API for the import system. The public __import__ builtin and importlib call into this layer; it is also the seam between the interpreter core and extension modules (.so/.pyd files). The 3.14 changes reorganise per-interpreter state and add stricter isolation for sub-interpreters.

Map

SymbolKindLines (approx)Notes
_Py_frozen_modulestruct20-30Describes one entry in the built-in frozen module table
_PyImport_FrozenModulesextern32Null-terminated array of _Py_frozen_module
_PyImport_FrozenBootstrapextern33Subset used during Py_Initialize before importlib is ready
_PyImport_FrozenStdlibextern34Standard-library frozen modules (added 3.11)
_PyImport_FixupBuiltinfunction decl40-44Register a newly loaded built-in module in sys.modules
_PyImport_FixupExtensionObjectfunction decl46-52Cache a loaded extension module for reuse across imports
_PyImport_LoadDynamicModuleWithSpecfunction decl54-62Load a .so/.pyd extension from a ModuleSpec
_PyImport_InterpreterDatastruct70-105Per-interpreter import state (3.14 rename from _import_state)
_PyImport_InterpreterData.modulesfield75The sys.modules dict for this interpreter
_PyImport_InterpreterData.modules_by_indexfield78List indexed by m_base.m_index for fast lookup
_PyImport_InterpreterData.importlibfield82Cached reference to the _bootstrap module
_PyImport_InterpreterData.import_funcfield85Cached __import__ callable
_PyImport_InterpreterData.pkgcontextfield90Current package for relative imports

Reading

Frozen module table

The frozen module table is a simple sorted array terminated by a {NULL, NULL, 0} sentinel.

// Include/internal/pycore_import.h:20
struct _Py_frozen_module {
const char *name;
const unsigned char *code; /* marshalled code object bytes */
int size; /* negative size means a package */
};

extern const struct _Py_frozen_module _PyImport_FrozenModules[];
extern const struct _Py_frozen_module _PyImport_FrozenBootstrap[];
extern const struct _Py_frozen_module _PyImport_FrozenStdlib[];

A negative size signals that the frozen entry is a package (it has an __path__). The bootstrap table is searched first during early initialization, before the full frozen table is consulted.

_PyImport_FixupExtensionObject

// Include/internal/pycore_import.h:46
extern int
_PyImport_FixupExtensionObject(
PyObject *mod,
PyObject *name, /* fully-qualified name, e.g. "numpy.core._multiarray" */
PyObject *filename, /* path to the .so */
PyObject *modules); /* sys.modules dict */

After dlopen and the module init function return a live PyModuleObject, this function stores it in two places: sys.modules[name] and the per-interpreter modules_by_index list. On a second import of the same extension, _PyImport_FindExtensionObject finds the cached entry and returns it without calling dlopen again.

_PyImport_InterpreterData (3.14)

The struct was renamed from _import_state in 3.13 to _PyImport_InterpreterData in 3.14 as part of the sub-interpreter isolation work. Each PyInterpreterState embeds one instance.

// Include/internal/pycore_import.h:70
struct _PyImport_InterpreterData {
PyObject *modules; /* sys.modules */
PyObject *modules_by_index; /* list, indexed by m_base.m_index */
PyObject *importlib; /* _bootstrap module */
PyObject *import_func; /* builtins.__import__ */
/* ... lock and path-related fields omitted for brevity ... */
};

In 3.14, modules_by_index is no longer shared across sub-interpreters. Each interpreter gets its own index space, which removes a class of use-after-free bugs when an extension is unloaded in one interpreter while another is still running.

gopy notes

  • vm/eval_import.go implements IMPORT_NAME and IMPORT_FROM opcodes. It currently resolves modules through the module/ package tree rather than the CPython frozen-table mechanism.
  • The frozen module table concept maps to stdlibinit/registry.go, which registers built-in Go module implementations at startup.
  • _PyImport_FixupExtensionObject has no direct equivalent yet; gopy modules are registered at compile time, not loaded via dlopen.
  • _PyImport_InterpreterData.pkgcontext for relative imports is partially handled in vm/eval_import.go via the __package__ attribute lookup.
  • The 3.14 per-interpreter index isolation is not yet relevant because gopy runs a single interpreter, but the field layout should be followed when multi-interpreter support is added.