Skip to main content

Include/internal/pycore_ast.h

cpython 3.14 @ ab2d84fe1023/Include/internal/pycore_ast.h

pycore_ast.h is machine-generated by Parser/asdl_c.py from Python/Python.asdl. It should never be edited by hand. The grammar file defines every AST node kind in an ASDL schema, and the generator emits this header together with the corresponding C source file. Changing the grammar and re-running the generator is the only supported workflow.

The header serves two audiences: the parser, which calls the _PyAST_* constructor functions to build a tree; and the compiler, which walks the tree by switching on the kind field of each union. Both sides agree on the same layout because they both include this single header. The PyArena allocator is threaded through every constructor so that the entire tree lives in one arena and can be freed in a single operation.

Validation and object-protocol functions at the bottom of the file form a thin public surface used by ast.py and by the C API. _PyAST_Validate enforces structural invariants after parsing, and PyAST_obj2mod / PyAST_mod2obj bridge between Python-level ast objects and the internal C structs.

Map​

LinesSymbolRolegopy
1file headerAuto-generation notice and include guardsn/a
15-55mod_ty, stmt_ty, expr_ty, ...Forward typedef declarations for all node pointer typesn/a
56-120asdl_*_seq types and _Py_asdl_*_seq_newTyped sequence containers for each node categoryn/a
121-900_PyAST_Module, _PyAST_FunctionDef, ...Constructor functions, one per grammar production (~70 total)n/a
901-1100struct definitions for all node kindsUnion layouts for mod, stmt, expr, pattern, etc.n/a
1101-1172_PyAST_Validate, _PyAST_ExprAsUnicode, PyAST_obj2mod, PyAST_mod2obj, _PyAST_GetDocStringValidation and object-protocol entrypointsn/a

Reading​

Forward declarations and typed sequences (lines 1 to 120)​

cpython 3.14 @ ab2d84fe1023/Include/internal/pycore_ast.h#L1-120

The file opens with a block of typedef struct _xxx *xxx_ty forward declarations that let the rest of the codebase use opaque pointer types. Following those are typed sequence structs (asdl_stmt_seq, asdl_expr_seq, and so on) each wrapping _ASDL_SEQ_HEAD from pycore_asdl.h. A matching _Py_asdl_*_seq_new(size, arena) allocator is declared alongside each sequence type.

typedef struct _mod *mod_ty;
typedef struct _stmt *stmt_ty;
typedef struct _expr *expr_ty;

typedef struct {
    _ASDL_SEQ_HEAD
    stmt_ty typed_elements[1];
} asdl_stmt_seq;

asdl_stmt_seq *_Py_asdl_stmt_seq_new(Py_ssize_t size, PyArena *arena);

Enumeration types (lines 21-55)​

cpython 3.14 @ ab2d84fe1023/Include/internal/pycore_ast.h#L21-55

Enums encode the fixed sets of operators and contexts. expr_context_ty has three values (Load, Store, Del). operator_ty covers all thirteen binary operators from Add through FloorDiv. cmpop_ty covers the ten comparison operators. These enums are embedded directly in expression nodes and are therefore part of the stable ABI surface that ast.py must match.

typedef enum _expr_context { Load=1, Store=2, Del=3 } expr_context_ty;
typedef enum _operator {
    Add=1, Sub=2, Mult=3, MatMult=4, Div=5, Mod=6, Pow=7,
    LShift=8, RShift=9, BitOr=10, BitXor=11, BitAnd=12,
    FloorDiv=13
} operator_ty;

Constructor functions (lines 121 to 900)​

cpython 3.14 @ ab2d84fe1023/Include/internal/pycore_ast.h#L121-900

Each grammar production has a corresponding constructor. All constructors follow the same signature convention: positional fields first, the owning PyArena *arena last. Constructors return NULL on allocation failure. Because the arena owns all memory, callers never free individual nodes. There are roughly 70 constructors covering modules, statements, expressions, exception handlers, pattern-match arms, and type parameters introduced in Python 3.12.

mod_ty _PyAST_Module(
    asdl_stmt_seq *body,
    asdl_type_ignore_seq *type_ignores,
    PyArena *arena);

expr_ty _PyAST_BoolOp(
    boolop_ty op,
    asdl_expr_seq *values,
    int lineno, int col_offset,
    int end_lineno, int end_col_offset,
    PyArena *arena);

Validation and object protocol (lines 1101 to 1172)​

cpython 3.14 @ ab2d84fe1023/Include/internal/pycore_ast.h#L1101-1172

_PyAST_Validate(mod_ty node, int flags) performs a post-parse structural check and is called by compile() before code generation. _PyAST_ExprAsUnicode(expr_ty e) converts an f-string expression node to a Unicode object for display in tracebacks. PyAST_mod2obj and PyAST_obj2mod convert between the internal C tree and Python-level ast module objects, making it possible for Python code to inspect or mutate the AST before compilation.

int _PyAST_Validate(mod_ty mod, int flags);
PyObject* _PyAST_ExprAsUnicode(expr_ty);
PyObject* _PyAST_GetDocString(asdl_stmt_seq *body);
PyObject* PyAST_mod2obj(mod_ty ast);
mod_ty PyAST_obj2mod(PyObject *ast, PyArena *arena, int mode);
int PyAST_Check(PyObject *obj);

gopy mirror​

Not yet ported.