Skip to main content

Python/compile.c

Map​

CPython symbolLines (approx)gopy counterpart
_PyAST_Compile / _PyAST_CompileObject353-410compile/compiler.go Compile
compiler_unit struct610-700compile/codegen.go compilerUnit
compiler struct700-760compile/codegen.go Compiler
new_compiler760-820compile/codegen.go newCompiler
compiler_mod / compile_codegen412-550compile/compiler.go compilerMod
compiler_set_qualname644-680compile/codegen.go setQualname
compiler_function1390-1500compile/codegen_stmt_funclike.go visitFunctionDef
compiler_class / codegen_class1515-1700compile/codegen_class.go visitClassDef
codegen_lambda1999-2050compile/codegen_stmt_funclike.go visitLambda
codegen_nameop3186-3287compile/codegen_expr_name.go visitName
codegen_add_yield_from472-510compile/codegen_expr_misc.go visitYieldFrom
compiler_arguments1311-1390compile/codegen_stmt_funclike.go emitArguments
codegen_make_closure923-990compile/codegen_stmt_funclike.go emitMakeClosure
assemble_exception_tablePython/assemble.c:157compile/assemble_exceptions.go assembleExceptionTable
codegen_comprehensionvariescompile/codegen_expr_comp.go

Reading​

The compiler pipeline​

cpython 3.14 @ ab2d84fe1023/

is the single public entry point. It runs three sequential passes and returns a PyCodeObject.

symtable.Build(ast)          // resolve every name to its scope
  → codegen walk             // emit instructions into a flowgraph
    → flowgraph.Optimize     // eliminate dead blocks, insert prefix ops
      → assemble             // linearise into bytes + exception table

In gopy compile/compiler.go Compile follows the same order: symtable.Build, then compileScope recursively for every nested scope, then flowgraph.Optimize, then assemble.Assemble.

The compiler holds a stack of compiler_unit objects, one per open scope. Entering a function or class pushes a new unit; exiting pops it and finalises the code object. gopy represents this with a compilerUnit value stored in Compiler.scope.

Name resolution and FAST / DEREF slots​

cpython 3.14 @ ab2d84fe1023/

maps a bare name to one of six opcode families depending on the scope flag the symtable stored for that name:

  • DEF_LOCAL with no free-variable status: LOAD_FAST / STORE_FAST / DELETE_FAST
  • CELL or FREE: LOAD_DEREF / STORE_DEREF / DELETE_DEREF and MAKE_CELL on function entry
  • DEF_GLOBAL or implicit global: LOAD_GLOBAL / STORE_GLOBAL
  • DEF_NONLOCAL: routed to the enclosing cell slot
  • class body fall-through: LOAD_NAME / STORE_NAME
// compile/codegen_expr_name.go
func (c *Compiler) visitNameLoad(id ast.Identifier, loc Loc) error {
    scope := c.scope.symEntry.GetScope(string(id))
    switch scope {
    case symtable.ScopeLocal:
        return c.emitFast(LOAD_FAST, id, loc)
    case symtable.ScopeCell, symtable.ScopeFree:
        return c.emitClosure(LOAD_DEREF, id, loc)
    ...
    }
}

Name mangling (__x inside a class body becoming _ClassName__x) is handled by symtable.Mangle before the name ever reaches codegen. That mirrors CPython's _Py_Mangle in Python/symtable.c:3207.

Function and class scope isolation​

cpython 3.14 @ ab2d84fe1023/

compiles the function body inside a fresh compiler_unit, then

cpython 3.14 @ ab2d84fe1023/

emits MAKE_CELL for every cell variable followed by MAKE_FUNCTION (or COPY_FREE_VARS) to lift free variables into the closure tuple.

For classes,

cpython 3.14 @ ab2d84fe1023/

wraps the body in a special scope where __class__ is a cell. gopy compile/codegen_class.go visitClassBody reproduces the same __class__ / __classdict__ preamble.

Comprehensions are isolated the same way: each listcomp, setcomp, dictcomp, or genexpr gets its own compiler_unit. The outer scope passes only the outermost iterable as an argument, and the inner code object is compiled separately. gopy compile/codegen_expr_comp.go emitInnerComprehensionCode follows this pattern. When the symtable marks a comprehension CompInlined, gopy skips the isolation and emits directly into the enclosing scope (CPython 3.12+ behaviour).

PEP 380 yield-from lowering and exception table emission​

yield from x cannot be a single opcode because the inner iterator may need to propagate throw() and close() calls. CPython lowers it to:

GET_YIELD_FROM_ITER
LOAD_CONST None          # initial send value
SEND         <end>       # loop head
YIELD_VALUE
RESUME       1
JUMP_BACKWARD <send>
<end>: END_SEND

gopy compile/codegen_expr_misc.go visitYieldFrom reproduces this exactly (CPython Python/codegen.c:472).

The exception table is a compact byte stream appended to the code object.

cpython 3.14 @ ab2d84fe1023/

walks the linearised instruction stream and emits variable-length entries encoding (start, length, target, depth, lasti) tuples using a base-128 varint. gopy compile/assemble_exceptions.go assembleExceptionTable is a line-for-line port.

gopy notes​

gopy splits Python/compile.c across several files:

  • compile/compiler.go holds the pipeline driver (Compile, compileScope).
  • compile/codegen.go holds Compiler, compilerUnit, enterScope, exitScope, setQualname.
  • compile/codegen_expr_name.go holds visitName and the six opcode-family helpers.
  • compile/codegen_class.go, codegen_stmt_funclike.go, codegen_expr_comp.go hold the corresponding compiler_* / codegen_* ports.
  • compile/assemble_exceptions.go ports Python/assemble.c exception-table emission.

The flowgraph passes (compile/flowgraph_passes.go, flowgraph_jumps.go) correspond to Python/flowgraph.c. Exception block tracking during codegen is in compile/codegen_fblock.go, mirroring CPython's fblockinfo stack.