Python/compile.c (part 13)

Source:

cpython 3.14 @ ab2d84fe1023/Python/compile.c ↗

This annotation covers comprehension compilation. See compile12_detail for compiler_function, compiler_class, and closure handling.

Map

Lines	Symbol	Role
1-80	compiler_comprehension	Entry point for all comprehension types
81-180	Inlined comprehension	Compile directly into enclosing scope (3.12+)
181-280	Non-inlined comprehension	Emit as nested function + CALL
281-380	compiler_listcomp	[x for x in it if cond]
381-600	compiler_genexp / compiler_setcomp / compiler_dictcomp	Other forms

Reading

compiler_comprehension

// CPython: Python/compile.c:5420 compiler_comprehension
static int
compiler_comprehension(struct compiler *c, expr_ty e, int type,
                       identifier name, asdl_comprehension_seq *generators,
                       expr_ty elt, expr_ty val)
{
    PySTEntryObject *entry = _PyST_Lookup(c->c_st, (PyObject *)e);
    int is_inlined = entry->ste_comp_inlined;
    if (is_inlined) {
        return compiler_comprehension_inlined(c, e, type, generators, elt, val);
    }
    return compiler_comprehension_nested(c, e, type, name, generators, elt, val);
}

Since Python 3.12, CPython inlines comprehensions that don't require a new scope (no walrus operator leaking into enclosing scope, no yield inside). The symbol table determines this at analysis time.

Inlined comprehension

// CPython: Python/compile.c:5460 compiler_comprehension_inlined
static int
compiler_comprehension_inlined(struct compiler *c, ...)
{
    /* No new frame: compile directly into the current function's bytecode.
       The iteration variable is local to the comprehension but in the
       enclosing frame's localsplus. */
    VISIT(c, expr, outermost_iter);   /* push the iterable */
    compiler_comprehension_generator(c, generators, 0, 0, elt, val, type);
    /* Result: list/set/dict/generator left on stack */
}

Inlined comprehensions don't create a nested function call. The iteration is compiled directly into the outer function's bytecode, with the loop variable stored in dedicated localsplus slots. This eliminates the overhead of creating and calling a nested function.

Non-inlined comprehension

// CPython: Python/compile.c:5510 compiler_comprehension_nested
static int
compiler_comprehension_nested(struct compiler *c, ...)
{
    /* Compile as: def <listcomp>(iter): ... */
    compiler_enter_scope(c, name, COMPILER_SCOPE_COMPREHENSION, ...);
    /* Inside new scope: .0 is the outermost iterator */
    compiler_comprehension_generator(c, generators, 0, 0, elt, val, type);
    ADDOP(c, RETURN_VALUE);
    compiler_exit_scope(c);
    /* In outer scope: push iterator, MAKE_FUNCTION, CALL */
    VISIT(c, expr, outermost_iter);
    ADDOP_I(c, GET_ITER, 0);
    ...
    ADDOP_I(c, CALL, 1);
}

Non-inlined comprehensions are compiled as nested functions. The outermost iterable is evaluated in the enclosing scope (before MAKE_FUNCTION) and passed as the first argument. Inner for clauses are evaluated inside the function.

compiler_listcomp

// CPython: Python/compile.c:5600 compiler_listcomp
static int
compiler_listcomp(struct compiler *c, expr_ty e)
{
    assert(e->kind == ListComp_kind);
    return compiler_comprehension(c, e, COMP_LISTCOMP, &_Py_ID(listcomp),
                                  e->v.ListComp.generators,
                                  e->v.ListComp.elt, NULL);
}

List, set, dict comprehensions, and generator expressions all route through compiler_comprehension with a different type constant (COMP_LISTCOMP, COMP_SETCOMP, COMP_DICTCOMP, COMP_GENEXP). The type controls which BUILD_LIST/SET_ADD/MAP_ADD/YIELD_VALUE opcodes are emitted.

Generator expressions

// CPython: Python/compile.c:5640 compiler_genexp
static int
compiler_genexp(struct compiler *c, expr_ty e)
{
    /* Generator expressions are always non-inlined:
       they need a frame to hold the suspended state */
    return compiler_comprehension(c, e, COMP_GENEXP, &_Py_ID(genexpr),
                                  e->v.GeneratorExp.generators,
                                  e->v.GeneratorExp.elt, NULL);
}

Generator expressions ((x for x in it)) are never inlined even in Python 3.12+: they need a suspended frame (generator object) to implement lazy evaluation. RETURN_GENERATOR is emitted at the start of the compiled function.

gopy notes

compiler_comprehension is in compile/codegen_expr.go. Inlined comprehensions are compiled with emitComprehensionInlined. Non-inlined call emitComprehensionNested which calls compileFunction for the inner scope. The type byte maps to LIST_APPEND/SET_ADD/MAP_ADD in compile/codegen_stmt.go.