Lib/ast.py (part 2)

Source:

cpython 3.14 @ ab2d84fe1023/Lib/ast.py ↗

This annotation covers AST traversal and transformation. See lib_ast_detail (part 1) for ast.parse, AST node types, ast.fix_missing_locations, and ast.dump.

Map

Lines	Symbol	Role
1-80	ast.NodeVisitor	Read-only AST traversal
81-160	ast.NodeTransformer	Mutating AST traversal
161-240	ast.literal_eval	Safely evaluate constant expressions
241-360	ast.unparse	Reconstruct source code from an AST
361-500	ast.walk	Iterate all nodes in the tree

Reading

NodeVisitor

# CPython: Lib/ast.py:480 NodeVisitor
class NodeVisitor:
    def visit(self, node):
        method = 'visit_' + node.__class__.__name__
        visitor = getattr(self, method, self.generic_visit)
        return visitor(node)

    def generic_visit(self, node):
        for field, value in ast.iter_fields(node):
            if isinstance(value, list):
                for item in value:
                    if isinstance(item, AST):
                        self.visit(item)
            elif isinstance(value, AST):
                self.visit(value)

NodeVisitor.visit dispatches to visit_FunctionDef, visit_Call, etc. based on the node class name. generic_visit descends into all child nodes. Subclasses override specific visit_X methods; unhandled nodes fall through to generic_visit.

NodeTransformer

# CPython: Lib/ast.py:520 NodeTransformer
class NodeTransformer(NodeVisitor):
    def generic_visit(self, node):
        for field, old_value in ast.iter_fields(node):
            if isinstance(old_value, list):
                new_values = []
                for old_node in old_value:
                    if isinstance(old_node, AST):
                        value = self.visit(old_node)
                        if value is None:
                            continue  # remove node
                        elif not isinstance(value, AST):
                            new_values.extend(value)  # replace with list
                            continue
                    new_values.append(value)
                old_value[:] = new_values
            elif isinstance(old_value, AST):
                new_node = self.visit(old_value)
                if new_node is None:
                    delattr(node, field)
                else:
                    setattr(node, field, new_node)
        return node

NodeTransformer returns the (possibly modified) node from each visitor. Return None to delete a node. Return a list to replace one statement with multiple. Used for macro expansion, instrumentation, and optimization passes.

ast.literal_eval

# CPython: Lib/ast.py:62 literal_eval
def literal_eval(node_or_string):
    """Safely evaluate an expression node or string containing only
    Python literals: strings, bytes, numbers, tuples, lists, dicts,
    sets, booleans, None, and Ellipsis."""
    def _convert(node):
        if isinstance(node, Constant):
            return node.value
        elif isinstance(node, Tuple):
            return tuple(map(_convert, node.elts))
        elif isinstance(node, List):
            return list(map(_convert, node.elts))
        elif isinstance(node, Set):
            return set(map(_convert, node.elts))
        elif isinstance(node, Dict):
            return dict(zip(map(_convert, node.keys), map(_convert, node.values)))
        elif isinstance(node, UnaryOp) and isinstance(node.op, (UAdd, USub)):
            val = _convert(node.operand)
            return +val if isinstance(node.op, UAdd) else -val
        raise ValueError(f'malformed node or string: {node!r}')
    ...

ast.literal_eval parses the string into an AST and evaluates only safe constant nodes. It cannot call functions, access attributes, or import modules. Safe for deserializing Python-literal config files.

ast.unparse

# CPython: Lib/ast.py:820 unparse
def unparse(ast_obj):
    unparser = _Unparser()
    return unparser.visit(ast_obj)

class _Unparser(NodeVisitor):
    def visit_BinOp(self, node):
        with self.require_parens(node.op, node):
            self.traverse(node.left)
            self.write(f" {self.binop[type(node.op).__name__]} ")
            self.traverse(node.right)

ast.unparse(tree) reconstructs syntactically valid Python source from an AST. The output is not pretty-printed (no newlines in expressions, minimal parentheses). Used for code generation and AST-to-source transformations.

gopy notes

NodeVisitor and NodeTransformer are pure Python via stdlib/ast.py. literal_eval uses the parser to build an AST then walks it in vm/eval_simple.go. ast.unparse is the pure Python implementation shipped in stdlib.