Modules/posixmodule.c

Source:

cpython 3.14 @ ab2d84fe1023/Modules/posixmodule.c ↗

posixmodule.c is the largest C file in CPython. It implements the os module on POSIX platforms and the nt module on Windows. A top-of-file #ifdef MS_WINDOWS block selects the module name; the rest of the file is shared between both platforms with per-platform sections guarded by further #ifdef blocks. The pure-Python os.py in Lib/ imports this extension and re-exports everything, adding path-manipulation helpers and os.path delegation.

Map

Symbol	Kind	Lines (approx)	Purpose
#ifdef MS_WINDOWS name selection	preprocessor	1-80	Sets MODNAME to "nt" or "posix"
stat_result / StatResult	struct + type	400-600	Holds os.stat() return values
os_stat_impl	function	2200-2320	os.stat(path, ...) implementation
os_lstat_impl	function	2325-2380	os.lstat variant (no symlink follow)
os_open_impl	function	3100-3200	os.open(path, flags, mode)
os_read_impl	function	3250-3310	os.read(fd, n)
os_write_impl	function	3315-3380	os.write(fd, data)
os_close_impl	function	3400-3430	os.close(fd)
DirEntry	struct + type	8500-8700	Single entry from os.scandir() iterator
ScandirIterator	struct + type	8750-8900	State for the os.scandir() iterator
os_scandir_impl	function	8950-9100	Opens directory and returns ScandirIterator
ScandirIterator_iternext	function	9105-9250	Yields DirEntry objects one at a time
DirEntry_stat	method	9300-9400	Cached stat() on a DirEntry
os_getcwd_impl	function	5100-5160	os.getcwd()
os_chdir_impl	function	5200-5240	os.chdir(path)
os_listdir_impl	function	8200-8350	os.listdir() (non-iterator, returns list)
path_converter	function	700-900	Converts path arg to fd or char* or wchar_t*
posix_module_exec	function	15800-16000	Module init: registers types, constants, os.environ

Reading

os.stat: stat_result struct and os_stat_impl

stat_result is a Python structseq type. structseq is CPython's fixed-length tuple subclass, used here so that st_mode, st_ino, etc. are accessible both by name and by index (for backward compatibility with os.stat()[0]).

// CPython: Modules/posixmodule.c:400 stat_result_desc
static PyStructSequence_Field stat_result_fields[] = {
    {"st_mode",  "protection bits"},
    {"st_ino",   "inode"},
    {"st_dev",   "device"},
    {"st_nlink", "number of hard links"},
    {"st_uid",   "user ID of file owner"},
    {"st_gid",   "group ID of file owner"},
    {"st_size",  "total size, in bytes"},
    ...
    {0}
};

os_stat_impl is generated by the Argument Clinic tool (Modules/clinic/posixmodule.c.h) and then calls os_do_stat, a shared helper that invokes fstatat(2) on POSIX or GetFileInformationByHandleEx on Windows. The result is converted to a stat_result via _Py_set_53bit_double for the timestamp fields, which preserves nanosecond resolution as a Python float without losing the integer second.

// CPython: Modules/posixmodule.c:2280 os_stat_impl
static PyObject *
os_stat_impl(PyObject *module, path_t *path, int dir_fd, int follow_symlinks)
{
    return posix_do_stat(module, "stat", path, follow_symlinks,
                         dir_fd, NULL);
}

os.open, os.read, os.write: POSIX wrappers

os_open_impl calls open(2) after translating the Python flags integer and mode integer directly. On Windows it calls _wopen with the same signature. The file descriptor returned is a plain C int; CPython wraps it in a Python int object but adds no fd-tracking abstraction at the C level.

// CPython: Modules/posixmodule.c:3120 os_open_impl
static int
os_open_impl(PyObject *module, path_t *path, int flags, int mode, int dir_fd)
{
    int fd;
    ...
    Py_BEGIN_ALLOW_THREADS
    fd = OPEN(path->narrow, flags, mode);
    Py_END_ALLOW_THREADS
    if (fd < 0)
        return posix_error();
    return fd;
}

Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS release the GIL around blocking syscalls. Both os_read_impl and os_write_impl follow the same pattern, calling read(2) / write(2) inside those macros.

// CPython: Modules/posixmodule.c:3260 os_read_impl
static PyObject *
os_read_impl(PyObject *module, int fd, Py_ssize_t length)
{
    Py_buffer buf;
    ...
    Py_BEGIN_ALLOW_THREADS
    n = read(fd, buf.buf, length);
    Py_END_ALLOW_THREADS
    ...
}

os_write_impl accepts a Py_buffer (any bytes-like object) and returns the number of bytes written as a Python int, matching POSIX write(2) partial-write semantics. It does not retry on EINTR; that is handled by a higher-level os.write wrapper in Lib/os.py.

DirEntry and os.scandir iterator

os.scandir() avoids the cost of calling stat(2) once per entry on most platforms. On Linux it uses getdents64(2) (via opendir/readdir); on Windows it uses FindFirstFileW/FindNextFileW. Both paths populate a DirEntry object with the file type and inode already known from the directory listing, so DirEntry.is_dir() and DirEntry.is_file() are typically free.

// CPython: Modules/posixmodule.c:8520 DirEntry struct
typedef struct {
    PyObject_HEAD
    PyObject *name;
    PyObject *path;
    PyObject *stat;
    PyObject *lstat;
#ifdef MS_WINDOWS
    struct _Py_stat_struct win32_lstat;
    uint64_t win32_file_index;
    ...
#else
    ino_t d_ino;
    unsigned char d_type;
#endif
} DirEntry;

ScandirIterator_iternext calls readdir(3) in a loop, skipping . and .., and constructs a new DirEntry for each result. The DirEntry.stat() method is lazy: it stores the result in DirEntry.stat (or DirEntry.lstat) on first access and returns the cached value on subsequent calls.

// CPython: Modules/posixmodule.c:9105 ScandirIterator_iternext
static PyObject *
ScandirIterator_iternext(ScandirIterator *iterator)
{
    ...
    while (1) {
        errno = 0;
        direntp = readdir(iterator->dirp);
        if (direntp == NULL) break;
        if (direntp->d_name[0] == '.' &&
            (direntp->d_name[1] == '\0' ||
             (direntp->d_name[1] == '.' && direntp->d_name[2] == '\0')))
            continue;
        return DirEntry_from_posix_info(iterator->module, iterator, direntp);
    }
    ...
}

os.path delegation and the WIN32/POSIX module name split

At the top of the file, #ifdef MS_WINDOWS controls two things: the C-level module name string and the selection of ntpath vs posixpath for os.path.

// CPython: Modules/posixmodule.c:60 module name selection
#ifdef MS_WINDOWS
#  define MODNAME "nt"
#else
#  define MODNAME "posix"
#endif

When posix_module_exec runs it sets sys.modules["posix"] (or "nt") to the newly-constructed module, then imports either posixpath or ntpath and binds the result to os.path on the Python side. The C code does not implement any path manipulation; it only wires up the delegation.

// CPython: Modules/posixmodule.c:15850 posix_module_exec path setup
static int
posix_module_exec(PyObject *module)
{
    ...
#ifdef MS_WINDOWS
    if (PyModule_AddObject(module, "path", import_and_init("ntpath")) < 0)
        return -1;
#else
    if (PyModule_AddObject(module, "path", import_and_init("posixpath")) < 0)
        return -1;
#endif
    ...
}

gopy notes

Status: not yet ported.

Planned package path: module/os/

The stat_result struct should map to a Go struct implementing the structseq protocol via objects/. The os_open_impl / os_read_impl / os_write_impl triad are direct wrappers around Go's os.OpenFile, (*os.File).Read, and (*os.File).Write, but the gopy layer must use raw file descriptors (via os.NewFile) to match CPython's fd-based API rather than the Go file handle model.

The scandir iterator is the highest-complexity item. It requires a Go struct holding an *os.File (used as a directory handle), with iternext calling (*os.File).Readdir(1) or the lower-level syscall.ReadDirent to preserve the d_type field needed for DirEntry.is_dir() without a stat call.

The #ifdef MS_WINDOWS name split is not needed in gopy because the os package abstracts over platforms, but the os.path delegation should still point to the appropriate pure-Python path module for the current OS.