Capsules

Capsules¶
Refer to Providing a C API for an Extension Module for more information on using these objects.
Added in version 3.1.
-
type PyCapsule¶
This subtype of
PyObject
represents an opaque value, useful for C extension modules which need to pass an opaque value (as a void* pointer) through Python code to other C code. It is often used to make a C function pointer defined in one module available to other modules, so the regular import mechanism can be used to access C APIs defined in dynamically loaded modules.
-
PyTypeObject PyCapsule_Type¶
- Part of the Stable ABI.
The type object corresponding to capsule objects. This is the same object as
types.CapsuleType
in the Python layer.
-
type PyCapsule_Destructor¶
- Part of the Stable ABI.
The type of a destructor callback for a capsule. Defined as:
typedef void (*PyCapsule_Destructor)(PyObject *);
See
PyCapsule_New()
for the semantics of PyCapsule_Destructor callbacks.
-
int PyCapsule_CheckExact(PyObject *p)¶
Return true if its argument is a
PyCapsule
. This function always succeeds.
-
PyObject *PyCapsule_New(void *pointer, const char *name, PyCapsule_Destructor destructor)¶
- Return value: New reference. Part of the Stable ABI.
Create a
PyCapsule
encapsulating the pointer. The pointer argument may not beNULL
.On failure, set an exception and return
NULL
.The name string may either be
NULL
or a pointer to a valid C string. If non-NULL
, this string must outlive the capsule. (Though it is permitted to free it inside the destructor.)If the destructor argument is not
NULL
, it will be called with the capsule as its argument when it is destroyed.If this capsule will be stored as an attribute of a module, the name should be specified as
modulename.attributename
. This will enable other modules to import the capsule usingPyCapsule_Import()
.
-
void *PyCapsule_GetPointer(PyObject *capsule, const char *name)¶
- Part of the Stable ABI.
Retrieve the pointer stored in the capsule. On failure, set an exception and return
NULL
.The name parameter must compare exactly to the name stored in the capsule. If the name stored in the capsule is
NULL
, the name passed in must also beNULL
. Python uses the C functionstrcmp()
to compare capsule names.
-
PyCapsule_Destructor PyCapsule_GetDestructor(PyObject *capsule)¶
- Part of the Stable ABI.
Return the current destructor stored in the capsule. On failure, set an exception and return
NULL
.It is legal for a capsule to have a
NULL
destructor. This makes aNULL
return code somewhat ambiguous; usePyCapsule_IsValid()
orPyErr_Occurred()
to disambiguate.
-
void *PyCapsule_GetContext(PyObject *capsule)¶
- Part of the Stable ABI.
Return the current context stored in the capsule. On failure, set an exception and return
NULL
.It is legal for a capsule to have a
NULL
context. This makes aNULL
return code somewhat ambiguous; usePyCapsule_IsValid()
orPyErr_Occurred()
to disambiguate.
-
const char *PyCapsule_GetName(PyObject *capsule)¶
- Part of the Stable ABI.
Return the current name stored in the capsule. On failure, set an exception and return
NULL
.It is legal for a capsule to have a
NULL
name. This makes aNULL
return code somewhat ambiguous; usePyCapsule_IsValid()
orPyErr_Occurred()
to disambiguate.
-
void *PyCapsule_Import(const char *name, int no_block)¶
- Part of the Stable ABI.
Import a pointer to a C object from a capsule attribute in a module. The name parameter should specify the full name to the attribute, as in
module.attribute
. The name stored in the capsule must match this string exactly.This function splits name on the
.
character, and imports the first element. It then processes further elements using attribute lookups.Return the capsule’s internal pointer on success. On failure, set an exception and return
NULL
.Note
If name points to an attribute of some submodule or subpackage, this submodule or subpackage must be previously imported using other means (for example, by using
PyImport_ImportModule()
) for the attribute lookups to succeed.Changed in version 3.3: no_block has no effect anymore.
-
int PyCapsule_IsValid(PyObject *capsule, const char *name)¶
- Part of the Stable ABI.
Determines whether or not capsule is a valid capsule. A valid capsule is non-
NULL
, passesPyCapsule_CheckExact()
, has a non-NULL
pointer stored in it, and its internal name matches the name parameter. (SeePyCapsule_GetPointer()
for information on how capsule names are compared.)In other words, if
PyCapsule_IsValid()
returns a true value, calls to any of the accessors (any function starting withPyCapsule_Get
) are guaranteed to succeed.Return a nonzero value if the object is valid and matches the name passed in. Return
0
otherwise. This function will not fail.
-
int PyCapsule_SetContext(PyObject *capsule, void *context)¶
- Part of the Stable ABI.
Set the context pointer inside capsule to context.
Return
0
on success. Return nonzero and set an exception on failure.
-
int PyCapsule_SetDestructor(PyObject *capsule, PyCapsule_Destructor destructor)¶
- Part of the Stable ABI.
Set the destructor inside capsule to destructor.
Return
0
on success. Return nonzero and set an exception on failure.
-
int PyCapsule_SetName(PyObject *capsule, const char *name)¶
- Part of the Stable ABI.
Set the name inside capsule to name. If non-
NULL
, the name must outlive the capsule. If the previous name stored in the capsule was notNULL
, no attempt is made to free it.Return
0
on success. Return nonzero and set an exception on failure.
-
int PyCapsule_SetPointer(PyObject *capsule, void *pointer)¶
- Part of the Stable ABI.
Set the void pointer inside capsule to pointer. The pointer may not be
NULL
.Return
0
on success. Return nonzero and set an exception on failure.


Streams

Streams¶
Source code: Lib/asyncio/streams.py
Streams are high-level async/await-ready primitives to work with network connections. Streams allow sending and receiving data without using callbacks or low-level protocols and transports.
Here is an example of a TCP echo client written using asyncio streams:
import asyncio
async def tcp_echo_client(message):
reader, writer = await asyncio.open_connection(
'127.0.0.1', 8888)
print(f'Send: {message!r}')
writer.write(message.encode())
await writer.drain()
data = await reader.read(100)
print(f'Received: {data.decode()!r}')
print('Close the connection')
writer.close()
await writer.wait_closed()
asyncio.run(tcp_echo_client('Hello World!'))
See also the Examples section below.
Stream Functions
The following top-level asyncio functions can be used to create and work with streams:
- async asyncio.open_connection(host=None, port=None, *, limit=None, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None, happy_eyeballs_delay=None, interleave=None)¶
Establish a network connection and return a pair of
(reader, writer)
objects.The returned reader and writer objects are instances of
StreamReader
andStreamWriter
classes.limit determines the buffer size limit used by the returned
StreamReader
instance. By default the limit is set to 64 KiB.The rest of the arguments are passed directly to
loop.create_connection()
.Note
The sock argument transfers ownership of the socket to the
StreamWriter
created. To close the socket, call itsclose()
method.Changed in version 3.7: Added the ssl_handshake_timeout parameter.
Changed in version 3.8: Added the happy_eyeballs_delay and interleave parameters.
Changed in version 3.10: Removed the loop parameter.
Changed in version 3.11: Added the ssl_shutdown_timeout parameter.
- async asyncio.start_server(client_connected_cb, host=None, port=None, *, limit=None, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None, keep_alive=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None, start_serving=True)¶
Start a socket server.
The client_connected_cb callback is called whenever a new client connection is established. It receives a
(reader, writer)
pair as two arguments, instances of theStreamReader
andStreamWriter
classes.client_connected_cb can be a plain callable or a coroutine function; if it is a coroutine function, it will be automatically scheduled as a
Task
.limit determines the buffer size limit used by the returned
StreamReader
instance. By default the limit is set to 64 KiB.The rest of the arguments are passed directly to
loop.create_server()
.Note
The sock argument transfers ownership of the socket to the server created. To close the socket, call the server’s
close()
method.Changed in version 3.7: Added the ssl_handshake_timeout and start_serving parameters.
Changed in version 3.10: Removed the loop parameter.
Changed in version 3.11: Added the ssl_shutdown_timeout parameter.
Changed in version 3.13: Added the keep_alive parameter.
Unix Sockets
- async asyncio.open_unix_connection(path=None, *, limit=None, ssl=None, sock=None, server_hostname=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None)¶
Establish a Unix socket connection and return a pair of
(reader, writer)
.Similar to
open_connection()
but operates on Unix sockets.See also the documentation of
loop.create_unix_connection()
.Note
The sock argument transfers ownership of the socket to the
StreamWriter
created. To close the socket, call itsclose()
method.Availability: Unix.
Changed in version 3.7: Added the ssl_handshake_timeout parameter. The path parameter can now be a path-like object
Changed in version 3.10: Removed the loop parameter.
Changed in version 3.11: Added the ssl_shutdown_timeout parameter.
- async asyncio.start_unix_server(client_connected_cb, path=None, *, limit=None, sock=None, backlog=100, ssl=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None, start_serving=True, cleanup_socket=True)¶
Start a Unix socket server.
Similar to
start_server()
but works with Unix sockets.If cleanup_socket is true then the Unix socket will automatically be removed from the filesystem when the server is closed, unless the socket has been replaced after the server has been created.
See also the documentation of
loop.create_unix_server()
.Note
The sock argument transfers ownership of the socket to the server created. To close the socket, call the server’s
close()
method.Availability: Unix.
Changed in version 3.7: Added the ssl_handshake_timeout and start_serving parameters. The path parameter can now be a path-like object.
Changed in version 3.10: Removed the loop parameter.
Changed in version 3.11: Added the ssl_shutdown_timeout parameter.
Changed in version 3.13: Added the cleanup_socket parameter.
StreamReader¶
- class asyncio.StreamReader¶
Represents a reader object that provides APIs to read data from the IO stream. As an asynchronous iterable, the object supports the
async for
statement.It is not recommended to instantiate StreamReader objects directly; use
open_connection()
andstart_server()
instead.- feed_eof()¶
Acknowledge the EOF.
- async read(n=-1)¶
Read up to n bytes from the stream.
If n is not provided or set to
-1
, read until EOF, then return all readbytes
. If EOF was received and the internal buffer is empty, return an emptybytes
object.If n is
0
, return an emptybytes
object immediately.If n is positive, return at most n available
bytes
as soon as at least 1 byte is available in the internal buffer. If EOF is received before any byte is read, return an emptybytes
object.
- async readline()¶
Read one line, where “line” is a sequence of bytes ending with
\n
.If EOF is received and
\n
was not found, the method returns partially read data.If EOF is received and the internal buffer is empty, return an empty
bytes
object.
- async readexactly(n)¶
Read exactly n bytes.
Raise an
IncompleteReadError
if EOF is reached before n can be read. Use theIncompleteReadError.partial
attribute to get the partially read data.
- async readuntil(separator=b'\n')¶
Read data from the stream until separator is found.
On success, the data and separator will be removed from the internal buffer (consumed). Returned data will include the separator at the end.
If the amount of data read exceeds the configured stream limit, a
LimitOverrunError
exception is raised, and the data is left in the internal buffer and can be read again.If EOF is reached before the complete separator is found, an
IncompleteReadError
exception is raised, and the internal buffer is reset. TheIncompleteReadError.partial
attribute may contain a portion of the separator.The separator may also be a tuple of separators. In this case the return value will be the shortest possible that has any separator as the suffix. For the purposes of
LimitOverrunError
, the shortest possible separator is considered to be the one that matched.Added in version 3.5.2.
Changed in version 3.13: The separator parameter may now be a
tuple
of separators.
- at_eof()¶
Return
True
if the buffer is empty andfeed_eof()
was called.
StreamWriter¶
- class asyncio.StreamWriter¶
Represents a writer object that provides APIs to write data to the IO stream.
It is not recommended to instantiate StreamWriter objects directly; use
open_connection()
andstart_server()
instead.- write(data)¶
The method attempts to write the data to the underlying socket immediately. If that fails, the data is queued in an internal write buffer until it can be sent.
The data buffer should be a bytes, bytearray, or C-contiguous one-dimensional memoryview object.
The method should be used along with the
drain()
method:stream.write(data) await stream.drain()
- writelines(data)¶
The method writes a list (or any iterable) of bytes to the underlying socket immediately. If that fails, the data is queued in an internal write buffer until it can be sent.
The method should be used along with the
drain()
method:stream.writelines(lines) await stream.drain()
- close()¶
The method closes the stream and the underlying socket.
The method should be used, though not mandatory, along with the
wait_closed()
method:stream.close() await stream.wait_closed()
- can_write_eof()¶
Return
True
if the underlying transport supports thewrite_eof()
method,False
otherwise.
- write_eof()¶
Close the write end of the stream after the buffered write data is flushed.
- transport¶
Return the underlying asyncio transport.
- get_extra_info(name, default=None)¶
Access optional transport information; see
BaseTransport.get_extra_info()
for details.
- async drain()¶
Wait until it is appropriate to resume writing to the stream. Example:
writer.write(data) await writer.drain()
This is a flow control method that interacts with the underlying IO write buffer. When the size of the buffer reaches the high watermark, drain() blocks until the size of the buffer is drained down to the low watermark and writing can be resumed. When there is nothing to wait for, the
drain()
returns immediately.
- async start_tls(sslcontext, *, server_hostname=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None)¶
Upgrade an existing stream-based connection to TLS.
Parameters:
sslcontext: a configured instance of
SSLContext
.server_hostname: sets or overrides the host name that the target server’s certificate will be matched against.
ssl_handshake_timeout is the time in seconds to wait for the TLS handshake to complete before aborting the connection.
60.0
seconds ifNone
(default).ssl_shutdown_timeout is the time in seconds to wait for the SSL shutdown to complete before aborting the connection.
30.0
seconds ifNone
(default).
Added in version 3.11.
Changed in version 3.12: Added the ssl_shutdown_timeout parameter.
- is_closing()¶
Return
True
if the stream is closed or in the process of being closed.Added in version 3.7.
Examples¶
TCP echo client using streams¶
TCP echo client using the asyncio.open_connection()
function:
import asyncio
async def tcp_echo_client(message):
reader, writer = await asyncio.open_connection(
'127.0.0.1', 8888)
print(f'Send: {message!r}')
writer.write(message.encode())
await writer.drain()
data = await reader.read(100)
print(f'Received: {data.decode()!r}')
print('Close the connection')
writer.close()
await writer.wait_closed()
asyncio.run(tcp_echo_client('Hello World!'))
See also
The TCP echo client protocol
example uses the low-level loop.create_connection()
method.
TCP echo server using streams¶
TCP echo server using the asyncio.start_server()
function:
import asyncio
async def handle_echo(reader, writer):
data = await reader.read(100)
message = data.decode()
addr = writer.get_extra_info('peername')
print(f"Received {message!r} from {addr!r}")
print(f"Send: {message!r}")
writer.write(data)
await writer.drain()
print("Close the connection")
writer.close()
await writer.wait_closed()
async def main():
server = await asyncio.start_server(
handle_echo, '127.0.0.1', 8888)
addrs = ', '.join(str(sock.getsockname()) for sock in server.sockets)
print(f'Serving on {addrs}')
async with server:
await server.serve_forever()
asyncio.run(main())
See also
The TCP echo server protocol
example uses the loop.create_server()
method.
Get HTTP headers¶
Simple example querying HTTP headers of the URL passed on the command line:
import asyncio
import urllib.parse
import sys
async def print_http_headers(url):
url = urllib.parse.urlsplit(url)
if url.scheme == 'https':
reader, writer = await asyncio.open_connection(
url.hostname, 443, ssl=True)
else:
reader, writer = await asyncio.open_connection(
url.hostname, 80)
query = (
f"HEAD {url.path or '/'} HTTP/1.0\r\n"
f"Host: {url.hostname}\r\n"
f"\r\n"
)
writer.write(query.encode('latin-1'))
while True:
line = await reader.readline()
if not line:
break
line = line.decode('latin1').rstrip()
if line:
print(f'HTTP header> {line}')
# Ignore the body, close the socket
writer.close()
await writer.wait_closed()
url = sys.argv[1]
asyncio.run(print_http_headers(url))
Usage:
python example.py http://example.com/path/page.html
or with HTTPS:
python example.py https://example.com/path/page.html
Register an open socket to wait for data using streams¶
Coroutine waiting until a socket receives data using the
open_connection()
function:
import asyncio
import socket
async def wait_for_data():
# Get a reference to the current event loop because
# we want to access low-level APIs.
loop = asyncio.get_running_loop()
# Create a pair of connected sockets.
rsock, wsock = socket.socketpair()
# Register the open socket to wait for data.
reader, writer = await asyncio.open_connection(sock=rsock)
# Simulate the reception of data from the network
loop.call_soon(wsock.send, 'abc'.encode())
# Wait for data
data = await reader.read(100)
# Got data, we are done: close the socket
print("Received:", data.decode())
writer.close()
await writer.wait_closed()
# Close the second socket
wsock.close()
asyncio.run(wait_for_data())
See also
The register an open socket to wait for data using a protocol example uses a low-level protocol and
the loop.create_connection()
method.
The watch a file descriptor for read events example uses the low-level
loop.add_reader()
method to watch a file descriptor.


More Control Flow Tools

4. More Control Flow Tools¶
As well as the while
statement just introduced, Python uses a few more
that we will encounter in this chapter.
4.1. if
Statements¶
Perhaps the most well-known statement type is the if
statement. For
example:
>>> x = int(input("Please enter an integer: "))
Please enter an integer: 42
>>> if x < 0:
... x = 0
... print('Negative changed to zero')
... elif x == 0:
... print('Zero')
... elif x == 1:
... print('Single')
... else:
... print('More')
...
More
There can be zero or more elif
parts, and the else
part is
optional. The keyword ‘elif
’ is short for ‘else if’, and is useful
to avoid excessive indentation. An if
… elif
…
elif
… sequence is a substitute for the switch
or
case
statements found in other languages.
If you’re comparing the same value to several constants, or checking for specific types or
attributes, you may also find the match
statement useful. For more
details see match Statements.
4.2. for
Statements¶
The for
statement in Python differs a bit from what you may be used
to in C or Pascal. Rather than always iterating over an arithmetic progression
of numbers (like in Pascal), or giving the user the ability to define both the
iteration step and halting condition (as C), Python’s for
statement
iterates over the items of any sequence (a list or a string), in the order that
they appear in the sequence. For example (no pun intended):
>>> # Measure some strings:
>>> words = ['cat', 'window', 'defenestrate']
>>> for w in words:
... print(w, len(w))
...
cat 3
window 6
defenestrate 12
Code that modifies a collection while iterating over that same collection can be tricky to get right. Instead, it is usually more straight-forward to loop over a copy of the collection or to create a new collection:
# Create a sample collection
users = {'Hans': 'active', 'Éléonore': 'inactive', '景太郎': 'active'}
# Strategy: Iterate over a copy
for user, status in users.copy().items():
if status == 'inactive':
del users[user]
# Strategy: Create a new collection
active_users = {}
for user, status in users.items():
if status == 'active':
active_users[user] = status
4.3. The range()
Function¶
If you do need to iterate over a sequence of numbers, the built-in function
range()
comes in handy. It generates arithmetic progressions:
>>> for i in range(5):
... print(i)
...
0
1
2
3
4
The given end point is never part of the generated sequence; range(10)
generates
10 values, the legal indices for items of a sequence of length 10. It
is possible to let the range start at another number, or to specify a different
increment (even negative; sometimes this is called the ‘step’):
>>> list(range(5, 10))
[5, 6, 7, 8, 9]
>>> list(range(0, 10, 3))
[0, 3, 6, 9]
>>> list(range(-10, -100, -30))
[-10, -40, -70]
To iterate over the indices of a sequence, you can combine range()
and
len()
as follows:
>>> a = ['Mary', 'had', 'a', 'little', 'lamb']
>>> for i in range(len(a)):
... print(i, a[i])
...
0 Mary
1 had
2 a
3 little
4 lamb
In most such cases, however, it is convenient to use the enumerate()
function, see Looping Techniques.
A strange thing happens if you just print a range:
>>> range(10)
range(0, 10)
In many ways the object returned by range()
behaves as if it is a list,
but in fact it isn’t. It is an object which returns the successive items of
the desired sequence when you iterate over it, but it doesn’t really make
the list, thus saving space.
We say such an object is iterable, that is, suitable as a target for
functions and constructs that expect something from which they can
obtain successive items until the supply is exhausted. We have seen that
the for
statement is such a construct, while an example of a function
that takes an iterable is sum()
:
>>> sum(range(4)) # 0 + 1 + 2 + 3
6
Later we will see more functions that return iterables and take iterables as
arguments. In chapter Data Structures, we will discuss list()
in more
detail.
4.4. break
and continue
Statements¶
The break
statement breaks out of the innermost enclosing
for
or while
loop:
>>> for n in range(2, 10):
... for x in range(2, n):
... if n % x == 0:
... print(f"{n} equals {x} * {n//x}")
... break
...
4 equals 2 * 2
6 equals 2 * 3
8 equals 2 * 4
9 equals 3 * 3
The continue
statement continues with the next
iteration of the loop:
>>> for num in range(2, 10):
... if num % 2 == 0:
... print(f"Found an even number {num}")
... continue
... print(f"Found an odd number {num}")
...
Found an even number 2
Found an odd number 3
Found an even number 4
Found an odd number 5
Found an even number 6
Found an odd number 7
Found an even number 8
Found an odd number 9
4.5. else
Clauses on Loops¶
In a for
or while
loop the break
statement
may be paired with an else
clause. If the loop finishes without
executing the break
, the else
clause executes.
In a for
loop, the else
clause is executed
after the loop finishes its final iteration, that is, if no break occurred.
In a while
loop, it’s executed after the loop’s condition becomes false.
In either kind of loop, the else
clause is not executed if the
loop was terminated by a break
. Of course, other ways of ending the
loop early, such as a return
or a raised exception, will also skip
execution of the else
clause.
This is exemplified in the following for
loop,
which searches for prime numbers:
>>> for n in range(2, 10):
... for x in range(2, n):
... if n % x == 0:
... print(n, 'equals', x, '*', n//x)
... break
... else:
... # loop fell through without finding a factor
... print(n, 'is a prime number')
...
2 is a prime number
3 is a prime number
4 equals 2 * 2
5 is a prime number
6 equals 2 * 3
7 is a prime number
8 equals 2 * 4
9 equals 3 * 3
(Yes, this is the correct code. Look closely: the else
clause belongs to
the for
loop, not the if
statement.)
One way to think of the else clause is to imagine it paired with the if
inside the loop. As the loop executes, it will run a sequence like
if/if/if/else. The if
is inside the loop, encountered a number of times. If
the condition is ever true, a break
will happen. If the condition is never
true, the else
clause outside the loop will execute.
When used with a loop, the else
clause has more in common with the else
clause of a try
statement than it does with that of if
statements: a try
statement’s else
clause runs when no exception
occurs, and a loop’s else
clause runs when no break
occurs. For more on
the try
statement and exceptions, see Handling Exceptions.
4.6. pass
Statements¶
The pass
statement does nothing. It can be used when a statement is
required syntactically but the program requires no action. For example:
>>> while True:
... pass # Busy-wait for keyboard interrupt (Ctrl+C)
...
This is commonly used for creating minimal classes:
>>> class MyEmptyClass:
... pass
...
Another place pass
can be used is as a place-holder for a function or
conditional body when you are working on new code, allowing you to keep thinking
at a more abstract level. The pass
is silently ignored:
>>> def initlog(*args):
... pass # Remember to implement this!
...
For this last case, many people use the ellipsis literal ...
instead of
pass
. This use has no special meaning to Python, and is not part of
the language definition (you could use any constant expression here), but
...
is used conventionally as a placeholder body as well.
See The Ellipsis Object.
4.7. match
Statements¶
A match
statement takes an expression and compares its value to successive
patterns given as one or more case blocks. This is superficially
similar to a switch statement in C, Java or JavaScript (and many
other languages), but it’s more similar to pattern matching in
languages like Rust or Haskell. Only the first pattern that matches
gets executed and it can also extract components (sequence elements
or object attributes) from the value into variables. If no case matches,
none of the branches is executed.
The simplest form compares a subject value against one or more literals:
def http_error(status):
match status:
case 400:
return "Bad request"
case 404:
return "Not found"
case 418:
return "I'm a teapot"
case _:
return "Something's wrong with the internet"
Note the last block: the “variable name” _
acts as a wildcard and
never fails to match.
You can combine several literals in a single pattern using |
(“or”):
case 401 | 403 | 404:
return "Not allowed"
Patterns can look like unpacking assignments, and can be used to bind variables:
# point is an (x, y) tuple
match point:
case (0, 0):
print("Origin")
case (0, y):
print(f"Y={y}")
case (x, 0):
print(f"X={x}")
case (x, y):
print(f"X={x}, Y={y}")
case _:
raise ValueError("Not a point")
Study that one carefully! The first pattern has two literals, and can
be thought of as an extension of the literal pattern shown above. But
the next two patterns combine a literal and a variable, and the
variable binds a value from the subject (point
). The fourth
pattern captures two values, which makes it conceptually similar to
the unpacking assignment (x, y) = point
.
If you are using classes to structure your data you can use the class name followed by an argument list resembling a constructor, but with the ability to capture attributes into variables:
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def where_is(point):
match point:
case Point(x=0, y=0):
print("Origin")
case Point(x=0, y=y):
print(f"Y={y}")
case Point(x=x, y=0):
print(f"X={x}")
case Point():
print("Somewhere else")
case _:
print("Not a point")
You can use positional parameters with some builtin classes that provide an
ordering for their attributes (e.g. dataclasses). You can also define a specific
position for attributes in patterns by setting the __match_args__
special
attribute in your classes. If it’s set to (“x”, “y”), the following patterns are all
equivalent (and all bind the y
attribute to the var
variable):
Point(1, var)
Point(1, y=var)
Point(x=1, y=var)
Point(y=var, x=1)
A recommended way to read patterns is to look at them as an extended form of what you
would put on the left of an assignment, to understand which variables would be set to
what.
Only the standalone names (like var
above) are assigned to by a match statement.
Dotted names (like foo.bar
), attribute names (the x=
and y=
above) or class names
(recognized by the “(…)” next to them like Point
above) are never assigned to.
Patterns can be arbitrarily nested. For example, if we have a short
list of Points, with __match_args__
added, we could match it like this:
class Point:
__match_args__ = ('x', 'y')
def __init__(self, x, y):
self.x = x
self.y = y
match points:
case []:
print("No points")
case [Point(0, 0)]:
print("The origin")
case [Point(x, y)]:
print(f"Single point {x}, {y}")
case [Point(0, y1), Point(0, y2)]:
print(f"Two on the Y axis at {y1}, {y2}")
case _:
print("Something else")
We can add an if
clause to a pattern, known as a “guard”. If the
guard is false, match
goes on to try the next case block. Note
that value capture happens before the guard is evaluated:
match point:
case Point(x, y) if x == y:
print(f"Y=X at {x}")
case Point(x, y):
print(f"Not on the diagonal")
Several other key features of this statement:
Like unpacking assignments, tuple and list patterns have exactly the same meaning and actually match arbitrary sequences. An important exception is that they don’t match iterators or strings.
Sequence patterns support extended unpacking:
[x, y, *rest]
and(x, y, *rest)
work similar to unpacking assignments. The name after*
may also be_
, so(x, y, *_)
matches a sequence of at least two items without binding the remaining items.Mapping patterns:
{"bandwidth": b, "latency": l}
captures the"bandwidth"
and"latency"
values from a dictionary. Unlike sequence patterns, extra keys are ignored. An unpacking like**rest
is also supported. (But**_
would be redundant, so it is not allowed.)Subpatterns may be captured using the
as
keyword:case (Point(x1, y1), Point(x2, y2) as p2): ...
will capture the second element of the input as
p2
(as long as the input is a sequence of two points)Most literals are compared by equality, however the singletons
True
,False
andNone
are compared by identity.Patterns may use named constants. These must be dotted names to prevent them from being interpreted as capture variables:
from enum import Enum class Color(Enum): RED = 'red' GREEN = 'green' BLUE = 'blue' color = Color(input("Enter your choice of 'red', 'blue' or 'green': ")) match color: case Color.RED: print("I see red!") case Color.GREEN: print("Grass is green") case Color.BLUE: print("I'm feeling the blues :(")
For a more detailed explanation and additional examples, you can look into PEP 636 which is written in a tutorial format.
4.8. Defining Functions¶
We can create a function that writes the Fibonacci series to an arbitrary boundary:
>>> def fib(n): # write Fibonacci series less than n
... """Print a Fibonacci series less than n."""
... a, b = 0, 1
... while a < n:
... print(a, end=' ')
... a, b = b, a+b
... print()
...
>>> # Now call the function we just defined:
>>> fib(2000)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597
The keyword def
introduces a function definition. It must be
followed by the function name and the parenthesized list of formal parameters.
The statements that form the body of the function start at the next line, and
must be indented.
The first statement of the function body can optionally be a string literal; this string literal is the function’s documentation string, or docstring. (More about docstrings can be found in the section Documentation Strings.) There are tools which use docstrings to automatically produce online or printed documentation, or to let the user interactively browse through code; it’s good practice to include docstrings in code that you write, so make a habit of it.
The execution of a function introduces a new symbol table used for the local
variables of the function. More precisely, all variable assignments in a
function store the value in the local symbol table; whereas variable references
first look in the local symbol table, then in the local symbol tables of
enclosing functions, then in the global symbol table, and finally in the table
of built-in names. Thus, global variables and variables of enclosing functions
cannot be directly assigned a value within a function (unless, for global
variables, named in a global
statement, or, for variables of enclosing
functions, named in a nonlocal
statement), although they may be
referenced.
The actual parameters (arguments) to a function call are introduced in the local symbol table of the called function when it is called; thus, arguments are passed using call by value (where the value is always an object reference, not the value of the object). [1] When a function calls another function, or calls itself recursively, a new local symbol table is created for that call.
A function definition associates the function name with the function object in the current symbol table. The interpreter recognizes the object pointed to by that name as a user-defined function. Other names can also point to that same function object and can also be used to access the function:
>>> fib
<function fib at 10042ed0>
>>> f = fib
>>> f(100)
0 1 1 2 3 5 8 13 21 34 55 89
Coming from other languages, you might object that fib
is not a function but
a procedure since it doesn’t return a value. In fact, even functions without a
return
statement do return a value, albeit a rather boring one. This
value is called None
(it’s a built-in name). Writing the value None
is
normally suppressed by the interpreter if it would be the only value written.
You can see it if you really want to using print()
:
>>> fib(0)
>>> print(fib(0))
None
It is simple to write a function that returns a list of the numbers of the Fibonacci series, instead of printing it:
>>> def fib2(n): # return Fibonacci series up to n
... """Return a list containing the Fibonacci series up to n."""
... result = []
... a, b = 0, 1
... while a < n:
... result.append(a) # see below
... a, b = b, a+b
... return result
...
>>> f100 = fib2(100) # call it
>>> f100 # write the result
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
This example, as usual, demonstrates some new Python features:
The
return
statement returns with a value from a function.return
without an expression argument returnsNone
. Falling off the end of a function also returnsNone
.The statement
result.append(a)
calls a method of the list objectresult
. A method is a function that ‘belongs’ to an object and is namedobj.methodname
, whereobj
is some object (this may be an expression), andmethodname
is the name of a method that is defined by the object’s type. Different types define different methods. Methods of different types may have the same name without causing ambiguity. (It is possible to define your own object types and methods, using classes, see Classes) The methodappend()
shown in the example is defined for list objects; it adds a new element at the end of the list. In this example it is equivalent toresult = result + [a]
, but more efficient.
4.9. More on Defining Functions¶
It is also possible to define functions with a variable number of arguments. There are three forms, which can be combined.
4.9.1. Default Argument Values¶
The most useful form is to specify a default value for one or more arguments. This creates a function that can be called with fewer arguments than it is defined to allow. For example:
def ask_ok(prompt, retries=4, reminder='Please try again!'):
while True:
reply = input(prompt)
if reply in {'y', 'ye', 'yes'}:
return True
if reply in {'n', 'no', 'nop', 'nope'}:
return False
retries = retries - 1
if retries < 0:
raise ValueError('invalid user response')
print(reminder)
This function can be called in several ways:
giving only the mandatory argument:
ask_ok('Do you really want to quit?')
giving one of the optional arguments:
ask_ok('OK to overwrite the file?', 2)
or even giving all arguments:
ask_ok('OK to overwrite the file?', 2, 'Come on, only yes or no!')
This example also introduces the in
keyword. This tests whether or
not a sequence contains a certain value.
The default values are evaluated at the point of function definition in the defining scope, so that
i = 5
def f(arg=i):
print(arg)
i = 6
f()
will print 5
.
Important warning: The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes. For example, the following function accumulates the arguments passed to it on subsequent calls:
def f(a, L=[]):
L.append(a)
return L
print(f(1))
print(f(2))
print(f(3))
This will print
[1]
[1, 2]
[1, 2, 3]
If you don’t want the default to be shared between subsequent calls, you can write the function like this instead:
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L
4.9.2. Keyword Arguments¶
Functions can also be called using keyword arguments
of the form kwarg=value
. For instance, the following function:
def parrot(voltage, state='a stiff', action='voom', type='Norwegian Blue'):
print("-- This parrot wouldn't", action, end=' ')
print("if you put", voltage, "volts through it.")
print("-- Lovely plumage, the", type)
print("-- It's", state, "!")
accepts one required argument (voltage
) and three optional arguments
(state
, action
, and type
). This function can be called in any
of the following ways:
parrot(1000) # 1 positional argument
parrot(voltage=1000) # 1 keyword argument
parrot(voltage=1000000, action='VOOOOOM') # 2 keyword arguments
parrot(action='VOOOOOM', voltage=1000000) # 2 keyword arguments
parrot('a million', 'bereft of life', 'jump') # 3 positional arguments
parrot('a thousand', state='pushing up the daisies') # 1 positional, 1 keyword
but all the following calls would be invalid:
parrot() # required argument missing
parrot(voltage=5.0, 'dead') # non-keyword argument after a keyword argument
parrot(110, voltage=220) # duplicate value for the same argument
parrot(actor='John Cleese') # unknown keyword argument
In a function call, keyword arguments must follow positional arguments.
All the keyword arguments passed must match one of the arguments
accepted by the function (e.g. actor
is not a valid argument for the
parrot
function), and their order is not important. This also includes
non-optional arguments (e.g. parrot(voltage=1000)
is valid too).
No argument may receive a value more than once.
Here’s an example that fails due to this restriction:
>>> def function(a):
... pass
...
>>> function(0, a=0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: function() got multiple values for argument 'a'
When a final formal parameter of the form **name
is present, it receives a
dictionary (see Mapping Types — dict) containing all keyword arguments except for
those corresponding to a formal parameter. This may be combined with a formal
parameter of the form *name
(described in the next subsection) which
receives a tuple containing the positional
arguments beyond the formal parameter list. (*name
must occur
before **name
.) For example, if we define a function like this:
def cheeseshop(kind, *arguments, **keywords):
print("-- Do you have any", kind, "?")
print("-- I'm sorry, we're all out of", kind)
for arg in arguments:
print(arg)
print("-" * 40)
for kw in keywords:
print(kw, ":", keywords[kw])
It could be called like this:
cheeseshop("Limburger", "It's very runny, sir.",
"It's really very, VERY runny, sir.",
shopkeeper="Michael Palin",
client="John Cleese",
sketch="Cheese Shop Sketch")
and of course it would print:
-- Do you have any Limburger ?
-- I'm sorry, we're all out of Limburger
It's very runny, sir.
It's really very, VERY runny, sir.
----------------------------------------
shopkeeper : Michael Palin
client : John Cleese
sketch : Cheese Shop Sketch
Note that the order in which the keyword arguments are printed is guaranteed to match the order in which they were provided in the function call.
4.9.3. Special parameters¶
By default, arguments may be passed to a Python function either by position or explicitly by keyword. For readability and performance, it makes sense to restrict the way arguments can be passed so that a developer need only look at the function definition to determine if items are passed by position, by position or keyword, or by keyword.
A function definition may look like:
def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):
----------- ---------- ----------
| | |
| Positional or keyword |
| - Keyword only
-- Positional only
where /
and *
are optional. If used, these symbols indicate the kind of
parameter by how the arguments may be passed to the function:
positional-only, positional-or-keyword, and keyword-only. Keyword parameters
are also referred to as named parameters.
4.9.3.1. Positional-or-Keyword Arguments¶
If /
and *
are not present in the function definition, arguments may
be passed to a function by position or by keyword.
4.9.3.2. Positional-Only Parameters¶
Looking at this in a bit more detail, it is possible to mark certain parameters
as positional-only. If positional-only, the parameters’ order matters, and
the parameters cannot be passed by keyword. Positional-only parameters are
placed before a /
(forward-slash). The /
is used to logically
separate the positional-only parameters from the rest of the parameters.
If there is no /
in the function definition, there are no positional-only
parameters.
Parameters following the /
may be positional-or-keyword or keyword-only.
4.9.3.3. Keyword-Only Arguments¶
To mark parameters as keyword-only, indicating the parameters must be passed
by keyword argument, place an *
in the arguments list just before the first
keyword-only parameter.
4.9.3.4. Function Examples¶
Consider the following example function definitions paying close attention to the
markers /
and *
:
>>> def standard_arg(arg):
... print(arg)
...
>>> def pos_only_arg(arg, /):
... print(arg)
...
>>> def kwd_only_arg(*, arg):
... print(arg)
...
>>> def combined_example(pos_only, /, standard, *, kwd_only):
... print(pos_only, standard, kwd_only)
The first function definition, standard_arg
, the most familiar form,
places no restrictions on the calling convention and arguments may be
passed by position or keyword:
>>> standard_arg(2)
2
>>> standard_arg(arg=2)
2
The second function pos_only_arg
is restricted to only use positional
parameters as there is a /
in the function definition:
>>> pos_only_arg(1)
1
>>> pos_only_arg(arg=1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: pos_only_arg() got some positional-only arguments passed as keyword arguments: 'arg'
The third function kwd_only_arg
only allows keyword arguments as indicated
by a *
in the function definition:
>>> kwd_only_arg(3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: kwd_only_arg() takes 0 positional arguments but 1 was given
>>> kwd_only_arg(arg=3)
3
And the last uses all three calling conventions in the same function definition:
>>> combined_example(1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: combined_example() takes 2 positional arguments but 3 were given
>>> combined_example(1, 2, kwd_only=3)
1 2 3
>>> combined_example(1, standard=2, kwd_only=3)
1 2 3
>>> combined_example(pos_only=1, standard=2, kwd_only=3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: combined_example() got some positional-only arguments passed as keyword arguments: 'pos_only'
Finally, consider this function definition which has a potential collision between the positional argument name
and **kwds
which has name
as a key:
def foo(name, **kwds):
return 'name' in kwds
There is no possible call that will make it return True
as the keyword 'name'
will always bind to the first parameter. For example:
>>> foo(1, **{'name': 2})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() got multiple values for argument 'name'
>>>
But using /
(positional only arguments), it is possible since it allows name
as a positional argument and 'name'
as a key in the keyword arguments:
>>> def foo(name, /, **kwds):
... return 'name' in kwds
...
>>> foo(1, **{'name': 2})
True
In other words, the names of positional-only parameters can be used in
**kwds
without ambiguity.
4.9.3.5. Recap¶
The use case will determine which parameters to use in the function definition:
def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):
As guidance:
Use positional-only if you want the name of the parameters to not be available to the user. This is useful when parameter names have no real meaning, if you want to enforce the order of the arguments when the function is called or if you need to take some positional parameters and arbitrary keywords.
Use keyword-only when names have meaning and the function definition is more understandable by being explicit with names or you want to prevent users relying on the position of the argument being passed.
For an API, use positional-only to prevent breaking API changes if the parameter’s name is modified in the future.
4.9.4. Arbitrary Argument Lists¶
Finally, the least frequently used option is to specify that a function can be called with an arbitrary number of arguments. These arguments will be wrapped up in a tuple (see Tuples and Sequences). Before the variable number of arguments, zero or more normal arguments may occur.
def write_multiple_items(file, separator, *args):
file.write(separator.join(args))
Normally, these variadic arguments will be last in the list of formal
parameters, because they scoop up all remaining input arguments that are
passed to the function. Any formal parameters which occur after the *args
parameter are ‘keyword-only’ arguments, meaning that they can only be used as
keywords rather than positional arguments.
>>> def concat(*args, sep="/"):
... return sep.join(args)
...
>>> concat("earth", "mars", "venus")
'earth/mars/venus'
>>> concat("earth", "mars", "venus", sep=".")
'earth.mars.venus'
4.9.5. Unpacking Argument Lists¶
The reverse situation occurs when the arguments are already in a list or tuple
but need to be unpacked for a function call requiring separate positional
arguments. For instance, the built-in range()
function expects separate
start and stop arguments. If they are not available separately, write the
function call with the *
-operator to unpack the arguments out of a list
or tuple:
>>> list(range(3, 6)) # normal call with separate arguments
[3, 4, 5]
>>> args = [3, 6]
>>> list(range(*args)) # call with arguments unpacked from a list
[3, 4, 5]
In the same fashion, dictionaries can deliver keyword arguments with the
**
-operator:
>>> def parrot(voltage, state='a stiff', action='voom'):
... print("-- This parrot wouldn't", action, end=' ')
... print("if you put", voltage, "volts through it.", end=' ')
... print("E's", state, "!")
...
>>> d = {"voltage": "four million", "state": "bleedin' demised", "action": "VOOM"}
>>> parrot(**d)
-- This parrot wouldn't VOOM if you put four million volts through it. E's bleedin' demised !
4.9.6. Lambda Expressions¶
Small anonymous functions can be created with the lambda
keyword.
This function returns the sum of its two arguments: lambda a, b: a+b
.
Lambda functions can be used wherever function objects are required. They are
syntactically restricted to a single expression. Semantically, they are just
syntactic sugar for a normal function definition. Like nested function
definitions, lambda functions can reference variables from the containing
scope:
>>> def make_incrementor(n):
... return lambda x: x + n
...
>>> f = make_incrementor(42)
>>> f(0)
42
>>> f(1)
43
The above example uses a lambda expression to return a function. Another use
is to pass a small function as an argument. For instance, list.sort()
takes a sorting key function key which can be a lambda function:
>>> pairs = [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')]
>>> pairs.sort(key=lambda pair: pair[1])
>>> pairs
[(4, 'four'), (1, 'one'), (3, 'three'), (2, 'two')]
4.9.7. Documentation Strings¶
Here are some conventions about the content and formatting of documentation strings.
The first line should always be a short, concise summary of the object’s purpose. For brevity, it should not explicitly state the object’s name or type, since these are available by other means (except if the name happens to be a verb describing a function’s operation). This line should begin with a capital letter and end with a period.
If there are more lines in the documentation string, the second line should be blank, visually separating the summary from the rest of the description. The following lines should be one or more paragraphs describing the object’s calling conventions, its side effects, etc.
The Python parser strips indentation from multi-line string literals when they serve as module, class, or function docstrings.
Here is an example of a multi-line docstring:
>>> def my_function():
... """Do nothing, but document it.
...
... No, really, it doesn't do anything:
...
... >>> my_function()
... >>>
... """
... pass
...
>>> print(my_function.__doc__)
Do nothing, but document it.
No, really, it doesn't do anything:
>>> my_function()
>>>
4.9.8. Function Annotations¶
Function annotations are completely optional metadata information about the types used by user-defined functions (see PEP 3107 and PEP 484 for more information).
Annotations are stored in the __annotations__
attribute of the function as a dictionary and have no effect on any other part of the
function. Parameter annotations are defined by a colon after the parameter name, followed
by an expression evaluating to the value of the annotation. Return annotations are
defined by a literal ->
, followed by an expression, between the parameter
list and the colon denoting the end of the def
statement. The
following example has a required argument, an optional argument, and the return
value annotated:
>>> def f(ham: str, eggs: str = 'eggs') -> str:
... print("Annotations:", f.__annotations__)
... print("Arguments:", ham, eggs)
... return ham + ' and ' + eggs
...
>>> f('spam')
Annotations: {'ham': <class 'str'>, 'return': <class 'str'>, 'eggs': <class 'str'>}
Arguments: spam eggs
'spam and eggs'
4.10. Intermezzo: Coding Style¶
Now that you are about to write longer, more complex pieces of Python, it is a good time to talk about coding style. Most languages can be written (or more concisely, formatted) in different styles; some are more readable than others. Making it easy for others to read your code is always a good idea, and adopting a nice coding style helps tremendously for that.
For Python, PEP 8 has emerged as the style guide that most projects adhere to; it promotes a very readable and eye-pleasing coding style. Every Python developer should read it at some point; here are the most important points extracted for you:
Use 4-space indentation, and no tabs.
4 spaces are a good compromise between small indentation (allows greater nesting depth) and large indentation (easier to read). Tabs introduce confusion, and are best left out.
Wrap lines so that they don’t exceed 79 characters.
This helps users with small displays and makes it possible to have several code files side-by-side on larger displays.
Use blank lines to separate functions and classes, and larger blocks of code inside functions.
When possible, put comments on a line of their own.
Use docstrings.
Use spaces around operators and after commas, but not directly inside bracketing constructs:
a = f(1, 2) + g(3, 4)
.Name your classes and functions consistently; the convention is to use
UpperCamelCase
for classes andlowercase_with_underscores
for functions and methods. Always useself
as the name for the first method argument (see A First Look at Classes for more on classes and methods).Don’t use fancy encodings if your code is meant to be used in international environments. Python’s default, UTF-8, or even plain ASCII work best in any case.
Likewise, don’t use non-ASCII characters in identifiers if there is only the slightest chance people speaking a different language will read or maintain the code.
Footnotes


Module Objects

Module Objects¶
-
PyTypeObject PyModule_Type¶
- Part of the Stable ABI.
This instance of
PyTypeObject
represents the Python module type. This is exposed to Python programs astypes.ModuleType
.
-
int PyModule_Check(PyObject *p)¶
Return true if p is a module object, or a subtype of a module object. This function always succeeds.
-
int PyModule_CheckExact(PyObject *p)¶
Return true if p is a module object, but not a subtype of
PyModule_Type
. This function always succeeds.
-
PyObject *PyModule_NewObject(PyObject *name)¶
- Return value: New reference. Part of the Stable ABI since version 3.7.
Return a new module object with
module.__name__
set to name. The module’s__name__
,__doc__
,__package__
and__loader__
attributes are filled in (all but__name__
are set toNone
). The caller is responsible for setting a__file__
attribute.Return
NULL
with an exception set on error.Added in version 3.3.
Changed in version 3.4:
__package__
and__loader__
are now set toNone
.
-
PyObject *PyModule_New(const char *name)¶
- Return value: New reference. Part of the Stable ABI.
Similar to
PyModule_NewObject()
, but the name is a UTF-8 encoded string instead of a Unicode object.
-
PyObject *PyModule_GetDict(PyObject *module)¶
- Return value: Borrowed reference. Part of the Stable ABI.
Return the dictionary object that implements module’s namespace; this object is the same as the
__dict__
attribute of the module object. If module is not a module object (or a subtype of a module object),SystemError
is raised andNULL
is returned.It is recommended extensions use other
PyModule_*
andPyObject_*
functions rather than directly manipulate a module’s__dict__
.The returned reference is borrowed from the module; it is valid until the module is destroyed.
-
PyObject *PyModule_GetNameObject(PyObject *module)¶
- Return value: New reference. Part of the Stable ABI since version 3.7.
Return module’s
__name__
value. If the module does not provide one, or if it is not a string,SystemError
is raised andNULL
is returned.Added in version 3.3.
-
const char *PyModule_GetName(PyObject *module)¶
- Part of the Stable ABI.
Similar to
PyModule_GetNameObject()
but return the name encoded to'utf-8'
.The returned buffer is only valid until the module is renamed or destroyed. Note that Python code may rename a module by setting its
__name__
attribute.
-
void *PyModule_GetState(PyObject *module)¶
- Part of the Stable ABI.
Return the “state” of the module, that is, a pointer to the block of memory allocated at module creation time, or
NULL
. SeePyModuleDef.m_size
.
-
PyModuleDef *PyModule_GetDef(PyObject *module)¶
- Part of the Stable ABI.
Return a pointer to the
PyModuleDef
struct from which the module was created, orNULL
if the module wasn’t created from a definition.On error, return
NULL
with an exception set. UsePyErr_Occurred()
to tell this case apart from a missingPyModuleDef
.
-
PyObject *PyModule_GetFilenameObject(PyObject *module)¶
- Return value: New reference. Part of the Stable ABI.
Return the name of the file from which module was loaded using module’s
__file__
attribute. If this is not defined, or if it is not a string, raiseSystemError
and returnNULL
; otherwise return a reference to a Unicode object.Added in version 3.2.
-
const char *PyModule_GetFilename(PyObject *module)¶
- Part of the Stable ABI.
Similar to
PyModule_GetFilenameObject()
but return the filename encoded to ‘utf-8’.The returned buffer is only valid until the module’s
__file__
attribute is reassigned or the module is destroyed.Deprecated since version 3.2:
PyModule_GetFilename()
raisesUnicodeEncodeError
on unencodable filenames, usePyModule_GetFilenameObject()
instead.
Module definitions¶
The functions in the previous section work on any module object, including modules imported from Python code.
Modules defined using the C API typically use a module definition,
PyModuleDef
– a statically allocated, constant “description” of
how a module should be created.
The definition is usually used to define an extension’s “main” module object (see Defining extension modules for details). It is also used to create extension modules dynamically.
Unlike PyModule_New()
, the definition allows management of
module state – a piece of memory that is allocated and cleared together
with the module object.
Unlike the module’s Python attributes, Python code cannot replace or delete
data stored in module state.
-
type PyModuleDef¶
- Part of the Stable ABI (including all members).
The module definition struct, which holds all information needed to create a module object. This structure must be statically allocated (or be otherwise guaranteed to be valid while any modules created from it exist). Usually, there is only one variable of this type for each extension module.
-
PyModuleDef_Base m_base¶
Always initialize this member to
PyModuleDef_HEAD_INIT
.
-
const char *m_name¶
Name for the new module.
-
const char *m_doc¶
Docstring for the module; usually a docstring variable created with
PyDoc_STRVAR
is used.
-
Py_ssize_t m_size¶
Module state may be kept in a per-module memory area that can be retrieved with
PyModule_GetState()
, rather than in static globals. This makes modules safe for use in multiple sub-interpreters.This memory area is allocated based on m_size on module creation, and freed when the module object is deallocated, after the
m_free
function has been called, if present.Setting it to a non-negative value means that the module can be re-initialized and specifies the additional amount of memory it requires for its state.
Setting
m_size
to-1
means that the module does not support sub-interpreters, because it has global state. Negativem_size
is only allowed when using legacy single-phase initialization or when creating modules dynamically.See PEP 3121 for more details.
-
PyMethodDef *m_methods¶
A pointer to a table of module-level functions, described by
PyMethodDef
values. Can beNULL
if no functions are present.
-
PyModuleDef_Slot *m_slots¶
An array of slot definitions for multi-phase initialization, terminated by a
{0, NULL}
entry. When using legacy single-phase initialization, m_slots must beNULL
.
-
traverseproc m_traverse¶
A traversal function to call during GC traversal of the module object, or
NULL
if not needed.This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (
Py_mod_exec
function). More precisely, this function is not called ifm_size
is greater than 0 and the module state (as returned byPyModule_GetState()
) isNULL
.Changed in version 3.9: No longer called before the module state is allocated.
-
inquiry m_clear¶
A clear function to call during GC clearing of the module object, or
NULL
if not needed.This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (
Py_mod_exec
function). More precisely, this function is not called ifm_size
is greater than 0 and the module state (as returned byPyModule_GetState()
) isNULL
.Like
PyTypeObject.tp_clear
, this function is not always called before a module is deallocated. For example, when reference counting is enough to determine that an object is no longer used, the cyclic garbage collector is not involved andm_free
is called directly.Changed in version 3.9: No longer called before the module state is allocated.
-
freefunc m_free¶
A function to call during deallocation of the module object, or
NULL
if not needed.This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (
Py_mod_exec
function). More precisely, this function is not called ifm_size
is greater than 0 and the module state (as returned byPyModule_GetState()
) isNULL
.Changed in version 3.9: No longer called before the module state is allocated.
-
PyModuleDef_Base m_base¶
Module slots¶
-
type PyModuleDef_Slot¶
- Part of the Stable ABI (including all members) since version 3.5.
-
int slot¶
A slot ID, chosen from the available values explained below.
-
void *value¶
Value of the slot, whose meaning depends on the slot ID.
Added in version 3.5.
-
int slot¶
The available slot types are:
-
Py_mod_create¶
- Part of the Stable ABI since version 3.5.
Specifies a function that is called to create the module object itself. The value pointer of this slot must point to a function of the signature:
-
PyObject *create_module(PyObject *spec, PyModuleDef *def)¶
The function receives a
ModuleSpec
instance, as defined in PEP 451, and the module definition. It should return a new module object, or set an error and returnNULL
.This function should be kept minimal. In particular, it should not call arbitrary Python code, as trying to import the same module again may result in an infinite loop.
Multiple
Py_mod_create
slots may not be specified in one module definition.If
Py_mod_create
is not specified, the import machinery will create a normal module object usingPyModule_New()
. The name is taken from spec, not the definition, to allow extension modules to dynamically adjust to their place in the module hierarchy and be imported under different names through symlinks, all while sharing a single module definition.There is no requirement for the returned object to be an instance of
PyModule_Type
. Any type can be used, as long as it supports setting and getting import-related attributes. However, onlyPyModule_Type
instances may be returned if thePyModuleDef
has non-NULL
m_traverse
,m_clear
,m_free
; non-zerom_size
; or slots other thanPy_mod_create
.Added in version 3.5.
-
PyObject *create_module(PyObject *spec, PyModuleDef *def)¶
-
Py_mod_exec¶
- Part of the Stable ABI since version 3.5.
Specifies a function that is called to execute the module. This is equivalent to executing the code of a Python module: typically, this function adds classes and constants to the module. The signature of the function is:
If multiple
Py_mod_exec
slots are specified, they are processed in the order they appear in the m_slots array.Added in version 3.5.
-
Py_mod_multiple_interpreters¶
- Part of the Stable ABI since version 3.12.
Specifies one of the following values:
-
Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED¶
The module does not support being imported in subinterpreters.
-
Py_MOD_MULTIPLE_INTERPRETERS_SUPPORTED¶
The module supports being imported in subinterpreters, but only when they share the main interpreter’s GIL. (See Isolating Extension Modules.)
-
Py_MOD_PER_INTERPRETER_GIL_SUPPORTED¶
The module supports being imported in subinterpreters, even when they have their own GIL. (See Isolating Extension Modules.)
This slot determines whether or not importing this module in a subinterpreter will fail.
Multiple
Py_mod_multiple_interpreters
slots may not be specified in one module definition.If
Py_mod_multiple_interpreters
is not specified, the import machinery defaults toPy_MOD_MULTIPLE_INTERPRETERS_SUPPORTED
.Added in version 3.12.
-
Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED¶
-
Py_mod_gil¶
- Part of the Stable ABI since version 3.13.
Specifies one of the following values:
-
Py_MOD_GIL_USED¶
The module depends on the presence of the global interpreter lock (GIL), and may access global state without synchronization.
-
Py_MOD_GIL_NOT_USED¶
The module is safe to run without an active GIL.
This slot is ignored by Python builds not configured with
--disable-gil
. Otherwise, it determines whether or not importing this module will cause the GIL to be automatically enabled. See Free-threaded CPython for more detail.Multiple
Py_mod_gil
slots may not be specified in one module definition.If
Py_mod_gil
is not specified, the import machinery defaults toPy_MOD_GIL_USED
.Added in version 3.13.
-
Py_MOD_GIL_USED¶
Creating extension modules dynamically¶
The following functions may be used to create a module outside of an extension’s initialization function. They are also used in single-phase initialization.
-
PyObject *PyModule_Create(PyModuleDef *def)¶
- Return value: New reference.
Create a new module object, given the definition in def. This is a macro that calls
PyModule_Create2()
with module_api_version set toPYTHON_API_VERSION
, or toPYTHON_ABI_VERSION
if using the limited API.
-
PyObject *PyModule_Create2(PyModuleDef *def, int module_api_version)¶
- Return value: New reference. Part of the Stable ABI.
Create a new module object, given the definition in def, assuming the API version module_api_version. If that version does not match the version of the running interpreter, a
RuntimeWarning
is emitted.Return
NULL
with an exception set on error.This function does not support slots. The
m_slots
member of def must beNULL
.Note
Most uses of this function should be using
PyModule_Create()
instead; only use this if you are sure you need it.
-
PyObject *PyModule_FromDefAndSpec(PyModuleDef *def, PyObject *spec)¶
- Return value: New reference.
This macro calls
PyModule_FromDefAndSpec2()
with module_api_version set toPYTHON_API_VERSION
, or toPYTHON_ABI_VERSION
if using the limited API.Added in version 3.5.
-
PyObject *PyModule_FromDefAndSpec2(PyModuleDef *def, PyObject *spec, int module_api_version)¶
- Return value: New reference. Part of the Stable ABI since version 3.7.
Create a new module object, given the definition in def and the ModuleSpec spec, assuming the API version module_api_version. If that version does not match the version of the running interpreter, a
RuntimeWarning
is emitted.Return
NULL
with an exception set on error.Note that this does not process execution slots (
Py_mod_exec
). BothPyModule_FromDefAndSpec
andPyModule_ExecDef
must be called to fully initialize a module.Note
Most uses of this function should be using
PyModule_FromDefAndSpec()
instead; only use this if you are sure you need it.Added in version 3.5.
-
int PyModule_ExecDef(PyObject *module, PyModuleDef *def)¶
- Part of the Stable ABI since version 3.7.
Process any execution slots (
Py_mod_exec
) given in def.Added in version 3.5.
-
PYTHON_API_VERSION¶
The C API version. Defined for backwards compatibility.
Currently, this constant is not updated in new Python versions, and is not useful for versioning. This may change in the future.
-
PYTHON_ABI_VERSION¶
Defined as
3
for backwards compatibility.Currently, this constant is not updated in new Python versions, and is not useful for versioning. This may change in the future.
Support functions¶
The following functions are provided to help initialize a module
state.
They are intended for a module’s execution slots (Py_mod_exec
),
the initialization function for legacy single-phase initialization,
or code that creates modules dynamically.
-
int PyModule_AddObjectRef(PyObject *module, const char *name, PyObject *value)¶
- Part of the Stable ABI since version 3.10.
Add an object to module as name. This is a convenience function which can be used from the module’s initialization function.
On success, return
0
. On error, raise an exception and return-1
.Example usage:
static int add_spam(PyObject *module, int value) { PyObject *obj = PyLong_FromLong(value); if (obj == NULL) { return -1; } int res = PyModule_AddObjectRef(module, "spam", obj); Py_DECREF(obj); return res; }
To be convenient, the function accepts
NULL
value with an exception set. In this case, return-1
and just leave the raised exception unchanged.The example can also be written without checking explicitly if obj is
NULL
:static int add_spam(PyObject *module, int value) { PyObject *obj = PyLong_FromLong(value); int res = PyModule_AddObjectRef(module, "spam", obj); Py_XDECREF(obj); return res; }
Note that
Py_XDECREF()
should be used instead ofPy_DECREF()
in this case, since obj can beNULL
.The number of different name strings passed to this function should be kept small, usually by only using statically allocated strings as name. For names that aren’t known at compile time, prefer calling
PyUnicode_FromString()
andPyObject_SetAttr()
directly. For more details, seePyUnicode_InternFromString()
, which may be used internally to create a key object.Added in version 3.10.
-
int PyModule_Add(PyObject *module, const char *name, PyObject *value)¶
- Part of the Stable ABI since version 3.13.
Similar to
PyModule_AddObjectRef()
, but “steals” a reference to value. It can be called with a result of function that returns a new reference without bothering to check its result or even saving it to a variable.Example usage:
if (PyModule_Add(module, "spam", PyBytes_FromString(value)) < 0) { goto error; }
Added in version 3.13.
-
int PyModule_AddObject(PyObject *module, const char *name, PyObject *value)¶
- Part of the Stable ABI.
Similar to
PyModule_AddObjectRef()
, but steals a reference to value on success (if it returns0
).The new
PyModule_Add()
orPyModule_AddObjectRef()
functions are recommended, since it is easy to introduce reference leaks by misusing thePyModule_AddObject()
function.Note
Unlike other functions that steal references,
PyModule_AddObject()
only releases the reference to value on success.This means that its return value must be checked, and calling code must
Py_XDECREF()
value manually on error.Example usage:
PyObject *obj = PyBytes_FromString(value); if (PyModule_AddObject(module, "spam", obj) < 0) { // If 'obj' is not NULL and PyModule_AddObject() failed, // 'obj' strong reference must be deleted with Py_XDECREF(). // If 'obj' is NULL, Py_XDECREF() does nothing. Py_XDECREF(obj); goto error; } // PyModule_AddObject() stole a reference to obj: // Py_XDECREF(obj) is not needed here.
Deprecated since version 3.13:
PyModule_AddObject()
is soft deprecated.
-
int PyModule_AddIntConstant(PyObject *module, const char *name, long value)¶
- Part of the Stable ABI.
Add an integer constant to module as name. This convenience function can be used from the module’s initialization function. Return
-1
with an exception set on error,0
on success.This is a convenience function that calls
PyLong_FromLong()
andPyModule_AddObjectRef()
; see their documentation for details.
-
int PyModule_AddStringConstant(PyObject *module, const char *name, const char *value)¶
- Part of the Stable ABI.
Add a string constant to module as name. This convenience function can be used from the module’s initialization function. The string value must be
NULL
-terminated. Return-1
with an exception set on error,0
on success.This is a convenience function that calls
PyUnicode_InternFromString()
andPyModule_AddObjectRef()
; see their documentation for details.
-
PyModule_AddIntMacro(module, macro)¶
Add an int constant to module. The name and the value are taken from macro. For example
PyModule_AddIntMacro(module, AF_INET)
adds the int constant AF_INET with the value of AF_INET to module. Return-1
with an exception set on error,0
on success.
-
PyModule_AddStringMacro(module, macro)¶
Add a string constant to module.
-
int PyModule_AddType(PyObject *module, PyTypeObject *type)¶
- Part of the Stable ABI since version 3.10.
Add a type object to module. The type object is finalized by calling internally
PyType_Ready()
. The name of the type object is taken from the last component oftp_name
after dot. Return-1
with an exception set on error,0
on success.Added in version 3.9.
-
int PyModule_AddFunctions(PyObject *module, PyMethodDef *functions)¶
- Part of the Stable ABI since version 3.7.
Add the functions from the
NULL
terminated functions array to module. Refer to thePyMethodDef
documentation for details on individual entries (due to the lack of a shared module namespace, module level “functions” implemented in C typically receive the module as their first parameter, making them similar to instance methods on Python classes).This function is called automatically when creating a module from
PyModuleDef
(such as when using Multi-phase initialization,PyModule_Create
, orPyModule_FromDefAndSpec
). Some module authors may prefer defining functions in multiplePyMethodDef
arrays; in that case they should call this function directly.The functions array must be statically allocated (or otherwise guaranteed to outlive the module object).
Added in version 3.5.
-
int PyModule_SetDocString(PyObject *module, const char *docstring)¶
- Part of the Stable ABI since version 3.7.
Set the docstring for module to docstring. This function is called automatically when creating a module from
PyModuleDef
(such as when using Multi-phase initialization,PyModule_Create
, orPyModule_FromDefAndSpec
).Return
0
on success. Return-1
with an exception set on error.Added in version 3.5.
-
int PyUnstable_Module_SetGIL(PyObject *module, void *gil)¶
- This is Unstable API. It may change without warning in minor releases.
Indicate that module does or does not support running without the global interpreter lock (GIL), using one of the values from
Py_mod_gil
. It must be called during module’s initialization function when using Legacy single-phase initialization. If this function is not called during module initialization, the import machinery assumes the module does not support running without the GIL. This function is only available in Python builds configured with--disable-gil
. Return-1
with an exception set on error,0
on success.Added in version 3.13.
Module lookup (single-phase initialization)¶
The legacy single-phase initialization initialization scheme creates singleton modules that can be looked up in the context of the current interpreter. This allows the module object to be retrieved later with only a reference to the module definition.
These functions will not work on modules created using multi-phase initialization, since multiple such modules can be created from a single definition.
-
PyObject *PyState_FindModule(PyModuleDef *def)¶
- Return value: Borrowed reference. Part of the Stable ABI.
Returns the module object that was created from def for the current interpreter. This method requires that the module object has been attached to the interpreter state with
PyState_AddModule()
beforehand. In case the corresponding module object is not found or has not been attached to the interpreter state yet, it returnsNULL
.
-
int PyState_AddModule(PyObject *module, PyModuleDef *def)¶
- Part of the Stable ABI since version 3.3.
Attaches the module object passed to the function to the interpreter state. This allows the module object to be accessible via
PyState_FindModule()
.Only effective on modules created using single-phase initialization.
Python calls
PyState_AddModule
automatically after importing a module that uses single-phase initialization, so it is unnecessary (but harmless) to call it from module initialization code. An explicit call is needed only if the module’s own init code subsequently callsPyState_FindModule
. The function is mainly intended for implementing alternative import mechanisms (either by calling it directly, or by referring to its implementation for details of the required state updates).If a module was attached previously using the same def, it is replaced by the new module.
The caller must have an attached thread state.
Return
-1
with an exception set on error,0
on success.Added in version 3.3.
-
int PyState_RemoveModule(PyModuleDef *def)¶
- Part of the Stable ABI since version 3.3.
Removes the module object created from def from the interpreter state. Return
-1
with an exception set on error,0
on success.The caller must have an attached thread state.
Added in version 3.3.