Using Python on Android

6. Using Python on Android¶
Python on Android is unlike Python on desktop platforms. On a desktop platform, Python is generally installed as a system resource that can be used by any user of that computer. Users then interact with Python by running a python executable and entering commands at an interactive prompt, or by running a Python script.
On Android, there is no concept of installing as a system resource. The only unit of software distribution is an “app”. There is also no console where you could run a python executable, or interact with a Python REPL.
As a result, the only way you can use Python on Android is in embedded mode – that
is, by writing a native Android application, embedding a Python interpreter
using libpython
, and invoking Python code using the Python embedding
API. The full Python interpreter, the standard library, and all
your Python code is then packaged into your app for its own private use.
The Python standard library has some notable omissions and restrictions on Android. See the API availability guide for details.
6.1. Adding Python to an Android app¶
Most app developers should use one of the following tools, which will provide a much easier experience:
If you’re sure you want to do all of this manually, read on. You can use the testbed app as a guide; each step below contains a link to the relevant file.
First, acquire a build of Python for Android:
The easiest way is to download an Android release from python.org. The
prefix
directory mentioned below is at the top level of the package.Or if you want to build it yourself, follow the instructions in Android/README.md. The
prefix
directory will be created undercross-build/HOST
.
Add code to your build.gradle file to copy the following items into your project. All except your own Python code can be copied from
prefix/lib
:In your JNI libraries:
libpython*.*.so
lib*_python.so
(external libraries such as OpenSSL)
In your assets:
python*.*
(the Python standard library)python*.*/site-packages
(your own Python code)
Add code to your app to extract the assets to the filesystem.
Add code to your app to start Python in embedded mode. This will need to be C code called via JNI.
6.2. Building a Python package for Android¶
Python packages can be built for Android as wheels and released on PyPI. The recommended tool for doing this is cibuildwheel, which automates all the details of setting up a cross-compilation environment, building the wheel, and testing it on an emulator.


Queues

Queues¶
Source code: Lib/asyncio/queues.py
asyncio queues are designed to be similar to classes of the
queue
module. Although asyncio queues are not thread-safe,
they are designed to be used specifically in async/await code.
Note that methods of asyncio queues don’t have a timeout parameter;
use asyncio.wait_for()
function to do queue operations with a
timeout.
See also the Examples section below.
Queue¶
- class asyncio.Queue(maxsize=0)¶
A first in, first out (FIFO) queue.
If maxsize is less than or equal to zero, the queue size is infinite. If it is an integer greater than
0
, thenawait put()
blocks when the queue reaches maxsize until an item is removed byget()
.Unlike the standard library threading
queue
, the size of the queue is always known and can be returned by calling theqsize()
method.Changed in version 3.10: Removed the loop parameter.
This class is not thread safe.
- maxsize¶
Number of items allowed in the queue.
- empty()¶
Return
True
if the queue is empty,False
otherwise.
- full()¶
Return
True
if there aremaxsize
items in the queue.If the queue was initialized with
maxsize=0
(the default), thenfull()
never returnsTrue
.
- async get()¶
Remove and return an item from the queue. If queue is empty, wait until an item is available.
Raises
QueueShutDown
if the queue has been shut down and is empty, or if the queue has been shut down immediately.
- get_nowait()¶
Return an item if one is immediately available, else raise
QueueEmpty
.
- async join()¶
Block until all items in the queue have been received and processed.
The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer coroutine calls
task_done()
to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero,join()
unblocks.
- async put(item)¶
Put an item into the queue. If the queue is full, wait until a free slot is available before adding the item.
Raises
QueueShutDown
if the queue has been shut down.
- put_nowait(item)¶
Put an item into the queue without blocking.
If no free slot is immediately available, raise
QueueFull
.
- qsize()¶
Return the number of items in the queue.
- shutdown(immediate=False)¶
Put a
Queue
instance into a shutdown mode.The queue can no longer grow. Future calls to
put()
raiseQueueShutDown
. Currently blocked callers ofput()
will be unblocked and will raiseQueueShutDown
in the formerly awaiting task.If immediate is false (the default), the queue can be wound down normally with
get()
calls to extract tasks that have already been loaded.And if
task_done()
is called for each remaining task, a pendingjoin()
will be unblocked normally.Once the queue is empty, future calls to
get()
will raiseQueueShutDown
.If immediate is true, the queue is terminated immediately. The queue is drained to be completely empty and the count of unfinished tasks is reduced by the number of tasks drained. If unfinished tasks is zero, callers of
join()
are unblocked. Also, blocked callers ofget()
are unblocked and will raiseQueueShutDown
because the queue is empty.Use caution when using
join()
with immediate set to true. This unblocks the join even when no work has been done on the tasks, violating the usual invariant for joining a queue.Added in version 3.13.
- task_done()¶
Indicate that a formerly enqueued work item is complete.
Used by queue consumers. For each
get()
used to fetch a work item, a subsequent call totask_done()
tells the queue that the processing on the work item is complete.If a
join()
is currently blocking, it will resume when all items have been processed (meaning that atask_done()
call was received for every item that had beenput()
into the queue).Raises
ValueError
if called more times than there were items placed in the queue.
Priority Queue¶
LIFO Queue¶
Exceptions¶
- exception asyncio.QueueEmpty¶
This exception is raised when the
get_nowait()
method is called on an empty queue.
- exception asyncio.QueueFull¶
Exception raised when the
put_nowait()
method is called on a queue that has reached its maxsize.
Examples¶
Queues can be used to distribute workload between several concurrent tasks:
import asyncio
import random
import time
async def worker(name, queue):
while True:
# Get a "work item" out of the queue.
sleep_for = await queue.get()
# Sleep for the "sleep_for" seconds.
await asyncio.sleep(sleep_for)
# Notify the queue that the "work item" has been processed.
queue.task_done()
print(f'{name} has slept for {sleep_for:.2f} seconds')
async def main():
# Create a queue that we will use to store our "workload".
queue = asyncio.Queue()
# Generate random timings and put them into the queue.
total_sleep_time = 0
for _ in range(20):
sleep_for = random.uniform(0.05, 1.0)
total_sleep_time += sleep_for
queue.put_nowait(sleep_for)
# Create three worker tasks to process the queue concurrently.
tasks = []
for i in range(3):
task = asyncio.create_task(worker(f'worker-{i}', queue))
tasks.append(task)
# Wait until the queue is fully processed.
started_at = time.monotonic()
await queue.join()
total_slept_for = time.monotonic() - started_at
# Cancel our worker tasks.
for task in tasks:
task.cancel()
# Wait until all worker tasks are cancelled.
await asyncio.gather(*tasks, return_exceptions=True)
print('====')
print(f'3 workers slept in parallel for {total_slept_for:.2f} seconds')
print(f'total expected sleep time: {total_sleep_time:.2f} seconds')
asyncio.run(main())


Using Python on Unix platforms

2. Using Python on Unix platforms¶
2.1. Getting and installing the latest version of Python¶
2.1.1. On Linux¶
Python comes preinstalled on most Linux distributions, and is available as a package on all others. However there are certain features you might want to use that are not available on your distro’s package. You can compile the latest version of Python from source.
In the event that the latest version of Python doesn’t come preinstalled and isn’t in the repositories as well, you can make packages for your own distro. Have a look at the following links:
See also
- https://www.debian.org/doc/manuals/maint-guide/first.en.html
for Debian users
- https://en.opensuse.org/Portal:Packaging
for OpenSuse users
- https://docs.fedoraproject.org/en-US/package-maintainers/Packaging_Tutorial_GNU_Hello/
for Fedora users
- https://slackbook.org/html/package-management-making-packages.html
for Slackware users
2.1.1.1. Installing IDLE¶
In some cases, IDLE might not be included in your Python installation.
For Debian and Ubuntu users:
sudo apt update sudo apt install idle
For Fedora, RHEL, and CentOS users:
sudo dnf install python3-idle
For SUSE and OpenSUSE users:
sudo zypper install python3-idle
For Alpine Linux users:
sudo apk add python3-idle
2.1.2. On FreeBSD and OpenBSD¶
FreeBSD users, to add the package use:
pkg install python3
OpenBSD users, to add the package use:
pkg_add -r python pkg_add ftp://ftp.openbsd.org/pub/OpenBSD/4.2/packages/<insert your architecture here>/python-<version>.tgz
For example i386 users get the 2.5.1 version of Python using:
pkg_add ftp://ftp.openbsd.org/pub/OpenBSD/4.2/packages/i386/python-2.5.1p2.tgz
2.2. Building Python¶
See also
If you want to contribute to CPython, refer to the devguide, which includes build instructions and other tips on setting up environment.
If you want to compile CPython yourself, first thing you should do is get the source. You can download either the latest release’s source or grab a fresh clone. You will also need to install the build requirements.
The build process consists of the usual commands:
./configure
make
make install
Configuration options and caveats for specific Unix platforms are extensively documented in the README.rst file in the root of the Python source tree.
Warning
make install
can overwrite or masquerade the python3
binary.
make altinstall
is therefore recommended instead of make install
since it only installs exec_prefix/bin/pythonversion
.
2.4. Miscellaneous¶
To easily use Python scripts on Unix, you need to make them executable, e.g. with
$ chmod +x script
and put an appropriate Shebang line at the top of the script. A good choice is usually
#!/usr/bin/env python3
which searches for the Python interpreter in the whole PATH
. However,
some Unices may not have the env command, so you may need to hardcode
/usr/bin/python3
as the interpreter path.
To use shell commands in your Python scripts, look at the subprocess
module.
2.5. Custom OpenSSL¶
To use your vendor’s OpenSSL configuration and system trust store, locate the directory with
openssl.cnf
file or symlink in/etc
. On most distribution the file is either in/etc/ssl
or/etc/pki/tls
. The directory should also contain acert.pem
file and/or acerts
directory.$ find /etc/ -name openssl.cnf -printf "%h\n" /etc/ssl
Download, build, and install OpenSSL. Make sure you use
install_sw
and notinstall
. Theinstall_sw
target does not overrideopenssl.cnf
.$ curl -O https://www.openssl.org/source/openssl-VERSION.tar.gz $ tar xzf openssl-VERSION $ pushd openssl-VERSION $ ./config \ --prefix=/usr/local/custom-openssl \ --libdir=lib \ --openssldir=/etc/ssl $ make -j1 depend $ make -j8 $ make install_sw $ popd
Build Python with custom OpenSSL (see the configure
--with-openssl
and--with-openssl-rpath
options)$ pushd python-3.x.x $ ./configure -C \ --with-openssl=/usr/local/custom-openssl \ --with-openssl-rpath=auto \ --prefix=/usr/local/python-3.x.x $ make -j8 $ make altinstall
Note
Patch releases of OpenSSL have a backwards compatible ABI. You don’t need to recompile Python to update OpenSSL. It’s sufficient to replace the custom OpenSSL installation with a newer version.


getopt
— C-style parser for command line options¶
Source code: Lib/getopt.py
Note
This module is considered feature complete. A more declarative and
extensible alternative to this API is provided in the optparse
module. Further functional enhancements for command line parameter
processing are provided either as third party modules on PyPI,
or else as features in the argparse
module.
This module helps scripts to parse the command line arguments in sys.argv
.
It supports the same conventions as the Unix getopt()
function (including
the special meanings of arguments of the form ‘-
’ and ‘--
‘). Long
options similar to those supported by GNU software may be used as well via an
optional third argument.
Users who are unfamiliar with the Unix getopt()
function should consider
using the argparse
module instead. Users who are familiar with the Unix
getopt()
function, but would like to get equivalent behavior while
writing less code and getting better help and error messages should consider
using the optparse
module. See Choosing an argument parsing library for
additional details.
This module provides two functions and an exception:
- getopt.getopt(args, shortopts, longopts=[])¶
Parses command line options and parameter list. args is the argument list to be parsed, without the leading reference to the running program. Typically, this means
sys.argv[1:]
. shortopts is the string of option letters that the script wants to recognize, with options that require an argument followed by a colon (':'
) and options that accept an optional argument followed by two colons ('::'
); i.e., the same format that Unixgetopt()
uses.Note
Unlike GNU
getopt()
, after a non-option argument, all further arguments are considered also non-options. This is similar to the way non-GNU Unix systems work.longopts, if specified, must be a list of strings with the names of the long options which should be supported. The leading
'--'
characters should not be included in the option name. Long options which require an argument should be followed by an equal sign ('='
). Long options which accept an optional argument should be followed by an equal sign and question mark ('=?'
). To accept only long options, shortopts should be an empty string. Long options on the command line can be recognized so long as they provide a prefix of the option name that matches exactly one of the accepted options. For example, if longopts is['foo', 'frob']
, the option--fo
will match as--foo
, but--f
will not match uniquely, soGetoptError
will be raised.The return value consists of two elements: the first is a list of
(option, value)
pairs; the second is the list of program arguments left after the option list was stripped (this is a trailing slice of args). Each option-and-value pair returned has the option as its first element, prefixed with a hyphen for short options (e.g.,'-x'
) or two hyphens for long options (e.g.,'--long-option'
), and the option argument as its second element, or an empty string if the option has no argument. The options occur in the list in the same order in which they were found, thus allowing multiple occurrences. Long and short options may be mixed.Changed in version 3.14: Optional arguments are supported.
- getopt.gnu_getopt(args, shortopts, longopts=[])¶
This function works like
getopt()
, except that GNU style scanning mode is used by default. This means that option and non-option arguments may be intermixed. Thegetopt()
function stops processing options as soon as a non-option argument is encountered.If the first character of the option string is
'+'
, or if the environment variablePOSIXLY_CORRECT
is set, then option processing stops as soon as a non-option argument is encountered.If the first character of the option string is
'-'
, non-option arguments that are followed by options are added to the list of option-and-value pairs as a pair that hasNone
as its first element and the list of non-option arguments as its second element. The second element of thegnu_getopt()
result is a list of program arguments after the last option.Changed in version 3.14: Support for returning intermixed options and non-option arguments in order.
- exception getopt.GetoptError¶
This is raised when an unrecognized option is found in the argument list or when an option requiring an argument is given none. The argument to the exception is a string indicating the cause of the error. For long options, an argument given to an option which does not require one will also cause this exception to be raised. The attributes
msg
andopt
give the error message and related option; if there is no specific option to which the exception relates,opt
is an empty string.
- exception getopt.error¶
Alias for
GetoptError
; for backward compatibility.
An example using only Unix style options:
>>> import getopt
>>> args = '-a -b -cfoo -d bar a1 a2'.split()
>>> args
['-a', '-b', '-cfoo', '-d', 'bar', 'a1', 'a2']
>>> optlist, args = getopt.getopt(args, 'abc:d:')
>>> optlist
[('-a', ''), ('-b', ''), ('-c', 'foo'), ('-d', 'bar')]
>>> args
['a1', 'a2']
Using long option names is equally easy:
>>> s = '--condition=foo --testing --output-file abc.def -x a1 a2'
>>> args = s.split()
>>> args
['--condition=foo', '--testing', '--output-file', 'abc.def', '-x', 'a1', 'a2']
>>> optlist, args = getopt.getopt(args, 'x', [
... 'condition=', 'output-file=', 'testing'])
>>> optlist
[('--condition', 'foo'), ('--testing', ''), ('--output-file', 'abc.def'), ('-x', '')]
>>> args
['a1', 'a2']
Optional arguments should be specified explicitly:
>>> s = '-Con -C --color=off --color a1 a2'
>>> args = s.split()
>>> args
['-Con', '-C', '--color=off', '--color', 'a1', 'a2']
>>> optlist, args = getopt.getopt(args, 'C::', ['color=?'])
>>> optlist
[('-C', 'on'), ('-C', ''), ('--color', 'off'), ('--color', '')]
>>> args
['a1', 'a2']
The order of options and non-option arguments can be preserved:
>>> s = 'a1 -x a2 a3 a4 --long a5 a6'
>>> args = s.split()
>>> args
['a1', '-x', 'a2', 'a3', 'a4', '--long', 'a5', 'a6']
>>> optlist, args = getopt.gnu_getopt(args, '-x:', ['long='])
>>> optlist
[(None, ['a1']), ('-x', 'a2'), (None, ['a3', 'a4']), ('--long', 'a5')]
>>> args
['a6']
In a script, typical usage is something like this:
import getopt, sys
def main():
try:
opts, args = getopt.getopt(sys.argv[1:], "ho:v", ["help", "output="])
except getopt.GetoptError as err:
# print help information and exit:
print(err) # will print something like "option -a not recognized"
usage()
sys.exit(2)
output = None
verbose = False
for o, a in opts:
if o == "-v":
verbose = True
elif o in ("-h", "--help"):
usage()
sys.exit()
elif o in ("-o", "--output"):
output = a
else:
assert False, "unhandled option"
process(args, output=output, verbose=verbose)
if __name__ == "__main__":
main()
Note that an equivalent command line interface could be produced with less code
and more informative help and error messages by using the optparse
module:
import optparse
if __name__ == '__main__':
parser = optparse.OptionParser()
parser.add_option('-o', '--output')
parser.add_option('-v', dest='verbose', action='store_true')
opts, args = parser.parse_args()
process(args, output=opts.output, verbose=opts.verbose)
A roughly equivalent command line interface for this case can also be
produced by using the argparse
module:
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('-o', '--output')
parser.add_argument('-v', dest='verbose', action='store_true')
parser.add_argument('rest', nargs='*')
args = parser.parse_args()
process(args.rest, output=args.output, verbose=args.verbose)
See Choosing an argument parsing library for details on how the argparse
version of this code differs in behaviour from the optparse
(and
getopt
) version.


— Manipulate raw audio data

audioop
— Manipulate raw audio data¶
Deprecated since version 3.11, removed in version 3.13.
This module is no longer part of the Python standard library. It was removed in Python 3.13 after being deprecated in Python 3.11. The removal was decided in PEP 594.
The last version of Python that provided the audioop
module was
Python 3.12.


— The password database

pwd
— The password database¶
This module provides access to the Unix user account and password database. It is available on all Unix versions.
Availability: Unix, not WASI, not iOS.
Password database entries are reported as a tuple-like object, whose attributes
correspond to the members of the passwd
structure (Attribute field below,
see <pwd.h>
):
Index |
Attribute |
Meaning |
|---|---|---|
0 |
|
Login name |
1 |
|
Optional encrypted password |
2 |
|
Numerical user ID |
3 |
|
Numerical group ID |
4 |
|
User name or comment field |
5 |
|
User home directory |
6 |
|
User command interpreter |
The uid and gid items are integers, all others are strings. KeyError
is
raised if the entry asked for cannot be found.
Note
In traditional Unix the field pw_passwd
usually contains a password
encrypted with a DES derived algorithm. However most
modern unices use a so-called shadow password system. On those unices the
pw_passwd field only contains an asterisk ('*'
) or the letter 'x'
where the encrypted password is stored in a file /etc/shadow
which is
not world readable. Whether the pw_passwd field contains anything useful is
system-dependent.
It defines the following items:
- pwd.getpwuid(uid)¶
Return the password database entry for the given numeric user ID.
- pwd.getpwnam(name)¶
Return the password database entry for the given user name.
- pwd.getpwall()¶
Return a list of all available password database entries, in arbitrary order.
See also
- Module
grp
An interface to the group database, similar to this.


An introduction to the ipaddress module

An introduction to the ipaddress module¶
- author:
Peter Moody
- author:
Nick Coghlan
Creating Address/Network/Interface objects¶
Since ipaddress
is a module for inspecting and manipulating IP addresses,
the first thing you’ll want to do is create some objects. You can use
ipaddress
to create objects from strings and integers.
A Note on IP Versions¶
For readers that aren’t particularly familiar with IP addressing, it’s important to know that the Internet Protocol (IP) is currently in the process of moving from version 4 of the protocol to version 6. This transition is occurring largely because version 4 of the protocol doesn’t provide enough addresses to handle the needs of the whole world, especially given the increasing number of devices with direct connections to the internet.
Explaining the details of the differences between the two versions of the protocol is beyond the scope of this introduction, but readers need to at least be aware that these two versions exist, and it will sometimes be necessary to force the use of one version or the other.
IP Host Addresses¶
Addresses, often referred to as “host addresses” are the most basic unit
when working with IP addressing. The simplest way to create addresses is
to use the ipaddress.ip_address()
factory function, which automatically
determines whether to create an IPv4 or IPv6 address based on the passed in
value:
>>> ipaddress.ip_address('192.0.2.1')
IPv4Address('192.0.2.1')
>>> ipaddress.ip_address('2001:DB8::1')
IPv6Address('2001:db8::1')
Addresses can also be created directly from integers. Values that will fit within 32 bits are assumed to be IPv4 addresses:
>>> ipaddress.ip_address(3221225985)
IPv4Address('192.0.2.1')
>>> ipaddress.ip_address(42540766411282592856903984951653826561)
IPv6Address('2001:db8::1')
To force the use of IPv4 or IPv6 addresses, the relevant classes can be invoked directly. This is particularly useful to force creation of IPv6 addresses for small integers:
>>> ipaddress.ip_address(1)
IPv4Address('0.0.0.1')
>>> ipaddress.IPv4Address(1)
IPv4Address('0.0.0.1')
>>> ipaddress.IPv6Address(1)
IPv6Address('::1')
Defining Networks¶
Host addresses are usually grouped together into IP networks, so
ipaddress
provides a way to create, inspect and manipulate network
definitions. IP network objects are constructed from strings that define the
range of host addresses that are part of that network. The simplest form
for that information is a “network address/network prefix” pair, where the
prefix defines the number of leading bits that are compared to determine
whether or not an address is part of the network and the network address
defines the expected value of those bits.
As for addresses, a factory function is provided that determines the correct IP version automatically:
>>> ipaddress.ip_network('192.0.2.0/24')
IPv4Network('192.0.2.0/24')
>>> ipaddress.ip_network('2001:db8::0/96')
IPv6Network('2001:db8::/96')
Network objects cannot have any host bits set. The practical effect of this
is that 192.0.2.1/24
does not describe a network. Such definitions are
referred to as interface objects since the ip-on-a-network notation is
commonly used to describe network interfaces of a computer on a given network
and are described further in the next section.
By default, attempting to create a network object with host bits set will
result in ValueError
being raised. To request that the
additional bits instead be coerced to zero, the flag strict=False
can
be passed to the constructor:
>>> ipaddress.ip_network('192.0.2.1/24')
Traceback (most recent call last):
...
ValueError: 192.0.2.1/24 has host bits set
>>> ipaddress.ip_network('192.0.2.1/24', strict=False)
IPv4Network('192.0.2.0/24')
While the string form offers significantly more flexibility, networks can also be defined with integers, just like host addresses. In this case, the network is considered to contain only the single address identified by the integer, so the network prefix includes the entire network address:
>>> ipaddress.ip_network(3221225984)
IPv4Network('192.0.2.0/32')
>>> ipaddress.ip_network(42540766411282592856903984951653826560)
IPv6Network('2001:db8::/128')
As with addresses, creation of a particular kind of network can be forced by calling the class constructor directly instead of using the factory function.
Host Interfaces¶
As mentioned just above, if you need to describe an address on a particular
network, neither the address nor the network classes are sufficient.
Notation like 192.0.2.1/24
is commonly used by network engineers and the
people who write tools for firewalls and routers as shorthand for “the host
192.0.2.1
on the network 192.0.2.0/24
”, Accordingly, ipaddress
provides a set of hybrid classes that associate an address with a particular
network. The interface for creation is identical to that for defining network
objects, except that the address portion isn’t constrained to being a network
address.
>>> ipaddress.ip_interface('192.0.2.1/24')
IPv4Interface('192.0.2.1/24')
>>> ipaddress.ip_interface('2001:db8::1/96')
IPv6Interface('2001:db8::1/96')
Integer inputs are accepted (as with networks), and use of a particular IP version can be forced by calling the relevant constructor directly.
Inspecting Address/Network/Interface Objects¶
You’ve gone to the trouble of creating an IPv(4|6)(Address|Network|Interface)
object, so you probably want to get information about it. ipaddress
tries to make doing this easy and intuitive.
Extracting the IP version:
>>> addr4 = ipaddress.ip_address('192.0.2.1')
>>> addr6 = ipaddress.ip_address('2001:db8::1')
>>> addr6.version
6
>>> addr4.version
4
Obtaining the network from an interface:
>>> host4 = ipaddress.ip_interface('192.0.2.1/24')
>>> host4.network
IPv4Network('192.0.2.0/24')
>>> host6 = ipaddress.ip_interface('2001:db8::1/96')
>>> host6.network
IPv6Network('2001:db8::/96')
Finding out how many individual addresses are in a network:
>>> net4 = ipaddress.ip_network('192.0.2.0/24')
>>> net4.num_addresses
256
>>> net6 = ipaddress.ip_network('2001:db8::0/96')
>>> net6.num_addresses
4294967296
Iterating through the “usable” addresses on a network:
>>> net4 = ipaddress.ip_network('192.0.2.0/24')
>>> for x in net4.hosts():
... print(x)
192.0.2.1
192.0.2.2
192.0.2.3
192.0.2.4
...
192.0.2.252
192.0.2.253
192.0.2.254
Obtaining the netmask (i.e. set bits corresponding to the network prefix) or the hostmask (any bits that are not part of the netmask):
>>> net4 = ipaddress.ip_network('192.0.2.0/24')
>>> net4.netmask
IPv4Address('255.255.255.0')
>>> net4.hostmask
IPv4Address('0.0.0.255')
>>> net6 = ipaddress.ip_network('2001:db8::0/96')
>>> net6.netmask
IPv6Address('ffff:ffff:ffff:ffff:ffff:ffff::')
>>> net6.hostmask
IPv6Address('::ffff:ffff')
Exploding or compressing the address:
>>> addr6.exploded
'2001:0db8:0000:0000:0000:0000:0000:0001'
>>> addr6.compressed
'2001:db8::1'
>>> net6.exploded
'2001:0db8:0000:0000:0000:0000:0000:0000/96'
>>> net6.compressed
'2001:db8::/96'
While IPv4 doesn’t support explosion or compression, the associated objects still provide the relevant properties so that version neutral code can easily ensure the most concise or most verbose form is used for IPv6 addresses while still correctly handling IPv4 addresses.
Networks as lists of Addresses¶
It’s sometimes useful to treat networks as lists. This means it is possible to index them like this:
>>> net4[1]
IPv4Address('192.0.2.1')
>>> net4[-1]
IPv4Address('192.0.2.255')
>>> net6[1]
IPv6Address('2001:db8::1')
>>> net6[-1]
IPv6Address('2001:db8::ffff:ffff')
It also means that network objects lend themselves to using the list membership test syntax like this:
if address in network:
# do something
Containment testing is done efficiently based on the network prefix:
>>> addr4 = ipaddress.ip_address('192.0.2.1')
>>> addr4 in ipaddress.ip_network('192.0.2.0/24')
True
>>> addr4 in ipaddress.ip_network('192.0.3.0/24')
False
Comparisons¶
ipaddress
provides some simple, hopefully intuitive ways to compare
objects, where it makes sense:
>>> ipaddress.ip_address('192.0.2.1') < ipaddress.ip_address('192.0.2.2')
True
A TypeError
exception is raised if you try to compare objects of
different versions or different types.
Using IP Addresses with other modules¶
Other modules that use IP addresses (such as socket
) usually won’t
accept objects from this module directly. Instead, they must be coerced to
an integer or string that the other module will accept:
>>> addr4 = ipaddress.ip_address('192.0.2.1')
>>> str(addr4)
'192.0.2.1'
>>> int(addr4)
3221225985
Getting more detail when instance creation fails¶
When creating address/network/interface objects using the version-agnostic
factory functions, any errors will be reported as ValueError
with
a generic error message that simply says the passed in value was not
recognized as an object of that type. The lack of a specific error is
because it’s necessary to know whether the value is supposed to be IPv4
or IPv6 in order to provide more detail on why it has been rejected.
To support use cases where it is useful to have access to this additional
detail, the individual class constructors actually raise the
ValueError
subclasses ipaddress.AddressValueError
and
ipaddress.NetmaskValueError
to indicate exactly which part of
the definition failed to parse correctly.
The error messages are significantly more detailed when using the class constructors directly. For example:
>>> ipaddress.ip_address("192.168.0.256")
Traceback (most recent call last):
...
ValueError: '192.168.0.256' does not appear to be an IPv4 or IPv6 address
>>> ipaddress.IPv4Address("192.168.0.256")
Traceback (most recent call last):
...
ipaddress.AddressValueError: Octet 256 (> 255) not permitted in '192.168.0.256'
>>> ipaddress.ip_network("192.168.0.1/64")
Traceback (most recent call last):
...
ValueError: '192.168.0.1/64' does not appear to be an IPv4 or IPv6 network
>>> ipaddress.IPv4Network("192.168.0.1/64")
Traceback (most recent call last):
...
ipaddress.NetmaskValueError: '64' is not a valid netmask
However, both of the module specific exceptions have ValueError
as their
parent class, so if you’re not concerned with the particular type of error,
you can still write code like the following:
try:
network = ipaddress.IPv4Network(address)
except ValueError:
print('address/netmask is invalid for IPv4:', address)


Superseded Modules

Superseded Modules¶
The modules described in this chapter have been superseded by other modules for most use cases, and are retained primarily to preserve backwards compatibility.
Modules may appear in this chapter because they only cover a limited subset of
a problem space, and a more generally applicable solution is available elsewhere
in the standard library (for example, getopt
covers the very specific
task of “mimic the C getopt()
API in Python”, rather than the broader
command line option parsing and argument parsing capabilities offered by
optparse
and argparse
).
Alternatively, modules may appear in this chapter because they are deprecated outright, and awaiting removal in a future release, or they are soft deprecated and their use is actively discouraged in new projects. With the removal of various obsolete modules through PEP 594, there are currently no modules in this latter category.


— Encode and decode XDR data

xdrlib
— Encode and decode XDR data¶
Deprecated since version 3.11, removed in version 3.13.
This module is no longer part of the Python standard library. It was removed in Python 3.13 after being deprecated in Python 3.11. The removal was decided in PEP 594.
The last version of Python that provided the xdrlib
module was
Python 3.12.


Download Python 3.14 documentation

Download Python 3.14 documentation
Last updated on: Feb 18, 2026 (17:01 UTC).
Download an archive containing all the documentation for this version of Python:
| Format | Packed as .zip | Packed as .tar.bz2 |
|---|---|---|
| HTML | Download | Download |
| Plain text | Download | Download |
| Texinfo | Download | Download |
| EPUB | Download |
We no longer provide pre-built PDFs of the documentation.
To build a PDF archive, follow the instructions in the
Developer's Guide
and run make dist-pdf
in the Doc/
directory of a copy of the CPython repository.
See the directory listing for file sizes.
Problems
Open an issue if you have comments or suggestions for the Python documentation.


Virtual Environments and Packages

12. Virtual Environments and Packages¶
12.1. Introduction¶
Python applications will often use packages and modules that don’t come as part of the standard library. Applications will sometimes need a specific version of a library, because the application may require that a particular bug has been fixed or the application may be written using an obsolete version of the library’s interface.
This means it may not be possible for one Python installation to meet the requirements of every application. If application A needs version 1.0 of a particular module but application B needs version 2.0, then the requirements are in conflict and installing either version 1.0 or 2.0 will leave one application unable to run.
The solution for this problem is to create a virtual environment, a self-contained directory tree that contains a Python installation for a particular version of Python, plus a number of additional packages.
Different applications can then use different virtual environments. To resolve the earlier example of conflicting requirements, application A can have its own virtual environment with version 1.0 installed while application B has another virtual environment with version 2.0. If application B requires a library be upgraded to version 3.0, this will not affect application A’s environment.
12.2. Creating Virtual Environments¶
The module used to create and manage virtual environments is called
venv
. venv
will install the Python version from which
the command was run (as reported by the --version
option).
For instance, executing the command with python3.12
will install
version 3.12.
To create a virtual environment, decide upon a directory where you want to
place it, and run the venv
module as a script with the directory path:
python -m venv tutorial-env
This will create the tutorial-env
directory if it doesn’t exist,
and also create directories inside it containing a copy of the Python
interpreter and various supporting files.
A common directory location for a virtual environment is .venv
.
This name keeps the directory typically hidden in your shell and thus
out of the way while giving it a name that explains why the directory
exists. It also prevents clashing with .env
environment variable
definition files that some tooling supports.
Once you’ve created a virtual environment, you may activate it.
On Windows, run:
tutorial-env\Scripts\activate
On Unix or MacOS, run:
source tutorial-env/bin/activate
(This script is written for the bash shell. If you use the
csh or fish shells, there are alternate
activate.csh
and activate.fish
scripts you should use
instead.)
Activating the virtual environment will change your shell’s prompt to show what
virtual environment you’re using, and modify the environment so that running
python
will get you that particular version and installation of Python.
For example:
$ source ~/envs/tutorial-env/bin/activate
(tutorial-env) $ python
Python 3.5.1 (default, May 6 2016, 10:59:36)
...
>>> import sys
>>> sys.path
['', '/usr/local/lib/python35.zip', ...,
'~/envs/tutorial-env/lib/python3.5/site-packages']
>>>
To deactivate a virtual environment, type:
deactivate
into the terminal.
12.3. Managing Packages with pip¶
You can install, upgrade, and remove packages using a program called
pip. By default pip
will install packages from the Python
Package Index. You can browse the Python
Package Index by going to it in your web browser.
pip
has a number of subcommands: “install”, “uninstall”,
“freeze”, etc. (Consult the Installing Python Modules guide for
complete documentation for pip
.)
You can install the latest version of a package by specifying a package’s name:
(tutorial-env) $ python -m pip install novas
Collecting novas
Downloading novas-3.1.1.3.tar.gz (136kB)
Installing collected packages: novas
Running setup.py install for novas
Successfully installed novas-3.1.1.3
You can also install a specific version of a package by giving the
package name followed by ==
and the version number:
(tutorial-env) $ python -m pip install requests==2.6.0
Collecting requests==2.6.0
Using cached requests-2.6.0-py2.py3-none-any.whl
Installing collected packages: requests
Successfully installed requests-2.6.0
If you re-run this command, pip
will notice that the requested
version is already installed and do nothing. You can supply a
different version number to get that version, or you can run python
-m pip install --upgrade
to upgrade the package to the latest version:
(tutorial-env) $ python -m pip install --upgrade requests
Collecting requests
Installing collected packages: requests
Found existing installation: requests 2.6.0
Uninstalling requests-2.6.0:
Successfully uninstalled requests-2.6.0
Successfully installed requests-2.7.0
python -m pip uninstall
followed by one or more package names will
remove the packages from the virtual environment.
python -m pip show
will display information about a particular package:
(tutorial-env) $ python -m pip show requests
---
Metadata-Version: 2.0
Name: requests
Version: 2.7.0
Summary: Python HTTP for Humans.
Home-page: http://python-requests.org
Author: Kenneth Reitz
Author-email: me@kennethreitz.com
License: Apache 2.0
Location: /Users/akuchling/envs/tutorial-env/lib/python3.4/site-packages
Requires:
python -m pip list
will display all of the packages installed in
the virtual environment:
(tutorial-env) $ python -m pip list
novas (3.1.1.3)
numpy (1.9.2)
pip (7.0.3)
requests (2.7.0)
setuptools (16.0)
python -m pip freeze
will produce a similar list of the installed packages,
but the output uses the format that python -m pip install
expects.
A common convention is to put this list in a requirements.txt
file:
(tutorial-env) $ python -m pip freeze > requirements.txt
(tutorial-env) $ cat requirements.txt
novas==3.1.1.3
numpy==1.9.2
requests==2.7.0
The requirements.txt
can then be committed to version control and
shipped as part of an application. Users can then install all the
necessary packages with install -r
:
(tutorial-env) $ python -m pip install -r requirements.txt
Collecting novas==3.1.1.3 (from -r requirements.txt (line 1))
...
Collecting numpy==1.9.2 (from -r requirements.txt (line 2))
...
Collecting requests==2.7.0 (from -r requirements.txt (line 3))
...
Installing collected packages: novas, numpy, requests
Running setup.py install for novas
Successfully installed novas-3.1.1.3 numpy-1.9.2 requests-2.7.0
pip
has many more options. Consult the Installing Python Modules
guide for complete documentation for pip
. When you’ve written
a package and want to make it available on the Python Package Index,
consult the Python packaging user guide.


Removed Modules

Removed Modules¶
The modules described in this chapter have been removed from the Python standard library. They are documented here to help people find replacements.
aifc
— Read and write AIFF and AIFC filesasynchat
— Asynchronous socket command/response handlerasyncore
— Asynchronous socket handleraudioop
— Manipulate raw audio datacgi
— Common Gateway Interface supportcgitb
— Traceback manager for CGI scriptschunk
— Read IFF chunked datacrypt
— Function to check Unix passwordsdistutils
— Building and installing Python modulesimghdr
— Determine the type of an imageimp
— Access the import internalsmailcap
— Mailcap file handlingmsilib
— Read and write Microsoft Installer filesnis
— Interface to Sun’s NIS (Yellow Pages)nntplib
— NNTP protocol clientossaudiodev
— Access to OSS-compatible audio devicespipes
— Interface to shell pipelinessmtpd
— SMTP Serversndhdr
— Determine type of sound filespwd
— The shadow password databasesunau
— Read and write Sun AU filestelnetlib
— Telnet clientuu
— Encode and decode uuencode filesxdrlib
— Encode and decode XDR data


Python HOWTOs

Python HOWTOs¶
Python HOWTOs are documents that cover a specific topic in-depth. Modeled on the Linux Documentation Project’s HOWTO collection, this collection is an effort to foster documentation that’s more detailed than the Python Library Reference.
General:
Advanced development:
Debugging and profiling:


— Terminal control functions

tty
— Terminal control functions¶
Source code: Lib/tty.py
The tty
module defines functions for putting the tty into cbreak and raw
modes.
Availability: Unix.
Because it requires the termios
module, it will work only on Unix.
The tty
module defines the following functions:
- tty.cfmakeraw(mode)¶
Convert the tty attribute list mode, which is a list like the one returned by
termios.tcgetattr()
, to that of a tty in raw mode.Added in version 3.12.
- tty.cfmakecbreak(mode)¶
Convert the tty attribute list mode, which is a list like the one returned by
termios.tcgetattr()
, to that of a tty in cbreak mode.This clears the
ECHO
andICANON
local mode flags in mode as well as setting the minimum input to 1 byte with no delay.Added in version 3.12.
Changed in version 3.12.2: The
ICRNL
flag is no longer cleared. This matches Linux and macOSstty cbreak
behavior and whatsetcbreak()
historically did.
- tty.setraw(fd, when=termios.TCSAFLUSH)¶
Change the mode of the file descriptor fd to raw. If when is omitted, it defaults to
termios.TCSAFLUSH
, and is passed totermios.tcsetattr()
. The return value oftermios.tcgetattr()
is saved before setting fd to raw mode; this value is returned.Changed in version 3.12: The return value is now the original tty attributes, instead of
None
.
- tty.setcbreak(fd, when=termios.TCSAFLUSH)¶
Change the mode of file descriptor fd to cbreak. If when is omitted, it defaults to
termios.TCSAFLUSH
, and is passed totermios.tcsetattr()
. The return value oftermios.tcgetattr()
is saved before setting fd to cbreak mode; this value is returned.This clears the
ECHO
andICANON
local mode flags as well as setting the minimum input to 1 byte with no delay.Changed in version 3.12: The return value is now the original tty attributes, instead of
None
.Changed in version 3.12.2: The
ICRNL
flag is no longer cleared. This restores the behavior of Python 3.11 and earlier as well as matching what Linux, macOS, & BSDs describe in theirstty(1)
man pages regarding cbreak mode.
See also
- Module
termios
Low-level terminal control interface.


Common Object Structures

Common Object Structures¶
There are a large number of structures which are used in the definition of object types for Python. This section describes these structures and how they are used.
Base object types and macros¶
All Python objects ultimately share a small number of fields at the beginning
of the object’s representation in memory. These are represented by the
PyObject
and PyVarObject
types, which are defined, in turn,
by the expansions of some macros also used, whether directly or indirectly, in
the definition of all other Python objects. Additional macros can be found
under reference counting.
-
type PyObject¶
- Part of the Limited API. (Only some members are part of the stable ABI.)
All object types are extensions of this type. This is a type which contains the information Python needs to treat a pointer to an object as an object. In a normal “release” build, it contains only the object’s reference count and a pointer to the corresponding type object. Nothing is actually declared to be a
PyObject
, but every pointer to a Python object can be cast to a PyObject*.The members must not be accessed directly; instead use macros such as
Py_REFCNT
andPy_TYPE
.-
Py_ssize_t ob_refcnt¶
- Part of the Stable ABI.
The object’s reference count, as returned by
Py_REFCNT
. Do not use this field directly; instead use functions and macros such asPy_REFCNT
,Py_INCREF()
andPy_DecRef()
.The field type may be different from
Py_ssize_t
, depending on build configuration and platform.
-
PyTypeObject *ob_type¶
- Part of the Stable ABI.
The object’s type. Do not use this field directly; use
Py_TYPE
andPy_SET_TYPE()
instead.
-
Py_ssize_t ob_refcnt¶
-
type PyVarObject¶
- Part of the Limited API. (Only some members are part of the stable ABI.)
An extension of
PyObject
that adds theob_size
field. This is intended for objects that have some notion of length.As with
PyObject
, the members must not be accessed directly; instead use macros such asPy_SIZE
,Py_REFCNT
andPy_TYPE
.-
Py_ssize_t ob_size¶
- Part of the Stable ABI.
A size field, whose contents should be considered an object’s internal implementation detail.
Do not use this field directly; use
Py_SIZE
instead.Object creation functions such as
PyObject_NewVar()
will generally set this field to the requested size (number of items). After creation, arbitrary values can be stored inob_size
usingPy_SET_SIZE
.To get an object’s publicly exposed length, as returned by the Python function
len()
, usePyObject_Length()
instead.
-
Py_ssize_t ob_size¶
-
PyObject_HEAD¶
This is a macro used when declaring new types which represent objects without a varying length. The PyObject_HEAD macro expands to:
PyObject ob_base;
See documentation of
PyObject
above.
-
PyObject_VAR_HEAD¶
This is a macro used when declaring new types which represent objects with a length that varies from instance to instance. The PyObject_VAR_HEAD macro expands to:
PyVarObject ob_base;
See documentation of
PyVarObject
above.
-
PyTypeObject PyBaseObject_Type¶
- Part of the Stable ABI.
The base class of all other objects, the same as
object
in Python.
-
int Py_Is(PyObject *x, PyObject *y)¶
- Part of the Stable ABI since version 3.10.
Test if the x object is the y object, the same as
x is y
in Python.Added in version 3.10.
-
int Py_IsNone(PyObject *x)¶
- Part of the Stable ABI since version 3.10.
Test if an object is the
None
singleton, the same asx is None
in Python.Added in version 3.10.
-
int Py_IsTrue(PyObject *x)¶
- Part of the Stable ABI since version 3.10.
Test if an object is the
True
singleton, the same asx is True
in Python.Added in version 3.10.
-
int Py_IsFalse(PyObject *x)¶
- Part of the Stable ABI since version 3.10.
Test if an object is the
False
singleton, the same asx is False
in Python.Added in version 3.10.
-
PyTypeObject *Py_TYPE(PyObject *o)¶
- Return value: Borrowed reference. Part of the Stable ABI since version 3.14.
Get the type of the Python object o.
The returned reference is borrowed from o. Do not release it with
Py_DECREF()
or similar.
-
int Py_IS_TYPE(PyObject *o, PyTypeObject *type)¶
Return non-zero if the object o type is type. Return zero otherwise. Equivalent to:
Py_TYPE(o) == type
.Added in version 3.9.
-
void Py_SET_TYPE(PyObject *o, PyTypeObject *type)¶
Set the type of object o to type, without any checking or reference counting.
This is a very low-level operation. Consider instead setting the Python attribute
__class__
usingPyObject_SetAttrString()
or similar.Note that assigning an incompatible type can lead to undefined behavior.
If type is a heap type, the caller must create a new reference to it. Similarly, if the old type of o is a heap type, the caller must release a reference to that type.
Added in version 3.9.
-
Py_ssize_t Py_SIZE(PyVarObject *o)¶
Get the
ob_size
field of o.Changed in version 3.11:
Py_SIZE()
is changed to an inline static function. The parameter type is no longer const PyVarObject*.
-
void Py_SET_SIZE(PyVarObject *o, Py_ssize_t size)¶
Set the
ob_size
field of o to size.Added in version 3.9.
-
PyObject_HEAD_INIT(type)¶
This is a macro which expands to initialization values for a new
PyObject
type. This macro expands to:_PyObject_EXTRA_INIT 1, type,
-
PyVarObject_HEAD_INIT(type, size)¶
This is a macro which expands to initialization values for a new
PyVarObject
type, including theob_size
field. This macro expands to:_PyObject_EXTRA_INIT 1, type, size,
Implementing functions and methods¶
-
type PyCFunction¶
- Part of the Stable ABI.
Type of the functions used to implement most Python callables in C. Functions of this type take two PyObject* parameters and return one such value. If the return value is
NULL
, an exception shall have been set. If notNULL
, the return value is interpreted as the return value of the function as exposed in Python. The function must return a new reference.The function signature is:
PyObject *PyCFunction(PyObject *self, PyObject *args);
-
type PyCFunctionWithKeywords¶
- Part of the Stable ABI.
Type of the functions used to implement Python callables in C with signature METH_VARARGS | METH_KEYWORDS. The function signature is:
PyObject *PyCFunctionWithKeywords(PyObject *self, PyObject *args, PyObject *kwargs);
-
type PyCFunctionFast¶
- Part of the Stable ABI since version 3.13.
Type of the functions used to implement Python callables in C with signature
METH_FASTCALL
. The function signature is:PyObject *PyCFunctionFast(PyObject *self, PyObject *const *args, Py_ssize_t nargs);
-
type PyCFunctionFastWithKeywords¶
- Part of the Stable ABI since version 3.13.
Type of the functions used to implement Python callables in C with signature METH_FASTCALL | METH_KEYWORDS. The function signature is:
PyObject *PyCFunctionFastWithKeywords(PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames);
-
type PyCMethod¶
Type of the functions used to implement Python callables in C with signature METH_METHOD | METH_FASTCALL | METH_KEYWORDS. The function signature is:
PyObject *PyCMethod(PyObject *self, PyTypeObject *defining_class, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
Added in version 3.9.
-
type PyMethodDef¶
- Part of the Stable ABI (including all members).
Structure used to describe a method of an extension type. This structure has four fields:
-
const char *ml_name¶
Name of the method.
-
PyCFunction ml_meth¶
Pointer to the C implementation.
-
int ml_flags¶
Flags bits indicating how the call should be constructed.
-
const char *ml_doc¶
Points to the contents of the docstring.
-
const char *ml_name¶
The ml_meth
is a C function pointer.
The functions may be of different
types, but they always return PyObject*. If the function is not of
the PyCFunction
, the compiler will require a cast in the method table.
Even though PyCFunction
defines the first parameter as
PyObject*, it is common that the method implementation uses the
specific C type of the self object.
The ml_flags
field is a bitfield which can include
the following flags.
The individual flags indicate either a calling convention or a binding
convention.
There are these calling conventions:
-
METH_VARARGS¶
- Part of the Stable ABI.
This is the typical calling convention, where the methods have the type
PyCFunction
. The function expects two PyObject* values. The first one is the self object for methods; for module functions, it is the module object. The second parameter (often called args) is a tuple object representing all arguments. This parameter is typically processed usingPyArg_ParseTuple()
orPyArg_UnpackTuple()
.
-
METH_KEYWORDS¶
Can only be used in certain combinations with other flags: METH_VARARGS | METH_KEYWORDS, METH_FASTCALL | METH_KEYWORDS and METH_METHOD | METH_FASTCALL | METH_KEYWORDS.
- METH_VARARGS | METH_KEYWORDS
Methods with these flags must be of type
PyCFunctionWithKeywords
. The function expects three parameters: self, args, kwargs where kwargs is a dictionary of all the keyword arguments or possiblyNULL
if there are no keyword arguments. The parameters are typically processed usingPyArg_ParseTupleAndKeywords()
.
-
METH_FASTCALL¶
- Part of the Stable ABI since version 3.7.
Fast calling convention supporting only positional arguments. The methods have the type
PyCFunctionFast
. The first parameter is self, the second parameter is a C array of PyObject* values indicating the arguments and the third parameter is the number of arguments (the length of the array).Added in version 3.7.
Changed in version 3.10:
METH_FASTCALL
is now part of the stable ABI.
- METH_FASTCALL | METH_KEYWORDS
Extension of
METH_FASTCALL
supporting also keyword arguments, with methods of typePyCFunctionFastWithKeywords
. Keyword arguments are passed the same way as in the vectorcall protocol: there is an additional fourth PyObject* parameter which is a tuple representing the names of the keyword arguments (which are guaranteed to be strings) or possiblyNULL
if there are no keywords. The values of the keyword arguments are stored in the args array, after the positional arguments.Added in version 3.7.
-
METH_METHOD¶
- Part of the Stable ABI since version 3.7.
Can only be used in the combination with other flags: METH_METHOD | METH_FASTCALL | METH_KEYWORDS.
- METH_METHOD | METH_FASTCALL | METH_KEYWORDS
Extension of METH_FASTCALL | METH_KEYWORDS supporting the defining class, that is, the class that contains the method in question. The defining class might be a superclass of
Py_TYPE(self)
.The method needs to be of type
PyCMethod
, the same as forMETH_FASTCALL | METH_KEYWORDS
withdefining_class
argument added afterself
.Added in version 3.9.
-
METH_NOARGS¶
- Part of the Stable ABI.
Methods without parameters don’t need to check whether arguments are given if they are listed with the
METH_NOARGS
flag. They need to be of typePyCFunction
. The first parameter is typically named self and will hold a reference to the module or object instance. In all cases the second parameter will beNULL
.The function must have 2 parameters. Since the second parameter is unused,
Py_UNUSED
can be used to prevent a compiler warning.
-
METH_O¶
- Part of the Stable ABI.
Methods with a single object argument can be listed with the
METH_O
flag, instead of invokingPyArg_ParseTuple()
with a"O"
argument. They have the typePyCFunction
, with the self parameter, and a PyObject* parameter representing the single argument.
These two constants are not used to indicate the calling convention but the binding when used with methods of classes. These may not be used for functions defined for modules. At most one of these flags may be set for any given method.
-
METH_CLASS¶
- Part of the Stable ABI.
The method will be passed the type object as the first parameter rather than an instance of the type. This is used to create class methods, similar to what is created when using the
classmethod()
built-in function.
-
METH_STATIC¶
- Part of the Stable ABI.
The method will be passed
NULL
as the first parameter rather than an instance of the type. This is used to create static methods, similar to what is created when using thestaticmethod()
built-in function.
One other constant controls whether a method is loaded in place of another definition with the same method name.
-
METH_COEXIST¶
- Part of the Stable ABI.
The method will be loaded in place of existing definitions. Without METH_COEXIST, the default is to skip repeated definitions. Since slot wrappers are loaded before the method table, the existence of a sq_contains slot, for example, would generate a wrapped method named
__contains__()
and preclude the loading of a corresponding PyCFunction with the same name. With the flag defined, the PyCFunction will be loaded in place of the wrapper object and will co-exist with the slot. This is helpful because calls to PyCFunctions are optimized more than wrapper object calls.
-
PyTypeObject PyCMethod_Type¶
The type object corresponding to Python C method objects. This is available as
types.BuiltinMethodType
in the Python layer.
-
int PyCMethod_Check(PyObject *op)¶
Return true if op is an instance of the
PyCMethod_Type
type or a subtype of it. This function always succeeds.
-
int PyCMethod_CheckExact(PyObject *op)¶
This is the same as
PyCMethod_Check()
, but does not account for subtypes.
-
PyObject *PyCMethod_New(PyMethodDef *ml, PyObject *self, PyObject *module, PyTypeObject *cls)¶
- Return value: New reference. Part of the Stable ABI since version 3.9.
Turn ml into a Python callable object. The caller must ensure that ml outlives the callable. Typically, ml is defined as a static variable.
The self parameter will be passed as the self argument to the C function in
ml->ml_meth
when invoked. self can beNULL
.The callable object’s
__module__
attribute can be set from the given module argument. module should be a Python string, which will be used as name of the module the function is defined in. If unavailable, it can be set toNone
orNULL
.See also
The cls parameter will be passed as the defining_class argument to the C function. Must be set if
METH_METHOD
is set onml->ml_flags
.Added in version 3.9.
-
PyTypeObject PyCFunction_Type¶
- Part of the Stable ABI.
The type object corresponding to Python C function objects. This is available as
types.BuiltinFunctionType
in the Python layer.
-
int PyCFunction_Check(PyObject *op)¶
Return true if op is an instance of the
PyCFunction_Type
type or a subtype of it. This function always succeeds.
-
int PyCFunction_CheckExact(PyObject *op)¶
This is the same as
PyCFunction_Check()
, but does not account for subtypes.
-
PyObject *PyCFunction_NewEx(PyMethodDef *ml, PyObject *self, PyObject *module)¶
- Return value: New reference. Part of the Stable ABI.
Equivalent to
PyCMethod_New(ml, self, module, NULL)
.
-
PyObject *PyCFunction_New(PyMethodDef *ml, PyObject *self)¶
- Return value: New reference. Part of the Stable ABI since version 3.4.
Equivalent to
PyCMethod_New(ml, self, NULL, NULL)
.
-
int PyCFunction_GetFlags(PyObject *func)¶
- Part of the Stable ABI.
Get the function’s flags on func as they were passed to
ml_flags
.If func is not a C function object, this fails with an exception. func must not be
NULL
.This function returns the function’s flags on success, and
-1
with an exception set on failure.
-
int PyCFunction_GET_FLAGS(PyObject *func)¶
This is the same as
PyCFunction_GetFlags()
, but without error or type checking.
-
PyCFunction PyCFunction_GetFunction(PyObject *func)¶
- Part of the Stable ABI.
Get the function pointer on func as it was passed to
ml_meth
.If func is not a C function object, this fails with an exception. func must not be
NULL
.This function returns the function pointer on success, and
NULL
with an exception set on failure.
-
int PyCFunction_GET_FUNCTION(PyObject *func)¶
This is the same as
PyCFunction_GetFunction()
, but without error or type checking.
-
PyObject *PyCFunction_GetSelf(PyObject *func)¶
- Part of the Stable ABI.
Get the “self” object on func. This is the object that would be passed to the first argument of a
PyCFunction
. For C function objects created through aPyMethodDef
on aPyModuleDef
, this is the resulting module object.If func is not a C function object, this fails with an exception. func must not be
NULL
.This function returns a borrowed reference to the “self” object on success, and
NULL
with an exception set on failure.
-
PyObject *PyCFunction_GET_SELF(PyObject *func)¶
This is the same as
PyCFunction_GetSelf()
, but without error or type checking.
Accessing attributes of extension types¶
-
type PyMemberDef¶
- Part of the Stable ABI (including all members).
Structure which describes an attribute of a type which corresponds to a C struct member. When defining a class, put a NULL-terminated array of these structures in the
tp_members
slot.Its fields are, in order:
-
const char *name¶
Name of the member. A NULL value marks the end of a
PyMemberDef[]
array.The string should be static, no copy is made of it.
-
int type¶
The type of the member in the C struct. See Member types for the possible values.
-
Py_ssize_t offset¶
The offset in bytes that the member is located on the type’s object struct.
-
int flags¶
Zero or more of the Member flags, combined using bitwise OR.
-
const char *doc¶
The docstring, or NULL. The string should be static, no copy is made of it. Typically, it is defined using
PyDoc_STR
.
By default (when
flags
is0
), members allow both read and write access. Use thePy_READONLY
flag for read-only access. Certain types, likePy_T_STRING
, implyPy_READONLY
. OnlyPy_T_OBJECT_EX
(and legacyT_OBJECT
) members can be deleted.For heap-allocated types (created using
PyType_FromSpec()
or similar),PyMemberDef
may contain a definition for the special member"__vectorcalloffset__"
, corresponding totp_vectorcall_offset
in type objects. This member must be defined withPy_T_PYSSIZET
, and eitherPy_READONLY
orPy_READONLY | Py_RELATIVE_OFFSET
. For example:static PyMemberDef spam_type_members[] = { {"__vectorcalloffset__", Py_T_PYSSIZET, offsetof(Spam_object, vectorcall), Py_READONLY}, {NULL} /* Sentinel */ };
(You may need to
#include <stddef.h>
foroffsetof()
.)The legacy offsets
tp_dictoffset
andtp_weaklistoffset
can be defined similarly using"__dictoffset__"
and"__weaklistoffset__"
members, but extensions are strongly encouraged to usePy_TPFLAGS_MANAGED_DICT
andPy_TPFLAGS_MANAGED_WEAKREF
instead.Changed in version 3.12:
PyMemberDef
is always available. Previously, it required including"structmember.h"
.Changed in version 3.14:
Py_RELATIVE_OFFSET
is now allowed for"__vectorcalloffset__"
,"__dictoffset__"
and"__weaklistoffset__"
. -
const char *name¶
-
PyObject *PyMember_GetOne(const char *obj_addr, struct PyMemberDef *m)¶
- Part of the Stable ABI.
Get an attribute belonging to the object at address obj_addr. The attribute is described by
PyMemberDef
m. ReturnsNULL
on error.Changed in version 3.12:
PyMember_GetOne
is always available. Previously, it required including"structmember.h"
.
-
int PyMember_SetOne(char *obj_addr, struct PyMemberDef *m, PyObject *o)¶
- Part of the Stable ABI.
Set an attribute belonging to the object at address obj_addr to object o. The attribute to set is described by
PyMemberDef
m. Returns0
if successful and a negative value on failure.Changed in version 3.12:
PyMember_SetOne
is always available. Previously, it required including"structmember.h"
.
Member flags¶
The following flags can be used with PyMemberDef.flags
:
-
Py_READONLY¶
- Part of the Stable ABI since version 3.12.
Not writable.
-
Py_AUDIT_READ¶
- Part of the Stable ABI since version 3.12.
Emit an
object.__getattr__
audit event before reading.
-
Py_RELATIVE_OFFSET¶
- Part of the Stable ABI since version 3.12.
Indicates that the
offset
of thisPyMemberDef
entry indicates an offset from the subclass-specific data, rather than fromPyObject
.Can only be used as part of the
Py_tp_members
slot
when creating a class using negativebasicsize
. It is mandatory in that case. When settingtp_members
from the slot during class creation, Python clears the flag and setsPyMemberDef.offset
to the offset from thePyObject
struct.
Changed in version 3.10: The RESTRICTED
, READ_RESTRICTED
and
WRITE_RESTRICTED
macros available with
#include "structmember.h"
are deprecated.
READ_RESTRICTED
and RESTRICTED
are equivalent to
Py_AUDIT_READ
; WRITE_RESTRICTED
does nothing.
Changed in version 3.12: The READONLY
macro was renamed to Py_READONLY
.
The PY_AUDIT_READ
macro was renamed with the Py_
prefix.
The new names are now always available.
Previously, these required #include "structmember.h"
.
The header is still available and it provides the old names.
Member types¶
PyMemberDef.type
can be one of the following macros corresponding
to various C types.
When the member is accessed in Python, it will be converted to the
equivalent Python type.
When it is set from Python, it will be converted back to the C type.
If that is not possible, an exception such as TypeError
or
ValueError
is raised.
Unless marked (D), attributes defined this way cannot be deleted
using e.g. del
or delattr()
.
Macro name |
C type |
Python type |
|---|---|---|
|
char |
|
|
short |
|
|
int |
|
|
long |
|
|
long long |
|
|
unsigned char |
|
|
unsigned int |
|
|
unsigned short |
|
|
unsigned long |
|
|
unsigned long long |
|
|
||
|
float |
|
|
double |
|
|
char (written as 0 or 1) |
|
|
const char* (*) |
|
|
const char[] (*) |
|
|
char (0-127) |
|
|
|
(*): Zero-terminated, UTF8-encoded C string. With
Py_T_STRING
the C representation is a pointer; withPy_T_STRING_INPLACE
the string is stored directly in the structure.(**): String of length 1. Only ASCII is accepted.
(RO): Implies
Py_READONLY
.(D): Can be deleted, in which case the pointer is set to
NULL
. Reading aNULL
pointer raisesAttributeError
.
Added in version 3.12: In previous versions, the macros were only available with
#include "structmember.h"
and were named without the Py_
prefix
(e.g. as T_INT
).
The header is still available and contains the old names, along with
the following deprecated types:
-
T_OBJECT¶
Like
Py_T_OBJECT_EX
, butNULL
is converted toNone
. This results in surprising behavior in Python: deleting the attribute effectively sets it toNone
.
-
T_NONE¶
Always
None
. Must be used withPy_READONLY
.
Defining Getters and Setters¶
-
type PyGetSetDef¶
- Part of the Stable ABI (including all members).
Structure to define property-like access for a type. See also description of the
PyTypeObject.tp_getset
slot.-
const char *name¶
attribute name
-
setter set¶
Optional C function to set or delete the attribute. If
NULL
, the attribute is read-only.
-
const char *doc¶
optional docstring
-
void *closure¶
Optional user data pointer, providing additional data for getter and setter.
-
const char *name¶
-
typedef PyObject *(*getter)(PyObject*, void*)¶
- Part of the Stable ABI.
The
get
function takes one PyObject* parameter (the instance) and a user data pointer (the associatedclosure
):It should return a new reference on success or
NULL
with a set exception on failure.
-
typedef int (*setter)(PyObject*, PyObject*, void*)¶
- Part of the Stable ABI.
set
functions take two PyObject* parameters (the instance and the value to be set) and a user data pointer (the associatedclosure
):In case the attribute should be deleted the second parameter is
NULL
. Should return0
on success or-1
with a set exception on failure.


Iterator Protocol

Iterator Protocol¶
There are two functions specifically for working with iterators.
-
int PyIter_Check(PyObject *o)¶
- Part of the Stable ABI since version 3.8.
Return non-zero if the object o can be safely passed to
PyIter_NextItem()
and0
otherwise. This function always succeeds.
-
int PyAIter_Check(PyObject *o)¶
- Part of the Stable ABI since version 3.10.
Return non-zero if the object o provides the
AsyncIterator
protocol, and0
otherwise. This function always succeeds.Added in version 3.10.
-
int PyIter_NextItem(PyObject *iter, PyObject **item)¶
- Part of the Stable ABI since version 3.14.
Return
1
and set item to a strong reference of the next value of the iterator iter on success. Return0
and set item toNULL
if there are no remaining values. Return-1
, set item toNULL
and set an exception on error.Added in version 3.14.
-
PyObject *PyIter_Next(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
This is an older version of
PyIter_NextItem()
, which is retained for backwards compatibility. PreferPyIter_NextItem()
.Return the next value from the iterator o. The object must be an iterator according to
PyIter_Check()
(it is up to the caller to check this). If there are no remaining values, returnsNULL
with no exception set. If an error occurs while retrieving the item, returnsNULL
and passes along the exception.
-
type PySendResult¶
The enum value used to represent different results of
PyIter_Send()
.Added in version 3.10.
-
PySendResult PyIter_Send(PyObject *iter, PyObject *arg, PyObject **presult)¶
- Part of the Stable ABI since version 3.10.
Sends the arg value into the iterator iter. Returns:
PYGEN_RETURN
if iterator returns. Return value is returned via presult.PYGEN_NEXT
if iterator yields. Yielded value is returned via presult.PYGEN_ERROR
if iterator has raised and exception. presult is set toNULL
.
Added in version 3.10.


Python Language Services

Python Language Services¶
Python provides a number of modules to assist in working with the Python language. These modules support tokenizing, parsing, syntax analysis, bytecode disassembly, and various other facilities.
These modules include:
ast
— Abstract syntax treessymtable
— Access to the compiler’s symbol tablestoken
— Constants used with Python parse treeskeyword
— Testing for Python keywordstokenize
— Tokenizer for Python sourcetabnanny
— Detection of ambiguous indentationpyclbr
— Python module browser supportpy_compile
— Compile Python source filescompileall
— Byte-compile Python librariesdis
— Disassembler for Python bytecodepickletools
— Tools for pickle developers


PyHash API

PyHash API¶
See also the PyTypeObject.tp_hash
member and Hashing of numeric types.
-
type Py_hash_t¶
Hash value type: signed integer.
Added in version 3.2.
-
type Py_uhash_t¶
Hash value type: unsigned integer.
Added in version 3.2.
-
Py_HASH_ALGORITHM¶
A numerical value indicating the algorithm for hashing of
str
,bytes
, andmemoryview
.The algorithm name is exposed by
sys.hash_info.algorithm
.Added in version 3.4.
-
Py_HASH_FNV¶
-
Py_HASH_SIPHASH24¶
-
Py_HASH_SIPHASH13¶
Numerical values to compare to
Py_HASH_ALGORITHM
to determine which algorithm is used for hashing. The hash algorithm can be configured via the configure--with-hash-algorithm
option.Added in version 3.4: Add
Py_HASH_FNV
andPy_HASH_SIPHASH24
.Added in version 3.11: Add
Py_HASH_SIPHASH13
.
-
Py_HASH_CUTOFF¶
Buffers of length in range
[1, Py_HASH_CUTOFF)
are hashed using DJBX33A instead of the algorithm described byPy_HASH_ALGORITHM
.A
Py_HASH_CUTOFF
of 0 disables the optimization.Py_HASH_CUTOFF
must be non-negative and less or equal than 7.
32-bit platforms should use a cutoff smaller than 64-bit platforms because it is easier to create colliding strings. A cutoff of 7 on 64-bit platforms and 5 on 32-bit platforms should provide a decent safety margin.
This corresponds to the
sys.hash_info.cutoff
constant.Added in version 3.4.
-
PyHASH_MODULUS¶
The Mersenne prime
P = 2**n -1
, used for numeric hash scheme.This corresponds to the
sys.hash_info.modulus
constant.Added in version 3.13.
-
PyHASH_BITS¶
The exponent
n
ofP
inPyHASH_MODULUS
.Added in version 3.13.
-
PyHASH_MULTIPLIER¶
Prime multiplier used in string and various other hashes.
Added in version 3.13.
-
PyHASH_INF¶
The hash value returned for a positive infinity.
This corresponds to the
sys.hash_info.inf
constant.Added in version 3.13.
-
PyHASH_IMAG¶
The multiplier used for the imaginary part of a complex number.
This corresponds to the
sys.hash_info.imag
constant.Added in version 3.13.
-
type PyHash_FuncDef¶
Hash function definition used by
PyHash_GetFuncDef()
.-
Py_hash_t (*const hash)(const void*, Py_ssize_t)¶
Hash function.
-
const char *name¶
Hash function name (UTF-8 encoded string).
This corresponds to the
sys.hash_info.algorithm
constant.
-
const int hash_bits¶
Internal size of the hash value in bits.
This corresponds to the
sys.hash_info.hash_bits
constant.
-
const int seed_bits¶
Size of seed input in bits.
This corresponds to the
sys.hash_info.seed_bits
constant.
Added in version 3.4.
-
Py_hash_t (*const hash)(const void*, Py_ssize_t)¶
-
PyHash_FuncDef *PyHash_GetFuncDef(void)¶
Get the hash function definition.
See also
PEP 456 “Secure and interchangeable hash algorithm”.
Added in version 3.4.
-
Py_hash_t Py_HashPointer(const void *ptr)¶
Hash a pointer value: process the pointer value as an integer (cast it to
uintptr_t
internally). The pointer is not dereferenced.The function cannot fail: it cannot return
-1
.Added in version 3.13.
-
Py_hash_t Py_HashBuffer(const void *ptr, Py_ssize_t len)¶
Compute and return the hash value of a buffer of len bytes starting at address ptr. The hash is guaranteed to match that of
bytes
,memoryview
, and other built-in objects that implement the buffer protocol.Use this function to implement hashing for immutable objects whose
tp_richcompare
function compares to another object’s buffer.len must be greater than or equal to
0
.This function always succeeds.
Added in version 3.14.
-
Py_hash_t PyObject_GenericHash(PyObject *obj)¶
Generic hashing function that is meant to be put into a type object’s
tp_hash
slot. Its result only depends on the object’s identity.CPython implementation detail: In CPython, it is equivalent to
Py_HashPointer()
.Added in version 3.13.


Curses C API

Curses C API¶
curses
exposes a small C interface for extension modules.
Consumers must include the header file py_curses.h
(which is not
included by default by Python.h
) and import_curses()
must
be invoked, usually as part of the module initialisation function, to populate
PyCurses_API
.
Warning
Neither the C API nor the pure Python curses
module are compatible
with subinterpreters.
-
import_curses()¶
Import the curses C API. The macro does not need a semi-colon to be called.
On success, populate the
PyCurses_API
pointer.On failure, set
PyCurses_API
to NULL and set an exception. The caller must check if an error occurred viaPyErr_Occurred()
:import_curses(); // semi-colon is optional but recommended if (PyErr_Occurred()) { /* cleanup */ }
-
void **PyCurses_API¶
Dynamically allocated object containing the curses C API. This variable is only available once
import_curses
succeeds.PyCurses_API[0]
corresponds toPyCursesWindow_Type
.PyCurses_API[1]
,PyCurses_API[2]
, andPyCurses_API[3]
are pointers to predicate functions of typeint (*)(void)
.When called, these predicates return whether
curses.setupterm()
,curses.initscr()
, andcurses.start_color()
have been called respectively.See also the convenience macros
PyCursesSetupTermCalled
,PyCursesInitialised
, andPyCursesInitialisedColor
.Note
The number of entries in this structure is subject to changes. Consider using
PyCurses_API_pointers
to check if new fields are available or not.
-
PyCurses_API_pointers¶
The number of accessible fields (
4
) inPyCurses_API
. This number is incremented whenever new fields are added.
-
PyTypeObject PyCursesWindow_Type¶
The heap type corresponding to
curses.window
.
-
int PyCursesWindow_Check(PyObject *op)¶
Return true if op is a
curses.window
instance, false otherwise.
The following macros are convenience macros expanding into C statements.
In particular, they can only be used as macro;
or macro
, but not
macro()
or macro();
.
-
PyCursesSetupTermCalled¶
Macro checking if
curses.setupterm()
has been called.The macro expansion is roughly equivalent to:
{ typedef int (*predicate_t)(void); predicate_t was_setupterm_called = (predicate_t)PyCurses_API[1]; if (!was_setupterm_called()) { return NULL; } }
-
PyCursesInitialised¶
Macro checking if
curses.initscr()
has been called.The macro expansion is roughly equivalent to:
{ typedef int (*predicate_t)(void); predicate_t was_initscr_called = (predicate_t)PyCurses_API[2]; if (!was_initscr_called()) { return NULL; } }
-
PyCursesInitialisedColor¶
Macro checking if
curses.start_color()
has been called.The macro expansion is roughly equivalent to:
{ typedef int (*predicate_t)(void); predicate_t was_start_color_called = (predicate_t)PyCurses_API[3]; if (!was_start_color_called()) { return NULL; } }
Internal data¶
The following objects are exposed by the C API but should be considered internal-only.
-
PyCurses_CAPSULE_NAME¶
Name of the curses capsule to pass to
PyCapsule_Import()
.Internal usage only. Use
import_curses
instead.