MS Windows Specific Services

pickletools — Tools for pickle developers
pickletools
msvcrt — Useful routines from the MS VC++ runtime
msvcrt
This chapter describes modules that are only available on MS Windows platforms.
winreg
winsound


MemoryView objects

MemoryView objects¶
A memoryview
object exposes the C level buffer interface as a Python object which can then be passed around like
any other object.
-
PyTypeObject PyMemoryView_Type¶
- Part of the Stable ABI.
This instance of
PyTypeObject
represents the Python memoryview type. This is the same object asmemoryview
in the Python layer.
-
PyObject *PyMemoryView_FromObject(PyObject *obj)¶
- Return value: New reference. Part of the Stable ABI.
Create a memoryview object from an object that provides the buffer interface. If obj supports writable buffer exports, the memoryview object will be read/write, otherwise it may be either read-only or read/write at the discretion of the exporter.
-
PyBUF_READ¶
- Part of the Stable ABI since version 3.11.
Flag to request a readonly buffer.
-
PyBUF_WRITE¶
- Part of the Stable ABI since version 3.11.
Flag to request a writable buffer.
-
PyObject *PyMemoryView_FromMemory(char *mem, Py_ssize_t size, int flags)¶
- Return value: New reference. Part of the Stable ABI since version 3.7.
Create a memoryview object using mem as the underlying buffer. flags can be one of
PyBUF_READ
orPyBUF_WRITE
.Added in version 3.3.
-
PyObject *PyMemoryView_FromBuffer(const Py_buffer *view)¶
- Return value: New reference. Part of the Stable ABI since version 3.11.
Create a memoryview object wrapping the given buffer structure view. For simple byte buffers,
PyMemoryView_FromMemory()
is the preferred function.
-
PyObject *PyMemoryView_GetContiguous(PyObject *obj, int buffertype, char order)¶
- Return value: New reference. Part of the Stable ABI.
Create a memoryview object to a contiguous chunk of memory (in either ‘C’ or ‘F’ortran order) from an object that defines the buffer interface. If memory is contiguous, the memoryview object points to the original memory. Otherwise, a copy is made and the memoryview points to a new bytes object.
buffertype can be one of
PyBUF_READ
orPyBUF_WRITE
.
-
int PyMemoryView_Check(PyObject *obj)¶
Return true if the object obj is a memoryview object. It is not currently allowed to create subclasses of
memoryview
. This function always succeeds.
-
Py_buffer *PyMemoryView_GET_BUFFER(PyObject *mview)¶
Return a pointer to the memoryview’s private copy of the exporter’s buffer. mview must be a memoryview instance; this macro doesn’t check its type, you must do it yourself or you will risk crashes.
-
PyObject *PyMemoryView_GET_BASE(PyObject *mview)¶
Return either a pointer to the exporting object that the memoryview is based on or
NULL
if the memoryview has been created by one of the functionsPyMemoryView_FromMemory()
orPyMemoryView_FromBuffer()
. mview must be a memoryview instance.


Policies

Policies¶
Warning
Policies are deprecated and will be removed in Python 3.16.
Users are encouraged to use the asyncio.run()
function
or the asyncio.Runner
with loop_factory to use
the desired loop implementation.
An event loop policy is a global object used to get and set the current event loop, as well as create new event loops. The default policy can be replaced with built-in alternatives to use different event loop implementations, or substituted by a custom policy that can override these behaviors.
The policy object gets and sets a separate event loop per context. This is per-thread by default, though custom policies could define context differently.
Custom event loop policies can control the behavior of
get_event_loop()
, set_event_loop()
, and new_event_loop()
.
Policy objects should implement the APIs defined
in the AbstractEventLoopPolicy
abstract base class.
Getting and Setting the Policy¶
The following functions can be used to get and set the policy for the current process:
- asyncio.get_event_loop_policy()¶
Return the current process-wide policy.
Deprecated since version 3.14: The
get_event_loop_policy()
function is deprecated and will be removed in Python 3.16.
- asyncio.set_event_loop_policy(policy)¶
Set the current process-wide policy to policy.
If policy is set to
None
, the default policy is restored.Deprecated since version 3.14: The
set_event_loop_policy()
function is deprecated and will be removed in Python 3.16.
Policy Objects¶
The abstract event loop policy base class is defined as follows:
- class asyncio.AbstractEventLoopPolicy¶
An abstract base class for asyncio policies.
- get_event_loop()¶
Get the event loop for the current context.
Return an event loop object implementing the
AbstractEventLoop
interface.This method should never return
None
.Changed in version 3.6.
- set_event_loop(loop)¶
Set the event loop for the current context to loop.
- new_event_loop()¶
Create and return a new event loop object.
This method should never return
None
.
Deprecated since version 3.14: The
AbstractEventLoopPolicy
class is deprecated and will be removed in Python 3.16.
asyncio ships with the following built-in policies:
- class asyncio.DefaultEventLoopPolicy¶
The default asyncio policy. Uses
SelectorEventLoop
on Unix andProactorEventLoop
on Windows.There is no need to install the default policy manually. asyncio is configured to use the default policy automatically.
Changed in version 3.8: On Windows,
ProactorEventLoop
is now used by default.Changed in version 3.14: The
get_event_loop()
method of the default asyncio policy now raises aRuntimeError
if there is no set event loop.Deprecated since version 3.14: The
DefaultEventLoopPolicy
class is deprecated and will be removed in Python 3.16.
- class asyncio.WindowsSelectorEventLoopPolicy¶
An alternative event loop policy that uses the
SelectorEventLoop
event loop implementation.Availability: Windows.
Deprecated since version 3.14: The
WindowsSelectorEventLoopPolicy
class is deprecated and will be removed in Python 3.16.
- class asyncio.WindowsProactorEventLoopPolicy¶
An alternative event loop policy that uses the
ProactorEventLoop
event loop implementation.Availability: Windows.
Deprecated since version 3.14: The
WindowsProactorEventLoopPolicy
class is deprecated and will be removed in Python 3.16.
Custom Policies¶
To implement a new event loop policy, it is recommended to subclass
DefaultEventLoopPolicy
and override the methods for which
custom behavior is wanted, e.g.:
class MyEventLoopPolicy(asyncio.DefaultEventLoopPolicy):
def get_event_loop(self):
"""Get the event loop.
This may be None or an instance of EventLoop.
"""
loop = super().get_event_loop()
# Do something with loop ...
return loop
asyncio.set_event_loop_policy(MyEventLoopPolicy())


— Determine the type of an image

imghdr
— Determine the type of an image¶
Deprecated since version 3.11, removed in version 3.13.
This module is no longer part of the Python standard library. It was removed in Python 3.13 after being deprecated in Python 3.11. The removal was decided in PEP 594.
Possible replacements are third-party libraries from PyPI: filetype, puremagic, or python-magic. These are not supported or maintained by the Python core team.
The last version of Python that provided the imghdr
module was
Python 3.12.


Context Variables Objects

Context Variables Objects¶
Added in version 3.7.
Changed in version 3.7.1:
Note
In Python 3.7.1 the signatures of all context variables
C APIs were changed to use PyObject
pointers instead
of PyContext
, PyContextVar
, and
PyContextToken
, e.g.:
// in 3.7.0:
PyContext *PyContext_New(void);
// in 3.7.1+:
PyObject *PyContext_New(void);
See bpo-34762 for more details.
This section details the public C API for the contextvars
module.
-
type PyContext¶
The C structure used to represent a
contextvars.Context
object.
-
type PyContextVar¶
The C structure used to represent a
contextvars.ContextVar
object.
-
type PyContextToken¶
The C structure used to represent a
contextvars.Token
object.
-
PyTypeObject PyContext_Type¶
The type object representing the context type.
-
PyTypeObject PyContextVar_Type¶
The type object representing the context variable type.
-
PyTypeObject PyContextToken_Type¶
The type object representing the context variable token type.
Type-check macros:
-
int PyContext_CheckExact(PyObject *o)¶
Return true if o is of type
PyContext_Type
. o must not beNULL
. This function always succeeds.
-
int PyContextVar_CheckExact(PyObject *o)¶
Return true if o is of type
PyContextVar_Type
. o must not beNULL
. This function always succeeds.
-
int PyContextToken_CheckExact(PyObject *o)¶
Return true if o is of type
PyContextToken_Type
. o must not beNULL
. This function always succeeds.
Context object management functions:
-
PyObject *PyContext_New(void)¶
- Return value: New reference.
Create a new empty context object. Returns
NULL
if an error has occurred.
-
PyObject *PyContext_Copy(PyObject *ctx)¶
- Return value: New reference.
Create a shallow copy of the passed ctx context object. Returns
NULL
if an error has occurred.
-
PyObject *PyContext_CopyCurrent(void)¶
- Return value: New reference.
Create a shallow copy of the current thread context. Returns
NULL
if an error has occurred.
-
int PyContext_Enter(PyObject *ctx)¶
Set ctx as the current context for the current thread. Returns
0
on success, and-1
on error.
-
int PyContext_Exit(PyObject *ctx)¶
Deactivate the ctx context and restore the previous context as the current context for the current thread. Returns
0
on success, and-1
on error.
-
int PyContext_AddWatcher(PyContext_WatchCallback callback)¶
Register callback as a context object watcher for the current interpreter. Return an ID which may be passed to
PyContext_ClearWatcher()
. In case of error (e.g. no more watcher IDs available), return-1
and set an exception.Added in version 3.14.
-
int PyContext_ClearWatcher(int watcher_id)¶
Clear watcher identified by watcher_id previously returned from
PyContext_AddWatcher()
for the current interpreter. Return0
on success, or-1
and set an exception on error (e.g. if the given watcher_id was never registered.)Added in version 3.14.
-
type PyContextEvent¶
Enumeration of possible context object watcher events:
Py_CONTEXT_SWITCHED
: The current context has switched to a different context. The object passed to the watch callback is the now-currentcontextvars.Context
object, or None if no context is current.
Added in version 3.14.
-
typedef int (*PyContext_WatchCallback)(PyContextEvent event, PyObject *obj)¶
Context object watcher callback function. The object passed to the callback is event-specific; see
PyContextEvent
for details.If the callback returns with an exception set, it must return
-1
; this exception will be printed as an unraisable exception usingPyErr_FormatUnraisable()
. Otherwise it should return0
.There may already be a pending exception set on entry to the callback. In this case, the callback should return
0
with the same exception still set. This means the callback may not call any other API that can set an exception unless it saves and clears the exception state first, and restores it before returning.Added in version 3.14.
Context variable functions:
-
PyObject *PyContextVar_New(const char *name, PyObject *def)¶
- Return value: New reference.
Create a new
ContextVar
object. The name parameter is used for introspection and debug purposes. The def parameter specifies a default value for the context variable, orNULL
for no default. If an error has occurred, this function returnsNULL
.
-
int PyContextVar_Get(PyObject *var, PyObject *default_value, PyObject **value)¶
Get the value of a context variable. Returns
-1
if an error has occurred during lookup, and0
if no error occurred, whether or not a value was found.If the context variable was found, value will be a pointer to it. If the context variable was not found, value will point to:
default_value, if not
NULL
;the default value of var, if not
NULL
;NULL
Except for
NULL
, the function returns a new reference.
-
PyObject *PyContextVar_Set(PyObject *var, PyObject *value)¶
- Return value: New reference.
Set the value of var to value in the current context. Returns a new token object for this change, or
NULL
if an error has occurred.
-
int PyContextVar_Reset(PyObject *var, PyObject *token)¶
Reset the state of the var context variable to that it was in before
PyContextVar_Set()
that returned the token was called. This function returns0
on success and-1
on error.


Modules

6. Modules¶
If you quit from the Python interpreter and enter it again, the definitions you have made (functions and variables) are lost. Therefore, if you want to write a somewhat longer program, you are better off using a text editor to prepare the input for the interpreter and running it with that file as input instead. This is known as creating a script. As your program gets longer, you may want to split it into several files for easier maintenance. You may also want to use a handy function that you’ve written in several programs without copying its definition into each program.
To support this, Python has a way to put definitions in a file and use them in a script or in an interactive instance of the interpreter. Such a file is called a module; definitions from a module can be imported into other modules or into the main module (the collection of variables that you have access to in a script executed at the top level and in calculator mode).
A module is a file containing Python definitions and statements. The file name
is the module name with the suffix .py
appended. Within a module, the
module’s name (as a string) is available as the value of the global variable
__name__
. For instance, use your favorite text editor to create a file
called fibo.py
in the current directory with the following contents:
# Fibonacci numbers module
def fib(n):
"""Write Fibonacci series up to n."""
a, b = 0, 1
while a < n:
print(a, end=' ')
a, b = b, a+b
print()
def fib2(n):
"""Return Fibonacci series up to n."""
result = []
a, b = 0, 1
while a < n:
result.append(a)
a, b = b, a+b
return result
Now enter the Python interpreter and import this module with the following command:
>>> import fibo
This does not add the names of the functions defined in fibo
directly to
the current namespace (see Python Scopes and Namespaces for more details);
it only adds the module name fibo
there. Using
the module name you can access the functions:
>>> fibo.fib(1000)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
>>> fibo.fib2(100)
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
>>> fibo.__name__
'fibo'
If you intend to use a function often you can assign it to a local name:
>>> fib = fibo.fib
>>> fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
6.1. More on Modules¶
A module can contain executable statements as well as function definitions. These statements are intended to initialize the module. They are executed only the first time the module name is encountered in an import statement. [1] (They are also run if the file is executed as a script.)
Each module has its own private namespace, which is used as the global namespace
by all functions defined in the module. Thus, the author of a module can
use global variables in the module without worrying about accidental clashes
with a user’s global variables. On the other hand, if you know what you are
doing you can touch a module’s global variables with the same notation used to
refer to its functions, modname.itemname
.
Modules can import other modules. It is customary but not required to place all
import
statements at the beginning of a module (or script, for that
matter). The imported module names, if placed at the top level of a module
(outside any functions or classes), are added to the module’s global namespace.
There is a variant of the import
statement that imports names from a
module directly into the importing module’s namespace. For example:
>>> from fibo import fib, fib2
>>> fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
This does not introduce the module name from which the imports are taken in the
local namespace (so in the example, fibo
is not defined).
There is even a variant to import all names that a module defines:
>>> from fibo import *
>>> fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
This imports all names except those beginning with an underscore (_
).
In most cases Python programmers do not use this facility since it introduces
an unknown set of names into the interpreter, possibly hiding some things
you have already defined.
Note that in general the practice of importing *
from a module or package is
frowned upon, since it often causes poorly readable code. However, it is okay to
use it to save typing in interactive sessions.
If the module name is followed by as
, then the name
following as
is bound directly to the imported module.
>>> import fibo as fib
>>> fib.fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
This is effectively importing the module in the same way that import fibo
will do, with the only difference of it being available as fib
.
It can also be used when utilising from
with similar effects:
>>> from fibo import fib as fibonacci
>>> fibonacci(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
Note
For efficiency reasons, each module is only imported once per interpreter
session. Therefore, if you change your modules, you must restart the
interpreter – or, if it’s just one module you want to test interactively,
use importlib.reload()
, e.g. import importlib;
importlib.reload(modulename)
.
6.1.1. Executing modules as scripts¶
When you run a Python module with
python fibo.py <arguments>
the code in the module will be executed, just as if you imported it, but with
the __name__
set to "__main__"
. That means that by adding this code at
the end of your module:
if __name__ == "__main__":
import sys
fib(int(sys.argv[1]))
you can make the file usable as a script as well as an importable module, because the code that parses the command line only runs if the module is executed as the “main” file:
$ python fibo.py 50
0 1 1 2 3 5 8 13 21 34
If the module is imported, the code is not run:
>>> import fibo
>>>
This is often used either to provide a convenient user interface to a module, or for testing purposes (running the module as a script executes a test suite).
6.1.2. The Module Search Path¶
When a module named spam
is imported, the interpreter first searches for
a built-in module with that name. These module names are listed in
sys.builtin_module_names
. If not found, it then searches for a file
named spam.py
in a list of directories given by the variable
sys.path
. sys.path
is initialized from these locations:
The directory containing the input script (or the current directory when no file is specified).
PYTHONPATH
(a list of directory names, with the same syntax as the shell variablePATH
).The installation-dependent default (by convention including a
site-packages
directory, handled by thesite
module).
More details are at The initialization of the sys.path module search path.
Note
On file systems which support symlinks, the directory containing the input script is calculated after the symlink is followed. In other words the directory containing the symlink is not added to the module search path.
After initialization, Python programs can modify sys.path
. The
directory containing the script being run is placed at the beginning of the
search path, ahead of the standard library path. This means that scripts in that
directory will be loaded instead of modules of the same name in the library
directory. This is an error unless the replacement is intended. See section
Standard Modules for more information.
6.1.3. “Compiled” Python files¶
To speed up loading modules, Python caches the compiled version of each module
in the __pycache__
directory under the name module.version.pyc
,
where the version encodes the format of the compiled file; it generally contains
the Python version number. For example, in CPython release 3.3 the compiled
version of spam.py would be cached as __pycache__/spam.cpython-33.pyc
. This
naming convention allows compiled modules from different releases and different
versions of Python to coexist.
Python checks the modification date of the source against the compiled version to see if it’s out of date and needs to be recompiled. This is a completely automatic process. Also, the compiled modules are platform-independent, so the same library can be shared among systems with different architectures.
Python does not check the cache in two circumstances. First, it always recompiles and does not store the result for the module that’s loaded directly from the command line. Second, it does not check the cache if there is no source module. To support a non-source (compiled only) distribution, the compiled module must be in the source directory, and there must not be a source module.
Some tips for experts:
You can use the
-O
or-OO
switches on the Python command to reduce the size of a compiled module. The-O
switch removes assert statements, the-OO
switch removes both assert statements and __doc__ strings. Since some programs may rely on having these available, you should only use this option if you know what you’re doing. “Optimized” modules have anopt-
tag and are usually smaller. Future releases may change the effects of optimization.A program doesn’t run any faster when it is read from a
.pyc
file than when it is read from a.py
file; the only thing that’s faster about.pyc
files is the speed with which they are loaded.The module
compileall
can create .pyc files for all modules in a directory.There is more detail on this process, including a flow chart of the decisions, in PEP 3147.
6.2. Standard Modules¶
Python comes with a library of standard modules, described in a separate
document, the Python Library Reference (“Library Reference” hereafter). Some
modules are built into the interpreter; these provide access to operations that
are not part of the core of the language but are nevertheless built in, either
for efficiency or to provide access to operating system primitives such as
system calls. The set of such modules is a configuration option which also
depends on the underlying platform. For example, the winreg
module is only
provided on Windows systems. One particular module deserves some attention:
sys
, which is built into every Python interpreter. The variables
sys.ps1
and sys.ps2
define the strings used as primary and secondary
prompts:
>>> import sys
>>> sys.ps1
'>>> '
>>> sys.ps2
'... '
>>> sys.ps1 = 'C> '
C> print('Yuck!')
Yuck!
C>
These two variables are only defined if the interpreter is in interactive mode.
The variable sys.path
is a list of strings that determines the interpreter’s
search path for modules. It is initialized to a default path taken from the
environment variable PYTHONPATH
, or from a built-in default if
PYTHONPATH
is not set. You can modify it using standard list
operations:
>>> import sys
>>> sys.path.append('/ufs/guido/lib/python')
6.3. The dir()
Function¶
The built-in function dir()
is used to find out which names a module
defines. It returns a sorted list of strings:
>>> import fibo, sys
>>> dir(fibo)
['__name__', 'fib', 'fib2']
>>> dir(sys)
['__breakpointhook__', '__displayhook__', '__doc__', '__excepthook__',
'__interactivehook__', '__loader__', '__name__', '__package__', '__spec__',
'__stderr__', '__stdin__', '__stdout__', '__unraisablehook__',
'_clear_type_cache', '_current_frames', '_debugmallocstats', '_framework',
'_getframe', '_git', '_home', '_xoptions', 'abiflags', 'addaudithook',
'api_version', 'argv', 'audit', 'base_exec_prefix', 'base_prefix',
'breakpointhook', 'builtin_module_names', 'byteorder', 'call_tracing',
'callstats', 'copyright', 'displayhook', 'dont_write_bytecode', 'exc_info',
'excepthook', 'exec_prefix', 'executable', 'exit', 'flags', 'float_info',
'float_repr_style', 'get_asyncgen_hooks', 'get_coroutine_origin_tracking_depth',
'getallocatedblocks', 'getdefaultencoding', 'getdlopenflags',
'getfilesystemencodeerrors', 'getfilesystemencoding', 'getprofile',
'getrecursionlimit', 'getrefcount', 'getsizeof', 'getswitchinterval',
'gettrace', 'hash_info', 'hexversion', 'implementation', 'int_info',
'intern', 'is_finalizing', 'last_traceback', 'last_type', 'last_value',
'maxsize', 'maxunicode', 'meta_path', 'modules', 'path', 'path_hooks',
'path_importer_cache', 'platform', 'prefix', 'ps1', 'ps2', 'pycache_prefix',
'set_asyncgen_hooks', 'set_coroutine_origin_tracking_depth', 'setdlopenflags',
'setprofile', 'setrecursionlimit', 'setswitchinterval', 'settrace', 'stderr',
'stdin', 'stdout', 'thread_info', 'unraisablehook', 'version', 'version_info',
'warnoptions']
Without arguments, dir()
lists the names you have defined currently:
>>> a = [1, 2, 3, 4, 5]
>>> import fibo
>>> fib = fibo.fib
>>> dir()
['__builtins__', '__name__', 'a', 'fib', 'fibo', 'sys']
Note that it lists all types of names: variables, modules, functions, etc.
dir()
does not list the names of built-in functions and variables. If you
want a list of those, they are defined in the standard module
builtins
:
>>> import builtins
>>> dir(builtins)
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException',
'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning',
'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError',
'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning',
'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False',
'FileExistsError', 'FileNotFoundError', 'FloatingPointError',
'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError',
'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError',
'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError',
'MemoryError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented',
'NotImplementedError', 'OSError', 'OverflowError',
'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError',
'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning',
'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError',
'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError',
'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError',
'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning',
'ValueError', 'Warning', 'ZeroDivisionError', '_', '__build_class__',
'__debug__', '__doc__', '__import__', '__name__', '__package__', 'abs',
'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable',
'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits',
'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit',
'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr',
'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass',
'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview',
'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property',
'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice',
'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars',
'zip']
6.4. Packages¶
Packages are a way of structuring Python’s module namespace by using “dotted
module names”. For example, the module name A.B
designates a submodule
named B
in a package named A
. Just like the use of modules saves the
authors of different modules from having to worry about each other’s global
variable names, the use of dotted module names saves the authors of multi-module
packages like NumPy or Pillow from having to worry about
each other’s module names.
Suppose you want to design a collection of modules (a “package”) for the uniform
handling of sound files and sound data. There are many different sound file
formats (usually recognized by their extension, for example: .wav
,
.aiff
, .au
), so you may need to create and maintain a growing
collection of modules for the conversion between the various file formats.
There are also many different operations you might want to perform on sound data
(such as mixing, adding echo, applying an equalizer function, creating an
artificial stereo effect), so in addition you will be writing a never-ending
stream of modules to perform these operations. Here’s a possible structure for
your package (expressed in terms of a hierarchical filesystem):
sound/ Top-level package
__init__.py Initialize the sound package
formats/ Subpackage for file format conversions
__init__.py
wavread.py
wavwrite.py
aiffread.py
aiffwrite.py
auread.py
auwrite.py
...
effects/ Subpackage for sound effects
__init__.py
echo.py
surround.py
reverse.py
...
filters/ Subpackage for filters
__init__.py
equalizer.py
vocoder.py
karaoke.py
...
When importing the package, Python searches through the directories on
sys.path
looking for the package subdirectory.
The __init__.py
files are required to make Python treat directories
containing the file as packages (unless using a namespace package, a
relatively advanced feature). This prevents directories with a common name,
such as string
, from unintentionally hiding valid modules that occur later
on the module search path. In the simplest case, __init__.py
can just be
an empty file, but it can also execute initialization code for the package or
set the __all__
variable, described later.
Users of the package can import individual modules from the package, for example:
import sound.effects.echo
This loads the submodule sound.effects.echo
. It must be referenced with
its full name.
sound.effects.echo.echofilter(input, output, delay=0.7, atten=4)
An alternative way of importing the submodule is:
from sound.effects import echo
This also loads the submodule echo
, and makes it available without its
package prefix, so it can be used as follows:
echo.echofilter(input, output, delay=0.7, atten=4)
Yet another variation is to import the desired function or variable directly:
from sound.effects.echo import echofilter
Again, this loads the submodule echo
, but this makes its function
echofilter()
directly available:
echofilter(input, output, delay=0.7, atten=4)
Note that when using from package import item
, the item can be either a
submodule (or subpackage) of the package, or some other name defined in the
package, like a function, class or variable. The import
statement first
tests whether the item is defined in the package; if not, it assumes it is a
module and attempts to load it. If it fails to find it, an ImportError
exception is raised.
Contrarily, when using syntax like import item.subitem.subsubitem
, each item
except for the last must be a package; the last item can be a module or a
package but can’t be a class or function or variable defined in the previous
item.
6.4.1. Importing * From a Package¶
Now what happens when the user writes from sound.effects import *
? Ideally,
one would hope that this somehow goes out to the filesystem, finds which
submodules are present in the package, and imports them all. This could take a
long time and importing sub-modules might have unwanted side-effects that should
only happen when the sub-module is explicitly imported.
The only solution is for the package author to provide an explicit index of the
package. The import
statement uses the following convention: if a package’s
__init__.py
code defines a list named __all__
, it is taken to be the
list of module names that should be imported when from package import *
is
encountered. It is up to the package author to keep this list up-to-date when a
new version of the package is released. Package authors may also decide not to
support it, if they don’t see a use for importing * from their package. For
example, the file sound/effects/__init__.py
could contain the following
code:
__all__ = ["echo", "surround", "reverse"]
This would mean that from sound.effects import *
would import the three
named submodules of the sound.effects
package.
Be aware that submodules might become shadowed by locally defined names. For
example, if you added a reverse
function to the
sound/effects/__init__.py
file, the from sound.effects import *
would only import the two submodules echo
and surround
, but not the
reverse
submodule, because it is shadowed by the locally defined
reverse
function:
__all__ = [
"echo", # refers to the 'echo.py' file
"surround", # refers to the 'surround.py' file
"reverse", # !!! refers to the 'reverse' function now !!!
]
def reverse(msg: str): # <-- this name shadows the 'reverse.py' submodule
return msg[::-1] # in the case of a 'from sound.effects import *'
If __all__
is not defined, the statement from sound.effects import *
does not import all submodules from the package sound.effects
into the
current namespace; it only ensures that the package sound.effects
has
been imported (possibly running any initialization code in __init__.py
)
and then imports whatever names are defined in the package. This includes any
names defined (and submodules explicitly loaded) by __init__.py
. It
also includes any submodules of the package that were explicitly loaded by
previous import
statements. Consider this code:
import sound.effects.echo
import sound.effects.surround
from sound.effects import *
In this example, the echo
and surround
modules are imported in the
current namespace because they are defined in the sound.effects
package
when the from...import
statement is executed. (This also works when
__all__
is defined.)
Although certain modules are designed to export only names that follow certain
patterns when you use import *
, it is still considered bad practice in
production code.
Remember, there is nothing wrong with using from package import
specific_submodule
! In fact, this is the recommended notation unless the
importing module needs to use submodules with the same name from different
packages.
6.4.2. Intra-package References¶
When packages are structured into subpackages (as with the sound
package
in the example), you can use absolute imports to refer to submodules of siblings
packages. For example, if the module sound.filters.vocoder
needs to use
the echo
module in the sound.effects
package, it can use from
sound.effects import echo
.
You can also write relative imports, with the from module import name
form
of import statement. These imports use leading dots to indicate the current and
parent packages involved in the relative import. From the surround
module for example, you might use:
from . import echo
from .. import formats
from ..filters import equalizer
Note that relative imports are based on the name of the current module’s package. Since the main module does not have a package, modules intended for use as the main module of a Python application must always use absolute imports.
6.4.3. Packages in Multiple Directories¶
Packages support one more special attribute, __path__
. This is
initialized to be a sequence of strings containing the name of the
directory holding the
package’s __init__.py
before the code in that file is executed. This
variable can be modified; doing so affects future searches for modules and
subpackages contained in the package.
While this feature is not often needed, it can be used to extend the set of modules found in a package.
Footnotes


— Compile Python source files

py_compile
— Compile Python source files¶
Source code: Lib/py_compile.py
The py_compile
module provides a function to generate a byte-code file
from a source file, and another function used when the module source file is
invoked as a script.
Though not often needed, this function can be useful when installing modules for shared use, especially if some of the users may not have permission to write the byte-code cache files in the directory containing the source code.
- exception py_compile.PyCompileError¶
Exception raised when an error occurs while attempting to compile the file.
- py_compile.compile(file, cfile=None, dfile=None, doraise=False, optimize=-1, invalidation_mode=PycInvalidationMode.TIMESTAMP, quiet=0)¶
Compile a source file to byte-code and write out the byte-code cache file. The source code is loaded from the file named file. The byte-code is written to cfile, which defaults to the PEP 3147/PEP 488 path, ending in
.pyc
. For example, if file is/foo/bar/baz.py
cfile will default to/foo/bar/__pycache__/baz.cpython-32.pyc
for Python 3.2. If dfile is specified, it is used instead of file as the name of the source file from which source lines are obtained for display in exception tracebacks. If doraise is true, aPyCompileError
is raised when an error is encountered while compiling file. If doraise is false (the default), an error string is written tosys.stderr
, but no exception is raised. This function returns the path to byte-compiled file, i.e. whatever cfile value was used.The doraise and quiet arguments determine how errors are handled while compiling file. If quiet is 0 or 1, and doraise is false, the default behaviour is enabled: an error string is written to
sys.stderr
, and the function returnsNone
instead of a path. If doraise is true, aPyCompileError
is raised instead. However if quiet is 2, no message is written, and doraise has no effect.If the path that cfile becomes (either explicitly specified or computed) is a symlink or non-regular file,
FileExistsError
will be raised. This is to act as a warning that import will turn those paths into regular files if it is allowed to write byte-compiled files to those paths. This is a side-effect of import using file renaming to place the final byte-compiled file into place to prevent concurrent file writing issues.optimize controls the optimization level and is passed to the built-in
compile()
function. The default of-1
selects the optimization level of the current interpreter.invalidation_mode should be a member of the
PycInvalidationMode
enum and controls how the generated bytecode cache is invalidated at runtime. The default isPycInvalidationMode.CHECKED_HASH
if theSOURCE_DATE_EPOCH
environment variable is set, otherwise the default isPycInvalidationMode.TIMESTAMP
.Changed in version 3.2: Changed default value of cfile to be PEP 3147-compliant. Previous default was file +
'c'
('o'
if optimization was enabled). Also added the optimize parameter.Changed in version 3.4: Changed code to use
importlib
for the byte-code cache file writing. This means file creation/writing semantics now match whatimportlib
does, e.g. permissions, write-and-move semantics, etc. Also added the caveat thatFileExistsError
is raised if cfile is a symlink or non-regular file.Changed in version 3.7: The invalidation_mode parameter was added as specified in PEP 552. If the
SOURCE_DATE_EPOCH
environment variable is set, invalidation_mode will be forced toPycInvalidationMode.CHECKED_HASH
.Changed in version 3.7.2: The
SOURCE_DATE_EPOCH
environment variable no longer overrides the value of the invalidation_mode argument, and determines its default value instead.Changed in version 3.8: The quiet parameter was added.
- class py_compile.PycInvalidationMode¶
An enumeration of possible methods the interpreter can use to determine whether a bytecode file is up to date with a source file. The
.pyc
file indicates the desired invalidation mode in its header. See Cached bytecode invalidation for more information on how Python invalidates.pyc
files at runtime.Added in version 3.7.
- TIMESTAMP¶
The
.pyc
file includes the timestamp and size of the source file, which Python will compare against the metadata of the source file at runtime to determine if the.pyc
file needs to be regenerated.
- CHECKED_HASH¶
The
.pyc
file includes a hash of the source file content, which Python will compare against the source at runtime to determine if the.pyc
file needs to be regenerated.
- UNCHECKED_HASH¶
Like
CHECKED_HASH
, the.pyc
file includes a hash of the source file content. However, Python will at runtime assume the.pyc
file is up to date and not validate the.pyc
against the source file at all.This option is useful when the
.pycs
are kept up to date by some system external to Python like a build system.
Command-Line Interface¶
This module can be invoked as a script to compile several source files. The files named in filenames are compiled and the resulting bytecode is cached in the normal manner. This program does not search a directory structure to locate source files; it only compiles files named explicitly. The exit status is nonzero if one of the files could not be compiled.
- <file> ... <fileN>¶
- -¶
Positional arguments are files to compile. If
-
is the only parameter, the list of files is taken from standard input.
- -q, --quiet¶
Suppress errors output.
Changed in version 3.2: Added support for -
.
Changed in version 3.10: Added support for -q
.
See also
- Module
compileall
Utilities to compile all Python source files in a directory tree.