What’s New in Python

What’s New in Python¶
The “What’s New in Python” series of essays takes tours through the most important changes between major Python versions. They are a “must read” for anyone wishing to stay up-to-date after a new release.
- What’s new in Python 3.14
- What’s New In Python 3.13
- What’s New In Python 3.12
- What’s New In Python 3.11
- Summary – Release highlights
- New Features
- New Features Related to Type Hints
- Other Language Changes
- Other CPython Implementation Changes
- New Modules
- Improved Modules
- Optimizations
- Faster CPython
- CPython bytecode changes
- Deprecated
- Pending Removal in Python 3.12
- Removed
- Porting to Python 3.11
- Build Changes
- C API Changes
- Notable changes in 3.11.4
- Notable changes in 3.11.5
- What’s New In Python 3.10
- Summary – Release highlights
- New Features
- New Features Related to Type Hints
- Other Language Changes
- New Modules
- Improved Modules
- Optimizations
- Deprecated
- Removed
- Porting to Python 3.10
- CPython bytecode changes
- Build Changes
- C API Changes
- Notable security feature in 3.10.7
- Notable security feature in 3.10.8
- Notable changes in 3.10.12
- What’s New In Python 3.9
- Summary – Release highlights
- You should check for DeprecationWarning in your code
- New Features
- Other Language Changes
- New Modules
- Improved Modules
- Optimizations
- Deprecated
- Removed
- Porting to Python 3.9
- Build Changes
- C API Changes
- Notable changes in Python 3.9.1
- Notable changes in Python 3.9.2
- Notable changes in Python 3.9.3
- Notable changes in Python 3.9.5
- Notable security feature in 3.9.14
- Notable changes in 3.9.17
- What’s New In Python 3.8
- Summary – Release highlights
- New Features
- Other Language Changes
- New Modules
- Improved Modules
- Optimizations
- Build and C API Changes
- Deprecated
- API and Feature Removals
- Porting to Python 3.8
- Notable changes in Python 3.8.1
- Notable changes in Python 3.8.2
- Notable changes in Python 3.8.3
- Notable changes in Python 3.8.8
- Notable changes in Python 3.8.9
- Notable changes in Python 3.8.10
- Notable changes in Python 3.8.10
- Notable changes in Python 3.8.12
- Notable security feature in 3.8.14
- Notable changes in 3.8.17
- What’s New In Python 3.7
- Summary – Release Highlights
- New Features
- Other Language Changes
- New Modules
- Improved Modules
- C API Changes
- Build Changes
- Optimizations
- Other CPython Implementation Changes
- Deprecated Python Behavior
- Deprecated Python modules, functions and methods
- Deprecated functions and types of the C API
- Platform Support Removals
- API and Feature Removals
- Module Removals
- Windows-only Changes
- Porting to Python 3.7
- Notable changes in Python 3.7.1
- Notable changes in Python 3.7.2
- Notable changes in Python 3.7.6
- Notable changes in Python 3.7.10
- Notable changes in Python 3.7.11
- Notable security feature in 3.7.14
- What’s New In Python 3.6
- Summary – Release highlights
- New Features
- Other Language Changes
- New Modules
- Improved Modules
- Optimizations
- Build and C API Changes
- Other Improvements
- Deprecated
- Removed
- Porting to Python 3.6
- Notable changes in Python 3.6.2
- Notable changes in Python 3.6.4
- Notable changes in Python 3.6.5
- Notable changes in Python 3.6.7
- Notable changes in Python 3.6.10
- Notable changes in Python 3.6.13
- Notable changes in Python 3.6.14
- What’s New In Python 3.5
- What’s New In Python 3.4
- What’s New In Python 3.3
- Summary – Release highlights
- PEP 405: Virtual Environments
- PEP 420: Implicit Namespace Packages
- PEP 3118: New memoryview implementation and buffer protocol documentation
- PEP 393: Flexible String Representation
- PEP 397: Python Launcher for Windows
- PEP 3151: Reworking the OS and IO exception hierarchy
- PEP 380: Syntax for Delegating to a Subgenerator
- PEP 409: Suppressing exception context
- PEP 414: Explicit Unicode literals
- PEP 3155: Qualified name for classes and functions
- PEP 412: Key-Sharing Dictionary
- PEP 362: Function Signature Object
- PEP 421: Adding sys.implementation
- Using importlib as the Implementation of Import
- Other Language Changes
- A Finer-Grained Import Lock
- Builtin functions and types
- New Modules
- Improved Modules
- Optimizations
- Build and C API Changes
- Deprecated
- Porting to Python 3.3
- What’s New In Python 3.2
- PEP 384: Defining a Stable ABI
- PEP 389: Argparse Command Line Parsing Module
- PEP 391: Dictionary Based Configuration for Logging
- PEP 3148: The
concurrent.futures
module - PEP 3147: PYC Repository Directories
- PEP 3149: ABI Version Tagged .so Files
- PEP 3333: Python Web Server Gateway Interface v1.0.1
- Other Language Changes
- New, Improved, and Deprecated Modules
- Multi-threading
- Optimizations
- Unicode
- Codecs
- Documentation
- IDLE
- Code Repository
- Build and C API Changes
- Porting to Python 3.2
- What’s New In Python 3.1
- What’s New In Python 3.0
- What’s New in Python 2.7
- The Future for Python 2.x
- Changes to the Handling of Deprecation Warnings
- Python 3.1 Features
- PEP 372: Adding an Ordered Dictionary to collections
- PEP 378: Format Specifier for Thousands Separator
- PEP 389: The argparse Module for Parsing Command Lines
- PEP 391: Dictionary-Based Configuration For Logging
- PEP 3106: Dictionary Views
- PEP 3137: The memoryview Object
- Other Language Changes
- New and Improved Modules
- Build and C API Changes
- Other Changes and Fixes
- Porting to Python 2.7
- New Features Added to Python 2.7 Maintenance Releases
- Acknowledgements
- What’s New in Python 2.6
- Python 3.0
- Changes to the Development Process
- PEP 343: The ‘with’ statement
- PEP 366: Explicit Relative Imports From a Main Module
- PEP 370: Per-user
site-packages
Directory - PEP 371: The
multiprocessing
Package - PEP 3101: Advanced String Formatting
- PEP 3105:
print
As a Function - PEP 3110: Exception-Handling Changes
- PEP 3112: Byte Literals
- PEP 3116: New I/O Library
- PEP 3118: Revised Buffer Protocol
- PEP 3119: Abstract Base Classes
- PEP 3127: Integer Literal Support and Syntax
- PEP 3129: Class Decorators
- PEP 3141: A Type Hierarchy for Numbers
- Other Language Changes
- New and Improved Modules
- Deprecations and Removals
- Build and C API Changes
- Porting to Python 2.6
- Acknowledgements
- What’s New in Python 2.5
- PEP 308: Conditional Expressions
- PEP 309: Partial Function Application
- PEP 314: Metadata for Python Software Packages v1.1
- PEP 328: Absolute and Relative Imports
- PEP 338: Executing Modules as Scripts
- PEP 341: Unified try/except/finally
- PEP 342: New Generator Features
- PEP 343: The ‘with’ statement
- PEP 352: Exceptions as New-Style Classes
- PEP 353: Using ssize_t as the index type
- PEP 357: The ‘__index__’ method
- Other Language Changes
- New, Improved, and Removed Modules
- Build and C API Changes
- Porting to Python 2.5
- Acknowledgements
- What’s New in Python 2.4
- PEP 218: Built-In Set Objects
- PEP 237: Unifying Long Integers and Integers
- PEP 289: Generator Expressions
- PEP 292: Simpler String Substitutions
- PEP 318: Decorators for Functions and Methods
- PEP 322: Reverse Iteration
- PEP 324: New subprocess Module
- PEP 327: Decimal Data Type
- PEP 328: Multi-line Imports
- PEP 331: Locale-Independent Float/String Conversions
- Other Language Changes
- New, Improved, and Deprecated Modules
- Build and C API Changes
- Porting to Python 2.4
- Acknowledgements
- What’s New in Python 2.3
- PEP 218: A Standard Set Datatype
- PEP 255: Simple Generators
- PEP 263: Source Code Encodings
- PEP 273: Importing Modules from ZIP Archives
- PEP 277: Unicode file name support for Windows NT
- PEP 278: Universal Newline Support
- PEP 279: enumerate()
- PEP 282: The logging Package
- PEP 285: A Boolean Type
- PEP 293: Codec Error Handling Callbacks
- PEP 301: Package Index and Metadata for Distutils
- PEP 302: New Import Hooks
- PEP 305: Comma-separated Files
- PEP 307: Pickle Enhancements
- Extended Slices
- Other Language Changes
- New, Improved, and Deprecated Modules
- Pymalloc: A Specialized Object Allocator
- Build and C API Changes
- Other Changes and Fixes
- Porting to Python 2.3
- Acknowledgements
- What’s New in Python 2.2
- Introduction
- PEPs 252 and 253: Type and Class Changes
- PEP 234: Iterators
- PEP 255: Simple Generators
- PEP 237: Unifying Long Integers and Integers
- PEP 238: Changing the Division Operator
- Unicode Changes
- PEP 227: Nested Scopes
- New and Improved Modules
- Interpreter Changes and Fixes
- Other Changes and Fixes
- Acknowledgements
- What’s New in Python 2.1
- Introduction
- PEP 227: Nested Scopes
- PEP 236: __future__ Directives
- PEP 207: Rich Comparisons
- PEP 230: Warning Framework
- PEP 229: New Build System
- PEP 205: Weak References
- PEP 232: Function Attributes
- PEP 235: Importing Modules on Case-Insensitive Platforms
- PEP 217: Interactive Display Hook
- PEP 208: New Coercion Model
- PEP 241: Metadata in Python Packages
- New and Improved Modules
- Other Changes and Fixes
- Acknowledgements
- What’s New in Python 2.0
- Introduction
- What About Python 1.6?
- New Development Process
- Unicode
- List Comprehensions
- Augmented Assignment
- String Methods
- Garbage Collection of Cycles
- Other Core Changes
- Porting to 2.0
- Extending/Embedding Changes
- Distutils: Making Modules Easy to Install
- XML Modules
- Module changes
- New modules
- IDLE Improvements
- Deleted and Deprecated Modules
- Acknowledgements
The “Changelog” is an HTML version of the file built from the contents of the Misc/NEWS.d directory tree, which contains all nontrivial changes to Python for the current version.
- Changelog
- Python next
- Python 3.14.3 final
- Python 3.14.2 final
- Python 3.14.1 final
- Python 3.14.0 final
- Python 3.14.0 release candidate 3
- Python 3.14.0 release candidate 2
- Python 3.14.0 release candidate 1
- Python 3.14.0 beta 4
- Python 3.14.0 beta 3
- Python 3.14.0 beta 2
- Python 3.14.0 beta 1
- Python 3.14.0 alpha 7
- Python 3.14.0 alpha 6
- Python 3.14.0 alpha 5
- Python 3.14.0 alpha 4
- Python 3.14.0 alpha 3
- Python 3.14.0 alpha 2
- Python 3.14.0 alpha 1
- Python 3.13.0 beta 1
- Python 3.13.0 alpha 6
- Python 3.13.0 alpha 5
- Python 3.13.0 alpha 4
- Python 3.13.0 alpha 3
- Python 3.13.0 alpha 2
- Python 3.13.0 alpha 1
- Python 3.12.0 beta 1
- Python 3.12.0 alpha 7
- Python 3.12.0 alpha 6
- Python 3.12.0 alpha 5
- Python 3.12.0 alpha 4
- Python 3.12.0 alpha 3
- Python 3.12.0 alpha 2
- Python 3.12.0 alpha 1
- Python 3.11.0 beta 1
- Python 3.11.0 alpha 7
- Python 3.11.0 alpha 6
- Python 3.11.0 alpha 5
- Python 3.11.0 alpha 4
- Python 3.11.0 alpha 3
- Python 3.11.0 alpha 2
- Python 3.11.0 alpha 1
- Python 3.10.0 beta 1
- Python 3.10.0 alpha 7
- Python 3.10.0 alpha 6
- Python 3.10.0 alpha 5
- Python 3.10.0 alpha 4
- Python 3.10.0 alpha 3
- Python 3.10.0 alpha 2
- Python 3.10.0 alpha 1
- Python 3.9.0 beta 1
- Python 3.9.0 alpha 6
- Python 3.9.0 alpha 5
- Python 3.9.0 alpha 4
- Python 3.9.0 alpha 3
- Python 3.9.0 alpha 2
- Python 3.9.0 alpha 1
- Python 3.8.0 beta 1
- Python 3.8.0 alpha 4
- Python 3.8.0 alpha 3
- Python 3.8.0 alpha 2
- Python 3.8.0 alpha 1
- Python 3.7.0 final
- Python 3.7.0 release candidate 1
- Python 3.7.0 beta 5
- Python 3.7.0 beta 4
- Python 3.7.0 beta 3
- Python 3.7.0 beta 2
- Python 3.7.0 beta 1
- Python 3.7.0 alpha 4
- Python 3.7.0 alpha 3
- Python 3.7.0 alpha 2
- Python 3.7.0 alpha 1
- Python 3.6.6 final
- Python 3.6.6 release candidate 1
- Python 3.6.5 final
- Python 3.6.5 release candidate 1
- Python 3.6.4 final
- Python 3.6.4 release candidate 1
- Python 3.6.3 final
- Python 3.6.3 release candidate 1
- Python 3.6.2 final
- Python 3.6.2 release candidate 2
- Python 3.6.2 release candidate 1
- Python 3.6.1 final
- Python 3.6.1 release candidate 1
- Python 3.6.0 final
- Python 3.6.0 release candidate 2
- Python 3.6.0 release candidate 1
- Python 3.6.0 beta 4
- Python 3.6.0 beta 3
- Python 3.6.0 beta 2
- Python 3.6.0 beta 1
- Python 3.6.0 alpha 4
- Python 3.6.0 alpha 3
- Python 3.6.0 alpha 2
- Python 3.6.0 alpha 1
- Python 3.5.5 final
- Python 3.5.5 release candidate 1
- Python 3.5.4 final
- Python 3.5.4 release candidate 1
- Python 3.5.3 final
- Python 3.5.3 release candidate 1
- Python 3.5.2 final
- Python 3.5.2 release candidate 1
- Python 3.5.1 final
- Python 3.5.1 release candidate 1
- Python 3.5.0 final
- Python 3.5.0 release candidate 4
- Python 3.5.0 release candidate 3
- Python 3.5.0 release candidate 2
- Python 3.5.0 release candidate 1
- Python 3.5.0 beta 4
- Python 3.5.0 beta 3
- Python 3.5.0 beta 2
- Python 3.5.0 beta 1
- Python 3.5.0 alpha 4
- Python 3.5.0 alpha 3
- Python 3.5.0 alpha 2
- Python 3.5.0 alpha 1


Mapping Protocol

Mapping Protocol¶
See also PyObject_GetItem()
, PyObject_SetItem()
and
PyObject_DelItem()
.
-
int PyMapping_Check(PyObject *o)¶
- Part of the Stable ABI.
Return
1
if the object provides the mapping protocol or supports slicing, and0
otherwise. Note that it returns1
for Python classes with a__getitem__()
method, since in general it is impossible to determine what type of keys the class supports. This function always succeeds.
-
Py_ssize_t PyMapping_Size(PyObject *o)¶
-
Py_ssize_t PyMapping_Length(PyObject *o)¶
- Part of the Stable ABI.
Returns the number of keys in object o on success, and
-1
on failure. This is equivalent to the Python expressionlen(o)
.
-
PyObject *PyMapping_GetItemString(PyObject *o, const char *key)¶
- Return value: New reference. Part of the Stable ABI.
This is the same as
PyObject_GetItem()
, but key is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.
-
int PyMapping_GetOptionalItem(PyObject *obj, PyObject *key, PyObject **result)¶
- Part of the Stable ABI since version 3.13.
Variant of
PyObject_GetItem()
which doesn’t raiseKeyError
if the key is not found.If the key is found, return
1
and set *result to a new strong reference to the corresponding value. If the key is not found, return0
and set *result toNULL
; theKeyError
is silenced. If an error other thanKeyError
is raised, return-1
and set *result toNULL
.Added in version 3.13.
-
int PyMapping_GetOptionalItemString(PyObject *obj, const char *key, PyObject **result)¶
- Part of the Stable ABI since version 3.13.
This is the same as
PyMapping_GetOptionalItem()
, but key is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.Added in version 3.13.
-
int PyMapping_SetItemString(PyObject *o, const char *key, PyObject *v)¶
- Part of the Stable ABI.
This is the same as
PyObject_SetItem()
, but key is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.
-
int PyMapping_DelItem(PyObject *o, PyObject *key)¶
This is an alias of
PyObject_DelItem()
.
-
int PyMapping_DelItemString(PyObject *o, const char *key)¶
This is the same as
PyObject_DelItem()
, but key is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.
-
int PyMapping_HasKeyWithError(PyObject *o, PyObject *key)¶
- Part of the Stable ABI since version 3.13.
Return
1
if the mapping object has the key key and0
otherwise. This is equivalent to the Python expressionkey in o
. On failure, return-1
.Added in version 3.13.
-
int PyMapping_HasKeyStringWithError(PyObject *o, const char *key)¶
- Part of the Stable ABI since version 3.13.
This is the same as
PyMapping_HasKeyWithError()
, but key is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.Added in version 3.13.
-
int PyMapping_HasKey(PyObject *o, PyObject *key)¶
- Part of the Stable ABI.
Return
1
if the mapping object has the key key and0
otherwise. This is equivalent to the Python expressionkey in o
. This function always succeeds.Note
Exceptions which occur when this calls the
__getitem__()
method are silently ignored. For proper error handling, usePyMapping_HasKeyWithError()
,PyMapping_GetOptionalItem()
orPyObject_GetItem()
instead.
-
int PyMapping_HasKeyString(PyObject *o, const char *key)¶
- Part of the Stable ABI.
This is the same as
PyMapping_HasKey()
, but key is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.Note
Exceptions that occur when this calls the
__getitem__()
method or while creating the temporarystr
object are silently ignored. For proper error handling, usePyMapping_HasKeyStringWithError()
,PyMapping_GetOptionalItemString()
orPyMapping_GetItemString()
instead.
-
PyObject *PyMapping_Keys(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
On success, return a list of the keys in object o. On failure, return
NULL
.Changed in version 3.7: Previously, the function returned a list or a tuple.
-
PyObject *PyMapping_Values(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
On success, return a list of the values in object o. On failure, return
NULL
.Changed in version 3.7: Previously, the function returned a list or a tuple.
-
PyObject *PyMapping_Items(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
On success, return a list of the items in object o, where each item is a tuple containing a key-value pair. On failure, return
NULL
.Changed in version 3.7: Previously, the function returned a list or a tuple.


Lexical analysis

2. Lexical analysis¶
A Python program is read by a parser. Input to the parser is a stream of tokens, generated by the lexical analyzer (also known as the tokenizer). This chapter describes how the lexical analyzer produces these tokens.
The lexical analyzer determines the program text’s encoding
(UTF-8 by default), and decodes the text into
source characters.
If the text cannot be decoded, a SyntaxError
is raised.
Next, the lexical analyzer uses the source characters to generate a stream of tokens. The type of a generated token generally depends on the next source character to be processed. Similarly, other special behavior of the analyzer depends on the first source character that hasn’t yet been processed. The following table gives a quick summary of these source characters, with links to sections that contain more information.
Character |
Next token (or other relevant documentation) |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2.1. Line structure¶
A Python program is divided into a number of logical lines.
2.1.1. Logical lines¶
The end of a logical line is represented by the token NEWLINE
.
Statements cannot cross logical line boundaries except where NEWLINE
is allowed by the syntax (e.g., between statements in compound statements).
A logical line is constructed from one or more physical lines by following
the explicit or implicit
line joining rules.
2.1.2. Physical lines¶
A physical line is a sequence of characters terminated by one the following end-of-line sequences:
the Unix form using ASCII LF (linefeed),
the Windows form using the ASCII sequence CR LF (return followed by linefeed),
the ‘Classic Mac OS’ form using the ASCII CR (return) character.
Regardless of platform, each of these sequences is replaced by a single ASCII LF (linefeed) character. (This is done even inside string literals.) Each line can use any of the sequences; they do not need to be consistent within a file.
The end of input also serves as an implicit terminator for the final physical line.
Formally:
newline: <ASCII LF> | <ASCII CR> <ASCII LF> | <ASCII CR>
2.1.4. Encoding declarations¶
If a comment in the first or second line of the Python script matches the
regular expression coding[=:]\s*([-\w.]+)
, this comment is processed as an
encoding declaration; the first group of this expression names the encoding of
the source code file. The encoding declaration must appear on a line of its
own. If it is the second line, the first line must also be a comment-only line.
The recommended forms of an encoding expression are
# -*- coding: <encoding-name> -*-
which is recognized also by GNU Emacs, and
# vim:fileencoding=<encoding-name>
which is recognized by Bram Moolenaar’s VIM.
If no encoding declaration is found, the default encoding is UTF-8. If the
implicit or explicit encoding of a file is UTF-8, an initial UTF-8 byte-order
mark (b'\xef\xbb\xbf'
) is ignored rather than being a syntax error.
If an encoding is declared, the encoding name must be recognized by Python (see Standard Encodings). The encoding is used for all lexical analysis, including string literals, comments and identifiers.
All lexical analysis, including string literals, comments and identifiers, works on Unicode text decoded using the source encoding. Any Unicode code point, except the NUL control character, can appear in Python source.
source_character: <any Unicode code point, except NUL>
2.1.5. Explicit line joining¶
Two or more physical lines may be joined into logical lines using backslash
characters (\
), as follows: when a physical line ends in a backslash that is
not part of a string literal or comment, it is joined with the following forming
a single logical line, deleting the backslash and the following end-of-line
character. For example:
if 1900 < year < 2100 and 1 <= month <= 12 \
and 1 <= day <= 31 and 0 <= hour < 24 \
and 0 <= minute < 60 and 0 <= second < 60: # Looks like a valid date
return 1
A line ending in a backslash cannot carry a comment. A backslash does not continue a comment. A backslash does not continue a token except for string literals (i.e., tokens other than string literals cannot be split across physical lines using a backslash). A backslash is illegal elsewhere on a line outside a string literal.
2.1.6. Implicit line joining¶
Expressions in parentheses, square brackets or curly braces can be split over more than one physical line without using backslashes. For example:
month_names = ['Januari', 'Februari', 'Maart', # These are the
'April', 'Mei', 'Juni', # Dutch names
'Juli', 'Augustus', 'September', # for the months
'Oktober', 'November', 'December'] # of the year
Implicitly continued lines can carry comments. The indentation of the continuation lines is not important. Blank continuation lines are allowed. There is no NEWLINE token between implicit continuation lines. Implicitly continued lines can also occur within triple-quoted strings (see below); in that case they cannot carry comments.
2.1.7. Blank lines¶
A logical line that contains only spaces, tabs, formfeeds and possibly a
comment, is ignored (i.e., no NEWLINE
token is generated).
During interactive input of statements, handling of a blank line may differ
depending on the implementation of the read-eval-print loop.
In the standard interactive interpreter, an entirely blank logical line (that
is, one containing not even whitespace or a comment) terminates a multi-line
statement.
2.1.8. Indentation¶
Leading whitespace (spaces and tabs) at the beginning of a logical line is used to compute the indentation level of the line, which in turn is used to determine the grouping of statements.
Tabs are replaced (from left to right) by one to eight spaces such that the total number of characters up to and including the replacement is a multiple of eight (this is intended to be the same rule as used by Unix). The total number of spaces preceding the first non-blank character then determines the line’s indentation. Indentation cannot be split over multiple physical lines using backslashes; the whitespace up to the first backslash determines the indentation.
Indentation is rejected as inconsistent if a source file mixes tabs and spaces
in a way that makes the meaning dependent on the worth of a tab in spaces; a
TabError
is raised in that case.
Cross-platform compatibility note: because of the nature of text editors on non-UNIX platforms, it is unwise to use a mixture of spaces and tabs for the indentation in a single source file. It should also be noted that different platforms may explicitly limit the maximum indentation level.
A formfeed character may be present at the start of the line; it will be ignored for the indentation calculations above. Formfeed characters occurring elsewhere in the leading whitespace have an undefined effect (for instance, they may reset the space count to zero).
The indentation levels of consecutive lines are used to generate
INDENT
and DEDENT
tokens, using a stack,
as follows.
Before the first line of the file is read, a single zero is pushed on the stack;
this will never be popped off again. The numbers pushed on the stack will
always be strictly increasing from bottom to top. At the beginning of each
logical line, the line’s indentation level is compared to the top of the stack.
If it is equal, nothing happens. If it is larger, it is pushed on the stack, and
one INDENT
token is generated. If it is smaller, it must be one of the
numbers occurring on the stack; all numbers on the stack that are larger are
popped off, and for each number popped off a DEDENT
token is generated.
At the end of the file, a DEDENT
token is generated for each number
remaining on the stack that is larger than zero.
Here is an example of a correctly (though confusingly) indented piece of Python code:
def perm(l):
# Compute the list of all permutations of l
if len(l) <= 1:
return [l]
r = []
for i in range(len(l)):
s = l[:i] + l[i+1:]
p = perm(s)
for x in p:
r.append(l[i:i+1] + x)
return r
The following example shows various indentation errors:
def perm(l): # error: first line indented
for i in range(len(l)): # error: not indented
s = l[:i] + l[i+1:]
p = perm(l[:i] + l[i+1:]) # error: unexpected indent
for x in p:
r.append(l[i:i+1] + x)
return r # error: inconsistent dedent
(Actually, the first three errors are detected by the parser; only the last
error is found by the lexical analyzer — the indentation of return r
does
not match a level popped off the stack.)
2.1.9. Whitespace between tokens¶
Except at the beginning of a logical line or in string literals, the whitespace characters space, tab and formfeed can be used interchangeably to separate tokens:
whitespace: ' ' | tab | formfeed
Whitespace is needed between two tokens only if their concatenation
could otherwise be interpreted as a different token. For example, ab
is one
token, but a b
is two tokens. However, +a
and + a
both produce
two tokens, +
and a
, as +a
is not a valid token.
2.1.10. End marker¶
At the end of non-interactive input, the lexical analyzer generates an
ENDMARKER
token.
2.2. Other tokens¶
Besides NEWLINE
, INDENT
and DEDENT
,
the following categories of tokens exist:
identifiers and keywords (NAME
), literals (such as
NUMBER
and STRING
), and other symbols
(operators and delimiters, OP
).
Whitespace characters (other than logical line terminators, discussed earlier)
are not tokens, but serve to delimit tokens.
Where ambiguity exists, a token comprises the longest possible string that
forms a legal token, when read from left to right.
2.3. Names (identifiers and keywords)¶
NAME
tokens represent identifiers, keywords, and
soft keywords.
Names are composed of the following characters:
uppercase and lowercase letters (
A-Z
anda-z
),the underscore (
_
),digits (
0
through9
), which cannot appear as the first character, andnon-ASCII characters. Valid names may only contain “letter-like” and “digit-like” characters; see Non-ASCII characters in names for details.
Names must contain at least one character, but have no upper length limit. Case is significant.
Formally, names are described by the following lexical definitions:
NAME:name_start
name_continue
* name_start: "a"..."z" | "A"..."Z" | "_" | <non-ASCII character> name_continue: name_start | "0"..."9" identifier: <NAME
, except keywords>
Note that not all names matched by this grammar are valid; see Non-ASCII characters in names for details.
2.3.1. Keywords¶
The following names are used as reserved words, or keywords of the language, and cannot be used as ordinary identifiers. They must be spelled exactly as written here:
False await else import pass
None break except in raise
True class finally is return
and continue for lambda try
as def from nonlocal while
assert del global not with
async elif if or yield
2.3.2. Soft Keywords¶
Added in version 3.10.
Some names are only reserved under specific contexts. These are known as soft keywords:
These syntactically act as keywords in their specific contexts, but this distinction is done at the parser level, not when tokenizing.
As soft keywords, their use in the grammar is possible while still preserving compatibility with existing code that uses these names as identifier names.
Changed in version 3.12: type
is now a soft keyword.
2.3.3. Reserved classes of identifiers¶
Certain classes of identifiers (besides keywords) have special meanings. These classes are identified by the patterns of leading and trailing underscore characters:
_*
Not imported by
from module import *
._
In a
case
pattern within amatch
statement,_
is a soft keyword that denotes a wildcard.Separately, the interactive interpreter makes the result of the last evaluation available in the variable
_
. (It is stored in thebuiltins
module, alongside built-in functions likeprint
.)Elsewhere,
_
is a regular identifier. It is often used to name “special” items, but it is not special to Python itself.Note
The name
_
is often used in conjunction with internationalization; refer to the documentation for thegettext
module for more information on this convention.It is also commonly used for unused variables.
__*__
System-defined names, informally known as “dunder” names. These names are defined by the interpreter and its implementation (including the standard library). Current system names are discussed in the Special method names section and elsewhere. More will likely be defined in future versions of Python. Any use of
__*__
names, in any context, that does not follow explicitly documented use, is subject to breakage without warning.__*
Class-private names. Names in this category, when used within the context of a class definition, are re-written to use a mangled form to help avoid name clashes between “private” attributes of base and derived classes. See section Identifiers (Names).
2.3.4. Non-ASCII characters in names¶
Names that contain non-ASCII characters need additional normalization
and validation beyond the rules and grammar explained
above.
For example, ř_1
, 蛇
, or साँप
are valid names, but r〰2
,
€
, or 🐍
are not.
This section explains the exact rules.
All names are converted into the normalization form NFKC while parsing.
This means that, for example, some typographic variants of characters are
converted to their “basic” form. For example, ﬁⁿₐˡᵢᶻₐᵗᵢᵒₙ
normalizes to
finalization
, so Python treats them as the same name:
>>> ﬁⁿₐˡᵢᶻₐᵗᵢᵒₙ = 3
>>> finalization
3
Note
Normalization is done at the lexical level only.
Run-time functions that take names as strings generally do not normalize
their arguments.
For example, the variable defined above is accessible at run time in the
globals()
dictionary as globals()["finalization"]
but not
globals()["ﬁⁿₐˡᵢᶻₐᵗᵢᵒₙ"]
.
Similarly to how ASCII-only names must contain only letters, digits and
the underscore, and cannot start with a digit, a valid name must
start with a character in the “letter-like” set xid_start
,
and the remaining characters must be in the “letter- and digit-like” set
xid_continue
.
These sets based on the XID_Start and XID_Continue sets as defined by the
Unicode standard annex UAX-31.
Python’s xid_start
additionally includes the underscore (_
).
Note that Python does not necessarily conform to UAX-31.
A non-normative listing of characters in the XID_Start and XID_Continue
sets as defined by Unicode is available in the DerivedCoreProperties.txt
file in the Unicode Character Database.
For reference, the construction rules for the xid_*
sets are given below.
The set id_start
is defined as the union of:
Unicode category
<Lu>
- uppercase letters (includesA
toZ
)Unicode category
<Ll>
- lowercase letters (includesa
toz
)Unicode category
<Lt>
- titlecase lettersUnicode category
<Lm>
- modifier lettersUnicode category
<Lo>
- other lettersUnicode category
<Nl>
- letter numbers{
"_"
} - the underscore<Other_ID_Start>
- an explicit set of characters in PropList.txt to support backwards compatibility
The set xid_start
then closes this set under NFKC normalization, by
removing all characters whose normalization is not of the form
id_start id_continue*
.
The set id_continue
is defined as the union of:
id_start
(see above)Unicode category
<Nd>
- decimal numbers (includes0
to9
)Unicode category
<Pc>
- connector punctuationsUnicode category
<Mn>
- nonspacing marksUnicode category
<Mc>
- spacing combining marks<Other_ID_Continue>
- another explicit set of characters in PropList.txt to support backwards compatibility
Again, xid_continue
closes this set under NFKC normalization.
Unicode categories use the version of the Unicode Character Database as
included in the unicodedata
module.
2.4. Literals¶
Literals are notations for constant values of some built-in types.
In terms of lexical analysis, Python has string, bytes and numeric literals.
Other “literals” are lexically denoted using keywords
(None
, True
, False
) and the special
ellipsis token (...
).
2.5. String and Bytes literals¶
String literals are text enclosed in single quotes ('
) or double
quotes ("
). For example:
"spam"
'eggs'
The quote used to start the literal also terminates it, so a string literal can only contain the other quote (except with escape sequences, see below). For example:
'Say "Hello", please.'
"Don't do that!"
Except for this limitation, the choice of quote character ('
or "
)
does not affect how the literal is parsed.
Inside a string literal, the backslash (\
) character introduces an
escape sequence, which has special meaning depending on the character
after the backslash.
For example, \"
denotes the double quote character, and does not end
the string:
>>> print("Say \"Hello\" to everyone!")
Say "Hello" to everyone!
See escape sequences below for a full list of such sequences, and more details.
2.5.1. Triple-quoted strings¶
Strings can also be enclosed in matching groups of three single or double quotes. These are generally referred to as triple-quoted strings:
"""This is a triple-quoted string."""
In triple-quoted literals, unescaped quotes are allowed (and are
retained), except that three unescaped quotes in a row terminate the literal,
if they are of the same kind ('
or "
) used at the start:
"""This string has "quotes" inside."""
Unescaped newlines are also allowed and retained:
'''This triple-quoted string
continues on the next line.'''
2.5.2. String prefixes¶
String literals can have an optional prefix that influences how the content of the literal is parsed, for example:
b"data"
f'{result=}'
The allowed prefixes are:
r
: Raw stringf
: Formatted string literal (“f-string”)t
: Template string literal (“t-string”)u
: No effect (allowed for backwards compatibility)
See the linked sections for details on each type.
Prefixes are case-insensitive (for example, ‘B
’ works the same as ‘b
’).
The ‘r
’ prefix can be combined with ‘f
’, ‘t
’ or ‘b
’, so ‘fr
’,
‘rf
’, ‘tr
’, ‘rt
’, ‘br
’, and ‘rb
’ are also valid prefixes.
Added in version 3.3: The 'rb'
prefix of raw bytes literals has been added as a synonym
of 'br'
.
Support for the unicode legacy literal (u'value'
) was reintroduced
to simplify the maintenance of dual Python 2.x and 3.x codebases.
See PEP 414 for more information.
2.5.3. Formal grammar¶
String literals, except “f-strings” and “t-strings”, are described by the following lexical definitions.
These definitions use negative lookaheads (!
)
to indicate that an ending quote ends the literal.
STRING: [stringprefix
] (stringcontent
) stringprefix: <("r" | "u" | "b" | "br" | "rb"), case-insensitive> stringcontent: | "'''" ( !"'''"longstringitem
)* "'''" | '"""' ( !'"""'longstringitem
)* '"""' | "'" ( !"'"stringitem
)* "'" | '"' ( !'"'stringitem
)* '"' stringitem:stringchar
|stringescapeseq
stringchar: <anysource_character
, except backslash and newline> longstringitem:stringitem
| newline stringescapeseq: "\" <anysource_character
>
Note that as in all lexical definitions, whitespace is significant. In particular, the prefix (if any) must be immediately followed by the starting quote.
2.5.4. Escape sequences¶
Unless an ‘r
’ or ‘R
’ prefix is present, escape sequences in string and
bytes literals are interpreted according to rules similar to those used by
Standard C. The recognized escape sequences are:
Escape Sequence |
Meaning |
|---|---|
|
|
|
|
|
|
|
|
|
ASCII Bell (BEL) |
|
ASCII Backspace (BS) |
|
ASCII Formfeed (FF) |
|
ASCII Linefeed (LF) |
|
ASCII Carriage Return (CR) |
|
ASCII Horizontal Tab (TAB) |
|
ASCII Vertical Tab (VT) |
|
|
|
|
|
|
|
|
|
2.5.4.1. Ignored end of line¶
A backslash can be added at the end of a line to ignore the newline:
>>> 'This string will not include \
... backslashes or newline characters.'
'This string will not include backslashes or newline characters.'
The same result can be achieved using triple-quoted strings, or parentheses and string literal concatenation.
2.5.4.2. Escaped characters¶
To include a backslash in a non-raw Python string
literal, it must be doubled. The \\
escape sequence denotes a single
backslash character:
>>> print('C:\\Program Files')
C:\Program Files
Similarly, the \'
and \"
sequences denote the single and double
quote character, respectively:
>>> print('\' and \"')
' and "
2.5.4.3. Octal character¶
The sequence \ooo
denotes a character with the octal (base 8)
value ooo:
>>> '\120'
'P'
Up to three octal digits (0 through 7) are accepted.
In a bytes literal, character means a byte with the given value. In a string literal, it means a Unicode character with the given value.
Changed in version 3.11: Octal escapes with value larger than 0o377
(255) produce a
DeprecationWarning
.
Changed in version 3.12: Octal escapes with value larger than 0o377
(255) produce a
SyntaxWarning
.
In a future Python version they will raise a SyntaxError
.
2.5.4.4. Hexadecimal character¶
The sequence \xhh
denotes a character with the hex (base 16)
value hh:
>>> '\x50'
'P'
Unlike in Standard C, exactly two hex digits are required.
In a bytes literal, character means a byte with the given value. In a string literal, it means a Unicode character with the given value.
2.5.4.5. Named Unicode character¶
The sequence \N{name}
denotes a Unicode character
with the given name:
>>> '\N{LATIN CAPITAL LETTER P}'
'P'
>>> '\N{SNAKE}'
'🐍'
This sequence cannot appear in bytes literals.
Changed in version 3.3: Support for name aliases has been added.
2.5.4.6. Hexadecimal Unicode characters¶
These sequences \uxxxx
and \Uxxxxxxxx
denote the
Unicode character with the given hex (base 16) value.
Exactly four digits are required for \u
; exactly eight digits are
required for \U
.
The latter can encode any Unicode character.
>>> '\u1234'
'ሴ'
>>> '\U0001f40d'
'🐍'
These sequences cannot appear in bytes literals.
2.5.4.7. Unrecognized escape sequences¶
Unlike in Standard C, all unrecognized escape sequences are left in the string unchanged, that is, the backslash is left in the result:
>>> print('\q')
\q
>>> list('\q')
['\\', 'q']
Note that for bytes literals, the escape sequences only recognized in string
literals (\N...
, \u...
, \U...
) fall into the category of
unrecognized escapes.
Changed in version 3.6: Unrecognized escape sequences produce a DeprecationWarning
.
Changed in version 3.12: Unrecognized escape sequences produce a SyntaxWarning
.
In a future Python version they will raise a SyntaxError
.
2.5.5. Bytes literals¶
Bytes literals are always prefixed with ‘b
’ or ‘B
’; they produce an
instance of the bytes
type instead of the str
type.
They may only contain ASCII characters; bytes with a numeric value of 128
or greater must be expressed with escape sequences (typically
Hexadecimal character or Octal character):
>>> b'\x89PNG\r\n\x1a\n'
b'\x89PNG\r\n\x1a\n'
>>> list(b'\x89PNG\r\n\x1a\n')
[137, 80, 78, 71, 13, 10, 26, 10]
Similarly, a zero byte must be expressed using an escape sequence (typically
\0
or \x00
).
2.5.6. Raw string literals¶
Both string and bytes literals may optionally be prefixed with a letter ‘r
’
or ‘R
’; such constructs are called raw string literals
and raw bytes literals respectively and treat backslashes as
literal characters.
As a result, in raw string literals, escape sequences
are not treated specially:
>>> r'\d{4}-\d{2}-\d{2}'
'\\d{4}-\\d{2}-\\d{2}'
Even in a raw literal, quotes can be escaped with a backslash, but the
backslash remains in the result; for example, r"\""
is a valid string
literal consisting of two characters: a backslash and a double quote; r"\"
is not a valid string literal (even a raw string cannot end in an odd number of
backslashes). Specifically, a raw literal cannot end in a single backslash
(since the backslash would escape the following quote character). Note also
that a single backslash followed by a newline is interpreted as those two
characters as part of the literal, not as a line continuation.
2.5.7. f-strings¶
Added in version 3.6.
Changed in version 3.8: Added the debug specifier (=
)
Changed in version 3.12: Many restrictions on expressions within f-strings have been removed. Notably, nested strings, comments, and backslashes are now permitted.
A formatted string literal or f-string is a string literal
that is prefixed with ‘f
’ or ‘F
’.
Unlike other string literals, f-strings do not have a constant value.
They may contain replacement fields delimited by curly braces {}
.
Replacement fields contain expressions which are evaluated at run time.
For example:
>>> who = 'nobody'
>>> nationality = 'Spanish'
>>> f'{who.title()} expects the {nationality} Inquisition!'
'Nobody expects the Spanish Inquisition!'
Any doubled curly braces ({{
or }}
) outside replacement fields
are replaced with the corresponding single curly brace:
>>> print(f'{{...}}')
{...}
Other characters outside replacement fields are treated like in ordinary string literals. This means that escape sequences are decoded (except when a literal is also marked as a raw string), and newlines are possible in triple-quoted f-strings:
>>> name = 'Galahad'
>>> favorite_color = 'blue'
>>> print(f'{name}:\t{favorite_color}')
Galahad: blue
>>> print(rf"C:\Users\{name}")
C:\Users\Galahad
>>> print(f'''Three shall be the number of the counting
... and the number of the counting shall be three.''')
Three shall be the number of the counting
and the number of the counting shall be three.
Expressions in formatted string literals are treated like regular
Python expressions.
Each expression is evaluated in the context where the formatted string literal
appears, in order from left to right.
An empty expression is not allowed, and both lambda
and
assignment expressions :=
must be surrounded by explicit parentheses:
>>> f'{(half := 1/2)}, {half * 42}'
'0.5, 21.0'
Reusing the outer f-string quoting type inside a replacement field is permitted:
>>> a = dict(x=2)
>>> f"abc {a["x"]} def"
'abc 2 def'
Backslashes are also allowed in replacement fields and are evaluated the same way as in any other context:
>>> a = ["a", "b", "c"]
>>> print(f"List a contains:\n{"\n".join(a)}")
List a contains:
a
b
c
It is possible to nest f-strings:
>>> name = 'world'
>>> f'Repeated:{f' hello {name}' * 3}'
'Repeated: hello world hello world hello world'
Portable Python programs should not use more than 5 levels of nesting.
CPython implementation detail: CPython does not limit nesting of f-strings.
Replacement expressions can contain newlines in both single-quoted and
triple-quoted f-strings and they can contain comments.
Everything that comes after a #
inside a replacement field
is a comment (even closing braces and quotes).
This means that replacement fields with comments must be closed in a
different line:
>>> a = 2
>>> f"abc{a # This comment }" continues until the end of the line
... + 3}"
'abc5'
After the expression, replacement fields may optionally contain:
a debug specifier – an equal sign (
=
), optionally surrounded by whitespace on one or both sides;a conversion specifier –
!s
,!r
or!a
; and/ora format specifier prefixed with a colon (
:
).
See the Standard Library section on f-strings for details on how these fields are evaluated.
As that section explains, format specifiers are passed as the second argument
to the format()
function to format a replacement field value.
For example, they can be used to specify a field width and padding characters
using the Format Specification Mini-Language:
>>> number = 14.3
>>> f'{number:20.7f}'
' 14.3000000'
Top-level format specifiers may include nested replacement fields:
>>> field_size = 20
>>> precision = 7
>>> f'{number:{field_size}.{precision}f}'
' 14.3000000'
These nested fields may include their own conversion fields and format specifiers:
>>> number = 3
>>> f'{number:{field_size}}'
' 3'
>>> f'{number:{field_size:05}}'
'00000000000000000003'
However, these nested fields may not include more deeply nested replacement fields.
Formatted string literals cannot be used as docstrings, even if they do not include expressions:
>>> def foo():
... f"Not a docstring"
...
>>> print(foo.__doc__)
None
See also
PEP 498 – Literal String Interpolation
PEP 701 – Syntactic formalization of f-strings
str.format()
, which uses a related format string mechanism.
2.5.8. t-strings¶
Added in version 3.14.
A template string literal or t-string is a string literal
that is prefixed with ‘t
’ or ‘T
’.
These strings follow the same syntax rules as
formatted string literals.
For differences in evaluation rules, see the
Standard Library section on t-strings
2.5.9. Formal grammar for f-strings¶
F-strings are handled partly by the lexical analyzer, which produces the
tokens FSTRING_START
, FSTRING_MIDDLE
and FSTRING_END
, and partly by the parser, which handles
expressions in the replacement field.
The exact way the work is split is a CPython implementation detail.
Correspondingly, the f-string grammar is a mix of lexical and syntactic definitions.
Whitespace is significant in these situations:
There may be no whitespace in
FSTRING_START
(between the prefix and quote).Whitespace in
FSTRING_MIDDLE
is part of the literal string contents.In
fstring_replacement_field
, iff_debug_specifier
is present, all whitespace after the opening brace until thef_debug_specifier
, as well as whitespace immediately followingf_debug_specifier
, is retained as part of the expression.CPython implementation detail: The expression is not handled in the tokenization phase; it is retrieved from the source code using locations of the
{
token and the token after=
.
The FSTRING_MIDDLE
definition uses
negative lookaheads (!
)
to indicate special characters (backslash, newline, {
, }
) and
sequences (f_quote
).
fstring:FSTRING_START
fstring_middle
*FSTRING_END
FSTRING_START:fstringprefix
("'" | '"' | "'''" | '"""') FSTRING_END:f_quote
fstringprefix: <("f" | "fr" | "rf"), case-insensitive> f_debug_specifier: '=' f_quote: <the quote character(s) used in FSTRING_START> fstring_middle: |fstring_replacement_field
|FSTRING_MIDDLE
FSTRING_MIDDLE: | (!"\" !newline
!'{' !'}' !f_quote
)source_character
|stringescapeseq
| "{{" | "}}" | <newline, in triple-quoted f-strings only> fstring_replacement_field: | '{'f_expression
[f_debug_specifier
] [fstring_conversion
] [fstring_full_format_spec
] '}' fstring_conversion: | "!" ("s" | "r" | "a") fstring_full_format_spec: | ':'fstring_format_spec
* fstring_format_spec: |FSTRING_MIDDLE
|fstring_replacement_field
f_expression: | ','.(conditional_expression
| "*"or_expr
)+ [","] |yield_expression
Note
In the above grammar snippet, the f_quote
and FSTRING_MIDDLE
rules
are context-sensitive – they depend on the contents of FSTRING_START
of the nearest enclosing fstring
.
Constructing a more traditional formal grammar from this template is left as an exercise for the reader.
The grammar for t-strings is identical to the one for f-strings, with t instead of f at the beginning of rule and token names and in the prefix.
tstring: TSTRING_START tstring_middle* TSTRING_END <rest of the t-string grammar is omitted; see above>
2.6. Numeric literals¶
NUMBER
tokens represent numeric literals, of which there are
three types: integers, floating-point numbers, and imaginary numbers.
NUMBER:integer
|floatnumber
|imagnumber
The numeric value of a numeric literal is the same as if it were passed as a
string to the int
, float
or complex
class
constructor, respectively.
Note that not all valid inputs for those constructors are also valid literals.
Numeric literals do not include a sign; a phrase like -1
is
actually an expression composed of the unary operator ‘-
’ and the literal
1
.
2.6.1. Integer literals¶
Integer literals denote whole numbers. For example:
7
3
2147483647
There is no limit for the length of integer literals apart from what can be stored in available memory:
7922816251426433759354395033679228162514264337593543950336
Underscores can be used to group digits for enhanced readability, and are ignored for determining the numeric value of the literal. For example, the following literals are equivalent:
100_000_000_000
100000000000
1_00_00_00_00_000
Underscores can only occur between digits.
For example, _123
, 321_
, and 123__321
are not valid literals.
Integers can be specified in binary (base 2), octal (base 8), or hexadecimal
(base 16) using the prefixes 0b
, 0o
and 0x
, respectively.
Hexadecimal digits 10 through 15 are represented by letters A
-F
,
case-insensitive. For example:
0b100110111
0b_1110_0101
0o177
0o377
0xdeadbeef
0xDead_Beef
An underscore can follow the base specifier.
For example, 0x_1f
is a valid literal, but 0_x1f
and 0x__1f
are
not.
Leading zeros in a non-zero decimal number are not allowed.
For example, 0123
is not a valid literal.
This is for disambiguation with C-style octal literals, which Python used
before version 3.0.
Formally, integer literals are described by the following lexical definitions:
integer:decinteger
|bininteger
|octinteger
|hexinteger
|zerointeger
decinteger:nonzerodigit
(["_"]digit
)* bininteger: "0" ("b" | "B") (["_"]bindigit
)+ octinteger: "0" ("o" | "O") (["_"]octdigit
)+ hexinteger: "0" ("x" | "X") (["_"]hexdigit
)+ zerointeger: "0"+ (["_"] "0")* nonzerodigit: "1"..."9" digit: "0"..."9" bindigit: "0" | "1" octdigit: "0"..."7" hexdigit:digit
| "a"..."f" | "A"..."F"
Changed in version 3.6: Underscores are now allowed for grouping purposes in literals.
2.6.2. Floating-point literals¶
Floating-point (float) literals, such as 3.14
or 1.5
, denote
approximations of real numbers.
They consist of integer and fraction parts, each composed of decimal digits.
The parts are separated by a decimal point, .
:
2.71828
4.0
Unlike in integer literals, leading zeros are allowed.
For example, 077.010
is legal, and denotes the same number as 77.01
.
As in integer literals, single underscores may occur between digits to help readability:
96_485.332_123
3.14_15_93
Either of these parts, but not both, can be empty. For example:
10. # (equivalent to 10.0)
.001 # (equivalent to 0.001)
Optionally, the integer and fraction may be followed by an exponent:
the letter e
or E
, followed by an optional sign, +
or -
,
and a number in the same format as the integer and fraction parts.
The e
or E
represents “times ten raised to the power of”:
1.0e3 # (represents 1.0×10³, or 1000.0)
1.166e-5 # (represents 1.166×10⁻⁵, or 0.00001166)
6.02214076e+23 # (represents 6.02214076×10²³, or 602214076000000000000000.)
In floats with only integer and exponent parts, the decimal point may be omitted:
1e3 # (equivalent to 1.e3 and 1.0e3)
0e0 # (equivalent to 0.)
Formally, floating-point literals are described by the following lexical definitions:
floatnumber: |digitpart
"." [digitpart
] [exponent
] | "."digitpart
[exponent
] |digitpart
exponent
digitpart:digit
(["_"]digit
)* exponent: ("e" | "E") ["+" | "-"]digitpart
Changed in version 3.6: Underscores are now allowed for grouping purposes in literals.
2.6.3. Imaginary literals¶
Python has complex number objects, but no complex literals. Instead, imaginary literals denote complex numbers with a zero real part.
For example, in math, the complex number 3+4.2i is written
as the real number 3 added to the imaginary number 4.2i.
Python uses a similar syntax, except the imaginary unit is written as j
rather than i:
3+4.2j
This is an expression composed
of the integer literal 3
,
the operator ‘+
’,
and the imaginary literal 4.2j
.
Since these are three separate tokens, whitespace is allowed between them:
3 + 4.2j
No whitespace is allowed within each token.
In particular, the j
suffix, may not be separated from the number
before it.
The number before the j
has the same syntax as a floating-point literal.
Thus, the following are valid imaginary literals:
4.2j
3.14j
10.j
.001j
1e100j
3.14e-10j
3.14_15_93j
Unlike in a floating-point literal the decimal point can be omitted if the imaginary number only has an integer part. The number is still evaluated as a floating-point number, not an integer:
10j
0j
1000000000000000000000000j # equivalent to 1e+24j
The j
suffix is case-insensitive.
That means you can use J
instead:
3.14J # equivalent to 3.14j
Formally, imaginary literals are described by the following lexical definition:
imagnumber: (floatnumber
|digitpart
) ("j" | "J")
2.7. Operators and delimiters¶
The following grammar defines operator and delimiter tokens,
that is, the generic OP
token type.
A list of these tokens and their names
is also available in the token
module documentation.
OP: | assignment_operator | bitwise_operator | comparison_operator | enclosing_delimiter | other_delimiter | arithmetic_operator | "..." | other_op assignment_operator: "+=" | "-=" | "*=" | "**=" | "/=" | "//=" | "%=" | "&=" | "|=" | "^=" | "<<=" | ">>=" | "@=" | ":=" bitwise_operator: "&" | "|" | "^" | "~" | "<<" | ">>" comparison_operator: "<=" | ">=" | "<" | ">" | "==" | "!=" enclosing_delimiter: "(" | ")" | "[" | "]" | "{" | "}" other_delimiter: "," | ":" | "!" | ";" | "=" | "->" arithmetic_operator: "+" | "-" | "**" | "*" | "//" | "/" | "%" other_op: "." | "@"
Note
Generally, operators are used to combine expressions, while delimiters serve other purposes. However, there is no clear, formal distinction between the two categories.
Some tokens can serve as either operators or delimiters, depending on usage.
For example, *
is both the multiplication operator and a delimiter used
for sequence unpacking, and @
is both the matrix multiplication and
a delimiter that introduces decorators.
For some tokens, the distinction is unclear.
For example, some people consider .
, (
, and )
to be delimiters, while others
see the getattr()
operator and the function call operator(s).
Some of Python’s operators, like and
, or
, and not in
, use
keyword tokens rather than “symbols” (operator tokens).
A sequence of three consecutive periods (...
) has a special
meaning as an Ellipsis
literal.
2.1.3. Comments¶
A comment starts with a hash character (
#
) that is not part of a string literal, and ends at the end of the physical line. A comment signifies the end of the logical line unless the implicit line joining rules are invoked. Comments are ignored by the syntax.


Call Graph Introspection

Call Graph Introspection¶
Source code: Lib/asyncio/graph.py
asyncio has powerful runtime call graph introspection utilities to trace the entire call graph of a running coroutine or task, or a suspended future. These utilities and the underlying machinery can be used from within a Python program or by external profilers and debuggers.
Added in version 3.14.
- asyncio.print_call_graph(future=None, /, *, file=None, depth=1, limit=None)¶
Print the async call graph for the current task or the provided
Task
orFuture
.This function prints entries starting from the top frame and going down towards the invocation point.
The function receives an optional future argument. If not passed, the current running task will be used.
If the function is called on the current task, the optional keyword-only depth argument can be used to skip the specified number of frames from top of the stack.
If the optional keyword-only limit argument is provided, each call stack in the resulting graph is truncated to include at most
abs(limit)
entries. If limit is positive, the entries left are the closest to the invocation point. If limit is negative, the topmost entries are left. If limit is omitted orNone
, all entries are present. If limit is0
, the call stack is not printed at all, only “awaited by” information is printed.If file is omitted or
None
, the function will print tosys.stdout
.Example:
The following Python code:
import asyncio async def test(): asyncio.print_call_graph() async def main(): async with asyncio.TaskGroup() as g: g.create_task(test(), name='test') asyncio.run(main())
will print:
* Task(name='test', id=0x1039f0fe0) + Call stack: | File 't2.py', line 4, in async test() + Awaited by: * Task(name='Task-1', id=0x103a5e060) + Call stack: | File 'taskgroups.py', line 107, in async TaskGroup.__aexit__() | File 't2.py', line 7, in async main()
- asyncio.format_call_graph(future=None, /, *, depth=1, limit=None)¶
Like
print_call_graph()
, but returns a string. If future isNone
and there’s no current task, the function returns an empty string.
- asyncio.capture_call_graph(future=None, /, *, depth=1, limit=None)¶
Capture the async call graph for the current task or the provided
Task
orFuture
.The function receives an optional future argument. If not passed, the current running task will be used. If there’s no current task, the function returns
None
.If the function is called on the current task, the optional keyword-only depth argument can be used to skip the specified number of frames from top of the stack.
Returns a
FutureCallGraph
data class object:FutureCallGraph(future, call_stack, awaited_by)
FrameCallGraphEntry(frame)
Where frame is a frame object of a regular Python function in the call stack.
Low level utility functions¶
To introspect an async call graph asyncio requires cooperation from
control flow structures, such as shield()
or TaskGroup
.
Any time an intermediate Future
object with low-level APIs like
Future.add_done_callback()
is
involved, the following two functions should be used to inform asyncio
about how exactly such intermediate future objects are connected with
the tasks they wrap or control.
- asyncio.future_add_to_awaited_by(future, waiter, /)¶
Record that future is awaited on by waiter.
Both future and waiter must be instances of
Future
orTask
or their subclasses, otherwise the call would have no effect.A call to
future_add_to_awaited_by()
must be followed by an eventual call to thefuture_discard_from_awaited_by()
function with the same arguments.


— The most common POSIX system calls

posix
— The most common POSIX system calls¶
This module provides access to operating system functionality that is standardized by the C Standard and the POSIX standard (a thinly disguised Unix interface).
Availability: Unix.
Do not import this module directly. Instead, import the module os
,
which provides a portable version of this interface. On Unix, the os
module provides a superset of the posix
interface. On non-Unix operating
systems the posix
module is not available, but a subset is always
available through the os
interface. Once os
is imported, there is
no performance penalty in using it instead of posix
. In addition,
os
provides some additional functionality, such as automatically calling
putenv()
when an entry in os.environ
is changed.
Errors are reported as exceptions; the usual exceptions are given for type
errors, while errors reported by the system calls raise OSError
.
Large File Support¶
Several operating systems (including AIX and Solaris) provide support for files that are larger than 2 GiB from a C programming model where int and long are 32-bit values. This is typically accomplished by defining the relevant size and offset types as 64-bit values. Such files are sometimes referred to as large files.
Large file support is enabled in Python when the size of an off_t
is
larger than a long and the long long is at least as large
as an off_t
.
It may be necessary to configure and compile Python with certain compiler flags
to enable this mode. For example, with Solaris 2.6 and 2.7 you need to do
something like:
CFLAGS="`getconf LFS_CFLAGS`" OPT="-g -O2 $CFLAGS" \
./configure
On large-file-capable Linux systems, this might work:
CFLAGS='-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64' OPT="-g -O2 $CFLAGS" \
./configure
Notable Module Contents¶
In addition to many functions described in the os
module documentation,
posix
defines the following data item:
- posix.environ¶
A dictionary representing the string environment at the time the interpreter was started. Keys and values are bytes on Unix and str on Windows. For example,
environ[b'HOME']
(environ['HOME']
on Windows) is the pathname of your home directory, equivalent togetenv("HOME")
in C.Modifying this dictionary does not affect the string environment passed on by
execv()
,popen()
orsystem()
; if you need to change the environment, passenviron
toexecve()
or add variable assignments and export statements to the command string forsystem()
orpopen()
.Changed in version 3.2: On Unix, keys and values are bytes.
Note
The
os
module provides an alternate implementation ofenviron
which updates the environment on modification. Note also that updatingos.environ
will render this dictionary obsolete. Use of theos
module version of this is recommended over direct access to theposix
module.


Sequence Protocol

Sequence Protocol¶
-
int PySequence_Check(PyObject *o)¶
- Part of the Stable ABI.
Return
1
if the object provides the sequence protocol, and0
otherwise. Note that it returns1
for Python classes with a__getitem__()
method, unless they aredict
subclasses, since in general it is impossible to determine what type of keys the class supports. This function always succeeds.
-
Py_ssize_t PySequence_Size(PyObject *o)¶
-
Py_ssize_t PySequence_Length(PyObject *o)¶
- Part of the Stable ABI.
Returns the number of objects in sequence o on success, and
-1
on failure. This is equivalent to the Python expressionlen(o)
.
-
PyObject *PySequence_Concat(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Return the concatenation of o1 and o2 on success, and
NULL
on failure. This is the equivalent of the Python expressiono1 + o2
.
-
PyObject *PySequence_Repeat(PyObject *o, Py_ssize_t count)¶
- Return value: New reference. Part of the Stable ABI.
Return the result of repeating sequence object o count times, or
NULL
on failure. This is the equivalent of the Python expressiono * count
.
-
PyObject *PySequence_InPlaceConcat(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Return the concatenation of o1 and o2 on success, and
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python expressiono1 += o2
.
-
PyObject *PySequence_InPlaceRepeat(PyObject *o, Py_ssize_t count)¶
- Return value: New reference. Part of the Stable ABI.
Return the result of repeating sequence object o count times, or
NULL
on failure. The operation is done in-place when o supports it. This is the equivalent of the Python expressiono *= count
.
-
PyObject *PySequence_GetItem(PyObject *o, Py_ssize_t i)¶
- Return value: New reference. Part of the Stable ABI.
Return the ith element of o, or
NULL
on failure. This is the equivalent of the Python expressiono[i]
.
-
PyObject *PySequence_GetSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2)¶
- Return value: New reference. Part of the Stable ABI.
Return the slice of sequence object o between i1 and i2, or
NULL
on failure. This is the equivalent of the Python expressiono[i1:i2]
.
-
int PySequence_SetItem(PyObject *o, Py_ssize_t i, PyObject *v)¶
- Part of the Stable ABI.
Assign object v to the ith element of o. Raise an exception and return
-1
on failure; return0
on success. This is the equivalent of the Python statemento[i] = v
. This function does not steal a reference to v.If v is
NULL
, the element is deleted, but this feature is deprecated in favour of usingPySequence_DelItem()
.
-
int PySequence_DelItem(PyObject *o, Py_ssize_t i)¶
- Part of the Stable ABI.
Delete the ith element of object o. Returns
-1
on failure. This is the equivalent of the Python statementdel o[i]
.
-
int PySequence_SetSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2, PyObject *v)¶
- Part of the Stable ABI.
Assign the sequence object v to the slice in sequence object o from i1 to i2. This is the equivalent of the Python statement
o[i1:i2] = v
.
-
int PySequence_DelSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2)¶
- Part of the Stable ABI.
Delete the slice in sequence object o from i1 to i2. Returns
-1
on failure. This is the equivalent of the Python statementdel o[i1:i2]
.
-
Py_ssize_t PySequence_Count(PyObject *o, PyObject *value)¶
- Part of the Stable ABI.
Return the number of occurrences of value in o, that is, return the number of keys for which
o[key] == value
. On failure, return-1
. This is equivalent to the Python expressiono.count(value)
.
-
int PySequence_Contains(PyObject *o, PyObject *value)¶
- Part of the Stable ABI.
Determine if o contains value. If an item in o is equal to value, return
1
, otherwise return0
. On error, return-1
. This is equivalent to the Python expressionvalue in o
.
-
int PySequence_In(PyObject *o, PyObject *value)¶
- Part of the Stable ABI.
Alias for
PySequence_Contains()
.Deprecated since version 3.14: The function is soft deprecated and should no longer be used to write new code.
-
Py_ssize_t PySequence_Index(PyObject *o, PyObject *value)¶
- Part of the Stable ABI.
Return the first index i for which
o[i] == value
. On error, return-1
. This is equivalent to the Python expressiono.index(value)
.
-
PyObject *PySequence_List(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Return a list object with the same contents as the sequence or iterable o, or
NULL
on failure. The returned list is guaranteed to be new. This is equivalent to the Python expressionlist(o)
.
-
PyObject *PySequence_Tuple(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Return a tuple object with the same contents as the sequence or iterable o, or
NULL
on failure. If o is a tuple, a new reference will be returned, otherwise a tuple will be constructed with the appropriate contents. This is equivalent to the Python expressiontuple(o)
.
-
PyObject *PySequence_Fast(PyObject *o, const char *m)¶
- Return value: New reference. Part of the Stable ABI.
Return the sequence or iterable o as an object usable by the other
PySequence_Fast*
family of functions. If the object is not a sequence or iterable, raisesTypeError
with m as the message text. ReturnsNULL
on failure.The
PySequence_Fast*
functions are thus named because they assume o is aPyTupleObject
or aPyListObject
and access the data fields of o directly.As a CPython implementation detail, if o is already a sequence or list, it will be returned.
-
Py_ssize_t PySequence_Fast_GET_SIZE(PyObject *o)¶
Returns the length of o, assuming that o was returned by
PySequence_Fast()
and that o is notNULL
. The size can also be retrieved by callingPySequence_Size()
on o, butPySequence_Fast_GET_SIZE()
is faster because it can assume o is a list or tuple.
-
PyObject *PySequence_Fast_GET_ITEM(PyObject *o, Py_ssize_t i)¶
- Return value: Borrowed reference.
Return the ith element of o, assuming that o was returned by
PySequence_Fast()
, o is notNULL
, and that i is within bounds.
-
PyObject **PySequence_Fast_ITEMS(PyObject *o)¶
Return the underlying array of PyObject pointers. Assumes that o was returned by
PySequence_Fast()
and o is notNULL
.Note, if a list gets resized, the reallocation may relocate the items array. So, only use the underlying array pointer in contexts where the sequence cannot change.
-
PyObject *PySequence_ITEM(PyObject *o, Py_ssize_t i)¶
- Return value: New reference.
Return the ith element of o or
NULL
on failure. Faster form ofPySequence_GetItem()
but without checking thatPySequence_Check()
on o is true and without adjustment for negative indices.


Input and Output

7. Input and Output¶
There are several ways to present the output of a program; data can be printed in a human-readable form, or written to a file for future use. This chapter will discuss some of the possibilities.
7.1. Fancier Output Formatting¶
So far we’ve encountered two ways of writing values: expression statements and
the print()
function. (A third way is using the write()
method
of file objects; the standard output file can be referenced as sys.stdout
.
See the Library Reference for more information on this.)
Often you’ll want more control over the formatting of your output than simply printing space-separated values. There are several ways to format output.
To use formatted string literals, begin a string with
f
orF
before the opening quotation mark or triple quotation mark. Inside this string, you can write a Python expression between{
and}
characters that can refer to variables or literal values.>>> year = 2016 >>> event = 'Referendum' >>> f'Results of the {year} {event}' 'Results of the 2016 Referendum'
The
str.format()
method of strings requires more manual effort. You’ll still use{
and}
to mark where a variable will be substituted and can provide detailed formatting directives, but you’ll also need to provide the information to be formatted. In the following code block there are two examples of how to format variables:>>> yes_votes = 42_572_654 >>> total_votes = 85_705_149 >>> percentage = yes_votes / total_votes >>> '{:-9} YES votes {:2.2%}'.format(yes_votes, percentage) ' 42572654 YES votes 49.67%'
Notice how the
yes_votes
are padded with spaces and a negative sign only for negative numbers. The example also printspercentage
multiplied by 100, with 2 decimal places and followed by a percent sign (see Format Specification Mini-Language for details).Finally, you can do all the string handling yourself by using string slicing and concatenation operations to create any layout you can imagine. The string type has some methods that perform useful operations for padding strings to a given column width.
When you don’t need fancy output but just want a quick display of some
variables for debugging purposes, you can convert any value to a string with
the repr()
or str()
functions.
The str()
function is meant to return representations of values which are
fairly human-readable, while repr()
is meant to generate representations
which can be read by the interpreter (or will force a SyntaxError
if
there is no equivalent syntax). For objects which don’t have a particular
representation for human consumption, str()
will return the same value as
repr()
. Many values, such as numbers or structures like lists and
dictionaries, have the same representation using either function. Strings, in
particular, have two distinct representations.
Some examples:
>>> s = 'Hello, world.'
>>> str(s)
'Hello, world.'
>>> repr(s)
"'Hello, world.'"
>>> str(1/7)
'0.14285714285714285'
>>> x = 10 * 3.25
>>> y = 200 * 200
>>> s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...'
>>> print(s)
The value of x is 32.5, and y is 40000...
>>> # The repr() of a string adds string quotes and backslashes:
>>> hello = 'hello, world\n'
>>> hellos = repr(hello)
>>> print(hellos)
'hello, world\n'
>>> # The argument to repr() may be any Python object:
>>> repr((x, y, ('spam', 'eggs')))
"(32.5, 40000, ('spam', 'eggs'))"
The string
module contains support for a simple templating approach
based upon regular expressions, via string.Template
.
This offers yet another way to substitute values into strings,
using placeholders like $x
and replacing them with values from a dictionary.
This syntax is easy to use, although it offers much less control for formatting.
7.1.1. Formatted String Literals¶
Formatted string literals (also called f-strings for
short) let you include the value of Python expressions inside a string by
prefixing the string with f
or F
and writing expressions as
{expression}
.
An optional format specifier can follow the expression. This allows greater control over how the value is formatted. The following example rounds pi to three places after the decimal:
>>> import math
>>> print(f'The value of pi is approximately {math.pi:.3f}.')
The value of pi is approximately 3.142.
Passing an integer after the ':'
will cause that field to be a minimum
number of characters wide. This is useful for making columns line up.
>>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678}
>>> for name, phone in table.items():
... print(f'{name:10} ==> {phone:10d}')
...
Sjoerd ==> 4127
Jack ==> 4098
Dcab ==> 7678
Other modifiers can be used to convert the value before it is formatted.
'!a'
applies ascii()
, '!s'
applies str()
, and '!r'
applies repr()
:
>>> animals = 'eels'
>>> print(f'My hovercraft is full of {animals}.')
My hovercraft is full of eels.
>>> print(f'My hovercraft is full of {animals!r}.')
My hovercraft is full of 'eels'.
The =
specifier can be used to expand an expression to the text of the
expression, an equal sign, then the representation of the evaluated expression:
>>> bugs = 'roaches'
>>> count = 13
>>> area = 'living room'
>>> print(f'Debugging {bugs=} {count=} {area=}')
Debugging bugs='roaches' count=13 area='living room'
See self-documenting expressions for more information
on the =
specifier. For a reference on these format specifications, see
the reference guide for the Format Specification Mini-Language.
7.1.2. The String format() Method¶
Basic usage of the str.format()
method looks like this:
>>> print('We are the {} who say "{}!"'.format('knights', 'Ni'))
We are the knights who say "Ni!"
The brackets and characters within them (called format fields) are replaced with
the objects passed into the str.format()
method. A number in the
brackets can be used to refer to the position of the object passed into the
str.format()
method.
>>> print('{0} and {1}'.format('spam', 'eggs'))
spam and eggs
>>> print('{1} and {0}'.format('spam', 'eggs'))
eggs and spam
If keyword arguments are used in the str.format()
method, their values
are referred to by using the name of the argument.
>>> print('This {food} is {adjective}.'.format(
... food='spam', adjective='absolutely horrible'))
This spam is absolutely horrible.
Positional and keyword arguments can be arbitrarily combined:
>>> print('The story of {0}, {1}, and {other}.'.format('Bill', 'Manfred',
... other='Georg'))
The story of Bill, Manfred, and Georg.
If you have a really long format string that you don’t want to split up, it
would be nice if you could reference the variables to be formatted by name
instead of by position. This can be done by simply passing the dict and using
square brackets '[]'
to access the keys.
>>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
>>> print('Jack: {0[Jack]:d}; Sjoerd: {0[Sjoerd]:d}; '
... 'Dcab: {0[Dcab]:d}'.format(table))
Jack: 4098; Sjoerd: 4127; Dcab: 8637678
This could also be done by passing the table
dictionary as keyword arguments with the **
notation.
>>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
>>> print('Jack: {Jack:d}; Sjoerd: {Sjoerd:d}; Dcab: {Dcab:d}'.format(**table))
Jack: 4098; Sjoerd: 4127; Dcab: 8637678
This is particularly useful in combination with the built-in function
vars()
, which returns a dictionary containing all local variables:
>>> table = {k: str(v) for k, v in vars().items()}
>>> message = " ".join([f'{k}: ' + '{' + k +'};' for k in table.keys()])
>>> print(message.format(**table))
__name__: __main__; __doc__: None; __package__: None; __loader__: ...
As an example, the following lines produce a tidily aligned set of columns giving integers and their squares and cubes:
>>> for x in range(1, 11):
... print('{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x))
...
1 1 1
2 4 8
3 9 27
4 16 64
5 25 125
6 36 216
7 49 343
8 64 512
9 81 729
10 100 1000
For a complete overview of string formatting with str.format()
, see
Format String Syntax.
7.1.3. Manual String Formatting¶
Here’s the same table of squares and cubes, formatted manually:
>>> for x in range(1, 11):
... print(repr(x).rjust(2), repr(x*x).rjust(3), end=' ')
... # Note use of 'end' on previous line
... print(repr(x*x*x).rjust(4))
...
1 1 1
2 4 8
3 9 27
4 16 64
5 25 125
6 36 216
7 49 343
8 64 512
9 81 729
10 100 1000
(Note that the one space between each column was added by the
way print()
works: it always adds spaces between its arguments.)
The str.rjust()
method of string objects right-justifies a string in a
field of a given width by padding it with spaces on the left. There are
similar methods str.ljust()
and str.center()
. These methods do
not write anything, they just return a new string. If the input string is too
long, they don’t truncate it, but return it unchanged; this will mess up your
column lay-out but that’s usually better than the alternative, which would be
lying about a value. (If you really want truncation you can always add a
slice operation, as in x.ljust(n)[:n]
.)
There is another method, str.zfill()
, which pads a numeric string on the
left with zeros. It understands about plus and minus signs:
>>> '12'.zfill(5)
'00012'
>>> '-3.14'.zfill(7)
'-003.14'
>>> '3.14159265359'.zfill(5)
'3.14159265359'
7.1.4. Old string formatting¶
The % operator (modulo) can also be used for string formatting.
Given format % values
(where format is a string),
%
conversion specifications in format are replaced with
zero or more elements of values.
This operation is commonly known as string
interpolation. For example:
>>> import math
>>> print('The value of pi is approximately %5.3f.' % math.pi)
The value of pi is approximately 3.142.
More information can be found in the printf-style String Formatting section.
7.2. Reading and Writing Files¶
open()
returns a file object, and is most commonly used with
two positional arguments and one keyword argument:
open(filename, mode, encoding=None)
>>> f = open('workfile', 'w', encoding="utf-8")
The first argument is a string containing the filename. The second argument is
another string containing a few characters describing the way in which the file
will be used. mode can be 'r'
when the file will only be read, 'w'
for only writing (an existing file with the same name will be erased), and
'a'
opens the file for appending; any data written to the file is
automatically added to the end. 'r+'
opens the file for both reading and
writing. The mode argument is optional; 'r'
will be assumed if it’s
omitted.
Normally, files are opened in text mode, that means, you read and write
strings from and to the file, which are encoded in a specific encoding.
If encoding is not specified, the default is platform dependent
(see open()
).
Because UTF-8 is the modern de-facto standard, encoding="utf-8"
is
recommended unless you know that you need to use a different encoding.
Appending a 'b'
to the mode opens the file in binary mode.
Binary mode data is read and written as bytes
objects.
You can not specify encoding when opening file in binary mode.
In text mode, the default when reading is to convert platform-specific line
endings (\n
on Unix, \r\n
on Windows) to just \n
. When writing in
text mode, the default is to convert occurrences of \n
back to
platform-specific line endings. This behind-the-scenes modification
to file data is fine for text files, but will corrupt binary data like that in
JPEG
or EXE
files. Be very careful to use binary mode when
reading and writing such files.
It is good practice to use the with
keyword when dealing
with file objects. The advantage is that the file is properly closed
after its suite finishes, even if an exception is raised at some
point. Using with
is also much shorter than writing
equivalent try
-finally
blocks:
>>> with open('workfile', encoding="utf-8") as f:
... read_data = f.read()
>>> # We can check that the file has been automatically closed.
>>> f.closed
True
If you’re not using the with
keyword, then you should call
f.close()
to close the file and immediately free up any system
resources used by it.
Warning
Calling f.write()
without using the with
keyword or calling
f.close()
might result in the arguments
of f.write()
not being completely written to the disk, even if the
program exits successfully.
After a file object is closed, either by a with
statement
or by calling f.close()
, attempts to use the file object will
automatically fail.
>>> f.close()
>>> f.read()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: I/O operation on closed file.
7.2.1. Methods of File Objects¶
The rest of the examples in this section will assume that a file object called
f
has already been created.
To read a file’s contents, call f.read(size)
, which reads some quantity of
data and returns it as a string (in text mode) or bytes object (in binary mode).
size is an optional numeric argument. When size is omitted or negative, the
entire contents of the file will be read and returned; it’s your problem if the
file is twice as large as your machine’s memory. Otherwise, at most size
characters (in text mode) or size bytes (in binary mode) are read and returned.
If the end of the file has been reached, f.read()
will return an empty
string (''
).
>>> f.read()
'This is the entire file.\n'
>>> f.read()
''
f.readline()
reads a single line from the file; a newline character (\n
)
is left at the end of the string, and is only omitted on the last line of the
file if the file doesn’t end in a newline. This makes the return value
unambiguous; if f.readline()
returns an empty string, the end of the file
has been reached, while a blank line is represented by '\n'
, a string
containing only a single newline.
>>> f.readline()
'This is the first line of the file.\n'
>>> f.readline()
'Second line of the file\n'
>>> f.readline()
''
For reading lines from a file, you can loop over the file object. This is memory efficient, fast, and leads to simple code:
>>> for line in f:
... print(line, end='')
...
This is the first line of the file.
Second line of the file
If you want to read all the lines of a file in a list you can also use
list(f)
or f.readlines()
.
f.write(string)
writes the contents of string to the file, returning
the number of characters written.
>>> f.write('This is a test\n')
15
Other types of objects need to be converted – either to a string (in text mode) or a bytes object (in binary mode) – before writing them:
>>> value = ('the answer', 42)
>>> s = str(value) # convert the tuple to string
>>> f.write(s)
18
f.tell()
returns an integer giving the file object’s current position in the file
represented as number of bytes from the beginning of the file when in binary mode and
an opaque number when in text mode.
To change the file object’s position, use f.seek(offset, whence)
. The position is computed
from adding offset to a reference point; the reference point is selected by
the whence argument. A whence value of 0 measures from the beginning
of the file, 1 uses the current file position, and 2 uses the end of the file as
the reference point. whence can be omitted and defaults to 0, using the
beginning of the file as the reference point.
>>> f = open('workfile', 'rb+')
>>> f.write(b'0123456789abcdef')
16
>>> f.seek(5) # Go to the 6th byte in the file
5
>>> f.read(1)
b'5'
>>> f.seek(-3, 2) # Go to the 3rd byte before the end
13
>>> f.read(1)
b'd'
In text files (those opened without a b
in the mode string), only seeks
relative to the beginning of the file are allowed (the exception being seeking
to the very file end with seek(0, 2)
) and the only valid offset values are
those returned from the f.tell()
, or zero. Any other offset value produces
undefined behaviour.
File objects have some additional methods, such as isatty()
and
truncate()
which are less frequently used; consult the Library
Reference for a complete guide to file objects.
7.2.2. Saving structured data with json
¶
Strings can easily be written to and read from a file. Numbers take a bit more
effort, since the read()
method only returns strings, which will have to
be passed to a function like int()
, which takes a string like '123'
and returns its numeric value 123. When you want to save more complex data
types like nested lists and dictionaries, parsing and serializing by hand
becomes complicated.
Rather than having users constantly writing and debugging code to save
complicated data types to files, Python allows you to use the popular data
interchange format called JSON (JavaScript Object Notation). The standard module called json
can take Python
data hierarchies, and convert them to string representations; this process is
called serializing. Reconstructing the data from the string representation
is called deserializing. Between serializing and deserializing, the
string representing the object may have been stored in a file or data, or
sent over a network connection to some distant machine.
Note
The JSON format is commonly used by modern applications to allow for data exchange. Many programmers are already familiar with it, which makes it a good choice for interoperability.
If you have an object x
, you can view its JSON string representation with a
simple line of code:
>>> import json
>>> x = [1, 'simple', 'list']
>>> json.dumps(x)
'[1, "simple", "list"]'
Another variant of the dumps()
function, called dump()
,
simply serializes the object to a text file. So if f
is a
text file object opened for writing, we can do this:
json.dump(x, f)
To decode the object again, if f
is a binary file or
text file object which has been opened for reading:
x = json.load(f)
Note
JSON files must be encoded in UTF-8. Use encoding="utf-8"
when opening
JSON file as a text file for both of reading and writing.
This simple serialization technique can handle lists and dictionaries, but
serializing arbitrary class instances in JSON requires a bit of extra effort.
The reference for the json
module contains an explanation of this.
See also
pickle
- the pickle module
Contrary to JSON, pickle is a protocol which allows the serialization of arbitrarily complex Python objects. As such, it is specific to Python and cannot be used to communicate with applications written in other languages. It is also insecure by default: deserializing pickle data coming from an untrusted source can execute arbitrary code, if the data was crafted by a skilled attacker.


— Read IFF chunked data

chunk
— Read IFF chunked data¶
Deprecated since version 3.11, removed in version 3.13.
This module is no longer part of the Python standard library. It was removed in Python 3.13 after being deprecated in Python 3.11. The removal was decided in PEP 594.
The last version of Python that provided the chunk
module was
Python 3.12.


— NNTP protocol client

nntplib
— NNTP protocol client¶
Deprecated since version 3.11, removed in version 3.13.
This module is no longer part of the Python standard library. It was removed in Python 3.13 after being deprecated in Python 3.11. The removal was decided in PEP 594.
The last version of Python that provided the nntplib
module was
Python 3.12.