ITookAPill's picture
PyComp First Commit
9273228
|
Raw
History Blame Contribute Delete
140 kB
# What\'s New In Python 3.11
Editor
: Pablo Galindo Salgado
> \* Anyone can add text to this document. Do not spend very much time on the wording of your changes, because your text will probably get rewritten to some degree.
>
> \* The maintainer will go through Misc/NEWS periodically and add changes; it\'s therefore more important to add your changes to Misc/NEWS than to this file.
>
> \* This is not a complete list of every single change; completeness is the purpose of Misc/NEWS. Some changes I consider too small or esoteric to include. If such a change is added to the text, I\'ll just remove it. (This is another reason you shouldn\'t spend too much time on writing your addition.)
>
> \* If you want to draw your new text to the attention of the maintainer, add \'XXX\' to the beginning of the paragraph or section.
>
> \* It\'s OK to just add a fragmentary note about a change. For example: \"XXX Describe the transmogrify() function added to the socket module.\" The maintainer will research the change and write the necessary text.
>
> \* You can comment out your additions if you like, but it\'s not necessary (especially when a final release is some months away).
>
> \* Credit the author of a patch or bugfix. Just the name is sufficient; the e-mail address isn\'t necessary.
>
> - It\'s helpful to add the bug/patch number as a comment:
>
> XXX Describe the transmogrify() function added to the socket module. (Contributed by P.Y. Developer in `12345`{.interpreted-text role="issue"}.)
>
> This saves the maintainer the effort of going through the Mercurial log when researching a change.
This article explains the new features in Python 3.11, compared to 3.10. Python 3.11 was released on October 24, 2022. For full details, see the `changelog <changelog>`{.interpreted-text role="ref"}.
## Summary \-- Release highlights {#whatsnew311-summary}
- Python 3.11 is between 10-60% faster than Python 3.10. On average, we measured a 1.25x speedup on the standard benchmark suite. See `whatsnew311-faster-cpython`{.interpreted-text role="ref"} for details.
New syntax features:
- `whatsnew311-pep654`{.interpreted-text role="ref"}
New built-in features:
- `whatsnew311-pep678`{.interpreted-text role="ref"}
New standard library modules:
- `680`{.interpreted-text role="pep"}: `tomllib`{.interpreted-text role="mod"} --- Support for parsing [TOML](https://toml.io/) in the Standard Library
Interpreter improvements:
- `whatsnew311-pep657`{.interpreted-text role="ref"}
- New `-P`{.interpreted-text role="option"} command line option and `PYTHONSAFEPATH`{.interpreted-text role="envvar"} environment variable to `disable automatically prepending potentially unsafe paths
<whatsnew311-pythonsafepath>`{.interpreted-text role="ref"} to `sys.path`{.interpreted-text role="data"}
New typing features:
- `whatsnew311-pep646`{.interpreted-text role="ref"}
- `whatsnew311-pep655`{.interpreted-text role="ref"}
- `whatsnew311-pep673`{.interpreted-text role="ref"}
- `whatsnew311-pep675`{.interpreted-text role="ref"}
- `whatsnew311-pep681`{.interpreted-text role="ref"}
Important deprecations, removals and restrictions:
- `594`{.interpreted-text role="pep"}: `Many legacy standard library modules have been deprecated
<whatsnew311-pep594>`{.interpreted-text role="ref"} and will be removed in Python 3.13
- `624`{.interpreted-text role="pep"}: `Py_UNICODE encoder APIs have been removed <whatsnew311-pep624>`{.interpreted-text role="ref"}
- `670`{.interpreted-text role="pep"}: `Macros converted to static inline functions <whatsnew311-pep670>`{.interpreted-text role="ref"}
## New Features {#whatsnew311-features}
### PEP 657: Fine-grained error locations in tracebacks {#whatsnew311-pep657}
When printing tracebacks, the interpreter will now point to the exact expression that caused the error, instead of just the line. For example:
``` python
Traceback (most recent call last):
File "distance.py", line 11, in <module>
print(manhattan_distance(p1, p2))
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "distance.py", line 6, in manhattan_distance
return abs(point_1.x - point_2.x) + abs(point_1.y - point_2.y)
^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'x'
```
Previous versions of the interpreter would point to just the line, making it ambiguous which object was `None`. These enhanced errors can also be helpful when dealing with deeply nested `dict`{.interpreted-text role="class"} objects and multiple function calls:
``` python
Traceback (most recent call last):
File "query.py", line 37, in <module>
magic_arithmetic('foo')
File "query.py", line 18, in magic_arithmetic
return add_counts(x) / 25
^^^^^^^^^^^^^
File "query.py", line 24, in add_counts
return 25 + query_user(user1) + query_user(user2)
^^^^^^^^^^^^^^^^^
File "query.py", line 32, in query_user
return 1 + query_count(db, response['a']['b']['c']['user'], retry=True)
~~~~~~~~~~~~~~~~~~^^^^^
TypeError: 'NoneType' object is not subscriptable
```
As well as complex arithmetic expressions:
``` python
Traceback (most recent call last):
File "calculation.py", line 54, in <module>
result = (x / y / z) * (a / b / c)
~~~~~~^~~
ZeroDivisionError: division by zero
```
Additionally, the information used by the enhanced traceback feature is made available via a general API, that can be used to correlate `bytecode`{.interpreted-text role="term"} `instructions <bytecodes>`{.interpreted-text role="ref"} with source code location. This information can be retrieved using:
- The `codeobject.co_positions`{.interpreted-text role="meth"} method in Python.
- The `PyCode_Addr2Location`{.interpreted-text role="c:func"} function in the C API.
See `657`{.interpreted-text role="pep"} for more details. (Contributed by Pablo Galindo, Batuhan Taskaya and Ammar Askar in `43950`{.interpreted-text role="issue"}.)
:::: note
::: title
Note
:::
This feature requires storing column positions in `codeobjects`{.interpreted-text role="ref"}, which may result in a small increase in interpreter memory usage and disk usage for compiled Python files. To avoid storing the extra information and deactivate printing the extra traceback information, use the `-X no_debug_ranges <-X>`{.interpreted-text role="option"} command line option or the `PYTHONNODEBUGRANGES`{.interpreted-text role="envvar"} environment variable.
::::
### PEP 654: Exception Groups and `except*` {#whatsnew311-pep654}
`654`{.interpreted-text role="pep"} introduces language features that enable a program to raise and handle multiple unrelated exceptions simultaneously. The builtin types `ExceptionGroup`{.interpreted-text role="exc"} and `BaseExceptionGroup`{.interpreted-text role="exc"} make it possible to group exceptions and raise them together, and the new `except* <except_star>`{.interpreted-text role="keyword"} syntax generalizes `except`{.interpreted-text role="keyword"} to match subgroups of exception groups.
See `654`{.interpreted-text role="pep"} for more details.
(Contributed by Irit Katriel in `45292`{.interpreted-text role="issue"}. PEP written by Irit Katriel, Yury Selivanov and Guido van Rossum.)
### PEP 678: Exceptions can be enriched with notes {#whatsnew311-pep678}
The `~BaseException.add_note`{.interpreted-text role="meth"} method is added to `BaseException`{.interpreted-text role="exc"}. It can be used to enrich exceptions with context information that is not available at the time when the exception is raised. The added notes appear in the default traceback.
See `678`{.interpreted-text role="pep"} for more details.
(Contributed by Irit Katriel in `45607`{.interpreted-text role="issue"}. PEP written by Zac Hatfield-Dodds.)
### Windows `py.exe` launcher improvements {#whatsnew311-windows-launcher}
The copy of the `launcher`{.interpreted-text role="ref"} included with Python 3.11 has been significantly updated. It now supports company/tag syntax as defined in `514`{.interpreted-text role="pep"} using the `-V:{<company>}/{<tag>}`{.interpreted-text role="samp"} argument instead of the limited `-{<major>}.{<minor>}`{.interpreted-text role="samp"}. This allows launching distributions other than `PythonCore`, the one hosted on [python.org](https://www.python.org).
When using `-V:` selectors, either company or tag can be omitted, but all installs will be searched. For example, `-V:OtherPython/` will select the \"best\" tag registered for `OtherPython`, while `-V:3.11` or `-V:/3.11` will select the \"best\" distribution with tag `3.11`.
When using the legacy `-{<major>}`{.interpreted-text role="samp"}, `-{<major>}.{<minor>}`{.interpreted-text role="samp"}, `-{<major>}-{<bitness>}`{.interpreted-text role="samp"} or `-{<major>}.{<minor>}-{<bitness>}`{.interpreted-text role="samp"} arguments, all existing behaviour should be preserved from past versions, and only releases from `PythonCore` will be selected. However, the `-64` suffix now implies \"not 32-bit\" (not necessarily x86-64), as there are multiple supported 64-bit platforms. 32-bit runtimes are detected by checking the runtime\'s tag for a `-32` suffix. All releases of Python since 3.5 have included this in their 32-bit builds.
## New Features Related to Type Hints[]{#new-feat-related-type-hints-311} {#whatsnew311-typing-features}
This section covers major changes affecting `484`{.interpreted-text role="pep"} type hints and the `typing`{.interpreted-text role="mod"} module.
### PEP 646: Variadic generics {#whatsnew311-pep646}
`484`{.interpreted-text role="pep"} previously introduced `~typing.TypeVar`{.interpreted-text role="data"}, enabling creation of generics parameterised with a single type. `646`{.interpreted-text role="pep"} adds `~typing.TypeVarTuple`{.interpreted-text role="data"}, enabling parameterisation with an *arbitrary* number of types. In other words, a `~typing.TypeVarTuple`{.interpreted-text role="data"} is a *variadic* type variable, enabling *variadic* generics.
This enables a wide variety of use cases. In particular, it allows the type of array-like structures in numerical computing libraries such as NumPy and TensorFlow to be parameterised with the array *shape*. Static type checkers will now be able to catch shape-related bugs in code that uses these libraries.
See `646`{.interpreted-text role="pep"} for more details.
(Contributed by Matthew Rahtz in `43224`{.interpreted-text role="issue"}, with contributions by Serhiy Storchaka and Jelle Zijlstra. PEP written by Mark Mendoza, Matthew Rahtz, Pradeep Kumar Srinivasan, and Vincent Siles.)
### PEP 655: Marking individual `TypedDict` items as required or not-required {#whatsnew311-pep655}
`~typing.Required`{.interpreted-text role="data"} and `~typing.NotRequired`{.interpreted-text role="data"} provide a straightforward way to mark whether individual items in a `~typing.TypedDict`{.interpreted-text role="class"} must be present. Previously, this was only possible using inheritance.
All fields are still required by default, unless the *total* parameter is set to `False`, in which case all fields are still not-required by default. For example, the following specifies a `!TypedDict`{.interpreted-text role="class"} with one required and one not-required key:
class Movie(TypedDict):
title: str
year: NotRequired[int]
m1: Movie = {"title": "Black Panther", "year": 2018} # OK
m2: Movie = {"title": "Star Wars"} # OK (year is not required)
m3: Movie = {"year": 2022} # ERROR (missing required field title)
The following definition is equivalent:
class Movie(TypedDict, total=False):
title: Required[str]
year: int
See `655`{.interpreted-text role="pep"} for more details.
(Contributed by David Foster and Jelle Zijlstra in `47087`{.interpreted-text role="issue"}. PEP written by David Foster.)
### PEP 673: `Self` type {#whatsnew311-pep673}
The new `~typing.Self`{.interpreted-text role="data"} annotation provides a simple and intuitive way to annotate methods that return an instance of their class. This behaves the same as the `~typing.TypeVar`{.interpreted-text role="class"}-based approach `specified in PEP 484 <484#annotating-instance-and-class-methods>`{.interpreted-text role="pep"}, but is more concise and easier to follow.
Common use cases include alternative constructors provided as `classmethod <classmethod>`{.interpreted-text role="func"}s, and `~object.__enter__`{.interpreted-text role="meth"} methods that return `self`:
class MyLock:
def __enter__(self) -> Self:
self.lock()
return self
...
class MyInt:
@classmethod
def fromhex(cls, s: str) -> Self:
return cls(int(s, 16))
...
`~typing.Self`{.interpreted-text role="data"} can also be used to annotate method parameters or attributes of the same type as their enclosing class.
See `673`{.interpreted-text role="pep"} for more details.
(Contributed by James Hilton-Balfe in `46534`{.interpreted-text role="issue"}. PEP written by Pradeep Kumar Srinivasan and James Hilton-Balfe.)
### PEP 675: Arbitrary literal string type {#whatsnew311-pep675}
The new `~typing.LiteralString`{.interpreted-text role="data"} annotation may be used to indicate that a function parameter can be of any literal string type. This allows a function to accept arbitrary literal string types, as well as strings created from other literal strings. Type checkers can then enforce that sensitive functions, such as those that execute SQL statements or shell commands, are called only with static arguments, providing protection against injection attacks.
For example, a SQL query function could be annotated as follows:
def run_query(sql: LiteralString) -> ...
...
def caller(
arbitrary_string: str,
query_string: LiteralString,
table_name: LiteralString,
) -> None:
run_query("SELECT * FROM students") # ok
run_query(query_string) # ok
run_query("SELECT * FROM " + table_name) # ok
run_query(arbitrary_string) # type checker error
run_query( # type checker error
f"SELECT * FROM students WHERE name = {arbitrary_string}"
)
See `675`{.interpreted-text role="pep"} for more details.
(Contributed by Jelle Zijlstra in `47088`{.interpreted-text role="issue"}. PEP written by Pradeep Kumar Srinivasan and Graham Bleaney.)
### PEP 681: Data class transforms {#whatsnew311-pep681}
`~typing.dataclass_transform`{.interpreted-text role="data"} may be used to decorate a class, metaclass, or a function that is itself a decorator. The presence of `@dataclass_transform()` tells a static type checker that the decorated object performs runtime \"magic\" that transforms a class, giving it `dataclass <dataclasses.dataclass>`{.interpreted-text role="func"}-like behaviors.
For example:
# The create_model decorator is defined by a library.
@typing.dataclass_transform()
def create_model(cls: Type[T]) -> Type[T]:
cls.__init__ = ...
cls.__eq__ = ...
cls.__ne__ = ...
return cls
# The create_model decorator can now be used to create new model classes:
@create_model
class CustomerModel:
id: int
name: str
c = CustomerModel(id=327, name="Eric Idle")
See `681`{.interpreted-text role="pep"} for more details.
(Contributed by Jelle Zijlstra in `91860`{.interpreted-text role="gh"}. PEP written by Erik De Bonte and Eric Traut.)
### PEP 563 may not be the future {#whatsnew311-pep563-deferred}
`563`{.interpreted-text role="pep"} Postponed Evaluation of Annotations (the `from __future__ import annotations` `future statement <future>`{.interpreted-text role="ref"}) that was originally planned for release in Python 3.10 has been put on hold indefinitely. See [this message from the Steering Council](https://mail.python.org/archives/list/python-dev@python.org/message/VIZEBX5EYMSYIJNDBF6DMUMZOCWHARSO/) for more information.
## Other Language Changes {#whatsnew311-other-lang-changes}
- Starred unpacking expressions can now be used in `for`{.interpreted-text role="keyword"} statements. (See `46725`{.interpreted-text role="issue"} for more details.)
- Asynchronous `comprehensions <comprehensions>`{.interpreted-text role="ref"} are now allowed inside comprehensions in `asynchronous functions <async def>`{.interpreted-text role="ref"}. Outer comprehensions implicitly become asynchronous in this case. (Contributed by Serhiy Storchaka in `33346`{.interpreted-text role="issue"}.)
- A `TypeError`{.interpreted-text role="exc"} is now raised instead of an `AttributeError`{.interpreted-text role="exc"} in `with`{.interpreted-text role="keyword"} statements and `contextlib.ExitStack.enter_context`{.interpreted-text role="meth"} for objects that do not support the `context manager`{.interpreted-text role="term"} protocol, and in `async with`{.interpreted-text role="keyword"} statements and `contextlib.AsyncExitStack.enter_async_context`{.interpreted-text role="meth"} for objects not supporting the `asynchronous context manager`{.interpreted-text role="term"} protocol. (Contributed by Serhiy Storchaka in `12022`{.interpreted-text role="issue"} and `44471`{.interpreted-text role="issue"}.)
- Added `object.__getstate__`{.interpreted-text role="meth"}, which provides the default implementation of the `!__getstate__`{.interpreted-text role="meth"} method. `copy`{.interpreted-text role="mod"}ing and `pickle`{.interpreted-text role="mod"}ing instances of subclasses of builtin types `bytearray`{.interpreted-text role="class"}, `set`{.interpreted-text role="class"}, `frozenset`{.interpreted-text role="class"}, `collections.OrderedDict`{.interpreted-text role="class"}, `collections.deque`{.interpreted-text role="class"}, `weakref.WeakSet`{.interpreted-text role="class"}, and `datetime.tzinfo`{.interpreted-text role="class"} now copies and pickles instance attributes implemented as `slots <__slots__>`{.interpreted-text role="term"}. This change has an unintended side effect: It trips up a small minority of existing Python projects not expecting `object.__getstate__`{.interpreted-text role="meth"} to exist. See the later comments on `70766`{.interpreted-text role="gh"} for discussions of what workarounds such code may need. (Contributed by Serhiy Storchaka in `26579`{.interpreted-text role="issue"}.)
::: {#whatsnew311-pythonsafepath}
- Added a `-P`{.interpreted-text role="option"} command line option and a `PYTHONSAFEPATH`{.interpreted-text role="envvar"} environment variable, which disable the automatic prepending to `sys.path`{.interpreted-text role="data"} of the script\'s directory when running a script, or the current directory when using `-c`{.interpreted-text role="option"} and `-m`{.interpreted-text role="option"}. This ensures only stdlib and installed modules are picked up by `import`{.interpreted-text role="keyword"}, and avoids unintentionally or maliciously shadowing modules with those in a local (and typically user-writable) directory. (Contributed by Victor Stinner in `57684`{.interpreted-text role="gh"}.)
- A `"z"` option was added to the `formatspec`{.interpreted-text role="ref"} that coerces negative to positive zero after rounding to the format precision. See `682`{.interpreted-text role="pep"} for more details. (Contributed by John Belmonte in `90153`{.interpreted-text role="gh"}.)
- Bytes are no longer accepted on `sys.path`{.interpreted-text role="data"}. Support broke sometime between Python 3.2 and 3.6, with no one noticing until after Python 3.10.0 was released. In addition, bringing back support would be problematic due to interactions between `-b`{.interpreted-text role="option"} and `sys.path_importer_cache`{.interpreted-text role="data"} when there is a mixture of `str`{.interpreted-text role="class"} and `bytes`{.interpreted-text role="class"} keys. (Contributed by Thomas Grainger in `91181`{.interpreted-text role="gh"}.)
:::
## Other CPython Implementation Changes {#whatsnew311-other-implementation-changes}
- The special methods `~object.__complex__`{.interpreted-text role="meth"} for `complex`{.interpreted-text role="class"} and `~object.__bytes__`{.interpreted-text role="meth"} for `bytes`{.interpreted-text role="class"} are implemented to support the `typing.SupportsComplex`{.interpreted-text role="class"} and `typing.SupportsBytes`{.interpreted-text role="class"} protocols. (Contributed by Mark Dickinson and Donghee Na in `24234`{.interpreted-text role="issue"}.)
- `siphash13` is added as a new internal hashing algorithm. It has similar security properties as `siphash24`, but it is slightly faster for long inputs. `str`{.interpreted-text role="class"}, `bytes`{.interpreted-text role="class"}, and some other types now use it as the default algorithm for `hash`{.interpreted-text role="func"}. `552`{.interpreted-text role="pep"} `hash-based .pyc files <pyc-invalidation>`{.interpreted-text role="ref"} now use `siphash13` too. (Contributed by Inada Naoki in `29410`{.interpreted-text role="issue"}.)
- When an active exception is re-raised by a `raise`{.interpreted-text role="keyword"} statement with no parameters, the traceback attached to this exception is now always `sys.exc_info()[1].__traceback__`. This means that changes made to the traceback in the current `except`{.interpreted-text role="keyword"} clause are reflected in the re-raised exception. (Contributed by Irit Katriel in `45711`{.interpreted-text role="issue"}.)
- The interpreter state\'s representation of handled exceptions (aka `exc_info` or `_PyErr_StackItem`) now only has the `exc_value` field; `exc_type` and `exc_traceback` have been removed, as they can be derived from `exc_value`. (Contributed by Irit Katriel in `45711`{.interpreted-text role="issue"}.)
- A new `command line option <install-quiet-option>`{.interpreted-text role="ref"}, `AppendPath`, has been added for the Windows installer. It behaves similarly to `PrependPath`, but appends the install and scripts directories instead of prepending them. (Contributed by Bastian Neuburger in `44934`{.interpreted-text role="issue"}.)
- The `PyConfig.module_search_paths_set`{.interpreted-text role="c:member"} field must now be set to `1` for initialization to use `PyConfig.module_search_paths`{.interpreted-text role="c:member"} to initialize `sys.path`{.interpreted-text role="data"}. Otherwise, initialization will recalculate the path and replace any values added to `module_search_paths`.
- The output of the `--help`{.interpreted-text role="option"} option now fits in 50 lines/80 columns. Information about `Python environment variables <using-on-envvars>`{.interpreted-text role="ref"} and `-X`{.interpreted-text role="option"} options is now available using the respective `--help-env`{.interpreted-text role="option"} and `--help-xoptions`{.interpreted-text role="option"} flags, and with the new `--help-all`{.interpreted-text role="option"}. (Contributed by Éric Araujo in `46142`{.interpreted-text role="issue"}.)
- Converting between `int`{.interpreted-text role="class"} and `str`{.interpreted-text role="class"} in bases other than 2 (binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal) now raises a `ValueError`{.interpreted-text role="exc"} if the number of digits in string form is above a limit to avoid potential denial of service attacks due to the algorithmic complexity. This is a mitigation for `2020-10735`{.interpreted-text role="cve"}. This limit can be configured or disabled by environment variable, command line flag, or `sys`{.interpreted-text role="mod"} APIs. See the `integer string conversion
length limitation <int_max_str_digits>`{.interpreted-text role="ref"} documentation. The default limit is 4300 digits in string form.
## New Modules {#whatsnew311-new-modules}
- `tomllib`{.interpreted-text role="mod"}: For parsing [TOML](https://toml.io/). See `680`{.interpreted-text role="pep"} for more details. (Contributed by Taneli Hukkinen in `40059`{.interpreted-text role="issue"}.)
- `wsgiref.types`{.interpreted-text role="mod"}: `WSGI <3333>`{.interpreted-text role="pep"}-specific types for static type checking. (Contributed by Sebastian Rittau in `42012`{.interpreted-text role="issue"}.)
## Improved Modules {#whatsnew311-improved-modules}
### asyncio {#whatsnew311-asyncio}
- Added the `~asyncio.TaskGroup`{.interpreted-text role="class"} class, an `asynchronous context manager <async-context-managers>`{.interpreted-text role="ref"} holding a group of tasks that will wait for all of them upon exit. For new code this is recommended over using `~asyncio.create_task`{.interpreted-text role="func"} and `~asyncio.gather`{.interpreted-text role="func"} directly. (Contributed by Yury Selivanov and others in `90908`{.interpreted-text role="gh"}.)
- Added `~asyncio.timeout`{.interpreted-text role="func"}, an asynchronous context manager for setting a timeout on asynchronous operations. For new code this is recommended over using `~asyncio.wait_for`{.interpreted-text role="func"} directly. (Contributed by Andrew Svetlov in `90927`{.interpreted-text role="gh"}.)
- Added the `~asyncio.Runner`{.interpreted-text role="class"} class, which exposes the machinery used by `~asyncio.run`{.interpreted-text role="func"}. (Contributed by Andrew Svetlov in `91218`{.interpreted-text role="gh"}.)
- Added the `~asyncio.Barrier`{.interpreted-text role="class"} class to the synchronization primitives in the asyncio library, and the related `~asyncio.BrokenBarrierError`{.interpreted-text role="exc"} exception. (Contributed by Yves Duprat and Andrew Svetlov in `87518`{.interpreted-text role="gh"}.)
- Added keyword argument *all_errors* to `asyncio.loop.create_connection`{.interpreted-text role="meth"} so that multiple connection errors can be raised as an `ExceptionGroup`{.interpreted-text role="exc"}.
- Added the `asyncio.StreamWriter.start_tls`{.interpreted-text role="meth"} method for upgrading existing stream-based connections to TLS. (Contributed by Ian Good in `34975`{.interpreted-text role="issue"}.)
- Added raw datagram socket functions to the event loop: `~asyncio.loop.sock_sendto`{.interpreted-text role="meth"}, `~asyncio.loop.sock_recvfrom`{.interpreted-text role="meth"} and `~asyncio.loop.sock_recvfrom_into`{.interpreted-text role="meth"}. These have implementations in `~asyncio.SelectorEventLoop`{.interpreted-text role="class"} and `~asyncio.ProactorEventLoop`{.interpreted-text role="class"}. (Contributed by Alex Grönholm in `46805`{.interpreted-text role="issue"}.)
- Added `~asyncio.Task.cancelling`{.interpreted-text role="meth"} and `~asyncio.Task.uncancel`{.interpreted-text role="meth"} methods to `~asyncio.Task`{.interpreted-text role="class"}. These are primarily intended for internal use, notably by `~asyncio.TaskGroup`{.interpreted-text role="class"}.
### contextlib {#whatsnew311-contextlib}
- Added non parallel-safe `~contextlib.chdir`{.interpreted-text role="func"} context manager to change the current working directory and then restore it on exit. Simple wrapper around `~os.chdir`{.interpreted-text role="func"}. (Contributed by Filipe Laíns in `25625`{.interpreted-text role="issue"})
### dataclasses {#whatsnew311-dataclasses}
- Change field default mutability check, allowing only defaults which are `hashable`{.interpreted-text role="term"} instead of any object which is not an instance of `dict`{.interpreted-text role="class"}, `list`{.interpreted-text role="class"} or `set`{.interpreted-text role="class"}. (Contributed by Eric V. Smith in `44674`{.interpreted-text role="issue"}.)
### datetime {#whatsnew311-datetime}
- Add `datetime.UTC`{.interpreted-text role="const"}, a convenience alias for `datetime.timezone.utc`{.interpreted-text role="attr"}. (Contributed by Kabir Kwatra in `91973`{.interpreted-text role="gh"}.)
- `datetime.date.fromisoformat`{.interpreted-text role="meth"}, `datetime.time.fromisoformat`{.interpreted-text role="meth"} and `datetime.datetime.fromisoformat`{.interpreted-text role="meth"} can now be used to parse most ISO 8601 formats (barring only those that support fractional hours and minutes). (Contributed by Paul Ganssle in `80010`{.interpreted-text role="gh"}.)
### enum {#whatsnew311-enum}
- Renamed `!EnumMeta`{.interpreted-text role="class"} to `~enum.EnumType`{.interpreted-text role="class"} (`!EnumMeta`{.interpreted-text role="class"} kept as an alias).
- Added `~enum.StrEnum`{.interpreted-text role="class"}, with members that can be used as (and must be) strings.
- Added `~enum.ReprEnum`{.interpreted-text role="class"}, which only modifies the `~object.__repr__`{.interpreted-text role="meth"} of members while returning their literal values (rather than names) for `~object.__str__`{.interpreted-text role="meth"} and `~object.__format__`{.interpreted-text role="meth"} (used by `str`{.interpreted-text role="func"}, `format`{.interpreted-text role="func"} and `f-string`{.interpreted-text role="term"}s).
- Changed `Enum.__format__() <enum.Enum.__format__>`{.interpreted-text role="meth"} (the default for `format`{.interpreted-text role="func"}, `str.format`{.interpreted-text role="meth"} and `f-string`{.interpreted-text role="term"}s) to always produce the same result as `Enum.__str__() <enum.Enum.__str__>`{.interpreted-text role="meth"}: for enums inheriting from `~enum.ReprEnum`{.interpreted-text role="class"} it will be the member\'s value; for all other enums it will be the enum and member name (e.g. `Color.RED`).
- Added a new *boundary* class parameter to `~enum.Flag`{.interpreted-text role="class"} enums and the `~enum.FlagBoundary`{.interpreted-text role="class"} enum with its options, to control how to handle out-of-range flag values.
- Added the `~enum.verify`{.interpreted-text role="func"} enum decorator and the `~enum.EnumCheck`{.interpreted-text role="class"} enum with its options, to check enum classes against several specific constraints.
- Added the `~enum.member`{.interpreted-text role="func"} and `~enum.nonmember`{.interpreted-text role="func"} decorators, to ensure the decorated object is/is not converted to an enum member.
- Added the `~enum.property`{.interpreted-text role="func"} decorator, which works like `property`{.interpreted-text role="func"} except for enums. Use this instead of `types.DynamicClassAttribute`{.interpreted-text role="func"}.
- Added the `~enum.global_enum`{.interpreted-text role="func"} enum decorator, which adjusts `~object.__repr__`{.interpreted-text role="meth"} and `~object.__str__`{.interpreted-text role="meth"} to show values as members of their module rather than the enum class. For example, `'re.ASCII'` for the `~re.ASCII`{.interpreted-text role="const"} member of `re.RegexFlag`{.interpreted-text role="class"} rather than `'RegexFlag.ASCII'`.
- Enhanced `~enum.Flag`{.interpreted-text role="class"} to support `len`{.interpreted-text role="func"}, iteration and `in`{.interpreted-text role="keyword"}/`not in`{.interpreted-text role="keyword"} on its members. For example, the following now works: `len(AFlag(3)) == 2 and list(AFlag(3)) == (AFlag.ONE, AFlag.TWO)`
- Changed `~enum.Enum`{.interpreted-text role="class"} and `~enum.Flag`{.interpreted-text role="class"} so that members are now defined before `~object.__init_subclass__`{.interpreted-text role="meth"} is called; `dir`{.interpreted-text role="func"} now includes methods, etc., from mixed-in data types.
- Changed `~enum.Flag`{.interpreted-text role="class"} to only consider primary values (power of two) canonical while composite values (`3`, `6`, `10`, etc.) are considered aliases; inverted flags are coerced to their positive equivalent.
### fcntl {#whatsnew311-fcntl}
- On FreeBSD, the `!F_DUP2FD`{.interpreted-text role="data"} and `!F_DUP2FD_CLOEXEC`{.interpreted-text role="data"} flags respectively are supported, the former equals to `dup2` usage while the latter set the `FD_CLOEXEC` flag in addition.
### fractions {#whatsnew311-fractions}
- Support [PEP 515](http://www.python.org/dev/peps/pep-0515/ "PEP 515")-style initialization of `~fractions.Fraction`{.interpreted-text role="class"} from string. (Contributed by Sergey B Kirpichev in `44258`{.interpreted-text role="issue"}.)
- `~fractions.Fraction`{.interpreted-text role="class"} now implements an `__int__` method, so that an `isinstance(some_fraction, typing.SupportsInt)` check passes. (Contributed by Mark Dickinson in `44547`{.interpreted-text role="issue"}.)
### functools {#whatsnew311-functools}
- `functools.singledispatch`{.interpreted-text role="func"} now supports `types.UnionType`{.interpreted-text role="class"} and `typing.Union`{.interpreted-text role="class"} as annotations to the dispatch argument.:
>>> from functools import singledispatch
>>> @singledispatch
... def fun(arg, verbose=False):
... if verbose:
... print("Let me just say,", end=" ")
... print(arg)
...
>>> @fun.register
... def _(arg: int | float, verbose=False):
... if verbose:
... print("Strength in numbers, eh?", end=" ")
... print(arg)
...
>>> from typing import Union
>>> @fun.register
... def _(arg: Union[list, set], verbose=False):
... if verbose:
... print("Enumerate this:")
... for i, elem in enumerate(arg):
... print(i, elem)
...
(Contributed by Yurii Karabas in `46014`{.interpreted-text role="issue"}.)
### gzip {#whatsnew311-gzip}
- The `gzip.compress`{.interpreted-text role="func"} function is now faster when used with the **mtime=0** argument as it delegates the compression entirely to a single `zlib.compress`{.interpreted-text role="func"} operation. There is one side effect of this change: The gzip file header contains an \"OS\" byte in its header. That was traditionally always set to a value of 255 representing \"unknown\" by the `gzip`{.interpreted-text role="mod"} module. Now, when using `~gzip.compress`{.interpreted-text role="func"} with **mtime=0**, it may be set to a different value by the underlying zlib C library Python was linked against. (See `112346`{.interpreted-text role="gh"} for details on the side effect.)
### hashlib {#whatsnew311-hashlib}
- `hashlib.blake2b`{.interpreted-text role="func"} and `hashlib.blake2s`{.interpreted-text role="func"} now prefer [libb2](https://www.blake2.net/) over Python\'s vendored copy. (Contributed by Christian Heimes in `47095`{.interpreted-text role="issue"}.)
- The internal `_sha3` module with SHA3 and SHAKE algorithms now uses *tiny_sha3* instead of the *Keccak Code Package* to reduce code and binary size. The `hashlib`{.interpreted-text role="mod"} module prefers optimized SHA3 and SHAKE implementations from OpenSSL. The change affects only installations without OpenSSL support. (Contributed by Christian Heimes in `47098`{.interpreted-text role="issue"}.)
- Add `hashlib.file_digest`{.interpreted-text role="func"}, a helper function for efficient hashing of files or file-like objects. (Contributed by Christian Heimes in `89313`{.interpreted-text role="gh"}.)
### IDLE and idlelib {#whatsnew311-idle}
- Apply syntax highlighting to `.pyi` files. (Contributed by Alex Waygood and Terry Jan Reedy in `45447`{.interpreted-text role="issue"}.)
- Include prompts when saving Shell with inputs and outputs. (Contributed by Terry Jan Reedy in `95191`{.interpreted-text role="gh"}.)
### inspect {#whatsnew311-inspect}
- Add `~inspect.getmembers_static`{.interpreted-text role="func"} to return all members without triggering dynamic lookup via the descriptor protocol. (Contributed by Weipeng Hong in `30533`{.interpreted-text role="issue"}.)
- Add `~inspect.ismethodwrapper`{.interpreted-text role="func"} for checking if the type of an object is a `~types.MethodWrapperType`{.interpreted-text role="class"}. (Contributed by Hakan Çelik in `29418`{.interpreted-text role="issue"}.)
- Change the frame-related functions in the `inspect`{.interpreted-text role="mod"} module to return new `~inspect.FrameInfo`{.interpreted-text role="class"} and `~inspect.Traceback`{.interpreted-text role="class"} class instances (backwards compatible with the previous `named tuple`{.interpreted-text role="term"}-like interfaces) that includes the extended `657`{.interpreted-text role="pep"} position information (end line number, column and end column). The affected functions are:
- `inspect.getframeinfo`{.interpreted-text role="func"}
- `inspect.getouterframes`{.interpreted-text role="func"}
- `inspect.getinnerframes`{.interpreted-text role="func"},
- `inspect.stack`{.interpreted-text role="func"}
- `inspect.trace`{.interpreted-text role="func"}
(Contributed by Pablo Galindo in `88116`{.interpreted-text role="gh"}.)
### locale {#whatsnew311-locale}
- Add `locale.getencoding`{.interpreted-text role="func"} to get the current locale encoding. It is similar to `locale.getpreferredencoding(False)` but ignores the `Python UTF-8 Mode <utf8-mode>`{.interpreted-text role="ref"}.
### logging {#whatsnew311-logging}
- Added `~logging.getLevelNamesMapping`{.interpreted-text role="func"} to return a mapping from logging level names (e.g. `'CRITICAL'`) to the values of their corresponding `levels`{.interpreted-text role="ref"} (e.g. `50`, by default). (Contributed by Andrei Kulakovin in `88024`{.interpreted-text role="gh"}.)
- Added a `~logging.handlers.SysLogHandler.createSocket`{.interpreted-text role="meth"} method to `~logging.handlers.SysLogHandler`{.interpreted-text role="class"}, to match `SocketHandler.createSocket()
<logging.handlers.SocketHandler.createSocket>`{.interpreted-text role="meth"}. It is called automatically during handler initialization and when emitting an event, if there is no active socket. (Contributed by Kirill Pinchuk in `88457`{.interpreted-text role="gh"}.)
### math {#whatsnew311-math}
- Add `math.exp2`{.interpreted-text role="func"}: return 2 raised to the power of x. (Contributed by Gideon Mitchell in `45917`{.interpreted-text role="issue"}.)
- Add `math.cbrt`{.interpreted-text role="func"}: return the cube root of x. (Contributed by Ajith Ramachandran in `44357`{.interpreted-text role="issue"}.)
- The behaviour of two `math.pow`{.interpreted-text role="func"} corner cases was changed, for consistency with the IEEE 754 specification. The operations `math.pow(0.0, -math.inf)` and `math.pow(-0.0, -math.inf)` now return `inf`. Previously they raised `ValueError`{.interpreted-text role="exc"}. (Contributed by Mark Dickinson in `44339`{.interpreted-text role="issue"}.)
- The `math.nan`{.interpreted-text role="data"} value is now always available. (Contributed by Victor Stinner in `46917`{.interpreted-text role="issue"}.)
### operator {#whatsnew311-operator}
- A new function `operator.call` has been added, such that `operator.call(obj, *args, **kwargs) == obj(*args, **kwargs)`. (Contributed by Antony Lee in `44019`{.interpreted-text role="issue"}.)
### os {#whatsnew311-os}
- On Windows, `os.urandom`{.interpreted-text role="func"} now uses `BCryptGenRandom()`, instead of `CryptGenRandom()` which is deprecated. (Contributed by Donghee Na in `44611`{.interpreted-text role="issue"}.)
### pathlib {#whatsnew311-pathlib}
- `~pathlib.Path.glob`{.interpreted-text role="meth"} and `~pathlib.Path.rglob`{.interpreted-text role="meth"} return only directories if *pattern* ends with a pathname components separator: `~os.sep`{.interpreted-text role="data"} or `~os.altsep`{.interpreted-text role="data"}. (Contributed by Eisuke Kawasima in `22276`{.interpreted-text role="issue"} and `33392`{.interpreted-text role="issue"}.)
### re {#whatsnew311-re}
- Atomic grouping (`(?>...)`) and possessive quantifiers (`*+`, `++`, `?+`, `{m,n}+`) are now supported in regular expressions. (Contributed by Jeffrey C. Jacobs and Serhiy Storchaka in `433030`{.interpreted-text role="issue"}.)
### shutil {#whatsnew311-shutil}
- Add optional parameter *dir_fd* in `shutil.rmtree`{.interpreted-text role="func"}. (Contributed by Serhiy Storchaka in `46245`{.interpreted-text role="issue"}.)
### socket {#whatsnew311-socket}
- Add CAN Socket support for NetBSD. (Contributed by Thomas Klausner in `30512`{.interpreted-text role="issue"}.)
- `~socket.create_connection`{.interpreted-text role="meth"} has an option to raise, in case of failure to connect, an `ExceptionGroup`{.interpreted-text role="exc"} containing all errors instead of only raising the last error. (Contributed by Irit Katriel in `29980`{.interpreted-text role="issue"}.)
### sqlite3 {#whatsnew311-sqlite3}
- You can now disable the authorizer by passing `None`{.interpreted-text role="const"} to `~sqlite3.Connection.set_authorizer`{.interpreted-text role="meth"}. (Contributed by Erlend E. Aasland in `44491`{.interpreted-text role="issue"}.)
- Collation name `~sqlite3.Connection.create_collation`{.interpreted-text role="meth"} can now contain any Unicode character. Collation names with invalid characters now raise `UnicodeEncodeError`{.interpreted-text role="exc"} instead of `sqlite3.ProgrammingError`{.interpreted-text role="exc"}. (Contributed by Erlend E. Aasland in `44688`{.interpreted-text role="issue"}.)
- `sqlite3`{.interpreted-text role="mod"} exceptions now include the SQLite extended error code as `~sqlite3.Error.sqlite_errorcode`{.interpreted-text role="attr"} and the SQLite error name as `~sqlite3.Error.sqlite_errorname`{.interpreted-text role="attr"}. (Contributed by Aviv Palivoda, Daniel Shahaf, and Erlend E. Aasland in `16379`{.interpreted-text role="issue"} and `24139`{.interpreted-text role="issue"}.)
- Add `~sqlite3.Connection.setlimit`{.interpreted-text role="meth"} and `~sqlite3.Connection.getlimit`{.interpreted-text role="meth"} to `sqlite3.Connection`{.interpreted-text role="class"} for setting and getting SQLite limits by connection basis. (Contributed by Erlend E. Aasland in `45243`{.interpreted-text role="issue"}.)
- `sqlite3`{.interpreted-text role="mod"} now sets `sqlite3.threadsafety`{.interpreted-text role="attr"} based on the default threading mode the underlying SQLite library has been compiled with. (Contributed by Erlend E. Aasland in `45613`{.interpreted-text role="issue"}.)
- `sqlite3`{.interpreted-text role="mod"} C callbacks now use unraisable exceptions if callback tracebacks are enabled. Users can now register an `unraisable hook handler <sys.unraisablehook>`{.interpreted-text role="func"} to improve their debug experience. (Contributed by Erlend E. Aasland in `45828`{.interpreted-text role="issue"}.)
- Fetch across rollback no longer raises `~sqlite3.InterfaceError`{.interpreted-text role="exc"}. Instead we leave it to the SQLite library to handle these cases. (Contributed by Erlend E. Aasland in `44092`{.interpreted-text role="issue"}.)
- Add `~sqlite3.Connection.serialize`{.interpreted-text role="meth"} and `~sqlite3.Connection.deserialize`{.interpreted-text role="meth"} to `sqlite3.Connection`{.interpreted-text role="class"} for serializing and deserializing databases. (Contributed by Erlend E. Aasland in `41930`{.interpreted-text role="issue"}.)
- Add `~sqlite3.Connection.create_window_function`{.interpreted-text role="meth"} to `sqlite3.Connection`{.interpreted-text role="class"} for creating aggregate window functions. (Contributed by Erlend E. Aasland in `34916`{.interpreted-text role="issue"}.)
- Add `~sqlite3.Connection.blobopen`{.interpreted-text role="meth"} to `sqlite3.Connection`{.interpreted-text role="class"}. `sqlite3.Blob`{.interpreted-text role="class"} allows incremental I/O operations on blobs. (Contributed by Aviv Palivoda and Erlend E. Aasland in `24905`{.interpreted-text role="issue"}.)
### string {#whatsnew311-string}
- Add `~string.Template.get_identifiers`{.interpreted-text role="meth"} and `~string.Template.is_valid`{.interpreted-text role="meth"} to `string.Template`{.interpreted-text role="class"}, which respectively return all valid placeholders, and whether any invalid placeholders are present. (Contributed by Ben Kehoe in `90465`{.interpreted-text role="gh"}.)
### sys {#whatsnew311-sys}
- `sys.exc_info`{.interpreted-text role="func"} now derives the `type` and `traceback` fields from the `value` (the exception instance), so when an exception is modified while it is being handled, the changes are reflected in the results of subsequent calls to `!exc_info`{.interpreted-text role="func"}. (Contributed by Irit Katriel in `45711`{.interpreted-text role="issue"}.)
- Add `sys.exception`{.interpreted-text role="func"} which returns the active exception instance (equivalent to `sys.exc_info()[1]`). (Contributed by Irit Katriel in `46328`{.interpreted-text role="issue"}.)
- Add the `sys.flags.safe_path <sys.flags>`{.interpreted-text role="data"} flag. (Contributed by Victor Stinner in `57684`{.interpreted-text role="gh"}.)
### sysconfig {#whatsnew311-sysconfig}
- Three new `installation schemes <installation_paths>`{.interpreted-text role="ref"} (*posix_venv*, *nt_venv* and *venv*) were added and are used when Python creates new virtual environments or when it is running from a virtual environment. The first two schemes (*posix_venv* and *nt_venv*) are OS-specific for non-Windows and Windows, the *venv* is essentially an alias to one of them according to the OS Python runs on. This is useful for downstream distributors who modify `sysconfig.get_preferred_scheme`{.interpreted-text role="func"}. Third party code that creates new virtual environments should use the new *venv* installation scheme to determine the paths, as does `venv`{.interpreted-text role="mod"}. (Contributed by Miro Hrončok in `45413`{.interpreted-text role="issue"}.)
### tempfile {#whatsnew311-tempfile}
- `~tempfile.SpooledTemporaryFile`{.interpreted-text role="class"} objects now fully implement the methods of `io.BufferedIOBase`{.interpreted-text role="class"} or `io.TextIOBase`{.interpreted-text role="class"} (depending on file mode). This lets them work correctly with APIs that expect file-like objects, such as compression modules. (Contributed by Carey Metcalfe in `70363`{.interpreted-text role="gh"}.)
### threading {#whatsnew311-threading}
- On Unix, if the `sem_clockwait()` function is available in the C library (glibc 2.30 and newer), the `threading.Lock.acquire`{.interpreted-text role="meth"} method now uses the monotonic clock (`time.CLOCK_MONOTONIC`{.interpreted-text role="const"}) for the timeout, rather than using the system clock (`time.CLOCK_REALTIME`{.interpreted-text role="const"}), to not be affected by system clock changes. (Contributed by Victor Stinner in `41710`{.interpreted-text role="issue"}.)
### time {#whatsnew311-time}
- On Unix, `time.sleep`{.interpreted-text role="func"} now uses the `clock_nanosleep()` or `nanosleep()` function, if available, which has a resolution of 1 nanosecond (10^-9^ seconds), rather than using `select()` which has a resolution of 1 microsecond (10^-6^ seconds). (Contributed by Benjamin Szőke and Victor Stinner in `21302`{.interpreted-text role="issue"}.)
- On Windows 8.1 and newer, `time.sleep`{.interpreted-text role="func"} now uses a waitable timer based on [high-resolution timers](https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/high-resolution-timers) which has a resolution of 100 nanoseconds (10^-7^ seconds). Previously, it had a resolution of 1 millisecond (10^-3^ seconds). (Contributed by Benjamin Szőke, Donghee Na, Eryk Sun and Victor Stinner in `21302`{.interpreted-text role="issue"} and `45429`{.interpreted-text role="issue"}.)
### tkinter {#whatsnew311-tkinter}
- Added method `info_patchlevel()` which returns the exact version of the Tcl library as a named tuple similar to `sys.version_info`{.interpreted-text role="data"}. (Contributed by Serhiy Storchaka in `91827`{.interpreted-text role="gh"}.)
### traceback {#whatsnew311-traceback}
- Add `traceback.StackSummary.format_frame_summary`{.interpreted-text role="func"} to allow users to override which frames appear in the traceback, and how they are formatted. (Contributed by Ammar Askar in `44569`{.interpreted-text role="issue"}.)
- Add `traceback.TracebackException.print`{.interpreted-text role="func"}, which prints the formatted `~traceback.TracebackException`{.interpreted-text role="exc"} instance to a file. (Contributed by Irit Katriel in `33809`{.interpreted-text role="issue"}.)
### typing {#whatsnew311-typing}
For major changes, see `new-feat-related-type-hints-311`{.interpreted-text role="ref"}.
- Add `typing.assert_never`{.interpreted-text role="func"} and `typing.Never`{.interpreted-text role="class"}. `typing.assert_never`{.interpreted-text role="func"} is useful for asking a type checker to confirm that a line of code is not reachable. At runtime, it raises an `AssertionError`{.interpreted-text role="exc"}. (Contributed by Jelle Zijlstra in `90633`{.interpreted-text role="gh"}.)
- Add `typing.reveal_type`{.interpreted-text role="func"}. This is useful for asking a type checker what type it has inferred for a given expression. At runtime it prints the type of the received value. (Contributed by Jelle Zijlstra in `90572`{.interpreted-text role="gh"}.)
- Add `typing.assert_type`{.interpreted-text role="func"}. This is useful for asking a type checker to confirm that the type it has inferred for a given expression matches the given type. At runtime it simply returns the received value. (Contributed by Jelle Zijlstra in `90638`{.interpreted-text role="gh"}.)
- `typing.TypedDict`{.interpreted-text role="data"} types can now be generic. (Contributed by Samodya Abeysiriwardane in `89026`{.interpreted-text role="gh"}.)
- `~typing.NamedTuple`{.interpreted-text role="class"} types can now be generic. (Contributed by Serhiy Storchaka in `43923`{.interpreted-text role="issue"}.)
- Allow subclassing of `typing.Any`{.interpreted-text role="class"}. This is useful for avoiding type checker errors related to highly dynamic class, such as mocks. (Contributed by Shantanu Jain in `91154`{.interpreted-text role="gh"}.)
- The `typing.final`{.interpreted-text role="func"} decorator now sets the `__final__` attributed on the decorated object. (Contributed by Jelle Zijlstra in `90500`{.interpreted-text role="gh"}.)
- The `typing.get_overloads`{.interpreted-text role="func"} function can be used for introspecting the overloads of a function. `typing.clear_overloads`{.interpreted-text role="func"} can be used to clear all registered overloads of a function. (Contributed by Jelle Zijlstra in `89263`{.interpreted-text role="gh"}.)
- The `~object.__init__`{.interpreted-text role="meth"} method of `~typing.Protocol`{.interpreted-text role="class"} subclasses is now preserved. (Contributed by Adrian Garcia Badarasco in `88970`{.interpreted-text role="gh"}.)
- The representation of empty tuple types (`Tuple[()]`) is simplified. This affects introspection, e.g. `get_args(Tuple[()])` now evaluates to `()` instead of `((),)`. (Contributed by Serhiy Storchaka in `91137`{.interpreted-text role="gh"}.)
- Loosen runtime requirements for type annotations by removing the callable check in the private `typing._type_check` function. (Contributed by Gregory Beauregard in `90802`{.interpreted-text role="gh"}.)
- `typing.get_type_hints`{.interpreted-text role="func"} now supports evaluating strings as forward references in `PEP 585 generic aliases <types-genericalias>`{.interpreted-text role="ref"}. (Contributed by Niklas Rosenstein in `85542`{.interpreted-text role="gh"}.)
- `typing.get_type_hints`{.interpreted-text role="func"} no longer adds `~typing.Optional`{.interpreted-text role="data"} to parameters with `None` as a default. (Contributed by Nikita Sobolev in `90353`{.interpreted-text role="gh"}.)
- `typing.get_type_hints`{.interpreted-text role="func"} now supports evaluating bare stringified `~typing.ClassVar`{.interpreted-text role="data"} annotations. (Contributed by Gregory Beauregard in `90711`{.interpreted-text role="gh"}.)
- `typing.no_type_check`{.interpreted-text role="func"} no longer modifies external classes and functions. It also now correctly marks classmethods as not to be type checked. (Contributed by Nikita Sobolev in `90729`{.interpreted-text role="gh"}.)
### unicodedata {#whatsnew311-unicodedata}
- The Unicode database has been updated to version 14.0.0. (Contributed by Benjamin Peterson in `45190`{.interpreted-text role="issue"}).
### unittest {#whatsnew311-unittest}
- Added methods `~unittest.TestCase.enterContext`{.interpreted-text role="meth"} and `~unittest.TestCase.enterClassContext`{.interpreted-text role="meth"} of class `~unittest.TestCase`{.interpreted-text role="class"}, method `~unittest.IsolatedAsyncioTestCase.enterAsyncContext`{.interpreted-text role="meth"} of class `~unittest.IsolatedAsyncioTestCase`{.interpreted-text role="class"} and function `unittest.enterModuleContext`{.interpreted-text role="func"}. (Contributed by Serhiy Storchaka in `45046`{.interpreted-text role="issue"}.)
### venv {#whatsnew311-venv}
- When new Python virtual environments are created, the *venv* `sysconfig installation scheme <installation_paths>`{.interpreted-text role="ref"} is used to determine the paths inside the environment. When Python runs in a virtual environment, the same installation scheme is the default. That means that downstream distributors can change the default sysconfig install scheme without changing behavior of virtual environments. Third party code that also creates new virtual environments should do the same. (Contributed by Miro Hrončok in `45413`{.interpreted-text role="issue"}.)
### warnings {#whatsnew311-warnings}
- `warnings.catch_warnings`{.interpreted-text role="func"} now accepts arguments for `warnings.simplefilter`{.interpreted-text role="func"}, providing a more concise way to locally ignore warnings or convert them to errors. (Contributed by Zac Hatfield-Dodds in `47074`{.interpreted-text role="issue"}.)
### zipfile {#whatsnew311-zipfile}
- Added support for specifying member name encoding for reading metadata in a `~zipfile.ZipFile`{.interpreted-text role="class"}\'s directory and file headers. (Contributed by Stephen J. Turnbull and Serhiy Storchaka in `28080`{.interpreted-text role="issue"}.)
- Added `ZipFile.mkdir() <zipfile.ZipFile.mkdir>`{.interpreted-text role="meth"} for creating new directories inside ZIP archives. (Contributed by Sam Ezeh in `49083`{.interpreted-text role="gh"}.)
- Added `~zipfile.Path.stem`{.interpreted-text role="attr"}, `~zipfile.Path.suffix`{.interpreted-text role="attr"} and `~zipfile.Path.suffixes`{.interpreted-text role="attr"} to `zipfile.Path`{.interpreted-text role="class"}. (Contributed by Miguel Brito in `88261`{.interpreted-text role="gh"}.)
## Optimizations {#whatsnew311-optimizations}
This section covers specific optimizations independent of the `whatsnew311-faster-cpython`{.interpreted-text role="ref"} project, which is covered in its own section.
- The compiler now optimizes simple `printf-style % formatting <old-string-formatting>`{.interpreted-text role="ref"} on string literals containing only the format codes `%s`, `%r` and `%a` and makes it as fast as a corresponding `f-string`{.interpreted-text role="term"} expression. (Contributed by Serhiy Storchaka in `28307`{.interpreted-text role="issue"}.)
- Integer division (`//`) is better tuned for optimization by compilers. It is now around 20% faster on x86-64 when dividing an `int`{.interpreted-text role="class"} by a value smaller than `2**30`. (Contributed by Gregory P. Smith and Tim Peters in `90564`{.interpreted-text role="gh"}.)
- `sum`{.interpreted-text role="func"} is now nearly 30% faster for integers smaller than `2**30`. (Contributed by Stefan Behnel in `68264`{.interpreted-text role="gh"}.)
- Resizing lists is streamlined for the common case, speeding up `list.append`{.interpreted-text role="meth"} by ≈15% and simple `list comprehension`{.interpreted-text role="term"}s by up to 20-30% (Contributed by Dennis Sweeney in `91165`{.interpreted-text role="gh"}.)
- Dictionaries don\'t store hash values when all keys are Unicode objects, decreasing `dict`{.interpreted-text role="class"} size. For example, `sys.getsizeof(dict.fromkeys("abcdefg"))` is reduced from 352 bytes to 272 bytes (23% smaller) on 64-bit platforms. (Contributed by Inada Naoki in `46845`{.interpreted-text role="issue"}.)
- Using `asyncio.DatagramProtocol`{.interpreted-text role="class"} is now orders of magnitude faster when transferring large files over UDP, with speeds over 100 times higher for a ≈60 MiB file. (Contributed by msoxzw in `91487`{.interpreted-text role="gh"}.)
- `math`{.interpreted-text role="mod"} functions `~math.comb`{.interpreted-text role="func"} and `~math.perm`{.interpreted-text role="func"} are now ≈10 times faster for large arguments (with a larger speedup for larger *k*). (Contributed by Serhiy Storchaka in `37295`{.interpreted-text role="issue"}.)
- The `statistics`{.interpreted-text role="mod"} functions `~statistics.mean`{.interpreted-text role="func"}, `~statistics.variance`{.interpreted-text role="func"} and `~statistics.stdev`{.interpreted-text role="func"} now consume iterators in one pass rather than converting them to a `list`{.interpreted-text role="class"} first. This is twice as fast and can save substantial memory. (Contributed by Raymond Hettinger in `90415`{.interpreted-text role="gh"}.)
- `unicodedata.normalize`{.interpreted-text role="func"} now normalizes pure-ASCII strings in constant time. (Contributed by Donghee Na in `44987`{.interpreted-text role="issue"}.)
## Faster CPython {#whatsnew311-faster-cpython}
CPython 3.11 is an average of [25% faster](https://github.com/faster-cpython/ideas#published-results) than CPython 3.10 as measured with the [pyperformance](https://github.com/python/pyperformance) benchmark suite, when compiled with GCC on Ubuntu Linux. Depending on your workload, the overall speedup could be 10-60%.
This project focuses on two major areas in Python: `whatsnew311-faster-startup`{.interpreted-text role="ref"} and `whatsnew311-faster-runtime`{.interpreted-text role="ref"}. Optimizations not covered by this project are listed separately under `whatsnew311-optimizations`{.interpreted-text role="ref"}.
### Faster Startup {#whatsnew311-faster-startup}
#### Frozen imports / Static code objects {#whatsnew311-faster-imports}
Python caches `bytecode`{.interpreted-text role="term"} in the `__pycache__ <tut-pycache>`{.interpreted-text role="ref"} directory to speed up module loading.
Previously in 3.10, Python module execution looked like this:
``` text
Read __pycache__ -> Unmarshal -> Heap allocated code object -> Evaluate
```
In Python 3.11, the core modules essential for Python startup are \"frozen\". This means that their `codeobjects`{.interpreted-text role="ref"} (and bytecode) are statically allocated by the interpreter. This reduces the steps in module execution process to:
``` text
Statically allocated code object -> Evaluate
```
Interpreter startup is now 10-15% faster in Python 3.11. This has a big impact for short-running programs using Python.
(Contributed by Eric Snow, Guido van Rossum and Kumar Aditya in many issues.)
### Faster Runtime {#whatsnew311-faster-runtime}
#### Cheaper, lazy Python frames {#whatsnew311-lazy-python-frames}
Python frames, holding execution information, are created whenever Python calls a Python function. The following are new frame optimizations:
- Streamlined the frame creation process.
- Avoided memory allocation by generously re-using frame space on the C stack.
- Streamlined the internal frame struct to contain only essential information. Frames previously held extra debugging and memory management information.
Old-style `frame objects <frame-objects>`{.interpreted-text role="ref"} are now created only when requested by debuggers or by Python introspection functions such as `sys._getframe`{.interpreted-text role="func"} and `inspect.currentframe`{.interpreted-text role="func"}. For most user code, no frame objects are created at all. As a result, nearly all Python functions calls have sped up significantly. We measured a 3-7% speedup in pyperformance.
(Contributed by Mark Shannon in `44590`{.interpreted-text role="issue"}.)
#### Inlined Python function calls[]{#inline-calls} {#whatsnew311-inline-calls}
During a Python function call, Python will call an evaluating C function to interpret that function\'s code. This effectively limits pure Python recursion to what\'s safe for the C stack.
In 3.11, when CPython detects Python code calling another Python function, it sets up a new frame, and \"jumps\" to the new code inside the new frame. This avoids calling the C interpreting function altogether.
Most Python function calls now consume no C stack space, speeding them up. In simple recursive functions like fibonacci or factorial, we observed a 1.7x speedup. This also means recursive functions can recurse significantly deeper (if the user increases the recursion limit with `sys.setrecursionlimit`{.interpreted-text role="func"}). We measured a 1-3% improvement in pyperformance.
(Contributed by Pablo Galindo and Mark Shannon in `45256`{.interpreted-text role="issue"}.)
#### PEP 659: Specializing Adaptive Interpreter {#whatsnew311-pep659}
`659`{.interpreted-text role="pep"} is one of the key parts of the Faster CPython project. The general idea is that while Python is a dynamic language, most code has regions where objects and types rarely change. This concept is known as *type stability*.
At runtime, Python will try to look for common patterns and type stability in the executing code. Python will then replace the current operation with a more specialized one. This specialized operation uses fast paths available only to those use cases/types, which generally outperform their generic counterparts. This also brings in another concept called *inline caching*, where Python caches the results of expensive operations directly in the `bytecode`{.interpreted-text role="term"}.
The specializer will also combine certain common instruction pairs into one superinstruction, reducing the overhead during execution.
Python will only specialize when it sees code that is \"hot\" (executed multiple times). This prevents Python from wasting time on run-once code. Python can also de-specialize when code is too dynamic or when the use changes. Specialization is attempted periodically, and specialization attempts are not too expensive, allowing specialization to adapt to new circumstances.
(PEP written by Mark Shannon, with ideas inspired by Stefan Brunthaler. See `659`{.interpreted-text role="pep"} for more information. Implementation by Mark Shannon and Brandt Bucher, with additional help from Irit Katriel and Dennis Sweeney.)
+-----------------------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+---------------------------------------------------------+
| Operation | Form | Specialization | Operation speedup (up to) | Contributor(s) |
+=======================+==============+=============================================================================================================================================================================================================================================+===========================+=========================================================+
| Binary operations | `x + x` | Binary add, multiply and subtract for common types such as `int`{.interpreted-text role="class"}, `float`{.interpreted-text role="class"} and `str`{.interpreted-text role="class"} take custom fast paths for their underlying types. | 10% | Mark Shannon, Donghee Na, Brandt Bucher, Dennis Sweeney |
| | | | | |
| | `x - x` | | | |
| | | | | |
| | `x * x` | | | |
+-----------------------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+---------------------------------------------------------+
| Subscript | `a[i]` | Subscripting container types such as `list`{.interpreted-text role="class"}, `tuple`{.interpreted-text role="class"} and `dict`{.interpreted-text role="class"} directly index the underlying data structures. | 10-25% | Irit Katriel, Mark Shannon |
| | | | | |
| | | Subscripting custom `~object.__getitem__`{.interpreted-text role="meth"} is also inlined similar to `inline-calls`{.interpreted-text role="ref"}. | | |
+-----------------------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+---------------------------------------------------------+
| Store subscript | `a[i] = z` | Similar to subscripting specialization above. | 10-25% | Dennis Sweeney |
+-----------------------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+---------------------------------------------------------+
| Calls | `f(arg)` | Calls to common builtin (C) functions and types such as `len`{.interpreted-text role="func"} and `str`{.interpreted-text role="class"} directly call their underlying C version. This avoids going through the internal calling convention. | 20% | Mark Shannon, Ken Jin |
| | | | | |
| | `C(arg)` | | | |
+-----------------------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+---------------------------------------------------------+
| Load global variable | `print` | The object\'s index in the globals/builtins namespace is cached. Loading globals and builtins require zero namespace lookups. | [^1] | Mark Shannon |
| | | | | |
| | `len` | | | |
+-----------------------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+---------------------------------------------------------+
| Load attribute | `o.attr` | Similar to loading global variables. The attribute\'s index inside the class/object\'s namespace is cached. In most cases, attribute loading will require zero namespace lookups. | [^2] | Mark Shannon |
+-----------------------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+---------------------------------------------------------+
| Load methods for call | `o.meth()` | The actual address of the method is cached. Method loading now has no namespace lookups \-- even for classes with long inheritance chains. | 10-20% | Ken Jin, Mark Shannon |
+-----------------------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+---------------------------------------------------------+
| Store attribute | `o.attr = z` | Similar to load attribute optimization. | 2% in pyperformance | Mark Shannon |
+-----------------------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+---------------------------------------------------------+
| Unpack Sequence | `*seq` | Specialized for common containers such as `list`{.interpreted-text role="class"} and `tuple`{.interpreted-text role="class"}. Avoids internal calling convention. | 8% | Brandt Bucher |
+-----------------------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+---------------------------------------------------------+
### Misc {#whatsnew311-faster-cpython-misc}
- Objects now require less memory due to lazily created object namespaces. Their namespace dictionaries now also share keys more freely. (Contributed Mark Shannon in `45340`{.interpreted-text role="issue"} and `40116`{.interpreted-text role="issue"}.)
- \"Zero-cost\" exceptions are implemented, eliminating the cost of `try`{.interpreted-text role="keyword"} statements when no exception is raised. (Contributed by Mark Shannon in `40222`{.interpreted-text role="issue"}.)
- A more concise representation of exceptions in the interpreter reduced the time required for catching an exception by about 10%. (Contributed by Irit Katriel in `45711`{.interpreted-text role="issue"}.)
- `re`{.interpreted-text role="mod"}\'s regular expression matching engine has been partially refactored, and now uses computed gotos (or \"threaded code\") on supported platforms. As a result, Python 3.11 executes the [pyperformance regular expression benchmarks](https://pyperformance.readthedocs.io/benchmarks.html#regex-dna) up to 10% faster than Python 3.10. (Contributed by Brandt Bucher in `91404`{.interpreted-text role="gh"}.)
### FAQ {#whatsnew311-faster-cpython-faq}
#### How should I write my code to utilize these speedups? {#faster-cpython-faq-my-code}
Write Pythonic code that follows common best practices; you don\'t have to change your code. The Faster CPython project optimizes for common code patterns we observe.
#### Will CPython 3.11 use more memory? {#faster-cpython-faq-memory}
Maybe not; we don\'t expect memory use to exceed 20% higher than 3.10. This is offset by memory optimizations for frame objects and object dictionaries as mentioned above.
#### I don\'t see any speedups in my workload. Why? {#faster-cpython-ymmv}
Certain code won\'t have noticeable benefits. If your code spends most of its time on I/O operations, or already does most of its computation in a C extension library like NumPy, there won\'t be significant speedups. This project currently benefits pure-Python workloads the most.
Furthermore, the pyperformance figures are a geometric mean. Even within the pyperformance benchmarks, certain benchmarks have slowed down slightly, while others have sped up by nearly 2x!
#### Is there a JIT compiler? {#faster-cpython-jit}
No. We\'re still exploring other optimizations.
### About {#whatsnew311-faster-cpython-about}
Faster CPython explores optimizations for `CPython`{.interpreted-text role="term"}. The main team is funded by Microsoft to work on this full-time. Pablo Galindo Salgado is also funded by Bloomberg LP to work on the project part-time. Finally, many contributors are volunteers from the community.
## CPython bytecode changes {#whatsnew311-bytecode-changes}
The bytecode now contains inline cache entries, which take the form of the newly-added `CACHE`{.interpreted-text role="opcode"} instructions. Many opcodes expect to be followed by an exact number of caches, and instruct the interpreter to skip over them at runtime. Populated caches can look like arbitrary instructions, so great care should be taken when reading or modifying raw, adaptive bytecode containing quickened data.
### New opcodes {#whatsnew311-added-opcodes}
- `!ASYNC_GEN_WRAP`{.interpreted-text role="opcode"}, `RETURN_GENERATOR`{.interpreted-text role="opcode"} and `SEND`{.interpreted-text role="opcode"}, used in generators and co-routines.
- `COPY_FREE_VARS`{.interpreted-text role="opcode"}, which avoids needing special caller-side code for closures.
- `JUMP_BACKWARD_NO_INTERRUPT`{.interpreted-text role="opcode"}, for use in certain loops where handling interrupts is undesirable.
- `MAKE_CELL`{.interpreted-text role="opcode"}, to create `cell-objects`{.interpreted-text role="ref"}.
- `CHECK_EG_MATCH`{.interpreted-text role="opcode"} and `!PREP_RERAISE_STAR`{.interpreted-text role="opcode"}, to handle the `new exception groups and except* <whatsnew311-pep654>`{.interpreted-text role="ref"} added in `654`{.interpreted-text role="pep"}.
- `PUSH_EXC_INFO`{.interpreted-text role="opcode"}, for use in exception handlers.
- `RESUME`{.interpreted-text role="opcode"}, a no-op, for internal tracing, debugging and optimization checks.
### Replaced opcodes {#whatsnew311-replaced-opcodes}
+-------------------------------------------------------------+--------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| Replaced Opcode(s) | New Opcode(s) | Notes |
+=============================================================+==============================================================+===================================================================================================================+
| | `!BINARY_*`{.interpreted-text role="opcode"} | `BINARY_OP`{.interpreted-text role="opcode"} | Replaced all numeric binary/in-place opcodes with a single opcode |
| | `!INPLACE_*`{.interpreted-text role="opcode"} | | |
+-------------------------------------------------------------+--------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | `!CALL_FUNCTION`{.interpreted-text role="opcode"} | | `CALL`{.interpreted-text role="opcode"} | Decouples argument shifting for methods from handling of keyword arguments; allows better specialization of calls |
| | `!CALL_FUNCTION_KW`{.interpreted-text role="opcode"} | | `!KW_NAMES`{.interpreted-text role="opcode"} | |
| | `!CALL_METHOD`{.interpreted-text role="opcode"} | | `!PRECALL`{.interpreted-text role="opcode"} | |
| | | `PUSH_NULL`{.interpreted-text role="opcode"} | |
+-------------------------------------------------------------+--------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | `!DUP_TOP`{.interpreted-text role="opcode"} | | `COPY`{.interpreted-text role="opcode"} | Stack manipulation instructions |
| | `!DUP_TOP_TWO`{.interpreted-text role="opcode"} | | `SWAP`{.interpreted-text role="opcode"} | |
| | `!ROT_TWO`{.interpreted-text role="opcode"} | | |
| | `!ROT_THREE`{.interpreted-text role="opcode"} | | |
| | `!ROT_FOUR`{.interpreted-text role="opcode"} | | |
| | `!ROT_N`{.interpreted-text role="opcode"} | | |
+-------------------------------------------------------------+--------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | `!JUMP_IF_NOT_EXC_MATCH`{.interpreted-text role="opcode"} | | `CHECK_EXC_MATCH`{.interpreted-text role="opcode"} | Now performs check but doesn\'t jump |
+-------------------------------------------------------------+--------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | `!JUMP_ABSOLUTE`{.interpreted-text role="opcode"} | | `JUMP_BACKWARD`{.interpreted-text role="opcode"} | See[^3]; `TRUE`, `FALSE`, `NONE` and `NOT_NONE` variants for each direction |
| | `!POP_JUMP_IF_FALSE`{.interpreted-text role="opcode"} | | `!POP_JUMP_BACKWARD_IF_*`{.interpreted-text role="opcode"} | |
| | `!POP_JUMP_IF_TRUE`{.interpreted-text role="opcode"} | | `!POP_JUMP_FORWARD_IF_*`{.interpreted-text role="opcode"} | |
+-------------------------------------------------------------+--------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | `!SETUP_WITH`{.interpreted-text role="opcode"} | `!BEFORE_WITH`{.interpreted-text role="opcode"} | `with`{.interpreted-text role="keyword"} block setup |
| | `!SETUP_ASYNC_WITH`{.interpreted-text role="opcode"} | | |
+-------------------------------------------------------------+--------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
### Changed/removed opcodes[]{#whatsnew311-removed-opcodes}[]{#whatsnew311-changed-opcodes} {#whatsnew311-changed-removed-opcodes}
- Changed `MATCH_CLASS`{.interpreted-text role="opcode"} and `MATCH_KEYS`{.interpreted-text role="opcode"} to no longer push an additional boolean value to indicate success/failure. Instead, `None` is pushed on failure in place of the tuple of extracted values.
- Changed opcodes that work with exceptions to reflect them now being represented as one item on the stack instead of three (see `89874`{.interpreted-text role="gh"}).
- Removed `!COPY_DICT_WITHOUT_KEYS`{.interpreted-text role="opcode"}, `!GEN_START`{.interpreted-text role="opcode"}, `!POP_BLOCK`{.interpreted-text role="opcode"}, `!SETUP_FINALLY`{.interpreted-text role="opcode"} and `!YIELD_FROM`{.interpreted-text role="opcode"}.
## Deprecated[]{#whatsnew311-deprecated} {#whatsnew311-python-api-deprecated}
This section lists Python APIs that have been deprecated in Python 3.11.
Deprecated C APIs are `listed separately <whatsnew311-c-api-deprecated>`{.interpreted-text role="ref"}.
### Language/Builtins[]{#whatsnew311-deprecated-language} {#whatsnew311-deprecated-builtins}
- Chaining `classmethod`{.interpreted-text role="class"} descriptors (introduced in `19072`{.interpreted-text role="issue"}) is now deprecated. It can no longer be used to wrap other descriptors such as `property`{.interpreted-text role="class"}. The core design of this feature was flawed and caused a number of downstream problems. To \"pass-through\" a `classmethod`{.interpreted-text role="class"}, consider using the `!__wrapped__`{.interpreted-text role="attr"} attribute that was added in Python 3.10. (Contributed by Raymond Hettinger in `89519`{.interpreted-text role="gh"}.)
- Octal escapes in string and bytes literals with values larger than `0o377` (255 in decimal) now produce a `DeprecationWarning`{.interpreted-text role="exc"}. In a future Python version, they will raise a `SyntaxWarning`{.interpreted-text role="exc"} and eventually a `SyntaxError`{.interpreted-text role="exc"}. (Contributed by Serhiy Storchaka in `81548`{.interpreted-text role="gh"}.)
- The delegation of `int`{.interpreted-text role="func"} to `~object.__trunc__`{.interpreted-text role="meth"} is now deprecated. Calling `int(a)` when `type(a)` implements `!__trunc__`{.interpreted-text role="meth"} but not `~object.__int__`{.interpreted-text role="meth"} or `~object.__index__`{.interpreted-text role="meth"} now raises a `DeprecationWarning`{.interpreted-text role="exc"}. (Contributed by Zackery Spytz in `44977`{.interpreted-text role="issue"}.)
### Modules {#whatsnew311-deprecated-modules}
::: {#whatsnew311-pep594}
- `594`{.interpreted-text role="pep"} led to the deprecations of the following modules slated for removal in Python 3.13:
------------------------------------------ ------------------------------------------ ---------------------------------------------- ----------------------------------------- --------------------------------------------
`!aifc`{.interpreted-text role="mod"} `!chunk`{.interpreted-text role="mod"} `!msilib`{.interpreted-text role="mod"} `!pipes`{.interpreted-text role="mod"} `!telnetlib`{.interpreted-text role="mod"}
`!audioop`{.interpreted-text role="mod"} `!crypt`{.interpreted-text role="mod"} `!nis`{.interpreted-text role="mod"} `!sndhdr`{.interpreted-text role="mod"} `!uu`{.interpreted-text role="mod"}
`!cgi`{.interpreted-text role="mod"} `!imghdr`{.interpreted-text role="mod"} `!nntplib`{.interpreted-text role="mod"} `!spwd`{.interpreted-text role="mod"} `!xdrlib`{.interpreted-text role="mod"}
`!cgitb`{.interpreted-text role="mod"} `!mailcap`{.interpreted-text role="mod"} `!ossaudiodev`{.interpreted-text role="mod"} `!sunau`{.interpreted-text role="mod"}
------------------------------------------ ------------------------------------------ ---------------------------------------------- ----------------------------------------- --------------------------------------------
(Contributed by Brett Cannon in `47061`{.interpreted-text role="issue"} and Victor Stinner in `68966`{.interpreted-text role="gh"}.)
- The `!asynchat`{.interpreted-text role="mod"}, `!asyncore`{.interpreted-text role="mod"} and `!smtpd`{.interpreted-text role="mod"} modules have been deprecated since at least Python 3.6. Their documentation and deprecation warnings have now been updated to note they will be removed in Python 3.12. (Contributed by Hugo van Kemenade in `47022`{.interpreted-text role="issue"}.)
- The `!lib2to3`{.interpreted-text role="mod"} package and `2to3` tool are now deprecated and may not be able to parse Python 3.10 or newer. See `617`{.interpreted-text role="pep"}, introducing the new PEG parser, for details. (Contributed by Victor Stinner in `40360`{.interpreted-text role="issue"}.)
- Undocumented modules `!sre_compile`{.interpreted-text role="mod"}, `!sre_constants`{.interpreted-text role="mod"} and `!sre_parse`{.interpreted-text role="mod"} are now deprecated. (Contributed by Serhiy Storchaka in `47152`{.interpreted-text role="issue"}.)
:::
### Standard Library {#whatsnew311-deprecated-stdlib}
- The following have been deprecated in `configparser`{.interpreted-text role="mod"} since Python 3.2. Their deprecation warnings have now been updated to note they will be removed in Python 3.12:
- the `!configparser.SafeConfigParser`{.interpreted-text role="class"} class
- the `!configparser.ParsingError.filename`{.interpreted-text role="attr"} property
- the `!configparser.RawConfigParser.readfp`{.interpreted-text role="meth"} method
(Contributed by Hugo van Kemenade in `45173`{.interpreted-text role="issue"}.)
- `!configparser.LegacyInterpolation`{.interpreted-text role="class"} has been deprecated in the docstring since Python 3.2, and is not listed in the `configparser`{.interpreted-text role="mod"} documentation. It now emits a `DeprecationWarning`{.interpreted-text role="exc"} and will be removed in Python 3.13. Use `configparser.BasicInterpolation`{.interpreted-text role="class"} or `configparser.ExtendedInterpolation`{.interpreted-text role="class"} instead. (Contributed by Hugo van Kemenade in `46607`{.interpreted-text role="issue"}.)
- The older set of `importlib.resources`{.interpreted-text role="mod"} functions were deprecated in favor of the replacements added in Python 3.9 and will be removed in a future Python version, due to not supporting resources located within package subdirectories:
- `!importlib.resources.contents`{.interpreted-text role="func"}
- `!importlib.resources.is_resource`{.interpreted-text role="func"}
- `!importlib.resources.open_binary`{.interpreted-text role="func"}
- `!importlib.resources.open_text`{.interpreted-text role="func"}
- `!importlib.resources.read_binary`{.interpreted-text role="func"}
- `!importlib.resources.read_text`{.interpreted-text role="func"}
- `!importlib.resources.path`{.interpreted-text role="func"}
- The `locale.getdefaultlocale`{.interpreted-text role="func"} function is deprecated and will be removed in Python 3.15. Use `locale.setlocale`{.interpreted-text role="func"}, `locale.getpreferredencoding(False) <locale.getpreferredencoding>`{.interpreted-text role="func"} and `locale.getlocale`{.interpreted-text role="func"} functions instead. (Contributed by Victor Stinner in `90817`{.interpreted-text role="gh"}.)
- The `!locale.resetlocale`{.interpreted-text role="func"} function is deprecated and will be removed in Python 3.13. Use `locale.setlocale(locale.LC_ALL, "")` instead. (Contributed by Victor Stinner in `90817`{.interpreted-text role="gh"}.)
- Stricter rules will now be applied for numerical group references and group names in `regular expressions <re-syntax>`{.interpreted-text role="ref"}. Only sequences of ASCII digits will now be accepted as a numerical reference, and the group name in `bytes`{.interpreted-text role="class"} patterns and replacement strings can only contain ASCII letters, digits and underscores. For now, a deprecation warning is raised for syntax violating these rules. (Contributed by Serhiy Storchaka in `91760`{.interpreted-text role="gh"}.)
- In the `re`{.interpreted-text role="mod"} module, the `!re.template`{.interpreted-text role="func"} function and the corresponding `!re.TEMPLATE`{.interpreted-text role="const"} and `!re.T`{.interpreted-text role="const"} flags are deprecated, as they were undocumented and lacked an obvious purpose. They will be removed in Python 3.13. (Contributed by Serhiy Storchaka and Miro Hrončok in `92728`{.interpreted-text role="gh"}.)
- `!turtle.settiltangle`{.interpreted-text role="func"} has been deprecated since Python 3.1; it now emits a deprecation warning and will be removed in Python 3.13. Use `turtle.tiltangle`{.interpreted-text role="func"} instead (it was earlier incorrectly marked as deprecated, and its docstring is now corrected). (Contributed by Hugo van Kemenade in `45837`{.interpreted-text role="issue"}.)
- `typing.Text`{.interpreted-text role="class"}, which exists solely to provide compatibility support between Python 2 and Python 3 code, is now deprecated. Its removal is currently unplanned, but users are encouraged to use `str`{.interpreted-text role="class"} instead wherever possible. (Contributed by Alex Waygood in `92332`{.interpreted-text role="gh"}.)
- The keyword argument syntax for constructing `typing.TypedDict`{.interpreted-text role="data"} types is now deprecated. Support will be removed in Python 3.13. (Contributed by Jingchen Ye in `90224`{.interpreted-text role="gh"}.)
- `!webbrowser.MacOSX`{.interpreted-text role="class"} is deprecated and will be removed in Python 3.13. It is untested, undocumented, and not used by `webbrowser`{.interpreted-text role="mod"} itself. (Contributed by Donghee Na in `42255`{.interpreted-text role="issue"}.)
- The behavior of returning a value from a `~unittest.TestCase`{.interpreted-text role="class"} and `~unittest.IsolatedAsyncioTestCase`{.interpreted-text role="class"} test methods (other than the default `None` value) is now deprecated.
- Deprecated the following not-formally-documented `unittest`{.interpreted-text role="mod"} functions, scheduled for removal in Python 3.13:
- `!unittest.findTestCases`{.interpreted-text role="func"}
- `!unittest.makeSuite`{.interpreted-text role="func"}
- `!unittest.getTestCaseNames`{.interpreted-text role="func"}
Use `~unittest.TestLoader`{.interpreted-text role="class"} methods instead:
- `unittest.TestLoader.loadTestsFromModule`{.interpreted-text role="meth"}
- `unittest.TestLoader.loadTestsFromTestCase`{.interpreted-text role="meth"}
- `unittest.TestLoader.getTestCaseNames`{.interpreted-text role="meth"}
(Contributed by Erlend E. Aasland in `5846`{.interpreted-text role="issue"}.)
- `!unittest.TestProgram.usageExit`{.interpreted-text role="meth"} is marked deprecated, to be removed in 3.13. (Contributed by Carlos Damázio in `67048`{.interpreted-text role="gh"}.)
## Pending Removal in Python 3.12[]{#whatsnew311-pending-removal} {#whatsnew311-python-api-pending-removal}
The following Python APIs have been deprecated in earlier Python releases, and will be removed in Python 3.12.
C APIs pending removal are `listed separately <whatsnew311-c-api-pending-removal>`{.interpreted-text role="ref"}.
- The `!asynchat`{.interpreted-text role="mod"} module
- The `!asyncore`{.interpreted-text role="mod"} module
- The `entire distutils package <distutils-deprecated>`{.interpreted-text role="ref"}
- The `!imp`{.interpreted-text role="mod"} module
- The `typing.io <typing.IO>`{.interpreted-text role="class"} namespace
- The `typing.re <typing.Pattern>`{.interpreted-text role="class"} namespace
- `!cgi.log`{.interpreted-text role="func"}
- `!importlib.find_loader`{.interpreted-text role="func"}
- `!importlib.abc.Loader.module_repr`{.interpreted-text role="meth"}
- `!importlib.abc.MetaPathFinder.find_module`{.interpreted-text role="meth"}
- `!importlib.abc.PathEntryFinder.find_loader`{.interpreted-text role="meth"}
- `!importlib.abc.PathEntryFinder.find_module`{.interpreted-text role="meth"}
- `!importlib.machinery.BuiltinImporter.find_module`{.interpreted-text role="meth"}
- `!importlib.machinery.BuiltinLoader.module_repr`{.interpreted-text role="meth"}
- `!importlib.machinery.FileFinder.find_loader`{.interpreted-text role="meth"}
- `!importlib.machinery.FileFinder.find_module`{.interpreted-text role="meth"}
- `!importlib.machinery.FrozenImporter.find_module`{.interpreted-text role="meth"}
- `!importlib.machinery.FrozenLoader.module_repr`{.interpreted-text role="meth"}
- `!importlib.machinery.PathFinder.find_module`{.interpreted-text role="meth"}
- `!importlib.machinery.WindowsRegistryFinder.find_module`{.interpreted-text role="meth"}
- `!importlib.util.module_for_loader`{.interpreted-text role="func"}
- `!importlib.util.set_loader_wrapper`{.interpreted-text role="func"}
- `!importlib.util.set_package_wrapper`{.interpreted-text role="func"}
- `!pkgutil.ImpImporter`{.interpreted-text role="class"}
- `!pkgutil.ImpLoader`{.interpreted-text role="class"}
- `!pathlib.Path.link_to`{.interpreted-text role="meth"}
- `!sqlite3.enable_shared_cache`{.interpreted-text role="func"}
- `!sqlite3.OptimizedUnicode`{.interpreted-text role="func"}
- `!PYTHONTHREADDEBUG`{.interpreted-text role="envvar"} environment variable
- The following deprecated aliases in `unittest`{.interpreted-text role="mod"}:
>
> Deprecated alias Method Name Deprecated in
> -------------------------- -------------------------------------------------------- ---------------
> `failUnless` `.assertTrue`{.interpreted-text role="meth"} 3.1
> `failIf` `.assertFalse`{.interpreted-text role="meth"} 3.1
> `failUnlessEqual` `.assertEqual`{.interpreted-text role="meth"} 3.1
> `failIfEqual` `.assertNotEqual`{.interpreted-text role="meth"} 3.1
> `failUnlessAlmostEqual` `.assertAlmostEqual`{.interpreted-text role="meth"} 3.1
> `failIfAlmostEqual` `.assertNotAlmostEqual`{.interpreted-text role="meth"} 3.1
> `failUnlessRaises` `.assertRaises`{.interpreted-text role="meth"} 3.1
> `assert_` `.assertTrue`{.interpreted-text role="meth"} 3.2
> `assertEquals` `.assertEqual`{.interpreted-text role="meth"} 3.2
> `assertNotEquals` `.assertNotEqual`{.interpreted-text role="meth"} 3.2
> `assertAlmostEquals` `.assertAlmostEqual`{.interpreted-text role="meth"} 3.2
> `assertNotAlmostEquals` `.assertNotAlmostEqual`{.interpreted-text role="meth"} 3.2
> `assertRegexpMatches` `.assertRegex`{.interpreted-text role="meth"} 3.2
> `assertRaisesRegexp` `.assertRaisesRegex`{.interpreted-text role="meth"} 3.2
> `assertNotRegexpMatches` `.assertNotRegex`{.interpreted-text role="meth"} 3.5
## Removed[]{#whatsnew311-removed} {#whatsnew311-python-api-removed}
This section lists Python APIs that have been removed in Python 3.11.
Removed C APIs are `listed separately <whatsnew311-c-api-removed>`{.interpreted-text role="ref"}.
- Removed the `!@asyncio.coroutine`{.interpreted-text role="func"} `decorator`{.interpreted-text role="term"} enabling legacy generator-based coroutines to be compatible with `async`{.interpreted-text role="keyword"} / `await`{.interpreted-text role="keyword"} code. The function has been deprecated since Python 3.8 and the removal was initially scheduled for Python 3.10. Use `async def`{.interpreted-text role="keyword"} instead. (Contributed by Illia Volochii in `43216`{.interpreted-text role="issue"}.)
- Removed `!asyncio.coroutines.CoroWrapper`{.interpreted-text role="class"} used for wrapping legacy generator-based coroutine objects in the debug mode. (Contributed by Illia Volochii in `43216`{.interpreted-text role="issue"}.)
- Due to significant security concerns, the *reuse_address* parameter of `asyncio.loop.create_datagram_endpoint`{.interpreted-text role="meth"}, disabled in Python 3.9, is now entirely removed. This is because of the behavior of the socket option `SO_REUSEADDR` in UDP. (Contributed by Hugo van Kemenade in `45129`{.interpreted-text role="issue"}.)
- Removed the `!binhex`{.interpreted-text role="mod"} module, deprecated in Python 3.9. Also removed the related, similarly-deprecated `binascii`{.interpreted-text role="mod"} functions:
- `!binascii.a2b_hqx`{.interpreted-text role="func"}
- `!binascii.b2a_hqx`{.interpreted-text role="func"}
- `!binascii.rlecode_hqx`{.interpreted-text role="func"}
- `!binascii.rldecode_hqx`{.interpreted-text role="func"}
The `binascii.crc_hqx`{.interpreted-text role="func"} function remains available.
(Contributed by Victor Stinner in `45085`{.interpreted-text role="issue"}.)
- Removed the `!distutils`{.interpreted-text role="mod"} `bdist_msi` command deprecated in Python 3.9. Use `bdist_wheel` (wheel packages) instead. (Contributed by Hugo van Kemenade in `45124`{.interpreted-text role="issue"}.)
- Removed the `~object.__getitem__`{.interpreted-text role="meth"} methods of `xml.dom.pulldom.DOMEventStream`{.interpreted-text role="class"}, `wsgiref.util.FileWrapper`{.interpreted-text role="class"} and `fileinput.FileInput`{.interpreted-text role="class"}, deprecated since Python 3.9. (Contributed by Hugo van Kemenade in `45132`{.interpreted-text role="issue"}.)
- Removed the deprecated `gettext`{.interpreted-text role="mod"} functions `!lgettext`{.interpreted-text role="func"}, `!ldgettext`{.interpreted-text role="func"}, `!lngettext`{.interpreted-text role="func"} and `!ldngettext`{.interpreted-text role="func"}. Also removed the `!bind_textdomain_codeset`{.interpreted-text role="func"} function, the `!NullTranslations.output_charset`{.interpreted-text role="meth"} and `!NullTranslations.set_output_charset`{.interpreted-text role="meth"} methods, and the *codeset* parameter of `!translation`{.interpreted-text role="func"} and `!install`{.interpreted-text role="func"}, since they are only used for the `!l*gettext`{.interpreted-text role="func"} functions. (Contributed by Donghee Na and Serhiy Storchaka in `44235`{.interpreted-text role="issue"}.)
- Removed from the `inspect`{.interpreted-text role="mod"} module:
- The `!getargspec`{.interpreted-text role="func"} function, deprecated since Python 3.0; use `inspect.signature`{.interpreted-text role="func"} or `inspect.getfullargspec`{.interpreted-text role="func"} instead.
- The `!formatargspec`{.interpreted-text role="func"} function, deprecated since Python 3.5; use the `inspect.signature`{.interpreted-text role="func"} function or the `inspect.Signature`{.interpreted-text role="class"} object directly.
- The undocumented `!Signature.from_builtin`{.interpreted-text role="meth"} and `!Signature.from_function`{.interpreted-text role="meth"} methods, deprecated since Python 3.5; use the `Signature.from_callable() <inspect.Signature.from_callable>`{.interpreted-text role="meth"} method instead.
(Contributed by Hugo van Kemenade in `45320`{.interpreted-text role="issue"}.)
- Removed the `~object.__class_getitem__`{.interpreted-text role="meth"} method from `pathlib.PurePath`{.interpreted-text role="class"}, because it was not used and added by mistake in previous versions. (Contributed by Nikita Sobolev in `46483`{.interpreted-text role="issue"}.)
- Removed the `!MailmanProxy`{.interpreted-text role="class"} class in the `!smtpd`{.interpreted-text role="mod"} module, as it is unusable without the external `!mailman`{.interpreted-text role="mod"} package. (Contributed by Donghee Na in `35800`{.interpreted-text role="issue"}.)
- Removed the deprecated `!split`{.interpreted-text role="meth"} method of `!_tkinter.TkappType`{.interpreted-text role="class"}. (Contributed by Erlend E. Aasland in `38371`{.interpreted-text role="issue"}.)
- Removed namespace package support from `unittest`{.interpreted-text role="mod"} discovery. It was introduced in Python 3.4 but has been broken since Python 3.7. (Contributed by Inada Naoki in `23882`{.interpreted-text role="issue"}.)
- Removed the undocumented private `!float.__set_format__`{.interpreted-text role="meth"} method, previously known as `!float.__setformat__`{.interpreted-text role="meth"} in Python 3.7. Its docstring said: \"You probably don\'t want to use this function. It exists mainly to be used in Python\'s test suite.\" (Contributed by Victor Stinner in `46852`{.interpreted-text role="issue"}.)
- The `!--experimental-isolated-subinterpreters`{.interpreted-text role="option"} configure flag (and corresponding `!EXPERIMENTAL_ISOLATED_SUBINTERPRETERS`{.interpreted-text role="c:macro"} macro) have been removed.
- `Pynche`{.interpreted-text role="pypi"} \-\-- The Pythonically Natural Color and Hue Editor \-\-- has been moved out of `Tools/scripts` and is [being developed independently](https://gitlab.com/warsaw/pynche/-/tree/main) from the Python source tree.
## Porting to Python 3.11[]{#whatsnew311-porting} {#whatsnew311-python-api-porting}
This section lists previously described changes and other bugfixes in the Python API that may require changes to your Python code.
Porting notes for the C API are `listed separately <whatsnew311-c-api-porting>`{.interpreted-text role="ref"}.
- `open`{.interpreted-text role="func"}, `io.open`{.interpreted-text role="func"}, `codecs.open`{.interpreted-text role="func"} and `fileinput.FileInput`{.interpreted-text role="class"} no longer accept `'U'` (\"universal newline\") in the file mode. In Python 3, \"universal newline\" mode is used by default whenever a file is opened in text mode, and the `'U'` flag has been deprecated since Python 3.3. The `newline parameter <open-newline-parameter>`{.interpreted-text role="ref"} to these functions controls how universal newlines work. (Contributed by Victor Stinner in `37330`{.interpreted-text role="issue"}.)
- `ast.AST`{.interpreted-text role="class"} node positions are now validated when provided to `compile`{.interpreted-text role="func"} and other related functions. If invalid positions are detected, a `ValueError`{.interpreted-text role="exc"} will be raised. (Contributed by Pablo Galindo in `93351`{.interpreted-text role="gh"})
- Prohibited passing non-`concurrent.futures.ThreadPoolExecutor`{.interpreted-text role="class"} executors to `asyncio.loop.set_default_executor`{.interpreted-text role="meth"} following a deprecation in Python 3.8. (Contributed by Illia Volochii in `43234`{.interpreted-text role="issue"}.)
- `calendar`{.interpreted-text role="mod"}: The `calendar.LocaleTextCalendar`{.interpreted-text role="class"} and `calendar.LocaleHTMLCalendar`{.interpreted-text role="class"} classes now use `locale.getlocale`{.interpreted-text role="func"}, instead of using `locale.getdefaultlocale`{.interpreted-text role="func"}, if no locale is specified. (Contributed by Victor Stinner in `46659`{.interpreted-text role="issue"}.)
- The `pdb`{.interpreted-text role="mod"} module now reads the `.pdbrc`{.interpreted-text role="file"} configuration file with the `'UTF-8'` encoding. (Contributed by Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి) in `41137`{.interpreted-text role="issue"}.)
- The *population* parameter of `random.sample`{.interpreted-text role="func"} must be a sequence, and automatic conversion of `set`{.interpreted-text role="class"}s to `list`{.interpreted-text role="class"}s is no longer supported. Also, if the sample size is larger than the population size, a `ValueError`{.interpreted-text role="exc"} is raised. (Contributed by Raymond Hettinger in `40465`{.interpreted-text role="issue"}.)
- The *random* optional parameter of `random.shuffle`{.interpreted-text role="func"} was removed. It was previously an arbitrary random function to use for the shuffle; now, `random.random`{.interpreted-text role="func"} (its previous default) will always be used.
- In `re`{.interpreted-text role="mod"} `re-syntax`{.interpreted-text role="ref"}, global inline flags (e.g. `(?i)`) can now only be used at the start of regular expressions. Using them elsewhere has been deprecated since Python 3.6. (Contributed by Serhiy Storchaka in `47066`{.interpreted-text role="issue"}.)
- In the `re`{.interpreted-text role="mod"} module, several long-standing bugs where fixed that, in rare cases, could cause capture groups to get the wrong result. Therefore, this could change the captured output in these cases. (Contributed by Ma Lin in `35859`{.interpreted-text role="issue"}.)
## Build Changes {#whatsnew311-build-changes}
- CPython now has `11`{.interpreted-text role="pep"} `Tier 3 support <11#tier-3>`{.interpreted-text role="pep"} for cross compiling to the [WebAssembly](https://webassembly.org/) platforms [Emscripten](https://emscripten.org/) (`wasm32-unknown-emscripten`, i.e. Python in the browser) and [WebAssembly System Interface (WASI)](https://wasi.dev/) (`wasm32-unknown-wasi`). The effort is inspired by previous work like [Pyodide](https://pyodide.org/). These platforms provide a limited subset of POSIX APIs; Python standard libraries features and modules related to networking, processes, threading, signals, mmap, and users/groups are not available or don\'t work. (Emscripten contributed by Christian Heimes and Ethan Smith in `84461`{.interpreted-text role="gh"} and WASI contributed by Christian Heimes in `90473`{.interpreted-text role="gh"}; platforms promoted in `95085`{.interpreted-text role="gh"})
- Building CPython now requires:
- A [C11](https://en.cppreference.com/w/c/11) compiler and standard library. [Optional C11 features](https://en.wikipedia.org/wiki/C11_(C_standard_revision)#Optional_features) are not required. (Contributed by Victor Stinner in `46656`{.interpreted-text role="issue"}, `45440`{.interpreted-text role="issue"} and `46640`{.interpreted-text role="issue"}.)
- Support for [IEEE 754](https://en.wikipedia.org/wiki/IEEE_754) floating-point numbers. (Contributed by Victor Stinner in `46917`{.interpreted-text role="issue"}.)
- The `!Py_NO_NAN`{.interpreted-text role="c:macro"} macro has been removed. Since CPython now requires IEEE 754 floats, NaN values are always available. (Contributed by Victor Stinner in `46656`{.interpreted-text role="issue"}.)
- The `tkinter`{.interpreted-text role="mod"} package now requires [Tcl/Tk](https://www.tcl.tk) version 8.5.12 or newer. (Contributed by Serhiy Storchaka in `46996`{.interpreted-text role="issue"}.)
- Build dependencies, compiler flags, and linker flags for most stdlib extension modules are now detected by `configure`{.interpreted-text role="program"}. libffi, libnsl, libsqlite3, zlib, bzip2, liblzma, libcrypt, Tcl/Tk, and uuid flags are detected by [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/) (when available). `tkinter`{.interpreted-text role="mod"} now requires a pkg-config command to detect development settings for [Tcl/Tk](https://www.tcl.tk) headers and libraries. (Contributed by Christian Heimes and Erlend Egeberg Aasland in `45847`{.interpreted-text role="issue"}, `45747`{.interpreted-text role="issue"}, and `45763`{.interpreted-text role="issue"}.)
- libpython is no longer linked against libcrypt. (Contributed by Mike Gilbert in `45433`{.interpreted-text role="issue"}.)
- CPython can now be built with the [ThinLTO](https://clang.llvm.org/docs/ThinLTO.html) option via passing `thin` to `--with-lto`{.interpreted-text role="option"}, i.e. `--with-lto=thin`. (Contributed by Donghee Na and Brett Holman in `44340`{.interpreted-text role="issue"}.)
- Freelists for object structs can now be disabled. A new `configure`{.interpreted-text role="program"} option `--without-freelists` can be used to disable all freelists except empty tuple singleton. (Contributed by Christian Heimes in `45522`{.interpreted-text role="issue"}.)
- `Modules/Setup` and `Modules/makesetup` have been improved and tied up. Extension modules can now be built through `makesetup`. All except some test modules can be linked statically into a main binary or library. (Contributed by Brett Cannon and Christian Heimes in `45548`{.interpreted-text role="issue"}, `45570`{.interpreted-text role="issue"}, `45571`{.interpreted-text role="issue"}, and `43974`{.interpreted-text role="issue"}.)
:::: note
::: title
Note
:::
Use the environment variables `!TCLTK_CFLAGS`{.interpreted-text role="envvar"} and `!TCLTK_LIBS`{.interpreted-text role="envvar"} to manually specify the location of Tcl/Tk headers and libraries. The `configure`{.interpreted-text role="program"} options `!--with-tcltk-includes`{.interpreted-text role="option"} and `!--with-tcltk-libs`{.interpreted-text role="option"} have been removed.
On RHEL 7 and CentOS 7 the development packages do not provide `tcl.pc` and `tk.pc`; use `TCLTK_LIBS="-ltk8.5 -ltkstub8.5 -ltcl8.5"`. The directory `Misc/rhel7` contains `.pc` files and instructions on how to build Python with RHEL 7\'s and CentOS 7\'s Tcl/Tk and OpenSSL.
::::
- CPython will now use 30-bit digits by default for the Python `int`{.interpreted-text role="class"} implementation. Previously, the default was to use 30-bit digits on platforms with `SIZEOF_VOID_P >= 8`, and 15-bit digits otherwise. It\'s still possible to explicitly request use of 15-bit digits via either the `--enable-big-digits`{.interpreted-text role="option"} option to the configure script or (for Windows) the `PYLONG_BITS_IN_DIGIT` variable in `PC/pyconfig.h`, but this option may be removed at some point in the future. (Contributed by Mark Dickinson in `45569`{.interpreted-text role="issue"}.)
## C API Changes {#whatsnew311-c-api}
### New Features {#whatsnew311-c-api-new-features}
- Add a new `PyType_GetName`{.interpreted-text role="c:func"} function to get type\'s short name. (Contributed by Hai Shi in `42035`{.interpreted-text role="issue"}.)
- Add a new `PyType_GetQualName`{.interpreted-text role="c:func"} function to get type\'s qualified name. (Contributed by Hai Shi in `42035`{.interpreted-text role="issue"}.)
- Add new `PyThreadState_EnterTracing`{.interpreted-text role="c:func"} and `PyThreadState_LeaveTracing`{.interpreted-text role="c:func"} functions to the limited C API to suspend and resume tracing and profiling. (Contributed by Victor Stinner in `43760`{.interpreted-text role="issue"}.)
- Added the `Py_Version`{.interpreted-text role="c:data"} constant which bears the same value as `PY_VERSION_HEX`{.interpreted-text role="c:macro"}. (Contributed by Gabriele N. Tornetta in `43931`{.interpreted-text role="issue"}.)
- `Py_buffer`{.interpreted-text role="c:type"} and APIs are now part of the limited API and the stable ABI:
- `PyObject_CheckBuffer`{.interpreted-text role="c:func"}
- `PyObject_GetBuffer`{.interpreted-text role="c:func"}
- `PyBuffer_GetPointer`{.interpreted-text role="c:func"}
- `PyBuffer_SizeFromFormat`{.interpreted-text role="c:func"}
- `PyBuffer_ToContiguous`{.interpreted-text role="c:func"}
- `PyBuffer_FromContiguous`{.interpreted-text role="c:func"}
- `PyObject_CopyData`{.interpreted-text role="c:func"}
- `PyBuffer_IsContiguous`{.interpreted-text role="c:func"}
- `PyBuffer_FillContiguousStrides`{.interpreted-text role="c:func"}
- `PyBuffer_FillInfo`{.interpreted-text role="c:func"}
- `PyBuffer_Release`{.interpreted-text role="c:func"}
- `PyMemoryView_FromBuffer`{.interpreted-text role="c:func"}
- `~PyBufferProcs.bf_getbuffer`{.interpreted-text role="c:member"} and `~PyBufferProcs.bf_releasebuffer`{.interpreted-text role="c:member"} type slots
(Contributed by Christian Heimes in `45459`{.interpreted-text role="issue"}.)
- Added the `PyType_GetModuleByDef`{.interpreted-text role="c:func"} function, used to get the module in which a method was defined, in cases where this information is not available directly (via `PyCMethod`{.interpreted-text role="c:type"}). (Contributed by Petr Viktorin in `46613`{.interpreted-text role="issue"}.)
- Add new functions to pack and unpack C double (serialize and deserialize): `PyFloat_Pack2`{.interpreted-text role="c:func"}, `PyFloat_Pack4`{.interpreted-text role="c:func"}, `PyFloat_Pack8`{.interpreted-text role="c:func"}, `PyFloat_Unpack2`{.interpreted-text role="c:func"}, `PyFloat_Unpack4`{.interpreted-text role="c:func"} and `PyFloat_Unpack8`{.interpreted-text role="c:func"}. (Contributed by Victor Stinner in `46906`{.interpreted-text role="issue"}.)
- Add new functions to get frame object attributes: `PyFrame_GetBuiltins`{.interpreted-text role="c:func"}, `PyFrame_GetGenerator`{.interpreted-text role="c:func"}, `PyFrame_GetGlobals`{.interpreted-text role="c:func"}, `PyFrame_GetLasti`{.interpreted-text role="c:func"}.
- Added two new functions to get and set the active exception instance: `PyErr_GetHandledException`{.interpreted-text role="c:func"} and `PyErr_SetHandledException`{.interpreted-text role="c:func"}. These are alternatives to `PyErr_SetExcInfo()`{.interpreted-text role="c:func"} and `PyErr_GetExcInfo()`{.interpreted-text role="c:func"} which work with the legacy 3-tuple representation of exceptions. (Contributed by Irit Katriel in `46343`{.interpreted-text role="issue"}.)
- Added the `PyConfig.safe_path`{.interpreted-text role="c:member"} member. (Contributed by Victor Stinner in `57684`{.interpreted-text role="gh"}.)
### Porting to Python 3.11 {#whatsnew311-c-api-porting}
::: {#whatsnew311-pep670}
- Some macros have been converted to static inline functions to avoid [macro pitfalls](https://gcc.gnu.org/onlinedocs/cpp/Macro-Pitfalls.html). The change should be mostly transparent to users, as the replacement functions will cast their arguments to the expected types to avoid compiler warnings due to static type checks. However, when the limited C API is set to \>=3.11, these casts are not done, and callers will need to cast arguments to their expected types. See `670`{.interpreted-text role="pep"} for more details. (Contributed by Victor Stinner and Erlend E. Aasland in `89653`{.interpreted-text role="gh"}.)
- `PyErr_SetExcInfo()`{.interpreted-text role="c:func"} no longer uses the `type` and `traceback` arguments, the interpreter now derives those values from the exception instance (the `value` argument). The function still steals references of all three arguments. (Contributed by Irit Katriel in `45711`{.interpreted-text role="issue"}.)
- `PyErr_GetExcInfo()`{.interpreted-text role="c:func"} now derives the `type` and `traceback` fields of the result from the exception instance (the `value` field). (Contributed by Irit Katriel in `45711`{.interpreted-text role="issue"}.)
- `_frozen`{.interpreted-text role="c:struct"} has a new `is_package` field to indicate whether or not the frozen module is a package. Previously, a negative value in the `size` field was the indicator. Now only non-negative values be used for `size`. (Contributed by Kumar Aditya in `46608`{.interpreted-text role="issue"}.)
- `_PyFrameEvalFunction`{.interpreted-text role="c:func"} now takes `_PyInterpreterFrame*` as its second parameter, instead of `PyFrameObject*`. See `523`{.interpreted-text role="pep"} for more details of how to use this function pointer type.
- `!PyCode_New`{.interpreted-text role="c:func"} and `!PyCode_NewWithPosOnlyArgs`{.interpreted-text role="c:func"} now take an additional `exception_table` argument. Using these functions should be avoided, if at all possible. To get a custom code object: create a code object using the compiler, then get a modified version with the `replace` method.
- `PyCodeObject`{.interpreted-text role="c:type"} no longer has the `co_code`, `co_varnames`, `co_cellvars` and `co_freevars` fields. Instead, use `PyCode_GetCode`{.interpreted-text role="c:func"}, `PyCode_GetVarnames`{.interpreted-text role="c:func"}, `PyCode_GetCellvars`{.interpreted-text role="c:func"} and `PyCode_GetFreevars`{.interpreted-text role="c:func"} respectively to access them via the C API. (Contributed by Brandt Bucher in `46841`{.interpreted-text role="issue"} and Ken Jin in `92154`{.interpreted-text role="gh"} and `94936`{.interpreted-text role="gh"}.)
- The old trashcan macros (`Py_TRASHCAN_SAFE_BEGIN`/`Py_TRASHCAN_SAFE_END`) are now deprecated. They should be replaced by the new macros `Py_TRASHCAN_BEGIN` and `Py_TRASHCAN_END`.
A tp_dealloc function that has the old macros, such as:
static void
mytype_dealloc(mytype *p)
{
PyObject_GC_UnTrack(p);
Py_TRASHCAN_SAFE_BEGIN(p);
...
Py_TRASHCAN_SAFE_END
}
should migrate to the new macros as follows:
static void
mytype_dealloc(mytype *p)
{
PyObject_GC_UnTrack(p);
Py_TRASHCAN_BEGIN(p, mytype_dealloc)
...
Py_TRASHCAN_END
}
Note that `Py_TRASHCAN_BEGIN` has a second argument which should be the deallocation function it is in.
To support older Python versions in the same codebase, you can define the following macros and use them throughout the code (credit: these were copied from the `mypy` codebase):
#if PY_VERSION_HEX >= 0x03080000
# define CPy_TRASHCAN_BEGIN(op, dealloc) Py_TRASHCAN_BEGIN(op, dealloc)
# define CPy_TRASHCAN_END(op) Py_TRASHCAN_END
#else
# define CPy_TRASHCAN_BEGIN(op, dealloc) Py_TRASHCAN_SAFE_BEGIN(op)
# define CPy_TRASHCAN_END(op) Py_TRASHCAN_SAFE_END(op)
#endif
- The `PyType_Ready`{.interpreted-text role="c:func"} function now raises an error if a type is defined with the `Py_TPFLAGS_HAVE_GC`{.interpreted-text role="c:macro"} flag set but has no traverse function (`PyTypeObject.tp_traverse`{.interpreted-text role="c:member"}). (Contributed by Victor Stinner in `44263`{.interpreted-text role="issue"}.)
- Heap types with the `Py_TPFLAGS_IMMUTABLETYPE`{.interpreted-text role="c:macro"} flag can now inherit the `590`{.interpreted-text role="pep"} vectorcall protocol. Previously, this was only possible for `static types <static-types>`{.interpreted-text role="ref"}. (Contributed by Erlend E. Aasland in `43908`{.interpreted-text role="issue"})
- Since `Py_TYPE()`{.interpreted-text role="c:func"} is changed to a inline static function, `Py_TYPE(obj) = new_type` must be replaced with `Py_SET_TYPE(obj, new_type)`: see the `Py_SET_TYPE()`{.interpreted-text role="c:func"} function (available since Python 3.9). For backward compatibility, this macro can be used:
#if PY_VERSION_HEX < 0x030900A4 && !defined(Py_SET_TYPE)
static inline void _Py_SET_TYPE(PyObject *ob, PyTypeObject *type)
{ ob->ob_type = type; }
#define Py_SET_TYPE(ob, type) _Py_SET_TYPE((PyObject*)(ob), type)
#endif
(Contributed by Victor Stinner in `39573`{.interpreted-text role="issue"}.)
- Since `Py_SIZE()`{.interpreted-text role="c:func"} is changed to a inline static function, `Py_SIZE(obj) = new_size` must be replaced with `Py_SET_SIZE(obj, new_size)`: see the `Py_SET_SIZE()`{.interpreted-text role="c:func"} function (available since Python 3.9). For backward compatibility, this macro can be used:
#if PY_VERSION_HEX < 0x030900A4 && !defined(Py_SET_SIZE)
static inline void _Py_SET_SIZE(PyVarObject *ob, Py_ssize_t size)
{ ob->ob_size = size; }
#define Py_SET_SIZE(ob, size) _Py_SET_SIZE((PyVarObject*)(ob), size)
#endif
(Contributed by Victor Stinner in `39573`{.interpreted-text role="issue"}.)
- `<Python.h>` no longer includes the header files `<stdlib.h>`, `<stdio.h>`, `<errno.h>` and `<string.h>` when the `Py_LIMITED_API` macro is set to `0x030b0000` (Python 3.11) or higher. C extensions should explicitly include the header files after `#include <Python.h>`. (Contributed by Victor Stinner in `45434`{.interpreted-text role="issue"}.)
- The non-limited API files `cellobject.h`, `classobject.h`, `code.h`, `context.h`, `funcobject.h`, `genobject.h` and `longintrepr.h` have been moved to the `Include/cpython` directory. Moreover, the `eval.h` header file was removed. These files must not be included directly, as they are already included in `Python.h`: `Include Files <api-includes>`{.interpreted-text role="ref"}. If they have been included directly, consider including `Python.h` instead. (Contributed by Victor Stinner in `35134`{.interpreted-text role="issue"}.)
- The `!PyUnicode_CHECK_INTERNED`{.interpreted-text role="c:func"} macro has been excluded from the limited C API. It was never usable there, because it used internal structures which are not available in the limited C API. (Contributed by Victor Stinner in `46007`{.interpreted-text role="issue"}.)
- The following frame functions and type are now directly available with `#include <Python.h>`, it\'s no longer needed to add `#include <frameobject.h>`:
- `PyFrame_Check`{.interpreted-text role="c:func"}
- `PyFrame_GetBack`{.interpreted-text role="c:func"}
- `PyFrame_GetBuiltins`{.interpreted-text role="c:func"}
- `PyFrame_GetGenerator`{.interpreted-text role="c:func"}
- `PyFrame_GetGlobals`{.interpreted-text role="c:func"}
- `PyFrame_GetLasti`{.interpreted-text role="c:func"}
- `PyFrame_GetLocals`{.interpreted-text role="c:func"}
- `PyFrame_Type`{.interpreted-text role="c:type"}
(Contributed by Victor Stinner in `93937`{.interpreted-text role="gh"}.)
:::
::: {#pyframeobject-3.11-hiding}
- The `PyFrameObject`{.interpreted-text role="c:type"} structure members have been removed from the public C API.
While the documentation notes that the `PyFrameObject`{.interpreted-text role="c:type"} fields are subject to change at any time, they have been stable for a long time and were used in several popular extensions.
In Python 3.11, the frame struct was reorganized to allow performance optimizations. Some fields were removed entirely, as they were details of the old implementation.
`PyFrameObject`{.interpreted-text role="c:type"} fields:
- `f_back`: use `PyFrame_GetBack`{.interpreted-text role="c:func"}.
- `f_blockstack`: removed.
- `f_builtins`: use `PyFrame_GetBuiltins`{.interpreted-text role="c:func"}.
- `f_code`: use `PyFrame_GetCode`{.interpreted-text role="c:func"}.
- `f_gen`: use `PyFrame_GetGenerator`{.interpreted-text role="c:func"}.
- `f_globals`: use `PyFrame_GetGlobals`{.interpreted-text role="c:func"}.
- `f_iblock`: removed.
- `f_lasti`: use `PyFrame_GetLasti`{.interpreted-text role="c:func"}. Code using `f_lasti` with `PyCode_Addr2Line()` should use `PyFrame_GetLineNumber`{.interpreted-text role="c:func"} instead; it may be faster.
- `f_lineno`: use `PyFrame_GetLineNumber`{.interpreted-text role="c:func"}
- `f_locals`: use `PyFrame_GetLocals`{.interpreted-text role="c:func"}.
- `f_stackdepth`: removed.
- `f_state`: no public API (renamed to `f_frame.f_state`).
- `f_trace`: no public API.
- `f_trace_lines`: use `PyObject_GetAttrString((PyObject*)frame, "f_trace_lines")`.
- `f_trace_opcodes`: use `PyObject_GetAttrString((PyObject*)frame, "f_trace_opcodes")`.
- `f_localsplus`: no public API (renamed to `f_frame.localsplus`).
- `f_valuestack`: removed.
The Python frame object is now created lazily. A side effect is that the `~frame.f_back`{.interpreted-text role="attr"} member must not be accessed directly, since its value is now also computed lazily. The `PyFrame_GetBack`{.interpreted-text role="c:func"} function must be called instead.
Debuggers that accessed the `~frame.f_locals`{.interpreted-text role="attr"} directly *must* call `PyFrame_GetLocals`{.interpreted-text role="c:func"} instead. They no longer need to call `!PyFrame_FastToLocalsWithError`{.interpreted-text role="c:func"} or `!PyFrame_LocalsToFast`{.interpreted-text role="c:func"}, in fact they should not call those functions. The necessary updating of the frame is now managed by the virtual machine.
Code defining `PyFrame_GetCode()` on Python 3.8 and older:
#if PY_VERSION_HEX < 0x030900B1
static inline PyCodeObject* PyFrame_GetCode(PyFrameObject *frame)
{
Py_INCREF(frame->f_code);
return frame->f_code;
}
#endif
Code defining `PyFrame_GetBack()` on Python 3.8 and older:
#if PY_VERSION_HEX < 0x030900B1
static inline PyFrameObject* PyFrame_GetBack(PyFrameObject *frame)
{
Py_XINCREF(frame->f_back);
return frame->f_back;
}
#endif
Or use the [pythoncapi_compat project](https://github.com/python/pythoncapi-compat) to get these two functions on older Python versions.
- Changes of the `PyThreadState`{.interpreted-text role="c:type"} structure members:
- `frame`: removed, use `PyThreadState_GetFrame`{.interpreted-text role="c:func"} (function added to Python 3.9 by `40429`{.interpreted-text role="issue"}). Warning: the function returns a `strong reference`{.interpreted-text role="term"}, need to call `Py_XDECREF`{.interpreted-text role="c:func"}.
- `tracing`: changed, use `PyThreadState_EnterTracing`{.interpreted-text role="c:func"} and `PyThreadState_LeaveTracing`{.interpreted-text role="c:func"} (functions added to Python 3.11 by `43760`{.interpreted-text role="issue"}).
- `recursion_depth`: removed, use `(tstate->recursion_limit - tstate->recursion_remaining)` instead.
- `stackcheck_counter`: removed.
Code defining `PyThreadState_GetFrame()` on Python 3.8 and older:
#if PY_VERSION_HEX < 0x030900B1
static inline PyFrameObject* PyThreadState_GetFrame(PyThreadState *tstate)
{
Py_XINCREF(tstate->frame);
return tstate->frame;
}
#endif
Code defining `PyThreadState_EnterTracing()` and `PyThreadState_LeaveTracing()` on Python 3.10 and older:
#if PY_VERSION_HEX < 0x030B00A2
static inline void PyThreadState_EnterTracing(PyThreadState *tstate)
{
tstate->tracing++;
#if PY_VERSION_HEX >= 0x030A00A1
tstate->cframe->use_tracing = 0;
#else
tstate->use_tracing = 0;
#endif
}
static inline void PyThreadState_LeaveTracing(PyThreadState *tstate)
{
int use_tracing = (tstate->c_tracefunc != NULL || tstate->c_profilefunc != NULL);
tstate->tracing--;
#if PY_VERSION_HEX >= 0x030A00A1
tstate->cframe->use_tracing = use_tracing;
#else
tstate->use_tracing = use_tracing;
#endif
}
#endif
Or use [the pythoncapi-compat project](https://github.com/python/pythoncapi-compat) to get these functions on old Python functions.
- Distributors are encouraged to build Python with the optimized Blake2 library [libb2](https://www.blake2.net/).
- The `PyConfig.module_search_paths_set`{.interpreted-text role="c:member"} field must now be set to 1 for initialization to use `PyConfig.module_search_paths`{.interpreted-text role="c:member"} to initialize `sys.path`{.interpreted-text role="data"}. Otherwise, initialization will recalculate the path and replace any values added to `module_search_paths`.
- `PyConfig_Read`{.interpreted-text role="c:func"} no longer calculates the initial search path, and will not fill any values into `PyConfig.module_search_paths`{.interpreted-text role="c:member"}. To calculate default paths and then modify them, finish initialization and use `PySys_GetObject`{.interpreted-text role="c:func"} to retrieve `sys.path`{.interpreted-text role="data"} as a Python list object and modify it directly.
:::
### Deprecated {#whatsnew311-c-api-deprecated}
- Deprecate the following functions to configure the Python initialization:
- `!PySys_AddWarnOptionUnicode`{.interpreted-text role="c:func"}
- `!PySys_AddWarnOption`{.interpreted-text role="c:func"}
- `!PySys_AddXOption`{.interpreted-text role="c:func"}
- `!PySys_HasWarnOptions`{.interpreted-text role="c:func"}
- `!PySys_SetArgvEx`{.interpreted-text role="c:func"}
- `!PySys_SetArgv`{.interpreted-text role="c:func"}
- `!PySys_SetPath`{.interpreted-text role="c:func"}
- `!Py_SetPath`{.interpreted-text role="c:func"}
- `!Py_SetProgramName`{.interpreted-text role="c:func"}
- `!Py_SetPythonHome`{.interpreted-text role="c:func"}
- `!Py_SetStandardStreamEncoding`{.interpreted-text role="c:func"}
- `!_Py_SetProgramFullPath`{.interpreted-text role="c:func"}
Use the new `PyConfig`{.interpreted-text role="c:type"} API of the `Python Initialization Configuration
<init-config>`{.interpreted-text role="ref"} instead (`587`{.interpreted-text role="pep"}). (Contributed by Victor Stinner in `88279`{.interpreted-text role="gh"}.)
- Deprecate the `ob_shash` member of the `PyBytesObject`{.interpreted-text role="c:type"}. Use `PyObject_Hash`{.interpreted-text role="c:func"} instead. (Contributed by Inada Naoki in `46864`{.interpreted-text role="issue"}.)
### Pending Removal in Python 3.12 {#whatsnew311-c-api-pending-removal}
The following C APIs have been deprecated in earlier Python releases, and will be removed in Python 3.12.
- `!PyUnicode_AS_DATA`{.interpreted-text role="c:func"}
- `!PyUnicode_AS_UNICODE`{.interpreted-text role="c:func"}
- `!PyUnicode_AsUnicodeAndSize`{.interpreted-text role="c:func"}
- `!PyUnicode_AsUnicode`{.interpreted-text role="c:func"}
- `!PyUnicode_FromUnicode`{.interpreted-text role="c:func"}
- `!PyUnicode_GET_DATA_SIZE`{.interpreted-text role="c:func"}
- `!PyUnicode_GET_SIZE`{.interpreted-text role="c:func"}
- `!PyUnicode_GetSize`{.interpreted-text role="c:func"}
- `!PyUnicode_IS_COMPACT`{.interpreted-text role="c:func"}
- `!PyUnicode_IS_READY`{.interpreted-text role="c:func"}
- `PyUnicode_READY`{.interpreted-text role="c:func"}
- `!PyUnicode_WSTR_LENGTH`{.interpreted-text role="c:func"}
- `!_PyUnicode_AsUnicode`{.interpreted-text role="c:func"}
- `!PyUnicode_WCHAR_KIND`{.interpreted-text role="c:macro"}
- `PyUnicodeObject`{.interpreted-text role="c:type"}
- `!PyUnicode_InternImmortal`{.interpreted-text role="c:func"}
### Removed {#whatsnew311-c-api-removed}
- `!PyFrame_BlockSetup`{.interpreted-text role="c:func"} and `!PyFrame_BlockPop`{.interpreted-text role="c:func"} have been removed. (Contributed by Mark Shannon in `40222`{.interpreted-text role="issue"}.)
- Remove the following math macros using the `errno` variable:
- `Py_ADJUST_ERANGE1()`
- `Py_ADJUST_ERANGE2()`
- `Py_OVERFLOWED()`
- `Py_SET_ERANGE_IF_OVERFLOW()`
- `Py_SET_ERRNO_ON_MATH_ERROR()`
(Contributed by Victor Stinner in `45412`{.interpreted-text role="issue"}.)
- Remove `Py_UNICODE_COPY()` and `Py_UNICODE_FILL()` macros, deprecated since Python 3.3. Use `PyUnicode_CopyCharacters()` or `memcpy()` (`wchar_t*` string), and `PyUnicode_Fill()` functions instead. (Contributed by Victor Stinner in `41123`{.interpreted-text role="issue"}.)
- Remove the `pystrhex.h` header file. It only contains private functions. C extensions should only include the main `<Python.h>` header file. (Contributed by Victor Stinner in `45434`{.interpreted-text role="issue"}.)
- Remove the `Py_FORCE_DOUBLE()` macro. It was used by the `Py_IS_INFINITY()` macro. (Contributed by Victor Stinner in `45440`{.interpreted-text role="issue"}.)
- The following items are no longer available when `Py_LIMITED_API`{.interpreted-text role="c:macro"} is defined:
- `PyMarshal_WriteLongToFile`{.interpreted-text role="c:func"}
- `PyMarshal_WriteObjectToFile`{.interpreted-text role="c:func"}
- `PyMarshal_ReadObjectFromString`{.interpreted-text role="c:func"}
- `PyMarshal_WriteObjectToString`{.interpreted-text role="c:func"}
- the `Py_MARSHAL_VERSION` macro
These are not part of the `limited API <limited-api-list>`{.interpreted-text role="ref"}.
(Contributed by Victor Stinner in `45474`{.interpreted-text role="issue"}.)
- Exclude `!PyWeakref_GET_OBJECT`{.interpreted-text role="c:func"} from the limited C API. It never worked since the `!PyWeakReference`{.interpreted-text role="c:type"} structure is opaque in the limited C API. (Contributed by Victor Stinner in `35134`{.interpreted-text role="issue"}.)
- Remove the `PyHeapType_GET_MEMBERS()` macro. It was exposed in the public C API by mistake, it must only be used by Python internally. Use the `PyTypeObject.tp_members` member instead. (Contributed by Victor Stinner in `40170`{.interpreted-text role="issue"}.)
- Remove the `HAVE_PY_SET_53BIT_PRECISION` macro (moved to the internal C API). (Contributed by Victor Stinner in `45412`{.interpreted-text role="issue"}.)
::: {#whatsnew311-pep624}
- Remove the `Py_UNICODE`{.interpreted-text role="c:type"} encoder APIs, as they have been deprecated since Python 3.3, are little used and are inefficient relative to the recommended alternatives.
The removed functions are:
- `!PyUnicode_Encode`{.interpreted-text role="func"}
- `!PyUnicode_EncodeASCII`{.interpreted-text role="func"}
- `!PyUnicode_EncodeLatin1`{.interpreted-text role="func"}
- `!PyUnicode_EncodeUTF7`{.interpreted-text role="func"}
- `!PyUnicode_EncodeUTF8`{.interpreted-text role="func"}
- `!PyUnicode_EncodeUTF16`{.interpreted-text role="func"}
- `!PyUnicode_EncodeUTF32`{.interpreted-text role="func"}
- `!PyUnicode_EncodeUnicodeEscape`{.interpreted-text role="func"}
- `!PyUnicode_EncodeRawUnicodeEscape`{.interpreted-text role="func"}
- `!PyUnicode_EncodeCharmap`{.interpreted-text role="func"}
- `!PyUnicode_TranslateCharmap`{.interpreted-text role="func"}
- `!PyUnicode_EncodeDecimal`{.interpreted-text role="func"}
- `!PyUnicode_TransformDecimalToASCII`{.interpreted-text role="func"}
See `624`{.interpreted-text role="pep"} for details and `migration guidance <624#alternative-apis>`{.interpreted-text role="pep"}. (Contributed by Inada Naoki in `44029`{.interpreted-text role="issue"}.)
:::
## Notable changes in 3.11.4
### tarfile
- The extraction methods in `tarfile`{.interpreted-text role="mod"}, and `shutil.unpack_archive`{.interpreted-text role="func"}, have a new a *filter* argument that allows limiting tar features than may be surprising or dangerous, such as creating files outside the destination directory. See `tarfile-extraction-filter`{.interpreted-text role="ref"} for details. In Python 3.12, use without the *filter* argument will show a `DeprecationWarning`{.interpreted-text role="exc"}. In Python 3.14, the default will switch to `'data'`. (Contributed by Petr Viktorin in `706`{.interpreted-text role="pep"}.)
## Notable changes in 3.11.5
### OpenSSL
- Windows builds and macOS installers from python.org now use OpenSSL 3.0.
[^1]: A similar optimization already existed since Python 3.8. 3.11 specializes for more forms and reduces some overhead.
[^2]: A similar optimization already existed since Python 3.10. 3.11 specializes for more forms. Furthermore, all attribute loads should be sped up by `45947`{.interpreted-text role="issue"}.
[^3]: All jump opcodes are now relative, including the existing `!JUMP_IF_TRUE_OR_POP`{.interpreted-text role="opcode"} and `!JUMP_IF_FALSE_OR_POP`{.interpreted-text role="opcode"}. The argument is now an offset from the current instruction rather than an absolute location.