What’s New In Python 3.0

What’s New In Python 3.0¶
- Author:
Guido van Rossum
This article explains the new features in Python 3.0, compared to 2.6. Python 3.0, also known as “Python 3000” or “Py3K”, is the first ever intentionally backwards incompatible Python release. Python 3.0 was released on December 3, 2008. There are more changes than in a typical release, and more that are important for all Python users. Nevertheless, after digesting the changes, you’ll find that Python really hasn’t changed all that much – by and large, we’re mostly fixing well-known annoyances and warts, and removing a lot of old cruft.
This article doesn’t attempt to provide a complete specification of all new features, but instead tries to give a convenient overview. For full details, you should refer to the documentation for Python 3.0, and/or the many PEPs referenced in the text. If you want to understand the complete implementation and design rationale for a particular feature, PEPs usually have more details than the regular documentation; but note that PEPs usually are not kept up-to-date once a feature has been fully implemented.
Due to time constraints this document is not as complete as it should
have been. As always for a new release, the Misc/NEWS
file in the
source distribution contains a wealth of detailed information about
every small thing that was changed.
Common Stumbling Blocks¶
This section lists those few changes that are most likely to trip you up if you’re used to Python 2.5.
Print Is A Function¶
The print
statement has been replaced with a print()
function, with keyword arguments to replace most of the special syntax
of the old print
statement (PEP 3105). Examples:
Old: print "The answer is", 2*2
New: print("The answer is", 2*2)
Old: print x, # Trailing comma suppresses newline
New: print(x, end=" ") # Appends a space instead of a newline
Old: print # Prints a newline
New: print() # You must call the function!
Old: print >>sys.stderr, "fatal error"
New: print("fatal error", file=sys.stderr)
Old: print (x, y) # prints repr((x, y))
New: print((x, y)) # Not the same as print(x, y)!
You can also customize the separator between items, e.g.:
print("There are <", 2**32, "> possibilities!", sep="")
which produces:
There are <4294967296> possibilities!
Note:
The
print()
function doesn’t support the “softspace” feature of the oldprint
statement. For example, in Python 2.x,print "A\n", "B"
would write"A\nB\n"
; but in Python 3.0,print("A\n", "B")
writes"A\n B\n"
.Initially, you’ll be finding yourself typing the old
print x
a lot in interactive mode. Time to retrain your fingers to typeprint(x)
instead!When using the
2to3
source-to-source conversion tool, allprint
statements are automatically converted toprint()
function calls, so this is mostly a non-issue for larger projects.
Views And Iterators Instead Of Lists¶
Some well-known APIs no longer return lists:
dict
methodsdict.keys()
,dict.items()
anddict.values()
return “views” instead of lists. For example, this no longer works:k = d.keys(); k.sort()
. Usek = sorted(d)
instead (this works in Python 2.5 too and is just as efficient).Also, the
dict.iterkeys()
,dict.iteritems()
anddict.itervalues()
methods are no longer supported.map()
andfilter()
return iterators. If you really need a list and the input sequences are all of equal length, a quick fix is to wrapmap()
inlist()
, e.g.list(map(...))
, but a better fix is often to use a list comprehension (especially when the original code useslambda
), or rewriting the code so it doesn’t need a list at all. Particularly tricky ismap()
invoked for the side effects of the function; the correct transformation is to use a regularfor
loop (since creating a list would just be wasteful).If the input sequences are not of equal length,
map()
will stop at the termination of the shortest of the sequences. For full compatibility withmap()
from Python 2.x, also wrap the sequences initertools.zip_longest()
, e.g.map(func, *sequences)
becomeslist(map(func, itertools.zip_longest(*sequences)))
.range()
now behaves likexrange()
used to behave, except it works with values of arbitrary size. The latter no longer exists.zip()
now returns an iterator.
Ordering Comparisons¶
Python 3.0 has simplified the rules for ordering comparisons:
The ordering comparison operators (
<
,<=
,>=
,>
) raise a TypeError exception when the operands don’t have a meaningful natural ordering. Thus, expressions like1 < ''
,0 > None
orlen <= len
are no longer valid, and e.g.None < None
raisesTypeError
instead of returningFalse
. A corollary is that sorting a heterogeneous list no longer makes sense – all the elements must be comparable to each other. Note that this does not apply to the==
and!=
operators: objects of different incomparable types always compare unequal to each other.sorted()
andlist.sort()
no longer accept the cmp argument providing a comparison function. Use the key argument instead. N.B. the key and reverse arguments are now “keyword-only”.The
cmp()
function should be treated as gone, and the__cmp__()
special method is no longer supported. Use__lt__()
for sorting,__eq__()
with__hash__()
, and other rich comparisons as needed. (If you really need thecmp()
functionality, you could use the expression(a > b) - (a < b)
as the equivalent forcmp(a, b)
.)
Integers¶
PEP 237: Essentially,
long
renamed toint
. That is, there is only one built-in integral type, namedint
; but it behaves mostly like the oldlong
type.PEP 238: An expression like
1/2
returns a float. Use1//2
to get the truncating behavior. (The latter syntax has existed for years, at least since Python 2.2.)The
sys.maxint
constant was removed, since there is no longer a limit to the value of integers. However,sys.maxsize
can be used as an integer larger than any practical list or string index. It conforms to the implementation’s “natural” integer size and is typically the same assys.maxint
in previous releases on the same platform (assuming the same build options).The
repr()
of a long integer doesn’t include the trailingL
anymore, so code that unconditionally strips that character will chop off the last digit instead. (Usestr()
instead.)Octal literals are no longer of the form
0720
; use0o720
instead.
Text Vs. Data Instead Of Unicode Vs. 8-bit¶
Everything you thought you knew about binary data and Unicode has changed.
Python 3.0 uses the concepts of text and (binary) data instead of Unicode strings and 8-bit strings. All text is Unicode; however encoded Unicode is represented as binary data. The type used to hold text is
str
, the type used to hold data isbytes
. The biggest difference with the 2.x situation is that any attempt to mix text and data in Python 3.0 raisesTypeError
, whereas if you were to mix Unicode and 8-bit strings in Python 2.x, it would work if the 8-bit string happened to contain only 7-bit (ASCII) bytes, but you would getUnicodeDecodeError
if it contained non-ASCII values. This value-specific behavior has caused numerous sad faces over the years.As a consequence of this change in philosophy, pretty much all code that uses Unicode, encodings or binary data most likely has to change. The change is for the better, as in the 2.x world there were numerous bugs having to do with mixing encoded and unencoded text. To be prepared in Python 2.x, start using
unicode
for all unencoded text, andstr
for binary or encoded data only. Then the2to3
tool will do most of the work for you.You can no longer use
u"..."
literals for Unicode text. However, you must useb"..."
literals for binary data.As the
str
andbytes
types cannot be mixed, you must always explicitly convert between them. Usestr.encode()
to go fromstr
tobytes
, andbytes.decode()
to go frombytes
tostr
. You can also usebytes(s, encoding=...)
andstr(b, encoding=...)
, respectively.Like
str
, thebytes
type is immutable. There is a separate mutable type to hold buffered binary data,bytearray
. Nearly all APIs that acceptbytes
also acceptbytearray
. The mutable API is based oncollections.MutableSequence
.All backslashes in raw string literals are interpreted literally. This means that
'\U'
and'\u'
escapes in raw strings are not treated specially. For example,r'\u20ac'
is a string of 6 characters in Python 3.0, whereas in 2.6,ur'\u20ac'
was the single “euro” character. (Of course, this change only affects raw string literals; the euro character is'\u20ac'
in Python 3.0.)The built-in
basestring
abstract type was removed. Usestr
instead. Thestr
andbytes
types don’t have functionality enough in common to warrant a shared base class. The2to3
tool (see below) replaces every occurrence ofbasestring
withstr
.Files opened as text files (still the default mode for
open()
) always use an encoding to map between strings (in memory) and bytes (on disk). Binary files (opened with ab
in the mode argument) always use bytes in memory. This means that if a file is opened using an incorrect mode or encoding, I/O will likely fail loudly, instead of silently producing incorrect data. It also means that even Unix users will have to specify the correct mode (text or binary) when opening a file. There is a platform-dependent default encoding, which on Unixy platforms can be set with theLANG
environment variable (and sometimes also with some other platform-specific locale-related environment variables). In many cases, but not all, the system default is UTF-8; you should never count on this default. Any application reading or writing more than pure ASCII text should probably have a way to override the encoding. There is no longer any need for using the encoding-aware streams in thecodecs
module.The initial values of
sys.stdin
,sys.stdout
andsys.stderr
are now unicode-only text files (i.e., they are instances ofio.TextIOBase
). To read and write bytes data with these streams, you need to use theirio.TextIOBase.buffer
attribute.Filenames are passed to and returned from APIs as (Unicode) strings. This can present platform-specific problems because on some platforms filenames are arbitrary byte strings. (On the other hand, on Windows filenames are natively stored as Unicode.) As a work-around, most APIs (e.g.
open()
and many functions in theos
module) that take filenames acceptbytes
objects as well as strings, and a few APIs have a way to ask for abytes
return value. Thus,os.listdir()
returns a list ofbytes
instances if the argument is abytes
instance, andos.getcwdb()
returns the current working directory as abytes
instance. Note that whenos.listdir()
returns a list of strings, filenames that cannot be decoded properly are omitted rather than raisingUnicodeError
.Some system APIs like
os.environ
andsys.argv
can also present problems when the bytes made available by the system is not interpretable using the default encoding. Setting theLANG
variable and rerunning the program is probably the best approach.PEP 3138: The
repr()
of a string no longer escapes non-ASCII characters. It still escapes control characters and code points with non-printable status in the Unicode standard, however.PEP 3120: The default source encoding is now UTF-8.
PEP 3131: Non-ASCII letters are now allowed in identifiers. (However, the standard library remains ASCII-only with the exception of contributor names in comments.)
The
StringIO
andcStringIO
modules are gone. Instead, import theio
module and useio.StringIO
orio.BytesIO
for text and data respectively.See also the Unicode HOWTO, which was updated for Python 3.0.
Overview Of Syntax Changes¶
This section gives a brief overview of every syntactic change in Python 3.0.
New Syntax¶
PEP 3107: Function argument and return value annotations. This provides a standardized way of annotating a function’s parameters and return value. There are no semantics attached to such annotations except that they can be introspected at runtime using the
__annotations__
attribute. The intent is to encourage experimentation through metaclasses, decorators or frameworks.PEP 3102: Keyword-only arguments. Named parameters occurring after
*args
in the parameter list must be specified using keyword syntax in the call. You can also use a bare*
in the parameter list to indicate that you don’t accept a variable-length argument list, but you do have keyword-only arguments.Keyword arguments are allowed after the list of base classes in a class definition. This is used by the new convention for specifying a metaclass (see next section), but can be used for other purposes as well, as long as the metaclass supports it.
PEP 3104:
nonlocal
statement. Usingnonlocal x
you can now assign directly to a variable in an outer (but non-global) scope.nonlocal
is a new reserved word.PEP 3132: Extended Iterable Unpacking. You can now write things like
a, b, *rest = some_sequence
. And even*rest, a = stuff
. Therest
object is always a (possibly empty) list; the right-hand side may be any iterable. Example:(a, *rest, b) = range(5)
This sets a to
0
, b to4
, and rest to[1, 2, 3]
.Dictionary comprehensions:
{k: v for k, v in stuff}
means the same thing asdict(stuff)
but is more flexible. (This is PEP 274 vindicated. :-)Set literals, e.g.
{1, 2}
. Note that{}
is an empty dictionary; useset()
for an empty set. Set comprehensions are also supported; e.g.,{x for x in stuff}
means the same thing asset(stuff)
but is more flexible.New octal literals, e.g.
0o720
(already in 2.6). The old octal literals (0720
) are gone.New binary literals, e.g.
0b1010
(already in 2.6), and there is a new corresponding built-in function,bin()
.Bytes literals are introduced with a leading
b
orB
, and there is a new corresponding built-in function,bytes()
.
Changed Syntax¶
PEP 3109 and PEP 3134: new
raise
statement syntax:raise [expr [from expr]]
. See below.as
andwith
are now reserved words. (Since 2.6, actually.)True
,False
, andNone
are reserved words. (2.6 partially enforced the restrictions onNone
already.)Change from
except
exc, var toexcept
excas
var. See PEP 3110.PEP 3115: New Metaclass Syntax. Instead of:
class C: __metaclass__ = M ...
you must now use:
class C(metaclass=M): ...
The module-global
__metaclass__
variable is no longer supported. (It was a crutch to make it easier to default to new-style classes without deriving every class fromobject
.)List comprehensions no longer support the syntactic form
[... for var in item1, item2, ...]
. Use[... for var in (item1, item2, ...)]
instead. Also note that list comprehensions have different semantics: they are closer to syntactic sugar for a generator expression inside alist()
constructor, and in particular the loop control variables are no longer leaked into the surrounding scope.The ellipsis (
...
) can be used as an atomic expression anywhere. (Previously it was only allowed in slices.) Also, it must now be spelled as...
. (Previously it could also be spelled as. . .
, by a mere accident of the grammar.)
Removed Syntax¶
PEP 3113: Tuple parameter unpacking removed. You can no longer write
def foo(a, (b, c)): ...
. Usedef foo(a, b_c): b, c = b_c
instead.Removed backticks (use
repr()
instead).Removed
<>
(use!=
instead).Removed keyword:
exec()
is no longer a keyword; it remains as a function. (Fortunately the function syntax was also accepted in 2.x.) Also note thatexec()
no longer takes a stream argument; instead ofexec(f)
you can useexec(f.read())
.Integer literals no longer support a trailing
l
orL
.String literals no longer support a leading
u
orU
.The
from
moduleimport
*
syntax is only allowed at the module level, no longer inside functions.The only acceptable syntax for relative imports is
from .[module] import name
. Allimport
forms not starting with.
are interpreted as absolute imports. (PEP 328)Classic classes are gone.
Changes Already Present In Python 2.6¶
Since many users presumably make the jump straight from Python 2.5 to Python 3.0, this section reminds the reader of new features that were originally designed for Python 3.0 but that were back-ported to Python 2.6. The corresponding sections in What’s New in Python 2.6 should be consulted for longer descriptions.
PEP 343: The ‘with’ statement. The
with
statement is now a standard feature and no longer needs to be imported from the__future__
. Also check out Writing Context Managers and The contextlib module.PEP 366: Explicit Relative Imports From a Main Module. This enhances the usefulness of the
-m
option when the referenced module lives in a package.PEP 3101: Advanced String Formatting. Note: the 2.6 description mentions the
format()
method for both 8-bit and Unicode strings. In 3.0, only thestr
type (text strings with Unicode support) supports this method; thebytes
type does not. The plan is to eventually make this the only API for string formatting, and to start deprecating the%
operator in Python 3.1.PEP 3105: print As a Function. This is now a standard feature and no longer needs to be imported from
__future__
. More details were given above.PEP 3110: Exception-Handling Changes. The
except
excas
var syntax is now standard andexcept
exc, var is no longer supported. (Of course, theas
var part is still optional.)PEP 3112: Byte Literals. The
b"..."
string literal notation (and its variants likeb'...'
,b"""..."""
, andbr"..."
) now produces a literal of typebytes
.PEP 3116: New I/O Library. The
io
module is now the standard way of doing file I/O. The built-inopen()
function is now an alias forio.open()
and has additional keyword arguments encoding, errors, newline and closefd. Also note that an invalid mode argument now raisesValueError
, notIOError
. The binary file object underlying a text file object can be accessed asf.buffer
(but beware that the text object maintains a buffer of itself in order to speed up the encoding and decoding operations).PEP 3118: Revised Buffer Protocol. The old builtin
buffer()
is now really gone; the new builtinmemoryview()
provides (mostly) similar functionality.PEP 3119: Abstract Base Classes. The
abc
module and the ABCs defined in thecollections
module plays a somewhat more prominent role in the language now, and built-in collection types likedict
andlist
conform to thecollections.MutableMapping
andcollections.MutableSequence
ABCs, respectively.PEP 3127: Integer Literal Support and Syntax. As mentioned above, the new octal literal notation is the only one supported, and binary literals have been added.
PEP 3141: A Type Hierarchy for Numbers. The
numbers
module is another new use of ABCs, defining Python’s “numeric tower”. Also note the newfractions
module which implementsnumbers.Rational
.
Library Changes¶
Due to time constraints, this document does not exhaustively cover the very extensive changes to the standard library. PEP 3108 is the reference for the major changes to the library. Here’s a capsule review:
Many old modules were removed. Some, like
gopherlib
(no longer used) andmd5
(replaced byhashlib
), were already deprecated by PEP 4. Others were removed as a result of the removal of support for various platforms such as Irix, BeOS and Mac OS 9 (see PEP 11). Some modules were also selected for removal in Python 3.0 due to lack of use or because a better replacement exists. See PEP 3108 for an exhaustive list.The
bsddb3
package was removed because its presence in the core standard library has proved over time to be a particular burden for the core developers due to testing instability and Berkeley DB’s release schedule. However, the package is alive and well, externally maintained at https://www.jcea.es/programacion/pybsddb.htm.Some modules were renamed because their old name disobeyed PEP 8, or for various other reasons. Here’s the list:
Old Name
New Name
_winreg
winreg
ConfigParser
configparser
copy_reg
copyreg
Queue
queue
SocketServer
socketserver
markupbase
_markupbase
repr
reprlib
test.test_support
test.support
A common pattern in Python 2.x is to have one version of a module implemented in pure Python, with an optional accelerated version implemented as a C extension; for example,
pickle
andcPickle
. This places the burden of importing the accelerated version and falling back on the pure Python version on each user of these modules. In Python 3.0, the accelerated versions are considered implementation details of the pure Python versions. Users should always import the standard version, which attempts to import the accelerated version and falls back to the pure Python version. Thepickle
/cPickle
pair received this treatment. Theprofile
module is on the list for 3.1. TheStringIO
module has been turned into a class in theio
module.Some related modules have been grouped into packages, and usually the submodule names have been simplified. The resulting new packages are:
dbm
(anydbm
,dbhash
,dbm
,dumbdbm
,gdbm
,whichdb
).html
(HTMLParser
,htmlentitydefs
).http
(httplib
,BaseHTTPServer
,CGIHTTPServer
,SimpleHTTPServer
,Cookie
,cookielib
).tkinter
(allTkinter
-related modules exceptturtle
). The target audience ofturtle
doesn’t really care abouttkinter
. Also note that as of Python 2.6, the functionality ofturtle
has been greatly enhanced.urllib
(urllib
,urllib2
,urlparse
,robotparse
).xmlrpc
(xmlrpclib
,DocXMLRPCServer
,SimpleXMLRPCServer
).
Some other changes to standard library modules, not covered by PEP 3108:
Killed
sets
. Use the built-inset()
class.Cleanup of the
sys
module: removedsys.exitfunc()
,sys.exc_clear()
,sys.exc_type
,sys.exc_value
,sys.exc_traceback
. (Note thatsys.last_type
etc. remain.)Cleanup of the
array.array
type: theread()
andwrite()
methods are gone; usefromfile()
andtofile()
instead. Also, the'c'
typecode for array is gone – use either'b'
for bytes or'u'
for Unicode characters.Cleanup of the
operator
module: removedsequenceIncludes()
andisCallable()
.Cleanup of the
thread
module:acquire_lock()
andrelease_lock()
are gone; useacquire()
andrelease()
instead.Cleanup of the
random
module: removed thejumpahead()
API.The
new
module is gone.The functions
os.tmpnam()
,os.tempnam()
andos.tmpfile()
have been removed in favor of thetempfile
module.The
tokenize
module has been changed to work with bytes. The main entry point is nowtokenize.tokenize()
, instead of generate_tokens.string.letters
and its friends (string.lowercase
andstring.uppercase
) are gone. Usestring.ascii_letters
etc. instead. (The reason for the removal is thatstring.letters
and friends had locale-specific behavior, which is a bad idea for such attractively named global “constants”.)Renamed module
__builtin__
tobuiltins
(removing the underscores, adding an ‘s’). The__builtins__
variable found in most global namespaces is unchanged. To modify a builtin, you should usebuiltins
, not__builtins__
!
PEP 3101: A New Approach To String Formatting¶
A new system for built-in string formatting operations replaces the
%
string formatting operator. (However, the%
operator is still supported; it will be deprecated in Python 3.1 and removed from the language at some later time.) Read PEP 3101 for the full scoop.
Changes To Exceptions¶
The APIs for raising and catching exception have been cleaned up and new powerful features added:
PEP 352: All exceptions must be derived (directly or indirectly) from
BaseException
. This is the root of the exception hierarchy. This is not new as a recommendation, but the requirement to inherit fromBaseException
is new. (Python 2.6 still allowed classic classes to be raised, and placed no restriction on what you can catch.) As a consequence, string exceptions are finally truly and utterly dead.Almost all exceptions should actually derive from
Exception
;BaseException
should only be used as a base class for exceptions that should only be handled at the top level, such asSystemExit
orKeyboardInterrupt
. The recommended idiom for handling all exceptions except for this latter category is to useexcept
Exception
.StandardError
was removed.Exceptions no longer behave as sequences. Use the
args
attribute instead.PEP 3109: Raising exceptions. You must now use
raise Exception(args)
instead ofraise Exception, args
. Additionally, you can no longer explicitly specify a traceback; instead, if you have to do this, you can assign directly to the__traceback__
attribute (see below).PEP 3110: Catching exceptions. You must now use
except SomeException as variable
instead ofexcept SomeException, variable
. Moreover, the variable is explicitly deleted when theexcept
block is left.PEP 3134: Exception chaining. There are two cases: implicit chaining and explicit chaining. Implicit chaining happens when an exception is raised in an
except
orfinally
handler block. This usually happens due to a bug in the handler block; we call this a secondary exception. In this case, the original exception (that was being handled) is saved as the__context__
attribute of the secondary exception. Explicit chaining is invoked with this syntax:raise SecondaryException() from primary_exception
(where primary_exception is any expression that produces an exception object, probably an exception that was previously caught). In this case, the primary exception is stored on the
__cause__
attribute of the secondary exception. The traceback printed when an unhandled exception occurs walks the chain of__cause__
and__context__
attributes and prints a separate traceback for each component of the chain, with the primary exception at the top. (Java users may recognize this behavior.)PEP 3134: Exception objects now store their traceback as the
__traceback__
attribute. This means that an exception object now contains all the information pertaining to an exception, and there are fewer reasons to usesys.exc_info()
(though the latter is not removed).A few exception messages are improved when Windows fails to load an extension module. For example,
error code 193
is now%1 is not a valid Win32 application
. Strings now deal with non-English locales.
Miscellaneous Other Changes¶
Operators And Special Methods¶
!=
now returns the opposite of==
, unless==
returnsNotImplemented
.The concept of “unbound methods” has been removed from the language. When referencing a method as a class attribute, you now get a plain function object.
__getslice__()
,__setslice__()
and__delslice__()
were killed. The syntaxa[i:j]
now translates toa.__getitem__(slice(i, j))
(or__setitem__()
or__delitem__()
, when used as an assignment or deletion target, respectively).PEP 3114: the standard
next()
method has been renamed to__next__()
.The
__oct__()
and__hex__()
special methods are removed –oct()
andhex()
use__index__()
now to convert the argument to an integer.Removed support for
__members__
and__methods__
.The function attributes named
func_X
have been renamed to use the__X__
form, freeing up these names in the function attribute namespace for user-defined attributes. To wit,func_closure
,func_code
,func_defaults
,func_dict
,func_doc
,func_globals
,func_name
were renamed to__closure__
,__code__
,__defaults__
,__dict__
,__doc__
,__globals__
,__name__
, respectively.__nonzero__()
is now__bool__()
.
Builtins¶
PEP 3135: New
super()
. You can now invokesuper()
without arguments and (assuming this is in a regular instance method defined inside aclass
statement) the right class and instance will automatically be chosen. With arguments, the behavior ofsuper()
is unchanged.PEP 3111:
raw_input()
was renamed toinput()
. That is, the newinput()
function reads a line fromsys.stdin
and returns it with the trailing newline stripped. It raisesEOFError
if the input is terminated prematurely. To get the old behavior ofinput()
, useeval(input())
.A new built-in function
next()
was added to call the__next__()
method on an object.The
round()
function rounding strategy and return type have changed. Exact halfway cases are now rounded to the nearest even result instead of away from zero. (For example,round(2.5)
now returns2
rather than3
.)round(x[, n])
now delegates tox.__round__([n])
instead of always returning a float. It generally returns an integer when called with a single argument and a value of the same type asx
when called with two arguments.Moved
intern()
tosys.intern()
.Removed:
apply()
. Instead ofapply(f, args)
usef(*args)
.Removed
callable()
. Instead ofcallable(f)
you can useisinstance(f, collections.Callable)
. Theoperator.isCallable()
function is also gone.Removed
coerce()
. This function no longer serves a purpose now that classic classes are gone.Removed
execfile()
. Instead ofexecfile(fn)
useexec(open(fn).read())
.Removed the
file
type. Useopen()
. There are now several different kinds of streams that open can return in theio
module.Removed
reduce()
. Usefunctools.reduce()
if you really need it; however, 99 percent of the time an explicitfor
loop is more readable.Removed
reload()
. Useimp.reload()
.Removed.
dict.has_key()
– use thein
operator instead.
Build and C API Changes¶
Due to time constraints, here is a very incomplete list of changes to the C API.
Support for several platforms was dropped, including but not limited to Mac OS 9, BeOS, RISCOS, Irix, and Tru64.
PEP 3118: New Buffer API.
PEP 3121: Extension Module Initialization & Finalization.
PEP 3123: Making
PyObject_HEAD
conform to standard C.No more C API support for restricted execution.
PyNumber_Coerce()
,PyNumber_CoerceEx()
,PyMember_Get()
, andPyMember_Set()
C APIs are removed.New C API
PyImport_ImportModuleNoBlock()
, works likePyImport_ImportModule()
but won’t block on the import lock (returning an error instead).Renamed the boolean conversion C-level slot and method:
nb_nonzero
is nownb_bool
.Removed
METH_OLDARGS
andWITH_CYCLE_GC
from the C API.
Performance¶
The net result of the 3.0 generalizations is that Python 3.0 runs the pystone benchmark around 10% slower than Python 2.5. Most likely the biggest cause is the removal of special-casing for small integers. There’s room for improvement, but it will happen after 3.0 is released!
Porting To Python 3.0¶
For porting existing Python 2.5 or 2.6 source code to Python 3.0, the best strategy is the following:
(Prerequisite:) Start with excellent test coverage.
Port to Python 2.6. This should be no more work than the average port from Python 2.x to Python 2.(x+1). Make sure all your tests pass.
(Still using 2.6:) Turn on the
-3
command line switch. This enables warnings about features that will be removed (or change) in 3.0. Run your test suite again, and fix code that you get warnings about until there are no warnings left, and all your tests still pass.Run the
2to3
source-to-source translator over your source code tree. Run the result of the translation under Python 3.0. Manually fix up any remaining issues, fixing problems until all tests pass again.
It is not recommended to try to write source code that runs unchanged
under both Python 2.6 and 3.0; you’d have to use a very contorted
coding style, e.g. avoiding print
statements, metaclasses,
and much more. If you are maintaining a library that needs to support
both Python 2.6 and Python 3.0, the best approach is to modify step 3
above by editing the 2.6 version of the source code and running the
2to3
translator again, rather than editing the 3.0 version of the
source code.
For porting C extensions to Python 3.0, please see Porting Extension Modules to Python 3.


— Function to check Unix passwords

crypt
— Function to check Unix passwords¶
Deprecated since version 3.11, removed in version 3.13.
This module is no longer part of the Python standard library. It was removed in Python 3.13 after being deprecated in Python 3.11. The removal was decided in PEP 594.
Applications can use the hashlib
module from the standard library.
Other possible replacements are third-party libraries from PyPI:
legacycrypt, bcrypt, or argon2-cffi.
These are not supported or maintained by the Python core team.
The last version of Python that provided the crypt
module was
Python 3.12.


Complex Number Objects

Complex Number Objects¶
Python’s complex number objects are implemented as two distinct types when viewed from the C API: one is the Python object exposed to Python programs, and the other is a C structure which represents the actual complex number value. The API provides functions for working with both.
Complex Numbers as C Structures¶
Note that the functions which accept these structures as parameters and return them as results do so by value rather than dereferencing them through pointers. This is consistent throughout the API.
-
type Py_complex¶
The C structure which corresponds to the value portion of a Python complex number object. Most of the functions for dealing with complex number objects use structures of this type as input or output values, as appropriate.
The structure is defined as:
typedef struct { double real; double imag; } Py_complex;
-
Py_complex _Py_c_sum(Py_complex left, Py_complex right)¶
Return the sum of two complex numbers, using the C
Py_complex
representation.
-
Py_complex _Py_c_diff(Py_complex left, Py_complex right)¶
Return the difference between two complex numbers, using the C
Py_complex
representation.
-
Py_complex _Py_c_neg(Py_complex num)¶
Return the negation of the complex number num, using the C
Py_complex
representation.
-
Py_complex _Py_c_prod(Py_complex left, Py_complex right)¶
Return the product of two complex numbers, using the C
Py_complex
representation.
-
Py_complex _Py_c_quot(Py_complex dividend, Py_complex divisor)¶
Return the quotient of two complex numbers, using the C
Py_complex
representation.If divisor is null, this method returns zero and sets
errno
toEDOM
.
-
Py_complex _Py_c_pow(Py_complex num, Py_complex exp)¶
Return the exponentiation of num by exp, using the C
Py_complex
representation.If num is null and exp is not a positive real number, this method returns zero and sets
errno
toEDOM
.Set
errno
toERANGE
on overflows.
Complex Numbers as Python Objects¶
-
PyTypeObject PyComplex_Type¶
- Part of the Stable ABI.
This instance of
PyTypeObject
represents the Python complex number type. It is the same object ascomplex
in the Python layer.
-
int PyComplex_Check(PyObject *p)¶
Return true if its argument is a
PyComplexObject
or a subtype ofPyComplexObject
. This function always succeeds.
-
int PyComplex_CheckExact(PyObject *p)¶
Return true if its argument is a
PyComplexObject
, but not a subtype ofPyComplexObject
. This function always succeeds.
-
PyObject *PyComplex_FromCComplex(Py_complex v)¶
- Return value: New reference.
Create a new Python complex number object from a C
Py_complex
value. ReturnNULL
with an exception set on error.
-
PyObject *PyComplex_FromDoubles(double real, double imag)¶
- Return value: New reference. Part of the Stable ABI.
Return a new
PyComplexObject
object from real and imag. ReturnNULL
with an exception set on error.
-
double PyComplex_RealAsDouble(PyObject *op)¶
- Part of the Stable ABI.
Return the real part of op as a C double.
If op is not a Python complex number object but has a
__complex__()
method, this method will first be called to convert op to a Python complex number object. If__complex__()
is not defined then it falls back to callPyFloat_AsDouble()
and returns its result.Upon failure, this method returns
-1.0
with an exception set, so one should callPyErr_Occurred()
to check for errors.Changed in version 3.13: Use
__complex__()
if available.
-
double PyComplex_ImagAsDouble(PyObject *op)¶
- Part of the Stable ABI.
Return the imaginary part of op as a C double.
If op is not a Python complex number object but has a
__complex__()
method, this method will first be called to convert op to a Python complex number object. If__complex__()
is not defined then it falls back to callPyFloat_AsDouble()
and returns0.0
on success.Upon failure, this method returns
-1.0
with an exception set, so one should callPyErr_Occurred()
to check for errors.Changed in version 3.13: Use
__complex__()
if available.
-
Py_complex PyComplex_AsCComplex(PyObject *op)¶
Return the
Py_complex
value of the complex number op.If op is not a Python complex number object but has a
__complex__()
method, this method will first be called to convert op to a Python complex number object. If__complex__()
is not defined then it falls back to__float__()
. If__float__()
is not defined then it falls back to__index__()
.Upon failure, this method returns
Py_complex
withreal
set to-1.0
and with an exception set, so one should callPyErr_Occurred()
to check for errors.Changed in version 3.8: Use
__index__()
if available.


— Common Gateway Interface support

cgi
— Common Gateway Interface support¶
Deprecated since version 3.11, removed in version 3.13.
This module is no longer part of the Python standard library. It was removed in Python 3.13 after being deprecated in Python 3.11. The removal was decided in PEP 594.
A fork of the module on PyPI can be used instead: legacy-cgi. This is a copy of the cgi module, no longer maintained or supported by the core Python team.
The last version of Python that provided the cgi
module was
Python 3.12.


— Interface to Sun’s NIS (Yellow Pages)

nis
— Interface to Sun’s NIS (Yellow Pages)¶
Deprecated since version 3.11, removed in version 3.13.
This module is no longer part of the Python standard library. It was removed in Python 3.13 after being deprecated in Python 3.11. The removal was decided in PEP 594.
The last version of Python that provided the nis
module was
Python 3.12.


Subprocesses

Subprocesses¶
Source code: Lib/asyncio/subprocess.py, Lib/asyncio/base_subprocess.py
This section describes high-level async/await asyncio APIs to create and manage subprocesses.
Here’s an example of how asyncio can run a shell command and obtain its result:
import asyncio
async def run(cmd):
proc = await asyncio.create_subprocess_shell(
cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE)
stdout, stderr = await proc.communicate()
print(f'[{cmd!r} exited with {proc.returncode}]')
if stdout:
print(f'[stdout]\n{stdout.decode()}')
if stderr:
print(f'[stderr]\n{stderr.decode()}')
asyncio.run(run('ls /zzz'))
will print:
['ls /zzz' exited with 1]
[stderr]
ls: /zzz: No such file or directory
Because all asyncio subprocess functions are asynchronous and asyncio provides many tools to work with such functions, it is easy to execute and monitor multiple subprocesses in parallel. It is indeed trivial to modify the above example to run several commands simultaneously:
async def main():
await asyncio.gather(
run('ls /zzz'),
run('sleep 1; echo "hello"'))
asyncio.run(main())
See also the Examples subsection.
Creating Subprocesses¶
- async asyncio.create_subprocess_exec(program, *args, stdin=None, stdout=None, stderr=None, limit=None, **kwds)¶
Create a subprocess.
The limit argument sets the buffer limit for
StreamReader
wrappers forstdout
andstderr
(ifsubprocess.PIPE
is passed to stdout and stderr arguments).Return a
Process
instance.See the documentation of
loop.subprocess_exec()
for other parameters.If the process object is garbage collected while the process is still running, the child process will be killed.
Changed in version 3.10: Removed the loop parameter.
- async asyncio.create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, limit=None, **kwds)¶
Run the cmd shell command.
The limit argument sets the buffer limit for
StreamReader
wrappers forstdout
andstderr
(ifsubprocess.PIPE
is passed to stdout and stderr arguments).Return a
Process
instance.See the documentation of
loop.subprocess_shell()
for other parameters.If the process object is garbage collected while the process is still running, the child process will be killed.
Important
It is the application’s responsibility to ensure that all whitespace and special characters are quoted appropriately to avoid shell injection vulnerabilities. The
shlex.quote()
function can be used to properly escape whitespace and special shell characters in strings that are going to be used to construct shell commands.Changed in version 3.10: Removed the loop parameter.
Note
Subprocesses are available for Windows if a ProactorEventLoop
is
used. See Subprocess Support on Windows
for details.
See also
asyncio also has the following low-level APIs to work with subprocesses:
loop.subprocess_exec()
, loop.subprocess_shell()
,
loop.connect_read_pipe()
, loop.connect_write_pipe()
,
as well as the Subprocess Transports
and Subprocess Protocols.
Constants¶
- asyncio.subprocess.PIPE¶
Can be passed to the stdin, stdout or stderr parameters.
If PIPE is passed to stdin argument, the
Process.stdin
attribute will point to aStreamWriter
instance.If PIPE is passed to stdout or stderr arguments, the
Process.stdout
andProcess.stderr
attributes will point toStreamReader
instances.
- asyncio.subprocess.STDOUT¶
Special value that can be used as the stderr argument and indicates that standard error should be redirected into standard output.
- asyncio.subprocess.DEVNULL¶
Special value that can be used as the stdin, stdout or stderr argument to process creation functions. It indicates that the special file
os.devnull
will be used for the corresponding subprocess stream.
Interacting with Subprocesses¶
Both create_subprocess_exec()
and create_subprocess_shell()
functions return instances of the Process class. Process is a high-level
wrapper that allows communicating with subprocesses and watching for
their completion.
- class asyncio.subprocess.Process¶
An object that wraps OS processes created by the
create_subprocess_exec()
andcreate_subprocess_shell()
functions.This class is designed to have a similar API to the
subprocess.Popen
class, but there are some notable differences:unlike Popen, Process instances do not have an equivalent to the
poll()
method;the
communicate()
andwait()
methods don’t have a timeout parameter: use thewait_for()
function;the
Process.wait()
method is asynchronous, whereassubprocess.Popen.wait()
method is implemented as a blocking busy loop;the universal_newlines parameter is not supported.
This class is not thread safe.
See also the Subprocess and Threads section.
- async wait()¶
Wait for the child process to terminate.
Set and return the
returncode
attribute.Note
This method can deadlock when using
stdout=PIPE
orstderr=PIPE
and the child process generates so much output that it blocks waiting for the OS pipe buffer to accept more data. Use thecommunicate()
method when using pipes to avoid this condition.
- async communicate(input=None)¶
Interact with process:
send data to stdin (if input is not
None
);closes stdin;
read data from stdout and stderr, until EOF is reached;
wait for process to terminate.
The optional input argument is the data (
bytes
object) that will be sent to the child process.Return a tuple
(stdout_data, stderr_data)
.If either
BrokenPipeError
orConnectionResetError
exception is raised when writing input into stdin, the exception is ignored. This condition occurs when the process exits before all data are written into stdin.If it is desired to send data to the process’ stdin, the process needs to be created with
stdin=PIPE
. Similarly, to get anything other thanNone
in the result tuple, the process has to be created withstdout=PIPE
and/orstderr=PIPE
arguments.Note, that the data read is buffered in memory, so do not use this method if the data size is large or unlimited.
Changed in version 3.12: stdin gets closed when
input=None
too.
- send_signal(signal)¶
Sends the signal signal to the child process.
Note
On Windows,
SIGTERM
is an alias forterminate()
.CTRL_C_EVENT
andCTRL_BREAK_EVENT
can be sent to processes started with a creationflags parameter which includesCREATE_NEW_PROCESS_GROUP
.
- terminate()¶
Stop the child process.
On POSIX systems this method sends
SIGTERM
to the child process.On Windows the Win32 API function
TerminateProcess()
is called to stop the child process.
- kill()¶
Kill the child process.
On POSIX systems this method sends
SIGKILL
to the child process.On Windows this method is an alias for
terminate()
.
- stdin¶
Standard input stream (
StreamWriter
) orNone
if the process was created withstdin=None
.
- stdout¶
Standard output stream (
StreamReader
) orNone
if the process was created withstdout=None
.
- stderr¶
Standard error stream (
StreamReader
) orNone
if the process was created withstderr=None
.
Warning
Use the
communicate()
method rather thanprocess.stdin.write()
,await process.stdout.read()
orawait process.stderr.read()
. This avoids deadlocks due to streams pausing reading or writing and blocking the child process.- pid¶
Process identification number (PID).
Note that for processes created by the
create_subprocess_shell()
function, this attribute is the PID of the spawned shell.
- returncode¶
Return code of the process when it exits.
A
None
value indicates that the process has not terminated yet.A negative value
-N
indicates that the child was terminated by signalN
(POSIX only).
Subprocess and Threads¶
Standard asyncio event loop supports running subprocesses from different threads by default.
On Windows subprocesses are provided by ProactorEventLoop
only (default),
SelectorEventLoop
has no subprocess support.
Note that alternative event loop implementations might have own limitations; please refer to their documentation.
See also
The Concurrency and multithreading in asyncio section.
Examples¶
An example using the Process
class to
control a subprocess and the StreamReader
class to read from
its standard output.
The subprocess is created by the create_subprocess_exec()
function:
import asyncio
import sys
async def get_date():
code = 'import datetime; print(datetime.datetime.now())'
# Create the subprocess; redirect the standard output
# into a pipe.
proc = await asyncio.create_subprocess_exec(
sys.executable, '-c', code,
stdout=asyncio.subprocess.PIPE)
# Read one line of output.
data = await proc.stdout.readline()
line = data.decode('ascii').rstrip()
# Wait for the subprocess exit.
await proc.wait()
return line
date = asyncio.run(get_date())
print(f"Current date: {date}")
See also the same example written using low-level APIs.


Porting Extension Modules to Python 3

Porting Extension Modules to Python 3¶
We recommend the following resources for porting extension modules to Python 3:
The Migrating C extensions chapter from Supporting Python 3: An in-depth guide, a book on moving from Python 2 to Python 3 in general, guides the reader through porting an extension module.
The Porting guide from the py3c project provides opinionated suggestions with supporting code.
Recommended third party tools offer abstractions over the Python’s C API. Extensions generally need to be re-written to use one of them, but the library then handles differences between various Python versions and implementations.


Object Life Cycle

Object Life Cycle¶
This section explains how a type’s slots relate to each other throughout the life of an object. It is not intended to be a complete canonical reference for the slots; instead, refer to the slot-specific documentation in Type Object Structures for details about a particular slot.
Life Events¶
The figure below illustrates the order of events that can occur throughout an object’s life. An arrow from A to B indicates that event B can occur after event A has occurred, with the arrow’s label indicating the condition that must be true for B to occur after A.
Explanation:
When a new object is constructed by calling its type:
tp_new
is called to create a new object.tp_alloc
is directly called bytp_new
to allocate the memory for the new object.tp_init
initializes the newly created object.tp_init
can be called again to re-initialize an object, if desired. Thetp_init
call can also be skipped entirely, for example by Python code calling__new__()
.
After
tp_init
completes, the object is ready to use.Some time after the last reference to an object is removed:
If an object is not marked as finalized, it might be finalized by marking it as finalized and calling its
tp_finalize
function. Python does not finalize an object when the last reference to it is deleted; usePyObject_CallFinalizerFromDealloc()
to ensure thattp_finalize
is always called.If the object is marked as finalized,
tp_clear
might be called by the garbage collector to clear references held by the object. It is not called when the object’s reference count reaches zero.tp_dealloc
is called to destroy the object. To avoid code duplication,tp_dealloc
typically calls intotp_clear
to free up the object’s references.When
tp_dealloc
finishes object destruction, it directly callstp_free
(usually set toPyObject_Free()
orPyObject_GC_Del()
automatically as appropriate for the type) to deallocate the memory.
The
tp_finalize
function is permitted to add a reference to the object if desired. If it does, the object is resurrected, preventing its pending destruction. (Onlytp_finalize
is allowed to resurrect an object;tp_clear
andtp_dealloc
cannot without calling intotp_finalize
.) Resurrecting an object may or may not cause the object’s finalized mark to be removed. Currently, Python does not remove the finalized mark from a resurrected object if it supports garbage collection (i.e., thePy_TPFLAGS_HAVE_GC
flag is set) but does remove the mark if the object does not support garbage collection; either or both of these behaviors may change in the future.tp_dealloc
can optionally calltp_finalize
viaPyObject_CallFinalizerFromDealloc()
if it wishes to reuse that code to help with object destruction. This is recommended because it guarantees thattp_finalize
is always called before destruction. See thetp_dealloc
documentation for example code.If the object is a member of a cyclic isolate and either
tp_clear
fails to break the reference cycle or the cyclic isolate is not detected (perhapsgc.disable()
was called, or thePy_TPFLAGS_HAVE_GC
flag was erroneously omitted in one of the involved types), the objects remain indefinitely uncollectable (they “leak”). Seegc.garbage
.
If the object is marked as supporting garbage collection (the
Py_TPFLAGS_HAVE_GC
flag is set in
tp_flags
), the following events are also possible:
The garbage collector occasionally calls
tp_traverse
to identify cyclic isolates.When the garbage collector discovers a cyclic isolate, it finalizes one of the objects in the group by marking it as finalized and calling its
tp_finalize
function, if it has one. This repeats until the cyclic isolate doesn’t exist or all of the objects have been finalized.tp_finalize
is permitted to resurrect the object by adding a reference from outside the cyclic isolate. The new reference causes the group of objects to no longer form a cyclic isolate (the reference cycle may still exist, but if it does the objects are no longer isolated).When the garbage collector discovers a cyclic isolate and all of the objects in the group have already been marked as finalized, the garbage collector clears one or more of the uncleared objects in the group (possibly concurrently) by calling each’s
tp_clear
function. This repeats as long as the cyclic isolate still exists and not all of the objects have been cleared.
Cyclic Isolate Destruction¶
Listed below are the stages of life of a hypothetical cyclic isolate
that continues to exist after each member object is finalized or cleared. It
is a memory leak if a cyclic isolate progresses through all of these stages; it should
vanish once all objects are cleared, if not sooner. A cyclic isolate can
vanish either because the reference cycle is broken or because the objects are
no longer isolated due to finalizer resurrection (see
tp_finalize
).
Reachable (not yet a cyclic isolate): All objects are in their normal, reachable state. A reference cycle could exist, but an external reference means the objects are not yet isolated.
Unreachable but consistent: The final reference from outside the cyclic group of objects has been removed, causing the objects to become isolated (thus a cyclic isolate is born). None of the group’s objects have been finalized or cleared yet. The cyclic isolate remains at this stage until some future run of the garbage collector (not necessarily the next run because the next run might not scan every object).
Mix of finalized and not finalized: Objects in a cyclic isolate are finalized one at a time, which means that there is a period of time when the cyclic isolate is composed of a mix of finalized and non-finalized objects. Finalization order is unspecified, so it can appear random. A finalized object must behave in a sane manner when non-finalized objects interact with it, and a non-finalized object must be able to tolerate the finalization of an arbitrary subset of its referents.
All finalized: All objects in a cyclic isolate are finalized before any of them are cleared.
Mix of finalized and cleared: The objects can be cleared serially or concurrently (but with the GIL held); either way, some will finish before others. A finalized object must be able to tolerate the clearing of a subset of its referents. PEP 442 calls this stage “cyclic trash”.
Leaked: If a cyclic isolate still exists after all objects in the group have been finalized and cleared, then the objects remain indefinitely uncollectable (see
gc.garbage
). It is a bug if a cyclic isolate reaches this stage—it means thetp_clear
methods of the participating objects have failed to break the reference cycle as required.
If tp_clear
did not exist, then Python would have no
way to safely break a reference cycle. Simply destroying an object in a cyclic
isolate would result in a dangling pointer, triggering undefined behavior when
an object referencing the destroyed object is itself destroyed. The clearing
step makes object destruction a two-phase process: first
tp_clear
is called to partially destroy the objects
enough to detangle them from each other, then
tp_dealloc
is called to complete the destruction.
Unlike clearing, finalization is not a phase of destruction. A finalized
object must still behave properly by continuing to fulfill its design
contracts. An object’s finalizer is allowed to execute arbitrary Python code,
and is even allowed to prevent the impending destruction by adding a reference.
The finalizer is only related to destruction by call order—if it runs, it runs
before destruction, which starts with tp_clear
(if
called) and concludes with tp_dealloc
.
The finalization step is not necessary to safely reclaim the objects in a cyclic isolate, but its existence makes it easier to design types that behave in a sane manner when objects are cleared. Clearing an object might necessarily leave it in a broken, partially destroyed state—it might be unsafe to call any of the cleared object’s methods or access any of its attributes. With finalization, only finalized objects can possibly interact with cleared objects; non-finalized objects are guaranteed to interact with only non-cleared (but potentially finalized) objects.
To summarize the possible interactions:
A non-finalized object might have references to or from non-finalized and finalized objects, but not to or from cleared objects.
A finalized object might have references to or from non-finalized, finalized, and cleared objects.
A cleared object might have references to or from finalized and cleared objects, but not to or from non-finalized objects.
Without any reference cycles, an object can be simply destroyed once its last
reference is deleted; the finalization and clearing steps are not necessary to
safely reclaim unused objects. However, it can be useful to automatically call
tp_finalize
and tp_clear
before destruction anyway because type design is simplified when all objects
always experience the same series of events regardless of whether they
participated in a cyclic isolate. Python currently only calls
tp_finalize
and tp_clear
as
needed to destroy a cyclic isolate; this may change in a future version.
Functions¶
To allocate and free memory, see Allocating Objects on the Heap.
-
void PyObject_CallFinalizer(PyObject *op)¶
Finalizes the object as described in
tp_finalize
. Call this function (orPyObject_CallFinalizerFromDealloc()
) instead of callingtp_finalize
directly because this function may deduplicate multiple calls totp_finalize
. Currently, calls are only deduplicated if the type supports garbage collection (i.e., thePy_TPFLAGS_HAVE_GC
flag is set); this may change in the future.Added in version 3.4.
-
int PyObject_CallFinalizerFromDealloc(PyObject *op)¶
Same as
PyObject_CallFinalizer()
but meant to be called at the beginning of the object’s destructor (tp_dealloc
). There must not be any references to the object. If the object’s finalizer resurrects the object, this function returns -1; no further destruction should happen. Otherwise, this function returns 0 and destruction can continue normally.Added in version 3.4.
See also
tp_dealloc
for example code.


Type Objects

Type Objects¶
-
type PyTypeObject¶
- Part of the Limited API (as an opaque struct).
The C structure of the objects used to describe built-in types.
-
PyTypeObject PyType_Type¶
- Part of the Stable ABI.
This is the type object for type objects; it is the same object as
type
in the Python layer.
-
int PyType_Check(PyObject *o)¶
Return non-zero if the object o is a type object, including instances of types derived from the standard type object. Return 0 in all other cases. This function always succeeds.
-
int PyType_CheckExact(PyObject *o)¶
Return non-zero if the object o is a type object, but not a subtype of the standard type object. Return 0 in all other cases. This function always succeeds.
-
unsigned int PyType_ClearCache()¶
- Part of the Stable ABI.
Clear the internal lookup cache. Return the current version tag.
-
unsigned long PyType_GetFlags(PyTypeObject *type)¶
- Part of the Stable ABI.
Return the
tp_flags
member of type. This function is primarily meant for use withPy_LIMITED_API
; the individual flag bits are guaranteed to be stable across Python releases, but access totp_flags
itself is not part of the limited API.Added in version 3.2.
Changed in version 3.4: The return type is now
unsigned long
rather thanlong
.
-
PyObject *PyType_GetDict(PyTypeObject *type)¶
Return the type object’s internal namespace, which is otherwise only exposed via a read-only proxy (
cls.__dict__
). This is a replacement for accessingtp_dict
directly. The returned dictionary must be treated as read-only.This function is meant for specific embedding and language-binding cases, where direct access to the dict is necessary and indirect access (e.g. via the proxy or
PyObject_GetAttr()
) isn’t adequate.Extension modules should continue to use
tp_dict
, directly or indirectly, when setting up their own types.Added in version 3.12.
-
void PyType_Modified(PyTypeObject *type)¶
- Part of the Stable ABI.
Invalidate the internal lookup cache for the type and all of its subtypes. This function must be called after any manual modification of the attributes or base classes of the type.
-
int PyType_AddWatcher(PyType_WatchCallback callback)¶
Register callback as a type watcher. Return a non-negative integer ID which must be passed to future calls to
PyType_Watch()
. In case of error (e.g. no more watcher IDs available), return-1
and set an exception.In free-threaded builds,
PyType_AddWatcher()
is not thread-safe, so it must be called at start up (before spawning the first thread).Added in version 3.12.
-
int PyType_ClearWatcher(int watcher_id)¶
Clear watcher identified by watcher_id (previously returned from
PyType_AddWatcher()
). Return0
on success,-1
on error (e.g. if watcher_id was never registered.)An extension should never call
PyType_ClearWatcher
with a watcher_id that was not returned to it by a previous call toPyType_AddWatcher()
.Added in version 3.12.
-
int PyType_Watch(int watcher_id, PyObject *type)¶
Mark type as watched. The callback granted watcher_id by
PyType_AddWatcher()
will be called wheneverPyType_Modified()
reports a change to type. (The callback may be called only once for a series of consecutive modifications to type, if_PyType_Lookup()
is not called on type between the modifications; this is an implementation detail and subject to change.)An extension should never call
PyType_Watch
with a watcher_id that was not returned to it by a previous call toPyType_AddWatcher()
.Added in version 3.12.
-
int PyType_Unwatch(int watcher_id, PyObject *type)¶
Mark type as not watched. This undoes a previous call to
PyType_Watch()
. type must not beNULL
.An extension should never call this function with a watcher_id that was not returned to it by a previous call to
PyType_AddWatcher()
.On success, this function returns
0
. On failure, this function returns-1
with an exception set.Added in version 3.12.
-
typedef int (*PyType_WatchCallback)(PyObject *type)¶
Type of a type-watcher callback function.
The callback must not modify type or cause
PyType_Modified()
to be called on type or any type in its MRO; violating this rule could cause infinite recursion.Added in version 3.12.
-
int PyType_HasFeature(PyTypeObject *o, int feature)¶
Return non-zero if the type object o sets the feature feature. Type features are denoted by single bit flags.
-
int PyType_FastSubclass(PyTypeObject *type, int flag)¶
Return non-zero if the type object type sets the subclass flag flag. Subclass flags are denoted by
Py_TPFLAGS_*_SUBCLASS
. This function is used by many_Check
functions for common types.See also
PyObject_TypeCheck()
, which is used as a slower alternative in_Check
functions for types that don’t come with subclass flags.
-
int PyType_IS_GC(PyTypeObject *o)¶
Return true if the type object includes support for the cycle detector; this tests the type flag
Py_TPFLAGS_HAVE_GC
.
-
int PyType_IsSubtype(PyTypeObject *a, PyTypeObject *b)¶
- Part of the Stable ABI.
Return true if a is a subtype of b.
This function only checks for actual subtypes, which means that
__subclasscheck__()
is not called on b. CallPyObject_IsSubclass()
to do the same check thatissubclass()
would do.
-
PyObject *PyType_GenericAlloc(PyTypeObject *type, Py_ssize_t nitems)¶
- Return value: New reference. Part of the Stable ABI.
Generic handler for the
tp_alloc
slot of a type object. Uses Python’s default memory allocation mechanism to allocate memory for a new instance, zeros the memory, then initializes the memory as if by callingPyObject_Init()
orPyObject_InitVar()
.Do not call this directly to allocate memory for an object; call the type’s
tp_alloc
slot instead.For types that support garbage collection (i.e., the
Py_TPFLAGS_HAVE_GC
flag is set), this function behaves likePyObject_GC_New
orPyObject_GC_NewVar
(except the memory is guaranteed to be zeroed before initialization), and should be paired withPyObject_GC_Del()
intp_free
. Otherwise, it behaves likePyObject_New
orPyObject_NewVar
(except the memory is guaranteed to be zeroed before initialization) and should be paired withPyObject_Free()
intp_free
.
-
PyObject *PyType_GenericNew(PyTypeObject *type, PyObject *args, PyObject *kwds)¶
- Return value: New reference. Part of the Stable ABI.
Generic handler for the
tp_new
slot of a type object. Creates a new instance using the type’stp_alloc
slot and returns the resulting object.
-
int PyType_Ready(PyTypeObject *type)¶
- Part of the Stable ABI.
Finalize a type object. This should be called on all type objects to finish their initialization. This function is responsible for adding inherited slots from a type’s base class. Return
0
on success, or return-1
and sets an exception on error.Note
If some of the base classes implements the GC protocol and the provided type does not include the
Py_TPFLAGS_HAVE_GC
in its flags, then the GC protocol will be automatically implemented from its parents. On the contrary, if the type being created does includePy_TPFLAGS_HAVE_GC
in its flags then it must implement the GC protocol itself by at least implementing thetp_traverse
handle.
-
PyObject *PyType_GetName(PyTypeObject *type)¶
- Return value: New reference. Part of the Stable ABI since version 3.11.
Return the type’s name. Equivalent to getting the type’s
__name__
attribute.Added in version 3.11.
-
PyObject *PyType_GetQualName(PyTypeObject *type)¶
- Return value: New reference. Part of the Stable ABI since version 3.11.
Return the type’s qualified name. Equivalent to getting the type’s
__qualname__
attribute.Added in version 3.11.
-
PyObject *PyType_GetFullyQualifiedName(PyTypeObject *type)¶
- Part of the Stable ABI since version 3.13.
Return the type’s fully qualified name. Equivalent to
f"{type.__module__}.{type.__qualname__}"
, ortype.__qualname__
iftype.__module__
is not a string or is equal to"builtins"
.Added in version 3.13.
-
PyObject *PyType_GetModuleName(PyTypeObject *type)¶
- Part of the Stable ABI since version 3.13.
Return the type’s module name. Equivalent to getting the
type.__module__
attribute.Added in version 3.13.
-
void *PyType_GetSlot(PyTypeObject *type, int slot)¶
- Part of the Stable ABI since version 3.4.
Return the function pointer stored in the given slot. If the result is
NULL
, this indicates that either the slot isNULL
, or that the function was called with invalid parameters. Callers will typically cast the result pointer into the appropriate function type.See
PyType_Slot.slot
for possible values of the slot argument.Added in version 3.4.
Changed in version 3.10:
PyType_GetSlot()
can now accept all types. Previously, it was limited to heap types.
-
PyObject *PyType_GetModule(PyTypeObject *type)¶
- Part of the Stable ABI since version 3.10.
Return the module object associated with the given type when the type was created using
PyType_FromModuleAndSpec()
.If no module is associated with the given type, sets
TypeError
and returnsNULL
.This function is usually used to get the module in which a method is defined. Note that in such a method,
PyType_GetModule(Py_TYPE(self))
may not return the intended result.Py_TYPE(self)
may be a subclass of the intended class, and subclasses are not necessarily defined in the same module as their superclass. SeePyCMethod
to get the class that defines the method. SeePyType_GetModuleByDef()
for cases whenPyCMethod
cannot be used.Added in version 3.9.
-
void *PyType_GetModuleState(PyTypeObject *type)¶
- Part of the Stable ABI since version 3.10.
Return the state of the module object associated with the given type. This is a shortcut for calling
PyModule_GetState()
on the result ofPyType_GetModule()
.If no module is associated with the given type, sets
TypeError
and returnsNULL
.If the type has an associated module but its state is
NULL
, returnsNULL
without setting an exception.Added in version 3.9.
-
PyObject *PyType_GetModuleByDef(PyTypeObject *type, struct PyModuleDef *def)¶
- Return value: Borrowed reference. Part of the Stable ABI since version 3.13.
Find the first superclass whose module was created from the given
PyModuleDef
def, and return that module.If no module is found, raises a
TypeError
and returnsNULL
.This function is intended to be used together with
PyModule_GetState()
to get module state from slot methods (such astp_init
ornb_add
) and other places where a method’s defining class cannot be passed using thePyCMethod
calling convention.The returned reference is borrowed from type, and will be valid as long as you hold a reference to type. Do not release it with
Py_DECREF()
or similar.Added in version 3.11.
-
int PyType_GetBaseByToken(PyTypeObject *type, void *token, PyTypeObject **result)¶
- Part of the Stable ABI since version 3.14.
Find the first superclass in type’s method resolution order whose
Py_tp_token
token is equal to the given one.If found, set *result to a new strong reference to it and return
1
.If not found, set *result to
NULL
and return0
.On error, set *result to
NULL
and return-1
with an exception set.
The result argument may be
NULL
, in which case *result is not set. Use this if you need only the return value.The token argument may not be
NULL
.Added in version 3.14.
-
int PyUnstable_Type_AssignVersionTag(PyTypeObject *type)¶
- This is Unstable API. It may change without warning in minor releases.
Attempt to assign a version tag to the given type.
Returns 1 if the type already had a valid version tag or a new one was assigned, or 0 if a new tag could not be assigned.
Added in version 3.12.
-
int PyType_SUPPORTS_WEAKREFS(PyTypeObject *type)¶
Return true if instances of type support creating weak references, false otherwise. This function always succeeds. type must not be
NULL
.See also
Creating Heap-Allocated Types¶
The following functions and structs are used to create heap types.
-
PyObject *PyType_FromMetaclass(PyTypeObject *metaclass, PyObject *module, PyType_Spec *spec, PyObject *bases)¶
- Part of the Stable ABI since version 3.12.
Create and return a heap type from the spec (see
Py_TPFLAGS_HEAPTYPE
).The metaclass metaclass is used to construct the resulting type object. When metaclass is
NULL
, the metaclass is derived from bases (or Py_tp_base[s] slots if bases isNULL
, see below).Metaclasses that override
tp_new
are not supported, except iftp_new
isNULL
.The bases argument can be used to specify base classes; it can either be only one class or a tuple of classes. If bases is
NULL
, thePy_tp_bases
slot is used instead. If that also isNULL
, thePy_tp_base
slot is used instead. If that also isNULL
, the new type derives fromobject
.The module argument can be used to record the module in which the new class is defined. It must be a module object or
NULL
. If notNULL
, the module is associated with the new type and can later be retrieved withPyType_GetModule()
. The associated module is not inherited by subclasses; it must be specified for each class individually.This function calls
PyType_Ready()
on the new type.Note that this function does not fully match the behavior of calling
type()
or using theclass
statement. With user-provided base types or metaclasses, prefer callingtype
(or the metaclass) overPyType_From*
functions. Specifically:__new__()
is not called on the new class (and it must be set totype.__new__
).__init__()
is not called on the new class.__init_subclass__()
is not called on any bases.__set_name__()
is not called on new descriptors.
Added in version 3.12.
-
PyObject *PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases)¶
- Return value: New reference. Part of the Stable ABI since version 3.10.
Equivalent to
PyType_FromMetaclass(NULL, module, spec, bases)
.Added in version 3.9.
Changed in version 3.10: The function now accepts a single class as the bases argument and
NULL
as thetp_doc
slot.Changed in version 3.12: The function now finds and uses a metaclass corresponding to the provided base classes. Previously, only
type
instances were returned.The
tp_new
of the metaclass is ignored. which may result in incomplete initialization. Creating classes whose metaclass overridestp_new
is deprecated.Changed in version 3.14: Creating classes whose metaclass overrides
tp_new
is no longer allowed.
-
PyObject *PyType_FromSpecWithBases(PyType_Spec *spec, PyObject *bases)¶
- Return value: New reference. Part of the Stable ABI since version 3.3.
Equivalent to
PyType_FromMetaclass(NULL, NULL, spec, bases)
.Added in version 3.3.
Changed in version 3.12: The function now finds and uses a metaclass corresponding to the provided base classes. Previously, only
type
instances were returned.The
tp_new
of the metaclass is ignored. which may result in incomplete initialization. Creating classes whose metaclass overridestp_new
is deprecated.Changed in version 3.14: Creating classes whose metaclass overrides
tp_new
is no longer allowed.
-
PyObject *PyType_FromSpec(PyType_Spec *spec)¶
- Return value: New reference. Part of the Stable ABI.
Equivalent to
PyType_FromMetaclass(NULL, NULL, spec, NULL)
.Changed in version 3.12: The function now finds and uses a metaclass corresponding to the base classes provided in Py_tp_base[s] slots. Previously, only
type
instances were returned.The
tp_new
of the metaclass is ignored. which may result in incomplete initialization. Creating classes whose metaclass overridestp_new
is deprecated.Changed in version 3.14: Creating classes whose metaclass overrides
tp_new
is no longer allowed.
-
int PyType_Freeze(PyTypeObject *type)¶
- Part of the Stable ABI since version 3.14.
Make a type immutable: set the
Py_TPFLAGS_IMMUTABLETYPE
flag.All base classes of type must be immutable.
On success, return
0
. On error, set an exception and return-1
.The type must not be used before it’s made immutable. For example, type instances must not be created before the type is made immutable.
Added in version 3.14.
-
type PyType_Spec¶
- Part of the Stable ABI (including all members).
Structure defining a type’s behavior.
-
const char *name¶
Name of the type, used to set
PyTypeObject.tp_name
.
-
int basicsize¶
If positive, specifies the size of the instance in bytes. It is used to set
PyTypeObject.tp_basicsize
.If zero, specifies that
tp_basicsize
should be inherited.If negative, the absolute value specifies how much space instances of the class need in addition to the superclass. Use
PyObject_GetTypeData()
to get a pointer to subclass-specific memory reserved this way. For negativebasicsize
, Python will insert padding when needed to meettp_basicsize
’s alignment requirements.Changed in version 3.12: Previously, this field could not be negative.
-
int itemsize¶
Size of one element of a variable-size type, in bytes. Used to set
PyTypeObject.tp_itemsize
. Seetp_itemsize
documentation for caveats.If zero,
tp_itemsize
is inherited. Extending arbitrary variable-sized classes is dangerous, since some types use a fixed offset for variable-sized memory, which can then overlap fixed-sized memory used by a subclass. To help prevent mistakes, inheritingitemsize
is only possible in the following situations:The base is not variable-sized (its
tp_itemsize
).The requested
PyType_Spec.basicsize
is positive, suggesting that the memory layout of the base class is known.The requested
PyType_Spec.basicsize
is zero, suggesting that the subclass does not access the instance’s memory directly.With the
Py_TPFLAGS_ITEMS_AT_END
flag.
-
unsigned int flags¶
Type flags, used to set
PyTypeObject.tp_flags
.If the
Py_TPFLAGS_HEAPTYPE
flag is not set,PyType_FromSpecWithBases()
sets it automatically.
-
PyType_Slot *slots¶
Array of
PyType_Slot
structures. Terminated by the special slot value{0, NULL}
.Each slot ID should be specified at most once.
-
const char *name¶
-
type PyType_Slot¶
- Part of the Stable ABI (including all members).
Structure defining optional functionality of a type, containing a slot ID and a value pointer.
-
int slot¶
A slot ID.
Slot IDs are named like the field names of the structures
PyTypeObject
,PyNumberMethods
,PySequenceMethods
,PyMappingMethods
andPyAsyncMethods
with an addedPy_
prefix. For example, use:Py_nb_add
to setPyNumberMethods.nb_add
An additional slot is supported that does not correspond to a
PyTypeObject
struct field:The following “offset” fields cannot be set using
PyType_Slot
:tp_weaklistoffset
(usePy_TPFLAGS_MANAGED_WEAKREF
instead if possible)tp_dictoffset
(usePy_TPFLAGS_MANAGED_DICT
instead if possible)tp_vectorcall_offset
(use"__vectorcalloffset__"
in PyMemberDef)
If it is not possible to switch to a
MANAGED
flag (for example, for vectorcall or to support Python older than 3.12), specify the offset inPy_tp_members
. See PyMemberDef documentation for details.The following internal fields cannot be set at all when creating a heap type:
tp_dict
,tp_mro
,tp_cache
,tp_subclasses
, andtp_weaklist
.
Setting
Py_tp_bases
orPy_tp_base
may be problematic on some platforms. To avoid issues, use the bases argument ofPyType_FromSpecWithBases()
instead.Changed in version 3.9: Slots in
PyBufferProcs
may be set in the unlimited API.Changed in version 3.11:
bf_getbuffer
andbf_releasebuffer
are now available under the limited API.Changed in version 3.14: The field
tp_vectorcall
can now be set usingPy_tp_vectorcall
. See the field’s documentation for details.
-
void *pfunc¶
The desired value of the slot. In most cases, this is a pointer to a function.
pfunc values may not be
NULL
, except for the following slots:Py_tp_token
(for clarity, preferPy_TP_USE_SPEC
rather thanNULL
)
-
int slot¶
-
Py_tp_token¶
- Part of the Stable ABI since version 3.14.
A
slot
that records a static memory layout ID for a class.If the
PyType_Spec
of the class is statically allocated, the token can be set to the spec using the special valuePy_TP_USE_SPEC
:static PyType_Slot foo_slots[] = { {Py_tp_token, Py_TP_USE_SPEC},
It can also be set to an arbitrary pointer, but you must ensure that:
The pointer outlives the class, so it’s not reused for something else while the class exists.
It “belongs” to the extension module where the class lives, so it will not clash with other extensions.
Use
PyType_GetBaseByToken()
to check if a class’s superclass has a given token – that is, check whether the memory layout is compatible.To get the token for a given class (without considering superclasses), use
PyType_GetSlot()
withPy_tp_token
.Added in version 3.14.
-
Py_TP_USE_SPEC¶
- Part of the Stable ABI since version 3.14.
Used as a value with
Py_tp_token
to set the token to the class’sPyType_Spec
. Expands toNULL
.Added in version 3.14.


— Asynchronous socket handler

asyncore
— Asynchronous socket handler¶
Deprecated since version 3.6, removed in version 3.12.
This module is no longer part of the Python standard library. It was removed in Python 3.12 after being deprecated in Python 3.6. The removal was decided in PEP 594.
Applications should use the asyncio
module instead.
The last version of Python that provided the asyncore
module was
Python 3.11.