Function Objects

Function Objects¶
There are a few functions specific to Python functions.
-
type PyFunctionObject¶
The C structure used for functions.
-
PyTypeObject PyFunction_Type¶
This is an instance of
PyTypeObject
and represents the Python function type. It is exposed to Python programmers astypes.FunctionType
.
-
int PyFunction_Check(PyObject *o)¶
Return true if o is a function object (has type
PyFunction_Type
). The parameter must not beNULL
. This function always succeeds.
-
PyObject *PyFunction_New(PyObject *code, PyObject *globals)¶
- Return value: New reference.
Return a new function object associated with the code object code. globals must be a dictionary with the global variables accessible to the function.
The function’s docstring and name are retrieved from the code object.
__module__
is retrieved from globals. The argument defaults, annotations and closure are set toNULL
.__qualname__
is set to the same value as the code object’sco_qualname
field.
-
PyObject *PyFunction_NewWithQualName(PyObject *code, PyObject *globals, PyObject *qualname)¶
- Return value: New reference.
As
PyFunction_New()
, but also allows setting the function object’s__qualname__
attribute. qualname should be a unicode object orNULL
; ifNULL
, the__qualname__
attribute is set to the same value as the code object’sco_qualname
field.Added in version 3.3.
-
PyObject *PyFunction_GetCode(PyObject *op)¶
- Return value: Borrowed reference.
Return the code object associated with the function object op.
-
PyObject *PyFunction_GetGlobals(PyObject *op)¶
- Return value: Borrowed reference.
Return the globals dictionary associated with the function object op.
-
PyObject *PyFunction_GetModule(PyObject *op)¶
- Return value: Borrowed reference.
Return a borrowed reference to the
__module__
attribute of the function object op. It can be NULL.This is normally a
string
containing the module name, but can be set to any other object by Python code.
-
PyObject *PyFunction_GetDefaults(PyObject *op)¶
- Return value: Borrowed reference.
Return the argument default values of the function object op. This can be a tuple of arguments or
NULL
.
-
int PyFunction_SetDefaults(PyObject *op, PyObject *defaults)¶
Set the argument default values for the function object op. defaults must be
Py_None
or a tuple.Raises
SystemError
and returns-1
on failure.
-
void PyFunction_SetVectorcall(PyFunctionObject *func, vectorcallfunc vectorcall)¶
Set the vectorcall field of a given function object func.
Warning: extensions using this API must preserve the behavior of the unaltered (default) vectorcall function!
Added in version 3.12.
-
PyObject *PyFunction_GetKwDefaults(PyObject *op)¶
- Return value: Borrowed reference.
Return the keyword-only argument default values of the function object op. This can be a dictionary of arguments or
NULL
.
-
int PyFunction_SetKwDefaults(PyObject *op, PyObject *defaults)¶
Set the keyword-only argument default values of the function object op. defaults must be a dictionary of keyword-only arguments or
Py_None
.This function returns
0
on success, and returns-1
with an exception set on failure.
-
PyObject *PyFunction_GetClosure(PyObject *op)¶
- Return value: Borrowed reference.
Return the closure associated with the function object op. This can be
NULL
or a tuple of cell objects.
-
int PyFunction_SetClosure(PyObject *op, PyObject *closure)¶
Set the closure associated with the function object op. closure must be
Py_None
or a tuple of cell objects.Raises
SystemError
and returns-1
on failure.
-
PyObject *PyFunction_GetAnnotations(PyObject *op)¶
- Return value: Borrowed reference.
Return the annotations of the function object op. This can be a mutable dictionary or
NULL
.
-
int PyFunction_SetAnnotations(PyObject *op, PyObject *annotations)¶
Set the annotations for the function object op. annotations must be a dictionary or
Py_None
.Raises
SystemError
and returns-1
on failure.
-
PyObject *PyFunction_GET_CODE(PyObject *op)¶
-
PyObject *PyFunction_GET_GLOBALS(PyObject *op)¶
-
PyObject *PyFunction_GET_MODULE(PyObject *op)¶
-
PyObject *PyFunction_GET_DEFAULTS(PyObject *op)¶
-
PyObject *PyFunction_GET_KW_DEFAULTS(PyObject *op)¶
-
PyObject *PyFunction_GET_CLOSURE(PyObject *op)¶
-
PyObject *PyFunction_GET_ANNOTATIONS(PyObject *op)¶
- Return value: Borrowed reference.
These functions are similar to their
PyFunction_Get*
counterparts, but do not do type checking. Passing anything other than an instance ofPyFunction_Type
is undefined behavior.
-
int PyFunction_AddWatcher(PyFunction_WatchCallback callback)¶
Register callback as a function watcher for the current interpreter. Return an ID which may be passed to
PyFunction_ClearWatcher()
. In case of error (e.g. no more watcher IDs available), return-1
and set an exception.Added in version 3.12.
-
int PyFunction_ClearWatcher(int watcher_id)¶
Clear watcher identified by watcher_id previously returned from
PyFunction_AddWatcher()
for the current interpreter. Return0
on success, or-1
and set an exception on error (e.g. if the given watcher_id was never registered.)Added in version 3.12.
-
type PyFunction_WatchEvent¶
Enumeration of possible function watcher events:
PyFunction_EVENT_CREATE
PyFunction_EVENT_DESTROY
PyFunction_EVENT_MODIFY_CODE
PyFunction_EVENT_MODIFY_DEFAULTS
PyFunction_EVENT_MODIFY_KWDEFAULTS
Added in version 3.12.
-
typedef int (*PyFunction_WatchCallback)(PyFunction_WatchEvent event, PyFunctionObject *func, PyObject *new_value)¶
Type of a function watcher callback function.
If event is
PyFunction_EVENT_CREATE
orPyFunction_EVENT_DESTROY
then new_value will beNULL
. Otherwise, new_value will hold a borrowed reference to the new value that is about to be stored in func for the attribute that is being modified.The callback may inspect but must not modify func; doing so could have unpredictable effects, including infinite recursion.
If event is
PyFunction_EVENT_CREATE
, then the callback is invoked after func has been fully initialized. Otherwise, the callback is invoked before the modification to func takes place, so the prior state of func can be inspected. The runtime is permitted to optimize away the creation of function objects when possible. In such cases no event will be emitted. Although this creates the possibility of an observable difference of runtime behavior depending on optimization decisions, it does not change the semantics of the Python code being executed.If event is
PyFunction_EVENT_DESTROY
, Taking a reference in the callback to the about-to-be-destroyed function will resurrect it, preventing it from being freed at this time. When the resurrected object is destroyed later, any watcher callbacks active at that time will be called again.If the callback sets an exception, it must return
-1
; this exception will be printed as an unraisable exception usingPyErr_WriteUnraisable()
. Otherwise it should return0
.There may already be a pending exception set on entry to the callback. In this case, the callback should return
0
with the same exception still set. This means the callback may not call any other API that can set an exception unless it saves and clears the exception state first, and restores it before returning.Added in version 3.12.


— Simple lexical analysis

shlex
— Simple lexical analysis¶
Source code: Lib/shlex.py
The shlex
class makes it easy to write lexical analyzers for
simple syntaxes resembling that of the Unix shell. This will often be useful
for writing minilanguages, (for example, in run control files for Python
applications) or for parsing quoted strings.
The shlex
module defines the following functions:
- shlex.split(s, comments=False, posix=True)¶
Split the string s using shell-like syntax. If comments is
False
(the default), the parsing of comments in the given string will be disabled (setting thecommenters
attribute of theshlex
instance to the empty string). This function operates in POSIX mode by default, but uses non-POSIX mode if the posix argument is false.Changed in version 3.12: Passing
None
for s argument now raises an exception, rather than readingsys.stdin
.
- shlex.join(split_command)¶
Concatenate the tokens of the list split_command and return a string. This function is the inverse of
split()
.>>> from shlex import join >>> print(join(['echo', '-n', 'Multiple words'])) echo -n 'Multiple words'
The returned value is shell-escaped to protect against injection vulnerabilities (see
quote()
).Added in version 3.8.
- shlex.quote(s)¶
Return a shell-escaped version of the string s. The returned value is a string that can safely be used as one token in a shell command line, for cases where you cannot use a list.
Warning
The
shlex
module is only designed for Unix shells.The
quote()
function is not guaranteed to be correct on non-POSIX compliant shells or shells from other operating systems such as Windows. Executing commands quoted by this module on such shells can open up the possibility of a command injection vulnerability.Consider using functions that pass command arguments with lists such as
subprocess.run()
withshell=False
.This idiom would be unsafe:
>>> filename = 'somefile; rm -rf ~' >>> command = 'ls -l {}'.format(filename) >>> print(command) # executed by a shell: boom! ls -l somefile; rm -rf ~
quote()
lets you plug the security hole:>>> from shlex import quote >>> command = 'ls -l {}'.format(quote(filename)) >>> print(command) ls -l 'somefile; rm -rf ~' >>> remote_command = 'ssh home {}'.format(quote(command)) >>> print(remote_command) ssh home 'ls -l '"'"'somefile; rm -rf ~'"'"''
The quoting is compatible with UNIX shells and with
split()
:>>> from shlex import split >>> remote_command = split(remote_command) >>> remote_command ['ssh', 'home', "ls -l 'somefile; rm -rf ~'"] >>> command = split(remote_command[-1]) >>> command ['ls', '-l', 'somefile; rm -rf ~']
Added in version 3.3.
The shlex
module defines the following class:
- class shlex.shlex(instream=None, infile=None, posix=False, punctuation_chars=False)¶
A
shlex
instance or subclass instance is a lexical analyzer object. The initialization argument, if present, specifies where to read characters from. It must be a file-/stream-like object withread()
andreadline()
methods, or a string. If no argument is given, input will be taken fromsys.stdin
. The second optional argument is a filename string, which sets the initial value of theinfile
attribute. If the instream argument is omitted or equal tosys.stdin
, this second argument defaults to “stdin”. The posix argument defines the operational mode: when posix is not true (default), theshlex
instance will operate in compatibility mode. When operating in POSIX mode,shlex
will try to be as close as possible to the POSIX shell parsing rules. The punctuation_chars argument provides a way to make the behaviour even closer to how real shells parse. This can take a number of values: the default value,False
, preserves the behaviour seen under Python 3.5 and earlier. If set toTrue
, then parsing of the characters();<>|&
is changed: any run of these characters (considered punctuation characters) is returned as a single token. If set to a non-empty string of characters, those characters will be used as the punctuation characters. Any characters in thewordchars
attribute that appear in punctuation_chars will be removed fromwordchars
. See Improved Compatibility with Shells for more information. punctuation_chars can be set only uponshlex
instance creation and can’t be modified later.Changed in version 3.6: The punctuation_chars parameter was added.
See also
- Module
configparser
Parser for configuration files similar to the Windows
.ini
files.
shlex Objects¶
A shlex
instance has the following methods:
- shlex.get_token()¶
Return a token. If tokens have been stacked using
push_token()
, pop a token off the stack. Otherwise, read one from the input stream. If reading encounters an immediate end-of-file,eof
is returned (the empty string (''
) in non-POSIX mode, andNone
in POSIX mode).
- shlex.push_token(str)¶
Push the argument onto the token stack.
- shlex.read_token()¶
Read a raw token. Ignore the pushback stack, and do not interpret source requests. (This is not ordinarily a useful entry point, and is documented here only for the sake of completeness.)
- shlex.sourcehook(filename)¶
When
shlex
detects a source request (seesource
below) this method is given the following token as argument, and expected to return a tuple consisting of a filename and an open file-like object.Normally, this method first strips any quotes off the argument. If the result is an absolute pathname, or there was no previous source request in effect, or the previous source was a stream (such as
sys.stdin
), the result is left alone. Otherwise, if the result is a relative pathname, the directory part of the name of the file immediately before it on the source inclusion stack is prepended (this behavior is like the way the C preprocessor handles#include "file.h"
).The result of the manipulations is treated as a filename, and returned as the first component of the tuple, with
open()
called on it to yield the second component. (Note: this is the reverse of the order of arguments in instance initialization!)This hook is exposed so that you can use it to implement directory search paths, addition of file extensions, and other namespace hacks. There is no corresponding ‘close’ hook, but a shlex instance will call the
close()
method of the sourced input stream when it returns EOF.For more explicit control of source stacking, use the
push_source()
andpop_source()
methods.
- shlex.push_source(newstream, newfile=None)¶
Push an input source stream onto the input stack. If the filename argument is specified it will later be available for use in error messages. This is the same method used internally by the
sourcehook()
method.
- shlex.pop_source()¶
Pop the last-pushed input source from the input stack. This is the same method used internally when the lexer reaches EOF on a stacked input stream.
- shlex.error_leader(infile=None, lineno=None)¶
This method generates an error message leader in the format of a Unix C compiler error label; the format is
'"%s", line %d: '
, where the%s
is replaced with the name of the current source file and the%d
with the current input line number (the optional arguments can be used to override these).This convenience is provided to encourage
shlex
users to generate error messages in the standard, parseable format understood by Emacs and other Unix tools.
Instances of shlex
subclasses have some public instance
variables which either control lexical analysis or can be used for debugging:
- shlex.commenters¶
The string of characters that are recognized as comment beginners. All characters from the comment beginner to end of line are ignored. Includes just
'#'
by default.
- shlex.wordchars¶
The string of characters that will accumulate into multi-character tokens. By default, includes all ASCII alphanumerics and underscore. In POSIX mode, the accented characters in the Latin-1 set are also included. If
punctuation_chars
is not empty, the characters~-./*?=
, which can appear in filename specifications and command line parameters, will also be included in this attribute, and any characters which appear inpunctuation_chars
will be removed fromwordchars
if they are present there. Ifwhitespace_split
is set toTrue
, this will have no effect.
- shlex.whitespace¶
Characters that will be considered whitespace and skipped. Whitespace bounds tokens. By default, includes space, tab, linefeed and carriage-return.
- shlex.escape¶
Characters that will be considered as escape. This will be only used in POSIX mode, and includes just
'\'
by default.
- shlex.quotes¶
Characters that will be considered string quotes. The token accumulates until the same quote is encountered again (thus, different quote types protect each other as in the shell.) By default, includes ASCII single and double quotes.
- shlex.escapedquotes¶
Characters in
quotes
that will interpret escape characters defined inescape
. This is only used in POSIX mode, and includes just'"'
by default.
- shlex.whitespace_split¶
If
True
, tokens will only be split in whitespaces. This is useful, for example, for parsing command lines withshlex
, getting tokens in a similar way to shell arguments. When used in combination withpunctuation_chars
, tokens will be split on whitespace in addition to those characters.Changed in version 3.8: The
punctuation_chars
attribute was made compatible with thewhitespace_split
attribute.
- shlex.infile¶
The name of the current input file, as initially set at class instantiation time or stacked by later source requests. It may be useful to examine this when constructing error messages.
- shlex.source¶
This attribute is
None
by default. If you assign a string to it, that string will be recognized as a lexical-level inclusion request similar to thesource
keyword in various shells. That is, the immediately following token will be opened as a filename and input will be taken from that stream until EOF, at which point theclose()
method of that stream will be called and the input source will again become the original input stream. Source requests may be stacked any number of levels deep.
- shlex.debug¶
If this attribute is numeric and
1
or more, ashlex
instance will print verbose progress output on its behavior. If you need to use this, you can read the module source code to learn the details.
- shlex.lineno¶
Source line number (count of newlines seen so far plus one).
- shlex.token¶
The token buffer. It may be useful to examine this when catching exceptions.
- shlex.eof¶
Token used to determine end of file. This will be set to the empty string (
''
), in non-POSIX mode, and toNone
in POSIX mode.
- shlex.punctuation_chars¶
A read-only property. Characters that will be considered punctuation. Runs of punctuation characters will be returned as a single token. However, note that no semantic validity checking will be performed: for example, ‘>>>’ could be returned as a token, even though it may not be recognised as such by shells.
Added in version 3.6.
Parsing Rules¶
When operating in non-POSIX mode, shlex
will try to obey the
following rules.
Quote characters are not recognized within words (
Do"Not"Separate
is parsed as the single wordDo"Not"Separate
);Escape characters are not recognized;
Enclosing characters in quotes preserve the literal value of all characters within the quotes;
Closing quotes separate words (
"Do"Separate
is parsed as"Do"
andSeparate
);If
whitespace_split
isFalse
, any character not declared to be a word character, whitespace, or a quote will be returned as a single-character token. If it isTrue
,shlex
will only split words in whitespaces;EOF is signaled with an empty string (
''
);It’s not possible to parse empty strings, even if quoted.
When operating in POSIX mode, shlex
will try to obey the
following parsing rules.
Quotes are stripped out, and do not separate words (
"Do"Not"Separate"
is parsed as the single wordDoNotSeparate
);Non-quoted escape characters (e.g.
'\'
) preserve the literal value of the next character that follows;Enclosing characters in quotes which are not part of
escapedquotes
(e.g."'"
) preserve the literal value of all characters within the quotes;Enclosing characters in quotes which are part of
escapedquotes
(e.g.'"'
) preserves the literal value of all characters within the quotes, with the exception of the characters mentioned inescape
. The escape characters retain their special meaning only when followed by the quote in use, or the escape character itself. Otherwise the escape character will be considered a normal character.EOF is signaled with a
None
value;Quoted empty strings (
''
) are allowed.
Improved Compatibility with Shells¶
Added in version 3.6.
The shlex
class provides compatibility with the parsing performed by
common Unix shells like bash
, dash
, and sh
. To take advantage of
this compatibility, specify the punctuation_chars
argument in the
constructor. This defaults to False
, which preserves pre-3.6 behaviour.
However, if it is set to True
, then parsing of the characters ();<>|&
is changed: any run of these characters is returned as a single token. While
this is short of a full parser for shells (which would be out of scope for the
standard library, given the multiplicity of shells out there), it does allow
you to perform processing of command lines more easily than you could
otherwise. To illustrate, you can see the difference in the following snippet:
>>> import shlex
>>> text = "a && b; c && d || e; f >'abc'; (def \"ghi\")"
>>> s = shlex.shlex(text, posix=True)
>>> s.whitespace_split = True
>>> list(s)
['a', '&&', 'b;', 'c', '&&', 'd', '||', 'e;', 'f', '>abc;', '(def', 'ghi)']
>>> s = shlex.shlex(text, posix=True, punctuation_chars=True)
>>> s.whitespace_split = True
>>> list(s)
['a', '&&', 'b', ';', 'c', '&&', 'd', '||', 'e', ';', 'f', '>', 'abc', ';',
'(', 'def', 'ghi', ')']
Of course, tokens will be returned which are not valid for shells, and you’ll need to implement your own error checks on the returned tokens.
Instead of passing True
as the value for the punctuation_chars parameter,
you can pass a string with specific characters, which will be used to determine
which characters constitute punctuation. For example:
>>> import shlex
>>> s = shlex.shlex("a && b || c", punctuation_chars="|")
>>> list(s)
['a', '&', '&', 'b', '||', 'c']
Note
When punctuation_chars
is specified, the wordchars
attribute is augmented with the characters ~-./*?=
. That is because these
characters can appear in file names (including wildcards) and command-line
arguments (e.g. --color=auto
). Hence:
>>> import shlex
>>> s = shlex.shlex('~/a && b-c --color=auto || d *.py?',
... punctuation_chars=True)
>>> list(s)
['~/a', '&&', 'b-c', '--color=auto', '||', 'd', '*.py?']
However, to match the shell as closely as possible, it is recommended to
always use posix
and whitespace_split
when using
punctuation_chars
, which will negate
wordchars
entirely.
For best effect, punctuation_chars
should be set in conjunction with
posix=True
. (Note that posix=False
is the default for
shlex
.)


— Tokenizer for Python source

tokenize
— Tokenizer for Python source¶
Source code: Lib/tokenize.py
The tokenize
module provides a lexical scanner for Python source code,
implemented in Python. The scanner in this module returns comments as tokens
as well, making it useful for implementing “pretty-printers”, including
colorizers for on-screen displays.
To simplify token stream handling, all operator and
delimiter tokens and Ellipsis
are returned using
the generic OP
token type. The exact
type can be determined by checking the exact_type
property on the
named tuple returned from tokenize.tokenize()
.
Warning
Note that the functions in this module are only designed to parse
syntactically valid Python code (code that does not raise when parsed
using ast.parse()
). The behavior of the functions in this module is
undefined when providing invalid Python code and it can change at any
point.
Tokenizing Input¶
The primary entry point is a generator:
- tokenize.tokenize(readline)¶
The
tokenize()
generator requires one argument, readline, which must be a callable object which provides the same interface as theio.IOBase.readline()
method of file objects. Each call to the function should return one line of input as bytes.The generator produces 5-tuples with these members: the token type; the token string; a 2-tuple
(srow, scol)
of ints specifying the row and column where the token begins in the source; a 2-tuple(erow, ecol)
of ints specifying the row and column where the token ends in the source; and the line on which the token was found. The line passed (the last tuple item) is the physical line. The 5 tuple is returned as a named tuple with the field names:type string start end line
.The returned named tuple has an additional property named
exact_type
that contains the exact operator type forOP
tokens. For all other token typesexact_type
equals the named tupletype
field.Changed in version 3.1: Added support for named tuples.
Changed in version 3.3: Added support for
exact_type
.tokenize()
determines the source encoding of the file by looking for a UTF-8 BOM or encoding cookie, according to PEP 263.
- tokenize.generate_tokens(readline)¶
Tokenize a source reading unicode strings instead of bytes.
Like
tokenize()
, the readline argument is a callable returning a single line of input. However,generate_tokens()
expects readline to return a str object rather than bytes.The result is an iterator yielding named tuples, exactly like
tokenize()
. It does not yield anENCODING
token.
All constants from the token
module are also exported from
tokenize
.
Another function is provided to reverse the tokenization process. This is useful for creating tools that tokenize a script, modify the token stream, and write back the modified script.
- tokenize.untokenize(iterable)¶
Converts tokens back into Python source code. The iterable must return sequences with at least two elements, the token type and the token string. Any additional sequence elements are ignored.
The result is guaranteed to tokenize back to match the input so that the conversion is lossless and round-trips are assured. The guarantee applies only to the token type and token string as the spacing between tokens (column positions) may change.
It returns bytes, encoded using the
ENCODING
token, which is the first token sequence output bytokenize()
. If there is no encoding token in the input, it returns a str instead.
tokenize()
needs to detect the encoding of source files it tokenizes. The
function it uses to do this is available:
- tokenize.detect_encoding(readline)¶
The
detect_encoding()
function is used to detect the encoding that should be used to decode a Python source file. It requires one argument, readline, in the same way as thetokenize()
generator.It will call readline a maximum of twice, and return the encoding used (as a string) and a list of any lines (not decoded from bytes) it has read in.
It detects the encoding from the presence of a UTF-8 BOM or an encoding cookie as specified in PEP 263. If both a BOM and a cookie are present, but disagree, a
SyntaxError
will be raised. Note that if the BOM is found,'utf-8-sig'
will be returned as an encoding.If no encoding is specified, then the default of
'utf-8'
will be returned.Use
open()
to open Python source files: it usesdetect_encoding()
to detect the file encoding.
- tokenize.open(filename)¶
Open a file in read only mode using the encoding detected by
detect_encoding()
.Added in version 3.2.
- exception tokenize.TokenError¶
Raised when either a docstring or expression that may be split over several lines is not completed anywhere in the file, for example:
"""Beginning of docstring
or:
[1, 2, 3
Command-Line Usage¶
Added in version 3.3.
The tokenize
module can be executed as a script from the command line.
It is as simple as:
python -m tokenize [-e] [filename.py]
The following options are accepted:
- -h, --help¶
show this help message and exit
- -e, --exact¶
display token names using the exact type
If filename.py
is specified its contents are tokenized to stdout.
Otherwise, tokenization is performed on stdin.
Examples¶
Example of a script rewriter that transforms float literals into Decimal objects:
from tokenize import tokenize, untokenize, NUMBER, STRING, NAME, OP
from io import BytesIO
def decistmt(s):
"""Substitute Decimals for floats in a string of statements.
>>> from decimal import Decimal
>>> s = 'print(+21.3e-5*-.1234/81.7)'
>>> decistmt(s)
"print (+Decimal ('21.3e-5')*-Decimal ('.1234')/Decimal ('81.7'))"
The format of the exponent is inherited from the platform C library.
Known cases are "e-007" (Windows) and "e-07" (not Windows). Since
we're only showing 12 digits, and the 13th isn't close to 5, the
rest of the output should be platform-independent.
>>> exec(s) #doctest: +ELLIPSIS
-3.21716034272e-0...7
Output from calculations with Decimal should be identical across all
platforms.
>>> exec(decistmt(s))
-3.217160342717258261933904529E-7
"""
result = []
g = tokenize(BytesIO(s.encode('utf-8')).readline) # tokenize the string
for toknum, tokval, _, _, _ in g:
if toknum == NUMBER and '.' in tokval: # replace NUMBER tokens
result.extend([
(NAME, 'Decimal'),
(OP, '('),
(STRING, repr(tokval)),
(OP, ')')
])
else:
result.append((toknum, tokval))
return untokenize(result).decode('utf-8')
Example of tokenizing from the command line. The script:
def say_hello():
print("Hello, World!")
say_hello()
will be tokenized to the following output where the first column is the range of the line/column coordinates where the token is found, the second column is the name of the token, and the final column is the value of the token (if any)
$ python -m tokenize hello.py
0,0-0,0: ENCODING 'utf-8'
1,0-1,3: NAME 'def'
1,4-1,13: NAME 'say_hello'
1,13-1,14: OP '('
1,14-1,15: OP ')'
1,15-1,16: OP ':'
1,16-1,17: NEWLINE '\n'
2,0-2,4: INDENT ' '
2,4-2,9: NAME 'print'
2,9-2,10: OP '('
2,10-2,25: STRING '"Hello, World!"'
2,25-2,26: OP ')'
2,26-2,27: NEWLINE '\n'
3,0-3,1: NL '\n'
4,0-4,0: DEDENT ''
4,0-4,9: NAME 'say_hello'
4,9-4,10: OP '('
4,10-4,11: OP ')'
4,11-4,12: NEWLINE '\n'
5,0-5,0: ENDMARKER ''
The exact token type names can be displayed using the -e
option:
$ python -m tokenize -e hello.py
0,0-0,0: ENCODING 'utf-8'
1,0-1,3: NAME 'def'
1,4-1,13: NAME 'say_hello'
1,13-1,14: LPAR '('
1,14-1,15: RPAR ')'
1,15-1,16: COLON ':'
1,16-1,17: NEWLINE '\n'
2,0-2,4: INDENT ' '
2,4-2,9: NAME 'print'
2,9-2,10: LPAR '('
2,10-2,25: STRING '"Hello, World!"'
2,25-2,26: RPAR ')'
2,26-2,27: NEWLINE '\n'
3,0-3,1: NL '\n'
4,0-4,0: DEDENT ''
4,0-4,9: NAME 'say_hello'
4,9-4,10: LPAR '('
4,10-4,11: RPAR ')'
4,11-4,12: NEWLINE '\n'
5,0-5,0: ENDMARKER ''
Example of tokenizing a file programmatically, reading unicode
strings instead of bytes with generate_tokens()
:
import tokenize
with tokenize.open('hello.py') as f:
tokens = tokenize.generate_tokens(f.readline)
for token in tokens:
print(token)
Or reading bytes directly with tokenize()
:
import tokenize
with open('hello.py', 'rb') as f:
tokens = tokenize.tokenize(f.readline)
for token in tokens:
print(token)


— Testing for Python keywords

keyword
— Testing for Python keywords¶
Source code: Lib/keyword.py
This module allows a Python program to determine if a string is a keyword or soft keyword.
- keyword.kwlist¶
Sequence containing all the keywords defined for the interpreter. If any keywords are defined to only be active when particular
__future__
statements are in effect, these will be included as well.
- keyword.issoftkeyword(s)¶
Return
True
if s is a Python soft keyword.Added in version 3.9.
- keyword.softkwlist¶
Sequence containing all the soft keywords defined for the interpreter. If any soft keywords are defined to only be active when particular
__future__
statements are in effect, these will be included as well.Added in version 3.9.


Buffer Protocol

Buffer Protocol¶
Certain objects available in Python wrap access to an underlying memory
array or buffer. Such objects include the built-in bytes
and
bytearray
, and some extension types like array.array
.
Third-party libraries may define their own types for special purposes, such
as image processing or numeric analysis.
While each of these types have their own semantics, they share the common characteristic of being backed by a possibly large memory buffer. It is then desirable, in some situations, to access that buffer directly and without intermediate copying.
Python provides such a facility at the C and Python level in the form of the buffer protocol. This protocol has two sides:
on the producer side, a type can export a “buffer interface” which allows objects of that type to expose information about their underlying buffer. This interface is described in the section Buffer Object Structures; for Python see Emulating buffer types.
on the consumer side, several means are available to obtain a pointer to the raw underlying data of an object (for example a method parameter). For Python see
memoryview
.
Simple objects such as bytes
and bytearray
expose their
underlying buffer in byte-oriented form. Other forms are possible; for example,
the elements exposed by an array.array
can be multi-byte values.
An example consumer of the buffer interface is the write()
method of file objects: any object that can export a series of bytes through
the buffer interface can be written to a file. While write()
only
needs read-only access to the internal contents of the object passed to it,
other methods such as readinto()
need write access
to the contents of their argument. The buffer interface allows objects to
selectively allow or reject exporting of read-write and read-only buffers.
There are two ways for a consumer of the buffer interface to acquire a buffer over a target object:
call
PyObject_GetBuffer()
with the right parameters;call
PyArg_ParseTuple()
(or one of its siblings) with one of they*
,w*
ors*
format codes.
In both cases, PyBuffer_Release()
must be called when the buffer
isn’t needed anymore. Failure to do so could lead to various issues such as
resource leaks.
Added in version 3.12: The buffer protocol is now accessible in Python, see
Emulating buffer types and memoryview
.
Buffer structure¶
Buffer structures (or simply “buffers”) are useful as a way to expose the binary data from another object to the Python programmer. They can also be used as a zero-copy slicing mechanism. Using their ability to reference a block of memory, it is possible to expose any data to the Python programmer quite easily. The memory could be a large, constant array in a C extension, it could be a raw block of memory for manipulation before passing to an operating system library, or it could be used to pass around structured data in its native, in-memory format.
Contrary to most data types exposed by the Python interpreter, buffers
are not PyObject
pointers but rather simple C structures. This
allows them to be created and copied very simply. When a generic wrapper
around a buffer is needed, a memoryview object
can be created.
For short instructions how to write an exporting object, see
Buffer Object Structures. For obtaining
a buffer, see PyObject_GetBuffer()
.
-
type Py_buffer¶
- Part of the Stable ABI (including all members) since version 3.11.
-
void *buf¶
A pointer to the start of the logical structure described by the buffer fields. This can be any location within the underlying physical memory block of the exporter. For example, with negative
strides
the value may point to the end of the memory block.For contiguous arrays, the value points to the beginning of the memory block.
-
PyObject *obj¶
A new reference to the exporting object. The reference is owned by the consumer and automatically released (i.e. reference count decremented) and set to
NULL
byPyBuffer_Release()
. The field is the equivalent of the return value of any standard C-API function.As a special case, for temporary buffers that are wrapped by
PyMemoryView_FromBuffer()
orPyBuffer_FillInfo()
this field isNULL
. In general, exporting objects MUST NOT use this scheme.
-
Py_ssize_t len¶
product(shape) * itemsize
. For contiguous arrays, this is the length of the underlying memory block. For non-contiguous arrays, it is the length that the logical structure would have if it were copied to a contiguous representation.Accessing
((char *)buf)[0] up to ((char *)buf)[len-1]
is only valid if the buffer has been obtained by a request that guarantees contiguity. In most cases such a request will bePyBUF_SIMPLE
orPyBUF_WRITABLE
.
-
int readonly¶
An indicator of whether the buffer is read-only. This field is controlled by the
PyBUF_WRITABLE
flag.
-
Py_ssize_t itemsize¶
Item size in bytes of a single element. Same as the value of
struct.calcsize()
called on non-NULL
format
values.Important exception: If a consumer requests a buffer without the
PyBUF_FORMAT
flag,format
will be set toNULL
, butitemsize
still has the value for the original format.If
shape
is present, the equalityproduct(shape) * itemsize == len
still holds and the consumer can useitemsize
to navigate the buffer.If
shape
isNULL
as a result of aPyBUF_SIMPLE
or aPyBUF_WRITABLE
request, the consumer must disregarditemsize
and assumeitemsize == 1
.
-
char *format¶
A NULL terminated string in
struct
module style syntax describing the contents of a single item. If this isNULL
,"B"
(unsigned bytes) is assumed.This field is controlled by the
PyBUF_FORMAT
flag.
-
int ndim¶
The number of dimensions the memory represents as an n-dimensional array. If it is
0
,buf
points to a single item representing a scalar. In this case,shape
,strides
andsuboffsets
MUST beNULL
. The maximum number of dimensions is given byPyBUF_MAX_NDIM
.
-
Py_ssize_t *shape¶
An array of
Py_ssize_t
of lengthndim
indicating the shape of the memory as an n-dimensional array. Note thatshape[0] * ... * shape[ndim-1] * itemsize
MUST be equal tolen
.Shape values are restricted to
shape[n] >= 0
. The caseshape[n] == 0
requires special attention. See complex arrays for further information.The shape array is read-only for the consumer.
-
Py_ssize_t *strides¶
An array of
Py_ssize_t
of lengthndim
giving the number of bytes to skip to get to a new element in each dimension.Stride values can be any integer. For regular arrays, strides are usually positive, but a consumer MUST be able to handle the case
strides[n] <= 0
. See complex arrays for further information.The strides array is read-only for the consumer.
-
Py_ssize_t *suboffsets¶
An array of
Py_ssize_t
of lengthndim
. Ifsuboffsets[n] >= 0
, the values stored along the nth dimension are pointers and the suboffset value dictates how many bytes to add to each pointer after de-referencing. A suboffset value that is negative indicates that no de-referencing should occur (striding in a contiguous memory block).If all suboffsets are negative (i.e. no de-referencing is needed), then this field must be
NULL
(the default value).This type of array representation is used by the Python Imaging Library (PIL). See complex arrays for further information how to access elements of such an array.
The suboffsets array is read-only for the consumer.
-
void *internal¶
This is for use internally by the exporting object. For example, this might be re-cast as an integer by the exporter and used to store flags about whether or not the shape, strides, and suboffsets arrays must be freed when the buffer is released. The consumer MUST NOT alter this value.
-
void *buf¶
Constants:
-
PyBUF_MAX_NDIM¶
- Part of the Stable ABI since version 3.11.
The maximum number of dimensions the memory represents. Exporters MUST respect this limit, consumers of multi-dimensional buffers SHOULD be able to handle up to
PyBUF_MAX_NDIM
dimensions. Currently set to 64.
Buffer request types¶
Buffers are usually obtained by sending a buffer request to an exporting
object via PyObject_GetBuffer()
. Since the complexity of the logical
structure of the memory can vary drastically, the consumer uses the flags
argument to specify the exact buffer type it can handle.
All Py_buffer
fields are unambiguously defined by the request
type.
request-independent fields¶
The following fields are not influenced by flags and must always be filled in
with the correct values: obj
, buf
,
len
, itemsize
, ndim
.
readonly, format¶
- PyBUF_WRITABLE¶
- Part of the Stable ABI since version 3.11.
Controls the
readonly
field. If set, the exporter MUST provide a writable buffer or else report failure. Otherwise, the exporter MAY provide either a read-only or writable buffer, but the choice MUST be consistent for all consumers. For example, PyBUF_SIMPLE | PyBUF_WRITABLE can be used to request a simple writable buffer.
- PyBUF_WRITEABLE¶
This is a soft deprecated alias to
PyBUF_WRITABLE
.
- PyBUF_FORMAT¶
- Part of the Stable ABI since version 3.11.
Controls the
format
field. If set, this field MUST be filled in correctly. Otherwise, this field MUST beNULL
.
PyBUF_WRITABLE
can be |’d to any of the flags in the next section.
Since PyBUF_SIMPLE
is defined as 0, PyBUF_WRITABLE
can be used as a stand-alone flag to request a simple writable buffer.
PyBUF_FORMAT
must be |’d to any of the flags except PyBUF_SIMPLE
, because
the latter already implies format B
(unsigned bytes). PyBUF_FORMAT
cannot be
used on its own.
shape, strides, suboffsets¶
The flags that control the logical structure of the memory are listed in decreasing order of complexity. Note that each flag contains all bits of the flags below it.
Request |
shape |
strides |
suboffsets |
|---|---|---|---|
|
yes |
yes |
if needed |
|
yes |
yes |
NULL |
|
yes |
NULL |
NULL |
|
NULL |
NULL |
NULL |
contiguity requests¶
C or Fortran contiguity can be explicitly requested, with and without stride information. Without stride information, the buffer must be C-contiguous.
Request |
shape |
strides |
suboffsets |
contig |
|---|---|---|---|---|
|
yes |
yes |
NULL |
C |
|
yes |
yes |
NULL |
F |
|
yes |
yes |
NULL |
C or F |
yes |
NULL |
NULL |
C |
compound requests¶
All possible requests are fully defined by some combination of the flags in the previous section. For convenience, the buffer protocol provides frequently used combinations as single flags.
In the following table U stands for undefined contiguity. The consumer would
have to call PyBuffer_IsContiguous()
to determine contiguity.
Request |
shape |
strides |
suboffsets |
contig |
readonly |
format |
|---|---|---|---|---|---|---|
|
yes |
yes |
if needed |
U |
0 |
yes |
|
yes |
yes |
if needed |
U |
1 or 0 |
yes |
|
yes |
yes |
NULL |
U |
0 |
yes |
|
yes |
yes |
NULL |
U |
1 or 0 |
yes |
|
yes |
yes |
NULL |
U |
0 |
NULL |
|
yes |
yes |
NULL |
U |
1 or 0 |
NULL |
|
yes |
NULL |
NULL |
C |
0 |
NULL |
|
yes |
NULL |
NULL |
C |
1 or 0 |
NULL |
Complex arrays¶
NumPy-style: shape and strides¶
The logical structure of NumPy-style arrays is defined by itemsize
,
ndim
, shape
and strides
.
If ndim == 0
, the memory location pointed to by buf
is
interpreted as a scalar of size itemsize
. In that case,
both shape
and strides
are NULL
.
If strides
is NULL
, the array is interpreted as
a standard n-dimensional C-array. Otherwise, the consumer must access an
n-dimensional array as follows:
ptr = (char *)buf + indices[0] * strides[0] + ... + indices[n-1] * strides[n-1];
item = *((typeof(item) *)ptr);
As noted above, buf
can point to any location within
the actual memory block. An exporter can check the validity of a buffer with
this function:
def verify_structure(memlen, itemsize, ndim, shape, strides, offset):
"""Verify that the parameters represent a valid array within
the bounds of the allocated memory:
char *mem: start of the physical memory block
memlen: length of the physical memory block
offset: (char *)buf - mem
"""
if offset % itemsize:
return False
if offset < 0 or offset+itemsize > memlen:
return False
if any(v % itemsize for v in strides):
return False
if ndim <= 0:
return ndim == 0 and not shape and not strides
if 0 in shape:
return True
imin = sum(strides[j]*(shape[j]-1) for j in range(ndim)
if strides[j] <= 0)
imax = sum(strides[j]*(shape[j]-1) for j in range(ndim)
if strides[j] > 0)
return 0 <= offset+imin and offset+imax+itemsize <= memlen
PIL-style: shape, strides and suboffsets¶
In addition to the regular items, PIL-style arrays can contain pointers
that must be followed in order to get to the next element in a dimension.
For example, the regular three-dimensional C-array char v[2][2][3]
can
also be viewed as an array of 2 pointers to 2 two-dimensional arrays:
char (*v[2])[2][3]
. In suboffsets representation, those two pointers
can be embedded at the start of buf
, pointing
to two char x[2][3]
arrays that can be located anywhere in memory.
Here is a function that returns a pointer to the element in an N-D array
pointed to by an N-dimensional index when there are both non-NULL
strides
and suboffsets:
void *get_item_pointer(int ndim, void *buf, Py_ssize_t *strides,
Py_ssize_t *suboffsets, Py_ssize_t *indices) {
char *pointer = (char*)buf;
int i;
for (i = 0; i < ndim; i++) {
pointer += strides[i] * indices[i];
if (suboffsets[i] >=0 ) {
pointer = *((char**)pointer) + suboffsets[i];
}
}
return (void*)pointer;
}


: Representing an email message

email.message
: Representing an email message¶
Source code: Lib/email/message.py
Added in version 3.6: [1]
The central class in the email
package is the EmailMessage
class, imported from the email.message
module. It is the base class for
the email
object model. EmailMessage
provides the core
functionality for setting and querying header fields, for accessing message
bodies, and for creating or modifying structured messages.
An email message consists of headers and a payload (which is also referred to as the content). Headers are RFC 5322 or RFC 6532 style field names and values, where the field name and value are separated by a colon. The colon is not part of either the field name or the field value. The payload may be a simple text message, or a binary object, or a structured sequence of sub-messages each with their own set of headers and their own payload. The latter type of payload is indicated by the message having a MIME type such as multipart/* or message/rfc822.
The conceptual model provided by an EmailMessage
object is that of an
ordered dictionary of headers coupled with a payload that represents the
RFC 5322 body of the message, which might be a list of sub-EmailMessage
objects. In addition to the normal dictionary methods for accessing the header
names and values, there are methods for accessing specialized information from
the headers (for example the MIME content type), for operating on the payload,
for generating a serialized version of the message, and for recursively walking
over the object tree.
The EmailMessage
dictionary-like interface is indexed by the header
names, which must be ASCII values. The values of the dictionary are strings
with some extra methods. Headers are stored and returned in case-preserving
form, but field names are matched case-insensitively. The keys are ordered,
but unlike a real dict, there can be duplicates. Additional methods are
provided for working with headers that have duplicate keys.
The payload is either a string or bytes object, in the case of simple message
objects, or a list of EmailMessage
objects, for MIME container
documents such as multipart/* and message/rfc822
message objects.
- class email.message.EmailMessage(policy=default)¶
If policy is specified use the rules it specifies to update and serialize the representation of the message. If policy is not set, use the
default
policy, which follows the rules of the email RFCs except for line endings (instead of the RFC mandated\r\n
, it uses the Python standard\n
line endings). For more information see thepolicy
documentation. [2]- as_string(unixfrom=False, maxheaderlen=None, policy=None)¶
Return the entire message flattened as a string. When optional unixfrom is true, the envelope header is included in the returned string. unixfrom defaults to
False
. For backward compatibility with the baseMessage
class maxheaderlen is accepted, but defaults toNone
, which means that by default the line length is controlled by themax_line_length
of the policy. The policy argument may be used to override the default policy obtained from the message instance. This can be used to control some of the formatting produced by the method, since the specified policy will be passed to theGenerator
.Flattening the message may trigger changes to the
EmailMessage
if defaults need to be filled in to complete the transformation to a string (for example, MIME boundaries may be generated or modified).Note that this method is provided as a convenience and may not be the most useful way to serialize messages in your application, especially if you are dealing with multiple messages. See
email.generator.Generator
for a more flexible API for serializing messages. Note also that this method is restricted to producing messages serialized as “7 bit clean” whenutf8
isFalse
, which is the default.Changed in version 3.6: the default behavior when maxheaderlen is not specified was changed from defaulting to 0 to defaulting to the value of max_line_length from the policy.
- __str__()¶
Equivalent to
as_string(policy=self.policy.clone(utf8=True))
. Allowsstr(msg)
to produce a string containing the serialized message in a readable format.Changed in version 3.4: the method was changed to use
utf8=True
, thus producing an RFC 6531-like message representation, instead of being a direct alias foras_string()
.
- as_bytes(unixfrom=False, policy=None)¶
Return the entire message flattened as a bytes object. When optional unixfrom is true, the envelope header is included in the returned string. unixfrom defaults to
False
. The policy argument may be used to override the default policy obtained from the message instance. This can be used to control some of the formatting produced by the method, since the specified policy will be passed to theBytesGenerator
.Flattening the message may trigger changes to the
EmailMessage
if defaults need to be filled in to complete the transformation to a string (for example, MIME boundaries may be generated or modified).Note that this method is provided as a convenience and may not be the most useful way to serialize messages in your application, especially if you are dealing with multiple messages. See
email.generator.BytesGenerator
for a more flexible API for serializing messages.
- __bytes__()¶
Equivalent to
as_bytes()
. Allowsbytes(msg)
to produce a bytes object containing the serialized message.
- is_multipart()¶
Return
True
if the message’s payload is a list of sub-EmailMessage
objects, otherwise returnFalse
. Whenis_multipart()
returnsFalse
, the payload should be a string object (which might be a CTE encoded binary payload). Note thatis_multipart()
returningTrue
does not necessarily mean that “msg.get_content_maintype() == ‘multipart’” will return theTrue
. For example,is_multipart
will returnTrue
when theEmailMessage
is of typemessage/rfc822
.
- set_unixfrom(unixfrom)¶
Set the message’s envelope header to unixfrom, which should be a string. (See
mboxMessage
for a brief description of this header.)
- get_unixfrom()¶
Return the message’s envelope header. Defaults to
None
if the envelope header was never set.
The following methods implement the mapping-like interface for accessing the message’s headers. Note that there are some semantic differences between these methods and a normal mapping (i.e. dictionary) interface. For example, in a dictionary there are no duplicate keys, but here there may be duplicate message headers. Also, in dictionaries there is no guaranteed order to the keys returned by
keys()
, but in anEmailMessage
object, headers are always returned in the order they appeared in the original message, or in which they were added to the message later. Any header deleted and then re-added is always appended to the end of the header list.These semantic differences are intentional and are biased toward convenience in the most common use cases.
Note that in all cases, any envelope header present in the message is not included in the mapping interface.
- __len__()¶
Return the total number of headers, including duplicates.
- __contains__(name)¶
Return
True
if the message object has a field named name. Matching is done without regard to case and name does not include the trailing colon. Used for thein
operator. For example:if 'message-id' in myMessage: print('Message-ID:', myMessage['message-id'])
- __getitem__(name)¶
Return the value of the named header field. name does not include the colon field separator. If the header is missing,
None
is returned; aKeyError
is never raised.Note that if the named field appears more than once in the message’s headers, exactly which of those field values will be returned is undefined. Use the
get_all()
method to get the values of all the extant headers named name.Using the standard (non-
compat32
) policies, the returned value is an instance of a subclass ofemail.headerregistry.BaseHeader
.
- __setitem__(name, val)¶
Add a header to the message with field name name and value val. The field is appended to the end of the message’s existing headers.
Note that this does not overwrite or delete any existing header with the same name. If you want to ensure that the new header is the only one present in the message with field name name, delete the field first, e.g.:
del msg['subject'] msg['subject'] = 'Python roolz!'
If the
policy
defines certain headers to be unique (as the standard policies do), this method may raise aValueError
when an attempt is made to assign a value to such a header when one already exists. This behavior is intentional for consistency’s sake, but do not depend on it as we may choose to make such assignments do an automatic deletion of the existing header in the future.
- __delitem__(name)¶
Delete all occurrences of the field with name name from the message’s headers. No exception is raised if the named field isn’t present in the headers.
- keys()¶
Return a list of all the message’s header field names.
- values()¶
Return a list of all the message’s field values.
- items()¶
Return a list of 2-tuples containing all the message’s field headers and values.
- get(name, failobj=None)¶
Return the value of the named header field. This is identical to
__getitem__()
except that optional failobj is returned if the named header is missing (failobj defaults toNone
).
Here are some additional useful header related methods:
- get_all(name, failobj=None)¶
Return a list of all the values for the field named name. If there are no such named headers in the message, failobj is returned (defaults to
None
).
- add_header(_name, _value, **_params)¶
Extended header setting. This method is similar to
__setitem__()
except that additional header parameters can be provided as keyword arguments. _name is the header field to add and _value is the primary value for the header.For each item in the keyword argument dictionary _params, the key is taken as the parameter name, with underscores converted to dashes (since dashes are illegal in Python identifiers). Normally, the parameter will be added as
key="value"
unless the value isNone
, in which case only the key will be added.If the value contains non-ASCII characters, the charset and language may be explicitly controlled by specifying the value as a three tuple in the format
(CHARSET, LANGUAGE, VALUE)
, whereCHARSET
is a string naming the charset to be used to encode the value,LANGUAGE
can usually be set toNone
or the empty string (see RFC 2231 for other possibilities), andVALUE
is the string value containing non-ASCII code points. If a three tuple is not passed and the value contains non-ASCII characters, it is automatically encoded in RFC 2231 format using aCHARSET
ofutf-8
and aLANGUAGE
ofNone
.Here is an example:
msg.add_header('Content-Disposition', 'attachment', filename='bud.gif')
This will add a header that looks like
Content-Disposition: attachment; filename="bud.gif"
An example of the extended interface with non-ASCII characters:
msg.add_header('Content-Disposition', 'attachment', filename=('iso-8859-1', '', 'Fußballer.ppt'))
- replace_header(_name, _value)¶
Replace a header. Replace the first header found in the message that matches _name, retaining header order and field name case of the original header. If no matching header is found, raise a
KeyError
.
- get_content_type()¶
Return the message’s content type, coerced to lower case of the form maintype/subtype. If there is no Content-Type header in the message return the value returned by
get_default_type()
. If the Content-Type header is invalid, returntext/plain
.(According to RFC 2045, messages always have a default type,
get_content_type()
will always return a value. RFC 2045 defines a message’s default type to be text/plain unless it appears inside a multipart/digest container, in which case it would be message/rfc822. If the Content-Type header has an invalid type specification, RFC 2045 mandates that the default type be text/plain.)
- get_content_maintype()¶
Return the message’s main content type. This is the maintype part of the string returned by
get_content_type()
.
- get_content_subtype()¶
Return the message’s sub-content type. This is the subtype part of the string returned by
get_content_type()
.
- get_default_type()¶
Return the default content type. Most messages have a default content type of text/plain, except for messages that are subparts of multipart/digest containers. Such subparts have a default content type of message/rfc822.
- set_default_type(ctype)¶
Set the default content type. ctype should either be text/plain or message/rfc822, although this is not enforced. The default content type is not stored in the Content-Type header, so it only affects the return value of the
get_content_type
methods when no Content-Type header is present in the message.
- set_param(param, value, header='Content-Type', requote=True, charset=None, language='', replace=False)¶
Set a parameter in the Content-Type header. If the parameter already exists in the header, replace its value with value. When header is
Content-Type
(the default) and the header does not yet exist in the message, add it, set its value to text/plain, and append the new parameter value. Optional header specifies an alternative header to Content-Type.If the value contains non-ASCII characters, the charset and language may be explicitly specified using the optional charset and language parameters. Optional language specifies the RFC 2231 language, defaulting to the empty string. Both charset and language should be strings. The default is to use the
utf8
charset andNone
for the language.If replace is
False
(the default) the header is moved to the end of the list of headers. If replace isTrue
, the header will be updated in place.Use of the requote parameter with
EmailMessage
objects is deprecated.Note that existing parameter values of headers may be accessed through the
params
attribute of the header value (for example,msg['Content-Type'].params['charset']
).Changed in version 3.4:
replace
keyword was added.
- del_param(param, header='content-type', requote=True)¶
Remove the given parameter completely from the Content-Type header. The header will be re-written in place without the parameter or its value. Optional header specifies an alternative to Content-Type.
Use of the requote parameter with
EmailMessage
objects is deprecated.
- get_filename(failobj=None)¶
Return the value of the
filename
parameter of the Content-Disposition header of the message. If the header does not have afilename
parameter, this method falls back to looking for thename
parameter on the Content-Type header. If neither is found, or the header is missing, then failobj is returned. The returned string will always be unquoted as peremail.utils.unquote()
.
- get_boundary(failobj=None)¶
Return the value of the
boundary
parameter of the Content-Type header of the message, or failobj if either the header is missing, or has noboundary
parameter. The returned string will always be unquoted as peremail.utils.unquote()
.
- set_boundary(boundary)¶
Set the
boundary
parameter of the Content-Type header to boundary.set_boundary()
will always quote boundary if necessary. AHeaderParseError
is raised if the message object has no Content-Type header.Note that using this method is subtly different from deleting the old Content-Type header and adding a new one with the new boundary via
add_header()
, becauseset_boundary()
preserves the order of the Content-Type header in the list of headers.
- get_content_charset(failobj=None)¶
Return the
charset
parameter of the Content-Type header, coerced to lower case. If there is no Content-Type header, or if that header has nocharset
parameter, failobj is returned.
- get_charsets(failobj=None)¶
Return a list containing the character set names in the message. If the message is a multipart, then the list will contain one element for each subpart in the payload, otherwise, it will be a list of length 1.
Each item in the list will be a string which is the value of the
charset
parameter in the Content-Type header for the represented subpart. If the subpart has no Content-Type header, nocharset
parameter, or is not of the text main MIME type, then that item in the returned list will be failobj.
- is_attachment()¶
Return
True
if there is a Content-Disposition header and its (case insensitive) value isattachment
,False
otherwise.Changed in version 3.4.2: is_attachment is now a method instead of a property, for consistency with
is_multipart()
.
- get_content_disposition()¶
Return the lowercased value (without parameters) of the message’s Content-Disposition header if it has one, or
None
. The possible values for this method are inline, attachment orNone
if the message follows RFC 2183.Added in version 3.5.
The following methods relate to interrogating and manipulating the content (payload) of the message.
- walk()¶
The
walk()
method is an all-purpose generator which can be used to iterate over all the parts and subparts of a message object tree, in depth-first traversal order. You will typically usewalk()
as the iterator in afor
loop; each iteration returns the next subpart.Here’s an example that prints the MIME type of every part of a multipart message structure:
>>> for part in msg.walk(): ... print(part.get_content_type()) multipart/report text/plain message/delivery-status text/plain text/plain message/rfc822 text/plain
walk
iterates over the subparts of any part whereis_multipart()
returnsTrue
, even thoughmsg.get_content_maintype() == 'multipart'
may returnFalse
. We can see this in our example by making use of the_structure
debug helper function:>>> from email.iterators import _structure >>> for part in msg.walk(): ... print(part.get_content_maintype() == 'multipart', ... part.is_multipart()) True True False False False True False False False False False True False False >>> _structure(msg) multipart/report text/plain message/delivery-status text/plain text/plain message/rfc822 text/plain
Here the
message
parts are notmultiparts
, but they do contain subparts.is_multipart()
returnsTrue
andwalk
descends into the subparts.
- get_body(preferencelist=('related', 'html', 'plain'))¶
Return the MIME part that is the best candidate to be the “body” of the message.
preferencelist must be a sequence of strings from the set
related
,html
, andplain
, and indicates the order of preference for the content type of the part returned.Start looking for candidate matches with the object on which the
get_body
method is called.If
related
is not included in preferencelist, consider the root part (or subpart of the root part) of any related encountered as a candidate if the (sub-)part matches a preference.When encountering a
multipart/related
, check thestart
parameter and if a part with a matching Content-ID is found, consider only it when looking for candidate matches. Otherwise consider only the first (default root) part of themultipart/related
.If a part has a Content-Disposition header, only consider the part a candidate match if the value of the header is
inline
.If none of the candidates matches any of the preferences in preferencelist, return
None
.Notes: (1) For most applications the only preferencelist combinations that really make sense are
('plain',)
,('html', 'plain')
, and the default('related', 'html', 'plain')
. (2) Because matching starts with the object on whichget_body
is called, callingget_body
on amultipart/related
will return the object itself unless preferencelist has a non-default value. (3) Messages (or message parts) that do not specify a Content-Type or whose Content-Type header is invalid will be treated as if they are of typetext/plain
, which may occasionally causeget_body
to return unexpected results.
- iter_attachments()¶
Return an iterator over all of the immediate sub-parts of the message that are not candidate “body” parts. That is, skip the first occurrence of each of
text/plain
,text/html
,multipart/related
, ormultipart/alternative
(unless they are explicitly marked as attachments via Content-Disposition: attachment), and return all remaining parts. When applied directly to amultipart/related
, return an iterator over the all the related parts except the root part (ie: the part pointed to by thestart
parameter, or the first part if there is nostart
parameter or thestart
parameter doesn’t match the Content-ID of any of the parts). When applied directly to amultipart/alternative
or a non-multipart
, return an empty iterator.
- iter_parts()¶
Return an iterator over all of the immediate sub-parts of the message, which will be empty for a non-
multipart
. (See alsowalk()
.)
- get_content(*args, content_manager=None, **kw)¶
Call the
get_content()
method of the content_manager, passing self as the message object, and passing along any other arguments or keywords as additional arguments. If content_manager is not specified, use thecontent_manager
specified by the currentpolicy
.
- set_content(*args, content_manager=None, **kw)¶
Call the
set_content()
method of the content_manager, passing self as the message object, and passing along any other arguments or keywords as additional arguments. If content_manager is not specified, use thecontent_manager
specified by the currentpolicy
.
Convert a non-
multipart
message into amultipart/related
message, moving any existing Content- headers and payload into a (new) first part of themultipart
. If boundary is specified, use it as the boundary string in the multipart, otherwise leave the boundary to be automatically created when it is needed (for example, when the message is serialized).
- make_alternative(boundary=None)¶
Convert a non-
multipart
or amultipart/related
into amultipart/alternative
, moving any existing Content- headers and payload into a (new) first part of themultipart
. If boundary is specified, use it as the boundary string in the multipart, otherwise leave the boundary to be automatically created when it is needed (for example, when the message is serialized).
- make_mixed(boundary=None)¶
Convert a non-
multipart
, amultipart/related
, or amultipart-alternative
into amultipart/mixed
, moving any existing Content- headers and payload into a (new) first part of themultipart
. If boundary is specified, use it as the boundary string in the multipart, otherwise leave the boundary to be automatically created when it is needed (for example, when the message is serialized).
If the message is a
multipart/related
, create a new message object, pass all of the arguments to itsset_content()
method, andattach()
it to themultipart
. If the message is a non-multipart
, callmake_related()
and then proceed as above. If the message is any other type ofmultipart
, raise aTypeError
. If content_manager is not specified, use thecontent_manager
specified by the currentpolicy
. If the added part has no Content-Disposition header, add one with the valueinline
.
- add_alternative(*args, content_manager=None, **kw)¶
If the message is a
multipart/alternative
, create a new message object, pass all of the arguments to itsset_content()
method, andattach()
it to themultipart
. If the message is a non-multipart
ormultipart/related
, callmake_alternative()
and then proceed as above. If the message is any other type ofmultipart
, raise aTypeError
. If content_manager is not specified, use thecontent_manager
specified by the currentpolicy
.
- add_attachment(*args, content_manager=None, **kw)¶
If the message is a
multipart/mixed
, create a new message object, pass all of the arguments to itsset_content()
method, andattach()
it to themultipart
. If the message is a non-multipart
,multipart/related
, ormultipart/alternative
, callmake_mixed()
and then proceed as above. If content_manager is not specified, use thecontent_manager
specified by the currentpolicy
. If the added part has no Content-Disposition header, add one with the valueattachment
. This method can be used both for explicit attachments (Content-Disposition: attachment) andinline
attachments (Content-Disposition: inline), by passing appropriate options to thecontent_manager
.
- clear()¶
Remove the payload and all of the headers.
- clear_content()¶
Remove the payload and all of the !Content- headers, leaving all other headers intact and in their original order.
EmailMessage
objects have the following instance attributes:- preamble¶
The format of a MIME document allows for some text between the blank line following the headers, and the first multipart boundary string. Normally, this text is never visible in a MIME-aware mail reader because it falls outside the standard MIME armor. However, when viewing the raw text of the message, or when viewing the message in a non-MIME aware reader, this text can become visible.
The preamble attribute contains this leading extra-armor text for MIME documents. When the
Parser
discovers some text after the headers but before the first boundary string, it assigns this text to the message’s preamble attribute. When theGenerator
is writing out the plain text representation of a MIME message, and it finds the message has a preamble attribute, it will write this text in the area between the headers and the first boundary. Seeemail.parser
andemail.generator
for details.Note that if the message object has no preamble, the preamble attribute will be
None
.
- epilogue¶
The epilogue attribute acts the same way as the preamble attribute, except that it contains text that appears between the last boundary and the end of the message. As with the
preamble
, if there is no epilog text this attribute will beNone
.
- defects¶
The defects attribute contains a list of all the problems found when parsing this message. See
email.errors
for a detailed description of the possible parsing defects.
- class email.message.MIMEPart(policy=default)¶
This class represents a subpart of a MIME message. It is identical to
EmailMessage
, except that no MIME-Version headers are added whenset_content()
is called, since sub-parts do not need their own MIME-Version headers.
Footnotes


Byte Array Objects

Byte Array Objects¶
-
PyTypeObject PyByteArray_Type¶
- Part of the Stable ABI.
This instance of
PyTypeObject
represents the Python bytearray type; it is the same object asbytearray
in the Python layer.
Type check macros¶
Direct API functions¶
-
PyObject *PyByteArray_FromObject(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Return a new bytearray object from any object, o, that implements the buffer protocol.
On failure, return
NULL
with an exception set.
-
PyObject *PyByteArray_FromStringAndSize(const char *string, Py_ssize_t len)¶
- Return value: New reference. Part of the Stable ABI.
Create a new bytearray object from string and its length, len.
On failure, return
NULL
with an exception set.
-
PyObject *PyByteArray_Concat(PyObject *a, PyObject *b)¶
- Return value: New reference. Part of the Stable ABI.
Concat bytearrays a and b and return a new bytearray with the result.
On failure, return
NULL
with an exception set.
-
Py_ssize_t PyByteArray_Size(PyObject *bytearray)¶
- Part of the Stable ABI.
Return the size of bytearray after checking for a
NULL
pointer.
-
char *PyByteArray_AsString(PyObject *bytearray)¶
- Part of the Stable ABI.
Return the contents of bytearray as a char array after checking for a
NULL
pointer. The returned array always has an extra null byte appended.
-
int PyByteArray_Resize(PyObject *bytearray, Py_ssize_t len)¶
- Part of the Stable ABI.
Resize the internal buffer of bytearray to len. Failure is a
-1
return with an exception set.Changed in version 3.14: A negative len will now result in an exception being set and -1 returned.
Macros¶
These macros trade safety for speed and they don’t check pointers.
-
char *PyByteArray_AS_STRING(PyObject *bytearray)¶
Similar to
PyByteArray_AsString()
, but without error checking.
-
Py_ssize_t PyByteArray_GET_SIZE(PyObject *bytearray)¶
Similar to
PyByteArray_Size()
, but without error checking.


email.mime
: Creating email and MIME objects from scratch¶
Source code: Lib/email/mime/
This module is part of the legacy (Compat32
) email API. Its functionality
is partially replaced by the contentmanager
in the new API, but
in certain applications these classes may still be useful, even in non-legacy
code.
Ordinarily, you get a message object structure by passing a file or some text to
a parser, which parses the text and returns the root message object. However
you can also build a complete message structure from scratch, or even individual
Message
objects by hand. In fact, you can also take an
existing structure and add new Message
objects, move them
around, etc. This makes a very convenient interface for slicing-and-dicing MIME
messages.
You can create a new object structure by creating Message
instances, adding attachments and all the appropriate headers manually. For MIME
messages though, the email
package provides some convenient subclasses to
make things easier.
Here are the classes:
- class email.mime.base.MIMEBase(_maintype, _subtype, *, policy=compat32, **_params)¶
Module:
email.mime.base
This is the base class for all the MIME-specific subclasses of
Message
. Ordinarily you won’t create instances specifically ofMIMEBase
, although you could.MIMEBase
is provided primarily as a convenient base class for more specific MIME-aware subclasses._maintype is the Content-Type major type (e.g. text or image), and _subtype is the Content-Type minor type (e.g. plain or gif). _params is a parameter key/value dictionary and is passed directly to
Message.add_header
.If policy is specified, (defaults to the
compat32
policy) it will be passed toMessage
.The
MIMEBase
class always adds a Content-Type header (based on _maintype, _subtype, and _params), and a MIME-Version header (always set to1.0
).Changed in version 3.6: Added policy keyword-only parameter.
- class email.mime.nonmultipart.MIMENonMultipart¶
Module:
email.mime.nonmultipart
A subclass of
MIMEBase
, this is an intermediate base class for MIME messages that are not multipart. The primary purpose of this class is to prevent the use of theattach()
method, which only makes sense for multipart messages. Ifattach()
is called, aMultipartConversionError
exception is raised.
- class email.mime.multipart.MIMEMultipart(_subtype='mixed', boundary=None, _subparts=None, *, policy=compat32, **_params)¶
Module:
email.mime.multipart
A subclass of
MIMEBase
, this is an intermediate base class for MIME messages that are multipart. Optional _subtype defaults to mixed, but can be used to specify the subtype of the message. A Content-Type header of multipart/_subtype will be added to the message object. A MIME-Version header will also be added.Optional boundary is the multipart boundary string. When
None
(the default), the boundary is calculated when needed (for example, when the message is serialized)._subparts is a sequence of initial subparts for the payload. It must be possible to convert this sequence to a list. You can always attach new subparts to the message by using the
Message.attach
method.Optional policy argument defaults to
compat32
.Additional parameters for the Content-Type header are taken from the keyword arguments, or passed into the _params argument, which is a keyword dictionary.
Changed in version 3.6: Added policy keyword-only parameter.
- class email.mime.application.MIMEApplication(_data, _subtype='octet-stream', _encoder=email.encoders.encode_base64, *, policy=compat32, **_params)¶
Module:
email.mime.application
A subclass of
MIMENonMultipart
, theMIMEApplication
class is used to represent MIME message objects of major type application. _data contains the bytes for the raw application data. Optional _subtype specifies the MIME subtype and defaults to octet-stream.Optional _encoder is a callable (i.e. function) which will perform the actual encoding of the data for transport. This callable takes one argument, which is the
MIMEApplication
instance. It should useget_payload()
andset_payload()
to change the payload to encoded form. It should also add any Content-Transfer-Encoding or other headers to the message object as necessary. The default encoding is base64. See theemail.encoders
module for a list of the built-in encoders.Optional policy argument defaults to
compat32
._params are passed straight through to the base class constructor.
Changed in version 3.6: Added policy keyword-only parameter.
- class email.mime.audio.MIMEAudio(_audiodata, _subtype=None, _encoder=email.encoders.encode_base64, *, policy=compat32, **_params)¶
Module:
email.mime.audio
A subclass of
MIMENonMultipart
, theMIMEAudio
class is used to create MIME message objects of major type audio. _audiodata contains the bytes for the raw audio data. If this data can be decoded as au, wav, aiff, or aifc, then the subtype will be automatically included in the Content-Type header. Otherwise you can explicitly specify the audio subtype via the _subtype argument. If the minor type could not be guessed and _subtype was not given, thenTypeError
is raised.Optional _encoder is a callable (i.e. function) which will perform the actual encoding of the audio data for transport. This callable takes one argument, which is the
MIMEAudio
instance. It should useget_payload()
andset_payload()
to change the payload to encoded form. It should also add any Content-Transfer-Encoding or other headers to the message object as necessary. The default encoding is base64. See theemail.encoders
module for a list of the built-in encoders.Optional policy argument defaults to
compat32
._params are passed straight through to the base class constructor.
Changed in version 3.6: Added policy keyword-only parameter.
- class email.mime.image.MIMEImage(_imagedata, _subtype=None, _encoder=email.encoders.encode_base64, *, policy=compat32, **_params)¶
Module:
email.mime.image
A subclass of
MIMENonMultipart
, theMIMEImage
class is used to create MIME message objects of major type image. _imagedata contains the bytes for the raw image data. If this data type can be detected (jpeg, png, gif, tiff, rgb, pbm, pgm, ppm, rast, xbm, bmp, webp, and exr attempted), then the subtype will be automatically included in the Content-Type header. Otherwise you can explicitly specify the image subtype via the _subtype argument. If the minor type could not be guessed and _subtype was not given, thenTypeError
is raised.Optional _encoder is a callable (i.e. function) which will perform the actual encoding of the image data for transport. This callable takes one argument, which is the
MIMEImage
instance. It should useget_payload()
andset_payload()
to change the payload to encoded form. It should also add any Content-Transfer-Encoding or other headers to the message object as necessary. The default encoding is base64. See theemail.encoders
module for a list of the built-in encoders.Optional policy argument defaults to
compat32
._params are passed straight through to the
MIMEBase
constructor.Changed in version 3.6: Added policy keyword-only parameter.
- class email.mime.message.MIMEMessage(_msg, _subtype='rfc822', *, policy=compat32)¶
Module:
email.mime.message
A subclass of
MIMENonMultipart
, theMIMEMessage
class is used to create MIME objects of main type message. _msg is used as the payload, and must be an instance of classMessage
(or a subclass thereof), otherwise aTypeError
is raised.Optional _subtype sets the subtype of the message; it defaults to rfc822.
Optional policy argument defaults to
compat32
.Changed in version 3.6: Added policy keyword-only parameter.
- class email.mime.text.MIMEText(_text, _subtype='plain', _charset=None, *, policy=compat32)¶
Module:
email.mime.text
A subclass of
MIMENonMultipart
, theMIMEText
class is used to create MIME objects of major type text. _text is the string for the payload. _subtype is the minor type and defaults to plain. _charset is the character set of the text and is passed as an argument to theMIMENonMultipart
constructor; it defaults tous-ascii
if the string contains onlyascii
code points, andutf-8
otherwise. The _charset parameter accepts either a string or aCharset
instance.Unless the _charset argument is explicitly set to
None
, the MIMEText object created will have both a Content-Type header with acharset
parameter, and a Content-Transfer-Encoding header. This means that a subsequentset_payload
call will not result in an encoded payload, even if a charset is passed in theset_payload
command. You can “reset” this behavior by deleting theContent-Transfer-Encoding
header, after which aset_payload
call will automatically encode the new payload (and add a new Content-Transfer-Encoding header).Optional policy argument defaults to
compat32
.Changed in version 3.5: _charset also accepts
Charset
instances.Changed in version 3.6: Added policy keyword-only parameter.


— Unix syslog library routines

syslog
— Unix syslog library routines¶
This module provides an interface to the Unix syslog
library routines.
Refer to the Unix manual pages for a detailed description of the syslog
facility.
Availability: Unix, not WASI, not iOS.
This module wraps the system syslog
family of routines. A pure Python
library that can speak to a syslog server is available in the
logging.handlers
module as SysLogHandler
.
The module defines the following functions:
- syslog.syslog(message)¶
- syslog.syslog(priority, message)
Send the string message to the system logger. A trailing newline is added if necessary. Each message is tagged with a priority composed of a facility and a level. The optional priority argument, which defaults to
LOG_INFO
, determines the message priority. If the facility is not encoded in priority using logical-or (LOG_INFO | LOG_USER
), the value given in theopenlog()
call is used.If
openlog()
has not been called prior to the call tosyslog()
,openlog()
will be called with no arguments.Raises an auditing event
syslog.syslog
with argumentspriority
,message
.Changed in version 3.2: In previous versions,
openlog()
would not be called automatically if it wasn’t called prior to the call tosyslog()
, deferring to the syslog implementation to callopenlog()
.Changed in version 3.12: This function is restricted in subinterpreters. (Only code that runs in multiple interpreters is affected and the restriction is not relevant for most users.)
openlog()
must be called in the main interpreter beforesyslog()
may be used in a subinterpreter. Otherwise it will raiseRuntimeError
.
- syslog.openlog([ident[, logoption[, facility]]])¶
Logging options of subsequent
syslog()
calls can be set by callingopenlog()
.syslog()
will callopenlog()
with no arguments if the log is not currently open.The optional ident keyword argument is a string which is prepended to every message, and defaults to
sys.argv[0]
with leading path components stripped. The optional logoption keyword argument (default is 0) is a bit field – see below for possible values to combine. The optional facility keyword argument (default isLOG_USER
) sets the default facility for messages which do not have a facility explicitly encoded.Raises an auditing event
syslog.openlog
with argumentsident
,logoption
,facility
.Changed in version 3.2: In previous versions, keyword arguments were not allowed, and ident was required.
Changed in version 3.12: This function is restricted in subinterpreters. (Only code that runs in multiple interpreters is affected and the restriction is not relevant for most users.) This may only be called in the main interpreter. It will raise
RuntimeError
if called in a subinterpreter.
- syslog.closelog()¶
Reset the syslog module values and call the system library
closelog()
.This causes the module to behave as it does when initially imported. For example,
openlog()
will be called on the firstsyslog()
call (ifopenlog()
hasn’t already been called), and ident and otheropenlog()
parameters are reset to defaults.Raises an auditing event
syslog.closelog
with no arguments.Changed in version 3.12: This function is restricted in subinterpreters. (Only code that runs in multiple interpreters is affected and the restriction is not relevant for most users.) This may only be called in the main interpreter. It will raise
RuntimeError
if called in a subinterpreter.
- syslog.setlogmask(maskpri)¶
Set the priority mask to maskpri and return the previous mask value. Calls to
syslog()
with a priority level not set in maskpri are ignored. The default is to log all priorities. The functionLOG_MASK(pri)
calculates the mask for the individual priority pri. The functionLOG_UPTO(pri)
calculates the mask for all priorities up to and including pri.Raises an auditing event
syslog.setlogmask
with argumentmaskpri
.
The module defines the following constants:
- syslog.LOG_EMERG¶
- syslog.LOG_ALERT¶
- syslog.LOG_CRIT¶
- syslog.LOG_ERR¶
- syslog.LOG_WARNING¶
- syslog.LOG_NOTICE¶
- syslog.LOG_INFO¶
- syslog.LOG_DEBUG¶
Priority levels (high to low).
- syslog.LOG_AUTH¶
- syslog.LOG_AUTHPRIV¶
- syslog.LOG_CRON¶
- syslog.LOG_DAEMON¶
- syslog.LOG_FTP¶
- syslog.LOG_INSTALL¶
- syslog.LOG_KERN¶
- syslog.LOG_LAUNCHD¶
- syslog.LOG_LPR¶
- syslog.LOG_MAIL¶
- syslog.LOG_NETINFO¶
- syslog.LOG_NEWS¶
- syslog.LOG_RAS¶
- syslog.LOG_REMOTEAUTH¶
- syslog.LOG_SYSLOG¶
- syslog.LOG_USER¶
- syslog.LOG_UUCP¶
- syslog.LOG_LOCAL0¶
- syslog.LOG_LOCAL1¶
- syslog.LOG_LOCAL2¶
- syslog.LOG_LOCAL3¶
- syslog.LOG_LOCAL4¶
- syslog.LOG_LOCAL5¶
- syslog.LOG_LOCAL6¶
- syslog.LOG_LOCAL7¶
Facilities, depending on availability in
<syslog.h>
forLOG_AUTHPRIV
,LOG_FTP
,LOG_NETINFO
,LOG_REMOTEAUTH
,LOG_INSTALL
andLOG_RAS
.Changed in version 3.13: Added
LOG_FTP
,LOG_NETINFO
,LOG_REMOTEAUTH
,LOG_INSTALL
,LOG_RAS
, andLOG_LAUNCHD
.
- syslog.LOG_PID¶
- syslog.LOG_CONS¶
- syslog.LOG_NDELAY¶
- syslog.LOG_ODELAY¶
- syslog.LOG_NOWAIT¶
- syslog.LOG_PERROR¶
Log options, depending on availability in
<syslog.h>
forLOG_ODELAY
,LOG_NOWAIT
andLOG_PERROR
.
Examples¶
Simple example¶
A simple set of examples:
import syslog
syslog.syslog('Processing started')
if error:
syslog.syslog(syslog.LOG_ERR, 'Processing started')
An example of setting some log options, these would include the process ID in logged messages, and write the messages to the destination facility used for mail logging:
syslog.openlog(logoption=syslog.LOG_PID, facility=syslog.LOG_MAIL)
syslog.syslog('E-mail processing initiated...')


: Representing character sets

email.charset
: Representing character sets¶
Source code: Lib/email/charset.py
This module is part of the legacy (Compat32
) email API. In the new
API only the aliases table is used.
The remaining text in this section is the original documentation of the module.
This module provides a class Charset
for representing character sets
and character set conversions in email messages, as well as a character set
registry and several convenience methods for manipulating this registry.
Instances of Charset
are used in several other modules within the
email
package.
Import this class from the email.charset
module.
- class email.charset.Charset(input_charset=DEFAULT_CHARSET)¶
Map character sets to their email properties.
This class provides information about the requirements imposed on email for a specific character set. It also provides convenience routines for converting between character sets, given the availability of the applicable codecs. Given a character set, it will do its best to provide information on how to use that character set in an email message in an RFC-compliant way.
Certain character sets must be encoded with quoted-printable or base64 when used in email headers or bodies. Certain character sets must be converted outright, and are not allowed in email.
Optional input_charset is as described below; it is always coerced to lower case. After being alias normalized it is also used as a lookup into the registry of character sets to find out the header encoding, body encoding, and output conversion codec to be used for the character set. For example, if input_charset is
iso-8859-1
, then headers and bodies will be encoded using quoted-printable and no output conversion codec is necessary. If input_charset iseuc-jp
, then headers will be encoded with base64, bodies will not be encoded, but output text will be converted from theeuc-jp
character set to theiso-2022-jp
character set.Charset
instances have the following data attributes:- input_charset¶
The initial character set specified. Common aliases are converted to their official email names (e.g.
latin_1
is converted toiso-8859-1
). Defaults to 7-bitus-ascii
.
- header_encoding¶
If the character set must be encoded before it can be used in an email header, this attribute will be set to
charset.QP
(for quoted-printable),charset.BASE64
(for base64 encoding), orcharset.SHORTEST
for the shortest of QP or BASE64 encoding. Otherwise, it will beNone
.
- body_encoding¶
Same as header_encoding, but describes the encoding for the mail message’s body, which indeed may be different than the header encoding.
charset.SHORTEST
is not allowed for body_encoding.
- output_charset¶
Some character sets must be converted before they can be used in email headers or bodies. If the input_charset is one of them, this attribute will contain the name of the character set output will be converted to. Otherwise, it will be
None
.
- input_codec¶
The name of the Python codec used to convert the input_charset to Unicode. If no conversion codec is necessary, this attribute will be
None
.
- output_codec¶
The name of the Python codec used to convert Unicode to the output_charset. If no conversion codec is necessary, this attribute will have the same value as the input_codec.
Charset
instances also have the following methods:- get_body_encoding()¶
Return the content transfer encoding used for body encoding.
This is either the string
quoted-printable
orbase64
depending on the encoding used, or it is a function, in which case you should call the function with a single argument, the Message object being encoded. The function should then set the Content-Transfer-Encoding header itself to whatever is appropriate.Returns the string
quoted-printable
if body_encoding isQP
, returns the stringbase64
if body_encoding isBASE64
, and returns the string7bit
otherwise.
- get_output_charset()¶
Return the output character set.
This is the output_charset attribute if that is not
None
, otherwise it is input_charset.
- header_encode(string)¶
Header-encode the string string.
The type of encoding (base64 or quoted-printable) will be based on the header_encoding attribute.
- header_encode_lines(string, maxlengths)¶
Header-encode a string by converting it first to bytes.
This is similar to
header_encode()
except that the string is fit into maximum line lengths as given by the argument maxlengths, which must be an iterator: each element returned from this iterator will provide the next maximum line length.
- body_encode(string)¶
Body-encode the string string.
The type of encoding (base64 or quoted-printable) will be based on the body_encoding attribute.
The
Charset
class also provides a number of methods to support standard operations and built-in functions.- __str__()¶
Returns input_charset as a string coerced to lower case.
__repr__()
is an alias for__str__()
.
The email.charset
module also provides the following functions for adding
new entries to the global character set, alias, and codec registries:
- email.charset.add_charset(charset, header_enc=None, body_enc=None, output_charset=None)¶
Add character properties to the global registry.
charset is the input character set, and must be the canonical name of a character set.
Optional header_enc and body_enc is either
charset.QP
for quoted-printable,charset.BASE64
for base64 encoding,charset.SHORTEST
for the shortest of quoted-printable or base64 encoding, orNone
for no encoding.SHORTEST
is only valid for header_enc. The default isNone
for no encoding.Optional output_charset is the character set that the output should be in. Conversions will proceed from input charset, to Unicode, to the output charset when the method
Charset.convert()
is called. The default is to output in the same character set as the input.Both input_charset and output_charset must have Unicode codec entries in the module’s character set-to-codec mapping; use
add_codec()
to add codecs the module does not know about. See thecodecs
module’s documentation for more information.The global character set registry is kept in the module global dictionary
CHARSETS
.
- email.charset.add_alias(alias, canonical)¶
Add a character set alias. alias is the alias name, e.g.
latin-1
. canonical is the character set’s canonical name, e.g.iso-8859-1
.The global charset alias registry is kept in the module global dictionary
ALIASES
.