3 Writing the Setup Script

distutils-term.html | dist.html | listing-packages.html | Distributing Python Modules | contents.html
---
# 3 Writing the Setup Script
The setup script is the centre of all activity in building,
distributing, and installing modules using the Distutils. The main
purpose of the setup script is to describe your module distribution to
the Distutils, so that the various commands that operate on your modules
do the right thing. As we saw in section 2.1 (simple-example.html#simple-example) above,
the setup script consists mainly of a call to setup(), and
most information supplied to the Distutils by the module developer is
supplied as keyword arguments to setup().
Here's a slightly more involved example, which we'll follow for the next
couple of sections: the Distutils' own setup script. (Keep in mind that
although the Distutils are included with Python 1.6 and later, they also
have an independent existence so that Python 1.5.2 users can use them to
install other module distributions. The Distutils' own setup script,
shown here, is used to install the package into Python 1.5.2.)
```text

#!/usr/bin/env python

from distutils.core import setup

setup (name = "Distutils",
version = "1.0",
description = "Python Distribution Utilities",
author = "Greg Ward",
author_email = "gward@python.net",
url = "http://www.python.org/sigs/distutils-sig/",

packages = ['distutils', 'distutils.command'],
)
```
There are only two differences between this and the trivial one-file
distribution presented in section 2.1 (simple-example.html#simple-example): more
meta-data, and the specification of pure Python modules by package,
rather than by module. This is important since the Distutils consist of
a couple of dozen modules split into (so far) two packages; an explicit
list of every module would be tedious to generate and difficult to
maintain.
Note that any pathnames (files or directories) supplied in the setup
script should be written using the Unix convention, i.e.
slash-separated. The Distutils will take care of converting this
platform-neutral representation into whatever is appropriate on your
current platform before actually using the pathname. This makes your
setup script portable across operating systems, which of course is one
of the major goals of the Distutils. In this spirit, all pathnames in
this document are slash-separated (Mac OS programmers should keep in
mind that the absence of a leading slash indicates a relative
path, the opposite of the Mac OS convention with colons).


2.1 A simple example

concepts.html | concepts.html | python-terms.html | Distributing Python Modules | contents.html
---
## 2.1 A simple example
The setup script is usually quite simple, although since it's written in
Python, there are no arbitrary limits to what you can do with it. If
all you want to do is distribute a module called foo, contained
in a file foo.py, then your setup script can be as little as
this:
```text

from distutils.core import setup
setup (name = "foo",
version = "1.0",
py_modules = ["foo"])
```
Some observations:
- most information that you supply to the Distutils is supplied as
keyword arguments to the setup() function
- those keyword arguments fall into two categories: package
meta-data (name, version number) and information about what's in the
package (a list of pure Python modules, in this case)
- modules are specified by module name, not filename (the same will
hold true for packages and extensions)
- it's recommended that you supply a little more meta-data, in
particular your name, email address and a URL for the project
To create a source distribution for this module, you would create a
setup script, setup.py, containing the above code, and run:
```text

python setup.py sdist
```
which will create an archive file (e.g., tarball on Unix, zip file on
Windows) containing your setup script, setup.py, and your module,
foo.py. The archive file will be named Foo-1.0.tar.gz (or
.zip), and will unpack into a directory Foo-1.0.
If an end-user wishes to install your foo module, all she has
to do is download Foo-1.0.tar.gz (or .zip), unpack it,
and--from the Foo-1.0 directory--run
```text

python setup.py install
```
which will ultimately copy foo.py to the appropriate directory
for third-party modules in their Python installation.
This simple example demonstrates some fundamental concepts of the
Distutils: first, both developers and installers have the same basic
user interface, i.e. the setup script. The difference is which
Distutils commands they use: the `sdist` command is
almost exclusively for module developers, while `install` is
more often for installers (although most developers will want to install
their own code occasionally).
If you want to make things really easy for your users, you can create
one or more built distributions for them. For instance, if you are
running on a Windows machine, and want to make things easy for other
Windows users, you can create an executable installer (the most
appropriate type of built distribution for this platform) with the
`bdist_wininst` command. For example:
```text

python setup.py bdist_wininst
```
will create an executable installer, Foo-1.0.win32.exe, in the
current directory.
** not implemented yet **
(Another way to create executable installers for Windows is with the
`bdist_wise` command, which uses Wise--the commercial
installer-generator used to create Python's own installer--to create
the installer. Wise-based installers are more appropriate for large,
industrial-strength applications that need the full capabilities of a
``real'' installer. `bdist_wininst` creates a self-extracting
zip file with a minimal user interface, which is enough for small- to
medium-sized module collections. You'll need to have version XXX of
Wise installed on your system for the `bdist_wise` command to
work; it's available from http://foo/bar/baz.)
Currently (Distutils 0.9.1), the are only other useful built
distribution format is RPM, implemented by the `bdist_rpm`
command. For example, the following command will create an RPM file
called Foo-1.0.noarch.rpm:
```text

python setup.py bdist_rpm
```
(This uses the `rpm` command, so has to be run on an RPM-based
system such as Red Hat Linux, SuSE Linux, or Mandrake Linux.)
You can find out what distribution formats are available at any time by
running
```text

python setup.py bdist --help-formats
```


7.3 Single extension module

pure-pkg.html | examples.html | multiple-ext.html | Distributing Python Modules | contents.html
---
## 7.3 Single extension module


5 Creating a Source Distribution

setup-config.html | dist.html | manifest.html | Distributing Python Modules | contents.html
---
# 5 Creating a Source Distribution
As shown in section 2.1 (simple-example.html#simple-example), you use the
`sdist` command to create a source distribution. In the
simplest case,
```text

python setup.py sdist
```
(assuming you haven't specified any `sdist` options in the setup
script or config file), `sdist` creates the archive of the
default format for the current platform. The default formats are:
You can specify as many formats as you like using the
--formats option, for example:
```text

python setup.py sdist --formats=gztar,zip
```
to create a gzipped tarball and a zip file. The available formats are:
Notes:
(1): default on Windows
(2): default on Unix


About this document ...

discussion.html | doc.html | Documenting Python | contents.html
---
# About this document ...
Documenting Python,
September 18, 2000, Release 1.6
This document was generated using the LaTeX2HTML (http://saftsack.fs.uni-bayreuth.de/~latex2ht/) translator.
LaTeX2HTML (http://saftsack.fs.uni-bayreuth.de/~latex2ht/) is Copyright ©
1993, 1994, 1995, 1996, 1997, Nikos
Drakos (http://cbl.leeds.ac.uk/nikos/personal.html), Computer Based Learning Unit, University of
Leeds, and Copyright © 1997, 1998, Ross
Moore (http://www.maths.mq.edu.au/~ross/), Mathematics Department, Macquarie University,
Sydney.
The application of LaTeX2HTML (http://saftsack.fs.uni-bayreuth.de/~latex2ht/) to the Python
documentation has been heavily tailored by Fred L. Drake,
Jr. Original navigation icons were contributed by Christopher
Petrilli.
---
## Comments and Questions
General comments and questions regarding this document should
be sent by email to python-docs@python.org (mailto:python-docs@python.org). If you find specific errors in
this document, please report the bug at the Python Bug
Tracker (http://sourceforge.net/bugs/?group_id=5470) at SourceForge (http://sourceforge.net/).
Questions regarding how to use the information in this
document should be sent to the Python news group, comp.lang.python (news:comp.lang.python), or the Python mailing list (http://www.python.org/mailman/listinfo/python-list) (which is gated to the newsgroup and
carries the same content).
For any of these channels, please be sure not to send HTML email.
Thanks.
---
discussion.html | doc.html | Documenting Python | contents.html
---


4 Document Classes

latex-primer.html | doc.html | node6.html | Documenting Python | contents.html
---
# 4 Document Classes
Two LATEX document classes are defined specifically for use with
the Python documentation. The `manual` class is for large
documents which are sectioned into chapters, and the `howto` class is for smaller documents.
The `manual` documents are larger and are used for most of the
standard documents. This document class is based on the standard
LATEX `report` class and is formatted very much like a long
technical report. The Python Reference
Manual (../ref/ref.html) is a good example of a `manual` document, and the
Python Library Reference (../lib/lib.html) is a large
example.
The `howto` documents are shorter, and don't have the large
structure of the `manual` documents. This class is based on
the standard LATEX `article` class and is formatted somewhat
like the Linux Documentation Project's ``HOWTO'' series as done
originally using the LinuxDoc software. The original intent for the
document class was that it serve a similar role as the LDP's HOWTO
series, but the applicability of the class turns out to be somewhat
more broad. This class is used for ``how-to'' documents (this
document is an example) and for shorter reference manuals for small,
fairly cohesive module libraries. Examples of the later use include
the standard Macintosh Library Modules (../mac/mac.html)
and
Using
Kerberos from Python (http://starship.python.org/crew/fdrake/manuals/krb5py/krb5py.html), which contains reference material for an
extension package. These documents are roughly equivalent to a
single chapter from a larger work.


Contents

doc.html | doc.html | node2.html | Documenting Python
---
## Contents


8.2 Discussion Forums

structured.html | futures.html | about.html | Documenting Python | contents.html
---
## 8.2 Discussion Forums
Discussion of the future of the Python documentation and related
topics takes place in the Documentation Special Interest Group, or
``Doc-SIG.'' Information on the group, including mailing list
archives and subscription information, is available at
http://www.python.org/sigs/doc-sig/. The SIG is open to all
interested parties.
Comments and bug reports on the standard documents should be sent
to python-docs@python.org. This may include comments
about formatting, content, grammatical and spelling errors, or
this document. You can also send comments on this document
directly to the author at fdrake@acm.org.


Documenting Python

../index.html | contents.html | Documenting Python | contents.html
---
# Documenting Python
Fred L. Drake, Jr.
Corporation for National Research Initiatives (CNRI)
1895 Preston White Drive, Reston, Va 20191, USA
E-mail: fdrake@acm.org
September 18, 2000
Release 1.6
### Abstract:
The Python language documentation has a substantial body of
documentation, much of it contributed by various authors. The markup
used for the Python documentation is based on LATEX and requires a
significant set of macros written specifically for documenting Python.
This document describes the macros introduced to support Python
documentation and how they should be used to support a wide range of
output formats.
This document describes the document classes and special markup used
in the Python documentation. Authors may use this guide, in
conjunction with the template files provided with the
distribution, to create or maintain whole documents or sections.


8 Future Directions

node19.html | doc.html | structured.html | Documenting Python | contents.html
---
# 8 Future Directions
The history of the Python documentation is full of changes, most of
which have been fairly small and evolutionary. There has been a
great deal of discussion about making large changes in the markup
languages and tools used to process the documentation. This section
deals with the nature of the changes and what appears to be the most
likely path of future development.


Documenting Python


5.9 Index-generating Markup

references.html | node6.html | node16.html | Documenting Python | contents.html
---
## 5.9 Index-generating Markup
Effective index generation for technical documents can be very
difficult, especially for someone familiar with the topic but not
the creation of indexes. Much of the difficulty arises in the
area of terminology: including the terms an expert would use for a
concept is not sufficient. Coming up with the terms that a novice
would look up is fairly difficult for an author who, typically, is
an expert in the area she is writing on.
The truly difficult aspects of index generation are not areas with
which the documentation tools can help. However, ease
of producing the index once content decisions are made is within
the scope of the tools. Markup is provided which the processing
software is able to use to generate a variety of kinds of index
entry with minimal effort. Additionally, many of the environments
described in section 5.2 (info-units.html#info-units), ``Information Units,'' will
generate appropriate entries into the general and module indexes.
The following macro can be used to control the generation of index
data, and should be used in the document preamble:
There are a number of macros that are useful for adding index
entries for particular concepts, many of which are specific to
programming languages or even Python.
Additional macros are provided which are useful for conveniently
creating general index entries which should appear at many places
in the index by rotating a list of words. These are simple macros
that simply use \index to build some number of index
entries. Index entries build using these macros contain both
primary and secondary text.


5.2 Information Units

meta-info.html | node6.html | node9.html | Documenting Python | contents.html
---
## 5.2 Information Units
XXX Explain terminology, or come up with something more ``lay.''
There are a number of environments used to describe specific
features provided by modules. Each environment requires
parameters needed to provide basic information about what is being
described, and the environment content should be the description.
Most of these environments make entries in the general index (if
one is being produced for the document); if no index entry is
desired, non-indexing variants are available for many of these
environments. The environments have names of the form
`feature desc`, and the non-indexing variants are named
`feature descni`. The available variants are explicitly
included in the list below.
For each of these environments, the first parameter, name,
provides the name by which the feature is accessed.
Environments which describe features of objects within a module,
such as object methods or data attributes, allow an optional
type name parameter. When the feature is an attribute of
class instances, type name only needs to be given if the
class was not the most recently described class in the module; the
name value from the most recent \classdesc is implied.
For features of built-in or extension types, the type name
value should always be provided. Another special case includes
methods and members of general ``protocols,'' such as the
formatter and writer protocols described for the
formatter module: these may be documented without any
specific implementation classes, and will always require the
type name parameter to be provided.


3 LATEX Primer

node3.html | doc.html | classes.html | Documenting Python | contents.html
---
# 3 LATEX Primer
This section is a brief introduction to LATEX concepts and
syntax, to provide authors enough information to author documents
productively without having to become ``TEXnicians.''
Perhaps the most important concept to keep in mind while marking up
Python documentation is the while TEX is unstructured, LATEX was
designed as a layer on top of TEX which specifically supports
structured markup. The Python-specific markup is intended to extend
the structure provided by standard LATEX document classes to
support additional information specific to Python.
LATEX documents contain two parts: the preamble and the body.
The preamble is used to specify certain metadata about the document
itself, such as the title, the list of authors, the date, and the
class the document belongs to. Additional information used
to control index generation and the use of bibliographic databases
can also be placed in the preamble. For most authors, the preamble
can be most easily created by copying it from an existing document
and modifying a few key pieces of information.
The class of a document is used to place a document within a
broad category of documents and set some fundamental formatting
properties. For Python documentation, two classes are used: the
`manual` class and the `howto` class. These classes also
define the additional markup used to document Python concepts and
structures. Specific information about these classes is provided in
section 4 (classes.html#classes), ``Document Classes,'' below. The first thing
in the preamble is the declaration of the document's class.
After the class declaration, a number of macros are used to
provide further information about the document and setup any
additional markup that is needed. No output is generated from the
preamble; it is an error to include free text in the preamble
because it would cause output.
The document body follows the preamble. This contains all the
printed components of the document marked up structurally.
XXX This section will discuss what the markup looks like, and
explain the difference between an environment and a macro.


5.1 Meta-information Markup

node6.html | node6.html | info-units.html | Documenting Python | contents.html
---
## 5.1 Meta-information Markup


5.4 Inline Markup

node9.html | node6.html | node11.html | Documenting Python | contents.html
---
## 5.4 Inline Markup
The macros described in this section are used to mark just about
anything interesting in the document text. They may be used in
headings (though anything involving hyperlinks should be avoided
there) as well as in the body text.


5.5 Module-specific Markup

node10.html | node6.html | node12.html | Documenting Python | contents.html
---
## 5.5 Module-specific Markup
The markup described in this section is used to provide information
about a module being documented. A typical use of this markup
appears at the top of the section used to document a module. A
typical example might look like this:
```text

\section{\module{spam} ---
Access to the SPAM facility}

\declaremodule{extension}{spam}
\platform{Unix}
\modulesynopsis{Access to the SPAM facility of \UNIX{}.}
\moduleauthor{Jane Doe}{jane.doe@frobnitz.org}
```


5.6 Library-level Markup

node11.html | node6.html | node13.html | Documenting Python | contents.html
---
## 5.6 Library-level Markup
This markup is used when describing a selection of modules. For
example, the Macintosh Library
Modules (../mac/mac.html) document uses this to help provide an overview of the
modules in the collection, and many chapters in the
Python Library Reference (../lib/lib.html) use it for
the same purpose.


5.7 Table Markup

node12.html | node6.html | references.html | Documenting Python | contents.html
---
## 5.7 Table Markup
There are three general-purpose table environments defined which
should be used whenever possible. These environments are defined
to provide tables of specific widths and some convenience for
formatting. These environments are not meant to be general
replacements for the standard LATEX table environments, but can
be used for an advantage when the documents are processed using
the tools for Python documentation processing. In particular, the
generated HTML looks good! There is also an advantage for the
eventual conversion of the documentation to SGML (see section
8 (futures.html#futures), ``Future Directions'').
Each environment is named \tablecols, where cols
is the number of columns in the table specified in lower-case
Roman numerals. Within each of these environments, an additional
macro, \linecols, is defined, where cols
matches the cols value of the corresponding table
environment. These are supported for cols values of
`ii`, `iii`, and `iv`. These environments are all
built on top of the \tabular environment.
Note that all tables in the standard Python documentation use
vertical lines between columns, and this must be specified in the
markup for each table. A general border around the outside of the
table is not used, but would be the responsibility of the
processor.
An additional table-like environment is \synopsistable. The
table generated by this environment contains two columns, and each
row is defined by an alternate definition of
\modulesynopsis. This environment is not normally used by
authors, but is created by the \localmoduletable macro.


6 Special Names

indexing.html | doc.html | node17.html | Documenting Python | contents.html
---
# 6 Special Names
Many special names are used in the Python documentation, including
the names of operating systems, programming languages, standards
bodies, and the like. Many of these were assigned LATEX macros
at some point in the distant past, and these macros lived on long
past their usefulness. In the current markup, these entities are
not assigned any special markup, but the preferred spellings are
given here to aid authors in maintaining the consistency of
presentation in the Python documentation.
POSIX: The name assigned to a particular group of standards. This is
always uppercase.
Python: The name of our favorite programming language is always
capitalized.
Unicode: The name of a character set and matching encoding. This is
always written capitalized.


7 Processing Tools

node16.html | doc.html | node18.html | Documenting Python | contents.html
---
# 7 Processing Tools


7.1 External Tools

node17.html | node17.html | node19.html | Documenting Python | contents.html
---
## 7.1 External Tools
Many tools are needed to be able to process the Python
documentation if all supported formats are required. This
section lists the tools used and when each is required. Consult
the Doc/README file to see if there are specific version
requirements for any of these.
dvips: This program is a typical part of TEX installations. It is
used to generate PostScript from the ``device independent''
.dvi files. It is needed for the conversion to
PostScript.
emacs: Emacs is the kitchen sink of programmers' editors, and a damn
fine kitchen sink it is. It also comes with some of the
processing needed to support the proper menu structures for
Texinfo documents when an info conversion is desired. This is
needed for the info conversion. Using xemacs
instead of FSF emacs may lead to instability in the
conversion, but that's because nobody seems to maintain the
Emacs Texinfo code in a portable manner.
latex: This is a world-class typesetter by Donald Knuth. It is used
for the conversion to PostScript, and is needed for the HTML
conversion as well (LATEX2HTML requires one of the
intermediate files it creates).
latex2html: Probably the longest Perl script anyone ever attempted to
maintain. This converts LATEX documents to HTML documents,
and does a pretty reasonable job. It is required for the
conversions to HTML and GNU info.
lynx: This is a text-mode Web browser which includes an
HTML-to-plain text conversion. This is used to convert
`howto` documents to text.
make: Just about any version should work for the standard documents,
but GNU make is required for the experimental
processes in Doc/tools/sgmlconv/, at least while
they're experimental.
makeindex: This is a standard program for converting LATEX index data
to a formatted index; it should be included with all LATEX
installations. It is needed for the PDF and PostScript
conversions.
makeinfo: GNU makeinfo is used to convert Texinfo documents to
GNU info files. Since Texinfo is used as an intermediate
format in the info conversion, this program is needed in that
conversion.
pdflatex: pdfTEX is a relatively new variant of TEX, and is used to
generate the PDF version of the manuals. It is typically
installed as part of most of the large TEX distributions.
pdflatex is pdfTEX using the LATEX format.
perl: Perl is required for LATEX2HTML and one of the scripts used
to post-process LATEX2HTML output, as well as the
HTML-to-Texinfo conversion. This is required for
the HTML and GNU info conversions.
python: Python is used for many of the scripts in the
Doc/tools/ directory; it is required for all
conversions. This shouldn't be a problem if you're interested
in writing documentation for Python!


7.2 Internal Tools

node18.html | node17.html | futures.html | Documenting Python | contents.html
---
## 7.2 Internal Tools
This section describes the various scripts that are used to
implement various stages of document processing or to orchestrate
entire build sequences. Most of these tools are only useful
in the context of building the standard documentation, but some
are more general.
mkhowto: This is the primary script used to format third-party
documents. It contains all the logic needed to ``get it
right.'' The proper way to use this script is to make a
symbolic link to it or run it in place; the actual script file
must be stored as part of the documentation source tree,
though it may be used to format documents outside the
tree. Use mkhowto --help
for a list of
command line options.
mkhowto can be used for both `howto` and
`manual` class documents. (For the later, be sure to get
the latest version from the Python CVS repository rather than
the version distributed in the latex-1.5.2.tgz source
archive.)
XXX Need more here.


1 Introduction

contents.html | doc.html | node3.html | Documenting Python | contents.html
---
# 1 Introduction
Python's documentation has long been considered to be good for a
free programming language. There are a number of reasons for this,
the most important being the early commitment of Python's creator,
Guido van Rossum, to providing documentation on the language and its
libraries, and the continuing involvement of the user community in
providing assistance for creating and maintaining documentation.
The involvement of the community takes many forms, from authoring to
bug reports to just plain complaining when the documentation could
be more complete or easier to use. All of these forms of input from
the community have proved useful during the time I've been involved
in maintaining the documentation.
This document is aimed at authors and potential authors of
documentation for Python. More specifically, it is for people
contributing to the standard documentation and developing additional
documents using the same tools as the standard documents. This
guide will be less useful for authors using the Python documentation
tools for topics other than Python, and less useful still for
authors not using the tools at all.
The material in this guide is intended to assist authors using the
Python documentation tools. It includes information on the source
distribution of the standard documentation, a discussion of the
document types, reference material on the markup defined in the
document classes, a list of the external tools needed for processing
documents, and reference material on the tools provided with the
documentation resources. At the end, there is also a section
discussing future directions for the Python documentation and where
to turn for more information.


2 Directory Structure

node2.html | doc.html | latex-primer.html | Documenting Python | contents.html
---
# 2 Directory Structure
The source distribution for the standard Python documentation
contains a large number of directories. While third-party documents
do not need to be placed into this structure or need to be placed
within a similar structure, it can be helpful to know where to look
for examples and tools when developing new documents using the
Python documentation tools. This section describes this directory
structure.
The documentation sources are usually placed within the Python
source distribution as the top-level directory Doc/, but
are not dependent on the Python source distribution in any way.
The Doc/ directory contains a few files and several
subdirectories. The files are mostly self-explanatory, including a
README and a Makefile. The directories fall into
three categories:


5 Special Markup Constructs

classes.html | doc.html | meta-info.html | Documenting Python | contents.html
---
# 5 Special Markup Constructs
The Python document classes define a lot of new environments and
macros. This section contains the reference material for these
facilities.


5.3 Showing Code Examples

info-units.html | node6.html | node10.html | Documenting Python | contents.html
---
## 5.3 Showing Code Examples
Examples of Python source code or interactive sessions are
represented as \verbatim environments. This environment
is a standard part of LATEX. It is important to only use
spaces for indentation in code examples since TEX drops tabs
instead of converting them to spaces.
Representing an interactive session requires including the prompts
and output along with the Python code. No special markup is
required for interactive sessions.
Within the \verbatim environment, characters special to
LATEX do not need to be specially marked in any way. The entire
example will be presented in a monospaced font; no attempt at
``pretty-printing'' is made, as the environment must work for
non-Python code and non-code displays.
The Python Documentation Special Interest Group has discussed a
number of approaches to creating pretty-printed code displays and
interactive sessions; see the Doc-SIG area on the Python Web site
for more information on this topic.


5.8 Reference List Markup

node13.html | node6.html | indexing.html | Documenting Python | contents.html
---
## 5.8 Reference List Markup
Many sections include a list of references to module documentation
or external documents. These lists are created using the
\seealso environment. This environment defines some
additional macros to support creating reference entries in a
reasonable manner.


8.1 Structured Documentation

futures.html | futures.html | discussion.html | Documenting Python | contents.html
---
## 8.1 Structured Documentation
Most of the small changes to the LATEX markup have been made
with an eye to divorcing the markup from the presentation, making
both a bit more maintainable. Over the course of 1998, a large
number of changes were made with exactly this in mind; previously,
changes had been made but in a less systematic manner and with
more concern for not needing to update the existing content. The
result has been a highly structured and semantically loaded markup
language implemented in LATEX. With almost no basic TEX or
LATEX markup in use, however, the markup syntax is about the
only evidence of LATEX in the actual document sources.
One side effect of this is that while we've been able to use
standard ``engines'' for manipulating the documents, such as
LATEX and LATEX2HTML, most of the actual transformations have
been created specifically for Python. The LATEX document
classes and LATEX2HTML support are both complete implementations
of the specific markup designed for these documents.
Combining highly customized markup with the somewhat esoteric
systems used to process the documents leads us to ask some
questions: Can we do this more easily? and, Can we do this
better? After a great deal of discussion with the community, we
have determined that actively pursuing modern structured
documentation systems is worth some investment of time.
There appear to be two real contenders in this arena: the Standard
General Markup Language (SGML), and the Extensible Markup Language
(XML). Both of these standards have advantages and disadvantages,
and many advantages are shared.
SGML offers advantages which may appeal most to authors,
especially those using ordinary text editors. There are also
additional abilities to define content models. A number of
high-quality tools with demonstrated maturity is available, but
most are not free; for those which are, portability issues remain
a problem.
The advantages of XML include the availability of a large number
of evolving tools. Unfortunately, many of the associated
standards are still evolving, and the tools will have to follow
along. This means that developing a robust tool set that uses
more than the basic XML 1.0 recommendation is not possible in the
short term. The promised availability of a wide variety of
high-quality tools which support some of the most important
related standards is not immediate. Many tools are likely to be
free.
XXX Eventual migration to SGML/XML.


About this document ...

embeddingInCplusplus.html | ext.html | Extending and Embedding the Python Interpreter | contents.html
---
# About this document ...
Extending and Embedding the Python Interpreter,
September 18, 2000, Release 1.6
This document was generated using the LaTeX2HTML (http://saftsack.fs.uni-bayreuth.de/~latex2ht/) translator.
LaTeX2HTML (http://saftsack.fs.uni-bayreuth.de/~latex2ht/) is Copyright ©
1993, 1994, 1995, 1996, 1997, Nikos
Drakos (http://cbl.leeds.ac.uk/nikos/personal.html), Computer Based Learning Unit, University of
Leeds, and Copyright © 1997, 1998, Ross
Moore (http://www.maths.mq.edu.au/~ross/), Mathematics Department, Macquarie University,
Sydney.
The application of LaTeX2HTML (http://saftsack.fs.uni-bayreuth.de/~latex2ht/) to the Python
documentation has been heavily tailored by Fred L. Drake,
Jr. Original navigation icons were contributed by Christopher
Petrilli.
---
## Comments and Questions
General comments and questions regarding this document should
be sent by email to python-docs@python.org (mailto:python-docs@python.org). If you find specific errors in
this document, please report the bug at the Python Bug
Tracker (http://sourceforge.net/bugs/?group_id=5470) at SourceForge (http://sourceforge.net/).
Questions regarding how to use the information in this
document should be sent to the Python news group, comp.lang.python (news:comp.lang.python), or the Python mailing list (http://www.python.org/mailman/listinfo/python-list) (which is gated to the newsgroup and
carries the same content).
For any of these channels, please be sure not to send HTML email.
Thanks.
---
embeddingInCplusplus.html | ext.html | Extending and Embedding the Python Interpreter | contents.html
---


1.3 Back to the Example

errors.html | intro.html | methodTable.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.3 Back to the Example
Going back to our example function, you should now be able to
understand this statement:
```text

if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
```
It returns NULL (the error indicator for functions returning
object pointers) if an error is detected in the argument list, relying
on the exception set by PyArg_ParseTuple(). Otherwise the
string value of the argument has been copied to the local variable
command. This is a pointer assignment and you are not supposed
to modify the string to which it points (so in Standard C, the variable
command should properly be declared as "const char
*command").
The next statement is a call to the Unix function
system(), passing it the string we just got from
PyArg_ParseTuple():
```text

sts = system(command);
```
Our spam.system() function must return the value of
sts as a Python object. This is done using the function
Py_BuildValue(), which is something like the inverse of
PyArg_ParseTuple(): it takes a format string and an
arbitrary number of C values, and returns a new Python object.
More info on Py_BuildValue() is given later.
```text

return Py_BuildValue("i", sts);
```
In this case, it will return an integer object. (Yes, even integers
are objects on the heap in Python!)
If you have a C function that returns no useful argument (a function
returning void), the corresponding Python function must return
`None`. You need this idiom to do so:
```text

Py_INCREF(Py_None);
return Py_None;
```
Py_None is the C name for the special Python object
`None`. It is a genuine Python object rather than a NULL
pointer, which means ``error'' in most contexts, as we have seen.


using-cobjects.html | ext.html | custom-interps.html | Extending and Embedding the Python Interpreter | contents.html
---
# 2. Building C and C++ Extensions on Unix
Starting in Python 1.4, Python provides a special make file for
building make files for building dynamically-linked extensions and
custom interpreters. The make file make file builds a make file
that reflects various system variables determined by configure when
the Python interpreter was built, so people building module's don't
have to resupply these settings. This vastly simplifies the process
of building extensions and custom interpreters on Unix systems.
The make file make file is distributed as the file
Misc/Makefile.pre.in in the Python source distribution. The
first step in building extensions or custom interpreters is to copy
this make file to a development directory containing extension module
source.
The make file make file, Makefile.pre.in uses metadata
provided in a file named Setup. The format of the Setup
file is the same as the Setup (or Setup.in) file
provided in the Modules/ directory of the Python source
distribution. The Setup file contains variable definitions:
```text

EC=/projects/ExtensionClass
```
and module description lines. It can also contain blank lines and
comment lines that start with "#".
A module description line includes a module name, source files,
options, variable references, and other input files, such
as libraries or object files. Consider a simple example:
```text

ExtensionClass ExtensionClass.c
```
This is the simplest form of a module definition line. It defines a
module, ExtensionClass, which has a single source file,
ExtensionClass.c.
This slightly more complex example uses an -I option to
specify an include directory:
```text

EC=/projects/ExtensionClass
cPersistence cPersistence.c -I$(EC)
```
This example also illustrates the format for variable references.
For systems that support dynamic linking, the Setup file should
begin:
```text

*shared*
```
to indicate that the modules defined in Setup are to be built
as dynamically linked modules. A line containing only "*static*"can be used to indicate the subsequently listed modules should be
statically linked.
Here is a complete Setup file for building a
cPersistent module:
```text

# Set-up file to build the cPersistence module.
# Note that the text should begin in the first column.
*shared*

# We need the path to the directory containing the ExtensionClass
# include file.
EC=/projects/ExtensionClass
cPersistence cPersistence.c -I$(EC)
```
After the Setup file has been created, Makefile.pre.in
is run with the "boot" target to create a make file:
```text

make -f Makefile.pre.in boot
```
This creates the file, Makefile. To build the extensions, simply
run the created make file:
```text

make
```
It's not necessary to re-run Makefile.pre.in if the
Setup file is changed. The make file automatically rebuilds
itself if the Setup file changes.


distributing.html | ext.html | win-cookbook.html | Extending and Embedding the Python Interpreter | contents.html
---
# 3. Building C and C++ Extensions on Windows
This chapter briefly explains how to create a Windows extension module
for Python using Microsoft Visual C++, and follows with more
detailed background information on how it works. The explanatory
material is useful for both the Windows programmer learning to build
Python extensions and the Unix programmer interested in producing
software which can be successfully built on both Unix and Windows.


1.9 The Py_BuildValue() Function

parseTupleAndKeywords.html | intro.html | refcounts.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.9 The Py_BuildValue() Function
This function is the counterpart to PyArg_ParseTuple(). It is
declared as follows:
```text

PyObject *Py_BuildValue(char *format, ...);
```
It recognizes a set of format units similar to the ones recognized by
PyArg_ParseTuple(), but the arguments (which are input to the
function, not output) must not be pointers, just values. It returns a
new Python object, suitable for returning from a C function called
from Python.
One difference with PyArg_ParseTuple(): while the latter
requires its first argument to be a tuple (since Python argument lists
are always represented as tuples internally),
Py_BuildValue() does not always build a tuple. It builds
a tuple only if its format string contains two or more format units.
If the format string is empty, it returns `None`; if it contains
exactly one format unit, it returns whatever object is described by
that format unit. To force it to return a tuple of size 0 or one,
parenthesize the format string.
In the following description, the quoted form is the format unit; the
entry in (round) parentheses is the Python object type that the format
unit will return; and the entry in [square] brackets is the type of
the C value(s) to be passed.
The characters space, tab, colon and comma are ignored in format
strings (but not within format units such as "s#"). This can be
used to make long format strings a tad more readable.
"s" (string) [char *]: Convert a null-terminated C string to a Python object. If the C
string pointer is NULL, `None` is returned.
"s#" (string) [char *, int]: Convert a C string and its length to a Python object. If the C string
pointer is NULL, the length is ignored and `None` is
returned.
"z" (string or `None`) [char *]: Same as "s".
"z#" (string or `None`) [char *, int]: Same as "s#".
"u" (Unicode string) [Py_UNICODE *]: Convert a null-terminated buffer of Unicode (UCS-2) data to a Python
Unicode object. If the Unicode buffer pointer is NULL,
`None` is returned.
"u#" (Unicode string) [Py_UNICODE *, int]: Convert a Unicode (UCS-2) data buffer and its length to a Python
Unicode object. If the Unicode buffer pointer is NULL, the length
is ignored and `None` is returned.
"u" (Unicode string) [Py_UNICODE *]: Convert a null-terminated buffer of Unicode (UCS-2) data to a Python Unicode
object. If the Unicode buffer pointer is NULL, `None` is returned.
"u#" (Unicode string) [Py_UNICODE *, int]: Convert a Unicode (UCS-2) data buffer and its length to a Python Unicode
object. If the Unicode buffer pointer is NULL, the length is ignored and
`None` is returned.
"i" (integer) [int]: Convert a plain C int to a Python integer object.
"b" (integer) [char]: Same as "i".
"h" (integer) [short int]: Same as "i".
"l" (integer) [long int]: Convert a C long int to a Python integer object.
"c" (string of length 1) [char]: Convert a C int representing a character to a Python string of
length 1.
"d" (float) [double]: Convert a C double to a Python floating point number.
"f" (float) [float]: Same as "d".
"O" (object) [PyObject *]: Pass a Python object untouched (except for its reference count, which
is incremented by one). If the object passed in is a NULL
pointer, it is assumed that this was caused because the call producing
the argument found an error and set an exception. Therefore,
Py_BuildValue() will return NULL but won't raise an
exception. If no exception has been raised yet,
PyExc_SystemError is set.
"S" (object) [PyObject *]: Same as "O".
"U" (object) [PyObject *]: Same as "O".
"N" (object) [PyObject *]: Same as "O", except it doesn't increment the reference count on
the object. Useful when the object is created by a call to an object
constructor in the argument list.
"O&" (object) [converter, anything]: Convert anything to a Python object through a converter
function. The function is called with anything (which should be
compatible with void *) as its argument and should return a
``new'' Python object, or NULL if an error occurred.
"(items)" (tuple) [matching-items]: Convert a sequence of C values to a Python tuple with the same number
of items.
"[items]" (list) [matching-items]: Convert a sequence of C values to a Python list with the same number
of items.
"{items}" (dictionary) [matching-items]: Convert a sequence of C values to a Python dictionary. Each pair of
consecutive C values adds one item to the dictionary, serving as key
and value, respectively.
If there is an error in the format string, the
PyExc_SystemError exception is raised and NULL returned.
Examples (to the left the call, to the right the resulting Python value):
```text

Py_BuildValue("") None
Py_BuildValue("i", 123) 123
Py_BuildValue("iii", 123, 456, 789) (123, 456, 789)
Py_BuildValue("s", "hello") 'hello'
Py_BuildValue("ss", "hello", "world") ('hello', 'world')
Py_BuildValue("s#", "hello", 4) 'hell'
Py_BuildValue("()") ()
Py_BuildValue("(i)", 123) (123,)
Py_BuildValue("(ii)", 123, 456) (123, 456)
Py_BuildValue("(i,i)", 123, 456) (123, 456)
Py_BuildValue("[i,i]", 123, 456) [123, 456]
Py_BuildValue("{s:i,s:i}",
"abc", 123, "def", 456) {'abc': 123, 'def': 456}
Py_BuildValue("((ii)(ii)) (ii)",
1, 2, 3, 4, 5, 6) (((1, 2), (3, 4)), (5, 6))
```


1.6 Calling Python Functions from C

compilation.html | intro.html | parseTuple.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.6 Calling Python Functions from C
So far we have concentrated on making C functions callable from
Python. The reverse is also useful: calling Python functions from C.
This is especially the case for libraries that support so-called
``callback'' functions. If a C interface makes use of callbacks, the
equivalent Python often needs to provide a callback mechanism to the
Python programmer; the implementation will require calling the Python
callback functions from a C callback. Other uses are also imaginable.
Fortunately, the Python interpreter is easily called recursively, and
there is a standard interface to call a Python function. (I won't
dwell on how to call the Python parser with a particular string as
input -- if you're interested, have a look at the implementation of
the -c command line option in Python/pythonmain.c
from the Python source code.)
Calling a Python function is easy. First, the Python program must
somehow pass you the Python function object. You should provide a
function (or some other interface) to do this. When this function is
called, save a pointer to the Python function object (be careful to
Py_INCREF() it!) in a global variable -- or wherever you
see fit. For example, the following function might be part of a module
definition:
```text

static PyObject *my_callback = NULL;

static PyObject *
my_set_callback(dummy, args)
PyObject *dummy, *args;
{
PyObject *result = NULL;
PyObject *temp;

if (PyArg_ParseTuple(args, "O:set_callback", &temp)) {
if (!PyCallable_Check(temp)) {
PyErr_SetString(PyExc_TypeError, "parameter must be callable");
return NULL;
}
Py_XINCREF(temp); /* Add a reference to new callback */
Py_XDECREF(my_callback); /* Dispose of previous callback */
my_callback = temp; /* Remember new callback */
/* Boilerplate to return "None" */
Py_INCREF(Py_None);
result = Py_None;
}
return result;
}
```
This function must be registered with the interpreter using the
METH_VARARGS flag; this is described in section
1.4 (methodTable.html#methodTable), ``The Module's Method Table and Initialization
Function.'' The PyArg_ParseTuple() function and its
arguments are documented in section 1.7 (parseTuple.html#parseTuple), ``Format Strings
for PyArg_ParseTuple().''
The macros Py_XINCREF() and Py_XDECREF()
increment/decrement the reference count of an object and are safe in
the presence of NULL pointers (but note that temp will not be
NULL in this context). More info on them in section
1.10 (refcounts.html#refcounts), ``Reference Counts.''
Later, when it is time to call the function, you call the C function
PyEval_CallObject(). This function has two arguments, both
pointers to arbitrary Python objects: the Python function, and the
argument list. The argument list must always be a tuple object, whose
length is the number of arguments. To call the Python function with
no arguments, pass an empty tuple; to call it with one argument, pass
a singleton tuple. Py_BuildValue() returns a tuple when its
format string consists of zero or more format codes between
parentheses. For example:
```text

int arg;
PyObject *arglist;
PyObject *result;
...
arg = 123;
...
/* Time to call the callback */
arglist = Py_BuildValue("(i)", arg);
result = PyEval_CallObject(my_callback, arglist);
Py_DECREF(arglist);
```
PyEval_CallObject() returns a Python object pointer: this is
the return value of the Python function. PyEval_CallObject() is
``reference-count-neutral'' with respect to its arguments. In the
example a new tuple was created to serve as the argument list, which
is Py_DECREF()-ed immediately after the call.
The return value of PyEval_CallObject() is ``new'': either it
is a brand new object, or it is an existing object whose reference
count has been incremented. So, unless you want to save it in a
global variable, you should somehow Py_DECREF() the result,
even (especially!) if you are not interested in its value.
Before you do this, however, it is important to check that the return
value isn't NULL. If it is, the Python function terminated by
raising an exception. If the C code that called
PyEval_CallObject() is called from Python, it should now
return an error indication to its Python caller, so the interpreter
can print a stack trace, or the calling Python code can handle the
exception. If this is not possible or desirable, the exception should
be cleared by calling PyErr_Clear(). For example:
```text

if (result == NULL)
return NULL; /* Pass error back */
...use result...
Py_DECREF(result);
```
Depending on the desired interface to the Python callback function,
you may also have to provide an argument list to
PyEval_CallObject(). In some cases the argument list is
also provided by the Python program, through the same interface that
specified the callback function. It can then be saved and used in the
same manner as the function object. In other cases, you may have to
construct a new tuple to pass as the argument list. The simplest way
to do this is to call Py_BuildValue(). For example, if
you want to pass an integral event code, you might use the following
code:
```text

PyObject *arglist;
...
arglist = Py_BuildValue("(l)", eventcode);
result = PyEval_CallObject(my_callback, arglist);
Py_DECREF(arglist);
if (result == NULL)
return NULL; /* Pass error back */
/* Here maybe use the result */
Py_DECREF(result);
```
Note the placement of "Py_DECREF(arglist)" immediately after the
call, before the error check! Also note that strictly spoken this
code is not complete: Py_BuildValue() may run out of
memory, and this should be checked.


1.5 Compilation and Linkage

methodTable.html | intro.html | callingPython.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.5 Compilation and Linkage
There are two more things to do before you can use your new extension:
compiling and linking it with the Python system. If you use dynamic
loading, the details depend on the style of dynamic loading your
system uses; see the chapters about building extension modules on
Unix (chapter 2 (building-on-unix.html#building-on-unix)) and Windows (chapter
3 (building-on-windows.html#building-on-windows)) for more information about this.
If you can't use dynamic loading, or if you want to make your module a
permanent part of the Python interpreter, you will have to change the
configuration setup and rebuild the interpreter. Luckily, this is
very simple: just place your file (spammodule.c for example) in
the Modules/ directory of an unpacked source distribution, add
a line to the file Modules/Setup.local describing your file:
```text

spam spammodule.o
```
and rebuild the interpreter by running make in the toplevel
directory. You can also run make in the Modules/
subdirectory, but then you must first rebuild Makefile
there by running `make Makefile'. (This is necessary each
time you change the Setup file.)
If your module requires additional libraries to link with, these can
be listed on the line in the configuration file as well, for instance:
```text

spam spammodule.o -lX11
```


Contents

front.html | ext.html | intro.html | Extending and Embedding the Python Interpreter
---
## Contents


1.11 Writing Extensions in C++

nullPointers.html | intro.html | using-cobjects.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.11 Writing Extensions in C++
It is possible to write extension modules in C++. Some restrictions
apply. If the main program (the Python interpreter) is compiled and
linked by the C compiler, global or static objects with constructors
cannot be used. This is not a problem if the main program is linked
by the C++ compiler. Functions that will be called by the
Python interpreter (in particular, module initalization functions)
have to be declared using `extern "C"`.
It is unnecessary to enclose the Python header files in
`extern "C" {...}` -- they use this form already if the symbol
"__cplusplus" is defined (all recent C++ compilers define this
symbol).


2.1 Building Custom Interpreters

building-on-unix.html | building-on-unix.html | module-defn-options.html | Extending and Embedding the Python Interpreter | contents.html
---
# 2.1 Building Custom Interpreters
The make file built by Makefile.pre.in can be run with the
"static" target to build an interpreter:
```text

make static
```
Any modules defined in the Setup file before the "*shared*" line
will be statically linked into the interpreter. Typically, a
"*shared*" line is omitted from the Setup file when a custom
interpreter is desired.


2.4 Distributing your extension modules

module-defn-example.html | building-on-unix.html | building-on-windows.html | Extending and Embedding the Python Interpreter | contents.html
---
# 2.4 Distributing your extension modules
When distributing your extension modules in source form, make sure to
include a Setup file. The Setup file should be named
Setup.in in the distribution. The make file make file,
Makefile.pre.in, will copy Setup.in to Setup.
Distributing a Setup.in file makes it easy for people to
customize the Setup file while keeping the original in
Setup.in.
It is a good idea to include a copy of Makefile.pre.in for
people who do not have a source distribution of Python.
Do not distribute a make file. People building your modules
should use Makefile.pre.in to build their own make file. A
README file included in the package should provide simple
instructions to perform the build.
Work is being done to make building and installing Python extensions
easier for all platforms; this work in likely to supplant the current
approach at some point in the future. For more information or to
participate in the effort, refer to
http://www.python.org/sigs/distutils-sig/ on the Python Web
site.


win-cookbook.html | building-on-windows.html | win-dlls.html | Extending and Embedding the Python Interpreter | contents.html
---
# 3.2 Differences Between Unix and Windows
Unix and Windows use completely different paradigms for run-time
loading of code. Before you try to build a module that can be
dynamically loaded, be aware of how your system works.
In Unix, a shared object (.so) file contains code to be used by the
program, and also the names of functions and data that it expects to
find in the program. When the file is joined to the program, all
references to those functions and data in the file's code are changed
to point to the actual locations in the program where the functions
and data are placed in memory. This is basically a link operation.
In Windows, a dynamic-link library (.dll) file has no dangling
references. Instead, an access to functions or data goes through a
lookup table. So the DLL code does not have to be fixed up at runtime
to refer to the program's memory; instead, the code already uses the
DLL's lookup table, and the lookup table is modified at runtime to
point to the functions and data.
In Unix, there is only one type of library file (.a) which
contains code from several object files (.o). During the link
step to create a shared object file (.so), the linker may find
that it doesn't know where an identifier is defined. The linker will
look for it in the object files in the libraries; if it finds it, it
will include all the code from that object file.
In Windows, there are two types of library, a static library and an
import library (both called .lib). A static library is like a
Unix .a file; it contains code to be included as necessary.
An import library is basically used only to reassure the linker that a
certain identifier is legal, and will be present in the program when
the DLL is loaded. So the linker uses the information from the
import library to build the lookup table for using identifiers that
are not included in the DLL. When an application or a DLL is linked,
an import library may be generated, which will need to be used for all
future DLLs that depend on the symbols in the application or DLL.
Suppose you are building two dynamic-load modules, B and C, which should
share another block of code A. On Unix, you would not pass
A.a to the linker for B.so and C.so; that would
cause it to be included twice, so that B and C would each have their
own copy. In Windows, building A.dll will also build
A.lib. You do pass A.lib to the linker for B and
C. A.lib does not contain code; it just contains information
which will be used at runtime to access A's code.
In Windows, using an import library is sort of like using "import
spam"; it gives you access to spam's names, but does not create a
separate copy. On Unix, linking with a library is more like
"from spam import *"; it does create a separate copy.


win-dlls.html | ext.html | embeddingInCplusplus.html | Extending and Embedding the Python Interpreter | contents.html
---
# 4. Embedding Python in Another Application
Embedding Python is similar to extending it, but not quite. The
difference is that when you extend Python, the main program of the
application is still the Python interpreter, while if you embed
Python, the main program may have nothing to do with Python --
instead, some parts of the application occasionally call the Python
interpreter to run some Python code.
So if you are embedding Python, you are providing your own main
program. One of the things this main program has to do is initialize
the Python interpreter. At the very least, you have to call the
function Py_Initialize() (on MacOS, call
PyMac_Initialize() instead). There are optional calls to
pass command line arguments to Python. Then later you can call the
interpreter from any part of the application.
There are several different ways to call the interpreter: you can pass
a string containing Python statements to
PyRun_SimpleString(), or you can pass a stdio file pointer
and a file name (for identification in error messages only) to
PyRun_SimpleFile(). You can also call the lower-level
operations described in the previous chapters to construct and use
Python objects.
A simple demo of embedding Python can be found in the directory
Demo/embed/ of the source distribution.


4.1 Embedding Python in C++

embedding.html | embedding.html | about.html | Extending and Embedding the Python Interpreter | contents.html
---
# 4.1 Embedding Python in C++
It is also possible to embed Python in a C++ program; precisely how this
is done will depend on the details of the C++ system used; in general you
will need to write the main program in C++, and use the C++ compiler
to compile and link your program. There is no need to recompile Python
itself using C++.


1.2 Intermezzo: Errors and Exceptions

simpleExample.html | intro.html | backToExample.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.2 Intermezzo: Errors and Exceptions
An important convention throughout the Python interpreter is the
following: when a function fails, it should set an exception condition
and return an error value (usually a NULL pointer). Exceptions
are stored in a static global variable inside the interpreter; if this
variable is NULL no exception has occurred. A second global
variable stores the ``associated value'' of the exception (the second
argument to raise). A third variable contains the stack
traceback in case the error originated in Python code. These three
variables are the C equivalents of the Python variables
`sys.exc_type`, `sys.exc_value` and `sys.exc_traceback` (see
the section on module sys in the
Python Library Reference (../lib/lib.html)). It is
important to know about them to understand how errors are passed
around.
The Python API defines a number of functions to set various types of
exceptions.
The most common one is PyErr_SetString(). Its arguments
are an exception object and a C string. The exception object is
usually a predefined object like PyExc_ZeroDivisionError. The
C string indicates the cause of the error and is converted to a
Python string object and stored as the ``associated value'' of the
exception.
Another useful function is PyErr_SetFromErrno(), which only
takes an exception argument and constructs the associated value by
inspection of the global variable errno. The most
general function is PyErr_SetObject(), which takes two object
arguments, the exception and its associated value. You don't need to
Py_INCREF() the objects passed to any of these functions.
You can test non-destructively whether an exception has been set with
PyErr_Occurred(). This returns the current exception object,
or NULL if no exception has occurred. You normally don't need
to call PyErr_Occurred() to see whether an error occurred in a
function call, since you should be able to tell from the return value.
When a function f that calls another function g detects
that the latter fails, f should itself return an error value
(e.g. NULL or `-1`). It should not call one of the
PyErr_*() functions -- one has already been called by g.
f's caller is then supposed to also return an error indication
to its caller, again without calling PyErr_*(),
and so on -- the most detailed cause of the error was already
reported by the function that first detected it. Once the error
reaches the Python interpreter's main loop, this aborts the currently
executing Python code and tries to find an exception handler specified
by the Python programmer.
(There are situations where a module can actually give a more detailed
error message by calling another PyErr_*() function, and in
such cases it is fine to do so. As a general rule, however, this is
not necessary, and can cause information about the cause of the error
to be lost: most operations can fail for a variety of reasons.)
To ignore an exception set by a function call that failed, the exception
condition must be cleared explicitly by calling PyErr_Clear().
The only time C code should call PyErr_Clear() is if it doesn't
want to pass the error on to the interpreter but wants to handle it
completely by itself (e.g. by trying something else or pretending
nothing happened).
Every failing malloc() call must be turned into an
exception -- the direct caller of malloc() (or
realloc()) must call PyErr_NoMemory() and
return a failure indicator itself. All the object-creating functions
(for example, PyInt_FromLong()) already do this, so this
note is only relevant to those who call malloc() directly.
Also note that, with the important exception of
PyArg_ParseTuple() and friends, functions that return an
integer status usually return a positive value or zero for success and
`-1` for failure, like Unix system calls.
Finally, be careful to clean up garbage (by making
Py_XDECREF() or Py_DECREF() calls for objects
you have already created) when you return an error indicator!
The choice of which exception to raise is entirely yours. There are
predeclared C objects corresponding to all built-in Python exceptions,
e.g. PyExc_ZeroDivisionError, which you can use directly. Of
course, you should choose exceptions wisely -- don't use
PyExc_TypeError to mean that a file couldn't be opened (that
should probably be PyExc_IOError). If something's wrong with
the argument list, the PyArg_ParseTuple() function usually
raises PyExc_TypeError. If you have an argument whose value
must be in a particular range or must satisfy other conditions,
PyExc_ValueError is appropriate.
You can also define a new exception that is unique to your module.
For this, you usually declare a static object variable at the
beginning of your file, e.g.
```text

static PyObject *SpamError;
```
and initialize it in your module's initialization function
(initspam()) with an exception object, e.g. (leaving out
the error checking for now):
```text

void
initspam()
{
PyObject *m, *d;

m = Py_InitModule("spam", SpamMethods);
d = PyModule_GetDict(m);
SpamError = PyErr_NewException("spam.error", NULL, NULL);
PyDict_SetItemString(d, "error", SpamError);
}
```
Note that the Python name for the exception object is
spam.error. The PyErr_NewException() function
may create either a string or class, depending on whether the
-X flag was passed to the interpreter. If
-X was used, SpamError will be a string object,
otherwise it will be a class object with the base class being
Exception, described in the
Python Library Reference (../lib/lib.html) under ``Built-in
Exceptions.''


../index.html | front.html | Extending and Embedding the Python Interpreter | contents.html
---
# Extending and Embedding the Python Interpreter
Guido van Rossum
Fred L. Drake, Jr., editor
BeOpen PythonLabs
E-mail: python-docs@python.org
September 18, 2000
Release 1.6


Front Matter

ext.html | ext.html | contents.html | Extending and Embedding the Python Interpreter | contents.html
---
# Front Matter
BEOPEN.COM TERMS AND CONDITIONS FOR PYTHON 2.0
BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1
1. This LICENSE AGREEMENT is between BeOpen.com (``BeOpen''), having an
office at 160 Saratoga Avenue, Santa Clara, CA 95051, and the
Individual or Organization (``Licensee'') accessing and otherwise
using this software in source or binary form and its associated
documentation (``the Software'').
2. Subject to the terms and conditions of this BeOpen Python License
Agreement, BeOpen hereby grants Licensee a non-exclusive,
royalty-free, world-wide license to reproduce, analyze, test, perform
and/or display publicly, prepare derivative works, distribute, and
otherwise use the Software alone or in any derivative version,
provided, however, that the BeOpen Python License is retained in the
Software, alone or in any derivative version prepared by Licensee.
3. BeOpen is making the Software available to Licensee on an ``AS IS''
basis. BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.
4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE
SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS
AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY
DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
5. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.
6. This License Agreement shall be governed by and interpreted in all
respects by the law of the State of California, excluding conflict of
law provisions. Nothing in this License Agreement shall be deemed to
create any relationship of agency, partnership, or joint venture
between BeOpen and Licensee. This License Agreement does not grant
permission to use BeOpen trademarks or trade names in a trademark
sense to endorse or promote products or services of Licensee, or any
third party. As an exception, the ``BeOpen Python'' logos available
at http://www.pythonlabs.com/logos.html may be used according to the
permissions granted on that web page.
7. By copying, installing or otherwise using the software, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.
CNRI OPEN SOURCE LICENSE AGREEMENT
Python 1.6 is made available subject to the terms and conditions in
CNRI's License Agreement. This Agreement together with Python 1.6 may
be located on the Internet using the following unique, persistent
identifier (known as a handle): 1895.22/1012. This Agreement may also
be obtained from a proxy server on the Internet using the following
URL: http://hdl.handle.net/1895.22/1012.
CWI PERMISSIONS STATEMENT AND DISCLAIMER
Copyright © 1991 - 1995, Stichting Mathematisch Centrum
Amsterdam, The Netherlands. All rights reserved.
Permission to use, copy, modify, and distribute this software and its
documentation for any purpose and without fee is hereby granted,
provided that the above copyright notice appear in all copies and that
both that copyright notice and this permission notice appear in
supporting documentation, and that the name of Stichting Mathematisch
Centrum or CWI not be used in advertising or publicity pertaining to
distribution of the software without specific, written prior
permission.
STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO
THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE
FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
Acknowledgements
The following people have contributed sections to this document: Jim
Fulton, Konrad Hinsen, Chris Phoenix, and Neil Schemenauer.
### Abstract:
Python is an interpreted, object-oriented programming language. This
document describes how to write modules in C or C++ to extend the
Python interpreter with new modules. Those modules can define new
functions but also new object types and their methods. The document
also describes how to embed the Python interpreter in another
application, for use as an extension language. Finally, it shows how
to compile and link extension modules so that they can be loaded
dynamically (at run time) into the interpreter, if the underlying
operating system supports this feature.
This document assumes basic knowledge about Python. For an informal
introduction to the language, see the
Python Tutorial (../tut/tut.html). The
Python Reference Manual (../ref/ref.html) gives a more
formal definition of the language. The
Python Library Reference (../lib/lib.html) documents the
existing object types, functions and modules (both built-in and
written in Python) that give the language its wide application range.
For a detailed description of the whole Python/C API, see the separate
Python/C API Reference Manual (../api/api.html).


1. Extending Python with C or C++

contents.html | ext.html | simpleExample.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1. Extending Python with C or C++
It is quite easy to add new built-in modules to Python, if you know
how to program in C. Such extension modules can do two things
that can't be done directly in Python: they can implement new built-in
object types, and they can call C library functions and system calls.
To support extensions, the Python API (Application Programmers
Interface) defines a set of functions, macros and variables that
provide access to most aspects of the Python run-time system. The
Python API is incorporated in a C source file by including the header
`"Python.h"`.
The compilation of an extension module depends on its intended use as
well as on your system setup; details are given in later chapters.


backToExample.html | intro.html | compilation.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.4 The Module's Method Table and Initialization Function
I promised to show how spam_system() is called from Python
programs. First, we need to list its name and address in a ``method
table'':
```text

static PyMethodDef SpamMethods[] = {
...
{"system", spam_system, METH_VARARGS},
...
{NULL, NULL} /* Sentinel */
};
```
Note the third entry ("METH_VARARGS"). This is a flag telling
the interpreter the calling convention to be used for the C
function. It should normally always be "METH_VARARGS" or
"METH_VARARGS | METH_KEYWORDS"; a value of `0` means that an
obsolete variant of PyArg_ParseTuple() is used.
When using only "METH_VARARGS", the function should expect
the Python-level parameters to be passed in as a tuple acceptable for
parsing via PyArg_ParseTuple(); more information on this
function is provided below.
The METH_KEYWORDS bit may be set in the third field if
keyword arguments should be passed to the function. In this case, the
C function should accept a third "PyObject *" parameter which
will be a dictionary of keywords. Use
PyArg_ParseTupleAndKeywords() to parse the arguments to
such a function.
The method table must be passed to the interpreter in the module's
initialization function. The initialization function must be named
initname(), where name is the name of the
module, and should be the only non-static item defined in
the module file:
```text

void
initspam()
{
(void) Py_InitModule("spam", SpamMethods);
}
```
Note that for C++, this method must be declared `extern "C"`.
When the Python program imports module spam for the first
time, initspam() is called. (See below for comments about
embedding Python.) It calls
Py_InitModule(), which creates a ``module object'' (which
is inserted in the dictionary `sys.modules` under the key
`"spam"`), and inserts built-in function objects into the newly
created module based upon the table (an array of PyMethodDef
structures) that was passed as its second argument.
Py_InitModule() returns a pointer to the module object
that it creates (which is unused here). It aborts with a fatal error
if the module could not be initialized satisfactorily, so the caller
doesn't need to check for errors.
When embedding Python, the initspam() function is not
called automatically unless there's an entry in the
_PyImport_Inittab table. The easiest way to handle this is to
statically initialize your statically-linked modules by directly
calling initspam() after the call to
Py_Initialize() or PyMac_Initialize():
```text

int main(int argc, char **argv)
{
/* Pass argv[0] to the Python interpreter */
Py_SetProgramName(argv[0]);

/* Initialize the Python interpreter. Required. */
Py_Initialize();

/* Add a static module */
initspam();
```
And example may be found in the file Demo/embed/demo.c in the
Python source distribution.
Note: Removing entries from `sys.modules` or importing
compiled modules into multiple interpreters within a process (or
following a fork() without an intervening
exec()) can create problems for some extension modules.
Extension module authors should exercise caution when initializing
internal data structures.
A more substantial example module is included in the Python source
distribution as Modules/xxmodule.c. This file may be used as a
template or simply read as an example. The modulator.py
script included in the source distribution or Windows install provides
a simple graphical user interface for declaring the functions and
objects which a module should implement, and can generate a template
which can be filled in. The script lives in the
Tools/modulator/ directory; see the README file there
for more information.


2.3 Example

module-defn-options.html | building-on-unix.html | distributing.html | Extending and Embedding the Python Interpreter | contents.html
---
# 2.3 Example
Here is a more complicated example from Modules/Setup.in:
```text

GMP=/ufs/guido/src/gmp
mpz mpzmodule.c -I$(GMP) $(GMP)/libgmp.a
```
which could also be written as:
```text

mpz mpzmodule.c -I$(GMP) -L$(GMP) -lgmp
```


2.2 Module Definition Options

custom-interps.html | building-on-unix.html | module-defn-example.html | Extending and Embedding the Python Interpreter | contents.html
---
# 2.2 Module Definition Options
Several compiler options are supported:
Other compiler options can be included (snuck in) by putting them
in variables.
Source files can include files with .c, .C, .cc,
.cpp, .cxx, and .c++ extensions.
Other input files include files with .a, .o, .sl,
and .so extensions.


1.10.4 NULL Pointers

thinIce.html | refcounts.html | cplusplus.html | Extending and Embedding the Python Interpreter | contents.html
---
## 1.10.4 NULL Pointers
In general, functions that take object references as arguments do not
expect you to pass them NULL pointers, and will dump core (or
cause later core dumps) if you do so. Functions that return object
references generally return NULL only to indicate that an
exception occurred. The reason for not testing for NULL
arguments is that functions often pass the objects they receive on to
other function -- if each function were to test for NULL,
there would be a lot of redundant tests and the code would run slower.
It is better to test for NULL only at the ``source'', i.e. when a
pointer that may be NULL is received, e.g. from
malloc() or from a function that may raise an exception.
The macros Py_INCREF() and Py_DECREF()
do not check for NULL pointers -- however, their variants
Py_XINCREF() and Py_XDECREF() do.
The macros for checking for a particular object type
(`Py type _Check()`) don't check for NULL pointers --
again, there is much code that calls several of these in a row to test
an object against various different expected types, and this would
generate redundant tests. There are no variants with NULL
checking.
The C function calling mechanism guarantees that the argument list
passed to C functions (`args` in the examples) is never
NULL -- in fact it guarantees that it is always a tuple.1.4 (#foot633)
It is a severe error to ever let a NULL pointer ``escape'' to
the Python user.


1.10.2 Ownership Rules

refcountsInPython.html | refcounts.html | thinIce.html | Extending and Embedding the Python Interpreter | contents.html
---
## 1.10.2 Ownership Rules
Whenever an object reference is passed into or out of a function, it
is part of the function's interface specification whether ownership is
transferred with the reference or not.
Most functions that return a reference to an object pass on ownership
with the reference. In particular, all functions whose function it is
to create a new object, e.g. PyInt_FromLong() and
Py_BuildValue(), pass ownership to the receiver. Even if in
fact, in some cases, you don't receive a reference to a brand new
object, you still receive ownership of the reference. For instance,
PyInt_FromLong() maintains a cache of popular values and can
return a reference to a cached item.
Many functions that extract objects from other objects also transfer
ownership with the reference, for instance
PyObject_GetAttrString(). The picture is less clear, here,
however, since a few common routines are exceptions:
PyTuple_GetItem(), PyList_GetItem(),
PyDict_GetItem(), and PyDict_GetItemString()
all return references that you borrow from the tuple, list or
dictionary.
The function PyImport_AddModule() also returns a borrowed
reference, even though it may actually create the object it returns:
this is possible because an owned reference to the object is stored in
`sys.modules`.
When you pass an object reference into another function, in general,
the function borrows the reference from you -- if it needs to store
it, it will use Py_INCREF() to become an independent
owner. There are exactly two important exceptions to this rule:
PyTuple_SetItem() and PyList_SetItem(). These
functions take over ownership of the item passed to them -- even if
they fail! (Note that PyDict_SetItem() and friends don't
take over ownership -- they are ``normal.'')
When a C function is called from Python, it borrows references to its
arguments from the caller. The caller owns a reference to the object,
so the borrowed reference's lifetime is guaranteed until the function
returns. Only when such a borrowed reference must be stored or passed
on, it must be turned into an owned reference by calling
Py_INCREF().
The object reference returned from a C function that is called from
Python must be an owned reference -- ownership is tranferred from the
function to its caller.


callingPython.html | intro.html | parseTupleAndKeywords.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.7 Format Strings for PyArg_ParseTuple()
The PyArg_ParseTuple() function is declared as follows:
```text

int PyArg_ParseTuple(PyObject *arg, char *format, ...);
```
The arg argument must be a tuple object containing an argument
list passed from Python to a C function. The format argument
must be a format string, whose syntax is explained below. The
remaining arguments must be addresses of variables whose type is
determined by the format string. For the conversion to succeed, the
arg object must match the format and the format must be
exhausted.
Note that while PyArg_ParseTuple() checks that the Python
arguments have the required types, it cannot check the validity of the
addresses of C variables passed to the call: if you make mistakes
there, your code will probably crash or at least overwrite random bits
in memory. So be careful!
A format string consists of zero or more ``format units''. A format
unit describes one Python object; it is usually a single character or
a parenthesized sequence of format units. With a few exceptions, a
format unit that is not a parenthesized sequence normally corresponds
to a single address argument to PyArg_ParseTuple(). In the
following description, the quoted form is the format unit; the entry
in (round) parentheses is the Python object type that matches the
format unit; and the entry in [square] brackets is the type of the C
variable(s) whose address should be passed. (Use the "&"operator to pass a variable's address.)
Note that any Python object references which are provided to the
caller are borrowed references; do not decrement their
reference count!
"s" (string) [char *]: Convert a Python string to a C pointer to a character string. You
must not provide storage for the string itself; a pointer to an
existing string is stored into the character pointer variable whose
address you pass. The C string is null-terminated. The Python string
must not contain embedded null bytes; if it does, a TypeError
exception is raised.
"s#" (string) [char *, int]: This variant on "s" stores into two C variables, the first one
a pointer to a character string, the second one its length. In this
case the Python string may contain embedded null bytes.
"z" (string or `None`) [char *]: Like "s", but the Python object may also be `None`, in which
case the C pointer is set to NULL.
"z#" (string or `None`) [char *, int]: This is to "s#" as "z" is to "s".
"u" (Unicode string) [Py_UNICODE *]: Convert a Python Unicode object to a C pointer to a null-terminated
buffer of Unicode (UCS-2) data. As with "s", there is no need
to provide storage for the Unicode data buffer; a pointer to the
existing Unicode data is stored into the Py_UNICODE pointer variable whose
address you pass.
"u#" (Unicode string) [Py_UNICODE *, int]: This variant on "u" stores into two C variables, the first one
a pointer to a Unicode data buffer, the second one its length.
"b" (integer) [char]: Convert a Python integer to a tiny int, stored in a C char.
"h" (integer) [short int]: Convert a Python integer to a C short int.
"i" (integer) [int]: Convert a Python integer to a plain C int.
"l" (integer) [long int]: Convert a Python integer to a C long int.
"c" (string of length 1) [char]: Convert a Python character, represented as a string of length 1, to a
C char.
"f" (float) [float]: Convert a Python floating point number to a C float.
"d" (float) [double]: Convert a Python floating point number to a C double.
"D" (complex) [Py_complex]: Convert a Python complex number to a C Py_complex structure.
"O" (object) [PyObject *]: Store a Python object (without any conversion) in a C object pointer.
The C program thus receives the actual object that was passed. The
object's reference count is not increased. The pointer stored is not
NULL.
"O!" (object) [typeobject, PyObject *]: Store a Python object in a C object pointer. This is similar to
"O", but takes two C arguments: the first is the address of a
Python type object, the second is the address of the C variable (of
type PyObject *) into which the object pointer is stored.
If the Python object does not have the required type,
TypeError is raised.
"O&" (object) [converter, anything]: Convert a Python object to a C variable through a converter
function. This takes two arguments: the first is a function, the
second is the address of a C variable (of arbitrary type), converted
to void *. The converter function in turn is called as
follows:
status`=`converter`(`object, address`);`
where object is the Python object to be converted and
address is the void * argument that was passed to
PyArg_ConvertTuple(). The returned status should be
`1` for a successful conversion and `0` if the conversion
has failed. When the conversion fails, the converter function
should raise an exception.
"S" (string) [PyStringObject *]: Like "O" but requires that the Python object is a string object.
Raises TypeError if the object is not a string object.
The C variable may also be declared as PyObject *.
"U" (Unicode string) [PyUnicodeObject *]: Like "O" but requires that the Python object is a Unicode object.
Raises TypeError if the object is not a Unicode object.
The C variable may also be declared as PyObject *.
"t#" (read-only character buffer) [char *, int]: Like "s#", but accepts any object which implements the read-only
buffer interface. The char * variable is set to point to the
first byte of the buffer, and the int is set to the length of
the buffer. Only single-segment buffer objects are accepted;
TypeError is raised for all others.
"w" (read-write character buffer) [char *]: Similar to "s", but accepts any object which implements the
read-write buffer interface. The caller must determine the length of
the buffer by other means, or use "w#" instead. Only
single-segment buffer objects are accepted; TypeError is
raised for all others.
"w#" (read-write character buffer) [char *, int]: Like "s#", but accepts any object which implements the
read-write buffer interface. The char * variable is set to
point to the first byte of the buffer, and the int is set to
the length of the buffer. Only single-segment buffer objects are
accepted; TypeError is raised for all others.
"(items)" (tuple) [matching-items]: The object must be a Python sequence whose length is the number of
format units in items. The C arguments must correspond to the
individual format units in items. Format units for sequences
may be nested.
Note: Prior to Python version 1.5.2, this format specifier
only accepted a tuple containing the individual parameters, not an
arbitrary sequence. Code which previously caused
TypeError to be raised here may now proceed without an
exception. This is not expected to be a problem for existing code.
It is possible to pass Python long integers where integers are
requested; however no proper range checking is done -- the most
significant bits are silently truncated when the receiving field is
too small to receive the value (actually, the semantics are inherited
from downcasts in C -- your mileage may vary).
A few other characters have a meaning in a format string. These may
not occur inside nested parentheses. They are:
"|": Indicates that the remaining arguments in the Python argument list are
optional. The C variables corresponding to optional arguments should
be initialized to their default value -- when an optional argument is
not specified, PyArg_ParseTuple() does not touch the contents
of the corresponding C variable(s).
":": The list of format units ends here; the string after the colon is used
as the function name in error messages (the ``associated value'' of
the exception that PyArg_ParseTuple() raises).
";": The list of format units ends here; the string after the colon is used
as the error message instead of the default error message.
Clearly, ":" and ";" mutually exclude each other.
Some example calls:
```text

int ok;
int i, j;
long k, l;
char *s;
int size;

ok = PyArg_ParseTuple(args, ""); /* No arguments */
/* Python call: f() */
```
```text

ok = PyArg_ParseTuple(args, "s", &s); /* A string */
/* Possible Python call: f('whoops!') */
```
```text

ok = PyArg_ParseTuple(args, "lls", &k, &l, &s); /* Two longs and a string */
/* Possible Python call: f(1, 2, 'three') */
```
```text

ok = PyArg_ParseTuple(args, "(ii)s#", &i, &j, &s, &size);
/* A pair of ints and a string, whose size is also returned */
/* Possible Python call: f((1, 2), 'three') */
```
```text

{
char *file;
char *mode = "r";
int bufsize = 0;
ok = PyArg_ParseTuple(args, "s|si", &file, &mode, &bufsize);
/* A string, and optionally another string and an integer */
/* Possible Python calls:
f('spam')
f('spam', 'w')
f('spam', 'wb', 100000) */
}
```
```text

{
int left, top, right, bottom, h, v;
ok = PyArg_ParseTuple(args, "((ii)(ii))(ii)",
&left, &top, &right, &bottom, &h, &v);
/* A rectangle and a point */
/* Possible Python call:
f(((0, 0), (400, 300)), (10, 10)) */
}
```
```text

{
Py_complex c;
ok = PyArg_ParseTuple(args, "D:myfunction", &c);
/* a complex, also providing a function name for errors */
/* Possible Python call: myfunction(1+2j) */
}
```


parseTuple.html | intro.html | buildValue.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.8 Keyword Parsing with PyArg_ParseTupleAndKeywords()
The PyArg_ParseTupleAndKeywords() function is declared as
follows:
```text

int PyArg_ParseTupleAndKeywords(PyObject *arg, PyObject *kwdict,
char *format, char **kwlist, ...);
```
The arg and format parameters are identical to those of the
PyArg_ParseTuple() function. The kwdict parameter
is the dictionary of keywords received as the third parameter from the
Python runtime. The kwlist parameter is a NULL-terminated
list of strings which identify the parameters; the names are matched
with the type information from format from left to right.
Note: Nested tuples cannot be parsed when using keyword
arguments! Keyword parameters passed in which are not present in the
kwlist will cause TypeError to be raised.
Here is an example module which uses keywords, based on an example by
Geoff Philbrick (philbrick@hks.com):
```text

#include <stdio.h>
#include "Python.h"

static PyObject *
keywdarg_parrot(self, args, keywds)
PyObject *self;
PyObject *args;
PyObject *keywds;
{
int voltage;
char *state = "a stiff";
char *action = "voom";
char *type = "Norwegian Blue";

static char *kwlist[] = {"voltage", "state", "action", "type", NULL};

if (!PyArg_ParseTupleAndKeywords(args, keywds, "i|sss", kwlist,
&voltage, &state, &action, &type))
return NULL;

printf("-- This parrot wouldn't %s if you put %i Volts through it.\n",
action, voltage);
printf("-- Lovely plumage, the %s -- It's %s!\n", type, state);

Py_INCREF(Py_None);

return Py_None;
}

static PyMethodDef keywdarg_methods[] = {
/* The cast of the function is necessary since PyCFunction values
* only take two PyObject* parameters, and keywdarg_parrot() takes
* three.
*/
{"parrot", (PyCFunction)keywdarg_parrot, METH_VARARGS|METH_KEYWORDS},
{NULL, NULL} /* sentinel */
};

void
initkeywdarg()
{
/* Create the module and add the functions */
Py_InitModule("keywdarg", keywdarg_methods);
}
```


1.10 Reference Counts

buildValue.html | intro.html | refcountsInPython.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.10 Reference Counts
In languages like C or C++, the programmer is responsible for
dynamic allocation and deallocation of memory on the heap. In C,
this is done using the functions malloc() and
free(). In C++, the operators new and
delete are used with essentially the same meaning; they are
actually implemented using malloc() and
free(), so we'll restrict the following discussion to the
latter.
Every block of memory allocated with malloc() should
eventually be returned to the pool of available memory by exactly one
call to free(). It is important to call
free() at the right time. If a block's address is
forgotten but free() is not called for it, the memory it
occupies cannot be reused until the program terminates. This is
called a memory leak. On the other hand, if a program calls
free() for a block and then continues to use the block, it
creates a conflict with re-use of the block through another
malloc() call. This is called using freed memory.
It has the same bad consequences as referencing uninitialized data --
core dumps, wrong results, mysterious crashes.
Common causes of memory leaks are unusual paths through the code. For
instance, a function may allocate a block of memory, do some
calculation, and then free the block again. Now a change in the
requirements for the function may add a test to the calculation that
detects an error condition and can return prematurely from the
function. It's easy to forget to free the allocated memory block when
taking this premature exit, especially when it is added later to the
code. Such leaks, once introduced, often go undetected for a long
time: the error exit is taken only in a small fraction of all calls,
and most modern machines have plenty of virtual memory, so the leak
only becomes apparent in a long-running process that uses the leaking
function frequently. Therefore, it's important to prevent leaks from
happening by having a coding convention or strategy that minimizes
this kind of errors.
Since Python makes heavy use of malloc() and
free(), it needs a strategy to avoid memory leaks as well
as the use of freed memory. The chosen method is called
reference counting. The principle is simple: every object
contains a counter, which is incremented when a reference to the
object is stored somewhere, and which is decremented when a reference
to it is deleted. When the counter reaches zero, the last reference
to the object has been deleted and the object is freed.
An alternative strategy is called automatic garbage collection.
(Sometimes, reference counting is also referred to as a garbage
collection strategy, hence my use of ``automatic'' to distinguish the
two.) The big advantage of automatic garbage collection is that the
user doesn't need to call free() explicitly. (Another claimed
advantage is an improvement in speed or memory usage -- this is no
hard fact however.) The disadvantage is that for C, there is no
truly portable automatic garbage collector, while reference counting
can be implemented portably (as long as the functions malloc()
and free() are available -- which the C Standard guarantees).
Maybe some day a sufficiently portable automatic garbage collector
will be available for C. Until then, we'll have to live with
reference counts.


1.10.1 Reference Counting in Python

refcounts.html | refcounts.html | ownershipRules.html | Extending and Embedding the Python Interpreter | contents.html
---
## 1.10.1 Reference Counting in Python
There are two macros, `Py_INCREF(x)` and `Py_DECREF(x)`,
which handle the incrementing and decrementing of the reference count.
Py_DECREF() also frees the object when the count reaches zero.
For flexibility, it doesn't call free() directly -- rather, it
makes a call through a function pointer in the object's type
object. For this purpose (and others), every object also contains a
pointer to its type object.
The big question now remains: when to use `Py_INCREF(x)` and
`Py_DECREF(x)`? Let's first introduce some terms. Nobody
``owns'' an object; however, you can own a reference to an
object. An object's reference count is now defined as the number of
owned references to it. The owner of a reference is responsible for
calling Py_DECREF() when the reference is no longer
needed. Ownership of a reference can be transferred. There are three
ways to dispose of an owned reference: pass it on, store it, or call
Py_DECREF(). Forgetting to dispose of an owned reference
creates a memory leak.
It is also possible to borrow1.2 (#foot571) a reference to an object. The borrower
of a reference should not call Py_DECREF(). The borrower must
not hold on to the object longer than the owner from which it was
borrowed. Using a borrowed reference after the owner has disposed of
it risks using freed memory and should be avoided
completely.1.3 (#foot907)
The advantage of borrowing over owning a reference is that you don't
need to take care of disposing of the reference on all possible paths
through the code -- in other words, with a borrowed reference you
don't run the risk of leaking when a premature exit is taken. The
disadvantage of borrowing over leaking is that there are some subtle
situations where in seemingly correct code a borrowed reference can be
used after the owner from which it was borrowed has in fact disposed
of it.
A borrowed reference can be changed into an owned reference by calling
Py_INCREF(). This does not affect the status of the owner from
which the reference was borrowed -- it creates a new owned reference,
and gives full owner responsibilities (i.e., the new owner must
dispose of the reference properly, as well as the previous owner).