python_doc_md/Intermediate/library/asyncio-dev.md

::: currentmodule
asyncio
:::

# Developing with asyncio {#asyncio-dev}

Asynchronous programming is different from classic \"sequential\" programming.

This page lists common mistakes and traps and explains how to avoid them.

## Debug Mode {#asyncio-debug-mode}

By default asyncio runs in production mode. In order to ease the development asyncio has a *debug mode*.

There are several ways to enable asyncio debug mode:

- Setting the `PYTHONASYNCIODEBUG`{.interpreted-text role="envvar"} environment variable to `1`.
- Using the `Python Development Mode <devmode>`{.interpreted-text role="ref"}.
- Passing `debug=True` to `asyncio.run`{.interpreted-text role="func"}.
- Calling `loop.set_debug`{.interpreted-text role="meth"}.

In addition to enabling the debug mode, consider also:

- setting the log level of the `asyncio logger <asyncio-logger>`{.interpreted-text role="ref"} to `logging.DEBUG`{.interpreted-text role="py:const"}, for example the following snippet of code can be run at startup of the application:

      logging.basicConfig(level=logging.DEBUG)

- configuring the `warnings`{.interpreted-text role="mod"} module to display `ResourceWarning`{.interpreted-text role="exc"} warnings. One way of doing that is by using the `-W`{.interpreted-text role="option"} `default` command line option.

When the debug mode is enabled:

- Many non-threadsafe asyncio APIs (such as `loop.call_soon`{.interpreted-text role="meth"} and `loop.call_at`{.interpreted-text role="meth"} methods) raise an exception if they are called from a wrong thread.
- The execution time of the I/O selector is logged if it takes too long to perform an I/O operation.
- Callbacks taking longer than 100 milliseconds are logged. The `loop.slow_callback_duration`{.interpreted-text role="attr"} attribute can be used to set the minimum execution duration in seconds that is considered \"slow\".

## Concurrency and Multithreading {#asyncio-multithreading}

An event loop runs in a thread (typically the main thread) and executes all callbacks and Tasks in its thread. While a Task is running in the event loop, no other Tasks can run in the same thread. When a Task executes an `await` expression, the running Task gets suspended, and the event loop executes the next Task.

To schedule a `callback`{.interpreted-text role="term"} from another OS thread, the `loop.call_soon_threadsafe`{.interpreted-text role="meth"} method should be used. Example:

    loop.call_soon_threadsafe(callback, *args)

Almost all asyncio objects are not thread safe, which is typically not a problem unless there is code that works with them from outside of a Task or a callback. If there\'s a need for such code to call a low-level asyncio API, the `loop.call_soon_threadsafe`{.interpreted-text role="meth"} method should be used, e.g.:

    loop.call_soon_threadsafe(fut.cancel)

To schedule a coroutine object from a different OS thread, the `run_coroutine_threadsafe`{.interpreted-text role="func"} function should be used. It returns a `concurrent.futures.Future`{.interpreted-text role="class"} to access the result:

    async def coro_func():
         return await asyncio.sleep(1, 42)

    # Later in another OS thread:

    future = asyncio.run_coroutine_threadsafe(coro_func(), loop)
    # Wait for the result:
    result = future.result()

To handle signals the event loop must be run in the main thread.

The `loop.run_in_executor`{.interpreted-text role="meth"} method can be used with a `concurrent.futures.ThreadPoolExecutor`{.interpreted-text role="class"} or `~concurrent.futures.InterpreterPoolExecutor`{.interpreted-text role="class"} to execute blocking code in a different OS thread without blocking the OS thread that the event loop runs in.

There is currently no way to schedule coroutines or callbacks directly from a different process (such as one started with `multiprocessing`{.interpreted-text role="mod"}). The `asyncio-event-loop-methods`{.interpreted-text role="ref"} section lists APIs that can read from pipes and watch file descriptors without blocking the event loop. In addition, asyncio\'s `Subprocess <asyncio-subprocess>`{.interpreted-text role="ref"} APIs provide a way to start a process and communicate with it from the event loop. Lastly, the aforementioned `loop.run_in_executor`{.interpreted-text role="meth"} method can also be used with a `concurrent.futures.ProcessPoolExecutor`{.interpreted-text role="class"} to execute code in a different process.

## Running Blocking Code {#asyncio-handle-blocking}

Blocking (CPU-bound) code should not be called directly. For example, if a function performs a CPU-intensive calculation for 1 second, all concurrent asyncio Tasks and IO operations would be delayed by 1 second.

An executor can be used to run a task in a different thread, including in a different interpreter, or even in a different process to avoid blocking the OS thread with the event loop. See the `loop.run_in_executor`{.interpreted-text role="meth"} method for more details.

## Logging {#asyncio-logger}

asyncio uses the `logging`{.interpreted-text role="mod"} module and all logging is performed via the `"asyncio"` logger.

The default log level is `logging.INFO`{.interpreted-text role="py:const"}, which can be easily adjusted:

    logging.getLogger("asyncio").setLevel(logging.WARNING)

Network logging can block the event loop. It is recommended to use a separate thread for handling logs or use non-blocking IO. For example, see `blocking-handlers`{.interpreted-text role="ref"}.

## Detect never-awaited coroutines {#asyncio-coroutine-not-scheduled}

When a coroutine function is called, but not awaited (e.g. `coro()` instead of `await coro()`) or the coroutine is not scheduled with `asyncio.create_task`{.interpreted-text role="meth"}, asyncio will emit a `RuntimeWarning`{.interpreted-text role="exc"}:

    import asyncio

    async def test():
        print("never scheduled")

    async def main():
        test()

    asyncio.run(main())

Output:

    test.py:7: RuntimeWarning: coroutine 'test' was never awaited
      test()

Output in debug mode:

    test.py:7: RuntimeWarning: coroutine 'test' was never awaited
    Coroutine created at (most recent call last)
      File "../t.py", line 9, in <module>
        asyncio.run(main(), debug=True)

      < .. >

      File "../t.py", line 7, in main
        test()
      test()

The usual fix is to either await the coroutine or call the `asyncio.create_task`{.interpreted-text role="meth"} function:

    async def main():
        await test()

## Detect never-retrieved exceptions

If a `Future.set_exception`{.interpreted-text role="meth"} is called but the Future object is never awaited on, the exception would never be propagated to the user code. In this case, asyncio would emit a log message when the Future object is garbage collected.

Example of an unhandled exception:

    import asyncio

    async def bug():
        raise Exception("not consumed")

    async def main():
        asyncio.create_task(bug())

    asyncio.run(main())

Output:

    Task exception was never retrieved
    future: <Task finished coro=<bug() done, defined at test.py:3>
      exception=Exception('not consumed')>

    Traceback (most recent call last):
      File "test.py", line 4, in bug
        raise Exception("not consumed")
    Exception: not consumed

`Enable the debug mode <asyncio-debug-mode>`{.interpreted-text role="ref"} to get the traceback where the task was created:

    asyncio.run(main(), debug=True)

Output in debug mode:

    Task exception was never retrieved
    future: <Task finished coro=<bug() done, defined at test.py:3>
        exception=Exception('not consumed') created at asyncio/tasks.py:321>

    source_traceback: Object created at (most recent call last):
      File "../t.py", line 9, in <module>
        asyncio.run(main(), debug=True)

    < .. >

    Traceback (most recent call last):
      File "../t.py", line 4, in bug
        raise Exception("not consumed")
    Exception: not consumed