What are and how to use coroutines in Python

Coroutines in Python are a way to perform asynchronous and concurrent tasks.

A coroutine in Python is a function that can be paused and then resumed, allowing other tasks to run in the meantime.

This ability to suspend and resume makes them ideal for tasks involving concurrency, such as asynchronous programming, data stream manipulation, and creating processing pipelines.

In a way, coroutines are similar to generators, but with additional capabilities. While generators produce a sequence of values, coroutines can consume values, produce values, and manage the flow of control.

Definition of a coroutine

Coroutines are defined using the async def keyword instead of def, and use await to suspend execution and wait for another task to complete.

# Basic example of a coroutine
import asyncio

async def my_coroutine():
    print("Starting the coroutine")
    await asyncio.sleep(1)
    print("Coroutine finished")

In this example, my_coroutine is an asynchronous function that uses await to pause its execution for 1 second (simulating an asynchronous task like an I/O wait or a network request).

Running coroutines

The asyncio module in Python provides a framework for writing asynchronous code using coroutines and performing efficient I/O operations.

To run a coroutine, we must add it to the asyncio event loop. The easiest way to do this is with asyncio.run().

import asyncio

async def greet():
    print("Hello world")
    await asyncio.sleep(1)
    print("From LuisLlamas")

asyncio.run(greet())

# output:
# hello world
# ...(wait for one second)...
# From LuisLlamas

In this example, greet is a coroutine that runs in the asyncio event loop. Execution pauses for 1 second between the two print statements.

Waiting for multiple coroutines

Coroutines are especially useful when multiple asynchronous operations need to be performed. This can be managed using asyncio.gather().

import asyncio

async def task(number):
    print(f"Starting task {number}")
    await asyncio.sleep(1)
    print(f"Finishing task {number}")

async def main():
    await asyncio.gather(task(1), task(2), task(3))

asyncio.run(main())

In this example, main waits for three task coroutines to complete in parallel.

Common applications of coroutines

Asynchronous Programming

Coroutines are essential for asynchronous programming, allowing multiple I/O operations to be performed without blocking the program flow.

import asyncio
import aiohttp

async def fetch_url(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()

async def main():
    html = await fetch_url('http://www.google.com')
    print(html)

asyncio.run(main())

Processing Pipelines

Coroutines can be used to build processing pipelines that manipulate data streams in real-time.

import asyncio

async def producer(queue):
    for i in range(5):
        await queue.put(i)
        await asyncio.sleep(1)

async def consumer(queue):
    while True:
        item = await queue.get()
        if item is None:
            break
        print(f"Consuming {item}")
        await asyncio.sleep(2)

async def main():
    queue = asyncio.Queue()
    await asyncio.gather(producer(queue), consumer(queue))

asyncio.run(main())

In this example, the producer produces items and places them in a queue, while the consumer consumes these items, demonstrating a simple pipeline.