Imagine you’re a chef in a busy restaurant, and you need to prepare multiple dishes at the same time. You have two options for managing your tasks:
Option 1: Subroutines (functions)
You can write down a list of steps for preparing each dish (like a recipe), and follow each list of steps sequentially. You start with the first dish, follow the steps from beginning to end, and then move on to the next dish. This is similar to how subroutines work — they execute one set of instructions at a time, from start to finish, without jumping back and forth between tasks.
Option 2: Coroutines
Alternatively, you can use a more flexible approach. You start preparing the first dish, but when you reach a step that takes a long time (like waiting for something to cook), you switch to the second dish and start working on that. When the first dish is ready for the next step, you switch back to it and continue. This is similar to how coroutines work — they can pause and resume tasks as needed, allowing them to work on multiple tasks concurrently.
In both cases, you’re still doing the same work (preparing multiple dishes), but coroutines allow you to manage your tasks more flexibly, by switching between tasks when needed, instead of following a strict, linear sequence.
Coroutines and asyncio go hand in hand in Python, as asyncio provides the infrastructure for running coroutines asynchronously. Here are a few key things to know about using coroutines with asyncio:
- async and await: In Python 3.5 and later, you can use the async and await keywords to define and work with coroutines. A coroutine is defined using the async keyword, and you can use the await keyword to suspend the coroutine and wait for the result of an asynchronous operation.
- Event loop: asyncio provides an event loop, which is responsible for managing and executing coroutines. You can use the asyncio.get_event_loop() function to get the current event loop, and the loop.run_until_complete(coroutine) method to run a coroutine until it is completed or until it yields.
- Tasks: asyncio provides a asyncio.Task class, which represents an ongoing, asynchronous task. You can use the loop.create_task(coroutine) method to create a new task from a coroutine, and then use the task object to manage the task, check its status, and get its result.
- Cooperative multitasking: asyncio relies on cooperative multitasking, where tasks voluntarily yield control back to the event loop, allowing other tasks to run. This allows asyncio to handle multiple tasks concurrently, without using multiple threads or processes.
import time
def task_one():
try:
print("Task One start")
start_time = time.time()
time.sleep(2)
end_time = time.time()
print(f"Task One end (Time: {end_time - start_time:.2f} seconds)")
except Exception as e:
print(f"Task One failed: {e}")
def task_two():
try:
print("Task Two start")
start_time = time.time()
time.sleep(1)
end_time = time.time()
print(f"Task Two end (Time: {end_time - start_time:.2f} seconds)")
except Exception as e:
print(f"Task Two failed: {e}")
def main():
# Create tasks explicitly
task_one()
task_two()
# Run the main function
main()This code demonstrates executing two tasks sequentially and timing their execution. It defines two functions: task_one() and task_two() that each print a start and end message and sleep for a few seconds to simulate work.
The main() function calls these two task functions explicitly in sequence. Before and after calling each task, it records the start and end time using time.time(). It prints the elapsed time for each task in seconds.
This allows measuring the execution time of each task. The try-except blocks catch and print any exceptions in the tasks.
The main() function is called at the bottom to execute the sequence of tasks. This code shows a simple pattern for executing and timing multiple tasks in sequence.
import asyncio
import time
async def task_one():
try:
print("Task One start")
start_time = time.time()
await asyncio.sleep(2)
end_time = time.time()
print(f"Task One end (Time: {end_time - start_time:.2f} seconds)")
return end_time - start_time
except Exception as e:
print(f"Task One failed: {e}")
return 0
async def task_two():
try:
print("Task Two start")
start_time = time.time()
await asyncio.sleep(1)
end_time = time.time()
print(f"Task Two end (Time: {end_time - start_time:.2f} seconds)")
return end_time - start_time
except Exception as e:
print(f"Task Two failed: {e}")
return 0
async def main():
try:
# Use asyncio.gather without awaiting individual tasks
results = await asyncio.gather(task_one(), task_two())
# Print the maximum time taken by any task
total_time = max(results)
print(f"Total time for both tasks: {total_time:.2f} seconds")
except Exception as e:
print(f"An error occurred: {e}")
asyncio.run(main())This code runs two async tasks concurrently using asyncio.gather() and calculates the total maximum execution time.
The task_one() and task_two() coroutines now return their individual elapsed times.
In main(), asyncio.gather() is used to run both tasks concurrently without awaiting each one separately.
The results are collected into a list containing each task’s duration.
The maximum duration is found using max() to get the total execution time for both tasks.
This shows how asyncio.gather() can efficiently run multiple coroutines in parallel. The total time is determined by the longest running task.
The individual task durations are also available to analyze each one’s performance. Useful technique for executing async tasks concurrently and calculating their combined execution time.
The key differences in results:
- Without asyncio, task 2 starts only after task 1 finishes, so total time is added.
- With asyncio, both tasks start nearly immediately, and run in parallel. Total time is governed by the longer task.
- Task 2 finishes first since it has a shorter duration of 1 second.
- But overall total time is reduced from 3 sec to 2 sec by running concurrently.
So asyncio allows efficient overlapping execution of multiple tasks, reducing overall execution time compared to sequential execution. The tasks can complete in different orders based on their durations.
For a more in-depth demonstration and visualization of how asyncio allows parallel execution of tasks, check out my video where I show examples using decorators and tqdm progress bars. The video walks through code examples in detail to illustrate the performance differences compared to sequential execution. Seeing the tasks execute concurrently in an animated demo helps build intuition for how asyncio speed ups programs with wait times. The use of decorators and progress bars adds nice polish to the async code as well.
Further reading: https://docs.python.org/3/library/asyncio-task.html
