Asynchronous Requests in Python

1. Asynchronous Requests in Python
2. Importance of Using Asynchronous Requests in Python
3. the Easiest Way to Write Asynchronous Requests in Python
4. Handle Asynchronous Requests in Python
5. Required Python Libraries for Asynchronous Requests
6. Asynchronous Requests vs Regular Requests
7. Async Requests in Python

Today, we will learn about asynchronous requests; this discussion will lead to the code example to see how we can write asynchronous requests in Python.

Asynchronous Requests in Python

Asynchronous requests are the heart of our system. We can think of them as follows:

When a user loads our site (website or web app), the user immediately starts seeing the content. But it’s not quite ready. So we load the content asynchronously in the background while the user continues interacting with the page.

These requests do not block the execution of subsequent code while waiting for a response. That means that other code can continue running while the request is processed.

It can be helpful when dealing with external resources, such as API calls, that may take some time to respond to. It can also make your code more responsive since the UI can continue to update while a request is being processed.

Importance of Using Asynchronous Requests in Python

Asynchronous requests are a great way to improve the performance of our Python application. When a request is made, the Python interpreter can continue to execute other code while the request is being processed.

It can cause a significant increase in speed, especially for applications that make a lot of requests. However, the following are a few essential points to remember when using asynchronous requests.

1. First, we must avoid making too many requests simultaneously. The interpreter can get overwhelmed and slow down if we make too many requests.
2. Second, we need to be prepared to handle errors. If a request fails, the interpreter cannot continue executing the code.

Overall, asynchronous requests are a great tool to improve the performance of our Python application. But, use it carefully; they can help our application run faster and more smoothly.

the Easiest Way to Write Asynchronous Requests in Python

Asynchronous requests are made using the asyncio module easily. In addition, python’s asyncio library provides tools to write asynchronous code. For example, we can use the asyncio.sleep() to pause a coroutine and the asyncio.wait() to wait for a coroutine to complete.

To write an asynchronous request, we need to first create a coroutine. We can do it using the asyncio.ensure_future() function. Once we have a coroutine, we can use the asyncio.sleep() function to pause it and the asyncio.wait() function to wait for it to complete.

Handle Asynchronous Requests in Python

First, if we want to run the asynchronous requests in Python, then you should install the python library of aiohttp by using the following command.

pip install aiohttp

We can use asynchronous requests to improve python applications’ performance. By making requests in parallel, we can dramatically speed up the process.

There are some different ways to handle asynchronous requests in Python. The most popular is the asyncio library. This library provides powerful tools for dealing with async requests.

Another popular option is the grequests library. This library is a bit simpler to use than asyncio, but it can be just as effective.

Which option we select will depend on our specific needs. But whichever we choose, we’re sure to see a significant performance boost by making our requests async.

import grequests

urls = [
    "http://www.heroku.com",
    "http://tablib.org",
    "http://httpbin.org",
    "http://python-requests.org",
    "http://kennethreitz.com",
]

rs = (grequests.get(u) for u in urls)
grequests.map(rs)

Required Python Libraries for Asynchronous Requests

We can use numerous Python libraries for making asynchronous requests. The most popular ones are aiohttp and asyncio.

aiohttp Library for Asynchronous Requests

aiohttp is a library that enables us to make asynchronous HTTP requests. It’s built on top of the asyncio and provides a simple interface for making HTTP requests.

asyncio Library for Asynchronous Requests

asyncio is a library that supports asynchronous programming in Python. It enables us to write asynchronous code and makes it easy to use libraries that support asyncio.

Both aiohttp and asyncio are available on PyPI and can be installed using pip.

import asyncio
import aiohttp
import json

from text_api_config import apikey

Asynchronous Requests vs Regular Requests

We can make two types of requests to a server: asynchronous and regular. Asynchronous requests are made in the background while the user still interacts with the page. Typical requests are made while the page is loading.

Asynchronous requests are generally faster and more efficient than regular requests since they don’t block the page from loading. However, they can be more complex to implement, and all browsers do not always support them.

Code Example:

import requests
import time

start_time = time.time()

for number in range(1, 151):
    url = f"https://pokeapi.co/api/v2/pokemon/{number}"
    resp = requests.get(url)
    pokemon = resp.json()
    print(pokemon["name"])

print("--- %s seconds ---" % (time.time() - start_time))

Output:

bulbasaur
ivysaur
venusaur
....
dragonair
dragonite
mewtwo
--- 68.17992424964905 seconds

the asyncio Module in Python

asyncio is a module for concurrent programming in Python. It provides a framework for managing concurrent threads, tasks, and events. asyncio is used to write programs that can perform multiple tasks concurrently.

asyncio is based on the concept of coroutines. A coroutine is a function that can suspend its execution and yield control back to the caller. It allows multiple coroutines to run concurrently.

asyncio provides tools for managing coroutines, including an event loop, task scheduler, and concurrent data structures.

asyncio is an efficient way to write concurrent programs. It is easy to use and can scale to large programs. asyncio is an excellent choice for programs requiring multiple tasks concurrently.

Code Example:

import asyncio
import aiohttp
import json
from text_api_config import apikey


async def gather_with_concurrency(n, *tasks):
    semaphore = asyncio.Semaphore(n)

    async def sem_task(task):
        async with semaphore:
            return await task

    return await asyncio.gather(*(sem_task(task) for task in tasks))

the aiohttp Module in Python

The aiohttp module is an asynchronous HTTP client/server for Python. It is built on asyncio and provides a simple API for working with HTTP.

The aiohttp module makes it easy to work with HTTP in Python. It provides a simple API that makes sending and receiving HTTP requests and responses easy.

The aiohttp module also provides a way to run an asynchronous HTTP server.

import asyncio
import aiohttp
import json
from text_api_config import apikey


async def main():
    conn = aiohttp.TCPConnector(limit=None, ttl_dns_cache=300)
    session = aiohttp.ClientSession(connector=conn)
    urls = [summarize_url, ner_url, mcp_url]
    conc_req = 3

Async Requests in Python

Code Example:

import queue


def task1(name, s_queue):
    if s_queue.empty():
        print(f"Task {name} has nothing to do")
    else:
        while not s_queue.empty():
            cnt = s_queue.get()
            total = 0
            for x in range(cnt):
                print(f"Task {name} is working now.")
                total += 1
            print(f"Task {name} is working with a total of: {total}")


def s_async():
    s_queue = queue.Queue()
    for work in [2, 5, 10, 15, 20]:
        s_queue.put(work)
    tasks = [
        (task1, "Async1", s_queue),
        (task1, "Async2", s_queue),
        (task1, "Async3", s_queue),
    ]
    for t, n, q in tasks:
        t(n, q)


if __name__ == "__main__":
    s_async()

Output:

Task Async1 is running now.
Task Async1 is running with a total of: 2
Task Async1 is running now
...
Task Async1 is running now.
Task Async1 is running with a total of: 5
Task Async1 is running now
...
Task Async1 is running now.
Task Async1 is running with a total of: 10
Task Async1 is running now
...
Task Async1 is running now.
Task Async1 is running with a total of: 15
Task Async1 is running now
...
Task Async1 is running now.
Task Async1 is running with a total of: 20
Task Async3 has nothing to do