Menu
Article Security Check

python requests library retry

Narkw Brahma

Narkw Brahma

• 1 min read

Python Requests Library Retry: How to Retry Requests in Python?

Python Requests library is a widely used library for making HTTP requests in Python. It is a powerful library that supports a lot of features such as HTTP/1.1, authentication, cookies, and much more. However, sometimes the server may not respond to the request, or the request may fail due to network issues. In such cases, it is essential to retry the request to get the desired response.

Retrying Requests with Retry Library

One way to retry requests in Python is to use the Retry library. This library provides a Retry object that can be used to configure the retry behavior.


import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

retry_strategy = Retry(
    total=3,
    backoff_factor=0.1,
    status_forcelist=[ 500, 502, 503, 504 ]
)

adapter = HTTPAdapter(max_retries=retry_strategy)
http = requests.Session()
http.mount("https://", adapter)
http.mount("http://", adapter)

response = http.get(url)

Here we have created a Retry object with a total of 3 retries and a backoff factor of 0.1. The status_forcelist parameter specifies which HTTP status codes should trigger a retry. In this case, we are retrying on status codes 500, 502, 503, and 504. We then create an HTTPAdapter object with the retry strategy and mount it to our HTTP session object. Finally, we make the request using our HTTP session object.

Retrying Requests with Exponential Backoff

Another way to retry requests in Python is to use exponential backoff. Exponential backoff is a retry strategy where the time between retries increases exponentially with each retry. This helps to avoid overloading the server with too many requests in a short period.


import requests
import time

def request_with_retry(url, retries=3):
    for i in range(retries):
        try:
            response = requests.get(url)
            response.raise_for_status()
            return response
        except requests.exceptions.RequestException as e:
            if i == retries - 1:
                raise e
            wait_time = 2 ** i
            print(f"Request failed, retrying in {wait_time} seconds...")
            time.sleep(wait_time)

response = request_with_retry(url)

In this code example, we define a request_with_retry function that takes a URL and a number of retries as input. We then loop through the retries and make the request using the requests library. If the request fails, we wait for a certain amount of time (which increases exponentially with each retry) before retrying the request. If all retries fail, we raise the original exception.

Share