Menu
Article Troubleshooting

python requests retry

Narkw Brahma

Narkw Brahma

• 2 min read

Python Requests Retry

Python Requests is one of the most commonly used libraries for making HTTP requests in Python. Sometimes, when we make a request to a server, we might not get a response due to network connectivity issues or server errors. In such cases, we might want to retry the request multiple times to ensure that we eventually get a response.

Fortunately, Python Requests provides us with the ability to retry requests using the retrying library. This library allows us to configure retry strategies for our requests.

Using Retry Strategy

To use the retry strategy in Python Requests, we first need to install the retrying library using pip:

pip install retrying

Once installed, we can use it in our code as follows:

import requests
from retrying import retry

@retry(stop_max_attempt_number=3, wait_fixed=1000)
def make_request_with_retry():
    response = requests.get('https://example.com')
    return response

response = make_request_with_retry()

In this code, we have defined a function named make_request_with_retry() that makes a GET request to 'https://example.com'. The @retry decorator above this function configures the retry strategy with the following options:

  • stop_max_attempt_number=3: Number of times to retry the request (in this case, 3 times)
  • wait_fixed=1000: Number of milliseconds to wait between retries (in this case, 1 second)

This means that if the request fails, it will be retried up to 3 times, with a 1-second delay between each retry.

There are other options available for configuring the retry strategy, such as stop_max_delay, wait_random_min, and wait_random_max. You can find more information about these options in the retrying documentation.

Using Retry Adapter

Another way to implement retrying in Python Requests is to use the built-in Retry adapter. This adapter provides us with more control over the retrying process.

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

retry_strategy = Retry(
    total=3,
    backoff_factor=1,
    status_forcelist=[ 500, 502, 503, 504 ]
)

adapter = HTTPAdapter(max_retries=retry_strategy)
http = requests.Session()
http.mount('https://', adapter)
http.mount('http://', adapter)

response = http.get('https://example.com')

In this code, we have defined a retry strategy using the Retry class. The options passed to this class include:

  • total=3: Number of times to retry the request (in this case, 3 times)
  • backoff_factor=1: Factor to apply to the delay between retries (in this case, the delay will be multiplied by 1, 2, and 4 for each retry)
  • status_forcelist=[ 500, 502, 503, 504 ]: List of HTTP status codes that should trigger a retry

We then create an instance of the HTTPAdapter class and pass our retry strategy to it. We then create a Session object and mount our adapter on it for both HTTP and HTTPS requests.

Finally, we make our request using the get() method of our Session object.

Using the retry adapter gives us more fine-grained control over the retrying process, and can be useful in cases where we need to modify the retry strategy depending on the outcome of previous retries.

Share