Menu
Article Troubleshooting

python requests retry example

Narkw Brahma

Narkw Brahma

• 1 min read

hljs.highlightAll();

Python Requests Retry Example

As a web developer, I have faced situations where I need to retry a request multiple times due to various reasons such as network errors or server issues. In such cases, Python Requests library provides a simple way to retry requests.

Retry with Maximum Number of Attempts

One way to retry a request is to use the requests.Session() object and call its mount() method to associate a Retry() object with the HTTP adapter.


import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

session = requests.Session()
retry = Retry(total=5, backoff_factor=0.1)
adapter = HTTPAdapter(max_retries=retry)
session.mount('http://', adapter)
session.mount('https://', adapter)

response = session.get('https://example.com')
print(response.status_code)

The above code will make a GET request to https://example.com and retry the request up to 5 times with an exponential backoff factor of 0.1 seconds between retries. If the maximum number of retries is exceeded, then the last error will be raised.

Retry with Custom Conditions

We can also customize the retry conditions based on the response status code or the exception raised. For example, we can retry only if the response status code is a server error (5xx) or a connection error (such as ConnectionError or Timeout).


import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

def retry_on_status_code(status_code):
    return 500 <= status_code <= 599

def retry_on_exception(exception):
    return isinstance(exception, (requests.exceptions.ConnectionError, requests.exceptions.Timeout))

session = requests.Session()

retry = Retry(total=5, backoff_factor=0.1, status_forcelist=[429, 500, 502, 503, 504], method_whitelist=["HEAD", "GET", "OPTIONS"])
retry.retry_on_status = retry_on_status_code
retry.retry_on_exception = retry_on_exception

adapter = HTTPAdapter(max_retries=retry)
session.mount('http://', adapter)
session.mount('https://', adapter)

response = session.get('https://example.com')
print(response.status_code)

In the above code, we define two custom functions retry_on_status_code() and retry_on_exception() to check whether to retry or not. We also specify a few conditions in the Retry() object such as retry only for server errors (5xx), and retry only for the specified HTTP methods. We then create an HTTP adapter and mount it to the session object.

By using these techniques, we can handle situations where a request fails due to various reasons and make sure that our application is resilient to such failures.

Share