Menu
Article Troubleshooting

retry in python requests

Narkw Brahma

Narkw Brahma

• 2 min read

How to Retry in Python Requests?

If you are working with Python Requests to send HTTP requests and receive responses, you might have come across situations where the request fails due to network issues, server errors, or other reasons. In such cases, retrying the request can help to get the desired response. Here's how you can retry in Python Requests.

Using Retry

Python Requests provides a built-in Retry class that you can use to configure and perform retries on failed requests. Here's an example:


      import requests
      from requests.adapters import HTTPAdapter
      from requests.packages.urllib3.util.retry import Retry
      
      retries = Retry(total=5, backoff_factor=0.1, status_forcelist=[ 500, 502, 503, 504 ])
      adapter = HTTPAdapter(max_retries=retries)
      http = requests.Session()
      http.mount("https://", adapter)
      
      response = http.get("https://www.example.com")
      print(response.text)

In this example, we create a Retry object with a total of 5 retries and a backoff factor of 0.1 (i.e., the time delay between retries increases exponentially with each retry). We also specify a list of HTTP status codes that should trigger a retry (i.e., 500, 502, 503, 504). Then, we create an HTTPAdapter object with the max_retries parameter set to the Retry object, and mount it to our requests.Session() object. Finally, we send a GET request to the example URL and print the response.

Using Decorators

Another way to retry in Python Requests is to use the @retry decorator provided by the retrying library. Here's an example:


      import requests
      from retrying import retry
      
      @retry(stop_max_attempt_number=5, wait_fixed=1000, retry_on_result=lambda x: x is None)
      def get_example():
          response = requests.get("https://www.example.com")
          if response.status_code == 200:
              return response.text
          else:
              return None
      
      print(get_example())

In this example, we define a function get_example() that uses the @retry decorator with the following parameters:

  • stop_max_attempt_number: the maximum number of retries (5 in this case)
  • wait_fixed: the fixed time delay between retries (1000 milliseconds in this case)
  • retry_on_result=lambda x: x is None: the condition for retrying (i.e., retry if the result is None)

Inside the function, we send a GET request to the example URL and check if the status code is 200. If it is, we return the response text; otherwise, we return None. If the result is None, the decorator will retry the function until either the maximum number of retries is reached or a non-None result is returned.

Conclusion

Retrying in Python Requests can be useful in situations where requests fail due to network issues or server errors. By using the built-in Retry class or the @retry decorator, you can configure and perform retries on failed requests. Keep in mind that retrying too many times can cause performance issues and/or trigger rate limiting, so use it judiciously.

Share