python requests timeout retry

Python Requests Timeout Retry

Python Requests is a popular library for making HTTP requests in Python. However, sometimes the server may fail to respond in time, leading to timeouts. In such situations, we may want to retry the request a certain number of times before giving up. This can be achieved using the timeout and retry features of the Requests library.

Timeout

The timeout feature allows us to set a timeout for the request. If the server fails to respond within this timeout, the request will be considered failed and an exception will be raised. To set a timeout, we can pass a tuple containing the timeout values for both the connection and read timeouts as an argument to the timeout parameter.

import requests

url = 'http://example.com'
timeout = (3, 5) # 3 seconds for connection timeout, 5 seconds for read timeout
response = requests.get(url, timeout=timeout)

print(response.status_code)

In the above example, we set a 3-second timeout for establishing the connection and a 5-second timeout for reading the response. If the server fails to respond within this time, a requests.exceptions.Timeout exception will be raised.

Retry

The retry feature allows us to retry a failed request a certain number of times before giving up. This can be useful in situations where the server is experiencing high traffic or is temporarily down. To use the retry feature, we need to install the requests_retry library.

pip install requests_retry

Once installed, we can use the Session object provided by requests_retry to make our requests. The Session object provides a retry method that takes an integer argument representing the maximum number of retries.

from requests_retry import Session

url = 'http://example.com'
session = Session()
session.retry = 3 # retry the request up to 3 times
response = session.get(url)

print(response.status_code)

In the above example, we create a Session object and set the maximum number of retries to 3. If the initial request fails, the session will automatically retry the request up to 3 times before giving up.

Combining Timeout and Retry

We can also combine the timeout and retry features to set a timeout for each retry. This can be useful in situations where the server may take longer to respond during high traffic periods.

from requests_retry import Session

url = 'http://example.com'
session = Session()
session.retry = 3 # retry the request up to 3 times
timeout = (3, 5) # 3 seconds for connection timeout, 5 seconds for read timeout
response = session.get(url, timeout=timeout)

print(response.status_code)

In the above example, we set a 3-second timeout for establishing the connection and a 5-second timeout for reading the response, and retry the request up to 3 times. If the server fails to respond within the timeout period, the request will be retried up to 3 times before giving up.