Menu
Article Troubleshooting

python requests with proxy

Narkw Brahma

Narkw Brahma

• 2 min read

Python Requests with Proxy

If you are working with web scraping or API calls, having a reliable proxy is essential. Python Requests is a popular library for making HTTP requests in Python, and it also provides support for using proxies. In this article, we will see how to use Python Requests with a proxy.

Using a Proxy with Python Requests

To use a proxy with Python Requests, you need to specify the proxy server's address while making the request. Here's an example:


import requests

proxies = {
    'http': 'http://10.10.1.10:3128',
    'https': 'https://10.10.1.11:1080',
}

response = requests.get('http://example.com', proxies=proxies)

print(response.content)

In the above code, we have specified two proxies - one for HTTP requests and one for HTTPS requests. You can also specify the proxy separately for each request by passing the proxy information as a parameter to the request method:


import requests

http_proxy = "http://10.10.1.10:3128"
https_proxy = "https://10.10.1.11:1080"

http_response = requests.get('http://example.com', proxies={'http': http_proxy})
https_response = requests.get('https://example.com', proxies={'https': https_proxy})

print(http_response.content)
print(https_response.content)

In this example, we have used two different proxies for HTTP and HTTPS requests.

Authentication with Proxies

If your proxy requires authentication, you can provide the username and password in the proxy URL. Here's an example:


import requests

proxies = {
    'http': 'http://username:[email protected]:3128',
    'https': 'https://username:[email protected]:1080',
}

response = requests.get('http://example.com', proxies=proxies)

print(response.content)

In the above code, we have specified the username and password in the proxy URL for both HTTP and HTTPS requests.

Handling Proxy Errors

Sometimes, the proxy may not be available or may return an error. In such cases, Python Requests will raise a `ProxyError` exception. You can catch this exception and handle it gracefully:


import requests

proxies = {
    'http': 'http://10.10.1.10:3128',
}

try:
    response = requests.get('http://example.com', proxies=proxies)
    print(response.content)
except requests.exceptions.ProxyError as e:
    print("Error connecting to proxy:", e)

In the above code, we have caught the `ProxyError` exception and printed an error message.

Conclusion

Using a proxy with Python Requests is simple and can be done by specifying the proxy server's address while making the request. If your proxy requires authentication, you can provide the username and password in the proxy URL. Handle any proxy errors to ensure that your script runs smoothly.

Share