Menu
Article Pro Tips

set proxy in python requests

Narkw Brahma

Narkw Brahma

• 1 min read

How to Set Proxy in Python Requests?

If you want to access a website from a location where it is blocked, you can use a proxy server to bypass the restriction. Python Requests module provides an easy way to set proxies while making HTTP requests. In this article, we will see how to set proxy in Python Requests module using different methods.

Method 1: Using proxies parameter

You can pass the proxy details as a dictionary to the proxies parameter while making a request. The dictionary should contain the scheme (http, https, etc.) and the URL of the proxy server.


import requests

proxies = {
    'http': 'http://10.10.1.10:3128',
    'https': 'http://10.10.1.10:1080',
}

response = requests.get('https://example.com', proxies=proxies)

In the above example, we have set two proxies - one for HTTP and one for HTTPS protocol. Change the IP address and port number according to your proxy server configuration.

Method 2: Using environment variables

If you want to set a global proxy for all requests made by Python Requests module, you can use environment variables. Set the HTTP_PROXY and HTTPS_PROXY environment variables with the proxy details.


import os
import requests

os.environ['HTTP_PROXY'] = 'http://10.10.1.10:3128'
os.environ['HTTPS_PROXY'] = 'http://10.10.1.10:1080'

response = requests.get('https://example.com')

In the above example, we have set the HTTP_PROXY and HTTPS_PROXY environment variables to the proxy details. Now, all requests made by Python Requests module will use this proxy server.

Method 3: Using a Proxy Pool

If you want to use a pool of proxies instead of a single proxy server, you can use the Python Requests-ProxyPool library. This library provides an easy way to rotate proxies with each request.

First, install the library using pip:


pip install requests requests-proxy

Now, you can use the ProxyPool class to create a pool of proxies:


from proxy_requests import ProxyRequests

proxy_pool = ProxyRequests('http://10.10.1.10:3128', 'http://10.10.1.10:1080')

response = proxy_pool.get('https://example.com')

In the above example, we have created a pool of two proxies - one for HTTP and one for HTTPS protocol. Now, each request made by the ProxyRequests object will use a different proxy from the pool.

Share