Menu
Article Security Check

how to pass proxy in requests python

Narkw Brahma

Narkw Brahma

• 2 min read

How to Pass Proxy in Requests Python

If you are working with web scraping or automation using Python, you might come across a situation where you need to use a proxy server to make requests. In such situations, the requests module in Python can be used to handle HTTP requests with ease. However, it is essential to pass the proxy information correctly to make the requests through the proxy server. In this article, we will discuss how to pass proxy in requests python using different methods.

Method 1: Passing Proxy as a Dictionary

The simplest way to use a proxy with requests is by passing a dictionary with the proxy information while making the request. The proxy dictionary should contain the following keys:

  • http: The proxy URL for HTTP requests.
  • https: The proxy URL for HTTPS requests.

The following code shows how to pass a proxy as a dictionary while making a request:


import requests

# Proxy dictionary
proxy_dict = {'http': 'http://:', 'https': 'https://:'}

# Request headers
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}

# Make request through proxy
response = requests.get('https://www.example.com', headers=headers, proxies=proxy_dict)

# Print response
print(response.content)

Method 2: Passing Proxy as an Environment Variable

Another way to use a proxy with requests is by setting the proxy information as an environment variable. Setting an environment variable makes it easier to switch between different proxy servers without modifying the code.

The environment variable that needs to be set for the proxy is:

  • HTTP_PROXY: The proxy URL for HTTP requests.
  • HTTPS_PROXY: The proxy URL for HTTPS requests.

The following code shows how to set the environment variable and make a request through the proxy:


import os
import requests

# Set environment variables
os.environ['HTTP_PROXY'] = 'http://:'
os.environ['HTTPS_PROXY'] = 'https://:'

# Request headers
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}

# Make request through proxy
response = requests.get('https://www.example.com', headers=headers)

# Print response
print(response.content)

Method 3: Passing Proxy with Authentication

If your proxy server requires authentication, you need to pass the authentication details along with the proxy URL. You can pass the authentication details in the proxy dictionary or set them as environment variables.

The authentication details that need to be passed are:

  • username: The username for authentication.
  • password: The password for authentication.

The following code shows how to pass the authentication details along with the proxy URL while making a request:


import requests

# Proxy dictionary with authentication
proxy_dict = {'http': 'http://:@:', 'https': 'https://:@:'}

# Request headers
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}

# Make request through proxy
response = requests.get('https://www.example.com', headers=headers, proxies=proxy_dict)

# Print response
print(response.content)

Alternatively, you can set the authentication details as environment variables and use them while making the request:


import os
import requests

# Set environment variables
os.environ['HTTP_PROXY'] = 'http://:@:'
os.environ['HTTPS_PROXY'] = 'https://:@:'

# Request headers
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}

# Make request through proxy
response = requests.get('https://www.example.com', headers=headers)

# Print response
print(response.content)

These are the different methods to pass proxy in requests python. Choose the one that suits your use case and ensure that the proxy information is correct to avoid any errors while making the requests.

Share