Menu
Article Features

python requests module with proxy

Narkw Brahma

Narkw Brahma

• 2 min read

Python Requests Module with Proxy

If you're working with Python and making HTTP requests, you may need to use a proxy to access certain resources. The Requests module is a popular library for making HTTP requests in Python, and it supports using a proxy through its proxies parameter.

Using a Proxy with Requests

To use a proxy with the Requests module, you need to pass a dictionary of proxy settings to the proxies parameter of the HTTP request method. The dictionary should contain the protocol (e.g. http or https) and the URL of the proxy server.


import requests

# Define proxy settings
proxies = {
    'http': 'http://username:[email protected]:port',
    'https': 'https://username:[email protected]:port'
}

# Make HTTP request using proxy
response = requests.get('https://example.com', proxies=proxies)

In this example, we've defined a dictionary of proxy settings with the protocol (http or https), the URL of the proxy server, and the username and password if required. We then pass this dictionary to the proxies parameter of the Requests HTTP request method.

Using a Proxy with Authentication

If your proxy server requires authentication, you can include the username and password in the URL of the proxy server in the proxies dictionary. Alternatively, you can use the auth parameter of the HTTP request method to pass in the authentication credentials separately.


import requests

# Define proxy settings with authentication
proxies = {
    'http': 'http://proxy-server.com:port',
    'https': 'https://proxy-server.com:port'
}

# Define authentication credentials
auth = requests.auth.HTTPProxyAuth('username', 'password')

# Make HTTP request using proxy with authentication
response = requests.get('https://example.com', proxies=proxies, auth=auth)

In this example, we've defined a dictionary of proxy settings with the protocol (http or https), the URL of the proxy server, and the port number. We then define the authentication credentials using the requests.auth.HTTPProxyAuth class and pass them to the auth parameter of the Requests HTTP request method.

Using a Proxy with Environment Variables

If you prefer to use environment variables to set your proxy settings, you can use the os.environ dictionary to get the values and pass them to the proxies parameter of the HTTP request method.


import os
import requests

# Get proxy settings from environment variables
http_proxy = os.environ.get('http_proxy')
https_proxy = os.environ.get('https_proxy')

# Define proxy settings dictionary
proxies = {
    'http': http_proxy,
    'https': https_proxy
}

# Make HTTP request using proxy with environment variables
response = requests.get('https://example.com', proxies=proxies)

In this example, we're using the os.environ dictionary to get the values of the http_proxy and https_proxy environment variables. We then define a dictionary of proxy settings with these values and pass it to the proxies parameter of the Requests HTTP request method.

Share