proxy in python request

Proxy in Python Request

If you are using Python to make HTTP requests, you may need to use a proxy server to access certain websites. A proxy server acts as an intermediary between your computer and the website you are requesting data from. It can help you bypass firewalls or hide your IP address.

Using Requests Library with Proxy

The most popular library for making HTTP requests in Python is requests. You can use it easily with a proxy server by passing the proxy URL as a parameter to the request function. Here is an example:

import requests

proxies = {
  "http": "http://10.10.1.10:3128",
  "https": "https://10.10.1.10:1080",
}

response = requests.get("https://www.example.com", proxies=proxies)

print(response.content)

In this example, we created a dictionary of proxies with keys http and https. The values are the URLs of the proxy servers we want to use. We passed this dictionary to the proxies parameter of the get function.

If your proxy server requires authentication, you can pass the username and password in the URL like this:

import requests

proxies = {
  "http": "http://user:[email protected]:3128",
  "https": "https://user:[email protected]:1080",
}

response = requests.get("https://www.example.com", proxies=proxies)

print(response.content)

Using urllib Library with Proxy

If you prefer to use the built-in urllib library instead of requests, you can still use a proxy server. Here is an example:

import urllib.request

proxy_handler = urllib.request.ProxyHandler({
    "http": "http://10.10.1.10:3128",
    "https": "https://10.10.1.10:1080",
})

opener = urllib.request.build_opener(proxy_handler)
response = opener.open("https://www.example.com")

print(response.read())

In this example, we created a ProxyHandler object with a dictionary of proxies and passed it to the build_opener function. We then used the open function of the resulting object to make the request.

Conclusion

Using a proxy server in Python can be very useful in certain situations. Whether you use requests or urllib, it is easy to configure your HTTP requests to go through a proxy server.