Menu
Article Case Studies

using headers in python requests

Narkw Brahma

Narkw Brahma

• 2 min read

Using Headers in Python Requests

Headers are an essential part of HTTP requests and responses. They provide additional information about the request or response, such as the content type or the user agent. In Python, headers can be easily added to HTTP requests using the requests library.

Adding Headers to Requests

To add headers to a request, you need to create a dictionary with the header names and values, and pass it to the headers parameter of the requests functions. Here is an example:


import requests

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}
response = requests.get('https://www.example.com', headers=headers)

In this example, we create a dictionary with a "User-Agent" header that specifies the browser's details. We then pass this dictionary to the headers parameter of the get function.

Inspecting Response Headers

When you receive a response from a server, it usually includes its own headers. These headers can be accessed using the response.headers attribute, which returns a dictionary-like object containing the headers.


import requests

response = requests.get('https://www.example.com')
print(response.headers)

This will output all the response headers in a dictionary-like format.

Using Multiple Headers

You can add multiple headers to a request by adding them to the headers dictionary:


import requests

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3', 'Accept-Encoding': 'gzip, deflate, br'}
response = requests.get('https://www.example.com', headers=headers)

In this example, we add a second header, "Accept-Encoding," to the headers dictionary.

Using Session Objects

Requests provides a Session object that you can use to persist headers across multiple requests. This can be useful if you need to make several requests to the same server and want to use the same headers:


import requests

s = requests.Session()
s.headers.update({'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'})

response1 = s.get('https://www.example.com/page1')
response2 = s.get('https://www.example.com/page2')

In this example, we create a Session object and add a User-Agent header to it. We then use this Session object to make two requests to the same server.

Conclusion

Headers are an integral part of HTTP requests and responses, and Python's requests library makes it easy to add and manipulate headers in your code. Whether you need to set a specific user agent or accept encoding, adding headers can help you get the data you need from the web.

Share