Menu
Article Case Studies

python requests library logging

Narkw Brahma

Narkw Brahma

• 2 min read

Python Requests Library Logging

If you are working with the Python Requests library, you may find yourself needing to log information about the requests and responses that are being sent and received. Logging can be useful for debugging, troubleshooting, and monitoring purposes.

How to Enable Logging in Python Requests Library?

To enable logging in the Python Requests library, you need to import the logging module and configure a logger instance for the requests module. Here's an example:


import requests
import logging

logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

response = requests.get("https://www.example.com")
  • The first two lines of code import the requests and logging modules.
  • The basicConfig() method configures the root logger with default settings.
  • The getLogger() method retrieves a logger instance for the root logger.
  • The setLevel() method sets the logging level to DEBUG.
  • The getLogger() method retrieves a logger instance for the urllib3 package used by requests.
  • The setLevel() method sets the logging level to DEBUG.
  • The propagate attribute is set to True to allow log messages to propagate up to the root logger.
  • The final line sends a GET request to a sample URL.

The logging output will include information about the request and response, including the URL, status code, headers, and content. Here's an example:


DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): www.example.com:443
DEBUG:urllib3.connectionpool:https://www.example.com:443 "GET / HTTP/1.1" 200 1270
DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): www.example.com:443
DEBUG:requests.packages.urllib3.connectionpool:https://www.example.com:443 "GET / HTTP/1.1" 200 1270

Alternative Method to Enable Logging in Python Requests Library

Another way to enable logging in the Python Requests library is to use a custom logging handler. Here's an example:


import requests
import logging

class RequestsLoggingHandler(logging.Handler):
    def emit(self, record):
        if record.name.startswith("requests"):
            print(record.message)

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logger.addHandler(RequestsLoggingHandler())

response = requests.get("https://www.example.com")
  • The first two lines of code import the requests and logging modules.
  • A custom logging handler is defined that checks if the logged record is from the requests module.
  • The getLogger() method retrieves a logger instance for the root logger.
  • The setLevel() method sets the logging level to DEBUG.
  • The custom logging handler is added to the logger using the addHandler() method.
  • The final line sends a GET request to a sample URL.

The logging output will include only log messages from the requests module. Here's an example:


DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): www.example.com:443
DEBUG:requests.packages.urllib3.connectionpool:https://www.example.com:443 "GET / HTTP/1.1" 200 1270
Share