Menu
Article Bugs

python requests post large file

Narkw Brahma

Narkw Brahma

• 1 min read

How to Post Large Files Using Python Requests Library

As a web developer, I have encountered a situation where I needed to upload large files using Python requests library. After some research and testing, I came up with multiple ways to post large files using Python requests library.

Method 1: Streaming Large Files

Streaming large files can be a good option when you don't want to load the entire file into memory. You can stream the file in chunks and send it to the server using the requests library.


import requests

url = 'https://example.com/upload'
data = {'file': ('large_file', open('large_file.mp4', 'rb'))}

response = requests.post(url, data=data, stream=True)

if response.status_code == 200:
    print('Success')
else:
    print('Error')

In the above code, we are opening the file in binary mode and passing it in the data parameter. We also set the stream parameter to True so that we can stream the file in chunks.

Method 2: Multipart POST Request

You can also use multipart POST request to upload large files using Python requests library. This method is easy to implement and does not require streaming the file.


import requests

url = 'https://example.com/upload'
files = {'file': open('large_file.mp4', 'rb')}

response = requests.post(url, files=files)

if response.status_code == 200:
    print('Success')
else:
    print('Error')

In the above code, we are passing the file in the files parameter. Requests library automatically sets the content type to multipart/form-data.

Method 3: Chunked Encoding

You can also use chunked encoding to upload large files using Python requests library. This method allows you to break the file into smaller chunks and send them one by one.


import requests

url = 'https://example.com/upload'
chunk_size = 1024 * 1024
with open('large_file.mp4', 'rb') as f:
    while True:
        chunk = f.read(chunk_size)
        if not chunk:
            break
        response = requests.post(url, data=chunk)

if response.status_code == 200:
    print('Success')
else:
    print('Error')

In the above code, we are reading the file in chunks of 1 MB and sending them one by one using requests.post() method.

These are the three methods that you can use to upload large files using Python requests library. Choose the method that suits your requirements and implement it in your project.

Share