Send a HTTP POST request with Python requests library

requests is an popular, simple yet powerful HTTP library for Python. It allows you to easily prepare and send complex HTTP requests in a few lines of code. Using requests library is a popular way to abstract away the complexity of managing query strings, arguments, form-encode POST data, and many other things when it comes to making HTTP requests.

In this article, we will show you a few ways to send a HTTP POST request in Python requests library, from the simple one to the more complex requests including multipart POST data.

Send a simple POST in Python requests

A POST request is typically sent via an HTML form and results in a change on the server. HTTP POST method is often used when submitting login or contact forms or uploading files and images to the server.

In order to send a simple HTTP POST request with some form-encoded data, use the post method with data argument.

post_data = {'key':'value'}
response = requests.post('https://httpbin.org/post', data=post_data)
print(response)

# OUTPUT
<Response [200]>

In the code snippet above, post_data is a Python dictionary contains all the POST data that you want to send out.

Send POST request with complex HTML form-encoded data

If you want to emulate a request sent by a HTML form, which contains form-encoded data, you can pass the data argument just like the example above. The data argument can also take in a tuple-based payload or a dictionary-based one with multiple elements in the same key.

payload_tuples = [('key1', 'value1'), ('key1', 'value2')]
r1 = requests.post('https://httpbin.org/post', data=payload_tuples)
payload_dict = {'key1': ['value1', 'value2']}
r2 = requests.post('https://httpbin.org/post', data=payload_dict)

print(r1.text == r2.text)
# OUTPUT
True

In the example above, the two tuple-based and dictionary-based payload does the same thing, which is sending a form-encoded data that use the same key for multiple field.

Send POST request with JSON data as string

There are times that you may want to send a JSON as string to a server via a POST request. For example, a few API only accepts complex data to be able to simplify the request body. If you pass in a string instead of a dict, the JSON data will be posted directly to the URL. In this example, we manually encode the JSON data to string with Python json library.

import json

url = 'https://api.example.com/endpoint'
payload = {'some': 'data'}

response = requests.post(url, data=json.dumps(payload))

If you’re using a more recent version of requests (>2.4.2), you can pass a json argument directly instead of manually encoding the dict.

url = 'https://api.github.com/some/endpoint'
payload = {'some': 'data'}

response = requests.post(url, json=payload)

Please note that the only difference between data and json argument is that the json will set the Content-Type in the header to application/json, while POST requests sent with data will have Content-Type: application/x-www-form-urlencoded header.

The json parameter would be ignored if either data or files is passed.

Send Multipart-Encoded POST request

A HTTP multipart request is a HTTP request that HTTP clients construct to send files and data over to a HTTP Server. It is commonly used by browsers and HTTP clients to upload files to the server. Those files can be photos, music, binary, text or a combination of them.

If you didn’t know how it would look like in raw form, here it is.

POST /test HTTP/1.1
Host: example
User-Agent: Mozilla/5.0 Gecko/2009042316 Firefox/3.0.10
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://example/example.htm
Content-Type: multipart/form-data; boundary=2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f
Content-Length: 514

--2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f
Content-Disposition: form-data; name="datafile1"; filename="a.gif"
Content-Type: image/gif

GIF87a.............,...........D..;
--2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f
Content-Disposition: form-data; name="datafile2"; filename="b.gif"
Content-Type: image/gif

GIF87a.............,...........D..;
--2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f
Content-Disposition: form-data; name="datafile3"; filename="c.gif"
Content-Type: image/gif

GIF87a.............,...........D..;
--2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f--

As you can see, the multipart data is big, and contains many files inside as well as their filenames. In requests, multipart-encoded POST request can be easily made using files argument.

url = 'https://httpbin.org/post'
files = {'datafile1': open('a.gif', 'rb')}

r = requests.post(url, files=files)
r.text

# OUTPUT
{
  ...
  "files": {
    "datafile1": "<binary_data>"
  },
  ...
}

Please note that we opened the file in binary mode. I is strongly recommended that you open files in binary mode because requests attempt to automatically provide the Content-Length header for you. This value should be set to the number of bytes in the file, not the number of text characters in the string version of it. Errors may occur if you open the file in text mode.

If you need to send filenames, content types and other file details as well, use a 3-element tuple as value in the files dict. The tuple order should be (filename, file_object, content_type, expiration).

url = 'https://httpbin.org/post'
file1_detail = ('a.gif', open('a.gif', 'rb'), 'image/gif', {'Expires': '0'})
files = {'datafile1': file1_detail}

r = requests.post(url, files=files)
r.text

# OUTPUT
{
  ...
  "files": {
    "datafile1": "<binary_data>"
  },
  ...
}

Send multiple files in a Multipart-Encoded POST request

You can send more than one file in a single POST with requests library as well. It’s a little different than when you send a single file. In this case, the files dictionary should be a list of 2-element tuples following (form_field_name, file_info) syntax, where file_info should be a 3-element tuple that contains file details.

url = 'https://httpbin.org/post'
multiple_files = [
...     ('images', ('1.png', open('1.png', 'rb'), 'image/png')),
...     ('images', ('2.png', open('2.png', 'rb'), 'image/png'))]
r = requests.post(url, files=multiple_files)
r.text

# OUTPUT
{
  ...
  'files': {'images': 'data:image/png;base64,iVBORw ....'}
  'Content-Type': 'multipart/form-data; boundary=3131623adb2043caaeb5538cc7aa0b3a',
  ...
}

Files should be opened in binary mode, otherwise Content-Length header can be set to an incorrect value and causing errors.

Send Large Multipart-Encoded POST request

There are times that your files is relatively big, you may want to stream the request instead of reading all of its contents into memory.

By default, requests does not support this, but there is a separate package which does – requests-toolbelt.

This use case is pretty rare, but if you want to find out more about it, you should read the toolbelt’s documentation. There are a section dedicated to Uploading Data via HTTP POST request, which contains tutorials on Streaming large multipart data and monitoring that process.