Requests is the de-facto standard when it comes to making HTTP requests in Python, especially since Python 3. The open-source library abstracts away the complexity of managing network connections and make sending HTTP requests a breeze.
Most of you may already know how to send HTTP POST, GET, as well as other type of HTTP requests. In this article, we will guide you through how to set timeout in requests, as well as managing exceptions.
Timeouts in Python requests
You can tell requests library to stop waiting for a response after a given amount of time by passing a number to the timeout parameter. If the requests library does not receive response in x seconds, it will raise a Timeout error.
It’s a best practice that production code should use this parameter in all network requests. Failure to do so can cause your program to hang indefinitely. If no timeout is specified explicitly, requests do not time out.
requests.get('https://example.com/', timeout=10)
# OUTPUT
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
requests.exceptions.Timeout: HTTPConnectionPool(host='example.com', port=80): Request timed out. (timeout=10)Code language: PHP (php)Catch timeout exception
Once a timeout value has been set, every request that doesn’t receive a response in the specified timeframe will raise a Timeout error. It’s important to handle this exception, otherwise your program would be terminated.
In order to catch Timeout errors in requests, you have to import the exception itself using from requests.exceptions import Timeout.
import requests
from requests.exceptions import Timeout
try:
requests.get('https://www.example.com, timeout=10)
except Timeout:
print('Timeout has been raised.')Code language: PHP (php)Advanced timeout handling
Most requests to external servers should have a timeout attached, in case the server is not responding in a timely manner. By default, requests do not time out unless a timeout value is set explicitly. Without a timeout, your code may hang for minutes or more.
The connect timeout is the number of seconds Requests will wait for your client to establish a connection to a remote machine (corresponding to the connect()) call on the socket. It’s a good practice to set connect timeouts to slightly larger than a multiple of 3, which is the default TCP packet retransmission window.
Once your client has connected to the server and sent the HTTP request, the read timeout is the number of seconds the client will wait for the server to send a response. (Specifically, it’s the number of seconds that the client will wait between bytes sent from the server. In 99.9% of cases, this is the time before the server sends the first byte).
If you specify a single value for the timeout, like this:
r = requests.get('https://github.com', timeout=5)Code language: JavaScript (javascript)The timeout value will be applied to both the connect and the read timeouts. Specify a tuple if you would like to set the values separately:
r = requests.get('https://github.com', timeout=(3.05, 27))Code language: JavaScript (javascript)Request timeout in async coroutines
Since there are a lot of beginners asked how to use requests in asynchronous programs or coroutines, we’ve made this section dedicated to answering just that.
requests is a blocking library, which means it should not be used in asynchronous coroutines. The proper libraries to use in this context is aiohttp and async-timeout. aiohttp is an asynchronous HTTP client/server framework, while async-timeout is a asyncio-compatible timeout context manager that can be used to wrap any coroutines inside a “timeout” context manager.
