Requests is the de-facto standard when it comes to making HTTP requests in Python, especially since Python 3. The open-source library abstracts away the complexity of managing network connections and make sending HTTP requests a breeze.
Most of you may already know how to send HTTP POST, GET, as well as other type of HTTP requests. In this article, we will guide you through how to set timeout in
requests, as well as managing exceptions.
Timeouts in Python requests
You can tell
requests library to stop waiting for a response after a given amount of time by passing a number to the
timeout parameter. If the
requests library does not receive response in
x seconds, it will raise a
It’s a best practice that production code should use this parameter in all network requests. Failure to do so can cause your program to hang indefinitely. If no
timeout is specified explicitly, requests do not time out.
requests.get('https://example.com/', timeout=10) # OUTPUT Traceback (most recent call last): File "<stdin>", line 1, in <module> requests.exceptions.Timeout: HTTPConnectionPool(host='example.com', port=80): Request timed out. (timeout=10)
Catch timeout exception
Once a timeout value has been set, every request that doesn’t receive a response in the specified timeframe will raise a
Timeout error. It’s important to handle this exception, otherwise your program would be terminated.
In order to catch
Timeout errors in
requests, you have to import the exception itself using
from requests.exceptions import Timeout.
import requests from requests.exceptions import Timeout try: requests.get('https://www.example.com, timeout=10) except Timeout: print('Timeout has been raised.')
Advanced timeout handling
Most requests to external servers should have a timeout attached, in case the server is not responding in a timely manner. By default, requests do not time out unless a timeout value is set explicitly. Without a timeout, your code may hang for minutes or more.
The connect timeout is the number of seconds Requests will wait for your client to establish a connection to a remote machine (corresponding to the connect()) call on the socket. It’s a good practice to set connect timeouts to slightly larger than a multiple of 3, which is the default TCP packet retransmission window.
Once your client has connected to the server and sent the HTTP request, the read timeout is the number of seconds the client will wait for the server to send a response. (Specifically, it’s the number of seconds that the client will wait between bytes sent from the server. In 99.9% of cases, this is the time before the server sends the first byte).
If you specify a single value for the timeout, like this:
r = requests.get('https://github.com', timeout=5)
The timeout value will be applied to both the
connect and the
read timeouts. Specify a tuple if you would like to set the values separately:
r = requests.get('https://github.com', timeout=(3.05, 27))
Request timeout in async coroutines
Since there are a lot of beginners asked how to use
requests in asynchronous programs or coroutines, we’ve made this section dedicated to answering just that.
requests is a blocking library, which means it should not be used in asynchronous coroutines. The proper libraries to use in this context is aiohttp and async-timeout. aiohttp is an asynchronous HTTP client/server framework, while async-timeout is a asyncio-compatible timeout context manager that can be used to wrap any coroutines inside a "timeout" context manager.