Summary
requests-cache is a persistent HTTP cache that provides an easy way to get better
performance with the python requests library.
Complete project documentation can be found at requests-cache.readthedocs.io.
Features
- 🍰 Ease of use: Keep using the
requestslibrary you're already familiar with. Add caching
with a drop-in replacement
forrequests.Session, or
install globally
to add transparent caching to allrequestsfunctions. - 🚀 Performance: Get sub-millisecond response times for cached responses. When they expire, you
still save time with
conditional requests. - 💾 Persistence: Works with several
storage backends
including SQLite, Redis, MongoDB, and DynamoDB; or save responses as plain JSON files, YAML,
and more - 🕗 Expiration: Use
Cache-Control
and other standard HTTP headers, define your own expiration schedule, keep your cache clutter-free
with backends that natively support TTL, or any combination of strategies - ⚙️ Customization: Works out of the box with zero config, but with a robust set of features for
configuring and extending the library to suit your needs - 🧩 Compatibility: Can be combined with other
popular libraries based on requests
Quickstart
First, install with pip:
pip install requests-cache
Then, use requests_cache.CachedSession
to make your requests. It behaves like a normal
requests.Session,
but with caching behavior.
To illustrate, we'll call an endpoint that adds a delay of 1 second, simulating a slow or
rate-limited website.
This takes 1 minute:
import requests
session = requests.Session()
for i in range(60):
session.get('https://httpbin.org/delay/1')
This takes 1 second:
import requests_cache
session = requests_cache.CachedSession('demo_cache')
for i in range(60):
session.get('https://httpbin.org/delay/1')
With caching, the response will be fetched once, saved to demo_cache.sqlite, and subsequent
requests will return the cached response near-instantly.
Patching
If you don't want to manage a session object, or just want to quickly test it out in your
application without modifying any code, requests-cache can also be installed globally, and all
requests will be transparently cached:
import requests
import requests_cache
requests_cache.install_cache('demo_cache')
requests.get('https://httpbin.org/delay/1')
Headers and Expiration
By default, requests-cache will keep cached responses indefinitely. In most cases, you will want to
use one of the two following strategies to balance cache freshness and performance:
Define exactly how long to keep responses:
Use the expire_after parameter to set a fixed expiration time for all responses:
from requests_cache import CachedSession
from datetime import timedelta
# Keep responses for 360 seconds
session = CachedSession('demo_cache', expire_after=360)
# Or use timedelta objects to specify other units of time
session = CachedSession('demo_cache', expire_after=timedelta(hours=1))
See Expiration for
more features and settings.
Use Cache-Control headers:
Use the cache_control parameter to enable automatic expiration based on Cache-Control and other
standard HTTP headers sent by the server:
from requests_cache import CachedSession
session = CachedSession('demo_cache', cache_control=True)
See Cache Headers
for more details.
Settings
The default settings work well for most use cases, but there are plenty of ways to customize
caching behavior when needed. Here is a quick example of some of the options available:
from datetime import timedelta
from requests_cache import CachedSession
session = CachedSession(
'demo_cache',
use_cache_dir=True, # Save files in the default user cache dir
cache_control=True, # Use Cache-Control response headers for expiration, if available
expire_after=timedelta(days=1), # Otherwise expire responses after one day
allowable_codes=[200, 400], # Cache 400 responses as a solemn reminder of your failures
allowable_methods=['GET', 'POST'], # Cache whatever HTTP methods you want
ignored_parameters=['api_key'], # Don't match this request param, and redact if from the cache
match_headers=['Accept-Language'], # Cache a different response per language
stale_if_error=True, # In case of request errors, use stale cache data if possible
)
Next Steps
To find out more about what you can do with requests-cache, see: