Skip to content

Commit cc45c11

Browse files
docs: document cache policy, scopes and pruning
Three new sections in the cache how-to - scopes, where the cache is stored, and listing/pruning - plus the cache_policy confval, the PYTEST_CACHE_POLICY, PYTEST_CACHE_HOME and PYTEST_HYPERLINKS environment variables, the two new options, and CacheScope in the API reference. The sample --cache-list output is a literal code block rather than regendoc output, since it is full of machine-specific paths. The regendoc-captured --help and ini listings are updated; nothing else moved, because the default resolved location is unchanged. One caveat is called out because it is not obvious and does bite: a project on an unmounted volume looks `orphaned`, which is the main reason pruning is never automatic. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 85000c8 commit cc45c11

2 files changed

Lines changed: 235 additions & 3 deletions

File tree

‎doc/en/how-to/cache.rst‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -332,6 +332,158 @@ servers where isolation and correctness is more important
332332
than speed.
333333

334334

335+
.. _cache_scopes:
336+
337+
Cache scopes
338+
------------
339+
340+
.. versionadded:: 9.2
341+
342+
Not everything in the cache is equally portable. Which tests exist, and which are skipped, depends on the
343+
interpreter running them, so the state behind :option:`--lf`, :option:`--nf` and :option:`--sw <--sw>` is
344+
only meaningful for the environment that recorded it.
345+
346+
pytest keeps such values apart *within* a single cache directory, rather than needing one cache directory
347+
per environment. Every cached value has a :class:`~pytest.CacheScope`:
348+
349+
.. list-table::
350+
:header-rows: 1
351+
352+
* - Scope
353+
- Valid for
354+
* - :attr:`CacheScope.SHARED <pytest.CacheScope.SHARED>`
355+
- the project, whatever runs it. The default.
356+
* - :attr:`CacheScope.PYTHON <pytest.CacheScope.PYTHON>`
357+
- one Python implementation and ``major.minor`` version
358+
* - :attr:`CacheScope.ENV <pytest.CacheScope.ENV>`
359+
- one environment, i.e. one :data:`sys.prefix`
360+
361+
``--lf``, ``--nf`` and ``--sw`` use ``ENV``. Running a project under a tox or nox matrix, or simply under
362+
two virtualenvs, therefore no longer has each run overwrite the previous one's last-failed set.
363+
364+
Plugins can do the same:
365+
366+
.. code-block:: python
367+
368+
def pytest_configure(config):
369+
# Valid anywhere the project is.
370+
config.cache.set("myplugin/schema-version", 3)
371+
372+
# Only valid for the environment that collected it.
373+
config.cache.set("myplugin/collected", ids, scope=pytest.CacheScope.ENV)
374+
375+
Reads must use the same scope they were written with. ``SHARED`` is the default, so existing plugins keep
376+
working and keep their existing on-disk location.
377+
378+
379+
.. _cache_location:
380+
381+
Where the cache is stored
382+
-------------------------
383+
384+
.. versionadded:: 9.2
385+
386+
By default the cache lives in ``.pytest_cache`` inside the :ref:`rootdir <rootdir>`. The
387+
:confval:`cache_policy` option chooses somewhere else by name:
388+
389+
.. code-block:: ini
390+
391+
[pytest]
392+
cache_policy = user
393+
394+
``local``
395+
``<rootdir>/.pytest_cache``. The default.
396+
397+
``user``
398+
A directory keyed by project inside the platform's user cache directory - ``$XDG_CACHE_HOME/pytest``
399+
or ``~/.cache/pytest`` on Linux, ``~/Library/Caches/pytest`` on macOS, ``%LOCALAPPDATA%\pytest\Cache``
400+
on Windows. Nothing at all is written into the project.
401+
402+
This requires the ``xdg`` extra::
403+
404+
pip install pytest[xdg]
405+
406+
:confval:`cache_dir` remains available and always wins: it is an explicit path, while ``cache_policy``
407+
only chooses a location when ``cache_dir`` is unset. Use it for anywhere the two policies do not name -
408+
it expands environment variables, so a cache inside the current virtualenv is:
409+
410+
.. code-block:: ini
411+
412+
[pytest]
413+
cache_dir = $VIRTUAL_ENV/.pytest_cache
414+
415+
To opt in for a whole machine without editing every project, set the environment variable instead:
416+
417+
.. code-block:: bash
418+
419+
export PYTEST_CACHE_POLICY=user
420+
421+
An explicit ``cache_policy`` in a config file still wins over it.
422+
423+
Under the ``user`` policy the directory is named after the project and a digest of its path, so it stays
424+
recognisable::
425+
426+
~/.cache/pytest/myproject-1a2b3c4d5e6f7a8b/
427+
428+
The key is the rootdir alone - not the interpreter, which is handled by :ref:`cache scopes <cache_scopes>`
429+
instead. One project therefore gets one cache directory however many environments run it. A *new*
430+
directory only appears if the project itself moves.
431+
432+
433+
.. _cache_pruning:
434+
435+
Listing and pruning caches
436+
--------------------------
437+
438+
.. versionadded:: 9.2
439+
440+
A cache stored inside the project is deleted along with the project. One stored under the ``user`` policy
441+
is not, so pytest can tell you what has accumulated:
442+
443+
.. code-block:: bash
444+
445+
pytest --cache-list
446+
447+
.. code-block:: text
448+
449+
user cache directory: /home/ronny/.cache/pytest
450+
451+
DIRECTORY SIZE LAST USED STATUS ORIGIN
452+
myproject-1a2b3c4d5e6f 12.4 MiB 2 days ok /home/ronny/src/myproject
453+
env-venv-9f8e7d6c 1.1 MiB 2 days ok /home/ronny/src/myproject/.venv
454+
env-py312-0a1b2c3d 840.1 KiB 94 days stale /home/ronny/src/myproject/.tox/py312
455+
oldthing-9f8e7d6c5b4a 840.0 KiB 31 days orphaned /home/ronny/src/oldthing
456+
457+
2 directories, 3 scopes, 13.2 MiB total
458+
459+
``orphaned`` means the origin project no longer exists, and ``stale`` means the environment a scope holds
460+
state for no longer exists. In terminals which support it, the directory and origin columns are clickable
461+
links.
462+
463+
Nothing is ever removed automatically. To remove things, say which:
464+
465+
.. code-block:: bash
466+
467+
pytest --cache-prune=stale # state for environments that are gone
468+
pytest --cache-prune=orphaned # caches for projects that are gone
469+
pytest --cache-prune='oldthing-*' # by name, or by origin path
470+
pytest --cache-prune=all
471+
472+
``--cache-prune`` requires a selector, so a bare invocation cannot delete anything, and it does not ask
473+
for confirmation once given one - use ``--cache-list`` as the preview. ``all`` never removes the cache
474+
directory of the project you run it from.
475+
476+
.. note::
477+
478+
A project on an unmounted network or removable volume looks ``orphaned``, and a virtualenv on one
479+
looks ``stale``. This is the main reason pruning is never automatic.
480+
481+
.. note::
482+
483+
``--cache-list`` and ``--cache-prune`` only see the user-level cache root. Caches placed somewhere
484+
else with :confval:`cache_dir` are not tracked, since pytest has no way to know where they all are.
485+
486+
335487
.. _cache stepwise:
336488

337489
Stepwise

‎doc/en/reference/reference.rst‎

Lines changed: 83 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -457,6 +457,11 @@ Under the hood, the cache plugin uses the simple
457457
.. autoclass:: pytest.Cache()
458458
:members:
459459

460+
How far a cached value travels is given by its :class:`pytest.CacheScope`; see :ref:`cache_scopes`.
461+
462+
.. autoclass:: pytest.CacheScope()
463+
:members:
464+
460465

461466
.. fixture:: doctest_namespace
462467

@@ -1215,6 +1220,20 @@ Environment variables that can be used to change pytest's behavior.
12151220
This is not meant to be set by users, but is set by pytest internally with the name of the current test so other
12161221
processes can inspect it, see :ref:`pytest current test env` for more information.
12171222

1223+
.. envvar:: PYTEST_CACHE_HOME
1224+
1225+
Overrides the user-level cache root used by ``cache_policy = user``, instead of asking the platform
1226+
for it. Useful for a shared, explicit location in CI. See :ref:`cache_location`.
1227+
1228+
.. versionadded:: 9.2
1229+
1230+
.. envvar:: PYTEST_CACHE_POLICY
1231+
1232+
Sets the default for :confval:`cache_policy`, so that the cache can be relocated for a whole machine
1233+
without editing every project. An explicit :confval:`cache_policy` still wins over it.
1234+
1235+
.. versionadded:: 9.2
1236+
12181237
.. envvar:: PYTEST_DEBUG
12191238

12201239
When set, pytest will print tracing and debug information.
@@ -1246,6 +1265,14 @@ Environment variables that can be used to change pytest's behavior.
12461265
Entry point names of installed plugins are now also accepted, in
12471266
addition to importable module names.
12481267

1268+
.. envvar:: PYTEST_HYPERLINKS
1269+
1270+
When set to ``1``, pytest emits OSC 8 terminal hyperlinks, so that paths it prints are clickable.
1271+
When set to ``0``, it never does. Otherwise they are emitted whenever terminal color is in use and
1272+
the terminal is not one known to mishandle them.
1273+
1274+
.. versionadded:: 9.2
1275+
12491276
.. envvar:: PYTEST_THEME
12501277

12511278
Sets a `pygment style <https://pygments.org/docs/styles/>`_ to use for the code output.
@@ -1373,14 +1400,38 @@ passed multiple times. The expected format is ``name=value``. For example::
13731400

13741401
.. confval:: cache_dir
13751402
:type: ``str``
1376-
:default: ``".pytest_cache"``
1403+
:default: ``""``
13771404

1378-
Sets the directory where the cache plugin's content is stored.
1405+
Sets the directory where the cache plugin's content is stored, overriding :confval:`cache_policy`.
13791406
Directory may be relative or absolute path. If setting relative path, then directory is created
13801407
relative to :ref:`rootdir <rootdir>`. Additionally, a path may contain environment
13811408
variables, that will be expanded. For more information about cache plugin
13821409
please refer to :ref:`cache_provider`.
13831410

1411+
When unset, the location is decided by :confval:`cache_policy` instead.
1412+
1413+
.. versionchanged:: 9.2
1414+
1415+
The default is now empty rather than ``".pytest_cache"``, so that leaving it unset can be told
1416+
apart from setting it. The resulting location is unchanged.
1417+
1418+
.. confval:: cache_policy
1419+
:type: ``Literal["local", "user"]``
1420+
:default: ``"local"``
1421+
1422+
.. versionadded:: 9.2
1423+
1424+
Chooses where the cache directory lives, without spelling out a path:
1425+
1426+
* ``local`` - ``<rootdir>/.pytest_cache``.
1427+
* ``user`` - a per-project directory inside the platform's user cache directory. Requires the
1428+
``xdg`` extra (``pip install pytest[xdg]``).
1429+
1430+
Ignored when :confval:`cache_dir` is set. The default can be set for a whole machine with the
1431+
:envvar:`PYTEST_CACHE_POLICY` environment variable.
1432+
1433+
See :ref:`cache_location` and :ref:`cache_pruning`.
1434+
13841435
.. confval:: collect_imported_tests
13851436
:type: ``bool``
13861437
:default: ``true``
@@ -3245,6 +3296,21 @@ Cache
32453296
Remove all cache contents at start of test run.
32463297
See :ref:`cache`.
32473298

3299+
.. option:: --cache-list
3300+
3301+
List the cache directories under the user-level cache root, don't perform collection or tests.
3302+
See :ref:`cache_pruning`.
3303+
3304+
.. versionadded:: 9.2
3305+
3306+
.. option:: --cache-prune=SELECTOR
3307+
3308+
Remove cache directories or scopes under the user-level cache root, don't perform collection or
3309+
tests. ``SELECTOR`` is ``all``, ``orphaned``, ``stale``, or a glob matched against the directory
3310+
name and the origin path. May be given more than once. See :ref:`cache_pruning`.
3311+
3312+
.. versionadded:: 9.2
3313+
32483314
Warnings
32493315
~~~~~~~~
32503316

@@ -3490,6 +3556,15 @@ All the command-line flags can also be obtained by running ``pytest --help``::
34903556
Show cache contents, don't perform collection or
34913557
tests. Optional argument: glob (default: '*').
34923558
--cache-clear Remove all cache contents at start of test run
3559+
--cache-list List the cache directories under the user-level
3560+
cache root, don't perform collection or tests.
3561+
--cache-prune=SELECTOR
3562+
Remove cache directories or scopes under the user-
3563+
level cache root, don't perform collection or tests.
3564+
SELECTOR is 'all', 'orphaned', 'stale', or a glob
3565+
matched against the directory name and the origin
3566+
path. May be given more than once. See --cache-list
3567+
first.
34933568
--lfnf, --last-failed-no-failures={all,none}
34943569
With ``--lf``, determines whether to execute tests
34953570
when there are no previously (known) failures or
@@ -3755,7 +3830,12 @@ All the command-line flags can also be obtained by running ``pytest --help``::
37553830
Option flags for doctests
37563831
doctest_encoding (string):
37573832
Encoding used for doctest files
3758-
cache_dir (string): Cache directory path
3833+
cache_dir (string): Cache directory path; overrides cache_policy
3834+
cache_policy ('local' | 'user'):
3835+
Where the cache directory lives: 'local'
3836+
(rootdir/.pytest_cache) or 'user' (the platform's
3837+
user cache directory, keyed by project). Ignored if
3838+
cache_dir is set.
37593839
log_level (string): Default value for --log-level
37603840
log_format (string): Default value for --log-format
37613841
log_date_format (string):

0 commit comments

Comments
 (0)