Suggestion: add new 'parametrization' fixture scope

Currently, a fixture can be scoped as function, class, module, or session.

I propose a fixture can also be scoped per parametrization.

Consider:

import time
@pytest.fixture
def slow_setup():
    time.sleep(1)
    return 'foo'

If I have a group of tests which I am parametrizing, but that would share this value, it'll do the setup for each one (as expected):

# takes 4 seconds
@pytest.mark.parametrize('bar', [1, 2, 3, 4])
def test_parametrized(slow_setup, bar):
    assert slow_setup == 'foo'

If I wanted to run the same group of tests but sharing the fixture, I only really have one option (considering that other tests in the same module need the fixture to be re-set up each time, so I can't use module or session scope): set the fixture scope to class, and put the tests inside a class:

@pytest.fixture(scope='class')
def slow_setup_cls():
    time.sleep(1)
    return 'foo'

# takes 1 second
class TestParametrized(object):
    @pytest.mark.parametrize('bar', [1, 2, 3, 4])
    def test_parametrized(self, slow_setup_cls, bar):
        assert slow_setup_cls == 'foo'

However, the meaning I really intend to convey is "this fixture should be shared across all parametrizations of a function". I propose the following:

@pytest.fixture(scope='parametrization')
def slow_setup_pmt():
    time.sleep(1)
    return 'foo'

# takes 1 second
@pytest.mark.parametrize('bar', [1, 2, 3, 4])
def test_parametrized(self, slow_setup_pmt, bar):
    assert slow_setup_pmt == 'foo'

This fixture scope conveys, to me: "All parametrizations of this test share the same setup and teardown with regards to this fixture."

i like the idea

currently the proposal has 2 semantical problems

there is no collection element for the function that is not parameterized (scoping selects the collection tree element based on a type belonging to the
parameterization can happen on more than one scope ( class/module/session scoped fixtures may be parameterized as well

@csaftoiu

Why not just have parametrized fixtures that have autouse as True (factoring in dependencies, of course) make multiple versions of the scopes they are designated for?

For example, currently, given the following module:

import pytest


@pytest.fixture(scope="class", autouse=True)
def before_param():
    print("in before_param")


class TestIt():

    @pytest.fixture(scope="class", autouse=True, params=["a", "b"])
    def param_fix(self, before_param, request):
        print("in param_fix")
        return request.param

    @pytest.fixture(scope="class", autouse=True)
    def other_fix(self, param_fix):
        print("in other_fix")

    def test_stuff(self, other_fix):
        assert True

You get the following flow:

test_module (module)
└── TestIt (class)
    └── before_param
        ├── param_fix["a"]
        |   └── other_fix
        |       └── test_stuff (function)
        └── param_fix["b"]
            └── other_fix
                └── test_stuff (function)

But since autouse is True, the whole of that scope should be impacted. That, to me, would mean that I want to go through that scope, in it's entirety, once for each param set, as each set should impact the whole scope on its own. Here's the flow I would expect in this case:

test_module (module)
├── TestIt["a"] (class)
|   └── before_param
|       └── param_fix
|           └── other_fix
|               └── test_stuff (function)
└── TestIt["b"] (class)
    └── before_param
        └── param_fix
            └── other_fix
                └── test_stuff (function)

In this case, param_fix made two versions of the TestIt class, one for each param set.

If autouse is not True, then just only make multiple versions of the individual tests that depend on it.

This covers @csaftoiu's example with pytest.mark.parametrize without adding a need to use another argument or alternate scope options.

So after doing some digging into pytest, it looks like my initial idea of having multiple versions of each scope would pretty much require a total rework of test collection/execution. But I did find what I think is a good place to make a small change so that autouse parametrized fixtures cause other fixtures for that scope to re-execute as well.

I'm writing up a pull request now to put in this change. If it's decided to not go this route, and instead add a new argument/scope to trigger this kind of behavior, I think the changes I've made so far can be easily modified to do that instead.

@RonnyPfannschmidt

i like the idea

currently the proposal has 2 semantical problems

there is no collection element for the function that is not parameterized (scoping selects the collection tree element based on a type belonging to the

parameterization can happen on more than one scope ( class/module/session scoped fixtures may be parameterized as well

Hi @RonnyPfannschmidt, I am not clear on the two semantical issues that adding a parametrization scope would raise. Could you provide an example of how this would be a problem?

conftest.py:

@pytest.fixtur(scope="param", params=["x", "y"])
def param_fix(fun):
   pass

test_a.py:

@pytest.fixture(scope="session", params=[1,2])
def fun():
  pass

@pytest.mark.parametrize("b", ["a", "b"], scope="module")
def test_param(fix_param, b):
  pass

what should be the order of tests and the setup/teardown in that case

@RonnyPfannschmidt

If I understand it correctly, it means that in terms of levels, it should be session > module > class > parameterization > function?

Then the example would not be valid @RonnyPfannschmidt , as test_param has a module scope, depending on a parametrization scope.

--

Actually the function now is in fact every single test case, and this parametrization scope would be in fact one per function definition.

In fact I think it would be more accurately defined as:

session (same fixture instance if its the same pytest run)
module (same fixture instance if its the same python module)
class (same fixture instance if its the same python class)
function (same fixture instance if its the same python function) - this would be the proposed here, although of course the name is already used to mean the following lower level.
None (always recreate the fixture instance) - the current pytest function scope

@toby-w

@toby-w @Sup3rGeo I think what @RonnyPfannschmidt is referring to, is the fact that you can apply a parametrization within multiple scopes (i.e. session/package/module/class/function), and having a "param" scope doesn't specify which level of parametrization you would want to target.

For example:

@pytest.fixture(scope="class", params=["a", "b"], autouse=True)
def some_class_lvl_fix(request):
    return request.param

@pytest.fixture(scope="param", autouse=True)
def some_param_lvl_fix():
    return something

class TestSomething():
    @pytest.fixture(scope="class", params=["x", "y"], autouse=True)
    def another_cls_lvl_fix(self, request):
        return request.param
    @pytest.fixture(scope="function", params=[1, 2])
    def some_test_param_fix(self, request):
        return request.param
    def test_param(self, some_test_param_fix):
        # test something while parametrized by some_test_param_fix
    def test_not_param(self):
        # test something else while not parametrized by some_test_param_fix

The question is what level of parametrization are you targeting with some_param_lvl_fix? Parametrization is happening at several points, and at multiple levels, but a scope of param is not specific enough to say where you want to actually have this fixture reside in the order of operations. You're right that the intention with what @csaftoiu wanted was just to address that weird scope for functions where parametrization doesn't behave the same way as larger scopes do (i.e. parametrization happening after a fixture of the same scope doesn't trigger re-execution of the prior fixture), but "param"/"parametrization" just isn't specific enough of a term.

I also don't think @csaftoiu's proposal would actually solve the problem, because a scope of "param"/"parametrization" would suggest that it happens for each parameter set, meaning it would still run the slow running fixture more than once. Granted, the "param" (or whatever it would be called) scope of that type could be used to go back to the current behavior, while the standard "function" level could be made to operate like the larger scopes. But that would likely break a lot of previously written tests, so it's a lose-lose situation.

@SalmonMode

@SalmonMode yep, so what I wrote was my understanding of the feature @csaftoiu described in this issue, which I am convinced is different from what @RonnyPfannschmidt understood.

As I understand, the OP wants to have this:

@pytest.fixture
def slow_setup():
    time.sleep(1)
    return 'foo'

@pytest.mark.parametrize('bar', [1, 2, 3, 4])
def test_parametrized_one(slow_setup, bar):
    assert slow_setup == 'foo'

@pytest.mark.parametrize('bar', [5, 6, 7, 8])
def test_parametrized_two(slow_setup, bar):
    assert slow_setup == 'foo'

With only two fixture invocations:

one for test_parametrized_one (and all its [1] [2] [3] [4] instances);
and also one for test_parametrized_two (and all its [5] [6] [7] [8] instances);

Currently, if the slow_setup scope is function, you are going to have 8 invocations. If it is module, then you are going to have only 1 invocation.

Thats why he uses the workaround of having classes:

@pytest.fixture(scope="class")
def slow_setup():
    time.sleep(1)
    return 'foo'

class one:
    @pytest.mark.parametrize('bar', [1, 2, 3, 4])
    def test_parametrized_one(self, slow_setup, bar):
        assert slow_setup == 'foo'

class two:
    @pytest.mark.parametrize('bar', [5, 6, 7, 8])
    def test_parametrized_two(self, slow_setup, bar):
        assert slow_setup == 'foo'

Then we get the desired two invocation scenario.

Superseded by the "definition" fixture scope in #14805 (part of #3926). @pytest.fixture(scope="definition") shares one instance across all parameter sets of a single test definition and nothing beyond it — the semantics requested here.

My 2016 objection that "there is no collection element for the function that is not parameterized" is resolved by the new ItemDefinition node, opted into via collect_function_definition.

The second objection — parametrized fixtures at class/module/session scope multiplying their whole enclosing scope — is a separate feature and not covered here; please open a fresh issue if that's the use case you're after.

closing as the definition scope will solve most of the perceived problem

Navigation Menu

Repository navigation

Suggestion: add new 'parametrization' fixture scope #1552

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Suggestion: add new 'parametrization' fixture scope #1552

Description

Activity

RonnyPfannschmidt commented on May 7, 2016

SalmonMode commented on Aug 24, 2018

SalmonMode commented on Aug 28, 2018

toby-w commented on Dec 14, 2018

RonnyPfannschmidt commented on Dec 14, 2018

Sup3rGeo commented on Mar 16, 2019

SalmonMode commented on Mar 16, 2019

Sup3rGeo commented on Mar 16, 2019

maxrothman commented on Apr 1, 2020

Wenzel commented on Oct 24, 2021

RonnyPfannschmidt commented on Aug 1, 2026

RonnyPfannschmidt commented on Aug 1, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions