Changelog
pytest-xdist 3.6.1 (2024-04-28)
Bug Fixes
#1071: Add backward compatibility for deadlock issue with the
execnet
newmain_thread_only
“execmodel” triggered when pytest-cov accesses rinfo.
pytest-xdist 3.6.0 (2024-04-19)
This release was YANKED due to a regression fixed in 3.6.1.
Features
#1027:
pytest-xdist
workers now always execute the tests in the main thread. Previously some tests might end up executing in a separate thread other thanmain
in the workers, due to some internal execnet` details. This can cause problems specially with async frameworks where the event loop is running in themain
thread (for example #620).
Bug Fixes
#1024: Added proper handling of
shouldstop
(such as set by--max-fail
) andshouldfail
conditions in workers. Previously, a worker might have continued executing further tests before the controller could terminate the session.#1028: Fixed compatibility issue between looponfail and editable installs.
#620: Use the new
main_thread_only
execnet
“execmodel” so that code which expects to only run in the main thread will now work as expected.#937: Fixed a bug where plugin would raise an incompatibility error with
--pdb
despite using-n0
.
Removals
Trivial Changes
#1020: pytest-xdist’s
setup.py
file is removed.If you relied on this file, e.g. to install pytest using
setup.py install
, please see Why you shouldn’t invoke setup.py directly for alternatives.#1057: The internals of pytest-xdist are now fully typed. The typing is not exposed yet.
#996: Adjusted license file format and content to ensure security scanners will identity the license.
pytest-xdist 3.5.0 (2023-11-21)
Features
#632:
--dist=loadscope
now sorts scopes by number of tests to assign largest scopes early – in many cases this should improve overall test session running time, as there is less chance of a large scope being left to be processed near the end of the session, leaving other workers idle.
pytest-xdist 3.4.0 (2023-11-11)
Features
pytest-xdist 3.3.1 (2023-05-19)
Bug Fixes
#907: Avoid remote calls during startup as
execnet
by default does not ensure remote affinity with the main thread and might accidentally schedule the pytest worker into a non-main thread, which breaks numerous frameworks, for exampleasyncio
,anyio
,PyQt/PySide
, etc.A more safe correction will require thread affinity in
execnet
(pytest-dev/execnet#96).
pytest-xdist 3.3.0 (2023-05-12)
Features
#555: Improved progress output when collecting nodes to be less verbose.
pytest-xdist 3.2.1 (2023-03-12)
Bug Fixes
#884: Fixed hang in
worksteal
scheduler.
pytest-xdist 3.2.0 (2023-02-07)
Improved Documentation
#863: Document limitations for debugging due to standard I/O of workers not being forwarded. Also, mention remote debugging as a possible workaround.
Features
#855: Users can now configure
load
scheduling precision using--maxschedchunk
command line option.#858: New
worksteal
scheduler, based on the idea of work stealing. It’s similar toload
scheduler, but it should handle tests with significantly differing duration better, and, at the same time, it should provide similar or better reuse of fixtures.
Trivial Changes
#870: Make the tests pass even when
$PYTEST_XDIST_AUTO_NUM_WORKERS
is set.
pytest-xdist 3.1.0 (2022-12-01)
Features
Removals
#842: Python 3.6 is no longer supported.
pytest-xdist 3.0.2 (2022-10-25)
Bug Fixes
#813: Cancel shutdown when a crashed worker is restarted.
Deprecations
#825: The
--rsyncdir
command line argument andrsyncdirs
config variable are deprecated.The rsync feature will be removed in pytest-xdist 4.0.
#826: The
--looponfail
command line argument andlooponfailroots
config variable are deprecated.The loop-on-fail feature will be removed in pytest-xdist 4.0.
Improved Documentation
Features
#792: The environment variable
PYTEST_XDIST_AUTO_NUM_WORKERS
can now be used to specify the default for-n auto
and-n logical
.#812: Partially restore old initial batch distribution algorithm in
LoadScheduling
.pytest orders tests for optimal sequential execution - i. e. avoiding unnecessary setup and teardown of fixtures. So executing tests in consecutive chunks is important for optimal performance.
In v1.14, initial test distribution in
LoadScheduling
was changed to round-robin, optimized for the corner case, when the number of tests is less than2 * number of nodes
. At the same time, it became worse for all other cases.For example: if some tests use some “heavy” fixture, and these tests fit into the initial batch, with round-robin distribution the fixture will be created
min(n_tests, n_workers)
times, no matter how many other tests there are.With the old algorithm (before v1.14), if there are enough tests not using the fixture, the fixture was created only once.
So restore the old behavior for typical cases where the number of tests is much greater than the number of workers (or, strictly speaking, when there are at least 2 tests for every node).
Removals
#468: The
--boxed
command-line option has been removed. If you still need this functionality, install pytest-forked separately.
Trivial Changes
pytest-xdist 2.5.0 (2021-12-10)
Deprecations and Removals
#468: The
--boxed
command line argument is deprecated. Install pytest-forked and use--forked
instead. pytest-xdist 3.0.0 will remove the--boxed
argument andpytest-forked
dependency.
Features
Trivial Changes
#708: Use
@pytest.hookspec
decorator to declare hook options innewhooks.py
to avoid warnings inpytest 7.0
.#719: Use up-to-date
setup.cfg
/pyproject.toml
packaging setup.#720: Require pytest>=6.2.0.
#721: Started using type annotations and mypy checking internally. The types are incomplete and not published.
pytest-xdist 2.4.0 (2021-09-20)
Features
#696: On Linux, the process title now changes to indicate the current worker state (running/idle).
Depends on the setproctitle package, which can be installed with
pip install pytest-xdist[setproctitle]
.#704: Add support for Python 3.10.
pytest-xdist 2.3.0 (2021-06-16)
Deprecations and Removals
#654: Python 3.5 is no longer supported.
Features
Bug Fixes
Trivial Changes
pytest-xdist 2.2.1 (2021-02-09)
Bug Fixes
#623: Gracefully handle the pending deprecation of Node.fspath by using config.rootpath for topdir.
pytest-xdist 2.2.0 (2020-12-14)
Features
#608: Internal errors in workers are now propagated to the master node.
pytest-xdist 2.1.0 (2020-08-25)
Features
#585: New
pytest_xdist_auto_num_workers
hook can be implemented by plugins orconftest.py
files to control the number of workers when--numprocesses=auto
is given in the command-line.
Trivial Changes
#585:
psutil
has proven to makepytest-xdist
installation in certain platforms and containers problematic, so to use it for automatic number of CPUs detection users need to install thepsutil
extra:pip install pytest-xdist[psutil]
pytest-xdist 2.0.0 (2020-08-12)
Deprecations and Removals
Features
#504: New functions
xdist.is_xdist_worker
,xdist.is_xdist_master
,xdist.get_xdist_worker_id
, to easily identify the current node.
Bug Fixes
Trivial Changes
#541:
pytest-xdist
now requirespytest>=6.0
.
pytest-xdist 1.34.0 (2020-07-27)
Features
#549: Make
--pdb
imply--dist no
, as the two options cannot really work together at the moment.
Bug Fixes
pytest-xdist 1.33.0 (2020-07-09)
Features
#554: Fix warnings support for upcoming pytest 6.0.
Trivial Changes
#548: SCM and CI files are no longer included in the source distribution.
pytest-xdist 1.32.0 (2020-05-03)
Deprecations and Removals
#475: Drop support for EOL Python 3.4.
Features
#524: Add testrun_uid fixture. This is a shared value that uniquely identifies a test run among all workers. This also adds a PYTEST_XDIST_TESTRUNUID environment variable that is accessible within a test as well as a command line option –testrunuid to manually set the value from outside.
pytest-xdist 1.31.0 (2019-12-19)
Features
#486: Add support for Python 3.8.
Bug Fixes
#491: Fix regression that caused custom plugin command-line arguments to be discarded when using
--tx
mode.
pytest-xdist 1.30.0 (2019-10-01)
Features
#448: Initialization between workers and master nodes is now more consistent, which fixes a number of long-standing issues related to startup with the
-c
option.Issues:
#6: Poor interaction between
-n#
and-c X.cfg
#445: pytest-xdist is not reporting the same nodeid as pytest does
This however only works with pytest 5.1 or later, as it required changes in pytest itself.
Bug Fixes
#467: Fix crash issues related to running xdist with the terminal plugin disabled.
pytest-xdist 1.29.0 (2019-06-14)
Features
#226:
--max-worker-restart
now assumes a more reasonable value (4 times the number of nodes) when not given explicitly. This prevents test suites from running forever when the suite crashes during collection.#435: When the test session is interrupted due to running out of workers, the reason is shown in the test summary for easier viewing.
#442: Compatibility fix for upcoming pytest 5.0:
session.exitstatus
is now anIntEnum
object.
Bug Fixes
#435: No longer show an internal error when we run out of workers due to crashes.
pytest-xdist 1.28.0 (2019-04-02)
Features
#426:
pytest-xdist
now uses the newpytest_report_to_serializable
andpytest_report_from_serializable
hooks frompytest 4.4
(still experimental). This will make report serialization more reliable and extensible.This also means that
pytest-xdist
now requirespytest>=4.4
.
pytest-xdist 1.27.0 (2019-02-15)
Features
pytest-xdist 1.26.1 (2019-01-28)
Bug Fixes
#406: Do not implement deprecated
pytest_logwarning
hook in pytest versions where it is deprecated.
pytest-xdist 1.26.0 (2019-01-11)
Features
#376: The current directory is no longer added
sys.path
for local workers, only for remote connections.This behavior is surprising because it makes xdist runs and non-xdist runs to potentially behave differently.
Bug Fixes
#379: Warning attributes are checked to make sure they can be dumped prior to serializing the warning for submission to the master node.
pytest-xdist 1.25.0 (2018-12-12)
Deprecations and Removals
#372: Pytest versions older than 3.6 are no longer supported.
Features
Bug Fixes
pytest-xdist 1.24.1 (2018-11-09)
Bug Fixes
#349: Correctly handle warnings created with arguments that can’t be serialized during the transfer from workers to master node.
pytest-xdist 1.24.0 (2018-10-18)
Features
#337: New
--maxprocesses
command-line option that limits the maximum number of workers when using--numprocesses=auto
.
Bug Fixes
#351: Fix scheduling deadlock in case of inter-test locking.
pytest-xdist 1.23.2 (2018-09-28)
Bug Fixes
#344: Fix issue where Warnings could cause pytest to fail if they do not set the args attribute correctly.
pytest-xdist 1.23.1 (2018-09-25)
Bug Fixes
#341: Fix warnings transfer between workers and master node with pytest >= 3.8.
pytest-xdist 1.23.0 (2018-08-23)
Features
#330: Improve collection performance by reducing the number of events sent to
master
node.
pytest-xdist 1.22.5 (2018-07-27)
Bug Fixes
#321: Revert change that dropped support for
pytest<3.4
and requiresix
.This change caused problems in some installations, and was a mistaken in the first place as we should not change version requirements in bug-fix releases unless they fix an actual bug.
pytest-xdist 1.22.4 (2018-07-27)
Bug Fixes
pytest-xdist 1.22.3 (2018-07-23)
Bug Fixes
Fix issue of virtualized or containerized environments not reporting the number of CPUs correctly. (#9)
Trivial Changes
Make all classes subclass from
object
and fixsuper()
call inLoadFileScheduling
; (#297)
pytest-xdist 1.22.2 (2018-02-26)
Bug Fixes
Add backward compatibility for
slaveoutput
attribute toWorkerController
instances. (#285)
pytest-xdist 1.22.1 (2018-02-19)
Bug Fixes
Fix issue when using
loadscope
orloadfile
where tests would fail to start if the first scope had only one test. (#257)
Trivial Changes
Change terminology used by
pytest-xdist
to master and worker in arguments and messages (for example--max-worker-reset
). (#234)
pytest-xdist 1.22.0 (2018-01-11)
Features
Add support for the
pytest_runtest_logfinish
hook which will be released in pytest 3.4. (#266)
pytest-xdist 1.21.0 (2017-12-22)
Deprecations and Removals
Drop support for EOL Python 2.6. (#259)
Features
New
--dist=loadfile
option which load-distributes test to workers grouped by the file the tests live in. (#242)
Bug Fixes
Fix accidental mutation of test report during serialization causing longrepr string-ification to break. (#241)
pytest-xdist 1.20.1 (2017-10-05)
Bug Fixes
pytest-xdist 1.20.0 (2017-08-17)
Features
xdist
now supports tests to log results multiple times, improving integration with plugins which require it like pytest-rerunfailures and flaky. (#206)
Bug Fixes
Fix issue where tests were being incorrectly identified if a worker crashed during the
teardown
stage of the test. (#124)
pytest-xdist 1.19.1 (2017-08-10)
Bug Fixes
Fix crash when transferring internal pytest warnings from workers to the master node. (#214)
pytest-xdist 1.19.0 (2017-08-09)
Deprecations and Removals
--boxed
functionality has been moved to a separate plugin, pytest-forked. This release now depends on `` pytest-forked`` and provides--boxed
as a backward compatibility option. (#1)
Features
Bug Fixes
Fix serialization of native tracebacks (
--tb=native
). (#196)
pytest-xdist 1.18.2 (2017-07-28)
Bug Fixes
pytest-xdist 1.18.1 (2017-07-05)
Bug Fixes
pytest-xdist 1.18.0 (2017-06-26)
pytest-xdist
now requirespytest>=3.0.0
.
Features
Add long option –numprocesses as alternative for -n. (#168)
Bug Fixes
Fix serialization and deserialization dropping longrepr details. (#133)
pytest-xdist 1.17.1 (2017-06-10)
Bug Fixes
Hot fix release reverting the change introduced by #124, unfortunately it broke a number of test suites so we are reversing this change while we investigate the problem. (#157)
Improved Documentation
Introduced
towncrier
forCHANGELOG
management. (#154)Added
HOWTORELEASE
documentation. (#155)
1.17.0
fix #124: xdist would mark test as complete after ‘call’ step. As a result, xdist could identify the wrong test as failing when test crashes at teardown. To address this issue, xdist now marks test as complete at teardown.
1.16.0
pytest-xdist
now requires pytest 2.7 or later.Add
worker_id
attribute in the TestReportnew hook:
pytest_xdist_make_scheduler(config, log)
, can return custom tests items distribution logic implementation. You can take a look at built-inLoadScheduling
andEachScheduling
implementations. Note that required scheduler class public API may change in nextpytest-xdist
versions.
1.15.0
new
worker_id
fixture, returns the id of the worker in a test or fixture. Thanks Jared Hellman for the PR.display progress during collection only when in a terminal, similar to pytest #1397 issue. Thanks Bruno Oliveira for the PR.
fix internal error message when
--maxfail
is used (#62, #65). Thanks Collin RM Stocks and Bryan A. Jones for reports and Bruno Oliveira for the PR.
1.14
new hook:
pytest_xdist_node_collection_finished(node, ids)
, called when a worker has finished collection. Thanks Omer Katz for the request and Bruno Oliveira for the PR.fix README display on pypi
fix #22: xdist now works if the internal tmpdir plugin is disabled. Thanks Bruno Oliveira for the PR.
fix #32: xdist now works if looponfail or boxed are disabled. Thanks Bruno Oliveira for the PR.
1.13.1
fix a regression -n 0 now disables xdist again
1.13
extended the tox matrix with the supported py.test versions
split up the plugin into 3 plugin’s to prepare the departure of boxed and looponfail.
looponfail will be a part of core and forked boxed will be replaced with a more reliable primitive based on xdist
conforming with new pytest-2.8 behavior of returning non-zero when all tests were skipped or deselected.
new “–max-slave-restart” option that can be used to control maximum number of times pytest-xdist can restart slaves due to crashes. Thanks to Anatoly Bubenkov for the report and Bruno Oliveira for the PR.
release as wheel
“-n” option now can be set to “auto” for automatic detection of number of cpus in the host system. Thanks Suloev Dmitry for the PR.
1.12
fix issue594: properly report errors when the test collection is random. Thanks Bruno Oliveira.
some internal test suite adaptation (to become forward compatible with the upcoming pytest-2.8)
1.11
fix pytest/xdist issue485 (also depends on py-1.4.22): attach stdout/stderr on –boxed processes that die.
fix pytest/xdist issue503: make sure that a node has usually two items to execute to avoid scoped fixtures to be torn down pre-maturely (fixture teardown/setup is “nextitem” sensitive). Thanks to Andreas Pelme for bug analysis and failing test.
restart crashed nodes by internally refactoring setup handling of nodes. Also includes better code documentation. Many thanks to Floris Bruynooghe for the complete PR.
1.10
add glob support for rsyncignores, add command line option to pass additional rsyncignores. Thanks Anatoly Bubenkov.
fix pytest issue382 - produce “pytest_runtest_logstart” event again in master. Thanks Aron Curzon.
fix pytest issue419 by sending/receiving indices into the test collection instead of node ids (which are not necessarily unique for functions parametrized with duplicate values)
send multiple “to test” indices in one network message to a slave and improve heuristics for sending chunks where the chunksize depends on the number of remaining tests rather than fixed numbers. This reduces the number of master -> node messages (but not the reverse direction)
1.9
changed LICENSE to MIT
fix duplicate reported test ids with –looponfailing (thanks Jeremy Thurgood)
fix pytest issue41: re-run tests on all file changes, not just randomly select ones like .py/.c.
fix pytest issue347: slaves running on top of Python3.2 will set PYTHONDONTWRITEYBTECODE to 1 to avoid import concurrency bugs.
1.8
fix pytest-issue93 - use the refined pytest-2.2.1 runtestprotocol interface to perform eager teardowns for test items.
1.7
fix incompatibilities with pytest-2.2.0 (allow multiple pytest_runtest_logreport reports for a test item)
1.6
terser collection reporting
fix issue34 - distributed testing with -p plugin now works correctly
fix race condition in looponfail mode where a concurrent file removal could cause a crash
1.5
adapt to and require pytest-2.0 changes, rsyncdirs and rsyncignore can now only be specified in [pytest] sections of ini files, see “py.test -h” for details.
major internal refactoring to match the pytest-2.0 event refactoring - perform test collection always at slave side instead of at the master - make python2/python3 bridging work, remove usage of pickling
improve initial reporting by using line-rewriting
remove all trailing whitespace from source
1.4
perform distributed testing related reporting in the plugin rather than having dist-related code in the generic py.test distribution
depend on execnet-1.0.7 which adds “env1:NAME=value” keys to gateway specification strings.
show detailed gateway setup and platform information only when “-v” or “–verbose” is specified.
1.3
fix –looponfailing - it would not actually run against the fully changed source tree when initial conftest files load application state.
adapt for py-1.3.1’s new –maxfailure option
1.2
fix issue79: sessionfinish/teardown hooks are now called systematically on the slave side
introduce a new data input/output mechanism to allow the master side to send and receive data from a slave.
fix race condition in underlying pickling/unpickling handling
use and require new register hooks facility of py.test>=1.3.0
require improved execnet>=1.0.6 because of various race conditions that can arise in xdist testing modes.
fix some python3 related pickling related race conditions
fix PyPI description
1.1
fix an indefinite hang which would wait for events although no events are pending - this happened if items arrive very quickly while the “reschedule-event” tried unconditionally avoiding a busy-loop and not schedule new work.
1.0
moved code out of py-1.1.1 into its own plugin
use a new, faster and more sensible model to do load-balancing of tests - now no magic “MAXITEMSPERHOST” is needed and load-testing works effectively even with very few tests.
cleaned up termination handling
make -x cause hard killing of test nodes to decrease wait time until the traceback shows up on first failure