[RFH] Running Python tests that require the source to be installed

[John Paul Adrian Glaubitz]

(Please CC as I'm not subscribed to debian-devel)

Hello,

I am maintaining the package src:kiwi [1] which hasn't been updated in
Debian for some time since upstream has added tests that only work when
the package source is installed into the test environment, i.e. available
through PYTHONPATH.

I have tried to adjust the environment variables in debian/rules as follows
and tried to invoke the testsuite from the test/unit subdirectory:

export PYTHONPATH = $(CURDIR)
export PYBUILD_TEST_ARGS_python3 = cd {build_dir}; python{version} -m discover

override_dh_auto_test:
ifeq ($(filter nocheck nobench, $(DEB_BUILD_OPTIONS)),)
/bin/sh -c "cd test/unit && pytest-3"
endif

However, that doesn't work and the tests still fail.

Does anyone have a suggestion how to solve? I would like to update the kiwi
package to the latest upstream version before the Trixie release.

Adrian

[1] https://salsa.debian.org/debian/kiwi

[Andrey Rakhmatullin]

export PYBUILD_BEFORE_TEST=cp -r {dir}/foo {dir}/bar {build_dir}
export PYBUILD_AFTER_TEST=rm -rf {build_dir}/foo {build_dir}/bar

Consider using pybuild.testfiels for this.

[Andrey Rakhmatullin]

Post by John Paul Adrian Glaubitz
(Please CC as I'm not subscribed to debian-devel)

(this is how I realized this is not debian-python@, consider using that
for more relevant coverage)

Post by John Paul Adrian Glaubitz
I am maintaining the package src:kiwi [1] which hasn't been updated in
Debian for some time since upstream has added tests that only work when
the package source is installed into the test environment, i.e. available
through PYTHONPATH.

I believe any or at least most Python unittests require the module being
tested to be available in sys.path and pybuild already helps with that, by
running tests in a directory with the package installed, is that not
enough? What are the specific problems you have?

Post by John Paul Adrian Glaubitz
I have tried to adjust the environment variables in debian/rules as follows
export PYTHONPATH = $(CURDIR)

$(CURDIR) is already in sys.path when running python -m.

Post by John Paul Adrian Glaubitz
export PYBUILD_TEST_ARGS_python3 = cd {build_dir}; python{version} -m discover
ifeq ($(filter nocheck nobench, $(DEB_BUILD_OPTIONS)),)
/bin/sh -c "cd test/unit && pytest-3"

You should use python* -m pytest (note that you need to run it for all
supported Python versions.

[John Paul Adrian Glaubitz]

Hi Andrey,

thanks a lot for the fast and already helpful reply!

Post by Andrey Rakhmatullin

Post by John Paul Adrian Glaubitz
(Please CC as I'm not subscribed to debian-devel)

for more relevant coverage)

Duly noted!

Post by Andrey Rakhmatullin

Post by John Paul Adrian Glaubitz
I am maintaining the package src:kiwi [1] which hasn't been updated in
Debian for some time since upstream has added tests that only work when
the package source is installed into the test environment, i.e. available
through PYTHONPATH.

I believe any or at least most Python unittests require the module being
tested to be available in sys.path and pybuild already helps with that, by
running tests in a directory with the package installed, is that not
enough? What are the specific problems you have?

I had to look it up and the mechanism used is called "entry_points". The kiwi
package adds such entry_points and wants to test them in its testsuite.

Thus, I need to figure out how to make those entry_points visible from the
build environment so that the testsuite can find and test them.

Is there any other Debian Python package that uses entry points?

Post by Andrey Rakhmatullin

Post by John Paul Adrian Glaubitz
I have tried to adjust the environment variables in debian/rules as follows
export PYTHONPATH = $(CURDIR)

$(CURDIR) is already in sys.path when running python -m.

Post by John Paul Adrian Glaubitz
export PYBUILD_TEST_ARGS_python3 = cd {build_dir}; python{version} -m discover
ifeq ($(filter nocheck nobench, $(DEB_BUILD_OPTIONS)),)
/bin/sh -c "cd test/unit && pytest-3"

You should use python* -m pytest (note that you need to run it for all
supported Python versions.

OK, thanks.

Adrian

[Timo Röhling]

Hi Adrian,

Post by John Paul Adrian Glaubitz
I had to look it up and the mechanism used is called
"entry_points". The kiwi package adds such entry_points and wants
to test them in its testsuite.
Thus, I need to figure out how to make those entry_points visible
from the build environment so that the testsuite can find and test
them.
Is there any other Debian Python package that uses entry points?

I ran into this issue with old-style setuptools packages (i.e.,
packages with a setup.py); AFAIK the entry_points mechanism needs
valid egg-info or dist-info metadata in the PYTHONPATH. My
workaround was


export PYBUILD_BEFORE_TEST=\
{interpreter} setup.py egg_info; \
cp -r {dir}/src/*.egg-info {build_dir}

export PYBUILD_AFTER_TEST=\
rm -r {dir}/src/*.egg-info {build_dir}/*.egg-info


Cheers
Timo

[John Paul Adrian Glaubitz]

Hi Timo,

Post by Timo Röhling
I ran into this issue with old-style setuptools packages (i.e.,
packages with a setup.py); AFAIK the entry_points mechanism needs
valid egg-info or dist-info metadata in the PYTHONPATH. My
workaround was
export PYBUILD_BEFORE_TEST=\
{interpreter} setup.py egg_info; \
cp -r {dir}/src/*.egg-info {build_dir}
export PYBUILD_AFTER_TEST=\
rm -r {dir}/src/*.egg-info {build_dir}/*.egg-info

Thanks, this is what I was looking for. I will give this a try.

Adrian

[John Paul Adrian Glaubitz]

Hi Timo,

Post by John Paul Adrian Glaubitz

Post by Timo Röhling
I ran into this issue with old-style setuptools packages (i.e.,
packages with a setup.py); AFAIK the entry_points mechanism needs
valid egg-info or dist-info metadata in the PYTHONPATH. My
workaround was
export PYBUILD_BEFORE_TEST=\
{interpreter} setup.py egg_info; \
cp -r {dir}/src/*.egg-info {build_dir}
export PYBUILD_AFTER_TEST=\
rm -r {dir}/src/*.egg-info {build_dir}/*.egg-info

Thanks, this is what I was looking for. I will give this a try.

I just realized that this doesn't work because kiwi doesn't have a setup.py
anymore but just uses pyproject.toml. Do you know how it works in this case?

Adrian

[John Paul Adrian Glaubitz]

Post by John Paul Adrian Glaubitz
I just realized that this doesn't work because kiwi doesn't have a setup.py
anymore but just uses pyproject.toml. Do you know how it works in this case?

OK, I just remove every custom override and let debhelper and pybuild just do it's
magic. Unfortunately, it turns out that the package now requires pytest_container
which doesn't exist in Debian.

*sigh*

Adrian

[Marcus Schäfer]

Hi,

Post by John Paul Adrian Glaubitz

Post by John Paul Adrian Glaubitz
I just realized that this doesn't work because kiwi doesn't have a setup.py
anymore but just uses pyproject.toml. Do you know how it works in this case?

OK, I just remove every custom override and let debhelper and pybuild just do it's
magic. Unfortunately, it turns out that the package now requires pytest_container
which doesn't exist in Debian.
*sigh*

Sorry for the trouble. A PR to fix the unneeded hard pytest_container
requirement is available here:

https://github.com/OSInside/kiwi/pull/2672

So that part will be fixed soon

Thanks much to all of you for the Debian packaging effort

Regards,
Marcus

[Stefano Rivera]

Hi John (2024.10.24_10:44:45_+0000)

Post by John Paul Adrian Glaubitz
I am maintaining the package src:kiwi [1] which hasn't been updated in
Debian for some time since upstream has added tests that only work when
the package source is installed into the test environment, i.e. available
through PYTHONPATH.

If it actually needs the .dist-info to be on path (fully installed), you
can try runnig the tests under tox. That's probably what the upstream
does.

tox tests in Debian package builds are a little different because we use
--system-site-packages virtualenvs, but they can be a good way to deal
with this.

Stefano

[John Paul Adrian Glaubitz]

Hello Stefano,

Post by Stefano Rivera
If it actually needs the .dist-info to be on path (fully installed), you
can try runnig the tests under tox. That's probably what the upstream
does.
tox tests in Debian package builds are a little different because we use
--system-site-packages virtualenvs, but they can be a good way to deal
with this.

Can you name any package using this mechanism so I can have a look?

I'm not really experienced with tox, so it would be great to have some
guidance in the form of sample code.

Adrian

[Stefano Rivera]

Hi John (2024.10.26_08:45:15_+0000)

Post by John Paul Adrian Glaubitz

Post by Stefano Rivera
tox tests in Debian package builds are a little different because we use
--system-site-packages virtualenvs, but they can be a good way to deal
with this.

Can you name any package using this mechanism so I can have a look?

wheel uses tox tests, in about the most straightforward way possible
(declare build-depends on tox).

Sometimes you'll need to patch tox.ini to add things to
allowlist_externals, because we're using --system-site-packages.

You'll also need to Build-Depend on everything that tox wants to install
into the virtualenv.

Some more examples:

$ reverse-depends -b tox
Reverse-Build-Depends
=====================
* anosql
* ceph
* custodia
* diskcache
* django-auth-ldap
* duplicity
* enlighten
* flask-jwt-simple
* git-imerge
* gitsome
* gnome-keysign
* gubbins
* mitmproxy
* pytest-datadir
* pytest-mypy-plugins
* python-django-solo
* python-json-log-formatter
* python-kdcproxy
* python-magic
* python-mrcfile
* python-nox
* python-pkginfo
* python-prettylog
* python-pyforge
* python-pyvmomi
* python-versioneer
* python-w3lib
* pytrainer
* rdflib-sqlalchemy
* reprotest
* sagemath
* sagenb-export
* sshtunnel
* tox-current-env
* wheel

Reverse-Build-Depends-Indep
===========================
* awscli
* python-bottle
* python-scrapy

Post by John Paul Adrian Glaubitz
I'm not really experienced with tox, so it would be great to have some
guidance in the form of sample code.
Adrian
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer
`. `' Physicist
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913