[build] setup.py should not modify the PYTHONPATH

[picca]

Same problem observed on PyFAI

here my solution as an example

diff --git a/setup.py b/setup.py
index 852c48d..f562f74 100644
--- a/setup.py
+++ b/setup.py
@@ -194,7 +194,6 @@ class BuildMan(Command):
         path.insert(0, os.path.abspath(build.build_lib))

         env = dict((str(k), str(v)) for k, v in os.environ.items())
-        env["PYTHONPATH"] = os.pathsep.join(path)

         import subprocess

Maybe it is possible to avoid the env variable but I di d not investigate this.

[vallsv]

silx-kit/pyFAI#646

As i said the build_lib path have to be provided to html2man. Then i don't see how it can work without. But it can be useful to see your command line.

[picca]

Yes my build system gicves the right PYTHONPATH.
another solutions would be to

add the build_lib to the PYTHONPATH instead of overriding it

something like

env ["PYTHONPATH"] = os.pathsep.join([path, env["PYTHONPATH"]])

warning not tested.

I

[vallsv]

What i dont understand is what is the content of build_lib in your case? Just None?

We should have a contain from build_man to execute build before. But in your case, you only use build_man without the build target right?

[picca]

I need to check what is the content of build_lib

I use build_man without build because it already check thaht build was run.
So the build system should see that the build target was already thanks to the right PYTHONPATH, I guess.

[vallsv]

        path = sys.path
        path.insert(0, os.path.abspath(build.build_lib))
        env["PYTHONPATH"] = os.pathsep.join(path)

But as you can see the python path is already complemented. No ?
sys.path is supposed to contains dirs from the PYTHONPATH no?

[picca]

Here the pydistutils file used to build the pacakge

[clean]
all=1
[build]
build-lib=/home/picca/Debian/silx/silx/.pybuild/pythonX.Y_2.7/build
[install]
force=1
install-layout=deb
install-scripts=/usr/bin
install-lib=/usr/lib/python2.7/dist-packages
[easy_install]
allow_hosts=None

[picca]

~$ PYTHONPATH='toto' python
Python 2.7.13 (default, Jan 19 2017, 14:48:08)
[GCC 6.3.0 20170118] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import sys
sys.path
['', '/home/picca/toto', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-i386-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7']

so yes...

[vallsv]

Then i dont really see how to fix the issue, cause i dont really see why it do not work in your case.

[picca]

When I print the env during the build, I get this

[...]
PYTHONPATH=/home/picca/Debian/silx/silx/.pybuild/pythonX.Y_2.7/build
GCJFLAGS=-g -O2 -fdebug-prefix-map=/home/picca/Debian/silx/silx=. -fstack-protector-strong
MFLAGS=-w
[...]

so the PYTHONPATH setup by pybuild is ok

[picca]

do not worry we will find :)

[vallsv]

Oh, BTW this thing in your path: '',, i already see that this thing was fully ignoring the folling paths in some cases. Do you know where does it come from? blahblah::blah instead of blahblah:blah. But maybe it was on Windows.

[vallsv]

Can you hack BuildMan to display some informations:

        print("PYTHONPATH", os.environ.get("PYTHONPATH", None))
        print("sys.path", sys.path)
        print("build_lib", build.build_lib, os.path.exists(build.build_lib))
        env = dict((str(k), str(v)) for k, v in os.environ.items())
        env["PYTHONPATH"] = os.pathsep.join(path)
        print("env[PYTHONPATH]", env["PYTHONPATH"])

[picca]

In fact I realize that this is a problem for build_man AND build_doc ...

I instrument the code and I tell you of rboth cases .

[picca]

here we go

running build_man
help2man: can't get `--help' info from /tmp/silx_n5O2QX
Try `--no-discard-stderr' if option outputs to stderr
('PYTHONPATH', '/home/picca/Debian/silx/silx/.pybuild/pythonX.Y_2.7/build')
('sys.path', ['/home/picca/Debian/silx/silx/build/lib.linux-i386-2.7', '/home/picca/Debian/silx/silx', '/home/picca/Debian/silx/silx/.pybuild/pythonX.Y_2.7/build', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-i386-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7'], False)
('env[PYTHONPATH]', '/home/picca/Debian/silx/silx/build/lib.linux-i386-2.7:/home/picca/Debian/silx/silx:/home/picca/Debian/silx/silx/.pybuild/pythonX.Y_2.7/build:/usr/lib/python2.7:/usr/lib/python2.7/plat-i386-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages/PILcompat:/usr/lib/python2.7/dist-packages/gtk-2.0:/usr/lib/pymodules/python2.7')
Traceback (most recent call last):
  File "setup.py", line 754, in <module>
    setup_package()
  File "setup.py", line 751, in setup_package
    setup(**setup_kwargs)
  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "setup.py", line 232, in run
    raise RuntimeError("Fail to generate man documentation")
RuntimeError: Fail to generate man documentation
E: pybuild pybuild:283: build: plugin distutils failed with: exit code=1: python2.7 setup.py build_man
Traceback (most recent call last):
  File "/usr/bin/pybuild", line 280, in main
    run(func, i, version, c)
  File "/usr/bin/pybuild", line 246, in run
    raise Exception(msg)
Exception: exit code=1: python2.7 setup.py build_man
dh_auto_build: pybuild --build -i python{version} -p 2.7 --after-build {interpreter} setup.py build_man returned exit code 13
debian/rules:29 : la recette pour la cible « override_dh_auto_build » a échouée
make[1]: *** [override_dh_auto_build] Erreur 25
make[1] : on quitte le répertoire « /home/picca/Debian/silx/silx »
debian/rules:17 : la recette pour la cible « build » a échouée
make: *** [build] Erreur 2
dpkg-buildpackage: erreur: debian/rules build a produit une erreur de sortie de type 2

So there is at lease two wrong directory before the right one in sys.path
PYTHONPATH is ok

[picca]

so the path added in front of sys.path

    path.insert(0, os.path.abspath(build.build_lib))

is the wrong path

'/home/picca/Debian/silx/silx/build/lib.linux-i386-2.7'

the right path is the one from PYTHONPATH which is given to setup.py via the pydist config file.
I am wondering what is the right way to implement a BuildXXX command in setup.py

[picca]

Writing a Build systems is touchy ;)

[picca]

Here just a example of small BuildMan target

http://sources.debian.net/src/cmdtest/0.27-1/setup.py/?hl=46#L46

I try this sort of minimalist command and report back

[vallsv]

Well, anyway i don't really understand the problem of adding a non existing path to the PYTHONPATH. It should just use the next one, then the next one... No? If it do not exists we can avoid to add it. That is why i put a os.path.exists in the logs.

[picca]

For now I build once more the module + extension just for the man pages and the sphinx doc.

I need to discuss about this problme with Piotr (the guy behind pybuild)

[vallsv]

Then i think the better think to do is to patch PYTHONPATH from the current PYTHONPATH (instead of using sys.path). The resulting path order should be better, and maybe it will fix your issue.

[payno]

This is no more the case. setup.py has been updated