Rip the stuff into separate repos

[KOLANICH]

Now it is problematic to reference python runtime as a package which should be installed from git, not pypi.

[GreyCat]

Can you elaborate what "stuff" you're targeting to "rip into separate repos", and what's the value in that?

[KOLANICH]

what's the value in that?

The value is PEP508 dependency specifier: kaitai.compress @ git+https://github.com/kaitai-io/kaitai_compress_python.git . It allows to maks packages be installed from a git repo rather than pypi. But not from a dir within a git repo.

Can you elaborate what "stuff" you're targeting to

this repo

"rip into separate repos"

move each folder into an own repo using git filter-branch

[KOLANICH]

@GreyCat

[generalmimon]

@KOLANICH:

The value is PEP508 dependency specifier: kaitai.compress @ git+https://github.com/kaitai-io/kaitai_compress_python.git . It allows to maks packages be installed from a git repo rather than pypi. But not from a dir within a git repo.

I'm not sure if pip supported it back in 2019 (when this issue was last active), perhaps not. But now it apparently does - see https://pip.pypa.io/en/stable/topics/vcs-support/#url-fragments (the pip.pypa.io site doesn't seem to support permalinks to a specific version, so see this section in vcs-support.md on GitHub in case the pip.pypa.io link no longer works):

URL fragments

pip looks at the subdirectory fragments of VCS URLs for specifying the path to the Python package, when it is not in the root of the VCS directory. eg: pkg_dir.

(...)

Example

If your repository layout is:

pkg_dir
├── setup.py  # setup.py for package "pkg"
└── some_module.py
other_dir
└── some_file
some_other_file

Then, to install from this repository, the syntax would be:

$ python -m pip install "pkg @ vcs+protocol://repo_url/#subdirectory=pkg_dir"

[KOLANICH]

I'm not sure if pip supported it back in 2019 (when this issue was last active), perhaps not. But now it apparently does

I was the one of the people who has started to use that feature since it got introduced. Subsequentially I felt like it is not such a good idea, because when a dep is specified this way, pip completely ignores the version that is already installed and always reinstalls. So now I just specify by name, but resolving packages to git repos should be done by a package manager, like ipi (which can use pip as a tool to install wheels, but usually a user should install a lib doing installation the right way without relying to pip (in fact it relies on pip internal machinery too for discovery of paths, but it is because there is no proper machinery in standalone libs ) ).

Also, since GitHub now allows partial fetches, it is not such a severe requirement. Though we cannot specify it in PEP 508 way, for ipi it can be specified another way and ipi can into partial fetches.

Please note: ipi is currently unfinished and broken. And even if it wasn't, it is still experimental.