Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build error with filenames containing unicode characters #345

Closed
bpabel opened this issue May 21, 2020 · 2 comments · Fixed by #346
Closed

Build error with filenames containing unicode characters #345

bpabel opened this issue May 21, 2020 · 2 comments · Fixed by #346
Milestone

Comments

@bpabel
Copy link
Contributor

bpabel commented May 21, 2020

I'm trying to build a project that has files that contain unicode characters in their filename. I can build the wheel successfully, but building the sdist results in the following error.

Screen Shot 2020-05-20 at 11 34 07 PM

$ flit build
Found 6 files tracked in git                                                                                                                                   I-flit.sdist
Traceback (most recent call last):
  File "/Users/brendan/.local/bin/flit", line 8, in <module>
    sys.exit(main())
  File "/Users/brendan/.local/pipx/venvs/flit/lib/python3.8/site-packages/flit/__init__.py", line 170, in main
    main(args.ini_file, formats=set(args.format or []),
  File "/Users/brendan/.local/pipx/venvs/flit/lib/python3.8/site-packages/flit/build.py", line 46, in main
    sdist_file = sb.build(dist_dir, gen_setup_py=gen_setup_py)
  File "/Users/brendan/.local/pipx/venvs/flit/lib/python3.8/site-packages/flit/sdist.py", line 223, in build
    return Path(super().build(str(target_dir), gen_setup_py=gen_setup_py))
  File "/Users/brendan/.local/pipx/venvs/flit/lib/python3.8/site-packages/flit_core/sdist.py", line 185, in build
    ti = tf.gettarinfo(path, arcname=pjoin(self.dir_name, relpath))
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/tarfile.py", line 1830, in gettarinfo
    statres = os.lstat(name)
FileNotFoundError: [Errno 2] No such file or directory: '"tests/No\\303\\253l.txt"'
@takluyver
Copy link
Member

Thanks. It looks like git ls-files quotes the filename for us when it's called here:

https://github.com/takluyver/flit/blob/cfd71d81be95300266c6e989cf2d1bf57aa637fc/flit/vcs/git.py#L6-L9

I see it has a -z option to give the raw filenames delimited by null bytes. That's probably what we need.

@bpabel bpabel changed the title Build error with files containing unicode characters Build error with filenames containing unicode characters May 21, 2020
bpabel added a commit to bpabel/flit that referenced this issue May 22, 2020
fix to the git vcs module to prevent git from double escaping
unicode chars in filenames so that flit can decode them correctly

fixes pypa#345
@bpabel
Copy link
Contributor Author

bpabel commented May 22, 2020

@takluyver You were exactly right, thanks! Changing that fixed the issue (see #346 )

@takluyver takluyver added this to the 3.0 milestone Sep 6, 2020
mawillcockson added a commit to mawillcockson/flit that referenced this issue Sep 29, 2021
git is isolated using the following environment, which are set globally
for the duration of that the new pytest fixture "tmp_git" is used:

GIT_CONFIG_GLOBAL
GIT_CONFIG_NOSYSTEM
HOME
GIT_AUTHOR_EMAIL
GIT_AUTHOR_NAME
GIT_AUTHOR_DATE
GIT_COMMITTER_EMAIL
GIT_COMMITTER_NAME
GIT_COMMITTER_DATE

GIT_DIR and GIT_WORK_TREE could be set so that git can be called using
any method, but instead a git() function is added that uses git's -C
command-line option.

These were taken from one of git's test scripts:
https://github.com/git/git/blob/cefe983a320c03d7843ac78e73bd513a27806845/t/test-lib.sh#L454-L461

There are probably other ways git can be isolated.

The repository is initialized with an empty commit, but this isn't
strictly necessary, it just makes some of the possible tests require
less setup.

The new tmp_project fixture copies the sample module from
./test/samples/module1_toml to the project and commits the files.

A test is added for pypa#345 as an example of how this can be used.

A pytest marker is added so that tests with either "needgit" or
"needsgit" in the name are skipped if python can't find an executable
named "git".

The tox configuration is changed and another pytest marker is added so
that those tests can be run by themselves:

tox -- -m needgit

or skipped:

tox -- -m "not needgit"

Type hints were added to help with development, but aren't necessary to
keep.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants