copy over directories using extra_path_metadata

[sanjiverat]

• I have searched the issues (including closed ones) and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.
• I am willing to lend a hand to help implement this feature.

Feature Request

The EXTRA_PATH_METADATA can be used to copy over regular html/JavaScript/css files to specific locations. But if I want to copy over an entire directory that contains the files, it is not possible. The fix is fairly simple and requires to change just a single line in utils.py in pelican module. I have given the diff below.
Incidentally, copying directories (and their contents) used to work on pelican 4.7.1. But when I tried to install the new version of pelican, I found that this was broken. The change below fixes it.

pelican/utils.py

def copy_file(source: str, destination: str) -> None:
    """Copy a file"""
    try:
-        shutil.copyfile(source, destination)
+        shutil.copy(source, destination)
    except OSError as e:
        logger.warning(
            "A problem occurred copying file %s to %s; %s", source, destination, e
        )

[justinmayer]

Hi @sanjiverat. To copy a directory of files, the proper method is probably STATIC_PATHS.

The change from shutil.copy2() to shutil.copyfile() was made in #3228. You can review that PR and the linked issues to understand why that change was made. Given the documented reasons within, your proposed change is not likely to be adopted.

If there is a reason that STATIC_PATHS does not address your use case, we would need a lot more detail to understand why.

[sanjiverat]

Thank you, @justinmayer
My use case was as follows:

I have a directory called "extra" inside my content folder. Inside this "extra" directory, I have folders "folder1" which has the files index.html and myscript.js and myscript.css. I typically put {'extra/folder1' : {'path': 'folder1/'}} in the EXTRA_PATH_METADATA, which results in folder1 being copied over to the root directory of the output (so that the url is accessible as http://example.com/folder1).
I also have "extra" in ARTICLE_EXCLUDES and in STATIC_PATHS.

I hope that this question is not completely silly, but how do I have python copy over a directory to a specific place (like the root folder), without going inside and trying to process the files inside? When I put it inside STATIC_PATHS, without the EXTRA_PATH_METADATA, it doesnt seem to work.

[sanjiverat]

I was checking the PR request. It changed copy2 to copyfile. As I understand it, copy2 (and copy) accepts a target directory path, but copyfile will not (https://docs.python.org/3/library/shutil.html#shutil.copy).

So the change also had the unintentional (?) consequence of changing the semantics of EXTRA_PATH_METADATA which had allowed directories to be moved to the appropriate location earlier.

So it seems to me that unless the copyfile is changed to copy (which will keep avoid copying metadata, but will copy the permissions), we can use the EXTRA_PATH_METADATA to only move files around (and not directories as we could earlier).

[sanjiverat]

My bad. I dont think even the earlier version was copying directories. It was copying the files one at a time into the specified directory. So if the directory had more than one level, then the copying wouldnt have worked.

If we want to copy the directories, we would probably need copytree or some such thing?

Anyway, I think the simpler thing is to specify that the EXTRA_PATH_METADATA should only be files (and not directories).