reddit - user profile download

[moeter87-real]

Hi,

i am pretty new to github and also to gallery-dl, so please forgive me if i doing/posting something (the) wrong (way) here. So i am using the lastest standalone windows version of gallery-dl (Version 1.23.5 - Executable). I know that you can configure the extractors in a config.json file (doen't really know where it has to be stored or if i always have to use -c on windows), i would like to configure the reddit extractor, that it downloads all files/media a user has posted in it's own directory.

If i just do something like ".\gallery-dl.exe https://www.reddit.com/user/username/submitted/" gallery-dl will create a ".\gallery-dl\reddit\u_username" but most files downloaded will be sortet in folders corresponding to the subreddit the media has been posted and not the u_username folder. I think that should be something that is possible to do with gallery-dl, the second thing (i am not so sure about this one) is, i would like to skip duplicate files, in best case by using file hashes instead of file or post names.

I could also install an linux vm if the usage of the windows version could become a problem.

Thanks for your efforts in advance and best regards
Alex

Edit:

Ok, i think i found a solution to my first and config file problem here #1862. Sorry, that i haven't also searched the issues before opening an discussion here.

[Hrxn]

I know that you can configure the extractors in a config.json file (doen't really know where it has to be stored or if i always have to use -c on windows)

You don't have to use any command line options. As specified in the configuration section, you can simply use one of the provided locations.
For Windows, place your file as one of these:

• %APPDATA%\gallery-dl\config.json
• %USERPROFILE%\gallery-dl\config.json
• %USERPROFILE%\gallery-dl.conf

If i just do something like ".\gallery-dl.exe https://www.reddit.com/user/username/submitted/" gallery-dl will create a ".\gallery-dl\reddit\u_username" but most files downloaded will be sortet in folders corresponding to the subreddit the media has been posted and not the u_username folder. I think that should be something that is possible to do with gallery-dl, the second thing (i am not so sure about this one) is, i would like to skip duplicate files, in best case by using file hashes instead of file or post names.

Yes, this is normal. Consider gallery-dl's perspective, if you try to extract from a reddit profile that actually contains not only content hosted on reddit itself - there are also URLs that point to other sites, like Imgur, Redgifs, etc.
gallery-dl recognizes these URLs if it's a supported site, and it continues with using the configuration options specified for that site which is external to reddit in our case. This is called running a child extractor, because, in our example here, the reddit extractor runs first, detects a Redgifs URL, and then hands it over to the Redgifs extractor..

This allows some very flexible user customizations, once you understand how it works.
Conceptually, it is nothing really complicated, all we have to do is use the collected metadata from the first extractor in the chain (i.e. reddit), and pass it along to the child extractor, so it can be reused there.

The option for that is called "parent-metadata", which needs to be added to the extractor used by the URLs you provide to gallery-dl:

{
  "extractor": {
    "reddit": {
      "subreddit": {
        "directory": ["Reddit", "R", "{subreddit}"],
        "filename": "{id}{num:?_//>02}_{title[:220]!t:R /_/}.{extension}"
      },
      "user": {
        "directory": ["Reddit", "U", "{author}"],
        "filename": "{id}{num:?_//>02}_{title[:220]!t:R /_/}.{extension}"
      },
      "refresh-token": "<your refresh token>",
      "comments": 0,
      "morecomments": false,
      "date-min": 0,
      "date-max": 253402210800,
      "date-format": "%Y-%m-%dT%H:%M:%S",
      "id-min": "0",
      "id-max": "zik0zj",
      "recursion": 0,
      "videos": true,


      "parent-metadata": "_reddit_"


    }
  }
}

And then, simply access this metadata in our child extractor, like this (I'm omitting the "reddit" section of the configuration file here):

{
    "extractor": {
        "redgifs":
        {
            "image":
            {
                "directory": {
                    "'_reddit_' in locals()": ["ClipsFromRedgifs"],
                    ""                      : ["Redgifs", "Clips", "Unsorted"]
                }
            },
            "filename": {
                "'_reddit_' in locals()": "{_reddit_[title][:180]!t:?/_/R /_/}{_reddit_[id]}.{extension}",
                "subcategory == 'user'" : "{filename}.{extension}",
                ""                      : "{filename}.{extension}"
            },
            "format": ["hd", "sd", "gif"],
            "fallback": true,
            "skip": true
        }
    }
}

[moeter87-real]

At first thanks for your fast an kind response.

With your suggested changes gallery-dl creates a subfolder in the main download directory named ClipsFromRedgifs, but the files that are created there doesn't contain the reddit username or something to identify them by. The most comfortable solution for me would be that gallery-dl creates a subfolder in the reddit users directory for other extractors. So that the folder structure for example looks something like this:

download root dir / reddit / u / username / redgifs / files.*
download root dir / reddit / u / username / imgur / files.*
download root dir / reddit / u / username / twitter / files.*

If i understand the config right it should look something like this, but i don`t know if can send the author value from the reddit extractor to the redgits extractor.

[...]

            "image":
            {
                "directory": {
                    "'_reddit_' in locals()": ["Reddit", "u", "{author}", "redgif"],
                    ""                      : ["Redgifs", "Clips", "Unsorted"]
                }

[...]

I tried this, but it doesn't seem to work. So instead files are downloaded to "download root dir / reddit / u / username / None/ files.*". So apparently i don't understand the configuration enought so far.

Best Regards

[Hrxn]

You have to use it like this: {_reddit_[author]}

[mikf]

The most comfortable solution for me would be that gallery-dl creates a subfolder in the reddit users directory for other extractors

That's what parent-directory is for.