-
Notifications
You must be signed in to change notification settings - Fork 87
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
下载文件名问题和zip描述信息改动等(掺入一些灵魂的 #54
Open
dynilath
wants to merge
73
commits into
fffonion:dev
Choose a base branch
from
dynilath:dev
base: dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
changed some SL
…to jpg format by exhentai, it leaves a problem.
…will be converted to .jpg file
…sidering write the id - name relational mapping into files.
ignore vs files
merge master into dev
glob.escape is not supported in python 2.x, use other method to filter files instead. output zipfile meta into xehdone, nothing more yet.
那么♂多,待我看看 |
The most noteworthy is that reload url had changed a bit. Now it can be mix-up of letters and numbers and dashes.
when load from h.json, failcode of task needs reset
fffonion
force-pushed
the
dev
branch
7 times, most recently
from
March 26, 2020 01:20
6689468
to
fa63e8f
Compare
fffonion
force-pushed
the
dev
branch
9 times, most recently
from
July 22, 2020 12:20
f61362f
to
9d96377
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
和之前的那个pr差不多,主要是修了些问题。
1,新的文件名控制。
在exhentai浏览画廊单独一页时,比较大的png都会变成jpg,比较小的png会保持png。具体是否会压缩的分界点和图片源的链接带宽有关系。这也即是说原本是png的画廊下载的文件实际是png还是jpg完全无法简单预计。
除此之外,gif文件也会被下载成jpg。
这里采用了另一种办法:在ehentai的画廊页面里,每一页预览的title里面包含了原本的文件名。单独一页中的顶部和底部显示的文件名是压缩转码后的文件名。我使用这些文件名取代了原本的rename_map机制。
具体工作方式如下:
此时如果出现重名,添加"_1"到文件名末尾,仍然重名则使用"_2",以此类推
2,zip描述信息改动
把h.json里面的一些信息搬到了zip里,包括gjname、gnname、tags、total、title、rename_ori、download_ori、url、fid_fname_map。最后这个task.fid_fname_map是上述的文件名控制用到的,用于描述第几页文件名为何。
这使得zip文件能够自我描述比较完整的画廊信息,在TASK_STATE_SCAN_PAGE阶段中,这些信息也用于检测zip文件是否需要重新下载等等。
这也使得载入其他方式转移来的文件成为可能,让xehentai有可能成为一个下载画廊管理软件。
3,scan_downloaded和prescan_downloaded
由于上述文件名控制原理,到TASK_STATE_SCAN_IMG的时候才能知道实际的文件名。
prescan_downloaded负责扫描zip文件的描述信息。通过以下四个方面决定如何操作该zip:
scan_downloaded主要负责下了一半的东西暂停继续。将下载目录中的文件和fid_2_file_size_map中的数据对比,如果通过对比则不会加入page_q。
4,一些小问题
画廊名称结尾有多个.时会被win系统全部吃掉。更改了一下filename_filter。
由于重新启动时,图片源的ip会发生变化,画廊会更新(尤其是ongoing的画廊),tag也会更新(尤其是早前下载的新画廊)。所以重新载入TASK_STATE_DOWNLOAD状态的任务时,改为退回到TASK_STATE_GET_META状态。