Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support server content file manifest via manifest.txt to reduce number of guessing #85

Open
Crystalwarrior opened this issue Sep 7, 2021 · 4 comments
Labels
enhancement New feature or request

Comments

@Crystalwarrior
Copy link

Crystalwarrior commented Sep 7, 2021

Right now, WebAO has to make ridiculous guesses as to what kind of file extension the char.ini is referring to, because AO is AO and the content is never referenced with its extension present.

For KFO, I created a batch tool to check if you're missing any files by comparing against a manifest.txt that I generate via commandline. The manifest.txt is generated like this:
dir/s/b > manifest.txt
Afterwards, I simply open it in a text editor (e.g. VisualStudio Code), Ctrl+F the absolute filepath, and replace it with empty space to make it all relative filepaths.
VbZMh3qO1q

This same manifest can be used to fully understand which exact file path the char.ini is referring to, making sure that WebAO always only makes one call to the exact file extension that it knows in advance. This will also massively reduce latency, as it no longer has to guess a .gif, .webp or .apng, and it also will understand if Case Cafe-style (a)/ (b)/ folders exist (#84).

Server owners would have the responsibility of making sure the manifest.txt exists in the asset repository they have. Otherwise, the old guessing algorithm will be used.

@Crystalwarrior
Copy link
Author

Crystalwarrior commented Sep 7, 2021

Here's the manifest txt I generated in the .gif
manifested.txt

(rename it to manifest.txt, I only named it manifested to not replace the manifest.txt I already have)

@Crystalwarrior Crystalwarrior added the enhancement New feature or request label Sep 7, 2021
@oldmud0
Copy link
Member

oldmud0 commented Sep 18, 2021

Hmm. This wouldn't be a bad idea after all.

@notcancername
Copy link

notcancername commented Jan 31, 2023

While talking about the possibility of supporting new formats, this came up:

(me): Perhaps WebAO should include an index? Just the output of `find` should already be a massive performance benefit.
[...]
(me): For the official client, the output of `find . -type f` is only 108K (2759 files).  This would be much more for a client like KFO, but even there the benefit from the index should outweigh the delay of making 3 HTTP requests for every animation.

I very much agree with this change. Making one request for every possible file extension to check for its existence is a massive hack caused by HTTP not having a proper mechanism for directory listings.

It worries me that this issue is apparently well-known (Salanto brought it up in response to an example addition of a format), but the obvious solution isn't implemented. I'm going to try and implement it.

@notcancername
Copy link

This is definitely not a good idea as-is. It's too big and will destroy users' mobile data plans. I'm looking into a better solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants