Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to fetch book data on upload #2333

Merged

Conversation

kieraneglin
Copy link
Contributor

@kieraneglin kieraneglin commented Nov 20, 2023

IMPORTANT NOTE

This PR is work-in-progress, mainly when it comes to the finishing touches in the UI. I will likely eventually need some UI direction, but for now the focus is on the general concept's viability.

Issue:

When importing media via the upload page, it can be tedious to import lots of media if you also want to ensure the author/series name is correct. This has the most impact to the structure of files stored on-disk - keeping this data correct will namespace media under the desired author/series directories. Keeps things clean!

A secondary issue comes from the details of media not actually being what you expect or may casually call it. For example, people (me) may call this series The Kingkiller Chronicles, according to Audible it's just Kingkiller Chronicle. Again, this mostly impacts keeping the directory structure of things well-formed and consistent.

Fix Description

I've added some UI to allow for automatic fetching of media data based on what data you have entered in so far (title is required). This fires on-selection of media as long as the title gets auto-populated! If it returns the wrong information, you can always edit it to your preference OR give it a little hint by updating, say, the author and fire a re-fetch manually.

As mentioned, this also provides a way to manually fetch details for a single piece of media. This is helpful for re-fetching media if-needed, but it's also good for when the uploaded filename is VERY different than the actual title of the media.

NOTE: this only works on audiobooks.

Additional Fixes:

  • Fixed directory input being smaller and misaligned compared to the series input on upload page
  • Added .trim modifier to upload form inputs to chop leading/trailing whitespace
  • Updated error string in the ItemUploadCard's getData method to use an I18n string rather than hardcoding

Regressions:

None known.

GitHub Issues Resolved:

None known.

Screen Captures

ABS.Auto-fetch.2.mov

@lukeIam
Copy link
Contributor

lukeIam commented Nov 24, 2023

@kieraneglin I was looking for some feature like that - would love such a feature.
Thanks for you work so far!

@kieraneglin kieraneglin marked this pull request as ready for review November 28, 2023 20:28
@kieraneglin kieraneglin changed the title [WIP] Add ability to fetch book data on upload Add ability to fetch book data on upload Nov 28, 2023
@advplyr
Copy link
Owner

advplyr commented Dec 1, 2023

This is working great and you taught me about the trim modifier which is awesome. My VS code has some different auto-formatting settings which is my fault (still have to go through #2183) but I made some minor UI adjustments to have more consistent spacing.

I came across one issue that has been an existing bug but got highlighted when fetching the proper title. I'm using Windows and I matched a book that had a : in the title. The bug is that this crashed the server since it isn't sanitizing the file path but I think this also should be handled client side somehow.

There is a feature request open to have the scanner also parse subtitles with : but I'm not sure if we should write files with that since we don't know if the filesystem will support it.

@kieraneglin
Copy link
Contributor Author

@advplyr great to hear! Regarding that bug, I don't necessarily agree that sanitizing file names should be a client-side concern for a few reasons:

  1. In general, any important validation should be done server side since you can always circumvent client-side validation
  2. The client isn't necessarily aware of what operating system the server is running on (nor should it be IMO), which ties into:
  3. Reserved characters, control characters, and reserved file names vary per-os. I believe: is totally fair game on UNIX, and you can't name a folder CON on Windows, for instance. Given this, it might be best to delegate sanitization to an existing package. Maybe the list is small enough to replicate the functionality in-app, but in any case this really feels like a concern of the server to me.

Please let me know what you think!

@kieraneglin
Copy link
Contributor Author

kieraneglin commented Dec 1, 2023

Ah, I see the issue - I forgot that the tempfile would also break things so filenames would need to be sanitized before they're written as a tempfile. I'll see what options are available server-side since I don't think it's ideal to be able to crash the server with a well-crafted filename, but as a last resort I'll update it to do client-side sanitizing

I was wrong here, the tempfiles upload with a safe filename. I just have to worry about what the tempfile is renamed to 🤙🏻

@advplyr
Copy link
Owner

advplyr commented Dec 1, 2023

I definitely agree this needs to be handled server side. I thought it may be useful to also check for this client side to better inform the user, but as you said it would be difficult for the client to know the proper way to sanitize.
We are sanitizing file paths for podcasts using the Windows restrictions. When I set that up I dug into it a bit and it seemed better to enforce the strictest since podcast episodes are auto-downloaded and the filepath is not a user input. The function for sanitizing the filename is

module.exports.sanitizeFilename = (filename, colonReplacement = ' - ') => {

We could force sanitize it that way server side or we could just catch the exception and return the error. The error will not happen until the upload completes which can be slow for some users.

Edit: the exception needs to be caught either way

@advplyr
Copy link
Owner

advplyr commented Dec 1, 2023

Oh yeah the sanitize filename function also exists client side https://github.com/advplyr/audiobookshelf/blob/master/client/plugins/init.client.js#L53

@advplyr
Copy link
Owner

advplyr commented Dec 2, 2023

Awesome, thanks!

@advplyr advplyr merged commit fbc2c2b into advplyr:master Dec 2, 2023
@kieraneglin kieraneglin deleted the ke/feature/upload-auto-fetch-data branch December 2, 2023 23:58
advplyr added a commit that referenced this pull request Dec 8, 2023
Follow up for sso-redirecturi and #2305 and #2333
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants