Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix accessing UID before first scan #102513

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Hilderin
Copy link
Contributor

@Hilderin Hilderin commented Feb 7, 2025

This PR should fix most of the issues where Godot tries to access an UID before the first scan is done and when the UID cache is missing, incomplete or invalid.

The idea is to execute a scan for UIDs on startup if an UID is used before the first scan. This scan is done a maximum of one time and, if executed, replaces the "normal" first scanning which loads the file structure of the project into first_scan_root_dir.

One potential problem with large project is that the scanning could take a while and the user has no feedback.

Also, the plugins are, maybe, not loaded when the scan of UIDs is executed. That means that it's possible that some custom loader will be missing, resulting in an impossibility to read some UIDs.

Tested with the MRP from #101677 and #100915.

@Hilderin Hilderin requested a review from a team as a code owner February 7, 2025 03:06
@Hilderin Hilderin added this to the 4.4 milestone Feb 7, 2025
@clayjohn clayjohn requested a review from KoBeWi February 7, 2025 03:46
@mihe
Copy link
Contributor

mihe commented Feb 7, 2025

One potential problem with large project is that the scanning could take a while and the user has no feedback.

For what it's worth, when testing this PR on a real-world project with thousands of resource files, I'm not seeing any noticeable regression in pure wall time when deleting uid_cache.bin and measuring from editor start to when all the various scan dialogs and progress bars are gone.

@Hilderin
Copy link
Contributor Author

Hilderin commented Feb 7, 2025

I did some tests this morning, and below are the added loading times for the editor when the UID cache is missing, and a UID is required before the first scan (e.g., run/main_scene project settings):

  • 3000 files (without .import and .uid): 1.5 sec
  • 60000 files (without .import and .uid): 48 sec

One way to improve this could be to avoid scanning the entire project when looking up a single UID and instead stop as soon as it is found. However, the challenge and potential drawback of this approach is that if multiple UIDs are needed, multiple rescans may be required.

That's why I suggest scanning everything. Technically, this "scan for UIDs" should not happen often, except in CI/CD environments where the UID cache will always be missing.

@mihe
Copy link
Contributor

mihe commented Feb 7, 2025

  • 3000 files (without .import and .uid): 1.5 sec
  • 60000 files (without .import and .uid): 48 sec

Right, yeah, the "real-world project" in question was definitely closer to 3k than 60k. A second here or there wasn't really what I was looking out for. :)

core/io/resource_uid.h Outdated Show resolved Hide resolved
@Hilderin Hilderin force-pushed the fix-accessing-uid-before-first-scan branch from 281cbdd to 749b57c Compare February 7, 2025 16:35
@KoBeWi
Copy link
Member

KoBeWi commented Feb 7, 2025

Fixes the issues.

Tested with my big project and the scan was not noticeable. I tested by deleting uid_cache.bin and filesystem_cache10 and I assume the project is supposed to hang on splash screen.

However there is one error remaining - if your splash screen path is stored as UID, you will still get an error about unrecognized UID.

@Hilderin Hilderin force-pushed the fix-accessing-uid-before-first-scan branch from 749b57c to 2ed5415 Compare February 7, 2025 17:37
@Hilderin
Copy link
Contributor Author

Hilderin commented Feb 7, 2025

Thanks for the review and testing.

Fixing the boot splash screen will be a bit trickier. The problem is that the logo is loaded before the resources format importers are loaded. If I move ResourceUID::scan_for_uid_on_startup = EditorFileSystem::scan_for_uid; just before setup_boot_logo, the ResourceFormatImporter::get_singleton()->get_recognized_extensions returns no extensions. That causes to not be able to get the UID for extensions like .tscn.

Since the issue is not supposed to happen that often, I did a little check before callingResourceUID::ensure_path on the boot logo path to prevent printing errors.

The alternative could be to display the boot logo later in the start sequence or put the loading of the ResourceFormatImporter sooner, but I fear that could lead to serious side effects and this close to a release candidate, I think it's a bad idea for this small issue.

@KoBeWi
Copy link
Member

KoBeWi commented Feb 7, 2025

Yeah, removing errors is enough.

main/main.cpp Outdated Show resolved Hide resolved
@KoBeWi
Copy link
Member

KoBeWi commented Feb 7, 2025

I did a little check before callingResourceUID::ensure_path on the boot logo path to prevent printing errors.

Did you test whether it fixes the errors? 🙃
image
If the UID is not available, it should make the path empty to fallback to default splash screen.

@Hilderin
Copy link
Contributor Author

Hilderin commented Feb 7, 2025

Really sorry about that, I went too fast! Tested correctly this time!

@Hilderin Hilderin force-pushed the fix-accessing-uid-before-first-scan branch from 2ed5415 to 1b6a0d8 Compare February 7, 2025 19:15
@Hilderin Hilderin force-pushed the fix-accessing-uid-before-first-scan branch from 1b6a0d8 to 9457666 Compare February 9, 2025 02:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants