Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add task manager #16570

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from
Open

add task manager #16570

wants to merge 1 commit into from

Conversation

wkpark
Copy link
Contributor

@wkpark wkpark commented Oct 20, 2024

based on https://github.com/lllyasviel/stable-diffusion-webui-forge/blob/main/modules_forge/main_thread.py

original author's comment in the source code:

This file is the main thread that handles all gradio calls for major t2i or i2i processing.
Other gradio calls (like those from extensions) are not influenced.
By using one single thread to process all major calls, model moving is significantly faster.

and it's commit:
lllyasviel/stable-diffusion-webui-forge@f06ba8e

from commit message:

original author said:
This will move all major gradio calls into the main thread rather than random gradio threads.
This ensures that all torch.module.to() are performed in main thread to completely possible avoid GPU fragments.
In my test now model moving is 0.7 ~ 1.2 seconds faster, which means all 6GB/8GB VRAM users will get 0.7 ~ 1.2 seconds faster per image on SDXL.

  • classified
  • fix workflow (all tests have been passed but coverage not work)
  • this way, gc.collect() will work as expected

extracted from PR #16484

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant