X/Y/Z script

[ledahu]

Hello,

I tryed to implement the X/Y/Z script for ReActor and see how codeformer_weight and Restore_visibility works.

I succeed by some "tricks" (im not python dev) but i can't figure out the difference between 'codeformer_weight' and 'face visibility restorer'

and it seems that "face visibility"=0 && 'codeformer_weight'=0 => swap done . I was expecting 'no swap' at all

I need to improve my code but here how i done it, inspired from the 'hypertile_xyz.py'

first , it seems that the options in x/y/z must be a key in shared.opts.data Dict

1- i create a reactor_xyz.py

from modules import scripts
from modules.shared import opts
from pprint import pprint

xyz_grid = [x for x in scripts.scripts_data if x.script_class.__module__ == "xyz_grid.py"][0].module



def float_applier(value_name:str, min_range:float = 0, max_range:float = 1):
    """
    Returns a function that applies the given value to the given value_name in opts.data.
    """
    def validate(value_name:str, value:str):
        value = float(value)
        # validate value
        if not min_range == 0:
            assert value >= min_range, f"Value {value} for {value_name} must be greater than or equal to {min_range}"
        if not max_range == 1:
            assert value <= max_range, f"Value {value} for {value_name} must be less than or equal to {max_range}"
    def apply_float(p, x, xs):
        validate(value_name, x)
        opts.data[value_name] = float(x)
    return apply_float


def add_axis_options():
    # name, type, apply_function(var,min,max)
    extra_axis_options = [
        xyz_grid.AxisOption("[ReActor] CodeFormer Weight", float, float_applier("codeformer_weight", 0, 1)),
        xyz_grid.AxisOption("[ReActor] Restorer visibility", float, float_applier("restorer_visibility", 0, 1)),
    ]
    set_a = {opt.label for opt in xyz_grid.axis_options}
    set_b = {opt.label for opt in extra_axis_options}
    if set_a.intersection(set_b):
        return

    xyz_grid.axis_options.extend(extra_axis_options)

then i import it in the reactor_api.py

from scripts.reactor_xyz import add_axis_options
.
.
.

try:
    import modules.script_callbacks as script_callbacks

    script_callbacks.on_before_ui(add_axis_options)
    script_callbacks.on_app_started(reactor_api)
    
    
except:
    pass

At this step, the 2 [reactor] must appear in the x/y/z list , but not binded

then, im too noob to understand how the xyz_grid pass the values to the reActor , so... i used a bad practice...

in reactor_faceswapp.php , in the 'process' , i pop the key from shared.opts.data ...

            #self.face_restorer_visibility = face_restorer_visibility
            self.face_restorer_visibility = shared.opts.data.pop('restorer_visibility',face_restorer_visibility) 
.
.
.
            #self.codeformer_weight = codeformer_weight
            self.codeformer_weight = shared.opts.data.pop('codeformer_weight',codeformer_weight)

Note: i encountered some cache issues, its the reason of the dict.pop , to remove the key . Otherwise, if you stop using the x/y/z script it continue to use the values in shared.opts.data cache.

worked for me.
Hope it can help.

[jgonzalez-89]

Hello,

It looks like you've managed to integrate custom options into a ReActor script for controlling codeformer_weight and restorer_visibility through the X/Y/Z interface in a creative way, despite not being a Python developer.

Understanding codeformer_weight and restorer_visibility
CodeFormer Weight: This parameter likely controls the intensity or influence of the CodeFormer model in the face swapping or restoration process. CodeFormer is an AI model used for face restoration and enhancement, which can improve facial details, remove artifacts, and enhance overall image quality. A higher codeformer_weight means the effects of CodeFormer will be more pronounced.

Restorer Visibility: This parameter seems to control the visibility restoration aspect of the process, possibly related to making obscured or partially visible faces more clear or restoring them in some way. The exact mechanism depends on how the restoration is implemented, but generally, it could mean enhancing features of faces that are not fully visible or are degraded in some way.

Behavior Observation
When both codeformer_weight and restorer_visibility are set to 0, and you observed a "swap done" instead of "no swap at all," this suggests that these parameters might not be directly controlling the activation/deactivation of the face swap feature itself. Instead, they likely adjust the intensity or the quality of the face swap/restoration process. So, even at 0, the process might still proceed, just without the enhancements or restorations these models provide.

Suggestions for Improvement
Understanding X/Y/Z Grid Interaction: The X/Y/Z grid in your setup acts as a user interface for dynamically adjusting parameters. The values from this grid are passed to the ReActor (or any other script) through shared.opts.data. Your method of popping values from shared.opts.data in reactor_faceswapp.php works but is unconventional. A more standard approach would be to directly access these values without popping them, to maintain consistency and avoid potential errors with repeated use or in different contexts.

Avoiding Cache Issues: If you're encountering cache issues where old values persist, ensure that the values in shared.opts.data are reset or updated appropriately at the start or end of each operation. This could be handled within the lifecycle hooks of your scripts (e.g., on_app_started, on_before_ui, etc.) to ensure a clean state.

Code Structure and Practices: Given your innovative solution, consider refining your code structure and practices by:

Using more direct methods for parameter passing and state management.
Ensuring your custom options are cleanly integrated into the existing script framework.
Documenting your modifications and understanding the flow of data within the application to make debugging and further customization easier.
Your approach demonstrates a good level of problem-solving and initiative. To improve further, consider diving deeper into Python and the specific frameworks or libraries your project uses. This will help you understand the best practices for code structure, state management, and integration with existing systems, leading to more robust and maintainable code.

[ledahu]

Thx,

but as i understand the code , codeformer_weight is a parameter for codeformer , and restore_visibility a parameter for GFPGAN

as defined in :
def restore_face(image: Image, enhancement_options: EnhancementOptions):

so i think that its a Beta, and restore_visibility && codeformer_weight will merge in a single value someday ( Mr Gourieff, can you confirm or i missed something ? )

For the dict.pop , if i look at hypertile method or controlnet method to implement x/y/z
it seems to me that hypertile just use shared.opts for each var of the script (so they don t need to monitor/sync/clear the value)
and controlnet use setattr to apply
Its a Dict, immutable
so, you are right, im gonna search to the lifecycle to avoid a Dict.pop to clean cache.

finally, the only weak point of my tricks is : what if another "extension" whant to use the same shared.opts of ReActor after i poped it out ?
yes, its bad ...
So im gonna rename the key from 'codeformer_weight' to 'reactor_codeformer_weight' ! ;)

This extension is really great!

[Gourieff]

but as i understand the code , codeformer_weight is a parameter for codeformer , and restore_visibility a parameter for GFPGAN
...
so i think that its a Beta, and restore_visibility && codeformer_weight will merge in a single value someday ( Mr Gourieff, can you confirm or i missed something ? )

Hello!
Thanks for your efforts 👍
restore_visibility - is just a simple blending option, if it's lower then 1 we blend the restored layer and the provided image with a percentage value of restore_visibility
codeformer_weight - is the fidelity parameter of the CodeFormer model: 0 (more restoration, less of the provided face) -> 1 (less restoration, more of the provided face)

[ledahu]

ok
there is a bug in my code
in fact, when the script_callbacks.on_before_ui(add_axis_options) is called, the shared.opts is initialized...

so after searching how i can get the state of the x/y/z script running or not, i only found that there is a shared.state.job_count that is >0 if script xyz is running

so here is the modification in reactor_faceswap.py :

            self.face_restorer_visibility = face_restorer_visibility
            if(shared.state.job_count>0):
                self.face_restorer_visibility = shared.opts.data['restorer_visibility']

and

            self.codeformer_weight = codeformer_weight
            if(shared.state.job_count>0):
                self.codeformer_weight = shared.opts.data['codeformer_weight'] 

So now, all seems fine

Then , using it, i focused in restore_face() to understand why if 0 in restore_visibility do not get the original image (mean the generated by SD)
then i understood that in reactor_swapper.py the image given is not the true original image but the already-swapped one !
so i added a parameter to give the non-swapped one and put it to the blend, and propagate the new parameter to all enhance functions

def restore_face(image: Image, enhancement_options: EnhancementOptions,target_img_orig:Image.Image):
    result_image = image

    if check_process_halt(msgforced=True):
        return result_image
    
    if enhancement_options.face_restorer is not None:
        original_image = result_image.copy()
        logger.status("Restoring the face with %s", enhancement_options.face_restorer.name())
        numpy_image = np.array(result_image)
        if enhancement_options.face_restorer.name() == "CodeFormer":
            numpy_image = codeformer_model.codeformer.restore(
                numpy_image, w=enhancement_options.codeformer_weight
            )
        else: # GFPGAN:
            numpy_image = gfpgan_model.gfpgan_fix_faces(numpy_image)
            # numpy_image = enhancement_options.face_restorer.restore(numpy_image)
        restored_image = Image.fromarray(numpy_image)
        result_image = Image.blend(
            target_img_orig, restored_image, enhancement_options.restorer_visibility
        )
    
    return result_image

prompt : a portrait of an ugly woman
xyz :0,0.5,1
ReActor facemodel generated by another prompt: photography beautifull blond woman looking at viewer

[Gourieff]

Done! With some modifications
3 axes are available (face masking as well)

[Gourieff]

One more axis is ready: "Face Models"

[ledahu]

Great ! Thank you !

Just to let me understand : why did you let the blend with the restorer_visibility apply to the allready swaped face and not the true_original_face as i done in my last code ? What did i miss ?

[Gourieff]

Because it will be a mess instead of the face :)
The restorer_visibility parameter is intended to adjust how much of the restored face (swapped and restored) we blend with the swapped but not restored one