Force UI Refresh on Browser

[Kriegbaum]

Hi, I am just getting started with trame. I have a question have dug through the existing discussions and I'm not quite understanding this concept though I suspect it is very solvable.

I have some MQTT devices that are controlled by a module that I wrote, I am attempting to use trame to provide a web interface, but other devices on the network can also change the state of these devices, so I would like the web interface to reflect those changes within a reasonable time after they happen. What I have now successfully gets the device changes into the server state, but the UI does not refresh until some other widget on the page is interacted with (Clicking on a button, pulling a slider, etc)

This is the class for the controls of one such device:

class FixtureControls:
    def __init__(self, fixture):
        self.fixture = fixture
        self.fixture_model = self.fixture.name.replace("-", "_")
        with html.Div(stye="padding: 10px;"):
            vuetify3.VSpacer()
            vuetify3.VColorPicker(v_model=(f"{self.fixture_model}_color", None), click=self.handle_color_change)
            vuetify3.VBtn(f"{fixture.name} On", click=self.handle_fixture_on)
            vuetify3.VBtn(f"{fixture.name} Off", click=self.handle_fixture_off)

    def handle_color_change(self):
        print(state[f"{self.fixture_model}_color"])
        color = rgb_hex_to_int(state[f"{self.fixture_model}_color"])
        self.fixture.set_color(color)

    def handle_fixture_on(self):
        self.fixture.on()

    def handle_fixture_off(self):
        self.fixture.off()

and the function that is called every time a device state is changed externally

def handle_device_update(device_name):
    print(f"Device: {device_name} updated")
    device = patch.fixtures.get(device_name) or patch.relays.get(device_name)
    device_model = device_name.replace("-", "_")
    if hasattr(device, 'state'):
        state.update({f"{device_model}_state": device.state})
    if hasattr(device, 'color'):
        state.update({f"{device_model}_color": rgb_int_to_hex(device.color)})
        state.dirty(f"{device_model}_color")

If, for instance, a light is blue and you can see that value in its color picker, another device on the network can change it to red. The UI will not react until another element on the page is interacted with, and then the color picker will snap to the correct color.

I have looked briefly at the StateQueue object here: https://github.com/Kitware/trame/blob/master/trame/app/asynchronous.py#L4-L5
But I am not fully grasping how this would be used for a case like this

[jourdain]

You don't need to tag your property as dirty since you are not mutating the object in place but simply replacing it.
What you are missing is the flush part but depending on when those method get executed (thread?), you may need to sync the execution to happen from the event-loop.

def handle_device_update(device_name):
    with state:
       print(f"Device: {device_name} updated")
       device = patch.fixtures.get(device_name) or patch.relays.get(device_name)
       device_model = device_name.replace("-", "_")
       if hasattr(device, 'state'):
           state[f"{device_model}_state"] = device.state
       if hasattr(device, 'color'):
           state[f"{device_model}_color"] = rgb_int_to_hex(device.color)

Also that section of the How to start guide is probably what you were looking for.

[Kriegbaum]

Hi jourdain, I appreciate your quick response. I've been working with your suggestions and I've made some progress.
The process with the lighting control is synchronous in a separate thread, and so I've been wrestling with getting it into an asnychronous environment. I must admit I still have a lot to learn about async.

That being said, I've added these

async def refresh_ui():
    print("refreshing ui...")
    with state:
        state.flush()
    print("UI refreshed")

loop = asyncio.new_event_loop()

and then in the handle_device_update function I've added

def handle_device_update(device_name):
    print(f"Device: {device_name} updated")
    device = patch.fixtures.get(device_name) or patch.relays.get(device_name)
    device_model = device_name.replace("-", "_")
    if hasattr(device, 'state'):
        state.update({f"{device_model}_state": device.state})
    if hasattr(device, 'color'):
        print(device.color)
        state.update({f"{device_model}_color": rgb_int_to_hex(device.color)})

    asyncio.set_event_loop(loop)
    loop.run_until_complete(refresh_ui())

When a device update arrives on the network, the ouput shows

Device: bedroom-iris updated
[0, 255, 255]
refreshing ui...
UI refreshed

But the UI is always one action behind, the UI will update to current state if I then keyboard interrupt in the terminal running the trame instance. I feel like I am very close! There is still something I am failing to understand though

[Kriegbaum]

For thoroughness, I've tried throwing a in a flush on all the objects that support it, this doesn't change the behavior at all

async def refresh_ui():
    print("refreshing ui...")
    with state:
        state.flush()
    server.js_call("flushState('bedroom_iris_color')")
    server.ui.flush_content()
    print("UI refreshed")

[Kriegbaum]

Okay, I have arrived at a solution that works!

    asyncio.set_event_loop(loop)
    loop.run_until_complete(refresh_ui())
    loop.run_until_complete(refresh_ui())

Just run the refresh function twice 😁
This satisfies me for what I am doing at the moment. If anyone out there has an idea about why this is happening I would be interested, but the performance impact of repeating this action appears to be negligible and this is okay for me

[jourdain]

Indeed async and Python can be tricky to grasp at the beginning. So base on what you are saying (hard to be sure of anything with the little info), it seems that you are starving the web server and therefore prevent any refresh. If you fix that part on your code that is synchronous, the code I share will be enough to fix your issue.

If you share your pulling loop, I might be able to help you convert into an async version and free-up the server so the refresh will work.

[sniper0110]

Hi,

What would change if I don't use async and use threading instead?

I have this code that handles uploading a file and also preparing input (conversion to nifti) to be visualized in the UI. When preparation starts, I want to show a VProgressCircular with some text next to it that describes what's happening. I control how to show the progresscircular with state.processing variable, and I control the text shown (inside a html span) using the state.message variable.

@state.change("uploaded_file")
def handle_file_upload(uploaded_file, **kwargs):

    print("file upload handler function has been called.")
    
    if uploaded_file:
        
        file_name = uploaded_file["name"]
        file_content = uploaded_file["content"]

        try:
            # Sanitize the file name to prevent security issues
            safe_file_name = os.path.basename(file_name)

            # Handle file_content based on its type
            if isinstance(file_content, str):
                # Check for data URI scheme and split if necessary
                if "," in file_content:
                    header, encoded = file_content.split(",", 1)
                else:
                    encoded = file_content

                # Decode the base64-encoded data
                data = base64.b64decode(encoded)
            elif isinstance(file_content, bytes):
                # Use the binary data directly
                data = file_content
            else:
                print("Unsupported type for file_content.")
                return

            # Save the file in the temporary directory
            file_path = os.path.join(temp_upload_dir, safe_file_name)
            with open(file_path, "wb") as f:
                f.write(data)

            print(f"File '{safe_file_name}' uploaded and saved to temporary directory.")

            # Get zip and convert to NIFTI
            data_handler = DataHandler(verbose=verbose, state=state)
            
            with state:
                state.processing = True
                state.message = "Processing DICOM.."
            # state.flush()
            threading.Thread(target=lambda: data_handler.prepare_input(file_path, temp_upload_dir, state)).start()
            
            # Update the state with the file path
            # state.uploaded_file_path = nifti_path

            # Trigger the pipeline update
            state.pipeline_needs_update = True
            
            # Enable segmentation button
            state.segment_btn_enabled = True
            
            state.output_3d_ready = False

            # Reset visualized 3D scene
            state.visualized_3d_structures = { 
                "Maxilla": None,
                "Mandible": None,
                "Upper_Teeth": None,
                "Lower_Teeth": None,
                "Canal": None
            }

            # Deactivate checkboxes
            state.Maxilla = False
            state.Mandible = False
            state.Upper_Teeth = False
            state.Lower_Teeth = False
            state.Canal = False

            state.enable_checkboxes = False

            
        except Exception as e:
            print(f"An error occurred while handling the uploaded file: {e}")

The preparation part happens in a separate thread because it could take some time. Once the preparation is done, I reset state.processing and state.message and I want that to be reflected in the UI (the progresscircular should disappear and text should become empty string). Unfortunately I haven't been able to get those changes reflected in the UI. Moreover, it seems that the app just halts in that level. No subsequent actions happen at all.

Here's the preprare_input code:

    def prepare_input(self, path_to_zip, output_dir, state):
        try:
            self.output_dir = os.path.join(output_dir, str(uuid.uuid4()))
            os.makedirs(output_dir, exist_ok=True)
            self.case_name = os.path.basename(path_to_zip).split('.')[0]
            
            # Provide the case name in the state
            state.case_name = self.case_name
            
            path_to_save_unzipped = os.path.join(self.output_dir, "dicom")
            os.makedirs(path_to_save_unzipped, exist_ok=True)

            # Unzip the zipped dicom folder in the given folder
            start = time.time()
            self.unzip_folder(path_to_zip, path_to_save_unzipped)
            if self.verbose:
                print("duration of unzipping: ", time.time() - start)

            # Find the first dicom series that contains at least 100 slices
            start = time.time()
            dicom_folder = self.find_dicom_series(path_to_save_unzipped)
            if self.verbose:
                print("duration of finding dicoms: ", time.time() - start)

            # Convert the dicom series into a nifti file
            start = time.time()
            os.makedirs(os.path.join(self.output_dir, "nifti_input"))
            input_nifti_path = os.path.join(self.output_dir, "nifti_input", self.case_name + '.nii.gz')
            self.dicom_to_nifti(dicom_folder, input_nifti_path)
            if self.verbose:
                print("duration for dicom to nifti: ", time.time() - start)
        except Exception as e:
            input_nifti_path = None
            print(f"Issue with the nifti preparation for the error {e}")
            print(traceback.format_exc())
        
        state.uploaded_file_path = input_nifti_path
        
        # state.processing = False
        # state.message = ""
        state.update({"processing": False, "message":""})
        
        return input_nifti_path

Any help would be appreciated.
Also, let me know if I should create a separate issue or discussion for this.

[jourdain]

Thanks for sharing the code but I don't have the time to look at it in more details. You sure can use threading, but you will need to sync your thread with the event loop in order to enable trame to properly sync with the front-end. In fact you can see such usage in the hot-reload section of that guide.

[jourdain]

You also have the option to wait till network is complete at key locations.