Pixel/CM Summer 2018 overview

[vigliensoni]

Test patch-wise classification method and Pixel

• Test Pixel with single images in Rodan
• Test patch-wise CM with single images in Rodan
• Test opening and correcting the output of CM in Pixel in a single workflow
• Test patch-wise CM with many resources in the job queue (several pages)
• Design iterative, incremental Rodan workflow to correct portions of images and reclassify
• Test CM and Pixel at large scale (full manuscript)

In order to do implement bullet point 5, we need to work on:

• Finish patch-wise CM trainer wrapper
• Implement iterative jobs in Rodan
• Fix memory issue in Rodan Docker container

[EricHanLiu]

Pixel works as a job in Rodan taking jpg/png/tiff resource and outputs 3 layer resources

[vigliensoni]

@EricHanLiu Did you actually perform a full test? Were you actually able to export all image layers and open them in RODAN?

[EricHanLiu]

All image layers can be exported (either as highlights, image data) and downloaded locally, or sent directly to Rodan to be piped into future jobs (saved as resources). The outputs are rgba+png files.

[EricHanLiu]

On step two we encountered a bug with Keras, and so the new workflow, which contains our patch-wise CM job, cannot finish. The issue is created and I'd like feedback from Jorge before trying a couple workarounds were presented on this issue.

[EricHanLiu]

Jorge suggested it's a version inconsistency error (which I saw mentioned a lot also on the issue page), so I will try to verify which backend Keras is using and play around with the settings to see if that fixes the issue, otherwise maybe deleting the optimizer

[EricHanLiu]

Keras uses Theano like suspected, tried second method which was to remove optimizer weights in each model file using a python script. This should fix the issue and allow us to checkoff the second part (test patch-wise CM with single images in Rodan)

[EricHanLiu]

Patch-wise classification works with single images in Rodan (using the demo pre-trained models)

[fujinaga]

👍

[EricHanLiu]

Using patch-wise CM as a Rodan job and piping the output layers (excluding background) into Pixel in a single workflow works, and the output of patch-wise CM can be corrected in Pixel and resubmitted to Rodan (or exported to CSV/highlight png/image-data png)

[EricHanLiu]

Simultaneously testing adding another resource in the queue for the patch-wise classifier to run after the first pixel image was corrected, will update once it finishes (or segfaults)

[martha-thomae]

Patch-wise CM works with multiple images on the queue, we tried with 3 sections of full TIFF images, it generated the 4 layers (background, staff-lines, music symbols, and text) for each of them correctly.

[EricHanLiu]

Also works when multiple images are input to patch-wise CM, and then piped into Pixel, multiple pixel jobs can be opened to correct classification and then resubmit to rodan

[vigliensoni]

@EricHanLiu One Pixel job at a time in series or many of them in parallel?

[EricHanLiu]

It depends on when the CM job finishes, but Pixel can be opened as soon as the CM job finishes and the last test we did (with two images) they finished simultaneously so two instances of Pixel could be opened with different classifications (although with the same image since Pixel only takes 1 image input in one job, so one of the classifications was off)
Since the fast CM doesn't output the original image, we can't load Pixel with the correct image for every job

[EricHanLiu]

Pixel now can export a background layer along with the other 3 layers, with some caveats (see this issue). We've added another port, so the output of Pixel can be piped directly into the fast CM trainer as ground truth.

[EricHanLiu]

Pixel now almost entirely works, only bug that remains is the erasing other layers one. However, multiple exports with edits in between can now be done without error, and export to image data successfully works.

[vigliensoni]

☀️ ☀️ ☀️

[EricHanLiu]

Pixel's background layer is now generated through pixel subtraction instead of path/shape subtraction, so it is now compatible with the Calvo classifier's output. Background layer generation is assigned to a button, so user has the option to generate it.

[EricHanLiu]

Iterative, incremental Rodan workflow depends on some sort of Rodan loop being implemented, as well as the patch-wise classifier being finished.

[EricHanLiu]

selectRegionLayer is implemented with a tutorial tooltip and restrictions on bad usage. Pixel is a submodule so pixel_wrapper is the latest version, but create/delete layer buttons were removed since they shouldn't be features within a workflow.

Number of default layers is generated based on either the number of input ports, or a prompt will ask the user how many layers. The input and output ports for layers are now optional, with max being 7 layers.

The background is only generated within the selection regions, and it has been optimized to only iterate within those regions. Generation has been moved to submitToRodan button and Pixel closes automatically after exporting.

Not sure what else needs to be done for this issue aside from fixing the Docker swap memory bug, and figuring out what exactly we're doing with the CM training

[EricHanLiu]

How should we proceed here with the next checkbox?

Design iterative, incremental Rodan workflow to correct portions of images and reclassify

[vigliensoni]

We've implemented part of the workflow, but we still need to work on: - [ ] Finish patch-wise CM trainer wrapper - [ ] Implement iterative jobs in Rodan - [ ] Fix memory issue in Rodan Docker container

…

On Thu, Jun 28, 2018 at 1:06 PM, Eric Han Liu ***@***.***> wrote: How should we proceed here with the next checkbox? Design iterative, incremental Rodan workflow to correct portions of images and reclassify — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#5 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAeCxeaYaO_PkqheOM1jf4KDt3dM1h7cks5uBQ0xgaJpZM4Tjlap> .