Skip to content

Commit

Permalink
Merge pull request #74 from joshmoore/about-update
Browse files Browse the repository at this point in the history
Update About text
  • Loading branch information
joshmoore authored Oct 28, 2024
2 parents 3dd8f95 + 57bddcc commit de8d2bc
Show file tree
Hide file tree
Showing 2 changed files with 41 additions and 26 deletions.
59 changes: 36 additions & 23 deletions ome2024-ngff-challenge/src/About.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -10,57 +10,70 @@

<ul>
<li>
Collect examples of OME-Zarr from multiple modalities, use cases, and
institutions, available from a single location.
Advancing the status of OME-Zarr specification proposals —
<a href="https://ngff.openmicroscopy.org/rfc/">Requests for Comment (RFC)</a> —
especially the adoption of the Zarr V3 specification
</li>
<li>
Test the utility (and lack thereof) of existing OME-Zarr metadata and
image viewers.
Collecting the URLs of converted examples of OME-Zarr from multiple modalities, use cases, and
institutions, all hosted by each participant themselves
</li>
<li>
Advance the adoption of the Zarr V3 specification, and also existing
OME-Zarr RFCs.
Testing the level of maturity of OME-Zarr conversion tools, image viewers,
and metadata in general
</li>
</ul>
<p>
The inspiration for the Challenge came from presentations at the OME Meeting
that demonstrated the level of adoption of OME-Zarr, but highlighted the
poor findability of the data.
that demonstrated the level of adoption of OME-Zarr and the relative ease of
conversion for some community members, but highlighted the often poor findability of the data
and the need for driving the OME-Zarr specification towards version 1.0.
<p>
A lofty goal of 1 Petabyte (PB) of data was set, mostly to make clear the scale of what <em>could be</em>.
</p>
<p>
Work on the challenge started in earnest in July 2024. All progress is
tracked in the <a href="https://github.com/ome/ome2024-ngff-challenge">ome2024-ngff-challenge</a>
repo. It was agreed that the results
of the OME-NGFF Challenge would be presented at the
repository. It was agreed that the deadline for submission would be such that the results
of the OME-NGFF Challenge could be presented at the
<a href="https://globalbioimaging.org/exchange-of-experience/exchange-of-experience-ix">
2024 Global BioImaging Meeting</a>.
</p>

<h2>Process</h2>
<p>
The Challenge was run via a series of virtual meetings coordinated on the
The Challenge was run via a series of virtual meetings open to all which were coordinated on the
Image.sc Forums, with all
<a href="https://hackmd.io/3emKqKQsT_2U35vLepzDEQ">notes and lists of participants</a>
available.
available. Initial meetings including that at OME2024 focused on defining the scope of
changes that would be made to the OME-Zarr format. A Python-based tool to convert data
to this format was built. Submissions were then collected in simple CSV files with a Zarr
URL per row.
</p>

<h2>Results</h2>
<p>
At the outset, we weren’t sure how much OME-Zarr data was available, if it
could be converted to Zarr V3, or how many organisations would participate.
In the end, we have far exceeded our expectations, with more than 0.5 PByte
of OME-Zarr made available, across a wide range of modalities.
At the outset, it was unclear how much OME-Zarr data was available, if it
could be converted to Zarr V3, and how many organisations would participate.
In the end, we are pleased with the more than 500 Terabytes of data which
have been made available, across a wide range of modalities.
</p>
<p>
To make the assembly as accessible as possible, we built the the OME-NGFF
Challenge Viewer [URL?}, which incorporates a CSV with locations of
datasets, and connects datasets to the OME NGFF Validator, for metadata
validation and viewing.
To make the collection as accessible as possible, the
<a href="https://ome.github.io/ome2024-ngff-challenge">OME-NGFF Challenge Viewer</a>,
parses submissions from contributors and provides a single view across
all Zarr files. Data can be searched by key-word,
filtered and sorted by various metadata and browsed with thumbnails,
all generated on the fly. Links allow opening of datasets with the OME NGFF
Validator for metadata validation and viewing.
Please let us know on image.sc or GitHub if you have any issues
or ideas.
</p>
<p>
Perhaps the most important outcome of the Challenge is the establishment of
a federated bioimage data system based on OME-Zarr. To our knowledge, this
is the largest federated bioimage data collection ever assembled.
Looking back, perhaps the most rewarding outcome of the Challenge is that with a remarkably
modest investment of time and cloud resources, we have almost inadvertently prototyped
a federated bioimage data system based on OME-Zarr, the largest one we know of.
We hope future challenges will continue to push the state-of-the-art forward.
</p>
</div>

Expand Down
8 changes: 5 additions & 3 deletions ome2024-ngff-challenge/src/Main.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -173,9 +173,10 @@
<div class="summary">
<h3 style="text-align:center">
{#if csvUrl == SAMPLES_HOME}
In the summer of 2024, the OME-NGFF project generated
<strong style="font-weight:600">{filesizeformat(totalBytes)}</strong> of data in Zarr v3 format
{/if}
<strong style="font-weight:600">{filesizeformat(totalBytes)}</strong>
of public bioimage data in the next major version of OME-Zarr,
including Zarr v3 and sharding, with distributed hosting.
{:else}
<div style="font-size: 90%">
Showing Collection:
<a href="{csvUrl}">{csvUrl.split("/").pop()}</a>
Expand All @@ -185,6 +186,7 @@
<a href="{window.location.origin + window.location.pathname}">Show all collections</a>
{/if}
</div>
{/if}
</h3>
<div class="textInputWrapper">
Expand Down

0 comments on commit de8d2bc

Please sign in to comment.