Skip to content

Latest commit

 

History

History
94 lines (57 loc) · 4.75 KB

8.md

File metadata and controls

94 lines (57 loc) · 4.75 KB

Peer Review of RO-8

Review 1

Quality of Writing

Is the text easy to follow? Are core concepts defined or referenced? Is it clear what is the author's contribution?

  • good

The paper is well written and easy to follow

Research Object / Zenodo

URL for a Research Object or Zenodo record provided?   Guidelines followed?   Open format (e.g. HTML)?   Sufficient metadata, e.g. links to software?   Some form of Data Package provided?   Add text below if you need to clarify your score.

  • sufficient (e.g. HTML, detailed Zenodo metadata)

The article was available in Zenodo and appropriately described there.

Overall evaluation

Please provide a brief review, including a justification for your scores. Both score and review text are required.

  • accept

This paper presents an approach for capturing all the aspects of a scientific publication as a Research Object in the neuroimaging domain. The approach is highly relevant for the workshop, and I look forward to its presentation. That said, I have many questions that remain unanswered in the abstract:

  • What is the difference between repro-pubs and research objects?
  • What are the assumptions of a repropub? For example, data size is usually bigger than what it's allowed in a GitHub release.
  • How difficult is to create a repropub? How accessible is to users?
  • Does the docker image include all the dependencies in the workflow of a repropub, or does a repropub require setting up a workflow environment that invokes the respective containers of each step?

Review 2

  • Reviewer: (anonymous)

Quality of Writing

Is the text easy to follow? Are core concepts defined or referenced? Is it clear what is the author's contribution?

  • good

Research Object / Zenodo

URL for a Research Object or Zenodo record provided?   Guidelines followed?   Open format (e.g. HTML)?   Sufficient metadata, e.g. links to software?   Some form of Data Package provided?   Add text below if you need to clarify your score.

  • sufficient (e.g. HTML, detailed Zenodo metadata)

Overall evaluation

Please provide a brief review, including a justification for your scores. Both score and review text are required.

  • weak accept

This abstract introduces the “ReproPub” concept as a publication that includes the provenance of the related experimental data, workflow, execution environment, and results, in order to support re-executability of the research findings.

The ReproPub concept is, in fact, in the same line and spirit as the research object concept. Thus, for instance, a research object can aggregate other research objects (called nested research objects).

However, even though technically a research object could represent each of the mentioned elements (data, workflow, execution environment, results), as proposed by the author, the main purpose of the research object is to encapsulate all the materials and methods relevant to a scientific investigation, the associated annotations and the context where such resources were used and produced (including provenance).

Hence, it’s more about the overarching aggregation mechanism (as the ReproPub concept), rather than the individual element encapsulation.

One question is regarding the proposed implementation for ReproPub. From the text and the proof of concept reference, it seems that the proposed approach is to have a typical publication (e.g., textual document), which includes references to where the individual elements can be accessed, e.g., GitHub, DockerHub, Zenodo, etc. This can, of course, improve human readability and understanding, allowing to potentially reproduce the investigation.

However, as ReproPubs are not addressing the formal representation of relations and metadata supporting machine-readable access, they may leverage further research objects in order to support features like (automatic) quality assessment, decay prevention, findability, indexing, enrichment, etc.

Hence, it will be go to analyse further possible complementarities/relations between ReproPubs and research objects, e.g., having a ReproPub as a named type of resource in a research object, potentially being able to be created automatically from a research object content.