ENH: API overhaul - foundations

[oesteban]

Summary

This PR continues the work initiated in #114 and #115 to create a new interface for SDCFlows. This overhaul has the vision of converting SDCFlows into some sort of subordinate pipeline to d/fMRIPrep, with a more independent functioning like sMRIPrep.

The idea is to consider fieldmaps a first-citizen input for which derivatives are generated at the output (on the same vein of, and effectively implementing #26).

CHANGES

• Moved estimation methods under the workflows module, to keep consistency with all NiPreps. Now, estimation methods are found as sdcflows.workflows.fit.{pepolar,fieldmap,syn}. Other than that, these workflows have not changed substantially from ENH: New objects for better representation of fieldmap estimation #114.
• New outputs submodule sdcflows.workflows.outputs that writes out reportlets and derivatives, following suit with higher-level NiPreps (s/f/dMRIPrep). The two workflows are exercised in the CircleCI tests*, and the artifacts are generated this way. The DerivativesDataSink in this module will need some updates to the JSON config of PyBIDS (within NiWorkflows) to allow some new names (e.g., to generate desc-preproc_magnitude files). Please check the JSON sidecars that are currently generated in CircleCI and whether they seem comprehensive enough - they forward IntendedFor metadata, link to corresponding coefficients files (only TOPUP for now), and link the appropriate, preprocessed reference (magnitude or epi average).
• Pilot draft of the new sdcflows.workflow.base. init_fmap_preproc_wf. At the moment, it only picks B₀ fieldmaps (i.e., _fieldmap, _phase1/2, _phasediff) and PEPOLAR if two or more _epi consistently named and with different direction entities are found under the same session. That PEPOLAR case is the only one that can be implicitly assumed in a somewhat safe manner. This pilot draft will be completed when [ENH] Allow encoding the fieldmapping intent of the protocol bids-standard/bids-specification#622 is finally merged - only then there will be a clear definition of fieldmap objects (see Implement B0FieldIdentifier in FieldmapEstimation object #125, Generate B0FieldIdentifier from IntendedFor fields, when it is missing #126).
• Separated out phase information manipulation functions to sdcflows.utils.phasemanip, to facilitate the implementation of (pretty basic) unit tests.
• General code cleanup (e.g., many utilities to manipulate fields and fieldmaps were not used at all).
• New sdcflows.workflows.apply.registration module: aligns the reference map of the fieldmap of choice (e.g., a magnitude image) to the reference EPI (e.g., an SBRef, a b=0 DWI, or a fMRIPrep's BOLDRef) with ANTs. The workflow resamples the fieldmap reference into the reference EPI's space for reporting/visualization objectives. Some scaffold has been left there to pick it up as start to continue with ENH: A 3D tensor B-Spline approximator and extrapolator #119. For now, attempting to use this path will result in a NotImplementedError.

Next steps

1. Once this is in, I'll continue to finalize ENH: A 3D tensor B-Spline approximator and extrapolator #119.
2. Instantiate actual workflows as a method of the FieldmapEstimation object.
3. Create an example integration with dMRIPrep
   • Showing how to generate EPI + DWI (or SBRef) PEPOLAR estimations
   • Showing how to generate SDC-SyN estimations
   • Exercising automated estimations (any kind of B₀ mapping or 2+ directions EPIs)
4. Create a PR for the integration with fMRIPrep
   • Showing how to generate EPI + BOLD (or SBRef) PEPOLAR estimations
   • Showing how to generate SDC-SyN estimations
   • Exercising automated estimations (any kind of B₀ mapping or 2+ directions EPIs)

---

* Although, pytest-cov does not seem to be registering coverage for interfaces run within workflows.

[codecov]

Codecov Report

Merging #123 (8c15f29) into master (bf8b995) will increase coverage by 14.06%.
The diff coverage is 62.96%.

@@             Coverage Diff             @@
##           master     #123       +/-   ##
===========================================
+ Coverage   57.23%   71.30%   +14.06%     
===========================================
  Files          14       14               
  Lines         968      784      -184     
  Branches      118       96       -22     
===========================================
+ Hits          554      559        +5     
+ Misses        394      200      -194     
- Partials       20       25        +5     

Flag	Coverage Δ
travis	58.29% <47.73%> (+8.70%)	⬆️
unittests	71.19% <62.96%> (+14.09%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
sdcflows/workflows/fit/fieldmap.py	66.66% <39.21%> (ø)
sdcflows/utils/phasemanip.py	59.42% <59.42%> (ø)
sdcflows/workflows/outputs.py	70.68% <67.30%> (+42.91%)	⬆️
sdcflows/interfaces/fmap.py	67.39% <67.85%> (+37.92%)	⬆️
sdcflows/workflows/apply/registration.py	82.60% <82.60%> (ø)
sdcflows/workflows/base.py	95.00% <94.44%> (+21.56%)	⬆️
sdcflows/workflows/fit/pepolar.py	63.51% <100.00%> (ø)
sdcflows/workflows/fit/syn.py	97.36% <100.00%> (ø)
... and 1 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bf8b995...8c15f29. Read the comment docs.

[pep8speaks]

Hello @oesteban, Thank you for updating!

Cheers! There are no style issues detected in this Pull Request. 🍻 To test for issues locally, pip install flake8 and then run flake8 sdcflows.

Comment last updated at 2020-11-22 20:23:20 UTC

[oesteban]

With @effigies off this week, @mgxd focused on Ni{Babies|Rodents} and having the dMRIPrep meeting tomorrow where I can catch up with @josephmje and @mattcieslak, I'm going to merge this into dev/1.4.0 and keep working on that branch.

I think that will shorten the span of time this overhaul is lingering and also allow me to make progress on other fronts.