Functions for collecting fMRIPrep and AfNI outputs

[tsalo]

In order to run tedana on partially preprocessed data (e.g., slice timing correction and motion correction) prior to any warping (e.g., distortion correction and normalization), we need to collect data partway through preprocessing, as well as transforms to finish preprocessing after we run tedana. At least with fMRIPrep, we'll need to run the full pipeline and then grab the relevant files from the working directory and save them to the output derivatives directory. I'm not sure how it should work with afni_proc.py though.

@emdupre started writing a script to do this for fMRIPrep here. We still need to finish, test, and convert it to a function.

@jbteves, do you have any thoughts on what we'll need to do for afni_proc outputs?

[tsalo]

I've added a notebook that collects relevant fMRIPrep files from the working directory and places them in the fMRIPrep derivatives folder. I'm not 100% sure that they're right, but @emdupre helped with this and they look good to me. I think all we need to do for that to be relatively complete is to write a function to split the 4D func files into 3D files, apply the native-to-reference, reference-to-T1w, and T1w-to-MNI transforms, and then merge the transformed 3D files back into 4D.

[jbteves]

Sorry for the long delay. For afni_proc, we can just write a short proc that does what we want and copy the last resulting image out of the folder. It should look something like this:

afni_proc.py -subj_id "$name"_oc \
-blocks tshift volreg \
-dsets_me_run $run1 \
-dsets_me_run $run2 \
-tshift_opts_ts -tpattern @$slicetiming \
-echo_times a b c \
-reg_echo 1 \
-volreg_base_dset $run1 \
-bash \
-scr_overwrite \
-execute

Which should then produce the BRIK/HEAD pairs we need. From there we can either (a) adapt tedana to read brik/head pairs if it doesn't already or (b) run 3dAfNItoNifti
So it's relatively straightforward for this use case. It gets more complicated trying to interface with tedana and then continue.

[tsalo]

We'll need to do coregistration and normalization too though (after tedana). Will you be able to run afni_proc again with just those steps?

[jbteves]

We're doing that on one of our projects now, except we're only running optimal combination of the data. You basically just need to copy things over to a new folder and then run a new afni_proc. Merging the two folders is tricky and therefore it gets a bit cumbersome during analysis, though, which is why it would be good to have a wrapper. It's also possible that I'm missing something since I'm fairly new to AfNI, but I think that's the best solution I have.

[tsalo]

That sounds good to me. Can you write those two afni_proc calls into separate functions with another one to merge them? It would be nice to only have to run afni_proc part 2 once and then to apply the transforms to the outputs from the many tedana runs we plan to have, but I don't think that's necessary if it's not possible.

[jbteves]

Merging them is a bit context-dependent. Frankly you probably only need the results of the second folder. But yes, I can work on the two scripts. I'll try to get test them and get them in by next Friday, if that's reasonable. If I don't have something on a PR by then please ping me to remind me-- I'm doing work + a data structures/algorithms course so it's easy for me to forget.

[jbteves]

Where should the files go in the codebase? Should I make a new directory?

[tsalo]

We should probably reorganize the repository at some point, but for the moment I think the top level is a fine place. You can name the files collect_afni_1.XX and collect_afni_2.XX (or something similar).

[jbteves]

Sorry for the repeated delays. afni_proc must be written in bash, but do you want the "harvesting" files to be in python for consistency? It looks like there's a notebook as well as a standard .py; is there a preference for either?

[tsalo]

Python would be nice. The notebook contains the currently working code, while the py file is outdated, but I plan to convert the notebook over to proper functions in a py file at some point.