-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH: Standalone CLI now estimates fieldmaps #408
Conversation
d988b05
to
82977fc
Compare
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #408 +/- ##
==========================================
- Coverage 80.59% 76.66% -3.93%
==========================================
Files 26 32 +6
Lines 2293 2824 +531
Branches 291 373 +82
==========================================
+ Hits 1848 2165 +317
- Misses 401 593 +192
- Partials 44 66 +22
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
216d148
to
df93c7b
Compare
Well, at df93c7b, this is surprisingly executing the estimation of GRE fieldmaps. EPI fieldmaps failed in the robust average, so I'll need to check what went wrong. |
43587af
to
3d65832
Compare
3d65832
to
ba77d98
Compare
bca8834
to
28cb7b4
Compare
28cb7b4
to
07887e0
Compare
# workflow construction jailed within a process preempts excessive VM buildup. | ||
from multiprocessing import Manager, Process | ||
|
||
with Manager() as mgr: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it needed to separate this, if you've already initialized the process pool?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could send it to the pool, that'd be fine. However, it's working this way in MRIQC.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My point was the pool is initialized, so isn't it safe to do this in the main process at this point? These managers tend to make debugging trickier, so if we can avoid it, I'd rather.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, yes, in theory we could use the main process without problems, indeed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, I was removing this context to build the workflow in the main thread, and it could be interesting to keep it this way, as the workflow is created while OMP_NUM_THREADS is set to 1. This saves great numpy buildup.
We could still move it to the main process, but then it would make sense to make sure numpy is imported with OMP_NUM_THREADS = 1 (and that is then overridden in subprocesses).
Co-authored-by: Chris Markiewicz <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Go ahead and merge this when you're ready. It would be nice if we could get away from the Manager()
indirection for building workflows eventually, but I don't want to hold this up on that.
The previous behavior (just "finding" estimators) is maintained with the argument
--dry-run
.