Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Importing worker takes a lot of time ... #2110

Open
kif opened this issue Mar 21, 2024 · 3 comments
Open

Importing worker takes a lot of time ... #2110

kif opened this issue Mar 21, 2024 · 3 comments
Assignees
Labels
performance profiling issues

Comments

@kif
Copy link
Member

kif commented Mar 21, 2024

Condition: ewoks worker running on slurm cluster; reported by Edgar.
Several seconds are reported ...

@kif kif added the performance profiling issues label Mar 21, 2024
@kif
Copy link
Member Author

kif commented Mar 21, 2024

python3 -X importtime -c "import pyFAI.worker" gives a hint ... the timings are in µs.

@kif
Copy link
Member Author

kif commented Mar 21, 2024

On lintaillefer, this takes: 2.1s out of which:

 193.521 ms	        fabio
 423.257 ms	      pyFAI.utils
 435.484 ms	    pyFAI
1466.502 ms	              silx.opencl.common
1466.716 ms	            silx.opencl
1466.735 ms	          silx.opencl.common
1467.515 ms	        pyFAI.opencl
1570.422 ms	      pyFAI.load_integrators
1574.027 ms	    pyFAI.azimuthalIntegrator
2109.442 ms	  pyFAI.worker

Apparently, it is the initialization of OpenCL (which probes all devices) which takes most of the time.

@kif
Copy link
Member Author

kif commented Mar 21, 2024

The selection of the OpenCL device can be tuned with OCL_ICD_VENDORS. In this example I point to an ICD for a FPGA board that I don't have:

  54.466 ms	      pyFAI.io.image
  59.140 ms	                numpy.core
 101.420 ms	              numpy
 102.072 ms	    pyFAI.average
 123.890 ms	            fabio.fabioimage
 124.134 ms	          fabio.fabioformats
 138.252 ms	          pkg_resources
 138.401 ms	        pyFAI.resources
 186.043 ms	        fabio
 401.359 ms	      pyFAI.utils
 414.103 ms	    pyFAI
 558.349 ms	  pyFAI.worker

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance profiling issues
Projects
None yet
Development

No branches or pull requests

2 participants