You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
One of the scripts in the examples/ folder of Accelerate or an officially supported no_trainer script in the examples folder of the transformers repo (such as run_no_trainer_glue.py)
My own task or dataset (give details below)
Reproduction
H5py is working as opening a file handler first, and then loading specific data when indexing the database. Since the H5py file handler is not supported to be opened/read in multiple threads at the same time, using h5py.File() with or without only opening an h5 if accelerator.is_local_main_process will cause RuntimeError: Unable to synchronously get dataspace or RuntimeError: Unable to synchronously open ....
Expected behavior
Being compatible with H5py.
The text was updated successfully, but these errors were encountered:
System Info
Information
Tasks
no_trainer
script in theexamples
folder of thetransformers
repo (such asrun_no_trainer_glue.py
)Reproduction
H5py is working as opening a file handler first, and then loading specific data when indexing the database. Since the H5py file handler is not supported to be opened/read in multiple threads at the same time, using
h5py.File()
with or without only opening an h5 ifaccelerator.is_local_main_process
will causeRuntimeError: Unable to synchronously get dataspace
orRuntimeError: Unable to synchronously open ...
.Expected behavior
Being compatible with H5py.
The text was updated successfully, but these errors were encountered: