Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new dataset #14

Open
anas2908 opened this issue Jan 9, 2024 · 3 comments
Open

new dataset #14

anas2908 opened this issue Jan 9, 2024 · 3 comments

Comments

@anas2908
Copy link

anas2908 commented Jan 9, 2024

running a new dataset for your model
getting error "h5py objects cannot be pickled"
All requirements are satisfied and are taken special care

@boheumd
Copy link
Owner

boheumd commented Jan 12, 2024

Hi, it seems that you need to set the num_workers=0 in this case.
https://stackoverflow.com/questions/60684061/typeerror-h5py-objects-cannot-be-pickled

@anas2908
Copy link
Author

The Training time becomes significantly slow it will take weeks, is there any other way to just directly dump .npy features in your model?

@boheumd
Copy link
Owner

boheumd commented Jan 12, 2024

Hi, did you store the features of the custom dataset in the h5py format? If it is in this case, maybe you can change the h5py file into numpy format, and then you can directly read from the numpy file when loading the dataset features.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants