Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add probe locations, REAME updates, and small cleanups. #17

Merged
merged 2 commits into from
Dec 21, 2023
Merged

Conversation

nwatters01
Copy link
Collaborator

Changes:

  • Add probe location (including orientation) to metadata['description'] and position to metadata['position'] for all probes.
  • README updates.
  • Remove dandi automatic upload. Unfortunately we can't use automatic upload on openmind because data transfer must be done with separate jobs on designated nodes.
  • Minor cleanups.

@CodyCBakerPhD
Copy link
Member

Remove dandi automatic upload. Unfortunately we can't use automatic upload on openmind because data transfer must be done with separate jobs on designated nodes.

That's unfortunate - was there any luck increasing the transfer speeds? It occurred to me that even on a good internet connection there can be other bottlenecks such as reading from disk (I've seen some low-quality USB connections for external drives go as low as ~20 MB/s)

@nwatters01
Copy link
Collaborator Author

Fast uploading does work on openmind, but it must be done on designated data transfer nodes. The compute nodes used for conversion cannot be used for uploading. This is totally fine --- I don't have to copy data locally or anything, I just have to run a separate uploading job on a different node on the cluster.

@nwatters01 nwatters01 merged commit d1ab151 into main Dec 21, 2023
1 check passed
@nwatters01 nwatters01 deleted the watters branch December 21, 2023 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants