np_codeocean

Tools for uploading and interacting with Mindscope Neuropixels experiments on Code Ocean

Requires running as admin on Windows in order to create remote-to-remote symlinks on the Isilon.

• upload CLI tool is provided, which uses the np_session interface to find and upload raw data for one ecephys session:

  pip install np_codeocean
  upload <session-id>
  

  where session-id is a valid input to np_session.Session():

  • a lims ID (1333741475)
  • a workgroups foldername (DRPilot_366122_20230101)
  • a path to a session folder ( \\allen\programs\mindscope\workgroups\np-exp\1333741475_719667_20240227)

• a folder of symlinks pointing to the raw data is created, with a new structure suitable for the KS2.5 sorting pipeline on Code Ocean

• the symlink folder, plus metadata, are entered into a csv file, which is submitted to http://aind-data-transfer-service, which in turn runs the aind-data-transfer tool on the HPC, which follows the symlinks to the original data, median-subtracts/scales/compresses ephys data, then uploads with the AWS CLI tool

• all compression/zipping acts on copies in temporary folders: the original raw data is not altered in anyway