Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CFA workflow for HPC -> JASMIN disk -> JASMIN tape -> JASMIN disk #128

Open
bnlawrence opened this issue Oct 19, 2023 · 2 comments
Open
Assignees
Milestone

Comments

@bnlawrence
Copy link
Collaborator

We need to document the expected workflow for data as

  • a cycle arrives from HPC onto JASMIN transfer disk
  • a CFA file is written for that data
  • a new CFA file is written which includes existing cycles (probably on tape) and the new data
  • user facing CFA files are updated per frequency
  • the data is migrated to tape
  • some data is brought back from a tape
  • a CFA file is updated for the presence of a new Quark (which CFA file)

A sketch of this is here:

IMG_5971

The initial use case is CANARI, but we want this to be a generic workflow.

@bnlawrence bnlawrence added this to the CANARI-MVP milestone Oct 19, 2023
@bnlawrence bnlawrence self-assigned this Oct 19, 2023
@bnlawrence
Copy link
Collaborator Author

Here are a couple of flowcharts expressing some of the stuff on that white board
workflow
workflow-1

@bnlawrence
Copy link
Collaborator Author

Already seen an error. Fragments are not moved from tape to disk, they're copied. Hence the stuff on disk in GWS is always a replicant of stuff on tape. Will update these and stick them in the repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant