Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add REANA workflow file #23

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

tpmccauley
Copy link

Not much more to say. This workflow has been tested using reana-client at reana.cern.ch.

reana.yaml Outdated
type: serial
specification:
steps:
- name: test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirm that the workflow works well 👍

AOD2NanoAODOutreachTool

and produces ROOT file:

$ reana-client ls -w AOD2NanoAODOutreachTool -h --filter name=output.root
NAME                                                 SIZE      LAST-MODIFIED
CMSSW_5_3_32/src/workspace/AOD2NanoAOD/output.root   8.7 GiB   2023-02-19T19:50:13

Two comments:

  1. The step of the workflow is called "test". Perhaps a better name could be "data-production" or "test-data-production" or something?

  2. The workflow produces a lot of files:

$ reana-client ls -w AOD2NanoAODOutreachTool | wc -l
1049

Most are from scramming etc located here:

$ reana-client ls -w AOD2NanoAODOutreachTool | grep CMSSW_5_3_32 | wc -l
923

It should not be necessary to keep them for long, so it may be useful to introduce a new workspace file retention configuration in reana.yaml, something like:

workspace:
  retention_days:
    CMSSW_5_3_32: 7

so that the CMSSW_5_3_32 directory would be automatically deleted after 7 days.

P.S. Just an example to be adapted based on what you would wish to preserve for shorter/longer, e.g. remove only CMSSW_5_3_32/.SCRAM, etc.

* change workspace name

* add retention rule

* closes cms-opendata-analyses#23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants