Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

restructure DagmanCreator and DagmanSubmitter #8892

Closed
4 tasks done
belforte opened this issue Jan 23, 2025 · 2 comments
Closed
4 tasks done

restructure DagmanCreator and DagmanSubmitter #8892

belforte opened this issue Jan 23, 2025 · 2 comments

Comments

@belforte
Copy link
Member

belforte commented Jan 23, 2025

after the changes for input_args.json in #8883 there is a a large amount of confusion and now-obsolete code in DagmanCreator

Take this chance to restructure and cleanup.
Main goals

  • stop using info dictionary everywhere to mean different things
  • instead of JOB_SUBMIT string and format where one needs to look in 3 or fours places to find out what goes in 'Job.submit' file, use an htcondor.Submit() object and classad.quote() and avoid as much as possible ad-hoc, unexplained, string manipulations
  • simplify and clarify what information is really needed to submit the DAG bootstrap job and rework the informaiton transfer from DagmanCreator to DagmanSubmitter. Ideally pass an htcondor.Submit() object with the classAds common to boths submissions, which DagmanSubmitter will extend as needed
  • address small "historical" differences and oddities in other parts of the code which rely/require specific ads with non-standard syntax/values
@belforte
Copy link
Member Author

belforte commented Jan 23, 2025

evolving code in my branch https://github.com/belforte/CRABServer/tree/CleanupAfterSwitchTo-input_args

so far I have a new DagmanSubmitter which reproduces current Job.submit file in a simple test task.
Next step is to work on communication with DagmanSubmitter

The try all tasks

@belforte
Copy link
Member Author

That branch is now able to submit a simple task.
Still need to worry about DagmanResubmitter.

first taks which succeeded: https://cmsweb-test2.cern.ch/crabserver/ui/task/250127_195607%3Abelforte_crab_20250127_205602

belforte added a commit to belforte/CRABServer that referenced this issue Jan 27, 2025
belforte added a commit to belforte/CRABServer that referenced this issue Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant