Main branch updates for Rocoto and ecflow workflows

[BenjaminBlake-NOAA]

DESCRIPTION OF CHANGES:

• The Rocoto and ecflow workflows have been successfully tested on WCOSS2 with these updates to the main branch. These updates are critical in order to use the main branch of rrfs-workflow. Once the real-time RRFS runs are turned back on, we should use the main branch of the workflow, which is closer to what we will deliver to NCO than what's in the dev-sci branch. A description of some of the changes is included here:
  • WGF (working group function) is added to the workflow. The four options are "det", "enkf", "ensf", and "firewx." There have been some discussions related to RRFSv2 about renaming ensf to enfcst or ensfcst, which can certainly be done at a later date (but not in this PR).
  • The jobid definitions are updated in the J-jobs. This was necessary for the ecflow workflow, which does not have taskid defined. Note that jobid will eventually be set to $job.$PBS_JOBID, but that update will be made alongside the adoption of the umbrella data structure for all tasks. So the current jobid/taskid definitions in the J-jobs are not the final definitions.
  • The mem_num definitions are updated in the J-jobs in order for the ecflow workflow to work properly.
  • RUNE is correctly defined as enkfrrfs in FV3LAM_wflow.xml.
  • Run directory names/locations were updated/fixed in FV3LAM_wflow.xml and several ex-scripts.
  • ENSCTRL_GESROOT is now correctly set to the deterministic forecast path.
• Comparison graphics for one Rocoto vs ecflow cycle are located here on rzdm: https://www.emc.ncep.noaa.gov/users/emc.campara/rrfs/2024120300/

TESTS CONDUCTED:

I tested the Rocoto workflow for the deterministic, ensemble DA, ensemble forecast, and fire weather components on WCOSS2. @lgannoaa tested the ecflow workflow for the same components using the same code base.

Machines/Platforms:

• WCOSS2
  • Cactus/Dogwood
  • Acorn
• RDHPCS
  • Hera
  • Jet
  • Orion
  • Hercules

Test cases:

• Engineering tests
  • Non-DA engineering test
  • DA engineering test
    • Retro
    • Ensemble
    • Parallel
• RRFS fire weather
• RRFS_A:
• RRFS_B:
• RTMA:
• Others:

ISSUE:

• Issues Add ecflow workflow support for WCOSS2 from production/RRFS.v1 branch #352 and Improve the ecflow workflow #428

CONTRIBUTORS (optional):

@lgannoaa was incredibly helpful and contributed greatly to this work.

[MatthewPyle-NOAA]

Lots of comments, but many of them are repeated a few times. Looks like a big step forward for RRFS, but I did have a questions and some requests for cleanup in a few small ways.

[lgannoaa]

Thank you Ben for great team work. runcyc_path is ok for me. Lin

…

On Tue, Dec 17, 2024 at 8:13 AM BenjaminBlake-NOAA ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ On scripts/exrrfs_prep_cyc.sh <#582 (comment)> : This is a change from Lin which fixes some incorrect directory paths, so I'm not actually sure what the r in r_path stands for. I agree that runcyc_path is more descriptive and I'll make this change wherever r_path is introduced. — Reply to this email directly, view it on GitHub <#582 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI62YCOPNAO5WNKYLRKDTAD2GAPPRAVCNFSM6AAAAABTQLMZQWVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDKMBYHA2TAMRZGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[lgannoaa]

Hi Ben, I recommend to keep it for now. You are right, that is for the umbrella data work. Lin

…

On Tue, Dec 17, 2024 at 8:23 AM BenjaminBlake-NOAA ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ On scripts/exrrfs_fsm.sh <#582 (comment)> : I think Lin wants to add these commented out lines to exrrfs_fsm.sh because they will be used in the future when we start using the umbrella data structure. But I can remove them unless Lin feels strongly about keeping them there now. @lgannoaa <https://github.com/lgannoaa> what do you think? — Reply to this email directly, view it on GitHub <#582 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI62YCLJXPDYKGLWBMKGZH32GAQWRAVCNFSM6AAAAABTQLMZQWVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDKMBYHA3TIMBQGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[MatthewPyle-NOAA]

Thanks for answering my questions and fixing a few minor things...

[MatthewPyle-NOAA]

Thanks for all of the work on this one...merging.