Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

4DEnVar Function Test #8

Open
XuLu-NOAA opened this issue Jun 12, 2024 · 3 comments
Open

4DEnVar Function Test #8

XuLu-NOAA opened this issue Jun 12, 2024 · 3 comments
Assignees
@XuLu-NOAA

Comments

@XuLu-NOAA
Copy link
Collaborator

XuLu-NOAA commented Jun 12, 2024 •
edited
Loading

This is to record the test of 4DEnVar function in JEDI using HAFS.
(1) A 3-hourly 10-member GDAS ensemble is first generated through atm_init_fgat_ens onto HAFS grid.
(2) A single dropsonde T observation is generated.
(3) Modify the following parts of the yaml file from 3DEnVar single ob test for 4DEnVar:

  1. cost type: 3D-Var -> cost type: 4D-Ens-Var
  2. Add subwindow:
    subwindow: PT3H
    parallel subwindows: false
  3. Provides all background times and files:
    states:
    - datetime: 2020-08-25T09:00:00Z
    filetype: fms restart
    datapath: Data/bkg
    filename_core: '20200825.090000.fv_core.res.tile1.nc'
    filename_trcr: '20200825.090000.fv_tracer.res.tile1.nc'
    filename_sfcd: '20200825.090000.sfc_data.nc'
    filename_sfcw: '20200825.090000.fv_srf_wnd.res.tile1.nc'
    filename_cplr: '20200825.090000.coupler.res'
    state variables: *3dvars
    - datetime: 2020-08-25T12:00:00Z
    filetype: fms restart
    datapath: Data/bkg
    filename_core: '20200825.120000.fv_core.res.tile1.nc'
    filename_trcr: '20200825.120000.fv_tracer.res.tile1.nc'
    filename_sfcd: '20200825.120000.sfc_data.nc'
    filename_sfcw: '20200825.120000.fv_srf_wnd.res.tile1.nc'
    filename_cplr: '20200825.120000.coupler.res'
    state variables: *3dvars
    - datetime: 2020-08-25T15:00:00Z
    filetype: fms restart
    datapath: Data/bkg
    filename_core: '20200825.150000.fv_core.res.tile1.nc'
    filename_trcr: '20200825.150000.fv_tracer.res.tile1.nc'
    filename_sfcd: '20200825.150000.sfc_data.nc'
    filename_sfcw: '20200825.150000.fv_srf_wnd.res.tile1.nc'
    filename_cplr: '20200825.150000.coupler.res'
    state variables: *3dvars
  4. Provide covariance ensemble times:
    background error:
    covariance model: ensemble
    members from template:
    template:
    states:
    - datetime: 2020-08-25T09:00:00Z
    filetype: fms restart
    state variables: *3dvars
    datapath: ens/mem%mem%/
    filename_core: '20200825.090000.fv_core.res.tile1.nc'
    filename_trcr: '20200825.090000.fv_tracer.res.tile1.nc'
    filename_sfcd: '20200825.090000.sfc_data.nc'
    filename_sfcw: '20200825.090000.fv_srf_wnd.res.tile1.nc'
    filename_cplr: '20200825.090000.coupler.res'
    - datetime: 2020-08-25T12:00:00Z
    filetype: fms restart
    state variables: *3dvars
    datapath: ens/mem%mem%/
    filename_core: '20200825.120000.fv_core.res.tile1.nc'
    filename_trcr: '20200825.120000.fv_tracer.res.tile1.nc'
    filename_sfcd: '20200825.120000.sfc_data.nc'
    filename_sfcw: '20200825.120000.fv_srf_wnd.res.tile1.nc'
    filename_cplr: '20200825.120000.coupler.res'
    - datetime: 2020-08-25T15:00:00Z
    filetype: fms restart
    state variables: *3dvars
    datapath: ens/mem%mem%/
    filename_core: '20200825.150000.fv_core.res.tile1.nc'
    filename_trcr: '20200825.150000.fv_tracer.res.tile1.nc'
    filename_sfcd: '20200825.150000.sfc_data.nc'
    filename_sfcw: '20200825.150000.fv_srf_wnd.res.tile1.nc'
    filename_cplr: '20200825.150000.coupler.res'
    pattern: '%mem%'
    nmembers: 10
    zero padding: 3
  5. Add time interpolation in observations/observers:
    get values:
    time interpolation: linear
  6. Modify the output section:
    output:
    filetype: fms restart
    datapath: Data/analysis/
    exp: 4denvar
    type: an
    first: PT0S
    date: 2020-08-25T09:00:00Z
    frequency: PT3H

The innovations show the 4DEnVar works fine. It is producing consistent first guess with the FGAT when positioning the single obs to different times. The analysis is also significantly different due to the covariance used.
image
image
@XuLu-NOAA XuLu-NOAA self-assigned this Jun 12, 2024
BinLiu-NOAA pushed a commit that referenced this issue Jun 18, 2024
@guillaumevernieres
Optionally run specified rocoto task as part of the ctests (#1121)
af7fb73 
Only tested with the coupled UFS and the marine DA tasks:
```
Test project /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/runs/prs/global-workflow/sorc/gdas.cd/build/gdas/test/gw-ci
  Test  #1: C48mx500_3DVarAOWCDA
  Test  #2: C96_atmaerosnowDA
  Test  #3: C96C48_ufs_hybatmDA
  Test  #4: C48mx500_3DVarAOWCDA_gdasfcst_202103241200
  Test  #5: C48mx500_3DVarAOWCDA_gdasprepoceanobs_202103241800
  Test  #6: C48mx500_3DVarAOWCDA_gdasocnanalprep_202103241800
  Test  #7: C48mx500_3DVarAOWCDA_gdasocnanalbmat_202103241800
  Test  #8: C48mx500_3DVarAOWCDA_gdasocnanalrun_202103241800
  Test  #9: C48mx500_3DVarAOWCDA_gdasocnanalchkpt_202103241800
  Test #10: C48mx500_3DVarAOWCDA_gdasocnanalpost_202103241800

Total Tests: 10
```
BinLiu-NOAA pushed a commit that referenced this issue Jun 18, 2024
@guillaumevernieres @CoryMartin-NOAA
Addin a GFSv17 ctest (#1130)
4598d91 
**What was done:**
- reorganization/code tidy to facilitate the addition of a GFSv17
prototype ctest
- exclude the new ctest by default, the user will have to run ```cmake
-DRUNGWCI=ON .``` on an old build to configure the new ctest

He're the current list:

```
 Test project /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/runs/prs/global-workflow/sorc/gdas.cd/build/gdas/test/gw-ci
  Test  #1: WCDA-3DVAR-C48mx500
  Test  #2: WCDA-3DVAR-C48mx500_gdasfcst_202103241200
  Test  #3: WCDA-3DVAR-C48mx500_gdasprepoceanobs_202103241800
  Test  #4: WCDA-3DVAR-C48mx500_gdasocnanalprep_202103241800
  Test  #5: WCDA-3DVAR-C48mx500_gdasocnanalbmat_202103241800
  Test  #6: WCDA-3DVAR-C48mx500_gdasocnanalrun_202103241800
  Test  #7: WCDA-3DVAR-C48mx500_gdasocnanalchkpt_202103241800
  Test  #8: WCDA-3DVAR-C48mx500_gdasocnanalpost_202103241800
  Test  #9: Aero-Snow-3DVAR-C96
  Test #10: Aero-Snow-3DVAR-C96_gdasfcst_202112201200
  Test #11: Atm-hyb-C96C48
  Test #12: Atm-hyb-C96C48_gdasfcst_202402231800
  Test #13: GFSv17-3DVAR-C384mx025
  Test #14: GFSv17-3DVAR-C384mx025_gdasfcst_202106300000
  Test #15: GFSv17-3DVAR-C384mx025_gdasprepoceanobs_202106300600
  Test #16: GFSv17-3DVAR-C384mx025_gdasocnanalprep_202106300600
  Test #17: GFSv17-3DVAR-C384mx025_gdasocnanalbmat_202106300600
  Test #18: GFSv17-3DVAR-C384mx025_gdasocnanalrun_202106300600
  Test #19: GFSv17-3DVAR-C384mx025_gdasocnanalchkpt_202106300600
  Test #20: GFSv17-3DVAR-C384mx025_gdasocnanalpost_202106300600
  Test #21: GFSv17-3DVAR-C384mx025_gdasocnanalvrfy_202106300600
  Test #22: GFSv17-3DVAR-C384mx025_gdasprep_202106300600
  Test #23: GFSv17-3DVAR-C384mx025_gdasanal_202106300600

Total Tests: 23
```

---------

Co-authored-by: Cory Martin <[email protected]>
@XuLu-NOAA
Copy link
Collaborator Author

XuLu-NOAA commented Jun 28, 2024 •
edited
Loading

Update the ensemble and background using the corrected halo file according to Issue #11.
image
The 4DEnVar result looks pretty reasonable. The increments and oma are consistent with the corresponding 3DEnVar when positioning the observations right at the background times:
image

@XuLu-NOAA
Copy link
Collaborator Author

But one major concern is the memory issue. Currently, 4DEnVar only works with 10 member and 10 iterations with only 1 inner loop. Otherwise, insufficient memory will crash the JEDI run.

@XuLu-NOAA XuLu-NOAA mentioned this issue Jul 2, 2024
Open
@XuLu-NOAA
Copy link
Collaborator Author

Additional test to confirm that the FGAT capability works fine in 4DEnVar. In this 4D_FGAT experiment, the ensembles at -3h and +3h are the same as the ones at hour 0. Therefore, all the changes from the observations time change is from FGAT part of the 4DEnVar exp.
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@XuLu-NOAA XuLu-NOAA

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@XuLu-NOAA