forked from NOAA-EMC/WW3
-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Added boundary checks to the SMC grid input files for ww3_grid, to ensure they comply with the limits of the nameslist. * Fb 360 calendar (#8) Changes to add support to 360-day and 365-day (no leap year) calendar - see ticket NOAA-EMC#209 * Additional CALTYPE namelist parameter in MISC section * New ww3_tc1 regtest. * RTD support for ww3_boun[dc] (#10) * Updated ww3_bound and ww3_bounc to handle model grids formulated on a rotated pole. * Manual and nml/inp files to updated clarify that ww3_bound/ww3_bounc only accept input spectra formulated on a standard pole grid. * Fb coupling time (#9) Updates to allow a coupling time step that is different from the model time step. * Includes new regtest (in ww3_tp2.14) for non-default oasis time step. * ww3_tp2.14 regtest added to matrix.base. * bug fix for ukmet development * add the matrix subsetter * clean-up * clean up * add another script which separate serial and parallel jobs and divide them * modify the script to remove ../model? after test completion. * bug fixes and adding ww3_tp2.17 to list_heavy * add if statement to remove matrix? and model? * Update matrix_divider_p.sh * Merge remote-tracking branch 'upstream/develop' into fb_matrix_divider * Merge remote-tracking branch 'upstream/develop' into fb_matrix_divider * put if check for ../model? inside matrix? loop * fix the bug for sed for model? * final fix for extra model? removal * add 2.21 to the list_heavy Co-authored-by: lewis.sampson <[email protected]> Co-authored-by: Juan Manuel Castillo Sanchez <[email protected]> Co-authored-by: Chris Bunney <[email protected]> Co-authored-by: Ali Abdolali <[email protected]>
- Loading branch information
5 people
authored
Mar 24, 2021
1 parent
854fc86
commit 78b0148
Showing
2 changed files
with
132 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| #!/bin/bash | ||
| # --------------------------------------------------------------------------- # | ||
| # matrix_divider simply divides the main matrix into subsets with a given # | ||
| # number of test (i.e. maxlist1 for mpi tests and maxlist2 for serial tests) # | ||
| # # | ||
| # Remarks: # | ||
| # - Once the matrix is generated, this script can be execute. The user should # | ||
| # define the maxlist and this script divide the matrix into matrix? and # | ||
| # submit the job using sbatch. # | ||
| # # | ||
| # Ali Abdolali # | ||
| # August 2018 # | ||
| # March 2021 # | ||
| # # | ||
| # Copyright 2013 National Weather Service (NWS), # | ||
| # National Oceanic and Atmospheric Administration. All rights # | ||
| # reserved. WAVEWATCH III is a trademark of the NWS. # | ||
| # No unauthorized use without permission. # | ||
| # # | ||
| # --------------------------------------------------------------------------- # | ||
| # --------------------------------------------------------------------------- # | ||
| # 1. clean up and definitions # | ||
| # --------------------------------------------------------------------------- # | ||
| cp matrix matrix.tmp | ||
| HOME=${PWD%/*} | ||
|
|
||
| maxlist1=48 | ||
| maxlist2=92 | ||
|
|
||
| #Put the job requirement/spec in "before" | ||
| sed -e "/run_test/,\$d" matrix.tmp > before | ||
| #Put the list of tests in "list" | ||
| command egrep 'ww3_tp2.14|ww3_tp2.17|ww3_tp2.21' matrix.tmp | cat >> list_heavy | ||
| awk '!/ww3_tp2.14/' matrix.tmp > tmpfile && mv tmpfile matrix.tmp | ||
| awk '!/ww3_tp2.17/' matrix.tmp > tmpfile && mv tmpfile matrix.tmp | ||
| awk '!/ww3_tp2.21/' matrix.tmp > tmpfile && mv tmpfile matrix.tmp | ||
| command egrep 'mpirun|mpiexec|MPI_LAUNCH' matrix.tmp | cat >> list_mpi | ||
| awk '!/mpirun|mpiexec|MPI_LAUNCH/' matrix.tmp > tmpfile && mv tmpfile matrix.tmp | ||
| split -dl $maxlist1 list_mpi list_mpi_ | ||
| rm list_mpi | ||
| matrixno1=$(ls list_mpi_* | wc -l) | ||
| echo "Total nummber of matrix with parallel tests = $(($matrixno1 + 1)); each includes $maxlist1 tests" | ||
| command egrep 'run_test' matrix.tmp | cat >> list_serial | ||
| split -dl $maxlist2 list_serial list_serial_ | ||
| rm list_serial | ||
| matrixno2=$(ls list_serial_* | wc -l) | ||
| echo "Total nummber of matrix with serial test = $matrixno2; each includes $maxlist2 tests" | ||
| rm matrix.tmp | ||
|
|
||
| # -------------------------------|ww3_tp2.21|ww3_tp2.21-------------------------------------------- # | ||
| # 2. Divide and dump in subsets # | ||
| # --------------------------------------------------------------------------- # | ||
| # parallel jobs | ||
| count=0 | ||
| for i in `seq -f '%02g' 0 "$((matrixno1 - 1))"`; do | ||
| #echo $i | ||
| #Replace matrix.out > matrix?.out, model > model? | ||
| (( count = count + 1 )) | ||
| if [ -f "matrix${count}" ]; then rm -f matrix${count}; fi | ||
| cat before >> matrix$count | ||
| cat list_mpi_$i >> matrix$count | ||
| sed -i 's/'matrix.out'/'matrix${count}.out'/gI' matrix$count | ||
| sed -i 's/'model'/'model${count}'/gI' matrix$count | ||
| echo " echo ' '" >> matrix$count | ||
| echo " echo ' **************************************************************'" >> matrix$count | ||
| echo " echo ' * end of WAVEWATCH III matrix$count of regression tests *'" >> matrix$count | ||
| echo " echo ' **************************************************************'" >> matrix$count | ||
| echo " echo ' '" >> matrix$count | ||
| echo "rm -rf ${HOME}/model${count}" >> matrix$count | ||
| #make sure ../model$count does not exist and copy a fresh copy | ||
| awk '1;/cd/ && !x {print " cp -r ../model ../model'$count'"; x=1;}' matrix$count > tmpfile && mv tmpfile matrix$count | ||
| awk '1;/cd/ && !x {print " if [ -d ../model'${count}' ]; then rm -rf ../model'${count}'; fi"; x=1;}' matrix$count > tmpfile && mv tmpfile matrix$count | ||
|
|
||
| echo " matrix$count prepared" | ||
| done | ||
|
|
||
| #serial jobs | ||
| for i in `seq -f '%02g' 0 "$((matrixno2 - 1))"`; do | ||
| #echo $i | ||
| #Replace matrix.out > matrix?.out, model > model? | ||
| (( count = count + 1 )) | ||
| if [ -f "matrix${count}" ]; then rm -f matrix${count}; fi | ||
| cat before >> matrix$count | ||
| cat list_serial_$i >> matrix$count | ||
| sed -i 's/'matrix.out'/'matrix${count}.out'/gI' matrix$count | ||
| sed -i 's/'model'/'model${count}'/gI' matrix$count | ||
| echo " echo ' '" >> matrix$count | ||
| echo " echo ' **************************************************************'" >> matrix$count | ||
| echo " echo ' * end of WAVEWATCH III matrix$count of regression tests *'" >> matrix$count | ||
| echo " echo ' **************************************************************'" >> matrix$count | ||
| echo " echo ' '" >> matrix$count | ||
| echo "rm -rf ${HOME}/model${count}" >> matrix$count | ||
| #make sure ../model$count does not exist and copy a fresh copy | ||
| awk '1;/cd/ && !x {print " cp -r ../model ../model'$count'"; x=1;}' matrix$count > tmpfile && mv tmpfile matrix$count | ||
| awk '1;/cd/ && !x {print " if [ -d ../model'${count}' ]; then rm -rf ../model'${count}'; fi"; x=1;}' matrix$count > tmpfile && mv tmpfile matrix$count | ||
| echo " matrix$count prepared" | ||
| done | ||
|
|
||
| #ww3_tp2.14 is separated, as it has dependency. | ||
| #ww3_tp2.17 and ww3_tp2.21 is separated, as it takes a long time to finish | ||
| (( count = count + 1 )) | ||
| if [ -f "matrix${count}" ]; then rm -f matrix${count}; fi | ||
| cat before >> matrix$count | ||
| cat list_heavy >> matrix$count | ||
| sed -i 's/'matrix.out'/'matrix${count}.out'/gI' matrix$count | ||
| sed -i 's/'model'/'model${count}'/gI' matrix$count | ||
| echo " echo ' '" >> matrix$count | ||
| echo " echo ' **************************************************************'" >> matrix$count | ||
| echo " echo ' * end of WAVEWATCH III matrix$count of regression tests *'" >> matrix$count | ||
| echo " echo ' **************************************************************'" >> matrix$count | ||
| echo " echo ' '" >> matrix$count | ||
| echo "rm -rf ${HOME}/model${count}" >> matrix$count | ||
| #make sure ../model$count does not exist and copy a fresh copy | ||
| awk '1;/cd/ && !x {print " cp -r ../model ../model'$count'"; x=1;}' matrix$count > tmpfile && mv tmpfile matrix$count | ||
| awk '1;/cd/ && !x {print " if [ -d ../model'${count}' ]; then rm -rf ../model'${count}'; fi"; x=1;}' matrix$count > tmpfile && mv tmpfile matrix$count | ||
| echo " matrix$count prepared" | ||
|
|
||
|
|
||
|
|
||
| rm before | ||
| rm list* | ||
|
|
||
| echo "file matrix is divided into $count subsets ...." | ||
|
|
||
| # --------------------------------------------------------------------------- # | ||
| # End to matrix_divider # | ||
| # --------------------------------------------------------------------------- # | ||
|
|