hindcasts

Say you want to run an ensemble of 30 members for part of a CESM hindcast. Building each of these models from scratch when you’re using CAM5 can take enormous amounts of time – anywhere from 5 to 15 minutes per experiment. To get around this, I’ve built a scaffold around a typical CESM build script that allows you to build the model just once and use those files in creating numerous ensembles. The ensemble script is included below; prior to this, build a case as you normally would (you’ll point the ensemble script to that case for build files later).

The most important points are to set up the initialization dates in arrays at the beginning (can be done more efficiently than I’ve done here) and also set an array of temperature perturbation values to be used by PERTLIM. Then, within a while loop, you can change the start date and temperature perturbation for each ensemble, and point towards a previously built version of the model rather than having to re-build 30 times.

#!/bin/csh

#Set initialization dates

set days={10,10,10,10,10,10,10,10,10,10,15,15,15,15,15,15,15,15,15,15,20,20,20,20,20,20,20,20,20,20}

set months={05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05,05}

set years={2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009,2009}

#Set temperature perturbation values

set valPert={0.00000000000001,0.00000000000002,0.00000000000003,0.00000000000004,0.00000000000005,0.00000000000006,0.00000000000007,0.00000000000008,0.00000000000009,0.00000000000010,0.00000000000011,0.00000000000012,0.00000000000013,0.00000000000014,0.00000000000015,0.00000000000016,0.00000000000017,0.00000000000018,0.00000000000019,0.00000000000020,0.00000000000021,0.00000000000022,0.00000000000023,0.00000000000024,0.00000000000025,0.00000000000026,0.00000000000027,0.00000000000028,0.00000000000029,0.00000000000030}

#Initialize counter/ensemble number “num”

@ num = 1

set run_time = 00:35:00

set queue = normal

set Np = 640

#Build 30 ensemble members within the while loop

while ($num < 31) #Set case name and date per ensemble

set run_start_date=”$years[$num]-$months[$num]-$days[$num]”

set name=”F_2000_CAM5_0.9×1.25_TestCase_$num”

#Set usual variables to define the case (CCSMTAG, CASE, CASESET, CASERES) and relevant directories (MACH,CCSMROOT,CASEROOT,PTMP,RUNDIR,etc.) [not shown here]

================================================== # create new case, configure, compile and run =================================================

rm -rf $CASEROOT

rm -rf $PTMP/cases/$CCSMTAG/$CASE

rm -rf $PTMP/$CASE

#——————

##create new case

#——————

cd $CCSMROOT/scripts ./create_newcase –case $CASEROOT -mach $MACH -res $CASERES -compset $CASESET -compiler intel -v

#——————

##set environment

#——————

cd $CASEROOT

set ntasks = $Np

./xmlchange -file env_mach_pes.xml -id NTASKS_ATM -val=$ntasks

./xmlchange -file env_mach_pes.xml -id NTASKS_LND -val=48

./xmlchange -file env_mach_pes.xml -id NTASKS_ICE -val=48

./xmlchange -file env_mach_pes.xml -id NTASKS_OCN -val=48

./xmlchange -file env_mach_pes.xml -id NTASKS_CPL -val=48

./xmlchange -file env_mach_pes.xml -id NTASKS_GLC -val=48

./xmlchange -file env_mach_pes.xml -id NTASKS_ROF -val=48

./xmlchange -file env_mach_pes.xml -id TOTALPES -val=$ntasks

set-run-opts:

cd $CASEROOT

./xmlchange -file env_run.xml -id CONTINUE_RUN -val ‘FALSE’

./xmlchange -file env_run.xml -id RESUBMIT -val ‘0’

./xmlchange -file env_run.xml -id STOP_N -val ’45’

./xmlchange -file env_run.xml -id STOP_OPTION -val ‘ndays’

./xmlchange -file env_run.xml -id REST_N -val ‘1’

./xmlchange -file env_run.xml -id REST_OPTION -val ‘nyears’

./xmlchange -file env_run.xml -id DOUT_L_MS -val ‘FALSE’

#Set start date for each ensemble

./xmlchange -file env_run.xml -id RUN_STARTDATE -val $run_start_date

./xmlchange -file env_run.xml -id DIN_LOC_ROOT -val $DATADIR

./xmlchange -file env_run.xml -id DOUT_S_ROOT -val $ARCHDIR

./xmlchange -file env_run.xml -id RUNDIR -val $RUNDIR

#—–

# Modify user namelist files

#—–

#Set initial conditions based on start date for each ensemble by ‘ncdata’; set temperature perturbation using ‘pertlim’

cat <<EOF >! user_nl_cam &camexp inithist = ‘YEARLY’

ncdata = ‘/work/04268/tg835671/inputdata/inic/F_2000_CAM5_0.9×1.25_ControlForInic.cam.i.$run_start_date-00000.nc’

pertlim = $valPert[$num]

nhtfrq = 0,-24,-3

mfilt = 1,1,8

fincl2 = ‘FSNS:A’, ‘FLNS:A’, ‘FSDS:A’, ‘FLDS:A’, ‘LHFLX:A’, ‘SHFLX:A’, ‘PRECT:A’, ‘QFLX:A’, ‘TREFHT:A’, ‘QREFHT:A’, ‘PS:A’, ‘Q:A’, ‘T:A’, ‘U:A’, ‘V:A’, ‘TMQ:A’, ‘LWCF:A’, ‘SWCF:A’, ‘PBLH:A’, ‘VQ:A’, ‘TAP:I’, ‘QAP:I’, ‘TTEND:A’, ‘DCQ:A’, ‘QRL:A’, ‘QRS:A’, ‘DTCOND:A’, ‘DTV:A’, ‘VD01:A’,’TS:A’,’OMEGA:A’,

fincl3 = ‘PRECT:A’,’LHFLX:A’,’SHFLX:A’,’PS:A’

/ EOF

#——————

##configure

#——————

config:

cd $CASEROOT

./cesm_setup

#Point the model to a previously built version of the model “F_2000_CAM5_0.9×1.25_EnsStart” and set build to TRUE.

./xmlchange -file env_build.xml -id EXEROOT -val $PTMP/F_2000_CAM5_0.9×1.25_EnsStart/bld

./xmlchange -file env_build.xml -id BUILD_COMPLETE -val ‘TRUE’

echo “CESM_setup ran okay for ensemble member $ensNum“

#——————

##Interactively build the model

#——————

#We no longer have to build the model!

#build:

#cd $CASEROOT

#./$CASE.build

cd $CASEROOT

sed -i “s/^#SBATCH -t .*/#SBATCH -t”$run_time” /” $CASE.run

sed -i “s/^#SBATCH -p .*/#SBATCH -p”$queue” /” $CASE.run

cd $CASEROOT

set bld_cmp = `grep BUILD_COMPLETE env_build.xml`

set split_str = `echo $bld_cmp | awk ‘{split($0,a,”=”); print a[3]}’`

set t_or_f = `echo $split_str | cut -c 2-5`

##if ( $t_or_f == “TRUE” ) then

##sbatch $CASE.run

##else

set t_or_f = `echo $split_str | cut -c 2-6` echo ‘Build not complete, BUILD_COMPLETE is:’ $t_or_f

#endif

#Increment ensemble number by 1

@ num++

end

“Random” temperature perturbations in CESM

CESM Ensemble Workflow