For a sensitivity study in a manuscript of ours that’s currently in the discussion phase at the HESS journal, we ran a couple of simulations with the Intermediate Complexity Atmospheric Research Model for the South Island of New Zealand. Horizontally the domain contained 205×225 grid points while the number of z-levels was varied between 7 and 25, corresponding to model top elevations above topography between 0.7 and 8.0 km (see Figure 2 in the linked manuscript).
Study Domain – 4x4km² grid spacing digital elevation model of the South Island of New Zealand
One of the main advantages of ICAR is that it is computationally rather inexpensive compared to full atmospheric models such as WRF for instance (see Gutmann et al., 2016). Whiĺe we did not set up our own WRF simulations for the South Island of New Zealand we nonetheless took a look at the core hours required for the ICAR runs. Core hours hereby meaning the amount of hours the simulation would have needed to complete on only one processor.
The simulations were run on one node of the NCAR’s cheyenne cluster which “…features 145,152 latest-generation Intel Xeon processor cores in 4,032 dual-socket nodes (36 cores/node) and 313 TB of total memory.”
Boxplot of core hours required for the ICAR simulation of one calendar month in dependence of the number of vertical levels. The whiskers indicate the range of the data while the upper boundary of the box shows the 75th percentile, the horizontal line the median and the lower boundary the 25h percentile.
The boxplot above is based on the core hours required to simulate each calendar month of 2014 separately. For instance, for 15 vertical levels the median number of core hours required to simulate one month is 27 hours. Not each calendar month requires the same amount of core hours, a spread in the data is clearly visible. This is not explicable by the difference in calendar days alone, most likely the stability criterion keeping the advection numerics stable may, in some cases, demand a smaller model time step, leading to an increased computational cost. Note that the perceived exponential increase in core hours in the plot above is due to the equidistant spacing of non-equidistant numbers on the x-axis. The increase is, in fact, linear.
So how many core hours does it require to simulate one year for the domain depicted above?
Number of core hours required for the ICAR simulation of one year for the South Island of New Zealand.
For the optimal model top setting of 4 km that we found for this domain (see Horak et al., 2018), ICAR needs about 200 core hours to simulate an entire year. Gutmann et al. (2016) calculated that ICAR, when run with 14 vertical model levels, is approximately 140 times faster than WRF (see Gutmann et al., 2016 for the WRF setup details). So for our domain this would translate to a similarly set up WRF requiring approximately 44,600 core hours to simulate one year. Compared to the ~200 core hours for a 12 vertical model levels this is a factor of about 223! Of course this is just extrapolated, we did not do any WRF simulations for our domain. However it gives a good idea of the potential time that may be saved when using ICAR instead of a full numerical model. Whether this is a viable option for every region on the globe is still a matter of investigation.
References
Gutmann, E., Barstad, I., Clark, M., Arnold, J., and Rasmussen, R.: The Intermediate Complexity Atmospheric Research Model (ICAR),
10 Journal of Hydrometeorology, 17, 957–973, https://doi.org/10.1175/JHM-D-15-0155.1, 2016.
Horak, J., Hofer, M., Maussion, F., Gutmann, E., Gohm, A., and Rotach, M. W.: Assessing the Added Value of the Intermediate Complexity Atmospheric Research Model (ICAR) for Precipitation in Complex Topography, Hydrol. Earth Syst. Sci. Discuss., https://doi.org/10.5194/hess-2018-612, in review, 2018.