How to compute SBUs usage on ECMWF cca/ccb

The accounting currency on ECMWF supercomputer is the SBU (System Billing Unit)
The accounting system is fully described here:
https://software.ecmwf.int/wiki/display/UDOC/HPC+accounting

Basically, "To ensure that computing resources are distributed equitably and to discourage irresponsible use users' jobs are charged" and

    SBU = P * (ELAPSED_TIME)  
 where  
 P = 50365140 / (3617 * 36 * 86400)       
 is a fixed proportionality factor, adjusted so that the system will deliver 50365140 units per day on a "typical"  
 ECMWF job mix on the ccb system (3617 * 36 is the number of physical CPUs of the cray system and 86400  
 the number of seconds in a day)  
 ELAPSED_TIME       
 is the elapsed time (real time in seconds) recorded for a job, multiplied by the number of physical CPUs used by the job  

Examples (copied from ECMWF User Documentation):
  1. A serial job using 1 hour (3600 seconds) of elapsed time will be charged about 16.11 SBU.
  2. A parallel job requesting 2 nodes, running for 3 hours elapsed time will be charged 3481.14 SBU.
Practically, how much does it cost my job?
Here is a recipe with the IFS global model and the WRF regional model deduced from experiments on cca/ccb:
1. one day (a 24-hour period) of simulation with IFS (model cycle 43r1) at TCO639  spectral resolution (cubic octahedral reduced Gaussian grid) with 91 vertical levels costs about 1466 SBU
2. one day (a 24-hour period) of simulation with WRF model at 3 km of horizontal resolution (400X440 rows X columns) with 60 vertical levels and 15 seconds as time step costs about 2540 SBU with the following PBS directives for the parallel job:
  •  EC_total_tasks=240
  •  EC_threads_per_task=1
  •  EC_hyperthreads=1
In other words:
1. a full ENS experiment (15-day forecast) with the IFS model at O639 spectral resolution costs about 15*1466*51~1.1 M SBU (that's only an approximation because after day 10 the resolution is lower (need to check this...))
2. a full WRF-ENS experiment (15-day forecast) with the above settings costs about 15*2540*51~1.9 M SBU

In other (other) words:
suppose you want to simulate an extreme event occurred on day X and you want to investigate the predictability of the event by running ENS and WRF-ENS (WRF ensemble nested in ENS), one, two, ..., seven days in advance (once per day) day X, you have to simulate a total number of (7*(7+1))/2 = 28 days.
So you need:
  • 28*1466*51 ~ 2.1 M SBU for the ENS experiment
  • 28*2540*51 ~ 3.6 M SBU for the WRF-ENS experiment
for a total cost of about 5.7 M SBU. Of course if you want to run the experiments twice per day (00 and 12 UTC), it will cost you about 11.4 M SBU.

Commenti

Post popolari in questo blog

How to run the WRF model using ERA5 (on model levels) as initial and boundary conditions

Yet another tutorial on how to install WRF

Code published via github and zenodo