How to compute SBUs usage on ECMWF cca/ccb
The accounting currency on ECMWF supercomputer is the SBU (System Billing Unit)
The accounting system is fully described here:
https://software.ecmwf.int/wiki/display/UDOC/HPC+accounting
Basically, "To ensure that computing resources are distributed equitably and to discourage irresponsible use users' jobs are charged" and
Examples (copied from ECMWF User Documentation):
Here is a recipe with the IFS global model and the WRF regional model deduced from experiments on cca/ccb:
1. one day (a 24-hour period) of simulation with IFS (model cycle 43r1) at TCO639 spectral resolution (cubic octahedral reduced Gaussian grid) with 91 vertical levels costs about 1466 SBU
2. one day (a 24-hour period) of simulation with WRF model at 3 km of horizontal resolution (400X440 rows X columns) with 60 vertical levels and 15 seconds as time step costs about 2540 SBU with the following PBS directives for the parallel job:
1. a full ENS experiment (15-day forecast) with the IFS model at O639 spectral resolution costs about 15*1466*51~1.1 M SBU (that's only an approximation because after day 10 the resolution is lower (need to check this...))
2. a full WRF-ENS experiment (15-day forecast) with the above settings costs about 15*2540*51~1.9 M SBU
In other (other) words:
suppose you want to simulate an extreme event occurred on day X and you want to investigate the predictability of the event by running ENS and WRF-ENS (WRF ensemble nested in ENS), one, two, ..., seven days in advance (once per day) day X, you have to simulate a total number of (7*(7+1))/2 = 28 days.
So you need:
The accounting system is fully described here:
https://software.ecmwf.int/wiki/display/UDOC/HPC+accounting
Basically, "To ensure that computing resources are distributed equitably and to discourage irresponsible use users' jobs are charged" and
SBU = P * (ELAPSED_TIME)
where
P = 50365140 / (3617 * 36 * 86400)
is a fixed proportionality factor, adjusted so that the system will deliver 50365140 units per day on a "typical"
ECMWF job mix on the ccb system (3617 * 36 is the number of physical CPUs of the cray system and 86400
the number of seconds in a day)
ELAPSED_TIME
is the elapsed time (real time in seconds) recorded for a job, multiplied by the number of physical CPUs used by the job
Examples (copied from ECMWF User Documentation):
- A serial job using 1 hour (3600 seconds) of elapsed time will be charged about 16.11 SBU.
- A parallel job requesting 2 nodes, running for 3 hours elapsed time will be charged 3481.14 SBU.
Here is a recipe with the IFS global model and the WRF regional model deduced from experiments on cca/ccb:
1. one day (a 24-hour period) of simulation with IFS (model cycle 43r1) at TCO639 spectral resolution (cubic octahedral reduced Gaussian grid) with 91 vertical levels costs about 1466 SBU
2. one day (a 24-hour period) of simulation with WRF model at 3 km of horizontal resolution (400X440 rows X columns) with 60 vertical levels and 15 seconds as time step costs about 2540 SBU with the following PBS directives for the parallel job:
- EC_total_tasks=240
- EC_threads_per_task=1
- EC_hyperthreads=1
1. a full ENS experiment (15-day forecast) with the IFS model at O639 spectral resolution costs about 15*1466*51~1.1 M SBU (that's only an approximation because after day 10 the resolution is lower (need to check this...))
2. a full WRF-ENS experiment (15-day forecast) with the above settings costs about 15*2540*51~1.9 M SBU
In other (other) words:
suppose you want to simulate an extreme event occurred on day X and you want to investigate the predictability of the event by running ENS and WRF-ENS (WRF ensemble nested in ENS), one, two, ..., seven days in advance (once per day) day X, you have to simulate a total number of (7*(7+1))/2 = 28 days.
So you need:
- 28*1466*51 ~ 2.1 M SBU for the ENS experiment
- 28*2540*51 ~ 3.6 M SBU for the WRF-ENS experiment
Commenti
Posta un commento