Automated estimation of job and app version characteristics

Goals

• eliminate the need for projects to supply FLOPs estimate for jobs
• eliminate the need for projects to supply FLOPS estimates for app versions (in app_plan())

Outline

Server

For each app, maintain

• flops_avg: the estimated average number of FLOPs used by the app's jobs
• flops_stdev: the standard deviation of the above

Note: if a project has different types of jobs for a given app, with widely differing durations, they should create separate apps for them. That will reduce the variance of the estimate.

Initially this is set to a high value (e.g. 1 GFLOP-day).

Update: whenever a complete job is reported, let x = duration * (host's flops_est). Update flops_avg in a way that favors decrease over increase. That way, hosts that execute the app efficiently (close to peak hardware speed) have a larger weight in the estimate.

Job completion time estimate: app.flops_avg / (host's flops_est for this app version)

client

for each app version, maintain

• flops_est: dynamic estimate of the real FLOPS of the app version on this host.

Initially this is based on the peak hardware speed, i.e. ngpus*(GPU peak FLOPs) + avg_ncus * (whetstone). Update: when a job finishes, let x = (job.flops / duration). Update accordingly (but cap at peak hardware speed).

Note: this replaces "duration correction factor".

protocol

Request message: add flops_est for each app version

Credit

Grant each validated job credit proportional to app.flops_avg