Credit options (proposal)

"Credit" is a number associated with completed jobs, reflecting how much floating-point computation was (or could have been) done. For CPU applications the basic formula is:

1 unit of credit = 1/200 day runtime on a CPU whose Whetstone benchmark is 1 GFLOPS.

Whetstone measures peak performance, and applications that do a lot of memory or disk access get lower FLOPS. So credit measures peak, not actual, FLOPs.

Credit is used for two purposes:

1. For users, to see their rate of progress, to compete with other users or teams, and to compare the performance of hosts.

2. To get an estimate of the peak performance available to a particular project, or of the volunteer host pool as a whole.

For 2) we care only about averages. For 1) we also care about parity between similar jobs; users get upset if someone else gets a lot more credit for a similar job.

BOINC provides 4 ways of determining credit. The choice (per app) depends on the properties of the app:

• If you can estimate a job's FLOPs in advance, use pre-assigned credit.

• Else if you can estimate a job's FLOPs after if completes, use post-assigned credit.

• Else if the app has only CPU versions, use runtime credit.

• Else use adaptive credit.

Pre-assigned credit

You can use this if the amount of computation done by each job is known in advance, e.g. if all jobs do the same computation. Measure the runtime on a machine with known Whetstone benchmarks. Pick a machine with enough RAM that you're not paging. The credit is then

(runtime in days)*benchmark*ncpus*200

ncpus is the number of CPUs used by the app version; use a sequential version if possible.

You can also use it if the runtime is a linear function of some job attribute (e.g. input file size) that's known in advance.

To specify:

• use the --credit argument to the create_work cmdline program
• if using the C++ API, assign wu.canonical_credit in the first argument.

You must run the app's validator with the --credit_from_wu option.

TODO: add to remote job submissions RPCs if anyone wants.

Post-assigned credit

Use this if you can estimate the FLOPs done by a completed job, based on the contents of its output files or stderr. For example, if your app has an outer loop, and you can measure (as above) the credit C due for each iteration, the job credit is C times the number of iterations performed.

To use this: in your validator, have the init_result() function return the credit for the job.

Runtime-based credit

Use this if the app has only CPU app versions. The "claimed credit" for a job instance is runtime*ncpus*benchmark.

To use this: pass the --credit_from_runtime option to the app's validator.

The app's efficiency (the ratio between peak FLOPS and actual FLOPS) can vary somewhat between hosts (e.g. because of different memory speeds, or because small RAM causes paging). Therefore there will be variation between claimed credit for identical jobs, but generally this will be a factor of 2 or so.

Runtime-based credit can't be used if the app has GPU versions because efficiency can vary by orders of magnitude between CPU and GPU versions.

Runtime-based credit is not cheat-proof.

Adaptive credit

Use this if you have GPU apps, and are unable to estimate FLOPs even after job completion. This method maintains performance statistics on a (host, app version) level, and uses these to normalize credit between CPU and GPU versions. See [CreditNew].

To use: this is the default.

If you use this, the adaptation will happen faster if you provide values for workunit fp_ops_est that are correlated with the actual FLOPs. Use a constant value if you're not sure.

Credit averaging

If you use replication, runtime-based and adaptive credit can produce different "claimed credit" for each job instance. The validator code averages these in tricky way I don't quite understand (Kevin invented it). It does not take the minimum. We should probably provide this as an option, to make runtime-based credit more cheat-proof. However, this won't work if you use adaptive replication, where many jobs have only one instance.