Changes between Version 19 and Version 20 of RemoteJobs


Ignore:
Timestamp:
Jul 27, 2011, 11:22:44 AM (13 years ago)
Author:
davea
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • RemoteJobs

    v19 v20  
    33
    44This document describes an API for remotely submitting jobs to a BOINC server.
    5 The APIs support the submission of large '''batches''' of jobs,
    6 such as parameter sweeps.
    7 The API can be used, for example, to create web interfaces for job submission,
    8 such as might be found in a science portal web site:
    9 
    10 [[Image(submit.png)]]
    11 
     5The API supports the submission of '''batches''' of jobs.
     6A batch could contain a single job, or many thousands of jobs.
    127Currently, the API has two restrictions:
    138
    … …  
    1712BOINC provides a PHP library that implements the API.
    1813The underlying web services are implemented as HTTP/XML RPCs,
    19 and it is easy to create bindings in other languages.
     14and it is possible to create bindings in other languages.
    2015
    21 This system is coupled with new
    22 [MultiUser multi-user project features].
    23 In particuler, users can submit jobs only if they have been given access
    24 (via a web interface) by project administrators,
    25 and admins can restrict the apps for which
    26 each user is allowed to submit jobs.
     16The API can be used, for example, to create web interfaces for job submission,
     17such as might be found in a science portal web site:
     18
     19[[Image(submit.png)]]
     20
     21This system is coupled with BOINC's [MultiUser multi-user project features].
     22This includes access control:
     23users can submit jobs only if they have been given access by project administrators,
     24and admins can restrict the apps for which each user is allowed to submit jobs.
    2725
    2826== PHP interface ==
    … …  
    3230which is independent of other BOINC code and can be used in the Portal web code.
    3331
    34 === boinc_estimate_batch() ===
     32=== boinc_submit_batch() ===
    3533
    36 Returns an estimate of the elapsed time required to complete a batch.
     34Submits a batch.
    3735
    3836Arguments: a "request object" whose fields include
    … …  
    4038 * '''authenticator''': the user's authenticator
    4139 * '''app_name''': the name of the application for which jobs are being submitted
    42  * '''jobs''': an array of job descriptors (see example)
     40 * '''batch_name''': a symbolic name for the batch.  Need not be unique.
     41 * '''jobs''': an array of job descriptors, each of which contains
     42  * '''rsc_fpops_est''': an estimate of the FLOPs used by the job
     43  * '''command_line''': command-line arguments to the application
     44  * '''input_files''': an array of input file descriptors, each of which contains
     45   * '''source''': a URL from which the BOINC server can download the file,
     46     or the path of the file on the BOINC server
     47   * '''name''': the [BoincFiles physical name] of the file
    4348
    44 Return value: a 2-element array containing
    45  * The elapsed time estimate, in seconds
     49Result: a 2-element array containing
     50 * The batch ID
    4651 * An error message (null if success)
    4752
    48 Example usage:
     53The following example submits a 10-job batch:
    4954{{{
    5055$req->project = "http://foo.bar.edu/test/";
    … …  
    7176}}}
    7277
    73 === boinc_submit_batch() ===
     78=== boinc_estimate_batch() ===
    7479
    75 Similar to '''boinc_estimate_batch()''', but submits the batch.
     80Returns an estimate of the elapsed time required to complete a batch.
    7681
    77 Result: a 2-element array containing
    78  * The batch ID
     82Arguments: same as '''boinc_submit_batch()'''
     83(only relevant fields need to be populated).
     84
     85Return value: a 2-element array containing
     86 * The elapsed time estimate, in seconds
    7987 * An error message (null if success)
    8088
    … …  
    8795
    8896Result: a 2-element array.
    89 The first element is an array of objects describing batches,
    90 with the following fields:
    91  * '''batch_id''': batch ID
    92  * '''fraction_done''': a number 0..1 giving the fraction of the batch that has been completed
     97The first element is an array of batch descriptor objects,
     98each with the following fields:
     99 * '''id''': batch ID
     100 * '''state''': values are
     101  * 1: in progress
     102  * 2: completed (all jobs either succeeded or had fatal errors)
     103  * 3: aborted
     104  * 4: retired
     105 * '''name''': the batch name
     106 * '''app_name''': the application name
    93107 * '''create_time''': when the batch was submitted
    94  * '''est_completion_time''': an updated estimate of completion time
     108 * '''est_completion_time''': current estimate of completion time
    95109 * '''njobs''': the number of jobs in the batch
     110 * '''fraction_done''': the fraction of the batch that has been completed (0..1)
     111 * '''nerror_jobs''': the number of jobs that had fatal errors
     112 * '''completion_time''': when the batch was completed
     113 * '''credit_estimate''': BOINC's initial estimate of the credit that
     114  would be granted to complete the batch, including replication
     115 * '''credit_canonical''': the actual credit granted to canonical instances
     116 * '''credit_total''': the actual credit granted to all instances
    96117
    97118=== boinc_query_batch() ===
    … …  
    104125
    105126Result: a 2-element array.
    106 The first element is an array of objects describing jobs
    107 within the batch, with the following fields:
    108 
    109  * '''job_id''': the job ID
    110  * '''canonical_instance_id''':
    111   if the job has been completed and validated, the ID of a valid instance
    112  * '''n_outfiles''': the number of output files
     127The first element is a batch descriptor object as described above,
     128with one additional field:
     129 * '''jobs''': an array of job descriptor objects, each one containing
     130  * '''id''': the database ID of the job's workunit record
     131  * '''canonical_instance_id''': if the job has a canonical result, its database ID
    113132
    114133=== boinc_query_job() ===
    … …  
    121140
    122141Result: a 2-element array.
    123 The first element is an object with the following fields:
    124  * '''instances''': an array of job instance descriptors
    125 
    126 Each job instance descriptor is an object with the following fields:
    127  * '''name''': the instance's name
    128  * '''id''': the ID of the corresponding result record
    129  * '''outfile''': a list of output file descriptors, each containing
    130   * '''size''': file size in bytes
     142The first element is a job descriptor object with the following fields:
     143 * '''instances''': an array of job instance descriptors, each containing:
     144  * '''name''': the instance's name
     145  * '''id''': the ID of the corresponding result record
     146  * '''state''': a string describing the instance's state
     147   (unsent, in progress, complete, etc.)
     148  * '''outfile''': if the instance is over,
     149   a list of output file descriptors, each containing
     150   * '''size''': file size in bytes
    131151
    132152=== boinc_abort_batch() ===
    … …  
    173193At a lower level, the APIs are accessed by sending a POST request,
    174194using HTTPS, to PROJECT_URL/submit.php.
     195The inputs and outputs of each function are XML documents.
     196The format of the request and reply XML documents
     197can be inferred from inc/submit.inc and user/submit.php.
     198
    175199Bindings of these RPCs can be implemented in languages other than PHP.
    176 
    177 The inputs and outputs of each function are XML documents.
    178 If an operation fails, the output is of the form
    179 {{{
    180 <error>
    181    <message>X</message>
    182 </error>
    183 }}}
    184 where X is an error message.
    185 
    186 All operations include the authenticator of the user making the request.
    187 This user must be marked as a "job submitter" in the BOINC database.
    188 
    189 === Describing a batch ===
    190 
    191 A job is described by
    192 {{{
    193 <job>
    194    <rsc_fpops_est>X</rsc_fpops_est>
    195    [<command_line>C</command_line>]
    196    [<input_file>
    197       <source>URL or path</source>
    198       <name>physical_name</name>
    199    </input_file>]
    200    ... other input files
    201 </job>
    202 }}}
    203 
    204 Each input file is specified either by a URL accessible to the server,
    205 or by a path on the server.
    206 
    207 A batch of jobs is described by:
    208 {{{
    209 <batch>
    210    <app_name>appname</app_name>
    211    <job>
    212       ...
    213    </job>
    214    ... more jobs
    215 </batch>
    216 }}}
    217 
    218 === Batch runtime estimation ===
    219 
    220 This function estimates the completion time of a batch.
    221 Input:
    222 {{{
    223 <batch_estimate>
    224    <authenticator>X</authenticator>
    225    <batch> ... </batch>
    226 </batch_estimate>
    227 }}}
    228 
    229 Output:
    230 {{{
    231 <estimate>
    232    <seconds>X</seconds>
    233 </estimate>
    234 }}}
    235 
    236 === Submitting a batch ===
    237 
    238 Input:
    239 {{{
    240 <batch_submit>
    241    <authenticator>X</authenticator>
    242    <batch> ... </batch>
    243 </batch_submit>
    244 }}}
    245 
    246 Output:
    247 {{{
    248 <batch_id>N</batch_id>
    249 }}}
    250 
    251 === Querying batches ===
    252 
    253 Input:
    254 {{{
    255 <query_batches>
    256    <authenticator>X</authenticator>
    257 </query_batches>
    258 }}}
    259 
    260 Output:
    261 {{{
    262 <batches>
    263    <batch>
    264       <id>N</id>
    265       <completed>0|1</completed>
    266       [<fraction_done>X</fraction_done>]
    267       {<completed_time>X</completed_time>]
    268    </batch>
    269    ...
    270 </batches>
    271 }}}
    272 
    273 === Querying a batch ===
    274 
    275 Input:
    276 {{{
    277 <query_batch>
    278    <authenticator>X</authenticator>
    279    <batch_id>N</batch_id>
    280 </query_batch>
    281 }}}
    282 
    283 Output:
    284 {{{
    285 <batch>
    286    <job>
    287       <id>N</id>
    288       <canonical_resultid>N</canonical_resultid>
    289       [<outfile>name</outfile>]
    290       ...
    291    </job>
    292 </batch>
    293 }}}
    294 
    295 === Aborting a batch ===
    296 
    297 Input:
    298 
    299 {{{
    300 <abort_batch>
    301    <authenticator>X</authenticator>
    302    <batch_id>N</batch_id>
    303 </query_batch>
    304 }}}
    305 
    306 == Implementation notes ==
    307 
    308 New tables
    309 
    310 {{{
    311 batch
    312   id
    313   create_time
    314   logical_start_time
    315   logical_end_time
    316   estimated_completion_time
    317   njobs
    318 
    319 user_submit
    320   user_id
    321   quota
    322   logical_start_time
    323   bool all_apps
    324 
    325 user_app
    326   user_id
    327   app_id
    328 }}}
    329