Changes between Version 57 and Version 58 of RemoteJobs


Ignore:
Timestamp:
Mar 2, 2017, 11:59:59 AM (8 years ago)
Author:
davea
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • RemoteJobs

    v57 v58  
    22
    33= RPCs for remote job submission =
    4 This document describes APIs for remotely submitting, monitoring, and controlling jobs on a BOINC server. The APIs supports the submission of '''batches''' of jobs, which may contain a single job or many thousands of jobs. Currently, the API has two restrictions:
     4This document describes APIs for remotely submitting, monitoring, and controlling jobs on a BOINC server.
     5The APIs supports the submission of '''batches''' of jobs, which may contain a single job or many thousands of jobs.
     6Currently, the API has two restrictions:
    57
    68 * All jobs in a batch must use the same application.
    79 * There can be no dependencies between jobs.
    810
    9 At the bottom level, the interface consists of Web RPCs. BOINC provides client-side bindings in PHP, C++, and Python. These bindings differ slightly; they expose different details of the Web RPCs.
    10 
    11 Interfaces for [wiki:RemoteInputFiles staging input files] and [wiki:RemoteOutputFiles fetching output files] are described separately.
    12 
    13 There are various options for managing input files. If you use [wiki:RemoteInputFiles#Job-basedfilemanagement Job-based file management], which maintains batch/file associations, the order of operations is:
     11At the bottom level, the interface consists of Web RPCs. BOINC provides client-side bindings in PHP, C++, and Python.
     12These bindings differ slightly; they expose different details of the Web RPCs.
     13
     14Interfaces for [wiki:RemoteInputFiles staging input files] and [wiki:RemoteOutputFiles fetching output files]
     15are described separately.
     16
     17There are various options for managing input files.
     18If you use [wiki:RemoteInputFiles#Job-basedfilemanagement Job-based file management],
     19which maintains batch/file associations, the order of operations is:
    1420
    1521 * Create a batch (initially empty); returns the batch ID.
    … …  
    2430 * '''Abort''' the batch (if you see errors, or if enough jobs have been finished) using abort_jobs() or abort_batch().
    2531 * [wiki:RemoteOutputFiles download output files].
    26  * '''Retire''' the batch using retire_batch(). This tells the server to clean up the files and job records associated with the batch, and to mark the batch as "retired"; retired batches are normally not shown in the web interface.
     32 * '''Retire''' the batch using retire_batch().
     33   This tells the server to clean up the files and job records associated with the batch,
     34   and to mark the batch as "retired"; retired batches are normally not shown in the web interface.
    2735
    2836== PHP interface ==
    29 The following functions are provided in the PHP file [/trac/browser/trunk/boinc/html/inc/submit.inc submit.inc], which is independent of other BOINC PHP code. The file html/user/submit_test.php has code to exercise and test these functions.
     37The following functions are provided in the PHP file [/trac/browser/trunk/boinc/html/inc/submit.inc submit.inc],
     38which is independent of other BOINC PHP code.
     39The file html/user/submit_test.php has code to exercise and test these functions.
    3040
    3141=== boinc_submit_batch() ===
    … …  
    3747 * '''authenticator''': the user's authenticator
    3848 * '''app_name''': the name of the application for which jobs are being submitted
    39  * '''batch_name''': a symbolic name for the batch. Must be unique. If omitted, a name of the form "batch_unixtime" will be used.
     49 * '''batch_name''': a symbolic name for the batch.
     50   Must be unique.
     51   If omitted, a name of the form "batch_unixtime" will be used.
    4052 * '''workunit_template_file''': an optional [wiki:JobTemplates input template file name].
    4153 * '''result_template_file''': an optional [wiki:JobTemplates output template file name].
    … …  
    4355   * '''rsc_fpops_est''': an estimate of the FLOPs used by the job
    4456   * '''command_line''': command-line arguments to the application
    45    * '''wu_template''': optional; the [http://boinc.berkeley.edu/trac/wiki/JobTemplates#Inputtemplates input template] to use for this job, as an XML string.
    46    * '''result_template''': optional; the [http://boinc.berkeley.edu/trac/wiki/JobTemplates#Outputtemplates output template] to use for this job, as an XML string.
     57   * '''wu_template''': optional; the [http://boinc.berkeley.edu/trac/wiki/JobTemplates#Inputtemplates input template]
     58     to use for this job, as an XML string.
     59   * '''result_template''': optional; the [http://boinc.berkeley.edu/trac/wiki/JobTemplates#Outputtemplates output template]
     60     to use for this job, as an XML string.
    4761   * '''input_files''': an array of input file descriptors, each of which contains
    4862     * '''mode''': "local", "semilocal", "local_staged", "inline", or "remote" (see below).
    … …  
    6478Input files can be supplied in any of the following ways:
    6579
    66  * '''local''': the file is on the BOINC server and is not [wiki:JobStage staged]. It's specified by its full path.
    67  * '''local_staged''': the filed has been [wiki:JobStage staged] on the BOINC server. It's specified by its physical name.
    68  * '''semilocal''': the file is on a data server that's accessible to the BOINC server but not necessarily to the outside world. The file is specified by its URL. It will be downloaded by the BOINC server during job submission, and served to clients from the BOINC server.
    69  * '''inline''': the file is included in the job submission request XML message. It will be served to clients from BOINC server.
    70  * '''remote''': the file is on a data server other than the BOINC server, and will be served to clients from that data server. It's specified by the URL, the file size, and the file MD5.
     80 * '''local''': the file is on the BOINC server and is not [wiki:JobStage staged].
     81   It's specified by its full path.
     82 * '''local_staged''': the filed has been [wiki:JobStage staged] on the BOINC server.
     83   It's specified by its physical name.
     84 * '''semilocal''': the file is on a data server that's accessible to the BOINC server but not necessarily to the outside world.
     85   The file is specified by its URL.
     86   It will be downloaded by the BOINC server during job submission,
     87   and served to clients from the BOINC server.
     88 * '''inline''': the file is included in the job submission request XML message.
     89   It will be served to clients from BOINC server.
     90 * '''remote''': the file is on a data server other than the BOINC server,
     91   and will be served to clients from that data server.
     92   It's specified by the URL, the file size, and the file MD5.
    7193
    7294The following mode has been proposed but is not implemented yet:
    7395
    74  * '''sandbox''': the file is in the user's [wiki:RemoteInputFiles#Per-userfilesandbox sandbox], and is specified by its name in the sandbox.
     96 * '''sandbox''': the file is in the user's [wiki:RemoteInputFiles#Per-userfilesandbox sandbox],
     97   and is specified by its name in the sandbox.
    7598
    7699The following example submits a 10-job batch:
    … …  
    139162 * '''nerror_jobs''': the number of jobs that had fatal errors
    140163 * '''completion_time''': when the batch was completed
    141  * '''credit_estimate''': BOINC's initial estimate of the credit that would be granted to complete the batch, including replication
     164 * '''credit_estimate''': BOINC's initial estimate of the credit that would be granted to complete the batch,
     165   including replication
    142166 * '''credit_canonical''': the actual credit granted to canonical instances
    143167 * '''credit_total''': the actual credit granted to all instances
    … …  
    150174 * '''project''' and '''authenticator''': as above
    151175 * '''batch_id''': specifies a batch.
    152  * '''get_cpu_time''' (optional): if nonzero, get CPU time of batch. This includes all job instances, and doesn't include GPU time, so it may not be meaningful.
     176 * '''get_cpu_time''' (optional): if nonzero, get CPU time of batch.
     177   This includes all job instances, and doesn't include GPU time, so it may not be meaningful.
    153178 * '''get_job_details''' (optional): if nonzero, return job details (see below).
    154179
    155 Result: a 2-element array. The first element is a batch descriptor object as described above, with one additional field:
     180Result: a 2-element array.
     181The first element is a batch descriptor object as described above, with an additional element:
    156182
    157183 * '''jobs''': an array of job descriptor objects, each one containing
    158184   * '''id''': the database ID of the job's workunit record
    159    * '''canonical_instance_id''': if the job has a canonical result, its database ID
     185   * '''canonical_instance_id''': if the job has a canonical instance, its database ID
    160186
    161187If '''get_job_details''' was set, the job descriptors also contain:
    162188
    163  * '''status''': "unsent", "in_progress", "error", or "done".
    164  * '''cpu_time''': if done, the CPU time
     189 * '''status''': "queued", "in_progress", "error", or "done".
     190 * '''cpu_time''': if done, the CPU time of canonical instance.
    165191 * '''exit_status''': if error, the exit status of one of the error instances.
    166192
    … …  
    175201 * '''job_id''': specifies a job.
    176202
    177 Result: a 2-element array. The first element is a job descriptor object with the following fields:
     203Result: a 2-element array.
     204The first element is a job descriptor object with the following fields:
    178205
    179206 * '''instances''': an array of job instance descriptors, each containing:
    … …  
    417444}}}
    418445=== query_batch_set() ===
    419 Return the status of the jobs in a given set of batches. This is used by the Condor interface; it's probably not useful outside of that.
     446Return the status of the jobs in a given set of batches.
     447This is used by the Condor interface; it's probably not useful outside of that.
    420448
    421449{{{
    … …  
    440468}}}
    441469== Python binding ==
    442 The file tools/submit_api.py contains a Python binding of some of above RPCs. For examples of its use, see tools/submit_api_test.py.
     470The file tools/submit_api.py contains a Python binding of some of above RPCs.
     471For examples of its use, see tools/submit_api_test.py.
    443472
    444473To build a description of a batch of jobs, use the BATCH_DESC, JOB_DESC, and FILE_DESC classes:
    … …  
    531560 * jobs (list of JOB_DESC)
    532561
    533 Available functions are listed below. Each function takes a request object whose attributes include at least project and authenticator.
     562Available functions are listed below.
     563Each function takes a request object whose attributes include at least project and authenticator.
    534564
    535565 abort_batch(req):: req attributes: batch_id
    … …  
    546576
    547577== HTTP/XML interface ==
    548 At a lower level, the APIs are accessed by sending a POST request, using HTTP or HTTPS, to PROJECT_URL/submit_rpc_handler.php. The inputs and outputs of each function are XML documents. The format of the request and reply XML documents can be inferred from user/submit_rpc_handler.php.
     578At a lower level, the APIs are accessed by sending a POST request, using HTTP or HTTPS,
     579to PROJECT_URL/submit_rpc_handler.php.
     580The inputs and outputs of each function are XML documents.
     581The format of the request and reply XML documents can be inferred from user/submit_rpc_handler.php.
    549582
    550583== Example web interface ==
    551 An example of a web interface for job submission and control, based on this API, can be found here: http://boinc.berkeley.edu/trac/browser/trunk/boinc/html/user/submit_example.php
    552 
    553 This example is functional and it shows how to use the API. However, you will have to modify it heavily for your particular applications and web site.
     584An example of a web interface for job submission and control,
     585based on this API, can be found here:
     586http://boinc.berkeley.edu/trac/browser/trunk/boinc/html/user/submit_example.php
     587
     588This example is functional and it shows how to use the API.
     589However, you will have to modify it for your particular applications and web site.