Remote job submission - overview

In early versions of BOINC, scientists submitted jobs to a BOINC server by logging in to the server and running programs or scripts. More recently, BOINC has been used to build systems, collectively called remote job submission systems, in which scientists can submit jobs without logging in to the BOINC server.

In all these systems, job submitters are identified using accounts on the BOINC project (the same kind of accounts as volunteers; job submitters can also act as resource providers). Access control is provided by BOINC's multi-user project features: users can submit jobs only if they have been given access by project administrators, and admins can restrict the apps for which each user is allowed to submit jobs. Users have quotas how resources are allocated to their jobs.

The structure and details of remote job submission systems vary widely. BOINC doesn't include a pre-packaged remote job submission system, but it provides a number of components that can be used to build such systems. We'll list some example systems, then describe the components.

Example: single-server portal

Users submit jobs using forms on the project web site:

These forms are application-specific; you must develop them yourself. Two examples are included in the BOINC distribution:

• html/user/tree_threader.php
• html/user/lammps.php

TODO: the above are full of application-specific complexity; provide a minimal/generic example.

The job-submission scripts use local PHP interfaces to authenticate users and to create jobs and batches.

Input files can be handled in any of several ways:

• Uploading them (from the submitter's computer) as part of the submission form. The submission script would then stage them.
• Using the user file sandbox mechanism.
• Serving them from a remote server.

Example: multi-server portal

In such systems, users submit jobs using forms on a web site other than the project web site.

The scripts implementing these forms would use Web RPCs to create batches and jobs.

Input files can be handled in any of several ways:

• Managing them using Web RPCs.
• Using the user file sandbox mechanism.
• Serving them from a remote server.

Example: Condor/BOINC bridge

In such systems, users submit jobs to a Condor system using any a command-line or GUI-based interface. The Condor system may, depending on load conditions, route the job to a BOINC project. This system uses web RPCs for all functions.

Components supplied by BOINC

Job submission control panel

The web page submit.php allows users to view submitted jobs and retrieve their output files. This link should be shown only to authorized users. You can do this, e.g., by putting the following on your home page (index.php):

$user = get_logged_in_user(false);
if ($user && BoincUserSubmit::lookup_userid($user->id);
    echo '
        <li><a href=submit.php>Job submission</a>
        <li><a href=sandbox.php>File sandbox</a>
    ';
}

(include the "File sandbox" link if you use this feature - see below).

Local PHP interfaces

Job-submission scripts to be run on the BOINC server are best implemented in PHP. BOINC provides a large set of PHP APIs for accessing the BOINC database, staging files, and so on.

For example, you can authorize a job submission request with code like

$user = get_logged_in_user();
$user_submit = BoincUserSubmit::lookup_userid($user->id);
if (!$user_submit) error_page("no submit access");
$app = BoincApp::lookup("name='lammps'");
if (!$app) error_page("no lammps app");
if (!$user_submit->submit_all) {
    $usa = BoincUserSubmitApp::lookup("user_id=$user->id and app_id=$app->id");
    if (!$usa) {
        error_page("no submit access");
    }
}

You can create a batch as follows:

$batch_id = BoincBatch::insert(
    "(user_id, create_time, njobs, name, app_id, state)
    values ($user->id, $now, $njobs, '$batch_name', $app->id, ".BATCH_STATE_IN_PROGRESS.")"
);

TODO: document these interfaces, or provide simpler examples.

Web RPCs for input file management

BOINC provides Web RPCs for remotely managing input files, and a C++ binding of this API.

Web RPCs for job submission

BOINC provides Web RPCs for creating and managing jobs.

Per-user file sandbox

BOINC provides a web-based system allowing users to remotely manage input files.