Changes between Version 1 and Version 2 of RemoteJob


Ignore:
Timestamp:
Sep 9, 2010, 2:29:36 AM (14 years ago)
Author:
tonig
Comment:

Explain rboinc templates

Legend:

Unmodified
Added
Removed
Modified
  • RemoteJob

    v1 v2  
    22
    33A group from Universitat Pompeu Fabra
    4 has developed a system for remote job submission and monitoring.
     4has developed ''RBoinc'', system for remote job submission and monitoring.
    55This system allows scientists to submit jobs (or groups of jobs)
    6 from a convenient command-line interface.
     6from a convenient command-line interface.
     7
     8In the following, we will use the ''scientist'' term to denote the individual who submits and administers the workunits on their workstation through the RBoinc 'client tools'. RBoinc client tools are not to be confused with BOINC clients (i.e. the slaves of the distributed computing architecture); for utmost clarity, we shall prefer the term ''scientist'' and ''scientist workstation'' to indicate the user of the  RBoinc client and their machine.
    79
    810The system (Perl-based) is in boinc/rboinc/.
     
    1214
    1315Powerpoint slides describing the system are
    14 [http://boinc.berkeley.edu/rboinc.pdf here].
     16[http://boinc.berkeley.edu/rboinc.pdf here].
     17For details please see the paper T. Giorgino, M. J. Harvey and G. De Fabritiis, ''Distributed computing as a virtual supercomputer: Tools to run and manage large-scale BOINC simulations'', Comp. Phys. Commun. 181, 1402 (2010).  [[http://boinc.berkeley.edu/rboinc.pdf pdf]]
    1518
    1619== Notes ==
     
    3437    * I implemented basic functions for authentication, but this is not finished yet
    3538    * file storage is optimized through hardlinking and pooling. Network transfers are not (but they could be)
     39
     40
     41== Syntax for workunit template files ==
     42
     43Workunit template files should be marked as RBoinc-enabled at the top. This marks the template as RBoinc-enabled and thus ''scientist-visible'' as an application.
     44
     45This is achieved prepending the following tag to the relevant workunit template:
     46
     47{{{
     48<rboinc application="md"
     49        description="Standard ACEMD run with optional DCD and PLUMED"/>
     50}}}
     51
     52The ''application'' attribute will be the user-visible name of the application (which may or may not coincide with BOINC application names). The scientist will identify this template through the {{{-app }}} command line switch.
     53
     54
     55=== File references ===
     56
     57Workunit template files are augmented with RBoinc-related settings. In the WU template, each {{{file_ref}}} element has now a child {{{rboinc element}}} as follows:
     58
     59{{{
     60#!xml
     61   <file_ref>
     62        <file_number>3</file_number>
     63        <open_name>input.vel</open_name>
     64        <copy_file/>
     65        <rboinc parameter_name="vel_file"
     66                parameter_description="Binary velocities"
     67                [ optional="true" ]
     68                />
     69    </file_ref>
     70}}}
     71
     72The ''parameter_name'' attribute is the command line parameter that will be required by the ''boinc_submit'' command for that file. The argument passed by the scientist on the command line to that parameter will be interpreted as a local file, transferred to the BOINC server, and associated to the given BOINC-handled file (in this case, number 3, with BOINC open name "input.vel").
     73
     74The ''parameter_description'' is a descriptive text returned by the command line client when the scientist requests help for the attributes supported by the given application.
     75
     76The optional ''optional'' flag specifies whether supplying the given file upon submission is mandatory or not. If not, it will be replaced by a (server-supplied) default file.
     77
     78
     79
     80== Syntax for results template files ==
     81
     82Results template files are annotated with RBoinc-specific tags which identify which results should be transferred back to the scientist's workstation. The same tags can be used to build output-input ''chains'', i.e. to automatically submit new workunits as continuations of successfully-completed ones.
     83
     84The syntax for the results template is as follows:
     85
     86{{{
     87#!xml
     88<file_info>
     89    <name><OUTFILE_0/></name>
     90    <generated_locally/>
     91    <upload_when_present/>
     92    <max_nbytes>100000000</max_nbytes>
     93    <url><UPLOAD_URL/></url>
     94    <gzip_when_done/>
     95    <rboinc aliases=".vel" chain="3"/>
     96</file_info>
     97}}}
     98
     99The ''chain'' attribute indicates that, upon successful WU completion, that output file should be used as a third input file for the next step in the chain. The ''aliases'' attribute (currently unimplemented) allows automatic renaming upon retrieval.
     100
     101For details on the chaining mechanism, please see the paper T. Giorgino, M. J. Harvey and G. De Fabritiis, ''Distributed computing as a virtual supercomputer: Tools to run and manage large-scale BOINC simulations'', Comp. Phys. Commun. 181, 1402 (2010).  [[http://boinc.berkeley.edu/rboinc.pdf pdf]]