Changes between Version 31 and Version 32 of WorkGeneration


Ignore:
Timestamp:
Sep 18, 2009, 3:56:32 PM (15 years ago)
Author:
davea
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • WorkGeneration

    v31 v32  
    11[[PageOutline]]
    2 = Submitting jobs =
    3 
    4 To submit a job:
    5 
    6  1. Write XML 'template files' that describe the job's input and outputs (typically the same template files can be used for many jobs).
    7  1. Create the job's input file(s), and put them in the right place (as determined by the file name) in the [DirHierarchy download directory hierarchy].
    8  1. Invoke a BOINC function or script that submits the job.
    9 
    10 Once this is done, BOINC takes over: it creates one or more instances of the job,
    11 distributes them to client hosts, collects the output files,
    12 finds a canonical instance, assimilates the canonical instance, and deletes files.
    13 
    14 Typically, steps 2) and 3) are done by a '''work generator''' program
    15 that creates lots of jobs.
    16 
    17 == Input and output template files == #templates
    18 
    19 An input template file has the form
    20 {{{
    21 <file_info>
    22     <number>0</number>
    23     [ <sticky/>, other attributes]
    24 </file_info>
    25 [ ... ]
    26 <workunit>
    27     <file_ref>
    28         <file_number>0</file_number>
    29         <open_name>NAME</open_name>
    30     </file_ref>
    31     [ ... ]
    32     [ <command_line>-flags xyz</command_line> ]
    33     [ <rsc_fpops_est>x</rsc_fpops_est> ]
    34     [ <rsc_fpops_bound>x</rsc_fpops_bound> ]
    35     [ <rsc_memory_bound>x</rsc_memory_bound> ]
    36     [ <rsc_disk_bound>x</rsc_disk_bound> ]
    37     [ <delay_bound>x</delay_bound> ]
    38     [ <min_quorum>x</min_quorum> ]
    39     [ <target_nresults>x</target_nresults> ]
    40     [ <max_error_results>x</max_error_results> ]
    41     [ <max_total_results>x</max_total_results> ]
    42     [ <max_success_results>x</max_success_results> ]
    43     [ <credit>X</credit> ]
    44 </workunit>
    45 }}}
    46 Elements and tags must be on separate lines as shown.
    47 The components are:
    48 
    49  <file_info>, <file_ref>::
    50 
    51         Each pair describes an [BoincFiles#Fileproperties input file] and [BoincFiles#Filereferences the way it's referenced].
    52 
    53  <command_line>::
    54 
    55         The command-line arguments to be passed to the main program.
    56 
    57  <credit>::
    58 
    59         The amount of credit to be granted for successful completion of this workunit. Use this only if you know in advance how many FLOPs it will take. Your [ValidationSimple validator] must use get_credit_from_wu() as its compute_granted_credit() function.
    60 
    61  Other elements::
    62         [JobIn Work unit attributes]
    63 
    64 An output template file has the form
    65 {{{
    66 <file_info>
    67     <name><OUTFILE_0/></name>
    68     <generated_locally/>
    69     <upload_when_present/>
    70     <max_nbytes>32768</max_nbytes>
    71     <url><UPLOAD_URL/></url>
    72 </file_info>
    73 <result>
    74     <file_ref>
    75         <file_name><OUTFILE_0/></file_name>
    76         <open_name>result.sah</open_name>
    77         [ <optional>0|1</optional> ]
    78         [ <no_validate>0|1</no_validate> ]
    79     </file_ref>
    80 </result>
    81 }}}
    82 
    83 Elements and tags must be on separate lines as shown.
    84 Each <file_info>, <file_ref> pair describes an output file.
    85 The elements include:
    86  '''<name>''' and '''<file_name>''':: the physical file name.  Typically use <OUTFILE_0>, <OUTFILE_1> etc.; this will be replaced with a generated name based on the job name.
    87  '''<generated_locally/>''':: always include this for output files.
    88  '''<max_nbytes>''':: maximum file size.  If the actual size exceeds this, the file will not be uploaded, and the job will be marked as an error.
    89  '''<url>''':: the URL of the file upload handler.  You may include this explicitly, or use '''<UPLOAD_URL/>''' to use the URL in your project's config.xml file.
    90  '''<open_name>''':: the "logical name" by which your application will reference the file.
    91  '''<optional>''':: if false, your application must create the file, otherwise the job will be marked as an error.
    92  '''<no_validate>''':: if true, don't include this file in the result validation process (relevant only if you are using the sample bitwise validator).
    93 
    94 Both elements may have other properties, see BoincFiles for the full list.
    95 
    96 Note: when a job is created, the name of its output template file is stored in the database.
    97 The file is read when instances of the job are created, which may happen days or weeks later.
    98 Thus, editing an output template file can affect existing jobs.
    99 If this is not desired, you must create a new output template file.
    100 
    101 You can safely remove a workunit template file after creating your last WU with it.
    102 However, you can't delete a result template file until any WU that refers to it is completed
    103 (i.e. no more replicas will be created).
    104 
    105 == Submitting a job manually == #creatework-tool
    106 
    107 To move an input file to the download directory, use
    108 {{{
    109 dir_hier_path filename
    110 }}}
    111 which prints the full pathname and creates the directory if needed (run this in the project's root directory). For example:
    112 {{{
    113 cp test_files/12ja04aa `bin/dir_hier_path 12ja04aa`
    114 }}}
    115 copies an input file from the test_files directory to the download directory hierarchy.
    116 
    117 To submit a job, run bin/create_work from the project root directory:
    118 {{{
    119 create_work [ arguments ] infile_1 ... infile_n
    120 }}}
    121 Mandatory arguments are:
    122     --appname name::
    123   application name
    124     --wu_name name::
    125         workunit name
    126     --wu_template filename::           
    127  WU template filename relative to project root; usually in templates/
    128     --result_template filename::
    129           result template filename, relative to project root; usually in templates/
    130 Optional arguments are:
    131     --batch n::
    132     --priority n::
    133 
    134 The following [JobIn job parameters] may be passed in the input template,
    135 or as command-line arguments to create_work,
    136 or not passed at all, in which case defaults will be used.
    137 
    138     --command_line "-flags foo"::
    139     --rsc_fpops_est x:: FLOPs estimate
    140     --rsc_fpops_bound x::
    141     --rsc_memory_bound x::
    142     --rsc_disk_bound x::
    143     --delay_bound x::
    144     --min_quorum x::
    145     --target_nresults x::
    146     --max_error_results x::
    147     --max_total_results x::
    148     --max_success_results x::
    149     --opaque N::
    150     --additional_xml 'x':: This can be used to supply, for example, <credit>12.4</credit>.
    151 
    152 == Submitting jobs from a C++ program == #cpp-workgen
    153 
    154 BOINC's library provides the functions:
    155 {{{
    156 // convert filename to path in download hierarchy
    157 //
    158 SCHED_CONFIG::download_path(
    159     const char* filename,
    160     char* path,
    161 );
    162 
    163 // submit a job
    164 //
    165 int create_work(
    166     DB_WORKUNIT& wu,
    167     const char* wu_template,                  // contents, not path
    168     const char* result_template_filename,     // relative to project root
    169     const char* result_template_filepath,     // absolute or relative to current dir
    170     const char** infiles,                     // array of input file names
    171     int ninfiles
    172     SCHED_CONFIG&,
    173     const char* command_line = NULL,
    174     const char* additional_xml = NULL
    175 );
    176 }}}
    177 The ''name'' and ''appid'' fields of the DB_WORKUNIT structure must always be initialized.
    178 Other job parameters may be passed either in the DB_WORKUNIT structure or in the input template file (the latter has priority).
    179 On a successful return, wu.id contains the database ID of the workunit.
    180 
    1812=== Making one workunit === #cpp-one
    1823