Changes between Version 11 and Version 12 of JobTemplates


Ignore:
Timestamp:
Jan 12, 2017, 1:46:51 AM (7 years ago)
Author:
davea
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • JobTemplates

    v11 v12  
    1414An input template file describes the job's input files, resource requirements, and scheduling parameters.
    1515
    16 Input files are of two types:
    17 
    18  * '''Variable files''' can vary between jobs.
    19    The job submission interfaces such as [JobSubmission#creatework-tool create_work()]
    20    supply the physical names of the input files for that job.
     16"Normal" input files are staged before job submission,
     17and their physical names are passed as arguments to the
     18job submission interfaces such as [JobSubmission#creatework-tool create_work()].
     19
     20There are also several special types of input files:
    2121
    2222 * '''Constant files''' are the same across multiple jobs.
    23    The physical name of the file is given in the input template;
    24    it is not passed to the job-submission interfaces.
     23   The physical name of the file is given in the input template,
     24   and is not passed to the job-submission interfaces.
    2525   The file must be staged before jobs are created with the template.
    2626   Constant files might be used for the files that make up a Docker image.
    27    
     27 * '''Remote files''': these are served from outside servers,
     28  i.e. not BOINC project servers.
     29  Their size and MD5 must be given explicitly since the files aren't locally available.
     30  There are two ways to specify remote files:
     31  * Pass a '''--remote_file URL nbytes MD5''' argument to create_work().
     32    The client will download the file from the given URL.
     33    Its physical name on the client will be "js_MD5".
     34  * Supply the URL, size, and MD5 in the input template.
     35    In this case the URL is of a directory;
     36    the name passed to create_work() will be appended to the URL,
     37    and will also be used as the physical name on the client.
     38    Note: in this case you're either generating an input template per job,
     39    or you're using the file like a constant file.
     40
    2841{{{
    2942<input_template>
    3043    <file_info>
    3144        <number>0</number>
    32         [ <physical_name>...</physical_name> ]
    3345        [ <gzip/> ]
     46        [ <gzipped_nbytes> N </gzipped_nbytes> ]
    3447        [ <sticky/> ]
    3548        [ <no_delete/> ]
    3649        [ <report_on_rpc/> ]
     50        [ <physical_name>...</physical_name> ]
    3751        [ <url>...</url> ]
    3852        [ <url>...</url> ]
    … …  
    6579Elements and tags must be on separate lines as shown.
    6680
     81=== <file_info> elements ===
     82
    6783Each '''<file_info>''' describes an [BoincFiles#Fileproperties input file]:
    6884
    69  '''<number>''':: use 0, 1, ...
    70  '''<physical_name>''':: for constant files, the physical name of the file.
     85 '''<number>''':: Use 0, 1, ...
    7186 '''<gzip/>''':: transfer the file in gzipped (compressed) format to reduce network usage.
    7287   You must [JobStage stage the file with the --gzip option].
    7388   Only 7.0+ clients can handle compressed transfers;
    7489   older clients will download the file in uncompressed form.
     90 '''<gzipped_nbytes>''':: if '''<gzip/>''' is specified, the size of the gzip file.
    7591 '''<sticky/>''':: if present, the file remains on the client after job is finished.
    7692 '''<no_delete/>''':: if present, the file is not deleted from the server after job is completed.  Use this if the file is used as input to more than one job.
    … …  
    7894   Include this for compatibility with old (pre-7.x) clients; 7.0+ clients report all sticky files.
    7995
    80 The following are used for files that are [JobStage staged] to
    81 a server (or servers) other than your BOINC server:
     96The following is used only for constant files:
     97 '''<physical_name>''':: the physical name of the file.
     98
     99The following are used for remote files specified in the input template:
    82100
    83101 '''<url>''':: specifies a directory (i.e. it should end with a /) to which the file name will
    … …  
    86104 '''<md5_cksum>''':: the file's MD5 checksum
    87105 '''<nbytes>''':: the file size.
    88  '''<gzipped_nbytes>''': if '''<gzip/>''' is specified, the size of the gzip file.
     106
     107=== <file_ref> elements ===
    89108
    90109The '''<file_ref>''' describes [BoincFiles#Filereferences the way the file is referenced]:
    91  '''<file_number>''':: 0, 1, etc.
     110 '''<file_number>''':: 0, 1, etc. (identifies the corresponding <file_info> element)
    92111 '''<open_name>''':: the [BoincFiles logical name] of the file
    93112 '''<copy_file>''':: if present, the file is copied into the job's slot directory
     113
     114=== Job parameters ===
    94115
    95116The job parameters include:
    … …  
    108129 '''<size_class>'''::
    109130  Specify the job's [MultiSize size class].
     131
     132=== Size limits ===
    110133
    111134The input template (substituted with filenames and URLs) is stored in a database field
    … …  
    190213(i.e. no more replicas will be created).
    191214
     215=== Size limits ===
     216
    192217The output template, substituted with filenames and URLs,
    193218is stored in a database field with a 64KB limit.
    … …  
    202227select max(length(xml_doc_in)), max(length(xml_doc_out)) from result;
    203228}}}
     229