Changes between Version 1 and Version 2 of BackendState


Ignore:
Timestamp:
Apr 25, 2007, 3:08:56 PM (18 years ago)
Author:
Nicolas
Comment:

Required manual changes to automatic conversion. HTML was the only way to get that table right.

Legend:

Unmodified
Added
Removed
Modified

  • BackendState

    v1 v2  
    44
    55
    6 === Workunit state variables ===
    7  Workunits parameters are described [JobIn here].
     6== Workunit state variables ==
     7Workunits parameters are described [JobIn here].
    88
    99Workunit state variables are as follows:
    1010
    11 
    12 
    13 ||'''canonical_resultid'''||The ID of the canonical result for this workunit, or zero.
     11{{{
     12#!html
     13<table border="1">
     14<tr><td><b>canonical_resultid</b></td><td>
     15}}}
     16The ID of the canonical result for this workunit, or zero.
    1417 * Initially zero
    1518 * Set by the validator (by check_set())
    16 
    17 ||||'''transition_time'''||The next time to check for state transitions for this WU.
     19{{{
     20#!html
     21</td></tr><tr><td><b>transition_time</b></td><td>
     22}}}
     23The next time to check for state transitions for this WU.
    1824 * Initially now.
    1925 * Set to now by scheduler when get a result for this WU.
    2026 * Set to min(current value, now + delay_bound) by scheduler     when send a result for this WU
    2127 * Set to min(x.sent_time + wu.delay_bound) over IN_PROGRESS results x     by transitioner when done handling this WU
    22  * Set to now by validator if it finds canonical result,     or if there is already a canonical result     and some other results have validate_state = INIT,     or if there is no consensus and the number of successful results     is > wu.max_success_results
    23 
    24 ||||'''file_delete_state'''||Indicates whether input files should be deleted.
     28 * Set to now by validator if it finds canonical result, or if there is already a canonical result and some other results have validate_state = INIT, or if there is no consensus and the number of successful results is > wu.max_success_results
     29{{{
     30#!html
     31</td></tr><tr><td><b>file_delete_state</b></td><td>
     32}}}
     33Indicates whether input files should be deleted.
    2534 * Initially INIT
    26  * Set to READY by transitioner when all results have server_state=OVER         and wu.assimilate_state=DONE         Note: db_purge purges a WU and all its results when         file_delete_state=DONE;         therefore it is critical that it only be set to DONE        if all results have server_state=OVER.
     35 * Set to READY by transitioner when all results have server_state=OVER         and wu.assimilate_state=DONE Note: db_purge purges a WU and all its results when file_delete_state=DONE; therefore it is critical that it only be set to DONE if all results have server_state=OVER.
    2736 * Set to DONE by file_deleter when it has attempted to delete files.
    28 
    29 ||||'''assimilate_state'''||Indicates whether the workunit should be assimilated.
     37{{{
     38#!html
     39</td></tr><tr><td><b>assimilate_state</b></td><td>
     40}}}
     41Indicates whether the workunit should be assimilated.
    3042 * Initially INIT
    31  * Set to READY by transitioner if wu.assimilate_state=INIT         and WU has error condition
    32  * Set to READY by validator when find canonical result         and wu.assimilate_state=INIT
     43 * Set to READY by transitioner if wu.assimilate_state=INIT and WU has error condition
     44 * Set to READY by validator when find canonical result and wu.assimilate_state=INIT
    3345 * Set to DONE by assimilator when done
    34 
    35 ||||'''need_validate'''||Indicates that the workunit has a result that needs validation.
     46{{{
     47#!html
     48</td></tr><tr><td><b>need_validate</b></td><td>
     49}}}
     50Indicates that the workunit has a result that needs validation.
    3651 * Initially FALSE
    3752 * Set to TRUE by transitioner if the number of success results         is at least wu.min_quorum and there is a success result         not validated yet
    3853 * Set to FALSE by validator
    39 
    40 ||||'''error_mask'''||A bit mask for error conditions.
     54{{{
     55#!html
     56</td></tr><tr><td><b>error_mask</b></td><td>
     57}}}
     58A bit mask for error conditions.
    4159 * Initially zero
    4260 * Transitioner sets COULDNT_SEND_RESULT if some result couldn't be sent.
    4361 * Transitioner sets TOO_MANY_RESULTS if too many error results
    4462 * Transitioner sets TOO_MANY_TOTAL_RESULTS if too many total results
    45  * Validator sets TOO_MANY_SUCCESS_RESULTS if no consensus         and too many success results
     63 * Validator sets TOO_MANY_SUCCESS_RESULTS if no consensus and too many success results
     64{{{
     65#!html
     66</td></tr></table>
     67}}}
    4668
    47 ||Workunit invariants:
    48 
     69Workunit invariants:
    4970
    5071 * eventually either canonical_resultid or error_mask is set
    … …  
    5475Notes on deletion of input files:
    5576
    56 
    5777 * Input files are eventually deleted, but only when all results have state=OVER (so that clients don't get download failures) and the WU has been assimilated (in case the project wants to examine input files in error cases).
    5878
    5979
    60 === Result state variable ===
    61  Result state variables are listed in the following table:
     80== Result state variable ==
    6281
    63 ||'''report_deadline'''||Give up on result (and possibly delete input files)     if don't get reply by this time.
     82Result state variables are listed in the following table:
     83{{{
     84#!html
     85<table border="1">
     86<tr><td><b>report_deadline</b></td><td>
     87}}}
     88Give up on result (and possibly delete input files) if don't get reply by this time.
    6489 * Set by scheduler to now + wu.delay_bound when send result
    65 
    66 ||||'''server_state'''||Values: UNSENT, IN_PROGRESS, OVER
     90{{{
     91#!html
     92</td></tr><tr><td><b>server_state</b></td><td>
     93}}}
     94Values: UNSENT, IN_PROGRESS, OVER
    6795 * Initially UNSENT
    6896 * Set by scheduler to IN_PROGRESS when send result
    … …  
    72100 * Set by transitioner to OVER if WU has error condition         and result.server_state=UNSENT
    73101 * Set by validator to OVER if WU has canonical result         and result.server_state=UNSENT
    74 
    75 ||||'''outcome'''||Values: SUCCESS, COULDNT_SEND, CLIENT_ERROR, NO_REPLY, DIDNT_NEED,     VALIDATE_ERROR, CLIENT_DETACHED.     Defined iff result.server_state=OVER
     102{{{
     103#!html
     104</td></tr><tr><td><b>outcome</b></td><td>
     105}}}
     106Values: SUCCESS, COULDNT_SEND, CLIENT_ERROR, NO_REPLY, DIDNT_NEED, VALIDATE_ERROR, CLIENT_DETACHED. Defined iff result.server_state=OVER
    76107 * Set by scheduler to SUCCESS if get reply and no client error
    77108 * Set by scheduler to CLIENT_ERROR if get reply and client error
    78109 * Set by scheduler to NO_REPLY if it thinks host has detached project.
    79  * Set by transitioner to NO_REPLY if server_state=IN_PROGRESS         and now < report_deadline
    80  * Set by transitioner to DIDNT_NEED if WU has error condition         and result.server_state=UNSENT
    81  * Set by validator to DIDNT_NEED if WU has canonical result         and result.server_state=UNSENT
    82  * Set by validator to VALIDATE_ERROR if outcome was initially         SUCCESS, but the validator had a permanent error reading a result file,         or a file had a syntax error.         Prevents the validator from trying again.
    83  * Set by scheduler to CLIENT_DETACHED if it gets a request         indicating that the client detached, then reattached
    84 
    85 ||||'''client_state'''||Records the client state (DOWNLOADING, DOWNLOADED,     COMPUTE_ERROR, UPLOADING, UPLOADED, ABORTED)     where an error occurred.     Defined if outcome is CLIENT_ERROR.     ||||'''file_delete_state'''||
     110 * Set by transitioner to NO_REPLY if server_state=IN_PROGRESS and now < report_deadline
     111 * Set by transitioner to DIDNT_NEED if WU has error condition and result.server_state=UNSENT
     112 * Set by validator to DIDNT_NEED if WU has canonical result and result.server_state=UNSENT
     113 * Set by validator to VALIDATE_ERROR if outcome was initially SUCCESS, but the validator had a permanent error reading a result file, or a file had a syntax error. Prevents the validator from trying again.
     114 * Set by scheduler to CLIENT_DETACHED if it gets a request indicating that the client detached, then reattached
     115{{{
     116#!html
     117</td></tr><tr><td><b>client_state</b></td><td>
     118}}}
     119Records the client state (DOWNLOADING, DOWNLOADED, COMPUTE_ERROR, UPLOADING, UPLOADED, ABORTED) where an error occurred. Defined if outcome is CLIENT_ERROR.
     120{{{
     121#!html
     122</td></tr><tr><td><b>file_delete_state</b></td><td>
     123}}}
    86124 * Initially INIT
    87  * Set by transitioner to READY if this is the canonical result,         and file_delete_state=INIT,         and wu.assimilate_state=DONE,         and all the results have server_state=OVER,         and all all the results with outcome=SUCCESS have validate_state<>INIT
    88  * Set by transitioner to READY if wu.assimilate_state=DONE         and result.outcome=CLIENT_ERROR         or result.validate_state!=INIT
    89 
    90 ||||'''validate_state'''||     Defined iff result.outcome=SUCCESS
     125 * Set by transitioner to READY if this is the canonical result, and file_delete_state=INIT, and wu.assimilate_state=DONE, and all the results have server_state=OVER, and all all the results with outcome=SUCCESS have validate_state<>INIT
     126 * Set by transitioner to READY if wu.assimilate_state=DONE and result.outcome=CLIENT_ERROR or result.validate_state!=INIT
     127{{{
     128#!html
     129</td></tr><tr><td><b>validate_state</b></td><td>
     130}}}
     131Defined iff result.outcome=SUCCESS
    91132 * Initially INIT
    92133 * Set by validator to VALID if outcome=SUCCESS and matches canonical result
    93134 * Set by validator to INVALID if outcome=SUCCESS and doesn't match canonical result
    94  * Set by transitioner to NO_CHECK if the WU had an error;     this avoids showing claimed credit as 'pending'.
    95  * Set by validator to ERROR if outcome=SUCCESS and         had a permanent error trying to read an output file,         or an output file had a syntax error.
    96  * Set by validator to INCONCLUSIVE if check_set()         didn't find a consensus in a set of results containing this one.
    97  * Set by scheduler to TOO_LATE if the result was reported         after the canonical result's files were deleted.
    98 
    99 ||
     135 * Set by transitioner to NO_CHECK if the WU had an error; this avoids showing claimed credit as 'pending'.
     136 * Set by validator to ERROR if outcome=SUCCESS and had a permanent error trying to read an output file, or an output file had a syntax error.
     137 * Set by validator to INCONCLUSIVE if check_set() didn't find a consensus in a set of results containing this one.
     138 * Set by scheduler to TOO_LATE if the result was reported after the canonical result's files were deleted.
     139{{{
     140#!html
     141</td></tr></table>
     142}}}
    100143
    101144Result invariants:
    … …  
    105148 * Output files are eventually deleted.
    106149
    107  Notes on deletion of output files:
     150Notes on deletion of output files:
    108151 * Non-canonical results can be deleted as soon as the WU is assimilated.
    109152 * Canonical results can be deleted only when all results have server_state=OVER and all success results are validated.
    110153 * If a result reply arrives after its timeout, the output files can be immediately deleted.
    111154
    112  How do we delete output files that arrive REALLY late? (e.g. uploaded after all results have timed out, and never reported)? Possible answer: let X = create time of oldest unassimilated WU. Any output files created before X can be deleted.
     155How do we delete output files that arrive REALLY late? (e.g. uploaded after all results have timed out, and never reported)? Possible answer: let X = create time of oldest unassimilated WU. Any output files created before X can be deleted.