Example 2: replication

This example uses the same type of jobs as example 1, but does replication to increase the accuracy of results. Every job is sent to at least two different volunteers.

We'll use the following replication policy:

• A job is considered finished if either
  • Two volunteers reported "no ellipse found";
  • Two volunteers found ellipses, and their centers are within 20 pixels;
• If ten instances have been performed and neither condition holds, the job is marked as "inconclusive" and no further instances are issued.

We'll use the following job distribution policy:

• Initially all jobs have the default priority (1)
• When we send the first instance of a job we set its priority to 2, so that we'll get another instance soon.
• When a 2nd or subsequent instance is issued we set the priority to 0.
• When an instance times out, or finished but without consensus, we set the priority to 2.

Setup

Create an application named bossa_example2. Create some jobs:

php bossa_example_make_jobs.php --app_name bossa_example2 --dir example

Callback functions

The file html/inc/bossa_example2.inc contains the application's callback functions:

    30  function job_issued($job, $inst, $user) {
    31      $insts = $job->get_instances();
    32      if (count($insts) == 1) {
    33          $job->set_priority(2);
    34      } else {
    35          $job->set_priority(0);
    36      }
    37  }

When an instance is finished, we check whether there are now two consistent responses. If so we mark the job as DONE. If there are 10 finished instances, we mark it as INCONCLUSIVE. Otherwise we set its priority to 2, so that we'll get another instance.

    39  function job_finished($job, $inst) {
    40      // record the user's response
    41      //
    42      $response = null;
    43      if (get_str('submit', true)) {
    44          $response->have_ellipse = 0;
    45      } else {
    46          $response->have_ellipse = 1;
    47          $response->cx = get_int('pic_x');
    48          $response->cy = get_int('pic_y');
    49      }
    50      $inst->set_opaque_data($response);
    51
    52      // see whether we have a consensus
    53      //
    54      $insts = $job->get_finished_instances();
    55      $n = count($insts);
    56
    57      $results = null;
    58      foreach ($insts as $inst) {
    59          $results[] = $inst->get_opaque_data();
    60      }
    61      for ($i=0; $i<$n-1; $i++) {
    62          $r1 = $results[$i];
    63          for ($j=$i+1; $j<$n; $j++) {
    64              $r2 = $results[$j];
    65              if (compatible($r1, $r2)) {
    66                  $job->set_state(BOSSA_JOB_DONE);
    67                  return;
    68              }
    69          }
    70      }
    71
    72      // no consensus - see whether we've reached replication limit
    73      //
    74      if ($n >= 10) {
    75          $job->set_state(BOSSA_JOB_INCONCLUSIVE);
    76          return;
    77      }
    78
    79      // still looking for consensus - allow another instance to be issued
    80      //
    81      $job->set_priority(2);
    82  }
    83
    84  // two results are compatible if neither found an ellipse,
    85  // or they both did and centers are within 20 pixels
    86  //
    87  function compatible($r1, $r2) {
    88      if ($r1->have_ellipse) {
    89          if ($r2->have_ellipse) {
    90              $dx = ($r1->cx - $r2->cx);
    91              $dy = ($r1->cy - $r2->cy);
    92              $dsq = $dx*$dx + $dy*$dy;
    93              return ($dsq < 400);
    94          } else return false;
    95      } else {
    96          return !$r2->have_ellipse;
    97      }
    98  }