Changes between Version 5 and Version 6 of BossaOverview

Timestamp:

Jul 20, 2008, 2:51:14 PM (16 years ago)

Author:

davea

Comment:

--

BossaOverview

@@ -18 +18 @@
 == The diverse requirements of distributed thinking ==
 
+Distributed thinking projects may have widely varying
+properties and requirements.
+
+First, they may require different orders of job distribution.
+Consider the two cases:
+
+ * Project A has a limited set of jobs and lots of volunteers.
+ * Project B has an unbounded stream of jobs and a limited set of volunteers.
+
+Project A will want to have each job performed about the same number of times.
+It will want to issues all jobs once,
+then issue them all a second time, and so on.
+In constrast, project B will want to issue the first job to a quorum of volunteers,
+then issue the second job to a quorum, and so on.
+
+Second, projects may want to assess the ability of each volunteer,
+and use this to determine how many replicas of each job to perform.
+Several factors might contribute to the ability estimate:
+
+ * The volunteer's performance on a training course.
+ * The volunteer's performance on a stream of "calibration jobs" (with known answers) intermixed with the job stream.
+ * The fraction of time a volunteer's response to a job agrees with the "correct" response, as determined by replication.
+
+In addition, the way in which ability is described may vary.
+In simple cases it might be a single number, e.g. and error rate.
+For tasks that involve feature detection we might want to track
+the rates of false positives and false negatives separately.
+For more complex tasks there could be arbitrarily many dimensions.
+
+Third, the way in which more able volunteers are used may differ.
+Two general possibilities:
+
+ '''Experts do the same job, only better'''::
+  For example, experts might be used to resolve cases in which no concensus is reached by other volunteers. Or they might be used to verify the identification of rare features.
+
+ '''Experts do more sophisticated jobs'''::
+ For example, non-experts might look for features, while experts classify them.
 
 == How Bossa addresses these requirements ==
+
+Bossa's design philosophy is to provide '''mechanisms''',
+and leave it up to the projects to define the '''policies'''.
+This is done using the following techniques:
+
+ * Project-specific information about jobs and volunteers is stored in PHP objects.  The structure of these objects is determined by the project.  Bossa stores them in a serialized form in the database.
+ * Project-specific policies are encoded in PHP '''callback functions''' that are invoked at various points (e.g., when jobs are issued, when they complete, and when they time out).
+
+The mechanisms provided by Bossa include:
+
+ * Jobs can be designated as '''calibration jobs''', and Bossa can be instructed to randomly mix a given rate of calibration jobs into the job stream.
+ * The order in which jobs are issued is determined by a floating-point priority.  The manipulation of these priorities is up to the project.
+ * Users can be assigned integer '''levels''', in which case each job has a vector of priorities, one per level.
+
+This framework enables all of the examples listed above
+(and others not yet conceived)
+to be implemented with a small amount of PHP programming.