Changes between Version 3 and Version 4 of PortalFeatures

Timestamp:

Oct 1, 2010, 2:42:44 PM (14 years ago)

Author:

davea

Comment:

--

PortalFeatures

@@ -32 +32 @@
  * A scientist's share is increased if they contribute to the portal,
    e.g. by participating in the message boards.
+
+The set of users may be large (1000s) and dynamic.
+Typically, only a few may have jobs at any given point.
 
 What are the precise semantics of computing share?
@@ -116 +119 @@
 and a button for aborting the batch.
 
-== Job DAGs ==
+=== Job DAGs ===
 
 Batches are a special case of Directed Acyclic Graphs (DAGs) of jobs.
@@ -139 +142 @@
 As an extension of the work described above,
 we could support DAGs rather than just batches.
+
+== Combining multiple scheduling types ==
+
+BOINC potentially supports the following types of work,
+each with its own scheduling mechanism:
+
+ * Job-stream: the input is a stream of jobs;
+   the goal is to maximize throughput and eventually finish all jobs.
+ * Batch: described above.
+ * Locality: a variant of job-stream in which
+   jobs are assigned according to the files resident on the client.
+ * Co-scheduling: like batch, except all jobs in each batch
+   must run simultaneously (e.g., MPI applications).
+
+A single project (such as a portal) may have work of some or all types.
+
+How should we handle multiple scheduling types -
+in particular, how can we maintain resource shares in their presence?
+
+One simple approach is to maintain, for each scheduling type T,
+the maximum debt D(T) of any user with an unsent job of type T.
+Then use this top-level policy:
+{{{
+for each scheduling type T in order of descending D(T)
+        send jobs of type T
+}}}
+How to maintain D(T):
+
+ * Job-stream: maintain in the feeder (consider only jobs in the cache)
+ * Locality: constraint: only 1 user can use locality scheduling
+ * Batch: maintain in batch module
+ * Co-scheduling: maintain in co-scheduling module
+
+== Maintaining debts ==
+
+A user's debt is adjusted each time a job is dispatched,
+based on the estimated FLOPS of the job.
+This quantity is recorded in the job record.
+
+When a job finishes or times out
+(at which point the actual computing done is known)
+the user debt is adjusted accordingly.