Changes between Version 5 and Version 6 of AssimilateIntro

Timestamp:

Sep 17, 2009, 4:04:35 PM (15 years ago)

Author:

davea

Comment:

--

AssimilateIntro

@@ -1 @@
-= Result assimilation =
+= Handling completed jobs =
 
-Projects must create one assimilator program per application. This is done by linking the program '''sched/assimilator.cpp''' with an application-specific function of the form
+Completed jobs are handled by programs called '''assimilators'''.
+These are generally application-specific:
+they might copy output files from the BOINC upload directory to a permanent location,
+or they might parse the output files and insert results into a database.
+
+== Creating an assimilator ==
+
+To create an assimilator, link the program '''sched/assimilator.cpp''' with a function of the form
 {{{
 int assimilate_handler(
@@ -8 +15 @@
 }}}
 
-This is called when either
- * The workunit has a nonzero [JobIn#Redundancyandschedulingattributes error mask] (indicating, for example, too many error results). In this case the handler might write a message to a log or send an email to the application developer.
- * The workunit has a canonical result. In this case wu.canonical_resultid will be nonzero, canonical_result will contain the canonical result. Your handler might, for example, parse the canonical result's output file and write its contents to a separate database.
+This function is called when either
+ * The workunit has a nonzero [JobIn#Redundancyandschedulingattributes error mask] (indicating, for example, too many error results). In this case the handler might write a message to a log or send an email to the project administrator.
+ * The workunit has a canonical result. In this case wu.canonical_resultid will be nonzero, and canonical_result will contain the canonical result.
 
-In both cases the 'results' vector will be populated with all the workunit's results (including unsuccessful and unsent ones). All files (both input and output) will generally be on disk.
+In both cases the 'results' vector will be populated with all the workunit's results (including unsuccessful and unsent ones).
+All files (both input and output) will generally be on disk.
 
 It's possible that both conditions might hold.
 
 The return values of `assimilate_handler()` are:
- * 0: the workunit will be marked as assimilated.
- * DEFER_ASSIMILATION: the workunit will not be marked as assimilated, and will be processed again when the next instance finishes.  This is useful for appliations where you want to see all the completed results.
- * Other nonzero values: the assimilator will log an error message and exit. Typically the function should return nonzero for any error condition. This way the system administrator can fix the problem before any completed or erroneous workunits are mis-handled by BOINC.
+ * 0: success: the workunit will be marked as assimilated.
+ * DEFER_ASSIMILATION: the workunit will be processed again when another instance finishes.  This is useful for appliations where you want to see all the completed results.
+ * Other nonzero values: the assimilator will log an error message and exit. Typically '''assimilate_handler()''' should return nonzero for any error condition. This way the system administrator can fix the problem before any completed or erroneous workunits are mis-handled by BOINC.
 
 You can use BOINC's [BackendUtilities back-end utility functions] to get file pathnames and open files.
+
+== Running assimilators ==
+
+Run assimilators as BOINC daemons: that is, add an entry
+{{{
+<daemon>
+   <cmd> my_assimilator -app APPNAME </cmd>
+</daemon>
+}}}
+to your project's [ProjectDaemons configuration file].
+
+Assimilators have the followiong command-line options:
+
+ -app name :: the application name
+
+ [ -mod N R ] :: process only jobs with mod(ID, N) == R
+ 
+ [ -d N ] :: set verbosity level (1 = least, 3 = most)
+
+ [ -dont_update_db ] :: don't mark jobs as assimilated (for testing)
+ 
+== The sample assimilator ==
+
+BOINC includes a sample assimilator, '''sample_assimilator'''.
+It does the following:
+
+ * For successful workunits, it writes the canonical instance's output files to the directory '''PROJECT/sample_results/'''.  If there is only one output file it is named WU_NAME.  If there are more than one they are named WU_NAME_0, WU_NAME_1, etc.  If there are no output files, an empty file WU_NAME_no_output_files is created.
+ * If the workunit failed (to many errors, etc.) it appends a line to '''sample_results/errors''' containing the workunit name and the error code.
+
+The sample assimilator can be used as a placeholder while you
+are developing your application.
+In some cases you may be able to use it in production.