| | 1 | = Remote job submission = |
| | 2 | |
| | 3 | A group from Universitat Pompeu Fabra |
| | 4 | has developed a system for remote job submission and monitoring. |
| | 5 | This system allows scientists to submit jobs (or groups of jobs) |
| | 6 | from a convenient command-line interface. |
| | 7 | |
| | 8 | The system (Perl-based) is in boinc/rboinc/. |
| | 9 | |
| | 10 | '''Warning: this system has been used only by its developers. |
| | 11 | It will take some work to get it working on other projects.''' |
| | 12 | |
| | 13 | Powerpoint slides describing the system are |
| | 14 | [http://boinc.berkeley.edu/rboinc.pdf here]. |
| | 15 | |
| | 16 | == Notes == |
| | 17 | |
| | 18 | The software should be fairly self-explanatory, but installation may be tricky. Here's a general overview |
| | 19 | |
| | 20 | * boinc_retrieve_server, boinc_submit_server run as cgi. The former, actually, also handles all administrative requests (stop, purge). |
| | 21 | * boinc_retrieve, boinc_submit, are the client components (ditto as above for admin requests) |
| | 22 | * Exchange of files between client and server is done through WEBDAV http extensions (a scratch area needs be setup for this) |
| | 23 | |
| | 24 | * Wus naming is important and enforced like this: NNN-UUU_GGG-XX-YY-RNDzzzz where |
| | 25 | * NN is the name of the workunit (sub-group) |
| | 26 | * UU is the submitter id |
| | 27 | * GGG is the group |
| | 28 | * XX is the current step in the chain |
| | 29 | * YY is the total n. of steps |
| | 30 | * zzzz is a random number (not needed,actually) |
| | 31 | * WUs are kept in a "workflow_directory", a subdir of the project dir, as per slide 22 of the Powerpoint. |
| | 32 | * Inside each dir a "process" bash file is created, which is executed by the assimilator with the name of the assimilated WU as its argument. It will create_work the next step for execution. |
| | 33 | * The main reason for using perl is that I preferred to use the XML::Simple module for (un-) xml-ing data structures over the network - it was useful for adding features on the fly keeping backwards compatibility |
| | 34 | * I implemented basic functions for authentication, but this is not finished yet |
| | 35 | * file storage is optimized through hardlinking and pooling. Network transfers are not (but they could be) |
