VM Apps
Notes taken by Ben Segal
Present
Uwe Beckert / Rechenkraft.net, Oliver Bock / MPI, Ignasi Buch / GRIB, Oscar Diaz / PUCP, Joseph Emeras / INRIA, David Garcia Quintas / CERN, Daniel Lombrana Gonzalez / Univ. Extremadura, Attila Marosi / SZTAKI, Nico Mittenzwey / Rechenkraft.net, Jarno Rantala / TUT, Michael Weber / Rechenkraft.net, and myself (12 in all).
(Others copied in this email are followers of the BOINC-VM discussions which began at last year's BOINC Workshop in Grenoble).
Introduction
Ben introduced the discussion by referring to work done on this topic over the last 3 years, most of which is documented on the BOINC Wiki at: http://boinc.berkeley.edu/trac/wiki/VmApps
The intention is to build a solid infrastructure in BOINC to support applications running in guest VM's hosted by a BOINC client, as compatibly as possible with applications running in the client itself. Our intention is to coordinate work as far as possible in this developing area to avoid needless duplication of effort.
In particular, two new pieces of work have been done at CERN this year and were reported at this BOINC Workshop (the presentations by David Garcia Quintas and Jarno Rantala). Their slides will be available at: http://boinc.berkeley.edu/trac/wiki/WorkShop09.
For additional information on these two projects see:
- VM Controller software (by David Garcia Quintas) at: http://boinc.berkeley.edu/trac/wiki/VirtualBox This is a general-purpose communication layer permitting host processes to control a hypervisor and exchange messages and files with it and all its guest VM's via an XML-RPC interface. It supports an open-ended set of hypervisors, with VirtualBox the first to be fully implemented.
- VM Wrapper program (by Jarno Rantala) at: http://boinc.berkeley.edu/trac/wiki/VmApps This is a rewritten BOINC Wrapper, back-compatible with the original Wrapper, but which launches and controls applications inside VM's if supplied with a suitably modified job.xml file. No changes are needed to the BOINC core client or server software.
Work To Do
The above software has already run an impressive demonstration at CERN, with a Linux BOINC client running jobs for a real physics experiment. It requires some more work to turn it into a rugged system supporting all major BOINC client platforms out on the Internet.
Specific "To Do" lists of work remaining will be posted on the BOINC Wiki shortly.
Proposed collaborations
Several members of the BOINC community have offered their collaboration to test and extend the above approach:
- Daniel Lombrana (U. Extremadura): To test with Windows clients and extend it to support the VMWare hypervisor.
- INRIA (Derrick Kondo and Joseph Emeras): To test with MacOSX clients and extend support to kQEMU.
- Attila Marosi (SZTAKI): To explore its use and development for a PhD thesis project.
- Jeremy Cowles (UC Berkeley): To help with automatic BOINC deployment of the Python components needed.
- CERN (Ben Segal and others): To use the system in conjunction with the CernVM image manager to run problems in High Energy Physics. (This system makes BOINC appear as a "Volunteer Cloud" and is described in a recent paper at:
( Information on the CernVM project can be found at: http://cernvm.web.cern.ch/cernvm/ )
Related areas of interest
During the BOF discussion, several areas of activity of interest for VM applications within BOINC were mentioned:
1) Image generation and management
- Attila Marosi (SZTAKI) and Oliver Bock (MPI) have experience with VM image size reduction and management using overlays.
- A colleague of Attila at SZTAKI (Gabor Kecskemeti) was active in a project to generate VM images, independent of CernVM.
- Joseph Emeras (INRIA) expressed interest in image generation using CernVM.
2) GPU's and VM's
- Oliver Bock (MPI) proposed a "standard VM image" to ease porting problems with GPU drivers on miscellaneous platforms.
3) Checkpointing
- Michael Weber and Nico Mittenzwey (Rechenkraft.net) proposed using VM's as a way to provide checkpointing for certain BOINC applications which could not be checkpointed by conventional means.
BOINC support for VM applications
Although our approach minimizes required changes to the BOINC infrastructure, there was a short discussion during the BOF report to the full meeting, concerning the need for hypervisor installation on BOINC client nodes before they could be used for VM applications. David Anderson said that "at the top of his list" of developments was a fix whereby an alert would be given to a volunteer attaching to a BOINC-VM project, requesting the installation of the required hypervisor before continuing.
Thanks to all the BOF attendees as well as to all those interested in working in development of VM support for BOINC.
