Deploying a BOINC server on the Amazon Elastic Computing Cloud

What is Cloud Computing?

Cloud Computing is a form of computation and storage that hides the complexity of hardware and software management from a user. Typically, cloud computing services allow you to rent computing or storage services. These services charge on a pay-per-use basis (roughly, about 0.10 per CPU hour or per GB/month). Cloud computing services such as Amazon's ​Elastic Compute Cloud (EC2) also allow one to tailor a computing environment through the use of virtual machines. In addition, Amazon offers a ​Simple Storage Service (S3) for enabling remote storage.

Why use a Cloud?

Hosting a BOINC server on a cloud is useful for the following reasons:

• It's cheaper for small to medium projects to rent time on a cloud versus paying for hardware, bandwidth, and electric power on one's own.
• It's easier and faster to use an existing OS image with the BOINC server already installed than to compile and configure it on one's own.

For an in-depth cost-benefit analysis, please read: ​Cost-Benefit Analysis of Cloud Computing versus Desktop Grids. Derrick Kondo, Bahman Javadi, Paul Malecot, Franck Cappello and David Anderson. To appear in 18th International Heterogeneity in Computing Workshop, May 25 2009, Rome.

To calculate the cost-benefits for your own application, you can use this ​Excel spreadsheet.

How to use a Cloud for a BOINC server?

We have created a custom virtual machine with a BOINC server installed and configured. This includes all server software prerequisites and the correct permissions and accounts already setup. All you need to do is to deploy this virtual machine image on the cloud and your server should be up and running. This guide describes how to deploy this Amazon Machine Image (AMI) over Amazon's EC2 cloud. This can be done in minutes. Testing this image deployement will cost you less then 0.50 USD so give it a try.

Prerequisites

​Create and setup an account with Amazon's EC2.

Deploying the BOINC Server Image

Start the instance (ami-2290774b) with the BOINC server (stable 6.3.14 JAN 28 2009) installed on Debian Etch:

ec2-run-instances ami-2290774b -k [key-pair]

Show the instance deployment status:

ec2-describe-instances 

After the instance status goes from pending to running (takes a few minutes), you should see the hostname of the form ec2-XXX-XXX-XXX-XXX.z-2.compute-1.amazonaws.com.

If you haven't already, open ports 22 and 80:

ec2-authorize default -p 22 
ec2-authorize default -p 80

At this point, you should be able to access the web server at the URL: http://ec2-XXX-XXX-XXX-XXX.z-2.compute-1.amazonaws.com/

Then login to the instance:

ssh -i [private_key corresponding to key-pair]  root@ec2-XXX-XXX-XXX-XXX.z-2.compute-1.amazonaws.com

As root,

su boincadm
cd

In the home directory is a README.txt file with important information.

Setting Up a Test Project

Here we describe how to create a test project using the uppercase application.

Create a test project uppercase:

cd ~/boinc/tools
./make_project --url_base http://ec2-XXX-XXX-XXX-XXX.z-2.compute-1.amazonaws.com --db_passwd boincadmpw --test_app cplan

Answer yes to all questions.

cd ~/projects/cplan
./bin/xadd
./bin/update_versions
./bin/start

To check that all daemons are running:

./bin/status

Now any BOINC client should be able to attach to the project and download workunits. For example, go to the project page (​http://ec2-XXX-XXX-XXX-XXX.z-2.compute-1.amazonaws.com/cplan) and create a user account.

Then with a Linux command-line client in ~boincadm/BOINC, run

cd ~/BOINC
./boinc_client  -no_gui_rpc -attach_project http://ec2-XXX-XXX-XXX-XXX.z-2.compute-1.amazonaws.com/cplan [account_key]

At this point, the client should attached to the project and begin downloading work.

Technical Notes

Below is not required reading, but we describe how and why the image was built.

BOINC Configuration

You'll find in the ~boincadm directory :

• BOINC: directory with client for testing purposes
• projects: directory where BOINC projects are installed
• boinc_server_stable_r16906_6.3.14: directory with server source code

Why Xen?

The general approach was to build a Debian EC2 AMI from a Xen image. (The alternative approach was to use an existing Debian AMI.) However, we chose the Xen approach primarily to be able to install and test things without having to worry about EC2 hourly costs. As a side-effect, one can use the Xen image either as a standalone server or over EC2 (as it's Xen based).

These links describe the setup involved: ​Debian from Xen Images, ​Bundle Debian AMI using Ubuntu

If you would like to use the Xen image directly, you can download the Xen image ​disk.img.tgz (1.5GB) and ​swap.img.tgz (.5MB), and config file ​boinc_debian_etch.cfg

and go from there.

The Xen image above was configured to use a bridged Ethernet connection and DHCP.

Hostname Configuration

The hostname can change each time an instance is started. As such, the start-up script /etc/rc.local was modified to query Amazon for the hostname and to set it at each boot.

IP Address

The IP address could change each time the instance is created. To associate a fixed address (regardless of the instance), one should use ​Amazon's Elastic IP Addresses.

In addition, one can configure the image to use dynamic DNS to associate a domain name with the instance. See the following links. ​Dynamic DNS with ZoneEdit ​Dynamic DNS Setup

SSH

SSH server host keys are (re)generated each time the instance is booted in /etc/rc.local. This is to help prevent man-in-the-middle-attacks.

rc.local is configured to randomly change the root password at each boot, and to download your ssh public key and cat it to the authorized_keys file to allow remote passwordless access.

To verify the identity of the host before logging in, you can run ec2-get-console-output to get the key fingerprint written to the console during the bootup process. The key fingerprints will look something like:

Restarting OpenBSD Secure Shell server: sshd.
------- KEY FINGERPRINTS -------
2048 28:f1:0c:51:95:82:c6:a1:a2:58:a3:37:7f:39:ef:f9 /etc/ssh/ssh_host_rsa_key.pub
1024 90:f2:98:4b:e6:c5:c7:7a:23:6e:ef:67:d0:be:72:a9 /etc/ssh/ssh_host_dsa_key.pub
--------------------------------

during bootup. Then you can compare this key fingerprint with the one reported by ssh when you log in to verify the host identity.

To do

• Consider setting up dynamic DNS by default in the bundle

Questions?

Email ​Derrick Kondo (derrick.kondo :: inria fr)