Superhost Design and Topology

T(DesignDocument)?

The superhost idea has been proposed numerous times, but has not yet been implemented. This document discusses concrete design ideas.

Problem

Large organisations wishing to run BOINC often don't want to open up their entire network to BOINC project servers, or allow unmanaged BOINC clients running on the network, potentially attached to unauthorised projects.

Solution

Promote one of the hosts to "superhost". All other hosts will only communicate with the superhost, and only the superhost requires Internet access.

Additional Benefits

• Internet traffic is reduced, since the superhost can communicate more efficiently with the project servers, and only needs to download a single copy of any file.
• Hosts can be controlled more tightly, since the superhost will have more information than is normally available to project schedulers.
• The superhost can manage a single work queue, improving turn around times.
• Normal work unit copy rules can be ignored, allowing a single work unit download to be run on multiple computers within the cluster. (See security section.)
• Validation can be delegated to the superhost, reducing uploads and server load. (See security section.)
• A company can run internal BOINC projects, and fall back on an external project if no work is available from the internal projects.

Topology

Security

For an untrusted cluster, the project must treat the superhost exactly the same as a single host. However, if there is a trust relationship between the project and the superhost, then the superhost can be treated as the sum of its hosts, with the capability to manage work unit redundancy on the project's behalf. Validation offers some technical challenges, but simple bitwise validation using homogeneous redundancy can be achieved easily.

Implementation

This design requires major server and client changes, in addition to the superhost application.

• The scheduler RPC schema will need extending to cope with more complex requests.
• The scheduler will have to treat such requests differently.
• The client will have to attach to a superhost (account manager RPCs may be useful here).

A simplified design could start by treating the superhost as a project from the point of view of the clients, and as a client from the point of view of the projects. Discussion on boinc_dev resulted in a consensus that this approach is impractical, since there would be so many special cases. This leaves designing the superhost as an end-to-end system.

SZTAKI have a hierarchical BOINC implementation. Source should be available soon.

Other Solutions

BOINC on JXTA by Marcin Cieślak (Thesis from Technical University of Wroclaw, Poland).

Abstract

Distributed computing systems allow acquiring resources that lie outside the capabilities of even the largest supercomputers. Indisputable example is prosperity of SETI@Home project which gathered at its peak ca. 850,000 users. Its success is partially related to well designed architecture – BOINC. Many projects work on this architecture, using all together the computing power of ca. 400 TeraFLOPs. This is more than the most hi-tech supercomputer. But even BOINC architecture struggles with scalability concerns. The creators may not have predicted the degree of projects’ popularity. The purpose of this work is to find new solutions for volunteer computing. This paper contains overview of existing systems which elements might be useful for this goal Next it describes BOINC parts along with propositions of changes which can benefit its effectiveness. In the second part has been shown how the popular JXTA platform can be utilized to completely redesign the existing structure. A complete project of new architecture has been introduced which has the same functionality as BOINC, but it overcomes its limitations and the resources are utilized better. This work also involves partial implementation and points out the directions of future development.