An organizational grid management system for BOINC

NOTE: This document is historical. OGM has moved ​here.

Introduction

OGM (TODO: pick a better name) is a system for grid computing on organizational resources, using BOINC.

OGM is designed for situations where some entities that own computers (Suppliers) have decided to let another entities (the Allocator) decide how their computer time is to be divided among a set of BOINC projects. With OGM, Suppliers are able to control the usage of their computers (for example, the hours during which it does BOINC computation). However, they have no control over which BOINC projects their computers contribute to; the Allocator makes that decision.

The BOINC projects to which computing power is given need not be related to either Suppliers or Allocator; they might be public projects like Climateprediction.net or Rosetta@home.

OGM is also designed for situations where the Suppliers' computers are in a public place, or are used by people not trusted by the Supplier; the users have no control over BOINC on the computer.

For example, Suppliers might be different departments in a university, each of which owns a set of desktop and laboratory PCs, and the Allocator might be a campus-wide committee that divides the resources among BOINC projects internal to the university. Or the Suppliers might be different companies, who have agreed to volunteer their PC resources to a philanthropic organization that divides them among public BOINC projects.

OGM is implemented using BOINC's Account Manager mechanism. The Allocator runs the OGM software on a server. The Suppliers run the BOINC client on their computers, and attach each client to the Allocator's account manager. The BOINC client on the computer periodically communicates with the OGM, which instructs it which projects to attach to, and the resource share for each attachment.

OGM entities

OGM defines the following entities:

• Supplier
• Pool: a set of computers belonging to the same supplier. Preferences can be set independently for different pools.
• Computer: a computer attached to the OGM; each one belongs to a particular Pool.
• User: an account on the OGM (usually corresponding to a person).
• Project: a BOINC project to which computing power is being supplied by the OGM.

OGM Users

OGM defines the following User roles:

1. Supplier Admin: this user can:
   • See the statistics of computers in his Pools
   • Change the BOINC preferences for his pools (e.g., time of day limits, disk/RAM usage, etc.).
2. Allocator Admin: this user can:
   • Create, edit, delete projects.
   • Set the resource share parameter for each Project.
3. Root: this user can do any task:
   • Create, delete, edit OGM users.
   • Do the tasks of a Supplier Admin
   • Do the tasks of an Allocator Admin

Resource allocation

In the current design, each of the OGM's Projects has a single "resource share" parameter. Every Computer is attached to all Projects, using these resource shares.

These resources shares may be dynamically adjusted by Allocator Admins. For example, if a Project has a short-term need for lots of computing, its resource share could be temporarily elevated, and later reduced to compensate.

In a future version of OGM, it would be possible to extend this model so that different resources shares are used on different Pools or Suppliers, for example so that Projects whose applications require particular hardware (e.g., large RAM) would be preferentially run on computers meeting those requirements.

Accounts and credit

Pools are defined by OGM accounts; the computers in a pool are attached to the same account.

The OGM has a single account on each Project, all with the same email address (i.e., there is not a separate account for each Supplier or Pool). This means that the OGM appears as a single user on each Project's "top users" list, and as a single user on aggregate statistics sites.

Thus, in order to allow Supplier Admins to view credit statistics for their Pools,

the OGM must obtain per-host credit statistics from all the Projects using RPCs; it can then group these however it wants.

Implementation

OGM has two main components: the server and the BOINC client. The tool employs the Account Manager protocol for all the communications.

host

• id int(11) KEY
• poolid int(11)
• supplier varchar(255)
• CPID varchar(255)
• host_cpid varchar(255)
• venue varchar(254)
• p_ncpus int(11)
• p_vendor varchar(255)
• p_model varchar(255)
• p_fpops float
• p_iops float
• os_name varchar(255)
• os_version varchar(255)

pool

• id int(11) KEY
• name varchar(255)
• supplier varchar(255)
• start_hour varchar(5)
• end_hour varchar(5)
• suspend_while_on_batteries tinyint(1)
• suspend_if_user_active tinyint(1)
• idle_time_to_run int(11)
• suspend_if_no_recent_input
• leave_apps_in_memory tinyint(1)
• cpu_scheduling_period_minutes int(11)
• max_cpus int(11)
• max_ncpus_pct int(11)
• cpu_usage_limit int(11)
• disk_max_used_gb float
• disk_min_free_gb float
• disk_max_used_pct int(11)
• disk_interval int(11)
• vm_max_used_pct int(11)
• ram_max_used_busy_pct int(11)
• ram_max_used_idle_pct int(11)
• work_buf_min_days int(11)
• work_buf_additional_days float
• confirm_before_connecting tinyint(1)
• hangup_if_dialed tinyint(1)
• max_bytes_sec_down float
• max_bytes_sec_up float
• net_start_hour varchar(5)
• net_end_hour varchar(5)
• dont_verify_images tinyint(1)

project

• id int(11) KEY
• name varchar(255)
• url varchar(255)
• signature varchar(1000)
• authenticator varchar(255)
• share tinyint(4)
• detach tinyint(1)
• update tinyint(1)

stats_host

• date timestamp
• id int(11)
• host_cpid varchar(255)
• supplier varchar(255)
• project varchar(255)
• total_credit float
• expavg_credit float
• expavg_time float

stats_user

• date timestamp
• project varchar(255)
• total_credit float
• expavg_credit float
• expavg_time float

user

• id int(11) KEY
• name varchar(255)
• password varchar(32)
• role varchar(255)
• supplier varchar(255)
• supp_auth varchar(255)