Explicit file management

BOINC allows you to explicitly manage files on client hosts. This might be used, e.g., to implement a high-latency distributed file system.

Retrieving file lists

To instruct a host to send a list of all persistent files, use the function

request_file_list(int host_id)

or the command line program (run from the project root directory)

request_file_list [ -host_id X ]

If -host_id is absent, get file lists for all hosts. A message is created for the specific host (or all hosts) and added to the msg_to_host table in the database. The upload message included in the next RPC reply to the host.

The file list will be included in the next scheduler RPC request. You must modify the scheduler to parse and store it.

Retrieving files

A persistent file can be retrieved from a specific host. This can be done using the function

get_file(int host_id, const char* file_name)

or the command line program (run in the project root dir)

get_file -host_id X -file_name Y

This program must be run in the project's root directory. get_file() creates a result with a name of the form:

get_FILENAME_HOSTID_TIME

Example: get_test.mpg_34_123456789 is a result representing the upload of test.mpg from host number 34 at time 1234567891.

An upload message is created for the specific host and added to the msg_to_host table in the database. This message instructs the client to upload the file and acknowledge a successful upload. The upload message included in the next RPC reply to the host. The message has the form:

<app>
    <name>FILE_MOVER</name>
</app>
<app_version>
    <app_name>FILE_MOVER</app_name>
    <version_num>BOINC_MAJOR_VERSION</version_num>
</app_version>
<file_info>
    <name>file_name</name>
    <url>upload_dir</url>
    <max_nbytes>1e10</max_nbytes>
    <upload_when_present/>
</file_info>
    RESULT_XML
<workunit>
    <name>result.name</name>
    <app_name>FILE_MOVER</app_name>
</workunit>

Include <msg_to_host/> in config.xml. Currently <ignore_upload_certificates/> needs to be included as there is no way to send upload certificates with these files.

Sending files

To send a file to a specific host, use the function

send_file(int host_id, const char* file_name)

or the command line program (run from the project root directory)

send_file -host_id X -file_name Y

send_file creates a result and initializes it with a name of the form send_FILENAME_HOSTID_TIME.

A message is created for the host and added to the msg_to_host table. This message instructs the client to download the file and acknowledge a successful download. The download message included in the next RPC reply to the host. The message has the form:

<app>
	<name>FILE_MOVER</name>
</app>
<app_version>
	<app_name>FILE_MOVER</app_name>
	<version_num>n</version_num>
</app_version>
<result>
    <wu_name>x</wu_name>
    <name>y</name>
</result>
<file_info>
	<name>file_name</name>
	<url>download_dir/file_name</url>
	<md5_cksum>md5</md5_cksum>
	<nbytes>file->nbytes</nbytes>
	<sticky/>
</file_info>
<workunit>
	<name>result.name</name>
	<app_name>FILE_MOVER</app_name>
	<file_ref>
		<file_name>file_name</file_name>
	</file_ref>
</workunit>

For this to work, you need to include <msg_to_host/> in config.xml.

Deleting files

To delete a file from a host, use the function

delete_file(int host_id, const char* file_name)

or the command line program (run from the project root dir)

delete_file -host_id X -file_name Y

delete_file() creates a message for the specific host and adds it to the msg_to_host table. This message instructs the client to delete the file. The message is included in the next scheduler reply to the host. The message XML has the form

<delete_file_info>file_name</delete_file_info>

For this to work, you need to include <msg_to_host/> in config.xml.