Client data model

This document describes proposed changes to the client so that it can handle distributed storage.

Current

generated_locally upload_when_present uploaded sticky urls

Problems

Proposed

FILE_INFO elements:

upload_urls download_urls

<file_info> elements in scheduler replies:

• if referenced from an app version or workunit, set download_if_not_present

• if referenced from a result, and no_upload is not present, set upload_when_present.

If we get an upuload request and the file isn't there, return error

Edge case: an input file that may have been generated by a previous job. If not, this job should generate it. Include such a file as an output, not an input. Input files have the property that if they're not present, we download them.