Step 1:

WATCH
We Are The Cloud Hosts

Goal: to offer a system for secure distributed object storage and retrieval, with public key based access control.

Allow users to create their own data storage clouds, and provide a standard interface for accessing information from these clouds, permitting third parties to access individual objects from the cloud according to a per-object access control list (which uses public keys as identities).

High level design:

1. A WATCH Cloud (WC) would be identified by a username and a domain, such as username@domain.com.

2. A WC would be composed of one or more WATCH Nodes (WN). WN provide some combination of possible services:
                A) List of other WN which may participate in this cloud.

        B) Object storage/retrieval (blobs of binary data with MIME type) -- these may be (and often are) encrypted.

                C) Encryption key storage/retrieval for objects. Note that key retrieval being separate from object retrieval means that untrusted hosts can store data with relative safety.

                D) Object name searching/listing (as access control permits)

                E) Connection bouncing services: If a WN is inaccessible due to NAT or other network addressing issues, a WN can optionally provide a connection between two WN (or a WN and a client) in a WC.

                F) A WN can (if desired by the WC administrator) choose to cache objects.

                G) Object transcoding services (this of course would require the server to have access to the data it would be transcoding). This could include photo resizing, audio bitrate changes, etc. Typically an infrastructure WN would not support this, but a home server that indexes photos would, etc.

        3. To gain access to a WC, a client (or server which will be acting as a client for this description) firsts get a list of domain-hosting WN, either via DNS (tbd; watch-xx.domain.com, perhaps?) or HTTP (tbd; http://www.domain.com/watch-query/ or similar?). Once a connection is made to a domain-hosting WN, the client provides its public key and queries a list of WN which provide WC services for username@domain.com’s cloud).

        4. All connections between WN and WN or clients should be over TCP, with SSL optional, however all transactions should be designed to use the public key identifies to protect object names, data, and encryption keys such that SSL is not required (to the extent that the objects in question are private -- unencrypted public objects should also be supported).

Make the following software (tbd: LGPL vs GPL vs BSD)

1) Develop a common C++ core library that implements the protocol and basic functionality

2) Develop a command line WN server for posix/win32/OSX, which can be used for domain-hosting WNs and/or content-providing WNs. This server would be designed to be as flexible as possible:

A) allow filesystem paths to be configured for content sharing, with regex translations to object paths.

B) allow automatic idle replication of objects/keys to other WNs according to rules (desired copies, quotas, etc).

C) bandwidth controls/schedules

3) Develop a command line WC client (store/retrieve objects).
4) Develop a graphical application for win32/OSX which can function as both a WN (server) and a client. TBD: identity management, initial key exchange, etc. Public key directories?

5) Provide compatibility interfaces for both ends, such as s3cmd compatibility for accessing objects from a WC, as well as allowing a WN server to use S3 as a backing store for its objects.

6) Keep external dependencies at a minimum for all of the above.

Step 2:

The applications (to add to the graphical WATCH application).

1) Integrate media viewer (photos, audio, video, etc. tbd: html).

2) Integrate IM/chat functionality

3) Integrate “newsfeeds”, which would be chronological lists of interesting objects (matching against the access control list for the requesting user). WN would be able to have special logic to filter this type of data by access control, etc.

Step 3: ????

Step 4: profit