>>24I am sorry. I often speak from my own context without establishing it. Uhmmm, so most of web 2.0 involves presenting information to users in a structured way that is easy to navigate. Users are capable of submitting various kinds of content to the site. But the site is always centralized. So I would like to make a framework of some sort, that would take care of the p2p stuff for such a distributed site. I am taking a lot of inspiration from the gnunet slides. Particularly where that one guy listed the tradoffs of a distributed service, and that his area of research is to make distributed software easier. So I'd like to make a general purpose framework that would be used to run a typical web 2.0 site.
There are two components, the framework, and the application.
Duties of the framework:
* manage the dht of peers that act as hosts for the service
** peers may go offline and their addresses may change.
* maintain a database of content items. From the point of view of the framework, a content item is a binary blob with a set of key values pairs that describe meta data associated to the blob. The framework may also store its own meta information in content items, in particular the time stamp of when the content item was imported into the peer's local database, and possibly the peer it was synced from. Each meta data field may be used as an index for search.
* sync new content items from the other known peers. Content may be filtered using meta data tags when it is synced. For example, if the peers maintained a bbs, the application may request the framework to sync only the threads from /prog/ and /lounge/, while ignoring unsynced threads from /newpol/. The framework will store the fetched content items and alert the application to their presence. The framework maintains steady communication with the other peers and awaits new content.
* Accept new content from the application. The application may be a local browser run directly by the user. The application could also be a http server, that presents users with a web interface to browse the local database of content. Users of the web application may submit data to the http server, which then wraps the data as an appropriate content item with meta data and passes it on to the framework, whom then handles propagation across the peers.
* Maintain a change log, which is a log of all changes to content sorted by time stamp. Other peers may request to sync all changes between a specified revision or time range. Other peers may request to only sync content within a time interval that has meta information matching a given set of filters.
* Cull content that is requested to be deleted by the original author of the content. Identity of the author would be authenticated somehow, perhaps by a cryptographic method. When content is submitted, the author could generate a random string, and a hash of it and attach the hash as meta information, DELETE_HASH. Then when the delete request is submitted to a peer, the delete request contains the original string generated by the author. The peer verifies the string matches the hash, deletes the content from local storage (if the framework is polite), alerts the application to the deleted content, and echos the delete request to the other peers if necessary. The delete request will also appear in the change log. The key of the deleted content item may be retained to give an appropriate error message if the application or a peer requests the deleted content item. Delete requests may also be received from the application.
* Meta data modification. Peers or the application may modify meta data values or add new key value pairs to existing content items. These changes are to be propagated to other peers.
* A moderation system. Moderators with identities may flag content items, contributing values to a moderation tag. Each content item contains a list of moderator designations, which contain a rating, categorization, comment, identity of the moderator and signature of the moderator.
* Remove only from local storage. If requested by the application or a local configuration utility, the framework may remove certain content items from local storage, but not indicate their deletion on the network. If the content is needed in the future, it can hopefully be fetched from a peer that still stores the content item. If all peers remove the same content item from local storage during the same time, the content is lost.
* accept peer configuration from the application or configuration utility. The moderation system may be extended to peers to identify malicious peers. Hopefully a white list of polite peers may be gathered and become large enough to ensure stable access to content items.
* When syncing, complicated queries may be provided for reading content filtered by meta data tags from other peers. When pushing content content items to the other peers, simply the presence of the new content item is communicated. Write access to peers is limited to introduction of new content items and delete requests (which may only be done by the original author).
* support meta data marking of content items that specify white and black lists for peers that are allowed to receive the content item.
Ok and now the roles of the application and how it may interact with the framework. A typical application would be a local browser or a http server.
* The application will treat the framework as a distributed database. Complex read-only queries may be executed, searching for content items based upon meta data tags. Searches may be performed locally and on peers (suitable for devices with the capability to run the framework and an application, but lack the storage to keep a local copy of the content items). In the typical example of a http server, the application may listen for GET requests, parse the uri, look up the relevant content items from the framework using either local or a peer's storage, and then assemble the fetched content items into a web page for the internet user.
* The application will create new content items with meta data and hand them to the framework. The framework will manage propagation of the content items.
* The application may perform a delete request on the framework. The framework shall then verify the delete request, perform the deletion locally, and propagate the delete request to the other peers.
* The application may write changes to the local storage managed by the framework. The framework will maintain a change list of affected content items.