/PROG/ SUMMER OF CODE

Greetings /prog/ !!

Some of you might be looking for FUN SUMMER PROJECTS[i]!![/i] Here is something you could try to program: let's build a global decentralized persistent "functional" (immutable) data store which works by exchanging messages. These messages could be serialized/stored/communicated however you want (we can always write adapters to have your programs communicate with eachother). But they need to satisfy following semantics:

__________________________________________________________________________________
Two basic messages are supported:

* define a new type of entity
* create a new entity

ENTITY TYPE
* Loosely corresponds to a table
* Has a name
* Has a number of (typed) fields
* Has a number of constrainst for entities.

ENTITY
* Loosely corresponds to a row in a table
* Has an entity type
* Is a mapping of field to value
* Satisfies the constraints from the entity type.

TYPE
* Fields in entity types have a TYPE
* You can not define extra types (although one of the types could be "a reference to an entity ABC which satisfies constraints DEF)
* Types will probably be things like: a unicode string, an ascii string, a number, a list of TYPE, a mapping of TYPE to TYPE, ...

CONSTRAINT
* A given entity satisfies a constraint at the time of creation.
* A constraint should be "positive" (not sure how to describe this). You could have a constraint that says "an entity of type X needs to exist with title = X" but you can NOT have a constraint that says "no entity with type X and property XYZ must exist". A server should be able to reject invalid entities without having access to all entities.
* Examples of constraints:
- list contains elem
- value bigger than value
- signature is valid for public key and data
- ...

(fucking syntax highlighting)
__________________________________________________________________________________

You'll need to design a way to serialize and transport these messages over existing protocols (think: email, IRC, plaintext pipes, HTML5 JSON APIs, soaps and so on). I think we might need a bit more work on the "types" and "constraints" part (perhaps we can steal from some existing system? any academics wanna chime in here?), but then we're good to go.

so itt: how do we represent types & constraints + pledges to implement the system along with what protocol you'll layer over. I'll do a RACKET implementation with a POSTGRES backend and an EMAIL transport.

feel free to bikeshed this to death, but don't let other people stop you from writing history!!

what are the applications of this?

>>1

immutable data store which works by exchanging messages

if it's immutable how can anything get done?

>>3
Detecting imperative idiots is easy.

>>2
the possibilities are ENDLESS

you could easily create a chan on top of the system; just define a "chan" entity type, a "board" entity type, "post" and a "reply". if you want to namefag, define a "profile" entity type which contains your public key and a "signature" type which holds the signatures.

(of course there should be some kind of way to query a server for various things but since it's expected that users run their own servers to connect to the mesh they can sort that out themselves)

>>4
You typed that message imperatively

>>6
But it's immutable, just like yours.

>>7
you mutated the board's database

This is a good idea until you consider what guarantees you're giving to your applications. What are the serialization characteristics? Is there transactionality? What about transaction replay?

This'd have to be some kind of a forking blockchain-type of affair. Distributed systems have come a long way since pre-buttcoin, but fucked if all of the fresh new knowledge isn't tainted by the cybercoin cult.

Are we resuscitating DistBB?

>>8
You should mutate yourself into not being a fucking uneducated hobo:

http://caitiem.com/2013/07/21/immutable-data-azure-table/

Immutability is the property of data to not change once it is created. Most storage systems provide basic C.R.U.D. (Create, Read, Update, Delete) functionality. However, to have truly Immutable data we should only use the Create and Read functionality provided by storage systems.

Nathan Marz makes an excellent case for why we need Immutable data in his talk at Strata Conf 2013, Human Fault – Tolerance. The basic premise is people make mistakes. We are fallible beings: we write code with bugs, we accidentally delete data from production, etc… He goes on to posit that the worst kinds of errors are those which cause Data Loss or Data Corruption, because it is impossible to recover from them. By having Immutable Data stores we minimize the impact bugs or mistakes can have on our systems.

https://orchestrate.io/blog/2014/02/13/immutability-and-why-it-matters/

In the past few years, immutability has entered the conversation in databases and data storage. Pat Helland of Salesforce gives a great talk on how “Immutability Changes Everything“. In HDFS, once a file is closed, it cannot be modified. For Datomic, a database designed by Rich Hickey, the creator of Clojure, immutability is a core concept. Datomic is a time-oriented database, wherein the entire database is treated as a time-ordered series of immutable objects. This simplifies reasoning about the state of data as all queries are performed at a particular point-in-time, and at any point-in-time, the database is consistent and unchanging.

>>11
Maybe you should be a bit more clear. An immutable database cannot be changed, but a database with immutable data could be useful.

>>3

if it's immutable how can anything get done?

brilliant ha ha ha

>>11

side note: we recently rolled out purge deletes which run counter to immutability, but allow users to ensure that data has been permanently removed from Orchestrate

So immutability is great except when it's not?

Right, here's why it's immutable. Each entity/entity type represents a concept, an idea. These are, by default, as in the real world, immutable. You can derive new ideas from old ideas, and a server can choose which ideas to retain, but once you said it it's out there: you cannot take it back.

This is also why there is a constraint on the constraints. Why you cannot have a "negative" constraint like "no record exists with characteristics XYZ". Because the server cannot be expected to hold all dreams ever dreamt.

>>9
The serialization characteristics are protocol dependent. You could serialize to JSON/XML/BINARY/POSTGRES/... whatever. There is no need for transactions, this is not a database used to store numbers. This is a repository of ideas.

/prog/ nuclear winter of code

>>14
$X is great except when it's not.