Return Styles: Pseud0ch, Terminal, Valhalla, NES, Geocities, Blue Moon. Entire thread

DistBB [PART II]

Name: sage 2013-09-06 2:55

DistBB [PART I] got deleted (probably because my Tor used the same exit node as the spammer). I bear no resentment towards Admin for I understand how utterly shitty this board's software is.

To Admin: could you please not use the ``delete all posts by IP'' button or at least hack up some way to protect posts against such deletion? If you don't mind, I understand you're quite busy.

Here is an update on the moderation system:

SPECIFICATION

A moderation post MUST have the string "!!mod-v1" as its `email' field, and it MUST be digitally signed (see the `pk-data' field). The `body' MUST match the following grammar:

body = tags-line "\n" moderation-line*
tags-line = tag-name (" " tag-name)*
tag-name = (any-character - "\0" - "\n" - " " - "," - "%" - "$" - "#")+
moderation-line = post-id (" " tag-index)+ "\n"
tag-index = nonnegative-integer

`post-id' MUST be the base64-encoded H160 of the post that is being tagged. Each `tag-index' is the zero-based index of the `tag-name' in `tag-line' that should be used to tag `post-id'.

The actual process for deciding whether to keep or delete a post is left up to the implementation.

EXAMPLE

Post "AAAA" is being tagged as "spam" and "worthless", post "BBBB" is being tagged as "spam" and "good", and post "CCCC" is being tagged as "worthless".

--begin post body
spam worthless good
AAAA 0 1
BBBB 0 2
CCCC 1
--end post body

SUGGESTED TAG NAMES

The purpose of the tags is to classify bad posts and annoyances to help people filter and control the content on their nodes. Dividing "A" and "A+" (good post) into "funny", "insightful" and so on is counterproductive since there is no reason why someone would want to configure their client to treat "funny" posts differently from "insightful" ones. The point of tags is also NOT to classify posts into topics.

Here are some suggested tag names along with their meanings:

"A+" : Post is very good, and should be kept at all costs.
"A" : Post is good.
"off" : Post is offtopic (in the case of strict thematic communities) or simply does not fit in, socially. This is very vague.
"spam" : The post is part of a disruptive flood where the contents of posts are computer-generated (e.g. the content is always the same or is chosen from a list).
"mspam" : The post is part of a disruptive flood where the contents of every post seems to be crafted manually by a human.
"prng" : While the post is not part of a computer or human driven flood, its contents are unreadable and it seems to have come out of "/dev/random".

IMPLEMENTATION SUGGESTIONS

Here follows an example of moderation system that can be used to decide posts' fates.

Using a system of rules, each post is assigned an integer rating. The decision of whether to keep or delete a post can be done as follows:

if the post's rating is nonnegative:
keep it
else:
if max(ratings of posts that reference it) >= abs(rating):
keep it
else:
delete it


In table 1 we can see an example of this policy.
highest rating of
referencing posts

\ 3 2 1 0 -1 -2 -3 n/a
\-------------------------
3 | k k k k k k k k
|
2 | k k k k k k k k
|
1 | k k k k k k k k
|
post rating 0 | k k k k k k k k
|
-1 | k k k D D D D D
|
-2 | k k D D D D D D
|
-3 | k D D D D D D D

Table 1. Deciding whether to keep or delete a post
according to its rating and the ratings of
the posts that reference it.


An example of system of rules used to determine posts' ratings could be the following. First see if any rule with the "force" modifier matches; if there is, then apply it directly. Otherwise, if any rule with positive rating matches, then the maximum matching rule wins; otherwise, the minimum matching rule wins. For example, if the rules that match a post have ratings {-3,-5,-6}, then the winning rating is -6. However, if the rules that match have ratings {1,2,-3,-5,-6}, then the winning rating is 2.

By default, posts are deleted after two weeks, in order to give the user a chance to review the deletion queue; "immediate" indicates that the posts should be deleted immediately.

--begin example
%trusted self bob ken
%acquaintances john uriel
%all %trusted-users %acquaintances
%annoying off mspam
%delete spam prng
%kill-list bertrand winston
%A A A+

pk self 10
tag self %A 10
tag self %delete -11,immediate,force
tag self %annoying -5,hide

pk %kill-list -5,immediate

pk %trusted 10

tag %trusted %A 5
tag %trusted %delete -5,immediate
tag %trusted %annoying -2,hide

tag %acquaintances %A 3
tag %acquaintances %delete -3
tag %acquaintances %annoying -1,hide

tag john,ken particular-annoyance -4,hide,immediate
tag %all some-annoyance-which-I-find-amusing 10
--end example

Name: Anonymous 2013-09-09 1:00

>>40
Honestly, where is the specification of the synchronizing protocol?
I can't find it on:
http://tablecat.ipyo.heliohost.org/lounge/read/1373068543/
git://repo.or.cz/prog.git
or
https://ivasiwlrjq5dxk6b.onion/p/distbb

We can use ncurses to build the UI with. We can even reverse engineer SyncTerm.

For NNTP, we just make the body of the message the commands, the subject the authentication/session (captcha'd with a message to PostServ@bbs.progrider.org). Heck, making the NNTP session, they can browse to session.token and use the new message as the captcha field without email.

IRC is for FAR in the future, and it is meant as an option, not a requirement.

Name: >>41 2013-09-09 1:10

LOL, there's been more repos:
http://repo.or.cz/w/4chanprog.git

I need time to sift through these. Right now I am looking for a good VPS host (free?), to test something in Scheme and Haskell. Plus, its past my bed time again. Someone should organize his notes, with links an stuff.

Name: polite bump 2013-09-09 1:19

>>41
Honestly, where is the specification of the synchronizing protocol?
https://ivasiwlrjq5dxk6b.onion/p/distbb/finfo?name=doc/protocol.txt

For NNTP, we just make the body of the message the commands, the subject the authentication/session (captcha'd with a message to PostServ@bbs.progrider.org). Heck, making the NNTP session, they can browse to session.token and use the new message as the captcha field without email.
I'm not sure what you're trying to do here. Are you trying to make a public NNTP gateway? If yes, then moderation stuff as well as anything that requires pk stuff won't work (unless either people send you their private keys or have special support for this in their NNTP clients). The best a public user can do is make an unsigned post, or hand over to you their private key so you can construct a signed post for them, or use a command line tool to manually and painstakingly construct a signed post, and then add it as an attachment or something.

Name: Anonymous 2013-09-09 9:46

>>41
Oh shit, I think I see what's going on. You can't click on that hash thing to see the file since you're not logged in. The URLs are disabled by default to prevent stupid spiders from trying to access every possible link (of which there are a lot).

Just click "log in" and log in as "anonymous" with the text CAPTCHA it gives you.

As a more durable solution, run

$ socat tcp4-listen:8000,bind=127.0.0.1,fork socks4a:127.0.0.1:ivasiwlrjq5dxk6b.onion:443,socksport=9050 &
$ fossil clone https://127.0.0.1:8000/p/distbb distbb.fossil


to clone the repository. Then of course do

$ mkdir distbb; cd distbb; fossil open ../distbb.fossil

to actually open the repo in a directory.

Name: the distbb guy 2013-09-10 13:18

I'm at a conundrum. Should the server itself support several clients (e.g. multiple non-privileged users and one administrator/moderator), or should each user simply run their own server?

Pros for multiple client support:
- Multiple users can use the same server directly without requiring extra post storage. Their preferences (ignore list, PK petname mapping) as well as their private keys stay private.
- Modular design.

Cons for multiple client support:
- Extra logic. The client program is separate.
- Extra complexity; this necessitates yet another protocol for client-server communication; the sync protocol is NOT sufficient as it does not support querying for "latest bumped threads" or "get all posts in this thread".

Name: the distbb guy 2013-09-10 13:30

Also, would one of you happen to have a copy of "Mathematics for Physicists" by Susan Lea? I've been looking all over the Internet, and all I found was a solutions manual and a torrent that gets stuck at 68%.

Name: Anonymous 2013-09-10 13:39

>>46
I do, hold on.

Name: >>47 2013-09-10 13:40

>>46
Nope, never mind. Different author.

Name: Anonymous 2013-09-10 17:20

>>45
I'd go for the unique server/multiple clients model. I'd run the server, but I'm sure there's at least one paranoid/lazy/schizophrenic ``faggot'' who will refuse to run it.

Name: Anonymous 2013-09-10 17:27

>>46
Nothing on libgen.net, nothing on ebookee, nothing on Google. Sorry.

Name: Anonymous 2013-09-10 20:11

>>47-48,50
Thank you for your efforts. <3

Nothing on libgen.net, nothing on ebookee, nothing on Google. Sorry.
I know, I really looked all over. But it's okay, I got a friend to lend me his dead tree copy for the semester.

>>49
I'd run the server, but I'm sure there's at least one paranoid/lazy/schizophrenic ``faggot'' who will refuse to run it.
This is a distributed board. Wouldn't a paranoid/schizophrenic ``faggot'' rather prefer to just run their own copy of the server (over some anonymization service such as Tor/I2P, of course) instead of depending on someone else? A lazy ``faggot'' might indeed opt not to run the server, however.

I've been thinking about the client/server model. The list of possible useful queries/features is hard to enumerate; while the simplest and most useful ones are obvious ("list recently bumped threads" or "list posts by thread"), the other ones might not be (e.g. "get threads in which posts match keywords" or "list posts by public key during a certain timeframe"), and extra complexity yields more places to attack/DoS. But then if someone really wants to do advanced queries they might as well just run a full server.

I don't know. I'm not even sure why someone would not just run a full server themself (disk space constraints?).

Note that this is all about the implementation I am writing, and that anyone else is free to reimplement stuff on their own or to modify my code however they want; da standards are public.

In the absence of an explicit client/server model, the various ``clients'' would access the sqlite database directly. That will not stop you from writing an IRC gateway or NNTP or public HTTP interface to it.

I might be overthinking this.

Name: Anonymous 2013-09-10 21:09

>>51
It's okay, making DistBBs is provably NP-hard.

Name: Anonymous 2013-09-10 21:42

>>45
What difference would it make? Each server, being distributed, should connect to many other servers, correct? If a node only want to act a client, it should simply disallow peers to connect to it, right? It won't propagate messages for the network, only receive them and send them to the servers it is peered with (of course, it could also run a HTTP or NNTP gateway or something). Am I misunderstanding something?

Name: polite bump 2013-09-10 22:10

>>53
Am I misunderstanding something?
Yes, and it's definitely my fault for misusing confusing terminology. Let's define some nice names.

Postserver: the program associated with the post database which synchronizes against other postservers. Its only other task is to use the moderation configuration set by the administrator to prune posts.

Reader: a program which uses the post database, possibly indirectly, letting the user (or administrator) access it, make new posts, moderate/tag posts, set PK-petname mappings, set ignore lists, add bookmarks, and so on.

The question in my post (>>45) is whether the Reader should be sufficiently decoupled from the Postserver (and thus from the post database) so that multiple unprivileged Readers can use the same post database (without security risks).

Without this decoupling, the post database (in particular, the sqlite3 database file) would also contain the administrator's preferences (e.g. PK-petname mappings, ignore lists, bookmarks, etc).

Name: >>54 2013-09-11 18:28

So, your thoughts on this? Should I stop making things complicated and just stick everything (including the administrator's preferences) in the main database since everyone will be running their own postserver anyway?

Name: Anonymous 2013-09-11 19:58

>>55
Everyone should use the same database IMO. I don't think having (locally) your own customized Redd- I mean, /prog/ is a good idea.

Name: Anonymous 2013-09-11 20:01

>>56
Which part of Distributed Bulletin Board do you not understand?

Name: Anonymous 2013-09-11 20:05

And by that I mean "decouple the reader and the postserver and keep a single post database".

Name: polite bump 2013-09-11 20:36

>>58
The point of decoupling the reader and the postserver is to allow multiple readers with different customizations and configurations (e.g. PK petnames, bookmarks, etc) efficient and direct access to the local postserver. Otherwise you can just use a web interface or an IRC gateway or a NNTP gateway or whatever one of you might come up with.

Name: Anonymous 2013-09-11 21:01

shame on you >>56-kun, you might want to read da distbb standard

Oops, I'm >>56 and I think I made the most retarded post in history. Sorry guys.

Name: Anonymous 2013-09-12 22:11

How's the project doing, DistBB-san?

Name: Anonymous 2013-09-13 5:39

>>61
Slow because my professors decided to give us assignments all at the same time. I'll be able to do more work on the weekend.

Also I'm not sure I've settled the reader/postserver decoupling dilemma, though I'm leaning towards the simpler ``just chuck it all in the same database'' which would indeed cover the most common case in which the administrator is also the sole ``power'' user.

Also, according to server logs, nobody has checked out (in both senses of the word) my repo to review the protocol and the various specs.

Name: Anonymous 2013-09-13 12:06

>>62
Because it's on Tor. Can't be bothered. I never have Tor running.

Name: Anonymous 2013-09-13 12:44

Yo Admin, could you mirror my repo for >>63-lazy?

Name: Anonymous 2013-09-13 20:19

>>64
Your request will be fulfilled sometime within the next 48 hours.
Along with other things. I've been busy this week.

Name: Anonymous 2013-09-13 20:25

>>65
It's okay, take your time.

Name: Anonymous 2013-09-13 21:31

>>62
What kind of degree are you getting?

Name: Anonymous 2013-09-13 22:27

>>67
Mathematics.

Name: Anonymous 2013-09-14 13:57

What's the point of the DistBB again? Don't get me wrong, the implementation looks fun, but other than the distributed moderation (which is kind of like ``customized hellbanning''), I don't see why we can't conform with a centralized textboard in Scheme.

Please convince me to regain interest in this project again.

Name: polite bump 2013-09-14 19:14

>>69
- Uniform representation of posts. No more imprecise scraping. All clients can also act as servers.
- Tripcodes are replaced by public keys (which have very little overhead thanks to the magic of DJB's Ed25519).
- Centralized textboard means centralized moderation. The moderator has to act sometimes, to remove things like spam or questionably legal content (esp. if they live in a shitty country). It is also psychologically difficult for a moderator not to turn into a gigantic ``faggot''. (The Admin of this board seems to be doing fine so far.) If and when that happens, the community is torn between staying and moving elsewhere.
- Don't put all your eggs in the same basket. If the administrator/moderator gets hit by a truck (or by less lethal real life circumstances) and can no longer host the board (or the post db gets corrupted/deleted and the admin was too dumb to make regular backups), the community must relocate in a rush, at which point they are once again at risk of ````faggot'' moderator syndrome''.
- Some communities (not /prog/) rely on active moderation, and they are most at risk of ````faggot'' moderator syndrome''. This should help them as well.

Name: polite sage 2013-09-14 19:18

>>69
The ``customized hellbanning'' is also very flexible; while one person may want to not see FV's or nikita's or mentishit's posts, they may happen to find tdavis' posts very entertaining.

Name: Anonymous 2013-09-14 20:30

>>70
Okay, it's clearer now. I wish you the best.

Name: Anonymous 2013-09-17 21:49

Polite bump and not-so-subtle request for development status.

Name: Anonymous 2013-09-17 21:50

>>73 Just go there yourself and commit, you can get all the updates you need, and actually work on the documentation, which needs to be polished.

Name: the distbb guy 2013-09-18 7:01

>>73
University work just kicked in, yo. I haven't worked much on it, but (interestingly) things have settled in my mind as to the design. Maybe my subconscious mind was working on it during sleep.

There is one component I still need help with. I need a simple CAPTCHA system for one-shot posting over Tor or anonymization services over the public HTTP gateway. This is critically important because it's the way you can announce your node so that other people pull posts from you (by default, the policy is to not pull posts during sync if you can't identify the peer you are communicating with).

Name: Anonymous 2013-09-18 14:28

>>75
So like in OTR or FISH (cipher), where a common key/password is made for the room or DHT, and if they don't have it, from solving the captcha, they only see is garbage.

Name: Anonymous 2013-09-18 14:37

>>76
What? No, like I said, the CAPTCHA thing is just to prevent people posting over anonymization networks from massively spamming. Instead, other people's postservers connect to you and pull the posts from you; this means that if you send them lots of spam, they'll be able to filter and delete your posts by source.

Name: Anonymous 2013-09-18 20:49

>>77
Ah so a post session. My bad.

Name: the distbb guy 2013-09-22 6:34

Keepalive post, not bumping.

The profs have ganged up on me with assignments again. I should be able to work on DistBB towards the middle of the week, in that short gap between submitting an assignment and getting a new one.

Name: the distbb guy 2013-09-24 20:46

What the fuck, they just gave us another assignment while the last one is still pending. fuck fuck fuck

Newer Posts
Don't change these.
Name: Email:
Entire Thread Thread List