Name: Anonymous 2020-12-17 11:16
ANUS - Anonymous News Universal Scraper
Public domain software by Anonymous, 2020
ANUS can be used to generate JSON backups of 2ch-style forums, which are:
a board of threads containing comments,
- where a board is a URL and title, and list of threads;
- a thread is a title, creation time, last reply time, comment count,
and list of comments;
- and a comment is a name, e-mail address, creation time, and message.
Given that 2ch-style forums do not generally provide an easy way to provide copies of boards, threads, and comments, nor is there a standard for 2ch-style boards, threads, and comments, ANUS attempts to provide solutions to these problems. Making copies of 2ch-style forums in JSON allows 2ch-style forum enthusiasts to archive discussions which are often ephemeral in nature, as well as feed the data into other kinds of software.
Example ways to use the data produced by ANUS:
Generate RSS feeds of new replies
Tweet / Mastodon new replies
Read forums offline in a desktop client
Restore a backup if site goes offline
Analyze comments
Supported forum software for scraping:
Kareha (4-ch)
Emanon (SAoVQ, Post Office)
Tinychan (/prog/)
This tool will later be used to both archive 2ch-style sites as well as bring their content into the textboard federation. This will simplify navigation of the Western 2ch community, and ensure preservation of its content.
JSON format:
thread:
{ id: {"url", "title", "replies", "last_reply"}}
comment:
{ id: {"name", "email", "time", "message"}}
Public domain software by Anonymous, 2020
ANUS can be used to generate JSON backups of 2ch-style forums, which are:
a board of threads containing comments,
- where a board is a URL and title, and list of threads;
- a thread is a title, creation time, last reply time, comment count,
and list of comments;
- and a comment is a name, e-mail address, creation time, and message.
Given that 2ch-style forums do not generally provide an easy way to provide copies of boards, threads, and comments, nor is there a standard for 2ch-style boards, threads, and comments, ANUS attempts to provide solutions to these problems. Making copies of 2ch-style forums in JSON allows 2ch-style forum enthusiasts to archive discussions which are often ephemeral in nature, as well as feed the data into other kinds of software.
Example ways to use the data produced by ANUS:
Generate RSS feeds of new replies
Tweet / Mastodon new replies
Read forums offline in a desktop client
Restore a backup if site goes offline
Analyze comments
Supported forum software for scraping:
Kareha (4-ch)
Emanon (SAoVQ, Post Office)
Tinychan (/prog/)
This tool will later be used to both archive 2ch-style sites as well as bring their content into the textboard federation. This will simplify navigation of the Western 2ch community, and ensure preservation of its content.
JSON format:
thread:
{ id: {"url", "title", "replies", "last_reply"}}
comment:
{ id: {"name", "email", "time", "message"}}