/prog/ challenge - Compressing Visual Novels

What makes VN companies come up with proprietary image/video/audio formats that compress worse than standard ones? Here's an example:

Proprietary format, original: 871KB
Proprietary format, recompressed[1]: 822KB
PNG: 676KB
Optimised PNG: 674KB
JPG (high): 221KB
JPG (medium): 125KB
JPG (low, artifacts start becoming obvious): 86.7KB

So they could be several times smaller while looking almost the same, and that's not even getting into things like difference encoding since the bulk of them are almost the same images with slight differences.

Here's the challenge: find an existing VN, and write a little utility to compress it to 1/2th its original size or less while maintaining all the content. You have until the end of the year.

[1] I RE'd the format and wrote a compressor for it, which performed better than the original while decompressing to the exact same data.

>>40
Either from mommy and daddy or from our taxes.

>>39
Higurashi and Umineko took four years each to make. Hardly a quick cash grab. Higurashi is still getting milked too. It will get a VR spin off somehow if Ryukishi07 is still alive by then. Umineko would make a good murder mystery game too.

There's also Fate. I don't know how that would work in VR though.

>>39
I don't really give a shit about VNs but that's a pretty dumb argument because most of game genres won't make sense in VR. shmups, strategy games or turn-based RPGs won't be jumping to VR either because why would they? it only makes sense for games that are fully 3D an played from a first-person perspective

>>33 nice dubs.

>>43
VR main genres will be Harem, Romance and Hentai
Probably along with "Create your 3D waifu" helper program

>>45
VR ANDRU!

Total images original size 1.05GB
After replacing duplicates with "symlinks" 534MB

This is ridiculously low-hanging fruit. It's not even hanging, it's sitting on the ground. A 5-minute postprocessor could save your disk space by up to 50%!

Next strategy I'm going to try: differencing successive images. There are many sequences of images which are basically the same except for some changes in things like facial expression. It's not even at the complexity of an H.264 encoder (or perhaps that would be an エッチ.264 encoder...)

Any bets on the resulting filesize?

>>32
At least Cudder can write a FizzBuzz app.

I decided to just throw them into ffmpeg's H.264 encoder at default settings to see what it could do, and... 32MB!!! HOLY FUCKING SHIT!!! THIRTY TWO TIMES SMALLER!!!

The resulting file even plays in any media player, since it's just a video (slideshow). Every frame looks identical to the original images.

1GB+ to 32MB. I knew they were bloated, but didn't think it would be that big of a difference. H.264 is amazing.

>>49

H.264 is amazing

and proprietary

>>49 Try HEVC/H.265

Check dubs

>>47,49
What VN is this?

>>50
The spec is freely available: https://www.itu.int/rec/T-REC-H.264-201602-I/en

>>53
http://www.anime-sharing.com/forum/hentai-cg-packs-24/%5Bng010%5D-%E5%85%AC%E7%B8%81%E5%B0%91%E5%A5%B3%EF%BD%9E%E6%A1%9C%E6%81%8B%E7%AF%87%EF%BD%9E-537160/

It was the first one in the list when I visited.

Check em

Check em

>>54
Just because the spec is visible doesn't mean that it's also open. H264 a proprietary spec that will cause liability in any public activity in the countries that enforce H264 patents.

Anyone tried DLI format https://sites.google.com/site/dlimagecomp/

Just the other day there was a libJPEG arbitrary code execution exploit released and here you are asking why no sensible developer wants to use open standard trash. A better thing to do would be just use DirectX texture files for everything, since it's developed and maintained by real programmers good enough to be paid, not college dropout failures who work for exposure and are terrified of showers.

>>59
because proprietary software has no exploits against it :^)

>>60
"With enough eyes all anii are haxxable" is dumb. There are more eyes watching commercial software because that's where the money is. Vulnerabilities that can be found usually are on the realm of static analysis these days. The tiny flaws don't get noticed in open source any mute than they do closed.

>>61
you know how I know you never worked in security for a major corporation? because you wouldn't have written this shit. here's the truth:
1. proprietary software uses open source libraries a lot - sometimes directly, occasionally developing their own solutions based on FOSS stuff if the license is too restrictive
2. corporate code monkeys are fucking horrible at their jobs
3. most vulnerabilities (as in individual errors in code that allow exploitation, not whole new classes of vulnerabilities) are found by fuzzing because finding vulns statistically gets exponentially harder with program size because programs are trees (this goes for both manual and automated tests)
4. because in corporate world everything is tied to a budget, bugs may go unfixed if the cost (of fixing) vs benefit (of preventing exploitation) doesn't look good. but because not all hackers have the same skillset, what looks like a tricky to fix DoS for a pentester might be a code execution for a russian black hat

>>62
statically, not statistically

>>63
staticly, nost statically

>>64
sticly, not staticly

>>65
dubs, check em

>>57
what if i implemented it slightly different?

>>57

in the countries that enforce H264 patents.

HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAAHAHAHAHAHAHAHAHA

I have 99 problems but an H.264 license ain't one.

...and chances are everyone already has paid someone for a H.264 license.

>>68
being proud of living in a 3rd world shithole

justcudderthings

C-dder is all talk and no action.

>>67
I don't know what specific change you'd do, but I would guess that you'd probably produce something "inferior" or it would be incompatible with the spec. In the interest of promoting open standards, I advocate encoding video using the completely open VP9 codec.

Check em

cudder? are you the dev of anonix or something from a cople of yes ago?

>>73
It was a couple of no ago, actually.

CheCk 'eM

dubs check

>>72,75-76
4 real this time.

>>77

77

Nice dubs.

22

Nice dubs.

>>73
I started Anonix/Anoncoreutils. The suckless guys took over from there a few years later, in fact I'm pretty sure the very first versions of sbase were based on Anoncoreutils.

>>71
VP9 would probably be only slightly inferior to H.264, but still make things two dozen times smaller.

Next thing to investigate: does Japanese compress better or worse than English text?

Not much can fit in 64K, but how about a 64MB procedurally generated VN...?

"Cudder" is not the same Cudder that started the (terrible) Anon* projects a few years back. I thought everyone knew this.