Another reason 64 bit is shit

A bit of background is in order. GHC allocates memory from the operating system in units of aligned megabytes. So it owns a collection of megabytes that might be anywhere in the address space. For every pointer it sees, the GC asks the question “is this a pointer into the heap?”, and the memory manager should return true for any of the memory that GHC has allocated, and false otherwise. This is the job of the HEAP_ALLOCED() macro.

On a 32-bit machine, answering the question is easy: a megabyte is 20 bits, so we only need a 12-bit lookup table, and even using a full byte for each entry, that’s just 4KB.

On a 64-bit machine, it’s much harder. Even taking advantage of the fact that only 48 bits are available address space on x86_64 architecture machines, that still leaves 28 bits, which is a 256MB table (32MB using bits instead of bytes). This is not likely to be an acceptable memory overhead.

https://simonmar.github.io/posts/2015-07-28-optimising-garbage-collection-overhead-in-sigma.html

>>18

If you "reserve" something, anyone sane will implicitly understand that you intend to use it in the future. How does attempting to reserve more memory than is available both in RAM and the pagefile combined on a typical system even make sense?

Spoken like a right apper. I solemnly pledge to read your name as "see-udder" henceforth.

There was already mentioned a clear and simple usecase: a memory manager that needs to maintain a certain structure of the program's address space.

The right way to do it: reserve a petabyte of memory for each arena you might need, then commit memory as it is actually allocated.

The wrong way to do it: have the OS refuse to reserve that much memory because it knows better, then try to determine how much memory you can reserve experimentally, with binary search. And you must know how many arenas you'll need in advance.

What makes it especially retarded is that the entire point of the overcommit hack is to allow management of your own address space (that shouldn't concern the OS in the slightest) separate from allocating virtual memory, which is exactly what the reserve/commit distinction does cleanly.