Another reason 64 bit is shit

A bit of background is in order. GHC allocates memory from the operating system in units of aligned megabytes. So it owns a collection of megabytes that might be anywhere in the address space. For every pointer it sees, the GC asks the question “is this a pointer into the heap?”, and the memory manager should return true for any of the memory that GHC has allocated, and false otherwise. This is the job of the HEAP_ALLOCED() macro.

On a 32-bit machine, answering the question is easy: a megabyte is 20 bits, so we only need a 12-bit lookup table, and even using a full byte for each entry, that’s just 4KB.

On a 64-bit machine, it’s much harder. Even taking advantage of the fact that only 48 bits are available address space on x86_64 architecture machines, that still leaves 28 bits, which is a 256MB table (32MB using bits instead of bytes). This is not likely to be an acceptable memory overhead.

https://simonmar.github.io/posts/2015-07-28-optimising-garbage-collection-overhead-in-sigma.html

>>39

If you reserve memory, the system rightly expects you to use it, and since memory is finite, there is less memory available for others to reserve.

Are you really that stupid or are you pretending?

Again, there are two, count 'em Hans, TWO different things you want to do with memory: first is managing your own process's address space layout, second is mapping some of your address space to actually usable memory.

The whole overcommit bullshit came into being because UNIX only provides one way to ask for memory (except for that whateverbrk thing, it's not different), but people realized that the first need is real too, so they made a hack: you can malloc whatever you want to get a neat contiguous range of memory reserved, and only when you actually access some part of it that part is committed as a mapping to the OS-managed virtual memory.

One of the reasons Windows is faster than Linux or other Unices is that they learned from their mistakes and gave you two separate tools, one for managing your address space, another for asking the OS to actually allocate some memory in your address space.

Now the funniest thing, Si Udder, is that you don't know how to call VirtualAlloc:

int main(int argc, char ** argv)
{
SIZE_T size = 2000i64 * 1000 * 1000 * 1000;
auto mem = VirtualAlloc(nullptr, size, MEM_RESERVE, PAGE_READWRITE);
printf("%Iu %p\n", size, mem);
return 0;
}

Totes allocates 2 petabytes of address space on my Windows 7. But not four for some reason, idk why, it doesn't seem to like any kind of obvious limit (40-bit real addressing should prevent even 1 petabyte, 48 should accommodate 4 petabytes, probs some Windows thing).