Any OS legal way to traverse an exec memory pool?

[Karlos]

Let's assume you are using CreatePool() / DeletePool() to manage memory allocation. If you wanted to traverse the allocations made so far so that you could, for example write to a debug log, is there any OS legal way to access the memory list?

The problem is that, according to the includes&autodocs, CreatePool() returns void* and DeletePool() takes void*, which tends to suggest the underlying implementation is completely private and I shouldn't assume anything about what is actually pointed to.

I can implement my own tracker easily enough if there is no sanctioned way of getting the information but that would seem like reinventing the wheel when it's clear a memory pool already tracks the allocations made within it.

[Karlos]

Any allocator you write is going to have to ask the OS for memory at some point. You can allocate large blocks from the OS infrequently and manage smaller allocations yourself from within those. In theory, the memory pool system was supposed to do away with the need for this.

[Karlos]

But... How can I be sure you are the real Piru? :-?

:lol: Actually, scrub that, only the real Piru would be this terse:

No.

As it happens, I am wrapping the pool and I have a tiny header for each allocation (I never use it for trivial allocation of a few bytes anyway), so as far as I can see there's no reason I couldn't add basic prev / next node pointers in there.

[Karlos]

So, it looks like I have to do my own tracking.

In the allocation system, so far, every allocation has a (private) 16-byte header that immediately precedes the allocated data. There may be some waste data before the header due to cache alignment code:

Code: [Select]


struct MemAllocation {
  uint32 identifier;      // used to identify this as an allocation from our system
  uint32 allocationSize;
  uint32 allocationFlags;
  void*  realAddress;     // the real base address that was returned from AllocPooled()
};



The allocation routine aligns the allocation to a cacheline boundary, so this structure is cache aligned, as is the space following it.

The identifier is a unique ID that allows the corresponding free method to ascertain if it is legal to release the memory pointed to. Basically if the uint32 value 16 bytes before the returned address (having done the obvious null checks first) matches this id, it is a block you can free. Of course, the free routine zeroes out this field so that if you ever did try to free it twice, the second free will fail gracefully.

I guess I can extend this and add MemAllocation* prev and MemAllocation* next, but that leaves a further 8 bytes to fill with useful data or rejig the alignment code. I suppose a 32-64 bytes overhead (inclusive of alignment padding) per allocation isn't so much these days.

Besides, I could always conditinally compile them in/out once the entire thing is sufficiently debugged.

[Karlos]

There's no way I'd consider patching the OS just to implement a feature for my own libraries. I think simply extending my header structure to include a pair of nodes and using the remaining bytes prior to the allocation base address as a wall (and some more afterwards) that can be removed once debugging is complete would be more than sufficient.

Having said that, I haven't had an out of bounds pointer bug in anything I've written for years ;-)