Skip to contentSkip to conversation

SLAB - have index_of bug at compile time.

PATCH Manfred Spraul 3 participants 4 messages Dec 26, 2005

lore
Share & tools
lore
#0 lore raw
Quoted 3 lines · Steven

So I'm going through ever line of code and examining it thoroughly, when I find something that could be improved

Try to find a kfree implementation that doesn't need virt_to_page(). This is a big restriction of the current implementation: It's in the hot path, but the operation is only fast on simple systems, not on numa systems without a big memory map. And it makes it impossible to use vmalloc memory as the basis for slabs, or to use slab for io memory.

--
Manfred

#1 lore raw
Quoted 5 lines · Manfred Spraul
Quoted 3 lines · Steven

So I'm going through ever line of code and examining it thoroughly, when I find something that could be improved

Quoted 7 lines

Try to find a kfree implementation that doesn't need virt_to_page(). This is a big restriction of the current implementation: It's in the hot path, but the operation is only fast on simple systems, not on numa systems without a big memory map.
And it makes it impossible to use vmalloc memory as the basis for slabs, or to use slab for io memory.

Unfortunately, that virt_to_page helps make kmalloc and kfree more efficient. Since it allows the use of the mem_map pages that map to the used memory to be used for storing information about the slab.

So removing virt_to_page means you need to store the information relative to the memory that is mapped. Thus you need to allocate more than is needed.

Your question refers to only kfree, which would not really be that difficult to remove the virt_to_page, since that would just need the extra memory _for each allocation_. But then you mention vmalloc and numa, where it is the generic use of virt_to_page through out slab.c that is the issue. I counted 14 direct uses of virt_to_page in slab.c (2.6.15-rc5). Now you need to find a way to store the information of the off slab descriptors and for slabs that are more than one page.

Changing the use of virt_to_page would probably hurt those that can use it, and the changes would not be accepted because of that. Unless you can keep the same speed and memory efficiency of those "simple systems".

Now, maybe NUMA and vmalloc might be a good reason to start a new allocation system along side of slab?

-- Steve

#2 lore raw

Hi Steven,

Quoted 2 lines · Steven Rostedt

Now, maybe NUMA and vmalloc might be a good reason to start a new allocation system along side of slab?

A better approach would probably be to introduce a vmem layer similar to what Solaris has to solve I/O memory and vmalloc issue. What NUMA issue are you referring to btw? I don't see any problem with the current design (in fact, I stole it for my magazine allocator too). It's just that the current implementation is bit hard to understand.

Pekka

#3 lore raw
Quoted 10 lines · Pekka Enberg

Hi Steven,

Quoted 4 lines · Steven Rostedt

Now, maybe NUMA and vmalloc might be a good reason to start a new allocation system along side of slab?

Quoted 8 lines

A better approach would probably be to introduce a vmem layer similar to what Solaris has to solve I/O memory and vmalloc issue. What NUMA issue are you referring to btw? I don't see any problem with the current design (in fact, I stole it for my magazine allocator too). It's just that the current implementation is bit hard to understand.

This is virt_to_page() on i386: the object address is in %esi

    lea    0x40000000(%esi),%eax
    mov    0x0,%edx [0x0 is actually mem_map]
    shr    $0xc,%eax
    shl    $0x5,%eax

Just read the mem_map pointer and a few calculations. And now retrieve the cachep pointer:
mov 0x18(%eax,%edx,1),%edx

With NUMA on i386 (GENERIC_ARCH)

   lea    0x40000000(%edi),%eax
   mov    %eax,%ebx
   shr    $0x1c,%eax
   movsbl 0x0(%eax),%eax [ 0x0 is physnode_map]
   shr    $0xc,%ebx
   mov    0x0(,%eax,4),%ecx [0x0 is node_data]
   mov    %ebx,%eax
   mov    0xaa0(%ecx),%edx
   sub    %edx,%eax
   mov    0xa98(%ecx),%edx
   shl    $0x5,%eax

4 memory accesses.
mov 0x18(%eax,%edx,1),%ebp

--
Manfred