Skip to contentSkip to conversation

Permit inode & dentry hash tables to be allocated > MAX_ORDER size

PATCH Andi Kleen 3 participants 6 messages Jun 12, 2004 – Jun 15, 2004 3 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

Andi Kleen

author
#3 lore raw
Quoted 3 lines · Martin J. Bligh

Allocating the big-assed hashes out of bootmem seems much cleaner to me, at least ...

Machines big enough that such big hashes make sense are probably NUMA. And on NUMA systems you imho should rather use node interleaving vmalloc(), not a bit physical allocation on a specific node for these hashes. This will avoid memory controller hot spots and avoid the problem completely. Likely it will perform better too.

-Andi

#4 lore raw
Quoted 8 lines
Quoted 2 lines

Allocating the big-assed hashes out of bootmem seems much cleaner to me, at least ...

Machines big enough that such big hashes make sense are probably NUMA. And on NUMA systems you imho should rather use node interleaving vmalloc(), not a bit physical allocation on a specific node for these hashes. This will avoid memory controller hot spots and avoid the problem completely. Likely it will perform better too.

I thought Manfred already fixed all that up, didn't he?

M.

#5 lore raw
In reply to Andi Kleen
Quoted 10 lines · Andi Kleen
Quoted 3 lines · Martin J. Bligh

Allocating the big-assed hashes out of bootmem seems much cleaner to me, at least ...

Machines big enough that such big hashes make sense are probably NUMA. And on NUMA systems you imho should rather use node interleaving vmalloc(), not a bit physical allocation on a specific node for these hashes. This will avoid memory controller hot spots and avoid the problem completely. Likely it will perform better too.

Since vmalloc() maps the pages with small pagetable entries (unlike most of the rest of the kernel address space), do you think the interleaving will outweigh any negative TLB effects?

-- Dave

author
#6 lore raw
Quoted 3 lines

Since vmalloc() maps the pages with small pagetable entries (unlike most of the rest of the kernel address space), do you think the interleaving will outweigh any negative TLB effects?

I think so, yes (assuming you run the benchmark on all CPUs)

-Andi

#7 lore raw
Quoted 5 lines
Quoted 3 lines

Since vmalloc() maps the pages with small pagetable entries (unlike most of the rest of the kernel address space), do you think the interleaving will outweigh any negative TLB effects?

I think so, yes (assuming you run the benchmark on all CPUs)

On the other hand, there's no reason we can't hack up a version of vmalloc to use large pages, and interleave only based on that.

M.

#8 lore raw
Quoted 8 lines · Martin J. Bligh
Quoted 5 lines
Quoted 3 lines

Since vmalloc() maps the pages with small pagetable entries (unlike most of the rest of the kernel address space), do you think the interleaving will outweigh any negative TLB effects?

I think so, yes (assuming you run the benchmark on all CPUs)

On the other hand, there's no reason we can't hack up a version of vmalloc to use large pages, and interleave only based on that.

Think about ppc64 where the large page size is 16MB. That might hurt interleaving a bit if the structure is only 32MB. It's better than *everything* on node 0, but not by much.

-- Dave