Skip to contentSkip to conversation

NUMA API for Linux

DISCUSSION Rajesh Venkatasubramanian 3 participants 4 messages Apr 8, 2004 2 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw
Quoted 14 lines · Hugh Dickins
Quoted 5 lines · Martin J. Bligh
Quoted 5 lines · Andrew Morton
Quoted 3 lines

Your patch takes the CONFIG_NUMA vma from 64 bytes to 68. It would be nice to pull those 4 bytes back somehow.

Quoted 7 lines
Quoted 5 lines

How significant is this vma size issue?

anon_vma objrmap will add 20 bytes to each vma (on 32-bit arches):
8 for prio_tree, 12 for anon_vma linkage in vma, sometimes another 12 for the anon_vma head itself.

Ewwww. Isn't some of that shared most of the time though?

Quoted 7 lines

The anon_vma head may well be shared with other vmas of the fork group. But the anon_vma linkage is a list_head and a pointer within the vma.

prio_tree is already using a union as much as it can (and a pointer where a list_head would simplify the code); Rajesh was thinking of reusing vm_private_data for one pointer, but I've gone and used it for nonlinear swapout.

I guess using vm_private_data for nonlinear is not a problem because we use list i_mmap_nonlinear for nonlinear vmas.

As you have found out vm_private_data is only used if vm_file != NULL or VM_RESERVED or VM_DONTEXPAND is set. I think we can revert back to i_mmap{_shared} list for such special cases and use prio_tree for others. I maybe missing something. Please teach me.

If anonmm is merged then I plan to seriously consider removing that 8 extra bytes for prio_tree. If anon_vma is merged, then I can easily point my finger at 12 more bytes added by anon_vma and be happy :)

I still think removing the 8 extra bytes used by prio_tree from vm_area_struct is possible.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

anonmm objrmap adds just the 8 bytes for prio_tree, remaining overhead 28 bytes per mm.

28 bytes per *mm* is nothing, and I still think the prio_tree is completely unneccesary. Nobody has ever demonstrated a real benchmark that needs it, as far as I recall.

I'm sure an Ingobench will shortly follow that observation.

Yeap. If Andrew didn't write his rmap-test.c and Ingo didn't write his test-mmap3.c, I wouldn't even have considered developing prio_tree.

Thanks,
Rajesh

#3 lore raw
Quoted 8 lines · Rajesh Venkatasubramanian

I guess using vm_private_data for nonlinear is not a problem because we use list i_mmap_nonlinear for nonlinear vmas.

As you have found out vm_private_data is only used if vm_file != NULL or VM_RESERVED or VM_DONTEXPAND is set. I think we can revert back to i_mmap{_shared} list for such special cases and use prio_tree for others. I maybe missing something. Please teach me.

Sorry, I don't understand what you're proposing here, and why? Oh, to save 4 bytes of vma by making the special cases use a list, no need for vm_set_head, and vm_private_data for the driver itself; but regular vmas use prio_tree, with vm_set_head in vm_private_data. Hmm, right now it's getting too much for me: can we keep it simplish for now, and come back to this later?

Hugh

#4 lore raw
Quoted 10 lines · Hugh Dickins
Quoted 8 lines · Rajesh Venkatasubramanian

I guess using vm_private_data for nonlinear is not a problem because we use list i_mmap_nonlinear for nonlinear vmas.

As you have found out vm_private_data is only used if vm_file != NULL or VM_RESERVED or VM_DONTEXPAND is set. I think we can revert back to i_mmap{_shared} list for such special cases and use prio_tree for others. I maybe missing something. Please teach me.

Quoted 4 lines

Sorry, I don't understand what you're proposing here, and why? Oh, to save 4 bytes of vma by making the special cases use a list, no need for vm_set_head, and vm_private_data for the driver itself; but regular vmas use prio_tree, with vm_set_head in vm_private_data.

Yeah. You are right.

Quoted 2 lines

Hmm, right now it's getting too much for me: can we keep it simplish for now, and come back to this later?

Yeah. This complicates the code further. That's why I didn't touch it now. If things settle down and if we really worry about sizeof vm_area_struct in the future, then we can remove the 8 bytes used by prio_tree.

Rajesh

#5 lore raw
In reply to Rajesh Venkatasubramanian
Quoted 2 lines · Rajesh Venkatasubramanian

8 extra bytes for prio_tree. If anon_vma is merged, then I can easily point my finger at 12 more bytes added by anon_vma and be happy :)

those 12 more bytes payoff providing better swapping performance, handle mremap transparently and make the code scale better, allowing even to _trivially_ move the page_table_lock into the vma without any downside (I mean performance downside, per-vma lock will cost one more bit of info in every vma).

There's no way to remove those 12 bytes without falling in the downsides of anonmm.

I don't see a real need to shrink the size of the prio_tree right now though if something useless and can be removed that's fine with me, what I'm trying to tell is that pointing the finger at 12 bytes of anon_vma doesn't sound a good argument since there's no way to remove them without falling back in the anonmm downsides.