NUMA API for Linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 14 lines · Hugh Dickins
Quoted 5 lines · Martin J. Bligh
Quoted 5 lines · Andrew Morton
Quoted 3 lines
Your patch takes the
CONFIG_NUMAvma from 64 bytes to 68. It would be nice to pull those 4 bytes back somehow.Quoted 7 lines
Quoted 5 lines
How significant is this vma size issue?
anon_vmaobjrmap will add 20 bytes to each vma (on 32-bit arches):
8 forprio_tree, 12 foranon_vmalinkage in vma, sometimes another 12 for theanon_vmahead itself.Ewwww. Isn't some of that shared most of the time though?
Quoted 7 lines
The
anon_vmahead may well be shared with other vmas of the fork group. But theanon_vmalinkage is alist_headand a pointer within the vma.
prio_treeis already using a union as much as it can (and a pointer where alist_headwould simplify the code); Rajesh was thinking of reusingvm_private_datafor one pointer, but I've gone and used it for nonlinear swapout.
I guess using vm_private_data for nonlinear is not a problem because we use list i_mmap_nonlinear for nonlinear vmas.
As you have found out vm_private_data is only used if vm_file != NULL or VM_RESERVED or VM_DONTEXPAND is set. I think we can revert back to i_mmap{_shared} list for such special cases and use prio_tree for others. I maybe missing something. Please teach me.
If anonmm is merged then I plan to seriously consider removing that 8 extra bytes for prio_tree. If anon_vma is merged, then I can easily point my finger at 12 more bytes added by anon_vma and be happy :)
I still think removing the 8 extra bytes used by prio_tree from vm_area_struct is possible.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
anonmm objrmap adds just the 8 bytes for
prio_tree, remaining overhead 28 bytes per mm.28 bytes per *mm* is nothing, and I still think the
prio_treeis completely unneccesary. Nobody has ever demonstrated a real benchmark that needs it, as far as I recall.I'm sure an Ingobench will shortly follow that observation.
Yeap. If Andrew didn't write his rmap-test.c and Ingo didn't write his test-mmap3.c, I wouldn't even have considered developing prio_tree.
Thanks,
Rajesh
Quoted 8 lines · Rajesh Venkatasubramanian
I guess using
vm_private_datafor nonlinear is not a problem because we use listi_mmap_nonlinearfor nonlinear vmas.As you have found out
vm_private_datais only used ifvm_file!= NULL orVM_RESERVEDorVM_DONTEXPANDis set. I think we can revert back to i_mmap{_shared} list for such special cases and useprio_treefor others. I maybe missing something. Please teach me.
Sorry, I don't understand what you're proposing here, and why? Oh, to save 4 bytes of vma by making the special cases use a list, no need for vm_set_head, and vm_private_data for the driver itself; but regular vmas use prio_tree, with vm_set_head in vm_private_data. Hmm, right now it's getting too much for me: can we keep it simplish for now, and come back to this later?
Hugh
Quoted 10 lines · Hugh Dickins
Quoted 8 lines · Rajesh Venkatasubramanian
I guess using
vm_private_datafor nonlinear is not a problem because we use listi_mmap_nonlinearfor nonlinear vmas.As you have found out
vm_private_datais only used ifvm_file!= NULL orVM_RESERVEDorVM_DONTEXPANDis set. I think we can revert back to i_mmap{_shared} list for such special cases and useprio_treefor others. I maybe missing something. Please teach me.
Quoted 4 lines
Sorry, I don't understand what you're proposing here, and why? Oh, to save 4 bytes of vma by making the special cases use a list, no need for
vm_set_head, andvm_private_datafor the driver itself; but regular vmas useprio_tree, withvm_set_headinvm_private_data.
Yeah. You are right.
Quoted 2 lines
Hmm, right now it's getting too much for me: can we keep it simplish for now, and come back to this later?
Yeah. This complicates the code further. That's why I didn't touch it now. If things settle down and if we really worry about sizeof vm_area_struct in the future, then we can remove the 8 bytes used by prio_tree.
Rajesh
Quoted 2 lines · Rajesh Venkatasubramanian
8 extra bytes for
prio_tree. Ifanon_vmais merged, then I can easily point my finger at 12 more bytes added byanon_vmaand be happy :)
those 12 more bytes payoff providing better swapping performance, handle mremap transparently and make the code scale better, allowing even to _trivially_ move the page_table_lock into the vma without any downside (I mean performance downside, per-vma lock will cost one more bit of info in every vma).
There's no way to remove those 12 bytes without falling in the downsides of anonmm.
I don't see a real need to shrink the size of the prio_tree right now though if something useless and can be removed that's fine with me, what I'm trying to tell is that pointing the finger at 12 bytes of anon_vma doesn't sound a good argument since there's no way to remove them without falling back in the anonmm downsides.