struct page field arrangement
Subsystems (2)
A change early last year reordered struct page so that ptl overlaps not only private, but also mapping. Since spinlock_t can be much larger, I'm wondering whether there's a reason to not also overlay the space index and lru take - are these used for anything on page table pages?
Thanks, Jan
Quoted 4 lines · Jan Beulich
A change early last year reordered struct page so that ptl overlaps not only private, but also mapping. Since
spinlock_tcan be much larger, I'm wondering whether there's a reason to not also overlay the space index and lru take - are these used for anything on page table pages?
Overlaying lru is a problem for for those architectures which use kmem_cache_alloc for their pagetables: arm26, powerpc, sparc64 and perhaps others (I just grepped quickly through include/asm*, didn't follow up those who have extern functions): since slab reuses the lru fields for its own purposes. Could perhaps be stacked somehow.
Overlaying index is fairly straightforward: the index field is fair game. In my original patches I did overlay index, but Andrew was strongly averse to the way I was doing it, and scaled things back, to private alone if I remember rightly, then relaxed a little to include mapping too. Way back then I made up a patch to overlay index too (when I saw Fedora going out with CONFIG_DEBUG_SPINLOCK), but I could never get it into a form where I felt it would satisfy Andrew; and grew increasingly dissatisfied with that approach myself.
I don't think further overlaying is the right answer really. But I do think it's a scandal that the size of struct page (in a DEBUG_SPINLOCK system) is governed by such a minority use of the struct page. Lacking a satisfying answer, I've just let it drift on until someone notices and complains.
kmalloc a separate spinlock structure when it's too big to fit in? Not such a good idea, since then there will tend to be false sharing of cachelines between them: simpler just to disable SPLIT_PTLOCK in that case.
I'm not happy with the status quo, but I don't know the right answer:
perhaps allow pagetable pages to use an undebugged spinlock variant?
Hugh
Having looked a little into the disabled SPLIT_PT_LOCK issue on Xen, I realized that is shouldn't be too difficult to re-enable it (at least in some cases).
One option would be to not use the mapping field of struct page for PageForeign but rather the index one (and to that effect was a mail I sent to Andrew Morton and Hugh Dickins that I got below answer on, basically indicating that it should be okay to use index here). This would allow removing the Xen conditional on SPLIT_PTLOCK_CPUS altogether.
The other option would be along the lines of attached patch (not checked whether it would apply as-is to -unstable, but it appears to work okay), allowing the splitting at least in non-debug configs.
Opinions?
Thanks, Jan
Quoted 1 line
Quoted 1 line
Quoted 1 line
Hugh Dickins <[email protected]> 28.02.07 22:08 >>>
Quoted 4 lines · Jan Beulich
A change early last year reordered struct page so that ptl overlaps not only private, but also mapping. Since
spinlock_tcan be much larger, I'm wondering whether there's a reason to not also overlay the space index and lru take - are these used for anything on page table pages?
Overlaying lru is a problem for for those architectures which use kmem_cache_alloc for their pagetables: arm26, powerpc, sparc64 and perhaps others (I just grepped quickly through include/asm*, didn't follow up those who have extern functions): since slab reuses the lru fields for its own purposes. Could perhaps be stacked somehow.
Overlaying index is fairly straightforward: the index field is fair game. In my original patches I did overlay index, but Andrew was strongly averse to the way I was doing it, and scaled things back, to private alone if I remember rightly, then relaxed a little to include mapping too. Way back then I made up a patch to overlay index too (when I saw Fedora going out with CONFIG_DEBUG_SPINLOCK), but I could never get it into a form where I felt it would satisfy Andrew; and grew increasingly dissatisfied with that approach myself.
I don't think further overlaying is the right answer really. But I do think it's a scandal that the size of struct page (in a DEBUG_SPINLOCK system) is governed by such a minority use of the struct page. Lacking a satisfying answer, I've just let it drift on until someone notices and complains.
kmalloc a separate spinlock structure when it's too big to fit in? Not such a good idea, since then there will tend to be false sharing of cachelines between them: simpler just to disable SPLIT_PTLOCK in that case.
I'm not happy with the status quo, but I don't know the right answer:
perhaps allow pagetable pages to use an undebugged spinlock variant?
Hugh
From: [email protected]
Subject: allow use of split page table lock in non-debug case
Patch-mainline: obsolete
include/linux/mm.h +5 -1
@@ -891,7 +891,11 @@ static inline pmd_t *pmd_alloc(struct mm#define pte_lock_init (_page) do { \ spin_lock_init(__pte_lockptr(_page)); \ } while (0) -#define pte_lock_deinit (page) ((page)->mapping = NULL) +#define pte_lock_deinit (_page) do { \ + if ((void *)&((struct page *)0)->mapping \ + < (void *)(__pte_lockptr((struct page *)0) + 1)) \ + (_page)->mapping = NULL; \ +} while (0)#define pte_lockptr (mm, pmd) ({(void)(mm); __pte_lockptr(pmd_page(*(pmd)));})#else /*
include/linux/page-flags.h +8 -0
@@ -254,8 +254,16 @@ static inline void SetPageUptodate(struc#define ClearPageUncached (page) clear_bit(PG_uncached, &(page)->flags)#define PageForeign (page) test_bit(PG_foreign, &(page)->flags) +#if NR_CPUS >=CONFIG_SPLIT_PTLOCK_CPUS +#define PageForeignCheck () \ + BUILD_BUG_ON((void *)&((struct page *)0)->mapping \ + < (void *)(__pte_lockptr((struct page *)0) + 1)) +#else +#define PageForeignCheck () ((void)0) +#endif #define SetPageForeign (page, dtor) do { \ set_bit(PG_foreign, &(page)->flags); \ + PageForeignCheck(); \ (page)->mapping = (void *)dtor; \ } while (0)#define ClearPageForeign (page) do { \
mm/Kconfig +2 -1
@@ -139,7 +139,8 @@ config SPLIT_PTLOCK_CPUS int default "4096" if ARM && !CPU_CACHE_VIPT default "4096" if PARISC && !PA20 - default "4096" if X86_XEN || X86_64_XEN + default "4096" if X86_XEN && (PREEMPT || DEBUG_SPINLOCK || DEBUG_LOCK_ALLOC) + default "4096" if X86_64_XEN && (DEBUG_SPINLOCK || DEBUG_LOCK_ALLOC) default "4" #
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Quoted 3 lines · Jan Beulich
Having looked a little into the disabled
SPLIT_PT_LOCKissue on Xen, I realized that is shouldn't be too difficult to re-enable it (at least in some cases).
How does pagetable locking work in modern Linux kernels? It seems that updates of ptes are protected by a per-page lock or the mm lock, and population of page directory entries is protected by the mm lock, but that there is no synchronisation with read-only pagetable walks. Does this mean that sections of pagetable hierarchy are never reaped from a process until it dies?
Can we confident that the mm_pin/mm_unpin code (which walks pagetables and has to find every page to make every one read-only or writable) is safe? Presumably for this to be true we need to be sure that noone can meanwhile concurrently be populating the pagetable we are walking with extra pgds/puds/pmds/ptes...
-- Keir
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Keir Fraser <[email protected]> 01.03.07 11:22 >>>
Quoted 3 lines · Jan Beulich
Having looked a little into the disabled
SPLIT_PT_LOCKissue on Xen, I realized that is shouldn't be too difficult to re-enable it (at least in some cases).
Quoted 6 lines
How does pagetable locking work in modern Linux kernels? It seems that updates of ptes are protected by a per-page lock or the mm lock, and population of page directory entries is protected by the mm lock, but that there is no synchronisation with read-only pagetable walks. Does this mean that sections of pagetable hierarchy are never reaped from a process until it dies?
Yes, that's my understanding.
Quoted 5 lines
Can we confident that the mm_pin/mm_unpin code (which walks pagetables and has to find every page to make every one read-only or writable) is safe? Presumably for this to be true we need to be sure that noone can meanwhile concurrently be populating the pagetable we are walking with extra pgds/puds/pmds/ptes...
Since the pin/unpin walking only cares about pgd/pud/pmd entries, synchronization is guaranteed through mm->page_table_lock. The pte lock is used only for leaf entries, which are of no concern to (un)pinning.
Jan
Quoted 10 lines · Jan Beulich
Quoted 5 lines
Can we confident that the mm_pin/mm_unpin code (which walks pagetables and has to find every page to make every one read-only or writable) is safe? Presumably for this to be true we need to be sure that noone can meanwhile concurrently be populating the pagetable we are walking with extra pgds/puds/pmds/ptes...
Since the pin/unpin walking only cares about pgd/pud/pmd entries, synchronization
is guaranteed throughmm->page_table_lock. The pte lock is used only for leaf entries, which are of no concern to (un)pinning.
Oh yes, of course.
By the way, I think your proposed patch looks okay. However we already use page->index for foreign pages, in netback. Perhaps page->lru is also fair game? :-)
K.
Quoted 3 lines
By the way, I think your proposed patch looks okay. However we already use
page->indexfor foreign pages, in netback. Perhapspage->lruis also fair game? :-)
Not generally, there are arch-es that use it for their own purposes (look back at the message from Hugh I originally forwarded). So even if none of Xen's currently supported would use it (I didn't check), I wouldn't want to introduce a lurking issue if anyone comes up with a new port.
I didn't check what private is being used for - while it can't be used for PageForeign's destructor pointer, perhaps netback could use that one rather than index? Or, even more likely, netback could use mapping if PageForeign was switched over to index (while mapping overlays ptl, that is benign to netback as it's not dealing with pte pages). But of course, this requires thorough auditing of other uses of index.
Jan
Quoted 6 lines · Jan Beulich
I didn't check what private is being used for - while it can't be used for PageForeign's destructor pointer, perhaps netback could use that one rather than index? Or, even more likely, netback could use mapping if PageForeign was switched over to index (while mapping overlays ptl, that is benign to netback as it's not dealing with pte pages). But of course, this requires thorough auditing of other uses of index.
Since PageForeign is only used for pagetables (for which you checked ->index should be okay) and for skbuff data (where we already use ->index in netback) then changing PageForeign to put the callback in ->index should be fine.
-- Keir