Corruption with MMOTS slub-bulk-allocation-from-per-cpu-partial-pages.patch
Subsystems (2)
It seems the patch from (inserted below):
http://ozlabs.org/~akpm/mmots/broken-out/slub-bulk-allocation-from-per-cpu-partial-pages.patch
Is not protecting access to c->partial "enough" (section is under local_irq_disable/enable). When exercising bulk API I can make it crash/corrupt memory when compiled with CONFIG_SLUB_CPU_PARTIAL=y
First I suspected:
object = get_freelist(s, c->page);
But the problem goes way with CONFIG_SLUB_CPU_PARTIAL=n
From: Christoph Lameter <[email protected]>
Subject: slub: bulk allocation from per cpu partial pages
Cover all of the per cpu objects available.
Expand the bulk allocation support to drain the per cpu partial pages while interrupts are off.
---
mm/slub.c | 36 +++++++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-)
mm/slub.c renamed from mm/slub.c~slub-bulk-allocation-from-per-cpu-partial-pages +33 -3
@@ -2769,15 +2769,45 @@ bool kmem_cache_alloc_bulk(struct kmem_cwhile (size) {void *object = c->freelist; -if (!object) -break ; +if (unlikely (!object)) { +/* +* Check if there remotely freed objects +* availalbe in the page. +*/ + object =get_freelist (s, c->page); + +if (!object) { +/* +* All objects in use lets check if +* we have other per cpu partial +* pages that have available +* objects. +*/ + c->page = c->partial; +if (!c->page) { +/* No per cpu objects left */ + c->freelist =NULL ; +break ; + } + +/* Next per cpu partial page */ + c->partial = c->page->next; + c->freelist =get_freelist (s, + c->page); +continue ; + } + + } + - c->freelist =get_freepointer (s, object); *p++ = object; size--;if (unlikely (flags & __GFP_ZERO))memset (object,0 , s->object_size); + + c->freelist =get_freepointer (s, object); + } c->tid =next_tid (c->tid);
_
Signature · 11 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Sr. Network Kernel Developer at Red Hat Author of http://www.iptv-analyzer.org LinkedIn: http://www.linkedin.com/in/brouer -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 71 lines · Jesper Dangaard Brouer
It seems the patch from (inserted below):
http://ozlabs.org/~akpm/mmots/broken-out/slub-bulk-allocation-from-per-cpu-partial-pages.patchIs not protecting access to
c->partial"enough" (section is under local_irq_disable/enable). When exercising bulk API I can make it crash/corrupt memory when compiled withCONFIG_SLUB_CPU_PARTIAL=yFirst I suspected:
object =get_freelist(s, c->page);
But the problem goes way withCONFIG_SLUB_CPU_PARTIAL=nFrom: Christoph Lameter <[email protected]>
Subject: slub: bulk allocation from per cpu partial pagesCover all of the per cpu objects available.
Expand the bulk allocation support to drain the per cpu partial pages while interrupts are off.
Signed-off-by Christoph Lameter <[email protected]>Cc Jesper Dangaard Brouer <[email protected]>Cc Pekka Enberg <[email protected]>Cc David Rientjes <[email protected]>Cc Joonsoo Kim <[email protected]>Signed-off-by Andrew Morton <[email protected]>---
mm/slub.c | 36 +++++++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-)diff -puN
mm/slub.c~slub-bulk-allocation-from-per-cpu-partial-pagesmm/slub.cmm/slub.c renamed from mm/slub.c~slub-bulk-allocation-from-per-cpu-partial-pages +30
-2@@ -2769,15 +2769,45 @@ bool kmem_cache_alloc_bulk(struct kmem_cwhile (size) {void *object = c->freelist; -if (!object) -break ; +if (unlikely (!object)) { +/* +* Check if there remotely freed objects +* availalbe in the page. +*/ + object =get_freelist (s, c->page); + +if (!object) { +/* +* All objects in use lets check if +* we have other per cpu partial +* pages that have available +* objects. +*/ + c->page = c->partial; +if (!c->page) { +/* No per cpu objects left */ + c->freelist =NULL ; +break ; + } + +/* Next per cpu partial page */ + c->partial = c->page->next; + c->freelist =get_freelist (s, + c->page); +continue ; + } + + } +
mm/slub.c bool kmem_cache_alloc_bulk(struct kmem_c · line 2810
Quoted 10 lines
mm/slub.c renamed from mm/slub.c~slub-bulk-allocation-from-per-cpu-partial-pages +3
-1- c->freelist =get_freepointer (s, object); *p++ = object; size--;if (unlikely (flags & __GFP_ZERO))memset (object,0 , s->object_size); + + c->freelist =get_freepointer (s, object); +
Hello,
get_freepointer() should be called before zeroing object. It may help your problem.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
To Andrew/Christoph, can we drop this patch? Then, I'll base my work on top of the previous patch. Which also need some bug fixes, as pointed out by Joonsoo.
(p.s. iif then also drop slub-bulk-allocation-from-per-cpu-partial-pages-fix.patch)
Quoted 85 lines · On Tue, 9 Jun 2015 09:22:59 +0900 Joonsoo Kim <[email protected]> wrote:
Quoted 71 lines · Jesper Dangaard Brouer
It seems the patch from (inserted below):
http://ozlabs.org/~akpm/mmots/broken-out/slub-bulk-allocation-from-per-cpu-partial-pages.patchIs not protecting access to
c->partial"enough" (section is under local_irq_disable/enable). When exercising bulk API I can make it crash/corrupt memory when compiled withCONFIG_SLUB_CPU_PARTIAL=yFirst I suspected:
object =get_freelist(s, c->page);
But the problem goes way withCONFIG_SLUB_CPU_PARTIAL=nFrom: Christoph Lameter <[email protected]>
Subject: slub: bulk allocation from per cpu partial pagesCover all of the per cpu objects available.
Expand the bulk allocation support to drain the per cpu partial pages while interrupts are off.
Signed-off-by Christoph Lameter <[email protected]>Cc Jesper Dangaard Brouer <[email protected]>Cc Pekka Enberg <[email protected]>Cc David Rientjes <[email protected]>Cc Joonsoo Kim <[email protected]>Signed-off-by Andrew Morton <[email protected]>---
mm/slub.c | 36 +++++++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-)diff -puN
mm/slub.c~slub-bulk-allocation-from-per-cpu-partial-pagesmm/slub.cmm/slub.c renamed from mm/slub.c~slub-bulk-allocation-from-per-cpu-partial-pages +30
-2@@ -2769,15 +2769,45 @@ bool kmem_cache_alloc_bulk(struct kmem_cwhile (size) {void *object = c->freelist; -if (!object) -break ; +if (unlikely (!object)) { +/* +* Check if there remotely freed objects +* availalbe in the page. +*/ + object =get_freelist (s, c->page); + +if (!object) { +/* +* All objects in use lets check if +* we have other per cpu partial +* pages that have available +* objects. +*/ + c->page = c->partial; +if (!c->page) { +/* No per cpu objects left */ + c->freelist =NULL ; +break ; + } + +/* Next per cpu partial page */ + c->partial = c->page->next; + c->freelist =get_freelist (s, + c->page); +continue ; + } + + } +mm/slub.c bool kmem_cache_alloc_bulk(struct kmem_c · line 2810
Quoted 10 lines
mm/slub.c renamed from mm/slub.c~slub-bulk-allocation-from-per-cpu-partial-pages +3
-1- c->freelist =get_freepointer (s, object); *p++ = object; size--;if (unlikely (flags & __GFP_ZERO))memset (object,0 , s->object_size); + + c->freelist =get_freepointer (s, object); +Hello,
Quoted 2 lines
get_freepointer()should be called before zeroing object. It may help your problem.
That is a bug, but I'm not invoking with __GFP_ZERO...
Signature · 11 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Sr. Network Kernel Developer at Red Hat Author of http://www.iptv-analyzer.org LinkedIn: http://www.linkedin.com/in/brouer -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>