CMA page migration failure due to buffers on bh_lru

RFC Gioh Kim 4 participants 8 messages Jun 24, 2014 – Jul 3, 2014 Memory management

Share & tools

Gioh Kim

author

2014-06-24 01:12 UTC #0 lore raw

Hello,

I am trying to apply CMA feature for my platform.
My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
Now my platform can use CMA area effectively.

But I have many failures to allocate memory from CMA area.
I found the same situation to Laura Abbott's patch descrbing, https://lkml.org/lkml/2012/8/31/313,
that releases buffer-heads attached at CPU's LRU list.

If Joonsoo's patch is applied and/or CMA feature is applied more and more, buffer-heads problem is going to be serious definitely.

Please look into the Laura's patch again.
I think it must be applied with Joonsoo's patch.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Michal Nazarewicz

2014-06-26 15:57 UTC #1 lore raw

Quoted 12 lines · Gioh Kim

Hello,
I am trying to apply CMA feature for my platform.
My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
Now my platform can use CMA area effectively.
But I have many failures to allocate memory from CMA area. I found the same situation to Laura Abbott's patch descrbing,
https://lkml.org/lkml/2012/8/31/313,
that releases buffer-heads attached at CPU's LRU list.

Quoted 5 lines

If Joonsoo's patch is applied and/or CMA feature is applied more and more, buffer-heads problem is going to be serious definitely.
Please look into the Laura's patch again.
I think it must be applied with Joonsoo's patch.

Just to make sure I understood you correctly, you're saying Laura's patch at <https://lkml.org/lkml/2012/8/31/313> fixes your issue?

Signature · 4 lines

Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo--

Gioh Kim

author

2014-06-26 23:23 UTC #2 lore raw

2014-06-27 오전 12:57, Michal Nazarewicz 쓴 글:

Quoted 13 lines · Gioh Kim

Quoted 12 lines
Hello,
I am trying to apply CMA feature for my platform.
My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
Now my platform can use CMA area effectively.
But I have many failures to allocate memory from CMA area. I found the same situation to Laura Abbott's patch descrbing,
https://lkml.org/lkml/2012/8/31/313,
that releases buffer-heads attached at CPU's LRU list.

Quoted 9 lines

Quoted 5 lines
If Joonsoo's patch is applied and/or CMA feature is applied more and more, buffer-heads problem is going to be serious definitely.
Please look into the Laura's patch again.
I think it must be applied with Joonsoo's patch.
Just to make sure I understood you correctly, you're saying Laura's patch at <https://lkml.org/lkml/2012/8/31/313> fixes your issue?

Yes, it is.

Laura Abbott

2014-06-29 19:49 UTC #3 lore raw

(cc-ing Hugh since he had comments on the patch before)

Quoted 19 lines · Gioh Kim

2014-06-27 i??i ? 12:57, Michal Nazarewicz i?' e,?:
Quoted 16 lines · Gioh Kim
Quoted 15 lines
Hello,
I am trying to apply CMA feature for my platform.
My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
Now my platform can use CMA area effectively.
But I have many failures to allocate memory from CMA area. I found the same situation to Laura Abbott's patch descrbing,
https://lkml.org/lkml/2012/8/31/313,
that releases buffer-heads attached at CPU's LRU list.
If Joonsoo's patch is applied and/or CMA feature is applied more and more, buffer-heads problem is going to be serious definitely.

Quoted 8 lines

Quoted 6 lines
Quoted 2 lines
Please look into the Laura's patch again.
I think it must be applied with Joonsoo's patch.
Just to make sure I understood you correctly, you're saying Laura's patch at <https://lkml.org/lkml/2012/8/31/313> fixes your issue?
Yes, it is.

I submitted this before and it was suggested that this was more related to filesystems

http://marc.info/?l=linaro-mm-sig&m=137645770708817&w=2

I never saw more discussion and pushed this into the 'CMA hacks' pile. So far we've been keeping the patch out of tree and it's useful to know that others have found the same problem. I'm willing to resubmit the patch for further discussion.

Thanks,
Laura

Signature · 8 lines

Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Gioh Kim

author

2014-07-01 01:07 UTC #4 lore raw

Hi,Laura.

I have a question.

Does the __evict_bh_lru() not need bh_lru_lock()?
The get_cpu_var() has already preenpt_disable() and can prevent other thread. But get_cpu_var cannot prevent IRQ context such like page-fault.
I think if a page-fault occured and a file is read in IRQ context it can change cpu-lru.

Is my concern correct?

2014-06-30 i??i ? 4:49, Laura Abbott i?' e,?:

Quoted 36 lines

(cc-ing Hugh since he had comments on the patch before)
Quoted 19 lines · Gioh Kim
2014-06-27 i??i ? 12:57, Michal Nazarewicz i?' e,?:
Quoted 16 lines · Gioh Kim
Quoted 15 lines
Hello,
I am trying to apply CMA feature for my platform.
My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
Now my platform can use CMA area effectively.
But I have many failures to allocate memory from CMA area. I found the same situation to Laura Abbott's patch descrbing,
https://lkml.org/lkml/2012/8/31/313,
that releases buffer-heads attached at CPU's LRU list.
If Joonsoo's patch is applied and/or CMA feature is applied more and more, buffer-heads problem is going to be serious definitely.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Please look into the Laura's patch again.
I think it must be applied with Joonsoo's patch.
Just to make sure I understood you correctly, you're saying Laura's patch at <https://lkml.org/lkml/2012/8/31/313> fixes your issue?
Yes, it is.
I submitted this before and it was suggested that this was more related to filesystems
http://marc.info/?l=linaro-mm-sig&m=137645770708817&w=2

Quoted 8 lines

I never saw more discussion and pushed this into the 'CMA hacks' pile. So far we've been keeping the patch out of tree and it's useful to know that others have found the same problem. I'm willing to resubmit the patch for further discussion.
Thanks,
Laura

Laura Abbott

2014-07-01 02:02 UTC #5 lore raw

Quoted 12 lines · Gioh Kim

Hi,Laura.
I have a question.
Does the __evict_bh_lru() not need bh_lru_lock()?
The get_cpu_var() has already preenpt_disable() and can prevent other thread. But get_cpu_var cannot prevent IRQ context such like page-fault.
I think if a page-fault occured and a file is read in IRQ context it can change cpu-lru.
Is my concern correct?

__evict_bh_lru is called via on_each_cpu_cond which I believe will disable irqs. I based the code on the existing invalidate_bh_lru which did not take the bh_lru_lock either. It's possible I missed something though.

Thanks,
Laura

Signature · 8 lines

Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Andrew Morton

maintainer

2014-07-02 05:46 UTC #6 lore raw

Quoted 10 lines · On Mon, 30 Jun 2014 19:02:45 -0700 Laura Abbott <[email protected]> wrote:

Quoted 9 lines · Gioh Kim
Hi,Laura.
I have a question.
Does the __evict_bh_lru() not need bh_lru_lock()?
The get_cpu_var() has already preenpt_disable() and can prevent other thread. But get_cpu_var cannot prevent IRQ context such like page-fault.
I think if a page-fault occured and a file is read in IRQ context it can change cpu-lru.

Quoted 7 lines

Quoted 3 lines
Is my concern correct?
__evict_bh_lru is called via on_each_cpu_cond which I believe will disable irqs. I based the code on the existing invalidate_bh_lru which did not take the bh_lru_lock either. It's possible I missed something though.

I fear that running on_each_cpu() within try_to_free_buffers() is going to be horridly expensive in some cases.

Maybe CMA can use invalidate_bh_lrus() to shoot down everything before trying the allocation attempt. That should increase the success rate greatly and doesn't burden page reclaim. The bh LRU isn't terribly important from a performance point of view, so emptying it occasionally won't hurt.

Gioh Kim

author

2014-07-03 07:34 UTC #7 lore raw

Hi, Laura,

I has replaced the evict_bh_lrus(bh) with invalidate_bh_lrus() and it is working fine. How about submit new patch with invalidate_bh_lrus()?
I would appreciate it.

2014-07-02 오후 2:46, Andrew Morton 쓴 글:

Quoted 30 lines · On Mon, 30 Jun 2014 19:02:45 -0700 Laura Abbott <[email protected]> wrote:

Quoted 11 lines · Gioh Kim
Quoted 9 lines
Hi,Laura.
I have a question.
Does the __evict_bh_lru() not need bh_lru_lock()?
The get_cpu_var() has already preenpt_disable() and can prevent other thread. But get_cpu_var cannot prevent IRQ context such like page-fault.
I think if a page-fault occured and a file is read in IRQ context it can change cpu-lru.
Quoted 7 lines
Quoted 3 lines
Is my concern correct?
__evict_bh_lru is called via on_each_cpu_cond which I believe will disable irqs. I based the code on the existing invalidate_bh_lru which did not take the bh_lru_lock either. It's possible I missed something though.
I fear that running on_each_cpu() within try_to_free_buffers() is going to be horridly expensive in some cases.
Maybe CMA can use invalidate_bh_lrus() to shoot down everything before trying the allocation attempt. That should increase the success rate greatly and doesn't burden page reclaim. The bh LRU isn't terribly important from a performance point of view, so emptying it occasionally won't hurt.