mm: cma: add some debug information for CMA
Subsystems (4)
v3 cover mm: cma: add some debug information for CMA
Hi all.
Here is the third version of a patch set that adds some debugging facility for CMA.
This patch set is based on v4.0-rc1 and Sasha Levin's patch set "mm: cma: debugfs access to CMA" v5 [1].
It is also available on git:git://github.com/stefanstrogin/linux -b cmainfo-v3
We want an interface to see a list of currently allocated CMA buffers and some useful information about them (like /proc/vmallocinfo but for physically contiguous buffers allocated with CMA).
For example. We want a big (megabytes) CMA buffer to be allocated in runtime in default CMA region. If someone already uses CMA then the big allocation could fail. If it happened then with such an interface we could find who used CMA at the moment of failure, who caused fragmentation and so on. Ftrace also would be helpful here, but with ftrace we can see the whole history of allocations and releases, whereas with this patch set we can see a snapshot of CMA region with actual information about its allocations.
These patches add some files to debugfs when CONFIG_CMA_DEBUGFS is enbled.
/sys/kernel/debug/cma/cma-<N>/used and cma/cma-<N>/maxchunk are added to show used size in pages and the biggest free chunk in each CMA region.
When CONFIG_CMA_BUFFER_LIST is enabled (depended on CONFIG_CMA_DEBUGFS) /sys/kernel/debug/cma/cma-<N>/buffers is added. It contains a list of currently allocated contiguous buffers for each CMA region (N stands for region number). Format is:
<base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)
Another added option is CONFIG_CMA_ALLOC_STACKTRACE. When it's enabled then stack traces are also saved when the allocations are made. The stack traces are added to each entry in cma/cma-<N>/buffers (see an example [3]). This can be used to see who and whence allocated each buffer.
Also added trace events for cma_alloc() and cma_release().
Changes from v2 (https://lkml.org/lkml/2015/2/12/647):
- Rebased on v4.0-rc1 and Sasha Levin's patch set v5 [1].
- Changed kmalloc() to vmalloc() in cma_buffer_list_read().
- Added CONFIG_CMA_BUFFER_LIST and CONFIG_CMA_ALLOC_STACKTRACE.
- Considered order_per_bit for returning page number in cma_get_used() and cma_get_maxchunk().
- Reordered the patches to make the one with trace events indepentent of others.
- Moved the prototypes of cma_buffer_list_add() and cma_buffer_list_del() from include/linux/cma.h to mm/cma.h.
- Various small changes.
Changes from v1 (aka "mm: cma: add /proc/cmainfo")
(https://lkml.org/lkml/2014/12/26/95):
- Rebased on v3.19 and Sasha Levin's patch set v4 [2].
- Moved debug code from cma.c to cma_debug.c.
- Moved cmainfo to debugfs and splited it by CMA region.
- Splited 'cmainfo' into 'buffers', 'used' and 'maxchunk'.
- Used CONFIG_CMA_DEBUGFS instead of CONFIG_CMA_DEBUG.
- Added trace events for cma_alloc() and cma_release().
- Don't use seq_files.
- A small change of debug output in cma_release().
- cma_buffer_list_del() now supports releasing chunks which ranges don't match allocations. E.g. we have buffer1: [0x0, 0x1], buffer2: [0x2, 0x3], then cma_buffer_list_del(cma, 0x1 /*or 0x0*/, 1 /*(or 2 or 3)*/) should work.
- Various small changes.
[1] https://lkml.org/lkml/2015/2/12/657
[2] https://lkml.org/lkml/2015/1/28/755
[3] root@debian:/sys/kernel/debug/cma# cat cma-0/buffers 0x2f400000 - 0x2f417000 (92 kB), allocated by pid 1 (swapper/0)
[<c1142c4b>] cma_alloc+0x1bb/0x200 [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40 [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160 [<c14456ce>] dmam_alloc_coherent+0xbe/0x100 [<c1487312>] ahci_port_start+0xe2/0x210 [<c146e0e0>] ata_host_start.part.28+0xc0/0x1a0 [<c1473650>] ata_host_activate+0xd0/0x110 [<c14881bf>] ahci_host_activate+0x3f/0x170 [<c14854e4>] ahci_init_one+0x764/0xab0 [<c12e415f>] pci_device_probe+0x6f/0xd0 [<c14378a8>] driver_probe_device+0x68/0x210 [<c1437b09>] __driver_attach+0x79/0x80 [<c1435eef>] bus_for_each_dev+0x4f/0x80 [<c143749e>] driver_attach+0x1e/0x20 [<c1437197>] bus_add_driver+0x157/0x200 [<c14381bd>] driver_register+0x5d/0xf0 <...> 0x2f41b000 - 0x2f41c000 (4 kB), allocated by pid 1264 (NetworkManager) [<c1142c4b>] cma_alloc+0x1bb/0x200 [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40 [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160 [<c14c5d13>] e1000_setup_all_tx_resources+0x93/0x540 [<c14c8021>] e1000_open+0x31/0x120 [<c16264cf>] __dev_open+0x9f/0x130 [<c16267ce>] __dev_change_flags+0x8e/0x150 [<c16268b8>] dev_change_flags+0x28/0x60 [<c1633ee0>] do_setlink+0x2a0/0x760 [<c1634acb>] rtnl_newlink+0x60b/0x7b0 [<c16314f4>] rtnetlink_rcv_msg+0x84/0x1f0 [<c164b58e>] netlink_rcv_skb+0x8e/0xb0 [<c1631461>] rtnetlink_rcv+0x21/0x30 [<c164af7a>] netlink_unicast+0x13a/0x1d0 [<c164b250>] netlink_sendmsg+0x240/0x3e0 [<c160cbfd>] do_sock_sendmsg+0xbd/0xe0
<...>
Dmitry Safonov (1):
mm: cma: add functions to get region pages counters
Stefan Strogin (3):
mm: cma: add trace events to debug physically-contiguous memory
allocations
mm: cma: add number of pages to debug message in cma_release()
mm: cma: add list of currently allocated CMA buffers to debugfs
include/linux/cma.h | 2 + include/trace/events/cma.h | 57 ++++++++++++++ mm/Kconfig | 17 +++++ mm/cma.c | 45 +++++++++++- mm/cma.h | 26 +++++++ mm/cma_debug.c | 180 +++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 326 insertions(+), 1 deletion(-) create mode 100644 include/trace/events/cma.h
Signature · 1 line
2.1.0
PATCH v3 1/4 mm: cma: add trace events to debug physically-contiguous memory allocations
Add trace events for cma_alloc() and cma_release().
--- include/trace/events/cma.h | 57 ++++++++++++++++++++++++++++++++++++++++++++++ mm/cma.c | 6 +++++ 2 files changed, 63 insertions(+) create mode 100644 include/trace/events/cma.h
include/trace/events/cma.h new file +57 -0
@@ -0,0 +1,57 @@ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM cma + +#if !defined(_TRACE_CMA_H) || defined(TRACE_HEADER_MULTI_READ ) +#define _TRACE_CMA_H + +#include <linux/types.h> +#include <linux/tracepoint.h> + +TRACE_EVENT (cma_alloc, + + TP_PROTO(struct cma *cma, unsigned long pfn, int count), + +TP_ARGS (cma, pfn, count), + +TP_STRUCT__entry ( + __field(unsigned long, pfn) + __field(unsigned long, count) + ), + +TP_fast_assign ( + __entry->pfn = pfn; + __entry->count = count; + ), + +TP_printk ("pfn=%lu page=%p count=%lu\n" , + __entry->pfn, +pfn_to_page (__entry->pfn), + __entry->count) +); + +TRACE_EVENT (cma_release, + + TP_PROTO(struct cma *cma, unsigned long pfn, int count), + +TP_ARGS (cma, pfn, count), + +TP_STRUCT__entry ( + __field(unsigned long, pfn) + __field(unsigned long, count) + ), + +TP_fast_assign ( + __entry->pfn = pfn; + __entry->count = count; + ), + +TP_printk ("pfn=%lu page=%p count=%lu\n" , + __entry->pfn, +pfn_to_page (__entry->pfn), + __entry->count) +); + +#endif /* _TRACE_CMA_H */ + +/* This part must be outside protection */ +#include <trace/define_trace.h>
mm/cma.c +6 -0
@@ -23,6 +23,7 @@# define DEBUG #endif #endif +#define CREATE_TRACE_POINTS #include <linux/memblock.h> #include <linux/err.h>
@@ -34,6 +35,7 @@#include <linux/cma.h> #include <linux/highmem.h> #include <linux/io.h> +#include <trace/events/cma.h> #include "cma.h"
@@ -408,6 +410,9 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align) start = bitmap_no + mask +1 ; } +if (page) +trace_cma_alloc (cma, pfn, count); +pr_debug ("%s(): returned %p\n" , __func__, page);return page; }
@@ -440,6 +445,7 @@ bool cma_release(struct cma *cma, struct page *pages, int count)free_contig_range (pfn, count);cma_clear_bitmap (cma, pfn, count); +trace_cma_release (cma, pfn, count);return true; }
Signature · 7 lines
2.1.0 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v3 1/4] mm: cma: add trace events to debug physically-contiguous memory allocations
Quoted 3 lines · Stefan Strogin
Add trace events for
cma_alloc()andcma_release().Signed-off-by Stefan Strogin <[email protected]>
Looks good to me but than again I don’t know much about trace points so perhaps someone else should ack it as well.
Quoted 5 lines
--- include/trace/events/cma.h | 57 ++++++++++++++++++++++++++++++++++++++++++++++ mm/cma.c | 6 +++++ 2 files changed, 63 insertions(+) create mode 100644 include/trace/events/cma.h
Signature · 10 lines
Best regards, _ _ .o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o ..o | Computer Science, Michał “mina86” Nazarewicz (o o) ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v3 1/4] mm: cma: add trace events to debug physically-contiguous memory allocations
In reply to Stefan StroginQuoted 103 lines · Stefan Strogin
Add trace events for
cma_alloc()andcma_release().Signed-off-by Stefan Strogin <[email protected]>--- include/trace/events/cma.h | 57 ++++++++++++++++++++++++++++++++++++++++++++++ mm/cma.c | 6 +++++ 2 files changed, 63 insertions(+) create mode 100644 include/trace/events/cma.hinclude/trace/events/cma.h new file +57
-0@@ -0,0 +1,57 @@ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM cma + +#if !defined(_TRACE_CMA_H) || defined(TRACE_HEADER_MULTI_READ ) +#define _TRACE_CMA_H + +#include <linux/types.h> +#include <linux/tracepoint.h> + +TRACE_EVENT (cma_alloc, + + TP_PROTO(struct cma *cma, unsigned long pfn, int count), + +TP_ARGS (cma, pfn, count), + +TP_STRUCT__entry ( + __field(unsigned long, pfn) + __field(unsigned long, count) + ), + +TP_fast_assign ( + __entry->pfn = pfn; + __entry->count = count; + ), + +TP_printk ("pfn=%lu page=%p count=%lu\n" , + __entry->pfn, +pfn_to_page (__entry->pfn), + __entry->count) +); + +TRACE_EVENT (cma_release, + + TP_PROTO(struct cma *cma, unsigned long pfn, int count), + +TP_ARGS (cma, pfn, count), + +TP_STRUCT__entry ( + __field(unsigned long, pfn) + __field(unsigned long, count) + ), + +TP_fast_assign ( + __entry->pfn = pfn; + __entry->count = count; + ), + +TP_printk ("pfn=%lu page=%p count=%lu\n" , + __entry->pfn, +pfn_to_page (__entry->pfn), + __entry->count) +); + +#endif /* _TRACE_CMA_H */ + +/* This part must be outside protection */ +#include <trace/define_trace.h> mm/cma.c +5
-0@@ -23,6 +23,7 @@# define DEBUG #endif #endif +#define CREATE_TRACE_POINTS #include <linux/memblock.h> #include <linux/err.h> @@ -34,6 +35,7 @@#include <linux/cma.h> #include <linux/highmem.h> #include <linux/io.h> +#include <trace/events/cma.h> #include "cma.h" @@ -408,6 +410,9 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align) start = bitmap_no + mask +1 ; } +if (page) +trace_cma_alloc (cma, pfn, count); +pr_debug ("%s(): returned %p\n" , __func__, page);return page; }
mm/cma.c bool cma_release(struct cma *cma, struct page *pages, int count) · line 450
Quoted 6 lines
mm/cma.c +1
-0@@ -440,6 +445,7 @@ bool cma_release(struct cma *cma, struct page *pages, int count)free_contig_range (pfn, count);cma_clear_bitmap (cma, pfn, count); +trace_cma_release (cma, pfn, count);return true;
Are we interested only in successful allocation and release ? Should we also have the trace point carry information regarding failure ?
-aneesh
Re: [PATCH v3 1/4] mm: cma: add trace events to debug physically-contiguous memory allocations
Hi Aneesh,
Quoted 6 lines · Aneesh Kumar K.V
Are we interested only in successful allocation and release ? Should we also have the trace point carry information regarding failure ?
-aneesh
I think we actually can be interested in tracing allocation failures too. Thanks for the remark.
Should it be smth like that?
@@ -408,6 +410,8 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
start = bitmap_no + mask + 1;
}
+ trace_cma_alloc(cma, page, count);
+
pr_debug("%s(): returned %p\n", __func__, page); return page;
}
and in include/trace/events/cma.h:
+TRACE_EVENT(cma_alloc,
<...>
+ TP_fast_assign(
+ __entry->page = page;
+ __entry->count = count;
+ ),
+
+ TP_printk("page=%p pfn=%lu count=%lu\n",
+ __entry->page, + __entry->page ? page_to_pfn(__entry->page) : 0, + __entry->count)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH v3 2/4 mm: cma: add number of pages to debug message in cma_release()
In reply to Stefan StroginIt's more useful to print address and number of pages which are being released, not only address.
--- mm/cma.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/cma.c +1 -1
@@ -434,7 +434,7 @@ bool cma_release(struct cma *cma, struct page *pages, int count)if (!cma || !pages)return false; -pr_debug ("%s(page %p)\n" , __func__, (void *)pages); +pr_debug ("%s(page %p, count %d)\n" , __func__, (void *)pages, count); pfn =page_to_pfn (pages);
Signature · 7 lines
2.1.0 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v3 2/4] mm: cma: add number of pages to debug message in cma_release()
Quoted 4 lines · Stefan Strogin
It's more useful to print address and number of pages which are being released, not only address.
Signed-off-by Stefan Strogin <[email protected]>
Quoted 20 lines
--- mm/cma.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/cma.c +1
-1@@ -434,7 +434,7 @@ bool cma_release(struct cma *cma, struct page *pages, int count)if (!cma || !pages)return false; -pr_debug ("%s(page %p)\n" , __func__, (void *)pages); +pr_debug ("%s(page %p, count %d)\n" , __func__, (void *)pages, count); pfn =page_to_pfn (pages);Signature · 1 line
2.1.0
Signature · 10 lines
Best regards, _ _ .o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o ..o | Computer Science, Michał “mina86” Nazarewicz (o o) ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH v3 3/4 mm: cma: add list of currently allocated CMA buffers to debugfs
In reply to Stefan StroginWhen CONFIG_CMA_BUFFER_LIST is configured a file is added to debugfs:
/sys/kernel/debug/cma/cma-<N>/buffers contains a list of currently allocated CMA buffers for each CMA region (N stands for number of CMA region).
Format is:
<base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)
When CONFIG_CMA_ALLOC_STACKTRACE is configured then stack traces are saved when the allocations are made. The stack traces are added to cma/cma-<N>/buffers for each buffer list entry.
Example:
root@debian:/sys/kernel/debug/cma# cat cma-0/buffers 0x2f400000 - 0x2f417000 (92 kB), allocated by pid 1 (swapper/0) [<c1142c4b>] cma_alloc+0x1bb/0x200 [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40 [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160 [<c14456ce>] dmam_alloc_coherent+0xbe/0x100 [<c1487312>] ahci_port_start+0xe2/0x210 [<c146e0e0>] ata_host_start.part.28+0xc0/0x1a0 [<c1473650>] ata_host_activate+0xd0/0x110 [<c14881bf>] ahci_host_activate+0x3f/0x170 [<c14854e4>] ahci_init_one+0x764/0xab0 [<c12e415f>] pci_device_probe+0x6f/0xd0 [<c14378a8>] driver_probe_device+0x68/0x210 [<c1437b09>] __driver_attach+0x79/0x80 [<c1435eef>] bus_for_each_dev+0x4f/0x80 [<c143749e>] driver_attach+0x1e/0x20 [<c1437197>] bus_add_driver+0x157/0x200 [<c14381bd>] driver_register+0x5d/0xf0 <...>
--- mm/Kconfig | 17 +++++++ mm/cma.c | 9 +++- mm/cma.h | 26 ++++++++++ mm/cma_debug.c | 156 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 207 insertions(+), 1 deletion(-)
mm/Kconfig +17 -0
@@ -523,6 +523,23 @@ config CMA_DEBUGFS help Turns on the DebugFS interface for CMA. +config CMA_BUFFER_LIST + bool "List of currently allocated CMA buffers in debugfs" + depends on CMA_DEBUGFS + help + /sys/kernel/debug/cma/cma-<N>/buffers contains a list of currently + allocated CMA buffers for each CMA region (N stands for number of + CMA region). + Format is: + <base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>) + +config CMA_ALLOC_STACKTRACE + bool "Add stack trace to CMA buffer list" + depends on CMA_BUFFER_LIST && STACKTRACE + help + Add stack traces saved at the moment of allocation for each buffer + listed in /sys/kernel/debug/cma/cma-<N>/buffers. + config CMA_AREAS int "Maximum count of the CMA areas" depends on CMA
mm/cma.c +8 -1
@@ -128,6 +128,10 @@ static int __init cma_activate_area(struct cma *cma)INIT_HLIST_HEAD (&cma->mem_head);spin_lock_init (&cma->mem_head_lock);#endif +#ifdef CONFIG_CMA_BUFFER_LIST +INIT_LIST_HEAD (&cma->buffer_list); +mutex_init (&cma->list_lock); +#endif return 0 ;
@@ -410,8 +414,10 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align) start = bitmap_no + mask +1 ; } -if (page) +if (page) {trace_cma_alloc (cma, pfn, count); +cma_buffer_list_add (cma, pfn, count); + }pr_debug ("%s(): returned %p\n" , __func__, page);return page;
@@ -446,6 +452,7 @@ bool cma_release(struct cma *cma, struct page *pages, int count)free_contig_range (pfn, count);cma_clear_bitmap (cma, pfn, count);trace_cma_release (cma, pfn, count); +cma_buffer_list_del (cma, pfn, count);return true; }
mm/cma.h +26 -0
@@ -1,6 +1,8 @@ #ifndef __MM_CMA_H__ #define __MM_CMA_H__ +#include <linux/sched.h> + struct cma { unsigned long base_pfn; unsigned long count;
@@ -11,8 +13,32 @@ struct cma {struct hlist_head mem_head;spinlock_t mem_head_lock;#endif +#ifdef CONFIG_CMA_BUFFER_LIST +struct list_head buffer_list; +struct mutex list_lock; +#endif }; +#ifdef CONFIG_CMA_BUFFER_LIST +struct cma_buffer { +unsigned long pfn; +unsigned long count; +pid_t pid; +char comm[TASK_COMM_LEN ]; +#ifdef CONFIG_CMA_ALLOC_STACKTRACE +unsigned long trace_entries[16 ]; +unsigned int nr_entries; +#endif +struct list_head list; +}; + +extern int cma_buffer_list_add (struct cma *cma,unsigned long pfn,int count); +extern void cma_buffer_list_del (struct cma *cma,unsigned long pfn,int count); +#else +#define cma_buffer_list_add (cma, pfn, count) { } +#define cma_buffer_list_del (cma, pfn, count) { } +#endif /* CONFIG_CMA_BUFFER_LIST */ +extern struct cma cma_areas[MAX_CMA_AREAS ];extern unsigned cma_area_count;
mm/cma_debug.c +156 -0
186 lines not on this page · raw message
Signature · 7 lines
2.1.0 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs
Quoted 50 lines · Stefan Strogin
When
CONFIG_CMA_BUFFER_LISTis configured a file is added to debugfs:
/sys/kernel/debug/cma/cma-<N>/buffers contains a list of currently allocated CMA buffers for each CMA region (N stands for number of CMA region).Format is:
<base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)When
CONFIG_CMA_ALLOC_STACKTRACEis configured then stack traces are saved when the allocations are made. The stack traces are added to cma/cma-<N>/buffers for each buffer list entry.Example:
root@debian:/sys/kernel/debug/cma# cat cma-0/buffers 0x2f400000 - 0x2f417000 (92 kB), allocated by pid 1 (swapper/0) [<c1142c4b>] cma_alloc+0x1bb/0x200 [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40 [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160 [<c14456ce>] dmam_alloc_coherent+0xbe/0x100 [<c1487312>] ahci_port_start+0xe2/0x210 [<c146e0e0>] ata_host_start.part.28+0xc0/0x1a0 [<c1473650>] ata_host_activate+0xd0/0x110 [<c14881bf>] ahci_host_activate+0x3f/0x170 [<c14854e4>] ahci_init_one+0x764/0xab0 [<c12e415f>] pci_device_probe+0x6f/0xd0 [<c14378a8>] driver_probe_device+0x68/0x210 [<c1437b09>] __driver_attach+0x79/0x80 [<c1435eef>] bus_for_each_dev+0x4f/0x80 [<c143749e>] driver_attach+0x1e/0x20 [<c1437197>] bus_add_driver+0x157/0x200 [<c14381bd>] driver_register+0x5d/0xf0 <...>Signed-off-by Stefan Strogin <[email protected]>mm/cma.h +6
-0@@ -11,8 +13,32 @@ struct cma {struct hlist_head mem_head;spinlock_t mem_head_lock;#endif +#ifdef CONFIG_CMA_BUFFER_LIST +struct list_head buffer_list; +struct mutex list_lock; +#endif }; +#ifdef CONFIG_CMA_BUFFER_LIST +struct cma_buffer {
mm/cma.h struct cma { · line 33
Quoted 10 lines
mm/cma.h +10
-0+unsigned long pfn; +unsigned long count; +pid_t pid; +char comm[TASK_COMM_LEN ]; +#ifdef CONFIG_CMA_ALLOC_STACKTRACE +unsigned long trace_entries[16 ]; +unsigned int nr_entries; +#endif +struct list_head list; +};
This structure is only ever used in cma_debug.c so is there a reason to define it in the header file?
Quoted 45 lines
mm/cma.h +41
-0+ +extern int cma_buffer_list_add (struct cma *cma,unsigned long pfn,int count); +extern void cma_buffer_list_del (struct cma *cma,unsigned long pfn,int count); +#else +#define cma_buffer_list_add (cma, pfn, count) { } +#define cma_buffer_list_del (cma, pfn, count) { } +#endif /* CONFIG_CMA_BUFFER_LIST */ +extern struct cma cma_areas[MAX_CMA_AREAS ];extern unsigned cma_area_count; +#ifdef CONFIG_CMA_BUFFER_LIST +static ssize_t cma_buffer_list_read (struct file *file,char __user *userbuf, +size_t count,loff_t *ppos) +{ +struct cma *cma = file->private_data; +struct cma_buffer *cmabuf; +char *buf; +int ret, n =0 ; +#ifdef CONFIG_CMA_ALLOC_STACKTRACE +struct stack_trace trace; +#endif + +if (*ppos <0 || !count) +return -EINVAL ; + + buf =vmalloc (count); +if (!buf) +return -ENOMEM ; + +mutex_lock (&cma->list_lock); +list_for_each_entry (cmabuf, &cma->buffer_list, list) { + n +=snprintf (buf + n, count - n, +"0x%llx - 0x%llx (%lu kB), allocated by pid %u (%s)\n" , + (unsigned long long )PFN_PHYS (cmabuf->pfn), + (unsigned long long )PFN_PHYS (cmabuf->pfn + + cmabuf->count), + (cmabuf->count *PAGE_SIZE ) >>10 , cmabuf->pid, + cmabuf->comm); + +#ifdef CONFIG_CMA_ALLOC_STACKTRACE + trace.nr_entries = cmabuf->nr_entries; + trace.entries = &cmabuf->trace_entries[0 ]; + n +=snprint_stack_trace (buf + n, count - n, &trace,0 );
mm/cma.h struct cma {
Quoted 10 lines
mm/cma.h +10
-0+ n +=snprintf (buf + n, count - n,"\n" ); +#endif + } +mutex_unlock (&cma->list_lock); + + ret =simple_read_from_buffer (userbuf, count, ppos, buf, n); +vfree (buf); + +return ret; +}
So in practice user space must allocate buffer big enough to read the whole file into memory. Calling read(2) with some count will never read anything past the first count bytes of the file.
Signature · 10 lines
Best regards, _ _ .o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o ..o | Computer Science, Michał “mina86” Nazarewicz (o o) ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs
Hi Michał,
Thank you for the answer.
Quoted 13 lines · Michal Nazarewicz
Quoted 12 lines · Stefan Strogin
mm/cma.h +4
-0@@ -11,8 +13,32 @@ struct cma {struct hlist_head mem_head;spinlock_t mem_head_lock;#endif +#ifdef CONFIG_CMA_BUFFER_LIST +struct list_head buffer_list; +struct mutex list_lock; +#endif };
Quoted 16 lines
mm/cma.h struct cma { · line 33
Quoted 12 lines
mm/cma.h +12
-0+#ifdef CONFIG_CMA_BUFFER_LIST +struct cma_buffer { +unsigned long pfn; +unsigned long count; +pid_t pid; +char comm[TASK_COMM_LEN ]; +#ifdef CONFIG_CMA_ALLOC_STACKTRACE +unsigned long trace_entries[16 ]; +unsigned int nr_entries; +#endif +struct list_head list; +};This structure is only ever used in
cma_debug.cso is there a reason to define it in the header file?
No, there isn't. Thanks. I'll move it to cma_debug.c
Quoted 51 lines
Quoted 51 lines
mm/cma.h +47
-0+ +extern int cma_buffer_list_add (struct cma *cma,unsigned long pfn,int count); +extern void cma_buffer_list_del (struct cma *cma,unsigned long pfn,int count); +#else +#define cma_buffer_list_add (cma, pfn, count) { } +#define cma_buffer_list_del (cma, pfn, count) { } +#endif /* CONFIG_CMA_BUFFER_LIST */ +extern struct cma cma_areas[MAX_CMA_AREAS ];extern unsigned cma_area_count; +#ifdef CONFIG_CMA_BUFFER_LIST +static ssize_t cma_buffer_list_read (struct file *file,char __user *userbuf, +size_t count,loff_t *ppos) +{ +struct cma *cma = file->private_data; +struct cma_buffer *cmabuf; +char *buf; +int ret, n =0 ; +#ifdef CONFIG_CMA_ALLOC_STACKTRACE +struct stack_trace trace; +#endif + +if (*ppos <0 || !count) +return -EINVAL ; + + buf =vmalloc (count); +if (!buf) +return -ENOMEM ; + +mutex_lock (&cma->list_lock); +list_for_each_entry (cmabuf, &cma->buffer_list, list) { + n +=snprintf (buf + n, count - n, +"0x%llx - 0x%llx (%lu kB), allocated by pid %u (%s)\n" , + (unsigned long long )PFN_PHYS (cmabuf->pfn), + (unsigned long long )PFN_PHYS (cmabuf->pfn + + cmabuf->count), + (cmabuf->count *PAGE_SIZE ) >>10 , cmabuf->pid, + cmabuf->comm); + +#ifdef CONFIG_CMA_ALLOC_STACKTRACE + trace.nr_entries = cmabuf->nr_entries; + trace.entries = &cmabuf->trace_entries[0 ]; + n +=snprint_stack_trace (buf + n, count - n, &trace,0 ); + n +=snprintf (buf + n, count - n,"\n" ); +#endif + } +mutex_unlock (&cma->list_lock); + + ret =simple_read_from_buffer (userbuf, count, ppos, buf, n);
Quoted 9 lines
mm/cma.h
Quoted 4 lines
mm/cma.h +4
-0+vfree (buf); + +return ret; +}So in practice user space must allocate buffer big enough to read the whole file into memory. Calling read(2) with some count will never read anything past the first count bytes of the file.
My fault. You are right.
I'm not sure how to do the output nice... I could use *ppos to point the number of next list entry to read (like that is used in read_page_owner()). But in this case the list could be changed before we finish reading, it's bad.
Or we could use seq_files like in v1, iterating over buffer_list entries. But seq_print_stack_trace() has to be added.
Re: [PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs
Quoted 7 lines · Stefan Strogin
My fault. You are right.
I'm not sure how to do the output nice... I could use *ppos to point the number of next list entry to read (like that is used inread_page_owner()). But in this case the list could be changed before we finish reading, it's bad.
Or we could useseq_fileslike in v1, iterating overbuffer_listentries. Butseq_print_stack_trace()has to be added.
I’m not that familiar with seq_* so my opinion may be ill-informed, but I feel that skipping some entries while (de)allocations happen is akin to process reading a file while some other process modifies it. This is a debug function so perhaps it’s acceptable that it may return garbage if not used carefully?
Signature · 10 lines
Best regards, _ _ .o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o ..o | Computer Science, Michał “mina86” Nazarewicz (o o) ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs
In reply to Stefan StroginQuoted 24 lines · Stefan Strogin
When
CONFIG_CMA_BUFFER_LISTis configured a file is added to debugfs:
/sys/kernel/debug/cma/cma-<N>/buffers contains a list of currently allocated CMA buffers for each CMA region (N stands for number of CMA region).Format is:
<base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)When
CONFIG_CMA_ALLOC_STACKTRACEis configured then stack traces are saved when the allocations are made. The stack traces are added to cma/cma-<N>/buffers for each buffer list entry.Example:
root@debian:/sys/kernel/debug/cma# cat cma-0/buffers 0x2f400000 - 0x2f417000 (92 kB), allocated by pid 1 (swapper/0) [<c1142c4b>] cma_alloc+0x1bb/0x200 [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40 [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160 [<c14456ce>] dmam_alloc_coherent+0xbe/0x100 [<c1487312>] ahci_port_start+0xe2/0x210 [<c146e0e0>] ata_host_start.part.28+0xc0/0x1a0 [<c1473650>] ata_host_activate+0xd0/0x110 [<c14881bf>] ahci_host_activate+0x3f/0x170 [<c14854e4>] ahci_init_one+0x764/0xab0
Quoted 8 lines
[<c12e415f>] pci_device_probe+0x6f/0xd0 [<c14378a8>] driver_probe_device+0x68/0x210 [<c1437b09>] __driver_attach+0x79/0x80 [<c1435eef>] bus_for_each_dev+0x4f/0x80 [<c143749e>] driver_attach+0x1e/0x20 [<c1437197>] bus_add_driver+0x157/0x200 [<c14381bd>] driver_register+0x5d/0xf0 <...>
A perf record -g will also give this information right ? To use this feature, one need to recompile the kernel anyway. So why not assume that user can always rerun the test with perf record -g and find the cma allocation point stack trace ?
-aneesh
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs
Hi Aneesh,
Quoted 35 lines · Aneesh Kumar K.V
Quoted 24 lines · Stefan Strogin
When
CONFIG_CMA_BUFFER_LISTis configured a file is added to debugfs:
/sys/kernel/debug/cma/cma-<N>/buffers contains a list of currently allocated CMA buffers for each CMA region (N stands for number of CMA region).Format is:
<base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)When
CONFIG_CMA_ALLOC_STACKTRACEis configured then stack traces are saved when the allocations are made. The stack traces are added to cma/cma-<N>/buffers for each buffer list entry.Example:
root@debian:/sys/kernel/debug/cma# cat cma-0/buffers 0x2f400000 - 0x2f417000 (92 kB), allocated by pid 1 (swapper/0) [<c1142c4b>] cma_alloc+0x1bb/0x200 [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40 [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160 [<c14456ce>] dmam_alloc_coherent+0xbe/0x100 [<c1487312>] ahci_port_start+0xe2/0x210 [<c146e0e0>] ata_host_start.part.28+0xc0/0x1a0 [<c1473650>] ata_host_activate+0xd0/0x110 [<c14881bf>] ahci_host_activate+0x3f/0x170 [<c14854e4>] ahci_init_one+0x764/0xab0Quoted 8 lines
[<c12e415f>] pci_device_probe+0x6f/0xd0 [<c14378a8>] driver_probe_device+0x68/0x210 [<c1437b09>] __driver_attach+0x79/0x80 [<c1435eef>] bus_for_each_dev+0x4f/0x80 [<c143749e>] driver_attach+0x1e/0x20 [<c1437197>] bus_add_driver+0x157/0x200 [<c14381bd>] driver_register+0x5d/0xf0 <...>
Quoted 7 lines
A perf record -g will also give this information right ? To use this feature, one need to recompile the kernel anyway. So why not assume that user can always rerun the test with perf record -g and find the cma allocation point stack trace ?
-aneesh
Excuse me for the delay.
I thought that 'perf record <command>' gathers data only for a command that it runs, does it? But we want to have information about all the allocations and releases from the boot time. IMHO it would be more reasonable to use ftrace for that. But after all the patch enables to see not a history of allocations and deallocations but a current state of CMA region.
As to recompilation, for example in our division this feature is enabled by default among other CONFIG_*_DEBUG features in debug versions of kernel.
Re: [PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs
Quoted 46 lines · Stefan Strogin
Hi Aneesh,
Quoted 35 lines · Aneesh Kumar K.V
Quoted 24 lines · Stefan Strogin
When
CONFIG_CMA_BUFFER_LISTis configured a file is added to debugfs:
/sys/kernel/debug/cma/cma-<N>/buffers contains a list of currently allocated CMA buffers for each CMA region (N stands for number of CMA region).Format is:
<base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)When
CONFIG_CMA_ALLOC_STACKTRACEis configured then stack traces are saved when the allocations are made. The stack traces are added to cma/cma-<N>/buffers for each buffer list entry.Example:
root@debian:/sys/kernel/debug/cma# cat cma-0/buffers 0x2f400000 - 0x2f417000 (92 kB), allocated by pid 1 (swapper/0) [<c1142c4b>] cma_alloc+0x1bb/0x200 [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40 [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160 [<c14456ce>] dmam_alloc_coherent+0xbe/0x100 [<c1487312>] ahci_port_start+0xe2/0x210 [<c146e0e0>] ata_host_start.part.28+0xc0/0x1a0 [<c1473650>] ata_host_activate+0xd0/0x110 [<c14881bf>] ahci_host_activate+0x3f/0x170 [<c14854e4>] ahci_init_one+0x764/0xab0Quoted 8 lines
[<c12e415f>] pci_device_probe+0x6f/0xd0 [<c14378a8>] driver_probe_device+0x68/0x210 [<c1437b09>] __driver_attach+0x79/0x80 [<c1435eef>] bus_for_each_dev+0x4f/0x80 [<c143749e>] driver_attach+0x1e/0x20 [<c1437197>] bus_add_driver+0x157/0x200 [<c14381bd>] driver_register+0x5d/0xf0 <...>Quoted 7 lines
A perf record -g will also give this information right ? To use this feature, one need to recompile the kernel anyway. So why not assume that user can always rerun the test with perf record -g and find the cma allocation point stack trace ?
-aneesh
Quoted 6 lines
Excuse me for the delay.
I thought that 'perf record <command>' gathers data only for a command that it runs, does it? But we want to have information about all the allocations and releases from the boot time.From boot time makes it interesting. Otherwise you could use perf
record. For ex:
./perf record -e kmem:kmalloc -g -a
./perf script
jbd2/dm-3-8 7666 [000] 4666.621521: kmem:kmalloc: call_site=c0000000003ce108 ptr=0xc000000fcd646360 bytes_req=96 bytes_alloc=96 gfp_flags=GFP_NOFS|GFP_ZERO
27f1dc .__kmalloc (/boot/vmlinux)
0 [unknown] ([unknown])
3de108 .ext4_find_extent (/boot/vmlinux)
3e4038 .ext4_ext_map_blocks (/boot/vmlinux)
3ab388 .ext4_map_blocks (/boot/vmlinux)
3ab970 ._ext4_get_block (/boot/vmlinux)
2df7e0 .generic_block_bmap (/boot/vmlinux)
3a7f4c .ext4_bmap (/boot/vmlinux)
2ba744 .bmap (/boot/vmlinux)
4293d4 .jbd2_journal_bmap (/boot/vmlinux)
41df84 .jbd2_journal_commit_transaction (/boot/vmlinux)
427620 .kjournald2 (/boot/vmlinux)
fbfa4 .kthread (/boot/vmlinux)
19568 .ret_from_kernel_thread (/boot/vmlinux)....
Quoted 6 lines
IMHO it would be more
reasonable to use ftrace for that. But after all the patch enables to see not a history of allocations and deallocations but a current state of CMA region.
As to recompilation, for example in our division this feature is enabled by default among other CONFIG_*_DEBUG features in debug versions of kernel.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH v3 4/4 mm: cma: add functions to get region pages counters
In reply to Stefan StroginFrom: Dmitry Safonov <[email protected]>
Here are two functions that provide interface to compute/get used size and size of biggest free chunk in cma region. Add that information to debugfs.
--- include/linux/cma.h | 2 ++ mm/cma.c | 30 ++++++++++++++++++++++++++++++ mm/cma_debug.c | 24 ++++++++++++++++++++++++ 3 files changed, 56 insertions(+)
include/linux/cma.h +2 -0
@@ -18,6 +18,8 @@ struct cma;extern unsigned long totalcma_pages;extern phys_addr_t cma_get_base (struct cma *cma);extern unsigned long cma_get_size (struct cma *cma); +extern unsigned long cma_get_used (struct cma *cma); +extern unsigned long cma_get_maxchunk (struct cma *cma); extern int __init cma_declare_contiguous(phys_addr_t base, phys_addr_t size, phys_addr_t limit,
mm/cma.c +30 -0
@@ -53,6 +53,36 @@ unsigned long cma_get_size(struct cma *cma)return cma->count <<PAGE_SHIFT ; } +unsigned long cma_get_used (struct cma *cma) +{ +unsigned long ret =0 ; + +mutex_lock (&cma->lock); +/* pages counter is smaller than sizeof(int) */ + ret =bitmap_weight (cma->bitmap, (int )cma->count); +mutex_unlock (&cma->lock); + +return ret << cma->order_per_bit; +} + +unsigned long cma_get_maxchunk (struct cma *cma) +{ +unsigned long maxchunk =0 ; +unsigned long start, end =0 ; + +mutex_lock (&cma->lock); +for (;;) { + start =find_next_zero_bit (cma->bitmap, cma->count, end); +if (start >= cma->count) +break ; + end =find_next_bit (cma->bitmap, cma->count, start); + maxchunk =max (end - start, maxchunk); + } +mutex_unlock (&cma->lock); + +return maxchunk << cma->order_per_bit; +} +static unsigned long cma_bitmap_aligned_mask (struct cma *cma,int align_order) {if (align_order <= cma->order_per_bit)
mm/cma_debug.c +24 -0
@@ -133,6 +133,28 @@ static int cma_debugfs_get(void *data, u64 *val)DEFINE_SIMPLE_ATTRIBUTE (cma_debugfs_fops, cma_debugfs_get,NULL ,"%llu\n" ); +static int cma_used_get (void *data,u64 *val) +{ +struct cma *cma = data; + + *val =cma_get_used (cma); + +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (cma_used_fops, cma_used_get,NULL ,"%llu\n" ); + +static int cma_maxchunk_get (void *data,u64 *val) +{ +struct cma *cma = data; + + *val =cma_get_maxchunk (cma); + +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (cma_maxchunk_fops, cma_maxchunk_get,NULL ,"%llu\n" ); +static void cma_add_to_cma_mem_list (struct cma *cma,struct cma_mem *mem) {spin_lock (&cma->mem_head_lock);
@@ -300,6 +322,8 @@ static void cma_debugfs_add_one(struct cma *cma, int idx) &cma->count, &cma_debugfs_fops);debugfs_create_file ("order_per_bit" ,S_IRUGO , tmp, &cma->order_per_bit, &cma_debugfs_fops); +debugfs_create_file ("used" ,S_IRUGO , tmp, cma, &cma_used_fops); +debugfs_create_file ("maxchunk" ,S_IRUGO , tmp, cma, &cma_maxchunk_fops);#ifdef CONFIG_CMA_BUFFER_LIST debugfs_create_file ("buffers" ,S_IRUGO , tmp, cma, &cma_buffer_list_fops);
Signature · 7 lines
2.1.0 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v3 4/4] mm: cma: add functions to get region pages counters
Quoted 7 lines · Stefan Strogin
From: Dmitry Safonov <[email protected]>
Here are two functions that provide interface to compute/get used size and size of biggest free chunk in cma region. Add that information to debugfs.
Signed-off-by Dmitry Safonov <[email protected]>Signed-off-by Stefan Strogin <[email protected]>
Quoted 5 lines
--- include/linux/cma.h | 2 ++ mm/cma.c | 30 ++++++++++++++++++++++++++++++ mm/cma_debug.c | 24 ++++++++++++++++++++++++ 3 files changed, 56 insertions(+)
Signature · 10 lines
Best regards, _ _ .o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o ..o | Computer Science, Michał “mina86” Nazarewicz (o o) ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>