Skip to contentSkip to conversation

mm: cma: support sysfs

PATCH Superseded Minchan Kim 3 participants 12 messages Mar 24, 2021

Subsystems (3)

v7 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

PATCH v7 1/1 mm: cma: support sysfs

Since CMA is getting used more widely, it's more important to keep monitoring CMA statistics for system health since it's directly related to user experience.

This patch introduces sysfs statistics for CMA, in order to provide some basic monitoring of the CMA allocator.

* the number of CMA page successful allocations
* the number of CMA page allocation failures

These two values allow the user to calcuate the allocation failure rate for each CMA area.

e.g.)
/sys/kernel/mm/cma/WIFI/alloc_pages_[success|fail] /sys/kernel/mm/cma/SENSOR/alloc_pages_[success|fail] /sys/kernel/mm/cma/BLUETOOTH/alloc_pages_[success|fail]

The cma_stat was intentionally allocated by dynamic allocation to harmonize with kobject lifetime management.
https://lore.kernel.org/linux-mm/[email protected]/

Addresses-Coverity: ("Dereference after null check")

---
Andrew, you could apply this patch after revering these patches:

mm: cma: fix potential null dereference on pointer cma
mm: cma: support sysfs

From v6 - https://lore.kernel.org/linux-mm/[email protected]/
From v5 - https://lore.kernel.org/linux-mm/[email protected]/
From v4 - https://lore.kernel.org/linux-mm/[email protected]/
* fix corruption - digetx@
* refactoring - digetx@, jhubbard@, willy@

From v3 - https://lore.kernel.org/linux-mm/[email protected]/
* fix ZERO_OR_NULL_PTR - kernel test robot
* remove prefix cma - david@
* resolve conflict with vmstat cma in mmotm - akpm@
* rename stat name with success|fail

From v2 - https://lore.kernel.org/linux-mm/[email protected]/
* sysfs doc and description modification - jhubbard

From v1 - https://lore.kernel.org/linux-mm/[email protected]/
* fix sysfs build and refactoring - willy
* rename and drop some attributes - jhubbard

 Documentation/ABI/testing/sysfs-kernel-mm-cma |  25 ++++
 mm/Kconfig                                    |   7 ++
 mm/Makefile                                   |   1 +
 mm/cma.c                                      |   8 +-
 mm/cma.h                                      |  23 ++++
 mm/cma_sysfs.c                                | 116 ++++++++++++++++++
 6 files changed, 178 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/ABI/testing/sysfs-kernel-mm-cma
 create mode 100644 mm/cma_sysfs.c
Documentation/ABI/testing/sysfs-kernel-mm-cma new file +25 -0
new file mode 100644
index 000000000000..02b2bb60c296
@@ -0,0 +1,25 @@
+What:		/sys/kernel/mm/cma/
+Date:		Feb 2021
+Contact:	Minchan Kim <[email protected]>
+Description:
+		/sys/kernel/mm/cma/ contains a subdirectory for each CMA
+		heap name (also sometimes called CMA areas).
+
+		Each CMA heap subdirectory (that is, each
+		/sys/kernel/mm/cma/<cma-heap-name> directory) contains the
+		following items:
+
+			alloc_pages_success
+			alloc_pages_fail
+
+What:		/sys/kernel/mm/cma/<cma-heap-name>/alloc_pages_success
+Date:		Feb 2021
+Contact:	Minchan Kim <[email protected]>
+Description:
+		the number of pages CMA API succeeded to allocate
+
+What:		/sys/kernel/mm/cma/<cma-heap-name>/alloc_pages_fail
+Date:		Feb 2021
+Contact:	Minchan Kim <[email protected]>
+Description:
+		the number of pages CMA API failed to allocate
mm/Kconfig +7 -0
index 23a0e3c98ff0..3823a2314256 100644
@@ -524,6 +524,13 @@ config CMA_DEBUGFS
 	help
 	  Turns on the DebugFS interface for CMA.
 
+config CMA_SYSFS
+	bool "CMA information through sysfs interface"
+	depends on CMA && SYSFS
+	help
+	  This option exposes some sysfs attributes to get information
+	  from CMA.
+
 config CMA_AREAS
 	int "Maximum count of the CMA areas"
 	depends on CMA
mm/Makefile +1 -0
index 9e284dba50ef..788c5ce5c0ef 100644
@@ -112,6 +112,7 @@ obj-$(CONFIG_CMA)	+= cma.o
 obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
 obj-$(CONFIG_PAGE_EXTENSION) += page_ext.o
 obj-$(CONFIG_CMA_DEBUGFS) += cma_debug.o
+obj-$(CONFIG_CMA_SYSFS) += cma_sysfs.o
 obj-$(CONFIG_USERFAULTFD) += userfaultfd.o
 obj-$(CONFIG_IDLE_PAGE_TRACKING) += page_idle.o
 obj-$(CONFIG_DEBUG_PAGE_REF) += debug_page_ref.o
mm/cma.c +6 -2
index 0361e289c31a..08c45157911a 100644
@@ -507,10 +507,14 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align,
 
 	pr_debug("%s(): returned %p\n", __func__, page);
 out:
-	if (page)
+	if (page) {
 		count_vm_event(CMA_ALLOC_SUCCESS);
-	else
+		cma_sysfs_account_success_pages(cma, count);
+	} else {
 		count_vm_event(CMA_ALLOC_FAIL);
+		if (cma)
+			cma_sysfs_account_fail_pages(cma, count);
+	}
 
 	return page;
 }
mm/cma.h +23 -0
index 42ae082cb067..37b9b7858c8e 100644
@@ -3,6 +3,12 @@
 #define __MM_CMA_H__
 
 #include <linux/debugfs.h>
+#include <linux/kobject.h>
+
+struct cma_kobject {
+	struct cma *cma;
+	struct kobject kobj;
+};
 
 struct cma {
 	unsigned long   base_pfn;
@@ -16,6 +22,14 @@ struct cma {
 	struct debugfs_u32_array dfs_bitmap;
 #endif
 	char name[CMA_MAX_NAME];
+#ifdef CONFIG_CMA_SYSFS
+	/* the number of CMA page successful allocations */
+	atomic64_t nr_pages_succeeded;
+	/* the number of CMA page allocation failures */
+	atomic64_t nr_pages_failed;
+	/* kobject requires dynamic object */
+	struct cma_kobject *cma_kobj;
+#endif
 };
 
 extern struct cma cma_areas[MAX_CMA_AREAS];
@@ -26,4 +40,13 @@ static inline unsigned long cma_bitmap_maxno(struct cma *cma)
 	return cma->count >> cma->order_per_bit;
 }
 
+#ifdef CONFIG_CMA_SYSFS
+void cma_sysfs_account_success_pages(struct cma *cma, unsigned long nr_pages);
+void cma_sysfs_account_fail_pages(struct cma *cma, unsigned long nr_pages);
+#else
+static inline void cma_sysfs_account_success_pages(struct cma *cma,
+						   unsigned long nr_pages) {};
+static inline void cma_sysfs_account_fail_pages(struct cma *cma,
+						unsigned long nr_pages) {};
+#endif
 #endif
mm/cma_sysfs.c new file +116 -0

116 lines not on this page · raw message

Signature · 1 line
2.31.0.291.g576ba9dcdaf-goog
Reviewed-by Tested-by on v7 1/1
#1 lore raw

24.03.2021 23:55, Minchan Kim пишет:

Quoted 18 lines

Since CMA is getting used more widely, it's more important to keep monitoring CMA statistics for system health since it's directly related to user experience.

This patch introduces sysfs statistics for CMA, in order to provide some basic monitoring of the CMA allocator.

* the number of CMA page successful allocations
* the number of CMA page allocation failures

These two values allow the user to calcuate the allocation failure rate for each CMA area.

e.g.)
/sys/kernel/mm/cma/WIFI/alloc_pages_[success|fail] /sys/kernel/mm/cma/SENSOR/alloc_pages_[success|fail] /sys/kernel/mm/cma/BLUETOOTH/alloc_pages_[success|fail]

Quoted 7 lines

The cma_stat was intentionally allocated by dynamic allocation to harmonize with kobject lifetime management.
https://lore.kernel.org/linux-mm/[email protected]/

The tags are incorrect, I haven't suggested this change.

Quoted 6 lines

Addresses-Coverity: ("Dereference after null check")

There are no dereferences fixed by this patch.

Quoted 3 lines

---

...

mm/cma.h line 10

Quoted 6 lines

#include <linux/debugfs.h>

mm/cma.h +5 -0
+#include <linux/kobject.h>
+
+struct cma_kobject {
+	struct cma *cma;
+	struct kobject kobj;

If you'll place the kobj as the first member of the struct, then container_of will be a no-op.

...

Quoted 8 lines
mm/cma_sysfs.c new file +8 -0
+#include <linux/cma.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "cma.h"
+
+void cma_sysfs_account_success_pages(struct cma *cma, unsigned long nr_pages)
+{

mm/cma_sysfs.c line 25

Quoted 10 lines
mm/cma_sysfs.c new file +10 -0
+	atomic64_add(nr_pages, &cma->nr_pages_succeeded);
+}
+
+void cma_sysfs_account_fail_pages(struct cma *cma, unsigned long nr_pages)
+{
+	atomic64_add(nr_pages, &cma->nr_pages_failed);
+}
+
+#define CMA_ATTR_RO(_name) \
+	static struct kobj_attribute _name##_attr = __ATTR_RO(_name)

nit: #defines and inlined helpers typically are placed at the top of the code, after includes.

mm/cma_sysfs.c line 33

Quoted 7 lines
mm/cma_sysfs.c new file +7 -0
+static inline struct cma *cma_from_kobj(struct kobject *kobj)
+{
+	struct cma_kobject *cma_kobj = container_of(kobj, struct cma_kobject,
+						    kobj);
+	struct cma *cma = cma_kobj->cma;
+
+	return cma;

nit: you can write this as:

return container_of(kobj, struct cma_kobject, kobj)->cma;

mm/cma_sysfs.c line 42

Quoted 9 lines
mm/cma_sysfs.c new file +9 -0
+}
+
+static ssize_t alloc_pages_success_show(struct kobject *kobj,
+					struct kobj_attribute *attr, char *buf)
+{
+	struct cma *cma = cma_from_kobj(kobj);
+
+	return sysfs_emit(buf, "%llu\n",
+			atomic64_read(&cma->nr_pages_succeeded));

nit: Algnment isn't right, should be better to write it as single line, IMO.

...

Quoted 13 lines
mm/cma_sysfs.c new file +13 -0
+static int __init cma_sysfs_init(void)
+{
+	struct kobject *cma_kobj_root;
+	struct cma_kobject *cma_kobj;
+	struct cma *cma;
+	int i, err;
+
+	cma_kobj_root = kobject_create_and_add("cma", mm_kobj);
+	if (!cma_kobj_root)
+		return -ENOMEM;
+
+	for (i = 0; i < cma_area_count; i++) {
+		cma_kobj = kzalloc(sizeof(*cma_kobj), GFP_KERNEL);

mm/cma_sysfs.c line 99

Quoted 10 lines
mm/cma_sysfs.c new file +10 -0
+		if (!cma_kobj) {
+			err = -ENOMEM;
+			goto out;
+		}
+
+		cma = &cma_areas[i];
+		cma->cma_kobj = cma_kobj;
+		cma_kobj->cma = cma;
+		err = kobject_init_and_add(&cma_kobj->kobj, &cma_ktype,
+				cma_kobj_root, "%s", cma->name);

nit: Previousy algnment of the code was better here.

Otherwise this is okay to me:

#3 lore raw
In reply to Dmitry Osipenko
Quoted 23 lines · Dmitry Osipenko

24.03.2021 23:55, Minchan Kim пишет:

Quoted 22 lines

Since CMA is getting used more widely, it's more important to keep monitoring CMA statistics for system health since it's directly related to user experience.

This patch introduces sysfs statistics for CMA, in order to provide some basic monitoring of the CMA allocator.

* the number of CMA page successful allocations
* the number of CMA page allocation failures

These two values allow the user to calcuate the allocation failure rate for each CMA area.

e.g.)
/sys/kernel/mm/cma/WIFI/alloc_pages_[success|fail] /sys/kernel/mm/cma/SENSOR/alloc_pages_[success|fail] /sys/kernel/mm/cma/BLUETOOTH/alloc_pages_[success|fail]

The cma_stat was intentionally allocated by dynamic allocation to harmonize with kobject lifetime management.
https://lore.kernel.org/linux-mm/[email protected]/

Quoted 5 lines
Quoted 3 lines

The tags are incorrect, I haven't suggested this change.

During the development, you have suggested many things to make it clean. That suggested-by couldn't represent all the detail but wanted to give credit for you, too since you spent the time to make it better.

Okay, since you didn't like it, I will remove it.

Quoted 9 lines
Quoted 7 lines

Addresses-Coverity: ("Dereference after null check")

There are no dereferences fixed by this patch.

Let me add this:
https://lore.kernel.org/linux-mm/[email protected]/

Quoted 6 lines
Quoted 4 lines

---

...

Quoted 9 lines

mm/cma.h line 10

Quoted 6 lines

#include <linux/debugfs.h>

mm/cma.h +5 -0
+#include <linux/kobject.h>
+
+struct cma_kobject {
+	struct cma *cma;
+	struct kobject kobj;

If you'll place the kobj as the first member of the struct, then container_of will be a no-op.

Cool.

Quoted 15 lines

...

Quoted 13 lines
mm/cma_sysfs.c new file +13 -0
+#include <linux/cma.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "cma.h"
+
+void cma_sysfs_account_success_pages(struct cma *cma, unsigned long nr_pages)
+{
+	atomic64_add(nr_pages, &cma->nr_pages_succeeded);
+}
+
+void cma_sysfs_account_fail_pages(struct cma *cma, unsigned long nr_pages)
+{
Quoted 8 lines

mm/cma_sysfs.c line 25

Quoted 5 lines
mm/cma_sysfs.c new file +5 -0
+	atomic64_add(nr_pages, &cma->nr_pages_failed);
+}
+
+#define CMA_ATTR_RO(_name) \
+	static struct kobj_attribute _name##_attr = __ATTR_RO(_name)

nit: #defines and inlined helpers typically are placed at the top of the code, after includes.

No problem since I should resend anyway.

Quoted 11 lines

mm/cma_sysfs.c line 33

Quoted 8 lines
mm/cma_sysfs.c new file +7 -0
+static inline struct cma *cma_from_kobj(struct kobject *kobj)
+{
+	struct cma_kobject *cma_kobj = container_of(kobj, struct cma_kobject,
+						    kobj);
+	struct cma *cma = cma_kobj->cma;
+
+	return cma;

nit: you can write this as:

Quoted 1 line

return container_of(kobj, struct cma_kobject, kobj)->cma;

Better.

Quoted 12 lines

mm/cma_sysfs.c line 42

Quoted 10 lines
mm/cma_sysfs.c new file +9 -0
+}
+
+static ssize_t alloc_pages_success_show(struct kobject *kobj,
+					struct kobj_attribute *attr, char *buf)
+{
+	struct cma *cma = cma_from_kobj(kobj);
+
+	return sysfs_emit(buf, "%llu\n",
+			atomic64_read(&cma->nr_pages_succeeded));

nit: Algnment isn't right, should be better to write it as single line, IMO.

Let me make it align rather than single line since I know someone who keep asking us to not overflow 80 columns unless it's special.

Quoted 19 lines

...

Quoted 17 lines
mm/cma_sysfs.c new file +17 -0
+static int __init cma_sysfs_init(void)
+{
+	struct kobject *cma_kobj_root;
+	struct cma_kobject *cma_kobj;
+	struct cma *cma;
+	int i, err;
+
+	cma_kobj_root = kobject_create_and_add("cma", mm_kobj);
+	if (!cma_kobj_root)
+		return -ENOMEM;
+
+	for (i = 0; i < cma_area_count; i++) {
+		cma_kobj = kzalloc(sizeof(*cma_kobj), GFP_KERNEL);
+		if (!cma_kobj) {
+			err = -ENOMEM;
+			goto out;
+		}
Quoted 8 lines

mm/cma_sysfs.c line 99

Quoted 6 lines
mm/cma_sysfs.c new file +6 -0
+
+		cma = &cma_areas[i];
+		cma->cma_kobj = cma_kobj;
+		cma_kobj->cma = cma;
+		err = kobject_init_and_add(&cma_kobj->kobj, &cma_ktype,
+				cma_kobj_root, "%s", cma->name);

nit: Previousy algnment of the code was better here.

Yub.

Quoted 5 lines

Otherwise this is okay to me:

Thanks!

#5 lore raw

25.03.2021 01:10, Dmitry Osipenko пишет:

Quoted 10 lines

25.03.2021 00:55, Minchan Kim пишет:

Quoted 4 lines
Quoted 1 line

There are no dereferences fixed by this patch.

Let me add this:
https://lore.kernel.org/linux-mm/[email protected]/

The tag is invalid now, since you squashed the fix. I think you may add "Co-developed-by: Colin Ian King <[email protected]>", but this also should require to add the s-b from Colin, if he doesn't mind.

Ah, I now see that the s-b tag is already there.

#6 lore raw
In reply to Minchan Kim

25.03.2021 00:55, Minchan Kim пишет:

Quoted 11 lines
Quoted 8 lines

mm/cma_sysfs.c line 42

Quoted 7 lines
mm/cma_sysfs.c new file +7 -0
+static ssize_t alloc_pages_success_show(struct kobject *kobj,
+					struct kobj_attribute *attr, char *buf)
+{
+	struct cma *cma = cma_from_kobj(kobj);
+
+	return sysfs_emit(buf, "%llu\n",
+			atomic64_read(&cma->nr_pages_succeeded));

nit: Algnment isn't right, should be better to write it as single line, IMO.

Let me make it align rather than single line since I know someone who keep asking us to not overflow 80 columns unless it's special.

I'm actually one of those guys who complains about 80 chars (where necessary), but in this particular case it only hurts readability of the code, IMO.

#7 lore raw
In reply to Minchan Kim

25.03.2021 00:55, Minchan Kim пишет:

Quoted 8 lines
Quoted 1 line

The tags are incorrect, I haven't suggested this change.

During the development, you have suggested many things to make it clean. That suggested-by couldn't represent all the detail but wanted to give credit for you, too since you spent the time to make it better.

Okay, since you didn't like it, I will remove it.

That's what the r-b tag is for, the suggested-by is about suggesting the patch in general.

Looking forward to the final v8, thank you for yours efforts!

#8 lore raw
In reply to Dmitry Osipenko
Quoted 10 lines · Dmitry Osipenko

...

mm/cma.h line 10

Quoted 5 lines
mm/cma.h +5 -0
+#include <linux/kobject.h>
+
+struct cma_kobject {
+	struct cma *cma;
+	struct kobject kobj;

If you'll place the kobj as the first member of the struct, then container_of will be a no-op.

However, *this does not matter*. Let's not get carried away. If container_of() ends up as a compile-time addition of +8, instead of +0, there is not going to be a visible effect in the world. Or do you have some perf data to the contrary?

Sometimes these kinds of things matter. But other times, they are just pointless to fret about, and this is once such case.

thanks,

Signature · 2 lines
John Hubbard
NVIDIA
#9 lore raw

25.03.2021 01:01, John Hubbard пишет:

Quoted 12 lines · Dmitry Osipenko
Quoted 10 lines

...

mm/cma.h line 10

Quoted 5 lines
mm/cma.h +5 -0
+#include <linux/kobject.h>
+
+struct cma_kobject {
+    struct cma *cma;
+    struct kobject kobj;

If you'll place the kobj as the first member of the struct, then container_of will be a no-op.

Quoted 7 lines

However, *this does not matter*. Let's not get carried away. If container_of() ends up as a compile-time addition of +8, instead of +0, there is not going to be a visible effect in the world. Or do you have some perf data to the contrary?

Sometimes these kinds of things matter. But other times, they are just pointless to fret about, and this is once such case.

Performance is out of question here, my main point is about maintaining a good coding style. Otherwise there is no point in not embedding kobj into cma struct as well, IMO.

#10 lore raw
Quoted 18 lines · Dmitry Osipenko

25.03.2021 01:01, John Hubbard пишет:

Quoted 17 lines · Dmitry Osipenko
Quoted 10 lines

...

mm/cma.h line 10

Quoted 5 lines
mm/cma.h +5 -0
+#include <linux/kobject.h>
+
+struct cma_kobject {
+    struct cma *cma;
+    struct kobject kobj;

If you'll place the kobj as the first member of the struct, then container_of will be a no-op.

However, *this does not matter*. Let's not get carried away. If container_of() ends up as a compile-time addition of +8, instead of +0, there is not going to be a visible effect in the world. Or do you have some perf data to the contrary?

Quoted 4 lines
Quoted 2 lines

Sometimes these kinds of things matter. But other times, they are just pointless to fret about, and this is once such case.

Performance is out of question here, my main point is about maintaining

In that case, there is even less reason to harass people about the order of members of a struct.

Quoted 2 lines

a good coding style. Otherwise there is no point in not embedding kobj into cma struct as well, IMO.

We really don't need to worry about the order of members in a struct, from a "coding style" point of view. It is a solid step too far.

Sorry if that sounds a little too direct. But this review has tended to go quite too far into nitpicks that are normally left as-is, and I've merely picked one that is particularly questionable. I realize that other coding communities have their own standards. Here, I'm explaining what I have observed about linux-mm and linux-kernel, which needs to be respected.

thanks,

Signature · 2 lines
John Hubbard
NVIDIA
#11 lore raw

25.03.2021 01:23, John Hubbard пишет:

Quoted 34 lines · Dmitry Osipenko
Quoted 18 lines

25.03.2021 01:01, John Hubbard пишет:

Quoted 17 lines · Dmitry Osipenko
Quoted 10 lines

...

mm/cma.h line 10

Quoted 5 lines
mm/cma.h +5 -0
+#include <linux/kobject.h>
+
+struct cma_kobject {
+    struct cma *cma;
+    struct kobject kobj;

If you'll place the kobj as the first member of the struct, then container_of will be a no-op.

However, *this does not matter*. Let's not get carried away. If container_of() ends up as a compile-time addition of +8, instead of +0, there is not going to be a visible effect in the world. Or do you have some perf data to the contrary?

Quoted 4 lines
Quoted 2 lines

Sometimes these kinds of things matter. But other times, they are just pointless to fret about, and this is once such case.

Performance is out of question here, my main point is about maintaining

In that case, there is even less reason to harass people about the order of members of a struct.

Quoted 2 lines

a good coding style. Otherwise there is no point in not embedding kobj into cma struct as well, IMO.

We really don't need to worry about the order of members in a struct, from a "coding style" point of view. It is a solid step too far.

Quoted 6 lines

Sorry if that sounds a little too direct. But this review has tended to go quite too far into nitpicks that are normally left as-is, and I've merely picked one that is particularly questionable. I realize that other coding communities have their own standards. Here, I'm explaining what I have observed about linux-mm and linux-kernel, which needs to be respected.

I tried to help as much as I could, sorry if this felt annoying to you or anyone else.

I assume that linux-mm maintainers, like any other maintainers, should skip all suggestions that are deemed as inappropriate to them.