Skip to contentSkip to conversation

Add decoder for GFP masks to tools.

PATCH In review Cody P Schafer 2 participants 5 messages Jan 8, 2013 – Jan 10, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Add decoder for GFP masks to tools.

I needed to decode some gfp_masks to debug an oom-killer invocation, and wrote this tool to avoid doing the decoding manually.

Bad things about this: slightly hacky use of code intended for use with ftrace, splitting linux/gfp.h into 2 parts.

Good things: No additional places need modification to keep the decoder up to date, no mistakes from manual gfp_mask decoding.

 include/linux/gfp-flags.h     | 143 +++++++++++++++++++++++++++++++++++++++++
 include/linux/gfp.h           | 144 +-----------------------------------------
 tools/gfp-decode/.gitignore   |   1 +
 tools/gfp-decode/Makefile     |  11 ++++
 tools/gfp-decode/gfp-decode.c |  89 ++++++++++++++++++++++++++
 5 files changed, 245 insertions(+), 143 deletions(-)

---

On the topic of writing Makefiles for tools/*, it would be extremely convienient if a system was setup which allowed simple specification of final executable(s) and the objects to compose them rather than having each tool write a custom Makefile.

For example (potentially):

TARGETS = gfp-decode
obj-gfp-decode = gfp-decode.o
include ../scripts/Makefile.magic

#1 lore raw

PATCH 1/2 gfp: split out defines for gfp flags into seperate header

Split the current linux/gfp.h into gfp.h and gfp-flags.h
All the defines for flags are placed into gfp-flags.h which is included from gfp.h

This split allows the use of the defines by userspace tools, such as a gfp decoder.

---
 include/linux/gfp-flags.h | 143 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/gfp.h       | 144 +---------------------------------------------
 2 files changed, 144 insertions(+), 143 deletions(-)
 create mode 100644 include/linux/gfp-flags.h
include/linux/gfp-flags.h new file +143 -0
new file mode 100644
index 0000000..572e4c0
@@ -0,0 +1,143 @@
+/* Plain integer GFP bitmasks. Do not use this directly. */
+#define ___GFP_DMA		0x01u
+#define ___GFP_HIGHMEM		0x02u
+#define ___GFP_DMA32		0x04u
+#define ___GFP_MOVABLE		0x08u
+#define ___GFP_WAIT		0x10u
+#define ___GFP_HIGH		0x20u
+#define ___GFP_IO		0x40u
+#define ___GFP_FS		0x80u
+#define ___GFP_COLD		0x100u
+#define ___GFP_NOWARN		0x200u
+#define ___GFP_REPEAT		0x400u
+#define ___GFP_NOFAIL		0x800u
+#define ___GFP_NORETRY		0x1000u
+#define ___GFP_MEMALLOC		0x2000u
+#define ___GFP_COMP		0x4000u
+#define ___GFP_ZERO		0x8000u
+#define ___GFP_NOMEMALLOC	0x10000u
+#define ___GFP_HARDWALL		0x20000u
+#define ___GFP_THISNODE		0x40000u
+#define ___GFP_RECLAIMABLE	0x80000u
+#define ___GFP_KMEMCG		0x100000u
+#define ___GFP_NOTRACK		0x200000u
+#define ___GFP_NO_KSWAPD	0x400000u
+#define ___GFP_OTHER_NODE	0x800000u
+#define ___GFP_WRITE		0x1000000u
+/* If the above are modified, __GFP_BITS_SHIFT may need updating */
+
+/*
+ * GFP bitmasks..
+ *
+ * Zone modifiers (see linux/mmzone.h - low three bits)
+ *
+ * Do not put any conditional on these. If necessary modify the definitions
+ * without the underscores and use them consistently. The definitions here may
+ * be used in bit comparisons.
+ */
+#define __GFP_DMA	((__force gfp_t)___GFP_DMA)
+#define __GFP_HIGHMEM	((__force gfp_t)___GFP_HIGHMEM)
+#define __GFP_DMA32	((__force gfp_t)___GFP_DMA32)
+#define __GFP_MOVABLE	((__force gfp_t)___GFP_MOVABLE)  /* Page is movable */
+#define GFP_ZONEMASK	(__GFP_DMA|__GFP_HIGHMEM|__GFP_DMA32|__GFP_MOVABLE)
+/*
+ * Action modifiers - doesn't change the zoning
+ *
+ * __GFP_REPEAT: Try hard to allocate the memory, but the allocation attempt
+ * _might_ fail.  This depends upon the particular VM implementation.
+ *
+ * __GFP_NOFAIL: The VM implementation _must_ retry infinitely: the caller
+ * cannot handle allocation failures.  This modifier is deprecated and no new
+ * users should be added.
+ *
+ * __GFP_NORETRY: The VM implementation must not retry indefinitely.
+ *
+ * __GFP_MOVABLE: Flag that this page will be movable by the page migration
+ * mechanism or reclaimed
+ */
+#define __GFP_WAIT	((__force gfp_t)___GFP_WAIT)	/* Can wait and reschedule? */
+#define __GFP_HIGH	((__force gfp_t)___GFP_HIGH)	/* Should access emergency pools? */
+#define __GFP_IO	((__force gfp_t)___GFP_IO)	/* Can start physical IO? */
+#define __GFP_FS	((__force gfp_t)___GFP_FS)	/* Can call down to low-level FS? */
+#define __GFP_COLD	((__force gfp_t)___GFP_COLD)	/* Cache-cold page required */
+#define __GFP_NOWARN	((__force gfp_t)___GFP_NOWARN)	/* Suppress page allocation failure warning */
+#define __GFP_REPEAT	((__force gfp_t)___GFP_REPEAT)	/* See above */
+#define __GFP_NOFAIL	((__force gfp_t)___GFP_NOFAIL)	/* See above */
+#define __GFP_NORETRY	((__force gfp_t)___GFP_NORETRY) /* See above */
+#define __GFP_MEMALLOC	((__force gfp_t)___GFP_MEMALLOC)/* Allow access to emergency reserves */
+#define __GFP_COMP	((__force gfp_t)___GFP_COMP)	/* Add compound page metadata */
+#define __GFP_ZERO	((__force gfp_t)___GFP_ZERO)	/* Return zeroed page on success */
+#define __GFP_NOMEMALLOC ((__force gfp_t)___GFP_NOMEMALLOC) /* Don't use emergency reserves.
+							 * This takes precedence over the
+							 * __GFP_MEMALLOC flag if both are
+							 * set
+							 */
+#define __GFP_HARDWALL   ((__force gfp_t)___GFP_HARDWALL) /* Enforce hardwall cpuset memory allocs */
+#define __GFP_THISNODE	((__force gfp_t)___GFP_THISNODE)/* No fallback, no policies */
+#define __GFP_RECLAIMABLE ((__force gfp_t)___GFP_RECLAIMABLE) /* Page is reclaimable */
+#define __GFP_NOTRACK	((__force gfp_t)___GFP_NOTRACK)  /* Don't track with kmemcheck */
+
+#define __GFP_NO_KSWAPD	((__force gfp_t)___GFP_NO_KSWAPD)
+#define __GFP_OTHER_NODE ((__force gfp_t)___GFP_OTHER_NODE) /* On behalf of other node */
+#define __GFP_KMEMCG	((__force gfp_t)___GFP_KMEMCG) /* Allocation comes from a memcg-accounted resource */
+#define __GFP_WRITE	((__force gfp_t)___GFP_WRITE)	/* Allocator intends to dirty page */
+
+/*
+ * This may seem redundant, but it's a way of annotating false positives vs.
+ * allocations that simply cannot be supported (e.g. page tables).
+ */
+#define __GFP_NOTRACK_FALSE_POSITIVE (__GFP_NOTRACK)
+
+#define __GFP_BITS_SHIFT 25	/* Room for N __GFP_FOO bits */
+#define __GFP_BITS_MASK ((__force gfp_t)((1 << __GFP_BITS_SHIFT) - 1))
+
+/* This equals 0, but use constants in case they ever change */
+#define GFP_NOWAIT	(GFP_ATOMIC & ~__GFP_HIGH)
+/* GFP_ATOMIC means both !wait (__GFP_WAIT not set) and use emergency pool */
+#define GFP_ATOMIC	(__GFP_HIGH)
+#define GFP_NOIO	(__GFP_WAIT)
+#define GFP_NOFS	(__GFP_WAIT | __GFP_IO)
+#define GFP_KERNEL	(__GFP_WAIT | __GFP_IO | __GFP_FS)
+#define GFP_TEMPORARY	(__GFP_WAIT | __GFP_IO | __GFP_FS | \
+			 __GFP_RECLAIMABLE)
+#define GFP_USER	(__GFP_WAIT | __GFP_IO | __GFP_FS | __GFP_HARDWALL)
+#define GFP_HIGHUSER	(__GFP_WAIT | __GFP_IO | __GFP_FS | __GFP_HARDWALL | \
+			 __GFP_HIGHMEM)
+#define GFP_HIGHUSER_MOVABLE	(__GFP_WAIT | __GFP_IO | __GFP_FS | \
+				 __GFP_HARDWALL | __GFP_HIGHMEM | \
+				 __GFP_MOVABLE)
+#define GFP_IOFS	(__GFP_IO | __GFP_FS)
+#define GFP_TRANSHUGE	(GFP_HIGHUSER_MOVABLE | __GFP_COMP | \
+			 __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN | \
+			 __GFP_NO_KSWAPD)
+
+#ifdef CONFIG_NUMA
+#define GFP_THISNODE	(__GFP_THISNODE | __GFP_NOWARN | __GFP_NORETRY)
+#else
+#define GFP_THISNODE	((__force gfp_t)0)
+#endif
+
+/* This mask makes up all the page movable related flags */
+#define GFP_MOVABLE_MASK (__GFP_RECLAIMABLE|__GFP_MOVABLE)
+
+/* Control page allocator reclaim behavior */
+#define GFP_RECLAIM_MASK (__GFP_WAIT|__GFP_HIGH|__GFP_IO|__GFP_FS|\
+			__GFP_NOWARN|__GFP_REPEAT|__GFP_NOFAIL|\
+			__GFP_NORETRY|__GFP_MEMALLOC|__GFP_NOMEMALLOC)
+
+/* Control slab gfp mask during early boot */
+#define GFP_BOOT_MASK (__GFP_BITS_MASK & ~(__GFP_WAIT|__GFP_IO|__GFP_FS))
+
+/* Control allocation constraints */
+#define GFP_CONSTRAINT_MASK (__GFP_HARDWALL|__GFP_THISNODE)
+
+/* Do not use these with a slab allocator */
+#define GFP_SLAB_BUG_MASK (__GFP_DMA32|__GFP_HIGHMEM|~__GFP_BITS_MASK)
+
+/* Flag - indicates that the buffer will be suitable for DMA.  Ignored on some
+   platforms, used as appropriate on others */
+
+#define GFP_DMA		__GFP_DMA
+
+/* 4GB DMA on some platforms */
+#define GFP_DMA32	__GFP_DMA32
include/linux/gfp.h +1 -143

150 lines not on this page · raw message

Signature · 1 line
1.8.0.3
#2 lore raw

PATCH 2/2 tools: add gfp mask decoder

In reply to Cody P Schafer

From: Cody P Schafer <[email protected]>

Allows decoding of gfp_masks printed, for example, when the oom-killer is invoked.

Example usage:

	[    6.464753] kthreadd invoked oom-killer: gfp_mask=0x1000d0, order=1, oom_score_adj=0
	$ ./gfp-decode 0x1000d0
	GFP_KERNEL|GFP_KMEMCG

Internally, it reuses some of the ftrace structure for decoding the gfp_mask.

---
 tools/gfp-decode/.gitignore   |  1 +
 tools/gfp-decode/Makefile     | 11 ++++++
 tools/gfp-decode/gfp-decode.c | 89 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 101 insertions(+)
 create mode 100644 tools/gfp-decode/.gitignore
 create mode 100644 tools/gfp-decode/Makefile
 create mode 100644 tools/gfp-decode/gfp-decode.c
tools/gfp-decode/.gitignore new file +1 -0
new file mode 100644
index 0000000..eca2466
@@ -0,0 +1 @@
+/gfp-decode
tools/gfp-decode/Makefile new file +11 -0
new file mode 100644
index 0000000..51f375a
@@ -0,0 +1,11 @@
+include ../scripts/Makefile.include
+
+all: $(OUTPUT)gfp-decode
+
+CC = $(CROSS_COMPILE)gcc
+ALL_CFLAGS = $(CFLAGS) $(EXTRA_WARNINGS) -Wall -Wextra -I../../include
+
+$(OUTPUT)gfp-decode: gfp-decode.c
+	$(QUIET_CC)$(CC) -o $@ $(ALL_CFLAGS) $(LDFLAGS) $<
+clean:
+	$(RM) $(OUTPUT)gfp-decode
tools/gfp-decode/gfp-decode.c new file +89 -0

89 lines not on this page · raw message

Signature · 1 line
1.8.0.3
#3 lore raw
In reply to Cody P Schafer
Quoted 9 lines · Cody P Schafer

I needed to decode some gfp_masks to debug an oom-killer invocation, and wrote this tool to avoid doing the decoding manually.

Bad things about this: slightly hacky use of code intended for use with ftrace, splitting linux/gfp.h into 2 parts.

Good things: No additional places need modification to keep the decoder up to date, no mistakes from manual gfp_mask decoding.

Any particular reason you did not use scripts/gfp-translate?

$ bash ./scripts/gfp-translate 0x1000d0
Source: /home/mel/git-public/linux-2.6
Parsing: 0x1000d0
#define ___GFP_WAIT		0x10
#define ___GFP_IO		0x40
#define ___GFP_FS		0x80
#define ___GFP_KMEMCG		0x100000

Script is dumb as rocks, requires access to the source code and does not translate flag combinations into something like GFP_KERNEL but it's usually sufficient. I'm not pushed either way but if you want to push your tool then the patch should also delete scripts/gfp-translate.

Signature · 2 lines
Mel Gorman
SUSE Labs
#4 lore raw

(Sorry Mel, you're getting this twice because I forgot CC's last time)

Quoted 1 line · Mel Gorman

Any particular reason you did not use scripts/gfp-translate?

Because I didn't look hard enough & didn't see it :(

Quoted 4 lines

Script is dumb as rocks, requires access to the source code and does not translate flag combinations into something like GFP_KERNEL but it's usually sufficient. I'm not pushed either way but if you want to push your tool then the patch should also delete scripts/gfp-translate.

No preference. Since we have gfp-translate already in tree, it makes sense just to keep using it.