Skip to contentSkip to conversation

mm: reduce /proc/pagetypeinfo ovehead

PATCH In review Michal Hocko 6 participants 10 messages Oct 25, 2019

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover mm: reduce /proc/pagetypeinfo ovehead

Hi,
Waiman Long has reported [1] that reading /proc/pagetypeinfo can severely interfere with the system and it might lead even to hard lockup detector firing up on a very large machines. Nevertheless small machines are not completely fine either because the operation requires to take the zone->lock IRQ safe spinlock and thus to interfere with both the IRQ delivery and the page allocator. The file is world readable which makes this kinda bad.

The immediate danger is addressed by making the file root readable only. This is a debugging aid so general audience shouldn't require it for a general operation. This is done in the first patch.

The potentially excessive time spent for free_list iteration is handled by capping the iteration loop. This should be fine for existing usecases because low numbers are usually of the primary interest. This is implemented in patch 2.

I am reposting these two patches with dropped RFC (previously posted [2]) and asking for inclusion. I have also dropped Mel's Ack from the second patch because there were quite some changes since he reviewed.

[1] http://lkml.kernel.org/r/[email protected]
[2] http://lkml.kernel.org/r/[email protected]

#1 lore raw

PATCH 1/2 mm, vmstat: hide /proc/pagetypeinfo from normal users

From: Michal Hocko <[email protected]>

/proc/pagetypeinfo is a debugging tool to examine internal page allocator state wrt to fragmentation. It is not very useful for any other use so normal users really do not need to read this file.

Waiman Long has noticed that reading this file can have negative side effects because zone->lock is necessary for gathering data and that
a) interferes with the page allocator and its users and b) can lead to hard lockups on large machines which have very long free_list.

Reduce both issues by simply not exporting the file to regular users.

Cc stable
Fixes 467c996c1e19 ("Print out statistics in relation to fragmentation avoidance to /proc/pagetypeinfo")
---
 mm/vmstat.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmstat.c +1 -1
index 6afc892a148a..4e885ecd44d1 100644
@@ -1972,7 +1972,7 @@ void __init init_mm_internals(void)
 #endif
 #ifdef CONFIG_PROC_FS
 	proc_create_seq("buddyinfo", 0444, NULL, &fragmentation_op);
-	proc_create_seq("pagetypeinfo", 0444, NULL, &pagetypeinfo_op);
+	proc_create_seq("pagetypeinfo", 0400, NULL, &pagetypeinfo_op);
 	proc_create_seq("vmstat", 0444, NULL, &vmstat_op);
 	proc_create_seq("zoneinfo", 0444, NULL, &zoneinfo_op);
 #endif
Signature · 1 line
2.20.1
#2 lore raw

Re: [PATCH 1/2] mm, vmstat: hide /proc/pagetypeinfo from normal users

Quoted 11 lines · Michal Hocko

From: Michal Hocko <[email protected]>

/proc/pagetypeinfo is a debugging tool to examine internal page allocator state wrt to fragmentation. It is not very useful for any other use so normal users really do not need to read this file.

Waiman Long has noticed that reading this file can have negative side effects because zone->lock is necessary for gathering data and that
a) interferes with the page allocator and its users and b) can lead to hard lockups on large machines which have very long free_list.

Quoted 4 lines

Reduce both issues by simply not exporting the file to regular users.

Quoted 25 lines
Fixes 467c996c1e19 ("Print out statistics in relation to fragmentation avoidance to /proc/pagetypeinfo")
---
 mm/vmstat.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmstat.c +1 -1
index 6afc892a148a..4e885ecd44d1 100644
@@ -1972,7 +1972,7 @@ void __init init_mm_internals(void)
 #endif
 #ifdef CONFIG_PROC_FS
 	proc_create_seq("buddyinfo", 0444, NULL, &fragmentation_op);
-	proc_create_seq("pagetypeinfo", 0444, NULL, &pagetypeinfo_op);
+	proc_create_seq("pagetypeinfo", 0400, NULL, &pagetypeinfo_op);
 	proc_create_seq("vmstat", 0444, NULL, &vmstat_op);
 	proc_create_seq("zoneinfo", 0444, NULL, &zoneinfo_op);
 #endif
maintainer
#3 lore raw

Re: [PATCH 1/2] mm, vmstat: hide /proc/pagetypeinfo from normal users

Quoted 12 lines · On Fri, 25 Oct 2019 09:33:26 +0200 Vlastimil Babka <[email protected]> wrote:
Quoted 11 lines · Michal Hocko

From: Michal Hocko <[email protected]>

/proc/pagetypeinfo is a debugging tool to examine internal page allocator state wrt to fragmentation. It is not very useful for any other use so normal users really do not need to read this file.

Waiman Long has noticed that reading this file can have negative side effects because zone->lock is necessary for gathering data and that
a) interferes with the page allocator and its users and b) can lead to hard lockups on large machines which have very long free_list.

Quoted 6 lines
Quoted 4 lines

Reduce both issues by simply not exporting the file to regular users.

As we don't really know how much damage this will cause, it would be nice to let it bake in mainline for a month or three before committing it to the -stable trees. But we don't have a process for that, apart from remembering to poke Greg at a suitable date.

Oh well. I guess that if someone is truly harmed by this change, they can just chmod /proc/pagetypeinfo back to 0444.

#4 lore raw

Re: [PATCH 1/2] mm, vmstat: hide /proc/pagetypeinfo from normal users

In reply to Michal Hocko
Quoted 32 lines · Michal Hocko

From: Michal Hocko <[email protected]>

/proc/pagetypeinfo is a debugging tool to examine internal page allocator state wrt to fragmentation. It is not very useful for any other use so normal users really do not need to read this file.

Waiman Long has noticed that reading this file can have negative side effects because zone->lock is necessary for gathering data and that
a) interferes with the page allocator and its users and b) can lead to hard lockups on large machines which have very long free_list.

Reduce both issues by simply not exporting the file to regular users.

Cc stable
Fixes 467c996c1e19 ("Print out statistics in relation to fragmentation avoidance to /proc/pagetypeinfo")
---
  mm/vmstat.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmstat.c +0 -0
index 6afc892a148a..4e885ecd44d1 100644

mm/vmstat.c void __init init_mm_internals(void) · line 1978

Quoted 8 lines
mm/vmstat.c +1 -1
@@ -1972,7 +1972,7 @@ void __init init_mm_internals(void)
  #endif
  #ifdef CONFIG_PROC_FS
  	proc_create_seq("buddyinfo", 0444, NULL, &fragmentation_op);
-	proc_create_seq("pagetypeinfo", 0444, NULL, &pagetypeinfo_op);
+	proc_create_seq("pagetypeinfo", 0400, NULL, &pagetypeinfo_op);
  	proc_create_seq("vmstat", 0444, NULL, &vmstat_op);
  	proc_create_seq("zoneinfo", 0444, NULL, &zoneinfo_op);
  #endif

Looks good too me (the ack list is already long enough :) )

Signature · 3 lines
Thanks,

David / dhildenb
#5 lore raw

PATCH 2/2 mm, vmstat: reduce zone->lock holding time by /proc/pagetypeinfo

In reply to Michal Hocko

From: Michal Hocko <[email protected]>

pagetypeinfo_showfree_print is called by zone->lock held in irq mode. This is not really nice because it blocks both any interrupts on that cpu and the page allocator. On large machines this might even trigger the hard lockup detector.

Considering the pagetypeinfo is a debugging tool we do not really need exact numbers here. The primary reason to look at the outuput is to see how pageblocks are spread among different migratetypes and low number of pages is much more interesting therefore putting a bound on the number of pages on the free_list sounds like a reasonable tradeoff.

The new output will simply tell
[...]
Node 6, zone Normal, type Movable >100000 >100000 >100000 >100000 41019 31560 23996 10054 3229 983 648

instead of
Node 6, zone Normal, type Movable 399568 294127 221558 102119 41019 31560 23996 10054 3229 983 648

The limit has been chosen arbitrary and it is a subject of a future change should there be a need for that.

While we are at it, also drop the zone lock after each free_list iteration which will help with the IRQ and page allocator responsiveness even further as the IRQ lock held time is always bound to those 100k pages.

---
 mm/vmstat.c | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)
mm/vmstat.c +20 -3
index 4e885ecd44d1..ddb89f4e0486 100644
@@ -1383,12 +1383,29 @@ static void pagetypeinfo_showfree_print(struct seq_file *m,
 			unsigned long freecount = 0;
 			struct free_area *area;
 			struct list_head *curr;
+			bool overflow = false;
 
 			area = &(zone->free_area[order]);
 
-			list_for_each(curr, &area->free_list[mtype])
-				freecount++;
-			seq_printf(m, "%6lu ", freecount);
+			list_for_each(curr, &area->free_list[mtype]) {
+				/*
+				 * Cap the free_list iteration because it might
+				 * be really large and we are under a spinlock
+				 * so a long time spent here could trigger a
+				 * hard lockup detector. Anyway this is a
+				 * debugging tool so knowing there is a handful
+				 * of pages in this order should be more than
+				 * sufficient
+				 */
+				if (++freecount >= 100000) {
+					overflow = true;
+					break;
+				}
+			}
+			seq_printf(m, "%s%6lu ", overflow ? ">" : "", freecount);
+			spin_unlock_irq(&zone->lock);
+			cond_resched();
+			spin_lock_irq(&zone->lock);
 		}
 		seq_putc(m, '\n');
 	}
Signature · 1 line
2.20.1
Acked-by on v1 2/2
#6 lore raw

Re: [PATCH 2/2] mm, vmstat: reduce zone->lock holding time by /proc/pagetypeinfo

Quoted 23 lines · Michal Hocko

From: Michal Hocko <[email protected]>

pagetypeinfo_showfree_print is called by zone->lock held in irq mode. This is not really nice because it blocks both any interrupts on that cpu and the page allocator. On large machines this might even trigger the hard lockup detector.

Considering the pagetypeinfo is a debugging tool we do not really need exact numbers here. The primary reason to look at the outuput is to see how pageblocks are spread among different migratetypes and low number of pages is much more interesting therefore putting a bound on the number of pages on the free_list sounds like a reasonable tradeoff.

The new output will simply tell
[...]
Node 6, zone Normal, type Movable >100000 >100000 >100000 >100000 41019 31560 23996 10054 3229 983 648

instead of
Node 6, zone Normal, type Movable 399568 294127 221558 102119 41019 31560 23996 10054 3229 983 648

The limit has been chosen arbitrary and it is a subject of a future change should there be a need for that.

Quoted 8 lines

While we are at it, also drop the zone lock after each free_list iteration which will help with the IRQ and page allocator responsiveness even further as the IRQ lock held time is always bound to those 100k pages.

Quoted 42 lines
---
 mm/vmstat.c | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)
mm/vmstat.c +20 -3
index 4e885ecd44d1..ddb89f4e0486 100644
@@ -1383,12 +1383,29 @@ static void pagetypeinfo_showfree_print(struct seq_file *m,
 			unsigned long freecount = 0;
 			struct free_area *area;
 			struct list_head *curr;
+			bool overflow = false;
 
 			area = &(zone->free_area[order]);
 
-			list_for_each(curr, &area->free_list[mtype])
-				freecount++;
-			seq_printf(m, "%6lu ", freecount);
+			list_for_each(curr, &area->free_list[mtype]) {
+				/*
+				 * Cap the free_list iteration because it might
+				 * be really large and we are under a spinlock
+				 * so a long time spent here could trigger a
+				 * hard lockup detector. Anyway this is a
+				 * debugging tool so knowing there is a handful
+				 * of pages in this order should be more than
+				 * sufficient
+				 */
+				if (++freecount >= 100000) {
+					overflow = true;
+					break;
+				}
+			}
+			seq_printf(m, "%s%6lu ", overflow ? ">" : "", freecount);
+			spin_unlock_irq(&zone->lock);
+			cond_resched();
+			spin_lock_irq(&zone->lock);
 		}
 		seq_putc(m, '\n');
 	}
Acked-by on v1 2/2
#7 lore raw

Re: [PATCH 2/2] mm, vmstat: reduce zone->lock holding time by /proc/pagetypeinfo

In reply to Michal Hocko
Quoted 48 lines · Michal Hocko

From: Michal Hocko <[email protected]>

pagetypeinfo_showfree_print is called by zone->lock held in irq mode. This is not really nice because it blocks both any interrupts on that cpu and the page allocator. On large machines this might even trigger the hard lockup detector.

Considering the pagetypeinfo is a debugging tool we do not really need exact numbers here. The primary reason to look at the outuput is to see how pageblocks are spread among different migratetypes and low number of pages is much more interesting therefore putting a bound on the number of pages on the free_list sounds like a reasonable tradeoff.

The new output will simply tell
[...]
Node 6, zone Normal, type Movable >100000 >100000 >100000 >100000 41019 31560 23996 10054 3229 983 648

instead of
Node 6, zone Normal, type Movable 399568 294127 221558 102119 41019 31560 23996 10054 3229 983 648

The limit has been chosen arbitrary and it is a subject of a future change should there be a need for that.

While we are at it, also drop the zone lock after each free_list iteration which will help with the IRQ and page allocator responsiveness even further as the IRQ lock held time is always bound to those 100k pages.

---
  mm/vmstat.c | 23 ++++++++++++++++++++---
  1 file changed, 20 insertions(+), 3 deletions(-)
mm/vmstat.c +1 -1
index 4e885ecd44d1..ddb89f4e0486 100644
@@ -1383,12 +1383,29 @@ static void pagetypeinfo_showfree_print(struct seq_file *m,
  			unsigned long freecount = 0;
  			struct free_area *area;
  			struct list_head *curr;
+			bool overflow = false;
 
  			area = &(zone->free_area[order]);
 
-			list_for_each(curr, &area->free_list[mtype])

mm/vmstat.c static void pagetypeinfo_showfree_print(struct seq_file *m, · line 1397

Quoted 10 lines
mm/vmstat.c +8 -2
-				freecount++;
-			seq_printf(m, "%6lu ", freecount);
+			list_for_each(curr, &area->free_list[mtype]) {
+				/*
+				 * Cap the free_list iteration because it might
+				 * be really large and we are under a spinlock
+				 * so a long time spent here could trigger a
+				 * hard lockup detector. Anyway this is a
+				 * debugging tool so knowing there is a handful
+				 * of pages in this order should be more than

"of this order" ?

mm/vmstat.c static void pagetypeinfo_showfree_print(struct seq_file *m, · line 1398

Quoted 1 line
mm/vmstat.c +1 -0
+				 * sufficient

s/sufficient"/sufficient." ?

Quoted 4 lines
mm/vmstat.c +3 -0
+				 */
+				if (++freecount >= 100000) {
+					overflow = true;

mm/vmstat.c static void pagetypeinfo_showfree_print(struct seq_file *m, · line 1411

Quoted 10 lines
mm/vmstat.c +7 -0
+					break;
+				}
+			}
+			seq_printf(m, "%s%6lu ", overflow ? ">" : "", freecount);
+			spin_unlock_irq(&zone->lock);
+			cond_resched();
+			spin_lock_irq(&zone->lock);
  		}
  		seq_putc(m, '\n');
  	}
Signature · 3 lines
Thanks,

David / dhildenb
Acked-by on v1 2/2
#8 lore raw

Re: [PATCH 2/2] mm, vmstat: reduce zone->lock holding time by /proc/pagetypeinfo

In reply to Michal Hocko
Quoted 75 lines · Michal Hocko

From: Michal Hocko <[email protected]>

pagetypeinfo_showfree_print is called by zone->lock held in irq mode. This is not really nice because it blocks both any interrupts on that cpu and the page allocator. On large machines this might even trigger the hard lockup detector.

Considering the pagetypeinfo is a debugging tool we do not really need exact numbers here. The primary reason to look at the outuput is to see how pageblocks are spread among different migratetypes and low number of pages is much more interesting therefore putting a bound on the number of pages on the free_list sounds like a reasonable tradeoff.

The new output will simply tell
[...]
Node 6, zone Normal, type Movable >100000 >100000 >100000 >100000 41019 31560 23996 10054 3229 983 648

instead of
Node 6, zone Normal, type Movable 399568 294127 221558 102119 41019 31560 23996 10054 3229 983 648

The limit has been chosen arbitrary and it is a subject of a future change should there be a need for that.

While we are at it, also drop the zone lock after each free_list iteration which will help with the IRQ and page allocator responsiveness even further as the IRQ lock held time is always bound to those 100k pages.

---
 mm/vmstat.c | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)
mm/vmstat.c +20 -3
index 4e885ecd44d1..ddb89f4e0486 100644
@@ -1383,12 +1383,29 @@ static void pagetypeinfo_showfree_print(struct seq_file *m,
 			unsigned long freecount = 0;
 			struct free_area *area;
 			struct list_head *curr;
+			bool overflow = false;
 
 			area = &(zone->free_area[order]);
 
-			list_for_each(curr, &area->free_list[mtype])
-				freecount++;
-			seq_printf(m, "%6lu ", freecount);
+			list_for_each(curr, &area->free_list[mtype]) {
+				/*
+				 * Cap the free_list iteration because it might
+				 * be really large and we are under a spinlock
+				 * so a long time spent here could trigger a
+				 * hard lockup detector. Anyway this is a
+				 * debugging tool so knowing there is a handful
+				 * of pages in this order should be more than
+				 * sufficient
+				 */
+				if (++freecount >= 100000) {
+					overflow = true;
+					break;
+				}
+			}
+			seq_printf(m, "%s%6lu ", overflow ? ">" : "", freecount);
+			spin_unlock_irq(&zone->lock);
+			cond_resched();
+			spin_lock_irq(&zone->lock);
 		}
 		seq_putc(m, '\n');
 	}
Signature · 1 line
2.20.1
Acked-by on v1 2/2
#9 lore raw

Re: [PATCH 2/2] mm, vmstat: reduce zone->lock holding time by /proc/pagetypeinfo

In reply to Michal Hocko
Quoted 23 lines · Michal Hocko

From: Michal Hocko <[email protected]>

pagetypeinfo_showfree_print is called by zone->lock held in irq mode. This is not really nice because it blocks both any interrupts on that cpu and the page allocator. On large machines this might even trigger the hard lockup detector.

Considering the pagetypeinfo is a debugging tool we do not really need exact numbers here. The primary reason to look at the outuput is to see how pageblocks are spread among different migratetypes and low number of pages is much more interesting therefore putting a bound on the number of pages on the free_list sounds like a reasonable tradeoff.

The new output will simply tell
[...]
Node 6, zone Normal, type Movable >100000 >100000 >100000 >100000 41019 31560 23996 10054 3229 983 648

instead of
Node 6, zone Normal, type Movable 399568 294127 221558 102119 41019 31560 23996 10054 3229 983 648

The limit has been chosen arbitrary and it is a subject of a future change should there be a need for that.

Quoted 8 lines

While we are at it, also drop the zone lock after each free_list iteration which will help with the IRQ and page allocator responsiveness even further as the IRQ lock held time is always bound to those 100k pages.

I think 100k is a very reasonable threshold.

Quoted 20 lines
---
 mm/vmstat.c | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)
mm/vmstat.c +2 -3
index 4e885ecd44d1..ddb89f4e0486 100644
@@ -1383,12 +1383,29 @@ static void pagetypeinfo_showfree_print(struct seq_file *m,
 			unsigned long freecount = 0;
 			struct free_area *area;
 			struct list_head *curr;
+			bool overflow = false;
 
 			area = &(zone->free_area[order]);
 
-			list_for_each(curr, &area->free_list[mtype])
-				freecount++;
-			seq_printf(m, "%6lu ", freecount);
+			list_for_each(curr, &area->free_list[mtype]) {

mm/vmstat.c static void pagetypeinfo_showfree_print(struct seq_file *m, · line 1400

Quoted 10 lines
mm/vmstat.c +10 -0
+				/*
+				 * Cap the free_list iteration because it might
+				 * be really large and we are under a spinlock
+				 * so a long time spent here could trigger a
+				 * hard lockup detector. Anyway this is a
+				 * debugging tool so knowing there is a handful
+				 * of pages in this order should be more than
+				 * sufficient
+				 */
+				if (++freecount >= 100000) {

I suppose it's most precise to check freecount > 1000000 to print >100000, but I doubt anybody cares :)

Quoted 11 lines
mm/vmstat.c +8 -0
+					overflow = true;
+					break;
+				}
+			}
+			seq_printf(m, "%s%6lu ", overflow ? ">" : "", freecount);
+			spin_unlock_irq(&zone->lock);
+			cond_resched();
+			spin_lock_irq(&zone->lock);
 		}
 		seq_putc(m, '\n');
 	}