Skip to contentSkip to conversation

mm: vmscan: Reduce throttling due to a failure to make progress

PATCH Superseded Mel Gorman 2 participants 3 messages Dec 2, 2021

Subsystems (7)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Mel Gorman

author
#0 lore raw

PATCH 1/1 mm: vmscan: Reduce throttling due to a failure to make progress

Mike Galbraith, Alexey Avramov and Darrick Wong all reported similar problems due to reclaim throttling for excessive lengths of time. In Alexey's case, a memory hog that should go OOM quickly stalls for several minutes before stalling. In Mike and Darrick's cases, a small memcg environment stalled excessively even though the system had enough memory overall.

Commit 69392a403f49 ("mm/vmscan: throttle reclaim when no progress is being made") introduced the problem although commit a19594ca4a8b ("mm/vmscan:
increase the timeout if page reclaim is not making progress") made it worse. Systems at or near an OOM state that cannot be recovered must reach OOM quickly and memcg should kill tasks if a memcg is near OOM.

To address this, only stall for the first zone in the zonelist, reduce the timeout to 1 tick for VMSCAN_THROTTLE_NOPROGRESS and only stall if the scan control nr_reclaimed is 0, kswapd is still active and there were excessive pages pending for writeback. If kswapd has stopped reclaiming due to excessive failures, do not stall at all so that OOM triggers relatively quickly. Similarly, if an LRU is simply congested, only lightly throttle similar to NOPROGRESS.

Alexey's original case was the most straight forward

for i in {1..3}; do tail /dev/zero; done

On vanilla 5.16-rc1, this test stalled heavily, after the patch the test completes in a few seconds similar to 5.15.

Alexey's second test case added watching a youtube video while tail runs 10 times. On 5.15, playback only jitters slightly, 5.16-rc1 stalls a lot with lots of frames missing and numerous audio glitches. With this patch applies, the video plays similarly to 5.15.

Fixes 69392a403f49 ("mm/vmscan: throttle reclaim when no progress is being made")
---
 include/linux/mmzone.h        |  1 +
 include/trace/events/vmscan.h |  4 ++-
 mm/vmscan.c                   | 64 ++++++++++++++++++++++++++++++-----
 3 files changed, 59 insertions(+), 10 deletions(-)
include/linux/mmzone.h +1 -0
index 58e744b78c2c..936dc0b6c226 100644
@@ -277,6 +277,7 @@ enum vmscan_throttle_state {
 	VMSCAN_THROTTLE_WRITEBACK,
 	VMSCAN_THROTTLE_ISOLATED,
 	VMSCAN_THROTTLE_NOPROGRESS,
+	VMSCAN_THROTTLE_CONGESTED,
 	NR_VMSCAN_THROTTLE,
 };
 
include/trace/events/vmscan.h +3 -1
index f25a6149d3ba..ca2e9009a651 100644
@@ -30,12 +30,14 @@
 #define _VMSCAN_THROTTLE_WRITEBACK	(1 << VMSCAN_THROTTLE_WRITEBACK)
 #define _VMSCAN_THROTTLE_ISOLATED	(1 << VMSCAN_THROTTLE_ISOLATED)
 #define _VMSCAN_THROTTLE_NOPROGRESS	(1 << VMSCAN_THROTTLE_NOPROGRESS)
+#define _VMSCAN_THROTTLE_CONGESTED	(1 << VMSCAN_THROTTLE_CONGESTED)
 
 #define show_throttle_flags(flags)						\
 	(flags) ? __print_flags(flags, "|",					\
 		{_VMSCAN_THROTTLE_WRITEBACK,	"VMSCAN_THROTTLE_WRITEBACK"},	\
 		{_VMSCAN_THROTTLE_ISOLATED,	"VMSCAN_THROTTLE_ISOLATED"},	\
-		{_VMSCAN_THROTTLE_NOPROGRESS,	"VMSCAN_THROTTLE_NOPROGRESS"}	\
+		{_VMSCAN_THROTTLE_NOPROGRESS,	"VMSCAN_THROTTLE_NOPROGRESS"},	\
+		{_VMSCAN_THROTTLE_CONGESTED,	"VMSCAN_THROTTLE_CONGESTED"}	\
 		) : "VMSCAN_THROTTLE_NONE"
 
 
mm/vmscan.c +55 -9

112 lines not on this page · raw message

Signature · 1 line
2.31.1
#1 lore raw
I love your patch! Perhaps something to improve:

Hi Mel,

I love your patch! Perhaps something to improve:

[auto build test WARNING on hnaz-mm/master]

url:    https://github.com/0day-ci/linux/commits/Mel-Gorman/mm-vmscan-Reduce-throttling-due-to-a-failure-to-make-progress/20211202-212004
base:   https://github.com/hnaz/linux-mm master
config: um-i386_defconfig (https://download.01.org/0day-ci/archive/20211202/202112022232.gARZWe0c-lkp(a)intel.com/config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0

reproduce (this is a W=1 build):
# https://github.com/0day-ci/linux/commit/01dada07590ae9c69a9415ba9af96d5ae184d861
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Mel-Gorman/mm-vmscan-Reduce-throttling-due-to-a-failure-to-make-progress/20211202-212004 git checkout 01dada07590ae9c69a9415ba9af96d5ae184d861
# save the config file to linux build tree
mkdir build_dir
make W=1 O=build_dir ARCH=um SUBARCH=i386 SHELL=/bin/bash

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

>> mm/vmscan.c:1024:6: warning: no previous prototype for 'skip_throttle_noprogress' [-Wmissing-prototypes]

1024 | bool skip_throttle_noprogress(pg_data_t *pgdat)
| ^~~~~~~~~~~~~~~~~~~~~~~~

vim +/skip_throttle_noprogress +1024 mm/vmscan.c

1023

> 1024	bool skip_throttle_noprogress(pg_data_t *pgdat)

1025 {
1026 int reclaimable = 0, write_pending = 0;
1027 int i;
1028
1029 /*
1030 * If kswapd is disabled, reschedule if necessary but do not
1031 * throttle as the system is likely near OOM.
1032 */
1033 if (pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES)
1034 return true;
1035
1036 /*
1037 * If there are a lot of dirty/writeback pages then do not
1038 * throttle as throttling will occur when the pages cycle
1039 * towards the end of the LRU if still under writeback.
1040 */
1041 for (i = 0; i < MAX_NR_ZONES; i++) {
1042 struct zone *zone = pgdat->node_zones + i;
1043
1044 if (!populated_zone(zone))
1045 continue;
1046
1047 reclaimable += zone_reclaimable_pages(zone);
1048 write_pending += zone_page_state_snapshot(zone,
1049 NR_ZONE_WRITE_PENDING);
1050 }
1051 if (2 * write_pending <= reclaimable)
1052 return true;
1053
1054 return false;
1055 }
1056

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/[email protected]

#2 lore raw
In reply to Mel Gorman
I love your patch! Perhaps something to improve:

Hi Mel,

I love your patch! Perhaps something to improve:

[auto build test WARNING on hnaz-mm/master]

url:    https://github.com/0day-ci/linux/commits/Mel-Gorman/mm-vmscan-Reduce-throttling-due-to-a-failure-to-make-progress/20211202-212004
base:   https://github.com/hnaz/linux-mm master
config: x86_64-buildonly-randconfig-r001-20211202 (https://download.01.org/0day-ci/archive/20211203/202112030001.HUiErCyK-lkp(a)intel.com/config)
compiler: clang version 14.0.0 (https://github.com/llvm/llvm-project 4b553297ef3ee4dc2119d5429adf3072e90fac38)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/01dada07590ae9c69a9415ba9af96d5ae184d861
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Mel-Gorman/mm-vmscan-Reduce-throttling-due-to-a-failure-to-make-progress/20211202-212004 git checkout 01dada07590ae9c69a9415ba9af96d5ae184d861
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

>> mm/vmscan.c:1024:6: warning: no previous prototype for function 'skip_throttle_noprogress' [-Wmissing-prototypes]
   bool skip_throttle_noprogress(pg_data_t *pgdat)
        ^

mm/vmscan.c:1024:1: note: declare 'static' if the function is not intended to be used outside of this translation unit bool skip_throttle_noprogress(pg_data_t *pgdat)
^
static
1 warning generated.

vim +/skip_throttle_noprogress +1024 mm/vmscan.c

1023

> 1024	bool skip_throttle_noprogress(pg_data_t *pgdat)

1025 {
1026 int reclaimable = 0, write_pending = 0;
1027 int i;
1028
1029 /*
1030 * If kswapd is disabled, reschedule if necessary but do not
1031 * throttle as the system is likely near OOM.
1032 */
1033 if (pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES)
1034 return true;
1035
1036 /*
1037 * If there are a lot of dirty/writeback pages then do not
1038 * throttle as throttling will occur when the pages cycle
1039 * towards the end of the LRU if still under writeback.
1040 */
1041 for (i = 0; i < MAX_NR_ZONES; i++) {
1042 struct zone *zone = pgdat->node_zones + i;
1043
1044 if (!populated_zone(zone))
1045 continue;
1046
1047 reclaimable += zone_reclaimable_pages(zone);
1048 write_pending += zone_page_state_snapshot(zone,
1049 NR_ZONE_WRITE_PENDING);
1050 }
1051 if (2 * write_pending <= reclaimable)
1052 return true;
1053
1054 return false;
1055 }
1056

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/[email protected]