Skip to contentSkip to conversation

mm: Require LRU reclaim progress before retrying direct reclaim

PATCH In review Matt Fleming 5 participants 13 messages Apr 10 – Apr 24

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm: Require LRU reclaim progress before retrying direct reclaim

From: Matt Fleming <[email protected]>

should_reclaim_retry() uses zone_reclaimable_pages() to estimate whether retrying reclaim could eventually satisfy an allocation. It's possible for reclaim to make minimal or no progress on an LRU type despite having ample reclaimable pages, e.g. anonymous pages when the only swap is RAM-backed (zram). This can cause the reclaim path to loop indefinitely.

Track LRU reclaim progress (anon vs file) through a new struct reclaim_progress passed out of try_to_free_pages(), and only count a type's reclaimable pages if at least reclaim_progress_pct% was actually reclaimed in the last cycle.

The threshold is exposed as /proc/sys/vm/reclaim_progress_pct (default 1, range 0-100). Setting 0 disables the gate and restores the previous behaviour. Environments with only RAM-backed swap (zram) and small memory may need a higher value to prevent futile anon LRU churn from keeping the allocator spinning.

---
 include/linux/swap.h |  13 +++++-
 mm/page_alloc.c      | 101 +++++++++++++++++++++++++++++++++++--------
 mm/vmscan.c          |  72 ++++++++++++++++++++++--------
 3 files changed, 146 insertions(+), 40 deletions(-)
include/linux/swap.h +11 -2
index 62fc7499b408..d46477365cd9 100644
@@ -368,9 +368,18 @@ void folio_mark_lazyfree(struct folio *folio);
 extern void swap_setup(void);
 
 /* linux/mm/vmscan.c */
+struct reclaim_progress {
+	unsigned long nr_reclaimed;
+	unsigned long nr_anon;
+	unsigned long nr_file;
+};
+
 extern unsigned long zone_reclaimable_pages(struct zone *zone);
-extern unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
-					gfp_t gfp_mask, nodemask_t *mask);
+extern unsigned long zone_reclaimable_file_pages(struct zone *zone);
+extern unsigned long zone_reclaimable_anon_pages(struct zone *zone);
+extern void try_to_free_pages(struct zonelist *zonelist, int order,
+			      gfp_t gfp_mask, nodemask_t *mask,
+			      struct reclaim_progress *progress);
 
 #define MEMCG_RECLAIM_MAY_SWAP (1 << 1)
 #define MEMCG_RECLAIM_PROACTIVE (1 << 2)
mm/page_alloc.c +82 -19

208 lines not on this page · raw message

mm/vmscan.c +53 -19

136 lines not on this page · raw message

Signature · 1 line
2.43.0
#1 lore raw
Quoted 13 lines · Matt Fleming

From: Matt Fleming <[email protected]>

should_reclaim_retry() uses zone_reclaimable_pages() to estimate whether retrying reclaim could eventually satisfy an allocation. It's possible for reclaim to make minimal or no progress on an LRU type despite having ample reclaimable pages, e.g. anonymous pages when the only swap is RAM-backed (zram). This can cause the reclaim path to loop indefinitely.

Track LRU reclaim progress (anon vs file) through a new struct reclaim_progress passed out of try_to_free_pages(), and only count a type's reclaimable pages if at least reclaim_progress_pct% was actually reclaimed in the last cycle.

Quoted 8 lines

The threshold is exposed as /proc/sys/vm/reclaim_progress_pct (default 1, range 0-100). Setting 0 disables the gate and restores the previous behaviour. Environments with only RAM-backed swap (zram) and small memory may need a higher value to prevent futile anon LRU churn from keeping the allocator spinning.

Hi Matt,

so have you tested it for your usecase with zram and have any observations how it helped, what values did you set etc?

Vlastimil

#2 lore raw
Quoted 5 lines · Vlastimil Babka (SUSE)

Hi Matt,

so have you tested it for your usecase with zram and have any observations how it helped, what values did you set etc?

Hey Vlastimil,

Yeah I've tested this out. So far, results have been positive -- I see system-wide OOM kills when memory is low and direct reclaim occurs, but not so many OOM kills that the SRE folks have started screaming at me.

I've only run with the proposed 1% value so far. I also ran a bunch of benchmarks alongside a memory hogging app that peridoically touches anoymous memory.

Workload rpp=0 rpp=1 Notes
----------------------------------------------------------------------------------------------
Kernel compile + anon hog Completed, no OOM Completed, Global OOM confirmed from
Global OOM fired __alloc_pages_slowpath

Memcached + anon hog 282k / 2.30M ops/s 562k / 3.53M ops/s Global OOM killed hog,
No OOM Global OOM fired then benchmark ran faster

Pure fio (5 reruns each)     median 3710 MiB/s  median 3702 MiB/s   No reproducible regression
Mixed fio + anon hog         2747 MiB/s         2915 MiB/s          Global OOM killed

unrelated services

reclaim_progress_pct=1 seems to help in these memory exhausted situations, and doesn't appear to cause a regression for the pure file workload case.

If you have any suggestions for other tests or benchmarks to run I'd be happy to do that.

Thanks,
Matt

#4 lore raw
In reply to Matt Fleming
Quoted 8 lines · Matt Fleming

From: Matt Fleming <[email protected]>

should_reclaim_retry() uses zone_reclaimable_pages() to estimate whether retrying reclaim could eventually satisfy an allocation. It's possible for reclaim to make minimal or no progress on an LRU type despite having ample reclaimable pages, e.g. anonymous pages when the only swap is RAM-backed (zram). This can cause the reclaim path to loop indefinitely.

Quoted 4 lines

Track LRU reclaim progress (anon vs file) through a new struct reclaim_progress passed out of try_to_free_pages(), and only count a type's reclaimable pages if at least reclaim_progress_pct% was actually reclaimed in the last cycle.

I think there is at least one problem with this heuristic: you are counting everything that hasn't made progress as "we cannot reclaim it". When in reality you can simply fail to make progress on any given folio as e.g it's referenced and we want to give it another spin in the LRU.

My theory (from merely reading the patch, maybe I missed something) is that a pathological case for this is a lot of folios added to the LRU in a row, that are set referenced (or dirty). Say SWAP_CLUSTER_MAX * MAX_RECLAIM_RETRIES
- it will simply OOM too early.

The other question is whether this effectively solves reclaim problems - some hard numbers would be great.

Signature · 1 line
Pedro
reviewer
#8 lore raw
In reply to Matt Fleming
Quoted 7 lines · Matt Fleming

From: Matt Fleming <[email protected]>

should_reclaim_retry() uses zone_reclaimable_pages() to estimate whether retrying reclaim could eventually satisfy an allocation. It's possible for reclaim to make minimal or no progress on an LRU type despite having ample reclaimable pages, e.g. anonymous pages when the only swap is RAM-backed (zram).

Or incompressible memory on zswap with writeback disabled or overcommitted memory.min.

Quoted 9 lines

This can cause the reclaim path to loop indefinitely.

Track LRU reclaim progress (anon vs file) through a new struct reclaim_progress passed out of try_to_free_pages(), and only count a type's reclaimable pages if at least reclaim_progress_pct% was actually reclaimed in the last cycle.

The threshold is exposed as /proc/sys/vm/reclaim_progress_pct (default 1, range 0-100).

Let's not expose any sysctl or user visible API for this heuristic. It will evolve and then this interface would be awkward and hard to remove.

Quoted 8 lines

Setting 0 disables the gate and restores the previous behaviour. Environments with only RAM-backed swap (zram) and small memory may need a higher value to prevent futile anon LRU churn from keeping the allocator spinning.

---

[...]

Quoted 7 lines
mm/page_alloc.c +1 -1
@@ -4637,7 +4672,24 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
 			!__cpuset_zone_allowed(zone, gfp_mask))
 				continue;
 
-		available = reclaimable = zone_reclaimable_pages(zone);
+		/*

mm/page_alloc.c should_reclaim_retry(gfp_t gfp_mask, unsigned order, · line 4685

Quoted 10 lines
mm/page_alloc.c +10 -0
+		 * Only count reclaimable pages from an LRU type if reclaim
+		 * actually made headway on that type in the last cycle.
+		 * This prevents the allocator from looping endlessly on
+		 * account of a large pool of pages that reclaim cannot make
+		 * progress on, e.g. anonymous pages when the only swap is
+		 * RAM-backed (zram).
+		 */
+		reclaimable = 0;
+		reclaimable_file = zone_reclaimable_file_pages(zone);
+		reclaimable_anon = zone_reclaimable_anon_pages(zone);

Here we are getting the current reclaimable pages.

mm/page_alloc.c should_reclaim_retry(gfp_t gfp_mask, unsigned order, · line 4690

Quoted 5 lines
mm/page_alloc.c +5 -0
+
+		if (reclaim_progress_sufficient(progress->nr_file, reclaimable_file))
+			reclaimable += reclaimable_file;
+		if (reclaim_progress_sufficient(progress->nr_anon, reclaimable_anon))
+			reclaimable += reclaimable_anon;

And here we are comparing the current reclaimable pages with last iteration. Is this intentional to keep things simple?

mm/page_alloc.c should_reclaim_retry(gfp_t gfp_mask, unsigned order, · line 4694

Quoted 4 lines
mm/page_alloc.c +2 -0
+
+		available = reclaimable;
 		available += zone_page_state_snapshot(zone, NR_FREE_PAGES);
 

Another heuristic we can play with is to also pass through the vmscan scan count. If for couple of consecutive iterations, we continue to see low reclaim efficiency, go for OOM. Also maybe compare the scan count with the watermark as I expect we don't see much difference scan count for consecutive reclaim iteration, so, it is a good representative of reclaimable memory.

The reclaim efficiency heuristic should handle the swap-on-zram or incomp-zswap-with-no-writeback. Treating scan count as proxy for reclaimable memory should handle the overcommitted memory.min case.

reviewer
#10 lore raw
In reply to Matt Fleming
Quoted 8 lines · Matt Fleming

From: Matt Fleming <[email protected]>

should_reclaim_retry() uses zone_reclaimable_pages() to estimate whether retrying reclaim could eventually satisfy an allocation. It's possible for reclaim to make minimal or no progress on an LRU type despite having ample reclaimable pages, e.g. anonymous pages when the only swap is RAM-backed (zram). This can cause the reclaim path to loop indefinitely.

I am still struggling to understand when zram-backed reclamation cannot make progress. Is it because zram is full, or because folio_alloc_swap() fails?

Or does zs_malloc() fail, causing pageout() to fail? Even incompressible pages are still written as ZRAM_HUGE pages and reclaimed successfully.

Quoted 5 lines

Track LRU reclaim progress (anon vs file) through a new struct reclaim_progress passed out of try_to_free_pages(), and only count a type's reclaimable pages if at least reclaim_progress_pct% was actually reclaimed in the last cycle.

I would rather detect what causes the lack of progress and implement a better fallback.

Quoted 9 lines

The threshold is exposed as /proc/sys/vm/reclaim_progress_pct (default 1, range 0-100). Setting 0 disables the gate and restores the previous behaviour. Environments with only RAM-backed swap (zram) and small memory may need a higher value to prevent futile anon LRU churn from keeping the allocator spinning.

Thanks
Barry

#5 lore raw
In reply to Pedro Falcato
Quoted 14 lines · Pedro Falcato
Quoted 8 lines · Matt Fleming

From: Matt Fleming <[email protected]>

should_reclaim_retry() uses zone_reclaimable_pages() to estimate whether retrying reclaim could eventually satisfy an allocation. It's possible for reclaim to make minimal or no progress on an LRU type despite having ample reclaimable pages, e.g. anonymous pages when the only swap is RAM-backed (zram). This can cause the reclaim path to loop indefinitely.

Quoted 4 lines

Track LRU reclaim progress (anon vs file) through a new struct reclaim_progress passed out of try_to_free_pages(), and only count a type's reclaimable pages if at least reclaim_progress_pct% was actually reclaimed in the last cycle.

Quoted 4 lines

I think there is at least one problem with this heuristic: you are counting everything that hasn't made progress as "we cannot reclaim it". When in reality you can simply fail to make progress on any given folio as e.g it's referenced and we want to give it another spin in the LRU.

The intention was that the percentage threshold would avoid giving up on reclaim as long as "sufficient" progress was made. This should allow for some folios to need another trip through the LRU but...

Quoted 4 lines

My theory (from merely reading the patch, maybe I missed something) is that a pathological case for this is a lot of folios added to the LRU in a row, that are set referenced (or dirty). Say SWAP_CLUSTER_MAX * MAX_RECLAIM_RETRIES
- it will simply OOM too early.

OK yeah I think I see the problem now: this heuristic applies the threshold against all reclaimable pages but that falls apart when doing SWAP_CLUSTER_MAX chunks of reclaim.

Quoted 2 lines

The other question is whether this effectively solves reclaim problems - some hard numbers would be great.

I shared some numbers in my reply to Vlastimil, but if there are other cases you'd like measured I'm happy to run them.

#9 lore raw
In reply to Shakeel Butt
Quoted 21 lines · Shakeel Butt
Quoted 7 lines · Matt Fleming
mm/page_alloc.c +1 -1
@@ -4637,7 +4672,24 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
 			!__cpuset_zone_allowed(zone, gfp_mask))
 				continue;
 
-		available = reclaimable = zone_reclaimable_pages(zone);
+		/*

mm/page_alloc.c should_reclaim_retry(gfp_t gfp_mask, unsigned order, · line 4685

Quoted 10 lines
mm/page_alloc.c +10 -0
+		 * Only count reclaimable pages from an LRU type if reclaim
+		 * actually made headway on that type in the last cycle.
+		 * This prevents the allocator from looping endlessly on
+		 * account of a large pool of pages that reclaim cannot make
+		 * progress on, e.g. anonymous pages when the only swap is
+		 * RAM-backed (zram).
+		 */
+		reclaimable = 0;
+		reclaimable_file = zone_reclaimable_file_pages(zone);
+		reclaimable_anon = zone_reclaimable_anon_pages(zone);

Here we are getting the current reclaimable pages.

Quoted 8 lines

mm/page_alloc.c should_reclaim_retry(gfp_t gfp_mask, unsigned order, · line 4690

Quoted 5 lines
mm/page_alloc.c +5 -0
+
+		if (reclaim_progress_sufficient(progress->nr_file, reclaimable_file))
+			reclaimable += reclaimable_file;
+		if (reclaim_progress_sufficient(progress->nr_anon, reclaimable_anon))
+			reclaimable += reclaimable_anon;

And here we are comparing the current reclaimable pages with last iteration. Is this intentional to keep things simple?

Yep, that was the intent.

Quoted 11 lines

mm/page_alloc.c should_reclaim_retry(gfp_t gfp_mask, unsigned order, · line 4694

Quoted 4 lines
mm/page_alloc.c +2 -0
+
+		available = reclaimable;
 		available += zone_page_state_snapshot(zone, NR_FREE_PAGES);
 

Another heuristic we can play with is to also pass through the vmscan scan count. If for couple of consecutive iterations, we continue to see low reclaim efficiency, go for OOM. Also maybe compare the scan count with the watermark as I expect we don't see much difference scan count for consecutive reclaim iteration, so, it is a good representative of reclaimable memory.

Quoted 3 lines

The reclaim efficiency heuristic should handle the swap-on-zram or incomp-zswap-with-no-writeback. Treating scan count as proxy for reclaimable memory should handle the overcommitted memory.min case.

Nice. I'll take a look at this.

reviewer
#6 lore raw
In reply to Matt Fleming
Quoted 1 line · Matt Fleming

On Wed, Apr 15, 2026 at 03:57:25PM +0100, Pedro Falcato wrote:

[...]

Quoted 9 lines
Quoted 5 lines

My theory (from merely reading the patch, maybe I missed something) is that a pathological case for this is a lot of folios added to the LRU in a row, that are set referenced (or dirty). Say SWAP_CLUSTER_MAX * MAX_RECLAIM_RETRIES
- it will simply OOM too early.

OK yeah I think I see the problem now: this heuristic applies the threshold against all reclaimable pages but that falls apart when doing SWAP_CLUSTER_MAX chunks of reclaim.

I am not sure I understand the pathological case. Yes SWAP_CLUSTER_MAX is requested amount of pages to reclaim but the kernel can potentially scan full memory twice to reclaim that much amount. Though those reclaimed pages can get stolen but that can still happen today before this patch.

reviewer
#11 lore raw
In reply to Barry Song
Quoted 10 lines · Barry Song
Quoted 8 lines · Matt Fleming

From: Matt Fleming <[email protected]>

should_reclaim_retry() uses zone_reclaimable_pages() to estimate whether retrying reclaim could eventually satisfy an allocation. It's possible for reclaim to make minimal or no progress on an LRU type despite having ample reclaimable pages, e.g. anonymous pages when the only swap is RAM-backed (zram). This can cause the reclaim path to loop indefinitely.

Quoted 7 lines

I am still struggling to understand when zram-backed reclamation cannot make progress. Is it because zram is full, or because folio_alloc_swap() fails?

Or does zs_malloc() fail, causing pageout() to fail? Even incompressible pages are still written as ZRAM_HUGE pages and reclaimed successfully.

We should have counters for these, right?

Quoted 9 lines
Quoted 6 lines

Track LRU reclaim progress (anon vs file) through a new struct reclaim_progress passed out of try_to_free_pages(), and only count a type's reclaimable pages if at least reclaim_progress_pct% was actually reclaimed in the last cycle.

I would rather detect what causes the lack of progress and implement a better fallback.

This is a good question. I think we have appropriate counters in /proc/vmstat for cases where pages keep getting recycled in the LRUs instead of reclaim.

Matt, do you see anything unexpected in /proc/vmstat?

#7 lore raw
In reply to Shakeel Butt
Quoted 9 lines · Shakeel Butt
Quoted 1 line · Matt Fleming

On Wed, Apr 15, 2026 at 03:57:25PM +0100, Pedro Falcato wrote:

[...]

Quoted 6 lines
Quoted 5 lines

My theory (from merely reading the patch, maybe I missed something) is that a pathological case for this is a lot of folios added to the LRU in a row, that are set referenced (or dirty). Say SWAP_CLUSTER_MAX * MAX_RECLAIM_RETRIES
- it will simply OOM too early.

Quoted 8 lines
Quoted 3 lines

OK yeah I think I see the problem now: this heuristic applies the threshold against all reclaimable pages but that falls apart when doing SWAP_CLUSTER_MAX chunks of reclaim.

I am not sure I understand the pathological case. Yes SWAP_CLUSTER_MAX is requested amount of pages to reclaim but the kernel can potentially scan full memory twice to reclaim that much amount. Though those reclaimed pages can get stolen but that can still happen today before this patch.

I see, yes, you are totally correct. Had a look at the vmscan code again and just realized I had missed some details.

Matt, please disregard :)

Signature · 1 line
Pedro
#3 lore raw
In reply to Matt Fleming
Quoted 4 lines · Matt Fleming
Quoted 3 lines · Vlastimil Babka (SUSE)

Hi Matt,

Quoted 8 lines
Quoted 2 lines

so have you tested it for your usecase with zram and have any observations how it helped, what values did you set etc?

Hey Vlastimil,

Yeah I've tested this out. So far, results have been positive -- I see system-wide OOM kills when memory is low and direct reclaim occurs, but not so many OOM kills that the SRE folks have started screaming at me.

Hmm...

Quoted 8 lines

I've only run with the proposed 1% value so far. I also ran a bunch of benchmarks alongside a memory hogging app that peridoically touches anoymous memory.

Workload rpp=0 rpp=1 Notes
----------------------------------------------------------------------------------------------
Kernel compile + anon hog Completed, no OOM Completed, Global OOM confirmed from
Global OOM fired __alloc_pages_slowpath

Completed in both cases... but was it faster? Also what got OOM killed, the hog?

Quoted 3 lines

Memcached + anon hog 282k / 2.30M ops/s 562k / 3.53M ops/s Global OOM killed hog,
No OOM Global OOM fired then benchmark ran faster

The improvement is nice. However even in the rpp=0 case there didn't seem to have been a thrashing so bad the system wouldn't recover.

I think this is minimally an argument against having it enabled by default, as by default we don't want to cause premature OOMs if the system is still working (And yes, we do have problems to recognize when it's not working, and actually doing OOM). But these tradeoffs for killing something to get better throughput on something else are good for certain kind of servers/workloads but not as a default.

And once you go that way then you might be better of looking at the PSI metrics that would be more holistic than this heuristic?

Quoted 13 lines
Pure fio (5 reruns each)     median 3710 MiB/s  median 3702 MiB/s   No reproducible regression
Mixed fio + anon hog         2747 MiB/s         2915 MiB/s          Global OOM killed

unrelated services

reclaim_progress_pct=1 seems to help in these memory exhausted situations, and doesn't appear to cause a regression for the pure file workload case.

If you have any suggestions for other tests or benchmarks to run I'd be happy to do that.

Thanks,
Matt

#12 lore raw
In reply to Shakeel Butt
Quoted 11 lines · Shakeel Butt
Quoted 8 lines · Barry Song

I am still struggling to understand when zram-backed reclamation cannot make progress. Is it because zram is full, or because folio_alloc_swap() fails?

Or does zs_malloc() fail, causing pageout() to fail? Even incompressible pages are still written as ZRAM_HUGE pages and reclaimed successfully.

We should have counters for these, right?

Let me try and provide some more data for this. It's hard to replicate on our production systems so I've resorted to creating a minimal Qemu repro that has 1GiB RAM and zram disk = 1GiB. The workload is a simple anon memory mapper that allocs 900MiB of memory and touches all pages for 60s.

zs_malloc
---------
None of the zs_malloc() calls failed and we made ~1.2M of them during the test. Here's a breakdown of allocation sizes:

@hist_zs_malloc_size:

[32, 64)         4831015 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[64, 128)            409 |                                                    |
[128, 256)          1090 |                                                    |
[256, 512)          2334 |                                                    |
[512, 1K)           5069 |                                                    |
[1K, 2K)           11174 |                                                    |
[2K, 4K)            2395 |                                                    |
[4K, 8K)             237 |                                                    |

During direct reclaim only:
@hist_zs_malloc_size_in_dr:

[32, 64)         1268042 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[64, 128)             52 |                                                    |
[128, 256)           149 |                                                    |
[256, 512)           292 |                                                    |
[512, 1K)           1234 |                                                    |
[1K, 2K)            3539 |                                                    |
[2K, 4K)            1156 |                                                    |
[4K, 8K)             135 |                                                    |

/sys/block/zram0/mm_stat
--------------------------

before: 4096       74    12288        0    12288        0        0        0        0
after:  42622976  9412667 10985472        0 34131968        0     1962        0      237

trace_mm_vmscan_lru_shrink_inactive
-----------------------------------

Anon LRU shrink events:                 397,949
  sum(args->nr_scanned):                11,837,216
  sum(args->nr_reclaimed):              4,871,775
  sum(args->nr_dirty):                  0
  sum(args->nr_writeback):              0
  sum(args->nr_congested):              0
  sum(args->nr_immediate):              0
  sum(args->nr_ref_keep):               5,200,896
  sum(args->nr_unmap_fail):             0
File LRU shrink events:                 2,632
  sum(args->nr_scanned):                26,048
  sum(args->nr_reclaimed):              12,681
  sum(args->nr_dirty):                  0
  sum(args->nr_writeback):              0
  sum(args->nr_congested):              0
  sum(args->nr_immediate):              0
  sum(args->nr_ref_keep):               476
  sum(args->nr_unmap_fail):             0
Quoted 5 lines
Quoted 2 lines

I would rather detect what causes the lack of progress and implement a better fallback.

This is a good question. I think we have appropriate counters in /proc/vmstat for cases where pages keep getting recycled in the LRUs instead of reclaim.

Here's the output of /proc/vmstat before and after the test runs.

nr_free_pages                             210,825 ->       206,742  (delta=-4,083)
nr_free_pages_blocks                      209,920 ->        65,536  (delta=-144,384)
nr_zone_inactive_anon                       1,685 ->           136  (delta=-1,549)
nr_zone_active_anon                            15 ->         3,774  (delta=3,759)
nr_zone_inactive_file                         329 ->           591  (delta=262)
nr_zone_active_file                           673 ->           504  (delta=-169)
nr_zspages                                      3 ->         2,716  (delta=2,713)
nr_inactive_anon                            1,685 ->           136  (delta=-1,549)
nr_active_anon                                 15 ->         3,774  (delta=3,759)
nr_inactive_file                              329 ->           591  (delta=262)
nr_active_file                                673 ->           504  (delta=-169)
nr_slab_reclaimable                         1,352 ->         2,037  (delta=685)
nr_slab_unreclaimable                       9,581 ->        11,689  (delta=2,108)
nr_anon_pages                               1,526 ->           262  (delta=-1,264)
nr_mapped                                     912 ->           442  (delta=-470)
nr_file_pages                               1,132 ->         4,760  (delta=3,628)
nr_shmem                                      162 ->         3,608  (delta=3,446)
nr_swapcached                                   0 ->            19  (delta=19)
nr_vmscan_write                                 0 ->     4,872,846  (delta=4,872,846)
nr_written                                      1 ->     4,853,727  (delta=4,853,726)
pgpgin                                      1,200 ->    19,035,312  (delta=19,034,112)
pgpgout                                         4 ->    19,414,908  (delta=19,414,904)
pswpin                                          0 ->     4,758,528  (delta=4,758,528)
pswpout                                         0 ->     4,853,726  (delta=4,853,726)
pgalloc_dma                                    32 ->        84,262  (delta=84,230)
pgalloc_dma32                              45,989 ->     5,095,307  (delta=5,049,318)
pgfree                                    269,896 ->     5,415,629  (delta=5,145,733)
pgactivate                                  2,820 ->        14,490  (delta=11,670)
pgdeactivate                                   10 ->        10,924  (delta=10,914)
pgfault                                    29,321 ->     5,088,427  (delta=5,059,106)
pgmajfault                                  3,750 ->     4,794,781  (delta=4,791,031)
pgrefill                                        0 ->        13,733  (delta=13,733)
pgreuse                                     3,333 ->         5,852  (delta=2,519)
pgsteal_kswapd                                  0 ->     3,605,552  (delta=3,605,552)
pgsteal_direct                                  0 ->     1,280,091  (delta=1,280,091)
pgscan_kswapd                                   0 ->     6,579,240  (delta=6,579,240)
pgscan_direct                                   0 ->     5,290,778  (delta=5,290,778)
pgscan_anon                                     0 ->    11,843,970  (delta=11,843,970)
pgscan_file                                     0 ->        26,048  (delta=26,048)
pgsteal_anon                                    0 ->     4,872,962  (delta=4,872,962)
pgsteal_file                                    0 ->        12,681  (delta=12,681)
allocstall_normal                               0 ->           110  (delta=110)
allocstall_movable                              0 ->        32,088  (delta=32,088)
oom_kill                                        0 ->             0  (delta=0)
workingset_nodes                                0 ->           302  (delta=302)
workingset_refault_anon                         0 ->     4,777,591  (delta=4,777,591)
workingset_refault_file                         0 ->           870  (delta=870)
workingset_activate_anon                        0 ->           487  (delta=487)
kswapd_low_wmark_hit_quickly                    0 ->            35  (delta=35)
kswapd_high_wmark_hit_quickly                   0 ->            99  (delta=99)
pageoutrun                                      0 ->           135  (delta=135)
pgmigrate_success                               0 ->        21,317  (delta=21,317)
compact_migrate_scanned                         0 ->        98,848  (delta=98,848)
compact_free_scanned                            0 ->       136,667  (delta=136,667)
swpin_zero                                      0 ->        19,069  (delta=19,069)
swpout_zero                                     0 ->        19,120  (delta=19,120)
swap_ra                                         0 ->            63  (delta=63)
swap_ra_hit                                     0 ->            26  (delta=26)

Happy to do any other tests or pull any other data for you to help.

Thanks,
Matt