why the count nr_file_pages is not equal to nr_inactive_file + nr_active_file ?

DISCUSSION Xishi Qiu 3 participants 8 messages May 13, 2016 – May 16, 2016 Memory management

Share & tools

Xishi Qiu

author

2016-05-13 03:58 UTC #0 lore raw

I find the count nr_file_pages is not equal to nr_inactive_file + nr_active_file. There are 8 cpus, 2 zones in my system.

I think may be the pagevec trigger the problem, but PAGEVEC_SIZE is only 14. Does anyone know the reason?

Thanks,
Xishi Qiu

nr_inactive_file 432444
nr_active_file 20659
total is 453103

nr_file_pages 462723

Aaron Lu

2016-05-13 07:01 UTC #1 lore raw

Quoted 5 lines · Xishi Qiu

I find the count nr_file_pages is not equal to nr_inactive_file + nr_active_file. There are 8 cpus, 2 zones in my system.
I think may be the pagevec trigger the problem, but PAGEVEC_SIZE is only 14. Does anyone know the reason?

One thing I can see is the ram backed filesystem where the page is counted as NR_FILE_PAGE but go into the anonymous LRU list instead of the file LRU list.

See function shmem_getpage_gfp.

An example:
[aaron@aaronlu ~]$ head -11 /proc/vmstat nr_free_pages 194472
nr_alloc_batch 58
nr_inactive_anon 483386
nr_active_anon 298161
nr_inactive_file 452791
nr_active_file 1942376
nr_unevictable 84
nr_mlock 84
nr_anon_pages 445332
nr_mapped 93553
nr_file_pages 2731481
[aaron@aaronlu ~]$ fallocate -l 400M /dev/shm/test [aaron@aaronlu ~]$ head -11 /proc/vmstat nr_free_pages 94808
nr_alloc_batch 838
nr_inactive_anon 582385
nr_active_anon 298371
nr_inactive_file 452795
nr_active_file 1942380
nr_unevictable 84
nr_mlock 84
nr_anon_pages 445543
nr_mapped 93658
nr_file_pages 2830488

The nr_file_pages increased with nr_inactive_anon while the nr_{in}active_file don't see much change.

Regards,
Aaron

Quoted 24 lines

Thanks,
Xishi Qiu
root@hi3650:/ # cat /proc/vmstat
nr_free_pages 54192
nr_inactive_anon 39830
nr_active_anon 28794
nr_inactive_file 432444
nr_active_file 20659
nr_unevictable 2363
nr_mlock 0
nr_anon_pages 65249
nr_mapped 19742
nr_file_pages 462723
nr_dirty 20
nr_writeback 0
...
nr_inactive_file 432444
nr_active_file 20659
total is 453103

Quoted 8 lines

nr_file_pages 462723
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Xishi Qiu

author

2016-05-13 07:32 UTC #2 lore raw

Quoted 40 lines · Aaron Lu

Quoted 5 lines · Xishi Qiu
I find the count nr_file_pages is not equal to nr_inactive_file + nr_active_file. There are 8 cpus, 2 zones in my system.
I think may be the pagevec trigger the problem, but PAGEVEC_SIZE is only 14. Does anyone know the reason?
One thing I can see is the ram backed filesystem where the page is counted as NR_FILE_PAGE but go into the anonymous LRU list instead of the file LRU list.
See function shmem_getpage_gfp.
An example:
[aaron@aaronlu ~]$ head -11 /proc/vmstat nr_free_pages 194472
nr_alloc_batch 58
nr_inactive_anon 483386
nr_active_anon 298161
nr_inactive_file 452791
nr_active_file 1942376
nr_unevictable 84
nr_mlock 84
nr_anon_pages 445332
nr_mapped 93553
nr_file_pages 2731481
[aaron@aaronlu ~]$ fallocate -l 400M /dev/shm/test [aaron@aaronlu ~]$ head -11 /proc/vmstat nr_free_pages 94808
nr_alloc_batch 838
nr_inactive_anon 582385
nr_active_anon 298371
nr_inactive_file 452795
nr_active_file 1942380
nr_unevictable 84
nr_mlock 84
nr_anon_pages 445543
nr_mapped 93658
nr_file_pages 2830488

Quoted 6 lines

The nr_file_pages increased with nr_inactive_anon while the nr_{in}active_file don't see much change.
Regards,
Aaron

Hi Aaron,

Thanks for your reply, but I find the count of nr_shmem is very small in my system.

root@hi3650:/ # cat /proc/vmstat
nr_free_pages 54192
nr_inactive_anon 39830
nr_active_anon 28794
nr_inactive_file 432444
nr_active_file 20659
nr_unevictable 2363
nr_mlock 0
nr_anon_pages 65249
nr_mapped 19742
nr_file_pages 462723
nr_dirty 20
nr_writeback 0
nr_slab_reclaimable 259333
nr_slab_unreclaimable 33463
nr_page_table_pages 3456
nr_kernel_stack 892
nr_unstable 0
nr_bounce 11
nr_vmscan_write 292032
nr_vmscan_immediate_reclaim 47204474
nr_writeback_temp 0
nr_isolated_anon 0
nr_isolated_file 0
nr_shmem 128
nr_dirtied 69574
nr_written 356299
nr_anon_transparent_hugepages 0
nr_free_cma 7519
nr_swapcache 41972
nr_dirty_threshold 6982
nr_dirty_background_threshold 99297

Michal Hocko

2016-05-16 09:57 UTC #3 lore raw

[Sorry I haven't noticed this answer before]

Quoted 6 lines · Xishi Qiu

On 2016/5/13 15:00, Aaron Lu wrote:
Hi Aaron,
Thanks for your reply, but I find the count of nr_shmem is very small in my system.

which kernel version is this? I remember that we used to account thp pages as NR_FILE_PAGE as well in the past.

I didn't get to look at your number more closely though.

Signature · 8 lines

Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Xishi Qiu

author

2016-05-16 10:01 UTC #4 lore raw

Quoted 7 lines · Michal Hocko

[Sorry I haven't noticed this answer before]
Quoted 4 lines · Xishi Qiu
On 2016/5/13 15:00, Aaron Lu wrote:
Hi Aaron,

Quoted 7 lines

Quoted 2 lines
Thanks for your reply, but I find the count of nr_shmem is very small in my system.
which kernel version is this? I remember that we used to account thp pages as NR_FILE_PAGE as well in the past.
I didn't get to look at your number more closely though.

Hi Michal,

It's android kernel, v3.10
I think the thp config is off.

Thanks,
Xishi Qiu

Michal Hocko

2016-05-16 11:16 UTC #5 lore raw

Quoted 15 lines · Xishi Qiu

Quoted 13 lines · Michal Hocko
[Sorry I haven't noticed this answer before]
Quoted 6 lines · Xishi Qiu
On 2016/5/13 15:00, Aaron Lu wrote:
Hi Aaron,
Thanks for your reply, but I find the count of nr_shmem is very small in my system.
which kernel version is this? I remember that we used to account thp pages as NR_FILE_PAGE as well in the past.

Quoted 6 lines

Quoted 1 line
I didn't get to look at your number more closely though.
Hi Michal,
It's android kernel, v3.10
I think the thp config is off.

Ble. Not enough sleep. I didn't mean thp but hugetlb pages. Sorry about the confusion. If even that is not the case then there is either an accounting bug or some fs doesn't put pages in the pagecache directly to the LRUs or thos pages are on unevictable list.

Signature · 8 lines

Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Michal Hocko

2016-05-16 09:31 UTC #6 lore raw

In reply to Xishi Qiu

Quoted 2 lines · Xishi Qiu

I find the count nr_file_pages is not equal to nr_inactive_file + nr_active_file. There are 8 cpus, 2 zones in my system.

Because they count shmem pages as well and those are living on the anon lru list (see shmem_add_to_page_cache).

Signature · 8 lines

Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Xishi Qiu

author

2016-05-16 09:59 UTC #7 lore raw

Quoted 6 lines · Michal Hocko

Quoted 2 lines · Xishi Qiu
I find the count nr_file_pages is not equal to nr_inactive_file + nr_active_file. There are 8 cpus, 2 zones in my system.
Because they count shmem pages as well and those are living on the anon lru list (see shmem_add_to_page_cache).

Hi Michal,

But the shmem seems very small.

nr_inactive_file 432444
nr_active_file 20659
nr_unevictable 2363
nr_shmem 128

nr_file_pages 462723

There is still 7129 pages difference.