Skip to contentSkip to conversation

lib: add SAFE_READ_PROC_STATUS macro

RFC PATCH New Li Wang 3 participants 7 messages Jan 25, 2021 – Feb 24, 2021

lore
Share & tools
lore

Li Wang

author
#0 lore raw

RFC PATCH 1/2 lib: add SAFE_READ_PROC_STATUS macro

To parse /proc/PID/status files, for example:

    SAFE_READ_PROC_STATUS(pid, "VmSwap:");
Cc Petr Vorel <[email protected]>
Cc Alexander Egorenkov <[email protected]>
---
 include/tst_safe_file_ops.h | 8 ++++++++
 1 file changed, 8 insertions(+)
include/tst_safe_file_ops.h +8 -0
index 894c16123..7a4076c4c 100644
@@ -25,6 +25,14 @@
                         &tst_rval); \
         tst_rval;})
 
+#define SAFE_READ_PROC_STATUS(pid, item) \
+       ({long tst_rval_; \
+        char tst_path_[128]; \
+        sprintf(tst_path_, "/proc/%d/status", pid); \
+        SAFE_FILE_LINES_SCANF(tst_path_, item " %ld", \
+                        &tst_rval_); \
+        tst_rval_;})
+
 #define FILE_PRINTF(path, fmt, ...) \
 	file_printf(__FILE__, __LINE__, \
 		    (path), (fmt), ## __VA_ARGS__)
Signature · 1 line
2.21.3
author
#1 lore raw

RFC PATCH 2/2 swapping01: check memory swap usage per process

Since previously swapping01 read the system FreeSwap for counting usage of swap-size, that's not precise on system especially with eating-memory daemon?in the background. Now, we try to check the 'VmmSwap' in proc/PID/status?per process, to get rid of?the potential influence from?other processes?which easily leads to false positive.

Cc Petr Vorel <[email protected]>
Cc Alexander Egorenkov <[email protected]>
---
 testcases/kernel/mem/swapping/swapping01.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
testcases/kernel/mem/swapping/swapping01.c +1 -2
index 8106f6466..0f693f313 100644
@@ -137,8 +137,7 @@ static void check_swapping(void)
 		i++;
 	}
 
-	swap_free_now = SAFE_READ_MEMINFO("SwapFree:");
-	swapped = swap_free_init - swap_free_now;
+	swapped = SAFE_READ_PROC_STATUS(pid, "VmSwap:");
 	if (swapped > mem_over_max) {
 		kill(pid, SIGCONT);
 		tst_brk(TFAIL, "heavy swapping detected: "
Signature · 1 line
2.21.3
#2 lore raw

RFC PATCH 2/2 swapping01: check memory swap usage per process

Hi Li,

Quoted 5 lines

Since previously swapping01 read the system FreeSwap for counting usage of swap-size, that's not precise on system especially with eating-memory daemon?in the background. Now, we try to check the 'VmmSwap' in proc/PID/status?per process, to get rid of?the potential influence from?other processes?which easily leads to false positive.

LGTM.

Kind regards,
Petr

Quoted 21 lines
Cc Petr Vorel <[email protected]>
Cc Alexander Egorenkov <[email protected]>
---
 testcases/kernel/mem/swapping/swapping01.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
testcases/kernel/mem/swapping/swapping01.c +1 -2
index 8106f6466..0f693f313 100644
@@ -137,8 +137,7 @@ static void check_swapping(void)
 		i++;
 	}
 
-	swap_free_now = SAFE_READ_MEMINFO("SwapFree:");
-	swapped = swap_free_init - swap_free_now;
+	swapped = SAFE_READ_PROC_STATUS(pid, "VmSwap:");
 	if (swapped > mem_over_max) {
 		kill(pid, SIGCONT);
 		tst_brk(TFAIL, "heavy swapping detected: "
#3 lore raw

RFC PATCH 2/2 swapping01: check memory swap usage per process

In reply to Li Wang

Hi!

Quoted 5 lines

Since previously swapping01 read the system FreeSwap for counting usage of swap-size, that's not precise on system especially with eating-memory daemon??in the background. Now, we try to check the 'VmmSwap' in proc/PID/status??per process, to get rid of??the potential influence from??other processes??which easily leads to false positive.

I've been looking for a while at the kernel commit:

commit 50a15981a1fac7e019ff7c3cba87531fb580f065

Author: Martin Schwidefsky <[email protected]>
Date:   Sun Jul 24 10:47:59 2011 +0200

[S390] reference bit testing for unmapped pages

For which this test seems to be a reproducer and as far as I can tell this fix is not correct.

If I understand correctly what we are trying to test here is that the kernel will not attempt to swap out unreferenced pages, so we have to, by definition, look at the system counters not the process ones.

Signature · 2 lines
Cyril Hrubis
[email protected]
author
#4 lore raw

RFC PATCH 2/2 swapping01: check memory swap usage per process

Hi Cyril,

Quoted 13 lines · Cyril Hrubis

Hi!

Quoted 5 lines

Since previously swapping01 read the system FreeSwap for counting usage of swap-size, that's not precise on system especially with eating-memory daemon??in the background. Now, we try to check the 'VmmSwap' in proc/PID/status??per process, to get rid of??the potential influence from??other processes??which easily leads to false positive.

I've been looking for a while at the kernel commit:

commit 50a15981a1fac7e019ff7c3cba87531fb580f065

Author: Martin Schwidefsky <[email protected]>
Date:   Sun Jul 24 10:47:59 2011 +0200
Quoted 9 lines

[S390] reference bit testing for unmapped pages

For which this test seems to be a reproducer and as far as I can tell this fix is not correct.

If I understand correctly what we are trying to test here is that the kernel will not attempt to swap out unreferenced pages, so we have to, by definition, look at the system counters not the process ones.

Thanks for point out this! Seems we were all working towards making the test robust but neglect it's a reproducer:).

As we have discussed many unsure things will take side effect to the system swap-counting, that's what we do NOT want to expect. Maybe, can we encapsulate all of the means in a process to avoid involving the whole system swap-counting. e.g.

Child:
to touch unreferenced pages (via
alloc&write&free 1.3*MemAvailable) [1]

         alloc&wirte 1.3*MemAvailable
         raise(SIGSTOP)
         ...

Parent:
waiting for child suspension
check child's VmSwap to see if heavy-swap occurs in a process
...

Does this make some sense to the test or, any else suggestion?

[1] As to perform alloc&write&free, the system pages will go through a completed life cycle from buddy-system to active-list to inactive-list then back to buddy-system, which reflect to a page status is theoretically like:
"inactive,unreferenced -> active,referenced -> ... ->inactive,unreferenced" so that might helpful to produce what the kernel target commit fixed situation.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20210126/8da66ebe/attachment.htm>
author
#5 lore raw

RFC PATCH 2/2 swapping01: check memory swap usage per process

Hi,

Quoted 49 lines · Li Wang

Hi Cyril,

Quoted 13 lines · Cyril Hrubis

Hi!

Quoted 5 lines

Since previously swapping01 read the system FreeSwap for counting usage of swap-size, that's not precise on system especially with eating-memory daemon??in the background. Now, we try to check the 'VmmSwap' in proc/PID/status??per process, to get rid of??the potential influence from??other processes??which easily leads to false positive.

I've been looking for a while at the kernel commit:

commit 50a15981a1fac7e019ff7c3cba87531fb580f065

Author: Martin Schwidefsky <[email protected]>
Date:   Sun Jul 24 10:47:59 2011 +0200
Quoted 9 lines

[S390] reference bit testing for unmapped pages

For which this test seems to be a reproducer and as far as I can tell this fix is not correct.

If I understand correctly what we are trying to test here is that the kernel will not attempt to swap out unreferenced pages, so we have to, by definition, look at the system counters not the process ones.

Thanks for point out this! Seems we were all working towards making the test robust but neglect it's a reproducer:).

As we have discussed many unsure things will take side effect to the system swap-counting, that's what we do NOT want to expect. Maybe, can we encapsulate all of the means in a process to avoid involving the whole system swap-counting. e.g.

Child:
to touch unreferenced pages (via
alloc&write&free 1.3*MemAvailable) [1]

         alloc&wirte 1.3*MemAvailable
         raise(SIGSTOP)
         ...

Parent:
waiting for child suspension
check child's VmSwap to see if heavy-swap occurs in a process
...

Does this make some sense to the test or, any else suggestion?

Quoted 8 lines

[1] As to perform alloc&write&free, the system pages will go through a completed life cycle from buddy-system to active-list to inactive-list then back to buddy-system, which reflect to a page status is theoretically like:
"inactive,unreferenced -> active,referenced -> ... ->inactive,unreferenced" so that might helpful to produce what the kernel target commit fixed situation.

To verify this assumption, I run the swapping01 with/without my patch-V2 against an unfix-kernel(2.6.39). Seems swap out unreferenced pages also could be occurred in a single process, which gives me some confidence to send out my patch V2.

-------
Reproduce the bug easily with an unfix-kernel

# uname  -r
2.6.39
# free -m
             total       used       free     shared    buffers     cached
Mem:          2006       1559        447          0         22       1359
-/+ buffers/cache:        178       1828
Swap:         4031         12       4019
# ./swapping01
tst_test.c:1263: TINFO: Timeout per run is 0h 05m 00s
swapping01.c:110: TINFO: available physical memory: 1896 MB
swapping01.c:113: TINFO: try to allocate: 2466 MB
swapping01.c:148: TFAIL: heavy swapping detected: 1905 MB swapped.
Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20210224/24561ebb/attachment-0001.htm>