Skip to contentSkip to conversation

numa: fix numa test error with non-continuous nodes

PATCH New Li Wang 5 participants 13 messages May 8, 2019 – Nov 10, 2021

lore
Share & tools
lore

Li Wang

author
#0 lore raw

PATCH v2 1/2 numa: fix numa test error with non-continuous nodes

Numa test failed on such machine which has non-continuous numa nodes, it gets wrong data because of the below syntax rule is not applicable to that special situation.
` numastat -p $pid |awk '/^Total/ {print $'$((node+2))'}'
In this patch, we fix that via a bit complex way of awk to get the allocated memory in specified node.

  # numactl -H
  available: 2 nodes (0,8)
  node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
               19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
               35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
               51 52 53 54 55 56 57 58 59 60 61 62 63
  node 0 size: 257741 MB
  node 0 free: 253158 MB
  node 8 cpus: 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
               80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95
               96 97 98 99 100 101 102 103 104 105 106 107 108
               109 110 111 112 113 114 115 116 117 118 119 120
               121 122 123 124 125 126 127
  node 8 size: 261752 MB
  node 8 free: 240933 MB
  node distances:
  node   0   8
    0:  10  40
    8:  40  10
  # numastat -p $pid
  Per-node process memory usage (in MBs) for PID 34168 (support_numa)
                             Node 0          Node 8           Total
                    --------------- --------------- ---------------
  Huge                         0.00            0.00            0.00
  Heap                         0.00            0.12            0.12
  Stack                        0.00            0.06            0.06
  Private                      1.62            1.50            3.12
  ----------------  --------------- --------------- ---------------
  Total                        1.62            1.69            3.31
Cc Chunyu Hu <[email protected]>
Cc Cyril Hrubis <[email protected]>
---
 testcases/kernel/numa/numa01.sh | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)
testcases/kernel/numa/numa01.sh +22 -4
index 33393ac8d..7f3dee849 100755
@@ -52,9 +52,18 @@ TST_NEEDS_CMDS="awk bc numactl numastat"
 extract_numastat_p()
 {
 	local pid=$1
-	local node=$(($2 + 2))
-
-	echo $(numastat -p $pid |awk '/^Total/ {print $'$node'}')
+	local node=$2
+
+	echo $(numastat -p $pid |		\
+		awk -v node=$node '/Node/ {	\
+		gsub("Node", "");		\
+		for (i=0; i<NF; i++)		\
+			if ($i == node)		\
+				col=i+1;	\
+			next			\
+		}				\
+		/^Total/ {print $col}'		\
+	)
 }
 
 check_for_support_numa()
@@ -363,7 +372,16 @@ test9()
 		pid=$!
 		TST_RETRY_FUNC "check_for_support_numa $pid" 0
 
-		Mem_huge=$(echo $(numastat -p $pid |awk '/^Huge/ {print $'$((node+2))'}'))
+		Mem_huge=$(echo $(numastat -p $pid |	\
+			awk -v node=$node '/Node/ {	\
+			gsub("Node", "");		\
+			for (i=0; i<NF; i++)		\
+				if ($i == node)		\
+					col=i+1;	\
+				next			\
+			}				\
+			/^Huge/ {print $col}')		\
+		)
 		Mem_huge=$((${Mem_huge%.*} * 1024))
 
 		if [ "$Mem_huge" -lt "$HPAGE_SIZE" ]; then
Signature · 1 line
2.20.1
author
#1 lore raw

PATCH v2 2/2 libltpnuma: remove restrictions on numa node-id

For some ppc64le systems, it has non-continuous numa nodes in hardware configuration. So we're hitting the below warnings while running set_mempolicy tests on that. To fix this issue, let's just remove restrictions on numa node-id in get_mempolicy().

Error Log
---------
tst_test.c:1096: INFO: Timeout per run is 0h 50m 00s tst_numa.c:190: INFO: Found 2 NUMA memory nodes set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 0 tst_numa.c:26: INFO: Node 0 allocated 16 pages tst_numa.c:26: INFO: Node 8 allocated 0 pages set_mempolicy01.c:82: PASS: child: Node 0 allocated 16 set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8
...
tst_numa.c:26: INFO: Node 0 allocated 0 pages tst_numa.c:26: INFO: Node 8 allocated 0 pages set_mempolicy01.c:86: FAIL: child: Node 8 allocated 0, expected 16

lscpu
-----
Architecture: ppc64le
...

CPU(s):              128
Core(s) per socket:  16
Socket(s):           2
NUMA node(s):        2
Model name:          POWER9, altivec supported
...
NUMA node0 CPU(s):   0-63
NUMA node8 CPU(s):   64-127
Cc Cyril Hrubis <[email protected]>
---
 libs/libltpnuma/tst_numa.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
libs/libltpnuma/tst_numa.c +3 -2
index 0ba6daf39..56c8640ff 100644
@@ -88,8 +88,9 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes,
 		if (ret < 0)
 			tst_brk(TBROK | TERRNO, "get_mempolicy() failed");
 
-		if (node < 0 || (unsigned int)node >= nodes->cnt) {
-			tst_res(TWARN, "get_mempolicy(...) returned invalid node %i\n", node);
+		if (node < 0) {
+			tst_res(TWARN,
+				"get_mempolicy(...) returned invalid node %i\n", node);
 			continue;
 		}
 
Signature · 1 line
2.20.1
#2 lore raw

PATCH v2 2/2 libltpnuma: remove restrictions on numa node-id

Quoted 46 lines · Li Wang

For some ppc64le systems, it has non-continuous numa nodes in hardware configuration. So we're hitting the below warnings while running set_mempolicy tests on that. To fix this issue, let's just remove restrictions on numa node-id in get_mempolicy().

Error Log
---------
tst_test.c:1096: INFO: Timeout per run is 0h 50m 00s tst_numa.c:190: INFO: Found 2 NUMA memory nodes set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 0 tst_numa.c:26: INFO: Node 0 allocated 16 pages tst_numa.c:26: INFO: Node 8 allocated 0 pages set_mempolicy01.c:82: PASS: child: Node 0 allocated 16 set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8
...
tst_numa.c:26: INFO: Node 0 allocated 0 pages tst_numa.c:26: INFO: Node 8 allocated 0 pages set_mempolicy01.c:86: FAIL: child: Node 8 allocated 0, expected 16

lscpu
-----
Architecture: ppc64le
...

CPU(s):              128
Core(s) per socket:  16
Socket(s):           2
NUMA node(s):        2
Model name:          POWER9, altivec supported
...
NUMA node0 CPU(s):   0-63
NUMA node8 CPU(s):   64-127
Cc Cyril Hrubis <[email protected]>
---
 libs/libltpnuma/tst_numa.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
libs/libltpnuma/tst_numa.c +0 -0
index 0ba6daf39..56c8640ff 100644

libs/libltpnuma/tst_numa.c void tst_nodemap_count_pages(struct tst_nodemap *nodes, · line 91

Quoted 6 lines
libs/libltpnuma/tst_numa.c +1 -2
@@ -88,8 +88,9 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes,
 		if (ret < 0)
 			tst_brk(TBROK | TERRNO, "get_mempolicy() failed");
 
-		if (node < 0 || (unsigned int)node >= nodes->cnt) {
-			tst_res(TWARN, "get_mempolicy(...) returned invalid node %i\n", node);
+		if (node < 0) {

It is better to have check for powerpc and remove restriction as it will still hold good for other arch.

-- Bala

Quoted 11 lines
libs/libltpnuma/tst_numa.c +2 -0
+			tst_res(TWARN,
+				"get_mempolicy(...) returned invalid node %i\n", node);
 			continue;
 		}
 
Signature · 5 lines
2.20.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
author
#3 lore raw

PATCH v2 2/2 libltpnuma: remove restrictions on numa node-id

Quoted 50 lines · Balamuruhan S
Quoted 49 lines · Li Wang

For some ppc64le systems, it has non-continuous numa nodes in hardware configuration. So we're hitting the below warnings while running set_mempolicy tests on that. To fix this issue, let's just remove restrictions on numa node-id in get_mempolicy().

Error Log
---------
tst_test.c:1096: INFO: Timeout per run is 0h 50m 00s tst_numa.c:190: INFO: Found 2 NUMA memory nodes set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 0 tst_numa.c:26: INFO: Node 0 allocated 16 pages tst_numa.c:26: INFO: Node 8 allocated 0 pages set_mempolicy01.c:82: PASS: child: Node 0 allocated 16 set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8
...
tst_numa.c:26: INFO: Node 0 allocated 0 pages tst_numa.c:26: INFO: Node 8 allocated 0 pages set_mempolicy01.c:86: FAIL: child: Node 8 allocated 0, expected 16

lscpu
-----
Architecture: ppc64le
...

CPU(s):              128
Core(s) per socket:  16
Socket(s):           2
NUMA node(s):        2
Model name:          POWER9, altivec supported
...
NUMA node0 CPU(s):   0-63
NUMA node8 CPU(s):   64-127
Cc Cyril Hrubis <[email protected]>
---
 libs/libltpnuma/tst_numa.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
libs/libltpnuma/tst_numa.c +0 -0
index 0ba6daf39..56c8640ff 100644
@@ -88,8 +88,9 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes,
              if (ret < 0)
                      tst_brk(TBROK | TERRNO, "get_mempolicy() failed");
 
Quoted 8 lines

libs/libltpnuma/tst_numa.c void tst_nodemap_count_pages(struct tst_nodemap *nodes, · line 92

Quoted 2 lines
libs/libltpnuma/tst_numa.c +0 -2
-             if (node < 0 || (unsigned int)node >= nodes->cnt) {
-                     tst_res(TWARN, "get_mempolicy(...) returned

invalid node %i\n", node);

libs/libltpnuma/tst_numa.c void tst_nodemap_count_pages(struct tst_nodemap *nodes, · line 91

Quoted 1 line
libs/libltpnuma/tst_numa.c +1 -0
+             if (node < 0) {

It is better to have check for powerpc and remove restriction as it will still hold good for other arch.

If the non-continuous numa nodes only appear in ppc machine, that should be necessary to check arch before removing the restriction.

then the patch probably like:

libs/libltpnuma/tst_numa.c +4 -0
@@ -88,7 +88,11 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes,
                if (ret < 0)
                        tst_brk(TBROK | TERRNO, "get_mempolicy() failed");
 
+#if defined (__powerpc64__) || (__powerpc__)
+               if (node < 0) {
+#elif
                if (node < 0 || (unsigned int)node >= nodes->cnt) {
+#endif
                        tst_res(TWARN, "get_mempolicy(...) returned invalid
node %i\n", node);
                        continue;
                }
Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20190509/e93d1f8f/attachment-0001.html>
#4 lore raw

PATCH v2 2/2 libltpnuma: remove restrictions on numa node-id

Quoted 70 lines · Li Wang
Quoted 50 lines · Balamuruhan S
Quoted 49 lines · Li Wang

For some ppc64le systems, it has non-continuous numa nodes in hardware configuration. So we're hitting the below warnings while running set_mempolicy tests on that. To fix this issue, let's just remove restrictions on numa node-id in get_mempolicy().

Error Log
---------
tst_test.c:1096: INFO: Timeout per run is 0h 50m 00s tst_numa.c:190: INFO: Found 2 NUMA memory nodes set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 0 tst_numa.c:26: INFO: Node 0 allocated 16 pages tst_numa.c:26: INFO: Node 8 allocated 0 pages set_mempolicy01.c:82: PASS: child: Node 0 allocated 16 set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8
...
tst_numa.c:26: INFO: Node 0 allocated 0 pages tst_numa.c:26: INFO: Node 8 allocated 0 pages set_mempolicy01.c:86: FAIL: child: Node 8 allocated 0, expected 16

lscpu
-----
Architecture: ppc64le
...

CPU(s):              128
Core(s) per socket:  16
Socket(s):           2
NUMA node(s):        2
Model name:          POWER9, altivec supported
...
NUMA node0 CPU(s):   0-63
NUMA node8 CPU(s):   64-127
Cc Cyril Hrubis <[email protected]>
---
 libs/libltpnuma/tst_numa.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
libs/libltpnuma/tst_numa.c +0 -0
index 0ba6daf39..56c8640ff 100644
@@ -88,8 +88,9 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes,
              if (ret < 0)
                      tst_brk(TBROK | TERRNO, "get_mempolicy() failed");
 
Quoted 8 lines

libs/libltpnuma/tst_numa.c void tst_nodemap_count_pages(struct tst_nodemap *nodes, · line 92

Quoted 2 lines
libs/libltpnuma/tst_numa.c +0 -2
-             if (node < 0 || (unsigned int)node >= nodes->cnt) {
-                     tst_res(TWARN, "get_mempolicy(...) returned

invalid node %i\n", node);

libs/libltpnuma/tst_numa.c void tst_nodemap_count_pages(struct tst_nodemap *nodes, · line 92

Quoted 1 line
libs/libltpnuma/tst_numa.c +1 -0
+             if (node < 0) {

It is better to have check for powerpc and remove restriction as it will still hold good for other arch.

If the non-continuous numa nodes only appear in ppc machine, that should be necessary to check arch before removing the restriction.

then the patch probably like:

libs/libltpnuma/tst_numa.c void tst_nodemap_count_pages(struct tst_nodemap *nodes, · line 95

Quoted 8 lines
libs/libltpnuma/tst_numa.c +4 -0
@@ -88,7 +88,11 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes,
                if (ret < 0)
                        tst_brk(TBROK | TERRNO, "get_mempolicy() failed");
 
+#if defined (__powerpc64__) || (__powerpc__)
+               if (node < 0) {
+#elif
                if (node < 0 || (unsigned int)node >= nodes->cnt) {
+#endif

Looks good to me. Thanks!

-- Bala

Quoted 9 lines
libs/libltpnuma/tst_numa.c +0 -0
                        tst_res(TWARN, "get_mempolicy(...) returned invalid
 node %i\n", node);
                        continue;
                }
Signature · 2 lines
Regards,
Li Wang
#5 lore raw

PATCH v2 2/2 libltpnuma: remove restrictions on numa node-id

In reply to Balamuruhan S

----- Original Message -----

Quoted 50 lines · Li Wang
Quoted 49 lines

For some ppc64le systems, it has non-continuous numa nodes in hardware configuration. So we're hitting the below warnings while running set_mempolicy tests on that. To fix this issue, let's just remove restrictions on numa node-id in get_mempolicy().

Error Log
---------
tst_test.c:1096: INFO: Timeout per run is 0h 50m 00s tst_numa.c:190: INFO: Found 2 NUMA memory nodes set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 0 tst_numa.c:26: INFO: Node 0 allocated 16 pages tst_numa.c:26: INFO: Node 8 allocated 0 pages set_mempolicy01.c:82: PASS: child: Node 0 allocated 16 set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8
...
tst_numa.c:26: INFO: Node 0 allocated 0 pages tst_numa.c:26: INFO: Node 8 allocated 0 pages set_mempolicy01.c:86: FAIL: child: Node 8 allocated 0, expected 16

lscpu
-----
Architecture: ppc64le
...

CPU(s):              128
Core(s) per socket:  16
Socket(s):           2
NUMA node(s):        2
Model name:          POWER9, altivec supported
...
NUMA node0 CPU(s):   0-63
NUMA node8 CPU(s):   64-127
Cc Cyril Hrubis <[email protected]>
---
 libs/libltpnuma/tst_numa.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
libs/libltpnuma/tst_numa.c +0 -0
index 0ba6daf39..56c8640ff 100644
@@ -88,8 +88,9 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes,
 		if (ret < 0)
 			tst_brk(TBROK | TERRNO, "get_mempolicy() failed");
 
Quoted 6 lines

libs/libltpnuma/tst_numa.c void tst_nodemap_count_pages(struct tst_nodemap *nodes, · line 91

Quoted 3 lines
libs/libltpnuma/tst_numa.c +1 -2
-		if (node < 0 || (unsigned int)node >= nodes->cnt) {
-			tst_res(TWARN, "get_mempolicy(...) returned invalid node %i\n", node);
+		if (node < 0) {

It is better to have check for powerpc and remove restriction as it will still hold good for other arch.

Unless memory gets offlined or test harness restricts LTP to run only on some nodes (via numactl). Here's example of former:

# uname -m
x86_64
# numactl -H
available: 8 nodes (0-7)
node 0 cpus: 0 4 8 12
node 0 size: 4061 MB
node 0 free: 3437 MB
node 1 cpus: 16 20 24 28
node 1 size: 0 MB
node 1 free: 0 MB
node 2 cpus: 1 5 9 13
node 2 size: 4096 MB
node 2 free: 3871 MB
node 3 cpus: 17 21 25 29
node 3 size: 0 MB
node 3 free: 0 MB
node 4 cpus: 2 6 10 14
node 4 size: 4096 MB
node 4 free: 3568 MB
node 5 cpus: 18 22 26 30
node 5 size: 0 MB
node 5 free: 0 MB
node 6 cpus: 19 23 27 31
node 6 size: 0 MB
node 6 free: 0 MB
node 7 cpus: 3 7 11 15
node 7 size: 4095 MB
node 7 free: 3802 MB

tst_test.c:1096: INFO: Timeout per run is 0h 05m 00s tst_numa.c:190: INFO: Found 4 NUMA memory nodes
...
tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 4 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 7

Quoted 19 lines

-- Bala

Quoted 11 lines
libs/libltpnuma/tst_numa.c +2 -1
+			tst_res(TWARN,
+				"get_mempolicy(...) returned invalid node %i\n", node);
 			continue;
 		}
 
--

2.20.1

--
Mailing list info: https://lists.linux.it/listinfo/ltp

--
Mailing list info: https://lists.linux.it/listinfo/ltp

#6 lore raw

PATCH v2 2/2 libltpnuma: remove restrictions on numa node-id

In reply to Balamuruhan S

Hi!

Quoted 2 lines

It is better to have check for powerpc and remove restriction as it will still hold good for other arch.

I would like to avoid adding arch specific ifdefs to the code, so for the release I've commited the original fix and we can add better checks later on. As I wrote checking that the numa id is <= than maximal used id would be much better and would work in all cases.

Signature · 2 lines
Cyril Hrubis
[email protected]
#7 lore raw

PATCH v2 1/2 numa: fix numa test error with non-continuous nodes

In reply to Li Wang
Quoted 59 lines · Li Wang

Numa test failed on such machine which has non-continuous numa nodes, it gets wrong data because of the below syntax rule is not applicable to that special situation.
` numastat -p $pid |awk '/^Total/ {print $'$((node+2))'}'
In this patch, we fix that via a bit complex way of awk to get the allocated memory in specified node.

  # numactl -H
  available: 2 nodes (0,8)
  node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
               19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
               35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
               51 52 53 54 55 56 57 58 59 60 61 62 63
  node 0 size: 257741 MB
  node 0 free: 253158 MB
  node 8 cpus: 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
               80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95
               96 97 98 99 100 101 102 103 104 105 106 107 108
               109 110 111 112 113 114 115 116 117 118 119 120
               121 122 123 124 125 126 127
  node 8 size: 261752 MB
  node 8 free: 240933 MB
  node distances:
  node   0   8
    0:  10  40
    8:  40  10
  # numastat -p $pid
  Per-node process memory usage (in MBs) for PID 34168 (support_numa)
                             Node 0          Node 8           Total
                    --------------- --------------- ---------------
  Huge                         0.00            0.00            0.00
  Heap                         0.00            0.12            0.12
  Stack                        0.00            0.06            0.06
  Private                      1.62            1.50            3.12
  ----------------  --------------- --------------- ---------------
  Total                        1.62            1.69            3.31
Cc Chunyu Hu <[email protected]>
Cc Cyril Hrubis <[email protected]>
---
 testcases/kernel/numa/numa01.sh | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)
testcases/kernel/numa/numa01.sh +3 -3
index 33393ac8d..7f3dee849 100755
@@ -52,9 +52,18 @@ TST_NEEDS_CMDS="awk bc numactl numastat"
 extract_numastat_p()
 {
 	local pid=$1
-	local node=$(($2 + 2))
-
-	echo $(numastat -p $pid |awk '/^Total/ {print $'$node'}')
+	local node=$2
+
+	echo $(numastat -p $pid |		\

testcases/kernel/numa/numa01.sh TST_NEEDS_CMDS="awk bc numactl numastat" · line 67

Quoted 10 lines
testcases/kernel/numa/numa01.sh +9 -0
+		awk -v node=$node '/Node/ {	\
+		gsub("Node", "");		\
+		for (i=0; i<NF; i++)		\
+			if ($i == node)		\
+				col=i+1;	\
+			next			\
+		}				\
+		/^Total/ {print $col}'		\
+	)
 }

If we can use extract_numastat_p0 with node_index then the existing code works, for example if can index while iterating $node_list and use this with extract_numastat_p0 then it should work.

This is just my thoughts, please feel free to correct me if I miss or it is something wrong. Thanks!

-- Bala

Quoted 26 lines
testcases/kernel/numa/numa01.sh +10 -1
 check_for_support_numa()
@@ -363,7 +372,16 @@ test9()
 		pid=$!
 		TST_RETRY_FUNC "check_for_support_numa $pid" 0
 
-		Mem_huge=$(echo $(numastat -p $pid |awk '/^Huge/ {print $'$((node+2))'}'))
+		Mem_huge=$(echo $(numastat -p $pid |	\
+			awk -v node=$node '/Node/ {	\
+			gsub("Node", "");		\
+			for (i=0; i<NF; i++)		\
+				if ($i == node)		\
+					col=i+1;	\
+				next			\
+			}				\
+			/^Huge/ {print $col}')		\
+		)
 		Mem_huge=$((${Mem_huge%.*} * 1024))
 
 		if [ "$Mem_huge" -lt "$HPAGE_SIZE" ]; then
Signature · 5 lines
2.20.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
author
#8 lore raw

PATCH v2 1/2 numa: fix numa test error with non-continuous nodes

Quoted 67 lines · Balamuruhan S
Quoted 38 lines · Li Wang

Numa test failed on such machine which has non-continuous numa nodes, it gets wrong data because of the below syntax rule is not applicable to that special situation.
` numastat -p $pid |awk '/^Total/ {print $'$((node+2))'}'
In this patch, we fix that via a bit complex way of awk to get the allocated memory in specified node.

  # numactl -H
  available: 2 nodes (0,8)
  node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
               19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
               35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
               51 52 53 54 55 56 57 58 59 60 61 62 63
  node 0 size: 257741 MB
  node 0 free: 253158 MB
  node 8 cpus: 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
               80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95
               96 97 98 99 100 101 102 103 104 105 106 107 108
               109 110 111 112 113 114 115 116 117 118 119 120
               121 122 123 124 125 126 127
  node 8 size: 261752 MB
  node 8 free: 240933 MB
  node distances:
  node   0   8
    0:  10  40
    8:  40  10
  # numastat -p $pid
  Per-node process memory usage (in MBs) for PID 34168 (support_numa)
                             Node 0          Node 8           Total
                    --------------- --------------- ---------------
  Huge                         0.00            0.00            0.00
  Heap                         0.00            0.12            0.12
  Stack                        0.00            0.06            0.06
  Private                      1.62            1.50            3.12
  ----------------  --------------- --------------- ---------------
  Total                        1.62            1.69            3.31

Review

Quoted 8 lines
Cc Chunyu Hu <[email protected]>
Cc Cyril Hrubis <[email protected]>
---
 testcases/kernel/numa/numa01.sh | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)
(quoted diff) +0 -0

b/testcases/kernel/numa/numa01.sh

Quoted 19 lines

index 33393ac8d..7f3dee849 100755

testcases/kernel/numa/numa01.sh +9 -3
@@ -52,9 +52,18 @@ TST_NEEDS_CMDS="awk bc numactl numastat"
 extract_numastat_p()
 {
      local pid=$1
-     local node=$(($2 + 2))
-
-     echo $(numastat -p $pid |awk '/^Total/ {print $'$node'}')
+     local node=$2
+
+     echo $(numastat -p $pid |               \
+             awk -v node=$node '/Node/ {     \
+             gsub("Node", "");               \
+             for (i=0; i<NF; i++)            \
+                     if ($i == node)         \
+                             col=i+1;        \
+                     next                    \
Quoted 9 lines

testcases/kernel/numa/numa01.sh TST_NEEDS_CMDS="awk bc numactl numastat" · line 67

Quoted 4 lines
testcases/kernel/numa/numa01.sh +3 -0
+             }                               \
+             /^Total/ {print $col}'          \
+     )
 }

If we can use extract_numastat_p0 with node_index then the existing code works, for example if can index while iterating $node_list and use this with extract_numastat_p0 then it should work.

Hi Bala,

Sorry, I don't fully understand what's you mean here :(. Could you explain something more for this method?

Quoted 34 lines

This is just my thoughts, please feel free to correct me if I miss or it is something wrong. Thanks!

-- Bala

testcases/kernel/numa/numa01.sh test9() · line 366

Quoted 7 lines
testcases/kernel/numa/numa01.sh +0 -1
 check_for_support_numa()
@@ -363,7 +372,16 @@ test9()
              pid=$!
              TST_RETRY_FUNC "check_for_support_numa $pid" 0
 
-             Mem_huge=$(echo $(numastat -p $pid |awk '/^Huge/ {print

$'$((node+2))'}'))

Quoted 19 lines
testcases/kernel/numa/numa01.sh +10 -1
+             Mem_huge=$(echo $(numastat -p $pid |    \
+                     awk -v node=$node '/Node/ {     \
+                     gsub("Node", "");               \
+                     for (i=0; i<NF; i++)            \
+                             if ($i == node)         \
+                                     col=i+1;        \
+                             next                    \
+                     }                               \
+                     /^Huge/ {print $col}')          \
+             )
              Mem_huge=$((${Mem_huge%.*} * 1024))
 
              if [ "$Mem_huge" -lt "$HPAGE_SIZE" ]; then
--

2.20.1

--
Mailing list info: https://lists.linux.it/listinfo/ltp

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20190509/81d3c6c9/attachment.html>
#9 lore raw

PATCH v2 1/2 numa: fix numa test error with non-continuous nodes

Quoted 80 lines · Li Wang
Quoted 67 lines · Balamuruhan S
Quoted 38 lines · Li Wang

Numa test failed on such machine which has non-continuous numa nodes, it gets wrong data because of the below syntax rule is not applicable to that special situation.
` numastat -p $pid |awk '/^Total/ {print $'$((node+2))'}'
In this patch, we fix that via a bit complex way of awk to get the allocated memory in specified node.

  # numactl -H
  available: 2 nodes (0,8)
  node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
               19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
               35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
               51 52 53 54 55 56 57 58 59 60 61 62 63
  node 0 size: 257741 MB
  node 0 free: 253158 MB
  node 8 cpus: 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
               80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95
               96 97 98 99 100 101 102 103 104 105 106 107 108
               109 110 111 112 113 114 115 116 117 118 119 120
               121 122 123 124 125 126 127
  node 8 size: 261752 MB
  node 8 free: 240933 MB
  node distances:
  node   0   8
    0:  10  40
    8:  40  10
  # numastat -p $pid
  Per-node process memory usage (in MBs) for PID 34168 (support_numa)
                             Node 0          Node 8           Total
                    --------------- --------------- ---------------
  Huge                         0.00            0.00            0.00
  Heap                         0.00            0.12            0.12
  Stack                        0.00            0.06            0.06
  Private                      1.62            1.50            3.12
  ----------------  --------------- --------------- ---------------
  Total                        1.62            1.69            3.31

Review

Quoted 8 lines
Cc Chunyu Hu <[email protected]>
Cc Cyril Hrubis <[email protected]>
---
 testcases/kernel/numa/numa01.sh | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)
(quoted diff) +0 -0

b/testcases/kernel/numa/numa01.sh

Quoted 19 lines

index 33393ac8d..7f3dee849 100755

testcases/kernel/numa/numa01.sh +9 -3
@@ -52,9 +52,18 @@ TST_NEEDS_CMDS="awk bc numactl numastat"
 extract_numastat_p()
 {
      local pid=$1
-     local node=$(($2 + 2))
-
-     echo $(numastat -p $pid |awk '/^Total/ {print $'$node'}')
+     local node=$2
+
+     echo $(numastat -p $pid |               \
+             awk -v node=$node '/Node/ {     \
+             gsub("Node", "");               \
+             for (i=0; i<NF; i++)            \
+                     if ($i == node)         \
+                             col=i+1;        \
+                     next                    \
Quoted 9 lines

testcases/kernel/numa/numa01.sh TST_NEEDS_CMDS="awk bc numactl numastat" · line 67

Quoted 4 lines
testcases/kernel/numa/numa01.sh +3 -0
+             }                               \
+             /^Total/ {print $col}'          \
+     )
 }

If we can use extract_numastat_p0 with node_index then the existing code works, for example if can index while iterating $node_list and use this with extract_numastat_p0 then it should work.

Quoted 4 lines

Hi Bala,

Sorry, I don't fully understand what's you mean here :(. Could you explain something more for this method?

instead of changing extract_numastat_p0() can do something like this ?

testcases/kernel/numa/numa01.sh +3 -1
index 33393ac8d..47c18edd6 100755
@@ -94,6 +94,7 @@ setup()
 test1()
 {
        Mem_curr=0
+       node_index=0
 
        for node in $nodes_list; do
                numactl --cpunodebind=$node --membind=$node support_numa alloc_1MB &
@@ -101,7 +102,8 @@ test1()
 
                TST_RETRY_FUNC "check_for_support_numa $pid" 0
 
-               Mem_curr=$(echo "$(extract_numastat_p $pid $node) * $MB" |bc)
+               Mem_curr=$(echo "$(extract_numastat_p $pid $node_index) * $MB" |bc)
+               let node_index++
                if [ $(echo "$Mem_curr < $MB" | bc) -eq 1 ]; then
                        tst_res TFAIL \
                                "NUMA memory allocated in node$node is less than expected"

-- Bala

Quoted 41 lines
Quoted 37 lines

This is just my thoughts, please feel free to correct me if I miss or it is something wrong. Thanks!

-- Bala

testcases/kernel/numa/numa01.sh test9() · line 366

Quoted 7 lines
testcases/kernel/numa/numa01.sh +0 -1
 check_for_support_numa()
@@ -363,7 +372,16 @@ test9()
              pid=$!
              TST_RETRY_FUNC "check_for_support_numa $pid" 0
 
-             Mem_huge=$(echo $(numastat -p $pid |awk '/^Huge/ {print

$'$((node+2))'}'))

Quoted 19 lines
testcases/kernel/numa/numa01.sh +10 -1
+             Mem_huge=$(echo $(numastat -p $pid |    \
+                     awk -v node=$node '/Node/ {     \
+                     gsub("Node", "");               \
+                     for (i=0; i<NF; i++)            \
+                             if ($i == node)         \
+                                     col=i+1;        \
+                             next                    \
+                     }                               \
+                     /^Huge/ {print $col}')          \
+             )
              Mem_huge=$((${Mem_huge%.*} * 1024))
 
              if [ "$Mem_huge" -lt "$HPAGE_SIZE" ]; then
--

2.20.1

--
Mailing list info: https://lists.linux.it/listinfo/ltp

Signature · 2 lines
Regards,
Li Wang
author
#10 lore raw

PATCH v2 1/2 numa: fix numa test error with non-continuous nodes

Hi Bala,

Quoted 54 lines · Balamuruhan S

[...snip...]

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
testcases/kernel/numa/numa01.sh +7 -3
@@ -52,9 +52,18 @@ TST_NEEDS_CMDS="awk bc numactl numastat"
 extract_numastat_p()
 {
      local pid=$1
-     local node=$(($2 + 2))
-
-     echo $(numastat -p $pid |awk '/^Total/ {print $'$node'}')
+     local node=$2
+
+     echo $(numastat -p $pid |               \
+             awk -v node=$node '/Node/ {     \
+             gsub("Node", "");               \
+             for (i=0; i<NF; i++)            \
+                     if ($i == node)         \
Quoted 8 lines
Quoted 8 lines

testcases/kernel/numa/numa01.sh TST_NEEDS_CMDS="awk bc numactl numastat" · line 67

Quoted 6 lines
testcases/kernel/numa/numa01.sh +5 -0
+                             col=i+1;        \
+                     next                    \
+             }                               \
+             /^Total/ {print $col}'          \
+     )
 }

If we can use extract_numastat_p0 with node_index then the existing

code

Quoted 7 lines
Quoted 3 lines

works, for example if can index while iterating $node_list and use this with extract_numastat_p0 then it should work.

Hi Bala,

Sorry, I don't fully understand what's you mean here :(. Could you

explain

Quoted 1 line

something more for this method?

instead of changing extract_numastat_p0() can do something like this ?

testcases/kernel/numa/numa01.sh +1 -0
index 33393ac8d..47c18edd6 100755
@@ -94,6 +94,7 @@ setup()
 test1()
 {
        Mem_curr=0
+       node_index=0
 
        for node in $nodes_list; do
                numactl --cpunodebind=$node --membind=$node support_numa

alloc_1MB &

testcases/kernel/numa/numa01.sh test1() · line 109

Quoted 9 lines
testcases/kernel/numa/numa01.sh +2 -1
@@ -101,7 +102,8 @@ test1()
 
                TST_RETRY_FUNC "check_for_support_numa $pid" 0
 
-               Mem_curr=$(echo "$(extract_numastat_p $pid $node) * $MB"
 |bc)
+               Mem_curr=$(echo "$(extract_numastat_p $pid $node_index) *
 $MB" |bc)
+               let node_index++
 

I guess it can be work, but the disadvantage of that is we have to involve a new variable(node_index) in each of the tests (from test1 to test10). Hence I don't think it is much better than my patch. For which way to go, I'd leave this to Cyril to make a choice. Or, maybe he has different thoughts on this:).

Anyway, thanks for your suggestion.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20190509/55f8a078/attachment.html>
#11 lore raw

PATCH v2 1/2 numa: fix numa test error with non-continuous nodes

Hi!

Quoted 18 lines
Quoted 18 lines
testcases/kernel/numa/numa01.sh +1 -0
index 33393ac8d..47c18edd6 100755
@@ -94,6 +94,7 @@ setup()
 test1()
 {
        Mem_curr=0
+       node_index=0
 
        for node in $nodes_list; do
                numactl --cpunodebind=$node --membind=$node support_numa

alloc_1MB &

testcases/kernel/numa/numa01.sh +0 -0
@@ -101,7 +102,8 @@ test1()
 
                TST_RETRY_FUNC "check_for_support_numa $pid" 0
 
Quoted 12 lines

testcases/kernel/numa/numa01.sh test1() · line 109

Quoted 6 lines
testcases/kernel/numa/numa01.sh +2 -1
-               Mem_curr=$(echo "$(extract_numastat_p $pid $node) * $MB"
 |bc)
+               Mem_curr=$(echo "$(extract_numastat_p $pid $node_index) *
 $MB" |bc)
+               let node_index++
 

I guess it can be work, but the disadvantage of that is we have to involve a new variable(node_index) in each of the tests (from test1 to test10). Hence I don't think it is much better than my patch. For which way to go, I'd leave this to Cyril to make a choice. Or, maybe he has different thoughts on this:).

I actually do not care that much about the numa01.sh tests, because these are broken in more ways than this and were never correct to begin with.

I've started to rewrite these into proper tests, the set_mempolicy() was first part of that effort, the mbind() tests are continuation of that and the end goal is to get rid of these broken tests eventually.

Signature · 2 lines
Cyril Hrubis
[email protected]
#12 lore raw

Hello,

Quoted 32 lines · Cyril Hrubis

Hi!

Quoted 18 lines
Quoted 18 lines
testcases/kernel/numa/numa01.sh +1 -0
index 33393ac8d..47c18edd6 100755
@@ -94,6 +94,7 @@ setup()
 test1()
 {
        Mem_curr=0
+       node_index=0
 
        for node in $nodes_list; do
                numactl --cpunodebind=$node --membind=$node support_numa

alloc_1MB &

testcases/kernel/numa/numa01.sh +0 -0
@@ -101,7 +102,8 @@ test1()
 
                TST_RETRY_FUNC "check_for_support_numa $pid" 0
 
Quoted 12 lines

testcases/kernel/numa/numa01.sh test1() · line 109

Quoted 6 lines
testcases/kernel/numa/numa01.sh +2 -1
-               Mem_curr=$(echo "$(extract_numastat_p $pid $node) * $MB"
 |bc)
+               Mem_curr=$(echo "$(extract_numastat_p $pid $node_index) *
 $MB" |bc)
+               let node_index++
 

I guess it can be work, but the disadvantage of that is we have to involve a new variable(node_index) in each of the tests (from test1 to test10). Hence I don't think it is much better than my patch. For which way to go, I'd leave this to Cyril to make a choice. Or, maybe he has different thoughts on this:).

Quoted 7 lines

I actually do not care that much about the numa01.sh tests, because these are broken in more ways than this and were never correct to begin with.

I've started to rewrite these into proper tests, the set_mempolicy() was first part of that effort, the mbind() tests are continuation of that and the end goal is to get rid of these broken tests eventually.

We have quite a few mbind and set_mempolicy tests now. So should we delete this test?

Quoted 4 lines
Signature · 3 lines
-- 
Cyril Hrubis
[email protected]
Signature · 5 lines
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp