Skip to contentSkip to conversation

shell: Add checkpoints api for new lib

PATCH Superseded Joerg Vehlow 2 participants 10 messages Aug 12, 2021 – Aug 24, 2021

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 shell: Add checkpoints api for new lib

From: Joerg Vehlow <[email protected]>

The checkpoint api is even mentioned in shell-test-api.txt, but with the old library used.
This also fixes the documentation.

memcg_lib.sh must be adapted in the same commit, because it already sets TST_NEEDS_CHECKPOINTS=1 and had the ipc initialization code. This would run the ipc initialization code twice.

---

Changes to v1:
- Delete ipc file in _tst_do_exit
- Replace TCID with TST_ID in ipc filename

 doc/shell-test-api.txt                        |  4 +-
 .../controllers/memcg/functional/memcg_lib.sh |  8 ----
 testcases/lib/tst_test.sh                     | 45 ++++++++++++++++++-
 3 files changed, 46 insertions(+), 11 deletions(-)
doc/shell-test-api.txt +2 -2
index bf297ab07..8d1bab5a1 100644
@@ -729,14 +729,14 @@ The shell library provides an implementation of the checkpoint interface
 compatible with the C version. All 'TST_CHECKPOINT_*' functions are available.
 
 In order to initialize checkpoints '$TST_NEEDS_CHECKPOINTS' must be set to '1'
-before the inclusion of 'test.sh':
+before the inclusion of 'tst_test.sh':
 
 [source,sh]
 -------------------------------------------------------------------------------
 #!/bin/sh
 
 TST_NEEDS_CHECKPOINTS=1
-. test.sh
+. tst_test.sh
 -------------------------------------------------------------------------------
 
 Since both the implementations are compatible, it's also possible to start
testcases/kernel/controllers/memcg/functional/memcg_lib.sh +0 -8
index dad66c798..e5fa215c3 100755
@@ -69,14 +69,6 @@ memcg_setup()
 		tst_brk TCONF "Either kernel does not support Memory Resource Controller or feature not enabled"
 	fi
 
-	# Setup IPC
-	LTP_IPC_PATH="/dev/shm/ltp_${TCID}_$$"
-	LTP_IPC_SIZE=$PAGESIZE
-	ROD_SILENT dd if=/dev/zero of="$LTP_IPC_PATH" bs="$LTP_IPC_SIZE" count=1
-	ROD_SILENT chmod 600 "$LTP_IPC_PATH"
-	export LTP_IPC_PATH
-	# Setup IPC end
-
 	ROD mkdir /dev/memcg
 	ROD mount -t cgroup -omemory memcg /dev/memcg
 
testcases/lib/tst_test.sh +44 -1

75 lines not on this page · raw message

Signature · 1 line
2.25.1
#1 lore raw

PATCH v2 2/2 pec: Improve reliability

From: Joerg Vehlow <[email protected]>

This fixes two problems:
1. It was possible, that the event listener was terminated, before all events were recorded.
The listener now terminates, after receiving the exit event of the generator. This also gets rid of the 100ms sleep in cn_pec.sh by
using proper synchronization points.
2. Sometimes exit events were generated in the wrong order, Adding a short delay seems to solve this. While this adds in a new sleep, it is just a 1 ms for default the execution.

---
 testcases/kernel/connectors/pec/cn_pec.sh     | 36 ++++---
 .../kernel/connectors/pec/event_generator.c   | 97 ++++++++++---------
 .../kernel/connectors/pec/pec_listener.c      | 56 ++++++++++-
 3 files changed, 128 insertions(+), 61 deletions(-)
testcases/kernel/connectors/pec/cn_pec.sh +22 -14
index 9b85a5c81..74e94dc5e 100755
@@ -16,12 +16,16 @@ TST_PARSE_ARGS=parse_args
 TST_USAGE=usage
 TST_NEEDS_ROOT=1
 TST_NEEDS_TMPDIR=1
+TST_NEEDS_CHECKPOINTS=1
 TST_TEST_DATA="fork exec exit uid gid"
 
 . tst_test.sh
 
 num_events=10
 
+LISTENER_ID=0
+GENERATOR_ID=1
+
 usage()
 {
 	cat << EOF
@@ -67,27 +71,31 @@ setup()
 test()
 {
 	local event=$2
-	local expected_events lis_rc pid fd_act failed act_nevents exp act
+	local gen_pid list_pid gen_rc lis_rc
+	local expected_events fd_act failed act_nevents exp act
 
 	tst_res TINFO "Testing $2 event (nevents=$num_events)"
 
-	pec_listener >lis_$event.log 2>lis_$event.err &
-	pid=$!
-	# wait for pec_listener to start listening
-	tst_sleep 100ms
+	event_generator -n $num_events -e $event -c $GENERATOR_ID >gen.log &
+	gen_pid=$!
+
+	pec_listener -p $gen_pid -c $LISTENER_ID >lis.log &
+	lis_pid=$!
 
-	ROD event_generator -n $num_events -e $event \>gen_$event.log 2\>gen_$event.err
+	TST_CHECKPOINT_WAIT $LISTENER_ID
+	TST_CHECKPOINT_WAKE $GENERATOR_ID
 
-	kill -s INT $pid 2> /dev/null
-	wait $pid
+	wait $gen_pid
+	gen_rc=$?
+	wait $lis_pid
 	lis_rc=$?
 
-	if [ ! -s gen_$event.log ]; then
-		tst_brk TBROK "failed to generate process events: $(cat gen_$event.err)"
+	if [ $gen_rc -ne 0 ]; then
+		tst_brk TBROK "failed to execute event_generator"
 	fi
 
 	if [ $lis_rc -ne 0 ]; then
-		tst_brk TBROK "failed to execute the listener: $(cat lis_$event.err)"
+		tst_brk TBROK "failed to execute pec_listener"
 	fi
 
 	# The listener writes the same messages as the generator, but it can
@@ -104,7 +112,7 @@ test()
 
 	fd_act=$(free_fd)
 	[ -z "$fd_act" ] && tst_brk TBROK "No free filehandle found"
-	eval "exec ${fd_act}<lis_$event.log"
+	eval "exec ${fd_act}<lis.log"
 
 	failed=0
 	act_nevents=0
@@ -122,7 +130,7 @@ test()
 			tst_res TFAIL "Event was not detected by the event listener: $exp"
 			break
 		fi
-	done <gen_$event.log
+	done <gen.log
 
 	eval "exec ${fd_act}<&-"
 
@@ -134,7 +142,7 @@ test()
 		fi
 	else
 		# TFAIL message is already printed in the loop above
-		cat lis_$event.log
+		cat lis.log
 	fi
 }
 
testcases/kernel/connectors/pec/event_generator.c +51 -46

169 lines not on this page · raw message

testcases/kernel/connectors/pec/pec_listener.c +55 -1

101 lines not on this page · raw message

Signature · 1 line
2.25.1
#2 lore raw

PATCH v2 2/2 pec: Improve reliability

Hi Joerg,

I suppose this one is superseded by adding volatile by Martin's fix in b1e7776bf ("connectors/pec_listener: Make exit_flag volatile")

Kind regards,
Petr

#3 lore raw

PATCH v2 2/2 pec: Improve reliability

Hi Petr,

Quoted 4 lines · Petr Vorel

Hi Joerg,

I suppose this one is superseded by adding volatile by Martin's fix in b1e7776bf ("connectors/pec_listener: Make exit_flag volatile")

No, not at all.
The first issue is a timing issue between the event generator -> kernel -> pec_listener,
that cannot be fixed by a volatile, only by correct synchronization. The second issue is also unrelated.

Martin's fix is only for the rare case, where the exit_flag could be optimized out by the compiler,
which would have made the event_listener run forever instead of terminating.

Joerg

#4 lore raw

PATCH v2 2/2 pec: Improve reliability

Hi Joerg,

FYI, this patch breaks build:

make[1]: Leaving directory 'ltp.git/lib'
/usr/lib64/gcc/x86_64-suse-linux/11/../../../../x86_64-suse-linux/bin/ld: ../../../../lib/libltp.a(tst_checkpoint.o):ltp.git/lib/tst_checkpoint.c:36: multiple definition of `tst_futexes'; /tmp/ccITQEiq.o:ltp.git/testcases/kernel/connectors/pec/event_generator.c:168: first defined here
collect2: error: ld returned 1 exit status
make: *** [../../../../include/mk/rules.mk:37: event_generator] Error 1

https://github.com/pevik/ltp/actions/runs/1161400737

Kind regards,
Petr

#5 lore raw

PATCH v2 2/2 pec: Improve reliability

Hi Petr,

Quoted 8 lines · Petr Vorel

Hi Joerg,

FYI, this patch breaks build:

make[1]: Leaving directory 'ltp.git/lib'
/usr/lib64/gcc/x86_64-suse-linux/11/../../../../x86_64-suse-linux/bin/ld: ../../../../lib/libltp.a(tst_checkpoint.o):ltp.git/lib/tst_checkpoint.c:36: multiple definition of `tst_futexes'; /tmp/ccITQEiq.o:ltp.git/testcases/kernel/connectors/pec/event_generator.c:168: first defined here
collect2: error: ld returned 1 exit status
make: *** [../../../../include/mk/rules.mk:37: event_generator] Error 1

Strange that it builds on my system... The symbol is indeed duplicated and a leftover from experimenting.
The declaration in event_generator.c can be removed.
Should I post a v3?

Joerg

#6 lore raw

PATCH v2 2/2 pec: Improve reliability

Hi Joerg,

Quoted 10 lines
Quoted 6 lines

FYI, this patch breaks build:

make[1]: Leaving directory 'ltp.git/lib'
/usr/lib64/gcc/x86_64-suse-linux/11/../../../../x86_64-suse-linux/bin/ld: ../../../../lib/libltp.a(tst_checkpoint.o):ltp.git/lib/tst_checkpoint.c:36: multiple definition of `tst_futexes'; /tmp/ccITQEiq.o:ltp.git/testcases/kernel/connectors/pec/event_generator.c:168: first defined here
collect2: error: ld returned 1 exit status
make: *** [../../../../include/mk/rules.mk:37: event_generator] Error 1

Strange that it builds on my system... The symbol is indeed duplicated and a leftover from experimenting.
The declaration in event_generator.c can be removed.
Should I post a v3?

Yes please.

Kind regards,
Petr

#7 lore raw

PATCH v2 1/2 shell: Add checkpoints api for new lib

In reply to Joerg Vehlow

Hi Joerg,

LGTM, good idea.

But I'd rename $LTP_IPC_PATH to $TST_IPC_PATH to follow conventions to new API (There are also some LTP_* definitions in the legacy C API which were renamed as TST_* in the new API).

testcases/lib/tst_test.sh _tst_do_exit() · line 52

Quoted 1 line
testcases/lib/tst_test.sh +1 -0
+	if [ -n "$TST_NEEDS_CHECKPOINTS" -a -f "${LTP_IPC_PATH}" ]; then

nit (remove unnecessary {}):

    if [ -n "$TST_NEEDS_CHECKPOINTS" -a -f "$TST_IPC_PATH" ]; then

testcases/lib/tst_test.sh _tst_do_exit() · line 53

Quoted 1 line
testcases/lib/tst_test.sh +1 -0
+		rm ${LTP_IPC_PATH}

and here:
rm $TST_IPC_PATH

Kind regards,
Petr

#8 lore raw

PATCH v2 1/2 shell: Add checkpoints api for new lib

Hi Petr,

Quoted 7 lines · Petr Vorel

Hi Joerg,

LGTM, good idea.

But I'd rename $LTP_IPC_PATH to $TST_IPC_PATH to follow conventions to new API (There are also some LTP_* definitions in the legacy C API which were renamed as TST_* in the new API).

This would also require changing the C API, because the environment variable is used for communicating with it.
I we want this, I guess it could be done in another changeset.

Quoted 4 lines

testcases/lib/tst_test.sh _tst_do_exit() · line 52

Quoted 3 lines
testcases/lib/tst_test.sh +1 -0
+	if [ -n "$TST_NEEDS_CHECKPOINTS" -a -f "${LTP_IPC_PATH}" ]; then

nit (remove unnecessary {}):

Could be changed while merging.

Quoted 5 lines
     if [ -n "$TST_NEEDS_CHECKPOINTS" -a -f "$TST_IPC_PATH" ]; then

testcases/lib/tst_test.sh _tst_do_exit() · line 53

Quoted 1 line
testcases/lib/tst_test.sh +1 -0
+		rm ${LTP_IPC_PATH}

and here:
rm $TST_IPC_PATH

Joerg

#9 lore raw

PATCH v2 1/2 shell: Add checkpoints api for new lib

Hi Joerg,

Quoted 5 lines

Hi Petr,

Quoted 1 line · Petr Vorel

Hi Joerg,

Quoted 8 lines
Quoted 5 lines

LGTM, good idea.

But I'd rename $LTP_IPC_PATH to $TST_IPC_PATH to follow conventions to new API (There are also some LTP_* definitions in the legacy C API which were renamed as TST_* in the new API).

This would also require changing the C API, because the environment variable is used for communicating with it.
I we want this, I guess it could be done in another changeset.

Ah, you right.

Quoted 3 lines
Quoted 2 lines

testcases/lib/tst_test.sh _tst_do_exit() · line 52

Quoted 1 line
testcases/lib/tst_test.sh +1 -0
+	if [ -n "$TST_NEEDS_CHECKPOINTS" -a -f "${LTP_IPC_PATH}" ]; then

nit (remove unnecessary {}):

Could be changed while merging.

Sure.

Kind regards,
Petr