Skip to contentSkip to conversation

kvm_stat issue running in the background

DISCUSSION Stefan Hajnoczi 2 participants 3 messages Aug 7, 2024 – Aug 9, 2024 1 message not in the archive KVM

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 3 lines · Claudio Fontana

Hello Stefan,

did not know where to report this, but the man page mentions you specifically so here I am.

Hi Claudio,
I wrote the man page for kvm_stat(1) but am not the maintainer of the tool. I have CCed the KVM mailing list although it's possible that no one actively maintains the tool :).

Quoted 11 lines

There seems to be an issue when kvm_stat is run with:

kvm_stat -p xxx -d -t -s yy -c -L FILENAME.csv &

specifically due to the ampersand (&), thus running in the background.

It seems that kvm_stat gets the interrupt signal (SIGINT), and does write as a result the output to disk, but then instead of terminating, it just hangs there forever.

That is strange. The only signal handler installed by kvm_stat is for SIGHUP, so Python should perform the default behavior for SIGINT and terminate. I'm not sure why the process would hang.

Quoted 5 lines

So to avoid ending up with a large number of kvm_stat processes lingering on the system, we needed to put a random sleep, and then send a SIGTERM to terminate the kvm_stat processes.

Just sending a SIGTERM (without the SIGINT) does terminate the kvm_stat processes, but NO DATA is written to disk (the files show as 0 size).

That makes sense since kvm_stat does not handle SIGTERM. The default SIGTERM behavior is to terminal and any output in Python's I/O buffers may not have been written to the file.

Maybe kvm_stat should catch SIGINT and SIGTERM. That would give it a chance to write out the log before terminating. Do you want to try implementing that?

Quoted 46 lines

This is the workaround script that we currently have:

----

#! /bin/bash

VM_PIDS=pgrep qemu-system-

for VM_PID in ${VM_PIDS} ; do
# warning: kvm_stat is very fragile, change with care kvm_stat -p ${VM_PID} -d -t -s 1 -c -L kvm_stat_${VM_PID}.csv &
done

if test "x${VM_PID}" != "x" ; then
echo "launched kvm_stat processes, capturing 10 seconds..." sleep 10
echo "signaling all kvm_stat processes to write to disk..." pkill -INT -P $$
sleep 5
sync
echo "signaling all kvm_stat processes to die..." pkill -TERM -P $$
echo "waiting for kvm_stat processes to exit..." while pgrep -P $$ > /dev/null; do
sleep 2
echo "still waiting for kvm_stat processes to exit..." done
fi

echo "Done."

----

Feel free to forward to the appropriate mailing list if needed,

thanks!

Claudio

Signature · 4 lines
Claudio Fontana
Engineering Manager Virtualization, SUSE Labs Core

SUSE Software Solutions Italy Srl
#2 lore raw

Hi,

Quoted 32 lines · Stefan Hajnoczi
Quoted 3 lines · Claudio Fontana

Hello Stefan,

did not know where to report this, but the man page mentions you specifically so here I am.

Hi Claudio,
I wrote the man page for kvm_stat(1) but am not the maintainer of the tool. I have CCed the KVM mailing list although it's possible that no one actively maintains the tool :).

Quoted 11 lines

There seems to be an issue when kvm_stat is run with:

kvm_stat -p xxx -d -t -s yy -c -L FILENAME.csv &

specifically due to the ampersand (&), thus running in the background.

It seems that kvm_stat gets the interrupt signal (SIGINT), and does write as a result the output to disk, but then instead of terminating, it just hangs there forever.

That is strange. The only signal handler installed by kvm_stat is for SIGHUP, so Python should perform the default behavior for SIGINT and terminate. I'm not sure why the process would hang.

Quoted 5 lines

So to avoid ending up with a large number of kvm_stat processes lingering on the system, we needed to put a random sleep, and then send a SIGTERM to terminate the kvm_stat processes.

Just sending a SIGTERM (without the SIGINT) does terminate the kvm_stat processes, but NO DATA is written to disk (the files show as 0 size).

Quoted 7 lines

That makes sense since kvm_stat does not handle SIGTERM. The default SIGTERM behavior is to terminal and any output in Python's I/O buffers may not have been written to the file.

Maybe kvm_stat should catch SIGINT and SIGTERM. That would give it a chance to write out the log before terminating. Do you want to try implementing that?I

I think SIGINT would be enough, sure I can give it a try,

Thanks,

C

Quoted 47 lines
Quoted 47 lines

This is the workaround script that we currently have:

----

#! /bin/bash

VM_PIDS=pgrep qemu-system-

for VM_PID in ${VM_PIDS} ; do
# warning: kvm_stat is very fragile, change with care kvm_stat -p ${VM_PID} -d -t -s 1 -c -L kvm_stat_${VM_PID}.csv &
done

if test "x${VM_PID}" != "x" ; then
echo "launched kvm_stat processes, capturing 10 seconds..." sleep 10
echo "signaling all kvm_stat processes to write to disk..." pkill -INT -P $$
sleep 5
sync
echo "signaling all kvm_stat processes to die..." pkill -TERM -P $$
echo "waiting for kvm_stat processes to exit..." while pgrep -P $$ > /dev/null; do
sleep 2
echo "still waiting for kvm_stat processes to exit..." done
fi

echo "Done."

----

Feel free to forward to the appropriate mailing list if needed,

thanks!

Claudio

Signature · 4 lines
Claudio Fontana
Engineering Manager Virtualization, SUSE Labs Core

SUSE Software Solutions Italy Srl
#3 lore raw
Quoted 39 lines · Claudio Fontana

Hi,

Quoted 36 lines · Stefan Hajnoczi
Quoted 3 lines · Claudio Fontana

Hello Stefan,

did not know where to report this, but the man page mentions you specifically so here I am.

Hi Claudio,
I wrote the man page for kvm_stat(1) but am not the maintainer of the tool. I have CCed the KVM mailing list although it's possible that no one actively maintains the tool :).

Quoted 11 lines

There seems to be an issue when kvm_stat is run with:

kvm_stat -p xxx -d -t -s yy -c -L FILENAME.csv &

specifically due to the ampersand (&), thus running in the background.

It seems that kvm_stat gets the interrupt signal (SIGINT), and does write as a result the output to disk, but then instead of terminating, it just hangs there forever.

That is strange. The only signal handler installed by kvm_stat is for SIGHUP, so Python should perform the default behavior for SIGINT and terminate. I'm not sure why the process would hang.

Quoted 5 lines

So to avoid ending up with a large number of kvm_stat processes lingering on the system, we needed to put a random sleep, and then send a SIGTERM to terminate the kvm_stat processes.

Just sending a SIGTERM (without the SIGINT) does terminate the kvm_stat processes, but NO DATA is written to disk (the files show as 0 size).

That makes sense since kvm_stat does not handle SIGTERM. The default SIGTERM behavior is to terminal and any output in Python's I/O buffers may not have been written to the file.

Quoted 5 lines
Quoted 3 lines

Maybe kvm_stat should catch SIGINT and SIGTERM. That would give it a chance to write out the log before terminating. Do you want to try implementing that?I

I think SIGINT would be enough, sure I can give it a try,

It seemed to make sense to process SIGINT and SIGTERM the same way in the end.

I sent a patch to the mailing list, Paolo and you:

https://marc.info/?l=kvm&m=172305139224516&w=2

There was some local problem with email, so the email might have reached the list but not you specifically also.

Thanks,

Claudio

Quoted 53 lines

Thanks,

C

Quoted 47 lines
Quoted 47 lines

This is the workaround script that we currently have:

----

#! /bin/bash

VM_PIDS=pgrep qemu-system-

for VM_PID in ${VM_PIDS} ; do
# warning: kvm_stat is very fragile, change with care kvm_stat -p ${VM_PID} -d -t -s 1 -c -L kvm_stat_${VM_PID}.csv &
done

if test "x${VM_PID}" != "x" ; then
echo "launched kvm_stat processes, capturing 10 seconds..." sleep 10
echo "signaling all kvm_stat processes to write to disk..." pkill -INT -P $$
sleep 5
sync
echo "signaling all kvm_stat processes to die..." pkill -TERM -P $$
echo "waiting for kvm_stat processes to exit..." while pgrep -P $$ > /dev/null; do
sleep 2
echo "still waiting for kvm_stat processes to exit..." done
fi

echo "Done."

----

Feel free to forward to the appropriate mailing list if needed,

thanks!

Claudio

Signature · 4 lines
Claudio Fontana
Engineering Manager Virtualization, SUSE Labs Core

SUSE Software Solutions Italy Srl