Skip to contentSkip to conversation

Virtual Machine Time Accounting

PATCH Superseded Laurent Vivier 7 participants 27 messages Aug 20, 2007 – Aug 4, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Virtual Machine Time Accounting

The aim of these four patches is to introduce Virtual Machine time accounting.

_Ingo_, as these patches modify files of the scheduler, could you have a look to them, please ?

[PATCH 1/4] as recent CPUs introduce a third running state, after "user" and "system", we need a new field, "guest", in cpustat to store the time used by the CPU to run virtual CPU. Modify /proc/stat to display this new field.

[PATCH 2/4] like for cpustat, introduce the "gtime" (guest time of the task) and "cgtime" (guest time of the task children) fields for the
tasks. Modify signal_struct and task_struct. Modify /proc/<pid>/stat to display these new fields.

[PATCH 3/4] modify account_system_time() to add cputime to cpustat->guest if we are running a VCPU. We add this cputime to cpustat->user instead of cpustat->system because this part of KVM code is in fact user code although it is executed in the kernel. We duplicate VCPU time between guest and user to allow an unmodified "top(1)" to display correct value. A modified "top(1)" is able to display good cpu user time and cpu guest time by subtracting cpu guest time from cpu user time. Update "gtime" and "cgtime" in signal_struct and task_struct accordingly.

[PATCH 4/4] Modify KVM to update guest time accounting.

Signature · 2 lines
------------- [email protected]  --------------
          "Software is hard" - Donald Knuth
#1 lore raw
Quoted 2 lines · Laurent Vivier

The aim of these four patches is to introduce Virtual Machine time accounting.

1-3 look good to me. 4 needs minor edits...

Signature · 8 lines
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
#2 lore raw
In reply to Laurent Vivier
Quoted 2 lines · Laurent Vivier

The aim of these four patches is to introduce Virtual Machine time accounting.

1-3 look good to me. 4 needs minor edits...

Signature · 1 line
error compiling committee.c: too many arguments to function
#3 lore raw
In reply to Laurent Vivier

Laurent> The aim of these four patches is to introduce Virtual Machine Laurent> time accounting.

So what does this buy us? What increased functionality?

Laurent> [PATCH 1/4] as recent CPUs introduce a third running state, Laurent> after "user" and "system", we need a new field, "guest", in Laurent> cpustat to store the time used by the CPU to run virtual Laurent> CPU. Modify /proc/stat to display this new field.

What happens when you have multiple guests, is this state cummulative?

I've glanced over these patches and the #ifdefs are ugly..., esp since the overhead seems to exist even if you don't run any Guest CPUs. Or whatever you're doing here.

Please justify this patch more in terms of what it provides and why it's a good thing to have.

Thanks,
John

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/

#4 lore raw

Hi John,

Quoted 6 lines · John Stoffel

Laurent> The aim of these four patches is to introduce Virtual Machine Laurent> time accounting.

So what does this buy us? What increased functionality?

functionnalities:

- allow to measure time spent by a CPU in a virtual CPU.
- allow to display in /proc/state this value by CPU
- allow to display in /proc/<pid>/state this value by process
- allow KVM to use these 3 previous functionnalities

Quoted 6 lines

Laurent> [PATCH 1/4] as recent CPUs introduce a third running state, Laurent> after "user" and "system", we need a new field, "guest", in Laurent> cpustat to store the time used by the CPU to run virtual Laurent> CPU. Modify /proc/stat to display this new field.

What happens when you have multiple guests, is this state cummulative?

Yes, like system time and user time are.

Quoted 1 line

I've glanced over these patches and the #ifdefs are ugly..., esp since

#ifdefs are ugly, I agree... I can remove them.
But "#ifdef" allows to save space and time (and it is not a beauty show).

Quoted 2 lines

the overhead seems to exist even if you don't run any Guest CPUs. Or whatever you're doing here.

Can you explain ?

Quoted 2 lines

Please justify this patch more in terms of what it provides and why it's a good thing to have.

Well... it provides the time spent by CPUs in virtual CPUs.
It is usefull only if you run Virtual Machine on your host (like KVM, lguest, kqemu...) to know how many time your CPUs dedicate to virtual machine execution. It's a good thing to have like user time and system time are good things to have... do you use 'top' or 'time' sometime ?

Laurent

Signature · 2 lines
------------- Laurent.Vivier-6ktuUTfB/[email protected]  --------------
          "Software is hard" - Donald Knuth

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/

_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

#5 lore raw
Quoted 7 lines · Laurent Vivier

functionnalities:

- allow to measure time spent by a CPU in a virtual CPU.
- allow to display in /proc/state this value by CPU
- allow to display in /proc/<pid>/state this value by process
- allow KVM to use these 3 previous functionnalities

So, currently time spent in a kvm guest is accumulated as qemu-kvm usertime, right? Given that qemu knows when its running in qemu vs guest context, couldn't it provide the breakdown between user and guest time (ditto lguest)?

J

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/

#6 lore raw
Quoted 9 lines · Jeremy Fitzhardinge
Quoted 7 lines · Laurent Vivier

functionnalities:

- allow to measure time spent by a CPU in a virtual CPU.
- allow to display in /proc/state this value by CPU
- allow to display in /proc/<pid>/state this value by process
- allow KVM to use these 3 previous functionnalities

Quoted 4 lines

So, currently time spent in a kvm guest is accumulated as qemu-kvm usertime, right? Given that qemu knows when its running in qemu vs guest context, couldn't it provide the breakdown between user and guest time (ditto lguest)?

No, unfortunately it's accumulated as system time.

Rusty.

#7 lore raw
In reply to Jeremy Fitzhardinge
Quoted 9 lines · Jeremy Fitzhardinge
Quoted 7 lines · Laurent Vivier

functionnalities:

- allow to measure time spent by a CPU in a virtual CPU.
- allow to display in /proc/state this value by CPU
- allow to display in /proc/<pid>/state this value by process
- allow KVM to use these 3 previous functionnalities

Quoted 4 lines

So, currently time spent in a kvm guest is accumulated as qemu-kvm usertime, right? Given that qemu knows when its running in qemu vs guest context, couldn't it provide the breakdown between user and guest time (ditto lguest)?

No, unfortunately it's accumulated as system time.

Rusty.

#8 lore raw
In reply to Jeremy Fitzhardinge
Quoted 11 lines · Jeremy Fitzhardinge
Quoted 8 lines · Laurent Vivier

functionnalities:

- allow to measure time spent by a CPU in a virtual CPU.
- allow to display in /proc/state this value by CPU
- allow to display in /proc/<pid>/state this value by process
- allow KVM to use these 3 previous functionnalities

Quoted 5 lines

So, currently time spent in a kvm guest is accumulated as qemu-kvm usertime, right? Given that qemu knows when its running in qemu vs guest context, couldn't it provide the breakdown between user and guest time (ditto lguest)?

qemu doesn't (and shouldn't) do accounting; that's best done by interrupt driven code.

The patches do account for guest time in a separate counter; guest time is added to both user time and the new counter. This allows an old 'top' to see guest time (accounted as user time), and a new 'top' to separate guest time and user time by performing the appropriate mathematical operation.

Signature · 1 line
error compiling committee.c: too many arguments to function
#9 lore raw
In reply to Jeremy Fitzhardinge
Quoted 11 lines · Jeremy Fitzhardinge
Quoted 8 lines · Laurent Vivier

functionnalities:

- allow to measure time spent by a CPU in a virtual CPU.
- allow to display in /proc/state this value by CPU
- allow to display in /proc/<pid>/state this value by process
- allow KVM to use these 3 previous functionnalities

Quoted 5 lines

So, currently time spent in a kvm guest is accumulated as qemu-kvm usertime, right? Given that qemu knows when its running in qemu vs guest context, couldn't it provide the breakdown between user and guest time (ditto lguest)?

qemu doesn't (and shouldn't) do accounting; that's best done by interrupt driven code.

The patches do account for guest time in a separate counter; guest time is added to both user time and the new counter. This allows an old 'top' to see guest time (accounted as user time), and a new 'top' to separate guest time and user time by performing the appropriate mathematical operation.

Signature · 1 line
error compiling committee.c: too many arguments to function
#10 lore raw
In reply to Laurent Vivier
Quoted 7 lines · Laurent Vivier

functionnalities:

- allow to measure time spent by a CPU in a virtual CPU.
- allow to display in /proc/state this value by CPU
- allow to display in /proc/<pid>/state this value by process
- allow KVM to use these 3 previous functionnalities

So, currently time spent in a kvm guest is accumulated as qemu-kvm usertime, right? Given that qemu knows when its running in qemu vs guest context, couldn't it provide the breakdown between user and guest time (ditto lguest)?

J

#11 lore raw
In reply to John Stoffel

Hi John,

Quoted 6 lines · John Stoffel

Laurent> The aim of these four patches is to introduce Virtual Machine Laurent> time accounting.

So what does this buy us? What increased functionality?

functionnalities:

- allow to measure time spent by a CPU in a virtual CPU.
- allow to display in /proc/state this value by CPU
- allow to display in /proc/<pid>/state this value by process
- allow KVM to use these 3 previous functionnalities

Quoted 6 lines

Laurent> [PATCH 1/4] as recent CPUs introduce a third running state, Laurent> after "user" and "system", we need a new field, "guest", in Laurent> cpustat to store the time used by the CPU to run virtual Laurent> CPU. Modify /proc/stat to display this new field.

What happens when you have multiple guests, is this state cummulative?

Yes, like system time and user time are.

Quoted 1 line

I've glanced over these patches and the #ifdefs are ugly..., esp since

#ifdefs are ugly, I agree... I can remove them.
But "#ifdef" allows to save space and time (and it is not a beauty show).

Quoted 2 lines

the overhead seems to exist even if you don't run any Guest CPUs. Or whatever you're doing here.

Can you explain ?

Quoted 2 lines

Please justify this patch more in terms of what it provides and why it's a good thing to have.

Well... it provides the time spent by CPUs in virtual CPUs.
It is usefull only if you run Virtual Machine on your host (like KVM, lguest, kqemu...) to know how many time your CPUs dedicate to virtual machine execution. It's a good thing to have like user time and system time are good things to have... do you use 'top' or 'time' sometime ?

Laurent

Signature · 2 lines
------------- [email protected]  --------------
          "Software is hard" - Donald Knuth

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/virtualization

#12 lore raw
In reply to Laurent Vivier

Laurent> The aim of these four patches is to introduce Virtual Machine Laurent> time accounting.

So what does this buy us? What increased functionality?

Laurent> [PATCH 1/4] as recent CPUs introduce a third running state, Laurent> after "user" and "system", we need a new field, "guest", in Laurent> cpustat to store the time used by the CPU to run virtual Laurent> CPU. Modify /proc/stat to display this new field.

What happens when you have multiple guests, is this state cummulative?

I've glanced over these patches and the #ifdefs are ugly..., esp since the overhead seems to exist even if you don't run any Guest CPUs. Or whatever you're doing here.

Please justify this patch more in terms of what it provides and why it's a good thing to have.

Thanks,
John

#13 lore raw
In reply to Laurent Vivier

These patches never seem to have made it onto LKML?!

Quoted 23 lines · Laurent Vivier

The aim of these four patches is to introduce Virtual Machine time accounting.

_Ingo_, as these patches modify files of the scheduler, could you have a look to them, please ?

[PATCH 1/4] as recent CPUs introduce a third running state, after "user" and "system", we need a new field, "guest", in cpustat to store the time used by the CPU to run virtual CPU. Modify /proc/stat to display this new field.

[PATCH 2/4] like for cpustat, introduce the "gtime" (guest time of the task) and "cgtime" (guest time of the task children) fields for the
tasks. Modify signal_struct and task_struct. Modify /proc/<pid>/stat to display these new fields.

[PATCH 3/4] modify account_system_time() to add cputime to cpustat->guest if we are running a VCPU. We add this cputime to cpustat->user instead of cpustat->system because this part of KVM code is in fact user code although it is executed in the kernel. We duplicate VCPU time between guest and user to allow an unmodified "top(1)" to display correct value. A modified "top(1)" is able to display good cpu user time and cpu guest time by subtracting cpu guest time from cpu user time. Update "gtime" and "cgtime" in signal_struct and task_struct accordingly.

Quoted 1 line

[PATCH 4/4] Modify KVM to update guest time accounting.

Isn't this exactly what CONFIG_VIRT_CPU_ACCOUNTING is about?

#14 lore raw
In reply to Laurent Vivier

These patches never seem to have made it onto LKML?!

Quoted 23 lines · Laurent Vivier

The aim of these four patches is to introduce Virtual Machine time accounting.

_Ingo_, as these patches modify files of the scheduler, could you have a look to them, please ?

[PATCH 1/4] as recent CPUs introduce a third running state, after "user" and "system", we need a new field, "guest", in cpustat to store the time used by the CPU to run virtual CPU. Modify /proc/stat to display this new field.

[PATCH 2/4] like for cpustat, introduce the "gtime" (guest time of the task) and "cgtime" (guest time of the task children) fields for the
tasks. Modify signal_struct and task_struct. Modify /proc/<pid>/stat to display these new fields.

[PATCH 3/4] modify account_system_time() to add cputime to cpustat->guest if we are running a VCPU. We add this cputime to cpustat->user instead of cpustat->system because this part of KVM code is in fact user code although it is executed in the kernel. We duplicate VCPU time between guest and user to allow an unmodified "top(1)" to display correct value. A modified "top(1)" is able to display good cpu user time and cpu guest time by subtracting cpu guest time from cpu user time. Update "gtime" and "cgtime" in signal_struct and task_struct accordingly.

Quoted 1 line

[PATCH 4/4] Modify KVM to update guest time accounting.

Isn't this exactly what CONFIG_VIRT_CPU_ACCOUNTING is about?

#15 lore raw
Quoted 26 lines · On Tue, 04 Aug 2009 16:16:38 +0200 Peter Zijlstra <[email protected]> wrote:

These patches never seem to have made it onto LKML?!

Quoted 23 lines · Laurent Vivier

The aim of these four patches is to introduce Virtual Machine time accounting.

_Ingo_, as these patches modify files of the scheduler, could you have a look to them, please ?

[PATCH 1/4] as recent CPUs introduce a third running state, after "user" and "system", we need a new field, "guest", in cpustat to store the time used by the CPU to run virtual CPU. Modify /proc/stat to display this new field.

[PATCH 2/4] like for cpustat, introduce the "gtime" (guest time of the task) and "cgtime" (guest time of the task children) fields for the
tasks. Modify signal_struct and task_struct. Modify /proc/<pid>/stat to display these new fields.

[PATCH 3/4] modify account_system_time() to add cputime to cpustat->guest if we are running a VCPU. We add this cputime to cpustat->user instead of cpustat->system because this part of KVM code is in fact user code although it is executed in the kernel. We duplicate VCPU time between guest and user to allow an unmodified "top(1)" to display correct value. A modified "top(1)" is able to display good cpu user time and cpu guest time by subtracting cpu guest time from cpu user time. Update "gtime" and "cgtime" in signal_struct and task_struct accordingly.

Quoted 3 lines
Quoted 1 line

[PATCH 4/4] Modify KVM to update guest time accounting.

Isn't this exactly what CONFIG_VIRT_CPU_ACCOUNTING is about?

Not really, CONFIG_VIRT_CPU_ACCOUNT is a mechanism to sort out the steal time in the >guest< and to increase the precision of the cpu accounting values in general. The patches from Laurent improve the code in the >host< that sorts out guest time vs. system time. The patches do make sense to me.

Signature · 4 lines
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.
#16 lore raw
In reply to Peter Zijlstra
Quoted 26 lines · On Tue, 04 Aug 2009 16:16:38 +0200 Peter Zijlstra <[email protected]> wrote:

These patches never seem to have made it onto LKML?!

Quoted 23 lines · Laurent Vivier

The aim of these four patches is to introduce Virtual Machine time accounting.

_Ingo_, as these patches modify files of the scheduler, could you have a look to them, please ?

[PATCH 1/4] as recent CPUs introduce a third running state, after "user" and "system", we need a new field, "guest", in cpustat to store the time used by the CPU to run virtual CPU. Modify /proc/stat to display this new field.

[PATCH 2/4] like for cpustat, introduce the "gtime" (guest time of the task) and "cgtime" (guest time of the task children) fields for the
tasks. Modify signal_struct and task_struct. Modify /proc/<pid>/stat to display these new fields.

[PATCH 3/4] modify account_system_time() to add cputime to cpustat->guest if we are running a VCPU. We add this cputime to cpustat->user instead of cpustat->system because this part of KVM code is in fact user code although it is executed in the kernel. We duplicate VCPU time between guest and user to allow an unmodified "top(1)" to display correct value. A modified "top(1)" is able to display good cpu user time and cpu guest time by subtracting cpu guest time from cpu user time. Update "gtime" and "cgtime" in signal_struct and task_struct accordingly.

Quoted 3 lines
Quoted 1 line

[PATCH 4/4] Modify KVM to update guest time accounting.

Isn't this exactly what CONFIG_VIRT_CPU_ACCOUNTING is about?

Not really, CONFIG_VIRT_CPU_ACCOUNT is a mechanism to sort out the steal time in the >guest< and to increase the precision of the cpu accounting values in general. The patches from Laurent improve the code in the >host< that sorts out guest time vs. system time. The patches do make sense to me.

Signature · 4 lines
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.
#17 lore raw
Quoted 29 lines · Martin Schwidefsky
Quoted 26 lines · On Tue, 04 Aug 2009 16:16:38 +0200 Peter Zijlstra <[email protected]> wrote:

These patches never seem to have made it onto LKML?!

Quoted 23 lines · Laurent Vivier

The aim of these four patches is to introduce Virtual Machine time accounting.

_Ingo_, as these patches modify files of the scheduler, could you have a look to them, please ?

[PATCH 1/4] as recent CPUs introduce a third running state, after "user" and "system", we need a new field, "guest", in cpustat to store the time used by the CPU to run virtual CPU. Modify /proc/stat to display this new field.

[PATCH 2/4] like for cpustat, introduce the "gtime" (guest time of the task) and "cgtime" (guest time of the task children) fields for the
tasks. Modify signal_struct and task_struct. Modify /proc/<pid>/stat to display these new fields.

[PATCH 3/4] modify account_system_time() to add cputime to cpustat->guest if we are running a VCPU. We add this cputime to cpustat->user instead of cpustat->system because this part of KVM code is in fact user code although it is executed in the kernel. We duplicate VCPU time between guest and user to allow an unmodified "top(1)" to display correct value. A modified "top(1)" is able to display good cpu user time and cpu guest time by subtracting cpu guest time from cpu user time. Update "gtime" and "cgtime" in signal_struct and task_struct accordingly.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

[PATCH 4/4] Modify KVM to update guest time accounting.

Isn't this exactly what CONFIG_VIRT_CPU_ACCOUNTING is about?

Not really, CONFIG_VIRT_CPU_ACCOUNT is a mechanism to sort out the steal time in the >guest< and to increase the precision of the cpu accounting values in general. The patches from Laurent improve the code in the >host< that sorts out guest time vs. system time.

Ah, that wasn't clear, it keeps mentioning guest all over the place ;-)

So its going to split user time into user and guest. Does that really make sense? For the host kernel it really is just another user process, no?

Quoted 1 line

The patches do make sense to me.

No idea there as they never made it to my lkml folder.

#18 lore raw
In reply to Martin Schwidefsky
Quoted 29 lines · Martin Schwidefsky
Quoted 26 lines · On Tue, 04 Aug 2009 16:16:38 +0200 Peter Zijlstra <[email protected]> wrote:

These patches never seem to have made it onto LKML?!

Quoted 23 lines · Laurent Vivier

The aim of these four patches is to introduce Virtual Machine time accounting.

_Ingo_, as these patches modify files of the scheduler, could you have a look to them, please ?

[PATCH 1/4] as recent CPUs introduce a third running state, after "user" and "system", we need a new field, "guest", in cpustat to store the time used by the CPU to run virtual CPU. Modify /proc/stat to display this new field.

[PATCH 2/4] like for cpustat, introduce the "gtime" (guest time of the task) and "cgtime" (guest time of the task children) fields for the
tasks. Modify signal_struct and task_struct. Modify /proc/<pid>/stat to display these new fields.

[PATCH 3/4] modify account_system_time() to add cputime to cpustat->guest if we are running a VCPU. We add this cputime to cpustat->user instead of cpustat->system because this part of KVM code is in fact user code although it is executed in the kernel. We duplicate VCPU time between guest and user to allow an unmodified "top(1)" to display correct value. A modified "top(1)" is able to display good cpu user time and cpu guest time by subtracting cpu guest time from cpu user time. Update "gtime" and "cgtime" in signal_struct and task_struct accordingly.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

[PATCH 4/4] Modify KVM to update guest time accounting.

Isn't this exactly what CONFIG_VIRT_CPU_ACCOUNTING is about?

Not really, CONFIG_VIRT_CPU_ACCOUNT is a mechanism to sort out the steal time in the >guest< and to increase the precision of the cpu accounting values in general. The patches from Laurent improve the code in the >host< that sorts out guest time vs. system time.

Ah, that wasn't clear, it keeps mentioning guest all over the place ;-)

So its going to split user time into user and guest. Does that really make sense? For the host kernel it really is just another user process, no?

Quoted 1 line

The patches do make sense to me.

No idea there as they never made it to my lkml folder.

#19 lore raw
Quoted 39 lines · On Tue, 04 Aug 2009 18:26:41 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 29 lines · Martin Schwidefsky
Quoted 26 lines · On Tue, 04 Aug 2009 16:16:38 +0200 Peter Zijlstra <[email protected]> wrote:

These patches never seem to have made it onto LKML?!

Quoted 23 lines · Laurent Vivier

The aim of these four patches is to introduce Virtual Machine time accounting.

_Ingo_, as these patches modify files of the scheduler, could you have a look to them, please ?

[PATCH 1/4] as recent CPUs introduce a third running state, after "user" and "system", we need a new field, "guest", in cpustat to store the time used by the CPU to run virtual CPU. Modify /proc/stat to display this new field.

[PATCH 2/4] like for cpustat, introduce the "gtime" (guest time of the task) and "cgtime" (guest time of the task children) fields for the
tasks. Modify signal_struct and task_struct. Modify /proc/<pid>/stat to display these new fields.

[PATCH 3/4] modify account_system_time() to add cputime to cpustat->guest if we are running a VCPU. We add this cputime to cpustat->user instead of cpustat->system because this part of KVM code is in fact user code although it is executed in the kernel. We duplicate VCPU time between guest and user to allow an unmodified "top(1)" to display correct value. A modified "top(1)" is able to display good cpu user time and cpu guest time by subtracting cpu guest time from cpu user time. Update "gtime" and "cgtime" in signal_struct and task_struct accordingly.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

[PATCH 4/4] Modify KVM to update guest time accounting.

Isn't this exactly what CONFIG_VIRT_CPU_ACCOUNTING is about?

Not really, CONFIG_VIRT_CPU_ACCOUNT is a mechanism to sort out the steal time in the >guest< and to increase the precision of the cpu accounting values in general. The patches from Laurent improve the code in the >host< that sorts out guest time vs. system time.

Quoted 5 lines

Ah, that wasn't clear, it keeps mentioning guest all over the place ;-)

So its going to split user time into user and guest. Does that really make sense? For the host kernel it really is just another user process, no?

The code (at least in parts) is already upstream. Look at the account_guest_time function:

static void account_guest_time(struct task_struct *p, cputime_t cputime,
cputime_t cputime_scaled)

{
        cputime64_t tmp;
        struct cpu_usage_stat *cpustat = &kstat_this_cpu.cpustat;

        tmp = cputime_to_cputime64(cputime);

        /* Add guest time to process. */
        p->utime = cputime_add(p->utime, cputime);
        p->utimescaled = cputime_add(p->utimescaled, cputime_scaled);
        account_group_user_time(p, cputime);
        p->gtime = cputime_add(p->gtime, cputime);

        /* Add guest time to cpustat. */
        cpustat->user = cputime64_add(cpustat->user, tmp);
        cpustat->guest = cputime64_add(cpustat->guest, tmp);
}

The cpu time for a guest is added to p->utime AND p->gtime. That is done not to break existing tools that know nothing about guest time. A guest time aware tool can subtract the p->gtime from p->utime to get the time spent by the process outside of the guest context.

Signature · 4 lines
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.
#20 lore raw
In reply to Peter Zijlstra
Quoted 39 lines · On Tue, 04 Aug 2009 18:26:41 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 29 lines · Martin Schwidefsky
Quoted 26 lines · On Tue, 04 Aug 2009 16:16:38 +0200 Peter Zijlstra <[email protected]> wrote:

These patches never seem to have made it onto LKML?!

Quoted 23 lines · Laurent Vivier

The aim of these four patches is to introduce Virtual Machine time accounting.

_Ingo_, as these patches modify files of the scheduler, could you have a look to them, please ?

[PATCH 1/4] as recent CPUs introduce a third running state, after "user" and "system", we need a new field, "guest", in cpustat to store the time used by the CPU to run virtual CPU. Modify /proc/stat to display this new field.

[PATCH 2/4] like for cpustat, introduce the "gtime" (guest time of the task) and "cgtime" (guest time of the task children) fields for the
tasks. Modify signal_struct and task_struct. Modify /proc/<pid>/stat to display these new fields.

[PATCH 3/4] modify account_system_time() to add cputime to cpustat->guest if we are running a VCPU. We add this cputime to cpustat->user instead of cpustat->system because this part of KVM code is in fact user code although it is executed in the kernel. We duplicate VCPU time between guest and user to allow an unmodified "top(1)" to display correct value. A modified "top(1)" is able to display good cpu user time and cpu guest time by subtracting cpu guest time from cpu user time. Update "gtime" and "cgtime" in signal_struct and task_struct accordingly.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

[PATCH 4/4] Modify KVM to update guest time accounting.

Isn't this exactly what CONFIG_VIRT_CPU_ACCOUNTING is about?

Not really, CONFIG_VIRT_CPU_ACCOUNT is a mechanism to sort out the steal time in the >guest< and to increase the precision of the cpu accounting values in general. The patches from Laurent improve the code in the >host< that sorts out guest time vs. system time.

Quoted 5 lines

Ah, that wasn't clear, it keeps mentioning guest all over the place ;-)

So its going to split user time into user and guest. Does that really make sense? For the host kernel it really is just another user process, no?

The code (at least in parts) is already upstream. Look at the account_guest_time function:

static void account_guest_time(struct task_struct *p, cputime_t cputime,
cputime_t cputime_scaled)

{
        cputime64_t tmp;
        struct cpu_usage_stat *cpustat = &kstat_this_cpu.cpustat;

        tmp = cputime_to_cputime64(cputime);

        /* Add guest time to process. */
        p->utime = cputime_add(p->utime, cputime);
        p->utimescaled = cputime_add(p->utimescaled, cputime_scaled);
        account_group_user_time(p, cputime);
        p->gtime = cputime_add(p->gtime, cputime);

        /* Add guest time to cpustat. */
        cpustat->user = cputime64_add(cpustat->user, tmp);
        cpustat->guest = cputime64_add(cpustat->guest, tmp);
}

The cpu time for a guest is added to p->utime AND p->gtime. That is done not to break existing tools that know nothing about guest time. A guest time aware tool can subtract the p->gtime from p->utime to get the time spent by the process outside of the guest context.

Signature · 4 lines
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.
#21 lore raw
Quoted 26 lines · Martin Schwidefsky
Quoted 3 lines

So its going to split user time into user and guest. Does that really make sense? For the host kernel it really is just another user process, no?

The code (at least in parts) is already upstream. Look at the account_guest_time function:

static void account_guest_time(struct task_struct *p, cputime_t cputime,
cputime_t cputime_scaled)

{
        cputime64_t tmp;
        struct cpu_usage_stat *cpustat = &kstat_this_cpu.cpustat;

        tmp = cputime_to_cputime64(cputime);

        /* Add guest time to process. */
        p->utime = cputime_add(p->utime, cputime);
        p->utimescaled = cputime_add(p->utimescaled, cputime_scaled);
        account_group_user_time(p, cputime);
        p->gtime = cputime_add(p->gtime, cputime);

        /* Add guest time to cpustat. */
        cpustat->user = cputime64_add(cpustat->user, tmp);
        cpustat->guest = cputime64_add(cpustat->guest, tmp);
}
Quoted 4 lines

The cpu time for a guest is added to p->utime AND p->gtime. That is done not to break existing tools that know nothing about guest time. A guest time aware tool can subtract the p->gtime from p->utime to get the time spent by the process outside of the guest context.

But why? How a vcpu anything other than yet another userspace process?

#22 lore raw
In reply to Martin Schwidefsky
Quoted 26 lines · Martin Schwidefsky
Quoted 3 lines

So its going to split user time into user and guest. Does that really make sense? For the host kernel it really is just another user process, no?

The code (at least in parts) is already upstream. Look at the account_guest_time function:

static void account_guest_time(struct task_struct *p, cputime_t cputime,
cputime_t cputime_scaled)

{
        cputime64_t tmp;
        struct cpu_usage_stat *cpustat = &kstat_this_cpu.cpustat;

        tmp = cputime_to_cputime64(cputime);

        /* Add guest time to process. */
        p->utime = cputime_add(p->utime, cputime);
        p->utimescaled = cputime_add(p->utimescaled, cputime_scaled);
        account_group_user_time(p, cputime);
        p->gtime = cputime_add(p->gtime, cputime);

        /* Add guest time to cpustat. */
        cpustat->user = cputime64_add(cpustat->user, tmp);
        cpustat->guest = cputime64_add(cpustat->guest, tmp);
}
Quoted 4 lines

The cpu time for a guest is added to p->utime AND p->gtime. That is done not to break existing tools that know nothing about guest time. A guest time aware tool can subtract the p->gtime from p->utime to get the time spent by the process outside of the guest context.

But why? How a vcpu anything other than yet another userspace process?

#23 lore raw
Quoted 28 lines · On Tue, 04 Aug 2009 19:33:14 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 26 lines · Martin Schwidefsky
Quoted 3 lines

So its going to split user time into user and guest. Does that really make sense? For the host kernel it really is just another user process, no?

The code (at least in parts) is already upstream. Look at the account_guest_time function:

static void account_guest_time(struct task_struct *p, cputime_t cputime,
cputime_t cputime_scaled)

{
        cputime64_t tmp;
        struct cpu_usage_stat *cpustat = &kstat_this_cpu.cpustat;

        tmp = cputime_to_cputime64(cputime);

        /* Add guest time to process. */
        p->utime = cputime_add(p->utime, cputime);
        p->utimescaled = cputime_add(p->utimescaled, cputime_scaled);
        account_group_user_time(p, cputime);
        p->gtime = cputime_add(p->gtime, cputime);

        /* Add guest time to cpustat. */
        cpustat->user = cputime64_add(cpustat->user, tmp);
        cpustat->guest = cputime64_add(cpustat->guest, tmp);
}
Quoted 6 lines
Quoted 4 lines

The cpu time for a guest is added to p->utime AND p->gtime. That is done not to break existing tools that know nothing about guest time. A guest time aware tool can subtract the p->gtime from p->utime to get the time spent by the process outside of the guest context.

But why? How a vcpu anything other than yet another userspace process?

Because you have two different contexts where you spent time, the "normal" process context and the "guest" process context. To know how much of your cpu time is spent on virtualization overhead is an interesting number.

Signature · 4 lines
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.
#24 lore raw
In reply to Peter Zijlstra
Quoted 28 lines · On Tue, 04 Aug 2009 19:33:14 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 26 lines · Martin Schwidefsky
Quoted 3 lines

So its going to split user time into user and guest. Does that really make sense? For the host kernel it really is just another user process, no?

The code (at least in parts) is already upstream. Look at the account_guest_time function:

static void account_guest_time(struct task_struct *p, cputime_t cputime,
cputime_t cputime_scaled)

{
        cputime64_t tmp;
        struct cpu_usage_stat *cpustat = &kstat_this_cpu.cpustat;

        tmp = cputime_to_cputime64(cputime);

        /* Add guest time to process. */
        p->utime = cputime_add(p->utime, cputime);
        p->utimescaled = cputime_add(p->utimescaled, cputime_scaled);
        account_group_user_time(p, cputime);
        p->gtime = cputime_add(p->gtime, cputime);

        /* Add guest time to cpustat. */
        cpustat->user = cputime64_add(cpustat->user, tmp);
        cpustat->guest = cputime64_add(cpustat->guest, tmp);
}
Quoted 6 lines
Quoted 4 lines

The cpu time for a guest is added to p->utime AND p->gtime. That is done not to break existing tools that know nothing about guest time. A guest time aware tool can subtract the p->gtime from p->utime to get the time spent by the process outside of the guest context.

But why? How a vcpu anything other than yet another userspace process?

Because you have two different contexts where you spent time, the "normal" process context and the "guest" process context. To know how much of your cpu time is spent on virtualization overhead is an interesting number.

Signature · 4 lines
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.
#25 lore raw
In reply to Peter Zijlstra
Quoted 29 lines · Peter Zijlstra
Quoted 26 lines · Martin Schwidefsky
Quoted 4 lines

So its going to split user time into user and guest. Does that really make sense? For the host kernel it really is just another user process, no?

The code (at least in parts) is already upstream. Look at the account_guest_time function:

static void account_guest_time(struct task_struct *p, cputime_t cputime,
cputime_t cputime_scaled)

{
        cputime64_t tmp;
        struct cpu_usage_stat *cpustat = &kstat_this_cpu.cpustat;

        tmp = cputime_to_cputime64(cputime);

        /* Add guest time to process. */
        p->utime = cputime_add(p->utime, cputime);
        p->utimescaled = cputime_add(p->utimescaled, cputime_scaled);
        account_group_user_time(p, cputime);
        p->gtime = cputime_add(p->gtime, cputime);

        /* Add guest time to cpustat. */
        cpustat->user = cputime64_add(cpustat->user, tmp);
        cpustat->guest = cputime64_add(cpustat->guest, tmp);
}
Quoted 8 lines
Quoted 5 lines

The cpu time for a guest is added to p->utime AND p->gtime. That is done not to break existing tools that know nothing about guest time. A guest time aware tool can subtract the p->gtime from p->utime to get the time spent by the process outside of the guest context.

But why? How a vcpu anything other than yet another userspace process?

Normally a running process can either be accumulating user or system time. From the host's perspective, the time spent running a vcpu is more or less a form of system time[*], but the CPU happens to be in an odd state (for kvm it would be in an actual completely different CPU mode; for lguest it would just be configured unusually).

[* I think? I assume qemu goes into a syscall for a while to run the vcpu, then returns to qemu userspace when it needs to handle some exception ]

I don't see that vcpu time is so particularly distinct from any other kind of system time that it really needs a whole special form of time accounting, unless the scheduler is being modified to schedule guests differently from normal processes. Otherwise it just seems like something that could be relegated to one of the existing tracing/benchmarking/profiling mechanisms.

(Full disclosure: these patches are completely irrelevant to Xen.)

J

#26 lore raw
In reply to Peter Zijlstra
Quoted 29 lines · Peter Zijlstra
Quoted 26 lines · Martin Schwidefsky
Quoted 4 lines

So its going to split user time into user and guest. Does that really make sense? For the host kernel it really is just another user process, no?

The code (at least in parts) is already upstream. Look at the account_guest_time function:

static void account_guest_time(struct task_struct *p, cputime_t cputime,
cputime_t cputime_scaled)

{
        cputime64_t tmp;
        struct cpu_usage_stat *cpustat = &kstat_this_cpu.cpustat;

        tmp = cputime_to_cputime64(cputime);

        /* Add guest time to process. */
        p->utime = cputime_add(p->utime, cputime);
        p->utimescaled = cputime_add(p->utimescaled, cputime_scaled);
        account_group_user_time(p, cputime);
        p->gtime = cputime_add(p->gtime, cputime);

        /* Add guest time to cpustat. */
        cpustat->user = cputime64_add(cpustat->user, tmp);
        cpustat->guest = cputime64_add(cpustat->guest, tmp);
}
Quoted 8 lines
Quoted 5 lines

The cpu time for a guest is added to p->utime AND p->gtime. That is done not to break existing tools that know nothing about guest time. A guest time aware tool can subtract the p->gtime from p->utime to get the time spent by the process outside of the guest context.

But why? How a vcpu anything other than yet another userspace process?

Normally a running process can either be accumulating user or system time. From the host's perspective, the time spent running a vcpu is more or less a form of system time[*], but the CPU happens to be in an odd state (for kvm it would be in an actual completely different CPU mode; for lguest it would just be configured unusually).

[* I think? I assume qemu goes into a syscall for a while to run the vcpu, then returns to qemu userspace when it needs to handle some exception ]

I don't see that vcpu time is so particularly distinct from any other kind of system time that it really needs a whole special form of time accounting, unless the scheduler is being modified to schedule guests differently from normal processes. Otherwise it just seems like something that could be relegated to one of the existing tracing/benchmarking/profiling mechanisms.

(Full disclosure: these patches are completely irrelevant to Xen.)

J