FPU LWP 0/5: patch description

RFC PATCH In review Wei Huang 3 participants 5 messages Apr 14, 2011 – Apr 15, 2011

lore

Share & tools

lore

Wei Huang

author

2011-04-14 20:37 UTC #0 lore raw

RFC PATCH 1/1 FPU LWP 0/5: patch description

The following patches support AMD lightweight profiling.

Because LWP isn't tracked by CR0.TS bit, we clean up the FPU code to handle lazy and unlazy FPU states differently. Lazy FPU state (such as SSE, YMM) is handled when #NM is triggered. Unlazy state, such as LWP, is saved and restored on each vcpu context switch. To simplify the code, we also add a mask option to xsave/xrstor function.

Thanks,
-Wei

Keir Fraser

2011-04-14 21:09 UTC #1 lore raw

Quoted 7 lines · Wei Huang

The following patches support AMD lightweight profiling.
Because LWP isn't tracked by CR0.TS bit, we clean up the FPU code to handle lazy and unlazy FPU states differently. Lazy FPU state (such as SSE, YMM) is handled when #NM is triggered. Unlazy state, such as LWP, is saved and restored on each vcpu context switch. To simplify the code, we also add a mask option to xsave/xrstor function.

How much cost is added to context switch paths in the (overwhelmingly likely) case that LWP is not being used by the guest? Is this adding a whole lot of unconditional overhead for a feature that noone uses?

-- Keir

Quoted 9 lines

Thanks,
-Wei
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

Wei Huang

author

2011-04-14 22:57 UTC #2 lore raw

Hi Keir,

I ran a quick test to calculate the overhead of __fpu_unlazy_save() and __fpu_unlazy_restore(), which are used to save/restore LWP state. Here are the results:

(1) tsc_total: total time used for context_switch() in x86/domain.c
(2) tsc_unlazy: total time used for __fpu_unlazy_save() + __fpu_unlazy_retore()

One example:
(XEN) tsc_unlazy=0x00000000008ae174
(XEN) tsc_total=0x00000001028b4907

So the overhead is about 0.2% of total time used by context_switch(). Of course, this is just one example. I would say the overhead ratio would be <1% for most cases.

Thanks,
-Wei

Quoted 26 lines · Keir Fraser

Quoted 7 lines · Wei Huang
The following patches support AMD lightweight profiling.
Because LWP isn't tracked by CR0.TS bit, we clean up the FPU code to handle lazy and unlazy FPU states differently. Lazy FPU state (such as SSE, YMM) is handled when #NM is triggered. Unlazy state, such as LWP, is saved and restored on each vcpu context switch. To simplify the code, we also add a mask option to xsave/xrstor function.
How much cost is added to context switch paths in the (overwhelmingly likely) case that LWP is not being used by the guest? Is this adding a whole lot of unconditional overhead for a feature that noone uses?
-- Keir
Quoted 9 lines
Thanks,
-Wei
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

Dan Magenheimer

2011-04-15 20:16 UTC #3 lore raw

Wait... a context switch takes over 4 billion cycles? Not likely!

And please check your division. I get the same answer from "dc" only when I use lowercase hex numbers and dc complains about unimplemented chars, else I get 0.033%... also unlikely.

Quoted 69 lines

-----Original Message-----
From: Wei Huang [mailto:[email protected]]
Sent: Thursday, April 14, 2011 4:57 PM
To: Keir Fraser
Cc: [email protected]
Subject: Re: [Xen-devel] [PATCH][RFC] FPU LWP 0/5: patch description
Hi Keir,
I ran a quick test to calculate the overhead of __fpu_unlazy_save() and __fpu_unlazy_restore(), which are used to save/restore LWP state. Here are the results:
(1) tsc_total: total time used for context_switch() in x86/domain.c
(2) tsc_unlazy: total time used for __fpu_unlazy_save() + __fpu_unlazy_retore()
One example:
(XEN) tsc_unlazy=0x00000000008ae174
(XEN) tsc_total=0x00000001028b4907
So the overhead is about 0.2% of total time used by context_switch(). Of
course, this is just one example. I would say the overhead ratio would be <1% for most cases.
Thanks,
-Wei
Quoted 6 lines · Keir Fraser
Quoted 4 lines · Wei Huang
The following patches support AMD lightweight profiling.
Because LWP isn't tracked by CR0.TS bit, we clean up the FPU code to handle lazy and unlazy FPU states differently. Lazy FPU state (such
as
Quoted 1 line
Quoted 1 line
SSE, YMM) is handled when #NM is triggered. Unlazy state, such as
LWP,
Quoted 1 line
Quoted 1 line
is saved and restored on each vcpu context switch. To simplify the
code,
Quoted 3 lines
Quoted 1 line
we also add a mask option to xsave/xrstor function.
How much cost is added to context switch paths in the (overwhelmingly likely) case that LWP is not being used by the guest? Is this adding
a whole
Quoted 15 lines
lot of unconditional overhead for a feature that noone uses?
-- Keir
Quoted 9 lines
Thanks,
-Wei
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

Huang2, Wei

author

2011-04-15 20:23 UTC #4 lore raw

Hi Dan,

This isn't the cycles of a single switch. This is the total cycle count (added) over a period. I randomly dumped the numbers when a guest was running.

Thanks,
-Wei

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Dan Magenheimer
Sent: Friday, April 15, 2011 3:16 PM
To: Huang2, Wei; Keir Fraser
Cc: [email protected]
Subject: RE: [Xen-devel] [PATCH][RFC] FPU LWP 0/5: patch description

Wait... a context switch takes over 4 billion cycles? Not likely!

And please check your division. I get the same answer from "dc" only when I use lowercase hex numbers and dc complains about unimplemented chars, else I get 0.033%... also unlikely.

Quoted 65 lines

-----Original Message-----
From: Wei Huang [mailto:[email protected]]
Sent: Thursday, April 14, 2011 4:57 PM
To: Keir Fraser
Cc: [email protected]
Subject: Re: [Xen-devel] [PATCH][RFC] FPU LWP 0/5: patch description
Hi Keir,
I ran a quick test to calculate the overhead of __fpu_unlazy_save() and __fpu_unlazy_restore(), which are used to save/restore LWP state. Here are the results:
(1) tsc_total: total time used for context_switch() in x86/domain.c
(2) tsc_unlazy: total time used for __fpu_unlazy_save() + __fpu_unlazy_retore()
One example:
(XEN) tsc_unlazy=0x00000000008ae174
(XEN) tsc_total=0x00000001028b4907
So the overhead is about 0.2% of total time used by context_switch(). Of
course, this is just one example. I would say the overhead ratio would be <1% for most cases.
Thanks,
-Wei
Quoted 6 lines · Keir Fraser
Quoted 4 lines · Wei Huang
The following patches support AMD lightweight profiling.
Because LWP isn't tracked by CR0.TS bit, we clean up the FPU code to handle lazy and unlazy FPU states differently. Lazy FPU state (such
as
Quoted 1 line
Quoted 1 line
SSE, YMM) is handled when #NM is triggered. Unlazy state, such as
LWP,
Quoted 1 line
Quoted 1 line
is saved and restored on each vcpu context switch. To simplify the
code,
Quoted 3 lines
Quoted 1 line
we also add a mask option to xsave/xrstor function.
How much cost is added to context switch paths in the (overwhelmingly likely) case that LWP is not being used by the guest? Is this adding
a whole
Quoted 15 lines
lot of unconditional overhead for a feature that noone uses?
-- Keir
Quoted 9 lines
Thanks,
-Wei
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

Quoted 4 lines

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel