x86/amd: fix crash as Xen Dom0 on AMD Trinity systems
Subsystems (3)
PATCH 1/1 x86/amd: fix crash as Xen Dom0 on AMD Trinity systems
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
This fixes a crash when using the kernel as a Xen Dom0 on affected Trinity systems. The wrmsrl_amd_safe is not properly paravirtualized yet (this will be fixed in another patch).
--- arch/x86/kernel/cpu/amd.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/amd.c +2 -2
@@ -586,9 +586,9 @@ static void __cpuinit init_amd(struct cpuinfo_x86 *c) !cpu_has(c, X86_FEATURE_TOPOEXT)) {u64 val; -if (!rdmsrl_amd_safe (0xc0011005 , &val)) { +if (!rdmsrl_safe (0xc0011005 , &val)) { val |=1ULL <<54 ; -wrmsrl_amd_safe (0xc0011005 , val); +checking_wrmsrl (0xc0011005 , val);rdmsrl (0xc0011005 , val);if (val & (1ULL <<54 )) {set_cpu_cap (c,X86_FEATURE_TOPOEXT );
Signature · 1 line
1.7.4.4
Quoted 7 lines
Quoted 1 line
Quoted 1 line · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
This fixes a crash when using the kernel as a Xen Dom0 on affected Trinity systems. Thewrmsrl_amd_safeis not properly paravirtualized yet (this will be fixed in another patch).
I'm not following: If the AMD variants (putting a special value into %edi) can be freely replaced by the non-AMD variants, why did the AMD special ones get used in the first place?
Further, I can't see how checking_wrmsrl() is being paravirtualized any better than wrmsrl_amd_safe() - both have nothing but an exception handling fixup attached to the wrmsr invocation. Care to point out what actual crash it is that was seen?
Finally, I would question whether re-enabling the topology extensions under Xen shouldn't be skipped altogether, perhaps even on Dom0 (as the hypervisor is controlling this MSR, but in any case on DomU - the hypervisor won't allow (read: ignore, not fault on) the write anyway (and will log a message for each (v)CPU that attempts this).
Jan
Quoted 22 lines
Signed-off-by Andre Przywara <[email protected]>Cc [email protected] # 3.4+--- arch/x86/kernel/cpu/amd.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)arch/x86/kernel/cpu/amd.c +2
-2@@ -586,9 +586,9 @@ static void __cpuinit init_amd(struct cpuinfo_x86 *c) !cpu_has(c, X86_FEATURE_TOPOEXT)) {u64 val; -if (!rdmsrl_amd_safe (0xc0011005 , &val)) { +if (!rdmsrl_safe (0xc0011005 , &val)) { val |=1ULL <<54 ; -wrmsrl_amd_safe (0xc0011005 , val); +checking_wrmsrl (0xc0011005 , val);rdmsrl (0xc0011005 , val);if (val & (1ULL <<54 )) {set_cpu_cap (c,X86_FEATURE_TOPOEXT );
Quoted 11 lines · Jan Beulich
Quoted 7 lines
Quoted 1 line
Quoted 1 line · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
This fixes a crash when using the kernel as a Xen Dom0 on affected Trinity systems. Thewrmsrl_amd_safeis not properly paravirtualized yet (this will be fixed in another patch).I'm not following: If the AMD variants (putting a special value into %edi) can be freely replaced by the non-AMD variants, why did the AMD special ones get used in the first place?
Older CPUs (K8) needed the AMD variants, starting with family 10h we can use the normal versions.
Quoted 4 lines
Further, I can't see how
checking_wrmsrl()is being paravirtualized any better thanwrmsrl_amd_safe()- both have nothing but an exception handling fixup attached to the wrmsr invocation. Care to point out what actual crash it is that was seen?
AFAIK, the difference is between the "l" and the regs version for rd/wrmsr. We have a patch already here to fix this. Will send it out soon. Jacob, can you comment on this?
The crash dump:
[ 1.601021] ------------[ cut here ]------------ [ 1.601025] kernel BUG at /buildrepos/linux-2.6-stable/arch/x86/include/asm/paravirt.h:133! [ 1.601031] invalid opcode: 0000 [#1] SMP [ 1.601038] CPU 0 [ 1.601041] Modules linked in: [ 1.601047] [ 1.601050] Pid: 0, comm: swapper/0 Not tainted 3.4.0 #1 AMD Virgo Platform/Annapurna [ 1.601061] RIP: e030:[<ffffffff8169b4fe>] [<ffffffff8169b4fe>] init_amd+0x21d/0x603 [ 1.601072] RSP: e02b:ffffffff81c01df8 EFLAGS: 00010246 [ 1.601076] RAX: 0000000000000000 RBX: ffffffff81ca7500 RCX: 0000000000000000 [ 1.601081] RDX: 0000000000000020 RSI: 000000000000c000 RDI: ffffffff81c01e78 [ 1.601086] RBP: ffffffff81c01ea8 R08: 0000000000004003 R09: 00000000ffffffff [ 1.601090] R10: 0000000000000000 R11: 00000000ffffffff R12: ffffffff81ca7574 [ 1.601095] R13: ffffffff81ca7514 R14: ffffffff81ca754c R15: ffffffff81ca756c [ 1.601103] FS: 0000000000000000(0000) GS:ffff8801ce600000(0000) knlGS:0000000000000000 [ 1.601108] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1.601112] CR2: 0000000000000000 CR3: 0000000001c0c000 CR4: 0000000000040660 [ 1.601117] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1.601122] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1.601127] Process swapper/0 (pid: 0, threadinfo ffffffff81c00000, task ffffffff81c14020) [ 1.601131] Stack: [ 1.601134] ffffffff81c01ea8 ffffffff8169a7d8 0000001600000017 0000000000000000 [ 1.601146] ffff8801c1c3ac00 ffff8801c1c002d0 ffffffff81c01e48 ffffffff81140118 [ 1.601157] ffffffff811415cc ffff8801c1c04220 ffff8801c1c02480 00000000000000d0 [ 1.601169] Call Trace: [ 1.601175] [<ffffffff8169a7d8>] ? get_cpu_cap+0x1f2/0x201 [ 1.601183] [<ffffffff81140118>] ? init_kmem_cache_cpus+0x3e/0x4d [ 1.601189] [<ffffffff811415cc>] ? sysfs_slab_alias+0x60/0x8d [ 1.601195] [<ffffffff8169aa3d>] identify_cpu+0x256/0x34d [ 1.601201] [<ffffffff81141649>] ? kmem_cache_alloc+0x50/0xe4 [ 1.601209] [<ffffffff81cd1c51>] identify_boot_cpu+0x10/0x3c [ 1.601216] [<ffffffff81cd2052>] check_bugs+0x9/0x2d [ 1.601222] [<ffffffff81cc7e08>] start_kernel+0x445/0x461 [ 1.601227] [<ffffffff81cc77d6>] ? kernel_init+0x1d8/0x1d8 [ 1.601233] [<ffffffff81cc72cf>] x86_64_start_reservations+0xba/0xc1 [ 1.601240] [<ffffffff81041797>] ? xen_setup_runstate_info+0x2c/0x36 [ 1.601247] [<ffffffff81ccbdc4>] xen_start_kernel+0x58b/0x592 [ 1.601251] Code: 0f 85 d3 00 00 00 31 c0 48 83 3d cd 5c 58 00 00 48 8d 7d b0 b9 08 00 00 00 f3 ab c7 45 b4 05 10 01 c0 c7 45 cc 3a 20 5a 9c 75 04 <0f> 0b eb fe 4c 8d 75 b0 4c 89 f7 ff 14 25 b0 11 c2 81 85 c0 8b [ 1.601374] RIP [<ffffffff8169b4fe>] init_amd+0x21d/0x603 [ 1.601381] RSP <ffffffff81c01df8> [ 1.601391] ---[ end trace a7919e7f17c0a725 ]--- [ 1.601397] Kernel panic - not syncing: Attempted to kill the idle task!
Quoted 6 lines
Finally, I would question whether re-enabling the topology extensions under Xen shouldn't be skipped altogether, perhaps even on Dom0 (as the hypervisor is controlling this MSR, but in any case on DomU - the hypervisor won't allow (read: ignore, not fault on) the write anyway (and will log a message for each (v)CPU that attempts this).
This is probably right. Let me think about this.
Thanks for picking this up.
Regards,
Andre.
Quoted 26 lines
Quoted 23 lines
Signed-off-by Andre Przywara<[email protected]>Cc [email protected] # 3.4+--- arch/x86/kernel/cpu/amd.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)arch/x86/kernel/cpu/amd.c +2
-2@@ -586,9 +586,9 @@ static void __cpuinit init_amd(struct cpuinfo_x86 *c) !cpu_has(c, X86_FEATURE_TOPOEXT)) {u64 val; -if (!rdmsrl_amd_safe (0xc0011005 ,&val)) { +if (!rdmsrl_safe (0xc0011005 ,&val)) { val |=1ULL <<54 ; -wrmsrl_amd_safe (0xc0011005 , val); +checking_wrmsrl (0xc0011005 , val);rdmsrl (0xc0011005 , val);if (val& (1ULL <<54 )) {set_cpu_cap (c,X86_FEATURE_TOPOEXT );
Signature · 2 lines
Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany
Quoted 10 lines
Quoted 1 line
Quoted 1 line · Andre Przywara
Quoted 4 lines · Jan Beulich
Further, I can't see how
checking_wrmsrl()is being paravirtualized any better thanwrmsrl_amd_safe()- both have nothing but an exception handling fixup attached to the wrmsr invocation. Care to point out what actual crash it is that was seen?AFAIK, the difference is between the "l" and the regs version for rd/wrmsr. We have a patch already here to fix this. Will send it out soon. Jacob, can you comment on this?
I see - the Xen code blindly overwrites pv_cpu_ops, despite not having initialized all members. That's an obvious oversight of the patch that introduced the _regs variants.
Plus having secondary instances of things like rdmsrl_amd_safe() in asm/paravirt.h seems pretty strange an approach (which was why initially I didn't spot how a crash could happen there) - only the lowest level functions should get re-implemented here.
Quoted 8 lines
Quoted 6 lines
Finally, I would question whether re-enabling the topology extensions under Xen shouldn't be skipped altogether, perhaps even on Dom0 (as the hypervisor is controlling this MSR, but in any case on DomU - the hypervisor won't allow (read: ignore, not fault on) the write anyway (and will log a message for each (v)CPU that attempts this).
This is probably right. Let me think about this.
I'll submit a respective hypervisor side patch soonish.
Jan
H. Peter Anvin
reviewerQuoted 10 lines · Jan Beulich
I see - the Xen code blindly overwrites
pv_cpu_ops, despite not having initialized all members. That's an obvious oversight of the patch that introduced the_regsvariants.Plus having secondary instances of things like
rdmsrl_amd_safe()inasm/paravirt.hseems pretty strange an approach (which was why initially I didn't spot how a crash could happen there) - only the lowest level functions should get re-implemented here.
This kinds of things are part of why Xen makes me want to cry regularly.
-hpa
Quoted 6 lines · H. Peter Anvin
Quoted 5 lines · Jan Beulich
I see - the Xen code blindly overwrites
pv_cpu_ops, despite not having initialized all members. That's an obvious oversight of the patch that introduced the_regsvariants.
Quoted 7 lines
Quoted 5 lines
Plus having secondary instances of things like
rdmsrl_amd_safe()inasm/paravirt.hseems pretty strange an approach (which was why initially I didn't spot how a crash could happen there) - only the lowest level functions should get re-implemented here.This kinds of things are part of why Xen makes me want to cry regularly.
It looks like an oversight by Borislav (177fed1ee8d727c39601ce9fc2299b4cb25a718e and 132ec92f) and Yinghai (b05f78f5c713eda2c34e495d92495ee4f1c3b5e1) where they added these wrappers way back in 2009!
Borislav Petkov
maintainerQuoted 13 lines · Konrad Rzeszutek Wilk
Quoted 12 lines · H. Peter Anvin
Quoted 10 lines · Jan Beulich
I see - the Xen code blindly overwrites
pv_cpu_ops, despite not having initialized all members. That's an obvious oversight of the patch that introduced the_regsvariants.Plus having secondary instances of things like
rdmsrl_amd_safe()inasm/paravirt.hseems pretty strange an approach (which was why initially I didn't spot how a crash could happen there) - only the lowest level functions should get re-implemented here.
Quoted 5 lines
Quoted 1 line
This kinds of things are part of why Xen makes me want to cry regularly.
It looks like an oversight by Borislav (
177fed1ee8d727c39601ce9fc2299b4cb25a718eand132ec92f) and Yinghai (b05f78f5c713eda2c34e495d92495ee4f1c3b5e1) where they added these wrappers way back in 2009!
This is exactly why xen has nothing to do in arch/x86/. It is not an oversight - I simply didn't test it on xen because I don't care about it.
Remember our last discussion about mcelog?
This current case should be a perfect example for why xen shouldn't be sprinkling code all over the place.
Signature · 2 lines
Regards/Gruss, Boris.
Quoted 3 lines
Quoted 1 line
Quoted 1 line · Borislav Petkov
This current case should be a perfect example for why xen shouldn't be sprinkling code all over the place.
Which means you're denying the benefits of para-virtualization (at the base system level; perhaps it's less of a problem for you when it comes to pv device drivers, which are generally standalone entities) as that's what distinguishes Xen from all other virtualization solutions Linux supports.
Jan
H. Peter Anvin
reviewerQuoted 10 lines · Jan Beulich
Quoted 3 lines
Quoted 1 line
Quoted 1 line · Borislav Petkov
This current case should be a perfect example for why xen shouldn't be sprinkling code all over the place.
Which means you're denying the benefits of para-virtualization (at the base system level; perhaps it's less of a problem for you when it comes to pv device drivers, which are generally standalone entities) as that's what distinguishes Xen from all other virtualization solutions Linux supports.
And it also denies the just atrocious cost of Xen grabbing random kernel internals and turning them into ABIs that we have to work around from that point on. This is particularly obnoxious because the cost is largely not borne by the Xen community but by the general Linux development.
-hpa
Borislav Petkov
maintainerQuoted 5 lines · Jan Beulich
Quoted 3 lines
Quoted 1 line
Quoted 1 line · Borislav Petkov
This current case should be a perfect example for why xen shouldn't be sprinkling code all over the place.
Which means you're denying the benefits of para-virtualization
Does it really mean that?
Quoted 4 lines
(at the base system level; perhaps it's less of a problem for you when it comes to pv device drivers, which are generally standalone entities) as that's what distinguishes Xen from all other virtualization solutions Linux supports.
All I'm saying is, xen should solve the whole deal of what it wants to do (whatever that is) _without_ and _agnostic_ from arch/x86/. Otherwise x86 changes break xen.
I couldn't care less about what distinguishes xen from all other solutions.
Signature · 2 lines
Regards/Gruss, Boris.
Quoted 16 lines · Andre Przywara
Quoted 11 lines · Jan Beulich
Quoted 7 lines
Quoted 1 line
Quoted 1 line · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
This fixes a crash when using the kernel as a Xen Dom0 on affected Trinity systems. Thewrmsrl_amd_safeis not properly paravirtualized yet (this will be fixed in another patch).I'm not following: If the AMD variants (putting a special value into %edi) can be freely replaced by the non-AMD variants, why did the AMD special ones get used in the first place?
Older CPUs (K8) needed the AMD variants, starting with family 10h we can use the normal versions.
Quoted 8 lines
Quoted 4 lines
Further, I can't see how
checking_wrmsrl()is being paravirtualized any better thanwrmsrl_amd_safe()- both have nothing but an exception handling fixup attached to the wrmsr invocation. Care to point out what actual crash it is that was seen?AFAIK, the difference is between the "l" and the regs version for rd/wrmsr. We have a patch already here to fix this. Will send it out soon. Jacob, can you comment on this?
Right, the checking_wrmsrl turns into wrmsr_safe which is paravirtualized but the rdmsrl_amd_safe which turns into rdmsr_regs is not paravirtualized by enlighten.
Quoted 111 lines
The crash dump:
[ 1.601021] ------------[ cut here ]------------ [ 1.601025] kernel BUG at /buildrepos/linux-2.6-stable/arch/x86/include/asm/paravirt.h:133! [ 1.601031] invalid opcode: 0000 [#1] SMP [ 1.601038] CPU 0 [ 1.601041] Modules linked in: [ 1.601047] [ 1.601050] Pid: 0, comm: swapper/0 Not tainted 3.4.0 #1 AMD Virgo Platform/Annapurna [ 1.601061] RIP: e030:[<ffffffff8169b4fe>] [<ffffffff8169b4fe>] init_amd+0x21d/0x603 [ 1.601072] RSP: e02b:ffffffff81c01df8 EFLAGS: 00010246 [ 1.601076] RAX: 0000000000000000 RBX: ffffffff81ca7500 RCX: 0000000000000000 [ 1.601081] RDX: 0000000000000020 RSI: 000000000000c000 RDI: ffffffff81c01e78 [ 1.601086] RBP: ffffffff81c01ea8 R08: 0000000000004003 R09: 00000000ffffffff [ 1.601090] R10: 0000000000000000 R11: 00000000ffffffff R12: ffffffff81ca7574 [ 1.601095] R13: ffffffff81ca7514 R14: ffffffff81ca754c R15: ffffffff81ca756c [ 1.601103] FS: 0000000000000000(0000) GS:ffff8801ce600000(0000) knlGS:0000000000000000 [ 1.601108] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1.601112] CR2: 0000000000000000 CR3: 0000000001c0c000 CR4: 0000000000040660 [ 1.601117] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1.601122] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1.601127] Process swapper/0 (pid: 0, threadinfo ffffffff81c00000, task ffffffff81c14020) [ 1.601131] Stack: [ 1.601134] ffffffff81c01ea8 ffffffff8169a7d8 0000001600000017 0000000000000000 [ 1.601146] ffff8801c1c3ac00 ffff8801c1c002d0 ffffffff81c01e48 ffffffff81140118 [ 1.601157] ffffffff811415cc ffff8801c1c04220 ffff8801c1c02480 00000000000000d0 [ 1.601169] Call Trace: [ 1.601175] [<ffffffff8169a7d8>] ? get_cpu_cap+0x1f2/0x201 [ 1.601183] [<ffffffff81140118>] ? init_kmem_cache_cpus+0x3e/0x4d [ 1.601189] [<ffffffff811415cc>] ? sysfs_slab_alias+0x60/0x8d [ 1.601195] [<ffffffff8169aa3d>] identify_cpu+0x256/0x34d [ 1.601201] [<ffffffff81141649>] ? kmem_cache_alloc+0x50/0xe4 [ 1.601209] [<ffffffff81cd1c51>] identify_boot_cpu+0x10/0x3c [ 1.601216] [<ffffffff81cd2052>] check_bugs+0x9/0x2d [ 1.601222] [<ffffffff81cc7e08>] start_kernel+0x445/0x461 [ 1.601227] [<ffffffff81cc77d6>] ? kernel_init+0x1d8/0x1d8 [ 1.601233] [<ffffffff81cc72cf>] x86_64_start_reservations+0xba/0xc1 [ 1.601240] [<ffffffff81041797>] ? xen_setup_runstate_info+0x2c/0x36 [ 1.601247] [<ffffffff81ccbdc4>] xen_start_kernel+0x58b/0x592 [ 1.601251] Code: 0f 85 d3 00 00 00 31 c0 48 83 3d cd 5c 58 00 0048 8d 7d b0 b9 08 00 00 00 f3
ab c7 45 b4 05 10 01 c0 c7 45 cc 3a 20 5a 9c 75 04 <0f> 0b eb fe 4c 8d 75 b0 4c 89 f7 ff 14 25 b0 11
c2 81 85 c0 8b[ 1.601374] RIP [<ffffffff8169b4fe>] init_amd+0x21d/0x603 [ 1.601381] RSP <ffffffff81c01df8> [ 1.601391] ---[ end trace a7919e7f17c0a725 ]--- [ 1.601397] Kernel panic - not syncing: Attempted to kill the idle task!Quoted 6 lines
Finally, I would question whether re-enabling the topology extensions under Xen shouldn't be skipped altogether, perhaps even on Dom0 (as the hypervisor is controlling this MSR, but in any case on DomU - the hypervisor won't allow (read: ignore, not fault on) the write anyway (and will log a message for each (v)CPU that attempts this).
This is probably right. Let me think about this.
Thanks for picking this up.
Regards,
Andre.Quoted 26 lines
Quoted 23 lines
Signed-off-by Andre Przywara<[email protected]>Cc [email protected] # 3.4+--- arch/x86/kernel/cpu/amd.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)arch/x86/kernel/cpu/amd.c +2
-2@@ -586,9 +586,9 @@ static void __cpuinit init_amd(struct cpuinfo_x86 *c) !cpu_has(c, X86_FEATURE_TOPOEXT)) {u64 val; -if (!rdmsrl_amd_safe (0xc0011005 ,&val)) { +if (!rdmsrl_safe (0xc0011005 ,&val)) { val |=1ULL <<54 ; -wrmsrl_amd_safe (0xc0011005 , val); +checking_wrmsrl (0xc0011005 , val);rdmsrl (0xc0011005 , val);if (val& (1ULL <<54 )) {set_cpu_cap (c,X86_FEATURE_TOPOEXT );Signature · 2 lines
Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany
Quoted 22 lines · Jacob Shin
Quoted 21 lines · Andre Przywara
Quoted 11 lines · Jan Beulich
Quoted 7 lines
Quoted 1 line
Quoted 1 line · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
This fixes a crash when using the kernel as a Xen Dom0 on affected Trinity systems. Thewrmsrl_amd_safeis not properly paravirtualized yet (this will be fixed in another patch).I'm not following: If the AMD variants (putting a special value into %edi) can be freely replaced by the non-AMD variants, why did the AMD special ones get used in the first place?
Older CPUs (K8) needed the AMD variants, starting with family 10h we can use the normal versions.
Quoted 4 lines
Further, I can't see how
checking_wrmsrl()is being paravirtualized any better thanwrmsrl_amd_safe()- both have nothing but an exception handling fixup attached to the wrmsr invocation. Care to point out what actual crash it is that was seen?
Quoted 7 lines
Quoted 3 lines
AFAIK, the difference is between the "l" and the regs version for rd/wrmsr. We have a patch already here to fix this. Will send it out soon. Jacob, can you comment on this?
Right, the
checking_wrmsrlturns intowrmsr_safewhich is paravirtualized but therdmsrl_amd_safewhich turns intordmsr_regsis not paravirtualized by enlighten.
So would a patch to implements the rdmsr_regs fix this crash?
Quoted 27 lines · Konrad Rzeszutek Wilk
Quoted 26 lines · Jacob Shin
Quoted 24 lines · Andre Przywara
Quoted 11 lines · Jan Beulich
Quoted 7 lines
Quoted 1 line
Quoted 1 line · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
This fixes a crash when using the kernel as a Xen Dom0 on affected Trinity systems. Thewrmsrl_amd_safeis not properly paravirtualized yet (this will be fixed in another patch).I'm not following: If the AMD variants (putting a special value into %edi) can be freely replaced by the non-AMD variants, why did the AMD special ones get used in the first place?
Older CPUs (K8) needed the AMD variants, starting with family 10h we can use the normal versions.
Quoted 4 lines
Further, I can't see how
checking_wrmsrl()is being paravirtualized any better thanwrmsrl_amd_safe()- both have nothing but an exception handling fixup attached to the wrmsr invocation. Care to point out what actual crash it is that was seen?AFAIK, the difference is between the "l" and the regs version for rd/wrmsr. We have a patch already here to fix this. Will send it out soon. Jacob, can you comment on this?
Quoted 5 lines
Quoted 3 lines
Right, the
checking_wrmsrlturns intowrmsr_safewhich is paravirtualized but therdmsrl_amd_safewhich turns intordmsr_regsis not paravirtualized by enlighten.So would a patch to implements the
rdmsr_regsfix this crash?
Yes, the following patch also fixed the crash for us:
Implement rdmsr_regs and wrmsr_regs for Xen pvops.
arch/x86/xen/enlighten.c +18 -2
@@ -945,9 +945,12 @@ static void xen_write_cr4(unsigned long cr4)native_write_cr4 (cr4); } -static int xen_write_msr_safe (unsigned int msr,unsigned low,unsigned high) +static int xen_wrmsr_safe_regs (u32 regs[8 ]) {int ret; +unsigned int msr = regs[1 ]; +unsigned low = regs[0 ]; +unsigned high = regs[2 ]; ret =0 ;
@@ -985,12 +988,23 @@ static int xen_write_msr_safe(unsigned int msr, unsigned low, unsigned high)break ;default : - ret =native_write_msr_safe (msr, low, high); + ret =native_wrmsr_safe_regs (regs); }return ret; } +static int xen_write_msr_safe (unsigned int msr,unsigned low,unsigned high) +{ +u32 regs[8 ] = {0 }; + + regs[0 ] = low; + regs[1 ] = msr; + regs[2 ] = high; + +return xen_wrmsr_safe_regs (regs); +} +void xen_setup_shared_info (void ) {if (!xen_feature (XENFEAT_auto_translated_physmap)) {
@@ -1116,7 +1130,9 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = { .wbinvd = native_wbinvd, .read_msr = native_read_msr_safe, + .rdmsr_regs = native_rdmsr_safe_regs, .write_msr = xen_write_msr_safe, + .wrmsr_regs = xen_wrmsr_safe_regs, .read_tsc = native_read_tsc, .read_pmc = native_read_pmc,
Quoted 3 lines
Yes, the following patch also fixed the crash for us:
Implement
rdmsr_regsandwrmsr_regsfor Xen pvops.
That needs more data. Such as the reason for it, the crash tombstone, and an analysis of the bug.
But at this point I am not sure what we are going to do.
I think Peter leans towards ripping the .rdmsr_regs/wdmsr_regs function out altogether (so altering the amd_rdmsr... to use the .rdmsr/.wrdmsr). At which point I think this would all work just fine?
I am tempted to write a patch that checks all the pv-cpu-ops to see if there are any that are NULL and throw a warning so that this does not hit us in the future - to be at least more proactive about this sort of thing.
Quoted 64 lines
Signed-off-by Jacob Shin <[email protected]>Cc [email protected] # 3.4+arch/x86/xen/enlighten.c +18
-2@@ -945,9 +945,12 @@ static void xen_write_cr4(unsigned long cr4)native_write_cr4 (cr4); } -static int xen_write_msr_safe (unsigned int msr,unsigned low,unsigned high) +static int xen_wrmsr_safe_regs (u32 regs[8 ]) {int ret; +unsigned int msr = regs[1 ]; +unsigned low = regs[0 ]; +unsigned high = regs[2 ]; ret =0 ;@@ -985,12 +988,23 @@ static int xen_write_msr_safe(unsigned int msr, unsigned low, unsigned high)break ;default : - ret =native_write_msr_safe (msr, low, high); + ret =native_wrmsr_safe_regs (regs); }return ret; } +static int xen_write_msr_safe (unsigned int msr,unsigned low,unsigned high) +{ +u32 regs[8 ] = {0 }; + + regs[0 ] = low; + regs[1 ] = msr; + regs[2 ] = high; + +return xen_wrmsr_safe_regs (regs); +} +void xen_setup_shared_info (void ) {if (!xen_feature (XENFEAT_auto_translated_physmap)) {@@ -1116,7 +1130,9 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = { .wbinvd = native_wbinvd, .read_msr = native_read_msr_safe, + .rdmsr_regs = native_rdmsr_safe_regs, .write_msr = xen_write_msr_safe, + .wrmsr_regs = xen_wrmsr_safe_regs, .read_tsc = native_read_tsc, .read_pmc = native_read_pmc,_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel
H. Peter Anvin
reviewerQuoted 7 lines · Konrad Rzeszutek Wilk
Quoted 3 lines
Yes, the following patch also fixed the crash for us:
Implement
rdmsr_regsandwrmsr_regsfor Xen pvops.That needs more data. Such as the reason for it, the crash tombstone, and an analysis of the bug.
Quoted 7 lines
But at this point I am not sure what we are going to do.
I think Peter leans towards ripping the .rdmsr_regs/wdmsr_regs function out altogether (so altering the
amd_rdmsr... to use the .rdmsr/.wrdmsr). At which point I think this would all work just fine?
No, you can't just do that. Rip them out as in trap and emulate.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 10 lines · H. Peter Anvin
Quoted 9 lines · Konrad Rzeszutek Wilk
Quoted 3 lines
Yes, the following patch also fixed the crash for us:
Implement
rdmsr_regsandwrmsr_regsfor Xen pvops.That needs more data. Such as the reason for it, the crash tombstone, and an analysis of the bug.
But at this point I am not sure what we are going to do.
Quoted 7 lines
Quoted 5 lines
I think Peter leans towards ripping the .rdmsr_regs/wdmsr_regs function out altogether (so altering the
amd_rdmsr... to use the .rdmsr/.wrdmsr). At which point I think this would all work just fine?No, you can't just do that. Rip them out as in trap and emulate.
Then this should do it:
arch/x86/xen/enlighten.c +3 -0
@@ -1116,7 +1116,10 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = { .wbinvd = native_wbinvd, .read_msr = native_read_msr_safe, + .rdmsr_regs = native_rdmsr_safe_regs, .write_msr = xen_write_msr_safe, + .wrmsr_regs = native_wrmsr_safe_regs, + .read_tsc = native_read_tsc, .read_pmc = native_read_pmc,
[tip:x86/urgent] x86, amd, xen: Avoid NULL pointer paravirt references
Commit-ID: 1ab46fd319bcf1fcd9fb6311727d532b580e4eba Gitweb: http://git.kernel.org/tip/1ab46fd319bcf1fcd9fb6311727d532b580e4eba Author: Konrad Rzeszutek Wilk <[email protected]> AuthorDate: Wed, 30 May 2012 18:23:56 -0400 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 30 May 2012 16:15:02 -0700
x86, amd, xen: Avoid NULL pointer paravirt references
Stub out MSR methods that aren't actually needed. This fixes a crash as Xen Dom0 on AMD Trinity systems. A bigger patch should be added to remove the paravirt machinery completely for the methods which apparently have no users!
--- arch/x86/xen/enlighten.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
arch/x86/xen/enlighten.c +3 -0
@@ -1116,7 +1116,10 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = { .wbinvd = native_wbinvd, .read_msr = native_read_msr_safe, + .rdmsr_regs = native_rdmsr_safe_regs, .write_msr = xen_write_msr_safe, + .wrmsr_regs = native_wrmsr_safe_regs, + .read_tsc = native_read_tsc, .read_pmc = native_read_pmc,
Borislav Petkov
maintainerQuoted 7 lines · Konrad Rzeszutek Wilk
Quoted 3 lines
Yes, the following patch also fixed the crash for us:
Implement
rdmsr_regsandwrmsr_regsfor Xen pvops.That needs more data. Such as the reason for it, the crash tombstone, and an analysis of the bug.
Quoted 6 lines
But at this point I am not sure what we are going to do.
I think Peter leans towards ripping the .rdmsr_regs/wdmsr_regs function out altogether (so altering the
amd_rdmsr... to use the .rdmsr/.wrdmsr). At which point I think this would all work just fine?
I wouldn't do that. Andre's patch switches to the rdmsrl_safe/checking_wrmsrl functions so you don't need to implement the _regs functions for pvops.
I'll send it now with corrected commit message.
Quoted 4 lines
I am tempted to write a patch that checks all the pv-cpu-ops to see if there are any that are NULL and throw a warning so that this does not hit us in the future - to be at least more proactive about this sort of thing.
This could be a prudent thing to do.
Signature · 2 lines
Regards/Gruss, Boris.
PATCH 1/1 x86, AMD: Fix crash as Xen Dom0 on AMD Trinity systems
From: Andre Przywara <[email protected]>
Switch to the standard {rd,wr}msr*_safe* variants which should've been used in the first place anyway and avoided unneeded excitation with xen.
--- arch/x86/kernel/cpu/amd.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/amd.c +2 -2
@@ -586,9 +586,9 @@ static void __cpuinit init_amd(struct cpuinfo_x86 *c) !cpu_has(c, X86_FEATURE_TOPOEXT)) {u64 val; -if (!rdmsrl_amd_safe (0xc0011005 , &val)) { +if (!rdmsrl_safe (0xc0011005 , &val)) { val |=1ULL <<54 ; -wrmsrl_amd_safe (0xc0011005 , val); +checking_wrmsrl (0xc0011005 , val);rdmsrl (0xc0011005 , val);if (val & (1ULL <<54 )) {set_cpu_cap (c,X86_FEATURE_TOPOEXT );
Signature · 1 line
1.7.9.3.362.g71319
H. Peter Anvin
reviewerQuoted 5 lines · Borislav Petkov
I wouldn't do that. Andre's patch switches to the rdmsrl_safe/checking_wrmsrl functions so you don't need to implement the
_regsfunctions for pvops.
But we just stated - it is wrong for K8?
-hpa
Borislav Petkov
maintainerQuoted 5 lines · H. Peter Anvin
Quoted 4 lines
I wouldn't do that. Andre's patch switches to the rdmsrl_safe/checking_wrmsrl functions so you don't need to implement the
_regsfunctions for pvops.But we just stated - it is wrong for K8?
Not executed on K8 - only on F15h, models 0x10 - 0x1f.
Signature · 2 lines
Regards/Gruss, Boris.
H. Peter Anvin
reviewerQuoted 9 lines · Borislav Petkov
Quoted 5 lines · H. Peter Anvin
Quoted 4 lines
I wouldn't do that. Andre's patch switches to the rdmsrl_safe/checking_wrmsrl functions so you don't need to implement the
_regsfunctions for pvops.But we just stated - it is wrong for K8?
Not executed on K8 - only on F15h, models
0x10-0x1f.
OK. But there is still the general problem, no?
-hpa
Borislav Petkov
maintainerQuoted 1 line · H. Peter Anvin
OK. But there is still the general problem, no?
With this patch xen crashes go away because they paravirt native_{read,write}_msr_safe.
The other place where we use the amd_safe variants is an obscure K8, revC and earlier fix for _some_ BIOSen and this hasn't bitten us yet so I'm assuming people haven't run xen on such boxes yet. Does it need fixing? Probably, if we really really have to.
Now, someone probably needs to paravirt the *safe_regs variants in case something else decides to use them. I don't know what to do here, do I want more paravirt code in there? No. I guess if this is done carefully and cleanly, then it should be ok but it can't be done like that - it needs to adhere to the current pv_cpu_ops thing which is already there.
Hmm...
Signature · 2 lines
Regards/Gruss, Boris.
Borislav Petkov
maintainerQuoted 4 lines · Borislav Petkov
The other place where we use the
amd_safevariants is an obscure K8, revC and earlier fix for _some_ BIOSen and this hasn't bitten us yet so I'm assuming people haven't run xen on such boxes yet. Does it need fixing? Probably, if we really really have to.
I'm being told we're safe here. Those boxes didn't have virtualization support yet (SVM) so this one doesn't need fixing.
Signature · 2 lines
Regards/Gruss, Boris.
H. Peter Anvin
reviewerQuoted 9 lines · Borislav Petkov
Quoted 4 lines · Borislav Petkov
The other place where we use the
amd_safevariants is an obscure K8, revC and earlier fix for _some_ BIOSen and this hasn't bitten us yet so I'm assuming people haven't run xen on such boxes yet. Does it need fixing? Probably, if we really really have to.I'm being told we're safe here. Those boxes didn't have virtualization support yet (SVM) so this one doesn't need fixing.
This is for PV, not for HVM (VT-x/SVM). HVM doesn't have this kind of drain bamage.
-hpa
Borislav Petkov
maintainerQuoted 2 lines · H. Peter Anvin
This is for PV, not for HVM (VT-x/SVM). HVM doesn't have this kind of drain bamage.
Frankly, I wouldn't want to fix this for K8 only - they're going away anyway. And besides, this is a BIOS fix for a very small subset of very early K8s. It probably shouldnt've been there in the first place but at the time I did it I wanted to fix the whole world (I'm much more reasonable now :-)).
IOW, I'd like to leave sleeping dogs lie if you don't mind.
Signature · 2 lines
Regards/Gruss, Boris.
H. Peter Anvin
reviewerQuoted 11 lines · Borislav Petkov
Quoted 1 line · H. Peter Anvin
OK. But there is still the general problem, no?
With this patch xen crashes go away because they paravirt native_{read,write}_msr_safe.
The other place where we use the
amd_safevariants is an obscure K8, revC and earlier fix for _some_ BIOSen and this hasn't bitten us yet so I'm assuming people haven't run xen on such boxes yet. Does it need fixing? Probably, if we really really have to.
Quoted 6 lines
Now, someone probably needs to paravirt the
*safe_regsvariants in case something else decides to use them. I don't know what to do here, do I want more paravirt code in there? No. I guess if this is done carefully and cleanly, then it should be ok but it can't be done like that - it needs to adhere to the currentpv_cpu_opsthing which is already there.
I thought I was being told that Xen would trap and emulate the rdmsr/wrmsr instructions. I guess they don't want to do that for the handful of performance-sensitive MSRs there are, but those don't use the *_regs variants.
-hpa
Quoted 5 lines
Quoted 5 lines
Now, someone probably needs to paravirt the
*safe_regsvariants in case something else decides to use them. I don't know what to do here, do I want more paravirt code in there? No. I guess if this is done carefully and cleanly, then it should be ok but it can't be done like that - it needs to adhere to the currentpv_cpu_opsthing which is already there.
Using the native variant seems the right thing to do.
Quoted 4 lines
Quoted 1 line
I thought I was being told that Xen would trap and emulate the rdmsr/wrmsr instructions. I guess they don't want to do that for the
It does.
Quoted 2 lines
handful of performance-sensitive MSRs there are, but those don't use the
*_regsvariants.
The underlaying issue (as I understand) was that .rdmsr_regs (and the corresponding write) was NULL and that caused the crash. This tiny patch should do it:
arch/x86/xen/enlighten.c +3 -0
@@ -1116,7 +1116,10 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = { .wbinvd = native_wbinvd, .read_msr = native_read_msr_safe, + .rdmsr_regs = native_rdmsr_safe_regs, .write_msr = xen_write_msr_safe, + .wrmsr_regs = native_wrmsr_safe_regs, + .read_tsc = native_read_tsc, .read_pmc = native_read_pmc,
H. Peter Anvin
reviewerQuoted 5 lines · Konrad Rzeszutek Wilk
arch/x86/xen/enlighten.c +0
-0
arch/x86/xen/enlighten.c static const struct pv_cpu_ops xen_cpu_ops __initconst = { · line 1125
Quoted 10 lines
arch/x86/xen/enlighten.c +3
-0@@ -1116,7 +1116,10 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = { .wbinvd = native_wbinvd, .read_msr = native_read_msr_safe, + .rdmsr_regs = native_rdmsr_safe_regs, .write_msr = xen_write_msr_safe, + .wrmsr_regs = native_wrmsr_safe_regs, + .read_tsc = native_read_tsc, .read_pmc = native_read_pmc,
Okay, tell me again:
why do we have these hooks again? Can we please weed out paravirt methods that have no users?
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 19 lines · H. Peter Anvin
Quoted 5 lines · Konrad Rzeszutek Wilk
arch/x86/xen/enlighten.c +0
-0arch/x86/xen/enlighten.c static const struct pv_cpu_ops xen_cpu_ops __initconst = { · line 1125
Quoted 10 lines
arch/x86/xen/enlighten.c +3
-0@@ -1116,7 +1116,10 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = { .wbinvd = native_wbinvd, .read_msr = native_read_msr_safe, + .rdmsr_regs = native_rdmsr_safe_regs, .write_msr = xen_write_msr_safe, + .wrmsr_regs = native_wrmsr_safe_regs, + .read_tsc = native_read_tsc, .read_pmc = native_read_pmc,Okay, tell me again:
Quoted 2 lines
why do we have these hooks again? Can we please weed out paravirt methods that have no users?
ping?
Ingo
Borislav Petkov
maintainerQuoted 22 lines · Ingo Molnar
Quoted 19 lines · H. Peter Anvin
Quoted 5 lines · Konrad Rzeszutek Wilk
arch/x86/xen/enlighten.c +0
-0arch/x86/xen/enlighten.c static const struct pv_cpu_ops xen_cpu_ops __initconst = { · line 1125
Quoted 10 lines
arch/x86/xen/enlighten.c +3
-0@@ -1116,7 +1116,10 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = { .wbinvd = native_wbinvd, .read_msr = native_read_msr_safe, + .rdmsr_regs = native_rdmsr_safe_regs, .write_msr = xen_write_msr_safe, + .wrmsr_regs = native_wrmsr_safe_regs, + .read_tsc = native_read_tsc, .read_pmc = native_read_pmc,Okay, tell me again:
Quoted 4 lines
Quoted 2 lines
why do we have these hooks again? Can we please weed out paravirt methods that have no users?
ping?
I think we solved all that and hpa wanted to queue them up shortly:
http://marc.info/?l=linux-kernel&m=133856342618027&w=2
If it's easier, I can send you a pull request later.
Thanks.
Signature · 2 lines
Regards/Gruss,
Boris.Quoted 23 lines · Borislav Petkov
Quoted 22 lines · Ingo Molnar
Quoted 19 lines · H. Peter Anvin
Quoted 5 lines · Konrad Rzeszutek Wilk
arch/x86/xen/enlighten.c +0
-0arch/x86/xen/enlighten.c static const struct pv_cpu_ops xen_cpu_ops __initconst = { · line 1125
Quoted 10 lines
arch/x86/xen/enlighten.c +3
-0@@ -1116,7 +1116,10 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = { .wbinvd = native_wbinvd, .read_msr = native_read_msr_safe, + .rdmsr_regs = native_rdmsr_safe_regs, .write_msr = xen_write_msr_safe, + .wrmsr_regs = native_wrmsr_safe_regs, + .read_tsc = native_read_tsc, .read_pmc = native_read_pmc,Okay, tell me again:
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
why do we have these hooks again? Can we please weed out paravirt methods that have no users?
ping?
I think we solved all that and hpa wanted to queue them up shortly:
Great, will wait for hpa then.
Thanks,
Ingo
Andre Przywara
Acked-byQuoted 26 lines · Konrad Rzeszutek Wilk
Quoted 5 lines
Quoted 5 lines
Now, someone probably needs to paravirt the
*safe_regsvariants in case something else decides to use them. I don't know what to do here, do I want more paravirt code in there? No. I guess if this is done carefully and cleanly, then it should be ok but it can't be done like that - it needs to adhere to the currentpv_cpu_opsthing which is already there.Using the native variant seems the right thing to do.
Quoted 4 lines
Quoted 1 line
I thought I was being told that Xen would trap and emulate the rdmsr/wrmsr instructions. I guess they don't want to do that for the
It does.
Quoted 2 lines
handful of performance-sensitive MSRs there are, but those don't use the
*_regsvariants.The underlaying issue (as I understand) was that .rdmsr_regs (and the corresponding write) was NULL and that caused the crash. This tiny patch should do it:
arch/x86/xen/enlighten.c +0
-0
arch/x86/xen/enlighten.c static const struct pv_cpu_ops xen_cpu_ops __initconst = { · line 1125
Quoted 10 lines
arch/x86/xen/enlighten.c +3
-0@@ -1116,7 +1116,10 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = { .wbinvd = native_wbinvd, .read_msr = native_read_msr_safe, + .rdmsr_regs = native_rdmsr_safe_regs, .write_msr = xen_write_msr_safe, + .wrmsr_regs = native_wrmsr_safe_regs, + .read_tsc = native_read_tsc, .read_pmc = native_read_pmc,
This works on the test machine.
Though I'd still like to have my original patch applied, because it makes the thing a bit cleaner.
And I made a patch to remove the {rd,wr}msr_regs hooks from paravirt_ops completely. Shall I send it out or do you want to make this part of larger patch series to clean up pvops?
Regards,
Andre.
Signature · 2 lines
Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany
Quoted 19 lines
Quoted 6 lines
arch/x86/xen/enlighten.c +0
-0arch/x86/xen/enlighten.c static const struct pv_cpu_ops xen_cpu_ops __initconst = { · line 1125
Quoted 10 lines
arch/x86/xen/enlighten.c +3
-0@@ -1116,7 +1116,10 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = { .wbinvd = native_wbinvd, .read_msr = native_read_msr_safe, + .rdmsr_regs = native_rdmsr_safe_regs, .write_msr = xen_write_msr_safe, + .wrmsr_regs = native_wrmsr_safe_regs, + .read_tsc = native_read_tsc, .read_pmc = native_read_pmc,Acked-by Andre Przywara <[email protected]>
Quoted 1 line
This works on the test machine.
Great! Thanks for doing a quick test for this.
Quoted 3 lines
Though I'd still like to have my original patch applied, because it makes the thing a bit cleaner.
OK. Please re-send with an up-to-date git commit as suggested by Peter.
Quoted 4 lines
And I made a patch to remove the {rd,wr}msr_regs hooks from
paravirt_opscompletely. Shall I send it out or do you want to make this part of larger patch series to clean up pvops?
Please do send it out. Thanks again!
Quoted 6 lines
Quoted 1 line
Quoted 1 line · Borislav Petkov
On Wed, May 30, 2012 at 11:00:23AM -0700, H. Peter Anvin wrote:
The other place where we use theamd_safevariants is an obscure K8, revC and earlier fix for _some_ BIOSen and this hasn't bitten us yet so I'm assuming people haven't run xen on such boxes yet. Does it need fixing? Probably, if we really really have to.
This again is something that shouldn't even be attempted under Xen. The hypervisor, unless really old, does this (and wouldn't allow the write by any domain - privileged or not - anyway).
There's one more user though - the code triggered by the "show_msr=" command line option. This one indeed requires rdmsr_safe_regs to be functional (albeit under Xen, once again, this won't work currently anyway for those MRS on old CPUs where the special key in %edi is required, which the emulation code in Xen doesn't support).
Jan
Borislav Petkov
maintainerQuoted 11 lines · Jan Beulich
Quoted 6 lines
Quoted 1 line
Quoted 1 line · Borislav Petkov
On Wed, May 30, 2012 at 11:00:23AM -0700, H. Peter Anvin wrote:
The other place where we use theamd_safevariants is an obscure K8, revC and earlier fix for _some_ BIOSen and this hasn't bitten us yet so I'm assuming people haven't run xen on such boxes yet. Does it need fixing? Probably, if we really really have to.This again is something that shouldn't even be attempted under Xen. The hypervisor, unless really old, does this (and wouldn't allow the write by any domain - privileged or not - anyway).
Quoted 6 lines
There's one more user though - the code triggered by the "show_msr=" command line option. This one indeed requires
rdmsr_safe_regsto be functional (albeit under Xen, once again, this won't work currently anyway for those MRS on old CPUs where the special key in %edi is required, which the emulation code in Xen doesn't support).
This doesn't look right. Yinghai, why does generic x86 code use rdmsrl_amd_safe - it should simply use rdmsrl_safe.
Signature · 2 lines
Regards/Gruss, Boris.
Quoted 5 lines
Quoted 1 line
Quoted 1 line · Konrad Rzeszutek Wilk
I am tempted to write a patch that checks all the pv-cpu-ops to see if there are any that are NULL and throw a warning so that this does not hit us in the future - to be at least more proactive about this sort of thing.
Perhaps rather than using C99 initializers, using old-style ones would be an alternative (assuming that the signatures of the respective entries [or at least immediately neighboring ones] are different), with a sentinel that is required to remain last (i.e. adding at the very end would be prohibited)?
Or rather than doing a full structure assignment, assign individual members directly to pv_cpu_ops (thus leaving everything that's not explicitly overridden at its "native" default)? After all, this is being done on __init code, so the few extra code bytes shouldn't matter much? (All this of course in the context of hpa's valid request that there be no unused paravirt hooks in the first place.)
Jan
H. Peter Anvin
reviewerQuoted 12 lines · Jan Beulich
Quoted 5 lines
Quoted 1 line
Quoted 1 line · Konrad Rzeszutek Wilk
I am tempted to write a patch that checks all the pv-cpu-ops to see if there are any that are NULL and throw a warning so that this does not hit us in the future - to be at least more proactive about this sort of thing.
Perhaps rather than using C99 initializers, using old-style ones would be an alternative (assuming that the signatures of the respective entries [or at least immediately neighboring ones] are different), with a sentinel that is required to remain last (i.e. adding at the very end would be prohibited)?
Quoted 8 lines
Or rather than doing a full structure assignment, assign individual members directly to
pv_cpu_ops(thus leaving everything that's not explicitly overridden at its "native" default)? After all, this is being done on__initcode, so the few extra code bytes shouldn't matter much? (All this of course in the context of hpa's valid request that there be no unused paravirt hooks in the first place.)
Actually there is a really easy way to do this with C99 initializers:
create a macro with all the default assignments, and put that one first.
This is because it is legal to have more than one C99 initializer for
the same member, the last one is the one that takes effect.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 6 lines · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
This fixes a crash when using the kernel as a Xen Dom0 on affected Trinity systems. Thewrmsrl_amd_safeis not properly paravirtualized yet (this will be fixed in another patch).
So with a rdmsrl_amd_safe and wrmsrl_amd_safe being implemented in the pv_cpu_ops - would this patch even be neccessary?
Quoted 32 lines
Signed-off-by Andre Przywara <[email protected]>Cc [email protected] # 3.4+--- arch/x86/kernel/cpu/amd.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)arch/x86/kernel/cpu/amd.c +2
-2@@ -586,9 +586,9 @@ static void __cpuinit init_amd(struct cpuinfo_x86 *c) !cpu_has(c, X86_FEATURE_TOPOEXT)) {u64 val; -if (!rdmsrl_amd_safe (0xc0011005 , &val)) { +if (!rdmsrl_safe (0xc0011005 , &val)) { val |=1ULL <<54 ; -wrmsrl_amd_safe (0xc0011005 , val); +checking_wrmsrl (0xc0011005 , val);rdmsrl (0xc0011005 , val);if (val & (1ULL <<54 )) {set_cpu_cap (c,X86_FEATURE_TOPOEXT );Signature · 8 lines
1.7.4.4 _______________________________________________ Xen-devel mailing list [email protected] http://lists.xen.org/xen-devel
H. Peter Anvin
reviewerQuoted 11 lines · Konrad Rzeszutek Wilk
Quoted 6 lines · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
This fixes a crash when using the kernel as a Xen Dom0 on affected Trinity systems. Thewrmsrl_amd_safeis not properly paravirtualized yet (this will be fixed in another patch).So with a
rdmsrl_amd_safeandwrmsrl_amd_safebeing implemented in thepv_cpu_ops- would this patch even be neccessary?
That is still bogus; a better thing would be to implement the _regs interface. Even better would be to trap and emulate rdmsr/wrmsr!
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 9 lines · H. Peter Anvin
Quoted 8 lines · Konrad Rzeszutek Wilk
Quoted 6 lines · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
This fixes a crash when using the kernel as a Xen Dom0 on affected Trinity systems. Thewrmsrl_amd_safeis not properly paravirtualized yet (this will be fixed in another patch).
Quoted 6 lines
Quoted 3 lines
So with a
rdmsrl_amd_safeandwrmsrl_amd_safebeing implemented in thepv_cpu_ops- would this patch even be neccessary?That is still bogus; a better thing would be to implement the
_regsinterface. Even better would be to trap and emulate rdmsr/wrmsr!
That is what I meant - implement these two:
rdmsr_regs = native_rdmsr_safe_regs,
.wrmsr_regs = native_wrmsr_safe_regs,
Xen already traps the rdmsr/wrms - I believe it just didn't do anything for this specific MSR.
Quoted 10 lines
Quoted 1 line
Quoted 1 line · H. Peter Anvin
Quoted 8 lines · Konrad Rzeszutek Wilk
Quoted 6 lines · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
This fixes a crash when using the kernel as a Xen Dom0 on affected Trinity systems. Thewrmsrl_amd_safeis not properly paravirtualized yet (this will be fixed in another patch).
Quoted 6 lines
Quoted 3 lines
So with a
rdmsrl_amd_safeandwrmsrl_amd_safebeing implemented in thepv_cpu_ops- would this patch even be neccessary?That is still bogus; a better thing would be to implement the
_regsinterface. Even better would be to trap and emulate rdmsr/wrmsr!
The crash is not on the wrmsr instruction, but on the paravirt layer finding a NULL pointer in one of the methods. Xen does trap and emulate (possibly just ignore) both instructions.
Jan
H. Peter Anvin
reviewerQuoted 3 lines · Jan Beulich
Xen does trap and emulate (possibly just ignore) both instructions.
Then why the fsck is there paravirt_crap on this?
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 7 lines
Quoted 1 line
Quoted 1 line · H. Peter Anvin
Quoted 3 lines · Jan Beulich
Xen does trap and emulate (possibly just ignore) both instructions.
Then why the fsck is there
paravirt_crapon this?
I have no clue. Jeremy, Konrad?
Jan
Quoted 9 lines · Jan Beulich
Quoted 7 lines
Quoted 1 line
Quoted 1 line · H. Peter Anvin
Quoted 3 lines · Jan Beulich
Xen does trap and emulate (possibly just ignore) both instructions.
Then why the fsck is there
paravirt_crapon this?I have no clue. Jeremy, Konrad?
No idea. The git 132ec92f says:
Borislav Petkov <[email protected]>
Date: Mon Aug 31 09:50:09 2009 +0200
x86, msr: Add rd/wrmsr interfaces with preset registers
native_{rdmsr,wrmsr}_safe_regs are two new interfaces which allow presetting of a subset of eight x86 GPRs before executing the rd/wrmsr instructions. This is needed at least on AMD K8 for accessing an erratum workaround MSR.
Originally based on an idea by H. Peter Anvin.
H. Peter Anvin
reviewerQuoted 3 lines · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
That is not what the *msr*_amd*() functions do.
NAK. This is a totally bogus patch.
-hpa
Borislav Petkov
maintainerQuoted 8 lines · H. Peter Anvin
Quoted 3 lines · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
That is not what the *msr*_amd*() functions do.
NAK. This is a totally bogus patch.
The *msr*_amd*() variants were used instead of the normal *msrl_safe variants although the AMD variants weren't needed there at all.
This has no issue on baremetal but breaks xen and this is how we caught this.
So the patch corrects the original patch so that xen is happy too.
Signature · 2 lines
Regards/Gruss, Boris.
H. Peter Anvin
reviewerQuoted 10 lines · Borislav Petkov
Quoted 8 lines · H. Peter Anvin
Quoted 3 lines · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
That is not what the *msr*_amd*() functions do.
NAK. This is a totally bogus patch.
Quoted 8 lines
The *msr*_amd*() variants were used instead of the normal
*msrl_safevariants although the AMD variants weren't needed there at all.This has no issue on baremetal but breaks xen and this is how we caught this.
So the patch corrects the original patch so that xen is happy too.
If so, then fix the description to match reality and we can take the patch.
-hpa
Borislav Petkov
maintainerQuoted 1 line · H. Peter Anvin
If so, then fix the description to match reality and we can take the patch.
Andre, would you please do that?
Thanks.
Signature · 2 lines
Regards/Gruss, Boris.
H. Peter Anvin
reviewerQuoted 5 lines · Borislav Petkov
Quoted 1 line · H. Peter Anvin
If so, then fix the description to match reality and we can take the patch.
Andre, would you please do that?
Ah, but now we get:
Quoted 6 lines
Quoted 3 lines
I'm not following: If the AMD variants (putting a special value into %edi) can be freely replaced by the non-AMD variants, why did the AMD special ones get used in the first place?
Older CPUs (K8) needed the AMD variants, starting with family 10h we can use the normal versions.
So no, not correct.
-hpa
Borislav Petkov
maintainerQuoted 8 lines · H. Peter Anvin
Quoted 6 lines
Quoted 3 lines
I'm not following: If the AMD variants (putting a special value into %edi) can be freely replaced by the non-AMD variants, why did the AMD special ones get used in the first place?
Older CPUs (K8) needed the AMD variants, starting with family 10h we can use the normal versions.
So no, not correct.
I don't see what the problem is: the amd* variants shouldn't have been used at all in the first place. Fullstop. The patch should've been doing *msrl_safe from the get-go. Regardless of family.
So what's the issue?
Signature · 2 lines
Regards/Gruss, Boris.
H. Peter Anvin
reviewerQuoted 9 lines · Andre Przywara
Because we are behind a family check before tweaking the topology bit, we can use the standard rd/wrmsr variants for the CPUID feature register.
This fixes a crash when using the kernel as a Xen Dom0 on affected Trinity systems. Thewrmsrl_amd_safeis not properly paravirtualized yet (this will be fixed in another patch).Signed-off-by Andre Przywara <[email protected]>Cc [email protected] # 3.4+
With Konrad's patch, this is no longer relevant, correct?
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.