Fix undefined operation VMXOFF during reboot and crash
PATCH 1/1 Fix undefined operation VMXOFF during reboot and crash
If a panic/reboot occurs when CR4 has VMX enabled, a VMXOFF is done on all CPUS, to allow the INIT IPI to function, since INIT is suppressed when CPUs are in VMX root operation. However, VMXOFF causes an undefined operation fault if the CPU is not in VMX operation, that is, VMXON has not been executed, or VMXOFF has been executed, but VMX is enabled. This fix makes the reboot work more reliably by modifying the #UD handler to skip the VMXOFF if VMX is enabled on the CPU and the VMXOFF is executed as part of cpu_emergency_vmxoff().
The logic in reboot.c is also corrected, since the point of forcing the processor out of VMX root operation is because when VMX root operation is enabled, the processor INIT signal is always masked. See Intel SDM section on differences between VMX Root operation and normal operation. Thus every CPU must be forced out of VMX operation. Since the CPU will hang rather than restart, a manual "reset" is the only way out of this state (or if there is a BMC, it can issue a RESET to the chip).
--- arch/x86/include/asm/virtext.h | 24 ++++++++++++---- arch/x86/kernel/reboot.c | 13 ++------- arch/x86/kernel/traps.c | 52 ++++++++++++++++++++++++++++++++-- 3 files changed, 71 insertions(+), 18 deletions(-)
arch/x86/include/asm/virtext.h +18 -6
@@ -13,12 +13,16 @@#ifndef _ASM_X86_VIRTEX_H#define _ASM_X86_VIRTEX_H +#include <linux/percpu.h> +#include <asm/processor.h> #include <asm/vmx.h> #include <asm/svm.h> #include <asm/tlbflush.h> +DECLARE_PER_CPU_READ_MOSTLY (int, doing_emergency_vmxoff); +/* * VMX functions: */
@@ -33,8 +37,8 @@ static inline int cpu_has_vmx(void)/** Disable VMX on the current CPU * * vmxoff causes a undefined-opcode exception if vmxon was not run -* on the CPU previously. Only call this function if you know VMX -* is enabled. +* on the CPU previously. Only call this function directly if you know VMX +* is enabled *and* CPU is in VMX root operation. */ static inline void cpu_vmxoff (void ) {
@@ -47,17 +51,25 @@ static inline int cpu_vmx_enabled(void)return __read_cr4 () &X86_CR4_VMXE ; } -/** Disable VMX if it is enabled on the current CPU +/** Force disable VMX if it is enabled on the current CPU. +* Note that if CPU is not in VMX root operation this +* VMXOFF will fault an undefined operation fault. +* So the 'doing_emergency_vmxoff' percpu flag is set, +* the trap handler for just restarts execution after +* the VMXOFF instruction. * -* You shouldn't call this if cpu_has_vmx() returns 0. +* You shouldn't call this directly if cpu_has_vmx() returns 0. */ static inline void __cpu_emergency_vmxoff (void ) { -if (cpu_vmx_enabled ()) +if (cpu_vmx_enabled ()) { +this_cpu_write (doing_emergency_vmxoff,1 );cpu_vmxoff (); +this_cpu_write (doing_emergency_vmxoff,0 ); + } } -/** Disable VMX if it is supported and enabled on the current CPU +/** Force disable VMX if it is supported and enabled on the current CPU */ static inline void cpu_emergency_vmxoff (void ) {
arch/x86/kernel/reboot.c +3 -10
@@ -540,21 +540,14 @@ static void emergency_vmx_disable_all(void)* * For safety, we will avoid running the nmi_shootdown_cpus() * stuff unnecessarily, but we don't have a way to check -* if other CPUs have VMX enabled. So we will call it only if the -* CPU we are running on has VMX enabled. -* -* We will miss cases where VMX is not enabled on all CPUs. This -* shouldn't do much harm because KVM always enable VMX on all -* CPUs anyway. But we can miss it on the small window where KVM -* is still enabling VMX. +* if other CPUs have VMX enabled. */ -if (cpu_has_vmx () &&cpu_vmx_enabled ()) { +if (cpu_has_vmx ()) {/* Disable VMX on this CPU. */ -cpu_vmxoff (); +cpu_emergency_vmxoff ();/* Halt and disable VMX on the other CPUs */ nmi_shootdown_cpus (vmxoff_nmi); - } }
arch/x86/kernel/traps.c +50 -2
83 lines not on this page · raw message
Signature · 1 line
2.26.2
kernel test robot
build botThank you for the patch! Perhaps something to improve:
Hi "David,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on next-20200610]
[cannot apply to tip/x86/core v5.7 v5.7-rc7 v5.7-rc6 v5.7]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/David-P-Reed/Fix-undefined-operation-VMXOFF-during-reboot-and-crash/20200611-022322 base: 976c320052ff22a7f9b8be0d56aefacb3a40f466 config: i386-tinyconfig (attached as .config) compiler: gcc-9 (Debian 9.3.0-13) 9.3.0 reproduce (this is a W=1 build): # save the attached .config to linux build tree make W=1 ARCH=i386
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>, old ones prefixed by <<):
arch/x86/kernel/traps.c:91:5: warning: no previous prototype for 'is_valid_bugaddr' [-Wmissing-prototypes] 91 | int is_valid_bugaddr(unsigned long addr)
| ^~~~~~~~~~~~~~~~arch/x86/kernel/traps.c: In function 'fixup_emergency_vmxoff':
>> arch/x86/kernel/traps.c:137:2: warning: 'static' is not at beginning of declaration [-Wold-style-declaration]
137 | const static u8 insn_vmxoff[3] = { 0x0f, 0x01, 0xc4 }; | ^~~~~
vim +/static +137 arch/x86/kernel/traps.c
121
122 /*
123 * Fix any unwanted undefined operation fault due to VMXOFF instruction that
124 * is needed to ensure that CPU is not in VMX root operation at time of
125 * a reboot/panic CPU reset. There is no safe and reliable way to know
126 * if a processor is in VMX root operation, other than to skip the
127 * VMXOFF. It is safe to just skip any VMXOFF that might generate this
128 * exception, when VMX operation is enabled in CR4. In the extremely
129 * rare case that a VMXOFF is erroneously executed while VMX is enabled,
130 * but VMXON has not been executed yet, the undefined opcode fault
131 * should not be missed by valid code, though it would be an error.
132 * To detect this, we could somehow restrict the instruction address
133 * to the specific use during reboot/panic.
134 */
135 static int fixup_emergency_vmxoff(struct pt_regs *regs, int trapnr)
136 {
> 137 const static u8 insn_vmxoff[3] = { 0x0f, 0x01, 0xc4 }; 138 u8 ud[3];
139
140 if (trapnr != X86_TRAP_UD)
141 return 0;
142 if (!cpu_vmx_enabled())
143 return 0;
144 if (!this_cpu_read(doing_emergency_vmxoff))
145 return 0;
146
147 /* undefined instruction must be in kernel and be VMXOFF */
148 if (regs->ip < TASK_SIZE_MAX)
149 return 0;
150 if (probe_kernel_address((u8 *)regs->ip, ud))
151 return 0;
152 if (memcmp(ud, insn_vmxoff, sizeof(insn_vmxoff)))
153 return 0;
154
155 regs->ip += sizeof(insn_vmxoff);
156 return 1;
157 }
158
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Hi David,
If you happen to make a v2 of this patch, there are a few comments that begin with "/**" but they are not kernel-doc comments, so they should instead begin with just "/*". Please see below.
(and you did not introduce this comment style here.)
Quoted 30 lines · David P. Reed
Signed-off-by David P. Reed <[email protected]>--- arch/x86/include/asm/virtext.h | 24 ++++++++++++---- arch/x86/kernel/reboot.c | 13 ++------- arch/x86/kernel/traps.c | 52 ++++++++++++++++++++++++++++++++-- 3 files changed, 71 insertions(+), 18 deletions(-)arch/x86/include/asm/virtext.h +4
-0@@ -13,12 +13,16 @@#ifndef _ASM_X86_VIRTEX_H#define _ASM_X86_VIRTEX_H +#include <linux/percpu.h> +#include <asm/processor.h> #include <asm/vmx.h> #include <asm/svm.h> #include <asm/tlbflush.h> +DECLARE_PER_CPU_READ_MOSTLY (int, doing_emergency_vmxoff); +/* * VMX functions: */
arch/x86/include/asm/virtext.h static inline int cpu_has_vmx(void) · line 37
Quoted 1 line
arch/x86/include/asm/virtext.h +0
-0@@ -33,8 +37,8 @@ static inline int cpu_has_vmx(void)/** Disable VMX on the current CPU
just
/* Disable VMX on the current CPU
Quoted 10 lines
arch/x86/include/asm/virtext.h +2
-2* * vmxoff causes a undefined-opcode exception if vmxon was not run -* on the CPU previously. Only call this function if you know VMX -* is enabled. +* on the CPU previously. Only call this function directly if you know VMX +* is enabled *and* CPU is in VMX root operation. */ static inline void cpu_vmxoff (void ) {
arch/x86/include/asm/virtext.h static inline int cpu_vmx_enabled(void) · line 54
Quoted 5 lines
arch/x86/include/asm/virtext.h +1
-1@@ -47,17 +51,25 @@ static inline int cpu_vmx_enabled(void)return __read_cr4 () &X86_CR4_VMXE ; } -/** Disable VMX if it is enabled on the current CPU +/** Force disable VMX if it is enabled on the current CPU.
just
/* Force disable VMX if it is enabled on the current CPU.
Quoted 11 lines
arch/x86/include/asm/virtext.h +6
-1+* Note that if CPU is not in VMX root operation this +* VMXOFF will fault an undefined operation fault. +* So the 'doing_emergency_vmxoff' percpu flag is set, +* the trap handler for just restarts execution after +* the VMXOFF instruction. * -* You shouldn't call this if cpu_has_vmx() returns 0. +* You shouldn't call this directly if cpu_has_vmx() returns 0. */ static inline void __cpu_emergency_vmxoff (void ) {
arch/x86/include/asm/virtext.h static inline int cpu_vmx_enabled(void) · line 72
Quoted 10 lines
arch/x86/include/asm/virtext.h +5
-2-if (cpu_vmx_enabled ()) +if (cpu_vmx_enabled ()) { +this_cpu_write (doing_emergency_vmxoff,1 );cpu_vmxoff (); +this_cpu_write (doing_emergency_vmxoff,0 ); + } } -/** Disable VMX if it is supported and enabled on the current CPU +/** Force disable VMX if it is supported and enabled on the current CPU
ditto.
arch/x86/include/asm/virtext.h static inline int cpu_vmx_enabled(void) · line 75
Quoted 3 lines
arch/x86/include/asm/virtext.h +0
-0*/ static inline void cpu_emergency_vmxoff (void ) {
thanks.
Signature · 1 line
~Randy
Quoted 25 lines · David P. Reed
arch/x86/kernel/traps.c +25
-0+/* +* Fix any unwanted undefined operation fault due to VMXOFF instruction that +* is needed to ensure that CPU is not in VMX root operation at time of +* a reboot/panic CPU reset. There is no safe and reliable way to know +* if a processor is in VMX root operation, other than to skip the +* VMXOFF. It is safe to just skip any VMXOFF that might generate this +* exception, when VMX operation is enabled in CR4. In the extremely +* rare case that a VMXOFF is erroneously executed while VMX is enabled, +* but VMXON has not been executed yet, the undefined opcode fault +* should not be missed by valid code, though it would be an error. +* To detect this, we could somehow restrict the instruction address +* to the specific use during reboot/panic. +*/ +static int fixup_emergency_vmxoff (struct pt_regs *regs,int trapnr) +{ +const static u8 insn_vmxoff[3 ] = {0x0f ,0x01 ,0xc4 }; +u8 ud[3 ]; + +if (trapnr !=X86_TRAP_UD ) +return 0 ; +if (!cpu_vmx_enabled ()) +return 0 ; +if (!this_cpu_read (doing_emergency_vmxoff)) +return 0 ; +
arch/x86/kernel/traps.c int fixup_bug(struct pt_regs *regs, int trapnr) · line 156
Quoted 10 lines
arch/x86/kernel/traps.c +10
-0+/* undefined instruction must be in kernel and be VMXOFF */ +if (regs->ip <TASK_SIZE_MAX ) +return 0 ; +if (probe_kernel_address ((u8 *)regs->ip, ud)) +return 0 ; +if (memcmp (ud, insn_vmxoff,sizeof (insn_vmxoff))) +return 0 ; + + regs->ip +=sizeof (insn_vmxoff); +return 1 ;
We have exception fixups to avoid exactly that kind of horrible workarounds all over the place.
static inline int cpu_vmxoff_safe(void)
{
int err;
asm volatile("2: vmxoff; xor %[err],%[err]\n"
"1:\n\t"
".section .fixup,\"ax\"\n\t"
"3: mov %[fault],%[err] ; jmp 1b\n\t"
".previous\n\t"
_ASM_EXTABLE(2b, 3b)
: [err] "=a" (err)
: [fault] "i" (-EFAULT)
: "memory");
return err;
}static inline void __cpu_emergency_vmxoff(void)
{
if (!cpu_vmx_enabled())
return;
if (!cpu_vmxoff_safe())
cr4_clear_bits(X86_CR4_VMXE);
}Problem solved.
Thanks,
tglx
Gah, I typed too slow :-)
Quoted 19 lines · Thomas Gleixner
We have exception fixups to avoid exactly that kind of horrible workarounds all over the place.
static inline int
cpu_vmxoff_safe(void){ int err; asm volatile("2: vmxoff; xor %[err],%[err]\n" "1:\n\t" ".section .fixup,\"ax\"\n\t" "3: mov %[fault],%[err] ; jmp 1b\n\t" ".previous\n\t" _ASM_EXTABLE(2b, 3b) : [err] "=a" (err) : [fault] "i" (-EFAULT) : "memory"); return err; }
Quoted 6 lines
static inline void
__cpu_emergency_vmxoff(void){ if (!cpu_vmx_enabled()) return; if (!cpu_vmxoff_safe()) cr4_clear_bits(X86_CR4_VMXE);
This bit is wrong, CR4.VMXE should be cleared even if VMXOFF faults, e.g. if this is called in NMI context and the NMI arrived in KVM code between VMXOFF and clearing CR4.VMXE.
All other VMXOFF faults are mode related, i.e. any fault is guaranteed to be due to the !post-VMXON check unless we're magically in RM, VM86, compat mode, or at CPL>0.
Quoted 7 lines
}
Problem solved.
Thanks,
tglx
Quoted 1 line · Sean Christopherson
Gah, I typed too slow :-)
Haha. I had the same thought.
Quoted 27 lines · Thomas Gleixner
Quoted 19 lines
We have exception fixups to avoid exactly that kind of horrible workarounds all over the place.
static inline int
cpu_vmxoff_safe(void){ int err; asm volatile("2: vmxoff; xor %[err],%[err]\n" "1:\n\t" ".section .fixup,\"ax\"\n\t" "3: mov %[fault],%[err] ; jmp 1b\n\t" ".previous\n\t" _ASM_EXTABLE(2b, 3b) : [err] "=a" (err) : [fault] "i" (-EFAULT) : "memory"); return err; }Quoted 6 lines
static inline void
__cpu_emergency_vmxoff(void){ if (!cpu_vmx_enabled()) return; if (!cpu_vmxoff_safe()) cr4_clear_bits(X86_CR4_VMXE);
Quoted 3 lines
This bit is wrong, CR4.VMXE should be cleared even if VMXOFF faults, e.g. if this is called in NMI context and the NMI arrived in KVM code between VMXOFF and clearing CR4.VMXE.
Oh, right.
Quoted 3 lines
All other VMXOFF faults are mode related, i.e. any fault is guaranteed to be due to the !post-VMXON check unless we're magically in RM, VM86, compat mode, or at CPL>0.
Your patch is simpler indeed.
Thanks,
tglx
Quoted 44 lines · David P. Reed
If a panic/reboot occurs when CR4 has VMX enabled, a VMXOFF is done on all CPUS, to allow the INIT IPI to function, since INIT is suppressed when CPUs are in VMX root operation. However, VMXOFF causes an undefined operation fault if the CPU is not in VMX operation, that is, VMXON has not been executed, or VMXOFF has been executed, but VMX is enabled. This fix makes the reboot work more reliably by modifying the #UD handler to skip the VMXOFF if VMX is enabled on the CPU and the VMXOFF is executed as part of
cpu_emergency_vmxoff().
The logic inreboot.cis also corrected, since the point of forcing the processor out of VMX root operation is because when VMX root operation is enabled, the processor INIT signal is always masked. See Intel SDM section on differences between VMX Root operation and normal operation. Thus every CPU must be forced out of VMX operation. Since the CPU will hang rather than restart, a manual "reset" is the only way out of this state (or if there is a BMC, it can issue a RESET to the chip).Signed-off-by David P. Reed <[email protected]>---
(quoted diff) +11
-3@@ -47,17 +51,25 @@ static inline int cpu_vmx_enabled(void) return __read_cr4() & X86_CR4_VMXE; } -/** Disable VMX if it is enabled on the current CPU +/** Force disable VMX if it is enabled on the current CPU. + * Note that if CPU is not in VMX root operation this + * VMXOFF will fault an undefined operation fault. + * So the 'doing_emergency_vmxoff' percpu flag is set, + * the trap handler for just restarts execution after + * the VMXOFF instruction. * - * You shouldn't call this if cpu_has_vmx() returns 0. + * You shouldn't call this directly if cpu_has_vmx() returns 0. */ static inline void __cpu_emergency_vmxoff(void) { - if (cpu_vmx_enabled()) + if (cpu_vmx_enabled()) { + this_cpu_write(doing_emergency_vmxoff, 1); cpu_vmxoff(); + this_cpu_write(doing_emergency_vmxoff, 0); + } }
...
Quoted 46 lines
(quoted diff) +39
-0+/* + * Fix any unwanted undefined operation fault due to VMXOFF instruction that + * is needed to ensure that CPU is not in VMX root operation at time of + * a reboot/panic CPU reset. There is no safe and reliable way to know + * if a processor is in VMX root operation, other than to skip the + * VMXOFF. It is safe to just skip any VMXOFF that might generate this + * exception, when VMX operation is enabled in CR4. In the extremely + * rare case that a VMXOFF is erroneously executed while VMX is enabled, + * but VMXON has not been executed yet, the undefined opcode fault + * should not be missed by valid code, though it would be an error. + * To detect this, we could somehow restrict the instruction address + * to the specific use during reboot/panic. + */ +static int fixup_emergency_vmxoff(struct pt_regs *regs, int trapnr) +{ + const static u8 insn_vmxoff[3] = { 0x0f, 0x01, 0xc4 }; + u8 ud[3]; + + if (trapnr != X86_TRAP_UD) + return 0; + if (!cpu_vmx_enabled()) + return 0; + if (!this_cpu_read(doing_emergency_vmxoff)) + return 0; + + /* undefined instruction must be in kernel and be VMXOFF */ + if (regs->ip < TASK_SIZE_MAX) + return 0; + if (probe_kernel_address((u8 *)regs->ip, ud)) + return 0; + if (memcmp(ud, insn_vmxoff, sizeof(insn_vmxoff))) + return 0; + + regs->ip += sizeof(insn_vmxoff); + return 1; +} + static nokprobe_inline int do_trap_no_signal(struct task_struct *tsk, int trapnr, const char *str, struct pt_regs *regs, long error_code)@@ -193,9 +234,16 @@ static void do_error_trap(struct pt_regs *regs, long error_code, char *str, /* * WARN*()s end up here; fix them up before we call the * notifier chain. + * Also, VMXOFF causes unwanted fault during reboot + * if VMX is enabled, but not in VMX root operation. Fix
Review static void do_error_trap(struct pt_regs *regs, long error_code, char *str, · line 246
Quoted 10 lines
(quoted diff) +7
-2+ * before calling notifier chain. */ - if (!user_mode(regs) && fixup_bug(regs, trapnr)) - return; + if (!user_mode(regs)) { + if (fixup_bug(regs, trapnr)) + return; + if (fixup_emergency_vmxoff(regs, trapnr)) + return; + }
Isn't this just a really kludgy way of doing fixup on vmxoff? E.g. wouldn't the below patch do the trick?
arch/x86/include/asm/virtext.h +6 -4
@@ -32,13 +32,15 @@ static inline int cpu_has_vmx(void)/** Disable VMX on the current CPU * -* vmxoff causes a undefined-opcode exception if vmxon was not run -* on the CPU previously. Only call this function if you know VMX -* is enabled. +* VMXOFF causes a #UD if the CPU is not post-VMXON, eat any #UDs to handle +* races with a hypervisor doing VMXOFF, e.g. if an NMI arrived between VMXOFF +* and clearing CR4.VMXE. */ static inline void cpu_vmxoff (void ) { - asmvolatile ("vmxoff" ); + asmvolatile ("1: vmxoff\n\t" +"2:\n\t" + _ASM_EXTABLE(1b, 2b));cr4_clear_bits (X86_CR4_VMXE ); }
Andy Lutomirski
NAKQuoted 10 lines · David P. Reed
If a panic/reboot occurs when CR4 has VMX enabled, a VMXOFF is done on all CPUS, to allow the INIT IPI to function, since INIT is suppressed when CPUs are in VMX root operation. However, VMXOFF causes an undefined operation fault if the CPU is not in VMX operation, that is, VMXON has not been executed, or VMXOFF has been executed, but VMX is enabled.
I’m surprised. Wouldn’t this mean that emergency reboots always fail it a VM is running? I would think someone would have noticed before.
Quoted 4 lines
This fix makes the reboot
work more reliably by modifying the #UD handler to skip the VMXOFF if VMX is enabled on the CPU and the VMXOFF is executed as part ofcpu_emergency_vmxoff().
NAK. See below.
Quoted 38 lines
The logic in
reboot.cis also corrected, since the point of forcing the processor out of VMX root operation is because when VMX root operation is enabled, the processor INIT signal is always masked. See Intel SDM section on differences between VMX Root operation and normal operation. Thus every CPU must be forced out of VMX operation. Since the CPU will hang rather than restart, a manual "reset" is the only way out of this state (or if there is a BMC, it can issue a RESET to the chip).Signed-off-by David P. Reed <[email protected]>---
arch/x86/include/asm/virtext.h | 24 ++++++++++++---- arch/x86/kernel/reboot.c | 13 ++------- arch/x86/kernel/traps.c | 52 ++++++++++++++++++++++++++++++++-- 3 files changed, 71 insertions(+), 18 deletions(-)arch/x86/include/asm/virtext.h +4
-0@@ -13,12 +13,16 @@#ifndef _ASM_X86_VIRTEX_H#define _ASM_X86_VIRTEX_H +#include <linux/percpu.h> +#include <asm/processor.h> #include <asm/vmx.h> #include <asm/svm.h> #include <asm/tlbflush.h> +DECLARE_PER_CPU_READ_MOSTLY (int, doing_emergency_vmxoff); +/* * VMX functions: */
arch/x86/include/asm/virtext.h static inline int cpu_has_vmx(void) · line 42
Quoted 8 lines
arch/x86/include/asm/virtext.h +2
-2@@ -33,8 +37,8 @@ static inline int cpu_has_vmx(void)/** Disable VMX on the current CPU * * vmxoff causes a undefined-opcode exception if vmxon was not run -* on the CPU previously. Only call this function if you know VMX -* is enabled. +* on the CPU previously. Only call this function directly if you know VMX +* is enabled *and* CPU is in VMX root operation. */
So presumably the bug is someone calling this inappropriatelet?
Quoted 16 lines
arch/x86/include/asm/virtext.h +7
-2static inline void cpu_vmxoff (void ) {@@ -47,17 +51,25 @@ static inline int cpu_vmx_enabled(void)return __read_ () &X86_CR4_VMXE ; } -/** Disable VMX if it is enabled on the current CPU +/** Force disable VMX if it is enabled on the current CPU. +* Note that if CPU is not in VMX root operation this +* VMXOFF will fault an undefined operation fault. +* So the 'doing_emergency_vmxoff' percpu flag is set, +* the trap handler for just restarts execution after +* the VMXOFF instruction. * -* You shouldn't call this if cpu_has_vmx() returns 0. +* You shouldn't call this directly if cpu_has_vmx() returns 0.
arch/x86/include/asm/virtext.h static inline int cpu_vmx_enabled(void) · line 70
Quoted 10 lines
arch/x86/include/asm/virtext.h +4
-1*/ static inline void __cpu_emergency_vmxoff (void ) { -if (cpu_vmx_enabled ()) +if (cpu_vmx_enabled ()) { +this_cpu_write (doing_emergency_vmxoff,1 );cpu_vmxoff (); +this_cpu_write (doing_emergency_vmxoff,0 ); + } }
NAK. Just write this in asm with an exception handler that does the right thing.
Please also try to identify the actual bug. Because I have a sneaking suspicion that you are running an out of tree module that has issues. If so, the patch should explain this.
Quoted 73 lines
arch/x86/include/asm/virtext.h +1
-1-/** Disable VMX if it is supported and enabled on the current CPU +/** Force disable VMX if it is supported and enabled on the current CPU */ static inline void cpu_emergency_vmxoff (void ) {arch/x86/kernel/reboot.c +3
-10@@ -540,21 +540,14 @@ static void emergency_vmx_disable_all(void)* * For safety, we will avoid running the nmi_shootdown_cpus() * stuff unnecessarily, but we don't have a way to check -* if other CPUs have VMX enabled. So we will call it only if the -* CPU we are running on has VMX enabled. -* -* We will miss cases where VMX is not enabled on all CPUs. This -* shouldn't do much harm because KVM always enable VMX on all -* CPUs anyway. But we can miss it on the small window where KVM -* is still enabling VMX. +* if other CPUs have VMX enabled. */ -if (cpu_has_vmx () &&cpu_vmx_enabled ()) { +if (cpu_has_vmx ()) {/* Disable VMX on this CPU. */ -cpu_vmxoff (); +cpu_emergency_vmxoff ();/* Halt and disable VMX on the other CPUs */ nmi_shootdown_cpus (vmxoff_nmi); - } }arch/x86/kernel/traps.c +9
-0@@ -39,6 +39,7 @@#include <linux/io.h> #include <linux/hardirq.h> #include <linux/atomic.h> +#include <linux/percpu.h> #include <asm/stacktrace.h> #include <asm/processor.h> @@ -59,6 +60,7 @@#include <asm/umip.h> #include <asm/insn.h> #include <asm/insn-eval.h> +#include <asm/virtext.h> #ifdef CONFIG_X86_64 #include <asm/x86_init.h> @@ -70,6 +72,8 @@#include <asm/proto.h> #endif +DEFINE_PER_CPU_READ_MOSTLY (int, doing_emergency_vmxoff) =0 ; +DECLARE_BITMAP (system_vectors,NR_VECTORS );static inline void cond_local_irq_enable (struct pt_regs *regs)@@ -115,6 +119,43 @@ int fixup_bug(struct pt_regs *regs, int trapnr)return 0 ; } +/* +* Fix any unwanted undefined operation fault due to VMXOFF instruction that +* is needed to ensure that CPU is not in VMX root operation at time of +* a reboot/panic CPU reset. There is no safe and reliable way to know +* if a processor is in VMX root operation, other than to skip the
arch/x86/kernel/traps.c int fixup_bug(struct pt_regs *regs, int trapnr) · line 136
Quoted 10 lines
arch/x86/kernel/traps.c +10
-0+* VMXOFF. It is safe to just skip any VMXOFF that might generate this +* exception, when VMX operation is enabled in CR4. In the extremely +* rare case that a VMXOFF is erroneously executed while VMX is enabled, +* but VMXON has not been executed yet, the undefined opcode fault +* should not be missed by valid code, though it would be an error. +* To detect this, we could somehow restrict the instruction address +* to the specific use during reboot/panic. +*/ +static int fixup_emergency_vmxoff (struct pt_regs *regs,int trapnr) +{
NAK.
Quoted 11 lines · Andy Lutomirski
Quoted 10 lines · David P. Reed
If a panic/reboot occurs when CR4 has VMX enabled, a VMXOFF is done on all CPUS, to allow the INIT IPI to function, since INIT is suppressed when CPUs are in VMX root operation. However, VMXOFF causes an undefined operation fault if the CPU is not in VMX operation, that is, VMXON has not been executed, or VMXOFF has been executed, but VMX is enabled.
Quoted 2 lines
I’m surprised. Wouldn’t this mean that emergency reboots always fail it a VM is running? I would think someone would have noticed before.
The call to cpu_vmxoff() is conditioned on CR4.VMXE==1, which KVM toggles in tandem with VMXON and VMXOFF. Out of tree hypervisors presumably do the same. That's obviously not atomic though, e.g. VMXOFF will #UD if the vmxoff_nmi() NMI arrives between CR4.VMXE=1 and VMXON, or between VMXOFF and CR4.VMXE=0.
Quoted 13 lines · Sean Christopherson
Quoted 11 lines · Andy Lutomirski
Quoted 10 lines · David P. Reed
If a panic/reboot occurs when CR4 has VMX enabled, a VMXOFF is done on all CPUS, to allow the INIT IPI to function, since INIT is suppressed when CPUs are in VMX root operation. However, VMXOFF causes an undefined operation fault if the CPU is not in VMX operation, that is, VMXON has not been executed, or VMXOFF has been executed, but VMX is enabled.
Quoted 8 lines
Quoted 2 lines
I’m surprised. Wouldn’t this mean that emergency reboots always fail it a VM is running? I would think someone would have noticed before.
The call to
cpu_vmxoff()is conditioned on CR4.VMXE==1, which KVM toggles in tandem with VMXON and VMXOFF. Out of tree hypervisors presumably do the same. That's obviously not atomic though, e.g. VMXOFF will #UD if thevmxoff_nmi()NMI arrives between CR4.VMXE=1 and VMXON, or between VMXOFF and CR4.VMXE=0.
It would be nice for the commit message to say "this happens when nmxoff_nmi() races with KVM's VMXON/VMXOFF toggling". Or the commit message should say something else if the bug happens for a different reason.
The race with KVM should be quite unusual, since it involves rebooting concurrently with loading or unloading KVM.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 5 lines · David P. Reed
To respond to Thomas Gleixner's suggestion about exception masking mechanism
- it may well be a better fix, but a) I used "BUG" as a model, and b) the exception masking is undocumented anywhere I can find. These are "static inline" routines, and only the "emergency" version needs protection, because you'd want a random VMXOFF to actually trap.
The only in-kernel usage of cpu_vmxoff() are for emergencies. And, the only reasonable source of faults on VMXOFF is that VMX is already off, i.e. for the kernel's usage, the goal is purely to ensure VMX is disabled, how we get there doesn't truly matter.
Quoted 10 lines
In at least one of the calls to emergency, it is stated that no locks may be taken at all because of where it was.
Further, I have a different patch that requires a scratch page per processor to exist, but which never takes a UD fault. (basically, it attempts VMXON first, and then does VMXOFF after VMXON, which ensures exit from VMX root mode, but VMXON needs a blank page to either succeed or fail without GP fault). If someone prefers that, it's local to the routine, but requires a new scratch page per processor be allocated. So after testing it, I decided in the interest of memory reduction that the masking of UD was preferable.
Please no, doing VMXON, even temporarily, could cause breakage. The CPU's VMCS cache isn't cleared on VMXOFF. Doing VMXON after kdump_nmi_callback() invokes cpu_crash_vmclear_loaded_vmcss() would create a window where VMPTRLD could succeed in a hypervisor and lead to memory corruption in the new kernel when the VMCS is evicted from the non-coherent VMCS cache.
Quoted 4 lines
I'm happy to resubmit the masking exception patch as version 2, if it works in my test case.
Advice?
Please test the below, which simply eats any exception on VMXOFF.
arch/x86/include/asm/virtext.h +6 -4
@@ -32,13 +32,15 @@ static inline int cpu_has_vmx(void)/** Disable VMX on the current CPU * -* vmxoff causes a undefined-opcode exception if vmxon was not run -* on the CPU previously. Only call this function if you know VMX -* is enabled. +* VMXOFF causes a #UD if the CPU is not post-VMXON, eat any #UDs to handle +* races with a hypervisor doing VMXOFF, e.g. if an NMI arrived between VMXOFF +* and clearing CR4.VMXE. */ static inline void cpu_vmxoff (void ) { - asmvolatile ("vmxoff" ); + asmvolatile ("1: vmxoff\n\t" +"2:\n\t" + _ASM_EXTABLE(1b, 2b));cr4_clear_bits (X86_CR4_VMXE ); }
Quoted 50 lines · Sean Christopherson
Quoted 5 lines · David P. Reed
To respond to Thomas Gleixner's suggestion about exception masking mechanism
- it may well be a better fix, but a) I used "BUG" as a model, and b) the exception masking is undocumented anywhere I can find. These are "static inline" routines, and only the "emergency" version needs protection, because you'd want a random VMXOFF to actually trap.The only in-kernel usage of
cpu_vmxoff()are for emergencies. And, the only reasonable source of faults on VMXOFF is that VMX is already off, i.e. for the kernel's usage, the goal is purely to ensure VMX is disabled, how we get there doesn't truly matter.Quoted 10 lines
In at least one of the calls to emergency, it is stated that no locks may be taken at all because of where it was.
Further, I have a different patch that requires a scratch page per processor to exist, but which never takes a UD fault. (basically, it attempts VMXON first, and then does VMXOFF after VMXON, which ensures exit from VMX root mode, but VMXON needs a blank page to either succeed or fail without GP fault). If someone prefers that, it's local to the routine, but requires a new scratch page per processor be allocated. So after testing it, I decided in the interest of memory reduction that the masking of UD was preferable.
Please no, doing VMXON, even temporarily, could cause breakage. The CPU's VMCS cache isn't cleared on VMXOFF. Doing VMXON after
kdump_nmi_callback()invokescpu_crash_vmclear_loaded_vmcss()would create a window where VMPTRLD could succeed in a hypervisor and lead to memory corruption in the new kernel when the VMCS is evicted from the non-coherent VMCS cache.Quoted 4 lines
I'm happy to resubmit the masking exception patch as version 2, if it works in my test case.
Advice?
Please test the below, which simply eats any exception on VMXOFF.
arch/x86/include/asm/virtext.h +2
-3@@ -32,13 +32,15 @@ static inline int cpu_has_vmx(void)/** Disable VMX on the current CPU * -* vmxoff causes a undefined-opcode exception if vmxon was not run -* on the CPU previously. Only call this function if you know VMX -* is enabled. +* VMXOFF causes a #UD if the CPU is not post-VMXON, eat any #UDs to handle +* races with a hypervisor doing VMXOFF, e.g. if an NMI arrived between VMXOFF
arch/x86/include/asm/virtext.h static inline int cpu_has_vmx(void) · line 45
Quoted 10 lines
arch/x86/include/asm/virtext.h +4
-1+* and clearing CR4.VMXE. */ static inline void cpu_vmxoff (void ) { - asmvolatile ("vmxoff" ); + asmvolatile ("1: vmxoff\n\t" +"2:\n\t" + _ASM_EXTABLE(1b, 2b));cr4_clear_bits (X86_CR4_VMXE ); }
I think that just eating exceptions like this is asking for trouble. How about having a separate cpu_emergency_vmxoff() that eats exceptions and leaving cpu_vmxoff() alone? Or make cpu_vmxoff() return an error on failure and have the normal caller WARN if there's an error.
Silently eating exceptions in the non-emergency path makes it too easy to regress something without noticing.
PATCH v2 1/1 Fix undefined operation VMXOFF during reboot and crash
If a panic/reboot occurs when CR4 has VMX enabled, a VMXOFF is done on all CPUS, to allow the INIT IPI to function, since INIT is suppressed when CPUs are in VMX root operation. Problem is that VMXOFF will causes undefined operation fault when CPU not in VMX operation, that is, VMXON has not been executed yet, or VMXOFF has been execute but VMX still enabled. Patch makes the reboot work more reliably by masking the exception on VMXOFF in the crash/panic/reboot path, which uses cpu_emergency_vmxoff(). Can happen with KVM due to a race, but that race is rare today. Problem discovered doing out-of-tree x-visor development that uses VMX in a novel way for kernel performance analysis.
The logic in reboot.c is also corrected, since the point of forcing the processor out of VMX root operation is to allow the INIT signal to be unmasked. See Intel SDM section on differences between VMX Root operation and normal operation. Thus every CPU must be forced out of VMX operation. Since the CPU may hang rather if INIT fails than restart, a manual hardware "reset" is the only way out of this state in a lights-out datacenter (well, if there is a BMC, it can issue a hardware RESET to the chip).
Style errors in original file fixed, at request of Randy Dunlap:
eliminate '/**' in non-kernel-doc comments.
--- arch/x86/include/asm/virtext.h | 40 ++++++++++++++++++++++++---------- arch/x86/kernel/reboot.c | 13 +++-------- 2 files changed, 32 insertions(+), 21 deletions(-)
arch/x86/include/asm/virtext.h +29 -11
@@ -30,15 +30,15 @@ static inline int cpu_has_vmx(void) } -/** Disable VMX on the current CPU +/* Disable VMX on the current CPU * -* vmxoff causes a undefined-opcode exception if vmxon was not run -* on the CPU previously. Only call this function if you know VMX -* is enabled. +* vmxoff causes an undefined-opcode exception if vmxon was not run +* on the CPU previously. Only call this function directly if you know VMX +* is enabled *and* CPU is in VMX root operation. */ static inline void cpu_vmxoff (void ) { - asmvolatile ("vmxoff" ); + asmvolatile ("vmxoff" :::"cc" ,"memory" );/* clears all flags on success */ cr4_clear_bits (X86_CR4_VMXE ); }
@@ -47,17 +47,35 @@ static inline int cpu_vmx_enabled(void)return __read_cr4 () &X86_CR4_VMXE ; } -/** Disable VMX if it is enabled on the current CPU -* -* You shouldn't call this if cpu_has_vmx() returns 0. +/* +* Safely disable VMX root operation if active +* Note that if CPU is not in VMX root operation this +* VMXOFF will fault an undefined operation fault, +* so use the exception masking facility to handle that RARE +* case. +* You shouldn't call this directly if cpu_has_vmx() returns 0 +*/ +static inline void cpu_vmxoff_safe (void ) +{ + asm volatile("1:vmxoff\n\t" /* clears all flags on success */ + "2:\n\t" + _ASM_EXTABLE(1b, 2b) + ::: "cc", "memory"); + cr4_clear_bits(X86_CR4_VMXE ); +} + +/* +* Force disable VMX if it is enabled on the current CPU, +* when it is unknown whether CPU is in VMX operation. */ static inline void __cpu_emergency_vmxoff (void ) { -if (cpu_vmx_enabled ()) -cpu_vmxoff (); +if (!cpu_vmx_enabled ()) +return ; +cpu_vmxoff_safe (); } -/** Disable VMX if it is supported and enabled on the current CPU +/* Force disable VMX if it is supported on current CPU */ static inline void cpu_emergency_vmxoff (void ) {
arch/x86/kernel/reboot.c +3 -10
@@ -540,21 +540,14 @@ static void emergency_vmx_disable_all(void)* * For safety, we will avoid running the nmi_shootdown_cpus() * stuff unnecessarily, but we don't have a way to check -* if other CPUs have VMX enabled. So we will call it only if the -* CPU we are running on has VMX enabled. -* -* We will miss cases where VMX is not enabled on all CPUs. This -* shouldn't do much harm because KVM always enable VMX on all -* CPUs anyway. But we can miss it on the small window where KVM -* is still enabling VMX. +* if other CPUs have VMX enabled. */ -if (cpu_has_vmx () &&cpu_vmx_enabled ()) { +if (cpu_has_vmx ()) {/* Disable VMX on this CPU. */ -cpu_vmxoff (); +cpu_emergency_vmxoff ();/* Halt and disable VMX on the other CPUs */ nmi_shootdown_cpus (vmxoff_nmi); - } }
Signature · 1 line
2.26.2
PATCH v2 1/1 Fix undefined operation VMXOFF during reboot and crash
In reply to Andy LutomirskiIf a panic/reboot occurs when CR4 has VMX enabled, a VMXOFF is done on all CPUS, to allow the INIT IPI to function, since INIT is suppressed when CPUs are in VMX root operation. Problem is that VMXOFF will causes undefined operation fault when CPU not in VMX operation, that is, VMXON has not been executed yet, or VMXOFF has been execute but VMX still enabled. Patch makes the reboot work more reliably by masking the exception on VMXOFF in the crash/panic/reboot path, which uses cpu_emergency_vmxoff(). Can happen with KVM due to a race, but that race is rare today. Problem discovered doing out-of-tree x-visor development that uses VMX in a novel way for kernel performance analysis.
The logic in reboot.c is also corrected, since the point of forcing the processor out of VMX root operation is to allow the INIT signal to be unmasked. See Intel SDM section on differences between VMX Root operation and normal operation. Thus every CPU must be forced out of VMX operation. Since the CPU may hang rather if INIT fails than restart, a manual hardware "reset" is the only way out of this state in a lights-out datacenter (well, if there is a BMC, it can issue a hardware RESET to the chip).
Style errors in original file fixed, at request of Randy Dunlap:
eliminate '/**' in non-kernel-doc comments.
--- arch/x86/include/asm/virtext.h | 40 ++++++++++++++++++++++++---------- arch/x86/kernel/reboot.c | 13 +++-------- 2 files changed, 32 insertions(+), 21 deletions(-)
arch/x86/include/asm/virtext.h +29 -11
@@ -30,15 +30,15 @@ static inline int cpu_has_vmx(void) } -/** Disable VMX on the current CPU +/* Disable VMX on the current CPU * -* vmxoff causes a undefined-opcode exception if vmxon was not run -* on the CPU previously. Only call this function if you know VMX -* is enabled. +* vmxoff causes an undefined-opcode exception if vmxon was not run +* on the CPU previously. Only call this function directly if you know VMX +* is enabled *and* CPU is in VMX root operation. */ static inline void cpu_vmxoff (void ) { - asmvolatile ("vmxoff" ); + asmvolatile ("vmxoff" :::"cc" ,"memory" );/* clears all flags on success */ cr4_clear_bits (X86_CR4_VMXE ); }
@@ -47,17 +47,35 @@ static inline int cpu_vmx_enabled(void)return __read_cr4 () &X86_CR4_VMXE ; } -/** Disable VMX if it is enabled on the current CPU -* -* You shouldn't call this if cpu_has_vmx() returns 0. +/* +* Safely disable VMX root operation if active +* Note that if CPU is not in VMX root operation this +* VMXOFF will fault an undefined operation fault, +* so use the exception masking facility to handle that RARE +* case. +* You shouldn't call this directly if cpu_has_vmx() returns 0 +*/ +static inline void cpu_vmxoff_safe (void ) +{ + asm volatile("1:vmxoff\n\t" /* clears all flags on success */ + "2:\n\t" + _ASM_EXTABLE(1b, 2b) + ::: "cc", "memory"); + cr4_clear_bits(X86_CR4_VMXE ); +} + +/* +* Force disable VMX if it is enabled on the current CPU, +* when it is unknown whether CPU is in VMX operation. */ static inline void __cpu_emergency_vmxoff (void ) { -if (cpu_vmx_enabled ()) -cpu_vmxoff (); +if (!cpu_vmx_enabled ()) +return ; +cpu_vmxoff_safe (); } -/** Disable VMX if it is supported and enabled on the current CPU +/* Force disable VMX if it is supported on current CPU */ static inline void cpu_emergency_vmxoff (void ) {
arch/x86/kernel/reboot.c +3 -10
@@ -540,21 +540,14 @@ static void emergency_vmx_disable_all(void)* * For safety, we will avoid running the nmi_shootdown_cpus() * stuff unnecessarily, but we don't have a way to check -* if other CPUs have VMX enabled. So we will call it only if the -* CPU we are running on has VMX enabled. -* -* We will miss cases where VMX is not enabled on all CPUs. This -* shouldn't do much harm because KVM always enable VMX on all -* CPUs anyway. But we can miss it on the small window where KVM -* is still enabling VMX. +* if other CPUs have VMX enabled. */ -if (cpu_has_vmx () &&cpu_vmx_enabled ()) { +if (cpu_has_vmx ()) {/* Disable VMX on this CPU. */ -cpu_vmxoff (); +cpu_emergency_vmxoff ();/* Halt and disable VMX on the other CPUs */ nmi_shootdown_cpus (vmxoff_nmi); - } }
Signature · 1 line
2.26.2
Quoted 25 lines · David P. Reed
arch/x86/include/asm/virtext.h +6
-8-/** Disable VMX on the current CPU +/* Disable VMX on the current CPU * -* vmxoff causes a undefined-opcode exception if vmxon was not run -* on the CPU previously. Only call this function if you know VMX -* is enabled. +* vmxoff causes an undefined-opcode exception if vmxon was not run +* on the CPU previously. Only call this function directly if you know VMX +* is enabled *and* CPU is in VMX root operation. */ static inline void cpu_vmxoff (void ) { - asmvolatile ("vmxoff" ); + asmvolatile ("vmxoff" :::"cc" ,"memory" );/* clears all flags on success */ cr4_clear_bits (X86_CR4_VMXE ); }@@ -47,17 +47,35 @@ static inline int cpu_vmx_enabled(void)return __read_cr4 () &X86_CR4_VMXE ; } -/** Disable VMX if it is enabled on the current CPU -* -* You shouldn't call this if cpu_has_vmx() returns 0. +/*
arch/x86/include/asm/virtext.h static inline int cpu_vmx_enabled(void) · line 60
Quoted 10 lines
arch/x86/include/asm/virtext.h +10
-0+* Safely disable VMX root operation if active +* Note that if CPU is not in VMX root operation this +* VMXOFF will fault an undefined operation fault, +* so use the exception masking facility to handle that RARE +* case. +* You shouldn't call this directly if cpu_has_vmx() returns 0 +*/ +static inline void cpu_vmxoff_safe (void ) +{ + asm volatile("1:vmxoff\n\t" /* clears all flags on success */
Eh, I wouldn't bother with the comment, there are a million other caveats with VMXOFF that are far more interesting.
arch/x86/include/asm/virtext.h static inline int cpu_vmx_enabled(void) · line 63
Quoted 3 lines
arch/x86/include/asm/virtext.h +3
-0+ "2:\n\t" + _ASM_EXTABLE(1b, 2b) + ::: "cc", "memory");
Adding the memory and flags clobber should be a separate patch.
arch/x86/include/asm/virtext.h static inline int cpu_vmx_enabled(void) · line 65
Quoted 2 lines
arch/x86/include/asm/virtext.h +2
-0+cr4_clear_bits (X86_CR4_VMXE ); +}
I don't see any value in safe/unsafe variants. The only in-kernel user of VMXOFF outside of the emergency flows is KVM, which has its own VMXOFF helper, i.e. all users of cpu_vmxoff() want the "safe" variant. Just add the exception fixup to cpu_vmxoff() and call it good.
arch/x86/include/asm/virtext.h static inline int cpu_vmx_enabled(void) · line 75
Quoted 12 lines
arch/x86/include/asm/virtext.h +7
-2+ +/* +* Force disable VMX if it is enabled on the current CPU, +* when it is unknown whether CPU is in VMX operation. */ static inline void __cpu_emergency_vmxoff (void ) { -if (cpu_vmx_enabled ()) -cpu_vmxoff (); +if (!cpu_vmx_enabled ()) +return ; +cpu_vmxoff_safe ();
Unnecessary churn.
Quoted 19 lines
arch/x86/include/asm/virtext.h +1
-1} -/** Disable VMX if it is supported and enabled on the current CPU +/* Force disable VMX if it is supported on current CPU */ static inline void cpu_emergency_vmxoff (void ) {arch/x86/kernel/reboot.c +0
-4@@ -540,21 +540,14 @@ static void emergency_vmx_disable_all(void)* * For safety, we will avoid running the nmi_shootdown_cpus() * stuff unnecessarily, but we don't have a way to check -* if other CPUs have VMX enabled. So we will call it only if the -* CPU we are running on has VMX enabled. -* -* We will miss cases where VMX is not enabled on all CPUs. This
arch/x86/kernel/reboot.c static void emergency_vmx_disable_all(void) · line 547
Quoted 10 lines
arch/x86/kernel/reboot.c +3
-5-* shouldn't do much harm because KVM always enable VMX on all -* CPUs anyway. But we can miss it on the small window where KVM -* is still enabling VMX. +* if other CPUs have VMX enabled. */ -if (cpu_has_vmx () &&cpu_vmx_enabled ()) { +if (cpu_has_vmx ()) {/* Disable VMX on this CPU. */ -cpu_vmxoff (); +cpu_emergency_vmxoff ();
This also needs to be in a separate patch. And it should use __cpu_emergency_vmxoff() instead of cpu_emergency_vmxoff().
Quoted 10 lines
arch/x86/kernel/reboot.c +0
-1/* Halt and disable VMX on the other CPUs */ nmi_shootdown_cpus (vmxoff_nmi); - } }Signature · 1 line
2.26.2