Skip to contentSkip to conversation

KVM: nVMX: fix the layout of struct kvm_vmx_nested_state_hdr

PATCH In review Vitaly Kuznetsov 4 participants 6 messages Jul 13, 2020 – Aug 30, 2020

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 KVM: nVMX: fix the layout of struct kvm_vmx_nested_state_hdr

Before commit 850448f35aaf ("KVM: nVMX: Fix VMX preemption timer migration") struct kvm_vmx_nested_state_hdr looked like:

struct kvm_vmx_nested_state_hdr {
        __u64 vmxon_pa;
        __u64 vmcs12_pa;
        struct {
                __u16 flags;
        } smm;
}

The ABI got broken by the above mentioned commit and an attempt to fix that was made in commit 83d31e5271ac ("KVM: nVMX: fixes for preemption timer migration") which made the structure look like:

struct kvm_vmx_nested_state_hdr {
        __u64 vmxon_pa;
        __u64 vmcs12_pa;
        struct {
                __u16 flags;
        } smm;
        __u32 flags;
        __u64 preemption_timer_deadline;
};

The problem with this layout is that before both changes compilers were allocating 24 bytes for this and although smm.flags is padded to 8 bytes, it is initialized as a 2 byte value. Chances are that legacy userspaces using old layout will be passing uninitialized bytes which will slip into what is now known as 'flags'.

Fixes 850448f35aaf ("KVM: nVMX: Fix VMX preemption timer migration")
Fixes 83d31e5271ac ("KVM: nVMX: fixes for preemption timer migration")

---
- The patch breaks ABI so it needs to go into 5.8.

- This is a successor of "[PATCH] KVM: nVMX: properly pad struct kvm_vmx_nested_state_hdr"

---
 Documentation/virt/kvm/api.rst  | 5 +++--
 arch/x86/include/uapi/asm/kvm.h | 5 +++--
 2 files changed, 6 insertions(+), 4 deletions(-)
Documentation/virt/kvm/api.rst +3 -2
index 320788f81a05..e75992ad856a 100644
@@ -4345,8 +4345,9 @@ Errors:
 	struct {
 		__u16 flags;
 	} smm;
-
-	__u32 flags;
+	__u16 pad16;
+	__u32 pad32;
+	__u64 flags;
 	__u64 preemption_timer_deadline;
   };
 
arch/x86/include/uapi/asm/kvm.h +3 -2
index 0780f97c1850..4ecc6bd49818 100644
@@ -414,8 +414,9 @@ struct kvm_vmx_nested_state_hdr {
 	struct {
 		__u16 flags;
 	} smm;
-
-	__u32 flags;
+	__u16 pad16;
+	__u32 pad32;
+	__u64 flags;
 	__u64 preemption_timer_deadline;
 };
 
Signature · 1 line
2.25.4
Reviewed-by on v2 1/1
#1 lore raw
Quoted 32 lines · Vitaly Kuznetsov

Before commit 850448f35aaf ("KVM: nVMX: Fix VMX preemption timer migration") struct kvm_vmx_nested_state_hdr looked like:

struct kvm_vmx_nested_state_hdr {
        __u64 vmxon_pa;
        __u64 vmcs12_pa;
        struct {
                __u16 flags;
        } smm;
}

The ABI got broken by the above mentioned commit and an attempt to fix that was made in commit 83d31e5271ac ("KVM: nVMX: fixes for preemption timer migration") which made the structure look like:

struct kvm_vmx_nested_state_hdr {
        __u64 vmxon_pa;
        __u64 vmcs12_pa;
        struct {
                __u16 flags;
        } smm;
        __u32 flags;
        __u64 preemption_timer_deadline;
};

The problem with this layout is that before both changes compilers were allocating 24 bytes for this and although smm.flags is padded to 8 bytes, it is initialized as a 2 byte value. Chances are that legacy userspaces using old layout will be passing uninitialized bytes which will slip into what is now known as 'flags'.

Quoted 4 lines
Fixes 850448f35aaf ("KVM: nVMX: Fix VMX preemption timer migration")
Fixes 83d31e5271ac ("KVM: nVMX: fixes for preemption timer migration")

Oops!

#2 lore raw
Quoted 34 lines · Jim Mattson
Quoted 32 lines · Vitaly Kuznetsov

Before commit 850448f35aaf ("KVM: nVMX: Fix VMX preemption timer migration") struct kvm_vmx_nested_state_hdr looked like:

struct kvm_vmx_nested_state_hdr {
        __u64 vmxon_pa;
        __u64 vmcs12_pa;
        struct {
                __u16 flags;
        } smm;
}

The ABI got broken by the above mentioned commit and an attempt to fix that was made in commit 83d31e5271ac ("KVM: nVMX: fixes for preemption timer migration") which made the structure look like:

struct kvm_vmx_nested_state_hdr {
        __u64 vmxon_pa;
        __u64 vmcs12_pa;
        struct {
                __u16 flags;
        } smm;
        __u32 flags;
        __u64 preemption_timer_deadline;
};

The problem with this layout is that before both changes compilers were allocating 24 bytes for this and although smm.flags is padded to 8 bytes, it is initialized as a 2 byte value. Chances are that legacy userspaces using old layout will be passing uninitialized bytes which will slip into what is now known as 'flags'.

Quoted 8 lines
Quoted 4 lines
Fixes 850448f35aaf ("KVM: nVMX: Fix VMX preemption timer migration")
Fixes 83d31e5271ac ("KVM: nVMX: fixes for preemption timer migration")

Oops!

Whatever happened to this?

#3 lore raw
Quoted 40 lines · Jim Mattson
Quoted 39 lines · Jim Mattson
Quoted 36 lines · Vitaly Kuznetsov

Before commit 850448f35aaf ("KVM: nVMX: Fix VMX preemption timer migration") struct kvm_vmx_nested_state_hdr looked like:

struct kvm_vmx_nested_state_hdr {
        __u64 vmxon_pa;
        __u64 vmcs12_pa;
        struct {
                __u16 flags;
        } smm;
}

The ABI got broken by the above mentioned commit and an attempt to fix that was made in commit 83d31e5271ac ("KVM: nVMX: fixes for preemption timer migration") which made the structure look like:

struct kvm_vmx_nested_state_hdr {
        __u64 vmxon_pa;
        __u64 vmcs12_pa;
        struct {
                __u16 flags;
        } smm;
        __u32 flags;
        __u64 preemption_timer_deadline;
};

The problem with this layout is that before both changes compilers were allocating 24 bytes for this and although smm.flags is padded to 8 bytes, it is initialized as a 2 byte value. Chances are that legacy userspaces using old layout will be passing uninitialized bytes which will slip into what is now known as 'flags'.

Fixes 850448f35aaf ("KVM: nVMX: Fix VMX preemption timer migration")
Fixes 83d31e5271ac ("KVM: nVMX: fixes for preemption timer migration")
Quoted 5 lines
Quoted 3 lines

Oops!

Whatever happened to this?

Paolo pushed an alternative solution for 5.8, commit 5e105c88ab485 ("KVM:
nVMX: check for invalid hdr.vmx.flags"). His argument was that there was no point in adding proper padding since we already broke the ABI, i.e. damage done.

So rather than pad the struct, which doesn't magically fix the ABI for old userspace, just check for unsupported flags. That gives decent odds of failing the ioctl() for old userspace if it's passing garbage (through no fault of its own), prevents new userspace from setting unsupported flags, and allows KVM to grow the struct by conditioning consumption of new fields on an associated flag.

maintainer
#4 lore raw
Quoted 11 lines · Sean Christopherson

Paolo pushed an alternative solution for 5.8, commit 5e105c88ab485 ("KVM:
nVMX: check for invalid hdr.vmx.flags"). His argument was that there was no point in adding proper padding since we already broke the ABI, i.e. damage done.

So rather than pad the struct, which doesn't magically fix the ABI for old userspace, just check for unsupported flags. That gives decent odds of failing the ioctl() for old userspace if it's passing garbage (through no fault of its own), prevents new userspace from setting unsupported flags, and allows KVM to grow the struct by conditioning consumption of new fields on an associated flag.

In general userspace (as a hygiene/future-proofing measure) should generally zero the contents of structs before filling in some fields only. There was no guarantee that smm wouldn't grow new fields that would have occupied the padding, for example. The general solution we use is flags fields and checking them.

(The original KVM_GET/SET_NESTED_STATE patches did add a generic flags fields, but not a VMX-specific one field).

Paolo

maintainer
#5 lore raw
In reply to Sean Christopherson
Quoted 11 lines · Sean Christopherson

Paolo pushed an alternative solution for 5.8, commit 5e105c88ab485 ("KVM:
nVMX: check for invalid hdr.vmx.flags"). His argument was that there was no point in adding proper padding since we already broke the ABI, i.e. damage done.

So rather than pad the struct, which doesn't magically fix the ABI for old userspace, just check for unsupported flags. That gives decent odds of failing the ioctl() for old userspace if it's passing garbage (through no fault of its own), prevents new userspace from setting unsupported flags, and allows KVM to grow the struct by conditioning consumption of new fields on an associated flag.

In general userspace (as a hygiene/future-proofing measure) should generally zero the contents of structs before filling in some fields only. There was no guarantee that smm wouldn't grow new fields that would have occupied the padding, for example. The general solution we use is flags fields and checking them.

(The original KVM_GET/SET_NESTED_STATE patches did add a generic flags fields, but not a VMX-specific one).

Paolo