Qemu SGX virtualization
Subsystems (2)
cover Qemu SGX virtualization
Since Sean Christopherson has left Intel and i am responsible for Qemu SGX upstream work. His @intel.com address will be bouncing and his new email( [email protected]) is also in CC lists.
This series is Qemu SGX virtualization implementation rebased on latest Qemu release.
You can find Qemu repo here:
https://github.com/intel/qemu-sgx.git upstream
If you want to try sgx, you can also get the kernel/kvm code from upstream branch of kvm-sgx repo on github:
https://github.com/intel/kvm-sgx.git upstream
Notice: The kvm SGX patch series has been queued by Paolo in 4/17. This Qemu series still depend on above kvm-sgx upstream branch, so the KVM_CAP_SGX_ATTRIBUTE will be changed to 196 once the linux release merge all kvm SGX series.
To simplify, you'd better install kvm-sgx on host and guest, which can support SGX on host and guest kernel. And to me, use below reference command to boot SGX guest:
#qemu-system-x86_64 \
...... \
-cpu host,+sgx_provisionkey \
-sgx-epc id=epc1,memdev=mem1 \
-object memory-backend-epc,id=mem1,size=64M,prealloc=onOverview
========
Intel Software Guard eXtensions (SGX) is a set of instructions and mechanisms for memory accesses in order to provide security accesses for sensitive applications and data. SGX allows an application to use it's pariticular address space as an *enclave*, which is a protected area provides confidentiality and integrity even in the presence of privileged malware. Accesses to the enclave memory area from any software not resident in the enclave are prevented, including those from privileged software.
SGX virtaulization
==================
The KVM SGX creates one new misc device, sgx_vepc, and Qemu will open '/dev/sgx_vepc' device node to mmap() host EPC memory to guest. The Qemu also adds 'sgx-epc' device to expose EPC sections to guest through CPUID and ACPI table. The Qemu SGX also supports multiple virtual EPC sections to guest, we just put them together physically contiguous for the sake of simplicity. The kernel SGX NUMA has been merged into Linux tip tree, we will support this function in the next phase.
Although the current host SGX subsystem can not support SGX2 feature, the KVM/Qemu implementation still expose this feature to guest. Guest SGX2 support doesn't have interaction with host kernel SGX driver, the SGX guest can normally use those new instructions.
As for SGX virtualization detailed infomation, please reference docs/intel-sgx.txt docuement(patch 32).
Sean Christopherson (22):
memory: Add RAM_PROTECTED flag to skip IOMMU mappings
hostmem: Add hostmem-epc as a backend for SGX EPC
i386: Add 'sgx-epc' device to expose EPC sections to guest
vl: Add "sgx-epc" option to expose SGX EPC sections to guest
i386: Add primary SGX CPUID and MSR defines
i386: Add SGX CPUID leaf FEAT_SGX_12_0_EAX
i386: Add SGX CPUID leaf FEAT_SGX_12_0_EBX
i386: Add SGX CPUID leaf FEAT_SGX_12_1_EAX
i386: Add get/set/migrate support for SGX_LEPUBKEYHASH MSRs
i386: Add feature control MSR dependency when SGX is enabled
i386: Update SGX CPUID info according to hardware/KVM/user input
linux-headers: Add placeholder for KVM_CAP_SGX_ATTRIBUTE
i386: kvm: Add support for exposing PROVISIONKEY to guest
i386: Propagate SGX CPUID sub-leafs to KVM
Adjust min CPUID level to 0x12 when SGX is enabled hw/i386/fw_cfg: Set SGX bits in feature control fw_cfg accordingly hw/i386/pc: Account for SGX EPC sections when calculating device
memory
i386/pc: Add e820 entry for SGX EPC section(s)
i386: acpi: Add SGX EPC entry to ACPI tables
q35: Add support for SGX EPC
i440fx: Add support for SGX EPC
doc: Add the SGX doc
Yang Zhong (10):
qom: Add memory-backend-epc ObjectOptions support
hostmem: Add the reset interface for EPC backend reset
sgx-epc: Add the reset interface for sgx-epc virt device
qmp: Add query-sgx command
hmp: Add 'info sgx' command
i386: Add sgx_get_info() interface
bitops: Support 32 and 64 bit mask macro
qmp: Add the qmp_query_sgx_capabilities()
Kconfig: Add CONFIG_SGX support
sgx-epc: Add the fill_device_info() callback support
backends/hostmem-epc.c | 106 ++++++ backends/hostmem-memfd.c | 2 +- backends/meson.build | 1 + default-configs/devices/i386-softmmu.mak | 1 + docs/intel-sgx.txt | 173 +++++++++ hmp-commands-info.hx | 15 + hw/i386/Kconfig | 5 + hw/i386/acpi-build.c | 22 ++ hw/i386/fw_cfg.c | 10 +- hw/i386/meson.build | 1 + hw/i386/pc.c | 15 +- hw/i386/pc_piix.c | 4 + hw/i386/pc_q35.c | 3 + hw/i386/sgx-epc.c | 451 +++++++++++++++++++++++ hw/i386/sgx-stub.c | 13 + hw/misc/ivshmem.c | 2 +- hw/remote/memory.c | 2 +- hw/vfio/common.c | 1 + include/exec/memory.h | 15 + include/hw/i386/pc.h | 10 + include/hw/i386/sgx-epc.h | 70 ++++ include/monitor/hmp.h | 1 + include/qemu/bitops.h | 7 + linux-headers/linux/kvm.h | 1 + monitor/hmp-cmds.c | 32 ++ monitor/qmp-cmds.c | 19 + qapi/machine.json | 26 +- qapi/misc.json | 61 +++ qapi/qom.json | 2 + qemu-options.hx | 8 + softmmu/globals.c | 1 + softmmu/memory.c | 12 +- softmmu/physmem.c | 2 +- softmmu/vl.c | 9 + stubs/meson.build | 1 + stubs/sgx-stub.c | 12 + target/i386/cpu.c | 168 ++++++++- target/i386/cpu.h | 16 + target/i386/kvm/kvm.c | 75 ++++ target/i386/kvm/kvm_i386.h | 2 + target/i386/machine.c | 20 + tests/qtest/qmp-cmd-test.c | 2 + 42 files changed, 1387 insertions(+), 12 deletions(-) create mode 100644 backends/hostmem-epc.c create mode 100644 docs/intel-sgx.txt create mode 100644 hw/i386/sgx-epc.c create mode 100644 hw/i386/sgx-stub.c create mode 100644 include/hw/i386/sgx-epc.h create mode 100644 stubs/sgx-stub.c
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 1/32 memory: Add RAM_PROTECTED flag to skip IOMMU mappings
From: Sean Christopherson <[email protected]>
Add a new RAMBlock flag to denote "protected" memory, i.e. memory that looks and acts like RAM but is inaccessible via normal mechanisms, including DMA. Use the flag to skip protected memory regions when mapping RAM for DMA in VFIO.
--- backends/hostmem-memfd.c | 2 +- hw/misc/ivshmem.c | 2 +- hw/remote/memory.c | 2 +- hw/vfio/common.c | 1 + include/exec/memory.h | 15 +++++++++++++++ softmmu/memory.c | 12 ++++++++++-- softmmu/physmem.c | 2 +- 7 files changed, 30 insertions(+), 6 deletions(-)
backends/hostmem-memfd.c +1 -1
@@ -55,7 +55,7 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) name =host_memory_backend_get_name (backend);memory_region_init_ram_from_fd (&backend->mr,OBJECT (backend), name, backend->size, - backend->share, fd,0 , errp); + backend->share, false, fd,0 , errp);g_free (name); }
hw/misc/ivshmem.c +1 -1
@@ -494,7 +494,7 @@ static void process_msg_shmem(IVShmemState *s, int fd, Error **errp)/* mmap the region and map into the BAR2 */ memory_region_init_ram_from_fd (&s->server_bar2,OBJECT (s), -"ivshmem.bar2" , size, true, fd,0 , +"ivshmem.bar2" , size, true, false, fd,0 , &local_err);if (local_err) {error_propagate (errp, local_err);
hw/remote/memory.c +1 -1
@@ -48,7 +48,7 @@ void remote_sysmem_reconfig(MPQemuMsg *msg, Error **errp) name =g_strdup_printf ("remote-mem-%u" , suffix++);memory_region_init_ram_from_fd (subregion,NULL , name, sysmem_info->sizes[region], - true, msg->fds[region], + true, false, msg->fds[region], sysmem_info->offsets[region], errp);
hw/vfio/common.c +1 -0
@@ -538,6 +538,7 @@ static bool vfio_listener_skipped_section(MemoryRegionSection *section) {return (!memory_region_is_ram (section->mr) && !memory_region_is_iommu (section->mr)) || +memory_region_is_protected (section->mr) ||/* * Sizing an enabled 64-bit BAR can cause spurious mappings to * addresses in the upper part of the 64-bit address space. These
include/exec/memory.h +15 -0
@@ -155,6 +155,9 @@ typedef struct IOMMUTLBEvent {*/ #define RAM_UF_WRITEPROTECT (1 << 6) +/* RAM that isn't accessible through normal means. */ +#define RAM_PROTECTED (1 << 7) + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, hwaddr start, hwaddr end,
@@ -1021,6 +1024,7 @@ void memory_region_init_ram_from_file(MemoryRegion *mr,* @name: the name of the region. * @size: size of the region. * @share: %true if memory must be mmaped with the MAP_SHARED flag +* @protected: %true if memory is protected and isn't treated like normal RAM * @fd: the fd to mmap. * @offset: offset within the file referenced by fd * @errp: pointer to Error*, to store an error if it happens.
@@ -1033,6 +1037,7 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr,const char *name, uint64_t size, bool share, + bool protected, int fd, ram_addr_t offset, Error **errp);
@@ -1321,6 +1326,16 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)return mr->rom_device && mr->romd_mode; } +/** +* memory_region_is_protected: check whether a memory region is protected +* +* Returns %true if a memory region is protected RAM and cannot be accessed +* via standard mechanisms, e.g. DMA. +* +* @mr: the memory region being queried +*/ +bool memory_region_is_protected (MemoryRegion *mr); +/** * memory_region_get_iommu: check whether a memory region is an iommu *
softmmu/memory.c +10 -2
@@ -1612,18 +1612,21 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr,const char *name, uint64_t size, bool share, + bool protected, int fd, ram_addr_t offset, Error **errp) { + uint32_t ram_flags = (share ? RAM_SHARED : 0) | + (protected ?RAM_PROTECTED :0 );Error *err =NULL ;memory_region_init (mr, owner, name, size); mr->ram = true; mr->terminates = true; mr->destructor = memory_region_destructor_ram; - mr->ram_block =qemu_ram_alloc_from_fd (size, mr, - share ?RAM_SHARED :0 , + mr->ram_block =qemu_ram_alloc_from_fd (size, mr, ram_flags, fd, offset, false, &err); +if (err) { mr->size =int128_zero ();object_unparent (OBJECT (mr));
@@ -1810,6 +1813,11 @@ bool memory_region_is_ram_device(MemoryRegion *mr)return mr->ram_device; } +bool memory_region_is_protected (MemoryRegion *mr) +{ +return mr->ram && (mr->ram_block->flags &RAM_PROTECTED ); +} +uint8_t memory_region_get_dirty_log_mask (MemoryRegion *mr) {uint8_t mask = mr->dirty_log_mask;
softmmu/physmem.c +1 -1
@@ -2022,7 +2022,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,int64_t file_size, file_align;/* Just support these ram flags by now. */ -assert ((ram_flags & ~(RAM_SHARED |RAM_PMEM )) ==0 ); +assert ((ram_flags & ~(RAM_SHARED |RAM_PMEM |RAM_PROTECTED )) ==0 );if (xen_enabled ()) {error_setg (errp,"-mem-path not supported with Xen" );
Signature · 1 line
2.29.2.334.gfaefdd61ec
Re: [RESEND PATCH 01/32] memory: Add RAM_PROTECTED flag to skip IOMMU mappings
Quoted 9 lines · Yang Zhong
From: Sean Christopherson <[email protected]>
Add a new RAMBlock flag to denote "protected" memory, i.e. memory that looks and acts like RAM but is inaccessible via normal mechanisms, including DMA. Use the flag to skip protected memory regions when mapping RAM for DMA in VFIO.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Yang Zhong <[email protected]>
Can you instead unify this with the "share" argument, and pass the argument down to qemu_ram_alloc_from_fd? This removes the "share ? RAM_SHARED : 0" conversion.
Paolo
Quoted 162 lines
--- backends/hostmem-memfd.c | 2 +- hw/misc/ivshmem.c | 2 +- hw/remote/memory.c | 2 +- hw/vfio/common.c | 1 + include/exec/memory.h | 15 +++++++++++++++ softmmu/memory.c | 12 ++++++++++-- softmmu/physmem.c | 2 +- 7 files changed, 30 insertions(+), 6 deletions(-)backends/hostmem-memfd.c +1
-1@@ -55,7 +55,7 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) name =host_memory_backend_get_name (backend);memory_region_init_ram_from_fd (&backend->mr,OBJECT (backend), name, backend->size, - backend->share, fd,0 , errp); + backend->share, false, fd,0 , errp);g_free (name); }hw/misc/ivshmem.c +1
-1@@ -494,7 +494,7 @@ static void process_msg_shmem(IVShmemState *s, int fd, Error **errp)/* mmap the region and map into the BAR2 */ memory_region_init_ram_from_fd (&s->server_bar2,OBJECT (s), -"ivshmem.bar2" , size, true, fd,0 , +"ivshmem.bar2" , size, true, false, fd,0 , &local_err);if (local_err) {error_propagate (errp, local_err);hw/remote/memory.c +1
-1@@ -48,7 +48,7 @@ void remote_sysmem_reconfig(MPQemuMsg *msg, Error **errp) name =g_strdup_printf ("remote-mem-%u" , suffix++);memory_region_init_ram_from_fd (subregion,NULL , name, sysmem_info->sizes[region], - true, msg->fds[region], + true, false, msg->fds[region], sysmem_info->offsets[region], errp);hw/vfio/common.c +1
-0@@ -538,6 +538,7 @@ static bool vfio_listener_skipped_section(MemoryRegionSection *section) {return (!memory_region_is_ram (section->mr) && !memory_region_is_iommu (section->mr)) || +memory_region_is_protected (section->mr) ||/* * Sizing an enabled 64-bit BAR can cause spurious mappings to * addresses in the upper part of the 64-bit address space. These include/exec/memory.h +15
-0@@ -155,6 +155,9 @@ typedef struct IOMMUTLBEvent {*/ #define RAM_UF_WRITEPROTECT (1 << 6) +/* RAM that isn't accessible through normal means. */ +#define RAM_PROTECTED (1 << 7) + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, hwaddr start, hwaddr end,@@ -1021,6 +1024,7 @@ void memory_region_init_ram_from_file(MemoryRegion *mr,* @name: the name of the region. * @size: size of the region. * @share: %true if memory must be mmaped with the MAP_SHARED flag +* @protected: %true if memory is protected and isn't treated like normal RAM * @fd: the fd to mmap. * @offset: offset within the file referenced by fd * @errp: pointer to Error*, to store an error if it happens. @@ -1033,6 +1037,7 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr,const char *name, uint64_t size, bool share, + bool protected, int fd, ram_addr_t offset, Error **errp);@@ -1321,6 +1326,16 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)return mr->rom_device && mr->romd_mode; } +/** +* memory_region_is_protected: check whether a memory region is protected +* +* Returns %true if a memory region is protected RAM and cannot be accessed +* via standard mechanisms, e.g. DMA. +* +* @mr: the memory region being queried +*/ +bool memory_region_is_protected (MemoryRegion *mr); +/** * memory_region_get_iommu: check whether a memory region is an iommu * softmmu/memory.c +10
-2@@ -1612,18 +1612,21 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr,const char *name, uint64_t size, bool share, + bool protected, int fd, ram_addr_t offset, Error **errp) { + uint32_t ram_flags = (share ? RAM_SHARED : 0) | + (protected ?RAM_PROTECTED :0 );Error *err =NULL ;memory_region_init (mr, owner, name, size); mr->ram = true; mr->terminates = true; mr->destructor = memory_region_destructor_ram; - mr->ram_block =qemu_ram_alloc_from_fd (size, mr, - share ?RAM_SHARED :0 , + mr->ram_block =qemu_ram_alloc_from_fd (size, mr, ram_flags, fd, offset, false, &err); +if (err) { mr->size =int128_zero ();object_unparent (OBJECT (mr));@@ -1810,6 +1813,11 @@ bool memory_region_is_ram_device(MemoryRegion *mr)return mr->ram_device; } +bool memory_region_is_protected (MemoryRegion *mr) +{ +return mr->ram && (mr->ram_block->flags &RAM_PROTECTED ); +} +uint8_t memory_region_get_dirty_log_mask (MemoryRegion *mr) {uint8_t mask = mr->dirty_log_mask;softmmu/physmem.c +1
-1@@ -2022,7 +2022,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,int64_t file_size, file_align;/* Just support these ram flags by now. */ -assert ((ram_flags & ~(RAM_SHARED |RAM_PMEM )) ==0 ); +assert ((ram_flags & ~(RAM_SHARED |RAM_PMEM |RAM_PROTECTED )) ==0 );if (xen_enabled ()) {error_setg (errp,"-mem-path not supported with Xen" );
Re: [RESEND PATCH 01/32] memory: Add RAM_PROTECTED flag to skip IOMMU mappings
Quoted 8 lines · Paolo Bonzini
Quoted 7 lines · Yang Zhong
From: Sean Christopherson <[email protected]>
Add a new RAMBlock flag to denote "protected" memory, i.e. memory that looks and acts like RAM but is inaccessible via normal mechanisms, including DMA. Use the flag to skip protected memory regions when mapping RAM for DMA in VFIO.
Quoted 7 lines
Quoted 2 lines
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Yang Zhong <[email protected]>Can you instead unify this with the "share" argument, and pass the argument down to
qemu_ram_alloc_from_fd? This removes the "share ?RAM_SHARED: 0" conversion.
Paolo, i will revert this patch, but in order to disable sgx epc memory region as DMA target, we still need to skip this memory region while adding this memory region.
So, my plan is to revert this patch, and then add one new bool variable in the struct MemoryRegion.
The new patch as below and it will be as patch 2 in the next version
diff --git a/backends/hostmem-epc.c b/backends/hostmem-epc.c
index 0c98812f0d..316e1ee688 100644
--- a/backends/hostmem-epc.c
+++ b/backends/hostmem-epc.c
@@ -67,6 +67,7 @@ void sgx_memory_backend_reset(HostMemoryBackend *backend, int fd,
}sgx_epc_backend_memory_alloc(backend, errp);
+ mr->sgx_epc_device = true;
}
static void sgx_epc_backend_instance_init(Object *obj)
diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 5bc5d29358..7a53855436 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -538,7 +538,7 @@ static bool vfio_listener_skipped_section(MemoryRegionSection *section)
{
return (!memory_region_is_ram(section->mr) &&
!memory_region_is_iommu(section->mr)) ||
+ section->mr->sgx_epc_device ||
/*
* Sizing an enabled 64-bit BAR can cause spurious mappings to
* addresses in the upper part of the 64-bit address space. These
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 2816e52be3..9bba0a1590 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -463,6 +463,7 @@ struct MemoryRegion {
bool readonly; /* For RAM regions */
bool nonvolatile;
bool rom_device;
+ bool sgx_epc_device;
bool flush_coalesced_mmio;
uint8_t dirty_log_mask;
bool is_iommu;If this patch is not suitable to skip this memory region, any suggestion is appreciated, thanks!
Yang
Quoted 156 lines
Paolo
Quoted 154 lines
--- backends/hostmem-memfd.c | 2 +- hw/misc/ivshmem.c | 2 +- hw/remote/memory.c | 2 +- hw/vfio/common.c | 1 + include/exec/memory.h | 15 +++++++++++++++ softmmu/memory.c | 12 ++++++++++-- softmmu/physmem.c | 2 +- 7 files changed, 30 insertions(+), 6 deletions(-)backends/hostmem-memfd.c +1
-1@@ -55,7 +55,7 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) name =host_memory_backend_get_name (backend);memory_region_init_ram_from_fd (&backend->mr,OBJECT (backend), name, backend->size, - backend->share, fd,0 , errp); + backend->share, false, fd,0 , errp);g_free (name); }hw/misc/ivshmem.c +1
-1@@ -494,7 +494,7 @@ static void process_msg_shmem(IVShmemState *s, int fd, Error **errp)/* mmap the region and map into the BAR2 */ memory_region_init_ram_from_fd (&s->server_bar2,OBJECT (s), -"ivshmem.bar2" , size, true, fd,0 , +"ivshmem.bar2" , size, true, false, fd,0 , &local_err);if (local_err) {error_propagate (errp, local_err);hw/remote/memory.c +1
-1@@ -48,7 +48,7 @@ void remote_sysmem_reconfig(MPQemuMsg *msg, Error **errp) name =g_strdup_printf ("remote-mem-%u" , suffix++);memory_region_init_ram_from_fd (subregion,NULL , name, sysmem_info->sizes[region], - true, msg->fds[region], + true, false, msg->fds[region], sysmem_info->offsets[region], errp);hw/vfio/common.c +1
-0@@ -538,6 +538,7 @@ static bool vfio_listener_skipped_section(MemoryRegionSection *section) {return (!memory_region_is_ram (section->mr) && !memory_region_is_iommu (section->mr)) || +memory_region_is_protected (section->mr) ||/* * Sizing an enabled 64-bit BAR can cause spurious mappings to * addresses in the upper part of the 64-bit address space. These include/exec/memory.h +15
-0@@ -155,6 +155,9 @@ typedef struct IOMMUTLBEvent {*/ #define RAM_UF_WRITEPROTECT (1 << 6) +/* RAM that isn't accessible through normal means. */ +#define RAM_PROTECTED (1 << 7) + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, hwaddr start, hwaddr end,@@ -1021,6 +1024,7 @@ void memory_region_init_ram_from_file(MemoryRegion *mr,* @name: the name of the region. * @size: size of the region. * @share: %true if memory must be mmaped with the MAP_SHARED flag +* @protected: %true if memory is protected and isn't treated like normal RAM * @fd: the fd to mmap. * @offset: offset within the file referenced by fd * @errp: pointer to Error*, to store an error if it happens. @@ -1033,6 +1037,7 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr,const char *name, uint64_t size, bool share, + bool protected, int fd, ram_addr_t offset, Error **errp);@@ -1321,6 +1326,16 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)return mr->rom_device && mr->romd_mode; } +/** +* memory_region_is_protected: check whether a memory region is protected +* +* Returns %true if a memory region is protected RAM and cannot be accessed +* via standard mechanisms, e.g. DMA. +* +* @mr: the memory region being queried +*/ +bool memory_region_is_protected (MemoryRegion *mr); +/** * memory_region_get_iommu: check whether a memory region is an iommu * softmmu/memory.c +10
-2@@ -1612,18 +1612,21 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr,const char *name, uint64_t size, bool share, + bool protected, int fd, ram_addr_t offset, Error **errp) { + uint32_t ram_flags = (share ? RAM_SHARED : 0) | + (protected ?RAM_PROTECTED :0 );Error *err =NULL ;memory_region_init (mr, owner, name, size); mr->ram = true; mr->terminates = true; mr->destructor = memory_region_destructor_ram; - mr->ram_block =qemu_ram_alloc_from_fd (size, mr, - share ?RAM_SHARED :0 , + mr->ram_block =qemu_ram_alloc_from_fd (size, mr, ram_flags, fd, offset, false, &err); +if (err) { mr->size =int128_zero ();object_unparent (OBJECT (mr));@@ -1810,6 +1813,11 @@ bool memory_region_is_ram_device(MemoryRegion *mr)return mr->ram_device; } +bool memory_region_is_protected (MemoryRegion *mr) +{ +return mr->ram && (mr->ram_block->flags &RAM_PROTECTED ); +} +uint8_t memory_region_get_dirty_log_mask (MemoryRegion *mr) {uint8_t mask = mr->dirty_log_mask;softmmu/physmem.c +1
-1@@ -2022,7 +2022,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,int64_t file_size, file_align;/* Just support these ram flags by now. */ -assert ((ram_flags & ~(RAM_SHARED |RAM_PMEM )) ==0 ); +assert ((ram_flags & ~(RAM_SHARED |RAM_PMEM |RAM_PROTECTED )) ==0 );if (xen_enabled ()) {error_setg (errp,"-mem-path not supported with Xen" );
Re: [RESEND PATCH 01/32] memory: Add RAM_PROTECTED flag to skip IOMMU mappings
Quoted 40 lines · Yang Zhong
Paolo, i will revert this patch, but in order to disable sgx epc memory region
as DMA target, we still need to skip this memory region while adding this memory
region.
So, my plan is to revert this patch, and then add one new bool variable in the
struct MemoryRegion.
The new patch as below and it will be as patch 2 in the next version
diff --git a/backends/hostmem-epc.c b/backends/hostmem-epc.c
index 0c98812f0d..316e1ee688 100644
--- a/backends/hostmem-epc.c
+++ b/backends/hostmem-epc.c
@@ -67,6 +67,7 @@ void sgx_memory_backend_reset(HostMemoryBackend *backend, int fd,
}
sgx_epc_backend_memory_alloc(backend, errp);
+ mr->sgx_epc_device = true;
}
static void sgx_epc_backend_instance_init(Object *obj)
diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 5bc5d29358..7a53855436 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -538,7 +538,7 @@ static bool vfio_listener_skipped_section(MemoryRegionSection *section)
{
return (!memory_region_is_ram(section->mr) &&
!memory_region_is_iommu(section->mr)) ||
+ section->mr->sgx_epc_device ||
/*
* Sizing an enabled 64-bit BAR can cause spurious mappings to
* addresses in the upper part of the 64-bit address space. These
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 2816e52be3..9bba0a1590 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -463,6 +463,7 @@ struct MemoryRegion {
bool readonly; /* For RAM regions */
bool nonvolatile;Quoted 8 lines
bool rom_device;
+ bool sgx_epc_device;
bool flush_coalesced_mmio;
uint8_t dirty_log_mask;
bool is_iommu;
If this patch is not suitable to skip this memory region, any suggestion is
appreciated, thanks!No, you can keep RAM_PROTECTED. I was just remarking on the new bool argument, and suggesting passing the flags (RAM_SHARED and RAM_PROTECTED) directly tot he memory_region_init* functions.
Paolo
Re: [RESEND PATCH 01/32] memory: Add RAM_PROTECTED flag to skip IOMMU mappings
Quoted 49 lines · Paolo Bonzini
Quoted 45 lines · Yang Zhong
Paolo, i will revert this patch, but in order to disable sgx epc memory region as DMA target, we still need to skip this memory region while adding this memory region.
So, my plan is to revert this patch, and then add one new bool variable in the struct MemoryRegion.
The new patch as below and it will be as patch 2 in the next version
diff --git a/backends/hostmem-epc.c b/backends/hostmem-epc.c index 0c98812f0d..316e1ee688 100644 --- a/backends/hostmem-epc.c +++ b/backends/hostmem-epc.c @@ -67,6 +67,7 @@ void sgx_memory_backend_reset(HostMemoryBackend *backend, int fd, }sgx_epc_backend_memory_alloc(backend, errp);+
mr->sgx_epc_device= true;
}static void
sgx_epc_backend_instance_init(Object *obj)
diff --gita/hw/vfio/common.cb/hw/vfio/common.c
index5bc5d29358..7a53855436100644
---a/hw/vfio/common.c
+++b/hw/vfio/common.c
@@ -538,7 +538,7 @@ static boolvfio_listener_skipped_section(MemoryRegionSection *section){ return (!memory_region_is_ram(section->mr) && !memory_region_is_iommu(section->mr)) || + section->mr->sgx_epc_device || /* * Sizing an enabled 64-bit BAR can cause spurious mappings to * addresses in the upper part of the 64-bit address space. These diff --git a/include/exec/memory.h b/include/exec/memory.h index 2816e52be3..9bba0a1590 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -463,6 +463,7 @@ struct MemoryRegion { bool readonly; /* For RAM regions */ bool nonvolatile; bool rom_device; + bool sgx_epc_device; bool flush_coalesced_mmio; uint8_t dirty_log_mask; bool is_iommu;Quoted 2 lines
If this patch is not suitable to skip this memory region, any suggestion is appreciated, thanks!
Quoted 4 lines
No, you can keep
RAM_PROTECTED. I was just remarking on the new bool argument, and suggesting passing the flags (RAM_SHAREDandRAM_PROTECTED) directly tot he memory_region_init* functions.
Paolo, thanks! If there is no new comments from community, i will send the V2 in the next week. thanks!
Yang
Quoted 1 line
Paolo
PATCH RESEND 2/32 hostmem: Add hostmem-epc as a backend for SGX EPC
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
EPC (Enclave Page Cahe) is a specialized type of memory used by Intel SGX (Software Guard Extensions). The SDM desribes EPC as:
The Enclave Page Cache (EPC) is the secure storage used to store enclave pages when they are a part of an executing enclave. For an EPC page, hardware performs additional access control checks to restrict access to the page. After the current page access checks and translations are performed, the hardware checks that the EPC page is accessible to the program currently executing. Generally an EPC page is only accessed by the owner of the executing enclave or an instruction which is setting up an EPC page.
Because of its unique requirements, Linux manages EPC separately from normal memory. Similar to memfd, the device /dev/sgx_vepc can be opened to obtain a file descriptor which can in turn be used to mmap() EPC memory.
--- backends/hostmem-epc.c | 90 ++++++++++++++++++++++++++++++++++++++++++ backends/meson.build | 1 + 2 files changed, 91 insertions(+) create mode 100644 backends/hostmem-epc.c
backends/hostmem-epc.c new file +90 -0
@@ -0,0 +1,90 @@ +/* +* QEMU host SGX EPC memory backend +* +* Copyright (C) 2019 Intel Corporation +* +* Authors: +* Sean Christopherson <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +*/ +#include <sys/ioctl.h> + +#include "qemu/osdep.h" +#include "qemu-common.h" +#include "qom/object_interfaces.h" +#include "qapi/error.h" +#include "sysemu/hostmem.h" + +#define TYPE_MEMORY_BACKEND_EPC "memory-backend-epc" + +#define MEMORY_BACKEND_EPC (obj) \ + OBJECT_CHECK(HostMemoryBackendEpc, (obj), TYPE_MEMORY_BACKEND_EPC) + +typedef struct HostMemoryBackendEpc HostMemoryBackendEpc ; + +struct HostMemoryBackendEpc { +HostMemoryBackend parent_obj; +}; + +static void +sgx_epc_backend_memory_alloc (HostMemoryBackend *backend,Error **errp) +{ +char *name; +int fd; + +if (!backend->size) { +error_setg (errp,"can't create backend with size 0" ); +return ; + } + + fd =qemu_open_old ("/dev/sgx_vepc" ,O_RDWR ); +if (fd <0 ) { +error_setg_errno (errp, errno, +"failed to open /dev/sgx_vepc to alloc SGX EPC" ); +return ; + } + + name =object_get_canonical_path (OBJECT (backend)); +memory_region_init_ram_from_fd (&backend->mr,OBJECT (backend), + name, backend->size, backend->share, true, + fd,0 , errp); +g_free (name); +} + +static void sgx_epc_backend_instance_init (Object *obj) +{ +HostMemoryBackend *m =MEMORY_BACKEND (obj); + + m->share = true; + m->merge = false; + m->dump = false; +} + +static void sgx_epc_backend_class_init (ObjectClass *oc,void *data) +{ +HostMemoryBackendClass *bc =MEMORY_BACKEND_CLASS (oc); + + bc->alloc = sgx_epc_backend_memory_alloc; +} + +static const TypeInfo sgx_epc_backed_info = { + .name =TYPE_MEMORY_BACKEND_EPC , + .parent =TYPE_MEMORY_BACKEND , + .instance_init = sgx_epc_backend_instance_init, + .class_init = sgx_epc_backend_class_init, + .instance_size =sizeof (HostMemoryBackendEpc), +}; + +static void register_types (void ) +{ +int fd =qemu_open_old ("/dev/sgx_vepc" ,O_RDWR ); +if (fd >=0 ) { +close (fd); + +type_register_static (&sgx_epc_backed_info); + } +} + +type_init (register_types);
backends/meson.build +1 -0
@@ -16,5 +16,6 @@ softmmu_ss.add(when: ['CONFIG_VHOST_USER', 'CONFIG_VIRTIO'], if_true: files('vho softmmu_ss.add(when: 'CONFIG_VIRTIO_CRYPTO', if_true: files('cryptodev-vhost.c')) softmmu_ss.add(when: ['CONFIG_VIRTIO_CRYPTO', 'CONFIG_VHOST_CRYPTO'], if_true: files('cryptodev-vhost-user.c')) softmmu_ss.add(when: 'CONFIG_GIO', if_true: [files('dbus-vmstate.c'), gio]) +softmmu_ss.add(when: 'CONFIG_LINUX', if_true: files('hostmem-epc.c')) subdir('tpm')
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 3/32 qom: Add memory-backend-epc ObjectOptions support
In reply to Yang ZhongAdd the new 'memory-backend-epc' user creatable QOM object in the ObjectOptions to support SGX, or the sgx backend object cannot bootup.
--- qapi/qom.json | 2 ++ 1 file changed, 2 insertions(+)
qapi/qom.json +2 -0
@@ -767,6 +767,7 @@ { 'name': 'memory-backend-memfd', 'if': 'defined(CONFIG_LINUX)' }, 'memory-backend-ram', + 'memory-backend-epc', 'pef-guest', 'pr-manager-helper', 'rng-builtin',
@@ -824,6 +825,7 @@ 'memory-backend-memfd': { 'type': 'MemoryBackendMemfdProperties', 'if': 'defined(CONFIG_LINUX)' }, 'memory-backend-ram': 'MemoryBackendProperties', + 'memory-backend-epc': 'MemoryBackendProperties', 'pr-manager-helper': 'PrManagerHelperProperties', 'rng-builtin': 'RngProperties', 'rng-egd': 'RngEgdProperties',
Signature · 1 line
2.29.2.334.gfaefdd61ec
Re: [RESEND PATCH 03/32] qom: Add memory-backend-epc ObjectOptions support
Quoted 8 lines · Yang Zhong
Add the new 'memory-backend-epc' user creatable QOM object in the ObjectOptions to support SGX, or the sgx backend object cannot bootup.
Signed-off-by Yang Zhong <[email protected]>--- qapi/qom.json | 2 ++ 1 file changed, 2 insertions(+)
Missing documentation, including a 'since 6.1' blurb, for the new addition.
Quoted 22 lines
qapi/qom.json +2
-0@@ -767,6 +767,7 @@ { 'name': 'memory-backend-memfd', 'if': 'defined(CONFIG_LINUX)' }, 'memory-backend-ram', + 'memory-backend-epc', 'pef-guest', 'pr-manager-helper', 'rng-builtin',@@ -824,6 +825,7 @@ 'memory-backend-memfd': { 'type': 'MemoryBackendMemfdProperties', 'if': 'defined(CONFIG_LINUX)' }, 'memory-backend-ram': 'MemoryBackendProperties', + 'memory-backend-epc': 'MemoryBackendProperties', 'pr-manager-helper': 'PrManagerHelperProperties', 'rng-builtin': 'RngProperties', 'rng-egd': 'RngEgdProperties',
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [RESEND PATCH 03/32] qom: Add memory-backend-epc ObjectOptions support
Quoted 5 lines · Eric Blake
Quoted 4 lines · Yang Zhong
Add the new 'memory-backend-epc' user creatable QOM object in the ObjectOptions to support SGX, or the sgx backend object cannot bootup.
Quoted 7 lines
Quoted 4 lines
Signed-off-by Yang Zhong <[email protected]>--- qapi/qom.json | 2 ++ 1 file changed, 2 insertions(+)Missing documentation, including a 'since 6.1' blurb, for the new addition.
Eric, thanks for your comments! i will add new MemoryBackendEpcProperties properties and documents for sgx epc. thanks!
Yang
Quoted 27 lines
Quoted 22 lines
qapi/qom.json +2
-0@@ -767,6 +767,7 @@ { 'name': 'memory-backend-memfd', 'if': 'defined(CONFIG_LINUX)' }, 'memory-backend-ram', + 'memory-backend-epc', 'pef-guest', 'pr-manager-helper', 'rng-builtin',@@ -824,6 +825,7 @@ 'memory-backend-memfd': { 'type': 'MemoryBackendMemfdProperties', 'if': 'defined(CONFIG_LINUX)' }, 'memory-backend-ram': 'MemoryBackendProperties', + 'memory-backend-epc': 'MemoryBackendProperties', 'pr-manager-helper': 'PrManagerHelperProperties', 'rng-builtin': 'RngProperties', 'rng-egd': 'RngEgdProperties',Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH RESEND 4/32 i386: Add 'sgx-epc' device to expose EPC sections to guest
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
SGX EPC is enumerated through CPUID, i.e. EPC "devices" need to be realized prior to realizing the vCPUs themselves, which occurs long before generic devices are parsed and realized. Because of this, do not allow 'sgx-epc' devices to be instantiated after vCPUS have been created.
The 'sgx-epc' device is essentially a placholder at this time, it will be fully implemented in a future patch along with a dedicated command to create 'sgx-epc' devices.
--- hw/i386/meson.build | 1 + hw/i386/sgx-epc.c | 161 ++++++++++++++++++++++++++++++++++++++ include/hw/i386/sgx-epc.h | 44 +++++++++++ 3 files changed, 206 insertions(+) create mode 100644 hw/i386/sgx-epc.c create mode 100644 include/hw/i386/sgx-epc.h
hw/i386/meson.build +1 -0
@@ -5,6 +5,7 @@ i386_ss.add(files( 'e820_memory_layout.c', 'multiboot.c', 'x86.c', + 'sgx-epc.c', )) i386_ss.add(when: 'CONFIG_X86_IOMMU', if_true: files('x86-iommu.c'),
hw/i386/sgx-epc.c new file +161 -0
161 lines not on this page · raw message
include/hw/i386/sgx-epc.h new file +44 -0
44 lines not on this page · raw message
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 5/32 vl: Add "sgx-epc" option to expose SGX EPC sections to guest
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
Because SGX EPC is enumerated through CPUID, EPC "devices" need to be realized prior to realizing the vCPUs themselves, i.e. long before generic devices are parsed and realized. From a virtualization perspective, the CPUID aspect also means that EPC sections cannot be hotplugged without paravirtualizing the guest kernel (hardware does not support hotplugging as EPC sections must be locked down during pre-boot to provide EPC's security properties).
So even though EPC sections could be realized through the generic -devices command, they need to be created much earlier for them to actually be usable by the guest. Place all EPC sections in a contiguous block, somewhat arbitrarily starting after RAM above 4g. Ensuring EPC is in a contiguous region simplifies calculations, e.g. device memory base, PCI hole, etc..., allows dynamic calculation of the total EPC size, e.g. exposing EPC to guests does not require -maxmem, and last but not least allows all of EPC to be enumerated in a single ACPI entry, which is expected by some kernels, e.g. Windows 7 and 8.
--- hw/i386/sgx-epc.c | 104 +++++++++++++++++++++++++++++++++++++- include/hw/i386/pc.h | 6 +++ include/hw/i386/sgx-epc.h | 16 ++++++ qemu-options.hx | 8 +++ softmmu/globals.c | 1 + softmmu/vl.c | 9 ++++ 6 files changed, 143 insertions(+), 1 deletion(-)
hw/i386/sgx-epc.c +103 -1
@@ -56,6 +56,8 @@ static void sgx_epc_realize(DeviceState *dev, Error **errp) {PCMachineState *pcms =PC_MACHINE (qdev_get_machine ());X86MachineState *x86ms =X86_MACHINE (pcms); +MemoryDeviceState *md =MEMORY_DEVICE (dev); +SGXEPCState *sgx_epc = pcms->sgx_epc;SGXEPCDevice *epc =SGX_EPC (dev);const char *path;
@@ -74,7 +76,18 @@ static void sgx_epc_realize(DeviceState *dev, Error **errp)return ; } -error_setg (errp,"'" TYPE_SGX_EPC "' not supported" ); + epc->addr = sgx_epc->base + sgx_epc->size; + +memory_region_add_subregion (&sgx_epc->mr, epc->addr - sgx_epc->base, +host_memory_backend_get_memory (epc->hostmem)); + +host_memory_backend_set_mapped (epc->hostmem, true); + + sgx_epc->sections = g_renew(SGXEPCDevice *, sgx_epc->sections, + sgx_epc->nr_sections +1 ); + sgx_epc->sections[sgx_epc->nr_sections++] = epc; + + sgx_epc->size +=memory_device_get_region_size (md, errp); }static void sgx_epc_unrealize (DeviceState *dev)
@@ -159,3 +172,92 @@ static void sgx_epc_register_types(void) }type_init (sgx_epc_register_types) + + +static int sgx_epc_set_property(void *opaque, const char *name, +const char *value,Error **errp) +{ +Object *obj = opaque; +Error *err =NULL ; + +object_property_parse (obj, name, value, &err); +if (err !=NULL ) { +error_propagate (errp, err); +return -1 ; + } +return 0 ; +} + +static int sgx_epc_init_func (void *opaque,QemuOpts *opts,Error **errp) +{ +Error *err =NULL ; +Object *obj; + + obj =object_new ("sgx-epc" ); + +qdev_set_id (DEVICE (obj),qemu_opts_id (opts)); + +if (qemu_opt_foreach (opts, sgx_epc_set_property, obj, &err)) { + goto out; + } + +object_property_set_bool (obj,"realized" , true, &err); + +out: +if (err !=NULL ) { +error_propagate (errp, err); + } +object_unref (obj); +return err !=NULL ?-1 :0 ; +} + +void pc_machine_init_sgx_epc (PCMachineState *pcms) +{ +SGXEPCState *sgx_epc; +X86MachineState *x86ms =X86_MACHINE (pcms); + + sgx_epc =g_malloc0 (sizeof (*sgx_epc)); + pcms->sgx_epc = sgx_epc; + + sgx_epc->base =0x100000000ULL + x86ms->above_4g_mem_size; + +memory_region_init (&sgx_epc->mr,OBJECT (pcms),"sgx-epc" ,UINT64_MAX ); +memory_region_add_subregion (get_system_memory (), sgx_epc->base, + &sgx_epc->mr); + +qemu_opts_foreach (qemu_find_opts ("sgx-epc" ), sgx_epc_init_func,NULL , + &error_fatal); + +if ((sgx_epc->base + sgx_epc->size) < sgx_epc->base) { +error_report ("Size of all 'sgx-epc' =0x%" PRIu64" causes EPC to wrap" , + sgx_epc->size); +exit (EXIT_FAILURE ); + } + +memory_region_set_size (&sgx_epc->mr, sgx_epc->size); +} + +static QemuOptsList sgx_epc_opts = { + .name ="sgx-epc" , + .implied_opt_name ="id" , + .head =QTAILQ_HEAD_INITIALIZER (sgx_epc_opts.head), + .desc = { + { + .name ="id" , + .type =QEMU_OPT_STRING , + .help ="SGX EPC section ID" , + },{ + .name ="memdev" , + .type =QEMU_OPT_STRING , + .help ="memory object backend" , + }, + {/* end of list */ } + }, +}; + +static void sgx_epc_register_opts (void ) +{ +qemu_add_opts (&sgx_epc_opts); +} + +opts_init (sgx_epc_register_opts);
include/hw/i386/pc.h +6 -0
@@ -12,6 +12,7 @@#include "hw/acpi/acpi_dev_interface.h" #include "hw/hotplug.h" #include "qom/object.h" +#include "hw/i386/sgx-epc.h" #define HPET_INTCAP "hpet-intcap"
@@ -53,6 +54,8 @@ typedef struct PCMachineState {/* ACPI Memory hotplug IO base address */ hwaddr memhp_io_base; + +SGXEPCState *sgx_epc; } PCMachineState;#define PC_MACHINE_ACPI_DEVICE_PROP "acpi-device"
@@ -197,6 +200,9 @@ bool pc_system_ovmf_table_find(const char *entry, uint8_t **data,void pc_madt_cpu_entry (AcpiDeviceIf *adev,int uid,const CPUArchIdList *apic_ids,GArray *entry); +/* sgx-epc.c */ +void pc_machine_init_sgx_epc (PCMachineState *pcms); +extern GlobalProperty pc_compat_5_2[];extern const size_t pc_compat_5_2_len;
include/hw/i386/sgx-epc.h +16 -0
@@ -41,4 +41,20 @@ typedef struct SGXEPCDevice { HostMemoryBackend *hostmem; } SGXEPCDevice; +/* +* @base: address in guest physical address space where EPC regions start +* @mr: address space container for memory devices +*/ +typedef struct SGXEPCState { +uint64_t base; +uint64_t size; + +MemoryRegion mr; + +struct SGXEPCDevice **sections; +int nr_sections; +}SGXEPCState ; + +extern int sgx_epc_enabled; +#endif
qemu-options.hx +8 -0
@@ -532,6 +532,14 @@ SRST Preallocate memory when using -mem-path. ERST +DEF("sgx-epc", HAS_ARG, QEMU_OPTION_sgx_epc, + "-sgx-epc memdev=memid[,id=epcid]\n", + QEMU_ARCH_I386) +SRST +``-sgx-epc memdev=@var{memid}[,id=@var{epcid}]`` + Define an SGX EPC section. +ERST + DEF("k", HAS_ARG, QEMU_OPTION_k, "-k language use keyboard layout (for example 'fr' for French)\n", QEMU_ARCH_ALL)
softmmu/globals.c +1 -0
@@ -70,3 +70,4 @@ bool qemu_uuid_set;uint32_t xen_domid;enum xen_mode xen_mode =XEN_EMULATE ;bool xen_domid_restrict; +int sgx_epc_enabled;
softmmu/vl.c +9 -0
@@ -74,6 +74,7 @@#include "hw/block/block.h" #include "hw/i386/x86.h" #include "hw/i386/pc.h" +#include "hw/i386/sgx-epc.h" #include "migration/misc.h" #include "migration/snapshot.h" #include "sysemu/tpm.h"
@@ -2891,6 +2892,14 @@ void qemu_init(int argc, char **argv, char **envp)case QEMU_OPTION_mem_prealloc: mem_prealloc =1 ;break ; +case QEMU_OPTION_sgx_epc: + opts =qemu_opts_parse_noisily (qemu_find_opts ("sgx-epc" ), + optarg, false); +if (!opts) { +exit (1 ); + } + sgx_epc_enabled =1 ; +break ;case QEMU_OPTION_d: log_mask = optarg;break ;
Signature · 1 line
2.29.2.334.gfaefdd61ec
Re: [RESEND PATCH 05/32] vl: Add "sgx-epc" option to expose SGX EPC sections to guest
Quoted 11 lines · Yang Zhong
hw/i386/sgx-epc.c +11
-0+void pc_machine_init_sgx_epc (PCMachineState *pcms) +{ +SGXEPCState *sgx_epc; +X86MachineState *x86ms =X86_MACHINE (pcms); + + sgx_epc =g_malloc0 (sizeof (*sgx_epc)); + pcms->sgx_epc = sgx_epc; + + sgx_epc->base =0x100000000ULL + x86ms->above_4g_mem_size; + +memory_region_init (&sgx_epc->mr,OBJECT (pcms),"sgx-epc" ,UINT64_MAX );
hw/i386/sgx-epc.c static void sgx_epc_register_types(void) · line 234
Quoted 10 lines
hw/i386/sgx-epc.c +10
-0+memory_region_add_subregion (get_system_memory (), sgx_epc->base, + &sgx_epc->mr); + +qemu_opts_foreach (qemu_find_opts ("sgx-epc" ), sgx_epc_init_func,NULL , + &error_fatal); + +if ((sgx_epc->base + sgx_epc->size) < sgx_epc->base) { +error_report ("Size of all 'sgx-epc' =0x%" PRIu64" causes EPC to wrap" , + sgx_epc->size); +exit (EXIT_FAILURE );
Or perhaps even drop completely the options and just do "-device sgx-epc,backend=epc1"?
Paolo
Re: [RESEND PATCH 05/32] vl: Add "sgx-epc" option to expose SGX EPC sections to guest
Quoted 17 lines · Paolo Bonzini
Quoted 16 lines · Yang Zhong
hw/i386/sgx-epc.c +16
-0+void pc_machine_init_sgx_epc (PCMachineState *pcms) +{ +SGXEPCState *sgx_epc; +X86MachineState *x86ms =X86_MACHINE (pcms); + + sgx_epc =g_malloc0 (sizeof (*sgx_epc)); + pcms->sgx_epc = sgx_epc; + + sgx_epc->base =0x100000000ULL + x86ms->above_4g_mem_size; + +memory_region_init (&sgx_epc->mr,OBJECT (pcms),"sgx-epc" ,UINT64_MAX ); +memory_region_add_subregion (get_system_memory (), sgx_epc->base, + &sgx_epc->mr); + +qemu_opts_foreach (qemu_find_opts ("sgx-epc" ), sgx_epc_init_func,NULL , + &error_fatal);
Quoted 8 lines
hw/i386/sgx-epc.c static void sgx_epc_register_types(void) · line 234
Quoted 5 lines
hw/i386/sgx-epc.c +5
-0+ +if ((sgx_epc->base + sgx_epc->size) < sgx_epc->base) { +error_report ("Size of all 'sgx-epc' =0x%" PRIu64" causes EPC to wrap" , + sgx_epc->size); +exit (EXIT_FAILURE );Or perhaps even drop completely the options and just do "-device sgx-epc,backend=epc1"?
Is there a way to process "-device sgx-epc..." before vCPUs are realized? The ordering problem was the only reason I added a dedicated option.
From the changelog:
Because SGX EPC is enumerated through CPUID, EPC "devices" need to be realized prior to realizing the vCPUs themselves, i.e. long before generic devices are parsed and realized.
So even though EPC sections could be realized through the generic -devices command, they need to be created much earlier for them to actually be usable by the guest.
Re: [RESEND PATCH 05/32] vl: Add "sgx-epc" option to expose SGX EPC sections to guest
Quoted 2 lines · Sean Christopherson
Is there a way to process "-device sgx-epc..." before vCPUs are realized? The ordering problem was the only reason I added a dedicated option.
If it's just CPUID, one possibility could be to mark the EPC sections specially in KVM_SET_USER_MEMORY_REGION and synthesize the leaves within KVM; or even look inside the VMA structs and detect EPC regions that way.
Otherwise, the -M solution would work.
Paolo
Quoted 9 lines
From the changelog:
Because SGX EPC is enumerated through CPUID, EPC "devices" need to be realized prior to realizing the vCPUs themselves, i.e. long before generic devices are parsed and realized.
So even though EPC sections could be realized through the generic -devices command, they need to be created much earlier for them to actually be usable by the guest.
Re: [RESEND PATCH 05/32] vl: Add "sgx-epc" option to expose SGX EPC sections to guest
Quoted 7 lines · Paolo Bonzini
Quoted 2 lines · Sean Christopherson
Is there a way to process "-device sgx-epc..." before vCPUs are realized? The ordering problem was the only reason I added a dedicated option.
If it's just CPUID, one possibility could be to mark the EPC sections specially in
KVM_SET_USER_MEMORY_REGIONand synthesize the leaves within KVM; or even look inside the VMA structs and detect EPC regions that way.
I experimented with those options, and a few others, and they all lack the flexibility of making EPC just another memory backend.
For synthesizing CPUID within KVM:
- Requires a vendor specific memory region flag for all architectures to work around a quirk of one userspace VMM.
- Pushes a lot of complexity into KVM, e.g. KVM needs to update CPUID in response to memslot changes, and needs to query memslots in response to CPUID changes.
- Does KVM or userspace define the section attributes, e.g. confidentiality, integrity, etc...? If KVM, are they hardcoded to match the host? What happens if a future Intel platform supports multiple EPC sections with different attributes? If userspace, how does userspace communicate the attributes?
- How does userspace know what KVM enumerated to the guest? See the whole KVM_GET_CPUID2 fiasco...
- Prevents userspace from enumerating EPC without a memslot, e.g. to trap on the first EPC access for tracking purposes.
For probing VMAs:
- In addition to the above issues, requires MMU notifier integration to update CPUID in response to a VMA change.
- Requires SGX subsystem to provide a helper to identify EPC VMAs.
In short, I feel very strongly that this is QEMU's problem to solve.
Quoted 14 lines
Otherwise, the -M solution would work.
Paolo
Quoted 9 lines
From the changelog:
Because SGX EPC is enumerated through CPUID, EPC "devices" need to be realized prior to realizing the vCPUs themselves, i.e. long before generic devices are parsed and realized.
So even though EPC sections could be realized through the generic -devices command, they need to be created much earlier for them to actually be usable by the guest.
Re: [RESEND PATCH 05/32] vl: Add "sgx-epc" option to expose SGX EPC sections to guest
Quoted 23 lines · Sean Christopherson
Quoted 3 lines
If it's just CPUID, one possibility could be to mark the EPC sections specially in
KVM_SET_USER_MEMORY_REGIONand synthesize the leaves within KVM; or even look inside the VMA structs and detect EPC regions that way.I experimented with those options, and a few others, and they all lack the flexibility of making EPC just another memory backend.
For synthesizing CPUID within KVM:
- Requires a vendor specific memory region flag for all architectures to work around a quirk of one userspace VMM.
- Pushes a lot of complexity into KVM, e.g. KVM needs to update CPUID in response to memslot changes, and needs to query memslots in response to CPUID changes.
- Does KVM or userspace define the section attributes, e.g. confidentiality, integrity, etc...? If KVM, are they hardcoded to match the host? What happens if a future Intel platform supports multiple EPC sections with different attributes? If userspace, how does userspace communicate the attributes?
- How does userspace know what KVM enumerated to the guest? See the wholeKVM_GET_CPUID2fiasco...
- Prevents userspace from enumerating EPC without a memslot, e.g. to trap on the first EPC access for tracking purposes.
Quoted 7 lines
For probing VMAs:
- In addition to the above issues, requires MMU notifier integration to update CPUID in response to a VMA change.
- Requires SGX subsystem to provide a helper to identify EPC VMAs.In short, I feel very strongly that this is QEMU's problem to solve.
Makes sense, thanks. (Of course this or other design comments could have been in the commit message too; but in all fairness it's never obvious which misguided ideas the reviewers could come up with).
Paolo
Re: [RESEND PATCH 05/32] vl: Add "sgx-epc" option to expose SGX EPC sections to guest
In reply to Yang ZhongQuoted 20 lines · Yang Zhong
From: Sean Christopherson <[email protected]>
Because SGX EPC is enumerated through CPUID, EPC "devices" need to be realized prior to realizing the vCPUs themselves, i.e. long before generic devices are parsed and realized. From a virtualization perspective, the CPUID aspect also means that EPC sections cannot be hotplugged without paravirtualizing the guest kernel (hardware does not support hotplugging as EPC sections must be locked down during pre-boot to provide EPC's security properties).
So even though EPC sections could be realized through the generic -devices command, they need to be created much earlier for them to actually be usable by the guest. Place all EPC sections in a contiguous block, somewhat arbitrarily starting after RAM above 4g. Ensuring EPC is in a contiguous region simplifies calculations, e.g. device memory base, PCI hole, etc..., allows dynamic calculation of the total EPC size, e.g. exposing EPC to guests does not require -maxmem, and last but not least allows all of EPC to be enumerated in a single ACPI entry, which is expected by some kernels, e.g. Windows 7 and 8.
Quoted 2 lines
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Yang Zhong <[email protected]>
This will have to change, since we don't really want to add more command line options. It will become an "array of strings" machine property, like
-M sgx-epc.0=epc1,sgx-epc.1=epc2
I was already planning support for this kind of compound property in -machine and -accel, so I'll clean up the patches and send them (but it will take a couple of weeks).
Paolo
Quoted 247 lines
--- hw/i386/sgx-epc.c | 104 +++++++++++++++++++++++++++++++++++++- include/hw/i386/pc.h | 6 +++ include/hw/i386/sgx-epc.h | 16 ++++++ qemu-options.hx | 8 +++ softmmu/globals.c | 1 + softmmu/vl.c | 9 ++++ 6 files changed, 143 insertions(+), 1 deletion(-)hw/i386/sgx-epc.c +103
-1@@ -56,6 +56,8 @@ static void sgx_epc_realize(DeviceState *dev, Error **errp) {PCMachineState *pcms =PC_MACHINE (qdev_get_machine ());X86MachineState *x86ms =X86_MACHINE (pcms); +MemoryDeviceState *md =MEMORY_DEVICE (dev); +SGXEPCState *sgx_epc = pcms->sgx_epc;SGXEPCDevice *epc =SGX_EPC (dev);const char *path;@@ -74,7 +76,18 @@ static void sgx_epc_realize(DeviceState *dev, Error **errp)return ; } -error_setg (errp,"'" TYPE_SGX_EPC "' not supported" ); + epc->addr = sgx_epc->base + sgx_epc->size; + +memory_region_add_subregion (&sgx_epc->mr, epc->addr - sgx_epc->base, +host_memory_backend_get_memory (epc->hostmem)); + +host_memory_backend_set_mapped (epc->hostmem, true); + + sgx_epc->sections = g_renew(SGXEPCDevice *, sgx_epc->sections, + sgx_epc->nr_sections +1 ); + sgx_epc->sections[sgx_epc->nr_sections++] = epc; + + sgx_epc->size +=memory_device_get_region_size (md, errp); }static void sgx_epc_unrealize (DeviceState *dev)@@ -159,3 +172,92 @@ static void sgx_epc_register_types(void) }type_init (sgx_epc_register_types) + + +static int sgx_epc_set_property(void *opaque, const char *name, +const char *value,Error **errp) +{ +Object *obj = opaque; +Error *err =NULL ; + +object_property_parse (obj, name, value, &err); +if (err !=NULL ) { +error_propagate (errp, err); +return -1 ; + } +return 0 ; +} + +static int sgx_epc_init_func (void *opaque,QemuOpts *opts,Error **errp) +{ +Error *err =NULL ; +Object *obj; + + obj =object_new ("sgx-epc" ); + +qdev_set_id (DEVICE (obj),qemu_opts_id (opts)); + +if (qemu_opt_foreach (opts, sgx_epc_set_property, obj, &err)) { + goto out; + } + +object_property_set_bool (obj,"realized" , true, &err); + +out: +if (err !=NULL ) { +error_propagate (errp, err); + } +object_unref (obj); +return err !=NULL ?-1 :0 ; +} + +void pc_machine_init_sgx_epc (PCMachineState *pcms) +{ +SGXEPCState *sgx_epc; +X86MachineState *x86ms =X86_MACHINE (pcms); + + sgx_epc =g_malloc0 (sizeof (*sgx_epc)); + pcms->sgx_epc = sgx_epc; + + sgx_epc->base =0x100000000ULL + x86ms->above_4g_mem_size; + +memory_region_init (&sgx_epc->mr,OBJECT (pcms),"sgx-epc" ,UINT64_MAX ); +memory_region_add_subregion (get_system_memory (), sgx_epc->base, + &sgx_epc->mr); + +qemu_opts_foreach (qemu_find_opts ("sgx-epc" ), sgx_epc_init_func,NULL , + &error_fatal); + +if ((sgx_epc->base + sgx_epc->size) < sgx_epc->base) { +error_report ("Size of all 'sgx-epc' =0x%" PRIu64" causes EPC to wrap" , + sgx_epc->size); +exit (EXIT_FAILURE ); + } + +memory_region_set_size (&sgx_epc->mr, sgx_epc->size); +} + +static QemuOptsList sgx_epc_opts = { + .name ="sgx-epc" , + .implied_opt_name ="id" , + .head =QTAILQ_HEAD_INITIALIZER (sgx_epc_opts.head), + .desc = { + { + .name ="id" , + .type =QEMU_OPT_STRING , + .help ="SGX EPC section ID" , + },{ + .name ="memdev" , + .type =QEMU_OPT_STRING , + .help ="memory object backend" , + }, + {/* end of list */ } + }, +}; + +static void sgx_epc_register_opts (void ) +{ +qemu_add_opts (&sgx_epc_opts); +} + +opts_init (sgx_epc_register_opts);include/hw/i386/pc.h +6
-0@@ -12,6 +12,7 @@#include "hw/acpi/acpi_dev_interface.h" #include "hw/hotplug.h" #include "qom/object.h" +#include "hw/i386/sgx-epc.h" #define HPET_INTCAP "hpet-intcap"@@ -53,6 +54,8 @@ typedef struct PCMachineState {/* ACPI Memory hotplug IO base address */ hwaddr memhp_io_base; + +SGXEPCState *sgx_epc; } PCMachineState;#define PC_MACHINE_ACPI_DEVICE_PROP "acpi-device"@@ -197,6 +200,9 @@ bool pc_system_ovmf_table_find(const char *entry, uint8_t **data,void pc_madt_cpu_entry (AcpiDeviceIf *adev,int uid,const CPUArchIdList *apic_ids,GArray *entry); +/* sgx-epc.c */ +void pc_machine_init_sgx_epc (PCMachineState *pcms); +extern GlobalProperty pc_compat_5_2[];extern const size_t pc_compat_5_2_len;include/hw/i386/sgx-epc.h +16
-0@@ -41,4 +41,20 @@ typedef struct SGXEPCDevice { HostMemoryBackend *hostmem; } SGXEPCDevice; +/* +* @base: address in guest physical address space where EPC regions start +* @mr: address space container for memory devices +*/ +typedef struct SGXEPCState { +uint64_t base; +uint64_t size; + +MemoryRegion mr; + +struct SGXEPCDevice **sections; +int nr_sections; +}SGXEPCState ; + +extern int sgx_epc_enabled; +#endif qemu-options.hx +8
-0@@ -532,6 +532,14 @@ SRST Preallocate memory when using -mem-path. ERST +DEF("sgx-epc", HAS_ARG, QEMU_OPTION_sgx_epc, + "-sgx-epc memdev=memid[,id=epcid]\n", + QEMU_ARCH_I386) +SRST +``-sgx-epc memdev=@var{memid}[,id=@var{epcid}]`` + Define an SGX EPC section. +ERST + DEF("k", HAS_ARG, QEMU_OPTION_k, "-k language use keyboard layout (for example 'fr' for French)\n", QEMU_ARCH_ALL)softmmu/globals.c +1
-0@@ -70,3 +70,4 @@ bool qemu_uuid_set;uint32_t xen_domid;enum xen_mode xen_mode =XEN_EMULATE ;bool xen_domid_restrict; +int sgx_epc_enabled;softmmu/vl.c +9
-0@@ -74,6 +74,7 @@#include "hw/block/block.h" #include "hw/i386/x86.h" #include "hw/i386/pc.h" +#include "hw/i386/sgx-epc.h" #include "migration/misc.h" #include "migration/snapshot.h" #include "sysemu/tpm.h" @@ -2891,6 +2892,14 @@ void qemu_init(int argc, char **argv, char **envp)case QEMU_OPTION_mem_prealloc: mem_prealloc =1 ;break ; +case QEMU_OPTION_sgx_epc: + opts =qemu_opts_parse_noisily (qemu_find_opts ("sgx-epc" ), + optarg, false); +if (!opts) { +exit (1 ); + } + sgx_epc_enabled =1 ; +break ;case QEMU_OPTION_d: log_mask = optarg;break ;
PATCH RESEND 6/32 i386: Add primary SGX CPUID and MSR defines
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
Add CPUID defines for SGX and SGX Launch Control (LC), as well as defines for their associated FEATURE_CONTROL MSR bits. Define the Launch Enclave Public Key Hash MSRs (LE Hash MSRs), which exist when SGX LC is present (in CPUID), and are writable when SGX LC is enabled (in FEATURE_CONTROL).
--- target/i386/cpu.c | 4 ++-- target/i386/cpu.h | 12 ++++++++++++ 2 files changed, 14 insertions(+), 2 deletions(-)
target/i386/cpu.c +2 -2
@@ -938,7 +938,7 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = { [FEAT_7_0_EBX] = { .type = CPUID_FEATURE_WORD, .feat_names = { -"fsgsbase" ,"tsc-adjust" ,NULL ,"bmi1" , +"fsgsbase" ,"tsc-adjust" ,"sgx" ,"bmi1" ,"hle" ,"avx2" ,NULL ,"smep" ,"bmi2" ,"erms" ,"invpcid" ,"rtm" , NULL, NULL, "mpx", NULL,
@@ -964,7 +964,7 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {"la57" ,NULL ,NULL ,NULL ,NULL ,NULL ,"rdpid" ,NULL ,"bus-lock-detect" ,"cldemote" ,NULL ,"movdiri" , - "movdir64b", NULL, NULL, "pks", + "movdir64b", NULL, "sgxlc", "pks", }, .cpuid = { .eax = 7,
target/i386/cpu.h +12 -0
@@ -360,9 +360,17 @@ typedef enum X86Seg {#define MSR_IA32_PKRS 0x6e1#define FEATURE_CONTROL_LOCKED (1<<0) +#define FEATURE_CONTROL_VMXON_ENABLED_INSIDE_SMX (1ULL << 1)#define FEATURE_CONTROL_VMXON_ENABLED_OUTSIDE_SMX (1<<2) +#define FEATURE_CONTROL_SGX_LC (1ULL << 17) +#define FEATURE_CONTROL_SGX (1ULL << 18)#define FEATURE_CONTROL_LMCE (1<<20) +#define MSR_IA32_SGXLEPUBKEYHASH0 0x8c +#define MSR_IA32_SGXLEPUBKEYHASH1 0x8d +#define MSR_IA32_SGXLEPUBKEYHASH2 0x8e +#define MSR_IA32_SGXLEPUBKEYHASH3 0x8f +#define MSR_P6_PERFCTR0 0xc1#define MSR_IA32_SMBASE 0x9e
@@ -689,6 +697,8 @@ typedef uint64_t FeatureWordArray[FEATURE_WORDS];/* Support RDFSBASE/RDGSBASE/WRFSBASE/WRGSBASE */ #define CPUID_7_0_EBX_FSGSBASE (1U << 0) +/* Support SGX */ +#define CPUID_7_0_EBX_SGX (1U << 2)/* 1st Group of Advanced Bit Manipulation Extensions */ #define CPUID_7_0_EBX_BMI1 (1U << 3)/* Hardware Lock Elision */
@@ -776,6 +786,8 @@ typedef uint64_t FeatureWordArray[FEATURE_WORDS];#define CPUID_7_0_ECX_MOVDIRI (1U << 27)/* Move 64 Bytes as Direct Store Instruction */ #define CPUID_7_0_ECX_MOVDIR64B (1U << 28) +/* Support SGX Launch Control */ +#define CPUID_7_0_ECX_SGX_LC (1U << 30)/* Protection Keys for Supervisor-mode Pages */ #define CPUID_7_0_ECX_PKS (1U << 31)
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 7/32 i386: Add SGX CPUID leaf FEAT_SGX_12_0_EAX
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
CPUID leaf 12_0_EAX is an Intel-defined feature bits leaf enumerating the CPU's SGX capabilities, e.g. supported SGX instruction sets. Currently there are four enumerated capabilities:
- SGX1 instruction set, i.e. "base" SGX
- SGX2 instruction set for dynamic EPC management
- ENCLV instruction set for VMM oversubscription of EPC
- ENCLS-C instruction set for thread safe variants of ENCLS
--- target/i386/cpu.c | 20 ++++++++++++++++++++ target/i386/cpu.h | 1 + 2 files changed, 21 insertions(+)
target/i386/cpu.c +20 -0
@@ -676,6 +676,7 @@ static void x86_cpu_vendor_words2str(char *dst, uint32_t vendor1,/* missing: CPUID_XSAVE_XSAVEC, CPUID_XSAVE_XSAVES */ #define TCG_14_0_ECX_FEATURES 0 +#define TCG_SGX_12_0_EAX_FEATURES 0 typedef enum FeatureWordType { CPUID_FEATURE_WORD,
@@ -1325,6 +1326,25 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = { .tcg_features = TCG_14_0_ECX_FEATURES, }, + [FEAT_SGX_12_0_EAX] = { + .type = CPUID_FEATURE_WORD, + .feat_names = { + "sgx1", "sgx2", NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + }, + .cpuid = { + .eax = 0x12, + .needs_ecx = true, .ecx = 0, + .reg = R_EAX, + }, + .tcg_features = TCG_SGX_12_0_EAX_FEATURES, + }, }; typedef struct FeatureMask {
target/i386/cpu.h +1 -0
@@ -549,6 +549,7 @@ typedef enum FeatureWord {FEAT_VMX_BASIC ,FEAT_VMX_VMFUNC ,FEAT_14_0_ECX , +FEAT_SGX_12_0_EAX ,/* CPUID[EAX=0x12,ECX=0].EAX (SGX) */ FEATURE_WORDS , } FeatureWord;
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 8/32 i386: Add SGX CPUID leaf FEAT_SGX_12_0_EBX
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
CPUID leaf 12_0_EBX is an Intel-defined feature bits leaf enumerating the platform's SGX extended capabilities. Currently there is a single capabilitiy:
- EXINFO: record information about #PFs and #GPs in the enclave's SSA
--- target/i386/cpu.c | 21 +++++++++++++++++++++ target/i386/cpu.h | 1 + 2 files changed, 22 insertions(+)
target/i386/cpu.c +21 -0
@@ -677,6 +677,7 @@ static void x86_cpu_vendor_words2str(char *dst, uint32_t vendor1,CPUID_XSAVE_XSAVEC, CPUID_XSAVE_XSAVES */ #define TCG_14_0_ECX_FEATURES 0#define TCG_SGX_12_0_EAX_FEATURES 0 +#define TCG_SGX_12_0_EBX_FEATURES 0 typedef enum FeatureWordType { CPUID_FEATURE_WORD,
@@ -1345,6 +1346,26 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = { }, .tcg_features = TCG_SGX_12_0_EAX_FEATURES, }, + + [FEAT_SGX_12_0_EBX] = { + .type = CPUID_FEATURE_WORD, + .feat_names = { + "sgx-exinfo" , NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + }, + .cpuid = { + .eax = 0x12, + .needs_ecx = true, .ecx = 0, + .reg = R_EBX, + }, + .tcg_features = TCG_SGX_12_0_EBX_FEATURES, + }, }; typedef struct FeatureMask {
target/i386/cpu.h +1 -0
@@ -550,6 +550,7 @@ typedef enum FeatureWord {FEAT_VMX_VMFUNC ,FEAT_14_0_ECX ,FEAT_SGX_12_0_EAX ,/* CPUID[EAX=0x12,ECX=0].EAX (SGX) */ +FEAT_SGX_12_0_EBX ,/* CPUID[EAX=0x12,ECX=0].EBX (SGX MISCSELECT[31:0]) */ FEATURE_WORDS , } FeatureWord;
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 9/32 i386: Add SGX CPUID leaf FEAT_SGX_12_1_EAX
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
CPUID leaf 12_1_EAX is an Intel-defined feature bits leaf enumerating the platform's SGX capabilities that may be utilized by an enclave, e.g. whether or not an enclave can gain access to the provision key. Currently there are six capabilities:
- INIT: set when the enclave has has been initialized by EINIT. Cannot be set by software, i.e. forced to zero in CPUID.
- DEBUG: permits a debugger to read/write into the enclave.
- MODE64BIT: the enclave runs in 64-bit mode
- PROVISIONKEY: grants has access to the provision key
- EINITTOKENKEY: grants access to the EINIT token key, i.e. the enclave can generate EINIT tokens
- KSS: Key Separation and Sharing enabled for the enclave.
Note that the entirety of CPUID.0x12.0x1, i.e. all registers, enumerates the allowed ATTRIBUTES (128 bits), but only bits 31:0 are directly exposed to the user (via FEAT_12_1_EAX). Bits 63:32 are currently all reserved and bits 127:64 correspond to the allowed XSAVE Feature Request Mask, which is calculated based on other CPU features, e.g. XSAVE, MPX, AVX, etc... and is not exposed to the user.
--- target/i386/cpu.c | 21 +++++++++++++++++++++ target/i386/cpu.h | 1 + 2 files changed, 22 insertions(+)
target/i386/cpu.c +21 -0
@@ -678,6 +678,7 @@ static void x86_cpu_vendor_words2str(char *dst, uint32_t vendor1,#define TCG_14_0_ECX_FEATURES 0#define TCG_SGX_12_0_EAX_FEATURES 0#define TCG_SGX_12_0_EBX_FEATURES 0 +#define TCG_SGX_12_1_EAX_FEATURES 0 typedef enum FeatureWordType { CPUID_FEATURE_WORD,
@@ -1366,6 +1367,26 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = { }, .tcg_features = TCG_SGX_12_0_EBX_FEATURES, }, + + [FEAT_SGX_12_1_EAX] = { + .type = CPUID_FEATURE_WORD, + .feat_names = { + NULL, "sgx-debug", "sgx-mode64", NULL, + "sgx-provisionkey", "sgx-tokenkey", NULL, "sgx-kss", + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + }, + .cpuid = { + .eax = 0x12, + .needs_ecx = true, .ecx = 1, + .reg = R_EAX, + }, + .tcg_features = TCG_SGX_12_1_EAX_FEATURES, + }, }; typedef struct FeatureMask {
target/i386/cpu.h +1 -0
@@ -551,6 +551,7 @@ typedef enum FeatureWord {FEAT_14_0_ECX ,FEAT_SGX_12_0_EAX ,/* CPUID[EAX=0x12,ECX=0].EAX (SGX) */ FEAT_SGX_12_0_EBX ,/* CPUID[EAX=0x12,ECX=0].EBX (SGX MISCSELECT[31:0]) */ +FEAT_SGX_12_1_EAX ,/* CPUID[EAX=0x12,ECX=1].EAX (SGX ATTRIBUTES[31:0]) */ FEATURE_WORDS , } FeatureWord;
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 10/32 i386: Add get/set/migrate support for SGX_LEPUBKEYHASH MSRs
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
On real hardware, on systems that supports SGX Launch Control, those MSRs are initialized to digest of Intel's signing key; on systems that don't support SGX Launch Control, those MSRs are not available but hardware always uses digest of Intel's signing key in EINIT.
KVM advertises SGX LC via CPUID if and only if the MSRs are writable. Unconditionally initialize those MSRs to digest of Intel's signing key when CPU is realized and reset to reflect the fact. This avoids potential bug in case kvm_arch_put_registers() is called before kvm_arch_get_registers() is called, in which case guest's virtual SGX_LEPUBKEYHASH MSRs will be set to 0, although KVM initializes those to digest of Intel's signing key by default, since KVM allows those MSRs to be updated by Qemu to support live migration.
Save/restore the SGX Launch Enclave Public Key Hash MSRs if SGX Launch Control (LC) is exposed to the guest. Likewise, migrate the MSRs if they are writable by the guest.
--- target/i386/cpu.c | 17 ++++++++++++++++- target/i386/cpu.h | 1 + target/i386/kvm/kvm.c | 22 ++++++++++++++++++++++ target/i386/machine.c | 20 ++++++++++++++++++++ 4 files changed, 59 insertions(+), 1 deletion(-)
target/i386/cpu.c +16 -1
@@ -6179,6 +6179,16 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, } } +#ifndef CONFIG_USER_ONLY +static void x86_cpu_set_sgxlepubkeyhash (CPUX86State *env) +{ + env->msr_ia32_sgxlepubkeyhash[0 ] =0xa6053e051270b7acULL ; + env->msr_ia32_sgxlepubkeyhash[1 ] =0x6cfbe8ba8b3b413dULL ; + env->msr_ia32_sgxlepubkeyhash[2 ] =0xc4916d99f2b3735dULL ; + env->msr_ia32_sgxlepubkeyhash[3 ] =0xd4f8c05909f9bb3bULL ; +} +#endif +static void x86_cpu_reset (DeviceState *dev) {CPUState *s =CPU (dev);
@@ -6310,6 +6320,8 @@ static void x86_cpu_reset(DeviceState *dev)if (kvm_enabled ()) {kvm_arch_reset_vcpu (cpu); } + +x86_cpu_set_sgxlepubkeyhash (env);#endif }
@@ -6922,6 +6934,10 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)/* Process Hyper-V enlightenments */ x86_cpu_hyperv_realize (cpu); +#ifndef CONFIG_USER_ONLY +x86_cpu_set_sgxlepubkeyhash (env); +#endif +cpu_exec_realizefn (cs, &local_err);if (local_err !=NULL ) {error_propagate (errp, local_err);
@@ -7559,7 +7575,6 @@ static const TypeInfo x86_cpu_type_info = { .class_init = x86_cpu_common_class_init, }; -/* "base" CPU model, used by query-cpu-model-expansion */ static void x86_cpu_base_class_init (ObjectClass *oc,void *data) {
target/i386/cpu.h +1 -0
@@ -1500,6 +1500,7 @@ typedef struct CPUX86State {uint64_t mcg_status;uint64_t msr_ia32_misc_enable;uint64_t msr_ia32_feature_control; +uint64_t msr_ia32_sgxlepubkeyhash[4 ];uint64_t msr_fixed_ctr_ctrl;uint64_t msr_global_ctrl;
target/i386/kvm/kvm.c +22 -0
@@ -3030,6 +3030,17 @@ static int kvm_put_msrs(X86CPU *cpu, int level) } } +if (env->features[FEAT_7_0_ECX ] &CPUID_7_0_ECX_SGX_LC ) { +kvm_msr_entry_add (cpu,MSR_IA32_SGXLEPUBKEYHASH0 , + env->msr_ia32_sgxlepubkeyhash[0 ]); +kvm_msr_entry_add (cpu,MSR_IA32_SGXLEPUBKEYHASH1 , + env->msr_ia32_sgxlepubkeyhash[1 ]); +kvm_msr_entry_add (cpu,MSR_IA32_SGXLEPUBKEYHASH2 , + env->msr_ia32_sgxlepubkeyhash[2 ]); +kvm_msr_entry_add (cpu,MSR_IA32_SGXLEPUBKEYHASH3 , + env->msr_ia32_sgxlepubkeyhash[3 ]); + } +/* Note: MSR_IA32_FEATURE_CONTROL is written separately, see * kvm_put_msr_feature_control. */ }
@@ -3369,6 +3380,13 @@ static int kvm_get_msrs(X86CPU *cpu) } } +if (env->features[FEAT_7_0_ECX ] &CPUID_7_0_ECX_SGX_LC ) { +kvm_msr_entry_add (cpu,MSR_IA32_SGXLEPUBKEYHASH0 ,0 ); +kvm_msr_entry_add (cpu,MSR_IA32_SGXLEPUBKEYHASH1 ,0 ); +kvm_msr_entry_add (cpu,MSR_IA32_SGXLEPUBKEYHASH2 ,0 ); +kvm_msr_entry_add (cpu,MSR_IA32_SGXLEPUBKEYHASH3 ,0 ); + } + ret =kvm_vcpu_ioctl (CPU (cpu),KVM_GET_MSRS , cpu->kvm_msr_buf);if (ret <0 ) {return ret;
@@ -3658,6 +3676,10 @@ static int kvm_get_msrs(X86CPU *cpu) case MSR_IA32_RTIT_ADDR0_A ... MSR_IA32_RTIT_ADDR3_B: env->msr_rtit_addrs[index -MSR_IA32_RTIT_ADDR0_A ] = msrs[i].data;break ; + case MSR_IA32_SGXLEPUBKEYHASH0 ... MSR_IA32_SGXLEPUBKEYHASH3: + env->msr_ia32_sgxlepubkeyhash[index -MSR_IA32_SGXLEPUBKEYHASH0 ] = + msrs[i].data; +break ; } }
target/i386/machine.c +20 -0
@@ -1396,6 +1396,25 @@ static const VMStateDescription vmstate_msr_tsx_ctrl = { } }; +static bool intel_sgx_msrs_needed (void *opaque) +{ +X86CPU *cpu = opaque; +CPUX86State *env = &cpu->env; + +return !!(env->features[FEAT_7_0_ECX ] &CPUID_7_0_ECX_SGX_LC ); +} + +static const VMStateDescription vmstate_msr_intel_sgx = { + .name ="cpu/intel_sgx" , + .version_id =1 , + .minimum_version_id =1 , + .needed = intel_sgx_msrs_needed, + .fields = (VMStateField []) { +VMSTATE_UINT64_ARRAY (env.msr_ia32_sgxlepubkeyhash,X86CPU ,4 ), +VMSTATE_END_OF_LIST () + } +}; +VMStateDescription vmstate_x86_cpu = { .name ="cpu" , .version_id =12 ,
@@ -1531,6 +1550,7 @@ VMStateDescription vmstate_x86_cpu = { &vmstate_nested_state, #endif &vmstate_msr_tsx_ctrl, + &vmstate_msr_intel_sgx,NULL } };
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 11/32 i386: Add feature control MSR dependency when SGX is enabled
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
SGX adds multiple flags to FEATURE_CONTROL to enable SGX and Flexible Launch Control.
--- target/i386/kvm/kvm.c | 5 +++++ 1 file changed, 5 insertions(+)
target/i386/kvm/kvm.c +5 -0
@@ -1789,6 +1789,11 @@ int kvm_arch_init_vcpu(CPUState *cs) !!(c->ecx &CPUID_EXT_SMX ); } + c =cpuid_find_entry (&cpuid_data.cpuid,7 ,0 ); +if (c && (c->ebx &CPUID_7_0_EBX_SGX )) { + has_msr_feature_control = true; + } +if (env->mcg_cap &MCG_LMCE_P ) { has_msr_mcg_ext_ctl = has_msr_feature_control = true; }
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 12/32 i386: Update SGX CPUID info according to hardware/KVM/user input
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
Expose SGX to the guest if and only if KVM is enabled and supports virtualization of SGX. While the majority of ENCLS can be emulated to some degree, because SGX uses a hardware-based root of trust, the attestation aspects of SGX cannot be emulated in software, i.e. ultimately emulation will fail as software cannot generate a valid quote/report. The complexity of partially emulating SGX in Qemu far outweighs the value added, e.g. an SGX specific simulator for userspace applications can emulate SGX for development and testing purposes.
Note, access to the PROVISIONKEY is not yet advertised to the guest as KVM blocks access to the PROVISIONKEY by default and requires userspace to provide additional credentials (via ioctl()) to expose PROVISIONKEY.
--- hw/i386/sgx-epc.c | 17 +++++++++ include/hw/i386/sgx-epc.h | 2 + target/i386/cpu.c | 77 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 96 insertions(+)
hw/i386/sgx-epc.c +17 -0
@@ -173,6 +173,23 @@ static void sgx_epc_register_types(void)type_init (sgx_epc_register_types) +int sgx_epc_get_section(int section_nr, uint64_t *addr, uint64_t *size) +{ +PCMachineState *pcms =PC_MACHINE (qdev_get_machine ()); +SGXEPCDevice *epc; + +if (pcms->sgx_epc ==NULL || pcms->sgx_epc->nr_sections <= section_nr) { +return 1 ; + } + + epc = pcms->sgx_epc->sections[section_nr]; + + *addr = epc->addr; + *size =memory_device_get_region_size (MEMORY_DEVICE (epc), &error_fatal); + +return 0 ; +} +static int sgx_epc_set_property (void *opaque,const char *name,const char *value,Error **errp)
include/hw/i386/sgx-epc.h +2 -0
@@ -57,4 +57,6 @@ typedef struct SGXEPCState {extern int sgx_epc_enabled; +int sgx_epc_get_section (int section_nr,uint64_t *addr,uint64_t *size); +#endif
target/i386/cpu.c +77 -0
95 lines not on this page · raw message
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 13/32 linux-headers: Add placeholder for KVM_CAP_SGX_ATTRIBUTE
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
KVM_CAP_SGX_ATTRIBUTE is a proposed capability for Intel SGX that can be used by userspace to enable privileged attributes, e.g. access to the PROVISIONKEY.
--- linux-headers/linux/kvm.h | 1 + 1 file changed, 1 insertion(+)
linux-headers/linux/kvm.h +1 -0
@@ -1056,6 +1056,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_ENFORCE_PV_FEATURE_CPUID 190#define KVM_CAP_SYS_HYPERV_CPUID 191#define KVM_CAP_DIRTY_LOG_RING 192 +#define KVM_CAP_SGX_ATTRIBUTE 195#ifdef KVM_CAP_IRQ_ROUTING
Signature · 1 line
2.29.2.334.gfaefdd61ec
Re: [RESEND PATCH 13/32] linux-headers: Add placeholder for KVM_CAP_SGX_ATTRIBUTE
Quoted 21 lines · Yang Zhong
From: Sean Christopherson <[email protected]>
KVM_CAP_SGX_ATTRIBUTEis a proposed capability for Intel SGX that can be used by userspace to enable privileged attributes, e.g. access to the PROVISIONKEY.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Yang Zhong <[email protected]>---
linux-headers/linux/kvm.h | 1 +1 file changed, 1 insertion(+)linux-headers/linux/kvm.h +1
-0@@ -1056,6 +1056,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_ENFORCE_PV_FEATURE_CPUID 190 #define KVM_CAP_SYS_HYPERV_CPUID 191 #define KVM_CAP_DIRTY_LOG_RING 192 +#define KVM_CAP_SGX_ATTRIBUTE 195
Quoted 2 lines
This hasn't been updated to 196.
Btw, should we just copy the latest kvm.h from linux kernel? 193-195 are missing if only define KVM_CAP_SGX_ATTRIBUTE here.
Re: [RESEND PATCH 13/32] linux-headers: Add placeholder for KVM_CAP_SGX_ATTRIBUTE
Quoted 13 lines · Kai Huang
Quoted 12 lines · Yang Zhong
From: Sean Christopherson <[email protected]>
KVM_CAP_SGX_ATTRIBUTEis a proposed capability for Intel SGX that can be used by userspace to enable privileged attributes, e.g. access to the PROVISIONKEY.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Yang Zhong <[email protected]>---
linux-headers/linux/kvm.h | 1 +1 file changed, 1 insertion(+)
Quoted 14 lines
Quoted 11 lines
linux-headers/linux/kvm.h +1
-0@@ -1056,6 +1056,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_ENFORCE_PV_FEATURE_CPUID 190 #define KVM_CAP_SYS_HYPERV_CPUID 191 #define KVM_CAP_DIRTY_LOG_RING 192 +#define KVM_CAP_SGX_ATTRIBUTE 195This hasn't been updated to 196.
As for KVM_CAP_SGX_ATTRIBUTE is still 195, i mentioned this in the coverletter, and this release still rebased on KVM-SGX upstream branch(5.12.0-rc3+). Since the KVM-SGX series has been merged into linux release in 5/1, i will rebase Qemu SGX on linux release in the next version patch series.
Quoted 3 lines
Btw, should we just copy the latest
kvm.hfrom linux kernel? 193-195 are missing if only defineKVM_CAP_SGX_ATTRIBUTEhere.
There are three macro definitions before KVM_CAP_SGX_ATTRIBUTE, and KVM_CAP_X86_BUS_LOCK_EXIT use case is being upstreamed by our colleague.
As for other two macro definitions, i am not sure if we need add those here. thanks!
Yang
PATCH RESEND 14/32 i386: kvm: Add support for exposing PROVISIONKEY to guest
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
If the guest want to fully use SGX, the guest needs to be able to access provisioning key. Add a new KVM_CAP_SGX_ATTRIBUTE to KVM to support provisioning key to KVM guests.
--- target/i386/cpu.c | 5 ++++- target/i386/kvm/kvm.c | 29 +++++++++++++++++++++++++++++ target/i386/kvm/kvm_i386.h | 2 ++ 3 files changed, 35 insertions(+), 1 deletion(-)
target/i386/cpu.c +4 -1
@@ -6015,7 +6015,10 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, *ecx |=XSTATE_FP_MASK |XSTATE_SSE_MASK ;/* Access to PROVISIONKEY requires additional credentials. */ - *eax &= ~(1U << 4); +if ((*eax & (1U <<4 )) && + !kvm_enable_sgx_provisioning (cs->kvm_state)) { + *eax &= ~(1U <<4 ); + } }#endif break ;
target/i386/kvm/kvm.c +29 -0
@@ -4555,6 +4555,35 @@ void kvm_arch_update_guest_debug(CPUState *cpu, struct kvm_guest_debug *dbg) } } +static bool has_sgx_provisioning; + +static bool __kvm_enable_sgx_provisioning (KVMState *s) +{ +int fd, ret; + +if (!kvm_vm_check_extension (s,KVM_CAP_SGX_ATTRIBUTE )) { +return false; + } + + fd =qemu_open_old ("/dev/sgx_provision" ,O_RDONLY ); +if (fd <0 ) { +return false; + } + + ret =kvm_vm_enable_cap (s,KVM_CAP_SGX_ATTRIBUTE ,0 , fd); +if (ret) { +error_report ("Could not enable SGX PROVISIONKEY: %s" ,strerror (-ret)); +exit (1 ); + } +close (fd); +return true; +} + +bool kvm_enable_sgx_provisioning (KVMState *s) +{ +return MEMORIZE (__kvm_enable_sgx_provisioning (s), has_sgx_provisioning); +} +static bool host_supports_vmx (void ) {uint32_t ecx, unused;
target/i386/kvm/kvm_i386.h +2 -0
@@ -50,4 +50,6 @@ bool kvm_hv_vpindex_settable(void);uint64_t kvm_swizzle_msi_ext_dest_id (uint64_t address); +bool kvm_enable_sgx_provisioning (KVMState *s); +#endif
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 15/32 i386: Propagate SGX CPUID sub-leafs to KVM
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
The SGX sub-leafs are enumerated at CPUID 0x12. Indices 0 and 1 are always present when SGX is supported, and enumerate SGX features and capabilities. Indices >=2 are directly correlated with the platform's EPC sections. Because the number of EPC sections is dynamic and user defined, the number of SGX sub-leafs is "NULL" terminated.
--- target/i386/kvm/kvm.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
target/i386/kvm/kvm.c +19 -0
@@ -1615,6 +1615,25 @@ int kvm_arch_init_vcpu(CPUState *cs) }break ;case 0x7 : +case 0x12 : +for (j =0 ; ; j++) { + c->function = i; + c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; + c->index = j; +cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); + +if (j >1 && (c->eax &0xf ) !=1 ) { +break ; + } + +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"cpuid_data is full, no space for " +"cpuid(eax:0x12,ecx:0x%x)\n" , j); +abort (); + } + c = &cpuid_data.entries[cpuid_i++]; + } +break ;case 0x14 : {uint32_t times;
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 16/32 Adjust min CPUID level to 0x12 when SGX is enabled
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
SGX capabilities are enumerated through CPUID_0x12.
--- target/i386/cpu.c | 5 +++++ 1 file changed, 5 insertions(+)
target/i386/cpu.c +5 -0
@@ -6741,6 +6741,11 @@ static void x86_cpu_expand_features(X86CPU *cpu, Error **errp)if (sev_enabled ()) {x86_cpu_adjust_level (cpu, &env->cpuid_min_xlevel,0x8000001F ); } + +/* SGX requires CPUID[0x12] for EPC enumeration */ +if (env->features[FEAT_7_0_EBX ] &CPUID_7_0_EBX_SGX ) { +x86_cpu_adjust_level (cpu, &env->cpuid_min_level,0x12 ); + } }/* Set cpuid_*level* based on cpuid_min_*level, if not explicitly set */
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 17/32 hw/i386/fw_cfg: Set SGX bits in feature control fw_cfg accordingly
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
Request SGX an SGX Launch Control to be enabled in FEATURE_CONTROL when the features are exposed to the guest. Our design is the SGX Launch Control bit will be unconditionally set in FEATURE_CONTROL, which is unlike host bios.
--- hw/i386/fw_cfg.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
hw/i386/fw_cfg.c +9 -1
@@ -157,7 +157,7 @@ void fw_cfg_build_feature_control(MachineState *ms, FWCfgState *fw_cfg) {X86CPU *cpu =X86_CPU (ms->possible_cpus->cpus[0 ].cpu);CPUX86State *env = &cpu->env; -uint32_t unused, ecx, edx; +uint32_t unused, ebx, ecx, edx;uint64_t feature_control_bits =0 ;uint64_t *val;
@@ -172,6 +172,14 @@ void fw_cfg_build_feature_control(MachineState *ms, FWCfgState *fw_cfg) feature_control_bits |=FEATURE_CONTROL_LMCE ; } +cpu_x86_cpuid (env,0x7 ,0 , &unused, &ebx, &ecx, &unused); +if (ebx &CPUID_7_0_EBX_SGX ) { + feature_control_bits |=FEATURE_CONTROL_SGX ; + } +if (ecx &CPUID_7_0_ECX_SGX_LC ) { + feature_control_bits |=FEATURE_CONTROL_SGX_LC ; + } +if (!feature_control_bits) {return ; }
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 18/32 hw/i386/pc: Account for SGX EPC sections when calculating device memory
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
Add helpers to detect if SGX EPC exists above 4g, and if so, where SGX EPC above 4g ends. Use the helpers to adjust the device memory range if SGX EPC exists above 4g.
For multiple virtual EPC sections, we just put them together physically contiguous for the simplicity because we don't support EPC NUMA affinity now. Once the SGX EPC NUMA support in the kernel SGX driver, we will support this in the future.
Note that SGX EPC is currently hardcoded to reside above 4g.
--- hw/i386/pc.c | 11 ++++++++++- include/hw/i386/sgx-epc.h | 7 +++++++ 2 files changed, 17 insertions(+), 1 deletion(-)
hw/i386/pc.c +10 -1
@@ -925,8 +925,15 @@ void pc_memory_init(PCMachineState *pcms,exit (EXIT_FAILURE ); } +if (pcms->sgx_epc !=NULL ) { + machine->device_memory->base =sgx_epc_above_4g_end (pcms->sgx_epc); + }else { + machine->device_memory->base = +0x100000000ULL + x86ms->above_4g_mem_size; + } + machine->device_memory->base = -ROUND_UP (0x100000000ULL + x86ms->above_4g_mem_size,1 * GiB); +ROUND_UP (machine->device_memory->base,1 * GiB);if (pcmc->enforce_aligned_dimm) {/* size device region assuming 1G page max alignment per slot */
@@ -1011,6 +1018,8 @@ uint64_t pc_pci_hole64_start(void)if (!pcmc->broken_reserved_end) { hole64_start +=memory_region_size (&ms->device_memory->mr); } + }else if (pcms->sgx_epc !=NULL ) { + hole64_start =sgx_epc_above_4g_end (pcms->sgx_epc); }else { hole64_start =0x100000000ULL + x86ms->above_4g_mem_size; }
include/hw/i386/sgx-epc.h +7 -0
@@ -59,4 +59,11 @@ extern int sgx_epc_enabled;int sgx_epc_get_section (int section_nr,uint64_t *addr,uint64_t *size); +static inline uint64_t sgx_epc_above_4g_end (SGXEPCState *sgx_epc) +{ +assert (sgx_epc !=NULL && sgx_epc->base >=0x100000000ULL ); + +return sgx_epc->base + sgx_epc->size; +} +#endif
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 19/32 i386/pc: Add e820 entry for SGX EPC section(s)
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
Note that SGX EPC is currently guaranteed to reside in a single contiguous chunk of memory regardless of the number of EPC sections.
--- hw/i386/pc.c | 4 ++++ 1 file changed, 4 insertions(+)
hw/i386/pc.c +4 -0
@@ -895,6 +895,10 @@ void pc_memory_init(PCMachineState *pcms,e820_add_entry (0x100000000ULL , x86ms->above_4g_mem_size,E820_RAM ); } +if (pcms->sgx_epc !=NULL ) { +e820_add_entry (pcms->sgx_epc->base, pcms->sgx_epc->size,E820_RESERVED ); + } +if (!pcmc->has_reserved_memory && (machine->ram_slots || (machine->maxram_size > machine->ram_size))) {
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 20/32 i386: acpi: Add SGX EPC entry to ACPI tables
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
The ACPI Device entry for SGX EPC is essentially a hack whose primary purpose is to provide software with a way to autoprobe SGX support, e.g. to allow software to implement SGX support as a driver. Details on the individual EPC sections are not enumerated through ACPI tables, i.e. software must enumerate the EPC sections via CPUID. Furthermore, software expects to see only a single EPC Device in the ACPI tables regardless of the number of EPC sections in the system.
However, several versions of Windows do rely on the ACPI tables to enumerate the address and size of the EPC. So, regardless of the number of EPC sections exposed to the guest, create exactly *one* EPC device with a _CRS entry that spans the entirety of all EPC sections (which are guaranteed to be contiguous in Qemu).
Note, NUMA support for EPC memory is intentionally not considered as enumerating EPC NUMA information is not yet defined for bare metal.
--- hw/i386/acpi-build.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
hw/i386/acpi-build.c +22 -0
@@ -1801,6 +1801,28 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,aml_append (sb_scope, dev); } +if (pcms->sgx_epc) { +uint64_t epc_base = pcms->sgx_epc->base; +uint64_t epc_size = pcms->sgx_epc->size; + + dev =aml_device ("EPC" ); +aml_append (dev,aml_name_decl ("_HID" ,aml_eisaid ("INT0E0C" ))); +aml_append (dev,aml_name_decl ("_STR" , +aml_unicode ("Enclave Page Cache 1.0" ))); + crs =aml_resource_template (); +aml_append (crs, +aml_qword_memory (AML_POS_DECODE ,AML_MIN_FIXED , +AML_MAX_FIXED ,AML_NON_CACHEABLE , +AML_READ_WRITE ,0 , epc_base, + epc_base + epc_size -1 ,0 , epc_size)); +aml_append (dev,aml_name_decl ("_CRS" , crs)); + + method =aml_method ("_STA" ,0 ,AML_NOTSERIALIZED ); +aml_append (method,aml_return (aml_int (0x0f ))); +aml_append (dev, method); + +aml_append (sb_scope, dev); + }aml_append (dsdt, sb_scope);/* copy AML table into ACPI tables blob and patch header there */
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 21/32 q35: Add support for SGX EPC
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
Enable SGX EPC virtualization, which is currently only support by KVM.
--- hw/i386/pc_q35.c | 3 +++ 1 file changed, 3 insertions(+)
hw/i386/pc_q35.c +3 -0
@@ -177,6 +177,9 @@ static void pc_q35_init(MachineState *machine) x86ms->below_4g_mem_size = machine->ram_size; } +if (sgx_epc_enabled) { +pc_machine_init_sgx_epc (pcms); + }x86_cpus_init (x86ms, pcmc->default_cpu_version);kvmclock_create (pcmc->kvmclock_create_always);
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 22/32 i440fx: Add support for SGX EPC
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
Enable SGX EPC virtualization, which is currently only support by KVM.
--- hw/i386/pc_piix.c | 4 ++++ 1 file changed, 4 insertions(+)
hw/i386/pc_piix.c +4 -0
@@ -156,6 +156,10 @@ static void pc_init1(MachineState *machine, } } +if (sgx_epc_enabled) { +pc_machine_init_sgx_epc (pcms); + } +x86_cpus_init (x86ms, pcmc->default_cpu_version);if (pcmc->kvmclock_enabled) {
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 23/32 hostmem: Add the reset interface for EPC backend reset
In reply to Yang ZhongAdd the sgx_memory_backend_reset() interface to handle EPC backend reset when VM is reset. This reset function will destroy previous backend memory region and re-mmap the EPC section for guest.
--- backends/hostmem-epc.c | 16 ++++++++++++++++ include/hw/i386/pc.h | 2 ++ 2 files changed, 18 insertions(+)
backends/hostmem-epc.c +16 -0
@@ -16,6 +16,7 @@#include "qom/object_interfaces.h" #include "qapi/error.h" #include "sysemu/hostmem.h" +#include "hw/i386/pc.h" #define TYPE_MEMORY_BACKEND_EPC "memory-backend-epc"
@@ -53,6 +54,21 @@ sgx_epc_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)g_free (name); } +void sgx_memory_backend_reset (HostMemoryBackend *backend,int fd, +Error **errp) +{ +MemoryRegion *mr = &backend->mr; + + mr->enabled = false; + +/* destroy the old memory region if it exist */ +if (fd >0 && mr->destructor) { + mr->destructor (mr); + } + +sgx_epc_backend_memory_alloc (backend, errp); +} +static void sgx_epc_backend_instance_init (Object *obj) {HostMemoryBackend *m =MEMORY_BACKEND (obj);
include/hw/i386/pc.h +2 -0
@@ -202,6 +202,8 @@ void pc_madt_cpu_entry(AcpiDeviceIf *adev, int uid,/* sgx-epc.c */ void pc_machine_init_sgx_epc (PCMachineState *pcms); +void sgx_memory_backend_reset (HostMemoryBackend *backend,int fd, +Error **errp);extern GlobalProperty pc_compat_5_2[];extern const size_t pc_compat_5_2_len;
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 24/32 sgx-epc: Add the reset interface for sgx-epc virt device
In reply to Yang ZhongIf the VM is reset, we need make sure sgx virt epc in clean status. Once the VM is reset, and sgx epc virt device will be reseted by reset callback registered by qemu_register_reset(). Since this epc virt device depend on backend, this reset will call backend reset interface to re-mmap epc to guest.
--- hw/i386/sgx-epc.c | 94 ++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 81 insertions(+), 13 deletions(-)
hw/i386/sgx-epc.c +81 -13
@@ -23,6 +23,9 @@#include "qemu/units.h" #include "target/i386/cpu.h" #include "exec/address-spaces.h" +#include "sysemu/reset.h" + +uint32_t epc_num;static Property sgx_epc_properties[] = {DEFINE_PROP_UINT64 (SGX_EPC_ADDR_PROP , SGXEPCDevice, addr,0 ),
@@ -52,12 +55,84 @@ static void sgx_epc_init(Object *obj) NULL, NULL, NULL); } +static void sgx_epc_del_subregion (DeviceState *dev) +{ +PCMachineState *pcms =PC_MACHINE (qdev_get_machine ()); +SGXEPCState *sgx_epc = pcms->sgx_epc; +SGXEPCDevice *epc =SGX_EPC (dev); + +/* del subregion and related operations */ +memory_region_del_subregion (&sgx_epc->mr, +host_memory_backend_get_memory (epc->hostmem)); +host_memory_backend_set_mapped (epc->hostmem, false); +g_free (sgx_epc->sections); + sgx_epc->sections =NULL ; + +/* multiple epc devices, only zero the first time */ +if (epc_num == sgx_epc->nr_sections) { + sgx_epc->size =0 ; + sgx_epc->nr_sections =0 ; + } +} + +static void sgx_epc_initialization (DeviceState *dev) +{ +PCMachineState *pcms =PC_MACHINE (qdev_get_machine ()); +SGXEPCState *sgx_epc = pcms->sgx_epc; +MemoryDeviceState *md =MEMORY_DEVICE (dev); +SGXEPCDevice *epc =SGX_EPC (dev); +Error *errp =NULL ; + +if (!epc->hostmem) { +error_setg (&errp,"'" SGX_EPC_MEMDEV_PROP "' property is not set" ); +return ; + } + + epc->addr = sgx_epc->base + sgx_epc->size; + +memory_region_add_subregion (&sgx_epc->mr, epc->addr - sgx_epc->base, +host_memory_backend_get_memory (epc->hostmem)); + +host_memory_backend_set_mapped (epc->hostmem, true); + + sgx_epc->sections = g_renew(SGXEPCDevice *, sgx_epc->sections, + sgx_epc->nr_sections +1 ); + sgx_epc->sections[sgx_epc->nr_sections++] = epc; + + sgx_epc->size +=memory_device_get_region_size (md, &errp); +} + +static void sgx_epc_reset (void *opaque) +{ +DeviceState *dev = opaque; +SGXEPCDevice *epc =SGX_EPC (dev); +Error *errp =NULL ; +int fd; + +if (!epc->hostmem) { +error_setg (&errp,"'" SGX_EPC_MEMDEV_PROP "' property is not set" ); +return ; + } + +/* delete subregion and related operations */ +sgx_epc_del_subregion (dev); + +/* reset sgx backend */ + fd =memory_region_get_fd (host_memory_backend_get_memory (epc->hostmem)); +sgx_memory_backend_reset (epc->hostmem, fd, &errp); +if (errp) { +error_setg (&errp,"failed to call sgx_memory_backend_reset" ); +return ; + } + +/* re-add subregion and related operations */ +sgx_epc_initialization (dev); +} +static void sgx_epc_realize (DeviceState *dev,Error **errp) {PCMachineState *pcms =PC_MACHINE (qdev_get_machine ());X86MachineState *x86ms =X86_MACHINE (pcms); -MemoryDeviceState *md =MEMORY_DEVICE (dev); -SGXEPCState *sgx_epc = pcms->sgx_epc;SGXEPCDevice *epc =SGX_EPC (dev);const char *path;
@@ -76,18 +151,11 @@ static void sgx_epc_realize(DeviceState *dev, Error **errp)return ; } - epc->addr = sgx_epc->base + sgx_epc->size; - -memory_region_add_subregion (&sgx_epc->mr, epc->addr - sgx_epc->base, -host_memory_backend_get_memory (epc->hostmem)); - -host_memory_backend_set_mapped (epc->hostmem, true); - - sgx_epc->sections = g_renew(SGXEPCDevice *, sgx_epc->sections, - sgx_epc->nr_sections +1 ); - sgx_epc->sections[sgx_epc->nr_sections++] = epc; +sgx_epc_initialization (dev); + epc_num++; - sgx_epc->size +=memory_device_get_region_size (md, errp); +/* register the reset callback for sgx reset */ +qemu_register_reset (sgx_epc_reset, dev); }static void sgx_epc_unrealize (DeviceState *dev)
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 25/32 qmp: Add query-sgx command
In reply to Yang ZhongThis QMP query command can be used by some userspaces to retrieve the SGX information when SGX is enabled on Intel platform.
--- monitor/qmp-cmds.c | 6 ++++++ qapi/misc.json | 42 ++++++++++++++++++++++++++++++++++++++ tests/qtest/qmp-cmd-test.c | 1 + 3 files changed, 49 insertions(+)
monitor/qmp-cmds.c +6 -0
@@ -351,3 +351,9 @@ void qmp_display_reload(DisplayReloadOptions *arg, Error **errp)abort (); } } + +SGXInfo *qmp_query_sgx (Error **errp) +{ +error_setg (errp,QERR_FEATURE_DISABLED ,"query-sgx" ); +return NULL ; +}
qapi/misc.json +42 -0
45 lines not on this page · raw message
tests/qtest/qmp-cmd-test.c +1 -0
@@ -100,6 +100,7 @@ static bool query_is_ignored(const char *cmd)/* Success depends on Host or Hypervisor SEV support */ "query-sev" ,"query-sev-capabilities" , +"query-sgx" , NULL };int i;
Signature · 1 line
2.29.2.334.gfaefdd61ec
Re: [RESEND PATCH 25/32] qmp: Add query-sgx command
Quoted 13 lines · Yang Zhong
This QMP query command can be used by some userspaces to retrieve the SGX information when SGX is enabled on Intel platform.
Signed-off-by Yang Zhong <[email protected]>--- monitor/qmp-cmds.c | 6 ++++++ qapi/misc.json | 42 ++++++++++++++++++++++++++++++++++++++ tests/qtest/qmp-cmd-test.c | 1 + 3 files changed, 49 insertions(+)+++
b/qapi/misc.json
qapi/misc.json line 528
Quoted 10 lines
qapi/misc.json +7
-0@@ -519,3 +519,45 @@ 'data': { '*option': 'str' }, 'returns': ['CommandLineOptionInfo'], 'allow-preconfig': true } + +## +# @SGXInfo: +# +# Information about intel Safe Guard eXtension (SGX) support +# +# @sgx: true if SGX is support
is supported
qapi/misc.json line 534
Quoted 6 lines
qapi/misc.json +6
-0+# +# @sgx1: true if SGX1 is support +# +# @sgx2: true if SGX2 is support +# +# @flc: true if FLC is support
again, for each of these too.
qapi/misc.json line 538
Quoted 4 lines
qapi/misc.json +4
-0+# +# @section-size: The EPC section size for guest +# +# Since: 5.1
6.1 (5.1 was last year).
Quoted 6 lines
qapi/misc.json +6
-0+## +{ 'struct': 'SGXInfo', + 'data': { 'sgx': 'bool', + 'sgx1': 'bool', + 'sgx2': 'bool', + 'flc': 'bool',
qapi/misc.json line 554
Quoted 10 lines
qapi/misc.json +10
-0+ 'section-size': 'uint64'}} + +## +# @query-sgx: +# +# Returns information about SGX +# +# Returns: @SGXInfo +# +# Since: 5.1
6.1
Quoted 22 lines
qapi/misc.json +9
-0+# +# Example: +# +# -> { "execute": "query-sgx" } +# <- { "return": { "sgx": true, "sgx1" : true, "sgx2" : true, +# "flc": true, "section-size" : 0 } } +# +## +{ 'command': 'query-sgx', 'returns': 'SGXInfo' }tests/qtest/qmp-cmd-test.c +1
-0@@ -100,6 +100,7 @@ static bool query_is_ignored(const char *cmd)/* Success depends on Host or Hypervisor SEV support */ "query-sev" ,"query-sev-capabilities" , +"query-sgx" , NULL };int i;
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [RESEND PATCH 25/32] qmp: Add query-sgx command
Quoted 12 lines · Eric Blake
Quoted 10 lines · Yang Zhong
This QMP query command can be used by some userspaces to retrieve the SGX information when SGX is enabled on Intel platform.
Signed-off-by Yang Zhong <[email protected]>--- monitor/qmp-cmds.c | 6 ++++++ qapi/misc.json | 42 ++++++++++++++++++++++++++++++++++++++ tests/qtest/qmp-cmd-test.c | 1 + 3 files changed, 49 insertions(+)
Quoted 15 lines
qapi/misc.json line 528
Quoted 12 lines
+++
b/qapi/misc.jsonqapi/misc.json +7
-0@@ -519,3 +519,45 @@ 'data': { '*option': 'str' }, 'returns': ['CommandLineOptionInfo'], 'allow-preconfig': true } + +## +# @SGXInfo: +# +# Information about intel Safe Guard eXtension (SGX) support +# +# @sgx: true if SGX is supportis supported
Eric, thanks for your comments! I will change this mistake. thanks!
Yang
Quoted 8 lines
qapi/misc.json line 534
Quoted 6 lines
qapi/misc.json +6
-0+# +# @sgx1: true if SGX1 is support +# +# @sgx2: true if SGX2 is support +# +# @flc: true if FLC is supportagain, for each of these too.
Ditto, thanks!
Yang
Quoted 8 lines
qapi/misc.json line 538
Quoted 5 lines
qapi/misc.json +4
-0+# +# @section-size: The EPC section size for guest +# +# Since: 5.16.1 (5.1 was last year).
Yes, i will change this version, thanks!
Yang
Quoted 10 lines
Quoted 10 lines
qapi/misc.json +10
-0+## +{ 'struct': 'SGXInfo', + 'data': { 'sgx': 'bool', + 'sgx1': 'bool', + 'sgx2': 'bool', + 'flc': 'bool', + 'section-size': 'uint64'}} + +## +# @query-sgx:
Quoted 8 lines
qapi/misc.json line 554
Quoted 6 lines
qapi/misc.json +6
-0+# +# Returns information about SGX +# +# Returns: @SGXInfo +# +# Since: 5.16.1
Ditto, thanks!
Yang
Quoted 28 lines
Quoted 23 lines
qapi/misc.json +9
-0+# +# Example: +# +# -> { "execute": "query-sgx" } +# <- { "return": { "sgx": true, "sgx1" : true, "sgx2" : true, +# "flc": true, "section-size" : 0 } } +# +## +{ 'command': 'query-sgx', 'returns': 'SGXInfo' }tests/qtest/qmp-cmd-test.c +1
-0@@ -100,6 +100,7 @@ static bool query_is_ignored(const char *cmd)/* Success depends on Host or Hypervisor SEV support */ "query-sev" ,"query-sev-capabilities" , +"query-sgx" , NULL };int i;Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH RESEND 26/32 hmp: Add 'info sgx' command
In reply to Yang ZhongThe command can be used to show the SGX information in the monitor when SGX is enabled on intel platform.
--- hmp-commands-info.hx | 15 +++++++++++++++ include/monitor/hmp.h | 1 + monitor/hmp-cmds.c | 6 ++++++ 3 files changed, 22 insertions(+)
hmp-commands-info.hx +15 -0
@@ -880,3 +880,18 @@ SRST ``info replay`` Display the record/replay information: mode and the current icount. ERST + +#if defined(TARGET_I386) + { + .name = "sgx", + .args_type = "", + .params = "", + .help = "show intel SGX information", + .cmd = hmp_info_sgx, + }, +#endif + +SRST + ``info sgx`` + Show intel SGX information. +ERST
include/monitor/hmp.h +1 -0
@@ -129,5 +129,6 @@ void hmp_info_replay(Monitor *mon, const QDict *qdict);void hmp_replay_break (Monitor *mon,const QDict *qdict);void hmp_replay_delete_break (Monitor *mon,const QDict *qdict);void hmp_replay_seek (Monitor *mon,const QDict *qdict); +void hmp_info_sgx (Monitor *mon,const QDict *qdict);#endif
monitor/hmp-cmds.c +6 -0
@@ -2226,3 +2226,9 @@ void hmp_info_memory_size_summary(Monitor *mon, const QDict *qdict) }hmp_handle_error (mon, err); } + +void hmp_info_sgx (Monitor *mon,const QDict *qdict) +{ +error_setg (errp,QERR_FEATURE_DISABLED ,"query-sgx" ); +return NULL ; +}
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 27/32 i386: Add sgx_get_info() interface
In reply to Yang ZhongAdd the sgx_get_info() interface for hmp and QMP usage, which will get the SGX info from this API.
--- hw/i386/sgx-epc.c | 22 ++++++++++++++++++++++ include/hw/i386/pc.h | 1 + include/hw/i386/sgx-epc.h | 1 + monitor/hmp-cmds.c | 20 ++++++++++++++++++-- monitor/qmp-cmds.c | 12 ++++++++++-- stubs/meson.build | 1 + stubs/sgx-stub.c | 7 +++++++ 7 files changed, 60 insertions(+), 4 deletions(-) create mode 100644 stubs/sgx-stub.c
hw/i386/sgx-epc.c +22 -0
@@ -322,6 +322,28 @@ void pc_machine_init_sgx_epc(PCMachineState *pcms)memory_region_set_size (&sgx_epc->mr, sgx_epc->size); } +SGXInfo *sgx_get_info (void ) +{ +SGXInfo *info; + + info =g_new0 (SGXInfo,1 ); +if (sgx_epc_enabled) { +PCMachineState *pcms =PC_MACHINE (qdev_get_machine ()); +SGXEPCState *sgx_epc = pcms->sgx_epc; + + info->sgx = true; + info->sgx1 = true; + info->sgx2 = true; + info->flc = true; + +if (sgx_epc) { + info->section_size = sgx_epc->size; + } + } + +return info; +} +static QemuOptsList sgx_epc_opts = { .name ="sgx-epc" , .implied_opt_name ="id" ,
include/hw/i386/pc.h +1 -0
@@ -204,6 +204,7 @@ void pc_madt_cpu_entry(AcpiDeviceIf *adev, int uid,void pc_machine_init_sgx_epc (PCMachineState *pcms);void sgx_memory_backend_reset (HostMemoryBackend *backend,int fd,Error **errp); +SGXInfo *sgx_get_info (void );extern GlobalProperty pc_compat_5_2[];extern const size_t pc_compat_5_2_len;
include/hw/i386/sgx-epc.h +1 -0
@@ -13,6 +13,7 @@#define QEMU_SGX_EPC_H #include "sysemu/hostmem.h" +#include "qapi/qapi-types-misc.h" #define TYPE_SGX_EPC "sgx-epc"#define SGX_EPC (obj) \
monitor/hmp-cmds.c +18 -2
@@ -2229,6 +2229,22 @@ void hmp_info_memory_size_summary(Monitor *mon, const QDict *qdict)void hmp_info_sgx (Monitor *mon,const QDict *qdict) { -error_setg (errp,QERR_FEATURE_DISABLED ,"query-sgx" ); -return NULL ; +SGXInfo *info =qmp_query_sgx (NULL ); + +if (info && info->sgx) { +monitor_printf (mon,"SGX support: %s\n" , + info->sgx ?"enabled" :"disabled" ); +monitor_printf (mon,"SGX1 support: %s\n" , + info->sgx1 ?"enabled" :"disabled" ); +monitor_printf (mon,"SGX2 support: %s\n" , + info->sgx2 ?"enabled" :"disabled" ); +monitor_printf (mon,"FLC support: %s\n" , + info->flc ?"enabled" :"disabled" ); +monitor_printf (mon,"size: %" PRIu64"\n" , + info->section_size); + }else { +monitor_printf (mon,"SGX is not enabled\n" ); + } + +qapi_free_SGXInfo (info); }
monitor/qmp-cmds.c +10 -2
@@ -40,6 +40,7 @@#include "qapi/qmp/qerror.h" #include "hw/mem/memory-device.h" #include "hw/acpi/acpi_dev_interface.h" +#include "hw/i386/pc.h" NameInfo *qmp_query_name (Error **errp) {
@@ -354,6 +355,13 @@ void qmp_display_reload(DisplayReloadOptions *arg, Error **errp)SGXInfo *qmp_query_sgx (Error **errp) { -error_setg (errp,QERR_FEATURE_DISABLED ,"query-sgx" ); -return NULL ; +SGXInfo *info; + + info =sgx_get_info (); +if (!info) { +error_setg (errp,"SGX features are not available" ); +return NULL ; + } + +return info; }
stubs/meson.build +1 -0
@@ -54,3 +54,4 @@ if have_system else stub_ss.add(files('qdev.c')) endif +stub_ss.add(files('sgx-stub.c'))
stubs/sgx-stub.c new file +7 -0
@@ -0,0 +1,7 @@ +#include "qemu/osdep.h" +#include "hw/i386/pc.h" + +SGXInfo *sgx_get_info (void ) +{ +return NULL ; +}
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 28/32 bitops: Support 32 and 64 bit mask macro
In reply to Yang ZhongThe Qemu should enable bit mask macro like Linux did in the kernel, the GENMASK(h, l) and GENMASK_ULL(h, l) will set the bit to 1 from l to h bit in the 32 bit or 64 bit long type.
--- include/qemu/bitops.h | 7 +++++++ 1 file changed, 7 insertions(+)
include/qemu/bitops.h +7 -0
@@ -18,6 +18,7 @@#define BITS_PER_BYTE CHAR_BIT#define BITS_PER_LONG (sizeof (unsigned long) * BITS_PER_BYTE) +#define BITS_PER_LONG_LONG 64#define BIT (nr) (1UL << (nr))#define BIT_ULL (nr) (1ULL << (nr))
@@ -28,6 +29,12 @@#define MAKE_64BIT_MASK (shift, length) \ (((~0ULL) >> (64 - (length))) << (shift)) +#define GENMASK (h, l) \ + (((~0UL) << (l)) & (~0UL >> (BITS_PER_LONG - 1 - (h)))) + +#define GENMASK_ULL (h, l) \ + (((~0ULL) << (l)) & (~0ULL >> (BITS_PER_LONG_LONG - 1 - (h)))) +/** * set_bit - Set a bit in memory * @nr: the bit to set
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 29/32 qmp: Add the qmp_query_sgx_capabilities()
In reply to Yang ZhongThe libvirt can use qmp_query_sgx_capabilities() to get the host sgx capabilitis.
--- hw/i386/sgx-epc.c | 66 ++++++++++++++++++++++++++++++++++++++ include/hw/i386/pc.h | 1 + monitor/qmp-cmds.c | 5 +++ qapi/misc.json | 19 +++++++++++ stubs/sgx-stub.c | 5 +++ tests/qtest/qmp-cmd-test.c | 1 + 6 files changed, 97 insertions(+)
hw/i386/sgx-epc.c +66 -0
@@ -27,6 +27,14 @@uint32_t epc_num; +#define SGX_MAX_EPC_SECTIONS 8 +#define SGX_CPUID_EPC_INVALID 0x0 + +/* A valid EPC section. */ +#define SGX_CPUID_EPC_SECTION 0x1 + +#define SGX_CPUID_EPC_MASK GENMASK(3, 0) +static Property sgx_epc_properties[] = {DEFINE_PROP_UINT64 (SGX_EPC_ADDR_PROP , SGXEPCDevice, addr,0 ), DEFINE_PROP_LINK(SGX_EPC_MEMDEV_PROP, SGXEPCDevice, hostmem,
@@ -344,6 +352,64 @@ SGXInfo *sgx_get_info(void)return info; } +static uint64_t sgx_calc_section_metric (uint64_t low,uint64_t high) +{ +return (low &GENMASK_ULL (31 ,12 )) + + ((high &GENMASK_ULL (19 ,0 )) <<32 ); +} + +static uint64_t sgx_calc_host_epc_section_size (void ) +{ +uint32_t i, type; +uint32_t eax, ebx, ecx, edx; +uint64_t size =0 ; + +for (i =0 ; i <SGX_MAX_EPC_SECTIONS ; i++) { +host_cpuid (0x12 , i +2 , &eax, &ebx, &ecx, &edx); + + type = eax &SGX_CPUID_EPC_MASK ; +if (type ==SGX_CPUID_EPC_INVALID ) { +break ; + } + +if (type !=SGX_CPUID_EPC_SECTION ) { +break ; + } + + size +=sgx_calc_section_metric (ecx, edx); + } + +return size; +} + +SGXInfo *sgx_get_capabilities (Error **errp) +{ +SGXInfo *info =NULL ; +uint32_t eax, ebx, ecx, edx; + +int fd =qemu_open_old ("/dev/sgx_vepc" ,O_RDWR ); +if (fd <0 ) { +error_setg (errp,"SGX is not enabled in KVM" ); +return NULL ; + } + + info =g_new0 (SGXInfo,1 ); +host_cpuid (0x7 ,0 , &eax, &ebx, &ecx, &edx); + + info->sgx = ebx & (1U <<2 ) ? true : false; + info->flc = ecx & (1U <<30 ) ? true : false; + +host_cpuid (0x12 ,0 , &eax, &ebx, &ecx, &edx); + info->sgx1 = eax & (1U <<0 ) ? true : false; + info->sgx2 = eax & (1U <<1 ) ? true : false; + + info->section_size =sgx_calc_host_epc_section_size (); + +close (fd); + +return info; +} +static QemuOptsList sgx_epc_opts = { .name ="sgx-epc" , .implied_opt_name ="id" ,
include/hw/i386/pc.h +1 -0
@@ -205,6 +205,7 @@ void pc_machine_init_sgx_epc(PCMachineState *pcms);void sgx_memory_backend_reset (HostMemoryBackend *backend,int fd,Error **errp);SGXInfo *sgx_get_info (void ); +SGXInfo *sgx_get_capabilities (Error **errp);extern GlobalProperty pc_compat_5_2[];extern const size_t pc_compat_5_2_len;
monitor/qmp-cmds.c +5 -0
@@ -365,3 +365,8 @@ SGXInfo *qmp_query_sgx(Error **errp)return info; } + +SGXInfo *qmp_query_sgx_capabilities (Error **errp) +{ +return sgx_get_capabilities (errp); +}
qapi/misc.json +19 -0
@@ -561,3 +561,22 @@ # ## { 'command': 'query-sgx', 'returns': 'SGXInfo' } + + +## +# @query-sgx-capabilities: +# +# Returns information from host SGX capabilities +# +# Returns: @SGXInfo +# +# Since: 5.1 +# +# Example: +# +# -> { "execute": "query-sgx-capabilities" } +# <- { "return": { "sgx": true, "sgx1" : true, "sgx2" : true, +# "flc": true, "section-size" : 0 } } +# +## +{ 'command': 'query-sgx-capabilities', 'returns': 'SGXInfo' }
stubs/sgx-stub.c +5 -0
@@ -5,3 +5,8 @@ SGXInfo *sgx_get_info(void) {return NULL ; } + +SGXInfo *sgx_get_capabilities (Error **errp) +{ +return NULL ; +}
tests/qtest/qmp-cmd-test.c +1 -0
@@ -101,6 +101,7 @@ static bool query_is_ignored(const char *cmd)"query-sev" ,"query-sev-capabilities" ,"query-sgx" , +"query-sgx-capabilities" , NULL };int i;
Signature · 1 line
2.29.2.334.gfaefdd61ec
Re: [RESEND PATCH 29/32] qmp: Add the qmp_query_sgx_capabilities()
Quoted 1 line · Yang Zhong
The libvirt can use
qmp_query_sgx_capabilities()to get the host
s/The libvirt/Libvirt/
Quoted 2 lines
sgx capabilitis.
capabilities
Quoted 16 lines
Signed-off-by Yang Zhong <[email protected]>--- hw/i386/sgx-epc.c | 66 ++++++++++++++++++++++++++++++++++++++ include/hw/i386/pc.h | 1 + monitor/qmp-cmds.c | 5 +++ qapi/misc.json | 19 +++++++++++ stubs/sgx-stub.c | 5 +++ tests/qtest/qmp-cmd-test.c | 1 + 6 files changed, 97 insertions(+)+++
b/qapi/misc.jsonqapi/misc.json +0
-0@@ -561,3 +561,22 @@ # ## { 'command': 'query-sgx', 'returns': 'SGXInfo' }
qapi/misc.json line 573
Quoted 10 lines
qapi/misc.json +10
-0+ + +## +# @query-sgx-capabilities: +# +# Returns information from host SGX capabilities +# +# Returns: @SGXInfo +# +# Since: 5.1
6.1
Quoted 12 lines
qapi/misc.json +9
-0+# +# Example: +# +# -> { "execute": "query-sgx-capabilities" } +# <- { "return": { "sgx": true, "sgx1" : true, "sgx2" : true, +# "flc": true, "section-size" : 0 } } +# +## +{ 'command': 'query-sgx-capabilities', 'returns': 'SGXInfo' }stubs/sgx-stub.c +0
-1--- a/stubs/sgx-stub.c
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [RESEND PATCH 29/32] qmp: Add the qmp_query_sgx_capabilities()
Quoted 10 lines · Eric Blake
Quoted 1 line · Yang Zhong
The libvirt can use
qmp_query_sgx_capabilities()to get the hosts/The libvirt/Libvirt/
Quoted 2 lines
sgx capabilitis.
capabilities
Eric, thanks for your comments! I will change those mistakes, thanks!
Yang
Quoted 20 lines
Quoted 20 lines
Signed-off-by Yang Zhong <[email protected]>--- hw/i386/sgx-epc.c | 66 ++++++++++++++++++++++++++++++++++++++ include/hw/i386/pc.h | 1 + monitor/qmp-cmds.c | 5 +++ qapi/misc.json | 19 +++++++++++ stubs/sgx-stub.c | 5 +++ tests/qtest/qmp-cmd-test.c | 1 + 6 files changed, 97 insertions(+)+++
b/qapi/misc.jsonqapi/misc.json +4
-0@@ -561,3 +561,22 @@ # ## { 'command': 'query-sgx', 'returns': 'SGXInfo' } + + +## +# @query-sgx-capabilities:
Quoted 9 lines
qapi/misc.json line 573
Quoted 6 lines
qapi/misc.json +6
-0+# +# Returns information from host SGX capabilities +# +# Returns: @SGXInfo +# +# Since: 5.16.1
Ditto, thanks!
Yang
Quoted 16 lines
Quoted 12 lines
qapi/misc.json +9
-0+# +# Example: +# +# -> { "execute": "query-sgx-capabilities" } +# <- { "return": { "sgx": true, "sgx1" : true, "sgx2" : true, +# "flc": true, "section-size" : 0 } } +# +## +{ 'command': 'query-sgx-capabilities', 'returns': 'SGXInfo' }stubs/sgx-stub.c +0
-1--- a/stubs/sgx-stub.cSignature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH RESEND 30/32 Kconfig: Add CONFIG_SGX support
In reply to Yang ZhongAdd new CONFIG_SGX for sgx support in the Qemu, and the Kconfig default enable sgx in the i386 platform.
--- backends/meson.build | 2 +- default-configs/devices/i386-softmmu.mak | 1 + hw/i386/Kconfig | 5 +++++ hw/i386/meson.build | 2 +- hw/i386/sgx-stub.c | 13 +++++++++++++ 5 files changed, 21 insertions(+), 2 deletions(-) create mode 100644 hw/i386/sgx-stub.c
backends/meson.build +1 -1
@@ -16,6 +16,6 @@ softmmu_ss.add(when: ['CONFIG_VHOST_USER', 'CONFIG_VIRTIO'], if_true: files('vho softmmu_ss.add(when: 'CONFIG_VIRTIO_CRYPTO', if_true: files('cryptodev-vhost.c')) softmmu_ss.add(when: ['CONFIG_VIRTIO_CRYPTO', 'CONFIG_VHOST_CRYPTO'], if_true: files('cryptodev-vhost-user.c')) softmmu_ss.add(when: 'CONFIG_GIO', if_true: [files('dbus-vmstate.c'), gio]) -softmmu_ss.add(when: 'CONFIG_LINUX', if_true: files('hostmem-epc.c')) +softmmu_ss.add(when: 'CONFIG_SGX', if_true: files('hostmem-epc.c')) subdir('tpm')
default-configs/devices/i386-softmmu.mak +1 -0
@@ -22,6 +22,7 @@ #CONFIG_TPM_CRB=n #CONFIG_TPM_TIS_ISA=n #CONFIG_VTD=n +#CONFIG_SGX=n # Boards: #
hw/i386/Kconfig +5 -0
@@ -2,6 +2,10 @@ config SEV bool depends on KVM +config SGX + bool + depends on KVM + config PC bool imply APPLESMC
@@ -17,6 +21,7 @@ config PC imply PVPANIC_ISA imply QXL imply SEV + imply SGX imply SGA imply TEST_DEVICES imply TPM_CRB
hw/i386/meson.build +1 -1
@@ -5,7 +5,6 @@ i386_ss.add(files( 'e820_memory_layout.c', 'multiboot.c', 'x86.c', - 'sgx-epc.c', )) i386_ss.add(when: 'CONFIG_X86_IOMMU', if_true: files('x86-iommu.c'),
@@ -17,6 +16,7 @@ i386_ss.add(when: 'CONFIG_Q35', if_true: files('pc_q35.c')) i386_ss.add(when: 'CONFIG_VMMOUSE', if_true: files('vmmouse.c')) i386_ss.add(when: 'CONFIG_VMPORT', if_true: files('vmport.c')) i386_ss.add(when: 'CONFIG_VTD', if_true: files('intel_iommu.c')) +i386_ss.add(when: 'CONFIG_SGX', if_true: files('sgx-epc.c'), if_false: files('sgx-stub.c')) i386_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-common.c')) i386_ss.add(when: 'CONFIG_ACPI_HW_REDUCED', if_true: files('generic_event_device_x86.c'))
hw/i386/sgx-stub.c new file +13 -0
@@ -0,0 +1,13 @@ +#include "qemu/osdep.h" +#include "hw/i386/pc.h" +#include "hw/i386/sgx-epc.h" + +void pc_machine_init_sgx_epc (PCMachineState *pcms) +{ +return ; +} + +int sgx_epc_get_section (int section_nr,uint64_t *addr,uint64_t *size) +{ +return 1 ; +}
Signature · 1 line
2.29.2.334.gfaefdd61ec
PATCH RESEND 31/32 sgx-epc: Add the fill_device_info() callback support
In reply to Yang ZhongSince there is no fill_device_info() callback support, and when we execute "info memory-devices" command in the monitor, the segfault will be found.
This patch will add this callback support and "info memory-devices" will show sgx epc memory exposed to guest. The result as below:
qemu) info memory-devices
Memory device [sgx-epc]: "epc1"
memaddr: 0x180000000
size: 29360128
memdev: /objects/mem1
Memory device [sgx-epc]: "epc2"
memaddr: 0x181c00000
size: 10485760
memdev: /objects/mem2
--- hw/i386/sgx-epc.c | 17 ++++++++++++++++- monitor/hmp-cmds.c | 10 ++++++++++ qapi/machine.json | 26 +++++++++++++++++++++++++- 3 files changed, 51 insertions(+), 2 deletions(-)
hw/i386/sgx-epc.c +16 -1
@@ -208,7 +208,22 @@ static MemoryRegion *sgx_epc_md_get_memory_region(MemoryDeviceState *md,static void sgx_epc_md_fill_device_info (const MemoryDeviceState *md,MemoryDeviceInfo *info) { -/* TODO */ +SgxEPCDeviceInfo *se =g_new0 (SgxEPCDeviceInfo,1 ); +SGXEPCDevice *epc =SGX_EPC (md); +const DeviceState *dev =DEVICE (md); + +if (dev->id) { + se->has_id = true; + se->id =g_strdup (dev->id); + } + + se->memaddr = epc->addr; + se->size =object_property_get_uint (OBJECT (epc),SGX_EPC_SIZE_PROP , +NULL ); + se->memdev =object_get_canonical_path (OBJECT (epc->hostmem)); + + info->u.sgx_epc.data = se; + info->type =MEMORY_DEVICE_INFO_KIND_SGX_EPC ; }static void sgx_epc_class_init (ObjectClass *oc,void *data)
monitor/hmp-cmds.c +10 -0
@@ -1819,6 +1819,7 @@ void hmp_info_memory_devices(Monitor *mon, const QDict *qdict)VirtioMEMDeviceInfo *vmi;MemoryDeviceInfo *value;PCDIMMDeviceInfo *di; +SgxEPCDeviceInfo *se;for (info = info_list; info; info = info->next) { value = info->value;
@@ -1866,6 +1867,15 @@ void hmp_info_memory_devices(Monitor *mon, const QDict *qdict) vmi->block_size);monitor_printf (mon," memdev: %s\n" , vmi->memdev);break ; +case MEMORY_DEVICE_INFO_KIND_SGX_EPC : + se = value->u.sgx_epc.data; +monitor_printf (mon,"Memory device [%s]: \"%s\"\n" , +MemoryDeviceInfoKind_str (value->type), + se->id ? se->id :"" ); +monitor_printf (mon," memaddr: 0x%" PRIx64"\n" , se->memaddr); +monitor_printf (mon," size: %" PRIu64"\n" , se->size); +monitor_printf (mon," memdev: %s\n" , se->memdev); +break ;default :g_assert_not_reached (); }
qapi/machine.json +25 -1
@@ -1184,6 +1184,29 @@ } } +## +# @SgxEPCDeviceInfo: +# +# Sgx EPC state information +# +# @id: device's ID +# +# @memaddr: physical address in memory, where device is mapped +# +# @size: size of memory that the device provides +# +# @memdev: memory backend linked with device +# +# Since: 5.1 +## +{ 'struct': 'SgxEPCDeviceInfo', + 'data': { '*id': 'str', + 'memaddr': 'size', + 'size': 'size', + 'memdev': 'str' + } +} + ## # @MemoryDeviceInfo: #
@@ -1198,7 +1221,8 @@ 'data': { 'dimm': 'PCDIMMDeviceInfo', 'nvdimm': 'PCDIMMDeviceInfo', 'virtio-pmem': 'VirtioPMEMDeviceInfo', - 'virtio-mem': 'VirtioMEMDeviceInfo' + 'virtio-mem': 'VirtioMEMDeviceInfo', + 'sgx-epc': 'SgxEPCDeviceInfo' } }
Signature · 1 line
2.29.2.334.gfaefdd61ec
Re: [RESEND PATCH 31/32] sgx-epc: Add the fill_device_info() callback support
Quoted 29 lines · Yang Zhong
Since there is no
fill_device_info()callback support, and when we execute "info memory-devices" command in the monitor, the segfault will be found.This patch will add this callback support and "info memory-devices" will show sgx epc memory exposed to guest. The result as below:
qemu) info memory-devices
Memory device [sgx-epc]: "epc1"
memaddr:0x180000000
size: 29360128
memdev: /objects/mem1
Memory device [sgx-epc]: "epc2"
memaddr:0x181c00000
size: 10485760
memdev: /objects/mem2Signed-off-by Yang Zhong <[email protected]>---
+++
b/qapi/machine.jsonqapi/machine.json +4
-0@@ -1184,6 +1184,29 @@ } } +## +# @SgxEPCDeviceInfo: +# +# Sgx EPC state information
qapi/machine.json line 1200
Quoted 10 lines
qapi/machine.json +10
-0+# +# @id: device's ID +# +# @memaddr: physical address in memory, where device is mapped +# +# @size: size of memory that the device provides +# +# @memdev: memory backend linked with device +# +# Since: 5.1
6.1
Quoted 23 lines
qapi/machine.json +11
-1+## +{ 'struct': 'SgxEPCDeviceInfo', + 'data': { '*id': 'str', + 'memaddr': 'size', + 'size': 'size', + 'memdev': 'str' + } +} + ## # @MemoryDeviceInfo: #@@ -1198,7 +1221,8 @@ 'data': { 'dimm': 'PCDIMMDeviceInfo', 'nvdimm': 'PCDIMMDeviceInfo', 'virtio-pmem': 'VirtioPMEMDeviceInfo', - 'virtio-mem': 'VirtioMEMDeviceInfo' + 'virtio-mem': 'VirtioMEMDeviceInfo', + 'sgx-epc': 'SgxEPCDeviceInfo' } }
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [RESEND PATCH 31/32] sgx-epc: Add the fill_device_info() callback support
Quoted 26 lines · Eric Blake
Quoted 25 lines · Yang Zhong
Since there is no
fill_device_info()callback support, and when we execute "info memory-devices" command in the monitor, the segfault will be found.This patch will add this callback support and "info memory-devices" will show sgx epc memory exposed to guest. The result as below:
qemu) info memory-devices
Memory device [sgx-epc]: "epc1"
memaddr:0x180000000
size: 29360128
memdev: /objects/mem1
Memory device [sgx-epc]: "epc2"
memaddr:0x181c00000
size: 10485760
memdev: /objects/mem2Signed-off-by Yang Zhong <[email protected]>---
+++
b/qapi/machine.json(quoted diff) +0
-0@@ -1184,6 +1184,29 @@ } }
Quoted 16 lines
Quoted 4 lines
qapi/machine.json +4
-0+## +# @SgxEPCDeviceInfo: +# +# Sgx EPC state informationqapi/machine.json line 1200
Quoted 10 lines
qapi/machine.json +10
-0+# +# @id: device's ID +# +# @memaddr: physical address in memory, where device is mapped +# +# @size: size of memory that the device provides +# +# @memdev: memory backend linked with device +# +# Since: 5.16.1
Eric, thanks for your comments, i will change this in the next version, thanks!
Yang
Quoted 29 lines
Quoted 24 lines
qapi/machine.json +11
-1+## +{ 'struct': 'SgxEPCDeviceInfo', + 'data': { '*id': 'str', + 'memaddr': 'size', + 'size': 'size', + 'memdev': 'str' + } +} + ## # @MemoryDeviceInfo: #@@ -1198,7 +1221,8 @@ 'data': { 'dimm': 'PCDIMMDeviceInfo', 'nvdimm': 'PCDIMMDeviceInfo', 'virtio-pmem': 'VirtioPMEMDeviceInfo', - 'virtio-mem': 'VirtioMEMDeviceInfo' + 'virtio-mem': 'VirtioMEMDeviceInfo', + 'sgx-epc': 'SgxEPCDeviceInfo' } }Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH RESEND 32/32 doc: Add the SGX doc
In reply to Yang ZhongFrom: Sean Christopherson <[email protected]>
--- docs/intel-sgx.txt | 173 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 173 insertions(+) create mode 100644 docs/intel-sgx.txt
docs/intel-sgx.txt new file +173 -0
@@ -0,0 +1,173 @@ +=============================== +Software Guard eXtensions (SGX) +=============================== + +Overview +======== + +Intel Software Guard eXtensions (SGX) is a set of instructions and mechanisms +for memory accesses in order to provide security accesses for sensitive +applications and data. SGX allows an application to use it's pariticular +address space as an *enclave*, which is a protected area provides confidentiality +and integrity even in the presence of privileged malware. Accesses to the +enclave memory area from any software not resident in the enclave are prevented, +including those from privileged software. + +Virtual SGX +=========== + +SGX feature is exposed to guest via SGX CPUID. Looking at SGX CPUID, we can +report the same CPUID info to guest as on host for most of SGX CPUID. With +reporting the same CPUID guest is able to use full capacity of SGX, and KVM +doesn't need to emulate those info. + +The guest's EPC base and size are determined by Qemu, and KVM needs Qemu to +notify such info to it before it can initialize SGX for guest. + +Virtual EPC +----------- + +By default, Qemu does not assign EPC to a VM, i.e. fully enabling SGX in a VM +requires explicit allocation of EPC to the VM. Similar to other specialized +memory types, e.g. hugetlbfs, EPC is exposed as a memory backend. For a number +of reasons, a EPC memory backend can only be realized via an 'sgx-epc' device. +Standard memory backend options such as prealloc are supported by EPC. + +SGX EPC is enumerated through CPUID, i.e. EPC "devices" need to be realized +prior to realizing the vCPUs themselves, which occurs long before generic +devices are parsed and realized. Because of this, 'sgx-epc' devices must be +created via the dedicated -sgx-epc command, i.e. cannot be created through +the generic -devices command. On the plus side, this limitation means that +EPC does not require -maxmem as EPC is not treated as {cold,hot}plugged memory. + +Qemu does not artificially restrict the number of EPC sections exposed to a +guest, e.g. Qemu will happily allow you to create 64 1M EPC sections. Be aware +that some kernels may not recognize all EPC sections, e.g. the Linux SGX driver +is hardwired to support only 8 EPC sections. + +The following Qemu snippet creates two EPC sections, with 64M pre-allocated +to the VM and an additional 28M mapped but not allocated: + + -object memory-backend-epc,id=mem1,size=64M,prealloc=on \ + -sgx-epc id=epc1,memdev=mem1 \ + -object memory-backend-epc,id=mem2,size=28M \ + -sgx-epc id=epc2,memdev=mem2 + +Note: + +The size and location of the virtual EPC are far less restricted compared +to physical EPC. Because physical EPC is protected via range registers, +the size of the physical EPC must be a power of two (though software sees +a subset of the full EPC, e.g. 92M or 128M) and the EPC must be naturally +aligned. KVM SGX's virtual EPC is purely a software construct and only +requires the size and location to be page aligned. Qemu enforces the EPC +size is a multiple of 4k and will ensure the base of the EPC is 4k aligned. +To simplify the implementation, EPC is always located above 4g in the guest +physical address space. + +Migration +--------- + +Qemu/KVM doesn't prevent live migrating SGX VMs, although from hardware's +perspective, SGX doesn't support live migration, since both EPC and the SGX +key hierarchy are bound to the physical platform. However live migration +can be supported in the sense if guest software stack can support recreating +enclaves when it suffers sudden lose of EPC; and if guest enclaves can detect +SGX keys being changed, and handle gracefully. For instance, when ERESUME fails +with #PF.SGX, guest software can gracefully detect it and recreate enclaves; +and when enclave fails to unseal sensitive information from outside, it can +detect such error and sensitive information can be provisioned to it again. + +CPUID +----- + +Due to its myriad dependencies, SGX is currently not listed as supported +in any of Qemu's built-in CPU configuration. To expose SGX (and SGX Launch +Control) to a guest, you must either use `-cpu host` to pass-through the +host CPU model, or explicitly enable SGX when using a built-in CPU model, +e.g. via `-cpu <model>,+sgx` or `-cpu <model>,+sgx,+sgxlc`. + +All SGX sub-features enumerated through CPUID, e.g. SGX2, MISCSELECT, +ATTRIBUTES, etc... can be restricted via CPUID flags. Be aware that enforcing +restriction of MISCSELECT, ATTRIBUTES and XFRM requires intercepting ECREATE, +i.e. may marginally reduce SGX performance in the guest. All SGX sub-features +controlled via -cpu are prefixed with "sgx", e.g.: + +$ qemu-system-x86_64 -cpu help | xargs printf "%s\n" | grep sgx + sgx + sgx-debug + sgx-encls-c + sgx-enclv + sgx-exinfo + sgx-kss + sgx-mode64 + sgx-provisionkey + sgx-tokenkey + sgx1 + sgx2 + sgxlc + +The following Qemu snippet passes through the host CPU (and host physical +address width) but restricts access to the provision and EINIT token keys: + + -cpu host,host-phys-bits,-sgx-provisionkey,-sgx-tokenkey + +Note: + +SGX sub-features cannot be emulated, i.e. sub-features that are not present +in hardware cannot be forced on via '-cpu'. + +Virtualize SGX Launch Control +----------------------------- + +Qemu SGX support for Launch Control (LC) is passive, in the sense that it +does not actively change the LC configuration. Qemu SGX provides the user +the ability to set/clear the CPUID flag (and by extension the associated +IA32_FEATURE_CONTROL MSR bit in fw_cfg) and saves/restores the LE Hash MSRs +when getting/putting guest state, but Qemu does not add new controls to +directly modify the LC configuration. Similar to hardware behavior, locking +the LC configuration to a non-Intel value is left to guest firmware. Unlike +host bios setting for SGX launch control(LC), there is no special bios setting +for SGX guest by our design. If host is in locked mode, we can still allow +creating VM with SGX. + +Feature Control +--------------- + +Qemu SGX updates the `etc/msr_feature_control` fw_cfg entry to set the SGX +(bit 18) and SGX LC (bit 17) flags based on their respective CPUID support, +i.e. existing guest firmware will automatically set SGX and SGX LC accordingly, +assuming said firmware supports fw_cfg.msr_feature_control. + +Launch a guest +============== + +To launch a SGX guest +${QEMU} \ + -cpu host,+sgx-provisionkey \ + -object memory-backend-epc,id=mem1,size=64M,prealloc=on \ + -sgx-epc id=epc1,memdev=mem1 \ + -object memory-backend-epc,id=mem2,size=28M \ + -sgx-epc id=epc2,memdev=mem2 + +Utilizing SGX in the guest requires a kernel/OS with SGX support. + +The support can be determined in guest by: +$ grep sgx /proc/cpuinfo + +Check the SGX epc info in the Guest: +$ dmesg | grep sgx +[ 1.242142] sgx: EPC section 0x180000000-0x181bfffff +[ 1.242319] sgx: EPC section 0x181c00000-0x1837fffff + +References +========== + +SGX Homepage: +https://software.intel.com/sgx + +SGX SDK: +https://github.com/intel/linux-sgx.git + +SGX SPEC: +Intel SDM Volume 3
Signature · 1 line
2.29.2.334.gfaefdd61ec