HSM driver for ACRN hypervisor
Subsystems (3)
v7 cover HSM driver for ACRN hypervisor
From: Shuo Liu <[email protected]>
ACRN is a Type 1 reference hypervisor stack, running directly on the bare-metal hardware, and is suitable for a variety of IoT and embedded device solutions.
ACRN implements a hybrid VMM architecture, using a privileged Service VM. The Service VM manages the system resources (CPU, memory, etc.) and I/O devices of User VMs. Multiple User VMs are supported, with each of them running Linux, Android OS or Windows. Both Service VM and User VMs are guest VM.
Below figure shows the architecture.
Service VM User VM
+----------------------------+ | +------------------+
| +--------------+ | | | |
| |ACRN userspace| | | | |
| +--------------+ | | | |
|-----------------ioctl------| | | | ...
|kernel space +----------+ | | | |
| | HSM | | | | Drivers |
| +----------+ | | | |
+--------------------|-------+ | +------------------+
+---------------------hypercall----------------------------------------+
| ACRN Hypervisor |
+----------------------------------------------------------------------+
| Hardware |
+----------------------------------------------------------------------+There is only one Service VM which could run Linux as OS.
In a typical case, the Service VM will be auto started when ACRN Hypervisor is booted. Then the ACRN userspace (an application running in Service VM) could be used to start/stop User VMs by communicating with ACRN Hypervisor Service Module (HSM).
ACRN Hypervisor Service Module (HSM) is a middle layer that allows the ACRN userspace and Service VM OS kernel to communicate with ACRN Hypervisor and manage different User VMs. This middle layer provides the following functionalities,
- Issues hypercalls to the hypervisor to manage User VMs:
* VM/vCPU management
* Memory management
* Device passthrough
* Interrupts injection
- I/O requests handling from User VMs.
- Exports ioctl through HSM char device.
- Exports function calls for other kernel modules
ACRN is focused on embedded system. So it doesn't support some features. E.g.,
- ACRN doesn't support VM migration.
- ACRN doesn't support vCPU migration.
This patch set adds the HSM to the Linux kernel.
I also added a simple example to launch a small guest (with several instructions as payload) on ACRN with demonstration ioctl usage.
The basic ARCN support was merged to upstream already.
https://lore.kernel.org/lkml/[email protected]/
ChangeLog:
v7:
- Typo fixed in documentation. (Boris)
- Verified some reserved fields of ioctls. (Greg)
- Rebased to v5.11-rc2.
v6:
- Added the cpuid.rst documentation (Boris)
- Deleted exported acrn_is_privileged_vm(), user detects feature bits by cpuid_eax() directly (Boris)
- Used 'g' as the hcall_id constrain in hypercall definitions (Boris, Segher)
- Removed unnecessary reserved fields from structures. Sorted some fields for alignment (Greg)
- Used built-in kernel guid_t types (Greg)
- Specified the endian of some fields in user/kernel interface structures (Greg)
- Removed the alignment attribute from user/kernel interface structures (Greg)
- Set reserved fields to zero (Greg)
- Added a ioctl interface usage sample in the last patch (Greg)
- Used pin_user_pages*() instead of get_user_pages*().
v5:
- Corrected typo in documentation.
- Removed unused pr_fmt().
- Used supported constraint with a explicit MOV to R8 at beginning of ASM for hypercall interface.
- Used dev_dbg() to replace dev_err() in places which might cause a DoS.
- Introduced acrn_vm_list_lock as a mutex for friendly review.
- Changed to use default attribute group list to add attribute files.
v4:
- Used acrn_dev.this_device directly for dev_*() (Reinette)
- Removed the odd usage of {get|put}_device() on &acrn_dev->this_device (Greg)
- Removed unused log code. (Greg)
- Corrected the return error values. (Greg)
- Mentioned that HSM relies hypervisor for sanity check in acrn_dev_ioctl() comments (Greg)
v3:
- Used {get|put}_device() helpers on &acrn_dev->this_device
- Moved unused code from front patches to later ones.
- Removed self-defined pr_fmt() and dev_fmt()
- Provided comments for acrn_vm_list_lock.
v2:
- Removed API version related code. (Dave)
- Replaced pr_*() by dev_*(). (Greg)
- Used -ENOTTY as the error code of unsupported ioctl. (Greg)
Shuo Liu (17):
docs: acrn: Introduce ACRN
x86/acrn: Introduce acrn_{setup, remove}_intr_handler() x86/acrn: Introduce hypercall interfaces
virt: acrn: Introduce ACRN HSM basic driver
virt: acrn: Introduce VM management interfaces
virt: acrn: Introduce an ioctl to set vCPU registers state
virt: acrn: Introduce EPT mapping management
virt: acrn: Introduce I/O request management
virt: acrn: Introduce PCI configuration space PIO accesses combiner
virt: acrn: Introduce interfaces for PCI device passthrough
virt: acrn: Introduce interrupt injection interfaces
virt: acrn: Introduce interfaces to query C-states and P-states
allowed by hypervisor
virt: acrn: Introduce I/O ranges operation interfaces
virt: acrn: Introduce ioeventfd
virt: acrn: Introduce irqfd
virt: acrn: Introduce an interface for Service VM to control vCPU sample/acrn: Introduce a sample of HSM ioctl interface usage
Yin Fengwei (1):
x86/acrn: Introduce acrn_cpuid_base() and hypervisor feature bits
.../userspace-api/ioctl/ioctl-number.rst | 1 + Documentation/virt/acrn/cpuid.rst | 46 ++ Documentation/virt/acrn/index.rst | 12 + Documentation/virt/acrn/introduction.rst | 43 ++ Documentation/virt/acrn/io-request.rst | 97 +++ Documentation/virt/index.rst | 1 + MAINTAINERS | 9 + arch/x86/include/asm/acrn.h | 78 +++ arch/x86/kernel/cpu/acrn.c | 16 +- drivers/virt/Kconfig | 2 + drivers/virt/Makefile | 1 + drivers/virt/acrn/Kconfig | 15 + drivers/virt/acrn/Makefile | 3 + drivers/virt/acrn/acrn_drv.h | 227 ++++++ drivers/virt/acrn/hsm.c | 445 ++++++++++++ drivers/virt/acrn/hypercall.h | 254 +++++++ drivers/virt/acrn/ioeventfd.c | 273 ++++++++ drivers/virt/acrn/ioreq.c | 645 ++++++++++++++++++ drivers/virt/acrn/irqfd.c | 235 +++++++ drivers/virt/acrn/mm.c | 306 +++++++++ drivers/virt/acrn/vm.c | 126 ++++ include/uapi/linux/acrn.h | 479 +++++++++++++ samples/acrn/Makefile | 12 + samples/acrn/guest.ld | 9 + samples/acrn/payload.ld | 9 + samples/acrn/vm-sample.c | 136 ++++ 26 files changed, 3479 insertions(+), 1 deletion(-) create mode 100644 Documentation/virt/acrn/cpuid.rst create mode 100644 Documentation/virt/acrn/index.rst create mode 100644 Documentation/virt/acrn/introduction.rst create mode 100644 Documentation/virt/acrn/io-request.rst create mode 100644 arch/x86/include/asm/acrn.h create mode 100644 drivers/virt/acrn/Kconfig create mode 100644 drivers/virt/acrn/Makefile create mode 100644 drivers/virt/acrn/acrn_drv.h create mode 100644 drivers/virt/acrn/hsm.c create mode 100644 drivers/virt/acrn/hypercall.h create mode 100644 drivers/virt/acrn/ioeventfd.c create mode 100644 drivers/virt/acrn/ioreq.c create mode 100644 drivers/virt/acrn/irqfd.c create mode 100644 drivers/virt/acrn/mm.c create mode 100644 drivers/virt/acrn/vm.c create mode 100644 include/uapi/linux/acrn.h create mode 100644 samples/acrn/Makefile create mode 100644 samples/acrn/guest.ld create mode 100644 samples/acrn/payload.ld create mode 100644 samples/acrn/vm-sample.c
base-commit: 9f1abbe97c08ba7ed609791627533a805a1b2c66
Signature · 1 line
2.28.0
PATCH v7 1/18 docs: acrn: Introduce ACRN
From: Shuo Liu <[email protected]>
Add documentation on the following aspects of ACRN:
1) A brief introduction on the architecture of ACRN.
2) I/O request handling in ACRN.
3) CPUID functions of ACRN.
To learn more about ACRN, please go to ACRN project website
https://projectacrn.org, or the documentation page
https://projectacrn.github.io/.
--- Documentation/virt/acrn/cpuid.rst | 46 +++++++++++ Documentation/virt/acrn/index.rst | 12 +++ Documentation/virt/acrn/introduction.rst | 43 +++++++++++ Documentation/virt/acrn/io-request.rst | 97 ++++++++++++++++++++++++ Documentation/virt/index.rst | 1 + MAINTAINERS | 7 ++ 6 files changed, 206 insertions(+) create mode 100644 Documentation/virt/acrn/cpuid.rst create mode 100644 Documentation/virt/acrn/index.rst create mode 100644 Documentation/virt/acrn/introduction.rst create mode 100644 Documentation/virt/acrn/io-request.rst
Documentation/virt/acrn/cpuid.rst new file +46 -0
@@ -0,0 +1,46 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +ACRN CPUID bits +=============== + +A guest VM running on an ACRN hypervisor, can check some of its features using +CPUID. + +ACRN cpuid functions are: + +function: 0x40000000 + +returns:: + + eax = 0x40000010 + ebx = 0x4e524341 + ecx = 0x4e524341 + edx = 0x4e524341 + +Note that this value in ebx, ecx and edx corresponds to the string +"ACRNACRNACRN". The value in eax corresponds to the maximum cpuid function +present in this leaf, and will be updated if more functions are added in the +future. + +function: define ACRN_CPUID_FEATURES (0x40000001) + +returns:: + + ebx, ecx, edx + eax = an OR'ed group of (1 << flag) + +where ``flag`` is defined as below: + +================================= =========== ================================ +flag value meaning +================================= =========== ================================ +ACRN_FEATURE_PRIVILEGED_VM 0 guest VM is a privileged VM +================================= =========== ================================ + +function: 0x40000010 + +returns:: + + ebx, ecx, edx + eax = (Virtual) TSC frequency in kHz.
Documentation/virt/acrn/index.rst new file +12 -0
@@ -0,0 +1,12 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +ACRN Hypervisor +=============== + +.. toctree:: + :maxdepth: 1 + + introduction + io-request + cpuid
Documentation/virt/acrn/introduction.rst new file +43 -0
43 lines not on this page · raw message
Documentation/virt/acrn/io-request.rst new file +97 -0
97 lines not on this page · raw message
Documentation/virt/index.rst +1 -0
@@ -12,6 +12,7 @@ Linux Virtualization Support paravirt_ops guest-halt-polling ne_overview + acrn/index .. only:: html and subproject
MAINTAINERS +7 -0
@@ -436,6 +436,13 @@ S: Orphan F: drivers/platform/x86/wmi.c F: include/uapi/linux/wmi.h +ACRN HYPERVISOR SERVICE MODULE +M: Shuo Liu <[email protected]> +L: [email protected] +S: Supported +W: https://projectacrn.org +F: Documentation/virt/acrn/ + AD1889 ALSA SOUND DRIVER L: [email protected] S: Maintained
Signature · 1 line
2.28.0
Randy Dunlap
reviewerRe: [PATCH v7 01/18] docs: acrn: Introduce ACRN
Hi,
Quoted 45 lines · On 1/5/21 11:50 PM, [email protected] wrote:
From: Shuo Liu <[email protected]>
Add documentation on the following aspects of ACRN:
1) A brief introduction on the architecture of ACRN.
2) I/O request handling in ACRN.
3) CPUID functions of ACRN.To learn more about ACRN, please go to ACRN project website
https://projectacrn.org, or the documentation page
https://projectacrn.github.io/.Signed-off-by Shuo Liu <[email protected]>Reviewed-by Zhi Wang <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Reviewed-by Borislav Petkov <[email protected]>Cc Dave Hansen <[email protected]>Cc Sen Christopherson <[email protected]>Cc Dan Williams <[email protected]>Cc Fengwei Yin <[email protected]>Cc Zhi Wang <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Yu Wang <[email protected]>Cc Reinette Chatre <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Randy Dunlap <[email protected]>--- Documentation/virt/acrn/cpuid.rst | 46 +++++++++++ Documentation/virt/acrn/index.rst | 12 +++ Documentation/virt/acrn/introduction.rst | 43 +++++++++++ Documentation/virt/acrn/io-request.rst | 97 ++++++++++++++++++++++++ Documentation/virt/index.rst | 1 + MAINTAINERS | 7 ++ 6 files changed, 206 insertions(+) create mode 100644 Documentation/virt/acrn/cpuid.rst create mode 100644 Documentation/virt/acrn/index.rst create mode 100644 Documentation/virt/acrn/introduction.rst create mode 100644 Documentation/virt/acrn/io-request.rstDocumentation/virt/acrn/cpuid.rst new file +0
-0
Documentation/virt/acrn/cpuid.rst line 7
Quoted 7 lines
Documentation/virt/acrn/cpuid.rst new file +7
-0@@ -0,0 +1,46 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +ACRN CPUID bits +=============== + +A guest VM running on an ACRN hypervisor, can check some of its features using
comma not needed/not wanted.
Quoted 6 lines
Documentation/virt/acrn/cpuid.rst new file +6
-0+CPUID. + +ACRN cpuid functions are: + +function: 0x40000000 +
Documentation/virt/acrn/cpuid.rst line 23
Quoted 10 lines
Documentation/virt/acrn/cpuid.rst new file +10
-0+returns:: + + eax = 0x40000010 + ebx = 0x4e524341 + ecx = 0x4e524341 + edx = 0x4e524341 + +Note that this value in ebx, ecx and edx corresponds to the string +"ACRNACRNACRN". The value in eax corresponds to the maximum cpuid function +present in this leaf, and will be updated if more functions are added in the
except for the 0x4000000 bit?
Quoted 171 lines
Documentation/virt/acrn/cpuid.rst new file +23
-0+future. + +function: define ACRN_CPUID_FEATURES (0x40000001) + +returns:: + + ebx, ecx, edx + eax = an OR'ed group of (1 << flag) + +where ``flag`` is defined as below: + +================================= =========== ================================ +flag value meaning +================================= =========== ================================ +ACRN_FEATURE_PRIVILEGED_VM 0 guest VM is a privileged VM +================================= =========== ================================ + +function: 0x40000010 + +returns:: + + ebx, ecx, edx + eax = (Virtual) TSC frequency in kHz.Documentation/virt/acrn/index.rst new file +12
-0@@ -0,0 +1,12 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +ACRN Hypervisor +=============== + +.. toctree:: + :maxdepth: 1 + + introduction + io-request + cpuidDocumentation/virt/acrn/introduction.rst new file +43
-0@@ -0,0 +1,43 @@ +.. SPDX-License-Identifier: GPL-2.0 + +ACRN Hypervisor Introduction +============================ + +The ACRN Hypervisor is a Type 1 hypervisor, running directly on bare-metal +hardware. It has a privileged management VM, called Service VM, to manage User +VMs and do I/O emulation. + +ACRN userspace is an application running in the Service VM that emulates +devices for a User VM based on command line configurations. ACRN Hypervisor +Service Module (HSM) is a kernel module in the Service VM which provides +hypervisor services to the ACRN userspace. + +Below figure shows the architecture. + +:: + + Service VM User VM + +----------------------------+ | +------------------+ + | +--------------+ | | | | + | |ACRN userspace| | | | | + | +--------------+ | | | | + |-----------------ioctl------| | | | ... + |kernel space +----------+ | | | | + | | HSM | | | | Drivers | + | +----------+ | | | | + +--------------------|-------+ | +------------------+ + +---------------------hypercall----------------------------------------+ + | ACRN Hypervisor | + +----------------------------------------------------------------------+ + | Hardware | + +----------------------------------------------------------------------+ + +ACRN userspace allocates memory for the User VM, configures and initializes the +devices used by the User VM, loads the virtual bootloader, initializes the +virtual CPU state and handles I/O request accesses from the User VM. It uses +ioctls to communicate with the HSM. HSM implements hypervisor services by +interacting with the ACRN Hypervisor via hypercalls. HSM exports a char device +interface (/dev/acrn_hsm) to userspace. + +The ACRN hypervisor is open for contribution from anyone. The source repo is +available at https://github.com/projectacrn/acrn-hypervisor.Documentation/virt/acrn/io-request.rst new file +75
-0@@ -0,0 +1,97 @@ +.. SPDX-License-Identifier: GPL-2.0 + +I/O request handling +==================== + +An I/O request of a User VM, which is constructed by the hypervisor, is +distributed by the ACRN Hypervisor Service Module to an I/O client +corresponding to the address range of the I/O request. Details of I/O request +handling are described in the following sections. + +1. I/O request +-------------- + +For each User VM, there is a shared 4-KByte memory region used for I/O requests +communication between the hypervisor and Service VM. An I/O request is a +256-byte structure buffer, which is 'struct acrn_io_request', that is filled by +an I/O handler of the hypervisor when a trapped I/O access happens in a User +VM. ACRN userspace in the Service VM first allocates a 4-KByte page and passes +the GPA (Guest Physical Address) of the buffer to the hypervisor. The buffer is +used as an array of 16 I/O request slots with each I/O request slot being 256 +bytes. This array is indexed by vCPU ID. + +2. I/O clients +-------------- + +An I/O client is responsible for handling User VM I/O requests whose accessed +GPA falls in a certain range. Multiple I/O clients can be associated with each +User VM. There is a special client associated with each User VM, called the +default client, that handles all I/O requests that do not fit into the range of +any other clients. The ACRN userspace acts as the default client for each User +VM. + +Below illustration shows the relationship between I/O requests shared buffer, +I/O requests and I/O clients. + +:: + + +------------------------------------------------------+ + | Service VM | + |+--------------------------------------------------+ | + || +----------------------------------------+ | | + || | shared page ACRN userspace | | | + || | +-----------------+ +------------+ | | | + || +----+->| acrn_io_request |<-+ default | | | | + || | | | +-----------------+ | I/O client | | | | + || | | | | ... | +------------+ | | | + || | | | +-----------------+ | | | + || | +-|--------------------------------------+ | | + ||---|----|-----------------------------------------| | + || | | kernel | | + || | | +----------------------+ | | + || | | | +-------------+ HSM | | | + || | +--------------+ | | | | + || | | | I/O clients | | | | + || | | | | | | | + || | | +-------------+ | | | + || | +----------------------+ | | + |+---|----------------------------------------------+ | + +----|-------------------------------------------------+ + | + +----|-------------------------------------------------+ + | +-+-----------+ | + | | I/O handler | ACRN Hypervisor | + | +-------------+ | + +------------------------------------------------------+ + +3. I/O request state transition +------------------------------- + +The state transitions of an ACRN I/O request are as follows. + +:: + + FREE -> PENDING -> PROCESSING -> COMPLETE -> FREE -> ... +
Documentation/virt/acrn/io-request.rst line 85
Quoted 10 lines
Documentation/virt/acrn/io-request.rst new file +10
-0+- FREE: this I/O request slot is empty +- PENDING: a valid I/O request is pending in this slot +- PROCESSING: the I/O request is being processed +- COMPLETE: the I/O request has been processed + +An I/O request in COMPLETE or FREE state is owned by the hypervisor. HSM and +ACRN userspace are in charge of processing the others. + +4. Processing flow of I/O requests +-------------------------------
add 3 more '-' above, please.
Quoted 17 lines
Documentation/virt/acrn/io-request.rst new file +12
-0+ +a. The I/O handler of the hypervisor will fill an I/O request with PENDING + state when a trapped I/O access happens in a User VM. +b. The hypervisor makes an upcall, which is a notification interrupt, to + the Service VM. +c. The upcall handler schedules a tasklet to dispatch I/O requests. +d. The tasklet looks for the PENDING I/O requests, assigns them to different + registered clients based on the address of the I/O accesses, updates + their state to PROCESSING, and notifies the corresponding client to handle. +e. The notified client handles the assigned I/O requests. +f. The HSM updates I/O requests states to COMPLETE and notifies the hypervisor + of the completion via hypercalls.Documentation/virt/index.rst +0
-0
Documentation/virt/index.rst Linux Virtualization Support · line 17
Quoted 6 lines
Documentation/virt/index.rst +1
-0@@ -12,6 +12,7 @@ Linux Virtualization Support paravirt_ops guest-halt-polling ne_overview + acrn/index .. only:: html and subproject
thanks for the docs.
Signature · 3 lines
~Randy You can't do anything without having to do something else first. -- Belefant's Law
Re: [PATCH v7 01/18] docs: acrn: Introduce ACRN
Hi Randy,
Thanks for review.
Quoted 40 lines · Randy Dunlap
Quoted 39 lines · On 1/5/21 11:50 PM, [email protected] wrote:
From: Shuo Liu <[email protected]>
Add documentation on the following aspects of ACRN:
1) A brief introduction on the architecture of ACRN.
2) I/O request handling in ACRN.
3) CPUID functions of ACRN.To learn more about ACRN, please go to ACRN project website
https://projectacrn.org, or the documentation page
https://projectacrn.github.io/.Signed-off-by Shuo Liu <[email protected]>Reviewed-by Zhi Wang <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Reviewed-by Borislav Petkov <[email protected]>Cc Dave Hansen <[email protected]>Cc Sen Christopherson <[email protected]>Cc Dan Williams <[email protected]>Cc Fengwei Yin <[email protected]>Cc Zhi Wang <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Yu Wang <[email protected]>Cc Reinette Chatre <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Randy Dunlap <[email protected]>--- Documentation/virt/acrn/cpuid.rst | 46 +++++++++++ Documentation/virt/acrn/index.rst | 12 +++ Documentation/virt/acrn/introduction.rst | 43 +++++++++++ Documentation/virt/acrn/io-request.rst | 97 ++++++++++++++++++++++++ Documentation/virt/index.rst | 1 + MAINTAINERS | 7 ++ 6 files changed, 206 insertions(+) create mode 100644 Documentation/virt/acrn/cpuid.rst create mode 100644 Documentation/virt/acrn/index.rst create mode 100644 Documentation/virt/acrn/introduction.rst create mode 100644 Documentation/virt/acrn/io-request.rst
Quoted 15 lines
Documentation/virt/acrn/cpuid.rst line 7
Quoted 13 lines
Documentation/virt/acrn/cpuid.rst new file +7
-0@@ -0,0 +1,46 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +ACRN CPUID bits +=============== + +A guest VM running on an ACRN hypervisor, can check some of its features usingcomma not needed/not wanted.
Removed.
Quoted 11 lines
Quoted 11 lines
Documentation/virt/acrn/cpuid.rst new file +10
-0+CPUID. + +ACRN cpuid functions are: + +function: 0x40000000 + +returns:: + + eax = 0x40000010 + ebx = 0x4e524341
Quoted 8 lines
Documentation/virt/acrn/cpuid.rst line 23
Quoted 6 lines
Documentation/virt/acrn/cpuid.rst new file +6
-0+ ecx = 0x4e524341 + edx = 0x4e524341 + +Note that this value in ebx, ecx and edx corresponds to the string +"ACRNACRNACRN". The value in eax corresponds to the maximum cpuid function +present in this leaf, and will be updated if more functions are added in theexcept for the
0x4000000bit?
It's the maximum cpuid function value. So it's 0x40000010.
Quoted 176 lines
Quoted 176 lines
Documentation/virt/acrn/cpuid.rst new file +23
-0+future. + +function: define ACRN_CPUID_FEATURES (0x40000001) + +returns:: + + ebx, ecx, edx + eax = an OR'ed group of (1 << flag) + +where ``flag`` is defined as below: + +================================= =========== ================================ +flag value meaning +================================= =========== ================================ +ACRN_FEATURE_PRIVILEGED_VM 0 guest VM is a privileged VM +================================= =========== ================================ + +function: 0x40000010 + +returns:: + + ebx, ecx, edx + eax = (Virtual) TSC frequency in kHz.Documentation/virt/acrn/index.rst new file +12
-0@@ -0,0 +1,12 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +ACRN Hypervisor +=============== + +.. toctree:: + :maxdepth: 1 + + introduction + io-request + cpuidDocumentation/virt/acrn/introduction.rst new file +43
-0@@ -0,0 +1,43 @@ +.. SPDX-License-Identifier: GPL-2.0 + +ACRN Hypervisor Introduction +============================ + +The ACRN Hypervisor is a Type 1 hypervisor, running directly on bare-metal +hardware. It has a privileged management VM, called Service VM, to manage User +VMs and do I/O emulation. + +ACRN userspace is an application running in the Service VM that emulates +devices for a User VM based on command line configurations. ACRN Hypervisor +Service Module (HSM) is a kernel module in the Service VM which provides +hypervisor services to the ACRN userspace. + +Below figure shows the architecture. + +:: + + Service VM User VM + +----------------------------+ | +------------------+ + | +--------------+ | | | | + | |ACRN userspace| | | | | + | +--------------+ | | | | + |-----------------ioctl------| | | | ... + |kernel space +----------+ | | | | + | | HSM | | | | Drivers | + | +----------+ | | | | + +--------------------|-------+ | +------------------+ + +---------------------hypercall----------------------------------------+ + | ACRN Hypervisor | + +----------------------------------------------------------------------+ + | Hardware | + +----------------------------------------------------------------------+ + +ACRN userspace allocates memory for the User VM, configures and initializes the +devices used by the User VM, loads the virtual bootloader, initializes the +virtual CPU state and handles I/O request accesses from the User VM. It uses +ioctls to communicate with the HSM. HSM implements hypervisor services by +interacting with the ACRN Hypervisor via hypercalls. HSM exports a char device +interface (/dev/acrn_hsm) to userspace. + +The ACRN hypervisor is open for contribution from anyone. The source repo is +available at https://github.com/projectacrn/acrn-hypervisor.Documentation/virt/acrn/io-request.rst new file +79
-0@@ -0,0 +1,97 @@ +.. SPDX-License-Identifier: GPL-2.0 + +I/O request handling +==================== + +An I/O request of a User VM, which is constructed by the hypervisor, is +distributed by the ACRN Hypervisor Service Module to an I/O client +corresponding to the address range of the I/O request. Details of I/O request +handling are described in the following sections. + +1. I/O request +-------------- + +For each User VM, there is a shared 4-KByte memory region used for I/O requests +communication between the hypervisor and Service VM. An I/O request is a +256-byte structure buffer, which is 'struct acrn_io_request', that is filled by +an I/O handler of the hypervisor when a trapped I/O access happens in a User +VM. ACRN userspace in the Service VM first allocates a 4-KByte page and passes +the GPA (Guest Physical Address) of the buffer to the hypervisor. The buffer is +used as an array of 16 I/O request slots with each I/O request slot being 256 +bytes. This array is indexed by vCPU ID. + +2. I/O clients +-------------- + +An I/O client is responsible for handling User VM I/O requests whose accessed +GPA falls in a certain range. Multiple I/O clients can be associated with each +User VM. There is a special client associated with each User VM, called the +default client, that handles all I/O requests that do not fit into the range of +any other clients. The ACRN userspace acts as the default client for each User +VM. + +Below illustration shows the relationship between I/O requests shared buffer, +I/O requests and I/O clients. + +:: + + +------------------------------------------------------+ + | Service VM | + |+--------------------------------------------------+ | + || +----------------------------------------+ | | + || | shared page ACRN userspace | | | + || | +-----------------+ +------------+ | | | + || +----+->| acrn_io_request |<-+ default | | | | + || | | | +-----------------+ | I/O client | | | | + || | | | | ... | +------------+ | | | + || | | | +-----------------+ | | | + || | +-|--------------------------------------+ | | + ||---|----|-----------------------------------------| | + || | | kernel | | + || | | +----------------------+ | | + || | | | +-------------+ HSM | | | + || | +--------------+ | | | | + || | | | I/O clients | | | | + || | | | | | | | + || | | +-------------+ | | | + || | +----------------------+ | | + |+---|----------------------------------------------+ | + +----|-------------------------------------------------+ + | + +----|-------------------------------------------------+ + | +-+-----------+ | + | | I/O handler | ACRN Hypervisor | + | +-------------+ | + +------------------------------------------------------+ + +3. I/O request state transition +------------------------------- + +The state transitions of an ACRN I/O request are as follows. + +:: + + FREE -> PENDING -> PROCESSING -> COMPLETE -> FREE -> ... + +- FREE: this I/O request slot is empty +- PENDING: a valid I/O request is pending in this slot +- PROCESSING: the I/O request is being processed +- COMPLETE: the I/O request has been processed
Quoted 8 lines
Documentation/virt/acrn/io-request.rst line 85
Quoted 6 lines
Documentation/virt/acrn/io-request.rst new file +6
-0+ +An I/O request in COMPLETE or FREE state is owned by the hypervisor. HSM and +ACRN userspace are in charge of processing the others. + +4. Processing flow of I/O requests +-------------------------------add 3 more '-' above, please.
Sure. Thanks.
shuo
PATCH v7 2/18 x86/acrn: Introduce acrn_{setup, remove}_intr_handler()
In reply to [email protected]From: Shuo Liu <[email protected]>
The ACRN Hypervisor builds an I/O request when a trapped I/O access happens in User VM. Then, ACRN Hypervisor issues an upcall by sending a notification interrupt to the Service VM. HSM in the Service VM needs to hook the notification interrupt to handle I/O requests.
Notification interrupts from ACRN Hypervisor are already supported and a, currently uninitialized, callback called.
Export two APIs for HSM to setup/remove its callback.
--- arch/x86/include/asm/acrn.h | 8 ++++++++ arch/x86/kernel/cpu/acrn.c | 14 ++++++++++++++ 2 files changed, 22 insertions(+) create mode 100644 arch/x86/include/asm/acrn.h
arch/x86/include/asm/acrn.h new file +8 -0
@@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_ACRN_H +#define _ASM_X86_ACRN_H + +void acrn_setup_intr_handler (void (*handler)(void )); +void acrn_remove_intr_handler (void ); + +#endif /* _ASM_X86_ACRN_H */
arch/x86/kernel/cpu/acrn.c +14 -0
@@ -10,6 +10,8 @@*/ #include <linux/interrupt.h> + +#include <asm/acrn.h> #include <asm/apic.h> #include <asm/cpufeatures.h> #include <asm/desc.h>
@@ -55,6 +57,18 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_acrn_hv_callback)set_irq_regs (old_regs); } +void acrn_setup_intr_handler (void (*handler)(void )) +{ + acrn_intr_handler = handler; +} +EXPORT_SYMBOL_GPL (acrn_setup_intr_handler); + +void acrn_remove_intr_handler (void ) +{ + acrn_intr_handler =NULL ; +} +EXPORT_SYMBOL_GPL (acrn_remove_intr_handler); +const __initconst struct hypervisor_x86 x86_hyper_acrn = { .name ="ACRN" , .detect = acrn_detect,
Signature · 1 line
2.28.0
PATCH v7 3/18 x86/acrn: Introduce acrn_cpuid_base() and hypervisor feature bits
In reply to [email protected]From: Yin Fengwei <[email protected]>
ACRN Hypervisor reports hypervisor features via CPUID leaf 0x40000001 which is similar to KVM. A VM can check if it's the privileged VM using the feature bits. The Service VM is the only privileged VM by design.
--- arch/x86/include/asm/acrn.h | 16 ++++++++++++++++ arch/x86/kernel/cpu/acrn.c | 2 +- 2 files changed, 17 insertions(+), 1 deletion(-)
arch/x86/include/asm/acrn.h +16 -0
@@ -2,7 +2,23 @@#ifndef _ASM_X86_ACRN_H#define _ASM_X86_ACRN_H +/* +* This CPUID returns feature bitmaps in EAX. +* Guest VM uses this to detect the appropriate feature bit. +*/ +#define ACRN_CPUID_FEATURES 0x40000001 +/* Bit 0 indicates whether guest VM is privileged */ +#define ACRN_FEATURE_PRIVILEGED_VM BIT(0) +void acrn_setup_intr_handler (void (*handler)(void ));void acrn_remove_intr_handler (void ); +static inline u32 acrn_cpuid_base (void ) +{ +if (boot_cpu_has (X86_FEATURE_HYPERVISOR )) +return hypervisor_cpuid_base ("ACRNACRNACRN" ,0 ); + +return 0 ; +} +#endif /* _ASM_X86_ACRN_H */
arch/x86/kernel/cpu/acrn.c +1 -1
@@ -21,7 +21,7 @@static u32 __init acrn_detect (void ) { -return hypervisor_cpuid_base ("ACRNACRNACRN" ,0 ); +return acrn_cpuid_base (); }static void __init acrn_init_platform (void )
Signature · 1 line
2.28.0
PATCH v7 4/18 x86/acrn: Introduce hypercall interfaces
In reply to [email protected]From: Shuo Liu <[email protected]>
The Service VM communicates with the hypervisor via conventional hypercalls. VMCALL instruction is used to make the hypercalls.
ACRN hypercall ABI:
* Hypercall number is in R8 register.
* Up to 2 parameters are in RDI and RSI registers.
* Return value is in RAX register.
Introduce the ACRN hypercall interfaces. Because GCC doesn't support R8 register as direct register constraints, use supported constraint as input with a explicit MOV to R8 in beginning of asm.
--- arch/x86/include/asm/acrn.h | 54 +++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+)
arch/x86/include/asm/acrn.h +54 -0
@@ -21,4 +21,58 @@ static inline u32 acrn_cpuid_base(void) return 0; } +/* +* Hypercalls for ACRN +* +* - VMCALL instruction is used to implement ACRN hypercalls. +* - ACRN hypercall ABI: +* - Hypercall number is passed in R8 register. +* - Up to 2 arguments are passed in RDI, RSI. +* - Return value will be placed in RAX. +* +* Because GCC doesn't support R8 register as direct register constraints, use +* supported constraint as input with a explicit MOV to R8 in beginning of asm. +*/ +static inline long acrn_hypercall0 (unsigned long hcall_id) +{ +long result; + + asmvolatile ("movl %1, %%r8d\n\t" +"vmcall\n\t" + :"=a" (result) + :"g" (hcall_id) + :"r8" ,"memory" ); + +return result; +} + +static inline long acrn_hypercall1 (unsigned long hcall_id, +unsigned long param1) +{ +long result; + + asmvolatile ("movl %1, %%r8d\n\t" +"vmcall\n\t" + :"=a" (result) + :"g" (hcall_id),"D" (param1) + :"r8" ,"memory" ); + +return result; +} + +static inline long acrn_hypercall2 (unsigned long hcall_id, +unsigned long param1, +unsigned long param2) +{ +long result; + + asmvolatile ("movl %1, %%r8d\n\t" +"vmcall\n\t" + :"=a" (result) + :"g" (hcall_id),"D" (param1),"S" (param2) + :"r8" ,"memory" ); + +return result; +} +#endif /* _ASM_X86_ACRN_H */
Signature · 1 line
2.28.0
PATCH v7 5/18 virt: acrn: Introduce ACRN HSM basic driver
In reply to [email protected]From: Shuo Liu <[email protected]>
ACRN Hypervisor Service Module (HSM) is a kernel module in Service VM which communicates with ACRN userspace through ioctls and talks to ACRN Hypervisor through hypercalls.
Add a basic HSM driver which allows Service VM userspace to communicate with ACRN. The following patches will add more ioctls, guest VM memory mapping caching, I/O request processing, ioeventfd and irqfd into this module. HSM exports a char device interface (/dev/acrn_hsm) to userspace.
--- MAINTAINERS | 1 + drivers/virt/Kconfig | 2 + drivers/virt/Makefile | 1 + drivers/virt/acrn/Kconfig | 14 ++++++ drivers/virt/acrn/Makefile | 3 ++ drivers/virt/acrn/acrn_drv.h | 18 ++++++++ drivers/virt/acrn/hsm.c | 87 ++++++++++++++++++++++++++++++++++++ 7 files changed, 126 insertions(+) create mode 100644 drivers/virt/acrn/Kconfig create mode 100644 drivers/virt/acrn/Makefile create mode 100644 drivers/virt/acrn/acrn_drv.h create mode 100644 drivers/virt/acrn/hsm.c
MAINTAINERS +1 -0
@@ -442,6 +442,7 @@ L: [email protected] S: Supported W: https://projectacrn.org F: Documentation/virt/acrn/ +F: drivers/virt/acrn/ AD1889 ALSA SOUND DRIVER L: [email protected]
drivers/virt/Kconfig +2 -0
@@ -34,4 +34,6 @@ config FSL_HV_MANAGER source "drivers/virt/vboxguest/Kconfig" source "drivers/virt/nitro_enclaves/Kconfig" + +source "drivers/virt/acrn/Kconfig" endif
drivers/virt/Makefile +1 -0
@@ -7,3 +7,4 @@ obj-$(CONFIG_FSL_HV_MANAGER) += fsl_hypervisor.o obj-y += vboxguest/ obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves/ +obj-$(CONFIG_ACRN_HSM) += acrn/
drivers/virt/acrn/Kconfig new file +14 -0
@@ -0,0 +1,14 @@ +# SPDX-License-Identifier: GPL-2.0 +config ACRN_HSM + tristate "ACRN Hypervisor Service Module" + depends on ACRN_GUEST + help + ACRN Hypervisor Service Module (HSM) is a kernel module which + communicates with ACRN userspace through ioctls and talks to + the ACRN Hypervisor through hypercalls. HSM will only run in + a privileged management VM, called Service VM, to manage User + VMs and do I/O emulation. Not required for simply running + under ACRN as a User VM. + + To compile as a module, choose M, the module will be called + acrn. If unsure, say N.
drivers/virt/acrn/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_ACRN_HSM) := acrn.o +acrn-y := hsm.o
drivers/virt/acrn/acrn_drv.h new file +18 -0
@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ACRN_HSM_DRV_H +#define __ACRN_HSM_DRV_H + +#include <linux/types.h> + +#define ACRN_INVALID_VMID (0xffffU) + +/** +* struct acrn_vm - Properties of ACRN User VM. +* @vmid: User VM ID +*/ +struct acrn_vm { +u16 vmid; +}; + +#endif /* __ACRN_HSM_DRV_H */
drivers/virt/acrn/hsm.c new file +87 -0
87 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH v7 6/18 virt: acrn: Introduce VM management interfaces
In reply to [email protected]From: Shuo Liu <[email protected]>
The VM management interfaces expose several VM operations to ACRN userspace via ioctls. For example, creating VM, starting VM, destroying VM and so on.
The ACRN Hypervisor needs to exchange data with the ACRN userspace during the VM operations. HSM provides VM operation ioctls to the ACRN userspace and communicates with the ACRN Hypervisor for VM operations via hypercalls.
HSM maintains a list of User VM. Each User VM will be bound to an existing file descriptor of /dev/acrn_hsm. The User VM will be destroyed when the file descriptor is closed.
--- .../userspace-api/ioctl/ioctl-number.rst | 1 + MAINTAINERS | 1 + drivers/virt/acrn/Makefile | 2 +- drivers/virt/acrn/acrn_drv.h | 21 ++++- drivers/virt/acrn/hsm.c | 76 +++++++++++++++++- drivers/virt/acrn/hypercall.h | 78 +++++++++++++++++++ drivers/virt/acrn/vm.c | 68 ++++++++++++++++ include/uapi/linux/acrn.h | 55 +++++++++++++ 8 files changed, 298 insertions(+), 4 deletions(-) create mode 100644 drivers/virt/acrn/hypercall.h create mode 100644 drivers/virt/acrn/vm.c create mode 100644 include/uapi/linux/acrn.h
Documentation/userspace-api/ioctl/ioctl-number.rst +1 -0
@@ -319,6 +319,7 @@ Code Seq# Include File Comments 0xA0 all linux/sdp/sdp.h Industrial Device Project <mailto:[email protected]> 0xA1 0 linux/vtpm_proxy.h TPM Emulator Proxy Driver +0xA2 all uapi/linux/acrn.h ACRN hypervisor 0xA3 80-8F Port ACL in development: <mailto:[email protected]> 0xA3 90-9F linux/dtlk.h
MAINTAINERS +1 -0
@@ -443,6 +443,7 @@ S: Supported W: https://projectacrn.org F: Documentation/virt/acrn/ F: drivers/virt/acrn/ +F: include/uapi/linux/acrn.h AD1889 ALSA SOUND DRIVER L: [email protected]
drivers/virt/acrn/Makefile +1 -1
@@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_ACRN_HSM) := acrn.o -acrn-y := hsm.o +acrn-y := hsm.o vm.o
drivers/virt/acrn/acrn_drv.h +20 -1
@@ -3,16 +3,35 @@#ifndef __ACRN_HSM_DRV_H#define __ACRN_HSM_DRV_H +#include <linux/acrn.h> +#include <linux/dev_printk.h> +#include <linux/miscdevice.h> #include <linux/types.h> +#include "hypercall.h" + +extern struct miscdevice acrn_dev; +#define ACRN_INVALID_VMID (0xffffU) +#define ACRN_VM_FLAG_DESTROYED 0U/** * struct acrn_vm - Properties of ACRN User VM. +* @list: Entry within global list of all VMs * @vmid: User VM ID +* @vcpu_num: Number of virtual CPUs in the VM +* @flags: Flags (ACRN_VM_FLAG_*) of the VM. This is VM flag management +* in HSM which is different from the &acrn_vm_creation.vm_flag. */ struct acrn_vm { -u16 vmid; +struct list_head list; +u16 vmid; +int vcpu_num; +unsigned long flags; }; +struct acrn_vm *acrn_vm_create (struct acrn_vm *vm, +struct acrn_vm_creation *vm_param); +int acrn_vm_destroy (struct acrn_vm *vm); +#endif /* __ACRN_HSM_DRV_H */
drivers/virt/acrn/hsm.c +74 -2
100 lines not on this page · raw message
drivers/virt/acrn/hypercall.h new file +78 -0
78 lines not on this page · raw message
drivers/virt/acrn/vm.c new file +68 -0
68 lines not on this page · raw message
include/uapi/linux/acrn.h new file +55 -0
55 lines not on this page · raw message
Signature · 1 line
2.28.0
Re: [PATCH v7 06/18] virt: acrn: Introduce VM management interfaces
Quoted 382 lines · On Wed, Jan 06, 2021 at 03:50:43PM +0800, [email protected] wrote:
From: Shuo Liu <[email protected]>
The VM management interfaces expose several VM operations to ACRN userspace via ioctls. For example, creating VM, starting VM, destroying VM and so on.
The ACRN Hypervisor needs to exchange data with the ACRN userspace during the VM operations. HSM provides VM operation ioctls to the ACRN userspace and communicates with the ACRN Hypervisor for VM operations via hypercalls.
HSM maintains a list of User VM. Each User VM will be bound to an existing file descriptor of
/dev/acrn_hsm. The User VM will be destroyed when the file descriptor is closed.Signed-off-by Shuo Liu <[email protected]>Reviewed-by Zhi Wang <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Cc Zhi Wang <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Yu Wang <[email protected]>Cc Reinette Chatre <[email protected]>Cc Greg Kroah-Hartman <[email protected]>--- .../userspace-api/ioctl/ioctl-number.rst | 1 + MAINTAINERS | 1 + drivers/virt/acrn/Makefile | 2 +- drivers/virt/acrn/acrn_drv.h | 21 ++++- drivers/virt/acrn/hsm.c | 76 +++++++++++++++++- drivers/virt/acrn/hypercall.h | 78 +++++++++++++++++++ drivers/virt/acrn/vm.c | 68 ++++++++++++++++ include/uapi/linux/acrn.h | 55 +++++++++++++ 8 files changed, 298 insertions(+), 4 deletions(-) create mode 100644 drivers/virt/acrn/hypercall.h create mode 100644 drivers/virt/acrn/vm.c create mode 100644 include/uapi/linux/acrn.hDocumentation/userspace-api/ioctl/ioctl-number.rst +1
-0@@ -319,6 +319,7 @@ Code Seq# Include File Comments 0xA0 all linux/sdp/sdp.h Industrial Device Project <mailto:[email protected]> 0xA1 0 linux/vtpm_proxy.h TPM Emulator Proxy Driver +0xA2 all uapi/linux/acrn.h ACRN hypervisor 0xA3 80-8F Port ACL in development: <mailto:[email protected]> 0xA3 90-9F linux/dtlk.hMAINTAINERS +1
-0@@ -443,6 +443,7 @@ S: Supported W: https://projectacrn.org F: Documentation/virt/acrn/ F: drivers/virt/acrn/ +F: include/uapi/linux/acrn.h AD1889 ALSA SOUND DRIVER L: [email protected]drivers/virt/acrn/Makefile +1
-1@@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_ACRN_HSM) := acrn.o -acrn-y := hsm.o +acrn-y := hsm.o vm.odrivers/virt/acrn/acrn_drv.h +20
-1@@ -3,16 +3,35 @@#ifndef __ACRN_HSM_DRV_H#define __ACRN_HSM_DRV_H +#include <linux/acrn.h> +#include <linux/dev_printk.h> +#include <linux/miscdevice.h> #include <linux/types.h> +#include "hypercall.h" + +extern struct miscdevice acrn_dev; +#define ACRN_INVALID_VMID (0xffffU) +#define ACRN_VM_FLAG_DESTROYED 0U/** * struct acrn_vm - Properties of ACRN User VM. +* @list: Entry within global list of all VMs * @vmid: User VM ID +* @vcpu_num: Number of virtual CPUs in the VM +* @flags: Flags (ACRN_VM_FLAG_*) of the VM. This is VM flag management +* in HSM which is different from the &acrn_vm_creation.vm_flag. */ struct acrn_vm { -u16 vmid; +struct list_head list; +u16 vmid; +int vcpu_num; +unsigned long flags; }; +struct acrn_vm *acrn_vm_create (struct acrn_vm *vm, +struct acrn_vm_creation *vm_param); +int acrn_vm_destroy (struct acrn_vm *vm); +#endif /* __ACRN_HSM_DRV_H */ drivers/virt/acrn/hsm.c +74
-2@@ -9,7 +9,6 @@* Yakui Zhao <[email protected]> */ -#include <linux/miscdevice.h> #include <linux/mm.h> #include <linux/module.h> #include <linux/slab.h> @@ -38,10 +37,82 @@ static int acrn_dev_open(struct inode *inode, struct file *filp)return 0 ; } +/* +* HSM relies on hypercall layer of the ACRN hypervisor to do the +* sanity check against the input parameters. +*/ +static long acrn_dev_ioctl (struct file *filp,unsigned int cmd, +unsigned long ioctl_param) +{ +struct acrn_vm *vm = filp->private_data; +struct acrn_vm_creation *vm_param; +int ret =0 ; + +if (vm->vmid ==ACRN_INVALID_VMID && cmd !=ACRN_IOCTL_CREATE_VM ) { +dev_dbg (acrn_dev.this_device, +"ioctl 0x%x: Invalid VM state!\n" , cmd); +return -EINVAL ; + } + +switch (cmd) { +case ACRN_IOCTL_CREATE_VM : + vm_param =memdup_user ((void __user *)ioctl_param, +sizeof (struct acrn_vm_creation )); +if (IS_ERR (vm_param)) +return PTR_ERR (vm_param); + +if ((vm_param->reserved0 | vm_param->reserved1) !=0 ) +return -EINVAL ; + + vm =acrn_vm_create (vm, vm_param); +if (!vm) { + ret = -EINVAL ; +kfree (vm_param); +break ; + } + +if (copy_to_user ((void __user *)ioctl_param, vm_param, +sizeof (struct acrn_vm_creation ))) { +acrn_vm_destroy (vm); + ret = -EFAULT ; + } + +kfree (vm_param); +break ; +case ACRN_IOCTL_START_VM : + ret =hcall_start_vm (vm->vmid); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to start VM %u!\n" , vm->vmid); +break ; +case ACRN_IOCTL_PAUSE_VM : + ret =hcall_pause_vm (vm->vmid); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to pause VM %u!\n" , vm->vmid); +break ; +case ACRN_IOCTL_RESET_VM : + ret =hcall_reset_vm (vm->vmid); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to restart VM %u!\n" , vm->vmid); +break ; +case ACRN_IOCTL_DESTROY_VM : + ret =acrn_vm_destroy (vm); +break ; +default : +dev_dbg (acrn_dev.this_device,"Unknown IOCTL 0x%x!\n" , cmd); + ret = -ENOTTY ; + } + +return ret; +} +static int acrn_dev_release (struct inode *inode,struct file *filp) {struct acrn_vm *vm = filp->private_data; +acrn_vm_destroy (vm);kfree (vm);return 0 ; }@@ -50,9 +121,10 @@ static const struct file_operations acrn_fops = { .owner = THIS_MODULE, .open = acrn_dev_open, .release = acrn_dev_release, + .unlocked_ioctl = acrn_dev_ioctl, }; -static struct miscdevice acrn_dev = { +struct miscdevice acrn_dev = { .minor =MISC_DYNAMIC_MINOR , .name ="acrn_hsm" , .fops = &acrn_fops,drivers/virt/acrn/hypercall.h new file +78
-0@@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* ACRN HSM: hypercalls of ACRN Hypervisor +*/ +#ifndef __ACRN_HSM_HYPERCALL_H +#define __ACRN_HSM_HYPERCALL_H +#include <asm/acrn.h> + +/* +* Hypercall IDs of the ACRN Hypervisor +*/ +#define _HC_ID (x, y) (((x) << 24) | (y)) + +#define HC_ID 0x80UL + +#define HC_ID_VM_BASE 0x10UL +#define HC_CREATE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x00) +#define HC_DESTROY_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x01) +#define HC_START_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x02) +#define HC_PAUSE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x03) +#define HC_RESET_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x05) + +/** +* hcall_create_vm() - Create a User VM +* @vminfo: Service VM GPA of info of User VM creation +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_create_vm (u64 vminfo) +{ +return acrn_hypercall1 (HC_CREATE_VM , vminfo); +} + +/** +* hcall_start_vm() - Start a User VM +* @vmid: User VM ID +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_start_vm (u64 vmid) +{ +return acrn_hypercall1 (HC_START_VM , vmid); +} + +/** +* hcall_pause_vm() - Pause a User VM +* @vmid: User VM ID +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_pause_vm (u64 vmid) +{ +return acrn_hypercall1 (HC_PAUSE_VM , vmid); +} + +/** +* hcall_destroy_vm() - Destroy a User VM +* @vmid: User VM ID +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_destroy_vm (u64 vmid) +{ +return acrn_hypercall1 (HC_DESTROY_VM , vmid); +} + +/** +* hcall_reset_vm() - Reset a User VM +* @vmid: User VM ID +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_reset_vm (u64 vmid) +{ +return acrn_hypercall1 (HC_RESET_VM , vmid); +} + +#endif /* __ACRN_HSM_HYPERCALL_H */ drivers/virt/acrn/vm.c new file +68
-0@@ -0,0 +1,68 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* ACRN_HSM: Virtual Machine management +* +* Copyright (C) 2020 Intel Corporation. All rights reserved. +* +* Authors: +* Jason Chen CJ <[email protected]> +* Yakui Zhao <[email protected]> +*/ +#include <linux/io.h> +#include <linux/mm.h> +#include <linux/slab.h> + +#include "acrn_drv.h" + +/* List of VMs */ +static LIST_HEAD (acrn_vm_list); +/* To protect acrn_vm_list */ +static DEFINE_MUTEX (acrn_vm_list_lock); + +struct acrn_vm *acrn_vm_create (struct acrn_vm *vm, +struct acrn_vm_creation *vm_param) +{ +int ret; + + ret =hcall_create_vm (virt_to_phys (vm_param)); +if (ret <0 || vm_param->vmid ==ACRN_INVALID_VMID ) { +dev_err (acrn_dev.this_device, +"Failed to create VM! Error: %d\n" , ret); +return NULL ; + } + + vm->vmid = vm_param->vmid; + vm->vcpu_num = vm_param->vcpu_num; + +mutex_lock (&acrn_vm_list_lock); +list_add (&vm->list, &acrn_vm_list); +mutex_unlock (&acrn_vm_list_lock); + +dev_dbg (acrn_dev.this_device,"VM %u created.\n" , vm->vmid); +return vm; +} + +int acrn_vm_destroy (struct acrn_vm *vm) +{ +int ret; + +if (vm->vmid ==ACRN_INVALID_VMID || +test_and_set_bit (ACRN_VM_FLAG_DESTROYED , &vm->flags)) +return 0 ; + +/* Remove from global VM list */ +mutex_lock (&acrn_vm_list_lock); +list_del_init (&vm->list); +mutex_unlock (&acrn_vm_list_lock); + + ret =hcall_destroy_vm (vm->vmid); +if (ret <0 ) { +dev_err (acrn_dev.this_device, +"Failed to destroy VM %u\n" , vm->vmid); +clear_bit (ACRN_VM_FLAG_DESTROYED , &vm->flags); +return ret; + } +dev_dbg (acrn_dev.this_device,"VM %u destroyed.\n" , vm->vmid); + vm->vmid =ACRN_INVALID_VMID ; +return 0 ; +}include/uapi/linux/acrn.h new file +0
-0
include/uapi/linux/acrn.h line 6
Quoted 6 lines
include/uapi/linux/acrn.h new file +6
-0@@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* +* Userspace interface for /dev/acrn_hsm - ACRN Hypervisor Service Module +* +* This file can be used by applications that need to communicate with the HSM +* via the ioctl interface.
No copyright notice on this file? Intel is being brave :)
include/uapi/linux/acrn.h line 18
Quoted 12 lines
include/uapi/linux/acrn.h new file +12
-0+*/ + +#ifndef _UAPI_ACRN_H +#define _UAPI_ACRN_H + +#include <linux/types.h> +#include <linux/uuid.h> + +/** +* struct acrn_vm_creation - Info to create a User VM +* @vmid: User VM ID returned from the hypervisor +* @reserved0: Reserved and must be 0
That's good, but you are not checking this. Why not?
include/uapi/linux/acrn.h line 20
Quoted 2 lines
include/uapi/linux/acrn.h new file +2
-0+* @vcpu_num: Number of vCPU in the VM. Return from hypervisor. +* @reserved1: Reserved and must be 0
Again, I failed to see a check, so this "must be" is not true.
include/uapi/linux/acrn.h line 25
Quoted 5 lines
include/uapi/linux/acrn.h new file +5
-0+* @uuid: UUID of the VM. Pass to hypervisor directly. +* @vm_flag: Flag of the VM creating. Pass to hypervisor directly. +* @ioreq_buf: Service VM GPA of I/O request buffer. Pass to +* hypervisor directly. +* @cpu_affinity: CPU affinity of the VM. Pass to hypervisor directly.
What format is this affinity?
include/uapi/linux/acrn.h line 36
Quoted 11 lines
include/uapi/linux/acrn.h new file +11
-0+*/ +struct acrn_vm_creation { +__u16 vmid; +__u16 reserved0; +__u16 vcpu_num; +__u16 reserved1; +guid_t uuid; +__u64 vm_flag; +__u64 ioreq_buf; +__u64 cpu_affinity; +};
As these values go outside the kernel, you need to specify the endian of them. You do that for other ioctl structures, but not this one, why?
thanks,
greg k-h
Re: [PATCH v7 06/18] virt: acrn: Introduce VM management interfaces
Quoted 383 lines · Greg Kroah-Hartman
Quoted 382 lines · On Wed, Jan 06, 2021 at 03:50:43PM +0800, [email protected] wrote:
From: Shuo Liu <[email protected]>
The VM management interfaces expose several VM operations to ACRN userspace via ioctls. For example, creating VM, starting VM, destroying VM and so on.
The ACRN Hypervisor needs to exchange data with the ACRN userspace during the VM operations. HSM provides VM operation ioctls to the ACRN userspace and communicates with the ACRN Hypervisor for VM operations via hypercalls.
HSM maintains a list of User VM. Each User VM will be bound to an existing file descriptor of
/dev/acrn_hsm. The User VM will be destroyed when the file descriptor is closed.Signed-off-by Shuo Liu <[email protected]>Reviewed-by Zhi Wang <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Cc Zhi Wang <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Yu Wang <[email protected]>Cc Reinette Chatre <[email protected]>Cc Greg Kroah-Hartman <[email protected]>--- .../userspace-api/ioctl/ioctl-number.rst | 1 + MAINTAINERS | 1 + drivers/virt/acrn/Makefile | 2 +- drivers/virt/acrn/acrn_drv.h | 21 ++++- drivers/virt/acrn/hsm.c | 76 +++++++++++++++++- drivers/virt/acrn/hypercall.h | 78 +++++++++++++++++++ drivers/virt/acrn/vm.c | 68 ++++++++++++++++ include/uapi/linux/acrn.h | 55 +++++++++++++ 8 files changed, 298 insertions(+), 4 deletions(-) create mode 100644 drivers/virt/acrn/hypercall.h create mode 100644 drivers/virt/acrn/vm.c create mode 100644 include/uapi/linux/acrn.hDocumentation/userspace-api/ioctl/ioctl-number.rst +1
-0@@ -319,6 +319,7 @@ Code Seq# Include File Comments 0xA0 all linux/sdp/sdp.h Industrial Device Project <mailto:[email protected]> 0xA1 0 linux/vtpm_proxy.h TPM Emulator Proxy Driver +0xA2 all uapi/linux/acrn.h ACRN hypervisor 0xA3 80-8F Port ACL in development: <mailto:[email protected]> 0xA3 90-9F linux/dtlk.hMAINTAINERS +1
-0@@ -443,6 +443,7 @@ S: Supported W: https://projectacrn.org F: Documentation/virt/acrn/ F: drivers/virt/acrn/ +F: include/uapi/linux/acrn.h AD1889 ALSA SOUND DRIVER L: [email protected]drivers/virt/acrn/Makefile +1
-1@@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_ACRN_HSM) := acrn.o -acrn-y := hsm.o +acrn-y := hsm.o vm.odrivers/virt/acrn/acrn_drv.h +20
-1@@ -3,16 +3,35 @@#ifndef __ACRN_HSM_DRV_H#define __ACRN_HSM_DRV_H +#include <linux/acrn.h> +#include <linux/dev_printk.h> +#include <linux/miscdevice.h> #include <linux/types.h> +#include "hypercall.h" + +extern struct miscdevice acrn_dev; +#define ACRN_INVALID_VMID (0xffffU) +#define ACRN_VM_FLAG_DESTROYED 0U/** * struct acrn_vm - Properties of ACRN User VM. +* @list: Entry within global list of all VMs * @vmid: User VM ID +* @vcpu_num: Number of virtual CPUs in the VM +* @flags: Flags (ACRN_VM_FLAG_*) of the VM. This is VM flag management +* in HSM which is different from the &acrn_vm_creation.vm_flag. */ struct acrn_vm { -u16 vmid; +struct list_head list; +u16 vmid; +int vcpu_num; +unsigned long flags; }; +struct acrn_vm *acrn_vm_create (struct acrn_vm *vm, +struct acrn_vm_creation *vm_param); +int acrn_vm_destroy (struct acrn_vm *vm); +#endif /* __ACRN_HSM_DRV_H */ drivers/virt/acrn/hsm.c +74
-2@@ -9,7 +9,6 @@* Yakui Zhao <[email protected]> */ -#include <linux/miscdevice.h> #include <linux/mm.h> #include <linux/module.h> #include <linux/slab.h> @@ -38,10 +37,82 @@ static int acrn_dev_open(struct inode *inode, struct file *filp)return 0 ; } +/* +* HSM relies on hypercall layer of the ACRN hypervisor to do the +* sanity check against the input parameters. +*/ +static long acrn_dev_ioctl (struct file *filp,unsigned int cmd, +unsigned long ioctl_param) +{ +struct acrn_vm *vm = filp->private_data; +struct acrn_vm_creation *vm_param; +int ret =0 ; + +if (vm->vmid ==ACRN_INVALID_VMID && cmd !=ACRN_IOCTL_CREATE_VM ) { +dev_dbg (acrn_dev.this_device, +"ioctl 0x%x: Invalid VM state!\n" , cmd); +return -EINVAL ; + } + +switch (cmd) { +case ACRN_IOCTL_CREATE_VM : + vm_param =memdup_user ((void __user *)ioctl_param, +sizeof (struct acrn_vm_creation )); +if (IS_ERR (vm_param)) +return PTR_ERR (vm_param); + +if ((vm_param->reserved0 | vm_param->reserved1) !=0 ) +return -EINVAL ; + + vm =acrn_vm_create (vm, vm_param); +if (!vm) { + ret = -EINVAL ; +kfree (vm_param); +break ; + } + +if (copy_to_user ((void __user *)ioctl_param, vm_param, +sizeof (struct acrn_vm_creation ))) { +acrn_vm_destroy (vm); + ret = -EFAULT ; + } + +kfree (vm_param); +break ; +case ACRN_IOCTL_START_VM : + ret =hcall_start_vm (vm->vmid); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to start VM %u!\n" , vm->vmid); +break ; +case ACRN_IOCTL_PAUSE_VM : + ret =hcall_pause_vm (vm->vmid); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to pause VM %u!\n" , vm->vmid); +break ; +case ACRN_IOCTL_RESET_VM : + ret =hcall_reset_vm (vm->vmid); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to restart VM %u!\n" , vm->vmid); +break ; +case ACRN_IOCTL_DESTROY_VM : + ret =acrn_vm_destroy (vm); +break ; +default : +dev_dbg (acrn_dev.this_device,"Unknown IOCTL 0x%x!\n" , cmd); + ret = -ENOTTY ; + } + +return ret; +} +static int acrn_dev_release (struct inode *inode,struct file *filp) {struct acrn_vm *vm = filp->private_data; +acrn_vm_destroy (vm);kfree (vm);return 0 ; }@@ -50,9 +121,10 @@ static const struct file_operations acrn_fops = { .owner = THIS_MODULE, .open = acrn_dev_open, .release = acrn_dev_release, + .unlocked_ioctl = acrn_dev_ioctl, }; -static struct miscdevice acrn_dev = { +struct miscdevice acrn_dev = { .minor =MISC_DYNAMIC_MINOR , .name ="acrn_hsm" , .fops = &acrn_fops,drivers/virt/acrn/hypercall.h new file +78
-0@@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* ACRN HSM: hypercalls of ACRN Hypervisor +*/ +#ifndef __ACRN_HSM_HYPERCALL_H +#define __ACRN_HSM_HYPERCALL_H +#include <asm/acrn.h> + +/* +* Hypercall IDs of the ACRN Hypervisor +*/ +#define _HC_ID (x, y) (((x) << 24) | (y)) + +#define HC_ID 0x80UL + +#define HC_ID_VM_BASE 0x10UL +#define HC_CREATE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x00) +#define HC_DESTROY_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x01) +#define HC_START_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x02) +#define HC_PAUSE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x03) +#define HC_RESET_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x05) + +/** +* hcall_create_vm() - Create a User VM +* @vminfo: Service VM GPA of info of User VM creation +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_create_vm (u64 vminfo) +{ +return acrn_hypercall1 (HC_CREATE_VM , vminfo); +} + +/** +* hcall_start_vm() - Start a User VM +* @vmid: User VM ID +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_start_vm (u64 vmid) +{ +return acrn_hypercall1 (HC_START_VM , vmid); +} + +/** +* hcall_pause_vm() - Pause a User VM +* @vmid: User VM ID +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_pause_vm (u64 vmid) +{ +return acrn_hypercall1 (HC_PAUSE_VM , vmid); +} + +/** +* hcall_destroy_vm() - Destroy a User VM +* @vmid: User VM ID +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_destroy_vm (u64 vmid) +{ +return acrn_hypercall1 (HC_DESTROY_VM , vmid); +} + +/** +* hcall_reset_vm() - Reset a User VM +* @vmid: User VM ID +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_reset_vm (u64 vmid) +{ +return acrn_hypercall1 (HC_RESET_VM , vmid); +} + +#endif /* __ACRN_HSM_HYPERCALL_H */ drivers/virt/acrn/vm.c new file +68
-0@@ -0,0 +1,68 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* ACRN_HSM: Virtual Machine management +* +* Copyright (C) 2020 Intel Corporation. All rights reserved. +* +* Authors: +* Jason Chen CJ <[email protected]> +* Yakui Zhao <[email protected]> +*/ +#include <linux/io.h> +#include <linux/mm.h> +#include <linux/slab.h> + +#include "acrn_drv.h" + +/* List of VMs */ +static LIST_HEAD (acrn_vm_list); +/* To protect acrn_vm_list */ +static DEFINE_MUTEX (acrn_vm_list_lock); + +struct acrn_vm *acrn_vm_create (struct acrn_vm *vm, +struct acrn_vm_creation *vm_param) +{ +int ret; + + ret =hcall_create_vm (virt_to_phys (vm_param)); +if (ret <0 || vm_param->vmid ==ACRN_INVALID_VMID ) { +dev_err (acrn_dev.this_device, +"Failed to create VM! Error: %d\n" , ret); +return NULL ; + } + + vm->vmid = vm_param->vmid; + vm->vcpu_num = vm_param->vcpu_num; + +mutex_lock (&acrn_vm_list_lock); +list_add (&vm->list, &acrn_vm_list); +mutex_unlock (&acrn_vm_list_lock); + +dev_dbg (acrn_dev.this_device,"VM %u created.\n" , vm->vmid); +return vm; +} + +int acrn_vm_destroy (struct acrn_vm *vm) +{ +int ret; + +if (vm->vmid ==ACRN_INVALID_VMID || +test_and_set_bit (ACRN_VM_FLAG_DESTROYED , &vm->flags)) +return 0 ; + +/* Remove from global VM list */ +mutex_lock (&acrn_vm_list_lock); +list_del_init (&vm->list); +mutex_unlock (&acrn_vm_list_lock); + + ret =hcall_destroy_vm (vm->vmid); +if (ret <0 ) { +dev_err (acrn_dev.this_device, +"Failed to destroy VM %u\n" , vm->vmid); +clear_bit (ACRN_VM_FLAG_DESTROYED , &vm->flags); +return ret; + } +dev_dbg (acrn_dev.this_device,"VM %u destroyed.\n" , vm->vmid); + vm->vmid =ACRN_INVALID_VMID ; +return 0 ; +}include/uapi/linux/acrn.h new file +0
-0@@ -0,0 +1,55 @@
Quoted 8 lines
include/uapi/linux/acrn.h line 6
Quoted 6 lines
include/uapi/linux/acrn.h new file +6
-0+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* +* Userspace interface for /dev/acrn_hsm - ACRN Hypervisor Service Module +* +* This file can be used by applications that need to communicate with the HSM +* via the ioctl interface. No copyright notice on this file? Intel is being brave :)
Will add it. Thanks.
Quoted 16 lines
Quoted 4 lines
include/uapi/linux/acrn.h new file +2
-0+*/ +include/uapi/linux/acrn.h line 18
Quoted 10 lines
include/uapi/linux/acrn.h new file +10
-0+#ifndef _UAPI_ACRN_H +#define _UAPI_ACRN_H + +#include <linux/types.h> +#include <linux/uuid.h> + +/** +* struct acrn_vm_creation - Info to create a User VM +* @vmid: User VM ID returned from the hypervisor +* @reserved0: Reserved and must be 0 That's good, but you are not checking this. Why not?
I did do the check in ACRN_IOCTL_CREATE_VM branch in acrn_dev_ioctl().
if ((vm_param->reserved0 | vm_param->reserved1) != 0) return -EINVAL;
Quoted 6 lines
include/uapi/linux/acrn.h line 20
Quoted 3 lines
include/uapi/linux/acrn.h new file +2
-0+* @vcpu_num: Number of vCPU in the VM. Return from hypervisor. +* @reserved1: Reserved and must be 0 Again, I failed to see a check, so this "must be" is not true.
Quoted 7 lines
include/uapi/linux/acrn.h line 25
Quoted 5 lines
include/uapi/linux/acrn.h new file +5
-0+* @uuid: UUID of the VM. Pass to hypervisor directly. +* @vm_flag: Flag of the VM creating. Pass to hypervisor directly. +* @ioreq_buf: Service VM GPA of I/O request buffer. Pass to +* hypervisor directly. +* @cpu_affinity: CPU affinity of the VM. Pass to hypervisor directly. What format is this affinity?
It's a bitmap. I will mention this in its documentation.
Quoted 15 lines
include/uapi/linux/acrn.h line 36
Quoted 12 lines
include/uapi/linux/acrn.h new file +11
-0+*/ +struct acrn_vm_creation { +__u16 vmid; +__u16 reserved0; +__u16 vcpu_num; +__u16 reserved1; +guid_t uuid; +__u64 vm_flag; +__u64 ioreq_buf; +__u64 cpu_affinity; +};As these values go outside the kernel, you need to specify the endian of them. You do that for other ioctl structures, but not this one, why?
From my understanding, these values are ARCH un-related. Both kernel and hypervisor are running on a same machine, so we can pass these values to hypervisor direclty as their endian are same.
For those ARCH related values, such as CPU registers, endian is specified.
Thanks
shuo
PATCH v7 7/18 virt: acrn: Introduce an ioctl to set vCPU registers state
In reply to [email protected]From: Shuo Liu <[email protected]>
A virtual CPU of User VM has different context due to the different registers state. ACRN userspace needs to set the virtual CPU registers state (e.g. giving a initial registers state to a virtual BSP of a User VM).
HSM provides an ioctl ACRN_IOCTL_SET_VCPU_REGS to do the virtual CPU registers state setting. The ioctl passes the registers state from ACRN userspace to the hypervisor directly.
--- drivers/virt/acrn/hsm.c | 21 ++++++++++- drivers/virt/acrn/hypercall.h | 13 +++++++ include/uapi/linux/acrn.h | 69 +++++++++++++++++++++++++++++++++++ 3 files changed, 102 insertions(+), 1 deletion(-)
drivers/virt/acrn/hsm.c +20 -1
@@ -9,6 +9,7 @@* Yakui Zhao <[email protected]> */ +#include <linux/io.h> #include <linux/mm.h> #include <linux/module.h> #include <linux/slab.h>
@@ -46,7 +47,8 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd, {struct acrn_vm *vm = filp->private_data;struct acrn_vm_creation *vm_param; -int ret =0 ; +struct acrn_vcpu_regs *cpu_regs; +int i, ret =0 ;if (vm->vmid ==ACRN_INVALID_VMID && cmd !=ACRN_IOCTL_CREATE_VM ) { dev_dbg(acrn_dev.this_device,
@@ -100,6 +102,23 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,case ACRN_IOCTL_DESTROY_VM : ret =acrn_vm_destroy (vm);break ; +case ACRN_IOCTL_SET_VCPU_REGS : + cpu_regs =memdup_user ((void __user *)ioctl_param, +sizeof (struct acrn_vcpu_regs )); +if (IS_ERR (cpu_regs)) +return PTR_ERR (cpu_regs); + +for (i =0 ; i <ARRAY_SIZE (cpu_regs->reserved); i++) +if (cpu_regs->reserved[i]) +return -EINVAL ; + + ret =hcall_set_vcpu_regs (vm->vmid,virt_to_phys (cpu_regs)); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to set regs state of VM%u!\n" , + vm->vmid); +kfree (cpu_regs); +break ;default :dev_dbg (acrn_dev.this_device,"Unknown IOCTL 0x%x!\n" , cmd); ret = -ENOTTY ;
drivers/virt/acrn/hypercall.h +13 -0
@@ -19,6 +19,7 @@#define HC_START_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x02)#define HC_PAUSE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x03)#define HC_RESET_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x05) +#define HC_SET_VCPU_REGS _HC_ID(HC_ID, HC_ID_VM_BASE + 0x06)/** * hcall_create_vm() - Create a User VM
@@ -75,4 +76,16 @@ static inline long hcall_reset_vm(u64 vmid) return acrn_hypercall1(HC_RESET_VM, vmid); } +/** +* hcall_set_vcpu_regs() - Set up registers of virtual CPU of a User VM +* @vmid: User VM ID +* @regs_state: Service VM GPA of registers state +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_set_vcpu_regs (u64 vmid,u64 regs_state) +{ +return acrn_hypercall2 (HC_SET_VCPU_REGS , vmid, regs_state); +} +#endif /* __ACRN_HSM_HYPERCALL_H */
include/uapi/linux/acrn.h +69 -0
80 lines not on this page · raw message
Signature · 1 line
2.28.0
Re: [PATCH v7 07/18] virt: acrn: Introduce an ioctl to set vCPU registers state
Quoted 150 lines · On Wed, Jan 06, 2021 at 03:50:44PM +0800, [email protected] wrote:
From: Shuo Liu <[email protected]>
A virtual CPU of User VM has different context due to the different registers state. ACRN userspace needs to set the virtual CPU registers state (e.g. giving a initial registers state to a virtual BSP of a User VM).
HSM provides an ioctl
ACRN_IOCTL_SET_VCPU_REGSto do the virtual CPU registers state setting. The ioctl passes the registers state from ACRN userspace to the hypervisor directly.Signed-off-by Shuo Liu <[email protected]>Reviewed-by Zhi Wang <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Cc Zhi Wang <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Yu Wang <[email protected]>Cc Reinette Chatre <[email protected]>Cc Greg Kroah-Hartman <[email protected]>--- drivers/virt/acrn/hsm.c | 21 ++++++++++- drivers/virt/acrn/hypercall.h | 13 +++++++ include/uapi/linux/acrn.h | 69 +++++++++++++++++++++++++++++++++++ 3 files changed, 102 insertions(+), 1 deletion(-)drivers/virt/acrn/hsm.c +20
-1@@ -9,6 +9,7 @@* Yakui Zhao <[email protected]> */ +#include <linux/io.h> #include <linux/mm.h> #include <linux/module.h> #include <linux/slab.h> @@ -46,7 +47,8 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd, {struct acrn_vm *vm = filp->private_data;struct acrn_vm_creation *vm_param; -int ret =0 ; +struct acrn_vcpu_regs *cpu_regs; +int i, ret =0 ;if (vm->vmid ==ACRN_INVALID_VMID && cmd !=ACRN_IOCTL_CREATE_VM ) { dev_dbg(acrn_dev.this_device,@@ -100,6 +102,23 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,case ACRN_IOCTL_DESTROY_VM : ret =acrn_vm_destroy (vm);break ; +case ACRN_IOCTL_SET_VCPU_REGS : + cpu_regs =memdup_user ((void __user *)ioctl_param, +sizeof (struct acrn_vcpu_regs )); +if (IS_ERR (cpu_regs)) +return PTR_ERR (cpu_regs); + +for (i =0 ; i <ARRAY_SIZE (cpu_regs->reserved); i++) +if (cpu_regs->reserved[i]) +return -EINVAL ; + + ret =hcall_set_vcpu_regs (vm->vmid,virt_to_phys (cpu_regs)); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to set regs state of VM%u!\n" , + vm->vmid); +kfree (cpu_regs); +break ;default :dev_dbg (acrn_dev.this_device,"Unknown IOCTL 0x%x!\n" , cmd); ret = -ENOTTY ;drivers/virt/acrn/hypercall.h +13
-0@@ -19,6 +19,7 @@#define HC_START_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x02)#define HC_PAUSE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x03)#define HC_RESET_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x05) +#define HC_SET_VCPU_REGS _HC_ID(HC_ID, HC_ID_VM_BASE + 0x06)/** * hcall_create_vm() - Create a User VM @@ -75,4 +76,16 @@ static inline long hcall_reset_vm(u64 vmid) return acrn_hypercall1(HC_RESET_VM, vmid); } +/** +* hcall_set_vcpu_regs() - Set up registers of virtual CPU of a User VM +* @vmid: User VM ID +* @regs_state: Service VM GPA of registers state +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_set_vcpu_regs (u64 vmid,u64 regs_state) +{ +return acrn_hypercall2 (HC_SET_VCPU_REGS , vmid, regs_state); +} +#endif /* __ACRN_HSM_HYPERCALL_H */ include/uapi/linux/acrn.h +42
-0@@ -35,6 +35,73 @@ struct acrn_vm_creation {__u64 cpu_affinity; }; +struct acrn_gp_regs { +__le64 rax; +__le64 rcx; +__le64 rdx; +__le64 rbx; +__le64 rsp; +__le64 rbp; +__le64 rsi; +__le64 rdi; +__le64 r8; +__le64 r9; +__le64 r10; +__le64 r11; +__le64 r12; +__le64 r13; +__le64 r14; +__le64 r15; +}; + +struct acrn_descriptor_ptr { +__le16 limit; +__le64 base; +__le16 reserved[3 ]; +} __attribute__ ((__packed__ )); + +struct acrn_regs { +struct acrn_gp_regs gprs; +struct acrn_descriptor_ptr gdt; +struct acrn_descriptor_ptr idt; + +__le64 rip; +__le64 cs_base; +__le64 cr0; +__le64 cr4; +__le64 cr3; +__le64 ia32_efer; +__le64 rflags; +__le64 reserved_64[4 ]; + +__le32 cs_ar; +__le32 cs_limit; +__le32 reserved_32[3 ];
include/uapi/linux/acrn.h struct acrn_vm_creation { · line 89
Quoted 10 lines
include/uapi/linux/acrn.h +10
-0+ +__le16 cs_sel; +__le16 ss_sel; +__le16 ds_sel; +__le16 es_sel; +__le16 fs_sel; +__le16 gs_sel; +__le16 ldt_sel; +__le16 tr_sel; +};
No documentation of what these variables are anywhere?
What about the reserved slots, what must they be set to?
include/uapi/linux/acrn.h struct acrn_vm_creation { · line 94
Quoted 5 lines
include/uapi/linux/acrn.h +5
-0+ +/** +* struct acrn_vcpu_regs - Info of vCPU registers state +* @vcpu_id: vCPU ID +* @reserved: Reserved and must be 0
You actually checked this, nice!
thanks,
greg k-h
Re: [PATCH v7 07/18] virt: acrn: Introduce an ioctl to set vCPU registers state
Quoted 155 lines · Greg Kroah-Hartman
Quoted 154 lines · On Wed, Jan 06, 2021 at 03:50:44PM +0800, [email protected] wrote:
From: Shuo Liu <[email protected]>
A virtual CPU of User VM has different context due to the different registers state. ACRN userspace needs to set the virtual CPU registers state (e.g. giving a initial registers state to a virtual BSP of a User VM).
HSM provides an ioctl
ACRN_IOCTL_SET_VCPU_REGSto do the virtual CPU registers state setting. The ioctl passes the registers state from ACRN userspace to the hypervisor directly.Signed-off-by Shuo Liu <[email protected]>Reviewed-by Zhi Wang <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Cc Zhi Wang <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Yu Wang <[email protected]>Cc Reinette Chatre <[email protected]>Cc Greg Kroah-Hartman <[email protected]>--- drivers/virt/acrn/hsm.c | 21 ++++++++++- drivers/virt/acrn/hypercall.h | 13 +++++++ include/uapi/linux/acrn.h | 69 +++++++++++++++++++++++++++++++++++ 3 files changed, 102 insertions(+), 1 deletion(-)drivers/virt/acrn/hsm.c +20
-1@@ -9,6 +9,7 @@* Yakui Zhao <[email protected]> */ +#include <linux/io.h> #include <linux/mm.h> #include <linux/module.h> #include <linux/slab.h> @@ -46,7 +47,8 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd, {struct acrn_vm *vm = filp->private_data;struct acrn_vm_creation *vm_param; -int ret =0 ; +struct acrn_vcpu_regs *cpu_regs; +int i, ret =0 ;if (vm->vmid ==ACRN_INVALID_VMID && cmd !=ACRN_IOCTL_CREATE_VM ) { dev_dbg(acrn_dev.this_device,@@ -100,6 +102,23 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,case ACRN_IOCTL_DESTROY_VM : ret =acrn_vm_destroy (vm);break ; +case ACRN_IOCTL_SET_VCPU_REGS : + cpu_regs =memdup_user ((void __user *)ioctl_param, +sizeof (struct acrn_vcpu_regs )); +if (IS_ERR (cpu_regs)) +return PTR_ERR (cpu_regs); + +for (i =0 ; i <ARRAY_SIZE (cpu_regs->reserved); i++) +if (cpu_regs->reserved[i]) +return -EINVAL ; + + ret =hcall_set_vcpu_regs (vm->vmid,virt_to_phys (cpu_regs)); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to set regs state of VM%u!\n" , + vm->vmid); +kfree (cpu_regs); +break ;default :dev_dbg (acrn_dev.this_device,"Unknown IOCTL 0x%x!\n" , cmd); ret = -ENOTTY ;drivers/virt/acrn/hypercall.h +13
-0@@ -19,6 +19,7 @@#define HC_START_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x02)#define HC_PAUSE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x03)#define HC_RESET_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x05) +#define HC_SET_VCPU_REGS _HC_ID(HC_ID, HC_ID_VM_BASE + 0x06)/** * hcall_create_vm() - Create a User VM @@ -75,4 +76,16 @@ static inline long hcall_reset_vm(u64 vmid) return acrn_hypercall1(HC_RESET_VM, vmid); } +/** +* hcall_set_vcpu_regs() - Set up registers of virtual CPU of a User VM +* @vmid: User VM ID +* @regs_state: Service VM GPA of registers state +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_set_vcpu_regs (u64 vmid,u64 regs_state) +{ +return acrn_hypercall2 (HC_SET_VCPU_REGS , vmid, regs_state); +} +#endif /* __ACRN_HSM_HYPERCALL_H */ include/uapi/linux/acrn.h +46
-0@@ -35,6 +35,73 @@ struct acrn_vm_creation {__u64 cpu_affinity; }; +struct acrn_gp_regs { +__le64 rax; +__le64 rcx; +__le64 rdx; +__le64 rbx; +__le64 rsp; +__le64 rbp; +__le64 rsi; +__le64 rdi; +__le64 r8; +__le64 r9; +__le64 r10; +__le64 r11; +__le64 r12; +__le64 r13; +__le64 r14; +__le64 r15; +}; + +struct acrn_descriptor_ptr { +__le16 limit; +__le64 base; +__le16 reserved[3 ]; +} __attribute__ ((__packed__ )); + +struct acrn_regs { +struct acrn_gp_regs gprs; +struct acrn_descriptor_ptr gdt; +struct acrn_descriptor_ptr idt; + +__le64 rip; +__le64 cs_base; +__le64 cr0; +__le64 cr4; +__le64 cr3; +__le64 ia32_efer; +__le64 rflags; +__le64 reserved_64[4 ]; + +__le32 cs_ar; +__le32 cs_limit; +__le32 reserved_32[3 ]; + +__le16 cs_sel; +__le16 ss_sel; +__le16 ds_sel;
Quoted 8 lines
include/uapi/linux/acrn.h struct acrn_vm_creation { · line 89
Quoted 6 lines
include/uapi/linux/acrn.h +6
-0+__le16 es_sel; +__le16 fs_sel; +__le16 gs_sel; +__le16 ldt_sel; +__le16 tr_sel; +};No documentation of what these variables are anywhere?
I will add the documentation to explain them in this file.
Quoted 2 lines
What about the reserved slots, what must they be set to?
Will document they shoud be zero and check them.
Thanks
shuo
PATCH v7 8/18 virt: acrn: Introduce EPT mapping management
In reply to [email protected]From: Shuo Liu <[email protected]>
The HSM provides hypervisor services to the ACRN userspace. While launching a User VM, ACRN userspace needs to allocate memory and request the ACRN Hypervisor to set up the EPT mapping for the VM.
A mapping cache is introduced for accelerating the translation between the Service VM kernel virtual address and User VM physical address.
From the perspective of the hypervisor, the types of GPA of User VM can be listed as following:
1) RAM region, which is used by User VM as system ram.
2) MMIO region, which is recognized by User VM as MMIO. MMIO region is used to be utilized for devices emulation.
Generally, User VM RAM regions mapping is set up before VM started and is released in the User VM destruction. MMIO regions mapping may be set and unset dynamically during User VM running.
To achieve this, ioctls ACRN_IOCTL_SET_MEMSEG and ACRN_IOCTL_UNSET_MEMSEG are introduced in HSM.
--- drivers/virt/acrn/Makefile | 2 +- drivers/virt/acrn/acrn_drv.h | 96 ++++++++++- drivers/virt/acrn/hsm.c | 15 ++ drivers/virt/acrn/hypercall.h | 14 ++ drivers/virt/acrn/mm.c | 306 ++++++++++++++++++++++++++++++++++ drivers/virt/acrn/vm.c | 4 + include/uapi/linux/acrn.h | 49 ++++++ 7 files changed, 476 insertions(+), 10 deletions(-) create mode 100644 drivers/virt/acrn/mm.c
drivers/virt/acrn/Makefile +1 -1
@@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_ACRN_HSM) := acrn.o -acrn-y := hsm.o vm.o +acrn-y := hsm.o vm.o mm.o
drivers/virt/acrn/acrn_drv.h +87 -9
113 lines not on this page · raw message
drivers/virt/acrn/hsm.c +15 -0
@@ -48,6 +48,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,struct acrn_vm *vm = filp->private_data;struct acrn_vm_creation *vm_param;struct acrn_vcpu_regs *cpu_regs; +struct acrn_vm_memmap memmap;int i, ret =0 ;if (vm->vmid ==ACRN_INVALID_VMID && cmd !=ACRN_IOCTL_CREATE_VM ) {
@@ -119,6 +120,20 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd, vm->vmid);kfree (cpu_regs);break ; +case ACRN_IOCTL_SET_MEMSEG : +if (copy_from_user (&memmap, (void __user *)ioctl_param, +sizeof (memmap))) +return -EFAULT ; + + ret =acrn_vm_memseg_map (vm, &memmap); +break ; +case ACRN_IOCTL_UNSET_MEMSEG : +if (copy_from_user (&memmap, (void __user *)ioctl_param, +sizeof (memmap))) +return -EFAULT ; + + ret =acrn_vm_memseg_unmap (vm, &memmap); +break ;default :dev_dbg (acrn_dev.this_device,"Unknown IOCTL 0x%x!\n" , cmd); ret = -ENOTTY ;
drivers/virt/acrn/hypercall.h +14 -0
@@ -21,6 +21,9 @@#define HC_RESET_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x05)#define HC_SET_VCPU_REGS _HC_ID(HC_ID, HC_ID_VM_BASE + 0x06) +#define HC_ID_MEM_BASE 0x40UL +#define HC_VM_SET_MEMORY_REGIONS _HC_ID(HC_ID, HC_ID_MEM_BASE + 0x02) +/** * hcall_create_vm() - Create a User VM * @vminfo: Service VM GPA of info of User VM creation
@@ -88,4 +91,15 @@ static inline long hcall_set_vcpu_regs(u64 vmid, u64 regs_state) return acrn_hypercall2(HC_SET_VCPU_REGS, vmid, regs_state); } +/** +* hcall_set_memory_regions() - Inform the hypervisor to set up EPT mappings +* @regions_pa: Service VM GPA of &struct vm_memory_region_batch +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_set_memory_regions (u64 regions_pa) +{ +return acrn_hypercall1 (HC_VM_SET_MEMORY_REGIONS , regions_pa); +} +#endif /* __ACRN_HSM_HYPERCALL_H */
drivers/virt/acrn/mm.c new file +306 -0
306 lines not on this page · raw message
drivers/virt/acrn/vm.c +4 -0
@@ -31,6 +31,7 @@ struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,return NULL ; } +mutex_init (&vm->regions_mapping_lock); vm->vmid = vm_param->vmid; vm->vcpu_num = vm_param->vcpu_num;
@@ -62,6 +63,9 @@ int acrn_vm_destroy(struct acrn_vm *vm)clear_bit (ACRN_VM_FLAG_DESTROYED , &vm->flags);return ret; } + +acrn_vm_all_ram_unmap (vm); +dev_dbg (acrn_dev.this_device,"VM %u destroyed.\n" , vm->vmid); vm->vmid =ACRN_INVALID_VMID ;return 0 ;
include/uapi/linux/acrn.h +49 -0
59 lines not on this page · raw message
Signature · 1 line
2.28.0
Re: [PATCH v7 08/18] virt: acrn: Introduce EPT mapping management
Quoted 14 lines · On Wed, Jan 06, 2021 at 03:50:45PM +0800, [email protected] wrote:
include/uapi/linux/acrn.h +14
-0+/** +* struct acrn_vm_memmap - A EPT memory mapping info for a User VM. +* @type: Type of the memory mapping (ACRM_MEMMAP_*). +* Pass to hypervisor directly. +* @attr: Attribute of the memory mapping. +* Pass to hypervisor directly. +* @user_vm_pa: Physical address of User VM. +* Pass to hypervisor directly. +* @service_vm_pa: Physical address of Service VM. +* Pass to hypervisor directly. +* @vma_base: VMA address of Service VM. Pass to hypervisor directly. +* @len: Length of the memory mapping. +* Pass to hypervisor directly. +*/
include/uapi/linux/acrn.h struct acrn_vcpu_regs { · line 147
Quoted 10 lines
include/uapi/linux/acrn.h +10
-0+struct acrn_vm_memmap { +__u32 type; +__u32 attr; +__u64 user_vm_pa; +union { +__u64 service_vm_pa; +__u64 vma_base; + }; +__u64 len; +};
Endianness?
Re: [PATCH v7 08/18] virt: acrn: Introduce EPT mapping management
Quoted 19 lines · Greg Kroah-Hartman
Quoted 18 lines · On Wed, Jan 06, 2021 at 03:50:45PM +0800, [email protected] wrote:
include/uapi/linux/acrn.h +18
-0+/** +* struct acrn_vm_memmap - A EPT memory mapping info for a User VM. +* @type: Type of the memory mapping (ACRM_MEMMAP_*). +* Pass to hypervisor directly. +* @attr: Attribute of the memory mapping. +* Pass to hypervisor directly. +* @user_vm_pa: Physical address of User VM. +* Pass to hypervisor directly. +* @service_vm_pa: Physical address of Service VM. +* Pass to hypervisor directly. +* @vma_base: VMA address of Service VM. Pass to hypervisor directly. +* @len: Length of the memory mapping. +* Pass to hypervisor directly. +*/ +struct acrn_vm_memmap { +__u32 type; +__u32 attr; +__u64 user_vm_pa;
Quoted 8 lines
include/uapi/linux/acrn.h struct acrn_vcpu_regs { · line 147
Quoted 6 lines
include/uapi/linux/acrn.h +6
-0+union { +__u64 service_vm_pa; +__u64 vma_base; + }; +__u64 len; +};Endianness?
Same consideration within patch 06/18. Need i specify these values? They are more like ioctls' paramter values.
Thanks
shuo
Quoted 1 line
PATCH v7 9/18 virt: acrn: Introduce I/O request management
In reply to [email protected]From: Shuo Liu <[email protected]>
An I/O request of a User VM, which is constructed by the hypervisor, is distributed by the ACRN Hypervisor Service Module to an I/O client corresponding to the address range of the I/O request.
For each User VM, there is a shared 4-KByte memory region used for I/O requests communication between the hypervisor and Service VM. An I/O request is a 256-byte structure buffer, which is 'struct acrn_io_request', that is filled by an I/O handler of the hypervisor when a trapped I/O access happens in a User VM. ACRN userspace in the Service VM first allocates a 4-KByte page and passes the GPA (Guest Physical Address) of the buffer to the hypervisor. The buffer is used as an array of 16 I/O request slots with each I/O request slot being 256 bytes. This array is indexed by vCPU ID.
An I/O client, which is 'struct acrn_ioreq_client', is responsible for handling User VM I/O requests whose accessed GPA falls in a certain range. Multiple I/O clients can be associated with each User VM. There is a special client associated with each User VM, called the default client, that handles all I/O requests that do not fit into the range of any other I/O clients. The ACRN userspace acts as the default client for each User VM.
The state transitions of a ACRN I/O request are as follows.
FREE -> PENDING -> PROCESSING -> COMPLETE -> FREE -> ...
FREE: this I/O request slot is empty
PENDING: a valid I/O request is pending in this slot
PROCESSING: the I/O request is being processed
COMPLETE: the I/O request has been processed
An I/O request in COMPLETE or FREE state is owned by the hypervisor. HSM and ACRN userspace are in charge of processing the others.
The processing flow of I/O requests are listed as following:
a) The I/O handler of the hypervisor will fill an I/O request with PENDING state when a trapped I/O access happens in a User VM.
b) The hypervisor makes an upcall, which is a notification interrupt, to the Service VM.
c) The upcall handler schedules a tasklet to dispatch I/O requests.
d) The tasklet looks for the PENDING I/O requests, assigns them to different registered clients based on the address of the I/O accesses, updates their state to PROCESSING, and notifies the corresponding client to handle.
e) The notified client handles the assigned I/O requests.
f) The HSM updates I/O requests states to COMPLETE and notifies the hypervisor of the completion via hypercalls.
--- drivers/virt/acrn/Makefile | 2 +- drivers/virt/acrn/acrn_drv.h | 82 ++++++ drivers/virt/acrn/hsm.c | 34 ++- drivers/virt/acrn/hypercall.h | 28 ++ drivers/virt/acrn/ioreq.c | 509 ++++++++++++++++++++++++++++++++++ drivers/virt/acrn/vm.c | 27 +- include/uapi/linux/acrn.h | 134 +++++++++ 7 files changed, 806 insertions(+), 10 deletions(-) create mode 100644 drivers/virt/acrn/ioreq.c
drivers/virt/acrn/Makefile +1 -1
@@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_ACRN_HSM) := acrn.o -acrn-y := hsm.o vm.o mm.o +acrn-y := hsm.o vm.o mm.o ioreq.o
drivers/virt/acrn/acrn_drv.h +82 -0
117 lines not on this page · raw message
drivers/virt/acrn/hsm.c +32 -2
@@ -48,6 +48,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,struct acrn_vm *vm = filp->private_data;struct acrn_vm_creation *vm_param;struct acrn_vcpu_regs *cpu_regs; +struct acrn_ioreq_notify notify;struct acrn_vm_memmap memmap;int i, ret =0 ;
@@ -134,6 +135,31 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd, ret =acrn_vm_memseg_unmap (vm, &memmap);break ; +case ACRN_IOCTL_CREATE_IOREQ_CLIENT : +if (vm->default_client) +return -EEXIST ; +if (!acrn_ioreq_client_create (vm,NULL ,NULL , true,"acrndm" )) + ret = -EINVAL ; +break ; +case ACRN_IOCTL_DESTROY_IOREQ_CLIENT : +if (vm->default_client) +acrn_ioreq_client_destroy (vm->default_client); +break ; +case ACRN_IOCTL_ATTACH_IOREQ_CLIENT : +if (vm->default_client) + ret =acrn_ioreq_client_wait (vm->default_client); +else + ret = -ENODEV ; +break ; +case ACRN_IOCTL_NOTIFY_REQUEST_FINISH : +if (copy_from_user (¬ify, (void __user *)ioctl_param, +sizeof (struct acrn_ioreq_notify ))) +return -EFAULT ; + ret =acrn_ioreq_request_default_complete (vm, notify.vcpu); +break ; +case ACRN_IOCTL_CLEAR_VM_IOREQ : +acrn_ioreq_request_clear (vm); +break ;default :dev_dbg (acrn_dev.this_device,"Unknown IOCTL 0x%x!\n" , cmd); ret = -ENOTTY ;
@@ -175,14 +201,18 @@ static int __init hsm_init(void)return -EPERM ; ret =misc_register (&acrn_dev); -if (ret) +if (ret) {pr_err ("Create misc dev failed!\n" ); +return ret; + } -return ret; +acrn_ioreq_intr_setup (); +return 0 ; } static void __exit hsm_exit(void) { +acrn_ioreq_intr_remove ();misc_deregister (&acrn_dev); }module_init (hsm_init);
drivers/virt/acrn/hypercall.h +28 -0
@@ -21,6 +21,10 @@#define HC_RESET_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x05)#define HC_SET_VCPU_REGS _HC_ID(HC_ID, HC_ID_VM_BASE + 0x06) +#define HC_ID_IOREQ_BASE 0x30UL +#define HC_SET_IOREQ_BUFFER _HC_ID(HC_ID, HC_ID_IOREQ_BASE + 0x00) +#define HC_NOTIFY_REQUEST_FINISH _HC_ID(HC_ID, HC_ID_IOREQ_BASE + 0x01) +#define HC_ID_MEM_BASE 0x40UL#define HC_VM_SET_MEMORY_REGIONS _HC_ID(HC_ID, HC_ID_MEM_BASE + 0x02)
@@ -91,6 +95,30 @@ static inline long hcall_set_vcpu_regs(u64 vmid, u64 regs_state)return acrn_hypercall2 (HC_SET_VCPU_REGS , vmid, regs_state); } +/** +* hcall_set_ioreq_buffer() - Set up the shared buffer for I/O Requests. +* @vmid: User VM ID +* @buffer: Service VM GPA of the shared buffer +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_set_ioreq_buffer (u64 vmid,u64 buffer) +{ +return acrn_hypercall2 (HC_SET_IOREQ_BUFFER , vmid, buffer); +} + +/** +* hcall_notify_req_finish() - Notify ACRN Hypervisor of I/O request completion. +* @vmid: User VM ID +* @vcpu: The vCPU which initiated the I/O request +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_notify_req_finish (u64 vmid,u64 vcpu) +{ +return acrn_hypercall2 (HC_NOTIFY_REQUEST_FINISH , vmid, vcpu); +} +/** * hcall_set_memory_regions() - Inform the hypervisor to set up EPT mappings * @regions_pa: Service VM GPA of &struct vm_memory_region_batch
drivers/virt/acrn/ioreq.c new file +509 -0
509 lines not on this page · raw message
drivers/virt/acrn/vm.c +20 -7
@@ -15,9 +15,12 @@#include "acrn_drv.h" /* List of VMs */ -static LIST_HEAD (acrn_vm_list); -/* To protect acrn_vm_list */ -static DEFINE_MUTEX (acrn_vm_list_lock); +LIST_HEAD (acrn_vm_list); +/* +* acrn_vm_list is read in a tasklet which dispatch I/O requests and is wrote +* in VM creation ioctl. Use the rwlock mechanism to protect it. +*/ +DEFINE_RWLOCK (acrn_vm_list_lock);struct acrn_vm *acrn_vm_create (struct acrn_vm *vm,struct acrn_vm_creation *vm_param)
@@ -32,12 +35,20 @@ struct acrn_vm *acrn_vm_create(struct acrn_vm *vm, }mutex_init (&vm->regions_mapping_lock); +INIT_LIST_HEAD (&vm->ioreq_clients); +spin_lock_init (&vm->ioreq_clients_lock); vm->vmid = vm_param->vmid; vm->vcpu_num = vm_param->vcpu_num; -mutex_lock (&acrn_vm_list_lock); +if (acrn_ioreq_init (vm, vm_param->ioreq_buf) <0 ) { +hcall_destroy_vm (vm_param->vmid); + vm->vmid =ACRN_INVALID_VMID ; +return NULL ; + } + +write_lock_bh (&acrn_vm_list_lock);list_add (&vm->list, &acrn_vm_list); -mutex_unlock (&acrn_vm_list_lock); +write_unlock_bh (&acrn_vm_list_lock);dev_dbg (acrn_dev.this_device,"VM %u created.\n" , vm->vmid);return vm;
@@ -52,9 +63,11 @@ int acrn_vm_destroy(struct acrn_vm *vm)return 0 ;/* Remove from global VM list */ -mutex_lock (&acrn_vm_list_lock); +write_lock_bh (&acrn_vm_list_lock);list_del_init (&vm->list); -mutex_unlock (&acrn_vm_list_lock); +write_unlock_bh (&acrn_vm_list_lock); + +acrn_ioreq_deinit (vm); ret =hcall_destroy_vm (vm->vmid);if (ret <0 ) {
include/uapi/linux/acrn.h +134 -0
146 lines not on this page · raw message
Signature · 1 line
2.28.0
Re: [PATCH v7 09/18] virt: acrn: Introduce I/O request management
Quoted 11 lines · On Wed, 06 Jan 2021, [email protected] wrote:
The processing flow of I/O requests are listed as following:
a) The I/O handler of the hypervisor will fill an I/O request with PENDING state when a trapped I/O access happens in a User VM.
b) The hypervisor makes an upcall, which is a notification interrupt, to the Service VM.
c) The upcall handler schedules a tasklet to dispatch I/O requests.
d) The tasklet looks for the PENDING I/O requests, assigns them to different registered clients based on the address of the I/O accesses, updates their state to PROCESSING, and notifies the corresponding client to handle.
Hmm so tasklets are deprecated (and have been for a while) and it's sad to see incoming new users in modern Linux. This wouldn't be the first one, however. We should be _removing_ users, not adding... In addition, this expands the whole tasklet_disable/enable() hacks.
Could this not be done in process context instead?
Thanks,
Davidlohr
Re: [PATCH v7 09/18] virt: acrn: Introduce I/O request management
Quoted 13 lines · Davidlohr Bueso
Quoted 11 lines · On Wed, 06 Jan 2021, [email protected] wrote:
The processing flow of I/O requests are listed as following:
a) The I/O handler of the hypervisor will fill an I/O request with PENDING state when a trapped I/O access happens in a User VM.
b) The hypervisor makes an upcall, which is a notification interrupt, to the Service VM.
c) The upcall handler schedules a tasklet to dispatch I/O requests.
d) The tasklet looks for the PENDING I/O requests, assigns them to different registered clients based on the address of the I/O accesses, updates their state to PROCESSING, and notifies the corresponding client to handle.
Quoted 4 lines
Hmm so tasklets are deprecated (and have been for a while) and it's sad to see incoming new users in modern Linux. This wouldn't be the first one, however. We should be _removing_ users, not adding... In addition, this expands the whole tasklet_disable/enable() hacks.
Sorry, i had not noticed that.
Quoted 2 lines
Could this not be done in process context instead?
It could be. The original consideration with tasklet was more about performance as the I/O requests dispatching is a hot code path. I think irq thread has little performance impact? I can have a try to convert the tasklet to irq thread.
Thanks
shuo
Re: [PATCH v7 09/18] virt: acrn: Introduce I/O request management
Quoted 7 lines · Shuo A Liu
Quoted 1 line · Davidlohr Bueso
Could this not be done in process context instead?
It could be. The original consideration with tasklet was more about performance as the I/O requests dispatching is a hot code path. I think irq thread has little performance impact? I can have a try to convert the tasklet to irq thread.
Yes, there is some added latency between when the work is scheduled and actually executed - however this should not be a problem for this scenario, and furthermore consider that tasklets do not guarantee performance as ksoftirqd comes in the picture under heavy load.
Thanks,
Davidlohr
PATCH v7 10/18 virt: acrn: Introduce PCI configuration space PIO accesses combiner
In reply to [email protected]From: Shuo Liu <[email protected]>
A User VM can access its virtual PCI configuration spaces via port IO approach, which has two following steps:
1) writes address into port 0xCF8
2) put/get data in/from port 0xCFC
To distribute a complete PCI configuration space access one time, HSM need to combine such two accesses together.
Combine two paired PIO I/O requests into one PCI I/O request and continue the I/O request distribution.
--- drivers/virt/acrn/acrn_drv.h | 2 + drivers/virt/acrn/ioreq.c | 76 ++++++++++++++++++++++++++++++++++++ include/uapi/linux/acrn.h | 15 +++++++ 3 files changed, 93 insertions(+)
drivers/virt/acrn/acrn_drv.h +2 -0
@@ -154,6 +154,7 @@ extern rwlock_t acrn_vm_list_lock;* @default_client: The default I/O request client * @ioreq_buf: I/O request shared buffer * @ioreq_page: The page of the I/O request shared buffer +* @pci_conf_addr: Address of a PCI configuration access emulation */ struct acrn_vm {struct list_head list;
@@ -168,6 +169,7 @@ struct acrn_vm {struct acrn_ioreq_client *default_client;struct acrn_io_request_buffer *ioreq_buf;struct page *ioreq_page; +u32 pci_conf_addr; }; struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
drivers/virt/acrn/ioreq.c +76 -0
88 lines not on this page · raw message
include/uapi/linux/acrn.h +15 -0
@@ -21,6 +21,7 @@#define ACRN_IOREQ_TYPE_PORTIO 0#define ACRN_IOREQ_TYPE_MMIO 1 +#define ACRN_IOREQ_TYPE_PCICFG 2#define ACRN_IOREQ_DIR_READ 0#define ACRN_IOREQ_DIR_WRITE 1
@@ -41,6 +42,18 @@ struct acrn_pio_request {__u32 value; }; +/* Need keep same header fields with pio_request */ +struct acrn_pci_request { +__u32 direction; +__u32 reserved[3 ]; +__u64 size; +__u32 value; +__u32 bus; +__u32 dev; +__u32 func; +__u32 reg; +}; +/** * struct acrn_io_request - 256-byte ACRN I/O request * @type: Type of this request (ACRN_IOREQ_TYPE_*).
@@ -49,6 +62,7 @@ struct acrn_pio_request {* @reserved0: Reserved fields. * @reqs: Union of different types of request. Byte offset: 64. * @reqs.pio_request: PIO request data of the I/O request. +* @reqs.pci_request: PCI configuration space request data of the I/O request. * @reqs.mmio_request: MMIO request data of the I/O request. * @reqs.data: Raw data of the I/O request. * @reserved1: Reserved fields.
@@ -108,6 +122,7 @@ struct acrn_io_request {__u32 reserved0[14 ];union {struct acrn_pio_request pio_request; +struct acrn_pci_request pci_request;struct acrn_mmio_request mmio_request;__u64 data[8 ]; } reqs;
Signature · 1 line
2.28.0
PATCH v7 11/18 virt: acrn: Introduce interfaces for PCI device passthrough
In reply to [email protected]From: Shuo Liu <[email protected]>
PCI device passthrough enables an OS in a virtual machine to directly access a PCI device in the host. It promises almost the native performance, which is required in performance-critical scenarios of ACRN.
HSM provides the following ioctls:
- Assign - ACRN_IOCTL_ASSIGN_PCIDEV
Pass data struct acrn_pcidev from userspace to the hypervisor, and inform the hypervisor to assign a PCI device to a User VM.
- De-assign - ACRN_IOCTL_DEASSIGN_PCIDEV
Pass data struct acrn_pcidev from userspace to the hypervisor, and inform the hypervisor to de-assign a PCI device from a User VM.
- Set a interrupt of a passthrough device - ACRN_IOCTL_SET_PTDEV_INTR Pass data struct acrn_ptdev_irq from userspace to the hypervisor, and inform the hypervisor to map a INTx interrupt of passthrough device of User VM.
- Reset passthrough device interrupt - ACRN_IOCTL_RESET_PTDEV_INTR Pass data struct acrn_ptdev_irq from userspace to the hypervisor, and inform the hypervisor to unmap a INTx interrupt of passthrough device of User VM.
--- drivers/virt/acrn/hsm.c | 50 ++++++++++++++++++++++++++++ drivers/virt/acrn/hypercall.h | 54 +++++++++++++++++++++++++++++++ include/uapi/linux/acrn.h | 61 +++++++++++++++++++++++++++++++++++ 3 files changed, 165 insertions(+)
drivers/virt/acrn/hsm.c +50 -0
@@ -49,7 +49,9 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,struct acrn_vm_creation *vm_param;struct acrn_vcpu_regs *cpu_regs;struct acrn_ioreq_notify notify; +struct acrn_ptdev_irq *irq_info;struct acrn_vm_memmap memmap; +struct acrn_pcidev *pcidev;int i, ret =0 ;if (vm->vmid ==ACRN_INVALID_VMID && cmd !=ACRN_IOCTL_CREATE_VM ) {
@@ -135,6 +137,54 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd, ret =acrn_vm_memseg_unmap (vm, &memmap);break ; +case ACRN_IOCTL_ASSIGN_PCIDEV : + pcidev =memdup_user ((void __user *)ioctl_param, +sizeof (struct acrn_pcidev )); +if (IS_ERR (pcidev)) +return PTR_ERR (pcidev); + + ret =hcall_assign_pcidev (vm->vmid,virt_to_phys (pcidev)); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to assign pci device!\n" ); +kfree (pcidev); +break ; +case ACRN_IOCTL_DEASSIGN_PCIDEV : + pcidev =memdup_user ((void __user *)ioctl_param, +sizeof (struct acrn_pcidev )); +if (IS_ERR (pcidev)) +return PTR_ERR (pcidev); + + ret =hcall_deassign_pcidev (vm->vmid,virt_to_phys (pcidev)); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to deassign pci device!\n" ); +kfree (pcidev); +break ; +case ACRN_IOCTL_SET_PTDEV_INTR : + irq_info =memdup_user ((void __user *)ioctl_param, +sizeof (struct acrn_ptdev_irq )); +if (IS_ERR (irq_info)) +return PTR_ERR (irq_info); + + ret =hcall_set_ptdev_intr (vm->vmid,virt_to_phys (irq_info)); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to configure intr for ptdev!\n" ); +kfree (irq_info); +break ; +case ACRN_IOCTL_RESET_PTDEV_INTR : + irq_info =memdup_user ((void __user *)ioctl_param, +sizeof (struct acrn_ptdev_irq )); +if (IS_ERR (irq_info)) +return PTR_ERR (irq_info); + + ret =hcall_reset_ptdev_intr (vm->vmid,virt_to_phys (irq_info)); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to reset intr for ptdev!\n" ); +kfree (irq_info); +break ;case ACRN_IOCTL_CREATE_IOREQ_CLIENT :if (vm->default_client)return -EEXIST ;
drivers/virt/acrn/hypercall.h +54 -0
64 lines not on this page · raw message
include/uapi/linux/acrn.h +61 -0
71 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH v7 12/18 virt: acrn: Introduce interrupt injection interfaces
In reply to [email protected]From: Shuo Liu <[email protected]>
ACRN userspace need to inject virtual interrupts into a User VM in devices emulation.
HSM needs provide interfaces to do so.
Introduce following interrupt injection interfaces:
ioctl ACRN_IOCTL_SET_IRQLINE:
Pass data from userspace to the hypervisor, and inform the hypervisor to inject a virtual IOAPIC GSI interrupt to a User VM.
ioctl ACRN_IOCTL_INJECT_MSI:
Pass data struct acrn_msi_entry from userspace to the hypervisor, and inform the hypervisor to inject a virtual MSI to a User VM.
ioctl ACRN_IOCTL_VM_INTR_MONITOR:
Set a 4-Kbyte aligned shared page for statistics information of interrupts of a User VM.
--- drivers/virt/acrn/acrn_drv.h | 4 ++++ drivers/virt/acrn/hsm.c | 40 ++++++++++++++++++++++++++++++++++ drivers/virt/acrn/hypercall.h | 41 +++++++++++++++++++++++++++++++++++ drivers/virt/acrn/vm.c | 36 ++++++++++++++++++++++++++++++ include/uapi/linux/acrn.h | 17 +++++++++++++++ 5 files changed, 138 insertions(+)
drivers/virt/acrn/acrn_drv.h +4 -0
@@ -155,6 +155,7 @@ extern rwlock_t acrn_vm_list_lock;* @ioreq_buf: I/O request shared buffer * @ioreq_page: The page of the I/O request shared buffer * @pci_conf_addr: Address of a PCI configuration access emulation +* @monitor_page: Page of interrupt statistics of User VM */ struct acrn_vm {struct list_head list;
@@ -170,6 +171,7 @@ struct acrn_vm {struct acrn_io_request_buffer *ioreq_buf;struct page *ioreq_page;u32 pci_conf_addr; +struct page *monitor_page; }; struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
@@ -196,4 +198,6 @@ struct acrn_ioreq_client *acrn_ioreq_client_create(struct acrn_vm *vm, const char *name);void acrn_ioreq_client_destroy (struct acrn_ioreq_client *client); +int acrn_msi_inject (struct acrn_vm *vm,u64 msi_addr,u64 msi_data); +#endif /* __ACRN_HSM_DRV_H */
drivers/virt/acrn/hsm.c +40 -0
@@ -51,7 +51,9 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,struct acrn_ioreq_notify notify;struct acrn_ptdev_irq *irq_info;struct acrn_vm_memmap memmap; +struct acrn_msi_entry *msi;struct acrn_pcidev *pcidev; +struct page *page;int i, ret =0 ;if (vm->vmid ==ACRN_INVALID_VMID && cmd !=ACRN_IOCTL_CREATE_VM ) {
@@ -185,6 +187,44 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd, "Failed to reset intr for ptdev!\n");kfree (irq_info);break ; +case ACRN_IOCTL_SET_IRQLINE : + ret =hcall_set_irqline (vm->vmid, ioctl_param); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to set interrupt line!\n" ); +break ; +case ACRN_IOCTL_INJECT_MSI : + msi =memdup_user ((void __user *)ioctl_param, +sizeof (struct acrn_msi_entry )); +if (IS_ERR (msi)) +return PTR_ERR (msi); + + ret =hcall_inject_msi (vm->vmid,virt_to_phys (msi)); +if (ret <0 ) +dev_dbg (acrn_dev.this_device, +"Failed to inject MSI!\n" ); +kfree (msi); +break ; +case ACRN_IOCTL_VM_INTR_MONITOR : + ret =pin_user_pages_fast (ioctl_param,1 , +FOLL_WRITE |FOLL_LONGTERM , &page); +if (unlikely (ret !=1 )) { +dev_dbg (acrn_dev.this_device, +"Failed to pin intr hdr buffer!\n" ); +return -EFAULT ; + } + + ret =hcall_vm_intr_monitor (vm->vmid,page_to_phys (page)); +if (ret <0 ) { +unpin_user_page (page); +dev_dbg (acrn_dev.this_device, +"Failed to monitor intr data!\n" ); +return ret; + } +if (vm->monitor_page) +unpin_user_page (vm->monitor_page); + vm->monitor_page = page; +break ;case ACRN_IOCTL_CREATE_IOREQ_CLIENT :if (vm->default_client)return -EEXIST ;
drivers/virt/acrn/hypercall.h +41 -0
53 lines not on this page · raw message
drivers/virt/acrn/vm.c +36 -0
@@ -68,6 +68,10 @@ int acrn_vm_destroy(struct acrn_vm *vm)write_unlock_bh (&acrn_vm_list_lock);acrn_ioreq_deinit (vm); +if (vm->monitor_page) { +put_page (vm->monitor_page); + vm->monitor_page =NULL ; + } ret =hcall_destroy_vm (vm->vmid);if (ret <0 ) {
@@ -83,3 +87,35 @@ int acrn_vm_destroy(struct acrn_vm *vm) vm->vmid =ACRN_INVALID_VMID ;return 0 ; } + +/** +* acrn_inject_msi() - Inject a MSI interrupt into a User VM +* @vm: User VM +* @msi_addr: The MSI address +* @msi_data: The MSI data +* +* Return: 0 on success, <0 on error +*/ +int acrn_msi_inject (struct acrn_vm *vm,u64 msi_addr,u64 msi_data) +{ +struct acrn_msi_entry *msi; +int ret; + +/* might be used in interrupt context, so use GFP_ATOMIC */ + msi =kzalloc (sizeof (*msi),GFP_ATOMIC ); +if (!msi) +return -ENOMEM ; + +/* +* msi_addr: addr[19:12] with dest vcpu id +* msi_data: data[7:0] with vector +*/ + msi->msi_addr = msi_addr; + msi->msi_data = msi_data; + ret =hcall_inject_msi (vm->vmid,virt_to_phys (msi)); +if (ret <0 ) +dev_err (acrn_dev.this_device, +"Failed to inject MSI to VM %u!\n" , vm->vmid); +kfree (msi); +return ret; +}
include/uapi/linux/acrn.h +17 -0
@@ -336,6 +336,16 @@ struct acrn_pcidev {__u32 bar[ACRN_PCI_NUM_BARS ]; }; +/** +* struct acrn_msi_entry - Info for injecting a MSI interrupt to a VM +* @msi_addr: MSI addr[19:12] with dest vCPU ID +* @msi_data: MSI data[7:0] with vector +*/ +struct acrn_msi_entry { +__u64 msi_addr; +__u64 msi_data; +}; +/* The ioctl type, documented in ioctl-number.rst */ #define ACRN_IOCTL_TYPE 0xA2
@@ -355,6 +365,13 @@ struct acrn_pcidev {#define ACRN_IOCTL_SET_VCPU_REGS \ _IOW(ACRN_IOCTL_TYPE, 0x16, struct acrn_vcpu_regs) +#define ACRN_IOCTL_INJECT_MSI \ + _IOW(ACRN_IOCTL_TYPE, 0x23, struct acrn_msi_entry) +#define ACRN_IOCTL_VM_INTR_MONITOR \ + _IOW(ACRN_IOCTL_TYPE, 0x24, unsigned long) +#define ACRN_IOCTL_SET_IRQLINE \ + _IOW(ACRN_IOCTL_TYPE, 0x25, __u64) +#define ACRN_IOCTL_NOTIFY_REQUEST_FINISH \ _IOW(ACRN_IOCTL_TYPE, 0x31, struct acrn_ioreq_notify)#define ACRN_IOCTL_CREATE_IOREQ_CLIENT \
Signature · 1 line
2.28.0
PATCH v7 13/18 virt: acrn: Introduce interfaces to query C-states and P-states allowed by hypervisor
In reply to [email protected]From: Shuo Liu <[email protected]>
The C-states and P-states data are used to support CPU power management. The hypervisor controls C-states and P-states for a User VM.
ACRN userspace need to query the data from the hypervisor to build ACPI tables for a User VM.
HSM provides ioctls for ACRN userspace to query C-states and P-states data obtained from the hypervisor.
--- drivers/virt/acrn/hsm.c | 69 +++++++++++++++++++++++++++++++++++ drivers/virt/acrn/hypercall.h | 12 ++++++ include/uapi/linux/acrn.h | 35 ++++++++++++++++++ 3 files changed, 116 insertions(+)
drivers/virt/acrn/hsm.c +69 -0
@@ -38,6 +38,67 @@ static int acrn_dev_open(struct inode *inode, struct file *filp)return 0 ; } +static int pmcmd_ioctl (u64 cmd,void __user *uptr) +{ +struct acrn_pstate_data *px_data; +struct acrn_cstate_data *cx_data; +u64 *pm_info; +int ret =0 ; + +switch (cmd &PMCMD_TYPE_MASK ) { +case ACRN_PMCMD_GET_PX_CNT : +case ACRN_PMCMD_GET_CX_CNT : + pm_info =kmalloc (sizeof (u64),GFP_KERNEL ); +if (!pm_info) +return -ENOMEM ; + + ret =hcall_get_cpu_state (cmd,virt_to_phys (pm_info)); +if (ret <0 ) { +kfree (pm_info); +break ; + } + +if (copy_to_user (uptr, pm_info,sizeof (u64))) + ret = -EFAULT ; +kfree (pm_info); +break ; +case ACRN_PMCMD_GET_PX_DATA : + px_data =kmalloc (sizeof (*px_data),GFP_KERNEL ); +if (!px_data) +return -ENOMEM ; + + ret =hcall_get_cpu_state (cmd,virt_to_phys (px_data)); +if (ret <0 ) { +kfree (px_data); +break ; + } + +if (copy_to_user (uptr, px_data,sizeof (*px_data))) + ret = -EFAULT ; +kfree (px_data); +break ; +case ACRN_PMCMD_GET_CX_DATA : + cx_data =kmalloc (sizeof (*cx_data),GFP_KERNEL ); +if (!cx_data) +return -ENOMEM ; + + ret =hcall_get_cpu_state (cmd,virt_to_phys (cx_data)); +if (ret <0 ) { +kfree (cx_data); +break ; + } + +if (copy_to_user (uptr, cx_data,sizeof (*cx_data))) + ret = -EFAULT ; +kfree (cx_data); +break ; +default : +break ; + } + +return ret; +} +/* * HSM relies on hypercall layer of the ACRN hypervisor to do the * sanity check against the input parameters.
@@ -54,6 +115,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,struct acrn_msi_entry *msi;struct acrn_pcidev *pcidev;struct page *page; +u64 cstate_cmd;int i, ret =0 ;if (vm->vmid ==ACRN_INVALID_VMID && cmd !=ACRN_IOCTL_CREATE_VM ) {
@@ -250,6 +312,13 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,case ACRN_IOCTL_CLEAR_VM_IOREQ :acrn_ioreq_request_clear (vm);break ; +case ACRN_IOCTL_PM_GET_CPU_STATE : +if (copy_from_user (&cstate_cmd, (void *)ioctl_param, +sizeof (cstate_cmd))) +return -EFAULT ; + + ret =pmcmd_ioctl (cstate_cmd, (void __user *)ioctl_param); +break ;default :dev_dbg (acrn_dev.this_device,"Unknown IOCTL 0x%x!\n" , cmd); ret = -ENOTTY ;
drivers/virt/acrn/hypercall.h +12 -0
@@ -39,6 +39,9 @@#define HC_ASSIGN_PCIDEV _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x05)#define HC_DEASSIGN_PCIDEV _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x06) +#define HC_ID_PM_BASE 0x80UL +#define HC_PM_GET_CPU_STATE _HC_ID(HC_ID, HC_ID_PM_BASE + 0x00) +/** * hcall_create_vm() - Create a User VM * @vminfo: Service VM GPA of info of User VM creation
@@ -225,4 +228,13 @@ static inline long hcall_reset_ptdev_intr(u64 vmid, u64 irq) return acrn_hypercall2(HC_RESET_PTDEV_INTR, vmid, irq); } +/* +* hcall_get_cpu_state() - Get P-states and C-states info from the hypervisor +* @state: Service VM GPA of buffer of P-states and C-states +*/ +static inline long hcall_get_cpu_state (u64 cmd,u64 state) +{ +return acrn_hypercall2 (HC_PM_GET_CPU_STATE , cmd, state); +} +#endif /* __ACRN_HSM_HYPERCALL_H */
include/uapi/linux/acrn.h +35 -0
@@ -346,6 +346,38 @@ struct acrn_msi_entry {__u64 msi_data; }; +struct acrn_acpi_generic_address { +__u8 space_id; +__u8 bit_width; +__u8 bit_offset; +__u8 access_size; +__u64 address; +} __attribute__ ((__packed__ )); + +struct acrn_cstate_data { +struct acrn_acpi_generic_address cx_reg; +__u8 type; +__u32 latency; +__u64 power; +}; + +struct acrn_pstate_data { +__u64 core_frequency; +__u64 power; +__u64 transition_latency; +__u64 bus_master_latency; +__u64 control; +__u64 status; +}; + +#define PMCMD_TYPE_MASK 0x000000ff +enum acrn_pm_cmd_type { +ACRN_PMCMD_GET_PX_CNT , +ACRN_PMCMD_GET_PX_DATA , +ACRN_PMCMD_GET_CX_CNT , +ACRN_PMCMD_GET_CX_DATA , +}; +/* The ioctl type, documented in ioctl-number.rst */ #define ACRN_IOCTL_TYPE 0xA2
@@ -397,4 +429,7 @@ struct acrn_msi_entry {#define ACRN_IOCTL_DEASSIGN_PCIDEV \ _IOW(ACRN_IOCTL_TYPE, 0x56, struct acrn_pcidev) +#define ACRN_IOCTL_PM_GET_CPU_STATE \ + _IOWR(ACRN_IOCTL_TYPE, 0x60, __u64) +#endif /* _UAPI_ACRN_H */
Signature · 1 line
2.28.0
PATCH v7 14/18 virt: acrn: Introduce I/O ranges operation interfaces
In reply to [email protected]From: Shuo Liu <[email protected]>
An I/O request of a User VM, which is constructed by hypervisor, is distributed by the ACRN Hypervisor Service Module to an I/O client corresponding to the address range of the I/O request.
I/O client maintains a list of address ranges. Introduce acrn_ioreq_range_{add,del}() to manage these address ranges.
--- drivers/virt/acrn/acrn_drv.h | 4 +++ drivers/virt/acrn/ioreq.c | 60 ++++++++++++++++++++++++++++++++++++ 2 files changed, 64 insertions(+)
drivers/virt/acrn/acrn_drv.h +4 -0
@@ -197,6 +197,10 @@ struct acrn_ioreq_client *acrn_ioreq_client_create(struct acrn_vm *vm, void *data, bool is_default, const char *name);void acrn_ioreq_client_destroy (struct acrn_ioreq_client *client); +int acrn_ioreq_range_add (struct acrn_ioreq_client *client, +u32 type,u64 start,u64 end); +void acrn_ioreq_range_del (struct acrn_ioreq_client *client, +u32 type,u64 start,u64 end);int acrn_msi_inject (struct acrn_vm *vm,u64 msi_addr,u64 msi_data);
drivers/virt/acrn/ioreq.c +60 -0
66 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH v7 15/18 virt: acrn: Introduce ioeventfd
In reply to [email protected]From: Shuo Liu <[email protected]>
ioeventfd is a mechanism to register PIO/MMIO regions to trigger an eventfd signal when written to by a User VM. ACRN userspace can register any arbitrary I/O address with a corresponding eventfd and then pass the eventfd to a specific end-point of interest for handling.
Vhost is a kernel-level virtio server which uses eventfd for signalling. To support vhost on ACRN, ioeventfd is introduced in HSM.
A new I/O client dedicated to ioeventfd is associated with a User VM during VM creation. HSM provides ioctls to associate an I/O region with a eventfd. The I/O client signals a eventfd once its corresponding I/O region is matched with an I/O request.
--- drivers/virt/acrn/Kconfig | 1 + drivers/virt/acrn/Makefile | 2 +- drivers/virt/acrn/acrn_drv.h | 10 ++ drivers/virt/acrn/hsm.c | 8 + drivers/virt/acrn/ioeventfd.c | 273 ++++++++++++++++++++++++++++++++++ drivers/virt/acrn/vm.c | 2 + include/uapi/linux/acrn.h | 29 ++++ 7 files changed, 324 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/acrn/ioeventfd.c
drivers/virt/acrn/Kconfig +1 -0
@@ -2,6 +2,7 @@ config ACRN_HSM tristate "ACRN Hypervisor Service Module" depends on ACRN_GUEST + select EVENTFD help ACRN Hypervisor Service Module (HSM) is a kernel module which communicates with ACRN userspace through ioctls and talks to
drivers/virt/acrn/Makefile +1 -1
@@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_ACRN_HSM) := acrn.o -acrn-y := hsm.o vm.o mm.o ioreq.o +acrn-y := hsm.o vm.o mm.o ioreq.o ioeventfd.o
drivers/virt/acrn/acrn_drv.h +10 -0
@@ -156,6 +156,9 @@ extern rwlock_t acrn_vm_list_lock;* @ioreq_page: The page of the I/O request shared buffer * @pci_conf_addr: Address of a PCI configuration access emulation * @monitor_page: Page of interrupt statistics of User VM +* @ioeventfds_lock: Lock to protect ioeventfds list +* @ioeventfds: List to link all hsm_ioeventfd +* @ioeventfd_client: I/O client for ioeventfds of the VM */ struct acrn_vm {struct list_head list;
@@ -172,6 +175,9 @@ struct acrn_vm {struct page *ioreq_page;u32 pci_conf_addr;struct page *monitor_page; +struct mutex ioeventfds_lock; +struct list_head ioeventfds; +struct acrn_ioreq_client *ioeventfd_client; }; struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
@@ -204,4 +210,8 @@ void acrn_ioreq_range_del(struct acrn_ioreq_client *client,int acrn_msi_inject (struct acrn_vm *vm,u64 msi_addr,u64 msi_data); +int acrn_ioeventfd_init (struct acrn_vm *vm); +int acrn_ioeventfd_config (struct acrn_vm *vm,struct acrn_ioeventfd *args); +void acrn_ioeventfd_deinit (struct acrn_vm *vm); +#endif /* __ACRN_HSM_DRV_H */
drivers/virt/acrn/hsm.c +8 -0
@@ -111,6 +111,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,struct acrn_vcpu_regs *cpu_regs;struct acrn_ioreq_notify notify;struct acrn_ptdev_irq *irq_info; +struct acrn_ioeventfd ioeventfd;struct acrn_vm_memmap memmap;struct acrn_msi_entry *msi;struct acrn_pcidev *pcidev;
@@ -319,6 +320,13 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd, ret =pmcmd_ioctl (cstate_cmd, (void __user *)ioctl_param);break ; +case ACRN_IOCTL_IOEVENTFD : +if (copy_from_user (&ioeventfd, (void __user *)ioctl_param, +sizeof (ioeventfd))) +return -EFAULT ; + + ret =acrn_ioeventfd_config (vm, &ioeventfd); +break ;default :dev_dbg (acrn_dev.this_device,"Unknown IOCTL 0x%x!\n" , cmd); ret = -ENOTTY ;
drivers/virt/acrn/ioeventfd.c new file +273 -0
273 lines not on this page · raw message
drivers/virt/acrn/vm.c +2 -0
@@ -50,6 +50,7 @@ struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,list_add (&vm->list, &acrn_vm_list);write_unlock_bh (&acrn_vm_list_lock); +acrn_ioeventfd_init (vm);dev_dbg (acrn_dev.this_device,"VM %u created.\n" , vm->vmid);return vm; }
@@ -67,6 +68,7 @@ int acrn_vm_destroy(struct acrn_vm *vm)list_del_init (&vm->list);write_unlock_bh (&acrn_vm_list_lock); +acrn_ioeventfd_deinit (vm);acrn_ioreq_deinit (vm);if (vm->monitor_page) {put_page (vm->monitor_page);
include/uapi/linux/acrn.h +29 -0
@@ -378,6 +378,32 @@ enum acrn_pm_cmd_type { ACRN_PMCMD_GET_CX_DATA, }; +#define ACRN_IOEVENTFD_FLAG_PIO 0x01 +#define ACRN_IOEVENTFD_FLAG_DATAMATCH 0x02 +#define ACRN_IOEVENTFD_FLAG_DEASSIGN 0x04 +/** +* struct acrn_ioeventfd - Data to operate a &struct hsm_ioeventfd +* @fd: The fd of eventfd associated with a hsm_ioeventfd +* @flags: Logical-OR of ACRN_IOEVENTFD_FLAG_* +* @addr: The start address of IO range of ioeventfd +* @len: The length of IO range of ioeventfd +* @reserved: Reserved +* @data: Data for data matching +* +* Without flag ACRN_IOEVENTFD_FLAG_DEASSIGN, ioctl ACRN_IOCTL_IOEVENTFD +* creates a &struct hsm_ioeventfd with properties originated from &struct +* acrn_ioeventfd. With flag ACRN_IOEVENTFD_FLAG_DEASSIGN, ioctl +* ACRN_IOCTL_IOEVENTFD destroys the &struct hsm_ioeventfd matching the fd. +*/ +struct acrn_ioeventfd { +__u32 fd; +__u32 flags; +__u64 addr; +__u32 len; +__u32 reserved; +__u64 data; +}; +/* The ioctl type, documented in ioctl-number.rst */ #define ACRN_IOCTL_TYPE 0xA2
@@ -432,4 +458,7 @@ enum acrn_pm_cmd_type {#define ACRN_IOCTL_PM_GET_CPU_STATE \ _IOWR(ACRN_IOCTL_TYPE, 0x60, __u64) +#define ACRN_IOCTL_IOEVENTFD \ + _IOW(ACRN_IOCTL_TYPE, 0x70, struct acrn_ioeventfd) +#endif /* _UAPI_ACRN_H */
Signature · 1 line
2.28.0
PATCH v7 16/18 virt: acrn: Introduce irqfd
In reply to [email protected]From: Shuo Liu <[email protected]>
irqfd is a mechanism to inject a specific interrupt to a User VM using a decoupled eventfd mechanism.
Vhost is a kernel-level virtio server which uses eventfd for interrupt injection. To support vhost on ACRN, irqfd is introduced in HSM.
HSM provides ioctls to associate a virtual Message Signaled Interrupt (MSI) with an eventfd. The corresponding virtual MSI will be injected into a User VM once the eventfd got signal.
--- drivers/virt/acrn/Makefile | 2 +- drivers/virt/acrn/acrn_drv.h | 10 ++ drivers/virt/acrn/hsm.c | 7 ++ drivers/virt/acrn/irqfd.c | 235 +++++++++++++++++++++++++++++++++++ drivers/virt/acrn/vm.c | 3 + include/uapi/linux/acrn.h | 15 +++ 6 files changed, 271 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/acrn/irqfd.c
drivers/virt/acrn/Makefile +1 -1
@@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_ACRN_HSM) := acrn.o -acrn-y := hsm.o vm.o mm.o ioreq.o ioeventfd.o +acrn-y := hsm.o vm.o mm.o ioreq.o ioeventfd.o irqfd.o
drivers/virt/acrn/acrn_drv.h +10 -0
@@ -159,6 +159,9 @@ extern rwlock_t acrn_vm_list_lock;* @ioeventfds_lock: Lock to protect ioeventfds list * @ioeventfds: List to link all hsm_ioeventfd * @ioeventfd_client: I/O client for ioeventfds of the VM +* @irqfds_lock: Lock to protect irqfds list +* @irqfds: List to link all hsm_irqfd +* @irqfd_wq: Workqueue for irqfd async shutdown */ struct acrn_vm {struct list_head list;
@@ -178,6 +181,9 @@ struct acrn_vm {struct mutex ioeventfds_lock;struct list_head ioeventfds;struct acrn_ioreq_client *ioeventfd_client; +struct mutex irqfds_lock; +struct list_head irqfds; +struct workqueue_struct *irqfd_wq; }; struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
@@ -214,4 +220,8 @@ int acrn_ioeventfd_init(struct acrn_vm *vm);int acrn_ioeventfd_config (struct acrn_vm *vm,struct acrn_ioeventfd *args);void acrn_ioeventfd_deinit (struct acrn_vm *vm); +int acrn_irqfd_init (struct acrn_vm *vm); +int acrn_irqfd_config (struct acrn_vm *vm,struct acrn_irqfd *args); +void acrn_irqfd_deinit (struct acrn_vm *vm); +#endif /* __ACRN_HSM_DRV_H */
drivers/virt/acrn/hsm.c +7 -0
@@ -115,6 +115,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,struct acrn_vm_memmap memmap;struct acrn_msi_entry *msi;struct acrn_pcidev *pcidev; +struct acrn_irqfd irqfd;struct page *page;u64 cstate_cmd;int i, ret =0 ;
@@ -327,6 +328,12 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd, ret =acrn_ioeventfd_config (vm, &ioeventfd);break ; +case ACRN_IOCTL_IRQFD : +if (copy_from_user (&irqfd, (void __user *)ioctl_param, +sizeof (irqfd))) +return -EFAULT ; + ret =acrn_irqfd_config (vm, &irqfd); +break ;default :dev_dbg (acrn_dev.this_device,"Unknown IOCTL 0x%x!\n" , cmd); ret = -ENOTTY ;
drivers/virt/acrn/irqfd.c new file +235 -0
235 lines not on this page · raw message
drivers/virt/acrn/vm.c +3 -0
@@ -51,6 +51,7 @@ struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,write_unlock_bh (&acrn_vm_list_lock);acrn_ioeventfd_init (vm); +acrn_irqfd_init (vm);dev_dbg (acrn_dev.this_device,"VM %u created.\n" , vm->vmid);return vm; }
@@ -69,7 +70,9 @@ int acrn_vm_destroy(struct acrn_vm *vm)write_unlock_bh (&acrn_vm_list_lock);acrn_ioeventfd_deinit (vm); +acrn_irqfd_deinit (vm);acrn_ioreq_deinit (vm); +if (vm->monitor_page) {put_page (vm->monitor_page); vm->monitor_page =NULL ;
include/uapi/linux/acrn.h +15 -0
@@ -404,6 +404,19 @@ struct acrn_ioeventfd {__u64 data; }; +#define ACRN_IRQFD_FLAG_DEASSIGN 0x01 +/** +* struct acrn_irqfd - Data to operate a &struct hsm_irqfd +* @fd: The fd of eventfd associated with a hsm_irqfd +* @flags: Logical-OR of ACRN_IRQFD_FLAG_* +* @msi: Info of MSI associated with the irqfd +*/ +struct acrn_irqfd { +__s32 fd; +__u32 flags; +struct acrn_msi_entry msi; +}; +/* The ioctl type, documented in ioctl-number.rst */ #define ACRN_IOCTL_TYPE 0xA2
@@ -460,5 +473,7 @@ struct acrn_ioeventfd {#define ACRN_IOCTL_IOEVENTFD \ _IOW(ACRN_IOCTL_TYPE, 0x70, struct acrn_ioeventfd) +#define ACRN_IOCTL_IRQFD \ + _IOW(ACRN_IOCTL_TYPE, 0x71, struct acrn_irqfd)#endif /* _UAPI_ACRN_H */
Signature · 1 line
2.28.0
PATCH v7 17/18 virt: acrn: Introduce an interface for Service VM to control vCPU
In reply to [email protected]From: Shuo Liu <[email protected]>
ACRN supports partition mode to achieve real-time requirements. In partition mode, a CPU core can be dedicated to a vCPU of User VM. The local APIC of the dedicated CPU core can be passthrough to the User VM. The Service VM controls the assignment of the CPU cores.
Introduce an interface for the Service VM to remove the control of CPU core from hypervisor perspective so that the CPU core can be a dedicated CPU core of User VM.
--- drivers/virt/acrn/hsm.c | 48 +++++++++++++++++++++++++++++++++++ drivers/virt/acrn/hypercall.h | 14 ++++++++++ 2 files changed, 62 insertions(+)
drivers/virt/acrn/hsm.c +48 -0
@@ -9,6 +9,7 @@* Yakui Zhao <[email protected]> */ +#include <linux/cpu.h> #include <linux/io.h> #include <linux/mm.h> #include <linux/module.h>
@@ -351,6 +352,52 @@ static int acrn_dev_release(struct inode *inode, struct file *filp)return 0 ; } +static ssize_t remove_cpu_store (struct device *dev, +struct device_attribute *attr, +const char *buf,size_t count) +{ +u64 cpu, lapicid; +int ret; + +if (kstrtoull (buf,0 , &cpu) <0 ) +return -EINVAL ; + +if (cpu >=num_possible_cpus () || cpu ==0 || !cpu_is_hotpluggable (cpu)) +return -EINVAL ; + +if (cpu_online (cpu)) +remove_cpu (cpu); + + lapicid =cpu_data (cpu).apicid; +dev_dbg (dev,"Try to remove cpu %lld with lapicid %lld\n" , cpu, lapicid); + ret =hcall_sos_remove_cpu (lapicid); +if (ret <0 ) { +dev_err (dev,"Failed to remove cpu %lld!\n" , cpu); + goto fail_remove; + } + +return count; + +fail_remove: +add_cpu (cpu); +return ret; +} +static DEVICE_ATTR_WO (remove_cpu); + +static struct attribute *acrn_attrs[] = { + &dev_attr_remove_cpu.attr, +NULL +}; + +static struct attribute_group acrn_attr_group = { + .attrs = acrn_attrs, +}; + +static const struct attribute_group *acrn_attr_groups[] = { + &acrn_attr_group, +NULL +}; +static const struct file_operations acrn_fops = { .owner =THIS_MODULE , .open = acrn_dev_open,
@@ -362,6 +409,7 @@ struct miscdevice acrn_dev = { .minor = MISC_DYNAMIC_MINOR, .name = "acrn_hsm", .fops = &acrn_fops, + .groups = acrn_attr_groups, };static int __init hsm_init (void )
drivers/virt/acrn/hypercall.h +14 -0
@@ -13,6 +13,9 @@#define HC_ID 0x80UL +#define HC_ID_GEN_BASE 0x0UL +#define HC_SOS_REMOVE_CPU _HC_ID(HC_ID, HC_ID_GEN_BASE + 0x01) +#define HC_ID_VM_BASE 0x10UL#define HC_CREATE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x00)#define HC_DESTROY_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x01)
@@ -42,6 +45,17 @@#define HC_ID_PM_BASE 0x80UL#define HC_PM_GET_CPU_STATE _HC_ID(HC_ID, HC_ID_PM_BASE + 0x00) +/** +* hcall_sos_remove_cpu() - Remove a vCPU of Service VM +* @cpu: The vCPU to be removed +* +* Return: 0 on success, <0 on failure +*/ +static inline long hcall_sos_remove_cpu (u64 cpu) +{ +return acrn_hypercall1 (HC_SOS_REMOVE_CPU , cpu); +} +/** * hcall_create_vm() - Create a User VM * @vminfo: Service VM GPA of info of User VM creation
Signature · 1 line
2.28.0
PATCH v7 18/18 sample/acrn: Introduce a sample of HSM ioctl interface usage
In reply to [email protected]From: Shuo Liu <[email protected]>
Launch a simple guest (with several instructions as payload) on ACRN with demonstration ioctl usage.
--- samples/acrn/Makefile | 12 ++++ samples/acrn/guest.ld | 9 +++ samples/acrn/payload.ld | 9 +++ samples/acrn/vm-sample.c | 136 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 166 insertions(+) create mode 100644 samples/acrn/Makefile create mode 100644 samples/acrn/guest.ld create mode 100644 samples/acrn/payload.ld create mode 100644 samples/acrn/vm-sample.c
samples/acrn/Makefile new file +12 -0
@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0 + +.PHONY: vm-sample + +vm-sample: vm-sample.o payload.o + $(CC) $^ -o $@ + +payload.o: payload.ld guest16.o + $(LD) -T $< -o $@ + +clean: + rm *.o vm-sample
samples/acrn/guest.ld new file +9 -0
@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +OUTPUT_FORMAT(binary) +SECTIONS +{ + .start : { *(.start) } + .text : { *(.text*) } + .rodata : { *(.rodata) } + .data : { *(.data) } +}
samples/acrn/payload.ld new file +9 -0
@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +SECTIONS +{ + .payload16 0 : { + guest16 = .; + guest16.o(.text) + guest16_end = .; + } +}
samples/acrn/vm-sample.c new file +136 -0
136 lines not on this page · raw message
Signature · 1 line
2.28.0