Documentation/kvm : Add documentation on Hypercalls
Subsystems (2)
PATCH 1/1 Documentation/kvm : Add documentation on Hypercalls
From: Raghavendra K T <[email protected]>
Thanks Alex for KVM_HC_FEATURES inputs and Jan for VAPIC_POLL_IRQ, and Peter (HPA) for suggesting hypercall ABI addition.
---
Please have a closer look at Hypercall ABI newly added
Changes since last post:
- Added hypercall ABI (Peter)
- made KVM_HC_VAPIC_POLL_IRQ active explicitly (Randy)
TODO: We need to add history details of each hypercall as suggested by HPA, which I could not trace easily. Hope it is easy for hypercall authors
Documentation/virtual/hypercalls.txt | 71 ++++++++++++++++++++++++++++++++++ 1 files changed, 71 insertions(+), 0 deletions(-) create mode 100644 Documentation/virtual/hypercalls.txt
Documentation/virtual/hypercalls.txt new file +71 -0
@@ -0,0 +1,71 @@ +Hypercall ABI: +============= +A brief look at calling conventions of X86, S390 and PPC +X86: + KVM Hypercalls have a three-byte sequence of either the vmrun or the vmmrun + instruction. The hypervisor can replace it with instructions that are + guaranteed to be supported. + + Up to four arguments may be passed in rbx, rcx, rdx, and rsi respectively. + The hypercall number should be placed in rax and the return value will be + placed in rax. No other registers will be clobbered unless explicitly stated + by the particular hypercall. + +S390: + R2-R7 are used for parameters 1-6. In addition, R1 is used for hypercall + number. The return value is written to R2. + + S390 uses diagnose instruction as hypercall (0x500) along with hypercall + number in R1. + + PoewerPC: + It uses R3-R10 and hypercall number in R11. R4-R11 are used as output registers. + Return value is placed in R3. + + KVM hypercalls uses 4 byte opcode, that are patched with 'hypercall-instructions' + property inside the device tree's /hypervisor node. + For more information refer to Documentation/virtual/kvm/ppc-pv.txt + +KVM Hypercalls Documentation +=========================== +The template for each hypercall is: +1. Hypercall name, value. +2. Architecture(s) +3. Status (deprecated, obsolete, active) +4. Purpose + +1. KVM_HC_VAPIC_POLL_IRQ +------------------------ +Value: 1 +Architecture: x86 +Status: active +Purpose: Trigger guest exit so that the host can check for pending +interrupts on reentry. + +2. KVM_HC_MMU_OP +------------------------ +Value: 2 +Architecture: x86 +Status: deprecated. +Purpose: Support MMU operations such as writing to PTE, +flushing TLB, release PT. + +3. KVM_HC_FEATURES +------------------------ +Value: 3 +Architecture: PPC +Status: active +Purpose: Expose hypercall availability to the guest. On x86 platforms, cpuid +used to enumerate which hypercalls are available. On PPC, either device tree +based lookup ( which is also what EPAPR dictates) OR KVM specific enumeration +mechanism (which is this hypercall) can be used. + +4. KVM_HC_PPC_MAP_MAGIC_PAGE +------------------------ +Value: 4 +Architecture: PPC +Status: active +Purpose: To enable communication between the hypervisor and guest there is a +shared page that contains parts of supervisor visible register state. +The guest can map this shared page to access its supervisor register through +memory using this hypercall.
Quoted 37 lines · Raghavendra K T
From: Raghavendra K T <[email protected]>
Thanks Alex for
KVM_HC_FEATURESinputs and Jan forVAPIC_POLL_IRQ, and Peter (HPA) for suggesting hypercall ABI addition.Signed-off-by Raghavendra K T <[email protected]>---
Please have a closer look at Hypercall ABI newly addedChanges since last post:
- Added hypercall ABI (Peter)
- madeKVM_HC_VAPIC_POLL_IRQactive explicitly (Randy)TODO: We need to add history details of each hypercall as suggested by HPA, which I could not trace easily. Hope it is easy for hypercall authors
Documentation/virtual/hypercalls.txt | 71 ++++++++++++++++++++++++++++++++++ 1 files changed, 71 insertions(+), 0 deletions(-)create mode 100644
Documentation/virtual/hypercalls.txtDocumentation/virtual/hypercalls.txt new file +11
-0@@ -0,0 +1,71 @@ +Hypercall ABI: +============= +A brief look at calling conventions of X86, S390 and PPC +X86: + KVM Hypercalls have a three-byte sequence of either the vmrun or the vmmrun + instruction. The hypervisor can replace it with instructions that are + guaranteed to be supported. + + Up to four arguments may be passed in rbx, rcx, rdx, and rsi respectively. + The hypercall number should be placed in rax and the return value will be + placed in rax. No other registers will be clobbered unless explicitly stated
Documentation/virtual/hypercalls.txt line 21
Quoted 10 lines
Documentation/virtual/hypercalls.txt new file +10
-0+ by the particular hypercall. + +S390: + R2-R7 are used for parameters 1-6. In addition, R1 is used for hypercall + number. The return value is written to R2. + + S390 uses diagnose instruction as hypercall (0x500) along with hypercall + number in R1. + + PoewerPC:
PowerPC
Documentation/virtual/hypercalls.txt line 27
Quoted 6 lines
Documentation/virtual/hypercalls.txt new file +6
-0+ It uses R3-R10 and hypercall number in R11. R4-R11 are used as output registers. + Return value is placed in R3. + + KVM hypercalls uses 4 byte opcode, that are patched with 'hypercall-instructions' + property inside the device tree's /hypervisor node. + For more information refer to Documentation/virtual/kvm/ppc-pv.txt
What exactly is this document supposed to cover? We have 3 different hypercall ABIs in KVM on PowerPC:
1) KVM hypercalls (ePAPR)
The ePAPR compliant hypercall implementation. This one is used for KVM specific hypercalls. All hypercalls get a KVM vendor prefix (42) and then the hypercall id. We also implement generic hypercalls here, like the ePAPR idle hcall. The instruction sequence for these is listed in the device tree, as you noted above. It's available on all targets.
2) PAPR hypercalls
To run server PowerPC PAPR guests (-M pseries in QEMU), we need to handle PAPR hypercalls. These are the same hypercalls that pHyp, the POWER hypervisor implements. Some of them are handled in the kernel, some are handled in user space. This is only available on book3s_64.
3) OSI hypercalls
In parallel to QEMU, there is another user of KVM on PowerPC: Mac-on-Linux. That project had its own hypercalls long before we came along with KVM, so to maintain compatibility we also support their hypercalls. All of these get forwarded to user space. This is only useful on book3s_32, but can be used with book3s_64 as well.
Documentation/virtual/hypercalls.txt line 39
Quoted 12 lines
Documentation/virtual/hypercalls.txt new file +12
-0+ +KVM Hypercalls Documentation +=========================== +The template for each hypercall is: +1. Hypercall name, value. +2. Architecture(s) +3. Status (deprecated, obsolete, active) +4. Purpose + +1. KVM_HC_VAPIC_POLL_IRQ +------------------------ +Value: 1
What are the value fields supposed to mean?
Alex
Quoted 33 lines
Documentation/virtual/hypercalls.txt new file +32
-0+Architecture: x86 +Status: active +Purpose: Trigger guest exit so that the host can check for pending +interrupts on reentry. + +2. KVM_HC_MMU_OP +------------------------ +Value: 2 +Architecture: x86 +Status: deprecated. +Purpose: Support MMU operations such as writing to PTE, +flushing TLB, release PT. + +3. KVM_HC_FEATURES +------------------------ +Value: 3 +Architecture: PPC +Status: active +Purpose: Expose hypercall availability to the guest. On x86 platforms, cpuid +used to enumerate which hypercalls are available. On PPC, either device tree +based lookup ( which is also what EPAPR dictates) OR KVM specific enumeration +mechanism (which is this hypercall) can be used. + +4. KVM_HC_PPC_MAP_MAGIC_PAGE +------------------------ +Value: 4 +Architecture: PPC +Status: active +Purpose: To enable communication between the hypervisor and guest there is a +shared page that contains parts of supervisor visible register state. +The guest can map this shared page to access its supervisor register through +memory using this hypercall.
Quoted 51 lines · Alexander Graf
Quoted 50 lines · Raghavendra K T
From: Raghavendra K T <[email protected]>
Thanks Alex for
KVM_HC_FEATURESinputs and Jan forVAPIC_POLL_IRQ, and Peter (HPA) for suggesting hypercall ABI addition.Signed-off-by Raghavendra K T <[email protected]>---
Please have a closer look at Hypercall ABI newly addedChanges since last post:
- Added hypercall ABI (Peter)
- madeKVM_HC_VAPIC_POLL_IRQactive explicitly (Randy)TODO: We need to add history details of each hypercall as suggested by HPA,
which I could not trace easily. Hope it is easy for hypercall authors
Documentation/virtual/hypercalls.txt| 71
++++++++++++++++++++++++++++++++++1 files changed, 71 insertions(+), 0 deletions(-)create mode 100644
Documentation/virtual/hypercalls.txtDocumentation/virtual/hypercalls.txt new file +5
-0@@ -0,0 +1,71 @@ +Hypercall ABI: +============= +A brief look at calling conventions of X86, S390 and PPC +X86: + KVM Hypercalls have a three-byte sequence of either the vmrun or thevmmrun
+ instruction. The hypervisor can replace it with instructions that are
+ guaranteed to be supported.
+
+ Up to four arguments may be passed in rbx, rcx, rdx, and rsi respectively.
+ The hypercall number should be placed in rax and the return value will be
+ placed in rax. No other registers will be clobbered unless explicitly stated
+ by the particular hypercall.Documentation/virtual/hypercalls.txt new file +3
-0+ +S390: + R2-R7 are used for parameters 1-6. In addition, R1 is used forhypercall
+ number. The return value is written to R2.
Quoted 8 lines
Quoted 6 lines
+
+ S390 uses diagnose instruction as hypercall (0x500) along with hypercall
+ number in R1.
+
+ PoewerPC:PowerPC
Sorry for the typo.
Quoted 10 lines
Quoted 9 lines
+ It uses R3-R10 and hypercall number in R11. R4-R11 are used as output registers.
+ Return value is placed in R3.
+
+ KVM hypercalls uses 4 byte opcode, that are patched with 'hypercall-instructions'
+ property inside the device tree's /hypervisor node.
+ For more information refer toDocumentation/virtual/kvm/ppc-pv.txt
Quoted 2 lines
What exactly is this document supposed to cover? We have 3 different hypercall ABIs in KVM on PowerPC:
Thanks for the detailed information on PPC hypercalls. My intention was to cover KVM specific hypercalls here.
Patch is wrongly generated against Documentation/virtual/ instead of Documentation/virtual/kvm will correct that.
Information you have provided should be very useful in understanding PPC hypercalls. Do you wish to have (2), (3) below goto some doc?
Quoted 23 lines
1) KVM hypercalls (ePAPR)
The ePAPR compliant hypercall implementation. This one is used for KVM specific hypercalls. All hypercalls get a KVM vendor prefix (42) and then the hypercall id. We also implement generic hypercalls here, like the ePAPR idle hcall. The instruction sequence for these is listed in the device tree, as you noted above. It's available on all targets.
2) PAPR hypercalls
To run server PowerPC PAPR guests (-M pseries in QEMU), we need to handle PAPR hypercalls. These are the same hypercalls that pHyp, the POWER hypervisor implements. Some of them are handled in the kernel, some are handled in user space. This is only available on
book3s_64.3) OSI hypercalls
In parallel to QEMU, there is another user of KVM on PowerPC:
Mac-on-Linux. That project had its own hypercalls long before we came along with KVM, so to maintain compatibility we also support their hypercalls. All of these get forwarded to user space. This is only useful onbook3s_32, but can be used withbook3s_64as well.
Quoted 14 lines
Documentation/virtual/hypercalls.txt line 39
Quoted 12 lines
Documentation/virtual/hypercalls.txt new file +12
-0+ +KVM Hypercalls Documentation +=========================== +The template for each hypercall is: +1. Hypercall name, value. +2. Architecture(s) +3. Status (deprecated, obsolete, active) +4. Purpose + +1. KVM_HC_VAPIC_POLL_IRQ +------------------------ +Value: 1What are the value fields supposed to mean?
Value field is just #define value of the hypercall in header. If that does not make much sense probably it can be dropped.
Quoted 26 lines · Raghavendra K T
From: Raghavendra K T <[email protected]>
Thanks Alex for
KVM_HC_FEATURESinputs and Jan forVAPIC_POLL_IRQ, and Peter (HPA) for suggesting hypercall ABI addition.Signed-off-by Raghavendra K T <[email protected]>---
Please have a closer look at Hypercall ABI newly addedChanges since last post:
- Added hypercall ABI (Peter)
- madeKVM_HC_VAPIC_POLL_IRQactive explicitly (Randy)TODO: We need to add history details of each hypercall as suggested by HPA, which I could not trace easily. Hope it is easy for hypercall authors
Documentation/virtual/hypercalls.txt | 71 ++++++++++++++++++++++++++++++++++ 1 files changed, 71 insertions(+), 0 deletions(-) create mode 100644 Documentation/virtual/hypercalls.txtDocumentation/virtual/hypercalls.txt new file +0
-0
Documentation/virtual/hypercalls.txt line 7
Quoted 7 lines
Documentation/virtual/hypercalls.txt new file +7
-0@@ -0,0 +1,71 @@ +Hypercall ABI: +============= +A brief look at calling conventions of X86, S390 and PPC +X86: + KVM Hypercalls have a three-byte sequence of either the vmrun or the vmmrun + instruction. The hypervisor can replace it with instructions that are + guaranteed to be supported.
vmcall.
Documentation/virtual/hypercalls.txt line 12
Quoted 5 lines
Documentation/virtual/hypercalls.txt new file +5
-0+ + Up to four arguments may be passed in rbx, rcx, rdx, and rsi respectively. + The hypercall number should be placed in rax and the return value will be + placed in rax. No other registers will be clobbered unless explicitly stated + by the particular hypercall.
It depends on the hypercall. It happens that current hypercalls use the four registers, but its not an ABI (hyper-v hypercalls uses r8, for example).
Quoted 21 lines · Marcelo Tosatti
Quoted 20 lines · Raghavendra K T
From: Raghavendra K T<[email protected]>
Thanks Alex for
KVM_HC_FEATURESinputs and Jan forVAPIC_POLL_IRQ, and Peter (HPA) for suggesting hypercall ABI addition.Signed-off-by Raghavendra K T<[email protected]>---
Please have a closer look at Hypercall ABI newly addedChanges since last post:
- Added hypercall ABI (Peter)
- madeKVM_HC_VAPIC_POLL_IRQactive explicitly (Randy)TODO: We need to add history details of each hypercall as suggested by HPA, which I could not trace easily. Hope it is easy for hypercall authors
Documentation/virtual/hypercalls.txt | 71 ++++++++++++++++++++++++++++++++++ 1 files changed, 71 insertions(+), 0 deletions(-)create mode 100644
Documentation/virtual/hypercalls.txt
Quoted 15 lines
Documentation/virtual/hypercalls.txt line 7
Quoted 13 lines
Documentation/virtual/hypercalls.txt new file +7
-0@@ -0,0 +1,71 @@ +Hypercall ABI: +============= +A brief look at calling conventions of X86, S390 and PPC +X86: + KVM Hypercalls have a three-byte sequence of either the vmrun or the vmmrun + instruction. The hypervisor can replace it with instructions that are + guaranteed to be supported.vmcall.
How is this:
KVM Hypercalls have a three-byte sequence of vmcall instruction. But
The hypervisor can replace it with the instructions that are guaranteed to be supported.
(Related question: Is it not that vmcall instruction is patched by 0f 01 09 vmmcall instruction sequence for amd?. So I think we need to change documentation I referred from arch/x86/include/asm/kvm_para.h. It should be vmcall / vmmcall instead of vmrun and vmmrun)
Quoted 10 lines
Documentation/virtual/hypercalls.txt line 12
Quoted 6 lines
Documentation/virtual/hypercalls.txt new file +5
-0+ + Up to four arguments may be passed in rbx, rcx, rdx, and rsi respectively. + The hypercall number should be placed in rax and the return value will be + placed in rax. No other registers will be clobbered unless explicitly stated + by the particular hypercall.It depends on the hypercall. It happens that current hypercalls use the four registers, but its not an ABI (hyper-v hypercalls uses r8, for example).
Okay, agree. I did not know about hyper-v. Above one is for X86 and KVM (and not X86 / hyper-v). So I should remove ABI and probably say,
KVM x86 Hypercall
==================
Up to four arguments may be passed in rbx, rcx, rdx, and rsi respectively. The hypercall number should be placed in rax and the return value will be placed in rax. No other registers will be clobbered unless explicitly stated by the particular hypercall.
Quoted 38 lines · Raghavendra K T
Quoted 21 lines · Marcelo Tosatti
Quoted 20 lines · Raghavendra K T
From: Raghavendra K T<[email protected]>
Thanks Alex for
KVM_HC_FEATURESinputs and Jan forVAPIC_POLL_IRQ, and Peter (HPA) for suggesting hypercall ABI addition.Signed-off-by Raghavendra K T<[email protected]>---
Please have a closer look at Hypercall ABI newly addedChanges since last post:
- Added hypercall ABI (Peter)
- madeKVM_HC_VAPIC_POLL_IRQactive explicitly (Randy)TODO: We need to add history details of each hypercall as suggested by HPA, which I could not trace easily. Hope it is easy for hypercall authors
Documentation/virtual/hypercalls.txt | 71 ++++++++++++++++++++++++++++++++++ 1 files changed, 71 insertions(+), 0 deletions(-) create mode 100644 Documentation/virtual/hypercalls.txtQuoted 15 lines
Documentation/virtual/hypercalls.txt line 7
Quoted 13 lines
Documentation/virtual/hypercalls.txt new file +7
-0@@ -0,0 +1,71 @@ +Hypercall ABI: +============= +A brief look at calling conventions of X86, S390 and PPC +X86: + KVM Hypercalls have a three-byte sequence of either the vmrun or the vmmrun + instruction. The hypervisor can replace it with instructions that are + guaranteed to be supported.vmcall.
Quoted 7 lines
How is this:
KVM Hypercalls have a three-byte sequence of vmcall instruction. But
The hypervisor can replace it with the instructions that are guaranteed to be supported.(Related question: Is it not that vmcall instruction is patched by 0f 01 09 vmmcall instruction sequence for amd?.
Yes, it is.
Linux guest binary contains the Intel vmcall instruction, which is patched on AMD to vmmcall.
Quoted 3 lines
So I think we need to
change documentation I referred fromarch/x86/include/asm/kvm_para.h. It should be vmcall / vmmcall instead of vmrun and vmmrun)
Yes.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Documentation/virtual/hypercalls.txt new file +5
-0+ + Up to four arguments may be passed in rbx, rcx, rdx, and rsi respectively. + The hypercall number should be placed in rax and the return value will be + placed in rax. No other registers will be clobbered unless explicitly stated + by the particular hypercall.
Quoted 6 lines
Quoted 3 lines
It depends on the hypercall. It happens that current hypercalls use the four registers, but its not an ABI (hyper-v hypercalls uses r8, for example).
Okay, agree. I did not know about hyper-v. Above one is for X86 and KVM (and not X86 / hyper-v). So I should remove ABI and probably say,
Just say "Linux x86 Hypercall", yeah.
Quoted 11 lines
KVM x86 Hypercall
==================
Up to four arguments may be passed in rbx, rcx, rdx, and rsi respectively. The hypercall number should be placed in rax and the return value will be placed in rax. No other registers will be clobbered unless explicitly stated by the particular hypercall.--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 10 lines · Marcelo Tosatti
Quoted 9 lines · Raghavendra K T
Quoted 8 lines · Marcelo Tosatti
Quoted 7 lines · Raghavendra K T
From: Raghavendra K T<[email protected]>
Thanks Alex for
KVM_HC_FEATURESinputs and Jan forVAPIC_POLL_IRQ, and Peter (HPA) for suggesting hypercall ABI addition.Signed-off-by Raghavendra K T<[email protected]>---
[...]
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
+
+ Up to four arguments may be passed in rbx, rcx, rdx, and rsi respectively.
+ The hypercall number should be placed in rax and the return value will be
+ placed in rax. No other registers will be clobbered unless explicitly stated
+ by the particular hypercall.
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
It depends on the hypercall. It happens that current hypercalls use the four registers, but its not an ABI (hyper-v hypercalls uses r8, for example).
Okay, agree. I did not know about hyper-v. Above one is for X86 and KVM (and not X86 / hyper-v). So I should remove ABI and probably say,
Just say "Linux x86 Hypercall", yeah.
Ok, Thanks Marcelo.will respin patches with the changes you suggested.
Also I would like add Alex's description on ppc hypercalls into Documentation/virtual/kvm/ppc-pv.txt with Alex from/sob or suggested-by in a separate patch unless I see any objection to that.
Quoted 25 lines · Raghavendra K T
Quoted 10 lines · Marcelo Tosatti
Quoted 9 lines · Raghavendra K T
Quoted 8 lines · Marcelo Tosatti
Quoted 7 lines · Raghavendra K T
From: Raghavendra K T<[email protected]>
Thanks Alex for
KVM_HC_FEATURESinputs and Jan forVAPIC_POLL_IRQ, and Peter (HPA) for suggesting hypercall ABI addition.Signed-off-by Raghavendra K T<[email protected]>---
[...]
Quoted 13 lines
Quoted 12 lines
Quoted 9 lines
Quoted 5 lines
+
+ Up to four arguments may be passed in rbx, rcx, rdx, and rsi respectively.
+ The hypercall number should be placed in rax and the return value will be
+ placed in rax. No other registers will be clobbered unless explicitly stated
+ by the particular hypercall.It depends on the hypercall. It happens that current hypercalls use the four registers, but its not an ABI (hyper-v hypercalls uses r8, for example).
Okay, agree. I did not know about hyper-v. Above one is for X86 and KVM (and not X86 / hyper-v). So I should remove ABI and probably say,
Quoted 9 lines
Quoted 2 lines
Just say "Linux x86 Hypercall", yeah.
Ok, Thanks Marcelo.will respin patches with the changes you suggested.
Also I would like add Alex's description on ppc hypercalls into
Documentation/virtual/kvm/ppc-pv.txtwith Alex from/sob or suggested-by in a separate patch unless I see any objection to that.
Yeah, don't stall this on me :)
Alex