Skip to contentSkip to conversation

Documentation: hyperv: Add basic info on Hyper-V enlightenments

PATCH In review Michael Kelley 3 participants 7 messages Jul 11, 2022 – Jul 17, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v2 cover Documentation: hyperv: Add basic info on Hyper-V enlightenments

This documentation is a high level overview to explain the basics of Linux running as a guest on Hyper-V. The intent is to document the forest, not the trees. The Hyper-V Top Level Functional Spec provides conceptual material and API details for the core Hyper-V hypervisor, and this documentation provides additional info on how that functionality is applied to Linux. Also, there's no public documentation on VMbus or the VMbus synthetic devices, so this documentation helps fill that gap at a conceptual level. This documentation is not API-level documentation, which can be seen in the code and associated comments.

More topics will be added in future patches, including:

* Miscellaneous synthetic devices like KVP, timesync, VSS, etc.
* Virtual PCI support
* Isolated/Confidential VMs
* UIO driver

If you think I'm missing a topic that fits into the overall approach as described, feel free to suggest text, or let me know and I can add it to my list.

Changes in v2:
* Updated clocks.rst to use section hierarchy that matches overview.rst and vmbus.rst [Wei Liu]

Michael Kelley (3):
Documentation: hyperv: Add overview of Hyper-V enlightenments
Documentation: hyperv: Add overview of VMbus
Documentation: hyperv: Add overview of clocks and timers

 Documentation/virt/hyperv/clocks.rst   |  73 ++++++++
 Documentation/virt/hyperv/index.rst    |  12 ++
 Documentation/virt/hyperv/overview.rst | 207 ++++++++++++++++++++++
 Documentation/virt/hyperv/vmbus.rst    | 303 +++++++++++++++++++++++++++++++++
 Documentation/virt/index.rst           |   1 +
 MAINTAINERS                            |   1 +
 6 files changed, 597 insertions(+)
 create mode 100644 Documentation/virt/hyperv/clocks.rst
 create mode 100644 Documentation/virt/hyperv/index.rst
 create mode 100644 Documentation/virt/hyperv/overview.rst
 create mode 100644 Documentation/virt/hyperv/vmbus.rst
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH v2 1/3 Documentation: hyperv: Add overview of Hyper-V enlightenments

Add an initial documentation topic for Linux enlightenments to run as a guest on Microsoft's Hyper-V hypervisor, linked under the "virt" documentation area. Update the virt doc index.rst and the MAINTAINERS file.

---
 Documentation/virt/hyperv/index.rst    |  10 ++
 Documentation/virt/hyperv/overview.rst | 207 +++++++++++++++++++++++++++++++++
 Documentation/virt/index.rst           |   1 +
 MAINTAINERS                            |   1 +
 4 files changed, 219 insertions(+)
 create mode 100644 Documentation/virt/hyperv/index.rst
 create mode 100644 Documentation/virt/hyperv/overview.rst
Documentation/virt/hyperv/index.rst new file +10 -0
new file mode 100644
index 0000000..991bee4
@@ -0,0 +1,10 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======================
+Hyper-V Enlightenments
+======================
+
+.. toctree::
+   :maxdepth: 1
+
+   overview
Documentation/virt/hyperv/overview.rst new file +207 -0

207 lines not on this page · raw message

Documentation/virt/index.rst +1 -0
index 492f092..2f1cffa 100644
@@ -14,6 +14,7 @@ Linux Virtualization Support
    ne_overview
    acrn/index
    coco/sev-guest
+   hyperv/index
 
 .. only:: html and subproject
 
MAINTAINERS +1 -0
index a6d3bd9..fd2e5ce 100644
@@ -9172,6 +9172,7 @@ S:	Supported
 T:	git git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git
 F:	Documentation/ABI/stable/sysfs-bus-vmbus
 F:	Documentation/ABI/testing/debugfs-hyperv
+F:	Documentation/virt/hyperv
 F:	Documentation/networking/device_drivers/ethernet/microsoft/netvsc.rst
 F:	arch/arm64/hyperv
 F:	arch/arm64/include/asm/hyperv-tlfs.h
Signature · 1 line
1.8.3.1
#2 lore raw

PATCH v2 2/3 Documentation: hyperv: Add overview of VMbus

In reply to Michael Kelley

Add documentation topic for using VMbus when running as a guest on Hyper-V.

---
 Documentation/virt/hyperv/index.rst |   1 +
 Documentation/virt/hyperv/vmbus.rst | 303 ++++++++++++++++++++++++++++++++++++
 2 files changed, 304 insertions(+)
 create mode 100644 Documentation/virt/hyperv/vmbus.rst
Documentation/virt/hyperv/index.rst +1 -0
index 991bee4..caa43ab 100644
@@ -8,3 +8,4 @@ Hyper-V Enlightenments
    :maxdepth: 1
 
    overview
+   vmbus
Documentation/virt/hyperv/vmbus.rst new file +303 -0

303 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#3 lore raw

PATCH v2 3/3 Documentation: hyperv: Add overview of clocks and timers

In reply to Michael Kelley

Add documentation topic for clocks and timers when running as a guest on Hyper-V.

---
 Documentation/virt/hyperv/clocks.rst | 73 ++++++++++++++++++++++++++++++++++++
 Documentation/virt/hyperv/index.rst  |  1 +
 2 files changed, 74 insertions(+)
 create mode 100644 Documentation/virt/hyperv/clocks.rst
Documentation/virt/hyperv/clocks.rst new file +73 -0
new file mode 100644
index 0000000..2da2879
@@ -0,0 +1,73 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Clocks and Timers
+=================
+
+arm64
+-----
+On arm64, Hyper-V virtualizes the ARMv8 architectural system counter
+and timer. Guest VMs use this virtualized hardware as the Linux
+clocksource and clockevents via the standard arm_arch_timer.c
+driver, just as they would on bare metal. Linux vDSO support for the
+architectural system counter is functional in guest VMs on Hyper-V.
+While Hyper-V also provides a synthetic system clock and four synthetic
+per-CPU timers as described in the TLFS, they are not used by the
+Linux kernel in a Hyper-V guest on arm64.  However, older versions
+of Hyper-V for arm64 only partially virtualize the ARMv8
+architectural timer, such that the timer does not generate
+interrupts in the VM. Because of this limitation, running current
+Linux kernel versions on these older Hyper-V versions requires an
+out-of-tree patch to use the Hyper-V synthetic clocks/timers instead.
+
+x86/x64
+-------
+On x86/x64, Hyper-V provides guest VMs with a synthetic system clock
+and four synthetic per-CPU timers as described in the TLFS. Hyper-V
+also provides access to the virtualized TSC via the RDTSC and
+related instructions. These TSC instructions do not trap to
+the hypervisor and so provide excellent performance in a VM.
+Hyper-V performs TSC calibration, and provides the TSC frequency
+to the guest VM via a synthetic MSR.  Hyper-V initialization code
+in Linux reads this MSR to get the frequency, so it skips TSC
+calibration and sets tsc_reliable. Hyper-V provides virtualized
+versions of the PIT (in Hyper-V  Generation 1 VMs only), local
+APIC timer, and RTC. Hyper-V does not provide a virtualized HPET in
+guest VMs.
+
+The Hyper-V synthetic system clock can be read via a synthetic MSR,
+but this access traps to the hypervisor. As a faster alternative,
+the guest can configure a memory page to be shared between the guest
+and the hypervisor.  Hyper-V populates this memory page with a
+64-bit scale value and offset value. To read the synthetic clock
+value, the guest reads the TSC and then applies the scale and offset
+as described in the Hyper-V TLFS. The resulting value advances
+at a constant 10 MHz frequency. In the case of a live migration
+to a host with a different TSC frequency, Hyper-V adjusts the
+scale and offset values in the shared page so that the 10 MHz
+frequency is maintained.
+
+Starting with Windows Server 2022 Hyper-V, Hyper-V uses hardware
+support for TSC frequency scaling to enable live migration of VMs
+across Hyper-V hosts where the TSC frequency may be different.
+When a Linux guest detects that this Hyper-V functionality is
+available, it prefers to use Linux's standard TSC-based clocksource.
+Otherwise, it uses the clocksource for the Hyper-V synthetic system
+clock implemented via the shared page (identified as
+"hyperv_clocksource_tsc_page").
+
+The Hyper-V synthetic system clock is available to user space via
+vDSO, and gettimeofday() and related system calls can execute
+entirely in user space.  The vDSO is implemented by mapping the
+shared page with scale and offset values into user space.  User
+space code performs the same algorithm of reading the TSC and
+appying the scale and offset to get the constant 10 MHz clock.
+
+Linux clockevents are based on Hyper-V synthetic timer 0. While
+Hyper-V offers 4 synthetic timers for each CPU, Linux only uses
+timer 0. Interrupts from stimer0 are recorded on the "HVS" line in
+/proc/interrupts.  Clockevents based on the virtualized PIT and
+local APIC timer also work, but the Hyper-V synthetic timer is
+preferred.
+
+The driver for the Hyper-V synthetic system clock and timers is
+drivers/clocksource/hyperv_timer.c.
Documentation/virt/hyperv/index.rst +1 -0
index caa43ab..4a7a1b7 100644
@@ -9,3 +9,4 @@ Hyper-V Enlightenments
 
    overview
    vmbus
+   clocks
Signature · 1 line
1.8.3.1
maintainer
#4 lore raw
In reply to Michael Kelley
Quoted 22 lines · Michael Kelley

This documentation is a high level overview to explain the basics of Linux running as a guest on Hyper-V. The intent is to document the forest, not the trees. The Hyper-V Top Level Functional Spec provides conceptual material and API details for the core Hyper-V hypervisor, and this documentation provides additional info on how that functionality is applied to Linux. Also, there's no public documentation on VMbus or the VMbus synthetic devices, so this documentation helps fill that gap at a conceptual level. This documentation is not API-level documentation, which can be seen in the code and associated comments.

More topics will be added in future patches, including:

* Miscellaneous synthetic devices like KVP, timesync, VSS, etc.
* Virtual PCI support
* Isolated/Confidential VMs
* UIO driver

If you think I'm missing a topic that fits into the overall approach as described, feel free to suggest text, or let me know and I can add it to my list.

Quoted 8 lines

Changes in v2:
* Updated clocks.rst to use section hierarchy that matches overview.rst and vmbus.rst [Wei Liu]

Michael Kelley (3):
Documentation: hyperv: Add overview of Hyper-V enlightenments
Documentation: hyperv: Add overview of VMbus
Documentation: hyperv: Add overview of clocks and timers

Content-wise all patches look good to me.

Jonathan, let me know how you would like to handle this series. I'm happy to carry them in hyperv-next.

Thanks,
Wei.

maintainer
#5 lore raw
Quoted 32 lines · Wei Liu
Quoted 22 lines · Michael Kelley

This documentation is a high level overview to explain the basics of Linux running as a guest on Hyper-V. The intent is to document the forest, not the trees. The Hyper-V Top Level Functional Spec provides conceptual material and API details for the core Hyper-V hypervisor, and this documentation provides additional info on how that functionality is applied to Linux. Also, there's no public documentation on VMbus or the VMbus synthetic devices, so this documentation helps fill that gap at a conceptual level. This documentation is not API-level documentation, which can be seen in the code and associated comments.

More topics will be added in future patches, including:

* Miscellaneous synthetic devices like KVP, timesync, VSS, etc.
* Virtual PCI support
* Isolated/Confidential VMs
* UIO driver

If you think I'm missing a topic that fits into the overall approach as described, feel free to suggest text, or let me know and I can add it to my list.

Quoted 8 lines

Changes in v2:
* Updated clocks.rst to use section hierarchy that matches overview.rst and vmbus.rst [Wei Liu]

Michael Kelley (3):
Documentation: hyperv: Add overview of Hyper-V enlightenments
Documentation: hyperv: Add overview of VMbus
Documentation: hyperv: Add overview of clocks and timers

Quoted 4 lines

Content-wise all patches look good to me.

Jonathan, let me know how you would like to handle this series. I'm happy to carry them in hyperv-next.

I went ahead and applied them while I was in the neighborhood.

Thanks,

jon

maintainer
#6 lore raw
Quoted 34 lines · Jonathan Corbet
Quoted 32 lines · Wei Liu
Quoted 30 lines · Michael Kelley

This documentation is a high level overview to explain the basics of Linux running as a guest on Hyper-V. The intent is to document the forest, not the trees. The Hyper-V Top Level Functional Spec provides conceptual material and API details for the core Hyper-V hypervisor, and this documentation provides additional info on how that functionality is applied to Linux. Also, there's no public documentation on VMbus or the VMbus synthetic devices, so this documentation helps fill that gap at a conceptual level. This documentation is not API-level documentation, which can be seen in the code and associated comments.

More topics will be added in future patches, including:

* Miscellaneous synthetic devices like KVP, timesync, VSS, etc.
* Virtual PCI support
* Isolated/Confidential VMs
* UIO driver

If you think I'm missing a topic that fits into the overall approach as described, feel free to suggest text, or let me know and I can add it to my list.

Changes in v2:
* Updated clocks.rst to use section hierarchy that matches overview.rst and vmbus.rst [Wei Liu]

Michael Kelley (3):
Documentation: hyperv: Add overview of Hyper-V enlightenments
Documentation: hyperv: Add overview of VMbus
Documentation: hyperv: Add overview of clocks and timers

Quoted 6 lines
Quoted 4 lines

Content-wise all patches look good to me.

Jonathan, let me know how you would like to handle this series. I'm happy to carry them in hyperv-next.

I went ahead and applied them while I was in the neighborhood.

Thanks!