Skip to contentSkip to conversation

Drivers: hv: Move Hyper-V clock/timer code to separate clocksource driver

PATCH Superseded Michael Kelley 2 participants 4 messages Mar 17, 2019 – Mar 21, 2019

Subsystems (7)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover Drivers: hv: Move Hyper-V clock/timer code to separate clocksource driver

This patch series moves Hyper-V clock/timer code to a separate Hyper-V clocksource driver. Previously, Hyper-V clock/timer code and data structures were mixed in with other Hyper-V code in the ISA independent drivers/hv code as well as in arch dependent code. The new Hyper-V clocksource driver is ISA independent, with a just few dependencies on arch specific functions. The patch series does not change any behavior or functionality -- it only reorganizes the existing code and fixes up the linkages. A few places outside of Hyper-V code are fixed up to use the new #include file structure.

This restructuring is in response to Marc Zyngier's review comments on supporting Hyper-V running on ARM64, and is a good idea in general. It increases the amount of code shared between the x86 and ARM64 architectures, and reduces the size of the new code for supporting Hyper-V on ARM64. A new version of the Hyper-V on ARM64 patches will follow once this clocksource restructuring is accepted.

The code is currently diff'ed against Linux 5.0. I'll rebase to linux-next once 5.1-rc1 is available.

Changes in v2:
* Revised commit short descriptions so the distinction between the two patches is clearer [GregKH]
* Renamed new clocksource driver files and functions to use existing "timer" and "stimer" names instead of introducing "syntimer". [Vitaly Kuznetsov]
* Introduced CONFIG_HYPER_TIMER to fix build problem when CONFIG_HYPERV=m [Vitaly Kuznetsov]
* Added "Suggested-by: Marc Zyngier"

Michael Kelley (2):
Drivers: hv: Create Hyper-V clocksource driver from existing
clockevents code
Drivers: hv: Move Hyper-V clocksource code to new clocksource driver

 MAINTAINERS                          |   2 +
 arch/x86/entry/vdso/vclock_gettime.c |   1 +
 arch/x86/entry/vdso/vma.c            |   2 +-
 arch/x86/hyperv/hv_init.c            |  91 +---------
 arch/x86/include/asm/hyperv-tlfs.h   |   6 +
 arch/x86/include/asm/mshyperv.h      |  80 ++-------
 arch/x86/kernel/cpu/mshyperv.c       |   2 +
 arch/x86/kvm/x86.c                   |   1 +
 drivers/clocksource/Makefile         |   1 +
 drivers/clocksource/hyperv_timer.c   | 328 +++++++++++++++++++++++++++++++++++
 drivers/hv/Kconfig                   |   3 +
 drivers/hv/hv.c                      | 154 ----------------
 drivers/hv/hyperv_vmbus.h            |   3 -
 drivers/hv/vmbus_drv.c               |  39 +++--
 include/clocksource/hyperv_timer.h   | 104 +++++++++++
 15 files changed, 485 insertions(+), 332 deletions(-)
 create mode 100644 drivers/clocksource/hyperv_timer.c
 create mode 100644 include/clocksource/hyperv_timer.h
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH v2 1/2 Drivers: hv: Create Hyper-V clocksource driver from existing clockevents code

Clockevents code for Hyper-V synthetic timers is currently mixed in with other Hyper-V code. Move the code to a Hyper-V specific driver in the "clocksource" directory. Update the VMbus driver to call initialization and cleanup routines since the Hyper-V synthetic timers are not independently enumerated in ACPI.

No behavior is changed and no new functionality is added.

---
 MAINTAINERS                        |   2 +
 arch/x86/include/asm/hyperv-tlfs.h |   6 ++
 arch/x86/kernel/cpu/mshyperv.c     |   2 +
 drivers/clocksource/Makefile       |   1 +
 drivers/clocksource/hyperv_timer.c | 206 +++++++++++++++++++++++++++++++++++++
 drivers/hv/Kconfig                 |   3 +
 drivers/hv/hv.c                    | 154 ---------------------------
 drivers/hv/hyperv_vmbus.h          |   3 -
 drivers/hv/vmbus_drv.c             |  39 +++----
 include/clocksource/hyperv_timer.h |  26 +++++
 10 files changed, 266 insertions(+), 176 deletions(-)
 create mode 100644 drivers/clocksource/hyperv_timer.c
 create mode 100644 include/clocksource/hyperv_timer.h
MAINTAINERS +2 -0
index 21ab064..63142da 100644
@@ -7159,6 +7159,7 @@ F:	arch/x86/include/asm/trace/hyperv.h
 F:	arch/x86/include/asm/hyperv-tlfs.h
 F:	arch/x86/kernel/cpu/mshyperv.c
 F:	arch/x86/hyperv
+F:	drivers/clocksource/hyperv_timer.c
 F:	drivers/hid/hid-hyperv.c
 F:	drivers/hv/
 F:	drivers/input/serio/hyperv-keyboard.c
@@ -7168,6 +7169,7 @@ F:	drivers/scsi/storvsc_drv.c
 F:	drivers/uio/uio_hv_generic.c
 F:	drivers/video/fbdev/hyperv_fb.c
 F:	net/vmw_vsock/hyperv_transport.c
+F:	include/clocksource/hyperv_timer.h
 F:	include/linux/hyperv.h
 F:	include/uapi/linux/hyperv.h
 F:	tools/hv/
arch/x86/include/asm/hyperv-tlfs.h +6 -0
index 2bdbbbc..ee62f57 100644
@@ -401,6 +401,12 @@ enum HV_GENERIC_SET_FORMAT {
 #define HV_STATUS_INVALID_CONNECTION_ID		18
 #define HV_STATUS_INSUFFICIENT_BUFFERS		19
 
+/*
+ * The Hyper-V TimeRefCount register and the TSC
+ * page provide a guest VM clock with 100ns tick rate
+ */
+#define HV_CLOCK_HZ (NSEC_PER_SEC/100)
+
 typedef struct _HV_REFERENCE_TSC_PAGE {
 	__u32 tsc_sequence;
 	__u32 res1;
arch/x86/kernel/cpu/mshyperv.c +2 -0
index e81a2db..f53a35a 100644
@@ -21,6 +21,7 @@
 #include <linux/irq.h>
 #include <linux/kexec.h>
 #include <linux/i8253.h>
+#include <linux/random.h>
 #include <asm/processor.h>
 #include <asm/hypervisor.h>
 #include <asm/hyperv-tlfs.h>
@@ -84,6 +85,7 @@ __visible void __irq_entry hv_stimer0_vector_handler(struct pt_regs *regs)
 	inc_irq_stat(hyperv_stimer0_count);
 	if (hv_stimer0_handler)
 		hv_stimer0_handler();
+	add_interrupt_randomness(HYPERV_STIMER0_VECTOR, 0);
 	ack_APIC_irq();
 
 	exiting_irq();
drivers/clocksource/Makefile +1 -0
index be6e0fb..cd284a4 100644
@@ -83,3 +83,4 @@ obj-$(CONFIG_ATCPIT100_TIMER)		+= timer-atcpit100.o
 obj-$(CONFIG_RISCV_TIMER)		+= timer-riscv.o
 obj-$(CONFIG_CSKY_MP_TIMER)		+= timer-mp-csky.o
 obj-$(CONFIG_GX6605S_TIMER)		+= timer-gx6605s.o
+obj-$(CONFIG_HYPERV_TIMER)		+= hyperv_timer.o
drivers/clocksource/hyperv_timer.c new file +206 -0

206 lines not on this page · raw message

drivers/hv/Kconfig +3 -0
index 1c1a251..c423e57 100644
@@ -10,6 +10,9 @@ config HYPERV
 	  Select this option to run Linux as a Hyper-V client operating
 	  system.
 
+config HYPERV_TIMER
+	def_bool HYPERV
+
 config HYPERV_TSCPAGE
        def_bool HYPERV && X86_64
 
drivers/hv/hv.c +0 -154

206 lines not on this page · raw message

drivers/hv/hyperv_vmbus.h +0 -3
index cb86b133..ffd4ad8 100644
@@ -151,7 +151,6 @@ struct hv_per_cpu_context {
 	 * per-cpu list of the channels based on their CPU affinity.
 	 */
 	struct list_head chan_list;
-	struct clock_event_device *clk_evt;
 };
 
 struct hv_context {
@@ -189,8 +188,6 @@ extern int hv_post_message(union hv_connection_id connection_id,
 
 extern int hv_synic_cleanup(unsigned int cpu);
 
-extern void hv_synic_clockevents_cleanup(void);
-
 /* Interface */
 
 
drivers/hv/vmbus_drv.c +20 -19
index 000b53e..e71b04b 100644
@@ -43,6 +43,7 @@
 #include <linux/kdebug.h>
 #include <linux/efi.h>
 #include <linux/random.h>
+#include <clocksource/hyperv_timer.h>
 #include "hyperv_vmbus.h"
 
 struct vmbus_dynid {
@@ -939,17 +940,6 @@ static void vmbus_onmessage_work(struct work_struct *work)
 	kfree(ctx);
 }
 
-static void hv_process_timer_expiration(struct hv_message *msg,
-					struct hv_per_cpu_context *hv_cpu)
-{
-	struct clock_event_device *dev = hv_cpu->clk_evt;
-
-	if (dev->event_handler)
-		dev->event_handler(dev);
-
-	vmbus_signal_eom(msg, HVMSG_TIMER_EXPIRED);
-}
-
 void vmbus_on_msg_dpc(unsigned long data)
 {
 	struct hv_per_cpu_context *hv_cpu = (void *)data;
@@ -1143,9 +1133,10 @@ static void vmbus_isr(void)
 
 	/* Check if there are actual msgs to be processed */
 	if (msg->header.message_type != HVMSG_NONE) {
-		if (msg->header.message_type == HVMSG_TIMER_EXPIRED)
-			hv_process_timer_expiration(msg, hv_cpu);
-		else
+		if (msg->header.message_type == HVMSG_TIMER_EXPIRED) {
+			hv_stimer0_isr();
+			vmbus_signal_eom(msg, HVMSG_TIMER_EXPIRED);
+		} else
 			tasklet_schedule(&hv_cpu->msg_dpc);
 	}
 
@@ -1248,8 +1239,8 @@ static int vmbus_bus_init(void)
 	if (ret)
 		goto err_alloc;
 	/*
-	 * Initialize the per-cpu interrupt state and
-	 * connect to the host.
+	 * Initialize the per-cpu interrupt state and stimer state.
+	 * Then connect to the host.
 	 */
 	ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "hyperv/vmbus:online",
 				hv_synic_init, hv_synic_cleanup);
@@ -1257,6 +1248,10 @@ static int vmbus_bus_init(void)
 		goto err_alloc;
 	hyperv_cpuhp_online = ret;
 
+	ret = hv_stimer_alloc(VMBUS_MESSAGE_SINT);
+	if (ret < 0)
+		goto err_stimer_alloc;
+
 	ret = vmbus_connect();
 	if (ret)
 		goto err_connect;
@@ -1301,6 +1296,8 @@ static int vmbus_bus_init(void)
 	return 0;
 
 err_connect:
+	hv_stimer_free();
+err_stimer_alloc:
 	cpuhp_remove_state(hyperv_cpuhp_online);
 err_alloc:
 	hv_synic_free();
@@ -1971,7 +1968,7 @@ static int vmbus_acpi_add(struct acpi_device *device)
 
 static void hv_kexec_handler(void)
 {
-	hv_synic_clockevents_cleanup();
+	hv_stimer_global_cleanup();
 	vmbus_initiate_unload(false);
 	vmbus_connection.conn_state = DISCONNECTED;
 	/* Make sure conn_state is set as hv_synic_cleanup checks for it */
@@ -1982,6 +1979,8 @@ static void hv_kexec_handler(void)
 
 static void hv_crash_handler(struct pt_regs *regs)
 {
+	int cpu;
+
 	vmbus_initiate_unload(true);
 	/*
 	 * In crash handler we can't schedule synic cleanup for all CPUs,
@@ -1989,7 +1988,9 @@ static void hv_crash_handler(struct pt_regs *regs)
 	 * for kdump.
 	 */
 	vmbus_connection.conn_state = DISCONNECTED;
-	hv_synic_cleanup(smp_processor_id());
+	cpu = smp_processor_id();
+	hv_stimer_cleanup(cpu);
+	hv_synic_cleanup(cpu);
 	hyperv_cleanup();
 };
 
@@ -2038,7 +2039,7 @@ static void __exit vmbus_exit(void)
 	hv_remove_kexec_handler();
 	hv_remove_crash_handler();
 	vmbus_connection.conn_state = DISCONNECTED;
-	hv_synic_clockevents_cleanup();
+	hv_stimer_global_cleanup();
 	vmbus_disconnect();
 	hv_remove_vmbus_irq();
 	for_each_online_cpu(cpu) {
include/clocksource/hyperv_timer.h new file +26 -0
new file mode 100644
index 0000000..1a5698c
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Definitions for the clocksource provided by the Hyper-V
+ * hypervisor to guest VMs, as described in the Hyper-V Top
+ * Level Functional Spec (TLFS).
+ *
+ * Copyright (C) 2019, Microsoft, Inc.
+ *
+ * Author:  Michael Kelley <[email protected]>
+ */
+
+#ifndef __CLKSOURCE_HYPERV_TIMER_H
+#define __CLKSOURCE_HYPERV_TIMER_H
+
+#define HV_MAX_MAX_DELTA_TICKS 0xffffffff
+#define HV_MIN_DELTA_TICKS 1
+
+/* Routines called by the VMbus driver */
+extern int hv_stimer_alloc(int sint);
+extern void hv_stimer_free(void);
+extern int hv_stimer_cleanup(unsigned int cpu);
+extern void hv_stimer_global_cleanup(void);
+extern void hv_stimer0_isr(void);
+
+#endif
Signature · 6 lines
1.8.3.1

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#2 lore raw

PATCH v2 2/2 Drivers: hv: Move Hyper-V clocksource code to new clocksource driver

In reply to Michael Kelley

Code for the Hyper-V specific clocksources is currently mixed in with other Hyper-V code. Move the code to the Hyper-V specific driver in the "clocksource" directory, while separating out ISA dependencies so that the clocksource driver remains ISA independent. Update the Hyper-V initialization code to call initialization and cleanup routines since the Hyper-V synthetic timers are not independently enumerated in ACPI. Update Hyper-V clocksource users KVM and VDSO to get definitions from a new include file.

No behavior is changed and no new functionality is added.

---
 arch/x86/entry/vdso/vclock_gettime.c |   1 +
 arch/x86/entry/vdso/vma.c            |   2 +-
 arch/x86/hyperv/hv_init.c            |  91 ++------------------------
 arch/x86/include/asm/mshyperv.h      |  80 ++++-------------------
 arch/x86/kvm/x86.c                   |   1 +
 drivers/clocksource/hyperv_timer.c   | 122 +++++++++++++++++++++++++++++++++++
 include/clocksource/hyperv_timer.h   |  78 ++++++++++++++++++++++
 7 files changed, 219 insertions(+), 156 deletions(-)
arch/x86/entry/vdso/vclock_gettime.c +1 -0
index 007b3fe9..59bb5d1 100644
@@ -21,6 +21,7 @@
 #include <linux/math64.h>
 #include <linux/time.h>
 #include <linux/kernel.h>
+#include <clocksource/hyperv_timer.h>
 
 #define gtod (&VVAR(vsyscall_gtod_data))
 
arch/x86/entry/vdso/vma.c +1 -1
index babc4e7..99b38e9 100644
@@ -22,7 +22,7 @@
 #include <asm/page.h>
 #include <asm/desc.h>
 #include <asm/cpufeature.h>
-#include <asm/mshyperv.h>
+#include <clocksource/hyperv_timer.h>
 
 #if defined(CONFIG_X86_64)
 unsigned int __read_mostly vdso64_enabled = 1;
arch/x86/hyperv/hv_init.c +5 -86

110 lines not on this page · raw message

arch/x86/include/asm/mshyperv.h +11 -69

90 lines not on this page · raw message

arch/x86/kvm/x86.c +1 -0
index 941f932..9e2b2c6 100644
@@ -70,6 +70,7 @@
 #include <asm/mshyperv.h>
 #include <asm/hypervisor.h>
 #include <asm/intel_pt.h>
+#include <clocksource/hyperv_timer.h>
 
 #define CREATE_TRACE_POINTS
 #include "trace.h"
drivers/clocksource/hyperv_timer.c +122 -0

131 lines not on this page · raw message

include/clocksource/hyperv_timer.h +78 -0

88 lines not on this page · raw message

Signature · 1 line
1.8.3.1
Reviewed-by on the v2 series
#3 lore raw
In reply to Michael Kelley
Quoted 20 lines · Michael Kelley

This patch series moves Hyper-V clock/timer code to a separate Hyper-V clocksource driver. Previously, Hyper-V clock/timer code and data structures were mixed in with other Hyper-V code in the ISA independent drivers/hv code as well as in arch dependent code. The new Hyper-V clocksource driver is ISA independent, with a just few dependencies on arch specific functions. The patch series does not change any behavior or functionality -- it only reorganizes the existing code and fixes up the linkages. A few places outside of Hyper-V code are fixed up to use the new #include file structure.

This restructuring is in response to Marc Zyngier's review comments on supporting Hyper-V running on ARM64, and is a good idea in general. It increases the amount of code shared between the x86 and ARM64 architectures, and reduces the size of the new code for supporting Hyper-V on ARM64. A new version of the Hyper-V on ARM64 patches will follow once this clocksource restructuring is accepted.

The code is currently diff'ed against Linux 5.0. I'll rebase to linux-next once 5.1-rc1 is available.

Quoted 8 lines

Changes in v2:
* Revised commit short descriptions so the distinction between the two patches is clearer [GregKH]
* Renamed new clocksource driver files and functions to use existing "timer" and "stimer" names instead of introducing "syntimer". [Vitaly Kuznetsov]
* Introduced CONFIG_HYPER_TIMER to fix build problem when CONFIG_HYPERV=m [Vitaly Kuznetsov]

Actually, it is "CONFIG_HYPERV_TIMER" but the typo appears only in this blurb :-)

Quoted 6 lines

* Added "Suggested-by: Marc Zyngier"

Michael Kelley (2):
Drivers: hv: Create Hyper-V clocksource driver from existing
clockevents code
Drivers: hv: Move Hyper-V clocksource code to new clocksource driver

Thanks for fixing v1 issues!

Quoted 19 lines
 MAINTAINERS                          |   2 +
 arch/x86/entry/vdso/vclock_gettime.c |   1 +
 arch/x86/entry/vdso/vma.c            |   2 +-
 arch/x86/hyperv/hv_init.c            |  91 +---------
 arch/x86/include/asm/hyperv-tlfs.h   |   6 +
 arch/x86/include/asm/mshyperv.h      |  80 ++-------
 arch/x86/kernel/cpu/mshyperv.c       |   2 +
 arch/x86/kvm/x86.c                   |   1 +
 drivers/clocksource/Makefile         |   1 +
 drivers/clocksource/hyperv_timer.c   | 328 +++++++++++++++++++++++++++++++++++
 drivers/hv/Kconfig                   |   3 +
 drivers/hv/hv.c                      | 154 ----------------
 drivers/hv/hyperv_vmbus.h            |   3 -
 drivers/hv/vmbus_drv.c               |  39 +++--
 include/clocksource/hyperv_timer.h   | 104 +++++++++++
 15 files changed, 485 insertions(+), 332 deletions(-)
 create mode 100644 drivers/clocksource/hyperv_timer.c
 create mode 100644 include/clocksource/hyperv_timer.h
Signature · 1 line
Vitaly