x86/xen: allow privcmd hypercalls to be preempted
Subsystems (2)
PATCH v6 1/1 x86/xen: allow privcmd hypercalls to be preempted
Hypercalls submitted by user space tools via the privcmd driver can take a long time (potentially many 10s of seconds) if the hypercall has many sub-operations.
A fully preemptible kernel may deschedule such as task in any upcall called from a hypercall continuation.
However, in a kernel with voluntary or no preemption, hypercall continuations in Xen allow event handlers to be run but the task issuing the hypercall will not be descheduled until the hypercall is complete and the ioctl returns to user space. These long running tasks may also trigger the kernel's soft lockup detection.
Add xen_preemptible_hcall_begin() and xen_preemptible_hcall_end() to bracket hypercalls that may be preempted. Use these in the privcmd driver.
When returning from an upcall, call xen_maybe_preempt_hcall() which adds a schedule point if if the current task was within a preemptible hypercall.
Since _cond_resched() can move the task to a different CPU, clear and set xen_in_preemptible_hcall around the call.
---
Changes in v6:
- Add an additional should_resched() check.
Changes in v5:
- Add xen_maybe_preempt_hcall() instead of copy-and-paste of assembly.
Changes in v4:
- Fix 32-bit build.
Changes in v3:
- Export xen_in_preemptible_hcall (to fix modular privcmd driver).
Changes in v2:
- Use per-cpu variable to mark preemptible regions
- Call preempt_schedule_irq() from the correct place in xen_hypervisor_callback
--- arch/x86/kernel/entry_32.S | 3 +++ arch/x86/kernel/entry_64.S | 3 +++ drivers/xen/Makefile | 2 +- drivers/xen/preempt.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ drivers/xen/privcmd.c | 2 ++ include/xen/xen-ops.h | 26 ++++++++++++++++++++++++++ 6 files changed, 79 insertions(+), 1 deletion(-) create mode 100644 drivers/xen/preempt.c
arch/x86/kernel/entry_32.S +3 -0
@@ -982,6 +982,9 @@ ENTRY(xen_hypervisor_callback) ENTRY(xen_do_upcall) 1: mov %esp, %eax call xen_evtchn_do_upcall +#ifndef CONFIG_PREEMPT + call xen_maybe_preempt_hcall +#endif jmp ret_from_intr CFI_ENDPROC ENDPROC(xen_hypervisor_callback)
arch/x86/kernel/entry_64.S +3 -0
@@ -1208,6 +1208,9 @@ ENTRY(xen_do_hypervisor_callback) # do_hypervisor_callback(struct *pt_regs) popq %rsp CFI_DEF_CFA_REGISTER rsp decl PER_CPU_VAR(irq_count) +#ifndef CONFIG_PREEMPT + call xen_maybe_preempt_hcall +#endif jmp error_exit CFI_ENDPROC END(xen_do_hypervisor_callback)
drivers/xen/Makefile +1 -1
@@ -2,7 +2,7 @@ ifeq ($(filter y, $(CONFIG_ARM) $(CONFIG_ARM64)),) obj-$(CONFIG_HOTPLUG_CPU) += cpu_hotplug.o endif obj-$(CONFIG_X86) += fallback.o -obj-y += grant-table.o features.o balloon.o manage.o +obj-y += grant-table.o features.o balloon.o manage.o preempt.o obj-y += events/ obj-y += xenbus/
drivers/xen/preempt.c new file +44 -0
44 lines not on this page · raw message
drivers/xen/privcmd.c +2 -0
@@ -56,10 +56,12 @@ static long privcmd_ioctl_hypercall(void __user *udata)if (copy_from_user (&hypercall, udata,sizeof (hypercall)))return -EFAULT ; +xen_preemptible_hcall_begin (); ret =privcmd_call (hypercall.op, hypercall.arg[0 ], hypercall.arg[1 ], hypercall.arg[2 ], hypercall.arg[3 ], hypercall.arg[4 ]); +xen_preemptible_hcall_end ();return ret; }
include/xen/xen-ops.h +26 -0
@@ -46,4 +46,30 @@ static inline efi_system_table_t __init *xen_efi_probe(void) }#endif +#ifdef CONFIG_PREEMPT + +static inline void xen_preemptible_hcall_begin (void ) +{ +} + +static inline void xen_preemptible_hcall_end (void ) +{ +} + +#else + +DECLARE_PER_CPU (bool, xen_in_preemptible_hcall); + +static inline void xen_preemptible_hcall_begin (void ) +{ +__this_cpu_write (xen_in_preemptible_hcall, true); +} + +static inline void xen_preemptible_hcall_end (void ) +{ +__this_cpu_write (xen_in_preemptible_hcall, false); +} + +#endif /* CONFIG_PREEMPT */ +#endif /* INCLUDE_XEN_OPS_H */
Signature · 1 line
1.7.10.4
Quoted 17 lines · David Vrabel
Hypercalls submitted by user space tools via the privcmd driver can take a long time (potentially many 10s of seconds) if the hypercall has many sub-operations.
A fully preemptible kernel may deschedule such as task in any upcall called from a hypercall continuation.
However, in a kernel with voluntary or no preemption, hypercall continuations in Xen allow event handlers to be run but the task issuing the hypercall will not be descheduled until the hypercall is complete and the ioctl returns to user space. These long running tasks may also trigger the kernel's soft lockup detection.
Add
xen_preemptible_hcall_begin()andxen_preemptible_hcall_end()to bracket hypercalls that may be preempted. Use these in the privcmd driver.
Quoted 8 lines
When returning from an upcall, call
xen_maybe_preempt_hcall()which adds a schedule point if if the current task was within a preemptible hypercall.Since
_cond_resched()can move the task to a different CPU, clear and setxen_in_preemptible_hcallaround the call.Signed-off-by David Vrabel <[email protected]>
Quoted 22 lines · Boris Ostrovsky
Quoted 21 lines · David Vrabel
Hypercalls submitted by user space tools via the privcmd driver can take a long time (potentially many 10s of seconds) if the hypercall has many sub-operations.
A fully preemptible kernel may deschedule such as task in any upcall called from a hypercall continuation.
However, in a kernel with voluntary or no preemption, hypercall continuations in Xen allow event handlers to be run but the task issuing the hypercall will not be descheduled until the hypercall is complete and the ioctl returns to user space. These long running tasks may also trigger the kernel's soft lockup detection.
Add
xen_preemptible_hcall_begin()andxen_preemptible_hcall_end()to bracket hypercalls that may be preempted. Use these in the privcmd driver.When returning from an upcall, call
xen_maybe_preempt_hcall()which adds a schedule point if if the current task was within a preemptible hypercall.
Quoted 6 lines
Quoted 4 lines
Since
_cond_resched()can move the task to a different CPU, clear and setxen_in_preemptible_hcallaround the call.Signed-off-by David Vrabel <[email protected]>Reviewed-by Boris Ostrovsky <[email protected]>
Thanks. Applied to stable/for-linus-3.20.
David