xen: spin in xen_flush_tlb_others if a target cpu is in gup_fast()

get_user_pages_fast() relies on cross-cpu tlb flushes being a barrier between clearing and setting a pte, and before freeing a pagetable page.

It normally implements this by disabling interrupts to suspend tlb flush IPIs. This doesn't work for us because we don't use kernel-visible IPIs for tlb flushes, so just spin manually until none of our target cpus are in gup_fast().

index aa16ef4..b0523f8 100644
@@ -50,6 +50,7 @@

#include <asm/setup.h>
#include <asm/paravirt.h>
#include <asm/linkage.h>
+#include <asm/mmu.h>

#include <asm/xen/hypercall.h>
#include <asm/xen/hypervisor.h>
@@ -1334,6 +1335,14 @@ static void xen_flush_tlb_others(const struct cpumask *cpus,

	MULTI_mmuext_op(mcs.mc, &args->op, 1, NULL, DOMID_SELF);

	xen_mc_issue(PARAVIRT_LAZY_MMU);

+
+ /*
+ * If we're racing with a get_user_pages_fast(), wait here
+ * until it has finishes so that it can use cross-cpu tlb
+ * flush as a barrier.
+ */
+ while (cpus_intersects(*in_gup_cpumask, *cpus))
+ cpu_relax();
}

static unsigned long xen_read_cr3(void)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>