Skip to contentSkip to conversation

alpha: fix stale TLB translations breaking copy-on-write and writeback

PATCH In review Magnus Lindholm 2 participants 14 messages Oct 10 – Oct 11

Subsystems (2)
lore
Share & tools
lore

Magnus Lindholm

author maintainer
#0 lore raw

v5 cover alpha: fix stale TLB translations breaking copy-on-write and writeback

On Alpha, stale TLB translations can break copy-on-write and shared-mapping
writeback: a multi-threaded process can lose its own stores, read data belonging to its child, or lose writes through a shared file mapping. The copy-on-write failures need more than one CPU; the writeback failure also happens on a uniprocessor.

This is the first of two series, based on for-next at 33465f6ab697 ("alpha: support the remaining kernel compression formats"). Apply all eight patches here, then the two patches in "alpha: complete the direct-mm shootdown fixes" v4. There is no interleaving or external prerequisite for this first series.

Patch 1 moves here from v2 of the direct-switch series:

https://lore.kernel.org/linux-alpha/[email protected]/

It loads the context when switch_mm() is called directly for current, before kthread_use_mm() can reach the hook added by patch 2. This closes the dependency that previously required applying one patch from the other series first. Its functional code and Matt's review and test tags are unchanged; v5 expands its comment and changelog to name all three callers.

Patch 2 completes the deferred-ASN handshake from finish_arch_post_lock_switch(), including the first switch to a newly created task. Its old location after alpha_switch_to() was bypassed when a new task reached schedule_tail(), leaving asn_lock set.

The hook runs after interrupts are enabled. Set need_new_asn whether switch_mm() reused or allocated an ASN, so a deferred IPI cannot leave a live hardware context with a zero slot. This is also required before the second series uses those slots to decide whether to skip shootdowns. Drop the preemptible() guard: Alpha has no kernel preemption and the direct-switch callers do not sleep before the hook, including when RCU_STRICT_GRACE_PERIOD enables PREEMPT_COUNT.

Patches 3, 5 and 6 test current->mm before issuing targeted tbi(), since a lazy active_mm does not establish which ASN is actually loaded. Patches 4, 7 and 8 retire the calling CPU's context when the local current-context test fails; smp_call_function() only visits the other CPUs.

Patch 4 deliberately precedes patch 5. The missing else branch must be present before lazy callers are routed to it, so no intermediate commit loses their local invalidate. This matters on EV7 where a foreign-context tbi() appeared to work. The full-mm and icache callers retain their active_mm tests because those paths load a new context instead of using a targeted tbi(). The second series separately changes their IPI handlers to retire slots on lazy CPUs rather than keeping those slots visible.

Testing

The combined TLB v5 and MMU v4 stack passed cross-build checks with GCC 15.0.1 for arch/alpha/kernel/, arch/alpha/mm/, kernel/sched/core.o and kernel/kthread.o: ALPHA_GENERIC SMP and UP with COMPACTION=n, and SMP with COMPACTION=y, RCU_STRICT_GRACE_PERIOD=y and PREEMPT_COUNT=y. Full ES40 kernel builds with COMPACTION=y and COMPACTION=n also passed. All ten patches
pass checkpatch without warnings, and sequential application reproduces the committed trees at both series boundaries.

The combined TLB v5 and MMU v4 stack also passed an ES40 SMP run with three CPUs online and COMPACTION=y: nine lost-store rounds, nine stale-read rounds and ten mkclean4 writeback rounds on ext4. Compaction ran throughout; pgmigrate_success increased by 7424. No new kernel warnings or errors were observed in the captured log. This exercises the migration path. A second boot with COMPACTION=n also passed nine lost-store, nine stale-read and ten writeback rounds, exercising the direct flush_tlb_page() writeback path. No new kernel warnings or errors were observed in that run either. UP was compile-tested only. The historical results below are from earlier revisions.

Historical results from the earlier stale-TLB series, on an ES40, EV68AL (21264C) Tsunami with three CPUs, v7.2-rc2 and v7.2-rc6:

before after
smoke test, stale-read check 7 of 9 rounds 0 of 9
lost stores 11 of 40 0 of 400

  writeback (mkclean4), SMP [*]       10 of 10        0 of 10
  writeback (mkclean4), UP  [*]       every round     0 of 8

tst-malloc-fork-deadlock-
malloc-check 8 of 10 fail 25/25 pass

[*] CONFIG_COMPACTION=n.

alpha-cow-smoketest.c uses pthreads and forked children. Each thread owns its slot, and the main thread reads only after joining. Slots are 128 bytes apart so threads share a page; writing once and reading repeatedly avoids hiding stale translations with another faulting write. It does not fail when restricted to one CPU.

mkclean4.c exercises patches 4 and 5 together on SMP, and patch 6 on UP. It compares a small MAP_SHARED mapping with its backing file after background writeback, with the writer and flusher pinned to the same CPU on SMP. It needs root and COMPACTION=n. With COMPACTION=y, Alpha overrides ptep_clear_flush() to use migrate_flush_tlb_page(), so folio_mkclean() does not exercise flush_tlb_page(). Earlier unpatched COMPACTION=y runs passed 103 rounds; separate regression runs under continuous compaction migrated 368522 folios without failures.

Matt's earlier ES47 (EV7) testing on v7.3-rc1 with one CPU online found no regressions in fork/COW, writeback and gdb breakpoint tests on SMP and UP builds. He could not reproduce the unpatched writeback failure despite counters showing foreign-context targeted invalidates. The fix rests on the tbi() contract, not on every implementation exhibiting the failure.

Patches 7 and 8 still have no isolated reproducer. A gdb breakpoint exerciser covered patch 8's path for regressions. Matt suggested move_pages() on another process's hugetlb mapping, with the caller pinned to the CPU the target last ran on, as a possible reproducer for patch 7. That is unverified; the proposed hugetlb support is not in this base.

Changes since v4:

- No functional changes; only comments, changelogs and review metadata.
- Drop stale Message-ID trailers from patches 1 and 3-8.
- Mention do_shoot_lazy_tlb() in patch 1's comment and changelog, and explain that Alpha does not select MMU_LAZY_TLB_SHOOTDOWN.
- Clarify patch 2's hook comment: kthread_use_mm() can call it with preemption enabled, but Alpha has no kernel preemption.
- Refer explicitly to patch 1 in patch 3's changelog.
- Rewrap patch 4's opening paragraph.
- Add Matt's Reviewed-by to patch 2 following his v4 review; retain the existing tags on the other seven patches. Matt has now reviewed the whole series and compile-tested v4 at every commit on SMP and UP.
- Update the companion-series reference to MMU v4.

Changes since v3:

- Move the reviewed direct-load fix from MMU v2 to patch 1, making the TLB and MMU series applicable in sequence.
- Drop the post-switch hook's preemptible() guard.
- Set need_new_asn on both reused and newly allocated ASN paths.
- Put the missing local flush_tlb_page() else branch before changing

    the current->mm test, as requested by Matt.

- Rebase on for-next and update numbering and dependency descriptions.
- Mention the proposed hugetlb caller without claiming a new reproducer.
- Drop the old review tag from the materially changed hook patch; retain tags on the other patches.

Thanks to Matt Turner for the review and EV7 testing.

v4: https://lore.kernel.org/linux-alpha/[email protected]/

v3: https://lore.kernel.org/linux-alpha/[email protected]/

Magnus Lindholm (8):
alpha: load the MMU context when switch_mm() switches the current task
alpha: run check_mmu_context() from finish_arch_post_lock_switch()
alpha: only use a targeted tbi() when the target mm is really current
alpha: invalidate the local context in flush_tlb_page()
alpha: fix the local TLB invalidate in flush_tlb_page()
alpha: fix the local TLB invalidate in the UP flush_tlb_page()
alpha: invalidate the local context in flush_tlb_mm()
alpha: invalidate the local context in flush_icache_user_page()

 arch/alpha/include/asm/mmu_context.h | 28 +++++++++++++++++++++++++---
 arch/alpha/include/asm/switch_to.h   |  1 -
 arch/alpha/include/asm/tlbflush.h    |  3 ++-
 arch/alpha/kernel/smp.c              | 15 +++++++++++++--
 4 files changed, 40 insertions(+), 7 deletions(-)

base-commit: 33465f6ab697abceafd9a340067a544d551c4412

Signature · 1 line
2.43.0
author maintainer
#1 lore raw

PATCH v5 1/8 alpha: load the MMU context when switch_mm() switches the current task

ev5_switch_mm() only prepares the incoming PCB. The context is installed by PAL_swpctx, which alpha_switch_to() issues against that PCB on the way out of the scheduler.

Direct callers reach switch_mm() without going through alpha_switch_to():
kthread_use_mm(), which borrows an mm for the current kernel thread, sched_force_init_mm() on the CPU-hotplug teardown path, and do_shoot_lazy_tlb(), which switches to init_mm from an IPI. None explicitly loads the context, so the task can carry on running
under whatever was loaded before while current->mm says otherwise.

The stale page-table root need not be swapper_pg_dir; it may belong to a user process that ran on the CPU earlier, and the kthread's user accesses can then read and write that process's memory wherever the stale mappings allow. Translations taken that way can also end up tagged with the borrowed mm's ASN: ev5_switch_mm() writes that ASN into the PCB, which the next PAL_swpctx installs against the stale ptbr. An address the stale tables do not map faults instead, and since do_page_fault() resolves faults against current->mm without reloading the context, the same access can fault again on return.

sched_force_init_mm() needs CONFIG_HOTPLUG_CPU and do_shoot_lazy_tlb() needs CONFIG_MMU_LAZY_TLB_SHOOTDOWN. Alpha selects neither, so only kthread_use_mm() is reachable in practice. Test the caller's identity rather than special-casing individual callers.

The scheduler passes the incoming task, which is not current until alpha_switch_to() runs; all three direct callers pass current. Test for that and load the context the way activate_mm() does. All three run with interrupts disabled across the switch, so this completes before any shootdown can be taken and needs no asn_lock handshake.

Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")
---
 arch/alpha/include/asm/mmu_context.h | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
arch/alpha/include/asm/mmu_context.h +12 -1
index eee8fe836a59..d366f89fee58 100644
@@ -130,6 +130,8 @@ __get_new_mm_context(struct mm_struct *mm, long cpu)
 	return next;
 }
 
+extern void __load_new_mm_context(struct mm_struct *);
+
 __EXTERN_INLINE void
 ev5_switch_mm(struct mm_struct *prev_mm, struct mm_struct *next_mm,
 	      struct task_struct *next)
@@ -139,6 +141,16 @@ ev5_switch_mm(struct mm_struct *prev_mm, struct mm_struct *next_mm,
 	unsigned long mmc;
 	long cpu = smp_processor_id();
 
+	/*
+	 * kthread_use_mm(), sched_force_init_mm() and do_shoot_lazy_tlb()
+	 * switch current's mm without alpha_switch_to(), which loads the
+	 * context. The latter two require options Alpha does not select.
+	 */
+	if (next == current) {
+		__load_new_mm_context(next_mm);
+		return;
+	}
+
 #ifdef CONFIG_SMP
 	cpu_data[cpu].asn_lock = 1;
 	barrier();
@@ -160,7 +172,6 @@ ev5_switch_mm(struct mm_struct *prev_mm, struct mm_struct *next_mm,
 	task_thread_info(next)->pcb.asn = mmc & HARDWARE_ASN_MASK;
 }
 
-extern void __load_new_mm_context(struct mm_struct *);
 asmlinkage void do_page_fault(unsigned long address, unsigned long mmcsr,
 			      long cause, struct pt_regs *regs);
 
Signature · 1 line
2.43.0
maintainer
#2 lore raw

Re: [PATCH v5 1/8] alpha: load the MMU context when switch_mm() switches the current task

Quoted 3 lines · Magnus Lindholm

do_shoot_lazy_tlb(), which switches to init_mm from an IPI. None explicitly loads the context, so the task can carry on running
under whatever was loaded before while current->mm says otherwise.

Two small things if you touch this changelog again when applying, neither worth a v6.

The second line above is short.

"while current->mm says otherwise" only describes kthread_use_mm(). The other two callers change active_mm and leave current->mm NULL.

Quoted 3 lines

Your Signed-off-by is last on patches 2-8 and first here.

author maintainer
#3 lore raw

Re: [PATCH v5 1/8] alpha: load the MMU context when switch_mm() switches the current task

Hi Matt,

Quoted 11 lines · Matt Turner
Quoted 3 lines · Magnus Lindholm

do_shoot_lazy_tlb(), which switches to init_mm from an IPI. None explicitly loads the context, so the task can carry on running
under whatever was loaded before while current->mm says otherwise.

Two small things if you touch this changelog again when applying, neither worth a v6.

The second line above is short.

Quoted 8 lines

"while current->mm says otherwise" only describes kthread_use_mm(). The other two callers change active_mm and leave current->mm NULL.

Quoted 3 lines

Your Signed-off-by is last on patches 2-8 and first here.

Thanks, I've reworded the changelog in for-next to distinguish kthread_use_mm(), which changes current->mm, from the other two callers, which change active_mm while leaving current->mm NULL. I've also fixed the line wrapping. There are no code changes for this patch.

The combined TLB v5 and MMU v4 stack has now also completed an extended SMP compaction soak on the ES40, with three CPUs online and CONFIG_COMPACTION=y.

Over an hour, all 23 cycles passed:
- 207 lost-store rounds
- 207 stale-read rounds
- 230 shared-mapping writeback rounds on ext4

Compaction was triggered repeatedly during each cycle, and pgmigrate_success increased by 193374. Continuous kernel logging showed no new warnings or errors, only the expected drop_caches notices from the writeback test. The writeback settings were restored and checked after every cycle.

Thanks,
Magnus

author maintainer
#4 lore raw

PATCH v5 2/8 alpha: run check_mmu_context() from finish_arch_post_lock_switch()

In reply to Magnus Lindholm

check_mmu_context() clears asn_lock and acts on need_new_asn, but it runs only as the tail of switch_to(), after alpha_switch_to() returns. A newly forked task never gets there: its first context switch resumes at ret_from_fork and reaches schedule_tail() without returning to the code following alpha_switch_to(). New kernel threads take the same path through schedule_tail().

asn_lock is left set on that CPU, so the new task can run with interrupts enabled while shootdown IPIs keep taking the deferred path. The handshake meant to complete those invalidations never runs.

Move check_mmu_context() to finish_arch_post_lock_switch(), which is called by finish_task_switch() for both existing and newly created tasks. Drop the now redundant call from switch_to().

The hook runs after finish_lock_switch() releases the rq lock and enables interrupts. An IPI arriving before asn_lock is cleared retires the slot through flush_tlb_other(); an IPI arriving afterwards can flush directly, since PAL_swpctx has already installed the incoming context.

Set need_new_asn whether ev5_switch_mm() reuses or allocates an ASN. An IPI in that interval can zero either slot. Without this, the allocated case can return to user space with a live ASN but a zero context slot, which a later context-based shootdown shortcut could mistake for an inactive CPU. check_mmu_context() reloads only when the slot is zero.

Call the hook unconditionally. Alpha selects ARCH_NO_PREEMPT, and neither kthread_use_mm() nor sched_force_init_mm() sleeps between the direct switch and this hook. RCU_STRICT_GRACE_PERIOD can enable PREEMPT_COUNT, but cannot make those callers migrate there. Testing preemptible() would only leave asn_lock set in that configuration.

This requires the preceding direct-switch fix, "alpha: load the MMU context when switch_mm() switches the current task". It installs the borrowed context before kthread_use_mm() reaches this hook, so clearing asn_lock cannot expose a targeted invalidate against the wrong context.

Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")
---
 arch/alpha/include/asm/mmu_context.h | 15 +++++++++++++--
 arch/alpha/include/asm/switch_to.h   |  1 -
 2 files changed, 13 insertions(+), 3 deletions(-)
arch/alpha/include/asm/mmu_context.h +13 -2
index d366f89fee58..f41d7c73fdd6 100644
@@ -162,8 +162,8 @@ ev5_switch_mm(struct mm_struct *prev_mm, struct mm_struct *next_mm,
 		next_mm->context[cpu] = mmc;
 	}
 #ifdef CONFIG_SMP
-	else
-		cpu_data[cpu].need_new_asn = 1;
+	/* A deferred shootdown can also invalidate a newly allocated ASN. */
+	cpu_data[cpu].need_new_asn = 1;
 #endif
 
 	/* Always update the PCB ASN.  Another thread may have allocated
@@ -192,6 +192,17 @@ do {								\
 #define check_mmu_context()  do { } while(0)
 #endif
 
+/*
+ * Also called from kthread_use_mm() with preemption enabled. Alpha has
+ * no kernel preemption, so the task is still on the CPU that ran
+ * switch_mm().
+ */
+#define finish_arch_post_lock_switch finish_arch_post_lock_switch
+static inline void finish_arch_post_lock_switch(void)
+{
+	check_mmu_context();
+}
+
 __EXTERN_INLINE void
 ev5_activate_mm(struct mm_struct *prev_mm, struct mm_struct *next_mm)
 {
arch/alpha/include/asm/switch_to.h +0 -1
index 762b7f975310..35c4b2c9d992 100644
@@ -9,7 +9,6 @@ extern struct task_struct *alpha_switch_to(unsigned long, struct task_struct *);
 #define switch_to(P,N,L)						 \
   do {									 \
     (L) = alpha_switch_to(virt_to_phys(&task_thread_info(N)->pcb), (P)); \
-    check_mmu_context();						 \
   } while (0)
 
 #endif /* __ALPHA_SWITCH_TO_H */
Signature · 1 line
2.43.0
maintainer
#5 lore raw

Re: [PATCH v5 2/8] alpha: run check_mmu_context() from finish_arch_post_lock_switch()

Quoted 4 lines · Magnus Lindholm

code following alpha_switch_to(). New kernel threads take the same path through schedule_tail().

asn_lock is left set on that CPU, so the new task can run with

I missed this in v4. New kernel threads do take that path, but context_switch() does not call switch_mm() for a task without an mm, so they never set asn_lock in the first place. Only a new user task leaves it set. As written, the two paragraphs read as if both do.

Changelog only, and not worth a v6. My Reviewed-by stands.

author maintainer
#6 lore raw

Re: [PATCH v5 2/8] alpha: run check_mmu_context() from finish_arch_post_lock_switch()

Hi Matt,

Quoted 5 lines · Matt Turner
Quoted 3 lines · Magnus Lindholm

code following alpha_switch_to(). New kernel threads take the same path through schedule_tail().

Quoted 8 lines
Quoted 1 line

asn_lock is left set on that CPU, so the new task can run with

I missed this in v4. New kernel threads do take that path, but context_switch() does not call switch_mm() for a task without an mm, so they never set asn_lock in the first place. Only a new user task leaves it set. As written, the two paragraphs read as if both do.

Changelog only, and not worth a v6. My Reviewed-by stands.

Thanks, fixed the changelog in for-next to make clear that only new user tasks leave asn_lock set. New kernel threads reach schedule_tail(), but skip switch_mm() and therefore do not set asn_lock.

No code changes; your Reviewed-by is retained.

Thanks,
Magnus

author maintainer
#7 lore raw

PATCH v5 3/8 alpha: only use a targeted tbi() when the target mm is really current

In reply to Magnus Lindholm

ipi_flush_tlb_page() gates a targeted tbi() on current->active_mm. tbi() acts on the address space context currently loaded on the CPU, so it is only guaranteed to reach an mm's translations when a thread of that mm is running there. current->active_mm is not sufficient: under lazy TLB an idle or kernel task keeps an mm as its active_mm while a different ASN is loaded, so the invalidate is issued against the wrong context and the stale entry can survive. Nothing retires the old ASN afterwards either, mm->context[cpu] still being valid, so the resuming thread can reuse it.

Test current->mm instead and otherwise fall back to flush_tlb_other(), which clears mm->context[cpu] and forces a fresh ASN at the next switch whatever is loaded now.

For an mm borrowed through kthread_use_mm(), patch 1 of this series installs the context before the task can take a shootdown IPI. The current->mm test therefore covers that case as well.

Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")
---
 arch/alpha/kernel/smp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/alpha/kernel/smp.c +2 -1
index ed06367ece57..1ad448105201 100644
@@ -669,7 +669,8 @@ ipi_flush_tlb_page(void *x)
 	struct flush_tlb_page_struct *data = x;
 	struct mm_struct * mm = data->mm;
 
-	if (mm == current->active_mm && !asn_locked())
+	/* A targeted tbi() needs a thread of MM to be current.  */
+	if (mm == current->mm && !asn_locked())
 		flush_tlb_current_page(mm, data->vma, data->addr);
 	else
 		flush_tlb_other(mm);
Signature · 1 line
2.43.0
author maintainer
#8 lore raw

PATCH v5 4/8 alpha: invalidate the local context in flush_tlb_page()

In reply to Magnus Lindholm

When the target mm is not the calling CPU's active_mm, flush_tlb_page() does nothing locally, and smp_call_function() reaches only the other CPUs. This CPU may still hold translations for the mm and can later reuse the old ASN together with them.

Add the missing else branch. flush_tlb_other() clears mm->context[cpu] so a fresh ASN is taken at the next switch. Add this before changing the current->active_mm test to current->mm: that change will also send lazy callers here, so their local invalidate must already be in place.

The uniprocessor implementations of flush_tlb_mm() and flush_icache_user_page() in asm/tlbflush.h and asm/cacheflush.h already contain exactly this branch.

Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")
---
 arch/alpha/kernel/smp.c | 3 +++
 1 file changed, 3 insertions(+)
arch/alpha/kernel/smp.c +3 -0
index 1ad448105201..e4ba0aab8a66 100644
@@ -697,6 +697,9 @@ flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
 			preempt_enable();
 			return;
 		}
+	} else {
+		/* smp_call_function() does not call back into this CPU.  */
+		flush_tlb_other(mm);
 	}
 
 	data.vma = vma;
Signature · 1 line
2.43.0
author maintainer
#9 lore raw

PATCH v5 5/8 alpha: fix the local TLB invalidate in flush_tlb_page()

In reply to Magnus Lindholm

flush_tlb_page() invalidates the calling CPU itself before asking the others, and gates that on current->active_mm. For a non-executable vma that means a targeted tbi(2, addr), which acts on the context currently loaded and is only guaranteed to invalidate the intended translations when the target mm's context is the loaded one. A lazy active_mm does not establish that, so as in ipi_flush_tlb_page() the target mm's stale translations can survive, and nothing forces the old ASN to be retired afterwards.

Test current->mm instead. A lazy caller then takes the flush_tlb_other() branch added by the previous patch, retiring its local context. Adding that branch first preserves the local invalidate at every step, including on EV7 where a targeted tbi() against a foreign ASN appeared to work.

Which callers reach here with a foreign mm depends on the configuration. folio_mkclean(), from the writeback flusher kworker that has no mm of its own, accounted for about half the calls during writeback of a shared mapping and none at all on anonymous memory. Those counts were measured with CONFIG_COMPACTION=n: with COMPACTION=y, asm/pgtable.h overrides ptep_clear_flush() to call migrate_flush_tlb_page(), which rendezvouses with every CPU and handles the context itself, so folio_mkclean() does not reach this function.

Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")
---
 arch/alpha/kernel/smp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/alpha/kernel/smp.c +2 -1
index e4ba0aab8a66..a5a42ae4a7d8 100644
@@ -684,7 +684,8 @@ flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
 
 	preempt_disable();
 
-	if (mm == current->active_mm) {
+	/* As in ipi_flush_tlb_page(): a targeted tbi() needs MM current.  */
+	if (mm == current->mm) {
 		flush_tlb_current_page(mm, vma, addr);
 		if (atomic_read(&mm->mm_users) <= 1) {
 			int cpu, this_cpu = smp_processor_id();
Signature · 1 line
2.43.0
author maintainer
#10 lore raw

PATCH v5 6/8 alpha: fix the local TLB invalidate in the UP flush_tlb_page()

In reply to Magnus Lindholm

The uniprocessor flush_tlb_page() in asm/tlbflush.h has the same wrong test as the SMP one: a targeted tbi() is only guaranteed to invalidate the intended translations when the target mm's context is loaded, but the gate is current->active_mm, which a lazily borrowed mm also satisfies.

Test current->mm instead. The else branch it falls to already exists here, so unlike the SMP side this is the whole fix. arch/alpha/kernel/smp.c is not built with CONFIG_SMP=n, so this is how the same defect reaches a uniprocessor, where the flusher kworker necessarily shares the only CPU with the writer.

Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")
---
 arch/alpha/include/asm/tlbflush.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/alpha/include/asm/tlbflush.h +2 -1
index 0c8529997f54..9ae1902faf8e 100644
@@ -87,7 +87,8 @@ flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
 {
 	struct mm_struct *mm = vma->vm_mm;
 
-	if (mm == current->active_mm)
+	/* A targeted tbi() needs a thread of MM to be current.  */
+	if (mm == current->mm)
 		flush_tlb_current_page(mm, vma, addr);
 	else
 		flush_tlb_other(mm);
Signature · 1 line
2.43.0
author maintainer
#11 lore raw

PATCH v5 7/8 alpha: invalidate the local context in flush_tlb_mm()

In reply to Magnus Lindholm

flush_tlb_mm() has the same caller-CPU omission that was fixed in flush_tlb_page(): when the target mm is not the calling CPU's active_mm nothing happens locally, and smp_call_function() handles only the other CPUs, so this CPU may later reuse the old ASN together with the translations it still holds.

The active_mm test itself is left alone here. That path calls flush_tlb_current(), which loads a fresh context through __load_new_mm_context() rather than issuing a targeted tbi() against whatever ASN happens to be loaded, so it does not depend on which context is current.

The uniprocessor implementation in asm/tlbflush.h already has the missing branch. Counted over a fork-heavy run, flush_tlb_mm() was entered with the mm not this CPU's active_mm 2934 times, and 632 times while otherwise idle.

The proposed Alpha hugetlb support routes hugetlb invalidations through flush_tlb_range(), which calls this function. Unmapping or migrating a hugetlb folio from another context will also need this local invalidate.

Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")
---
 arch/alpha/kernel/smp.c | 3 +++
 1 file changed, 3 insertions(+)
arch/alpha/kernel/smp.c +3 -0
index a5a42ae4a7d8..988e397b0b8a 100644
@@ -649,6 +649,9 @@ flush_tlb_mm(struct mm_struct *mm)
 			preempt_enable();
 			return;
 		}
+	} else {
+		/* smp_call_function() does not call back into this CPU.  */
+		flush_tlb_other(mm);
 	}
 
 	smp_call_function(ipi_flush_tlb_mm, mm, 1);
Signature · 1 line
2.43.0
author maintainer
#12 lore raw

PATCH v5 8/8 alpha: invalidate the local context in flush_icache_user_page()

In reply to Magnus Lindholm

flush_icache_user_page() has the same caller-CPU omission that the previous patch fixed in flush_tlb_mm(): when the target mm is not the calling CPU's active_mm nothing happens locally, and smp_call_function() handles only the other CPUs, so this CPU may later reuse the old ASN together with the translations it still holds.

This matters here in particular because the function exists for operating on another process's mappings: the comment above it describes setting breakpoints through ptrace, and access_remote_vm() reaches it through copy_to_user_page(). The calling CPU is therefore often running something other than the target mm.

As in flush_tlb_mm(), the uniprocessor implementation in asm/cacheflush.h already has the missing case.

No imb() is needed, here or in ipi_flush_icache_page(). Alpha's user-space I-cache flush works by allocating a new ASN rather than by invalidating the I-cache: the entries stay, but they are tagged with the old ASN and can no longer match. An imb() is only required when the ASN space wraps and numbers are reused, and __get_new_mm_context() already does one in that case.

Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")
---
 arch/alpha/kernel/smp.c | 3 +++
 1 file changed, 3 insertions(+)
arch/alpha/kernel/smp.c +3 -0
index 988e397b0b8a..e21bc3920bec 100644
@@ -758,6 +758,9 @@ flush_icache_user_page(struct vm_area_struct *vma, struct page *page,
 			preempt_enable();
 			return;
 		}
+	} else {
+		/* smp_call_function() does not call back into this CPU.  */
+		flush_tlb_other(mm);
 	}
 
 	smp_call_function(ipi_flush_icache_page, mm, 1);
Signature · 1 line
2.43.0
maintainer
#13 lore raw
In reply to Magnus Lindholm
Quoted 1 line · Magnus Lindholm

Changes since v4:

One thing in your tree rather than in the posting: the eight commits in for-next (7500493539b4..29e34d2b1b87) each carry your Signed-off-by twice, once before the Link: trailer and once after it. Worth fixing before the pull request.