Please backport 9e249c484128 ("iommu/amd: serialize sequence allocation under concurrent TLB invalidations") to 6.12.y
Subsystems (4)
Hi
After updating to 6.12.85 in Debian we got a regression report, where frequently a user had "AMD-Vi: Completion-Wait loop timed out" logged. The report is at: https://bugs.debian.org/1135313
While investigating it looks d2a0cac10597 ("iommu/amd: move wait_on_sem() out of spinlock") got backported to 6.12.y but a followup for it not.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1135313#43 confirmed that a backport of 9e249c484128 solved the problem. Now that one does not apply cleanly. As an alternative the following two might work, the first is a clean cherry-pick, the second on top needed a slight adjustment since in 6.12.y f32fe7cb0198 is not present.
it would be though ideal if the maintainers can confirm this is the way to go.
Regards,
Salvatore
From c1eddc8c4ee7dc0ef651d87c51676fe4cd60a9df Mon Sep 17 00:00:00 2001
From: Uros Bizjak <[email protected]>
Date: Mon, 7 Oct 2024 10:43:31 +0200
Subject: [PATCH 1/2] iommu/amd: Use atomic64_inc_return() in iommu.c
commit 5ce73c524f5fb5abd7b1bfed0115474b4fb437b4 upstream.
Use atomic64_inc_return(&ref) instead of atomic64_add_return(1, &ref) to use optimized implementation and ease register pressure around the primitive for targets that implement optimized variant.
--- drivers/iommu/amd/iommu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/iommu/amd/iommu.c +2 -2
@@ -1266,7 +1266,7 @@ static int iommu_completion_wait(struct amd_iommu *iommu)if (!iommu->need_sync)return 0 ; - data =atomic64_add_return (1 , &iommu->cmd_sem_val); + data =atomic64_inc_return (&iommu->cmd_sem_val);build_completion_wait (&cmd, iommu, data);raw_spin_lock_irqsave (&iommu->lock, flags);
@@ -2929,7 +2929,7 @@ static void iommu_flush_irt_and_complete(struct amd_iommu *iommu, u16 devid)return ;build_inv_irt (&cmd, devid); - data =atomic64_add_return (1 , &iommu->cmd_sem_val); + data =atomic64_inc_return (&iommu->cmd_sem_val);build_completion_wait (&cmd2, iommu, data);raw_spin_lock_irqsave (&iommu->lock, flags);
Signature · 1 line
2.53.0
From c0e5ddae730cd75b2c806655a8ced16148bafa2c Mon Sep 17 00:00:00 2001
From: Ankit Soni <[email protected]>
Date: Thu, 22 Jan 2026 15:30:38 +0000
Subject: [PATCH 2/2] iommu/amd: serialize sequence allocation under concurrent
TLB invalidations
commit 9e249c48412828e807afddc21527eb734dc9bd3d upstream.
With concurrent TLB invalidations, completion wait randomly gets timed out because cmd_sem_val was incremented outside the IOMMU spinlock, allowing CMD_COMPL_WAIT commands to be queued out of sequence and breaking the ordering assumption in wait_on_sem().
Move the cmd_sem_val increment under iommu->lock so completion sequence allocation is serialized with command queuing.
And remove the unnecessary return.
[Salvatore Bonaccorso: Backport to v6.12.y where f32fe7cb0198 ("iommu/amd: Add support to remap/unmap IOMMU buffers for kdump") is not present]
--- drivers/iommu/amd/amd_iommu_types.h | 2 +- drivers/iommu/amd/init.c | 2 +- drivers/iommu/amd/iommu.c | 18 ++++++++++++------ 3 files changed, 14 insertions(+), 8 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +1 -1
@@ -781,7 +781,7 @@ struct amd_iommu {u32 flags;volatile u64 *cmd_sem; -atomic64_t cmd_sem_val; +u64 cmd_sem_val;#ifdef CONFIG_AMD_IOMMU_DEBUGFS /* DebugFS Info */
drivers/iommu/amd/init.c +1 -1
@@ -1742,7 +1742,7 @@ static int __init init_iommu_one(struct amd_iommu *iommu, struct ivhd_header *h, iommu->pci_seg = pci_seg;raw_spin_lock_init (&iommu->lock); -atomic64_set (&iommu->cmd_sem_val,0 ); + iommu->cmd_sem_val =0 ;/* Add IOMMU to internal data structures */ list_add_tail (&iommu->list, &amd_iommu_list);
drivers/iommu/amd/iommu.c +12 -6
@@ -1252,6 +1252,12 @@ static int iommu_queue_command(struct amd_iommu *iommu, struct iommu_cmd *cmd)return iommu_queue_command_sync (iommu, cmd, true); } +static u64 get_cmdsem_val (struct amd_iommu *iommu) +{ +lockdep_assert_held (&iommu->lock); +return ++iommu->cmd_sem_val; +} +/* * This function queues a completion wait command into the command * buffer of an IOMMU
@@ -1266,11 +1272,11 @@ static int iommu_completion_wait(struct amd_iommu *iommu)if (!iommu->need_sync)return 0 ; - data =atomic64_inc_return (&iommu->cmd_sem_val); -build_completion_wait (&cmd, iommu, data); -raw_spin_lock_irqsave (&iommu->lock, flags); + data =get_cmdsem_val (iommu); +build_completion_wait (&cmd, iommu, data); + ret =__iommu_queue_command_sync (iommu, &cmd, false);raw_spin_unlock_irqrestore (&iommu->lock, flags);
@@ -2929,10 +2935,11 @@ static void iommu_flush_irt_and_complete(struct amd_iommu *iommu, u16 devid)return ;build_inv_irt (&cmd, devid); - data =atomic64_inc_return (&iommu->cmd_sem_val); -build_completion_wait (&cmd2, iommu, data);raw_spin_lock_irqsave (&iommu->lock, flags); + data =get_cmdsem_val (iommu); +build_completion_wait (&cmd2, iommu, data); + ret =__iommu_queue_command_sync (iommu, &cmd, true);if (ret) goto out_err;
@@ -2946,7 +2953,6 @@ static void iommu_flush_irt_and_complete(struct amd_iommu *iommu, u16 devid) out_err:raw_spin_unlock_irqrestore (&iommu->lock, flags); -return ; } static void set_dte_irq_entry(struct amd_iommu *iommu, u16 devid,
Signature · 1 line
2.53.0
Sasha Levin
maintainerQuoted 5 lines
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1135313#43 confirmed that a backport of
9e249c484128solved the problem. Now that one does not apply cleanly. As an alternative the following two might work, the first is a clean cherry-pick, the second on top needed a slight adjustment since in 6.12.yf32fe7cb0198is not present.
Both patches queued for 6.12.y.
I extended the same backport to 6.6.y as well - it carries d2a0cac10597 and the same older atomic64_add_return(1,..) pattern, so it is exposed to the same race.
--
Sasha