Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2055E374E62 for ; Mon, 6 Jul 2026 15:07:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.74 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783350455; cv=none; b=FvUflSVtb6o23RiudEpNyIDdSBk3p0EkM/WlzHGMbyLP/KCw+msWQdo4bloUr24Dhcb8p3EajRj3jsgZjFis0SVbXgZnXD9zNsSMkNjECnb1VsZSIQID+z00lPzw88rjhGUO/qyQdt5QQ2wvk/nq8B6tn74MgkEhZpjZjFBPoyY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783350455; c=relaxed/simple; bh=lKQonR4ehLSrdg7Q9EQmcaFN4H/cXaBULLkvW6+uEfg=; h=Date:Mime-Version:Message-ID:Subject:From:To:Cc:Content-Type; b=p+LVGc2uR5EG+7IS9ScEvZAg3HeMkgzIY1Uz2yiZ4Hg9oVpTOeD/dtETT7CvaUGGHIv4Mf6Ge+rzp5gB5GlD3EwWGGqgf8iYtLYTBXS0FpNaqfgyNyUBOtyx3d9odNrJDPyxQehJQ1ZptEZMNj9tR5jhPVGW0mstetCK8UjqiEw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--mclapinski.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=gNXg9xj7; arc=none smtp.client-ip=209.85.218.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--mclapinski.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gNXg9xj7" Received: by mail-ej1-f74.google.com with SMTP id a640c23a62f3a-c12690ff1fbso249548166b.2 for ; Mon, 06 Jul 2026 08:07:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1783350452; x=1783955252; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=1lx/Tz8Zmq/2rMr5850VaaKZJq/iHDsvqGoxRZtlKNQ=; b=gNXg9xj7+SlDJdi5kvhakccOvVz0nu/EODuYo+RVvp3zS/j8yeJfz/NRuilbhe9K8y MIk56PRlvPgc9Au5TdV/jfgpXzhWp3CYaRH6LDlzsGo7kX8YH8lMU8DQbAOSrf6uNsDl XSYLiVMQ3QiV9nM4Dd/hne8rTnBbFtJpRXMzba+qLeDGVzShtO4k5wT1FiRjvh0/O42F 4qIS//xgz8U0TFdQez4gHbL2qR4ivear7pTudmGyhPdkZRujIRrvnHOqx109O4f8PdRv y/MuR2clytRkZZt40lHVdaO+XcfBr6qvrT7WhXmsu/HCeKcD1wm0oXQAM1MSTmMZDMm8 jO6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783350452; x=1783955252; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=1lx/Tz8Zmq/2rMr5850VaaKZJq/iHDsvqGoxRZtlKNQ=; b=HOds50ZzrzbwG+REBF0eX7Z/F36Gi9l+s43Ab4UVh7OKA5FS4CJ7NmYty/mJHaP0i0 2Jn8D1D834fB/WZjd5z6xKd0z2pY9i7ZAshLwJig+AZrSWrp1kUqDJOd+dRU2Jjr/X84 3W1yCH8yliVPheT3n+7uNeQsOEV+/tj2DzapQwPSJ7bIl3+YKPqJSz2ZH+gbG8dACGpA 3cY/1kR2P5ndjfXoJupH7q76Aiwx6yJkzSzJWvjflOUYXMLgSHlcUQ6s+T+8GBSF+2HK KcpSgpZgoU3wygoKXumwy1Pd4r/M7gfEfPVf9ISz8ezL823YL5j4WMUHOF5v3jmU9N/N enKQ== X-Gm-Message-State: AOJu0Yzk5HRwh6y05TWHFJdCgfcoohuj1TC9KMuFjc5UaiMdHDZi1lvp 0jSxIYb16N0Xdh8ayWm9cbIv2B+c5AkTm2uw+T9ddJ5DZzNS5j/tYcrUN/nXCp9zcgfM8IDUxxu F2R+tjINp5rGzDqxV6j/x3A== X-Received: from edqu19.prod.google.com ([2002:aa7:d893:0:b0:698:878f:fb60]) (user=mclapinski job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:bac5:b0:c12:15b5:876a with SMTP id a640c23a62f3a-c15a67fa6f0mr51152166b.17.1783350452355; Mon, 06 Jul 2026 08:07:32 -0700 (PDT) Date: Mon, 6 Jul 2026 17:07:27 +0200 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.55.0.795.g602f6c329a-goog Message-ID: <20260706150727.4098614-1-mclapinski@google.com> Subject: [PATCH v2] iommu/iova: Move CPU magazine init to first insert From: Michal Clapinski To: Robin Murphy , "Joerg Roedel (AMD)" , Will Deacon , iommu@lists.linux.dev Cc: linux-kernel@vger.kernel.org, Logan Odell , Michal Clapinski Content-Type: text/plain; charset="UTF-8" From: Logan Odell A large amount of memory may be allocated for these magazines on machines with a lot of IOMMU groups and CPU cores. Not all may be used as some devices may be unused or be bound to drivers that do not use the DMA-API. Furthermore, some drivers may not use all levels or CPUs. Move the initialization of the loaded and prev magazines for each CPU on the first attempt to try to insert a freed IOVA to them. Signed-off-by: Logan Odell Signed-off-by: Michal Clapinski --- v2: only rebase --- drivers/iommu/iova.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index 021daf6528de..1a4bbf45dbb9 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -621,6 +621,9 @@ iova_magazine_free_pfns(struct iova_magazine *mag, struct iova_domain *iovad) unsigned long flags; int i; + if (!mag) + return; + spin_lock_irqsave(&iovad->iova_rbtree_lock, flags); for (i = 0 ; i < mag->size; ++i) { @@ -737,14 +740,7 @@ int iova_domain_init_rcaches(struct iova_domain *iovad) } for_each_possible_cpu(cpu) { cpu_rcache = per_cpu_ptr(rcache->cpu_rcaches, cpu); - spin_lock_init(&cpu_rcache->lock); - cpu_rcache->loaded = iova_magazine_alloc(GFP_KERNEL); - cpu_rcache->prev = iova_magazine_alloc(GFP_KERNEL); - if (!cpu_rcache->loaded || !cpu_rcache->prev) { - ret = -ENOMEM; - goto out_err; - } } } @@ -777,8 +773,20 @@ static bool __iova_rcache_insert(struct iova_domain *iovad, cpu_rcache = raw_cpu_ptr(rcache->cpu_rcaches); spin_lock_irqsave(&cpu_rcache->lock, flags); + if (!cpu_rcache->loaded) { + cpu_rcache->loaded = iova_magazine_alloc(GFP_ATOMIC | __GFP_NOWARN); + if (!cpu_rcache->loaded) + goto unlock; + } + if (!iova_magazine_full(cpu_rcache->loaded)) { can_insert = true; + } else if (!cpu_rcache->prev) { + cpu_rcache->prev = iova_magazine_alloc(GFP_ATOMIC | __GFP_NOWARN); + if (!cpu_rcache->prev) + goto unlock; + swap(cpu_rcache->prev, cpu_rcache->loaded); + can_insert = true; } else if (!iova_magazine_full(cpu_rcache->prev)) { swap(cpu_rcache->prev, cpu_rcache->loaded); can_insert = true; @@ -799,6 +807,7 @@ static bool __iova_rcache_insert(struct iova_domain *iovad, if (can_insert) iova_magazine_push(cpu_rcache->loaded, iova_pfn); +unlock: spin_unlock_irqrestore(&cpu_rcache->lock, flags); return can_insert; @@ -831,9 +840,9 @@ static unsigned long __iova_rcache_get(struct iova_rcache *rcache, cpu_rcache = raw_cpu_ptr(rcache->cpu_rcaches); spin_lock_irqsave(&cpu_rcache->lock, flags); - if (!iova_magazine_empty(cpu_rcache->loaded)) { + if (cpu_rcache->loaded && !iova_magazine_empty(cpu_rcache->loaded)) { has_pfn = true; - } else if (!iova_magazine_empty(cpu_rcache->prev)) { + } else if (cpu_rcache->prev && !iova_magazine_empty(cpu_rcache->prev)) { swap(cpu_rcache->prev, cpu_rcache->loaded); has_pfn = true; } else { -- 2.55.0.795.g602f6c329a-goog