Skip to contentSkip to conversation

Add DMA attributes tracking

PATCH Applied Leon Romanovsky 4 participants 16 messages May 1 – May 8

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover Add DMA attributes tracking

The DMA API expects that callers use the same attributes when mapping and unmapping. Add tracking to verify this and catch mismatches.

Thanks

---
Changes in v2:
- Split original patch to be a series
- Added all unmap flows
- Skipped DMA attributes which exist only in map path
- Added ntb patch
- Link to v1: https://patch.msgid.link/[email protected]

---
Leon Romanovsky (6):
ntb: Store original DMA address for future release
ntb: Use consistent DMA attributes when freeing DMA mappings
dma-debug: Remove unused DMA attribute parameter
dma-debug: Record DMA attributes in debug entry
dma-debug: Feed DMA attribute for unmapping flows too
dma-debug: Ensure mappings are created and released with matching attributes

 drivers/ntb/ntb_transport.c | 10 +++---
 kernel/dma/debug.c          | 87 +++++++++++++++++++++++++++++++--------------
 kernel/dma/debug.h          | 25 ++++++-------
 kernel/dma/mapping.c        | 10 +++---
 4 files changed, 85 insertions(+), 47 deletions(-)

---
base-commit: 254f49634ee16a731174d2ae34bc50bd5f45e731
change-id: 20260323-dma-attrs-debug-85e282d6f3bb

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>
#1 lore raw

PATCH v2 1/6 ntb: Store original DMA address for future release

From: Leon Romanovsky <[email protected]>

The DMA API requires that dma_free_attrs receive the exact dma_handle originally returned by the allocation function. Do not modify it.

Fixes fc5d1829f9bf ("NTB: transport: Try harder to alloc an aligned MW buffer")
---
 drivers/ntb/ntb_transport.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/ntb/ntb_transport.c +3 -1
index 7cabc82305d6b..771eb7d9f4356 100644
@@ -223,6 +223,7 @@ struct ntb_transport_mw {
 	void *alloc_addr;
 	void *virt_addr;
 	dma_addr_t dma_addr;
+	dma_addr_t original_dma_addr;
 };
 
 struct ntb_transport_client_dev {
@@ -766,7 +767,7 @@ static void ntb_free_mw(struct ntb_transport_ctx *nt, int num_mw)
 
 	ntb_mw_clear_trans(nt->ndev, PIDX, num_mw);
 	dma_free_coherent(dma_dev, mw->alloc_size,
-			  mw->alloc_addr, mw->dma_addr);
+			  mw->alloc_addr, mw->original_dma_addr);
 	mw->xlat_size = 0;
 	mw->buff_size = 0;
 	mw->alloc_size = 0;
@@ -802,6 +803,7 @@ static int ntb_alloc_mw_buffer(struct ntb_transport_mw *mw,
 		return -ENOMEM;
 	}
 	virt_addr = alloc_addr;
+	mw->original_dma_addr = dma_addr;
 
 	/*
 	 * we must ensure that the memory address allocated is BAR size
Signature · 1 line
2.53.0
maintainer Reviewed-by on v2 1/6
#2 lore raw

Re: [PATCH v2 1/6] ntb: Store original DMA address for future release

Quoted 7 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

The DMA API requires that dma_free_attrs receive the exact dma_handle originally returned by the allocation function. Do not modify it.

Fixes fc5d1829f9bf ("NTB: transport: Try harder to alloc an aligned MW buffer")
Quoted 34 lines
---
 drivers/ntb/ntb_transport.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/ntb/ntb_transport.c +3 -1
index 7cabc82305d6b..771eb7d9f4356 100644
@@ -223,6 +223,7 @@ struct ntb_transport_mw {
 	void *alloc_addr;
 	void *virt_addr;
 	dma_addr_t dma_addr;
+	dma_addr_t original_dma_addr;
 };
 
 struct ntb_transport_client_dev {
@@ -766,7 +767,7 @@ static void ntb_free_mw(struct ntb_transport_ctx *nt, int num_mw)
 
 	ntb_mw_clear_trans(nt->ndev, PIDX, num_mw);
 	dma_free_coherent(dma_dev, mw->alloc_size,
-			  mw->alloc_addr, mw->dma_addr);
+			  mw->alloc_addr, mw->original_dma_addr);
 	mw->xlat_size = 0;
 	mw->buff_size = 0;
 	mw->alloc_size = 0;
@@ -802,6 +803,7 @@ static int ntb_alloc_mw_buffer(struct ntb_transport_mw *mw,
 		return -ENOMEM;
 	}
 	virt_addr = alloc_addr;
+	mw->original_dma_addr = dma_addr;
 
 	/*
 	 * we must ensure that the memory address allocated is BAR size
#3 lore raw

PATCH v2 2/6 ntb: Use consistent DMA attributes when freeing DMA mappings

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

The counterpart of dma_alloc_attrs() is dma_free_attrs(), which must receive the same DMA attributes used during allocation. The code previously used dma_free_coherent(), which does not accept or apply any DMA attributes.

Fixes 061a785a114f ("ntb: Force physically contiguous allocation of rx ring buffers")
---
 drivers/ntb/ntb_transport.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/ntb/ntb_transport.c +4 -4
index 771eb7d9f4356..f59f926d4bfaa 100644
@@ -766,8 +766,8 @@ static void ntb_free_mw(struct ntb_transport_ctx *nt, int num_mw)
 		return;
 
 	ntb_mw_clear_trans(nt->ndev, PIDX, num_mw);
-	dma_free_coherent(dma_dev, mw->alloc_size,
-			  mw->alloc_addr, mw->original_dma_addr);
+	dma_free_attrs(dma_dev, mw->alloc_size, mw->alloc_addr,
+		       mw->original_dma_addr, DMA_ATTR_FORCE_CONTIGUOUS);
 	mw->xlat_size = 0;
 	mw->buff_size = 0;
 	mw->alloc_size = 0;
@@ -828,8 +828,8 @@ static int ntb_alloc_mw_buffer(struct ntb_transport_mw *mw,
 	return 0;
 
 err:
-	dma_free_coherent(ntb_dev, mw->alloc_size, alloc_addr, dma_addr);
-
+	dma_free_attrs(ntb_dev, mw->alloc_size, alloc_addr, dma_addr,
+		       DMA_ATTR_FORCE_CONTIGUOUS);
 	return rc;
 }
 
Signature · 1 line
2.53.0
maintainer Reviewed-by on v2 2/6
#4 lore raw

Re: [PATCH v2 2/6] ntb: Use consistent DMA attributes when freeing DMA mappings

Quoted 9 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

The counterpart of dma_alloc_attrs() is dma_free_attrs(), which must receive the same DMA attributes used during allocation. The code previously used dma_free_coherent(), which does not accept or apply any DMA attributes.

Fixes 061a785a114f ("ntb: Force physically contiguous allocation of rx ring buffers")
Quoted 31 lines
---
 drivers/ntb/ntb_transport.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/ntb/ntb_transport.c +4 -4
index 771eb7d9f4356..f59f926d4bfaa 100644
@@ -766,8 +766,8 @@ static void ntb_free_mw(struct ntb_transport_ctx *nt, int num_mw)
 		return;
 
 	ntb_mw_clear_trans(nt->ndev, PIDX, num_mw);
-	dma_free_coherent(dma_dev, mw->alloc_size,
-			  mw->alloc_addr, mw->original_dma_addr);
+	dma_free_attrs(dma_dev, mw->alloc_size, mw->alloc_addr,
+		       mw->original_dma_addr, DMA_ATTR_FORCE_CONTIGUOUS);
 	mw->xlat_size = 0;
 	mw->buff_size = 0;
 	mw->alloc_size = 0;
@@ -828,8 +828,8 @@ static int ntb_alloc_mw_buffer(struct ntb_transport_mw *mw,
 	return 0;
 
 err:
-	dma_free_coherent(ntb_dev, mw->alloc_size, alloc_addr, dma_addr);
-
+	dma_free_attrs(ntb_dev, mw->alloc_size, alloc_addr, dma_addr,
+		       DMA_ATTR_FORCE_CONTIGUOUS);
 	return rc;
 }
 
#5 lore raw

PATCH v2 3/6 dma-debug: Remove unused DMA attribute parameter

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

debug_dma_alloc_pages() always receives a DMA attribute value of 0, because dma_alloc_pages() never receives any attributes from its callers. As preparation for upcoming patches, remove this unused attribute from the debug routine.

---
 kernel/dma/debug.c   | 5 ++---
 kernel/dma/debug.h   | 6 ++----
 kernel/dma/mapping.c | 2 +-
 3 files changed, 5 insertions(+), 8 deletions(-)
kernel/dma/debug.c +2 -3
index 1a725edbbbf6a..3b53495337f5c 100644
@@ -1567,8 +1567,7 @@ void debug_dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
 
 void debug_dma_alloc_pages(struct device *dev, struct page *page,
 			   size_t size, int direction,
-			   dma_addr_t dma_addr,
-			   unsigned long attrs)
+			   dma_addr_t dma_addr)
 {
 	struct dma_debug_entry *entry;
 
@@ -1586,7 +1585,7 @@ void debug_dma_alloc_pages(struct device *dev, struct page *page,
 	entry->dev_addr  = dma_addr;
 	entry->direction = direction;
 
-	add_dma_entry(entry, attrs);
+	add_dma_entry(entry, 0);
 }
 
 void debug_dma_free_pages(struct device *dev, struct page *page,
kernel/dma/debug.h +2 -4
index da7be0bddcf67..24b8610850fbd 100644
@@ -47,8 +47,7 @@ extern void debug_dma_sync_sg_for_device(struct device *dev,
 					 int nelems, int direction);
 extern void debug_dma_alloc_pages(struct device *dev, struct page *page,
 				  size_t size, int direction,
-				  dma_addr_t dma_addr,
-				  unsigned long attrs);
+				  dma_addr_t dma_addr);
 extern void debug_dma_free_pages(struct device *dev, struct page *page,
 				 size_t size, int direction,
 				 dma_addr_t dma_addr);
@@ -113,8 +112,7 @@ static inline void debug_dma_sync_sg_for_device(struct device *dev,
 
 static inline void debug_dma_alloc_pages(struct device *dev, struct page *page,
 					 size_t size, int direction,
-					 dma_addr_t dma_addr,
-					 unsigned long attrs)
+					 dma_addr_t dma_addr)
 {
 }
 
kernel/dma/mapping.c +1 -1
index 23ed8eb9233e5..6cbefbd4158c8 100644
@@ -733,7 +733,7 @@ struct page *dma_alloc_pages(struct device *dev, size_t size,
 	if (page) {
 		trace_dma_alloc_pages(dev, page_to_virt(page), *dma_handle,
 				      size, dir, gfp, 0);
-		debug_dma_alloc_pages(dev, page, size, dir, *dma_handle, 0);
+		debug_dma_alloc_pages(dev, page, size, dir, *dma_handle);
 	} else {
 		trace_dma_alloc_pages(dev, NULL, 0, size, dir, gfp, 0);
 	}
Signature · 1 line
2.53.0
Reviewed-by on v2 3/6
#6 lore raw

Re: [PATCH v2 3/6] dma-debug: Remove unused DMA attribute parameter

Quoted 79 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

debug_dma_alloc_pages() always receives a DMA attribute value of 0, because dma_alloc_pages() never receives any attributes from its callers. As preparation for upcoming patches, remove this unused attribute from the debug routine.

---

kernel/dma/debug.c   | 5 ++---
kernel/dma/debug.h   | 6 ++----
kernel/dma/mapping.c | 2 +-
3 files changed, 5 insertions(+), 8 deletions(-)
kernel/dma/debug.c +2 -3
index 1a725edbbbf6a..3b53495337f5c 100644
@@ -1567,8 +1567,7 @@ void debug_dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
 
 void debug_dma_alloc_pages(struct device *dev, struct page *page,
 			   size_t size, int direction,
-			   dma_addr_t dma_addr,
-			   unsigned long attrs)
+			   dma_addr_t dma_addr)
 {
 	struct dma_debug_entry *entry;
 
@@ -1586,7 +1585,7 @@ void debug_dma_alloc_pages(struct device *dev, struct page *page,
 	entry->dev_addr  = dma_addr;
 	entry->direction = direction;
 
-	add_dma_entry(entry, attrs);
+	add_dma_entry(entry, 0);
 }
 
 void debug_dma_free_pages(struct device *dev, struct page *page,
kernel/dma/debug.h +2 -4
index da7be0bddcf67..24b8610850fbd 100644
@@ -47,8 +47,7 @@ extern void debug_dma_sync_sg_for_device(struct device *dev,
 					 int nelems, int direction);
 extern void debug_dma_alloc_pages(struct device *dev, struct page *page,
 				  size_t size, int direction,
-				  dma_addr_t dma_addr,
-				  unsigned long attrs);
+				  dma_addr_t dma_addr);
 extern void debug_dma_free_pages(struct device *dev, struct page *page,
 				 size_t size, int direction,
 				 dma_addr_t dma_addr);
@@ -113,8 +112,7 @@ static inline void debug_dma_sync_sg_for_device(struct device *dev,
 
 static inline void debug_dma_alloc_pages(struct device *dev, struct page *page,
 					 size_t size, int direction,
-					 dma_addr_t dma_addr,
-					 unsigned long attrs)
+					 dma_addr_t dma_addr)
 {
 }
 
kernel/dma/mapping.c +1 -1
index 23ed8eb9233e5..6cbefbd4158c8 100644
@@ -733,7 +733,7 @@ struct page *dma_alloc_pages(struct device *dev, size_t size,
 	if (page) {
 		trace_dma_alloc_pages(dev, page_to_virt(page), *dma_handle,
 				      size, dir, gfp, 0);
-		debug_dma_alloc_pages(dev, page, size, dir, *dma_handle, 0);
+		debug_dma_alloc_pages(dev, page, size, dir, *dma_handle);
 	} else {
 		trace_dma_alloc_pages(dev, NULL, 0, size, dir, gfp, 0);
 	}
Signature · 2 lines
-- 
2.53.0
#7 lore raw

PATCH v2 4/6 dma-debug: Record DMA attributes in debug entry

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

To enable reliable comparison of DMA attributes between map and unmap operations, store the attribute value in dma_debug_entry.

---
 kernel/dma/debug.c | 48 +++++++++++++++++++++++++++++-------------------
 1 file changed, 29 insertions(+), 19 deletions(-)
kernel/dma/debug.c +29 -19
index 3b53495337f5c..f07e6a1e9fbab 100644
@@ -63,7 +63,7 @@ enum map_err_types {
  * @sg_mapped_ents: 'mapped_ents' from dma_map_sg
  * @paddr: physical start address of the mapping
  * @map_err_type: track whether dma_mapping_error() was checked
- * @is_cache_clean: driver promises not to write to buffer while mapped
+ * @attrs: dma attributes
  * @stack_len: number of backtrace entries in @stack_entries
  * @stack_entries: stack of backtrace history
  */
@@ -78,7 +78,7 @@ struct dma_debug_entry {
 	int		 sg_mapped_ents;
 	phys_addr_t	 paddr;
 	enum map_err_types map_err_type;
-	bool		 is_cache_clean;
+	unsigned long	 attrs;
 #ifdef CONFIG_STACKTRACE
 	unsigned int	stack_len;
 	unsigned long	stack_entries[DMA_DEBUG_STACKTRACE_ENTRIES];
@@ -478,6 +478,9 @@ static int active_cacheline_insert(struct dma_debug_entry *entry,
 				   bool *overlap_cache_clean)
 {
 	phys_addr_t cln = to_cacheline_number(entry);
+	bool is_cache_clean = entry->attrs &
+			      (DMA_ATTR_DEBUGGING_IGNORE_CACHELINES |
+			       DMA_ATTR_REQUIRE_COHERENT);
 	unsigned long flags;
 	int rc;
 
@@ -495,12 +498,15 @@ static int active_cacheline_insert(struct dma_debug_entry *entry,
 	if (rc == -EEXIST) {
 		struct dma_debug_entry *existing;
 
-		active_cacheline_inc_overlap(cln, entry->is_cache_clean);
+		active_cacheline_inc_overlap(cln, is_cache_clean);
 		existing = radix_tree_lookup(&dma_active_cacheline, cln);
 		/* A lookup failure here after we got -EEXIST is unexpected. */
 		WARN_ON(!existing);
 		if (existing)
-			*overlap_cache_clean = existing->is_cache_clean;
+			*overlap_cache_clean =
+				existing->attrs &
+				(DMA_ATTR_DEBUGGING_IGNORE_CACHELINES |
+				 DMA_ATTR_REQUIRE_COHERENT);
 	}
 	spin_unlock_irqrestore(&radix_lock, flags);
 
@@ -544,12 +550,13 @@ void debug_dma_dump_mappings(struct device *dev)
 			if (!dev || dev == entry->dev) {
 				cln = to_cacheline_number(entry);
 				dev_info(entry->dev,
-					 "%s idx %d P=%pa D=%llx L=%llx cln=%pa %s %s\n",
+					 "%s idx %d P=%pa D=%llx L=%llx cln=%pa %s %s attrs=0x%lx\n",
 					 type2name[entry->type], idx,
 					 &entry->paddr, entry->dev_addr,
 					 entry->size, &cln,
 					 dir2name[entry->direction],
-					 maperr2str[entry->map_err_type]);
+					 maperr2str[entry->map_err_type],
+					 entry->attrs);
 			}
 		}
 		spin_unlock_irqrestore(&bucket->lock, flags);
@@ -575,14 +582,15 @@ static int dump_show(struct seq_file *seq, void *v)
 		list_for_each_entry(entry, &bucket->list, list) {
 			cln = to_cacheline_number(entry);
 			seq_printf(seq,
-				   "%s %s %s idx %d P=%pa D=%llx L=%llx cln=%pa %s %s\n",
+				   "%s %s %s idx %d P=%pa D=%llx L=%llx cln=%pa %s %s attrs=0x%lx\n",
 				   dev_driver_string(entry->dev),
 				   dev_name(entry->dev),
 				   type2name[entry->type], idx,
 				   &entry->paddr, entry->dev_addr,
 				   entry->size, &cln,
 				   dir2name[entry->direction],
-				   maperr2str[entry->map_err_type]);
+				   maperr2str[entry->map_err_type],
+				   entry->attrs);
 		}
 		spin_unlock_irqrestore(&bucket->lock, flags);
 	}
@@ -594,16 +602,14 @@ DEFINE_SHOW_ATTRIBUTE(dump);
  * Wrapper function for adding an entry to the hash.
  * This function takes care of locking itself.
  */
-static void add_dma_entry(struct dma_debug_entry *entry, unsigned long attrs)
+static void add_dma_entry(struct dma_debug_entry *entry)
 {
+	unsigned long attrs = entry->attrs;
 	bool overlap_cache_clean;
 	struct hash_bucket *bucket;
 	unsigned long flags;
 	int rc;
 
-	entry->is_cache_clean = attrs & (DMA_ATTR_DEBUGGING_IGNORE_CACHELINES |
-					 DMA_ATTR_REQUIRE_COHERENT);
-
 	bucket = get_hash_bucket(entry, &flags);
 	hash_bucket_add(bucket, entry);
 	put_hash_bucket(bucket, flags);
@@ -612,9 +618,10 @@ static void add_dma_entry(struct dma_debug_entry *entry, unsigned long attrs)
 	if (rc == -ENOMEM) {
 		pr_err_once("cacheline tracking ENOMEM, dma-debug disabled\n");
 		global_disable = true;
-	} else if (rc == -EEXIST &&
-		   !(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
-		   !(entry->is_cache_clean && overlap_cache_clean) &&
+	} else if (rc == -EEXIST && !(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
+		   !(attrs & (DMA_ATTR_DEBUGGING_IGNORE_CACHELINES |
+			      DMA_ATTR_REQUIRE_COHERENT) &&
+		     overlap_cache_clean) &&
 		   dma_get_cache_alignment() >= L1_CACHE_BYTES &&
 		   !(IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC) &&
 		     is_swiotlb_active(entry->dev))) {
@@ -1250,6 +1257,7 @@ void debug_dma_map_phys(struct device *dev, phys_addr_t phys, size_t size,
 	entry->size      = size;
 	entry->direction = direction;
 	entry->map_err_type = MAP_ERR_NOT_CHECKED;
+	entry->attrs     = attrs;
 
 	if (!(attrs & DMA_ATTR_MMIO)) {
 		check_for_stack(dev, phys);
@@ -1258,7 +1266,7 @@ void debug_dma_map_phys(struct device *dev, phys_addr_t phys, size_t size,
 			check_for_illegal_area(dev, phys_to_virt(phys), size);
 	}
 
-	add_dma_entry(entry, attrs);
+	add_dma_entry(entry);
 }
 
 void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
@@ -1345,10 +1353,11 @@ void debug_dma_map_sg(struct device *dev, struct scatterlist *sg,
 		entry->direction      = direction;
 		entry->sg_call_ents   = nents;
 		entry->sg_mapped_ents = mapped_ents;
+		entry->attrs          = attrs;
 
 		check_sg_segment(dev, s);
 
-		add_dma_entry(entry, attrs);
+		add_dma_entry(entry);
 	}
 }
 
@@ -1440,8 +1449,9 @@ void debug_dma_alloc_coherent(struct device *dev, size_t size,
 	entry->size      = size;
 	entry->dev_addr  = dma_addr;
 	entry->direction = DMA_BIDIRECTIONAL;
+	entry->attrs     = attrs;
 
-	add_dma_entry(entry, attrs);
+	add_dma_entry(entry);
 }
 
 void debug_dma_free_coherent(struct device *dev, size_t size,
@@ -1585,7 +1595,7 @@ void debug_dma_alloc_pages(struct device *dev, struct page *page,
 	entry->dev_addr  = dma_addr;
 	entry->direction = direction;
 
-	add_dma_entry(entry, 0);
+	add_dma_entry(entry);
 }
 
 void debug_dma_free_pages(struct device *dev, struct page *page,
Signature · 1 line
2.53.0
Reviewed-by on v2 4/6
#8 lore raw

Re: [PATCH v2 4/6] dma-debug: Record DMA attributes in debug entry

kernel/dma/debug.c void debug_dma_alloc_coherent(struct device *dev, size_t size,

Quoted 158 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

To enable reliable comparison of DMA attributes between map and unmap operations, store the attribute value in dma_debug_entry.

---
kernel/dma/debug.c | 48 +++++++++++++++++++++++++++++-------------------

1 file changed, 29 insertions(+), 19 deletions(-)
kernel/dma/debug.c +26 -17
index 3b53495337f5c..f07e6a1e9fbab 100644
@@ -63,7 +63,7 @@ enum map_err_types {
 * @sg_mapped_ents: 'mapped_ents' from dma_map_sg
 * @paddr: physical start address of the mapping
 * @map_err_type: track whether dma_mapping_error() was checked
- * @is_cache_clean: driver promises not to write to buffer while mapped
+ * @attrs: dma attributes
 * @stack_len: number of backtrace entries in @stack_entries
 * @stack_entries: stack of backtrace history
 */
@@ -78,7 +78,7 @@ struct dma_debug_entry {
 	int		 sg_mapped_ents;
 	phys_addr_t	 paddr;
 	enum map_err_types map_err_type;
-	bool		 is_cache_clean;
+	unsigned long	 attrs;
 #ifdef CONFIG_STACKTRACE
 	unsigned int	stack_len;
 	unsigned long	stack_entries[DMA_DEBUG_STACKTRACE_ENTRIES];
@@ -478,6 +478,9 @@ static int active_cacheline_insert(struct dma_debug_entry *entry,
 				   bool *overlap_cache_clean)
 {
 	phys_addr_t cln = to_cacheline_number(entry);
+	bool is_cache_clean = entry->attrs &
+			      (DMA_ATTR_DEBUGGING_IGNORE_CACHELINES |
+			       DMA_ATTR_REQUIRE_COHERENT);
 	unsigned long flags;
 	int rc;
 
@@ -495,12 +498,15 @@ static int active_cacheline_insert(struct dma_debug_entry *entry,
 	if (rc == -EEXIST) {
 		struct dma_debug_entry *existing;
 
-		active_cacheline_inc_overlap(cln, entry->is_cache_clean);
+		active_cacheline_inc_overlap(cln, is_cache_clean);
 		existing = radix_tree_lookup(&dma_active_cacheline, cln);
 		/* A lookup failure here after we got -EEXIST is unexpected. */
 		WARN_ON(!existing);
 		if (existing)
-			*overlap_cache_clean = existing->is_cache_clean;
+			*overlap_cache_clean =
+				existing->attrs &
+				(DMA_ATTR_DEBUGGING_IGNORE_CACHELINES |
+				 DMA_ATTR_REQUIRE_COHERENT);
 	}
 	spin_unlock_irqrestore(&radix_lock, flags);
 
@@ -544,12 +550,13 @@ void debug_dma_dump_mappings(struct device *dev)
 			if (!dev || dev == entry->dev) {
 				cln = to_cacheline_number(entry);
 				dev_info(entry->dev,
-					 "%s idx %d P=%pa D=%llx L=%llx cln=%pa %s %s\n",
+					 "%s idx %d P=%pa D=%llx L=%llx cln=%pa %s %s attrs=0x%lx\n",
 					 type2name[entry->type], idx,
 					 &entry->paddr, entry->dev_addr,
 					 entry->size, &cln,
 					 dir2name[entry->direction],
-					 maperr2str[entry->map_err_type]);
+					 maperr2str[entry->map_err_type],
+					 entry->attrs);
 			}
 		}
 		spin_unlock_irqrestore(&bucket->lock, flags);
@@ -575,14 +582,15 @@ static int dump_show(struct seq_file *seq, void *v)
 		list_for_each_entry(entry, &bucket->list, list) {
 			cln = to_cacheline_number(entry);
 			seq_printf(seq,
-				   "%s %s %s idx %d P=%pa D=%llx L=%llx cln=%pa %s %s\n",
+				   "%s %s %s idx %d P=%pa D=%llx L=%llx cln=%pa %s %s attrs=0x%lx\n",
 				   dev_driver_string(entry->dev),
 				   dev_name(entry->dev),
 				   type2name[entry->type], idx,
 				   &entry->paddr, entry->dev_addr,
 				   entry->size, &cln,
 				   dir2name[entry->direction],
-				   maperr2str[entry->map_err_type]);
+				   maperr2str[entry->map_err_type],
+				   entry->attrs);
 		}
 		spin_unlock_irqrestore(&bucket->lock, flags);
 	}
@@ -594,16 +602,14 @@ DEFINE_SHOW_ATTRIBUTE(dump);
 * Wrapper function for adding an entry to the hash.
 * This function takes care of locking itself.
 */
-static void add_dma_entry(struct dma_debug_entry *entry, unsigned long attrs)
+static void add_dma_entry(struct dma_debug_entry *entry)
 {
+	unsigned long attrs = entry->attrs;
 	bool overlap_cache_clean;
 	struct hash_bucket *bucket;
 	unsigned long flags;
 	int rc;
 
-	entry->is_cache_clean = attrs & (DMA_ATTR_DEBUGGING_IGNORE_CACHELINES |
-					 DMA_ATTR_REQUIRE_COHERENT);
-
 	bucket = get_hash_bucket(entry, &flags);
 	hash_bucket_add(bucket, entry);
 	put_hash_bucket(bucket, flags);
@@ -612,9 +618,10 @@ static void add_dma_entry(struct dma_debug_entry *entry, unsigned long attrs)
 	if (rc == -ENOMEM) {
 		pr_err_once("cacheline tracking ENOMEM, dma-debug disabled\n");
 		global_disable = true;
-	} else if (rc == -EEXIST &&
-		   !(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
-		   !(entry->is_cache_clean && overlap_cache_clean) &&
+	} else if (rc == -EEXIST && !(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
+		   !(attrs & (DMA_ATTR_DEBUGGING_IGNORE_CACHELINES |
+			      DMA_ATTR_REQUIRE_COHERENT) &&
+		     overlap_cache_clean) &&
 		   dma_get_cache_alignment() >= L1_CACHE_BYTES &&
 		   !(IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC) &&
 		     is_swiotlb_active(entry->dev))) {
@@ -1250,6 +1257,7 @@ void debug_dma_map_phys(struct device *dev, phys_addr_t phys, size_t size,
 	entry->size      = size;
 	entry->direction = direction;
 	entry->map_err_type = MAP_ERR_NOT_CHECKED;
+	entry->attrs     = attrs;
 
 	if (!(attrs & DMA_ATTR_MMIO)) {
 		check_for_stack(dev, phys);
@@ -1258,7 +1266,7 @@ void debug_dma_map_phys(struct device *dev, phys_addr_t phys, size_t size,
 			check_for_illegal_area(dev, phys_to_virt(phys), size);
 	}
 
-	add_dma_entry(entry, attrs);
+	add_dma_entry(entry);
 }
 
 void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
@@ -1345,10 +1353,11 @@ void debug_dma_map_sg(struct device *dev, struct scatterlist *sg,
 		entry->direction      = direction;
 		entry->sg_call_ents   = nents;
 		entry->sg_mapped_ents = mapped_ents;
+		entry->attrs          = attrs;
 
 		check_sg_segment(dev, s);
 
-		add_dma_entry(entry, attrs);
+		add_dma_entry(entry);
 	}
 }
 
@@ -1440,8 +1449,9 @@ void debug_dma_alloc_coherent(struct device *dev, size_t size,

Unrelated to this patch/series, but I am wondering whether we should rename this function to debug_dma_alloc_attrs() as it is called from dma_alloc_attrs().

Quoted 24 lines
kernel/dma/debug.c +3 -2
 	entry->size      = size;
 	entry->dev_addr  = dma_addr;
 	entry->direction = DMA_BIDIRECTIONAL;
+	entry->attrs     = attrs;
 
-	add_dma_entry(entry, attrs);
+	add_dma_entry(entry);
 }
 
 void debug_dma_free_coherent(struct device *dev, size_t size,
@@ -1585,7 +1595,7 @@ void debug_dma_alloc_pages(struct device *dev, struct page *page,
 	entry->dev_addr  = dma_addr;
 	entry->direction = direction;
 
-	add_dma_entry(entry, 0);
+	add_dma_entry(entry);
 }
 
 void debug_dma_free_pages(struct device *dev, struct page *page,
Signature · 2 lines
-- 
2.53.0
#9 lore raw

PATCH v2 5/6 dma-debug: Feed DMA attribute for unmapping flows too

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

There are multiple unmapping flows which didn't provide DMA attributes, which limited DMA debug code to compare the mapping and unmapping attributes. Let's fix it.

---
 kernel/dma/debug.c   | 13 ++++++++-----
 kernel/dma/debug.h   | 19 +++++++++++--------
 kernel/dma/mapping.c |  8 ++++----
 3 files changed, 23 insertions(+), 17 deletions(-)
kernel/dma/debug.c +8 -5
index f07e6a1e9fbab..3dfed51c3d9aa 100644
@@ -1307,8 +1307,8 @@ void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
 }
 EXPORT_SYMBOL(debug_dma_mapping_error);
 
-void debug_dma_unmap_phys(struct device *dev, dma_addr_t dma_addr,
-			  size_t size, int direction)
+void debug_dma_unmap_phys(struct device *dev, dma_addr_t dma_addr, size_t size,
+			  int direction, unsigned long attrs)
 {
 	struct dma_debug_entry ref = {
 		.type           = dma_debug_phy,
@@ -1316,6 +1316,7 @@ void debug_dma_unmap_phys(struct device *dev, dma_addr_t dma_addr,
 		.dev_addr       = dma_addr,
 		.size           = size,
 		.direction      = direction,
+		.attrs          = attrs,
 	};
 
 	if (unlikely(dma_debug_disabled()))
@@ -1381,7 +1382,7 @@ static int get_nr_mapped_entries(struct device *dev,
 }
 
 void debug_dma_unmap_sg(struct device *dev, struct scatterlist *sglist,
-			int nelems, int dir)
+			int nelems, int dir, unsigned long attrs)
 {
 	struct scatterlist *s;
 	int mapped_ents = 0, i;
@@ -1399,6 +1400,7 @@ void debug_dma_unmap_sg(struct device *dev, struct scatterlist *sglist,
 			.size           = sg_dma_len(s),
 			.direction      = dir,
 			.sg_call_ents   = nelems,
+			.attrs          = attrs,
 		};
 
 		if (mapped_ents && i >= mapped_ents)
@@ -1454,8 +1456,8 @@ void debug_dma_alloc_coherent(struct device *dev, size_t size,
 	add_dma_entry(entry);
 }
 
-void debug_dma_free_coherent(struct device *dev, size_t size,
-			 void *virt, dma_addr_t dma_addr)
+void debug_dma_free_coherent(struct device *dev, size_t size, void *virt,
+			     dma_addr_t dma_addr, unsigned long attrs)
 {
 	struct dma_debug_entry ref = {
 		.type           = dma_debug_coherent,
@@ -1463,6 +1465,7 @@ void debug_dma_free_coherent(struct device *dev, size_t size,
 		.dev_addr       = dma_addr,
 		.size           = size,
 		.direction      = DMA_BIDIRECTIONAL,
+		.attrs          = attrs,
 	};
 
 	/* handle vmalloc and linear addresses */
kernel/dma/debug.h +11 -8
index 24b8610850fbd..13e384633c32a 100644
@@ -14,21 +14,22 @@ extern void debug_dma_map_phys(struct device *dev, phys_addr_t phys,
 			       unsigned long attrs);
 
 extern void debug_dma_unmap_phys(struct device *dev, dma_addr_t addr,
-				 size_t size, int direction);
+				 size_t size, int direction,
+				 unsigned long attrs);
 
 extern void debug_dma_map_sg(struct device *dev, struct scatterlist *sg,
 			     int nents, int mapped_ents, int direction,
 			     unsigned long attrs);
 
 extern void debug_dma_unmap_sg(struct device *dev, struct scatterlist *sglist,
-			       int nelems, int dir);
+			       int nelems, int dir, unsigned long attrs);
 
 extern void debug_dma_alloc_coherent(struct device *dev, size_t size,
 				     dma_addr_t dma_addr, void *virt,
 				     unsigned long attrs);
 
-extern void debug_dma_free_coherent(struct device *dev, size_t size,
-				    void *virt, dma_addr_t addr);
+extern void debug_dma_free_coherent(struct device *dev, size_t size, void *virt,
+				    dma_addr_t addr, unsigned long attrs);
 
 extern void debug_dma_sync_single_for_cpu(struct device *dev,
 					  dma_addr_t dma_handle, size_t size,
@@ -59,7 +60,8 @@ static inline void debug_dma_map_phys(struct device *dev, phys_addr_t phys,
 }
 
 static inline void debug_dma_unmap_phys(struct device *dev, dma_addr_t addr,
-					size_t size, int direction)
+					size_t size, int direction,
+					unsigned long attrs)
 {
 }
 
@@ -70,8 +72,8 @@ static inline void debug_dma_map_sg(struct device *dev, struct scatterlist *sg,
 }
 
 static inline void debug_dma_unmap_sg(struct device *dev,
-				      struct scatterlist *sglist,
-				      int nelems, int dir)
+				      struct scatterlist *sglist, int nelems,
+				      int dir, unsigned long attrs)
 {
 }
 
@@ -82,7 +84,8 @@ static inline void debug_dma_alloc_coherent(struct device *dev, size_t size,
 }
 
 static inline void debug_dma_free_coherent(struct device *dev, size_t size,
-					   void *virt, dma_addr_t addr)
+					   void *virt, dma_addr_t addr,
+					   unsigned long attrs)
 {
 }
 
kernel/dma/mapping.c +4 -4
index 6cbefbd4158c8..f010b3cc0ece4 100644
@@ -225,7 +225,7 @@ void dma_unmap_phys(struct device *dev, dma_addr_t addr, size_t size,
 	else if (ops->unmap_phys)
 		ops->unmap_phys(dev, addr, size, dir, attrs);
 	trace_dma_unmap_phys(dev, addr, size, dir, attrs);
-	debug_dma_unmap_phys(dev, addr, size, dir);
+	debug_dma_unmap_phys(dev, addr, size, dir, attrs);
 }
 EXPORT_SYMBOL_GPL(dma_unmap_phys);
 
@@ -351,7 +351,7 @@ void dma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg,
 
 	BUG_ON(!valid_dma_direction(dir));
 	trace_dma_unmap_sg(dev, sg, nents, dir, attrs);
-	debug_dma_unmap_sg(dev, sg, nents, dir);
+	debug_dma_unmap_sg(dev, sg, nents, dir, attrs);
 	if (dma_map_direct(dev, ops) ||
 	    arch_dma_unmap_sg_direct(dev, sg, nents))
 		dma_direct_unmap_sg(dev, sg, nents, dir, attrs);
@@ -693,7 +693,7 @@ void dma_free_attrs(struct device *dev, size_t size, void *cpu_addr,
 	if (!cpu_addr)
 		return;
 
-	debug_dma_free_coherent(dev, size, cpu_addr, dma_handle);
+	debug_dma_free_coherent(dev, size, cpu_addr, dma_handle, attrs);
 	if (dma_alloc_direct(dev, ops) || arch_dma_free_direct(dev, dma_handle))
 		dma_direct_free(dev, size, cpu_addr, dma_handle, attrs);
 	else if (use_dma_iommu(dev))
@@ -840,7 +840,7 @@ void dma_free_noncontiguous(struct device *dev, size_t size,
 		struct sg_table *sgt, enum dma_data_direction dir)
 {
 	trace_dma_free_sgt(dev, sgt, size, dir);
-	debug_dma_unmap_sg(dev, sgt->sgl, sgt->orig_nents, dir);
+	debug_dma_unmap_sg(dev, sgt->sgl, sgt->orig_nents, dir, 0);
 
 	if (use_dma_iommu(dev))
 		iommu_dma_free_noncontiguous(dev, size, sgt, dir);
Signature · 1 line
2.53.0
Reviewed-by on v2 5/6
#10 lore raw

Re: [PATCH v2 5/6] dma-debug: Feed DMA attribute for unmapping flows too

Quoted 179 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

There are multiple unmapping flows which didn't provide DMA attributes, which limited DMA debug code to compare the mapping and unmapping attributes. Let's fix it.

---

kernel/dma/debug.c   | 13 ++++++++-----
kernel/dma/debug.h   | 19 +++++++++++--------
kernel/dma/mapping.c |  8 ++++----
3 files changed, 23 insertions(+), 17 deletions(-)
kernel/dma/debug.c +8 -5
index f07e6a1e9fbab..3dfed51c3d9aa 100644
@@ -1307,8 +1307,8 @@ void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
 }
 EXPORT_SYMBOL(debug_dma_mapping_error);
 
-void debug_dma_unmap_phys(struct device *dev, dma_addr_t dma_addr,
-			  size_t size, int direction)
+void debug_dma_unmap_phys(struct device *dev, dma_addr_t dma_addr, size_t size,
+			  int direction, unsigned long attrs)
 {
 	struct dma_debug_entry ref = {
 		.type           = dma_debug_phy,
@@ -1316,6 +1316,7 @@ void debug_dma_unmap_phys(struct device *dev, dma_addr_t dma_addr,
 		.dev_addr       = dma_addr,
 		.size           = size,
 		.direction      = direction,
+		.attrs          = attrs,
 	};
 
 	if (unlikely(dma_debug_disabled()))
@@ -1381,7 +1382,7 @@ static int get_nr_mapped_entries(struct device *dev,
 }
 
 void debug_dma_unmap_sg(struct device *dev, struct scatterlist *sglist,
-			int nelems, int dir)
+			int nelems, int dir, unsigned long attrs)
 {
 	struct scatterlist *s;
 	int mapped_ents = 0, i;
@@ -1399,6 +1400,7 @@ void debug_dma_unmap_sg(struct device *dev, struct scatterlist *sglist,
 			.size           = sg_dma_len(s),
 			.direction      = dir,
 			.sg_call_ents   = nelems,
+			.attrs          = attrs,
 		};
 
 		if (mapped_ents && i >= mapped_ents)
@@ -1454,8 +1456,8 @@ void debug_dma_alloc_coherent(struct device *dev, size_t size,
 	add_dma_entry(entry);
 }
 
-void debug_dma_free_coherent(struct device *dev, size_t size,
-			 void *virt, dma_addr_t dma_addr)
+void debug_dma_free_coherent(struct device *dev, size_t size, void *virt,
+			     dma_addr_t dma_addr, unsigned long attrs)
 {
 	struct dma_debug_entry ref = {
 		.type           = dma_debug_coherent,
@@ -1463,6 +1465,7 @@ void debug_dma_free_coherent(struct device *dev, size_t size,
 		.dev_addr       = dma_addr,
 		.size           = size,
 		.direction      = DMA_BIDIRECTIONAL,
+		.attrs          = attrs,
 	};
 
 	/* handle vmalloc and linear addresses */
kernel/dma/debug.h +11 -8
index 24b8610850fbd..13e384633c32a 100644
@@ -14,21 +14,22 @@ extern void debug_dma_map_phys(struct device *dev, phys_addr_t phys,
 			       unsigned long attrs);
 
 extern void debug_dma_unmap_phys(struct device *dev, dma_addr_t addr,
-				 size_t size, int direction);
+				 size_t size, int direction,
+				 unsigned long attrs);
 
 extern void debug_dma_map_sg(struct device *dev, struct scatterlist *sg,
 			     int nents, int mapped_ents, int direction,
 			     unsigned long attrs);
 
 extern void debug_dma_unmap_sg(struct device *dev, struct scatterlist *sglist,
-			       int nelems, int dir);
+			       int nelems, int dir, unsigned long attrs);
 
 extern void debug_dma_alloc_coherent(struct device *dev, size_t size,
 				     dma_addr_t dma_addr, void *virt,
 				     unsigned long attrs);
 
-extern void debug_dma_free_coherent(struct device *dev, size_t size,
-				    void *virt, dma_addr_t addr);
+extern void debug_dma_free_coherent(struct device *dev, size_t size, void *virt,
+				    dma_addr_t addr, unsigned long attrs);
 
 extern void debug_dma_sync_single_for_cpu(struct device *dev,
 					  dma_addr_t dma_handle, size_t size,
@@ -59,7 +60,8 @@ static inline void debug_dma_map_phys(struct device *dev, phys_addr_t phys,
 }
 
 static inline void debug_dma_unmap_phys(struct device *dev, dma_addr_t addr,
-					size_t size, int direction)
+					size_t size, int direction,
+					unsigned long attrs)
 {
 }
 
@@ -70,8 +72,8 @@ static inline void debug_dma_map_sg(struct device *dev, struct scatterlist *sg,
 }
 
 static inline void debug_dma_unmap_sg(struct device *dev,
-				      struct scatterlist *sglist,
-				      int nelems, int dir)
+				      struct scatterlist *sglist, int nelems,
+				      int dir, unsigned long attrs)
 {
 }
 
@@ -82,7 +84,8 @@ static inline void debug_dma_alloc_coherent(struct device *dev, size_t size,
 }
 
 static inline void debug_dma_free_coherent(struct device *dev, size_t size,
-					   void *virt, dma_addr_t addr)
+					   void *virt, dma_addr_t addr,
+					   unsigned long attrs)
 {
 }
 
kernel/dma/mapping.c +4 -4
index 6cbefbd4158c8..f010b3cc0ece4 100644
@@ -225,7 +225,7 @@ void dma_unmap_phys(struct device *dev, dma_addr_t addr, size_t size,
 	else if (ops->unmap_phys)
 		ops->unmap_phys(dev, addr, size, dir, attrs);
 	trace_dma_unmap_phys(dev, addr, size, dir, attrs);
-	debug_dma_unmap_phys(dev, addr, size, dir);
+	debug_dma_unmap_phys(dev, addr, size, dir, attrs);
 }
 EXPORT_SYMBOL_GPL(dma_unmap_phys);
 
@@ -351,7 +351,7 @@ void dma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg,
 
 	BUG_ON(!valid_dma_direction(dir));
 	trace_dma_unmap_sg(dev, sg, nents, dir, attrs);
-	debug_dma_unmap_sg(dev, sg, nents, dir);
+	debug_dma_unmap_sg(dev, sg, nents, dir, attrs);
 	if (dma_map_direct(dev, ops) ||
 	    arch_dma_unmap_sg_direct(dev, sg, nents))
 		dma_direct_unmap_sg(dev, sg, nents, dir, attrs);
@@ -693,7 +693,7 @@ void dma_free_attrs(struct device *dev, size_t size, void *cpu_addr,
 	if (!cpu_addr)
 		return;
 
-	debug_dma_free_coherent(dev, size, cpu_addr, dma_handle);
+	debug_dma_free_coherent(dev, size, cpu_addr, dma_handle, attrs);
 	if (dma_alloc_direct(dev, ops) || arch_dma_free_direct(dev, dma_handle))
 		dma_direct_free(dev, size, cpu_addr, dma_handle, attrs);
 	else if (use_dma_iommu(dev))
@@ -840,7 +840,7 @@ void dma_free_noncontiguous(struct device *dev, size_t size,
 		struct sg_table *sgt, enum dma_data_direction dir)
 {
 	trace_dma_free_sgt(dev, sgt, size, dir);
-	debug_dma_unmap_sg(dev, sgt->sgl, sgt->orig_nents, dir);
+	debug_dma_unmap_sg(dev, sgt->sgl, sgt->orig_nents, dir, 0);
 
 	if (use_dma_iommu(dev))
 		iommu_dma_free_noncontiguous(dev, size, sgt, dir);
Signature · 2 lines
-- 
2.53.0
#11 lore raw

PATCH v2 6/6 dma-debug: Ensure mappings are created and released with matching attributes

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

The DMA API expects that callers use the same attributes when mapping and unmapping. Add tracking to verify this and catch mismatches.

---
 kernel/dma/debug.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
kernel/dma/debug.c +23 -0
index 3dfed51c3d9aa..c38efc1ac8d6c 100644
@@ -1074,6 +1074,29 @@ static void check_unmap(struct dma_debug_entry *ref)
 			   type2name[entry->type]);
 	}
 
+	/*
+	 * This may be no bug in reality - but DMA API still expects
+	 * that entry is unmapped with same attributes as it was mapped.
+	 *
+	 * DMA_ATTR_UNMAP_VALID lists the attributes that must be identical
+	 * between map and unmap. Any attribute outside this set (e.g.
+	 * DMA_ATTR_NO_WARN, DMA_ATTR_SKIP_CPU_SYNC) is allowed to differ.
+	 */
+#define DMA_ATTR_UNMAP_VALID                                               \
+	(DMA_ATTR_NO_KERNEL_MAPPING | DMA_ATTR_FORCE_CONTIGUOUS |          \
+	 DMA_ATTR_MMIO | DMA_ATTR_REQUIRE_COHERENT | DMA_ATTR_PRIVILEGED | \
+	 DMA_ATTR_CC_SHARED)
+	if ((ref->attrs & DMA_ATTR_UNMAP_VALID) !=
+	    (entry->attrs & DMA_ATTR_UNMAP_VALID)) {
+		err_printk(ref->dev, entry,
+			   "device driver frees "
+			   "DMA memory with different attributes "
+			   "[device address=0x%016llx] [size=%llu bytes] "
+			   "[mapped with 0x%lx] [unmapped with 0x%lx]\n",
+			   ref->dev_addr, ref->size, entry->attrs, ref->attrs);
+	}
+#undef DMA_ATTR_UNMAP_VALID
+
 	hash_bucket_del(entry);
 	put_hash_bucket(bucket, flags);
 
Signature · 1 line
2.53.0
Reviewed-by on v2 6/6
#12 lore raw

Re: [PATCH v2 6/6] dma-debug: Ensure mappings are created and released with matching attributes

Quoted 49 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

The DMA API expects that callers use the same attributes when mapping and unmapping. Add tracking to verify this and catch mismatches.

---
kernel/dma/debug.c | 23 +++++++++++++++++++++++

1 file changed, 23 insertions(+)
kernel/dma/debug.c +23 -0
index 3dfed51c3d9aa..c38efc1ac8d6c 100644
@@ -1074,6 +1074,29 @@ static void check_unmap(struct dma_debug_entry *ref)
 			   type2name[entry->type]);
 	}
 
+	/*
+	 * This may be no bug in reality - but DMA API still expects
+	 * that entry is unmapped with same attributes as it was mapped.
+	 *
+	 * DMA_ATTR_UNMAP_VALID lists the attributes that must be identical
+	 * between map and unmap. Any attribute outside this set (e.g.
+	 * DMA_ATTR_NO_WARN, DMA_ATTR_SKIP_CPU_SYNC) is allowed to differ.
+	 */
+#define DMA_ATTR_UNMAP_VALID                                               \
+	(DMA_ATTR_NO_KERNEL_MAPPING | DMA_ATTR_FORCE_CONTIGUOUS |          \
+	 DMA_ATTR_MMIO | DMA_ATTR_REQUIRE_COHERENT | DMA_ATTR_PRIVILEGED | \
+	 DMA_ATTR_CC_SHARED)
+	if ((ref->attrs & DMA_ATTR_UNMAP_VALID) !=
+	    (entry->attrs & DMA_ATTR_UNMAP_VALID)) {
+		err_printk(ref->dev, entry,
+			   "device driver frees "
+			   "DMA memory with different attributes "
+			   "[device address=0x%016llx] [size=%llu bytes] "
+			   "[mapped with 0x%lx] [unmapped with 0x%lx]\n",
+			   ref->dev_addr, ref->size, entry->attrs, ref->attrs);
+	}
+#undef DMA_ATTR_UNMAP_VALID
+
 	hash_bucket_del(entry);
 	put_hash_bucket(bucket, flags);
 
Signature · 2 lines
-- 
2.53.0
maintainer
#13 lore raw
In reply to Leon Romanovsky
Quoted 2 lines · Leon Romanovsky

The DMA API expects that callers use the same attributes when mapping and unmapping. Add tracking to verify this and catch mismatches.

I would like to merge this to dma-mapping-for-next, but I have a question about ntb patches - Jon, Dave, Allen: do You plan to take them (either to the fixes or to the -next) or do You want me to take them together with dma patches?

Quoted 24 lines

Thanks

---
Changes in v2:
- Split original patch to be a series
- Added all unmap flows
- Skipped DMA attributes which exist only in map path
- Added ntb patch
- Link to v1: https://protect2.fireeye.com/v1/url?k=70e722e7-116c37d7-70e6a9a8-000babffaa23-c39f7f9efe32c6a2&q=1&e=c79241ff-9ff4-40ae-8e29-753dd9d59f23&u=https%3A%2F%2Fpatch.msgid.link%2F20260323-dma-attrs-debug-v1-1-6275228ca300%40nvidia.com

---
Leon Romanovsky (6):
ntb: Store original DMA address for future release
ntb: Use consistent DMA attributes when freeing DMA mappings
dma-debug: Remove unused DMA attribute parameter
dma-debug: Record DMA attributes in debug entry
dma-debug: Feed DMA attribute for unmapping flows too
dma-debug: Ensure mappings are created and released with matching attributes

 drivers/ntb/ntb_transport.c | 10 +++---
 kernel/dma/debug.c          | 87 +++++++++++++++++++++++++++++++--------------
 kernel/dma/debug.h          | 25 ++++++-------
 kernel/dma/mapping.c        | 10 +++---
 4 files changed, 85 insertions(+), 47 deletions(-)
Quoted 9 lines

---
base-commit: 254f49634ee16a731174d2ae34bc50bd5f45e731
change-id: 20260323-dma-attrs-debug-85e282d6f3bb

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>

Best regards

Signature · 2 lines
Marek Szyprowski, PhD
Samsung R&D Institute Poland
maintainer
#14 lore raw
Quoted 7 lines · Marek Szyprowski
Quoted 2 lines · Leon Romanovsky

The DMA API expects that callers use the same attributes when mapping and unmapping. Add tracking to verify this and catch mismatches.

I would like to merge this to dma-mapping-for-next, but I have a question about ntb patches - Jon, Dave, Allen: do You plan to take them (either to the fixes or to the -next) or do You want me to take them together with dma patches?

Please go ahead and take them together. Thanks!

Quoted 36 lines
Quoted 26 lines

Thanks

---
Changes in v2:
- Split original patch to be a series
- Added all unmap flows
- Skipped DMA attributes which exist only in map path
- Added ntb patch
- Link to v1: https://protect2.fireeye.com/v1/url?k=70e722e7-116c37d7-70e6a9a8-000babffaa23-c39f7f9efe32c6a2&q=1&e=c79241ff-9ff4-40ae-8e29-753dd9d59f23&u=https%3A%2F%2Fpatch.msgid.link%2F20260323-dma-attrs-debug-v1-1-6275228ca300%40nvidia.com

---
Leon Romanovsky (6):
ntb: Store original DMA address for future release
ntb: Use consistent DMA attributes when freeing DMA mappings
dma-debug: Remove unused DMA attribute parameter
dma-debug: Record DMA attributes in debug entry
dma-debug: Feed DMA attribute for unmapping flows too
dma-debug: Ensure mappings are created and released with matching attributes

 drivers/ntb/ntb_transport.c | 10 +++---
 kernel/dma/debug.c          | 87 +++++++++++++++++++++++++++++++--------------
 kernel/dma/debug.h          | 25 ++++++-------
 kernel/dma/mapping.c        | 10 +++---
 4 files changed, 85 insertions(+), 47 deletions(-)
Quoted 9 lines

---
base-commit: 254f49634ee16a731174d2ae34bc50bd5f45e731
change-id: 20260323-dma-attrs-debug-85e282d6f3bb

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>

Best regards

maintainer Applied
#15 lore raw
Quoted 8 lines · Dave Jiang
Quoted 6 lines · Marek Szyprowski
Quoted 2 lines · Leon Romanovsky

The DMA API expects that callers use the same attributes when mapping and unmapping. Add tracking to verify this and catch mismatches.

I would like to merge this to dma-mapping-for-next, but I have a question about ntb patches - Jon, Dave, Allen: do You plan to take them (either to the fixes or to the -next) or do You want me to take them together with dma patches?

Please go ahead and take them together. Thanks!

Applied all patches to dma-mapping-for-next. Thanks!

Best regards

Signature · 2 lines
Marek Szyprowski, PhD
Samsung R&D Institute Poland