Skip to contentSkip to conversation

dma: mark unmapped DMA scatter/gather invalid

PATCH In review Marc Orr 3 participants 6 messages Jan 11, 2021

Subsystems (2)
lore
Share & tools
lore

Marc Orr

author
#0 lore raw

PATCH 1/1 dma: mark unmapped DMA scatter/gather invalid

This patch updates dma_direct_unmap_sg() to mark each scatter/gather entry invalid, after it's unmapped. This fixes two issues:

1. It makes the unmapping code able to tolerate a double unmap.
2. It prevents the NVMe driver from erroneously treating an unmapped DMA address as mapped.

The bug that motivated this patch was the following sequence, which occurred within the NVMe driver, with the kernel flag swiotlb=force.

* NVMe driver calls dma_direct_map_sg()
* dma_direct_map_sg() fails part way through the scatter gather/list
* dma_direct_map_sg() calls dma_direct_unmap_sg() to unmap any entries succeeded.
* NVMe driver calls dma_direct_unmap_sg(), redundantly, leading to a double unmap, which is a bug.

With this patch, a hadoop workload running on a cluster of three AMD SEV VMs, is able to succeed. Without the patch, the hadoop workload suffers application-level and even VM-level failures.

---
 kernel/dma/direct.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/dma/direct.c +3 -1
index 0a4881e59aa7..3d9b17fe5771 100644
@@ -374,9 +374,11 @@ void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
 	struct scatterlist *sg;
 	int i;
 
-	for_each_sg(sgl, sg, nents, i)
+	for_each_sg(sgl, sg, nents, i) {
 		dma_direct_unmap_page(dev, sg->dma_address, sg_dma_len(sg), dir,
 			     attrs);
+		sg->dma_address = DMA_MAPPING_ERROR;
+	}
 }
 EXPORT_SYMBOL(dma_direct_unmap_sg);
 #endif
Signature · 1 line
2.30.0.284.gd98b1dd5eaa7-goog
reviewer
#1 lore raw
Quoted 2 lines · Marc Orr

This patch updates dma_direct_unmap_sg() to mark each scatter/gather entry invalid, after it's unmapped. This fixes two issues:

s/fixes/bodges around (badly)/

Quoted 7 lines

1. It makes the unmapping code able to tolerate a double unmap.
2. It prevents the NVMe driver from erroneously treating an unmapped DMA address as mapped.

The bug that motivated this patch was the following sequence, which occurred within the NVMe driver, with the kernel flag swiotlb=force.

Quoted 6 lines

* NVMe driver calls dma_direct_map_sg()
* dma_direct_map_sg() fails part way through the scatter gather/list
* dma_direct_map_sg() calls dma_direct_unmap_sg() to unmap any entries succeeded.
* NVMe driver calls dma_direct_unmap_sg(), redundantly, leading to a double unmap, which is a bug.

So why not just fix that actual bug?

Quoted 17 lines

With this patch, a hadoop workload running on a cluster of three AMD SEV VMs, is able to succeed. Without the patch, the hadoop workload suffers application-level and even VM-level failures.

---
  kernel/dma/direct.c | 4 +++-
  1 file changed, 3 insertions(+), 1 deletion(-)
kernel/dma/direct.c +0 -0
index 0a4881e59aa7..3d9b17fe5771 100644

kernel/dma/direct.c void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl, · line 380

Quoted 8 lines
kernel/dma/direct.c +2 -1
@@ -374,9 +374,11 @@ void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
  	struct scatterlist *sg;
  	int i;
 
-	for_each_sg(sgl, sg, nents, i)
+	for_each_sg(sgl, sg, nents, i) {
  		dma_direct_unmap_page(dev, sg->dma_address, sg_dma_len(sg), dir,
  			     attrs);
+		sg->dma_address = DMA_MAPPING_ERROR;

There are more DMA API backends than just dma-direct, so while this might help paper over bugs when SWIOTLB is in use, it's not going to have any effect when those same bugs are hit under other circumstances. Once again, the moral of the story is that effort is better spent just fixing the bugs ;)

Robin.

kernel/dma/direct.c void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl, · line 384

Quoted 5 lines
kernel/dma/direct.c +1 -0
+	}
  }
  EXPORT_SYMBOL(dma_direct_unmap_sg);
  #endif

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

maintainer
#5 lore raw
In reply to Marc Orr
Quoted 21 lines · Marc Orr

This patch updates dma_direct_unmap_sg() to mark each scatter/gather entry invalid, after it's unmapped. This fixes two issues:

1. It makes the unmapping code able to tolerate a double unmap.
2. It prevents the NVMe driver from erroneously treating an unmapped DMA address as mapped.

The bug that motivated this patch was the following sequence, which occurred within the NVMe driver, with the kernel flag swiotlb=force.

* NVMe driver calls dma_direct_map_sg()
* dma_direct_map_sg() fails part way through the scatter gather/list
* dma_direct_map_sg() calls dma_direct_unmap_sg() to unmap any entries succeeded.
* NVMe driver calls dma_direct_unmap_sg(), redundantly, leading to a double unmap, which is a bug.

With this patch, a hadoop workload running on a cluster of three AMD SEV VMs, is able to succeed. Without the patch, the hadoop workload suffers application-level and even VM-level failures.

Quoted 7 lines
---
 kernel/dma/direct.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

<formletter>

This is not the correct way to submit patches for inclusion in the stable kernel tree. Please read:
https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
for how to do this properly.

</formletter>
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author
#2 lore raw
In reply to Robin Murphy
Quoted 1 line · Marc Orr

minus [email protected], per gregkh@'s email.

Quoted 8 lines
Quoted 8 lines
kernel/dma/direct.c +0 -0
index 0a4881e59aa7..3d9b17fe5771 100644
@@ -374,9 +374,11 @@ void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
      struct scatterlist *sg;
      int i;
 
Quoted 11 lines

kernel/dma/direct.c void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl, · line 380

Quoted 5 lines
kernel/dma/direct.c +2 -1
-     for_each_sg(sgl, sg, nents, i)
+     for_each_sg(sgl, sg, nents, i) {
              dma_direct_unmap_page(dev, sg->dma_address, sg_dma_len(sg), dir,
                           attrs);
+             sg->dma_address = DMA_MAPPING_ERROR;

There are more DMA API backends than just dma-direct, so while this might help paper over bugs when SWIOTLB is in use, it's not going to have any effect when those same bugs are hit under other circumstances. Once again, the moral of the story is that effort is better spent just fixing the bugs ;)

Thanks for the quick feedback. What is the correct fix? I understand the first half. The NVMe driver should be updated to not call unmap on an address that has already been unmapped within the DMA direct code. Where I'm less certain is how to communicate to the NVMe driver that the mapping failed. In particular, the NVMe code explicitly checks if the first DMA address in the scatter/gather list is set to DMA_MAPPING_ERROR. Thus, don't we need to update the DMA direct code to propagate DMA_MAPPING_ERROR back up to the driver, via the scatter/gather struct?

I skimmed arch/arm/mm/dma-mapping.c, just now. I can see that this code sets the address within the scatter/gather struct to DMA_MAPPING_ERROR before trying to map an IO address and write it into the struct. Is this a good example to follow?

Thanks,
Marc

reviewer
#3 lore raw
Quoted 24 lines · Marc Orr
Quoted 1 line · Marc Orr

minus [email protected], per gregkh@'s email.

Quoted 8 lines
Quoted 8 lines
kernel/dma/direct.c +0 -0
index 0a4881e59aa7..3d9b17fe5771 100644
@@ -374,9 +374,11 @@ void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
       struct scatterlist *sg;
       int i;
 
Quoted 11 lines

kernel/dma/direct.c void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl, · line 380

Quoted 5 lines
kernel/dma/direct.c +2 -1
-     for_each_sg(sgl, sg, nents, i)
+     for_each_sg(sgl, sg, nents, i) {
               dma_direct_unmap_page(dev, sg->dma_address, sg_dma_len(sg), dir,
                            attrs);
+             sg->dma_address = DMA_MAPPING_ERROR;

There are more DMA API backends than just dma-direct, so while this might help paper over bugs when SWIOTLB is in use, it's not going to have any effect when those same bugs are hit under other circumstances. Once again, the moral of the story is that effort is better spent just fixing the bugs ;)

Quoted 9 lines

Thanks for the quick feedback. What is the correct fix? I understand the first half. The NVMe driver should be updated to not call unmap on an address that has already been unmapped within the DMA direct code. Where I'm less certain is how to communicate to the NVMe driver that the mapping failed. In particular, the NVMe code explicitly checks if the first DMA address in the scatter/gather list is set to DMA_MAPPING_ERROR. Thus, don't we need to update the DMA direct code to propagate DMA_MAPPING_ERROR back up to the driver, via the scatter/gather struct?

Erm, you check the return value of dma_map_sg(). If it's zero, the request failed; if it's nonzero, that's how many DMA segments you now have to process. See Documentation/core-api/dma-api.rst.

The only guarantee offered about the state of the scatterlist itself is that if it is successfully mapped, then the dma_address and dma_length fields are valid for that many segments, and if that is fewer than the total number of physical segments then the next one after the final DMA segment will have a dma_length of 0. In particular there are no guarantees at all about the state if the mapping was unsuccessful.

If a driver is failing to keep track of the success/failure status and later down the line trying to guess what to do with a list that may or may not have been mapped, then frankly that driver should be redesigned because that is a terrible anti-pattern. At the very very least it should explicitly encode its own "known bad" state upon failure that it can then reliably recognise later.

Robin.

Quoted 8 lines

I skimmed arch/arm/mm/dma-mapping.c, just now. I can see that this code sets the address within the scatter/gather struct to DMA_MAPPING_ERROR before trying to map an IO address and write it into the struct. Is this a good example to follow?

Thanks,
Marc

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author
#4 lore raw
Quoted 41 lines
Quoted 20 lines
Quoted 8 lines
Quoted 8 lines
kernel/dma/direct.c +0 -0
index 0a4881e59aa7..3d9b17fe5771 100644
@@ -374,9 +374,11 @@ void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
       struct scatterlist *sg;
       int i;
 
Quoted 11 lines

kernel/dma/direct.c void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl, · line 380

Quoted 5 lines
kernel/dma/direct.c +2 -1
-     for_each_sg(sgl, sg, nents, i)
+     for_each_sg(sgl, sg, nents, i) {
               dma_direct_unmap_page(dev, sg->dma_address, sg_dma_len(sg), dir,
                            attrs);
+             sg->dma_address = DMA_MAPPING_ERROR;

There are more DMA API backends than just dma-direct, so while this might help paper over bugs when SWIOTLB is in use, it's not going to have any effect when those same bugs are hit under other circumstances. Once again, the moral of the story is that effort is better spent just fixing the bugs ;)

Quoted 9 lines

Thanks for the quick feedback. What is the correct fix? I understand the first half. The NVMe driver should be updated to not call unmap on an address that has already been unmapped within the DMA direct code. Where I'm less certain is how to communicate to the NVMe driver that the mapping failed. In particular, the NVMe code explicitly checks if the first DMA address in the scatter/gather list is set to DMA_MAPPING_ERROR. Thus, don't we need to update the DMA direct code to propagate DMA_MAPPING_ERROR back up to the driver, via the scatter/gather struct?

Erm, you check the return value of dma_map_sg(). If it's zero, the request failed; if it's nonzero, that's how many DMA segments you now have to process. See Documentation/core-api/dma-api.rst.

The only guarantee offered about the state of the scatterlist itself is that if it is successfully mapped, then the dma_address and dma_length fields are valid for that many segments, and if that is fewer than the total number of physical segments then the next one after the final DMA segment will have a dma_length of 0. In particular there are no guarantees at all about the state if the mapping was unsuccessful.

Quoted 6 lines

If a driver is failing to keep track of the success/failure status and later down the line trying to guess what to do with a list that may or may not have been mapped, then frankly that driver should be redesigned because that is a terrible anti-pattern. At the very very least it should explicitly encode its own "known bad" state upon failure that it can then reliably recognise later.

Got it now. I'll get to work on a patch for the NVMe driver to fix the bug. Thanks for all of these pointers. They are immensely helpful.

Thanks,
Marc
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu