Skip to contentSkip to conversation

dma-direct: zero out DMA_ATTR_NO_KERNEL_MAPPING buf

PATCH In review Hillf Danton 4 participants 8 messages Sep 4, 2020 – Sep 7, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 dma-direct: zero out DMA_ATTR_NO_KERNEL_MAPPING buf

The DMA buffer allocated is always cleared in DMA core and this is making DMA_ATTR_NO_KERNEL_MAPPING non-special.

Fixes d98849aff879 ("dma-direct: handle DMA_ATTR_NO_KERNEL_MAPPING in common code")
Cc Kees Cook <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Marek Szyprowski <[email protected]>
Cc James Bottomley <[email protected]>

---

kernel/dma/direct.c +8 -0
@@ -178,9 +178,17 @@ void *dma_direct_alloc_pages(struct devi
 
 	if ((attrs & DMA_ATTR_NO_KERNEL_MAPPING) &&
 	    !force_dma_unencrypted(dev)) {
+		int i;
+
 		/* remove any dirty cache lines on the kernel alias */
 		if (!PageHighMem(page))
 			arch_dma_prep_coherent(page, size);
+
+		for (i = 0; i < size/PAGE_SIZE; i++) {
+			ret = kmap_atomic(page + i);
+			memset(ret, 0, PAGE_SIZE);
+			kunmap_atomic(ret);
+		}
 		/* return the page pointer as the opaque cookie */
 		ret = page;
 		goto done;

--

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#1 lore raw
Quoted 19 lines · Hillf Danton

The DMA buffer allocated is always cleared in DMA core and this is making DMA_ATTR_NO_KERNEL_MAPPING non-special.

Fixes d98849aff879 ("dma-direct: handle DMA_ATTR_NO_KERNEL_MAPPING

in common code")

Cc Kees Cook <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Marek Szyprowski <[email protected]>
Cc James Bottomley <[email protected]>

---

kernel/dma/direct.c +1 -0
@@ -178,9 +178,17 @@ void *dma_direct_alloc_pages(struct devi
 
 	if ((attrs & DMA_ATTR_NO_KERNEL_MAPPING) &&
 	    !force_dma_unencrypted(dev)) {
+		int i;

kernel/dma/direct.c void *dma_direct_alloc_pages(struct devi · line 191

Quoted 10 lines
kernel/dma/direct.c +6 -0
+
 		/* remove any dirty cache lines on the kernel alias
 */
 		if (!PageHighMem(page))
 			arch_dma_prep_coherent(page, size);
+
+		for (i = 0; i < size/PAGE_SIZE; i++) {
+			ret = kmap_atomic(page + i);
+			memset(ret, 0, PAGE_SIZE);
+			kunmap_atomic(ret);

This is massively expensive on PARISC and likely other VIPT/VIVT architectures. What's the reason for clearing it? This could also be really inefficient even on PIPT architectures if the memory is device remote.

If we really have to do this, it should likely be done in the arch or driver hooks because there are potentially more efficient ways we can do this knowing how the architecture behaves.

James

#2 lore raw
Quoted 17 lines · On Fri, 04 Sep 2020 08:34:39 -0700 James Bottomley wrote:
Quoted 16 lines · Hillf Danton

The DMA buffer allocated is always cleared in DMA core and this is making DMA_ATTR_NO_KERNEL_MAPPING non-special.

Fixes d98849aff879 ("dma-direct: handle DMA_ATTR_NO_KERNEL_MAPPING

in common code")

Cc Kees Cook <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Marek Szyprowski <[email protected]>
Cc James Bottomley <[email protected]>

---

kernel/dma/direct.c +0 -0
@@ -178,9 +178,17 @@ void *dma_direct_alloc_pages(struct devi
 
Quoted 13 lines

kernel/dma/direct.c void *dma_direct_alloc_pages(struct devi · line 190

Quoted 13 lines
kernel/dma/direct.c +7 -0
 	if ((attrs & DMA_ATTR_NO_KERNEL_MAPPING) &&
 	    !force_dma_unencrypted(dev)) {
+		int i;
+
 		/* remove any dirty cache lines on the kernel alias
 */
 		if (!PageHighMem(page))
 			arch_dma_prep_coherent(page, size);
+
+		for (i = 0; i < size/PAGE_SIZE; i++) {
+			ret = kmap_atomic(page + i);
+			memset(ret, 0, PAGE_SIZE);
+			kunmap_atomic(ret);

Hi James

Quoted 3 lines

This is massively expensive on PARISC and likely other VIPT/VIVT architectures.

Correct.

Quoted 1 line

What's the reason for clearing it? This could also be

	/* we always manually zero the memory once we are done: */
	gfp &= ~__GFP_ZERO;
	gfp |= dma_direct_optimal_gfp_mask(dev, dev->coherent_dma_mask,
					   &phys_limit);
Quoted 6 lines

really inefficient even on PIPT architectures if the memory is device remote.

If we really have to do this, it should likely be done in the arch or driver hooks because there are potentially more efficient ways we can do this knowing how the architecture behaves.

I'm open to any vintage ideas in your mind wrt clearing dma buf e.g on platforms like PARISC. Or feel free to offload me the work if it makes sense to you who are rich of PARISC knowledge.

Hillf

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#3 lore raw
Quoted 39 lines · Hillf Danton
Quoted 18 lines · On Fri, 04 Sep 2020 08:34:39 -0700 James Bottomley wrote:
Quoted 17 lines · Hillf Danton

The DMA buffer allocated is always cleared in DMA core and this is making DMA_ATTR_NO_KERNEL_MAPPING non-special.

Fixes d98849aff879 ("dma-direct: handle

DMA_ATTR_NO_KERNEL_MAPPING
in common code")

Cc Kees Cook <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Marek Szyprowski <[email protected]>
Cc James Bottomley <[email protected]>

---

kernel/dma/direct.c +0 -0
@@ -178,9 +178,17 @@ void *dma_direct_alloc_pages(struct devi
 
Quoted 14 lines
Quoted 7 lines
kernel/dma/direct.c +2 -0
 	if ((attrs & DMA_ATTR_NO_KERNEL_MAPPING) &&
 	    !force_dma_unencrypted(dev)) {
+		int i;
+
 		/* remove any dirty cache lines on the kernel

alias
*/

kernel/dma/direct.c void *dma_direct_alloc_pages(struct devi · line 190

Quoted 7 lines
kernel/dma/direct.c +5 -0
 		if (!PageHighMem(page))
 			arch_dma_prep_coherent(page, size);
+
+		for (i = 0; i < size/PAGE_SIZE; i++) {
+			ret = kmap_atomic(page + i);
+			memset(ret, 0, PAGE_SIZE);
+			kunmap_atomic(ret);

Hi James

Quoted 3 lines

This is massively expensive on PARISC and likely other VIPT/VIVT architectures.

Quoted 8 lines

Correct.

Quoted 1 line

What's the reason for clearing it? This could also be

	/* we always manually zero the memory once we are done: */
	gfp &= ~__GFP_ZERO;
	gfp |= dma_direct_optimal_gfp_mask(dev, dev->coherent_dma_mask,
					   &phys_limit);

That's not a reason ... that comment was put in for coherent mappings. What is the reason we should incur all this expense for clearing pages which aren't unmapped in the kernel, because we can update the comment?
The usual rationale for kernel mapped pages is security, because they
may leak information but unmapped pages shouldn't have this problem.

Quoted 10 lines
Quoted 6 lines

really inefficient even on PIPT architectures if the memory is device remote.

If we really have to do this, it should likely be done in the arch or driver hooks because there are potentially more efficient ways we can do this knowing how the architecture behaves.

I'm open to any vintage ideas in your mind wrt clearing dma buf e.g on platforms like PARISC. Or feel free to offload me the work if it makes sense to you who are rich of PARISC knowledge.

OK, I've cc'd linux-arch because this is a problem for more than just parisc. However, not having to do it is the best solution ... sort of the doctor, doctor it hurts when I do this answer.

James

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#4 lore raw
In reply to Hillf Danton

[resend with correct linux-arch address]

Quoted 39 lines · Hillf Danton
Quoted 18 lines · On Fri, 04 Sep 2020 08:34:39 -0700 James Bottomley wrote:
Quoted 17 lines · Hillf Danton

The DMA buffer allocated is always cleared in DMA core and this is making DMA_ATTR_NO_KERNEL_MAPPING non-special.

Fixes d98849aff879 ("dma-direct: handle

DMA_ATTR_NO_KERNEL_MAPPING
in common code")

Cc Kees Cook <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Marek Szyprowski <[email protected]>
Cc James Bottomley <[email protected]>

---

kernel/dma/direct.c +0 -0
@@ -178,9 +178,17 @@ void *dma_direct_alloc_pages(struct devi
 
Quoted 14 lines
Quoted 7 lines
kernel/dma/direct.c +2 -0
 	if ((attrs & DMA_ATTR_NO_KERNEL_MAPPING) &&
 	    !force_dma_unencrypted(dev)) {
+		int i;
+
 		/* remove any dirty cache lines on the kernel

alias
*/

kernel/dma/direct.c void *dma_direct_alloc_pages(struct devi · line 190

Quoted 7 lines
kernel/dma/direct.c +5 -0
 		if (!PageHighMem(page))
 			arch_dma_prep_coherent(page, size);
+
+		for (i = 0; i < size/PAGE_SIZE; i++) {
+			ret = kmap_atomic(page + i);
+			memset(ret, 0, PAGE_SIZE);
+			kunmap_atomic(ret);

Hi James

Quoted 3 lines

This is massively expensive on PARISC and likely other VIPT/VIVT architectures.

Quoted 8 lines

Correct.

Quoted 1 line

What's the reason for clearing it? This could also be

	/* we always manually zero the memory once we are done: */
	gfp &= ~__GFP_ZERO;
	gfp |= dma_direct_optimal_gfp_mask(dev, dev->coherent_dma_mask,
					   &phys_limit);

That's not a reason ... that comment was put in for coherent mappings. What is the reason we should incur all this expense for clearing pages which aren't unmapped in the kernel, because we can update the comment?
The usual rationale for kernel mapped pages is security, because they
may leak information but unmapped pages shouldn't have this problem.

Quoted 10 lines
Quoted 6 lines

really inefficient even on PIPT architectures if the memory is device remote.

If we really have to do this, it should likely be done in the arch or driver hooks because there are potentially more efficient ways we can do this knowing how the architecture behaves.

I'm open to any vintage ideas in your mind wrt clearing dma buf e.g on platforms like PARISC. Or feel free to offload me the work if it makes sense to you who are rich of PARISC knowledge.

OK, I've cc'd linux-arch because this is a problem for more than just parisc. However, not having to do it is the best solution ... sort of the doctor, doctor it hurts when I do this answer.

James

#5 lore raw
Quoted 3 lines · On Sat, 05 Sep 2020 08:50:42 -0700 James Bottomley wrote:

[resend with correct linux-arch address]

Quoted 1 line · Hillf Danton

On Fri, 04 Sep 2020 08:34:39 -0700 James Bottomley wrote:

[...]

Quoted 7 lines
Quoted 7 lines

Hi James

Quoted 3 lines

This is massively expensive on PARISC and likely other VIPT/VIVT architectures.

Correct.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

What's the reason for clearing it? This could also be

	/* we always manually zero the memory once we are done: */
	gfp &= ~__GFP_ZERO;
	gfp |= dma_direct_optimal_gfp_mask(dev, dev->coherent_dma_mask,
					   &phys_limit);

That's not a reason ... that comment was put in for coherent mappings.

It's quite likely that I misread it.

Quoted 2 lines

What is the reason we should incur all this expense for clearing pages which aren't unmapped in the kernel, because we can update the comment?

A tough question.

Quoted 2 lines

The usual rationale for kernel mapped pages is security, because they may leak information but unmapped pages shouldn't have this problem.

We can turn to Marek for some more light on the concerns you have about security, particularly linked to DMA_ATTR_NO_KERNEL_MAPPING, after the patch [1] fails to answer your question.

[1] Subject: [PATCH 1/3] common: DMA-mapping: add DMA_ATTR_NO_KERNEL_MAPPING attribute
https://lore.kernel.org/lkml/[email protected]/

Quoted 12 lines
Quoted 11 lines
Quoted 7 lines

really inefficient even on PIPT architectures if the memory is device remote.

If we really have to do this, it should likely be done in the arch or driver hooks because there are potentially more efficient ways we can do this knowing how the architecture behaves.

I'm open to any vintage ideas in your mind wrt clearing dma buf e.g on platforms like PARISC. Or feel free to offload me the work if it makes sense to you who are rich of PARISC knowledge.

Quoted 5 lines

OK, I've cc'd linux-arch because this is a problem for more than just parisc. However, not having to do it is the best solution ... sort of the doctor, doctor it hurts when I do this answer.

James

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

maintainer
#6 lore raw
In reply to James Bottomley

Hi James,

Quoted 45 lines · James Bottomley

[resend with correct linux-arch address]

Quoted 36 lines · Hillf Danton
Quoted 18 lines · On Fri, 04 Sep 2020 08:34:39 -0700 James Bottomley wrote:
Quoted 17 lines · Hillf Danton

The DMA buffer allocated is always cleared in DMA core and this is making DMA_ATTR_NO_KERNEL_MAPPING non-special.

Fixes d98849aff879 ("dma-direct: handle

DMA_ATTR_NO_KERNEL_MAPPING
in common code")

Cc Kees Cook <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Marek Szyprowski <[email protected]>
Cc James Bottomley <[email protected]>

---

kernel/dma/direct.c +0 -0
@@ -178,9 +178,17 @@ void *dma_direct_alloc_pages(struct devi
 
Quoted 14 lines
Quoted 9 lines
kernel/dma/direct.c +2 -0
  	if ((attrs & DMA_ATTR_NO_KERNEL_MAPPING) &&
  	    !force_dma_unencrypted(dev)) {
+		int i;
+
  		/* remove any dirty cache lines on the kernel

alias
*/
if (!PageHighMem(page))
arch_dma_prep_coherent(page, size);

kernel/dma/direct.c void *dma_direct_alloc_pages(struct devi · line 190

Quoted 5 lines
kernel/dma/direct.c +5 -0
+
+		for (i = 0; i < size/PAGE_SIZE; i++) {
+			ret = kmap_atomic(page + i);
+			memset(ret, 0, PAGE_SIZE);
+			kunmap_atomic(ret);

Hi James

Quoted 2 lines

This is massively expensive on PARISC and likely other VIPT/VIVT architectures.

Quoted 7 lines

Correct.

Quoted 1 line

What's the reason for clearing it? This could also be

	/* we always manually zero the memory once we are done: */
	gfp &= ~__GFP_ZERO;
	gfp |= dma_direct_optimal_gfp_mask(dev, dev->coherent_dma_mask,
					   &phys_limit);
Quoted 5 lines

That's not a reason ... that comment was put in for coherent mappings. What is the reason we should incur all this expense for clearing pages which aren't unmapped in the kernel, because we can update the comment?
The usual rationale for kernel mapped pages is security, because they
may leak information but unmapped pages shouldn't have this problem.

Any dma_alloc_attrs() buffer might be mmaped to userspace, so the security reason is still valid. Possible lack if kernel mapping was only a hint that driver doesn't need it, so it might be skipped on some architectures, where creating it requires significant resources (i.e. vmalloc area).

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

really inefficient even on PIPT architectures if the memory is device remote.

If we really have to do this, it should likely be done in the arch or driver hooks because there are potentially more efficient ways we can do this knowing how the architecture behaves.

Quoted 6 lines
Quoted 3 lines

I'm open to any vintage ideas in your mind wrt clearing dma buf e.g on platforms like PARISC. Or feel free to offload me the work if it makes sense to you who are rich of PARISC knowledge.

OK, I've cc'd linux-arch because this is a problem for more than just parisc. However, not having to do it is the best solution ... sort of the doctor, doctor it hurts when I do this answer.

Best regards

Signature · 8 lines
Marek Szyprowski, PhD
Samsung R&D Institute Poland


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
#7 lore raw
Quoted 11 lines · Marek Szyprowski
Quoted 5 lines

That's not a reason ... that comment was put in for coherent mappings. What is the reason we should incur all this expense for clearing pages which aren't unmapped in the kernel, because we can update the comment?
The usual rationale for kernel mapped pages is security, because they
may leak information but unmapped pages shouldn't have this problem.

Any dma_alloc_attrs() buffer might be mmaped to userspace, so the security reason is still valid. Possible lack if kernel mapping was only a hint that driver doesn't need it, so it might be skipped on some architectures, where creating it requires significant resources (i.e. vmalloc area).

Yes. It seems actually mapping it to userspace in media/drm drivers seems to be one of the big use cases. There other one is memory not used by the host at all and just as an extra buffer for hardware so that the PCIe device can cut down on DRAM cost. For that could potentially skip the zeroing, but then again you'd need to trust the device, which with thunderbolt might not always be a good idea.