Skip to contentSkip to conversation

dma-buf: heaps: system: zero pages after set_memory_decrypted()

PATCH In review Ying Xiao 5 participants 5 messages Nov 17 – Nov 18

Subsystems (3)
lore
Share & tools
lore

Ying Xiao

author
#0 lore raw

PATCH 1/1 dma-buf: heaps: system: zero pages after set_memory_decrypted()

system_heap_allocate() allocates pages via order_flags[] with __GFP_ZERO while the direct map still has the memory encryption bit set. For the system_cc_shared heap, set_memory_decrypted() subsequently clears the encryption bit without re-zeroing, leaving ciphertext in DRAM on confidential computing (CoCo) guests instead of plaintext zeros.

Skip __GFP_ZERO during alloc_pages() for cc_shared allocations, and zero the pages once in system_heap_set_page_decrypted() after set_memory_decrypted() succeeds.

Fixes 78b30c50a7ac ("dma-buf: heaps: system: add system_cc_shared heap for explicitly shared memory")
---
 drivers/dma-buf/heaps/system_heap.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)
drivers/dma-buf/heaps/system_heap.c +20 -4
index c8959eadc71d..5f98d0df11e7 100644
@@ -72,11 +72,19 @@ static int system_heap_set_page_decrypted(struct page *page)
 	int ret;
 
 	ret = set_memory_decrypted(addr, nr_pages);
-	if (ret)
+	if (ret) {
 		pr_warn_ratelimited("dma-buf system heap: failed to decrypt page at %p\n",
 				    page_address(page));
+		return ret;
+	}
 
-	return ret;
+	/*
+	 * Clearing the encryption bit does not clear DRAM, leaving ciphertext
+	 * from when the page was encrypted. Zero it now that it is decrypted.
+	 */
+	memset((void *)addr, 0, (size_t)nr_pages << PAGE_SHIFT);
+
+	return 0;
 }
 
 static int system_heap_set_page_encrypted(struct page *page)
@@ -376,7 +384,8 @@ static const struct dma_buf_ops system_heap_buf_ops = {
 };
 
 static struct page *alloc_largest_available(unsigned long size,
-					    unsigned int max_order)
+					    unsigned int max_order,
+					    bool cc_shared)
 {
 	struct page *page;
 	int i;
@@ -388,6 +397,12 @@ static struct page *alloc_largest_available(unsigned long size,
 		if (max_order < orders[i])
 			continue;
 		flags = order_flags[i];
+		/*
+		 * Zeroing while encrypted is wasted since decryption leaves
+		 * ciphertext; system_heap_set_page_decrypted() zeroes instead.
+		 */
+		if (cc_shared)
+			flags &= ~__GFP_ZERO;
 		if (mem_accounting)
 			flags |= __GFP_ACCOUNT;
 		page = alloc_pages(flags, orders[i]);
@@ -438,7 +453,8 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
 			goto free_buffer;
 		}
 
-		page = alloc_largest_available(size_remaining, max_order);
+		page = alloc_largest_available(size_remaining, max_order,
+					       cc_shared_buffer(buffer));
 		if (!page)
 			goto free_buffer;
 
Signature · 1 line
2.56.0.679.gc527c65306-goog
#1 lore raw
Thank you for your contribution! Sashiko AI review found no new issues in this patch, but noticed 1 related pre-existing issue it does not introduce: - [Critical] dma-buf: heaps: double encryption…

Thank you for your contribution! Sashiko AI review found no new issues in this patch, but noticed 1 related pre-existing issue it does not introduce:
- [Critical] dma-buf: heaps: double encryption and memory leak in system_heap_allocate() [1]
--

commit 8e29228b9a5d6486d110b63bcefe3df39d060082
Author: Ying Xiao <[email protected]>

dma-buf: heaps: system: zero pages after set_memory_decrypted()

This commit skips __GFP_ZERO when allocating pages for the system_cc_shared heap and zeroes the pages in system_heap_set_page_decrypted() after set_memory_decrypted() succeeds.

Quoted 4 lines
drivers/dma-buf/heaps/system_heap.c +0 -0
index c8959eadc71d1..5f98d0df11e76 100644

[ ... ]

drivers/dma-buf/heaps/system_heap.c static struct dma_buf *system_heap_allocate(struct dma_heap *heap, · line 460

Quoted 10 lines
drivers/dma-buf/heaps/system_heap.c +2 -1
@@ -438,7 +453,8 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
 			goto free_buffer;
 		}
 
-		page = alloc_largest_available(size_remaining, max_order);
+		page = alloc_largest_available(size_remaining, max_order,
+					       cc_shared_buffer(buffer));
 		if (!page)
 			goto free_buffer;
 

[Severity: Critical]
This problem wasn't introduced by this patch, but does the free_pages error path in system_heap_allocate() call system_heap_set_page_encrypted() on pages that were never decrypted [1] when system_heap_set_page_decrypted() fails partway through the scatterlist?

drivers/dma-buf/heaps/system_heap.c:system_heap_allocate() {
    ...
	if (cc_shared_buffer(buffer)) {
		for_each_sgtable_sg(table, sg, i) {
			ret = system_heap_set_page_decrypted(sg_page(sg));
			if (ret)
				goto free_pages;
		}
	}
    ...
free_pages:
	for_each_sgtable_sg(table, sg, i) {
		struct page *p = sg_page(sg);
    ...
		if (cc_shared_buffer(buffer) &&
		    system_heap_set_page_encrypted(p))
			continue;
		__free_pages(p, compound_order(p));
	}
    ...
}

At free_pages, for_each_sgtable_sg() resets i to 0 and iterates over the entire scatterlist instead of only the entries that were decrypted. Calling system_heap_set_page_encrypted() on pages that are still encrypted can fault in confidential computing guests, or fail and skip __free_pages(), leaking the pages.

[1] https://sashiko.dev/#/bug/linux-ece74ffb-d452-4ab2-b56e-aa5701dddcf3

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Acked-by on v1 1/1
#2 lore raw
In reply to Ying Xiao
Quoted 6 lines · Ying Xiao

system_heap_allocate() allocates pages via order_flags[] with __GFP_ZERO while the direct map still has the memory encryption bit set. For the system_cc_shared heap, set_memory_decrypted() subsequently clears the encryption bit without re-zeroing, leaving ciphertext in DRAM on confidential computing (CoCo) guests instead of plaintext zeros.

Quoted 7 lines

Skip __GFP_ZERO during alloc_pages() for cc_shared allocations, and zero the pages once in system_heap_set_page_decrypted() after set_memory_decrypted() succeeds.

Fixes 78b30c50a7ac ("dma-buf: heaps: system: add system_cc_shared heap for explicitly shared memory")
Quoted 54 lines
---
 drivers/dma-buf/heaps/system_heap.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)
drivers/dma-buf/heaps/system_heap.c +18 -3
index c8959eadc71d..5f98d0df11e7 100644
@@ -72,11 +72,19 @@ static int system_heap_set_page_decrypted(struct page *page)
 	int ret;
 
 	ret = set_memory_decrypted(addr, nr_pages);
-	if (ret)
+	if (ret) {
 		pr_warn_ratelimited("dma-buf system heap: failed to decrypt page at %p\n",
 				    page_address(page));
+		return ret;
+	}
 
-	return ret;
+	/*
+	 * Clearing the encryption bit does not clear DRAM, leaving ciphertext
+	 * from when the page was encrypted. Zero it now that it is decrypted.
+	 */
+	memset((void *)addr, 0, (size_t)nr_pages << PAGE_SHIFT);
+
+	return 0;
 }
 
 static int system_heap_set_page_encrypted(struct page *page)
@@ -376,7 +384,8 @@ static const struct dma_buf_ops system_heap_buf_ops = {
 };
 
 static struct page *alloc_largest_available(unsigned long size,
-					    unsigned int max_order)
+					    unsigned int max_order,
+					    bool cc_shared)
 {
 	struct page *page;
 	int i;
@@ -388,6 +397,12 @@ static struct page *alloc_largest_available(unsigned long size,
 		if (max_order < orders[i])
 			continue;
 		flags = order_flags[i];
+		/*
+		 * Zeroing while encrypted is wasted since decryption leaves
+		 * ciphertext; system_heap_set_page_decrypted() zeroes instead.
+		 */
+		if (cc_shared)
+			flags &= ~__GFP_ZERO;
 		if (mem_accounting)
 			flags |= __GFP_ACCOUNT;
 		page = alloc_pages(flags, orders[i]);

drivers/dma-buf/heaps/system_heap.c static struct dma_buf *system_heap_allocate(struct dma_heap *heap, · line 460

Quoted 9 lines
drivers/dma-buf/heaps/system_heap.c +2 -1
@@ -438,7 +453,8 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
 			goto free_buffer;
 		}
 
-		page = alloc_largest_available(size_remaining, max_order);
+		page = alloc_largest_available(size_remaining, max_order,
+					       cc_shared_buffer(buffer));
 		if (!page)
 			goto free_buffer;
 

Best regards

Signature · 2 lines
Marek Szyprowski, PhD
Samsung R&D Institute Poland
Reviewed-by on v1 1/1
#3 lore raw
In reply to Ying Xiao
Quoted 6 lines · Tue, Nov 17, 2026 at 04:54:31PM +0100, [email protected] wrote:

system_heap_allocate() allocates pages via order_flags[] with __GFP_ZERO while the direct map still has the memory encryption bit set. For the system_cc_shared heap, set_memory_decrypted() subsequently clears the encryption bit without re-zeroing, leaving ciphertext in DRAM on confidential computing (CoCo) guests instead of plaintext zeros.

Quoted 7 lines

Skip __GFP_ZERO during alloc_pages() for cc_shared allocations, and zero the pages once in system_heap_set_page_decrypted() after set_memory_decrypted() succeeds.

Fixes 78b30c50a7ac ("dma-buf: heaps: system: add system_cc_shared heap for explicitly shared memory")
Reviewed-by on v1 1/1
#4 lore raw
In reply to Ying Xiao
Quoted 10 lines · Ying Xiao

system_heap_allocate() allocates pages via order_flags[] with __GFP_ZERO while the direct map still has the memory encryption bit set. For the system_cc_shared heap, set_memory_decrypted() subsequently clears the encryption bit without re-zeroing, leaving ciphertext in DRAM on confidential computing (CoCo) guests instead of plaintext zeros.

Skip __GFP_ZERO during alloc_pages() for cc_shared allocations, and zero the pages once in system_heap_set_page_decrypted() after set_memory_decrypted() succeeds.

Quoted 6 lines
Fixes 78b30c50a7ac ("dma-buf: heaps: system: add system_cc_shared heap for explicitly shared memory")
---
 drivers/dma-buf/heaps/system_heap.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

This seems fine, but Aneesh is fixing this in a more complete way with this other series.

Jason