Skip to contentSkip to conversation

Clear shared pages after private-to-shared conversion

PATCH In review Steven Price 5 participants 11 messages Aug 20

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

v2 cover Clear shared pages after private-to-shared conversion

set_memory_decrypted() doesn't (currently) guarantee to preserve or zero memory, but both the GICv3 ITS driver and the system_cc_shared dma-buf heap currently allocate memory with __GFP_ZERO followed by calling set_memory_decrypted(). On an Arm CCA system with MEC this can cause ciphertext to be visible to the guest rather than the expected zeros.

Patches 1 and 3 fix this by zeroing after the set_memory_decrypted() call.

Patches 2 and 4 fix other related bugs that Sashiko found. Patch 2 fixes the issue that set_memory_decrypted() can be a sleeping call, so moves the allocation out of an atomic context.

Patch 4 deals with the situation where set_memory_decrypted() fails and the rollback path could attempt to re-encrypt memory which was never decrypted.

I've sorted the patches by area, but there's no (semantic) dependency between them.

Changes in v2:
* Switched to use BIT(n) rather than 1 << n in the GICv3 change.
* Added Jason's R-b.
* Patches 2 and 4 are new.

v1: https://lore.kernel.org/r/[email protected]

Steven Price (4):
irqchip/gic-v3-its: Zero shared pages after conversion irqchip/gic-v3-its: Allocate VPE tables from sleepable context
dma-buf: heaps: Zero system shared heap pages after conversion
dma-buf: heaps: Fix shared system heap allocation rollback

 drivers/dma-buf/heaps/system_heap.c | 24 +++++++++++++-----
 drivers/irqchip/irq-gic-v3-its.c    | 39 ++++++++++++++++-------------
 2 files changed, 40 insertions(+), 23 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 1/4 irqchip/gic-v3-its: Zero shared pages after conversion

its_alloc_pages_node() passes __GFP_ZERO to the page allocator before calling set_memory_decrypted(). This assumes that converting a page from private to shared preserves its contents.

For Arm CCA with MEC (Memory Encryption Contexts) the key used to access the page will change, and so by default the visible data will change. The host could ensure that it zeros the page, but rather than relying on the host's behaviour it's best if the guest simply zeros after the decryption rather than before. Specifically in this case the ITS tables are required to be zeroed.

Mask out __GFP_ZERO from the allocation request, and do the zeroing as a separate step after decryption.

Fixes b08e2f42e86b ("irqchip/gic-v3-its: Share ITS tables with a non-trusted hypervisor")
---
 drivers/irqchip/irq-gic-v3-its.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +7 -2
index 6f5811aae59c..a055837832bc 100644
@@ -213,16 +213,18 @@ static gfp_t gfp_flags_quirk;
 static struct page *its_alloc_pages_node(int node, gfp_t gfp,
 					 unsigned int order)
 {
+	bool want_zero = gfp & __GFP_ZERO;
 	struct page *page;
 	int ret = 0;
 
-	page = alloc_pages_node(node, gfp | gfp_flags_quirk, order);
+	page = alloc_pages_node(node, (gfp & ~__GFP_ZERO) | gfp_flags_quirk,
+				order);
 
 	if (!page)
 		return NULL;
 
 	ret = set_memory_decrypted((unsigned long)page_address(page),
-				   1 << order);
+				   BIT(order));
 	/*
 	 * If set_memory_decrypted() fails then we don't know what state the
 	 * page is in, so we can't free it. Instead we leak it.
@@ -231,6 +233,9 @@ static struct page *its_alloc_pages_node(int node, gfp_t gfp,
 	if (ret)
 		return NULL;
 
+	if (want_zero)
+		clear_pages(page_address(page), BIT(order));
+
 	return page;
 }
 
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH v2 1/4] irqchip/gic-v3-its: Zero shared pages after conversion

Quoted 5 lines · Steven Price
drivers/irqchip/irq-gic-v3-its.c +0 -0
index 6f5811aae59c..a055837832bc 100644

drivers/irqchip/irq-gic-v3-its.c static gfp_t gfp_flags_quirk; · line 221

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +3 -1
@@ -213,16 +213,18 @@ static gfp_t gfp_flags_quirk;
 static struct page *its_alloc_pages_node(int node, gfp_t gfp,
 					 unsigned int order)
 {
+	bool want_zero = gfp & __GFP_ZERO;
 	struct page *page;
 	int ret = 0;
 
-	page = alloc_pages_node(node, gfp | gfp_flags_quirk, order);
+	page = alloc_pages_node(node, (gfp & ~__GFP_ZERO) | gfp_flags_quirk,
+				order);

I don't think pKVM does any scrubbing on set_memory_decrypted(), so it potentially exposes confidential guest data before it reaches clear_pages() below.

Quoted 22 lines
drivers/irqchip/irq-gic-v3-its.c +4 -1
 	if (!page)
 		return NULL;
 
 	ret = set_memory_decrypted((unsigned long)page_address(page),
-				   1 << order);
+				   BIT(order));
 	/*
 	 * If set_memory_decrypted() fails then we don't know what state the
 	 * page is in, so we can't free it. Instead we leak it.
@@ -231,6 +233,9 @@ static struct page *its_alloc_pages_node(int node, gfp_t gfp,
 	if (ret)
 		return NULL;
 
+	if (want_zero)
+		clear_pages(page_address(page), BIT(order));
+
 	return page;
 }
 
Signature · 1 line
2.43.0
Signature · 1 line
Catalin
#3 lore raw

Re: [PATCH v2 1/4] irqchip/gic-v3-its: Zero shared pages after conversion

Quoted 13 lines · Catalin Marinas
Quoted 12 lines · Steven Price
drivers/irqchip/irq-gic-v3-its.c +1 -0
index 6f5811aae59c..a055837832bc 100644
@@ -213,16 +213,18 @@ static gfp_t gfp_flags_quirk;
 static struct page *its_alloc_pages_node(int node, gfp_t gfp,
 					 unsigned int order)
 {
+	bool want_zero = gfp & __GFP_ZERO;
 	struct page *page;
 	int ret = 0;
 
Quoted 7 lines

drivers/irqchip/irq-gic-v3-its.c static gfp_t gfp_flags_quirk; · line 221

Quoted 3 lines
drivers/irqchip/irq-gic-v3-its.c +2 -1
-	page = alloc_pages_node(node, gfp | gfp_flags_quirk, order);
+	page = alloc_pages_node(node, (gfp & ~__GFP_ZERO) | gfp_flags_quirk,
+				order);

I don't think pKVM does any scrubbing on set_memory_decrypted(), so it potentially exposes confidential guest data before it reaches clear_pages() below.

IMHO that has got to be handled in the arch code implementing set memory decrypted.

Further pointing that maybe we should have an alloc decrypted so we can at least try to minimize the number of times we write to this memory. :(

Jason

#4 lore raw

Re: [PATCH v2 1/4] irqchip/gic-v3-its: Zero shared pages after conversion

Quoted 22 lines · Jason Gunthorpe
Quoted 13 lines · Catalin Marinas
Quoted 12 lines · Steven Price
drivers/irqchip/irq-gic-v3-its.c +1 -0
index 6f5811aae59c..a055837832bc 100644
@@ -213,16 +213,18 @@ static gfp_t gfp_flags_quirk;
 static struct page *its_alloc_pages_node(int node, gfp_t gfp,
 					 unsigned int order)
 {
+	bool want_zero = gfp & __GFP_ZERO;
 	struct page *page;
 	int ret = 0;
 
Quoted 7 lines

drivers/irqchip/irq-gic-v3-its.c static gfp_t gfp_flags_quirk; · line 221

Quoted 3 lines
drivers/irqchip/irq-gic-v3-its.c +2 -1
-	page = alloc_pages_node(node, gfp | gfp_flags_quirk, order);
+	page = alloc_pages_node(node, (gfp & ~__GFP_ZERO) | gfp_flags_quirk,
+				order);

I don't think pKVM does any scrubbing on set_memory_decrypted(), so it potentially exposes confidential guest data before it reaches clear_pages() below.

Quoted 2 lines

IMHO that has got to be handled in the arch code implementing set memory decrypted.

I guess that's a better separation. It probably needs to clear the MTE tags as well, it's not great to leak them (though not as bad as leaking data).

OTOH, pKVM would no longer need the clear_pages() afterwards since there's no encryption key changed. Not too bad, it's not a hot path.

Quoted 3 lines

Further pointing that maybe we should have an alloc decrypted so we can at least try to minimize the number of times we write to this memory. :(

This would be better.

Signature · 1 line
Catalin
#5 lore raw

PATCH v2 2/4 irqchip/gic-v3-its: Allocate VPE tables from sleepable context

In reply to Steven Price

The VPE L1 table is allocated from the CPU-starting hotplug state, where interrupts are disabled. Although the page allocation uses GFP_ATOMIC, its_alloc_pages() subsequently calls set_memory_decrypted(), which can sleep while splitting the arm64 linear map.

Move the allocation to the existing CPU-online callback, which runs in sleepable context, and use GFP_KERNEL for both allocations performed there. Register the callback even without EFI, since it is now also responsible for VPE table allocation.

Fixes b08e2f42e86b ("irqchip/gic-v3-its: Share ITS tables with a non-trusted hypervisor")
---
 drivers/irqchip/irq-gic-v3-its.c | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +15 -15
index a055837832bc..71515dbff9ec 100644
@@ -2932,7 +2932,7 @@ static int allocate_vpe_l1_table(void)
 	if (val & GICR_VPROPBASER_4_1_VALID)
 		goto out;
 
-	gic_data_rdist()->vpe_table_mask = kzalloc_obj(cpumask_t, GFP_ATOMIC);
+	gic_data_rdist()->vpe_table_mask = kzalloc_obj(cpumask_t, GFP_KERNEL);
 	if (!gic_data_rdist()->vpe_table_mask)
 		return -ENOMEM;
 
@@ -2999,7 +2999,7 @@ static int allocate_vpe_l1_table(void)
 
 	pr_debug("np = %d, npg = %lld, psz = %d, epp = %d, esz = %d\n",
 		 np, npg, psz, epp, esz);
-	page = its_alloc_pages(GFP_ATOMIC | __GFP_ZERO, get_order(np * PAGE_SIZE));
+	page = its_alloc_pages(GFP_KERNEL | __GFP_ZERO, get_order(np * PAGE_SIZE));
 	if (!page)
 		return -ENOMEM;
 
@@ -3268,16 +3268,6 @@ static void its_cpu_init_lpis(void)
 		val = its_clear_vpend_valid(vlpi_base, 0, 0);
 	}
 
-	if (allocate_vpe_l1_table()) {
-		/*
-		 * If the allocation has failed, we're in massive trouble.
-		 * Disable direct injection, and pray that no VM was
-		 * already running...
-		 */
-		gic_rdists->has_rvpeid = false;
-		gic_rdists->has_vlpis = false;
-	}
-
 	/* Make sure the GIC has seen the above */
 	dsb(sy);
 	gic_data_rdist()->flags |= RD_LOCAL_LPI_ENABLED;
@@ -5452,6 +5442,19 @@ static int its_cpu_memreserve_lpi(unsigned int cpu)
 	if (gic_data_rdist()->flags & RD_LOCAL_MEMRESERVE_DONE)
 		return 0;
 
+	if (allocate_vpe_l1_table()) {
+		/*
+		 * If the allocation has failed, we're in massive trouble.
+		 * Disable direct injection, and pray that no VM was
+		 * already running...
+		 */
+		gic_rdists->has_rvpeid = false;
+		gic_rdists->has_vlpis = false;
+	}
+
+	if (!efi_enabled(EFI_CONFIG_TABLES))
+		goto out;
+
 	pend_page = gic_data_rdist()->pend_page;
 	if (WARN_ON(!pend_page)) {
 		ret = -ENOMEM;
@@ -5793,9 +5796,6 @@ int __init its_lpi_memreserve_init(void)
 {
 	int state;
 
-	if (!efi_enabled(EFI_CONFIG_TABLES))
-		return 0;
-
 	if (list_empty(&its_nodes))
 		return 0;
 
Signature · 1 line
2.43.0
#6 lore raw

Re: [PATCH v2 2/4] irqchip/gic-v3-its: Allocate VPE tables from sleepable context

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Moving VPE L1 table allocation to a late CPU hotplug state exposes uninitialized GICv4.1…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Moving VPE L1 table allocation to a late CPU hotplug state exposes uninitialized GICv4.1 redistributor state to concurrent VPE mappings. --

--- Patch [2]: [PATCH v2 2/4] irqchip/gic-v3-its: Allocate VPE tables from sleepable context --- commit 7ed963cf27a623f2c21ad61ef0162e3eddac65ae
Author: Steven Price <[email protected]>

irqchip/gic-v3-its: Allocate VPE tables from sleepable context

This patch moves the VPE L1 table allocation from the early CPU-starting hotplug state to the later CPU-online callback. This permits the use of GFP_KERNEL allocations, as its_alloc_pages() can sleep. The CPU-online callback is now registered unconditionally to handle this setup.

Quoted 4 lines
drivers/irqchip/irq-gic-v3-its.c +0 -0
index bfd58097261a7..6452888b34fcb 100644

[ ... ]

Quoted 17 lines
drivers/irqchip/irq-gic-v3-its.c +0 -10
@@ -3268,16 +3268,6 @@ static void its_cpu_init_lpis(void)
 		val = its_clear_vpend_valid(vlpi_base, 0, 0);
 	}
 
-	if (allocate_vpe_l1_table()) {
-		/*
-		 * If the allocation has failed, we're in massive trouble.
-		 * Disable direct injection, and pray that no VM was
-		 * already running...
-		 */
-		gic_rdists->has_rvpeid = false;
-		gic_rdists->has_vlpis = false;
-	}
-
 	/* Make sure the GIC has seen the above */
 	dsb(sy);
 	gic_data_rdist()->flags |= RD_LOCAL_LPI_ENABLED;

[ ... ]

Quoted 4 lines
drivers/irqchip/irq-gic-v3-its.c +0 -0
@@ -5454,6 +5444,19 @@ static int its_cpu_memreserve_lpi(unsigned int cpu)
 	if (gic_data_rdist()->flags & RD_LOCAL_MEMRESERVE_DONE)
 		return 0;
 

drivers/irqchip/irq-gic-v3-its.c static int its_cpu_memreserve_lpi(unsigned int cpu) · line 5456

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +10 -0
+	if (allocate_vpe_l1_table()) {
+		/*
+		 * If the allocation has failed, we're in massive trouble.
+		 * Disable direct injection, and pray that no VM was
+		 * already running...
+		 */
+		gic_rdists->has_rvpeid = false;
+		gic_rdists->has_vlpis = false;
+	}
+

[Severity: High]
Does moving this allocation to the CPUHP_AP_ONLINE_DYN callback expose uninitialized GICv4.1 redistributor state to concurrent VPE mappings?

The CPUHP_AP_ONLINE_DYN state runs for its_cpu_memreserve_lpi() in the hotplug thread asynchronously after the CPU has already been marked online in cpu_online_mask.

Could a concurrent process changing IRQ affinities (like irqbalance) observe the CPU in cpu_online_mask and route a virtual interrupt to it via its_vpe_set_affinity() before the hotplug thread has executed this table allocation?

If a VMAPP command executes before allocate_vpe_l1_table() programs the GICR_VPROPBASER, it appears the hardware could use stale or uninitialized physical addresses, leading to unpredictable behavior or dropped interrupts.

Quoted 3 lines
drivers/irqchip/irq-gic-v3-its.c +0 -0
 	if (!efi_enabled(EFI_CONFIG_TABLES))
 		goto out;
 
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#7 lore raw

Re: [PATCH v2 2/4] irqchip/gic-v3-its: Allocate VPE tables from sleepable context

In reply to Steven Price
Quoted 10 lines · Steven Price

The VPE L1 table is allocated from the CPU-starting hotplug state, where interrupts are disabled. Although the page allocation uses GFP_ATOMIC, its_alloc_pages() subsequently calls set_memory_decrypted(), which can sleep while splitting the arm64 linear map.

Move the allocation to the existing CPU-online callback, which runs in sleepable context, and use GFP_KERNEL for both allocations performed there. Register the callback even without EFI, since it is now also responsible for VPE table allocation.

Ok, Sashiko pointed out this is bunk:

Quoted 7 lines

Does moving this allocation to the CPUHP_AP_ONLINE_DYN callback expose uninitialized GICv4.1 redistributor state to concurrent VPE mappings?

The CPUHP_AP_ONLINE_DYN state runs for its_cpu_memreserve_lpi() in the hotplug thread asynchronously after the CPU has already been marked online in cpu_online_mask.

Quoted 8 lines

Could a concurrent process changing IRQ affinities (like irqbalance) observe the CPU in cpu_online_mask and route a virtual interrupt to it via its_vpe_set_affinity() before the hotplug thread has executed this table allocation?

If a VMAPP command executes before allocate_vpe_l1_table() programs the GICR_VPROPBASER, it appears the hardware could use stale or uninitialized physical addresses, leading to unpredictable behavior or dropped interrupts.

Please ignore this patch. I thought this was a bit too simple to fix :( I guess some sort of preallocation might be the solution.

Thanks,
Steve

Quoted 75 lines
Fixes b08e2f42e86b ("irqchip/gic-v3-its: Share ITS tables with a non-trusted hypervisor")
---
 drivers/irqchip/irq-gic-v3-its.c | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +15 -15
index a055837832bc..71515dbff9ec 100644
@@ -2932,7 +2932,7 @@ static int allocate_vpe_l1_table(void)
 	if (val & GICR_VPROPBASER_4_1_VALID)
 		goto out;
 
-	gic_data_rdist()->vpe_table_mask = kzalloc_obj(cpumask_t, GFP_ATOMIC);
+	gic_data_rdist()->vpe_table_mask = kzalloc_obj(cpumask_t, GFP_KERNEL);
 	if (!gic_data_rdist()->vpe_table_mask)
 		return -ENOMEM;
 
@@ -2999,7 +2999,7 @@ static int allocate_vpe_l1_table(void)
 
 	pr_debug("np = %d, npg = %lld, psz = %d, epp = %d, esz = %d\n",
 		 np, npg, psz, epp, esz);
-	page = its_alloc_pages(GFP_ATOMIC | __GFP_ZERO, get_order(np * PAGE_SIZE));
+	page = its_alloc_pages(GFP_KERNEL | __GFP_ZERO, get_order(np * PAGE_SIZE));
 	if (!page)
 		return -ENOMEM;
 
@@ -3268,16 +3268,6 @@ static void its_cpu_init_lpis(void)
 		val = its_clear_vpend_valid(vlpi_base, 0, 0);
 	}
 
-	if (allocate_vpe_l1_table()) {
-		/*
-		 * If the allocation has failed, we're in massive trouble.
-		 * Disable direct injection, and pray that no VM was
-		 * already running...
-		 */
-		gic_rdists->has_rvpeid = false;
-		gic_rdists->has_vlpis = false;
-	}
-
 	/* Make sure the GIC has seen the above */
 	dsb(sy);
 	gic_data_rdist()->flags |= RD_LOCAL_LPI_ENABLED;
@@ -5452,6 +5442,19 @@ static int its_cpu_memreserve_lpi(unsigned int cpu)
 	if (gic_data_rdist()->flags & RD_LOCAL_MEMRESERVE_DONE)
 		return 0;
 
+	if (allocate_vpe_l1_table()) {
+		/*
+		 * If the allocation has failed, we're in massive trouble.
+		 * Disable direct injection, and pray that no VM was
+		 * already running...
+		 */
+		gic_rdists->has_rvpeid = false;
+		gic_rdists->has_vlpis = false;
+	}
+
+	if (!efi_enabled(EFI_CONFIG_TABLES))
+		goto out;
+
 	pend_page = gic_data_rdist()->pend_page;
 	if (WARN_ON(!pend_page)) {
 		ret = -ENOMEM;
@@ -5793,9 +5796,6 @@ int __init its_lpi_memreserve_init(void)
 {
 	int state;
 
-	if (!efi_enabled(EFI_CONFIG_TABLES))
-		return 0;
-
 	if (list_empty(&its_nodes))
 		return 0;
 
#8 lore raw

PATCH v2 3/4 dma-buf: heaps: Zero system shared heap pages after conversion

In reply to Steven Price

The system_cc_shared heap allocates pages with __GFP_ZERO before converting them from private to shared with set_memory_decrypted(). This assumes that the conversion preserves the contents of the pages.

For Arm CCA with MEC (Memory Encryption Contexts) the key used to access the page will change, and so by default the visible data will change. The host could ensure that it zeros the page after decryption, but rather than relying on the host's behaviour it's best if the guest simply zeros after the decryption rather than before.

For CC shared buffers, defer zeroing until each page has been converted successfully. For other buffers keep the existing behaviour.

Fixes 78b30c50a7ac ("dma-buf: heaps: system: add system_cc_shared heap for explicitly shared memory")
---
 drivers/dma-buf/heaps/system_heap.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
drivers/dma-buf/heaps/system_heap.c +11 -3
index c8959eadc71d..f14930904089 100644
@@ -376,7 +376,8 @@ static const struct dma_buf_ops system_heap_buf_ops = {
 };
 
 static struct page *alloc_largest_available(unsigned long size,
-					    unsigned int max_order)
+					    unsigned int max_order,
+					    bool defer_zero)
 {
 	struct page *page;
 	int i;
@@ -388,6 +389,9 @@ static struct page *alloc_largest_available(unsigned long size,
 		if (max_order < orders[i])
 			continue;
 		flags = order_flags[i];
+		/* Decryption can change the contents, so clear it afterwards. */
+		if (defer_zero)
+			flags &= ~__GFP_ZERO;
 		if (mem_accounting)
 			flags |= __GFP_ACCOUNT;
 		page = alloc_pages(flags, orders[i]);
@@ -438,7 +442,8 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
 			goto free_buffer;
 		}
 
-		page = alloc_largest_available(size_remaining, max_order);
+		page = alloc_largest_available(size_remaining, max_order,
+					       cc_shared_buffer(buffer));
 		if (!page)
 			goto free_buffer;
 
@@ -461,9 +466,12 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
 
 	if (cc_shared_buffer(buffer)) {
 		for_each_sgtable_sg(table, sg, i) {
-			ret = system_heap_set_page_decrypted(sg_page(sg));
+			page = sg_page(sg);
+			ret = system_heap_set_page_decrypted(page);
 			if (ret)
 				goto free_pages;
+
+			clear_pages(page_address(page), 1 << compound_order(page));
 		}
 	}
 
Signature · 1 line
2.43.0
#9 lore raw

PATCH v2 4/4 dma-buf: heaps: Fix shared system heap allocation rollback

In reply to Steven Price

If converting one of the allocated pages to shared memory fails, the cleanup path attempts to convert every page back to private memory. Pages after the failed page have not been converted yet, so attempting to convert them back can fail and cause otherwise reusable memory to be leaked.

Count the pages converted successfully and only convert those and the failed allocation back during cleanup. If converting the failed allocation back succeeds it can be freed safely; otherwise it is leaked because its state is unknown. Allocations that were not converted can be freed directly.

Fixes 78b30c50a7ac ("dma-buf: heaps: system: add system_cc_shared heap for explicitly shared memory")
---
 drivers/dma-buf/heaps/system_heap.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
drivers/dma-buf/heaps/system_heap.c +7 -3
index f14930904089..8d3ffeb64e00 100644
@@ -418,6 +418,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
 	struct scatterlist *sg;
 	struct list_head pages;
 	struct page *page, *tmp_page;
+	int nr_decrypted = 0;
 	int i, ret = -ENOMEM;
 
 	buffer = kzalloc_obj(*buffer);
@@ -472,6 +473,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
 				goto free_pages;
 
 			clear_pages(page_address(page), 1 << compound_order(page));
+			nr_decrypted++;
 		}
 	}
 
@@ -496,9 +498,11 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
 		 * Intentionally leak pages that cannot be re-encrypted
 		 * to prevent shared memory from being reused.
 		 */
-		if (cc_shared_buffer(buffer) &&
-		    system_heap_set_page_encrypted(p))
-			continue;
+		if (cc_shared_buffer(buffer)) {
+			if (i <= nr_decrypted &&
+			    system_heap_set_page_encrypted(p))
+				continue;
+		}
 		__free_pages(p, compound_order(p));
 	}
 	sg_free_table(table);
Signature · 1 line
2.43.0
Reviewed-by on v2 4/4
#10 lore raw

Re: [PATCH v2 4/4] dma-buf: heaps: Fix shared system heap allocation rollback

Quoted 12 lines · Steven Price

If converting one of the allocated pages to shared memory fails, the cleanup path attempts to convert every page back to private memory. Pages after the failed page have not been converted yet, so attempting to convert them back can fail and cause otherwise reusable memory to be leaked.

Count the pages converted successfully and only convert those and the failed allocation back during cleanup. If converting the failed allocation back succeeds it can be freed safely; otherwise it is leaked because its state is unknown. Allocations that were not converted can be freed directly.

Quoted 5 lines
Fixes 78b30c50a7ac ("dma-buf: heaps: system: add system_cc_shared heap for explicitly shared memory")
---
 drivers/dma-buf/heaps/system_heap.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

Jason