Skip to contentSkip to conversation

Add encrypt/decrypt support for vmalloc regions

PATCH RESEND In review Shanker Donthineni 4 participants 12 messages Aug 11, 2025

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Add encrypt/decrypt support for vmalloc regions

This series fixes two related issues in the handling of memory encryption attributes on ARM64 systems with Confidential Compute Architecture (CCA) enabled.

Currently, certain code paths assume that the target address for set_memory_encrypted()/set_memory_decrypted() is always part of the linear mapping. This assumption breaks on ARM64 when CONFIG_DMA_DIRECT_REMAP=y or when operating on vmalloc-backed memory regions, leading to incorrect encryption attribute updates and, in some cases, system crashes.

Patch 1 addresses an ARM64-specific bug in the DMA atomic pool expansion path, where the wrong virtual address was passed to the memory encryption helpers. On CCA-enabled systems, this resulted in corruption or crashes during pool expansion.

Patch 2 extends the ARM64 memory attribute change functions to handle vmalloc and other non-linear mappings correctly. A new helper is introduced to detect such mappings, translate them to physical pages, and update attributes on a per-page basis. This resolves failures and crashes in scenarios such as NVMe device pass-through where vmalloc-backed memory is involved CONFIG_DMA_DIRECT_REMAP=y.

Both changes have been tested on an ARM64 CCA-enabled environment with NVMe pass-through feature.

Shanker Donthineni (2):
dma/pool: Use vmap() address for memory encryption helpers on ARM64
arm64: Add encrypt/decrypt support for vmalloc regions

 arch/arm64/mm/pageattr.c | 55 +++++++++++++++++++++++++++++++++++-----
 kernel/dma/pool.c        |  8 +++---
 2 files changed, 52 insertions(+), 11 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH RESEND 1/2 dma/pool: Use vmap() address for memory encryption helpers on ARM64

In atomic_pool_expand(), set_memory_encrypted()/set_memory_decrypted() are currently called with page_to_virt(page). On ARM64 with CONFIG_DMA_DIRECT_REMAP=y, the atomic pool is mapped via vmap(), so page_to_virt(page) does not reference the actual mapped region.

Using this incorrect address can cause encryption attribute updates to be applied to the wrong memory region. On ARM64 systems with memory encryption enabled (e.g. CCA), this can lead to data corruption or crashes.

Fix this by using the vmap() address ('addr') on ARM64 when invoking the memory encryption helpers, while retaining the existing page_to_virt(page) usage for other architectures.

Fixes 76a19940bd62 ("dma-direct: atomic allocations must come from atomic coherent pools")
---
 kernel/dma/pool.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/dma/pool.c +4 -4
index 7b04f7575796b..ba08a301590fd 100644
@@ -81,6 +81,7 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
 {
 	unsigned int order;
 	struct page *page = NULL;
+	void *vaddr;
 	void *addr;
 	int ret = -ENOMEM;
 
@@ -113,8 +114,8 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
 	 * Memory in the atomic DMA pools must be unencrypted, the pools do not
 	 * shrink so no re-encryption occurs in dma_direct_free().
 	 */
-	ret = set_memory_decrypted((unsigned long)page_to_virt(page),
-				   1 << order);
+	vaddr = IS_ENABLED(CONFIG_ARM64) ? addr : page_to_virt(page);
+	ret = set_memory_decrypted((unsigned long)vaddr, 1 << order);
 	if (ret)
 		goto remove_mapping;
 	ret = gen_pool_add_virt(pool, (unsigned long)addr, page_to_phys(page),
@@ -126,8 +127,7 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
 	return 0;
 
 encrypt_mapping:
-	ret = set_memory_encrypted((unsigned long)page_to_virt(page),
-				   1 << order);
+	ret = set_memory_encrypted((unsigned long)vaddr, 1 << order);
 	if (WARN_ON_ONCE(ret)) {
 		/* Decrypt succeeded but encrypt failed, purposely leak */
 		goto out;
Signature · 1 line
2.25.1
#2 lore raw

Re: [RESEND PATCH 1/2] dma/pool: Use vmap() address for memory encryption helpers on ARM64

Quoted 33 lines · Shanker Donthineni

In atomic_pool_expand(), set_memory_encrypted()/set_memory_decrypted() are currently called with page_to_virt(page). On ARM64 with CONFIG_DMA_DIRECT_REMAP=y, the atomic pool is mapped via vmap(), so page_to_virt(page) does not reference the actual mapped region.

Using this incorrect address can cause encryption attribute updates to be applied to the wrong memory region. On ARM64 systems with memory encryption enabled (e.g. CCA), this can lead to data corruption or crashes.

Fix this by using the vmap() address ('addr') on ARM64 when invoking the memory encryption helpers, while retaining the existing page_to_virt(page) usage for other architectures.

Fixes 76a19940bd62 ("dma-direct: atomic allocations must come from atomic coherent pools")
---
 kernel/dma/pool.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/dma/pool.c +1 -0
index 7b04f7575796b..ba08a301590fd 100644
@@ -81,6 +81,7 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
 {
 	unsigned int order;
 	struct page *page = NULL;
+	void *vaddr;
 	void *addr;
 	int ret = -ENOMEM;
 

kernel/dma/pool.c static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size, · line 117

Quoted 6 lines
kernel/dma/pool.c +1 -2
@@ -113,8 +114,8 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
 	 * Memory in the atomic DMA pools must be unencrypted, the pools do not
 	 * shrink so no re-encryption occurs in dma_direct_free().
 	 */
-	ret = set_memory_decrypted((unsigned long)page_to_virt(page),
-				   1 << order);
+	vaddr = IS_ENABLED(CONFIG_ARM64) ? addr : page_to_virt(page);

There's address calculation just before this code:

#ifdef CONFIG_DMA_DIRECT_REMAP
	addr = dma_common_contiguous_remap(page, pool_size,
					   pgprot_dmacoherent(PAGE_KERNEL),
					   __builtin_return_address(0));
	if (!addr)
		goto free_page;
#else
	addr = page_to_virt(page);

#endif

It should be enough to s/page_to_virt(page)/addr in the call to set_memory_decrypted().

Quoted 17 lines
kernel/dma/pool.c +2 -2
+	ret = set_memory_decrypted((unsigned long)vaddr, 1 << order);
 	if (ret)
 		goto remove_mapping;
 	ret = gen_pool_add_virt(pool, (unsigned long)addr, page_to_phys(page),
@@ -126,8 +127,7 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
 	return 0;
 
 encrypt_mapping:
-	ret = set_memory_encrypted((unsigned long)page_to_virt(page),
-				   1 << order);
+	ret = set_memory_encrypted((unsigned long)vaddr, 1 << order);
 	if (WARN_ON_ONCE(ret)) {
 		/* Decrypt succeeded but encrypt failed, purposely leak */
 		goto out;
Signature · 1 line
2.25.1
Signature · 2 lines
Sincerely yours,
Mike.
reviewer
#3 lore raw

Re: [RESEND PATCH 1/2] dma/pool: Use vmap() address for memory encryption helpers on ARM64

Quoted 51 lines · Mike Rapoport
Quoted 33 lines · Shanker Donthineni

In atomic_pool_expand(), set_memory_encrypted()/set_memory_decrypted() are currently called with page_to_virt(page). On ARM64 with CONFIG_DMA_DIRECT_REMAP=y, the atomic pool is mapped via vmap(), so page_to_virt(page) does not reference the actual mapped region.

Using this incorrect address can cause encryption attribute updates to be applied to the wrong memory region. On ARM64 systems with memory encryption enabled (e.g. CCA), this can lead to data corruption or crashes.

Fix this by using the vmap() address ('addr') on ARM64 when invoking the memory encryption helpers, while retaining the existing page_to_virt(page) usage for other architectures.

Fixes 76a19940bd62 ("dma-direct: atomic allocations must come from atomic coherent pools")
---
  kernel/dma/pool.c | 8 ++++----
  1 file changed, 4 insertions(+), 4 deletions(-)
kernel/dma/pool.c +1 -0
index 7b04f7575796b..ba08a301590fd 100644
@@ -81,6 +81,7 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
  {
  	unsigned int order;
  	struct page *page = NULL;
+	void *vaddr;
  	void *addr;
  	int ret = -ENOMEM;
 

kernel/dma/pool.c static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size, · line 117

Quoted 6 lines
kernel/dma/pool.c +1 -2
@@ -113,8 +114,8 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
  	 * Memory in the atomic DMA pools must be unencrypted, the pools do not
  	 * shrink so no re-encryption occurs in dma_direct_free().
  	 */
-	ret = set_memory_decrypted((unsigned long)page_to_virt(page),
-				   1 << order);
+	vaddr = IS_ENABLED(CONFIG_ARM64) ? addr : page_to_virt(page);

There's address calculation just before this code:

#ifdef CONFIG_DMA_DIRECT_REMAP
	addr = dma_common_contiguous_remap(page, pool_size,
					   pgprot_dmacoherent(PAGE_KERNEL),
					   __builtin_return_address(0));
	if (!addr)
		goto free_page;
#else
	addr = page_to_virt(page);
Quoted 4 lines

#endif

It should be enough to s/page_to_virt(page)/addr in the call to set_memory_decrypted().

Indeed, and either way this is clearly a DMA_DIRECT_REMAP concern rather than just an ARM64 one.

Thanks,
Robin.

Quoted 18 lines
Quoted 17 lines
kernel/dma/pool.c +2 -2
+	ret = set_memory_decrypted((unsigned long)vaddr, 1 << order);
  	if (ret)
  		goto remove_mapping;
  	ret = gen_pool_add_virt(pool, (unsigned long)addr, page_to_phys(page),
@@ -126,8 +127,7 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
  	return 0;
 
  encrypt_mapping:
-	ret = set_memory_encrypted((unsigned long)page_to_virt(page),
-				   1 << order);
+	ret = set_memory_encrypted((unsigned long)vaddr, 1 << order);
  	if (WARN_ON_ONCE(ret)) {
  		/* Decrypt succeeded but encrypt failed, purposely leak */
  		goto out;
Signature · 1 line
2.25.1
maintainer
#4 lore raw

Re: [RESEND PATCH 1/2] dma/pool: Use vmap() address for memory encryption helpers on ARM64

In reply to Shanker Donthineni
Quoted 33 lines · Shanker Donthineni

In atomic_pool_expand(), set_memory_encrypted()/set_memory_decrypted() are currently called with page_to_virt(page). On ARM64 with CONFIG_DMA_DIRECT_REMAP=y, the atomic pool is mapped via vmap(), so page_to_virt(page) does not reference the actual mapped region.

Using this incorrect address can cause encryption attribute updates to be applied to the wrong memory region. On ARM64 systems with memory encryption enabled (e.g. CCA), this can lead to data corruption or crashes.

Fix this by using the vmap() address ('addr') on ARM64 when invoking the memory encryption helpers, while retaining the existing page_to_virt(page) usage for other architectures.

Fixes 76a19940bd62 ("dma-direct: atomic allocations must come from atomic coherent pools")
---
 kernel/dma/pool.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/dma/pool.c +1 -0
index 7b04f7575796b..ba08a301590fd 100644
@@ -81,6 +81,7 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
 {
 	unsigned int order;
 	struct page *page = NULL;
+	void *vaddr;
 	void *addr;
 	int ret = -ENOMEM;
 

kernel/dma/pool.c static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size, · line 118

Quoted 7 lines
kernel/dma/pool.c +2 -2
@@ -113,8 +114,8 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
 	 * Memory in the atomic DMA pools must be unencrypted, the pools do not
 	 * shrink so no re-encryption occurs in dma_direct_free().
 	 */
-	ret = set_memory_decrypted((unsigned long)page_to_virt(page),
-				   1 << order);
+	vaddr = IS_ENABLED(CONFIG_ARM64) ? addr : page_to_virt(page);
+	ret = set_memory_decrypted((unsigned long)vaddr, 1 << order);

At least with arm CCA, there are two aspects to setting the memory encrypted/decrypted: an RMM (realm management monitor) call and setting of the attributes of the stage 1 mapping. The RMM call doesn't care about the virtual address, only the (intermediate) physical address, so having page_to_virt(page) here is fine.

The second part is setting the (fake) attribute for this mapping (top bit of the IPA space). Can we not instead just call:

	addr = dma_common_contiguous_remap(page, pool_size,
					   pgprot_decrypted(pgprot_dmacoherent(PAGE_KERNEL)),
					   __builtin_return_address(0));

in the atomic pool code? The advantage is that we keep the set_memory_decrypted() call on the linear map so that we change its attributes as well.

I want avoid walking the page tables for vmap regions if possible in the arm64 set_memory_* implementation. At some point I was proposing a GFP_DECRYPTED flag for allocations but never got around to post a patch (and implement vmalloc() support):

https://lore.kernel.org/linux-arm-kernel/[email protected]/

Signature · 1 line
Catalin
#5 lore raw

Re: [RESEND PATCH 1/2] dma/pool: Use vmap() address for memory encryption helpers on ARM64

Hi Catalin,

Quoted 51 lines · Catalin Marinas

External email: Use caution opening links or attachments

Quoted 33 lines · Shanker Donthineni

In atomic_pool_expand(), set_memory_encrypted()/set_memory_decrypted() are currently called with page_to_virt(page). On ARM64 with CONFIG_DMA_DIRECT_REMAP=y, the atomic pool is mapped via vmap(), so page_to_virt(page) does not reference the actual mapped region.

Using this incorrect address can cause encryption attribute updates to be applied to the wrong memory region. On ARM64 systems with memory encryption enabled (e.g. CCA), this can lead to data corruption or crashes.

Fix this by using the vmap() address ('addr') on ARM64 when invoking the memory encryption helpers, while retaining the existing page_to_virt(page) usage for other architectures.

Fixes 76a19940bd62 ("dma-direct: atomic allocations must come from atomic coherent pools")
---
  kernel/dma/pool.c | 8 ++++----
  1 file changed, 4 insertions(+), 4 deletions(-)
kernel/dma/pool.c +1 -0
index 7b04f7575796b..ba08a301590fd 100644
@@ -81,6 +81,7 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
  {
       unsigned int order;
       struct page *page = NULL;
+     void *vaddr;
       void *addr;
       int ret = -ENOMEM;
 

kernel/dma/pool.c static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size, · line 118

Quoted 7 lines
kernel/dma/pool.c +2 -2
@@ -113,8 +114,8 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
        * Memory in the atomic DMA pools must be unencrypted, the pools do not
        * shrink so no re-encryption occurs in dma_direct_free().
        */
-     ret = set_memory_decrypted((unsigned long)page_to_virt(page),
-                                1 << order);
+     vaddr = IS_ENABLED(CONFIG_ARM64) ? addr : page_to_virt(page);
+     ret = set_memory_decrypted((unsigned long)vaddr, 1 << order);

At least with arm CCA, there are two aspects to setting the memory encrypted/decrypted: an RMM (realm management monitor) call and setting of the attributes of the stage 1 mapping. The RMM call doesn't care about the virtual address, only the (intermediate) physical address, so having page_to_virt(page) here is fine.

Quoted 7 lines

The second part is setting the (fake) attribute for this mapping (top bit of the IPA space). Can we not instead just call:

addr = dma_common_contiguous_remap(page, pool_size,
pgprot_decrypted(pgprot_dmacoherent(PAGE_KERNEL)), __builtin_return_address(0));

Thanks for the simple fix, it resolves the crash issue. I’ve posted the v2 patch and dropped patch 2/2 which was added to support non-linear memory regions in pageattr.c.

Quoted 13 lines

in the atomic pool code? The advantage is that we keep the set_memory_decrypted() call on the linear map so that we change its attributes as well.

I want avoid walking the page tables for vmap regions if possible in the arm64 set_memory_* implementation. At some point I was proposing a GFP_DECRYPTED flag for allocations but never got around to post a patch (and implement vmalloc() support):

https://lore.kernel.org/linux-arm-kernel/[email protected]/

--
Catalin

#6 lore raw

PATCH RESEND 2/2 arm64: Add encrypt/decrypt support for vmalloc regions

In reply to Shanker Donthineni

On ARM64 systems with CCA (Confidential Compute Architecture) enabled, the kernel may need to change the encryption attributes of memory regions. The existing implementation of set_memory_encrypted() and set_memory_decrypted() assumes that the input address is part of the linear mapping region '__is_lm_address()', and fails with -EINVAL otherwise.

This breaks use cases where the memory region resides in the vmalloc area, which is mapped in non-linear mapping region.

This patch introduces a new helper, realm_set_memory(), which detects whether the given address is from a non-linear mapping. If so, it uses vmalloc_to_page() to resolve each page’s physical address and applies attribute changes one page at a time. For the linear address regions, it maintains the existing fast-path.

This change ensures that encrypted/decrypted memory attribute updates correctly for all memory regions, including those allocated via vmap(), module allocations, or other vmalloc-backed paths.

Call stack of Realm crash, QEMU hypervisor + NVME device (emulated):
...
Freeing unused kernel memory: 6336K
Run /sbin/init as init process

 Internal error: synchronous external abort: 0000000096000250 [#1]  SMP
 Modules linked in:
 CPU: 0 UID: 0 PID: 64 Comm: lsblk Not tainted 6.15.5 #2 PREEMPT(undef)
 Hardware name: linux,dummy-virt (DT)
 pstate: 43400005 (nZcv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--)
 pc : __pi_memset_generic+0x16c/0x188
 lr : dma_alloc_from_pool+0xd0/0x1b8
 sp : ffff80008335b350
 x29: ffff80008335b350 x28: ffff800083162000 x27: ffff80008335b3c0
 x26: ffff80008144f000 x25: ffff8000801a27e8 x24: ffff800081e14000
 x23: ffffc1ffc0000000 x22: 0000000000001000 x21: ffff800081458310
 x20: 0000000042a40000 x19: ffff00000232fcc0 x18: 0000000000200000
 x17: 00000000000120c0 x16: ffff0000795520c0 x15: 0000000000000000
 x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
 x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000
 x8 : ffff800083162000 x7 : 0000000000000000 x6 : 000000000000003f
 x5 : 0000000000000040 x4 : 0000000000000000 x3 : 0000000000000004
 x2 : 0000000000000fc0 x1 : 0000000000000000 x0 : ffff800083162000
 Call trace:
   __pi_memset_generic+0x16c/0x188 (P)
   dma_direct_alloc_from_pool+0xc4/0x230
   dma_direct_alloc+0x80/0x4a0
   dma_alloc_attrs+0x94/0x238
   dma_pool_alloc+0x128/0x258
   nvme_prep_rq.part.0+0x5f0/0x950
   nvme_queue_rq+0x78/0x1e8
   blk_mq_dispatch_rq_list+0x10c/0x6f0
   __blk_mq_sched_dispatch_requests+0x4a0/0x580
   blk_mq_sched_dispatch_requests+0x38/0xa0
   blk_mq_run_hw_queue+0x288/0x2f8
   blk_mq_flush_plug_list+0x134/0x630
   __blk_flush_plug+0x100/0x168
   blk_finish_plug+0x40/0x60
   read_pages+0x1a0/0x2b0
   page_cache_ra_unbounded+0x1f8/0x268
   force_page_cache_ra+0xa4/0xe0
   page_cache_sync_ra+0x48/0x268
   filemap_get_pages+0xf4/0x7a0
   filemap_read+0xf0/0x448
   blkdev_read_iter+0x8c/0x1a8
   vfs_read+0x288/0x330
   ksys_read+0x78/0x118
   __arm64_sys_read+0x24/0x40
   invoke_syscall+0x50/0x120
   el0_svc_common.constprop.0+0x48/0xf0
   do_el0_svc+0x24/0x38
   el0_svc+0x34/0xf8
   el0t_64_sync_handler+0x10c/0x138
   el0t_64_sync+0x1ac/0x1b0
---
 arch/arm64/mm/pageattr.c | 55 +++++++++++++++++++++++++++++++++++-----
 1 file changed, 48 insertions(+), 7 deletions(-)
arch/arm64/mm/pageattr.c +48 -7
index 04d4a8f676db4..65c3322a86b49 100644
@@ -202,21 +202,26 @@ int set_direct_map_default_noflush(struct page *page)
 				   PAGE_SIZE, change_page_range, &data);
 }
 
+/*
+ * Common function for setting memory encryption or decryption attributes.
+ *
+ * @addr:        Virtual start address of the memory region
+ * @start:       Corresponding physical start address
+ * @numpages:    Number of pages to update
+ * @encrypt:     If true, set memory as encrypted; if false, decrypt
+ */
 static int __set_memory_enc_dec(unsigned long addr,
+				phys_addr_t start,
 				int numpages,
 				bool encrypt)
 {
 	unsigned long set_prot = 0, clear_prot = 0;
-	phys_addr_t start, end;
+	phys_addr_t end;
 	int ret;
 
 	if (!is_realm_world())
 		return 0;
 
-	if (!__is_lm_address(addr))
-		return -EINVAL;
-
-	start = __virt_to_phys(addr);
 	end = start + numpages * PAGE_SIZE;
 
 	if (encrypt)
@@ -248,9 +253,45 @@ static int __set_memory_enc_dec(unsigned long addr,
 				      __pgprot(0));
 }
 
+/*
+ * Wrapper for __set_memory_enc_dec() that handles both linear-mapped
+ * and vmalloc/module memory regions.
+ *
+ * If the address is in the linear map, we can directly compute the
+ * physical address. If not (e.g. vmalloc memory), we walk each page
+ * and call the attribute update individually.
+ */
+static int realm_set_memory(unsigned long addr, int numpages, bool encrypt)
+{
+	phys_addr_t start;
+	struct page *page;
+	int ret, i;
+
+	if (__is_lm_address(addr)) {
+		start = __virt_to_phys(addr);
+		return __set_memory_enc_dec(addr, start, numpages, encrypt);
+	}
+
+	for (i = 0; i < numpages; i++) {
+		page = vmalloc_to_page((void *)addr);
+		if (!page)
+			return -EINVAL;
+
+		start = page_to_phys(page);
+		ret = __set_memory_enc_dec(addr, start, 1, encrypt);
+		if (ret)
+			return ret;
+
+		addr += PAGE_SIZE;
+	}
+
+	return 0;
+}
+
 static int realm_set_memory_encrypted(unsigned long addr, int numpages)
 {
-	int ret = __set_memory_enc_dec(addr, numpages, true);
+	int ret = realm_set_memory(addr, numpages, true);
+
 
 	/*
 	 * If the request to change state fails, then the only sensible cause
@@ -264,7 +305,7 @@ static int realm_set_memory_encrypted(unsigned long addr, int numpages)
 
 static int realm_set_memory_decrypted(unsigned long addr, int numpages)
 {
-	int ret = __set_memory_enc_dec(addr, numpages, false);
+	int ret = realm_set_memory(addr, numpages, false);
 
 	WARN(ret, "Failed to decrypt memory, %d pages will be leaked",
 	     numpages);
Signature · 1 line
2.25.1
#7 lore raw

Re: [RESEND PATCH 2/2] arm64: Add encrypt/decrypt support for vmalloc regions

Quoted 132 lines · Shanker Donthineni

On ARM64 systems with CCA (Confidential Compute Architecture) enabled, the kernel may need to change the encryption attributes of memory regions. The existing implementation of set_memory_encrypted() and set_memory_decrypted() assumes that the input address is part of the linear mapping region '__is_lm_address()', and fails with -EINVAL otherwise.

This breaks use cases where the memory region resides in the vmalloc area, which is mapped in non-linear mapping region.

This patch introduces a new helper, realm_set_memory(), which detects whether the given address is from a non-linear mapping. If so, it uses vmalloc_to_page() to resolve each page’s physical address and applies attribute changes one page at a time. For the linear address regions, it maintains the existing fast-path.

This change ensures that encrypted/decrypted memory attribute updates correctly for all memory regions, including those allocated via vmap(), module allocations, or other vmalloc-backed paths.

Call stack of Realm crash, QEMU hypervisor + NVME device (emulated):
...
Freeing unused kernel memory: 6336K
Run /sbin/init as init process

 Internal error: synchronous external abort: 0000000096000250 [#1]  SMP
 Modules linked in:
 CPU: 0 UID: 0 PID: 64 Comm: lsblk Not tainted 6.15.5 #2 PREEMPT(undef)
 Hardware name: linux,dummy-virt (DT)
 pstate: 43400005 (nZcv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--)
 pc : __pi_memset_generic+0x16c/0x188
 lr : dma_alloc_from_pool+0xd0/0x1b8
 sp : ffff80008335b350
 x29: ffff80008335b350 x28: ffff800083162000 x27: ffff80008335b3c0
 x26: ffff80008144f000 x25: ffff8000801a27e8 x24: ffff800081e14000
 x23: ffffc1ffc0000000 x22: 0000000000001000 x21: ffff800081458310
 x20: 0000000042a40000 x19: ffff00000232fcc0 x18: 0000000000200000
 x17: 00000000000120c0 x16: ffff0000795520c0 x15: 0000000000000000
 x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
 x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000
 x8 : ffff800083162000 x7 : 0000000000000000 x6 : 000000000000003f
 x5 : 0000000000000040 x4 : 0000000000000000 x3 : 0000000000000004
 x2 : 0000000000000fc0 x1 : 0000000000000000 x0 : ffff800083162000
 Call trace:
   __pi_memset_generic+0x16c/0x188 (P)
   dma_direct_alloc_from_pool+0xc4/0x230
   dma_direct_alloc+0x80/0x4a0
   dma_alloc_attrs+0x94/0x238
   dma_pool_alloc+0x128/0x258
   nvme_prep_rq.part.0+0x5f0/0x950
   nvme_queue_rq+0x78/0x1e8
   blk_mq_dispatch_rq_list+0x10c/0x6f0
   __blk_mq_sched_dispatch_requests+0x4a0/0x580
   blk_mq_sched_dispatch_requests+0x38/0xa0
   blk_mq_run_hw_queue+0x288/0x2f8
   blk_mq_flush_plug_list+0x134/0x630
   __blk_flush_plug+0x100/0x168
   blk_finish_plug+0x40/0x60
   read_pages+0x1a0/0x2b0
   page_cache_ra_unbounded+0x1f8/0x268
   force_page_cache_ra+0xa4/0xe0
   page_cache_sync_ra+0x48/0x268
   filemap_get_pages+0xf4/0x7a0
   filemap_read+0xf0/0x448
   blkdev_read_iter+0x8c/0x1a8
   vfs_read+0x288/0x330
   ksys_read+0x78/0x118
   __arm64_sys_read+0x24/0x40
   invoke_syscall+0x50/0x120
   el0_svc_common.constprop.0+0x48/0xf0
   do_el0_svc+0x24/0x38
   el0_svc+0x34/0xf8
   el0t_64_sync_handler+0x10c/0x138
   el0t_64_sync+0x1ac/0x1b0
---
 arch/arm64/mm/pageattr.c | 55 +++++++++++++++++++++++++++++++++++-----
 1 file changed, 48 insertions(+), 7 deletions(-)
arch/arm64/mm/pageattr.c +23 -5
index 04d4a8f676db4..65c3322a86b49 100644
@@ -202,21 +202,26 @@ int set_direct_map_default_noflush(struct page *page)
 				   PAGE_SIZE, change_page_range, &data);
 }
 
+/*
+ * Common function for setting memory encryption or decryption attributes.
+ *
+ * @addr:        Virtual start address of the memory region
+ * @start:       Corresponding physical start address
+ * @numpages:    Number of pages to update
+ * @encrypt:     If true, set memory as encrypted; if false, decrypt
+ */
 static int __set_memory_enc_dec(unsigned long addr,
+				phys_addr_t start,
 				int numpages,
 				bool encrypt)
 {
 	unsigned long set_prot = 0, clear_prot = 0;
-	phys_addr_t start, end;
+	phys_addr_t end;
 	int ret;
 
 	if (!is_realm_world())
 		return 0;
 
-	if (!__is_lm_address(addr))
-		return -EINVAL;
-
-	start = __virt_to_phys(addr);
 	end = start + numpages * PAGE_SIZE;
 
 	if (encrypt)
@@ -248,9 +253,45 @@ static int __set_memory_enc_dec(unsigned long addr,
 				      __pgprot(0));
 }
 
+/*
+ * Wrapper for __set_memory_enc_dec() that handles both linear-mapped
+ * and vmalloc/module memory regions.
+ *
+ * If the address is in the linear map, we can directly compute the
+ * physical address. If not (e.g. vmalloc memory), we walk each page
+ * and call the attribute update individually.
+ */
+static int realm_set_memory(unsigned long addr, int numpages, bool encrypt)
+{
+	phys_addr_t start;
+	struct page *page;
+	int ret, i;

arch/arm64/mm/pageattr.c static int __set_memory_enc_dec(unsigned long addr, · line 278

Quoted 10 lines
arch/arm64/mm/pageattr.c +10 -0
+
+	if (__is_lm_address(addr)) {
+		start = __virt_to_phys(addr);
+		return __set_memory_enc_dec(addr, start, numpages, encrypt);
+	}
+
+	for (i = 0; i < numpages; i++) {
+		page = vmalloc_to_page((void *)addr);
+		if (!page)
+			return -EINVAL;

It would be faster to get_vm_area(addr) and iterate over its ->pages rather than call vmalloc_to_page every time

Quoted 32 lines
arch/arm64/mm/pageattr.c +15 -2
+
+		start = page_to_phys(page);
+		ret = __set_memory_enc_dec(addr, start, 1, encrypt);
+		if (ret)
+			return ret;
+
+		addr += PAGE_SIZE;
+	}
+
+	return 0;
+}
+
 static int realm_set_memory_encrypted(unsigned long addr, int numpages)
 {
-	int ret = __set_memory_enc_dec(addr, numpages, true);
+	int ret = realm_set_memory(addr, numpages, true);
+
 
 	/*
 	 * If the request to change state fails, then the only sensible cause
@@ -264,7 +305,7 @@ static int realm_set_memory_encrypted(unsigned long addr, int numpages)
 
 static int realm_set_memory_decrypted(unsigned long addr, int numpages)
 {
-	int ret = __set_memory_enc_dec(addr, numpages, false);
+	int ret = realm_set_memory(addr, numpages, false);
 
 	WARN(ret, "Failed to decrypt memory, %d pages will be leaked",
 	     numpages);
Signature · 1 line
2.25.1
Signature · 2 lines
Sincerely yours,
Mike.
#8 lore raw

Re: [RESEND PATCH 2/2] arm64: Add encrypt/decrypt support for vmalloc regions

Hi Mike

Quoted 141 lines · Mike Rapoport

External email: Use caution opening links or attachments

Quoted 137 lines · Shanker Donthineni

On ARM64 systems with CCA (Confidential Compute Architecture) enabled, the kernel may need to change the encryption attributes of memory regions. The existing implementation of set_memory_encrypted() and set_memory_decrypted() assumes that the input address is part of the linear mapping region '__is_lm_address()', and fails with -EINVAL otherwise.

This breaks use cases where the memory region resides in the vmalloc area, which is mapped in non-linear mapping region.

This patch introduces a new helper, realm_set_memory(), which detects whether the given address is from a non-linear mapping. If so, it uses vmalloc_to_page() to resolve each page’s physical address and applies attribute changes one page at a time. For the linear address regions, it maintains the existing fast-path.

This change ensures that encrypted/decrypted memory attribute updates correctly for all memory regions, including those allocated via vmap(), module allocations, or other vmalloc-backed paths.

Call stack of Realm crash, QEMU hypervisor + NVME device (emulated):
...
Freeing unused kernel memory: 6336K
Run /sbin/init as init process

  Internal error: synchronous external abort: 0000000096000250 [#1]  SMP
  Modules linked in:
  CPU: 0 UID: 0 PID: 64 Comm: lsblk Not tainted 6.15.5 #2 PREEMPT(undef)
  Hardware name: linux,dummy-virt (DT)
  pstate: 43400005 (nZcv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--)
  pc : __pi_memset_generic+0x16c/0x188
  lr : dma_alloc_from_pool+0xd0/0x1b8
  sp : ffff80008335b350
  x29: ffff80008335b350 x28: ffff800083162000 x27: ffff80008335b3c0
  x26: ffff80008144f000 x25: ffff8000801a27e8 x24: ffff800081e14000
  x23: ffffc1ffc0000000 x22: 0000000000001000 x21: ffff800081458310
  x20: 0000000042a40000 x19: ffff00000232fcc0 x18: 0000000000200000
  x17: 00000000000120c0 x16: ffff0000795520c0 x15: 0000000000000000
  x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
  x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000
  x8 : ffff800083162000 x7 : 0000000000000000 x6 : 000000000000003f
  x5 : 0000000000000040 x4 : 0000000000000000 x3 : 0000000000000004
  x2 : 0000000000000fc0 x1 : 0000000000000000 x0 : ffff800083162000
  Call trace:
    __pi_memset_generic+0x16c/0x188 (P)
    dma_direct_alloc_from_pool+0xc4/0x230
    dma_direct_alloc+0x80/0x4a0
    dma_alloc_attrs+0x94/0x238
    dma_pool_alloc+0x128/0x258
    nvme_prep_rq.part.0+0x5f0/0x950
    nvme_queue_rq+0x78/0x1e8
    blk_mq_dispatch_rq_list+0x10c/0x6f0
    __blk_mq_sched_dispatch_requests+0x4a0/0x580
    blk_mq_sched_dispatch_requests+0x38/0xa0
    blk_mq_run_hw_queue+0x288/0x2f8
    blk_mq_flush_plug_list+0x134/0x630
    __blk_flush_plug+0x100/0x168
    blk_finish_plug+0x40/0x60
    read_pages+0x1a0/0x2b0
    page_cache_ra_unbounded+0x1f8/0x268
    force_page_cache_ra+0xa4/0xe0
    page_cache_sync_ra+0x48/0x268
    filemap_get_pages+0xf4/0x7a0
    filemap_read+0xf0/0x448
    blkdev_read_iter+0x8c/0x1a8
    vfs_read+0x288/0x330
    ksys_read+0x78/0x118
    __arm64_sys_read+0x24/0x40
    invoke_syscall+0x50/0x120
    el0_svc_common.constprop.0+0x48/0xf0
    do_el0_svc+0x24/0x38
    el0_svc+0x34/0xf8
    el0t_64_sync_handler+0x10c/0x138
    el0t_64_sync+0x1ac/0x1b0
---
  arch/arm64/mm/pageattr.c | 55 +++++++++++++++++++++++++++++++++++-----
  1 file changed, 48 insertions(+), 7 deletions(-)
arch/arm64/mm/pageattr.c +28 -5
index 04d4a8f676db4..65c3322a86b49 100644
@@ -202,21 +202,26 @@ int set_direct_map_default_noflush(struct page *page)
                                  PAGE_SIZE, change_page_range, &data);
  }
 
+/*
+ * Common function for setting memory encryption or decryption attributes.
+ *
+ * @addr:        Virtual start address of the memory region
+ * @start:       Corresponding physical start address
+ * @numpages:    Number of pages to update
+ * @encrypt:     If true, set memory as encrypted; if false, decrypt
+ */
  static int __set_memory_enc_dec(unsigned long addr,
+                             phys_addr_t start,
                               int numpages,
                               bool encrypt)
  {
       unsigned long set_prot = 0, clear_prot = 0;
-     phys_addr_t start, end;
+     phys_addr_t end;
       int ret;
 
       if (!is_realm_world())
               return 0;
 
-     if (!__is_lm_address(addr))
-             return -EINVAL;
-
-     start = __virt_to_phys(addr);
       end = start + numpages * PAGE_SIZE;
 
       if (encrypt)
@@ -248,9 +253,45 @@ static int __set_memory_enc_dec(unsigned long addr,
                                     __pgprot(0));
  }
 
+/*
+ * Wrapper for __set_memory_enc_dec() that handles both linear-mapped
+ * and vmalloc/module memory regions.
+ *
+ * If the address is in the linear map, we can directly compute the
+ * physical address. If not (e.g. vmalloc memory), we walk each page
+ * and call the attribute update individually.
+ */
+static int realm_set_memory(unsigned long addr, int numpages, bool encrypt)
+{
+     phys_addr_t start;
+     struct page *page;
+     int ret, i;
+
+     if (__is_lm_address(addr)) {
+             start = __virt_to_phys(addr);
+             return __set_memory_enc_dec(addr, start, numpages, encrypt);
+     }
Quoted 9 lines

arch/arm64/mm/pageattr.c static int __set_memory_enc_dec(unsigned long addr, · line 278

Quoted 5 lines
arch/arm64/mm/pageattr.c +5 -0
+
+     for (i = 0; i < numpages; i++) {
+             page = vmalloc_to_page((void *)addr);
+             if (!page)
+                     return -EINVAL;

It would be faster to get_vm_area(addr) and iterate over its ->pages rather than call vmalloc_to_page every time

Thanks Mike for suggestion, I'll do it in the next patch.

Quoted 36 lines
Quoted 30 lines
arch/arm64/mm/pageattr.c +15 -3
+
+             start = page_to_phys(page);
+             ret = __set_memory_enc_dec(addr, start, 1, encrypt);
+             if (ret)
+                     return ret;
+
+             addr += PAGE_SIZE;
+     }
+
+     return 0;
+}
+
  static int realm_set_memory_encrypted(unsigned long addr, int numpages)
  {
-     int ret = __set_memory_enc_dec(addr, numpages, true);
+     int ret = realm_set_memory(addr, numpages, true);
+
 
       /*
        * If the request to change state fails, then the only sensible cause
@@ -264,7 +305,7 @@ static int realm_set_memory_encrypted(unsigned long addr, int numpages)
 
  static int realm_set_memory_decrypted(unsigned long addr, int numpages)
  {
-     int ret = __set_memory_enc_dec(addr, numpages, false);
+     int ret = realm_set_memory(addr, numpages, false);
 
       WARN(ret, "Failed to decrypt memory, %d pages will be leaked",
            numpages);
--
Quoted 2 lines

2.25.1

--
Sincerely yours,
Mike.

reviewer
#9 lore raw

Re: [RESEND PATCH 2/2] arm64: Add encrypt/decrypt support for vmalloc regions

In reply to Shanker Donthineni
Quoted 37 lines · Shanker Donthineni

On ARM64 systems with CCA (Confidential Compute Architecture) enabled, the kernel may need to change the encryption attributes of memory regions. The existing implementation of set_memory_encrypted() and set_memory_decrypted() assumes that the input address is part of the linear mapping region '__is_lm_address()', and fails with -EINVAL otherwise.

This breaks use cases where the memory region resides in the vmalloc area, which is mapped in non-linear mapping region.

This patch introduces a new helper, realm_set_memory(), which detects whether the given address is from a non-linear mapping. If so, it uses vmalloc_to_page() to resolve each page’s physical address and applies attribute changes one page at a time. For the linear address regions, it maintains the existing fast-path.

This change ensures that encrypted/decrypted memory attribute updates correctly for all memory regions, including those allocated via vmap(), module allocations, or other vmalloc-backed paths.

Call stack of Realm crash, QEMU hypervisor + NVME device (emulated):
...
Freeing unused kernel memory: 6336K
Run /sbin/init as init process

  Internal error: synchronous external abort: 0000000096000250 [#1]  SMP
  Modules linked in:
  CPU: 0 UID: 0 PID: 64 Comm: lsblk Not tainted 6.15.5 #2 PREEMPT(undef)
  Hardware name: linux,dummy-virt (DT)
  pstate: 43400005 (nZcv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--)
  pc : __pi_memset_generic+0x16c/0x188
  lr : dma_alloc_from_pool+0xd0/0x1b8
  sp : ffff80008335b350
  x29: ffff80008335b350 x28: ffff800083162000 x27: ffff80008335b3c0
  x26: ffff80008144f000 x25: ffff8000801a27e8 x24: ffff800081e14000
  x23: ffffc1ffc0000000 x22: 0000000000001000 x21: ffff800081458310
  x20: 0000000042a40000 x19: ffff00000232fcc0 x18: 0000000000200000
  x17: 00000000000120c0 x16: ffff0000795520c0 x15: 0000000000000000
Quoted 8 lines
  x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
  x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000
  x8 : ffff800083162000 x7 : 0000000000000000 x6 : 000000000000003f
  x5 : 0000000000000040 x4 : 0000000000000000 x3 : 0000000000000004
  x2 : 0000000000000fc0 x1 : 0000000000000000 x0 : ffff800083162000
  Call trace:
    __pi_memset_generic+0x16c/0x188 (P)
    dma_direct_alloc_from_pool+0xc4/0x230

But isn't that exactly the case that patch #1 is supposed to have fixed? From a quick scan of set_memory_decrypted() callers I don't see anything obvious jumping out - can you clarify who you think needs this for reasons other than papering over bugs in the DMA layer?

Thanks,
Robin.

Quoted 126 lines
    dma_direct_alloc+0x80/0x4a0
    dma_alloc_attrs+0x94/0x238
    dma_pool_alloc+0x128/0x258
    nvme_prep_rq.part.0+0x5f0/0x950
    nvme_queue_rq+0x78/0x1e8
    blk_mq_dispatch_rq_list+0x10c/0x6f0
    __blk_mq_sched_dispatch_requests+0x4a0/0x580
    blk_mq_sched_dispatch_requests+0x38/0xa0
    blk_mq_run_hw_queue+0x288/0x2f8
    blk_mq_flush_plug_list+0x134/0x630
    __blk_flush_plug+0x100/0x168
    blk_finish_plug+0x40/0x60
    read_pages+0x1a0/0x2b0
    page_cache_ra_unbounded+0x1f8/0x268
    force_page_cache_ra+0xa4/0xe0
    page_cache_sync_ra+0x48/0x268
    filemap_get_pages+0xf4/0x7a0
    filemap_read+0xf0/0x448
    blkdev_read_iter+0x8c/0x1a8
    vfs_read+0x288/0x330
    ksys_read+0x78/0x118
    __arm64_sys_read+0x24/0x40
    invoke_syscall+0x50/0x120
    el0_svc_common.constprop.0+0x48/0xf0
    do_el0_svc+0x24/0x38
    el0_svc+0x34/0xf8
    el0t_64_sync_handler+0x10c/0x138
    el0t_64_sync+0x1ac/0x1b0
---
  arch/arm64/mm/pageattr.c | 55 +++++++++++++++++++++++++++++++++++-----
  1 file changed, 48 insertions(+), 7 deletions(-)
arch/arm64/mm/pageattr.c +48 -7
index 04d4a8f676db4..65c3322a86b49 100644
@@ -202,21 +202,26 @@ int set_direct_map_default_noflush(struct page *page)
  				   PAGE_SIZE, change_page_range, &data);
  }
 
+/*
+ * Common function for setting memory encryption or decryption attributes.
+ *
+ * @addr:        Virtual start address of the memory region
+ * @start:       Corresponding physical start address
+ * @numpages:    Number of pages to update
+ * @encrypt:     If true, set memory as encrypted; if false, decrypt
+ */
  static int __set_memory_enc_dec(unsigned long addr,
+				phys_addr_t start,
  				int numpages,
  				bool encrypt)
  {
  	unsigned long set_prot = 0, clear_prot = 0;
-	phys_addr_t start, end;
+	phys_addr_t end;
  	int ret;
 
  	if (!is_realm_world())
  		return 0;
 
-	if (!__is_lm_address(addr))
-		return -EINVAL;
-
-	start = __virt_to_phys(addr);
  	end = start + numpages * PAGE_SIZE;
 
  	if (encrypt)
@@ -248,9 +253,45 @@ static int __set_memory_enc_dec(unsigned long addr,
  				      __pgprot(0));
  }
 
+/*
+ * Wrapper for __set_memory_enc_dec() that handles both linear-mapped
+ * and vmalloc/module memory regions.
+ *
+ * If the address is in the linear map, we can directly compute the
+ * physical address. If not (e.g. vmalloc memory), we walk each page
+ * and call the attribute update individually.
+ */
+static int realm_set_memory(unsigned long addr, int numpages, bool encrypt)
+{
+	phys_addr_t start;
+	struct page *page;
+	int ret, i;
+
+	if (__is_lm_address(addr)) {
+		start = __virt_to_phys(addr);
+		return __set_memory_enc_dec(addr, start, numpages, encrypt);
+	}
+
+	for (i = 0; i < numpages; i++) {
+		page = vmalloc_to_page((void *)addr);
+		if (!page)
+			return -EINVAL;
+
+		start = page_to_phys(page);
+		ret = __set_memory_enc_dec(addr, start, 1, encrypt);
+		if (ret)
+			return ret;
+
+		addr += PAGE_SIZE;
+	}
+
+	return 0;
+}
+
  static int realm_set_memory_encrypted(unsigned long addr, int numpages)
  {
-	int ret = __set_memory_enc_dec(addr, numpages, true);
+	int ret = realm_set_memory(addr, numpages, true);
+
 
  	/*
  	 * If the request to change state fails, then the only sensible cause
@@ -264,7 +305,7 @@ static int realm_set_memory_encrypted(unsigned long addr, int numpages)
 
  static int realm_set_memory_decrypted(unsigned long addr, int numpages)
  {
-	int ret = __set_memory_enc_dec(addr, numpages, false);
+	int ret = realm_set_memory(addr, numpages, false);
 
  	WARN(ret, "Failed to decrypt memory, %d pages will be leaked",
  	     numpages);
#10 lore raw

Re: [RESEND PATCH 2/2] arm64: Add encrypt/decrypt support for vmalloc regions

Hi Robin,

Quoted 50 lines · Robin Murphy

External email: Use caution opening links or attachments

Quoted 42 lines · Shanker Donthineni

On ARM64 systems with CCA (Confidential Compute Architecture) enabled, the kernel may need to change the encryption attributes of memory regions. The existing implementation of set_memory_encrypted() and set_memory_decrypted() assumes that the input address is part of the linear mapping region '__is_lm_address()', and fails with -EINVAL otherwise.

This breaks use cases where the memory region resides in the vmalloc area, which is mapped in non-linear mapping region.

This patch introduces a new helper, realm_set_memory(), which detects whether the given address is from a non-linear mapping. If so, it uses vmalloc_to_page() to resolve each page’s physical address and applies attribute changes one page at a time. For the linear address regions, it maintains the existing fast-path.

This change ensures that encrypted/decrypted memory attribute updates correctly for all memory regions, including those allocated via vmap(), module allocations, or other vmalloc-backed paths.

Call stack of Realm crash, QEMU hypervisor + NVME device (emulated):
  ...
  Freeing unused kernel memory: 6336K
  Run /sbin/init as init process

  Internal error: synchronous external abort: 0000000096000250 [#1]  SMP
  Modules linked in:
  CPU: 0 UID: 0 PID: 64 Comm: lsblk Not tainted 6.15.5 #2 PREEMPT(undef)
  Hardware name: linux,dummy-virt (DT)
  pstate: 43400005 (nZcv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--)
  pc : __pi_memset_generic+0x16c/0x188
  lr : dma_alloc_from_pool+0xd0/0x1b8

  sp : ffff80008335b350
  x29: ffff80008335b350 x28: ffff800083162000 x27: ffff80008335b3c0
  x26: ffff80008144f000 x25: ffff8000801a27e8 x24: ffff800081e14000
  x23: ffffc1ffc0000000 x22: 0000000000001000 x21: ffff800081458310
  x20: 0000000042a40000 x19: ffff00000232fcc0 x18: 0000000000200000
  x17: 00000000000120c0 x16: ffff0000795520c0 x15: 0000000000000000
  x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
  x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000 x8 : ffff800083162000 x7 : 0000000000000000 x6 : 000000000000003f x5 : 0000000000000040 x4 : 0000000000000000 x3 : 0000000000000004 x2 : 0000000000000fc0 x1 : 0000000000000000 x0 : ffff800083162000

Quoted 3 lines
  Call trace:
    __pi_memset_generic+0x16c/0x188 (P)
    dma_direct_alloc_from_pool+0xc4/0x230
Quoted 5 lines

But isn't that exactly the case that patch #1 is supposed to have fixed?
From a quick scan of set_memory_decrypted() callers I don't see
anything obvious jumping out - can you clarify who you think needs this for reasons other than papering over bugs in the DMA layer?

Patch #1 fixes the passing of the correct mapped address (via vmalloc/vmap), prevent this specific crash. However, Realm boot still fails because __set_memory_enc_dec() returns -EINVAL when the requested address is not part of the linear mapping. Both patches are required to fully resolve the issue. Patch #2 is to support shared (decrypted) pages in vmalloced regions.

Thanks,
Shanker

reviewer
#11 lore raw

Re: [RESEND PATCH 2/2] arm64: Add encrypt/decrypt support for vmalloc regions

Quoted 59 lines · Shanker Donthineni

Hi Robin,

Quoted 50 lines · Robin Murphy

External email: Use caution opening links or attachments

Quoted 42 lines · Shanker Donthineni

On ARM64 systems with CCA (Confidential Compute Architecture) enabled, the kernel may need to change the encryption attributes of memory regions. The existing implementation of set_memory_encrypted() and set_memory_decrypted() assumes that the input address is part of the linear mapping region '__is_lm_address()', and fails with -EINVAL otherwise.

This breaks use cases where the memory region resides in the vmalloc area, which is mapped in non-linear mapping region.

This patch introduces a new helper, realm_set_memory(), which detects whether the given address is from a non-linear mapping. If so, it uses vmalloc_to_page() to resolve each page’s physical address and applies attribute changes one page at a time. For the linear address regions, it maintains the existing fast-path.

This change ensures that encrypted/decrypted memory attribute updates correctly for all memory regions, including those allocated via vmap(), module allocations, or other vmalloc-backed paths.

Call stack of Realm crash, QEMU hypervisor + NVME device (emulated):
  ...
  Freeing unused kernel memory: 6336K
  Run /sbin/init as init process

  Internal error: synchronous external abort: 0000000096000250 [#1]  SMP
  Modules linked in:
  CPU: 0 UID: 0 PID: 64 Comm: lsblk Not tainted 6.15.5 #2 PREEMPT(undef)
  Hardware name: linux,dummy-virt (DT)
  pstate: 43400005 (nZcv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--)
  pc : __pi_memset_generic+0x16c/0x188
  lr : dma_alloc_from_pool+0xd0/0x1b8

  sp : ffff80008335b350
  x29: ffff80008335b350 x28: ffff800083162000 x27: ffff80008335b3c0
  x26: ffff80008144f000 x25: ffff8000801a27e8 x24: ffff800081e14000
  x23: ffffc1ffc0000000 x22: 0000000000001000 x21: ffff800081458310
  x20: 0000000042a40000 x19: ffff00000232fcc0 x18: 0000000000200000
  x17: 00000000000120c0 x16: ffff0000795520c0 x15: 0000000000000000
  x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
  x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000 x8 : ffff800083162000 x7 : 0000000000000000 x6 : 000000000000003f x5 : 0000000000000040 x4 : 0000000000000000 x3 : 0000000000000004 x2 : 0000000000000fc0 x1 : 0000000000000000 x0 : ffff800083162000

Quoted 3 lines
  Call trace:
    __pi_memset_generic+0x16c/0x188 (P)
    dma_direct_alloc_from_pool+0xc4/0x230
Quoted 5 lines

But isn't that exactly the case that patch #1 is supposed to have fixed? From a quick scan of set_memory_decrypted() callers I don't see anything obvious jumping out - can you clarify who you think needs this for reasons other than papering over bugs in the DMA layer?

Quoted 5 lines

Patch #1 fixes the passing of the correct mapped address (via vmalloc/vmap), prevent this specific crash. However, Realm boot still fails because __set_memory_enc_dec() returns -EINVAL when the requested address is not part of the linear mapping. Both patches are required to fully resolve the issue. Patch #2 is to support shared (decrypted) pages in vmalloced regions.

Right, sorry for perhaps being unclear - the half-formed idea I was heading towards is that if patch #1 doesn't actually make DMA pools work then I'm not necessarily sure it's the right fix as-is.

In fact, looking at the code again, I think it probably shouldn't be relying on set_memory at all in the remap case, but instead using pgprot_decrypted(), same as the regular non-pool path in dma_direct_alloc().

Thanks,
Robin.