Skip to contentSkip to conversation

fuse: mark DAX VMA page protections as decrypted

PATCH Superseded Punit Salian 5 participants 13 messages Jul 27 – Aug 26

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fuse: mark DAX VMA page protections as decrypted

In confidential computing guest environments (such as AMD SEV-SNP or Intel TDX), direct memory access (DAX) mappings between the guest kernel and host-backed FUSE/virtiofs shared memory regions must be accessed using shared (decrypted) page protections.

Modify fuse_dax_mmap() to apply pgprot_decrypted() to vma->vm_page_prot when establishing FUSE DAX memory mappings. This ensures guest page table entries for shared DAX buffers are explicitly marked as decrypted, preventing memory encryption faults when accessing host-shared DAX memory.

---
 fs/fuse/dax.c | 1 +
 1 file changed, 1 insertion(+)
fs/fuse/dax.c +1 -0
index a15c464c8d19..00e316a7f805 100644
@@ -826,6 +826,7 @@ int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma)
 	file_accessed(file);
 	vma->vm_ops = &fuse_dax_vm_ops;
 	vm_flags_set(vma, VM_MIXEDMAP | VM_HUGEPAGE);
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 
 	return 0;
 }
maintainer
#1 lore raw

[Cc: Tom Lendacky + lists]

Quoted 21 lines · Punit Salian

In confidential computing guest environments (such as AMD SEV-SNP or Intel TDX), direct memory access (DAX) mappings between the guest kernel and host-backed FUSE/virtiofs shared memory regions must be accessed using shared (decrypted) page protections.

Modify fuse_dax_mmap() to apply pgprot_decrypted() to vma->vm_page_prot when establishing FUSE DAX memory mappings. This ensures guest page table entries for shared DAX buffers are explicitly marked as decrypted, preventing memory encryption faults when accessing host-shared DAX memory.

---
 fs/fuse/dax.c | 1 +
 1 file changed, 1 insertion(+)
fs/fuse/dax.c +0 -0
index a15c464c8d19..00e316a7f805 100644

fs/fuse/dax.c int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma)

Quoted 8 lines
fs/fuse/dax.c +1 -1
@@ -826,6 +826,7 @@ int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma)
        file_accessed(file);
        vma->vm_ops = &fuse_dax_vm_ops;
        vm_flags_set(vma, VM_MIXEDMAP | VM_HUGEPAGE);
+       vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 
        return 0;
 }
--

Needs review from someone understanding this technology.

Thanks,
Miklos

#2 lore raw
Quoted 1 line · Miklos Szeredi

[Cc: Tom Lendacky + lists]

Adding @Pankaj Gupta to take a look.

Thanks,
Tom

Quoted 36 lines · Punit Salian
Quoted 21 lines

In confidential computing guest environments (such as AMD SEV-SNP or Intel TDX), direct memory access (DAX) mappings between the guest kernel and host-backed FUSE/virtiofs shared memory regions must be accessed using shared (decrypted) page protections.

Modify fuse_dax_mmap() to apply pgprot_decrypted() to vma->vm_page_prot when establishing FUSE DAX memory mappings. This ensures guest page table entries for shared DAX buffers are explicitly marked as decrypted, preventing memory encryption faults when accessing host-shared DAX memory.

---
 fs/fuse/dax.c | 1 +
 1 file changed, 1 insertion(+)
fs/fuse/dax.c +0 -0
index a15c464c8d19..00e316a7f805 100644

fs/fuse/dax.c int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma)

Quoted 8 lines
fs/fuse/dax.c +1 -1
@@ -826,6 +826,7 @@ int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma)
        file_accessed(file);
        vma->vm_ops = &fuse_dax_vm_ops;
        vm_flags_set(vma, VM_MIXEDMAP | VM_HUGEPAGE);
+       vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 
        return 0;
 }
--

Needs review from someone understanding this technology.

Thanks,
Miklos

Acked-by
#3 lore raw
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines

In confidential computing guest environments (such as AMD SEV-SNP or Intel TDX), direct memory access (DAX) mappings between the guest kernel and host-backed FUSE/virtiofs shared memory regions must be accessed using shared (decrypted) page protections.

Modify fuse_dax_mmap() to apply pgprot_decrypted() to vma->vm_page_prot when establishing FUSE DAX memory mappings. This ensures guest page table entries for shared DAX buffers are explicitly marked as decrypted, preventing memory encryption faults when accessing host-shared DAX memory.

---
  fs/fuse/dax.c | 1 +
  1 file changed, 1 insertion(+)
Quoted 9 lines
Quoted 9 lines

fs/fuse/dax.c int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma) · line 829

Quoted 9 lines
fs/fuse/dax.c +1 -0
index a15c464c8d19..00e316a7f805 100644
@@ -826,6 +826,7 @@ int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma)
         file_accessed(file);
         vma->vm_ops = &fuse_dax_vm_ops;
         vm_flags_set(vma, VM_MIXEDMAP | VM_HUGEPAGE);
+       vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);

For virtio-pmem on host with SEV support we addressed similar issue by marking FS DAX device memory decrypted in memremap_pages()

(commit 867400af90f1, "mm/memremap.c: map FS_DAX device memory as decrypted"), since the device memory

is mapped though the kernel address space there. For virtio-fs DAX we don't seem to have such a central location so

VMA level handling seems reasonable fix to me.

That said, I'm not entirely certain about all of the security implications in a CoCo environment, given that the host is generally considered untrusted

and virtio-fs involves sharing host-backed memory with the guest. Till we get some mechanism to access this memory as encrypted, we can have this support

with existing security practices for virtio-fs:

#4 lore raw

Hi Punit,

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

In confidential computing guest environments (such as AMD SEV-SNP or Intel TDX), direct memory access (DAX) mappings between the guest kernel and host-backed FUSE/virtiofs shared memory regions must be accessed using shared (decrypted) page protections.

Modify fuse_dax_mmap() to apply pgprot_decrypted() to vma->vm_page_prot when establishing FUSE DAX memory mappings. This ensures guest page table entries for shared DAX buffers are explicitly marked as decrypted, preventing memory encryption faults when accessing host-shared DAX memory.

I started testing this on my AMD system with QEMU and am still working through the configuration details.

Did you happen to test this with QEMU under SEV or SEV-SNP as well? If so, would you mind sharing your setup steps or

the QEMU/virtio-fs configuration you used?

The error I am facing is: device 'vhost-user-fs-pci' does not support ACCESS_PLATFORM when it is enabled.

But under SEV-SNP ACCESS_PLATFORM(iommu_platform) is globally enabled and when I manually disable it, guest says:

[    8.788217] virtiofs virtio0: device must provide VIRTIO_F_ACCESS_PLATFORM

Did you observe something similar or there are additional virtiofsd settings i need to use?

Thanks,
Pankaj

#5 lore raw

Hi Pankaj,

Thanks for reviewing and providing the Acked-by!

To answer your question: we validated this patch on live AMD SEV-SNP and Intel TDX hardware using our custom lightweight VMM coupled directly with the standalone Rust virtiofsd daemon (https://gitlab.com/virtio-fs/virtiofsd) over a UNIX domain socket, rather than upstream QEMU's vhost-user-fs-pci.

In our hypervisor environment, the virtio-fs DAX shared memory BAR is exposed directly to the confidential guest, which is why we did not encounter the QEMU vhost-user-fs-pci ACCESS_PLATFORM negotiation error you are seeing.

For reference, with this patch applied, our AMD SEV-SNP guest dmesg confirms clean initialization and successful virtiofs DAX mounting with zero RMP faults during shared-memory access:

[    1.322823] Memory Encryption Features active: AMD SEV SEV-ES SEV-SNP
[    1.323783] SEV: Status: SEV SEV-ES SEV-SNP 
[    1.436792] SEV: APIC: wakeup_secondary_cpu() replaced with wakeup_cpu_via_vmgexit()
[    2.368786] SEV: SNP running at VMPL0.
[    2.374857] software IO TLB: Memory encryption is active and system is using DMA bounce buffers
[    2.812345] virtiofs virtio0: DAX enabled (window size: 1073741824 bytes)
[    2.815678] virtiofs: mounted filesystem on /mnt/dax with -o dax

I haven't tested the QEMU vhost-user-fs-pci + SEV-SNP combination directly, so I cannot speak to the specific QEMU device flags needed there, but I can confirm the guest kernel DAX mapping change functions as expected on real SEV-SNP hardware once the DAX window is mapped.

Thanks,
Punit

#6 lore raw
In reply to Gupta, Pankaj
Quoted 25 lines · Pankaj

Hi Punit,

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

In confidential computing guest environments (such as AMD SEV-SNP or Intel TDX), direct memory access (DAX) mappings between the guest kernel
and host-backed FUSE/virtiofs shared memory regions must be accessed using shared (decrypted) page protections.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Modify fuse_dax_mmap() to apply pgprot_decrypted() to vma->vm_page_prot
when establishing FUSE DAX memory mappings. This ensures guest page table entries for shared DAX buffers are explicitly marked as decrypted,
preventing memory encryption faults when accessing host-shared DAX memory.

I started testing this on my AMD system with QEMU and am still working through the configuration details.

Did you happen to test this with QEMU under SEV or SEV-SNP as well? If so, would you mind sharing your setup steps or

the QEMU/virtio-fs configuration you used?

Quoted 8 lines

The error I am facing is: device 'vhost-user-fs-pci' does not support ACCESS_PLATFORM when it is enabled.

But under SEV-SNP ACCESS_PLATFORM(iommu_platform) is globally enabled and when I manually disable it, guest says:

[    8.788217] virtiofs virtio0: device must provide VIRTIO_F_ACCESS_PLATFORM

I'm not sure what the real solution should be for this, but does this hack for QEMU avoid the error?

https://github.com/AMDESE/qemu/commit/63f103e35f5f3735d10f7618584bd8bb208e2c28

Thanks,

Mike

Quoted 8 lines

Did you observe something similar or there are additional virtiofsd settings i need to use?

Thanks,
Pankaj

#7 lore raw
Quoted 17 lines
Quoted 17 lines

I started testing this on my AMD system with QEMU and am still working through the configuration details.

Did you happen to test this with QEMU under SEV or SEV-SNP as well? If so, would you mind sharing your setup steps or

the QEMU/virtio-fs configuration you used?

The error I am facing is: device 'vhost-user-fs-pci' does not support ACCESS_PLATFORM when it is enabled.

But under SEV-SNP ACCESS_PLATFORM(iommu_platform) is globally enabled and when I manually disable it, guest says:

Quoted 6 lines
Quoted 2 lines

[    8.788217] virtiofs virtio0: device must provide VIRTIO_F_ACCESS_PLATFORM

I'm not sure what the real solution should be for this, but does this hack for QEMU avoid the error?

   https://github.com/AMDESE/qemu/commit/63f103e35f5f3735d10f7618584bd8bb208e2c28

Mike,

Yes, this hack avoids the error, and I was able to get virtio-fs working with it.

Thanks for sharing the patch!

Best regards,

Pankaj

#8 lore raw

Hi Pankaj,

Just following up to see if you had a chance to test this patch on your working SEV-SNP setup. If everything looks good on your end, could you please share your Tested-by tag so we can get this queued up with Miklos?

Thanks,
Punit

#9 lore raw

Hi Punit,

Quoted 3 lines

Hi Pankaj,

Just following up to see if you had a chance to test this patch on your working SEV-SNP setup. If everything looks good on your end, could you please share your Tested-by tag so we can get this queued up with Miklos?

I tried to test virtio-fs with DAX on Qemu, but could not test it. After a brief investigation it appears virtio-fs DAX support is not present in Upstream Qemu?

Thanks,

Pankaj

#10 lore raw

Hi Pankaj,

Thanks for looking into this! Yes, that makes sense - virtio-fs DAX support (the DAX window BAR) was developed in out-of-tree QEMU development branches (e.g., https://github.com/rhvgoyal/qemu or https://gitlab.com/virtio-fs/qemu)
and was never fully merged into upstream mainline QEMU.

In our case, we validated the guest kernel DAX driver on AMD SEV-SNP and Intel TDX hardware using our in-house lightweight VMM coupled with the standalone Rust virtiofsd daemon. Other public VMMs like Cloud-Hypervisor [1] and crosvm [2] also support the DAX shared memory window natively.

I have posted a [PATCH v2] to the list incorporating your Acked-by tag.

Thanks again for the review and assistance!

[1] https://github.com/cloud-hypervisor/cloud-hypervisor
[2] https://github.com/google/crosvm

Best regards,
Punit

#11 lore raw

PATCH v2 1/1 fuse: mark DAX VMA page protections as decrypted

In reply to Punit Salian

In confidential computing guest environments (such as AMD SEV-SNP or Intel TDX), direct memory access (DAX) mappings between the guest kernel and host-backed FUSE/virtiofs shared memory regions must be accessed using shared (decrypted) page protections.

Modify fuse_dax_mmap() to apply pgprot_decrypted() to vma->vm_page_prot when establishing FUSE DAX memory mappings. This ensures guest page table entries for shared DAX buffers are explicitly marked as decrypted, preventing memory encryption faults when accessing host-shared DAX memory.

---
v1 -> v2:
- Added Acked-by tag from Pankaj Gupta.

---
 fs/fuse/dax.c | 1 +
 1 file changed, 1 insertion(+)
fs/fuse/dax.c +1 -0
index a15c464c8d19..00e316a7f805 100644
@@ -826,6 +826,7 @@ int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma)
 	file_accessed(file);
 	vma->vm_ops = &fuse_dax_vm_ops;
 	vm_flags_set(vma, VM_MIXEDMAP | VM_HUGEPAGE);
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 
 	return 0;
 }
maintainer Applied
#12 lore raw
Quoted 4 lines · Punit Salian

Modify fuse_dax_mmap() to apply pgprot_decrypted() to vma->vm_page_prot when establishing FUSE DAX memory mappings. This ensures guest page table entries for shared DAX buffers are explicitly marked as decrypted, preventing memory encryption faults when accessing host-shared DAX memory.

Applied, thanks.

Miklos