vhost: fix caching attributes of MMIO regions by setting them explicitly
Subsystems (5)
PATCH 1/1 vhost: fix caching attributes of MMIO regions by setting them explicitly
Explicitly set non-cached caching attributes for MMIO regions. Default write-back mode can cause CPU to cache device memory, causing invalid reads and unpredictable behavior.
Invalid read and write issues were observed on ARM64 when mapping the notification area to userspace via mmap.
---
Originally sent to net-next, now redirected to vhost tree per Jason Wang's suggestion.
drivers/vhost/vdpa.c | 1 + 1 file changed, 1 insertion(+)
drivers/vhost/vdpa.c +1 -0
@@ -1527,6 +1527,7 @@ static int vhost_vdpa_mmap(struct file *file, struct vm_area_struct *vma)if (vma->vm_end - vma->vm_start != notify.size)return -ENOTSUPP ; + vma->vm_page_prot =pgprot_noncached (vma->vm_page_prot);vm_flags_set (vma,VM_IO |VM_PFNMAP |VM_DONTEXPAND |VM_DONTDUMP ); vma->vm_ops = &vhost_vdpa_vm_ops;return 0 ;
Signature · 1 line
2.48.1
Hi Michael,
Just a ping on this patch. Would appreciate your review when you get a chance.
Thanks
Quoted 39 lines · Kommula Shiva Shankar
Prioritize security for external emails:
Confirm sender and content safety before clicking links or opening attachments Report Suspicious
Explicitly set non-cached caching attributes for MMIO regions. Default write-back mode can cause CPU to cache device memory, causing invalid reads and unpredictable behavior.Invalid read and write issues were observed on ARM64 when mapping the notification area to userspace via mmap.
Signed-off-by Kommula Shiva Shankar <[email protected]>Acked-by Jason Wang <[email protected]>---
Originally sent to net-next, now redirected to vhost tree per Jason Wang's suggestion.
drivers/vhost/vdpa.c| 1 +1 file changed, 1 insertion(+)drivers/vhost/vdpa.c +1
-0@@ -1527,6 +1527,7 @@ static int vhost_vdpa_mmap(struct file *file, struct vm_area_struct *vma)if (vma->vm_end - vma->vm_start != notify.size)return -ENOTSUPP ; + vma->vm_page_prot =pgprot_noncached (vma->vm_page_prot);vm_flags_set (vma,VM_IO |VM_PFNMAP |VM_DONTEXPAND |VM_DONTDUMP ); vma->vm_ops = &vhost_vdpa_vm_ops;return 0 ;Signature · 1 line
2.48.1
Michael S. Tsirkin
maintainerQuoted 6 lines · Kommula Shiva Shankar
Explicitly set non-cached caching attributes for MMIO regions. Default write-back mode can cause CPU to cache device memory, causing invalid reads and unpredictable behavior.
Invalid read and write issues were observed on ARM64 when mapping the notification area to userspace via mmap.
device memory in question is the VQ kick, yes? So if it is cached, the kick can get delayed, but how is this causing "invalid read and write issues"? What is read/written exactly?
Quoted 3 lines
Signed-off-by Kommula Shiva Shankar <[email protected]>Acked-by Jason Wang <[email protected]>
I also worry a bit about regressing on other hardware. Cc nvidia guys.
Quoted 21 lines
---
Originally sent to net-next, now redirected to vhost tree per Jason Wang's suggestion.drivers/vhost/vdpa.c | 1 + 1 file changed, 1 insertion(+)drivers/vhost/vdpa.c +1
-0@@ -1527,6 +1527,7 @@ static int vhost_vdpa_mmap(struct file *file, struct vm_area_struct *vma)if (vma->vm_end - vma->vm_start != notify.size)return -ENOTSUPP ; + vma->vm_page_prot =pgprot_noncached (vma->vm_page_prot);vm_flags_set (vma,VM_IO |VM_PFNMAP |VM_DONTEXPAND |VM_DONTDUMP ); vma->vm_ops = &vhost_vdpa_vm_ops;return 0 ;Signature · 1 line
2.48.1
________________________________________
From: Michael S. Tsirkin <[email protected]>
Sent: Tuesday, January 13, 2026 13:00
To: Shiva Shankar Kommula
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Jerin Jacob; Nithin Kumar Dabilpuram; Srujana Challa; [email protected]; [email protected]
Subject: [EXTERNAL] Re: [PATCH] vhost: fix caching attributes of MMIO regions by setting them explicitly
On Fri, Jan 02, 2026 at 12: 27: 03PM +0530, Kommula Shiva Shankar wrote: > Explicitly set non-cached caching attributes for MMIO regions. > Default write-back mode can cause CPU to cache device memory, > causing invalid reads and unpredictable ZjQcmQRYFpfptBannerStart
Prioritize security for external emails:
Confirm sender and content safety before clicking links or opening attachments
<https://us-phishalarm-ewt.proofpoint.com/EWT/v1/CRVmXkqW!tc3Z1f8UYnX61G-8-Z36D0TwOyXGKE_ybqkwtAsNGONThDzce1-BzQFJIDlDn_iWLJpe2FHKyeofP3u77neSt706ScpI5Ec$>
Report Suspicious
ZjQcmQRYFpfptBannerEnd
Quoted 11 lines · Kommula Shiva Shankar
Explicitly set non-cached caching attributes for MMIO regions. Default write-back mode can cause CPU to cache device memory, causing invalid reads and unpredictable behavior.
Invalid read and write issues were observed on ARM64 when mapping the notification area to userspace via mmap.
device memory in question is the VQ kick, yes? So if it is cached, the kick can get delayed, but how is this causing "invalid read and write issues"? What is read/written exactly?
This is the VQ notification address.
on ARM64, when memory mapped as write back, notification writes never reached the device. Reads on the device side sees stale values.
Quoted 29 lines
Signed-off-by Kommula Shiva Shankar <[email protected]>Acked-by Jason Wang <[email protected]>I also worry a bit about regressing on other hardware. Cc nvidia guys.
---
Originally sent to net-next, now redirected to vhost tree per Jason Wang's suggestion.drivers/vhost/vdpa.c | 1 + 1 file changed, 1 insertion(+)drivers/vhost/vdpa.c +1
-1@@ -1527,6 +1527,7 @@ static int vhost_vdpa_mmap(struct file *file, struct vm_area_struct *vma)if (vma->vm_end - vma->vm_start != notify.size)return -ENOTSUPP ; + vma->vm_page_prot =pgprot_noncached (vma->vm_page_prot);vm_flags_set (vma,VM_IO |VM_PFNMAP |VM_DONTEXPAND |VM_DONTDUMP ); vma->vm_ops = &vhost_vdpa_vm_ops;return 0 ; --2.48.1
Jason Gunthorpe
Reviewed-byQuoted 8 lines · Michael S. Tsirkin
Quoted 2 lines
Signed-off-by Kommula Shiva Shankar <[email protected]>Acked-by Jason Wang <[email protected]>I also worry a bit about regressing on other hardware. Cc nvidia guys.
drivers/vhost/vdpa.c static int vhost_vdpa_mmap(struct file *file, struct vm_area_struct *vma) · line 1531
Quoted 2 lines
drivers/vhost/vdpa.c +1
-0+ vma->vm_page_prot =pgprot_noncached (vma->vm_page_prot);vm_flags_set (vma,VM_IO |VM_PFNMAP |VM_DONTEXPAND |VM_DONTDUMP );
This is definitely required and correct if notify.addr comes from a PCI BAR address.
You need to trace the origin of that memory in all the drivers to determine if it is OK or not.
For instance mlx5 is:
kick_addr = mdev->bar_addr + offset; res->phys_kick_addr = kick_addr; [..] addr = (phys_addr_t)ndev->mvdev.res.phys_kick_addr;
"bar_addr" is PCI memory so this patch is correct and required for mlx5.
ifcvf:
hw->notify_base_pa = pci_resource_start(pdev, cap.bar) +
le32_to_cpu(cap.offset);
[..]
hw->vring[i].notify_pa = hw->notify_base_pa +
notify_off * hw->notify_off_multiplier;
[..]
area.addr = vf->vring[idx].notify_pa;octep:
oct_hw->notify_base_pa = pci_resource_start(pdev, cap.bar) +
le32_to_cpu(cap.offset);
[..]
oct_hw->vqs[i].notify_pa = oct_hw->notify_base_pa +
notify_off * oct_hw->notify_off_multiplier;
[..]
area.addr = oct_hw->vqs[idx].notify_pa;pds:
No idea, it is messed up though:
area.addr = pdsv->vqs[qid].notify_pa;
struct pds_vdpa_vq_info {
dma_addr_t notify_pa; Can't cast dma_addr_t to phys_addr_t!
virtio_pci:
Also messed up:
notify.addr = vp_vdpa->vring[qid].notify_pa;
struct vp_vring {
resource_size_t notify_pa; phys_addr is not a resource_size_t
Guessing pds and virtio_pci are also both fine, even if I gave up trying to figure out where notify_pa gets set from in the end.
So the patch is OK
Jason