New: When passthrough device to KVM guest with iommufd+hugepage, more hugepage are used than assigned, and DMAR error reported from host dmesg
https://bugzilla.kernel.org/show_bug.cgi?id=220776
Bug ID: 220776
Summary: When passthrough device to KVM guest with
iommufd+hugepage, more hugepage are used than assigned, and DMAR error reported from host dmesg
Product: Virtualization
Version: unspecified
Hardware: Intel
OS: Linux
Status: NEW
Severity: normal
Priority: P3
Component: kvm
Assignee: [email protected]
Reporter: [email protected]
Regression: No
Environment:
kernel: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
master dc77806cf3b
qemu: https://gitlab.com/qemu-project/qemu.git master 909772f0a5
Bug detail description:
When passthrough device to KVM guest with iommufd+hugepage, if 4G hugepage is assigned in qemu command, but 5G+ hugepage are used by qemu process. Meanwhile, error message reported from device driver in guest, and DMAR error reported from host dmesg.
I reproduced this issue on both Intel I210 NIC and Broadcom BCM57416 NIC:
Intel Corporation I210 Gigabit Network Connection (rev 03) Broadcom Inc. and subsidiaries BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller (rev 01)
Reproduce steps:
Create 8G 2M hugepage:
echo 4096 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages mount -t hugetlbfs hugetlbfs /dev/hugepages -o pagesize=2M
Bind I210 NIC 2a:00.0 to vfio-pci:
modprobe vfio-pci
echo 0000:3e:00.0 > /sys/bus/pci/devices/0000\:2a\:00.0/driver/unbind echo 8086 1533 > /sys/bus/pci/drivers/vfio-pci/new_id
Check host hugepage status:
[root@gnr-sp-2s-605 fchen]# cat /proc/meminfo | grep -i huge
AnonHugePages: 98304 kB ShmemHugePages: 0 kB FileHugePages: 0 kB HugePages_Total: 4096 HugePages_Free: 4096 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 8388608 kB
Boot Guest with I210 assigned(iommufd) and 4G hugepage:
/home/fchen/qemu/build/qemu-system-x86_64 \
-name legacy,debug-threads=on \
-machine q35 \
-m 4G -mem-path /dev/hugepages/ \
-accel kvm \
-smp 4 -cpu host \
-drive file=${img},if=none,id=virtio-disk0 \
-device virtio-blk-pci,drive=virtio-disk0 \
-monitor telnet:127.0.0.1:45454,nowait,server \
-vnc :1 \
-object iommufd,id=iommufd0 \
-device vfio-pci,host=3e:00.0,iommufd=iommufd0 \
-serial stdioError log:
Check host hugepage status after guest boot:
[root@gnr-sp-2s-605 ~]# cat /proc/meminfo | grep -i huge
AnonHugePages: 116736 kB ShmemHugePages: 0 kB FileHugePages: 0 kB HugePages_Total: 4096 HugePages_Free: 1408 HugePages_Rsvd: 1408 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 8388608 kB
Below guest and host dmesg error can only be captured when this NIC is connected with calble(IP available in host). If no cable connected in the NIC, no such error, only hugepage size error can be found.
guest dmesg:
[ 14.713154] igb 0000:00:04.0: Detected Tx Unit Hang
[ 14.713154] Tx Queue <1>
[ 14.713154] TDH <0>
[ 14.713154] TDT <1>
[ 14.713154] next_to_use <1>
[ 14.713154] next_to_clean <0>
[ 14.713154] buffer_info[next_to_clean]
[ 14.713154] time_stamp <fffb9477>
[ 14.713154] next_to_watch <0000000015eda9d3>
[ 14.713154] jiffies <fffba4aa>
[ 14.713154] desc.status <1b8000>
...
We can also see below error in some systems' host dmesg:
host dmesg:
[23560.951863] DMAR: DRHD: handling fault status reg 2 [23560.957541] DMAR: [DMA Write NO_PASID] Request device [3e:00.0] fault addr
0x7efef4f08000 [fault reason 0x79] SM: Read/Write permission error in second-level paging entry
Expected result:
1. No Error in guest and host dmesg.
2. If without iommufd, the free hugepages are 2048, which means QEMU process used 4G hugepage indeed. But now with iommufd, it used 2688(4096-1408) 2M(5.25G in total) hugepages.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 220776] When passthrough device to KVM guest with iommufd+hugepage, more hugepage are used than assigned, and DMAR error reported from host dmesg
https://bugzilla.kernel.org/show_bug.cgi?id=220776
[email protected] changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]
--- Comment #1 from [email protected] ---
After some bisect, it is found that this issue is introduced by the support of dma_map_file in iommufd [1][2]. With hugetlbfs as backend, it will go to the path of IOMMU_IOAS_MAP_FILE ioctl. If we fallback to the IOMMU_IOAS_MAP ioctl, there's no such problem.
[1]
https://lore.kernel.org/qemu-devel/[email protected]/
[2]
https://lore.kernel.org/all/[email protected]/
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
Re: [Bug 220776] When passthrough device to KVM guest with iommufd+hugepage, more hugepage are used than assigned, and DMAR error reported from host dmesg
Involve the author [email protected] of related patches to take a look.
Thanks,
Fan
Quoted 19 lines · On 11/11/2025 5:01 PM, [email protected] wrote:
https://bugzilla.kernel.org/show_bug.cgi?id=220776
[email protected] changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]--- Comment #1 from [email protected] ---
After some bisect, it is found that this issue is introduced by the support ofdma_map_filein iommufd [1][2]. With hugetlbfs as backend, it will go to the path ofIOMMU_IOAS_MAP_FILEioctl. If we fallback to theIOMMU_IOAS_MAPioctl, there's no such problem.[1]
https://lore.kernel.org/qemu-devel/[email protected]/
[2]
https://lore.kernel.org/all/[email protected]/
[Bug 220776] When passthrough device to KVM guest with iommufd+hugepage, more hugepage are used than assigned, and DMAR error reported from host dmesg
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=220776
--- Comment #2 from Chen, Fan ([email protected]) --- Involve the author [email protected] of related patches to take a look.
Thanks,
Fan
Quoted 23 lines · On 11/11/2025 5:01 PM, [email protected] wrote:
https://bugzilla.kernel.org/show_bug.cgi?id=220776
[email protected] changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]--- Comment #1 from [email protected] ---
After some bisect, it is found that this issue is introduced by the support ofdma_map_filein iommufd [1][2]. With hugetlbfs as backend, it will go to the path ofIOMMU_IOAS_MAP_FILEioctl. If we fallback to theIOMMU_IOAS_MAPioctl,
there's no such problem.[1]
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 220776] When passthrough device to KVM guest with iommufd+hugepage, more hugepage are used than assigned, and DMAR error reported from host dmesg
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=220776
Cédric Le Goater ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]
--- Comment #3 from Cédric Le Goater ([email protected]) --- Hello,
For QEMU related topics, the best is to open an "issue" on :
https://gitlab.com/qemu-project/qemu/-/work_items/
and add a label to it, "VFIO" in this case. Then send a patch. FYI, Steven has retired.
Thanks,
C.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 220776] When passthrough device to KVM guest with iommufd+hugepage, more hugepage are used than assigned, and DMAR error reported from host dmesg
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=220776
--- Comment #4 from [email protected] --- The fix patch is now merged.
https://lore.kernel.org/qemu-devel/[email protected]/
(commit id: e6c47bebdf8628e635e1ba970919ca96d572dbbe)
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 220776] When passthrough device to KVM guest with iommufd+hugepage, more hugepage are used than assigned, and DMAR error reported from host dmesg
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=220776
--- Comment #5 from Chen, Fan ([email protected]) --- Verified on the latest upstream QEMU 405c32d2b18a683ba36301351af75125d9afda08, test passed.
Thanks for the suggestion, I didn't know it is a QEMU issue when I open this ticket, I will do more analysis before file ticket here, thanks again.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 220776] When passthrough device to KVM guest with iommufd+hugepage, more hugepage are used than assigned, and DMAR error reported from host dmesg
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=220776
Chen, Fan ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |CODE_FIXSignature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.