eal: add option to force IOVA as PA mode
PATCH 1/1 eal: add option to force IOVA as PA mode
From: Chas Williams <[email protected]>
The IOMMU in some machines report that they can only support limited widths. IOVA virtual addresses may exceed this width making the use of IOVA virtual addresses difficult. The option CONFIG_RTE_LIBRTE_USE_PHYS_IOVA can used to force IOVA physical address usage.
--- config/common_base | 5 +++++ lib/librte_eal/bsdapp/eal/eal.c | 4 ++++ lib/librte_eal/linuxapp/eal/eal.c | 4 ++++ 3 files changed, 13 insertions(+)
config/common_base +5 -0
@@ -820,3 +820,8 @@ CONFIG_RTE_APP_CRYPTO_PERF=y # Compile the eventdev application # CONFIG_RTE_APP_EVENTDEV=y + +# +# Force IOVA physical addresses in VFIO +# +CONFIG_RTE_LIBRTE_USE_PHYS_IOVA=n
lib/librte_eal/bsdapp/eal/eal.c +4 -0
@@ -569,8 +569,12 @@ rte_eal_init(int argc, char **argv)return -1 ; } +#ifdef RTE_LIBRTE_USE_PHYS_IOVA +rte_eal_get_configuration ()->iova_mode =RTE_IOVA_PA ; +#else /* autodetect the iova mapping mode (default is iova_pa) */ rte_eal_get_configuration ()->iova_mode =rte_bus_get_iommu_class (); +#endif if (internal_config.no_hugetlbfs == 0 && internal_config.process_type != RTE_PROC_SECONDARY &&
lib/librte_eal/linuxapp/eal/eal.c +4 -0
@@ -804,8 +804,12 @@ rte_eal_init(int argc, char **argv)return -1 ; } +#ifdef RTE_LIBRTE_USE_PHYS_IOVA +rte_eal_get_configuration ()->iova_mode =RTE_IOVA_PA ; +#else /* autodetect the iova mapping mode (default is iova_pa) */ rte_eal_get_configuration ()->iova_mode =rte_bus_get_iommu_class (); +#endif /* Workaround for KNI which requires physical address to work */ if (rte_eal_get_configuration()->iova_mode == RTE_IOVA_VA &&
Signature · 1 line
2.9.5
30/11/2017 03:57, Chas Williams:
Quoted 7 lines
From: Chas Williams <[email protected]>
The IOMMU in some machines report that they can only support limited widths. IOVA virtual addresses may exceed this width making the use of IOVA virtual addresses difficult. The option
CONFIG_RTE_LIBRTE_USE_PHYS_IOVAcan used to force IOVA physical address usage.
Which machines are you talking about?
A run-time option may be a better solution.
The particular machine in my case is a Dell Optiplex 790 which is fairly similar to another system that has basically the same problem. The IOMMU advertises that is can only handle 39 bits of addressing. The DMAR address tables have a width of 36 bits, so all is well when using IOVA physical addresses. With IOVA virtual addresses, they sometimes go beyond the 39 bit boundary. I was under the impression that IOMMU widths are more typically 48 bits, but I appear to have some low end systems with a simpler IOMMU.
I could make it a runtime option.
Quoted 13 lines · Thomas Monjalon
30/11/2017 03:57, Chas Williams:
Quoted 7 lines
From: Chas Williams <[email protected]>
The IOMMU in some machines report that they can only support limited widths. IOVA virtual addresses may exceed this width making the use of IOVA virtual addresses difficult. The option
CONFIG_RTE_LIBRTE_USE_PHYS_IOVAcan used to force IOVA physical address usage.Which machines are you talking about?
A run-time option may be a better solution.
03/12/2017 01:23, Chas Williams:
Quoted 9 lines
The particular machine in my case is a Dell Optiplex 790 which is fairly similar to another system that has basically the same problem. The IOMMU advertises that is can only handle 39 bits of addressing. The DMAR address tables have a width of 36 bits, so all is well when using IOVA physical addresses. With IOVA virtual addresses, they sometimes go beyond the 39 bit boundary. I was under the impression that IOMMU widths are more typically 48 bits, but I appear to have some low end systems with a simpler IOMMU.
Quoted 16 lines
I could make it a runtime option.
Quoted 11 lines · Thomas Monjalon
30/11/2017 03:57, Chas Williams:
Quoted 7 lines
From: Chas Williams <[email protected]>
The IOMMU in some machines report that they can only support limited widths. IOVA virtual addresses may exceed this width making the use of IOVA virtual addresses difficult. The option
CONFIG_RTE_LIBRTE_USE_PHYS_IOVAcan used to force IOVA physical address usage.Which machines are you talking about?
A run-time option may be a better solution.
Please check the patch from Maxime:
https://dpdk.org/dev/patchwork/patch/33192/
Thanks, and yes I already saw that. That seems like a much better solution. I will give it a spin in my box with this problem.
Quoted 34 lines · Thomas Monjalon
03/12/2017 01:23, Chas Williams:
Quoted 3 lines
The particular machine in my case is a Dell Optiplex 790 which is fairly similar to another system that has basically the same problem. The IOMMU advertises that is can only handle 39 bits of addressing. The DMAR
address
Quoted 4 lines
tables have a width of 36 bits, so all is well when using IOVA physical addresses. With IOVA virtual addresses, they sometimes go beyond the 39 bit boundary. I was under the impression that IOMMU widths are more typically 48 bits, but I appear to have some low end systems with a
simpler
Quoted 18 lines
IOMMU.
I could make it a runtime option.
Quoted 11 lines · Thomas Monjalon
30/11/2017 03:57, Chas Williams:
Quoted 7 lines
From: Chas Williams <[email protected]>
The IOMMU in some machines report that they can only support limited widths. IOVA virtual addresses may exceed this width making the use of IOVA virtual addresses difficult. The option
CONFIG_RTE_LIBRTE_USE_PHYS_IOVAcan used to force IOVA physical address usage.Which machines are you talking about?
A run-time option may be a better solution.Please check the patch from Maxime:
https://dpdk.org/dev/patchwork/patch/33192/
Quoted 8 lines · Chas Williams
From: Chas Williams <[email protected]>
The IOMMU in some machines report that they can only support limited widths. IOVA virtual addresses may exceed this width making the use of IOVA virtual addresses difficult. The option
CONFIG_RTE_LIBRTE_USE_PHYS_IOVAcan used to force IOVA physical address usage.
Is it possible to dynamically detect the machine limitation in bus iova code instead of compile time option?
Quoted 11 lines · Hemant Agrawal
Quoted 9 lines · Chas Williams
From: Chas Williams <[email protected]>
The IOMMU in some machines report that they can only support limited widths. IOVA virtual addresses may exceed this width making the use of IOVA virtual addresses difficult. The option
CONFIG_RTE_LIBRTE_USE_PHYS_IOVAcan used to force IOVA physical address usage.
Quoted 4 lines
Is it possible to dynamically detect the machine limitation in bus iova code instead of compile time option?
Yes, I think you can examine the "cap" of the IOMMU group in /sys and determine the bit width. I will have to check on that.