vfio: retry creating sPAPR DMA window
PATCH 1/1 vfio: retry creating sPAPR DMA window
sPAPR allows only page_shift from VFIO_IOMMU_SPAPR_TCE_GET_INFO ioctl. However, Linux 4.17 or before returns incorrect page_shift for Power9. I added the code for retrying creation of sPAPR DMA window.
--- lib/librte_eal/linux/eal/eal_vfio.c | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)
lib/librte_eal/linux/eal/eal_vfio.c +23 -3
@@ -1448,9 +1448,29 @@ vfio_spapr_create_new_dma_window(int vfio_container_fd,/* create new DMA window */ ret =ioctl (vfio_container_fd,VFIO_IOMMU_SPAPR_TCE_CREATE , create);if (ret) { -RTE_LOG (ERR ,EAL ," cannot create new DMA window, " -"error %i (%s)\n" , errno,strerror (errno)); -return -1 ; +/* try possible page_shift and levels for workaround */ +uint32_t levels; + +for (levels =1 ; levels <= info.ddw.levels; levels++) { +uint32_t pgsizes = info.ddw.pgsizes; + +while (pgsizes !=0 ) { + create->page_shift =31 -__builtin_clz (pgsizes); + create->levels = levels; + ret =ioctl (vfio_container_fd, +VFIO_IOMMU_SPAPR_TCE_CREATE , create); +if (!ret) +break ; + pgsizes &= ~(1 << create->page_shift); + } +if (!ret) +break ; + } +if (ret) { +RTE_LOG (ERR ,EAL ," cannot create new DMA window, " +"error %i (%s)\n" , errno,strerror (errno)); +return -1 ; + } }if (create->start_addr !=0 ) {
Signature · 1 line
2.17.1
Please, any review or ack for this patch?
07/06/2019 04:28, Takeshi Yoshimura:
Quoted 8 lines
sPAPR allows only
page_shiftfromVFIO_IOMMU_SPAPR_TCE_GET_INFOioctl. However, Linux 4.17 or before returns incorrectpage_shiftfor Power9. I added the code for retrying creation of sPAPR DMA window.Signed-off-by Takeshi Yoshimura <[email protected]>--- lib/librte_eal/linux/eal/eal_vfio.c | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)
Quoted 12 lines
Please, any review or ack for this patch?
07/06/2019 04:28, Takeshi Yoshimura:
Quoted 8 lines
sPAPR allows only
page_shiftfromVFIO_IOMMU_SPAPR_TCE_GET_INFOioctl. However, Linux 4.17 or before returns incorrectpage_shiftfor Power9. I added the code for retrying creation of sPAPR DMA window.Signed-off-by Takeshi Yoshimura <[email protected]>--- lib/librte_eal/linux/eal/eal_vfio.c | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)
I've shared my comments with Takeshi and I don't think the patch is ready in it's current state. It appears to modify x86 functionality and I'm not comfortable with signing off on that since it's in an anrea I'm not familiar with.
Dave
08/07/2019 18:47, David Christensen:
Quoted 7 lines
Quoted 7 lines
Please, any review or ack for this patch?
07/06/2019 04:28, Takeshi Yoshimura:
Quoted 4 lines
sPAPR allows only
page_shiftfromVFIO_IOMMU_SPAPR_TCE_GET_INFOioctl. However, Linux 4.17 or before returns incorrectpage_shiftfor Power9. I added the code for retrying creation of sPAPR DMA window.
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Signed-off-by Takeshi Yoshimura <[email protected]>I've shared my comments with Takeshi and I don't think the patch is ready in it's current state. It appears to modify x86 functionality and I'm not comfortable with signing off on that since it's in an anrea I'm not familiar with.
It is a function specific to sPAPR. So there is no risk for x86 or Arm. As you didn't share your comments publicly, this patch has been merged in 19.08-rc1.
Quoted 15 lines · Thomas Monjalon
08/07/2019 18:47, David Christensen:
Quoted 7 lines
Quoted 7 lines
Please, any review or ack for this patch?
07/06/2019 04:28, Takeshi Yoshimura:
Quoted 4 lines
sPAPR allows only
page_shiftfromVFIO_IOMMU_SPAPR_TCE_GET_INFOioctl. However, Linux 4.17 or before returns incorrectpage_shiftfor Power9. I added the code for retrying creation of sPAPR DMA window.Quoted 6 lines
Quoted 1 line
Quoted 1 line
Signed-off-by Takeshi Yoshimura <[email protected]>I've shared my comments with Takeshi and I don't think the patch is ready in it's current state. It appears to modify x86 functionality and I'm not comfortable with signing off on that since it's in an anrea I'm not familiar with.
Quoted 5 lines
It is a function specific to sPAPR. So there is no risk for x86 or Arm. As you didn't share your comments publicly, this patch has been merged in 19.08-rc1.
I have a suspicion that David has confused this patch with another one, which did indeed touch a lot of code (but still didn't touch x86 AFAIR).
Signature · 2 lines
Thanks, Anatoly
Quoted 6 lines · Takeshi Yoshimura
sPAPR allows only
page_shiftfromVFIO_IOMMU_SPAPR_TCE_GET_INFOioctl. However, Linux 4.17 or before returns incorrectpage_shiftfor Power9. I added the code for retrying creation of sPAPR DMA window.Signed-off-by Takeshi Yoshimura <[email protected]>---
This doesn't affect any code outside of sPAPR and looks sane, so
Signature · 2 lines
Thanks, Anatoly
Thomas Monjalon
Applied05/07/2019 10:15, Burakov, Anatoly:
Quoted 11 lines · Takeshi Yoshimura
Quoted 6 lines
sPAPR allows only
page_shiftfromVFIO_IOMMU_SPAPR_TCE_GET_INFOioctl. However, Linux 4.17 or before returns incorrectpage_shiftfor Power9. I added the code for retrying creation of sPAPR DMA window.Signed-off-by Takeshi Yoshimura <[email protected]>---
This doesn't affect any code outside of sPAPR and looks sane, so
Acked-by Anatoly Burakov <[email protected]>
Applied, thanks
Quoted 15 lines · Takeshi Yoshimura
sPAPR allows only
page_shiftfromVFIO_IOMMU_SPAPR_TCE_GET_INFOioctl. However, Linux 4.17 or before returns incorrectpage_shiftfor Power9. I added the code for retrying creation of sPAPR DMA window.Signed-off-by Takeshi Yoshimura <[email protected]>--- lib/librte_eal/linux/eal/eal_vfio.c | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)lib/librte_eal/linux/eal/eal_vfio.c +0
-0@@ -1448,9 +1448,29 @@ vfio_spapr_create_new_dma_window(int vfio_container_fd,/* create new DMA window */
lib/librte_eal/linux/eal/eal_vfio.c vfio_spapr_create_new_dma_window(int vfio_container_fd, · line 1455
Quoted 10 lines
lib/librte_eal/linux/eal/eal_vfio.c +5
-3ret =ioctl (vfio_container_fd,VFIO_IOMMU_SPAPR_TCE_CREATE , create);if (ret) { -RTE_LOG (ERR ,EAL ," cannot create new DMA window, " -"error %i (%s)\n" , errno,strerror (errno)); -return -1 ; +/* try possible page_shift and levels for workaround */ +uint32_t levels; + +for (levels =1 ; levels <= info.ddw.levels; levels++) { +uint32_t pgsizes = info.ddw.pgsizes;
+CC POWER maintainer from MAINTAINERS file.
This is failing compilation on some older distros (Fedora 20, Ubuntu 14.04):
/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c: In function ‘vfio_spapr_create_new_dma_window’:/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c:1451:34: error: ‘struct vfio_iommu_spapr_tce_info’ has no member named ‘ddw’ for (levels = 1; levels <= info.ddw.levels; levels++) {
^/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c:1452:27: error: ‘struct vfio_iommu_spapr_tce_info’ has no member named ‘ddw’ uint32_t pgsizes = info.ddw.pgsizes;
^/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c: At top level:
cc1: error: unrecognized command line option "-Wno-address-of-packed-member" [-Werror]
cc1: all warnings being treated as errors
make[7]: *** [eal_vfio.o] Error 1
make[7]: *** Waiting for unfinished jobs....
make[6]: *** [eal] Error 2
make[5]: *** [linux] Error 2
make[4]: *** [librte_eal] Error 2
make[3]: *** [lib] Error 2
make[2]: *** [all] Error 2
make[1]: *** [pre_install] Error 2
make: *** [install] Error 2
Looking at the eal_vfio.h we are handling case of missing sPAPR info/ddw structures, but this isn't simply a case of a missing struct; rather it is an issue of differing definitions between what recent kernels have, and what older kernels had.
@Thomas Do we still support these older distros?
Signature · 2 lines
Thanks, Anatoly
10/07/2019 12:32, Burakov, Anatoly:
Quoted 57 lines · Takeshi Yoshimura
Quoted 15 lines
sPAPR allows only
page_shiftfromVFIO_IOMMU_SPAPR_TCE_GET_INFOioctl. However, Linux 4.17 or before returns incorrectpage_shiftfor Power9. I added the code for retrying creation of sPAPR DMA window.Signed-off-by Takeshi Yoshimura <[email protected]>--- lib/librte_eal/linux/eal/eal_vfio.c | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)lib/librte_eal/linux/eal/eal_vfio.c +0
-0@@ -1448,9 +1448,29 @@ vfio_spapr_create_new_dma_window(int vfio_container_fd,/* create new DMA window */ lib/librte_eal/linux/eal/eal_vfio.c vfio_spapr_create_new_dma_window(int vfio_container_fd, · line 1455
Quoted 10 lines
lib/librte_eal/linux/eal/eal_vfio.c +5
-3ret =ioctl (vfio_container_fd,VFIO_IOMMU_SPAPR_TCE_CREATE , create);if (ret) { -RTE_LOG (ERR ,EAL ," cannot create new DMA window, " -"error %i (%s)\n" , errno,strerror (errno)); -return -1 ; +/* try possible page_shift and levels for workaround */ +uint32_t levels; + +for (levels =1 ; levels <= info.ddw.levels; levels++) { +uint32_t pgsizes = info.ddw.pgsizes;+CC POWER maintainer from
MAINTAINERSfile.This is failing compilation on some older distros (Fedora 20, Ubuntu 14.04):
/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c: In function ‘vfio_spapr_create_new_dma_window’:/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c:1451:34: error: ‘struct vfio_iommu_spapr_tce_info’ has no member named ‘ddw’
for (levels = 1; levels <= info.ddw.levels; levels++) {
^/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c:1452:27: error: ‘struct vfio_iommu_spapr_tce_info’ has no member named ‘ddw’uint32_tpgsizes = info.ddw.pgsizes;
^/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c: At top level:
cc1: error: unrecognized command line option "-Wno-address-of-packed-member" [-Werror]
cc1: all warnings being treated as errors
make[7]: *** [eal_vfio.o] Error 1
make[7]: *** Waiting for unfinished jobs....
make[6]: *** [eal] Error 2
make[5]: *** [linux] Error 2
make[4]: *** [librte_eal] Error 2
make[3]: *** [lib] Error 2
make[2]: *** [all] Error 2
make[1]: *** [pre_install] Error 2
make: *** [install] Error 2
Quoted 6 lines
Looking at the
eal_vfio.hwe are handling case of missing sPAPR info/ddw structures, but this isn't simply a case of a missing struct; rather it is an issue of differing definitions between what recent kernels have, and what older kernels had.@Thomas Do we still support these older distros?
If the kernel is not maintained, we don't support it. For most distros, Linux < 3.16 is not maintained.
Quoted 63 lines · Thomas Monjalon
10/07/2019 12:32, Burakov, Anatoly:
Quoted 62 lines · Takeshi Yoshimura
Quoted 15 lines
sPAPR allows only
page_shiftfromVFIO_IOMMU_SPAPR_TCE_GET_INFOioctl. However, Linux 4.17 or before returns incorrectpage_shiftfor Power9. I added the code for retrying creation of sPAPR DMA window.Signed-off-by Takeshi Yoshimura <[email protected]>--- lib/librte_eal/linux/eal/eal_vfio.c | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)lib/librte_eal/linux/eal/eal_vfio.c +0
-0@@ -1448,9 +1448,29 @@ vfio_spapr_create_new_dma_window(int vfio_container_fd,/* create new DMA window */ lib/librte_eal/linux/eal/eal_vfio.c vfio_spapr_create_new_dma_window(int vfio_container_fd, · line 1455
Quoted 10 lines
lib/librte_eal/linux/eal/eal_vfio.c +5
-3ret =ioctl (vfio_container_fd,VFIO_IOMMU_SPAPR_TCE_CREATE , create);if (ret) { -RTE_LOG (ERR ,EAL ," cannot create new DMA window, " -"error %i (%s)\n" , errno,strerror (errno)); -return -1 ; +/* try possible page_shift and levels for workaround */ +uint32_t levels; + +for (levels =1 ; levels <= info.ddw.levels; levels++) { +uint32_t pgsizes = info.ddw.pgsizes;+CC POWER maintainer from
MAINTAINERSfile.This is failing compilation on some older distros (Fedora 20, Ubuntu 14.04):
/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c: In function ‘vfio_spapr_create_new_dma_window’:/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c:1451:34: error: ‘struct vfio_iommu_spapr_tce_info’ has no member named ‘ddw’
for (levels = 1; levels <= info.ddw.levels; levels++) {
^/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c:1452:27: error: ‘struct vfio_iommu_spapr_tce_info’ has no member named ‘ddw’uint32_tpgsizes = info.ddw.pgsizes;
^/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c: At top level:
cc1: error: unrecognized command line option "-Wno-address-of-packed-member" [-Werror]
cc1: all warnings being treated as errors
make[7]: *** [eal_vfio.o] Error 1
make[7]: *** Waiting for unfinished jobs....
make[6]: *** [eal] Error 2
make[5]: *** [linux] Error 2
make[4]: *** [librte_eal] Error 2
make[3]: *** [lib] Error 2
make[2]: *** [all] Error 2
make[1]: *** [pre_install] Error 2
make: *** [install] Error 2Looking at the
eal_vfio.hwe are handling case of missing sPAPR info/ddw structures, but this isn't simply a case of a missing struct; rather it is an issue of differing definitions between what recent kernels have, and what older kernels had.
Quoted 5 lines
Quoted 1 line
@Thomas Do we still support these older distros?
If the kernel is not maintained, we don't support it. For most distros, Linux < 3.16 is not maintained.
This is on kernel 3.13 (Ubuntu 14.04 LTS which according to Canonical's published schedule [1] has stopped receiving even maintenance updates, and is only updated for select customers). It was never an LTS kernel to begin with, i think.
[1] https://ubuntu.com/about/release-cycle
Signature · 2 lines
Thanks, Anatoly
10/07/2019 14:35, Burakov, Anatoly:
Quoted 66 lines · Thomas Monjalon
Quoted 65 lines
10/07/2019 12:32, Burakov, Anatoly:
Quoted 63 lines · Takeshi Yoshimura
Quoted 15 lines
sPAPR allows only
page_shiftfromVFIO_IOMMU_SPAPR_TCE_GET_INFOioctl. However, Linux 4.17 or before returns incorrectpage_shiftfor Power9. I added the code for retrying creation of sPAPR DMA window.Signed-off-by Takeshi Yoshimura <[email protected]>--- lib/librte_eal/linux/eal/eal_vfio.c | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)lib/librte_eal/linux/eal/eal_vfio.c +0
-0@@ -1448,9 +1448,29 @@ vfio_spapr_create_new_dma_window(int vfio_container_fd,/* create new DMA window */ lib/librte_eal/linux/eal/eal_vfio.c vfio_spapr_create_new_dma_window(int vfio_container_fd, · line 1455
Quoted 10 lines
lib/librte_eal/linux/eal/eal_vfio.c +5
-3ret =ioctl (vfio_container_fd,VFIO_IOMMU_SPAPR_TCE_CREATE , create);if (ret) { -RTE_LOG (ERR ,EAL ," cannot create new DMA window, " -"error %i (%s)\n" , errno,strerror (errno)); -return -1 ; +/* try possible page_shift and levels for workaround */ +uint32_t levels; + +for (levels =1 ; levels <= info.ddw.levels; levels++) { +uint32_t pgsizes = info.ddw.pgsizes;+CC POWER maintainer from
MAINTAINERSfile.This is failing compilation on some older distros (Fedora 20, Ubuntu 14.04):
/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c: In function ‘vfio_spapr_create_new_dma_window’:/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c:1451:34: error: ‘struct vfio_iommu_spapr_tce_info’ has no member named ‘ddw’
for (levels = 1; levels <= info.ddw.levels; levels++) {
^/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c:1452:27: error: ‘struct vfio_iommu_spapr_tce_info’ has no member named ‘ddw’uint32_tpgsizes = info.ddw.pgsizes;
^/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c: At top level:
cc1: error: unrecognized command line option "-Wno-address-of-packed-member" [-Werror]
cc1: all warnings being treated as errors
make[7]: *** [eal_vfio.o] Error 1
make[7]: *** Waiting for unfinished jobs....
make[6]: *** [eal] Error 2
make[5]: *** [linux] Error 2
make[4]: *** [librte_eal] Error 2
make[3]: *** [lib] Error 2
make[2]: *** [all] Error 2
make[1]: *** [pre_install] Error 2
make: *** [install] Error 2Looking at the
eal_vfio.hwe are handling case of missing sPAPR info/ddw structures, but this isn't simply a case of a missing struct; rather it is an issue of differing definitions between what recent kernels have, and what older kernels had.@Thomas Do we still support these older distros?
Quoted 7 lines
Quoted 2 lines
If the kernel is not maintained, we don't support it. For most distros, Linux < 3.16 is not maintained.
This is on kernel 3.13 (Ubuntu 14.04 LTS which according to Canonical's published schedule [1] has stopped receiving even maintenance updates, and is only updated for select customers). It was never an LTS kernel to begin with, i think.
So we can ignore this issue.
Quoted 70 lines · Thomas Monjalon
10/07/2019 14:35, Burakov, Anatoly:
Quoted 69 lines · Thomas Monjalon
Quoted 67 lines
10/07/2019 12:32, Burakov, Anatoly:
Quoted 57 lines · Takeshi Yoshimura
Quoted 15 lines
sPAPR allows only
page_shiftfromVFIO_IOMMU_SPAPR_TCE_GET_INFOioctl. However, Linux 4.17 or before returns incorrectpage_shiftfor Power9. I added the code for retrying creation of sPAPR DMA window.Signed-off-by Takeshi Yoshimura <[email protected]>--- lib/librte_eal/linux/eal/eal_vfio.c | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)lib/librte_eal/linux/eal/eal_vfio.c +0
-0@@ -1448,9 +1448,29 @@ vfio_spapr_create_new_dma_window(int vfio_container_fd,/* create new DMA window */ lib/librte_eal/linux/eal/eal_vfio.c vfio_spapr_create_new_dma_window(int vfio_container_fd, · line 1455
Quoted 10 lines
lib/librte_eal/linux/eal/eal_vfio.c +5
-3ret =ioctl (vfio_container_fd,VFIO_IOMMU_SPAPR_TCE_CREATE , create);if (ret) { -RTE_LOG (ERR ,EAL ," cannot create new DMA window, " -"error %i (%s)\n" , errno,strerror (errno)); -return -1 ; +/* try possible page_shift and levels for workaround */ +uint32_t levels; + +for (levels =1 ; levels <= info.ddw.levels; levels++) { +uint32_t pgsizes = info.ddw.pgsizes;+CC POWER maintainer from
MAINTAINERSfile.This is failing compilation on some older distros (Fedora 20, Ubuntu 14.04):
/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c: In function ‘vfio_spapr_create_new_dma_window’:/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c:1451:34: error: ‘struct vfio_iommu_spapr_tce_info’ has no member named ‘ddw’
for (levels = 1; levels <= info.ddw.levels; levels++) {
^/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c:1452:27: error: ‘struct vfio_iommu_spapr_tce_info’ has no member named ‘ddw’uint32_tpgsizes = info.ddw.pgsizes;
^/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c: At top level:
cc1: error: unrecognized command line option "-Wno-address-of-packed-member" [-Werror]
cc1: all warnings being treated as errors
make[7]: *** [eal_vfio.o] Error 1
make[7]: *** Waiting for unfinished jobs....
make[6]: *** [eal] Error 2
make[5]: *** [linux] Error 2
make[4]: *** [librte_eal] Error 2
make[3]: *** [lib] Error 2
make[2]: *** [all] Error 2
make[1]: *** [pre_install] Error 2
make: *** [install] Error 2Quoted 6 lines
Looking at the
eal_vfio.hwe are handling case of missing sPAPR info/ddw structures, but this isn't simply a case of a missing struct; rather it is an issue of differing definitions between what recent kernels have, and what older kernels had.@Thomas Do we still support these older distros?
If the kernel is not maintained, we don't support it. For most distros, Linux < 3.16 is not maintained.
Quoted 7 lines
Quoted 4 lines
This is on kernel 3.13 (Ubuntu 14.04 LTS which according to Canonical's published schedule [1] has stopped receiving even maintenance updates, and is only updated for select customers). It was never an LTS kernel to begin with, i think.
So we can ignore this issue.
It seems that it's also reproducible on Ubuntu 14.04 with kernel 3.19. Which is not an upstream LTS kernel, but it's >3.16.
It also looks like it may be an issue for us specifically, so i would suggest reverting the patch for rc2 if the submitter/maintainers don't come back and fix it.
Signature · 2 lines
Thanks, Anatoly
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
Quoted 1 line
@Thomas Do we still support these older distros?
If the kernel is not maintained, we don't support it. For most distros, Linux < 3.16 is not maintained.
This is on kernel 3.13 (Ubuntu 14.04 LTS which according to Canonical's published schedule [1] has stopped receiving even maintenance updates, and is only updated for select customers). It was never an LTS kernel to begin with, i think.
Quoted 6 lines
Quoted 2 lines
So we can ignore this issue.
It seems that it's also reproducible on Ubuntu 14.04 with kernel 3.19. Which is not an upstream LTS kernel, but it's >3.16.
It also looks like it may be an issue for us specifically, so i would suggest reverting the patch for rc2 if the submitter/maintainers don't come back and fix it.
Just to confirm, these failures are on x86 builds, is that correct? Power 9 systems aren't supported by these older distros, Linux kernel 4.6 was the first kernel with Power 9 support.
Dave
-----"Burakov, Anatoly" <[email protected]> wrote: -----
Quoted 106 lines
To: Thomas Monjalon <[email protected]>
From: "Burakov, Anatoly" <[email protected]>
Date: 07/10/2019 09:01AM
Cc: Takeshi Yoshimura <[email protected]>, [email protected], David Christensen <[email protected]>
Subject: [EXTERNAL] Re: [dpdk-dev] [PATCH] vfio: retry creating sPAPR DMA windowQuoted 5 lines · Thomas Monjalon
10/07/2019 14:35, Burakov, Anatoly:
Quoted 4 lines · Thomas Monjalon
Quoted 3 lines
10/07/2019 12:32, Burakov, Anatoly:
Quoted 2 lines · Takeshi Yoshimura
Quoted 1 line
sPAPR allows only
page_shiftfromVFIO_IOMMU_SPAPR_TCE_GET_INFOioctl.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
However, Linux 4.17 or before returns incorrect
page_shiftforPower9.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
I added the code for retrying creation of sPAPR DMA window.
Signed-off-by Takeshi Yoshimura <[email protected]>--- lib/librte_eal/linux/eal/eal_vfio.c | 26+++++++++++++++++++++++---
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Review
Quoted 3 lines
1 file changed, 23 insertions(+), 3 deletions(-)(quoted diff) +0
-0
b/lib/librte_eal/linux/eal/eal_vfio.cQuoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
lib/librte_eal/linux/eal/eal_vfio.c vfio_spapr_create_new_dma_window(int
Quoted 4 lines
index
6892a2c14..f16c5c3c0100644lib/librte_eal/linux/eal/eal_vfio.c +0
-0@@ -1448,9 +1448,29 @@ vfio_spapr_create_new_dma_window(int
vfio_container_fd,Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
lib/librte_eal/linux/eal/eal_vfio.c vfio_spapr_create_new_dma_window(int · line 1449
Quoted 2 lines
lib/librte_eal/linux/eal/eal_vfio.c +0
-0/* create new DMA window */ ret = ioctl(vfio_container_fd,
VFIO_IOMMU_SPAPR_TCE_CREATE, create);Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
lib/librte_eal/linux/eal/eal_vfio.c vfio_spapr_create_new_dma_window(int vfio_container_fd, · line 1455
Quoted 9 lines
lib/librte_eal/linux/eal/eal_vfio.c +5
-3if (ret) { -RTE_LOG (ERR ,EAL ," cannot create new DMA window, " -"error %i (%s)\n" , errno,strerror (errno)); -return -1 ; +/* try possible page_shift and levels for workaround */ +uint32_t levels; + +for (levels =1 ; levels <= info.ddw.levels; levels++) { +uint32_t pgsizes = info.ddw.pgsizes;+CC POWER maintainer from
MAINTAINERSfile.Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
This is failing compilation on some older distros (Fedora 20,
Ubuntu 14.04):
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c: In function ‘vfio_spapr_create_new_dma_window’:/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c:1451:34: error:‘struct
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
vfio_iommu_spapr_tce_info’ has no member named ‘ddw’ for (levels = 1; levels <= info.ddw.levels; levels++) { ^
/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c:1452:27: error:‘struct
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
vfio_iommu_spapr_tce_info’ has no member named ‘ddw’
uint32_tpgsizes = info.ddw.pgsizes;
^/root/dpdk/lib/librte_eal/linux/eal/eal_vfio.c: At top level:
cc1: error: unrecognized command line option "-Wno-address-of-packed-member" [-Werror]
cc1: all warnings being treated as errors
make[7]: *** [eal_vfio.o] Error 1
make[7]: *** Waiting for unfinished jobs.... make[6]: *** [eal] Error 2
make[5]: *** [linux] Error 2
make[4]: *** [librte_eal] Error 2
make[3]: *** [lib] Error 2
make[2]: *** [all] Error 2
make[1]: *** [pre_install] Error 2
make: *** [install] Error 2Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Looking at the
eal_vfio.hwe are handling case of missing sPAPRinfo/ddw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
structures, but this isn't simply a case of a missing struct;
rather it
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
is an issue of differing definitions between what recent kernels
have,
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
and what older kernels had.
@Thomas Do we still support these older distros?
If the kernel is not maintained, we don't support it. For most distros, Linux < 3.16 is not maintained.
This is on kernel 3.13 (Ubuntu 14.04 LTS which according to
Canonical's
Quoted 1 line
Quoted 1 line
published schedule [1] has stopped receiving even maintenance
updates,
Quoted 1 line
Quoted 1 line
and is only updated for select customers). It was never an LTS
kernel to
Quoted 4 lines
Quoted 1 line
begin with, i think.
So we can ignore this issue.
It seems that it's also reproducible on Ubuntu 14.04 with kernel 3.19.
Which is not an upstream LTS kernel, but it's >3.16.
Quoted 11 lines
It also looks like it may be an issue for us specifically, so i would
suggest reverting the patch for rc2 if the submitter/maintainers don't
come back and fix it.Signature · 2 lines
Thanks, Anatoly
Hi Anatoly,
Thank you for your report and analysis. This can be avoided by adding #ifdef VFIO_IOMMU_SPAPR_INFO_DDW. I will quickly submit the fix soon after this email.
Regards,
Takeshi