Skip to contentSkip to conversation

doc: document vfio-pci usage with QAT PMD

PATCH Superseded Adam Dybkowski 5 participants 16 messages Sep 8, 2020 – Oct 28, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 doc: document vfio-pci usage with QAT PMD

This patch marks the old igb-uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

---
 doc/guides/cryptodevs/qat.rst | 33 ++++++++++++++++++++++-----------
 1 file changed, 22 insertions(+), 11 deletions(-)
doc/guides/cryptodevs/qat.rst +22 -11
index e5d2cf499..bd4743aa7 100644
@@ -462,7 +462,7 @@ Check that the VFs are available for use. For example ``lspci -d:37c9`` should
 list 48 VF devices available for a ``C62x`` device.
 
 To complete the installation follow the instructions in
-`Binding the available VFs to the DPDK UIO driver`_.
+`Binding the available VFs to the vfio-pci driver`_.
 
 .. Note::
 
@@ -534,7 +534,7 @@ Confirm the presence of 48 VF devices - 16 per PF::
     lspci -d:37c9
 
 
-To complete the installation - follow instructions in `Binding the available VFs to the DPDK UIO driver`_.
+To complete the installation - follow instructions in `Binding the available VFs to the vfio-pci driver`_.
 
 .. Note::
 
@@ -584,10 +584,21 @@ To complete the installation - follow instructions in `Binding the available VFs
       sudo yum install kernel-devel-`uname -r`
 
 
-Binding the available VFs to the DPDK UIO driver
+Binding the available VFs to the vfio-pci driver
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Unbind the VFs from the stock driver so they can be bound to the uio driver.
+Notice:
+
+* Please note that due to security issues, the usage of older DPDK igb-uio
+  driver is not recommended. This document shows how to use the more secure
+  vfio-pci driver.
+* If QAT fails to bind to vfio-pci on Linux kernel 5.9+, please see the
+  QATE-39220 and QATE-7495 issues in
+  `01.org doc <https://01.org/sites/default/files/downloads/336211-015-qatsoftwareforlinux-rn-hwv1.7-final.pdf>`_
+  which details the constraint about trusted guests and add `disable_denylist=1`
+  to the vfio-pci params to use QAT. See also `this patch description <https://lkml.org/lkml/2020/7/23/1155>`_.
+
+Unbind the VFs from the stock driver so they can be bound to the vfio-pci driver.
 
 For an Intel(R) QuickAssist Technology DH895xCC device
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -635,25 +646,25 @@ VFs are different adjust the unbind command below::
         done; \
     done
 
-Bind to the DPDK uio driver
+Bind to the vfio-pci driver
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Install the DPDK igb_uio driver, bind the VF PCI Device id to it and use lspci
-to confirm the VF devices are now in use by igb_uio kernel driver,
+Load the vfio-pci driver, bind the VF PCI Device id to it and use lspci
+to confirm the VF devices are now in use by vfio-pci kernel driver,
 e.g. for the C62x device::
 
     cd to the top-level DPDK directory
     modprobe uio
-    insmod ./build/kmod/igb_uio.ko
-    echo "8086 37c9" > /sys/bus/pci/drivers/igb_uio/new_id
+    modprobe vfio-pci
+    echo "8086 37c9" > /sys/bus/pci/drivers/vfio-pci/new_id
     lspci -vvd:37c9
 
 
-Another way to bind the VFs to the DPDK UIO driver is by using the
+Another way to bind the VFs to the vfio-pci driver is by using the
 ``dpdk-devbind.py`` script::
 
     cd to the top-level DPDK directory
-    ./usertools/dpdk-devbind.py -b igb_uio 0000:03:01.1
+    ./usertools/dpdk-devbind.py -b vfio-pci 0000:03:01.1
 
 Testing
 ~~~~~~~
Signature · 1 line
2.25.1
Acked-by on v1 1/1
#1 lore raw

Hi Adam,

Quoted 15 lines

-----Original Message-----
From: Dybkowski, AdamX <[email protected]>
Sent: Tuesday, September 8, 2020 3:21 PM
To: [email protected]; Trahe, Fiona <[email protected]>; [email protected]; Mcnamara, John <[email protected]>
Cc: Dybkowski, AdamX <[email protected]>
Subject: [PATCH] doc: document vfio-pci usage with QAT PMD

This patch marks the old igb-uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

Quoted 2 lines

---

//snip///

doc/guides/cryptodevs/qat.rst VFs are different adjust the unbind command below:: · line 658

Quoted 9 lines
doc/guides/cryptodevs/qat.rst +3 -2
+Load the vfio-pci driver, bind the VF PCI Device id to it and use lspci
+to confirm the VF devices are now in use by vfio-pci kernel driver,
 e.g. for the C62x device::
 
     cd to the top-level DPDK directory
     modprobe uio
-    insmod ./build/kmod/igb_uio.ko
-    echo "8086 37c9" > /sys/bus/pci/drivers/igb_uio/new_id
+    modprobe vfio-pci

[Fiona] it would be useful to also add a note here - something like
"Use modprobe vfio-pci disable_denylist=1" from kernel 5.9 onwards. See note in Section xxx above"

Apart from this

#2 lore raw

v2 cover doc: document vfio-pci usage with QAT PMD

In reply to Adam Dybkowski

This patch marks the old igb-uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

v2:
* add example command with disable_denylist option

Adam Dybkowski (1):
doc: document vfio-pci usage with QAT PMD

 doc/guides/cryptodevs/qat.rst | 36 ++++++++++++++++++++++++-----------
 1 file changed, 25 insertions(+), 11 deletions(-)
Signature · 1 line
2.25.1
#3 lore raw

PATCH v2 1/1 doc: document vfio-pci usage with QAT PMD

This patch marks the old igb-uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

---
 doc/guides/cryptodevs/qat.rst | 36 ++++++++++++++++++++++++-----------
 1 file changed, 25 insertions(+), 11 deletions(-)
doc/guides/cryptodevs/qat.rst +25 -11
index e5d2cf499..8dd7eb083 100644
@@ -462,7 +462,7 @@ Check that the VFs are available for use. For example ``lspci -d:37c9`` should
 list 48 VF devices available for a ``C62x`` device.
 
 To complete the installation follow the instructions in
-`Binding the available VFs to the DPDK UIO driver`_.
+`Binding the available VFs to the vfio-pci driver`_.
 
 .. Note::
 
@@ -534,7 +534,7 @@ Confirm the presence of 48 VF devices - 16 per PF::
     lspci -d:37c9
 
 
-To complete the installation - follow instructions in `Binding the available VFs to the DPDK UIO driver`_.
+To complete the installation - follow instructions in `Binding the available VFs to the vfio-pci driver`_.
 
 .. Note::
 
@@ -584,10 +584,21 @@ To complete the installation - follow instructions in `Binding the available VFs
       sudo yum install kernel-devel-`uname -r`
 
 
-Binding the available VFs to the DPDK UIO driver
+Binding the available VFs to the vfio-pci driver
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Unbind the VFs from the stock driver so they can be bound to the uio driver.
+Notice:
+
+* Please note that due to security issues, the usage of older DPDK igb-uio
+  driver is not recommended. This document shows how to use the more secure
+  vfio-pci driver.
+* If QAT fails to bind to vfio-pci on Linux kernel 5.9+, please see the
+  QATE-39220 and QATE-7495 issues in
+  `01.org doc <https://01.org/sites/default/files/downloads/336211-015-qatsoftwareforlinux-rn-hwv1.7-final.pdf>`_
+  which details the constraint about trusted guests and add `disable_denylist=1`
+  to the vfio-pci params to use QAT. See also `this patch description <https://lkml.org/lkml/2020/7/23/1155>`_.
+
+Unbind the VFs from the stock driver so they can be bound to the vfio-pci driver.
 
 For an Intel(R) QuickAssist Technology DH895xCC device
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -635,25 +646,28 @@ VFs are different adjust the unbind command below::
         done; \
     done
 
-Bind to the DPDK uio driver
+Bind to the vfio-pci driver
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Install the DPDK igb_uio driver, bind the VF PCI Device id to it and use lspci
-to confirm the VF devices are now in use by igb_uio kernel driver,
+Load the vfio-pci driver, bind the VF PCI Device id to it and use lspci
+to confirm the VF devices are now in use by vfio-pci kernel driver,
 e.g. for the C62x device::
 
     cd to the top-level DPDK directory
     modprobe uio
-    insmod ./build/kmod/igb_uio.ko
-    echo "8086 37c9" > /sys/bus/pci/drivers/igb_uio/new_id
+    modprobe vfio-pci
+    echo "8086 37c9" > /sys/bus/pci/drivers/vfio-pci/new_id
     lspci -vvd:37c9
 
+Use ``modprobe vfio-pci disable_denylist=1`` from kernel 5.9 onwards.
+See note in the section `Binding the available VFs to the vfio-pci driver`_
+above.
 
-Another way to bind the VFs to the DPDK UIO driver is by using the
+Another way to bind the VFs to the vfio-pci driver is by using the
 ``dpdk-devbind.py`` script::
 
     cd to the top-level DPDK directory
-    ./usertools/dpdk-devbind.py -b igb_uio 0000:03:01.1
+    ./usertools/dpdk-devbind.py -b vfio-pci 0000:03:01.1
 
 Testing
 ~~~~~~~
Signature · 1 line
2.25.1
Acked-by on the v2 series
#5 lore raw
In reply to Adam Dybkowski
Quoted 8 lines

-----Original Message-----
From: Dybkowski, AdamX <[email protected]>
Sent: Tuesday, September 8, 2020 5:20 PM
To: [email protected]; Trahe, Fiona <[email protected]>; [email protected]; Mcnamara, John <[email protected]>
Cc: Dybkowski, AdamX <[email protected]>
Subject: [PATCH v2 0/1] doc: document vfio-pci usage with QAT PMD

Quoted 4 lines

...

 doc/guides/cryptodevs/qat.rst | 36 ++++++++++++++++++++++++-----------
 1 file changed, 25 insertions(+), 11 deletions(-)
#4 lore raw
In reply to Adam Dybkowski
Quoted 10 lines · Adam Dybkowski

This patch marks the old igb-uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

---

<snip>

Quoted 3 lines

cd to the top-level DPDK directory
modprobe uio

No need to modprobe uio if you're using VFIO.

doc/guides/cryptodevs/qat.rst VFs are different adjust the unbind command below:: · line 659

Quoted 4 lines
doc/guides/cryptodevs/qat.rst +2 -2
-    insmod ./build/kmod/igb_uio.ko
-    echo "8086 37c9" > /sys/bus/pci/drivers/igb_uio/new_id
+    modprobe vfio-pci
+    echo "8086 37c9" > /sys/bus/pci/drivers/vfio-pci/new_id

This seems like very outdated instructions from when we had a PCI ID filtering in igb_uio. Devbind way is better in every way, so why keep this at all?

Quoted 17 lines
      lspci -vvd:37c9
doc/guides/cryptodevs/qat.rst +5 -2
+Use ``modprobe vfio-pci disable_denylist=1`` from kernel 5.9 onwards.
+See note in the section `Binding the available VFs to the vfio-pci driver`_
+above.
 
-Another way to bind the VFs to the DPDK UIO driver is by using the
+Another way to bind the VFs to the vfio-pci driver is by using the
  ``dpdk-devbind.py`` script::
 
      cd to the top-level DPDK directory
-    ./usertools/dpdk-devbind.py -b igb_uio 0000:03:01.1
+    ./usertools/dpdk-devbind.py -b vfio-pci 0000:03:01.1
 
  Testing
  ~~~~~~~
Signature · 2 lines
Thanks,
Anatoly
#6 lore raw

v3 cover doc: document vfio-pci usage with QAT PMD

In reply to Adam Dybkowski

This patch marks the old igb_uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

v2:
* add example command with disable_denylist option

v3:
* simplify binding/unbinding command examples by using python scripts
* remove the description of old binding method

Adam Dybkowski (1):
doc: document vfio-pci usage with QAT PMD

 doc/guides/cryptodevs/qat.rst | 63 +++++++++++++++++++----------------
 1 file changed, 34 insertions(+), 29 deletions(-)
Signature · 1 line
2.25.1
#7 lore raw

PATCH v3 1/1 doc: document vfio-pci usage with QAT PMD

This patch marks the old igb_uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

---
 doc/guides/cryptodevs/qat.rst | 63 +++++++++++++++++++----------------
 1 file changed, 34 insertions(+), 29 deletions(-)
doc/guides/cryptodevs/qat.rst +34 -29
index e5d2cf499..dbbdec1c7 100644
@@ -462,7 +462,7 @@ Check that the VFs are available for use. For example ``lspci -d:37c9`` should
 list 48 VF devices available for a ``C62x`` device.
 
 To complete the installation follow the instructions in
-`Binding the available VFs to the DPDK UIO driver`_.
+`Binding the available VFs to the vfio-pci driver`_.
 
 .. Note::
 
@@ -534,7 +534,8 @@ Confirm the presence of 48 VF devices - 16 per PF::
     lspci -d:37c9
 
 
-To complete the installation - follow instructions in `Binding the available VFs to the DPDK UIO driver`_.
+To complete the installation - follow instructions in
+`Binding the available VFs to the vfio-pci driver`_.
 
 .. Note::
 
@@ -584,10 +585,21 @@ To complete the installation - follow instructions in `Binding the available VFs
       sudo yum install kernel-devel-`uname -r`
 
 
-Binding the available VFs to the DPDK UIO driver
+Binding the available VFs to the vfio-pci driver
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Unbind the VFs from the stock driver so they can be bound to the uio driver.
+Note:
+
+* Please note that due to security issues, the usage of older DPDK igb_uio
+  driver is not recommended. This document shows how to use the more secure
+  vfio-pci driver.
+* If QAT fails to bind to vfio-pci on Linux kernel 5.9+, please see the
+  QATE-39220 and QATE-7495 issues in
+  `01.org doc <https://01.org/sites/default/files/downloads/336211-015-qatsoftwareforlinux-rn-hwv1.7-final.pdf>`_
+  which details the constraint about trusted guests and add `disable_denylist=1`
+  to the vfio-pci params to use QAT. See also `this patch description <https://lkml.org/lkml/2020/7/23/1155>`_.
+
+Unbind the VFs from the stock driver so they can be bound to the vfio-pci driver.
 
 For an Intel(R) QuickAssist Technology DH895xCC device
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -595,10 +607,10 @@ For an Intel(R) QuickAssist Technology DH895xCC device
 The unbind command below assumes ``BDFs`` of ``03:01.00-03:04.07``, if your
 VFs are different adjust the unbind command below::
 
+    cd to the top-level DPDK directory
     for device in $(seq 1 4); do \
         for fn in $(seq 0 7); do \
-            echo -n 0000:03:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:03\:0${device}.${fn}/driver/unbind; \
+            usertools/dpdk-devbind.py -u 0000:03:0${device}.${fn}; \
         done; \
     done
 
@@ -609,16 +621,12 @@ The unbind command below assumes ``BDFs`` of ``1a:01.00-1a:02.07``,
 ``3d:01.00-3d:02.07`` and ``3f:01.00-3f:02.07``, if your VFs are different
 adjust the unbind command below::
 
+    cd to the top-level DPDK directory
     for device in $(seq 1 2); do \
         for fn in $(seq 0 7); do \
-            echo -n 0000:1a:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:1a\:0${device}.${fn}/driver/unbind; \
-
-            echo -n 0000:3d:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:3d\:0${device}.${fn}/driver/unbind; \
-
-            echo -n 0000:3f:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:3f\:0${device}.${fn}/driver/unbind; \
+            usertools/dpdk-devbind.py -u 0000:1a:0${device}.${fn}; \
+            usertools/dpdk-devbind.py -u 0000:3d:0${device}.${fn}; \
+            usertools/dpdk-devbind.py -u 0000:3f:0${device}.${fn}; \
         done; \
     done
 
@@ -628,32 +636,29 @@ For Intel(R) QuickAssist Technology C3xxx or 200xx or D15xx device
 The unbind command below assumes ``BDFs`` of ``01:01.00-01:02.07``, if your
 VFs are different adjust the unbind command below::
 
+    cd to the top-level DPDK directory
     for device in $(seq 1 2); do \
         for fn in $(seq 0 7); do \
-            echo -n 0000:01:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:01\:0${device}.${fn}/driver/unbind; \
+            usertools/dpdk-devbind.py -u 0000:01:0${device}.${fn}; \
         done; \
     done
 
-Bind to the DPDK uio driver
+Bind to the vfio-pci driver
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Install the DPDK igb_uio driver, bind the VF PCI Device id to it and use lspci
-to confirm the VF devices are now in use by igb_uio kernel driver,
+Load the vfio-pci driver, bind the VF PCI Device id to it using the
+``dpdk-devbind.py`` script then use the ``--status`` option
+to confirm the VF devices are now in use by vfio-pci kernel driver,
 e.g. for the C62x device::
 
     cd to the top-level DPDK directory
-    modprobe uio
-    insmod ./build/kmod/igb_uio.ko
-    echo "8086 37c9" > /sys/bus/pci/drivers/igb_uio/new_id
-    lspci -vvd:37c9
-
+    modprobe vfio-pci
+    usertools/dpdk-devbind.py -b vfio-pci 0000:03:01.1
+    usertools/dpdk-devbind.py --status
 
-Another way to bind the VFs to the DPDK UIO driver is by using the
-``dpdk-devbind.py`` script::
-
-    cd to the top-level DPDK directory
-    ./usertools/dpdk-devbind.py -b igb_uio 0000:03:01.1
+Use ``modprobe vfio-pci disable_denylist=1`` from kernel 5.9 onwards.
+See note in the section `Binding the available VFs to the vfio-pci driver`_
+above.
 
 Testing
 ~~~~~~~
Signature · 1 line
2.25.1
Acked-by on v3 1/1
#8 lore raw
Quoted 9 lines · Adam Dybkowski

This patch marks the old igb_uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

---

Signature · 2 lines
Thanks,
Anatoly
#9 lore raw
In reply to Adam Dybkowski

Hi Adam,

Quoted 9 lines

This patch marks the old igb_uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

---

Could you please rebase this patch?

Regards,
Akhil

#11 lore raw

v4 cover doc: document vfio-pci usage with QAT PMD

In reply to Adam Dybkowski

This patch marks the old igb_uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

v2:
* add example command with disable_denylist option

v3:
* simplify binding/unbinding command examples by using python scripts
* remove the description of old binding method

v4:
* rebase

Adam Dybkowski (1):
doc: document vfio-pci usage with QAT PMD

 doc/guides/cryptodevs/qat.rst | 63 +++++++++++++++++++----------------
 1 file changed, 34 insertions(+), 29 deletions(-)
Signature · 1 line
2.25.1
#12 lore raw

PATCH v4 1/1 doc: document vfio-pci usage with QAT PMD

This patch marks the old igb-uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

---
 doc/guides/cryptodevs/qat.rst | 63 +++++++++++++++++++----------------
 1 file changed, 34 insertions(+), 29 deletions(-)
doc/guides/cryptodevs/qat.rst +34 -29
index e5d2cf499..dbbdec1c7 100644
@@ -462,7 +462,7 @@ Check that the VFs are available for use. For example ``lspci -d:37c9`` should
 list 48 VF devices available for a ``C62x`` device.
 
 To complete the installation follow the instructions in
-`Binding the available VFs to the DPDK UIO driver`_.
+`Binding the available VFs to the vfio-pci driver`_.
 
 .. Note::
 
@@ -534,7 +534,8 @@ Confirm the presence of 48 VF devices - 16 per PF::
     lspci -d:37c9
 
 
-To complete the installation - follow instructions in `Binding the available VFs to the DPDK UIO driver`_.
+To complete the installation - follow instructions in
+`Binding the available VFs to the vfio-pci driver`_.
 
 .. Note::
 
@@ -584,10 +585,21 @@ To complete the installation - follow instructions in `Binding the available VFs
       sudo yum install kernel-devel-`uname -r`
 
 
-Binding the available VFs to the DPDK UIO driver
+Binding the available VFs to the vfio-pci driver
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Unbind the VFs from the stock driver so they can be bound to the uio driver.
+Note:
+
+* Please note that due to security issues, the usage of older DPDK igb-uio
+  driver is not recommended. This document shows how to use the more secure
+  vfio-pci driver.
+* If QAT fails to bind to vfio-pci on Linux kernel 5.9+, please see the
+  QATE-39220 and QATE-7495 issues in
+  `01.org doc <https://01.org/sites/default/files/downloads/336211-015-qatsoftwareforlinux-rn-hwv1.7-final.pdf>`_
+  which details the constraint about trusted guests and add `disable_denylist=1`
+  to the vfio-pci params to use QAT. See also `this patch description <https://lkml.org/lkml/2020/7/23/1155>`_.
+
+Unbind the VFs from the stock driver so they can be bound to the vfio-pci driver.
 
 For an Intel(R) QuickAssist Technology DH895xCC device
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -595,10 +607,10 @@ For an Intel(R) QuickAssist Technology DH895xCC device
 The unbind command below assumes ``BDFs`` of ``03:01.00-03:04.07``, if your
 VFs are different adjust the unbind command below::
 
+    cd to the top-level DPDK directory
     for device in $(seq 1 4); do \
         for fn in $(seq 0 7); do \
-            echo -n 0000:03:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:03\:0${device}.${fn}/driver/unbind; \
+            usertools/dpdk-devbind.py -u 0000:03:0${device}.${fn}; \
         done; \
     done
 
@@ -609,16 +621,12 @@ The unbind command below assumes ``BDFs`` of ``1a:01.00-1a:02.07``,
 ``3d:01.00-3d:02.07`` and ``3f:01.00-3f:02.07``, if your VFs are different
 adjust the unbind command below::
 
+    cd to the top-level DPDK directory
     for device in $(seq 1 2); do \
         for fn in $(seq 0 7); do \
-            echo -n 0000:1a:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:1a\:0${device}.${fn}/driver/unbind; \
-
-            echo -n 0000:3d:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:3d\:0${device}.${fn}/driver/unbind; \
-
-            echo -n 0000:3f:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:3f\:0${device}.${fn}/driver/unbind; \
+            usertools/dpdk-devbind.py -u 0000:1a:0${device}.${fn}; \
+            usertools/dpdk-devbind.py -u 0000:3d:0${device}.${fn}; \
+            usertools/dpdk-devbind.py -u 0000:3f:0${device}.${fn}; \
         done; \
     done
 
@@ -628,32 +636,29 @@ For Intel(R) QuickAssist Technology C3xxx or 200xx or D15xx device
 The unbind command below assumes ``BDFs`` of ``01:01.00-01:02.07``, if your
 VFs are different adjust the unbind command below::
 
+    cd to the top-level DPDK directory
     for device in $(seq 1 2); do \
         for fn in $(seq 0 7); do \
-            echo -n 0000:01:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:01\:0${device}.${fn}/driver/unbind; \
+            usertools/dpdk-devbind.py -u 0000:01:0${device}.${fn}; \
         done; \
     done
 
-Bind to the DPDK uio driver
+Bind to the vfio-pci driver
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Install the DPDK igb_uio driver, bind the VF PCI Device id to it and use lspci
-to confirm the VF devices are now in use by igb_uio kernel driver,
+Load the vfio-pci driver, bind the VF PCI Device id to it using the
+``dpdk-devbind.py`` script then use the ``--status`` option
+to confirm the VF devices are now in use by vfio-pci kernel driver,
 e.g. for the C62x device::
 
     cd to the top-level DPDK directory
-    modprobe uio
-    insmod ./build/kmod/igb_uio.ko
-    echo "8086 37c9" > /sys/bus/pci/drivers/igb_uio/new_id
-    lspci -vvd:37c9
-
+    modprobe vfio-pci
+    usertools/dpdk-devbind.py -b vfio-pci 0000:03:01.1
+    usertools/dpdk-devbind.py --status
 
-Another way to bind the VFs to the DPDK UIO driver is by using the
-``dpdk-devbind.py`` script::
-
-    cd to the top-level DPDK directory
-    ./usertools/dpdk-devbind.py -b igb_uio 0000:03:01.1
+Use ``modprobe vfio-pci disable_denylist=1`` from kernel 5.9 onwards.
+See note in the section `Binding the available VFs to the vfio-pci driver`_
+above.
 
 Testing
 ~~~~~~~
Signature · 1 line
2.25.1
#10 lore raw
In reply to Akhil Goyal

OK, v4 was just sent.

Adam

Quoted 21 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Thursday, 8 October, 2020 23:29
To: Dybkowski, AdamX <[email protected]>; Trahe, Fiona <[email protected]>; [email protected]; Burakov, Anatoly <[email protected]>
Subject: RE: [PATCH v3 1/1] doc: document vfio-pci usage with QAT PMD

Hi Adam,

Quoted 8 lines

This patch marks the old igb_uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead. It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

---

Could you please rebase this patch?

Regards,
Akhil

#13 lore raw

v5 cover doc: document vfio-pci usage with QAT PMD

In reply to Adam Dybkowski

This patch marks the old igb_uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

v2:
* add example command with disable_denylist option

v3:
* simplify binding/unbinding command examples by using python scripts
* remove the description of old binding method

v4:
* rebase

v5:
* rebase against github dpdk main branch

Adam Dybkowski (1):
doc: document vfio-pci usage with QAT PMD

 doc/guides/cryptodevs/qat.rst | 64 +++++++++++++++++++----------------
 1 file changed, 35 insertions(+), 29 deletions(-)

--
2.25.1

#14 lore raw

PATCH v5 1/1 doc: document vfio-pci usage with QAT PMD

This patch marks the old igb-uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

---
 doc/guides/cryptodevs/qat.rst | 64 +++++++++++++++++++----------------
 1 file changed, 35 insertions(+), 29 deletions(-)
doc/guides/cryptodevs/qat.rst +35 -29
index 7c5629319..dbbdec1c7 100644
@@ -462,7 +462,7 @@ Check that the VFs are available for use. For example ``lspci -d:37c9`` should
 list 48 VF devices available for a ``C62x`` device.
 
 To complete the installation follow the instructions in
-`Binding the available VFs to the DPDK UIO driver`_.
+`Binding the available VFs to the vfio-pci driver`_.
 
 .. Note::
 
@@ -534,7 +534,8 @@ Confirm the presence of 48 VF devices - 16 per PF::
     lspci -d:37c9
 
 
-To complete the installation - follow instructions in `Binding the available VFs to the DPDK UIO driver`_.
+To complete the installation - follow instructions in
+`Binding the available VFs to the vfio-pci driver`_.
 
 .. Note::
 
@@ -584,10 +585,21 @@ To complete the installation - follow instructions in `Binding the available VFs
       sudo yum install kernel-devel-`uname -r`
 
 
-Binding the available VFs to the DPDK UIO driver
+Binding the available VFs to the vfio-pci driver
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Unbind the VFs from the stock driver so they can be bound to the uio driver.
+Note:
+
+* Please note that due to security issues, the usage of older DPDK igb-uio
+  driver is not recommended. This document shows how to use the more secure
+  vfio-pci driver.
+* If QAT fails to bind to vfio-pci on Linux kernel 5.9+, please see the
+  QATE-39220 and QATE-7495 issues in
+  `01.org doc <https://01.org/sites/default/files/downloads/336211-015-qatsoftwareforlinux-rn-hwv1.7-final.pdf>`_
+  which details the constraint about trusted guests and add `disable_denylist=1`
+  to the vfio-pci params to use QAT. See also `this patch description <https://lkml.org/lkml/2020/7/23/1155>`_.
+
+Unbind the VFs from the stock driver so they can be bound to the vfio-pci driver.
 
 For an Intel(R) QuickAssist Technology DH895xCC device
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -595,10 +607,10 @@ For an Intel(R) QuickAssist Technology DH895xCC device
 The unbind command below assumes ``BDFs`` of ``03:01.00-03:04.07``, if your
 VFs are different adjust the unbind command below::
 
+    cd to the top-level DPDK directory
     for device in $(seq 1 4); do \
         for fn in $(seq 0 7); do \
-            echo -n 0000:03:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:03\:0${device}.${fn}/driver/unbind; \
+            usertools/dpdk-devbind.py -u 0000:03:0${device}.${fn}; \
         done; \
     done
 
@@ -609,16 +621,12 @@ The unbind command below assumes ``BDFs`` of ``1a:01.00-1a:02.07``,
 ``3d:01.00-3d:02.07`` and ``3f:01.00-3f:02.07``, if your VFs are different
 adjust the unbind command below::
 
+    cd to the top-level DPDK directory
     for device in $(seq 1 2); do \
         for fn in $(seq 0 7); do \
-            echo -n 0000:1a:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:1a\:0${device}.${fn}/driver/unbind; \
-
-            echo -n 0000:3d:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:3d\:0${device}.${fn}/driver/unbind; \
-
-            echo -n 0000:3f:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:3f\:0${device}.${fn}/driver/unbind; \
+            usertools/dpdk-devbind.py -u 0000:1a:0${device}.${fn}; \
+            usertools/dpdk-devbind.py -u 0000:3d:0${device}.${fn}; \
+            usertools/dpdk-devbind.py -u 0000:3f:0${device}.${fn}; \
         done; \
     done
 
@@ -628,31 +636,29 @@ For Intel(R) QuickAssist Technology C3xxx or 200xx or D15xx device
 The unbind command below assumes ``BDFs`` of ``01:01.00-01:02.07``, if your
 VFs are different adjust the unbind command below::
 
+    cd to the top-level DPDK directory
     for device in $(seq 1 2); do \
         for fn in $(seq 0 7); do \
-            echo -n 0000:01:0${device}.${fn} > \
-            /sys/bus/pci/devices/0000\:01\:0${device}.${fn}/driver/unbind; \
+            usertools/dpdk-devbind.py -u 0000:01:0${device}.${fn}; \
         done; \
     done
 
-Bind to the DPDK uio driver
+Bind to the vfio-pci driver
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Install the DPDK igb_uio driver, bind the VF PCI Device id to it and use lspci
-to confirm the VF devices are now in use by igb_uio kernel driver,
+Load the vfio-pci driver, bind the VF PCI Device id to it using the
+``dpdk-devbind.py`` script then use the ``--status`` option
+to confirm the VF devices are now in use by vfio-pci kernel driver,
 e.g. for the C62x device::
 
-    modprobe uio
-    insmod igb_uio.ko
-    echo "8086 37c9" > /sys/bus/pci/drivers/igb_uio/new_id
-    lspci -vvd:37c9
-
-
-Another way to bind the VFs to the DPDK UIO driver is by using the
-``dpdk-devbind.py`` script::
-
     cd to the top-level DPDK directory
-    ./usertools/dpdk-devbind.py -b igb_uio 0000:03:01.1
+    modprobe vfio-pci
+    usertools/dpdk-devbind.py -b vfio-pci 0000:03:01.1
+    usertools/dpdk-devbind.py --status
+
+Use ``modprobe vfio-pci disable_denylist=1`` from kernel 5.9 onwards.
+See note in the section `Binding the available VFs to the vfio-pci driver`_
+above.
 
 Testing
 ~~~~~~~

--
2.25.1

Applied
#15 lore raw
Quoted 9 lines

Subject: [PATCH v5 1/1] doc: document vfio-pci usage with QAT PMD

This patch marks the old igb-uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead.
It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later.

Quoted 3 lines

---

Applied to dpdk-next-crypto
Thanks.