eal: Support running as unprivileged user

PATCH Superseded Ben Walker 4 participants 9 messages Jan 3, 2017 – Mar 9, 2017

Share & tools

Ben Walker

author

2017-01-03 22:30 UTC #0 lore raw

PATCH 1/1 eal: Support running as unprivileged user

For Linux kernel 4.0 and newer, the ability to obtain physical page frame numbers for unprivileged users from /proc/self/pagemap was removed. Instead, when an IOMMU is present, simply choose our own DMA addresses instead.

Signed-off-by Ben Walker <[email protected]>

---
 lib/librte_eal/common/eal_private.h      | 12 ++++++
 lib/librte_eal/linuxapp/eal/eal_memory.c | 71 +++++++++++++++++++++++---------
 lib/librte_eal/linuxapp/eal/eal_pci.c    |  6 ++-
 3 files changed, 68 insertions(+), 21 deletions(-)

lib/librte_eal/common/eal_private.h +12 -0

index 9e7d8f6..8b2d323 100644
@@ -34,6 +34,7 @@
 #ifndef _EAL_PRIVATE_H_
 #define _EAL_PRIVATE_H_
 
+#include <stdbool.h>
 #include <stdio.h>
 #include <rte_pci.h>

@@ -301,4 +302,15 @@ int rte_eal_hugepage_init(void);
  */
 int rte_eal_hugepage_attach(void);
 
+/**
+ * Returns true if the system is able to obtain
+ * physical addresses. Return false if using DMA
+ * addresses through an IOMMU.
+ *
+ * Drivers based on uio will not load unless physical
+ * addresses are obtainable. It is only possible to get
+ * physical addresses when running as a privileged user.
+ */
+bool rte_eal_using_phys_addrs(void);
+
 #endif /* _EAL_PRIVATE_H_ */

lib/librte_eal/linuxapp/eal/eal_memory.c +52 ~~-19~~

131 lines not on this page · raw message

lib/librte_eal/linuxapp/eal/eal_pci.c +4 -2

index 4350134..cfbfedf 100644
@@ -99,8 +99,10 @@ rte_eal_pci_map_device(struct rte_pci_device *dev)
 		break;
 	case RTE_KDRV_IGB_UIO:
 	case RTE_KDRV_UIO_GENERIC:
-		/* map resources for devices that use uio */
-		ret = pci_uio_map_resource(dev);
+		if (rte_eal_using_phys_addrs()) {
+			/* map resources for devices that use uio */
+			ret = pci_uio_map_resource(dev);
+		}
 		break;
 	default:
 		RTE_LOG(DEBUG, EAL,

Signature · 1 line

2.9.3

Ben Walker

author

2017-01-03 22:56 UTC #1 lore raw

PATCH v2 1/1 eal: Support running as unprivileged user

Signed-off-by Ben Walker <[email protected]>

---
 lib/librte_eal/common/eal_private.h      | 12 +++++
 lib/librte_eal/linuxapp/eal/eal_memory.c | 75 +++++++++++++++++++++++---------
 lib/librte_eal/linuxapp/eal/eal_pci.c    |  6 ++-
 3 files changed, 71 insertions(+), 22 deletions(-)

lib/librte_eal/common/eal_private.h +12 -0

index 9e7d8f6..8b2d323 100644
@@ -34,6 +34,7 @@
 #ifndef _EAL_PRIVATE_H_
 #define _EAL_PRIVATE_H_
 
+#include <stdbool.h>
 #include <stdio.h>
 #include <rte_pci.h>

@@ -301,4 +302,15 @@ int rte_eal_hugepage_init(void);
  */
 int rte_eal_hugepage_attach(void);
 
+/**
+ * Returns true if the system is able to obtain
+ * physical addresses. Return false if using DMA
+ * addresses through an IOMMU.
+ *
+ * Drivers based on uio will not load unless physical
+ * addresses are obtainable. It is only possible to get
+ * physical addresses when running as a privileged user.
+ */
+bool rte_eal_using_phys_addrs(void);
+
 #endif /* _EAL_PRIVATE_H_ */

lib/librte_eal/linuxapp/eal/eal_memory.c +55 ~~-20~~

141 lines not on this page · raw message

lib/librte_eal/linuxapp/eal/eal_pci.c +4 -2

index 4350134..cfbfedf 100644
@@ -99,8 +99,10 @@ rte_eal_pci_map_device(struct rte_pci_device *dev)
 		break;
 	case RTE_KDRV_IGB_UIO:
 	case RTE_KDRV_UIO_GENERIC:
-		/* map resources for devices that use uio */
-		ret = pci_uio_map_resource(dev);
+		if (rte_eal_using_phys_addrs()) {
+			/* map resources for devices that use uio */
+			ret = pci_uio_map_resource(dev);
+		}
 		break;
 	default:
 		RTE_LOG(DEBUG, EAL,

Signature · 1 line

2.9.3

Sergio Gonzalez Monroy

2017-01-18 17:05 UTC #2 lore raw

Quoted 15 lines · Ben Walker

For Linux kernel 4.0 and newer, the ability to obtain physical page frame numbers for unprivileged users from /proc/self/pagemap was removed. Instead, when an IOMMU is present, simply choose our own DMA addresses instead.
Signed-off-by Ben Walker <[email protected]>
lib/librte_eal/linuxapp/eal/eal_memory.c +5 -0
@@ -255,6 +257,22 @@ find_physaddrs(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi)
  }
 
  /*
+ * For each hugepage in hugepg_tbl, fill the physaddr value sequentially.
+ */
+static int
+set_physaddrs(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi)
+{

lib/librte_eal/linuxapp/eal/eal_memory.c find_physaddrs(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi) · line 274

Quoted 10 lines

lib/librte_eal/linuxapp/eal/eal_memory.c +10 -0

+	unsigned int i;
+	phys_addr_t addr = 0;
+
+	for (i = 0; i < hpi->num_pages[0]; i++) {
+		hugepg_tbl[i].physaddr = addr;
+		addr += hugepg_tbl[i].size;
+	}
+	return 0;
+}
+

Sorry for late catch, but when having multiple hugepage sizes, both would be mapped from address 0. I thin making the var static should be enough.
Also I would prefer some randomness on the start address instead of always 0.

Thanks,
Sergio

Ben Walker

author

2017-01-31 17:41 UTC #3 lore raw

PATCH v3 1/1 eal: Support running as unprivileged user

In reply to Ben Walker

Signed-off-by Ben Walker <[email protected]>

---
 lib/librte_eal/common/eal_private.h      | 12 +++++
 lib/librte_eal/linuxapp/eal/eal_memory.c | 75 +++++++++++++++++++++++---------
 lib/librte_eal/linuxapp/eal/eal_pci.c    |  6 ++-
 3 files changed, 71 insertions(+), 22 deletions(-)

lib/librte_eal/common/eal_private.h +12 -0

index 9e7d8f6..8b2d323 100644
@@ -34,6 +34,7 @@
 #ifndef _EAL_PRIVATE_H_
 #define _EAL_PRIVATE_H_
 
+#include <stdbool.h>
 #include <stdio.h>
 #include <rte_pci.h>

@@ -301,4 +302,15 @@ int rte_eal_hugepage_init(void);
  */
 int rte_eal_hugepage_attach(void);
 
+/**
+ * Returns true if the system is able to obtain
+ * physical addresses. Return false if using DMA
+ * addresses through an IOMMU.
+ *
+ * Drivers based on uio will not load unless physical
+ * addresses are obtainable. It is only possible to get
+ * physical addresses when running as a privileged user.
+ */
+bool rte_eal_using_phys_addrs(void);
+
 #endif /* _EAL_PRIVATE_H_ */

lib/librte_eal/linuxapp/eal/eal_memory.c +55 ~~-20~~

141 lines not on this page · raw message

lib/librte_eal/linuxapp/eal/eal_pci.c +4 -2

index e2fc219..61d55b9 100644
@@ -99,8 +99,10 @@ rte_eal_pci_map_device(struct rte_pci_device *dev)
 		break;
 	case RTE_KDRV_IGB_UIO:
 	case RTE_KDRV_UIO_GENERIC:
-		/* map resources for devices that use uio */
-		ret = pci_uio_map_resource(dev);
+		if (rte_eal_using_phys_addrs()) {
+			/* map resources for devices that use uio */
+			ret = pci_uio_map_resource(dev);
+		}
 		break;
 	default:
 		RTE_LOG(DEBUG, EAL,

Signature · 1 line

2.9.3

Ben Walker

author

2017-01-31 17:44 UTC #4 lore raw

PATCH v4 1/1 eal: Support running as unprivileged user

Signed-off-by Ben Walker <[email protected]>

---
 lib/librte_eal/common/eal_private.h      | 12 +++++
 lib/librte_eal/linuxapp/eal/eal_memory.c | 75 +++++++++++++++++++++++---------
 lib/librte_eal/linuxapp/eal/eal_pci.c    |  6 ++-
 3 files changed, 71 insertions(+), 22 deletions(-)

lib/librte_eal/common/eal_private.h +12 -0

index 9e7d8f6..8b2d323 100644
@@ -34,6 +34,7 @@
 #ifndef _EAL_PRIVATE_H_
 #define _EAL_PRIVATE_H_
 
+#include <stdbool.h>
 #include <stdio.h>
 #include <rte_pci.h>

@@ -301,4 +302,15 @@ int rte_eal_hugepage_init(void);
  */
 int rte_eal_hugepage_attach(void);
 
+/**
+ * Returns true if the system is able to obtain
+ * physical addresses. Return false if using DMA
+ * addresses through an IOMMU.
+ *
+ * Drivers based on uio will not load unless physical
+ * addresses are obtainable. It is only possible to get
+ * physical addresses when running as a privileged user.
+ */
+bool rte_eal_using_phys_addrs(void);
+
 #endif /* _EAL_PRIVATE_H_ */

lib/librte_eal/linuxapp/eal/eal_memory.c +55 ~~-20~~

141 lines not on this page · raw message

lib/librte_eal/linuxapp/eal/eal_pci.c +4 -2

index e2fc219..61d55b9 100644
@@ -99,8 +99,10 @@ rte_eal_pci_map_device(struct rte_pci_device *dev)
 		break;
 	case RTE_KDRV_IGB_UIO:
 	case RTE_KDRV_UIO_GENERIC:
-		/* map resources for devices that use uio */
-		ret = pci_uio_map_resource(dev);
+		if (rte_eal_using_phys_addrs()) {
+			/* map resources for devices that use uio */
+			ret = pci_uio_map_resource(dev);
+		}
 		break;
 	default:
 		RTE_LOG(DEBUG, EAL,

Signature · 1 line

2.9.3

Sergio Gonzalez Monroy

Acked-by on v4 1/1

2017-02-17 14:59 UTC #5 lore raw

Quoted 11 lines · Ben Walker

For Linux kernel 4.0 and newer, the ability to obtain physical page frame numbers for unprivileged users from /proc/self/pagemap was removed. Instead, when an IOMMU is present, simply choose our own DMA addresses instead.
Signed-off-by Ben Walker <[email protected]>
---
  lib/librte_eal/common/eal_private.h      | 12 +++++
  lib/librte_eal/linuxapp/eal/eal_memory.c | 75 +++++++++++++++++++++++---------
  lib/librte_eal/linuxapp/eal/eal_pci.c    |  6 ++-
  3 files changed, 71 insertions(+), 22 deletions(-)

Acked-by Sergio Gonzalez Monroy <[email protected]>

PS: Please keep a summary of changes made in each version on future patch sets (after the triple dash --- )

Thomas Monjalon

Applied

2017-03-09 16:10 UTC #6 lore raw

2017-02-17 14:59, Sergio Gonzalez Monroy:

Quoted 6 lines · Ben Walker

Quoted 5 lines
For Linux kernel 4.0 and newer, the ability to obtain physical page frame numbers for unprivileged users from /proc/self/pagemap was removed. Instead, when an IOMMU is present, simply choose our own DMA addresses instead.

Quoted 8 lines

Quoted 6 lines
Signed-off-by Ben Walker <[email protected]>
---
  lib/librte_eal/common/eal_private.h      | 12 +++++
  lib/librte_eal/linuxapp/eal/eal_memory.c | 75 +++++++++++++++++++++++---------
  lib/librte_eal/linuxapp/eal/eal_pci.c    |  6 ++-
  3 files changed, 71 insertions(+), 22 deletions(-)
Acked-by Sergio Gonzalez Monroy <[email protected]>

Applied, thanks

Stephen Hemminger

2017-02-17 19:28 UTC #7 lore raw

In reply to Ben Walker

lib/librte_eal/linuxapp/eal/eal_memory.c int rte_xen_dom0_supported(void) · line 140

Quoted 6 lines · On Tue, 31 Jan 2017 10:44:53 -0700 Ben Walker <[email protected]> wrote:

lib/librte_eal/linuxapp/eal/eal_memory.c +3 -2

+	if (physaddr == RTE_BAD_PHYS_ADDR) {
 		RTE_LOG(ERR, EAL,
-			"Cannot open /proc/self/pagemap: %s. "
-			"virt2phys address translation will not work\n",
+			"Cannot obtain physical addresses: %s. "
+			"Only vfio will function.\n",

Please don't split a single error message across multiple lines. It makes it harder for user to find the source code lines with simple grep. Better to just have one long line for format string, or better yet be less wordy.

Yes, the existing DPDK code has lots of these issues.

Thomas Monjalon

2017-03-09 16:11 UTC #8 lore raw

2017-02-17 11:28, Stephen Hemminger:

Quoted 14 lines

lib/librte_eal/linuxapp/eal/eal_memory.c int rte_xen_dom0_supported(void) · line 140
Quoted 6 lines · On Tue, 31 Jan 2017 10:44:53 -0700 Ben Walker <[email protected]> wrote:
lib/librte_eal/linuxapp/eal/eal_memory.c +3 -2
+	if (physaddr == RTE_BAD_PHYS_ADDR) {
 		RTE_LOG(ERR, EAL,
-			"Cannot open /proc/self/pagemap: %s. "
-			"virt2phys address translation will not work\n",
+			"Cannot obtain physical addresses: %s. "
+			"Only vfio will function.\n",
Please don't split a single error message across multiple lines. It makes it harder for user to find the source code lines with simple grep. Better to just have one long line for format string, or better yet be less wordy.

Quoted 1 line

Yes, the existing DPDK code has lots of these issues.

You're right.
Here the split is acceptable as there is strerror(errno) in the middle.