Skip to contentSkip to conversation

Vyatta patches

PATCH Applied Stephen Hemminger 7 participants 39 messages May 30, 2013 – Jul 19, 2013 26 messages not in the archive

lore
Share & tools
lore
#0 lore raw

cover Vyatta patches

This is a subset of the Vyatta patches we use. They include some bug fixes and simple changes to make life easier.

One fix not included is the NUMA cpu assignment. The original Intel code in 1.2 incorrectly used /proc/cpuinfo to try and assign CPU's to NUMA socket. The problem is that /proc/cpuinfo physical_id corresponds to what the BIOS tells the kernel and is intended for messages only. For example, on our Dell boxes the first CPU and only CPU is reported as physical_id 1! The fix is to use sysfs instead, and Intel did incorporate my fix in the next DPDK version, and don't want to confuse this code base by putting in a conflicting change.

#1 lore raw

PATCH 4/8 igb: workaround errata with wthresh on 82576

The 82576 has known issues which require the write threshold to be set to 1. See:
http://download.intel.com/design/network/specupdt/82576_SPECUPDATE.pdf

If not then single packets will hang in transmit ring until more arrive. Simple tests like ping will fail.

---
 lib/librte_pmd_e1000/em_rxtx.c |    5 +++++
 1 file changed, 5 insertions(+)
lib/librte_pmd_igb/e1000_rxtx.c +5 -0
@@ -1240,6 +1240,8 @@ eth_igb_tx_queue_setup(struct rte_eth_de
 	txq->pthresh = tx_conf->tx_thresh.pthresh;
 	txq->hthresh = tx_conf->tx_thresh.hthresh;
 	txq->wthresh = tx_conf->tx_thresh.wthresh;
+	if (txq->wthresh > 0 && hw->mac.type == e1000_82576)
+		txq->wthresh = 1;
 	txq->queue_id = queue_idx;
 	txq->port_id = dev->data->port_id;
 
@@ -1384,6 +1386,9 @@ eth_igb_rx_queue_setup(struct rte_eth_de
 	rxq->pthresh = rx_conf->rx_thresh.pthresh;
 	rxq->hthresh = rx_conf->rx_thresh.hthresh;
 	rxq->wthresh = rx_conf->rx_thresh.wthresh;
+	if (rxq->wthresh > 0 && hw->mac.type == e1000_82576)
+		rxq->wthresh = 1;
+
 	rxq->rx_free_thresh = rx_conf->rx_free_thresh;
 	rxq->queue_id = queue_idx;
 	rxq->port_id = dev->data->port_id;
#6 lore raw

PATCH 2/7 rte_timer: optimize for empty case

In reply to Stephen Hemminger

In many application there are no timers queued, and the call to rte_timer_managecan be optimized in that case avoid reading HPET and lock overhead.

---
 lib/librte_timer/rte_timer.c |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
lib/librte_timer/rte_timer.c +6 -1
@@ -404,9 +404,14 @@ void rte_timer_manage(void)
 	union rte_timer_status status;
 	struct rte_timer *tim, *tim2;
 	unsigned lcore_id = rte_lcore_id();
-	uint64_t cur_time = rte_get_hpet_cycles();
+	uint64_t cur_time;
 	int ret;
 
+	/* optimize for the case where per-cpu list is empty */
+	if (LIST_EMPTY(&priv_timer[lcore_id].pending))
+		return;
+
+	cur_time = rte_get_hpet_cycles();
 	__TIMER_STAT_ADD(manage, 1);
 
 	/* browse ordered list, add expired timers in 'expired' list */
#11 lore raw

PATCH 3/7 optimize log/panic

In reply to Stephen Hemminger

Both logging and calls to panic are never in the critical path. Use the GCC attribute cold to mark these functions as cold, which generates more optimised code.

---
 lib/librte_eal/common/include/rte_debug.h |    1 +
 lib/librte_eal/common/include/rte_log.h   |    1 +
 2 files changed, 2 insertions(+)
lib/librte_eal/common/include/rte_debug.h +1 -0
@@ -87,6 +87,7 @@ void rte_dump_registers(void);
  * documentation.
  */
 void __rte_panic(const char *funcname , const char *format, ...)
+	__attribute__((cold))
 	__attribute__((noreturn))
 	__attribute__((format(printf, 2, 3)));
 
lib/librte_eal/common/include/rte_log.h +1 -0
@@ -216,6 +216,7 @@ int rte_log_add_in_history(const char *b
  *   - Negative on error.
  */
 int rte_log(uint32_t level, uint32_t logtype, const char *format, ...)
+	__attribute__((cold))
 	__attribute__((format(printf, 3, 4)));
 
 /**
#16 lore raw

PATCH 4/7 eal: support different modules

In reply to Stephen Hemminger

An additional change is that failure to detect module should not be fatal, but an error like other EAL setup problems, allowing the application to recover.

---
 app/chkincs/test_pci.c                  |    3 ++-
 app/test/test_pci.c                     |    4 ++--
 lib/librte_eal/common/include/rte_pci.h |    5 ++---
 lib/librte_eal/linuxapp/eal/eal_pci.c   |    9 ++-------
 lib/librte_pmd_igb/e1000_ethdev.c       |    2 +-
 lib/librte_pmd_ixgbe/ixgbe_ethdev.c     |    4 ++--
 6 files changed, 11 insertions(+), 16 deletions(-)
lib/librte_eal/common/include/rte_pci.h +2 -3
@@ -151,12 +151,11 @@ struct rte_pci_driver {
 	pci_devinit_t *devinit;                 /**< Device init. function. */
 	struct rte_pci_id *id_table;            /**< ID table, NULL terminated. */
 	uint32_t drv_flags;                     /**< Flags contolling handling of device. */
+	const char *module_name;		/**< Associated kernel module */
 };
 
-/** Device needs igb_uio kernel module */
-#define RTE_PCI_DRV_NEED_IGB_UIO 0x0001
 /** Device driver must be registered several times until failure */
-#define RTE_PCI_DRV_MULTIPLE 0x0002
+#define RTE_PCI_DRV_MULTIPLE 0x0001
 
 /**
  * Probe the PCI bus for registered drivers.
lib/librte_eal/linuxapp/eal/eal_pci.c +2 -7
@@ -76,7 +76,7 @@
  * This code is used to simulate a PCI probe by parsing information in
  * sysfs. Moreover, when a registered driver matches a device, the
  * kernel driver currently using it is unloaded and replaced by
- * igb_uio module, which is a very minimal userland driver for Intel
+ * a module, which is a very minimal userland driver for Intel
  * network card, only providing access to PCI BAR to applications, and
  * enabling bus master.
  */
@@ -84,8 +84,6 @@
 
 #define PROC_MODULES "/proc/modules"
 
-#define IGB_UIO_NAME "igb_uio"
-
 #define UIO_NEWID "/sys/bus/pci/drivers/%s/new_id"
 #define UIO_BIND  "/sys/bus/pci/drivers/%s/bind"
 
@@ -700,12 +698,9 @@ int
 rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev)
 {
 	struct rte_pci_id *id_table;
-	const char *module_name = NULL;
+	const char *module_name = dr->module_name;
 	int uio_status = -1;
 
-	if (dr->drv_flags & RTE_PCI_DRV_NEED_IGB_UIO)
-		module_name = IGB_UIO_NAME;
-
 	uio_status = pci_uio_check_module(module_name);
 
 	for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) {
lib/librte_pmd_ixgbe/ixgbe_ethdev.c +2 -2
@@ -547,7 +547,7 @@ static struct eth_driver rte_ixgbe_pmd =
 	{
 		.name = "rte_ixgbe_pmd",
 		.id_table = pci_id_ixgbe_map,
-		.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
+		.module_name = "igb_uio",
 	},
 	.eth_dev_init = eth_ixgbe_dev_init,
 	.dev_private_size = sizeof(struct ixgbe_adapter),
@@ -560,7 +560,7 @@ static struct eth_driver rte_ixgbevf_pmd
 	{
 		.name = "rte_ixgbevf_pmd",
 		.id_table = pci_id_ixgbevf_map,
-		.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
+		.module_name = "igb_uio",
 	},
 	.eth_dev_init = eth_ixgbevf_dev_init,
 	.dev_private_size = sizeof(struct ixgbe_adapter),
app/test/test_pci.c +2 -2
@@ -69,7 +69,7 @@ static int my_driver_init(struct rte_pci
 			  struct rte_pci_device *dev);
 
 /*
- * To test cases where RTE_PCI_DRV_NEED_IGB_UIO is set, and isn't set, two
+ * To test cases where module_name is set, and isn't set, two
  * drivers are created (one with IGB devices, the other with IXGBE devices).
  */
 
@@ -101,7 +101,7 @@ struct rte_pci_driver my_driver = {
 	.name = "test_driver",
 	.devinit = my_driver_init,
 	.id_table = my_driver_id,
-	.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
+	.module_name = "igb_uio",
 };
 
 struct rte_pci_driver my_driver2 = {
app/chkincs/test_pci.c +2 -1
@@ -64,7 +64,8 @@ struct rte_pci_driver my_driver = {
 	"test_driver",
 	my_driver_init,
 	my_driver_id,
-	RTE_PCI_DRV_NEED_IGB_UIO,
+	0,
+	"igb_uio",
 };
 
 static int
lib/librte_pmd_igb/e1000_ethdev.c +1 -1
@@ -336,7 +336,7 @@ static struct eth_driver rte_igb_pmd = {
 	{
 		.name = "rte_igb_pmd",
 		.id_table = pci_id_igb_map,
-		.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
+		.module_name = "igb_uio",
 	},
 	.eth_dev_init = eth_igb_dev_init,
 	.dev_private_size = sizeof(struct e1000_adapter),
#33 lore raw

PATCH 5/7 pci: support multiple PCI regions per device

In reply to Stephen Hemminger
---
 app/test-pmd/config.c                   |    2 
 app/test-pmd/testpmd.h                  |    8 -
 lib/librte_eal/common/eal_common_pci.c  |   11 +
 lib/librte_eal/common/include/rte_pci.h |    7 
 lib/librte_eal/linuxapp/eal/eal_pci.c   |  244 +++++++++++++++++++-------------
 lib/librte_pmd_ixgbe/ixgbe_ethdev.c     |    4 
 6 files changed, 169 insertions(+), 107 deletions(-)
lib/librte_eal/common/eal_common_pci.c +9 -2
@@ -121,12 +121,19 @@ rte_eal_pci_probe(void)
 static int
 pci_dump_one_device(struct rte_pci_device *dev)
 {
+	int i;
+
 	printf(PCI_PRI_FMT, dev->addr.domain, dev->addr.bus,
 	       dev->addr.devid, dev->addr.function);
 	printf(" - vendor:%x device:%x\n", dev->id.vendor_id,
 	       dev->id.device_id);
-	printf("   %16.16"PRIx64" %16.16"PRIx64"\n",
-	       dev->mem_resource.phys_addr, dev->mem_resource.len);
+
+	for (i = 0; i < PCI_MEM_RESOURCE; i++) {
+		printf("   %16.16"PRIx64" %16.16"PRIx64"\n",
+		       dev->mem_resource[i].phys_addr,
+		       dev->mem_resource[i].len);
+	}
+
 	return 0;
 }
 
lib/librte_eal/common/include/rte_pci.h +6 -1
@@ -50,6 +50,7 @@ extern "C" {
 #include <sys/queue.h>
 #include <stdint.h>
 #include <inttypes.h>
+#include <limits.h>
 #include <rte_interrupts.h>
 
 TAILQ_HEAD(pci_device_list, rte_pci_device); /**< PCI devices in D-linked Q. */
@@ -81,6 +82,8 @@ struct rte_pci_resource {
 
 /** Maximum number of PCI resources. */
 #define PCI_MAX_RESOURCE 7
+/** Maximum number of PCI memory resources. */
+#define PCI_MEM_RESOURCE 5
 
 /**
  * A structure describing an ID for a PCI driver. Each driver provides a
@@ -110,10 +113,12 @@ struct rte_pci_device {
 	TAILQ_ENTRY(rte_pci_device) next;       /**< Next probed PCI device. */
 	struct rte_pci_addr addr;               /**< PCI location. */
 	struct rte_pci_id id;                   /**< PCI ID. */
-	struct rte_pci_resource mem_resource;   /**< PCI Memory Resource */
+	struct rte_pci_resource mem_resource[PCI_MEM_RESOURCE];
+						/**< PCI Memory Resource */
 	struct rte_intr_handle intr_handle;     /**< Interrupt handle */
 	const struct rte_pci_driver *driver;    /**< Associated driver */
 	unsigned int blacklisted:1;             /**< Device is blacklisted */
+	char uio_name[PATH_MAX];		/**< Associated UIO device name */
 };
 
 /** Any PCI device identifier (vendor, device, ...) */
lib/librte_eal/linuxapp/eal/eal_pci.c +147 -97

320 lines not on this page · raw message

lib/librte_pmd_ixgbe/ixgbe_ethdev.c +2 -2
@@ -369,7 +369,7 @@ eth_ixgbe_dev_init(__attribute__((unused
 	/* Vendor and Device ID need to be set before init of shared code */
 	hw->device_id = pci_dev->id.device_id;
 	hw->vendor_id = pci_dev->id.vendor_id;
-	hw->hw_addr = (void *)pci_dev->mem_resource.addr;
+	hw->hw_addr = pci_dev->mem_resource[0].addr;
 
 	/* Initialize the shared code */
 	diag = ixgbe_init_shared_code(hw);
@@ -490,7 +490,7 @@ eth_ixgbevf_dev_init(__attribute__((unus
 
 	hw->device_id = pci_dev->id.device_id;
 	hw->vendor_id = pci_dev->id.vendor_id;
-	hw->hw_addr = (void *)pci_dev->mem_resource.addr;
+	hw->hw_addr = pci_dev->mem_resource[0].addr;
 
 	/* Initialize the shared code */
 	diag = ixgbe_init_shared_code(hw);
app/test-pmd/config.c +1 -1
@@ -180,7 +180,7 @@ port_reg_off_is_invalid(portid_t port_id
 		       (unsigned)reg_off);
 		return 1;
 	}
-	pci_len = ports[port_id].dev_info.pci_dev->mem_resource.len;
+	pci_len = ports[port_id].dev_info.pci_dev->mem_resource[0].len;
 	if (reg_off >= pci_len) {
 		printf("Port %d: register offset %u (0x%X) out of port PCI "
 		       "resource (length=%"PRIu64")\n",
app/test-pmd/testpmd.h +4 -4
@@ -304,8 +304,8 @@ port_pci_reg_read(struct rte_port *port,
 	void *reg_addr;
 	uint32_t reg_v;
 
-	reg_addr = (void *)((char *)port->dev_info.pci_dev->mem_resource.addr +
-			    reg_off);
+	reg_addr = (char *)port->dev_info.pci_dev->mem_resource[0].addr
+		+ reg_off;
 	reg_v = *((volatile uint32_t *)reg_addr);
 	return rte_le_to_cpu_32(reg_v);
 }
@@ -318,8 +318,8 @@ port_pci_reg_write(struct rte_port *port
 {
 	void *reg_addr;
 
-	reg_addr = (void *)((char *)port->dev_info.pci_dev->mem_resource.addr +
-			    reg_off);
+	reg_addr = (char *)port->dev_info.pci_dev->mem_resource[0].addr
+		    + reg_off;
 	*((volatile uint32_t *)reg_addr) = rte_cpu_to_le_32(reg_v);
 }
 
#49 lore raw

PATCH 6/7 igb_uio: pci_block_user_cfg_access is unsafe, remove it

In reply to Stephen Hemminger

Using pci_block_user_cfg_access in IRQ context is unsafe. In fact if kernel is compiled with option to check for might_sleep() it causes a warning and a backtrace. The problem Intel was trying to solve here can not be solved with these functions; better to just remove them.

lib/librte_eal/linuxapp/igb_uio/igb_uio.c +0 -17
@@ -28,15 +28,6 @@
 #include <linux/msi.h>
 #include <linux/version.h>
 
-/* Some function names changes between 3.2.0 and 3.3.0... */
-#if LINUX_VERSION_CODE < KERNEL_VERSION(3,3,0)
-#define PCI_LOCK pci_block_user_cfg_access
-#define PCI_UNLOCK pci_unblock_user_cfg_access
-#else
-#define PCI_LOCK pci_cfg_access_lock
-#define PCI_UNLOCK pci_cfg_access_unlock
-#endif
-
 /**
  * MSI-X related macros, copy from linux/pci_regs.h in kernel 2.6.39,
  * but none of them in kernel 2.6.35.
@@ -170,14 +161,9 @@ igbuio_pci_irqcontrol(struct uio_info *i
 {
 	unsigned long flags;
 	struct rte_uio_pci_dev *udev = igbuio_get_uio_pci_dev(info);
-	struct pci_dev *pdev = udev->pdev;
 
 	spin_lock_irqsave(&udev->lock, flags);
-	PCI_LOCK(pdev);
-
 	igbuio_set_interrupt_mask(udev, irq_state);
-
-	PCI_UNLOCK(pdev);
 	spin_unlock_irqrestore(&udev->lock, flags);
 
 	return 0;
@@ -198,8 +184,6 @@ igbuio_pci_irqhandler(int irq, struct ui
 	uint16_t status;
 
 	spin_lock_irqsave(&udev->lock, flags);
-	/* block userspace PCI config reads/writes */
-	PCI_LOCK(pdev);
 
 	/* for legacy mode, interrupt maybe shared */
 	if (udev->mode == IGBUIO_LEGACY_INTR_MODE) {
@@ -214,7 +198,6 @@ igbuio_pci_irqhandler(int irq, struct ui
 	ret = IRQ_HANDLED;
 done:
 	/* unblock userspace PCI config reads/writes */
-	PCI_UNLOCK(pdev);
 	spin_unlock_irqrestore(&udev->lock, flags);
 	printk(KERN_INFO "irq 0x%x %s\n", irq, (ret == IRQ_HANDLED) ? "handled" : "not handled");
 
#50 lore raw

PATCH 7/7 eal: add ability to override DPDK syslog parameters

In reply to Stephen Hemminger

By default, DPDK based applications would only allow logging to syslog as "rte", DAEMON; but for any production application more control is desired to allow using actual application name and overriding the facility.

---
 lib/librte_eal/common/include/eal_private.h            |    2 
 lib/librte_eal/linuxapp/eal/eal.c                      |   58 ++++++++++++++++-
 lib/librte_eal/linuxapp/eal/eal_log.c                  |    4 -
 lib/librte_eal/linuxapp/eal/include/eal_internal_cfg.h |    1 
 4 files changed, 61 insertions(+), 4 deletions(-)
lib/librte_eal/common/include/eal_private.h +1 -1
@@ -115,7 +115,7 @@ int rte_eal_log_early_init(void);
  * @return
  *   0 on success, negative on error
  */
-int rte_eal_log_init(void);
+int rte_eal_log_init(const char *id, int facility);
 
 /**
  * Init the default log stream
lib/librte_eal/linuxapp/eal/eal.c +57 -1

112 lines not on this page · raw message

lib/librte_eal/linuxapp/eal/eal_log.c +2 -2
@@ -119,7 +119,7 @@ static cookie_io_functions_t console_log
  * once memzones are available.
  */
 int
-rte_eal_log_init(void)
+rte_eal_log_init(const char *id, int facility)
 {
 	FILE *log_stream;
 
@@ -127,7 +127,7 @@ rte_eal_log_init(void)
 	if (log_stream == NULL)
 		return -1;
 
-	openlog("rte", LOG_NDELAY | LOG_PID, LOG_DAEMON);
+	openlog(id, LOG_NDELAY | LOG_PID, facility);
 
 	if (rte_eal_common_log_init(log_stream) < 0)
 		return -1;
lib/librte_eal/linuxapp/eal/include/eal_internal_cfg.h +1 -0
@@ -65,6 +65,7 @@ struct internal_config {
 	volatile unsigned no_hpet;        /* true to disable HPET */
 	volatile unsigned vmware_tsc_map; /* true to use VMware TSC mapping instead of native TSC */
 	volatile unsigned no_shconf;      /* true if there is no shared config */
+	volatile int syslog_facility;	  /* facility passed to openlog() */
 	volatile enum rte_proc_type_t process_type; /* multi-process proc type */
 	const char *hugefile_prefix;      /* the base filename of hugetlbfs files */
 	const char *hugepage_dir;         /* specific hugetlbfs directory to use */
#56 lore raw
In reply to a message not in the archive

Hello,

30/05/2013 10:12, Stephen Hemminger :

Quoted 2 lines

This is a subset of the Vyatta patches we use. They include some bug fixes and simple changes to make life easier.

Thank you

Quoted 8 lines

One fix not included is the NUMA cpu assignment. The original Intel code in 1.2 incorrectly used /proc/cpuinfo to try and assign CPU's to NUMA socket. The problem is that /proc/cpuinfo physical_id corresponds to what the BIOS tells the kernel and is intended for messages only. For example, on our Dell boxes the first CPU and only CPU is reported as physical_id 1! The fix is to use sysfs instead, and Intel did incorporate my fix in the next DPDK version, and don't want to confuse this code base by putting in a conflicting change.

If I understand well, Intel could release this patch among other ones and they don't want to handle this patch separately.
Maybe we could apply it now but if it's not urgent and if Intel plan to release it soon, we may wait for the Intel drop.

Signature · 1 line
Thomas
#57 lore raw
In reply to a message not in the archive
Quoted 6 lines · Stephen Hemminger

One fix not included is the NUMA cpu assignment. The original Intel code in 1.2 incorrectly used /proc/cpuinfo to try and assign CPU's to NUMA socket. The problem is that /proc/cpuinfo physical_id corresponds to what the BIOS tells the kernel and is intended for messages only. For example, on our Dell boxes the first CPU and only CPU is reported as physical_id 1! The fix is to use sysfs instead,

Hi Stephen,

Are you using the /sys/devices/system/node/nodeX/cpuX/topology/physical_package_id special file ?

If so, then it is only usable starting from Kernel 3.3 according to this fix:
http://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/?id=64be4c1c2428e148de6081af235e2418e6a66dda

The value returned for kernel prior to 3.3 is really not better than the one from /proc/cpuinfo, so that's kind of a robbing Peter to pay Paul example...

There is obviously a problem with the NUMA node ID detection right now since both /proc/cpuinfo and /sys can return incorrect values, however the node and cpu numbering is always good in the kernel boot log from what I know. So there might be a better way to find the real node ID whatever the kernel version used.

Cheers,

Signature · 1 line
Damien Millescamps
#59 lore raw
In reply to a message not in the archive
Quoted 21 lines · On Fri, 31 May 2013 11:29:09 +0200 Damien Millescamps…
Quoted 6 lines · Stephen Hemminger

One fix not included is the NUMA cpu assignment. The original Intel code in 1.2 incorrectly used /proc/cpuinfo to try and assign CPU's to NUMA socket. The problem is that /proc/cpuinfo physical_id corresponds to what the BIOS tells the kernel and is intended for messages only. For example, on our Dell boxes the first CPU and only CPU is reported as physical_id 1! The fix is to use sysfs instead,

Hi Stephen,

Are you using the /sys/devices/system/node/nodeX/cpuX/topology/physical_package_id special file ?

If so, then it is only usable starting from Kernel 3.3 according to this fix:
http://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/?id=64be4c1c2428e148de6081af235e2418e6a66dda

The value returned for kernel prior to 3.3 is really not better than the one from /proc/cpuinfo, so that's kind of a robbing Peter to pay Paul example...

Quoted 7 lines

There is obviously a problem with the NUMA node ID detection right now since both /proc/cpuinfo and /sys can return incorrect values, however the node and cpu numbering is always good in the kernel boot log from what I know. So there might be a better way to find the real node ID whatever the kernel version used.

Cheers,

You need to use /sys/devices/system/cpu/cpuN/topology/physical_package_id as documented in kernel (Documentation/cputopology.txt).
This was confirmed by several kernel developers including Andi Kleen from Intel. The value in /proc/cpuinfo comes from the APCI tables and is the value reported by the BIOS. There are machines that report socket 1 and 2. Haven't played with older kernels, but the topology information in sysfs has existed since 2.6.16.

The final Intel solution was to use physical_packate_id and fall back to /proc/cpuinfo as last resort.

#61 lore raw
In reply to a message not in the archive
Quoted 5 lines · Stephen Hemminger

This was confirmed by several kernel developers including Andi Kleen from Intel. The value in /proc/cpuinfo comes from the APCI tables and is the value reported by the BIOS. There are machines that report socket 1 and 2. Haven't played with older kernels, but the topology information in sysfs has existed since 2.6.16.

Hi Stephen,

You didn't really address my point, because the Linux documentation says something doesn't mean it works in the real world.

If you take the example of the hugepages, they have been in the Linux kernel since 2.6.28, but the last fix was pushed in 2.6.37 and before 2.6.33 it is just impossible to do something with hugepages. If you take a look at section 2.3.2 of the DPDK getting started:
http://www.intel.com/content/dam/www/public/us/en/documents/guides/intel-dpdk-getting-started-guide.pdf,
you will find that this issue is at least partially addressed.

So the point is that the topology information being theoretically correct on all kernel Linux supported by the DPDK, but the information being practically wrong on versions prior to kernel 3.3 means that the "fix" is breaking the support for a quite huge range of kernel versions... So maybe there might be a better way to get this node information on all kernel still currently supported by the DPDK.

Cheers,

Signature · 1 line
Damien Millescamps
#63 lore raw
In reply to a message not in the archive
Quoted 18 lines · On Fri, 31 May 2013 18:44:03 +0200 Damien Millescamps…
Quoted 5 lines · Stephen Hemminger

This was confirmed by several kernel developers including Andi Kleen from Intel. The value in /proc/cpuinfo comes from the APCI tables and is the value reported by the BIOS. There are machines that report socket 1 and 2. Haven't played with older kernels, but the topology information in sysfs has existed since 2.6.16.

Hi Stephen,

You didn't really address my point, because the Linux documentation says something doesn't mean it works in the real world.

If you take the example of the hugepages, they have been in the Linux kernel since 2.6.28, but the last fix was pushed in 2.6.37 and before 2.6.33 it is just impossible to do something with hugepages. If you take a look at section 2.3.2 of the DPDK getting started:
http://www.intel.com/content/dam/www/public/us/en/documents/guides/intel-dpdk-getting-started-guide.pdf,
you will find that this issue is at least partially addressed.

Quoted 8 lines

So the point is that the topology information being theoretically correct on all kernel Linux supported by the DPDK, but the information being practically wrong on versions prior to kernel 3.3 means that the "fix" is breaking the support for a quite huge range of kernel versions... So maybe there might be a better way to get this node information on all kernel still currently supported by the DPDK.

Cheers,

I will just state what we saw. /proc/cpuinfo was wrong on several Dell machines but sysfs is correct. And since are working on a product, not generic DPDK, there is no motivation to work on older kernels.

I can dig out the email back and forth with the kernel developers to show that /proc/cpuinfo is not intended to report anything useful for NUMA related API if you want.

#18 lore raw

Re: [PATCH 4/7] eal: support different modules

In reply to a message not in the archive

comments inlined.

Quoted 18 lines · Stephen Hemminger

An additional change is that failure to detect module should not be fatal, but an error like other EAL setup problems, allowing the application to recover.

---
 app/chkincs/test_pci.c                  |    3 ++-
 app/test/test_pci.c                     |    4 ++--
 lib/librte_eal/common/include/rte_pci.h |    5 ++---
 lib/librte_eal/linuxapp/eal/eal_pci.c   |    9 ++-------
 lib/librte_pmd_igb/e1000_ethdev.c       |    2 +-
 lib/librte_pmd_ixgbe/ixgbe_ethdev.c     |    4 ++--
 6 files changed, 11 insertions(+), 16 deletions(-)

lib/librte_eal/common/include/rte_pci.h struct rte_pci_driver { · line 154

Quoted 4 lines
lib/librte_eal/common/include/rte_pci.h +1 -0
@@ -151,12 +151,11 @@ struct rte_pci_driver {
 	pci_devinit_t *devinit;                 /**< Device init. function. */
 	struct rte_pci_id *id_table;            /**< ID table, NULL terminated. */
 	uint32_t drv_flags;                     /**< Flags contolling handling of device. */
+	const char *module_name;		/**< Associated kernel module */

You solution only permits for one module to be checked during initialization, while the former solution using flags could be easily extended to check for more than one module.
However it is true that there is a problem with this module check since it is historically hard-coded for "igb_uio".

Some better solution could be to have a new flag that will check for a NULL terminated array of module_name.

lib/librte_eal/common/include/rte_pci.h struct rte_pci_driver { · line 158

Quoted 7 lines
lib/librte_eal/common/include/rte_pci.h +1 -3
 };
 
-/** Device needs igb_uio kernel module */
-#define RTE_PCI_DRV_NEED_IGB_UIO 0x0001
 /** Device driver must be registered several times until failure */
-#define RTE_PCI_DRV_MULTIPLE 0x0002
+#define RTE_PCI_DRV_MULTIPLE 0x0001

You are breaking a public API here, and I don't see any technical reason to do so. The RTE_PCI_DRV_NEED_IGB_UIO flag could be deprecated, but there is no way its value could be recycled into an already existing flag.

Quoted 102 lines
lib/librte_eal/common/include/rte_pci.h +0 -0
 /**
  * Probe the PCI bus for registered drivers.
lib/librte_eal/linuxapp/eal/eal_pci.c +2 -7
@@ -76,7 +76,7 @@
  * This code is used to simulate a PCI probe by parsing information in
  * sysfs. Moreover, when a registered driver matches a device, the
  * kernel driver currently using it is unloaded and replaced by
- * igb_uio module, which is a very minimal userland driver for Intel
+ * a module, which is a very minimal userland driver for Intel
  * network card, only providing access to PCI BAR to applications, and
  * enabling bus master.
  */
@@ -84,8 +84,6 @@
 
 #define PROC_MODULES "/proc/modules"
 
-#define IGB_UIO_NAME "igb_uio"
-
 #define UIO_NEWID "/sys/bus/pci/drivers/%s/new_id"
 #define UIO_BIND  "/sys/bus/pci/drivers/%s/bind"
 
@@ -700,12 +698,9 @@ int
 rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev)
 {
 	struct rte_pci_id *id_table;
-	const char *module_name = NULL;
+	const char *module_name = dr->module_name;
 	int uio_status = -1;
 
-	if (dr->drv_flags & RTE_PCI_DRV_NEED_IGB_UIO)
-		module_name = IGB_UIO_NAME;
-
 	uio_status = pci_uio_check_module(module_name);
 
 	for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) {
lib/librte_pmd_ixgbe/ixgbe_ethdev.c +2 -2
@@ -547,7 +547,7 @@ static struct eth_driver rte_ixgbe_pmd =
 	{
 		.name = "rte_ixgbe_pmd",
 		.id_table = pci_id_ixgbe_map,
-		.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
+		.module_name = "igb_uio",
 	},
 	.eth_dev_init = eth_ixgbe_dev_init,
 	.dev_private_size = sizeof(struct ixgbe_adapter),
@@ -560,7 +560,7 @@ static struct eth_driver rte_ixgbevf_pmd
 	{
 		.name = "rte_ixgbevf_pmd",
 		.id_table = pci_id_ixgbevf_map,
-		.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
+		.module_name = "igb_uio",
 	},
 	.eth_dev_init = eth_ixgbevf_dev_init,
 	.dev_private_size = sizeof(struct ixgbe_adapter),
app/test/test_pci.c +2 -2
@@ -69,7 +69,7 @@ static int my_driver_init(struct rte_pci
 			  struct rte_pci_device *dev);
 
 /*
- * To test cases where RTE_PCI_DRV_NEED_IGB_UIO is set, and isn't set, two
+ * To test cases where module_name is set, and isn't set, two
  * drivers are created (one with IGB devices, the other with IXGBE devices).
  */
 
@@ -101,7 +101,7 @@ struct rte_pci_driver my_driver = {
 	.name = "test_driver",
 	.devinit = my_driver_init,
 	.id_table = my_driver_id,
-	.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
+	.module_name = "igb_uio",
 };
 
 struct rte_pci_driver my_driver2 = {
app/chkincs/test_pci.c +2 -1
@@ -64,7 +64,8 @@ struct rte_pci_driver my_driver = {
 	"test_driver",
 	my_driver_init,
 	my_driver_id,
-	RTE_PCI_DRV_NEED_IGB_UIO,
+	0,
+	"igb_uio",
 };
 
 static int
lib/librte_pmd_igb/e1000_ethdev.c +1 -1
@@ -336,7 +336,7 @@ static struct eth_driver rte_igb_pmd = {
 	{
 		.name = "rte_igb_pmd",
 		.id_table = pci_id_igb_map,
-		.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
+		.module_name = "igb_uio",
 	},
 	.eth_dev_init = eth_igb_dev_init,
 	.dev_private_size = sizeof(struct e1000_adapter),
#64 lore raw
In reply to a message not in the archive

30/05/2013 19:12, Stephen Hemminger :

Quoted 2 lines

This is a subset of the Vyatta patches we use. They include some bug fixes and simple changes to make life easier.

It would be nice to have these 7 patches in the next release (dpdk-1.2.3r3). Reviewers are welcomed !

Signature · 1 line
Thomas
#20 lore raw

Re: [PATCH 4/7] eal: support different modules

In reply to a message not in the archive
Quoted 134 lines · On Mon, 03 Jun 2013 10:58:01 +0200 Damien Millescamps…

comments inlined.

Quoted 18 lines · Stephen Hemminger

An additional change is that failure to detect module should not be fatal, but an error like other EAL setup problems, allowing the application to recover.

---
 app/chkincs/test_pci.c                  |    3 ++-
 app/test/test_pci.c                     |    4 ++--
 lib/librte_eal/common/include/rte_pci.h |    5 ++---
 lib/librte_eal/linuxapp/eal/eal_pci.c   |    9 ++-------
 lib/librte_pmd_igb/e1000_ethdev.c       |    2 +-
 lib/librte_pmd_ixgbe/ixgbe_ethdev.c     |    4 ++--
 6 files changed, 11 insertions(+), 16 deletions(-)

lib/librte_eal/common/include/rte_pci.h struct rte_pci_driver { · line 154

Quoted 4 lines
lib/librte_eal/common/include/rte_pci.h +1 -0
@@ -151,12 +151,11 @@ struct rte_pci_driver {
 	pci_devinit_t *devinit;                 /**< Device init. function. */
 	struct rte_pci_id *id_table;            /**< ID table, NULL terminated. */
 	uint32_t drv_flags;                     /**< Flags contolling handling of device. */
+	const char *module_name;		/**< Associated kernel module */

You solution only permits for one module to be checked during initialization, while the former solution using flags could be easily extended to check for more than one module.
However it is true that there is a problem with this module check since it is historically hard-coded for "igb_uio".

Some better solution could be to have a new flag that will check for a NULL terminated array of module_name.

lib/librte_eal/common/include/rte_pci.h struct rte_pci_driver { · line 158

Quoted 7 lines
lib/librte_eal/common/include/rte_pci.h +1 -3
 };
 
-/** Device needs igb_uio kernel module */
-#define RTE_PCI_DRV_NEED_IGB_UIO 0x0001
 /** Device driver must be registered several times until failure */
-#define RTE_PCI_DRV_MULTIPLE 0x0002
+#define RTE_PCI_DRV_MULTIPLE 0x0001

You are breaking a public API here, and I don't see any technical reason to do so. The RTE_PCI_DRV_NEED_IGB_UIO flag could be deprecated, but there is no way its value could be recycled into an already existing flag.

Quoted 88 lines
lib/librte_eal/common/include/rte_pci.h +0 -0
 /**
  * Probe the PCI bus for registered drivers.
lib/librte_eal/linuxapp/eal/eal_pci.c +2 -7
@@ -76,7 +76,7 @@
  * This code is used to simulate a PCI probe by parsing information in
  * sysfs. Moreover, when a registered driver matches a device, the
  * kernel driver currently using it is unloaded and replaced by
- * igb_uio module, which is a very minimal userland driver for Intel
+ * a module, which is a very minimal userland driver for Intel
  * network card, only providing access to PCI BAR to applications, and
  * enabling bus master.
  */
@@ -84,8 +84,6 @@
 
 #define PROC_MODULES "/proc/modules"
 
-#define IGB_UIO_NAME "igb_uio"
-
 #define UIO_NEWID "/sys/bus/pci/drivers/%s/new_id"
 #define UIO_BIND  "/sys/bus/pci/drivers/%s/bind"
 
@@ -700,12 +698,9 @@ int
 rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev)
 {
 	struct rte_pci_id *id_table;
-	const char *module_name = NULL;
+	const char *module_name = dr->module_name;
 	int uio_status = -1;
 
-	if (dr->drv_flags & RTE_PCI_DRV_NEED_IGB_UIO)
-		module_name = IGB_UIO_NAME;
-
 	uio_status = pci_uio_check_module(module_name);
 
 	for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) {
lib/librte_pmd_ixgbe/ixgbe_ethdev.c +2 -2
@@ -547,7 +547,7 @@ static struct eth_driver rte_ixgbe_pmd =
 	{
 		.name = "rte_ixgbe_pmd",
 		.id_table = pci_id_ixgbe_map,
-		.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
+		.module_name = "igb_uio",
 	},
 	.eth_dev_init = eth_ixgbe_dev_init,
 	.dev_private_size = sizeof(struct ixgbe_adapter),
@@ -560,7 +560,7 @@ static struct eth_driver rte_ixgbevf_pmd
 	{
 		.name = "rte_ixgbevf_pmd",
 		.id_table = pci_id_ixgbevf_map,
-		.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
+		.module_name = "igb_uio",
 	},
 	.eth_dev_init = eth_ixgbevf_dev_init,
 	.dev_private_size = sizeof(struct ixgbe_adapter),
app/test/test_pci.c +2 -2
@@ -69,7 +69,7 @@ static int my_driver_init(struct rte_pci
 			  struct rte_pci_device *dev);
 
 /*
- * To test cases where RTE_PCI_DRV_NEED_IGB_UIO is set, and isn't set, two
+ * To test cases where module_name is set, and isn't set, two
  * drivers are created (one with IGB devices, the other with IXGBE devices).
  */
 
@@ -101,7 +101,7 @@ struct rte_pci_driver my_driver = {
 	.name = "test_driver",
 	.devinit = my_driver_init,
 	.id_table = my_driver_id,
-	.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
+	.module_name = "igb_uio",
 };
 
 struct rte_pci_driver my_driver2 = {
app/chkincs/test_pci.c +2 -1
@@ -64,7 +64,8 @@ struct rte_pci_driver my_driver = {
 	"test_driver",
 	my_driver_init,
 	my_driver_id,
-	RTE_PCI_DRV_NEED_IGB_UIO,
+	0,
+	"igb_uio",
 };
 
Quoted 15 lines
Quoted 6 lines
app/chkincs/test_pci.c +0 -0
 static int

lib/librte_pmd_igb/e1000_ethdev.c static struct eth_driver rte_igb_pmd = { · line 342

Quoted 8 lines
lib/librte_pmd_igb/e1000_ethdev.c +1 -1
@@ -336,7 +336,7 @@ static struct eth_driver rte_igb_pmd = {
 	{
 		.name = "rte_igb_pmd",
 		.id_table = pci_id_igb_map,
-		.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
+		.module_name = "igb_uio",
 	},
 	.eth_dev_init = eth_igb_dev_init,
 	.dev_private_size = sizeof(struct e1000_adapter),

It is one module per driver which works quite well. The code identifies the module to load based on the PCI id.

#27 lore raw

Re: [PATCH 4/7] eal: support different modules

In reply to a message not in the archive
Quoted 10 lines · Damien Millescamps

lib/librte_eal/common/include/rte_pci.h struct rte_pci_driver { · line 158

Quoted 7 lines
  };
lib/librte_eal/common/include/rte_pci.h +1 -3
-/** Device needs igb_uio kernel module */
-#define RTE_PCI_DRV_NEED_IGB_UIO 0x0001
  /** Device driver must be registered several times until failure */
-#define RTE_PCI_DRV_MULTIPLE 0x0002
+#define RTE_PCI_DRV_MULTIPLE 0x0001

You are breaking a public API here, and I don't see any technical reason to do so. The RTE_PCI_DRV_NEED_IGB_UIO flag could be deprecated, but there is no way its value could be recycled into an already existing flag.

Is breaking the API a bad thing in this context? IMHO the initialization APIs need work before they're general enough and perpetually supporting the current ones seems like an unnecessary burden. I'm trying to understand the general guidelines of the project.

(and nittily, recycling flag values is fine for static-only libs as long as you remove the old macro, but of course removal is the API breakage you mentioned)

#29 lore raw

Re: [PATCH 4/7] eal: support different modules

In reply to a message not in the archive

03/06/2013 18:08, Antti Kantee :

Quoted 14 lines · Damien Millescamps
Quoted 5 lines
Quoted 5 lines
lib/librte_eal/common/include/rte_pci.h +0 -2
-/** Device needs igb_uio kernel module */
-#define RTE_PCI_DRV_NEED_IGB_UIO 0x0001
 
  /** Device driver must be registered several times until failure */
Quoted 7 lines

lib/librte_eal/common/include/rte_pci.h struct rte_pci_driver { · line 158

Quoted 2 lines
lib/librte_eal/common/include/rte_pci.h +1 -1
-#define RTE_PCI_DRV_MULTIPLE 0x0002
+#define RTE_PCI_DRV_MULTIPLE 0x0001

You are breaking a public API here, and I don't see any technical reason to do so. The RTE_PCI_DRV_NEED_IGB_UIO flag could be deprecated, but there is no way its value could be recycled into an already existing flag.

Quoted 8 lines

Is breaking the API a bad thing in this context? IMHO the initialization APIs need work before they're general enough and perpetually supporting the current ones seems like an unnecessary burden. I'm trying to understand the general guidelines of the project.

(and nittily, recycling flag values is fine for static-only libs as long as you remove the old macro, but of course removal is the API breakage you mentioned)

Yes, DPDK is a young project but breaking API should be always justified. In this case it is not mandatory to change it.

Signature · 1 line
Thomas
#22 lore raw

Re: [PATCH 4/7] eal: support different modules

In reply to a message not in the archive

03/06/2013 17:41, Stephen Hemminger :

Quoted 21 lines
Quoted 11 lines · On Mon, 03 Jun 2013 10:58:01 +0200 Damien Millescamps…
Quoted 11 lines

--- a/lib/librte_eal/common/include/rte_pci.h 2013-05-29
08:45:38.000000000 -0700 +++
b/lib/librte_eal/common/include/rte_pci.h 2013-05-29
09:02:50.000000000 -0700 @@ -151,12 +151,11 @@ struct rte_pci_driver {

(quoted diff) +0 -0
 	pci_devinit_t *devinit;                 /**< Device init. function.
 	*/ struct rte_pci_id *id_table;            /**< ID table, NULL
 	terminated. */ uint32_t drv_flags;                     /**< Flags
 	contolling handling of device. */
Quoted 7 lines

lib/librte_eal/common/include/rte_pci.h struct rte_pci_driver { · line 154

Quoted 1 line
lib/librte_eal/common/include/rte_pci.h +1 -0
+	const char *module_name;		/**< Associated kernel module */

You solution only permits for one module to be checked during initialization, while the former solution using flags could be easily extended to check for more than one module.
However it is true that there is a problem with this module check since it is historically hard-coded for "igb_uio".

Quoted 2 lines

It is one module per driver which works quite well. The code identifies the module to load based on the PCI id.

There are cases where we need more than one kernel module for 2 reasons:
- in this case, igb_uio is built and inserted without dependency on uio
- some other PMDs/NICs, several modules without dependency links could be
needed

Please could you refactor your patch accordingly ? Thanks

Signature · 1 line
Thomas
#31 lore raw

Re: [PATCH 4/7] eal: support different modules

In reply to a message not in the archive
Quoted 20 lines · On Mon, 3 Jun 2013 18:29:02 +0200 Thomas Monjalon…

03/06/2013 18:08, Antti Kantee :

Quoted 19 lines · Damien Millescamps
Quoted 5 lines
Quoted 5 lines
(quoted diff) +0 -2
-/** Device needs igb_uio kernel module */
-#define RTE_PCI_DRV_NEED_IGB_UIO 0x0001
 
  /** Device driver must be registered several times until failure */
Quoted 7 lines

Review

Quoted 2 lines
(quoted diff) +1 -1
-#define RTE_PCI_DRV_MULTIPLE 0x0002
+#define RTE_PCI_DRV_MULTIPLE 0x0001

You are breaking a public API here, and I don't see any technical reason to do so. The RTE_PCI_DRV_NEED_IGB_UIO flag could be deprecated, but there is no way its value could be recycled into an already existing flag.

Is breaking the API a bad thing in this context? IMHO the initialization APIs need work before they're general enough and perpetually supporting the current ones seems like an unnecessary burden. I'm trying to understand the general guidelines of the project.

Quoted 7 lines
Quoted 3 lines

(and nittily, recycling flag values is fine for static-only libs as long as you remove the old macro, but of course removal is the API breakage you mentioned)

Yes, DPDK is a young project but breaking API should be always justified. In this case it is not mandatory to change it.

This is a source project, there is no fixed ABI.

#24 lore raw

Re: [PATCH 4/7] eal: support different modules

In reply to a message not in the archive
Quoted 25 lines · On Mon, 3 Jun 2013 18:36:52 +0200 Thomas Monjalon…

03/06/2013 17:41, Stephen Hemminger :

Quoted 21 lines
Quoted 11 lines · On Mon, 03 Jun 2013 10:58:01 +0200 Damien Millescamps…
Quoted 11 lines

--- a/lib/librte_eal/common/include/rte_pci.h 2013-05-29
08:45:38.000000000 -0700 +++
b/lib/librte_eal/common/include/rte_pci.h 2013-05-29
09:02:50.000000000 -0700 @@ -151,12 +151,11 @@ struct rte_pci_driver {

(quoted diff) +0 -0
 	pci_devinit_t *devinit;                 /**< Device init. function.
 	*/ struct rte_pci_id *id_table;            /**< ID table, NULL
 	terminated. */ uint32_t drv_flags;                     /**< Flags
 	contolling handling of device. */
Quoted 7 lines

Review

Quoted 1 line
(quoted diff) +1 -0
+	const char *module_name;		/**< Associated kernel module */

You solution only permits for one module to be checked during initialization, while the former solution using flags could be easily extended to check for more than one module.
However it is true that there is a problem with this module check since it is historically hard-coded for "igb_uio".

Quoted 2 lines

It is one module per driver which works quite well. The code identifies the module to load based on the PCI id.

Quoted 7 lines

There are cases where we need more than one kernel module for 2 reasons:
- in this case, igb_uio is built and inserted without dependency on uio
- some other PMDs/NICs, several modules without dependency links could be
needed

Please could you refactor your patch accordingly ? Thanks

There is nothing that says a PMD could load a second module, but why not put that special case code in the the driver. Or make it a list of strings.

#32 lore raw

Re: [PATCH 4/7] eal: support different modules

In reply to a message not in the archive
Quoted 11 lines · Thomas Monjalon
Quoted 8 lines

Is breaking the API a bad thing in this context? IMHO the initialization APIs need work before they're general enough and perpetually supporting the current ones seems like an unnecessary burden. I'm trying to understand the general guidelines of the project.

(and nittily, recycling flag values is fine for static-only libs as long as you remove the old macro, but of course removal is the API breakage you mentioned)

Yes, DPDK is a young project but breaking API should be always justified. In this case it is not mandatory to change it.

Ok, I was writing with the premise that Stephen's patch would be accepted.

#26 lore raw

Re: [PATCH 4/7] eal: support different modules

In reply to a message not in the archive
Quoted 2 lines · Stephen Hemminger

There is nothing that says a PMD could load a second module, but why not put that special case code in the the driver. Or make it a list of strings.

There is nothing that says anything about PMD and loading a single module. However, the Getting Started Guide refers to "dymanic kernel tweaks" citing "modules", all in a plural form:
/"Running an Intel®DPDK application requires some kernel configuration customization (done at build time) and some dynamic kernel tweaks (modules, procfs)./"

A list of strings can be implemented using a "null terminated array", as suggested in my review. So yes, it could be a good solution.

Signature · 1 line
Damien Millescamps
Reviewed-by
#3 lore raw

Re: [PATCH 1/7] [PATCH 4/8] igb: workaround errata with wthresh on 82576

In reply to a message not in the archive
Quoted 8 lines · Stephen Hemminger

The 82576 has known issues which require the write threshold to be set to 1. See:
http://download.intel.com/design/network/specupdt/82576_SPECUPDATE.pdf

If not then single packets will hang in transmit ring until more arrive. Simple tests like ping will fail.

It looks good. Thank you.

Reviewed-by
#13 lore raw

Re: [PATCH 3/7] optimize log/panic

In reply to a message not in the archive
Quoted 5 lines · Stephen Hemminger

Both logging and calls to panic are never in the critical path. Use the GCC attribute cold to mark these functions as cold, which generates more optimised code.

It does not hurt to move both to a cold section. TODO: some other init and setup functions should be cold too.

Reviewed-by
#52 lore raw

Re: [PATCH 7/7] eal: add ability to override DPDK syslog parameters

In reply to a message not in the archive
Quoted 6 lines · Stephen Hemminger

By default, DPDK based applications would only allow logging to syslog as "rte", DAEMON; but for any production application more control is desired to allow using actual application name and overriding the facility.

#36 lore raw

Re: [PATCH 5/7] pci: support multiple PCI regions per device

In reply to a message not in the archive

Hi Stephen,

Overall this patch is very nice. My only comment on this one is why do you limit the max number of memory resources to 5 ?
The PCI configuration space permits to store up to 6 base addresses.

Quoted 1 line

+#define PCI_MEM_RESOURCE 5

Please, can you add a log/comment with your patch, too ?

Cheers,

Signature · 1 line
Damien Millescamps
#38 lore raw

Re: [PATCH 5/7] pci: support multiple PCI regions per device

In reply to a message not in the archive
Quoted 6 lines · On Wed, 05 Jun 2013 16:50:03 +0200 Damien Millescamps…

Hi Stephen,

Overall this patch is very nice. My only comment on this one is why do you limit the max number of memory resources to 5 ?
The PCI configuration space permits to store up to 6 base addresses.

Quoted 6 lines
Quoted 1 line

+#define PCI_MEM_RESOURCE 5

Please, can you add a log/comment with your patch, too ?

Cheers,

Only because I was trying to save some space, and I didn't see any hardware with that many useful regions. Also the kernel UIO driver has some control over which regions get exposed.

#40 lore raw

Re: [PATCH 5/7] pci: support multiple PCI regions per device

In reply to a message not in the archive
Quoted 9 lines · Stephen Hemminger
Quoted 6 lines · On Wed, 05 Jun 2013 16:50:03 +0200 Damien Millescamps…

Hi Stephen,

Overall this patch is very nice. My only comment on this one is why do you limit the max number of memory resources to 5 ?
The PCI configuration space permits to store up to 6 base addresses.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

+#define PCI_MEM_RESOURCE 5

Please, can you add a log/comment with your patch, too ?

Cheers,

Only because I was trying to save some space, and I didn't see any hardware with that many useful regions. Also the kernel UIO driver has some control over which regions get exposed.

I agree that hardware generally don't use that much BAR for the PCIe. However, this is only a matter of 20 to 24 Bytes, so I don't see any reason not defining this macro as per the PCI standard value.

Could you add a commit log and change that so it can be ack'd and pushed in the DPDK repository ?

Thanks,

Signature · 1 line
Damien Millescamps
#42 lore raw

Re: [PATCH 5/7] pci: support multiple PCI regions per device

In reply to a message not in the archive
Quoted 19 lines · On Wed, 05 Jun 2013 20:05:15 +0200 Damien Millescamps…
Quoted 9 lines · Stephen Hemminger
Quoted 6 lines · On Wed, 05 Jun 2013 16:50:03 +0200 Damien Millescamps…

Hi Stephen,

Overall this patch is very nice. My only comment on this one is why do you limit the max number of memory resources to 5 ?
The PCI configuration space permits to store up to 6 base addresses.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

+#define PCI_MEM_RESOURCE 5

Please, can you add a log/comment with your patch, too ?

Cheers,

Only because I was trying to save some space, and I didn't see any hardware with that many useful regions. Also the kernel UIO driver has some control over which regions get exposed.

Quoted 8 lines

I agree that hardware generally don't use that much BAR for the PCIe. However, this is only a matter of 20 to 24 Bytes, so I don't see any reason not defining this macro as per the PCI standard value.

Could you add a commit log and change that so it can be ack'd and pushed in the DPDK repository ?

Thanks,

Go ahead and change the 5 to a 7.
No point in another resend of whole pile.

#15 lore raw

Re: [PATCH 3/7] optimize log/panic

In reply to a message not in the archive

05/06/2013 16:34, Vincent JARDIN :

Quoted 7 lines · Stephen Hemminger
Quoted 1 line

It does not hurt to move both to a cold section. TODO: some other init and setup functions should be cold too.

applied with modified title:
log: optimize log/panic with attribute cold

Signature · 1 line
Thomas
#44 lore raw

Re: [PATCH 5/7] pci: support multiple PCI regions per device

In reply to a message not in the archive

Hi
Any idea on the ETA for this patch? I'm presuming it's all good to be
pulled into the mainline otherwise?

Thanks
Som

Quoted 36 lines · Stephen Hemminger
Quoted 10 lines · On Wed, 05 Jun 2013 20:05:15 +0200 Damien Millescamps…
Quoted 9 lines · Stephen Hemminger
Quoted 6 lines · On Wed, 05 Jun 2013 16:50:03 +0200 Damien Millescamps…

Hi Stephen,

Overall this patch is very nice. My only comment on this one is why do you limit the max number of memory resources to 5 ?
The PCI configuration space permits to store up to 6 base addresses.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 1 line

+#define PCI_MEM_RESOURCE 5

Please, can you add a log/comment with your patch, too ?

Cheers,

Only because I was trying to save some space, and I didn't see any

hardware

Quoted 1 line
Quoted 1 line

with that many useful regions. Also the kernel UIO driver has some

control

Quoted 10 lines
Quoted 1 line

over which regions get exposed.

I agree that hardware generally don't use that much BAR for the PCIe. However, this is only a matter of 20 to 24 Bytes, so I don't see any reason not defining this macro as per the PCI standard value.

Could you add a commit log and change that so it can be ack'd and pushed in the DPDK repository ?

Thanks,

Go ahead and change the 5 to a 7.
No point in another resend of whole pile.

#46 lore raw

Re: [PATCH 5/7] pci: support multiple PCI regions per device

In reply to a message not in the archive

Hello,

18/06/2013 03:28, somnath kotur :

Quoted 2 lines

Any idea on the ETA for this patch? I'm presuming it's all good to be
pulled into the mainline otherwise?

We are waiting for a commit log.
Please Stephen, could you fix the number of PCI_MEM_RESOURCE and provide a commit log ?

Thank you

Signature · 1 line
Thomas
#48 lore raw

Re: [PATCH 5/7] pci: support multiple PCI regions per device

In reply to a message not in the archive

Intel incorporated this into later version of DPDK (not released yet), let me respin the patch so that the API comes out the same. I don't want to cause pain on the assumption that the open and Intel versions will want to stay converged.

Also, want to support I/O regions which are necessary when doing virtio devices natively.