Skip to contentSkip to conversation

Extend --lcores to run on cores > RTE_MAX_LCORE

PATCH In review David Marchand 9 participants 22 messages Dec 2, 2019 – Jun 10, 2020

lore
Share & tools
lore
#0 lore raw

cover Extend --lcores to run on cores > RTE_MAX_LCORE

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

The --lcores does not need to care about the underlying cores, remove this limitation.

Signature · 18 lines
David Marchand

David Marchand (4):
  eal/windows: fix cpuset macro name
  eal: do not cache lcore detection state
  eal: display all detected cores at startup
  eal: remove limitation on cpuset with --lcores

 drivers/bus/fslmc/portal/dpaa2_hw_dpio.c   | 31 +++++----
 lib/librte_eal/common/eal_common_lcore.c   | 12 +++-
 lib/librte_eal/common/eal_common_options.c | 73 +++++++++++-----------
 lib/librte_eal/common/eal_common_thread.c  |  4 +-
 lib/librte_eal/common/eal_private.h        |  1 -
 lib/librte_eal/windows/eal/include/sched.h |  8 +--
 6 files changed, 68 insertions(+), 61 deletions(-)

-- 
2.23.0
#1 lore raw

PATCH 1/4 eal/windows: fix cpuset macro name

Fix the name of CPU_SETSIZE in hope we can reuse it in other parts of the dpdk manipulating some rte_cpuset_t.

Fixes 4dc2b4d2a4cd ("eal/windows: add headers for compatibility")
---
 lib/librte_eal/windows/eal/include/sched.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
lib/librte_eal/windows/eal/include/sched.h +4 -4
index 257060594c..29868c93d1 100644
@@ -14,8 +14,8 @@
 extern "C" {
 #endif
 
-#ifndef CPU_SET_SIZE
-#define CPU_SET_SIZE RTE_MAX_LCORE
+#ifndef CPU_SETSIZE
+#define CPU_SETSIZE RTE_MAX_LCORE
 #endif
 
 #define _BITS_PER_SET (sizeof(long long) * 8)
@@ -26,7 +26,7 @@ extern "C" {
 #define _WHICH_BIT(b) ((b) & (_BITS_PER_SET - 1))
 
 typedef struct _rte_cpuset_s {
-	long long _bits[_NUM_SETS(CPU_SET_SIZE)];
+	long long _bits[_NUM_SETS(CPU_SETSIZE)];
 } rte_cpuset_t;
 
 #define CPU_SET(b, s) ((s)->_bits[_WHICH_SET(b)] |= (1LL << _WHICH_BIT(b)))
@@ -35,7 +35,7 @@ typedef struct _rte_cpuset_s {
 	do {								\
 		unsigned int _i;					\
 									\
-		for (_i = 0; _i < _NUM_SETS(CPU_SET_SIZE); _i++)	\
+		for (_i = 0; _i < _NUM_SETS(CPU_SETSIZE); _i++)		\
 			(s)->_bits[_i] = 0LL;				\
 	} while (0)
 
Signature · 1 line
2.23.0
#2 lore raw

PATCH 2/4 eal: do not cache lcore detection state

In reply to David Marchand

We use this state in control path only for services cores and -c/-l options.
The value is not updated when using --lcores.

Use the internal helper where needed.

---
 lib/librte_eal/common/eal_common_lcore.c   |  4 +---
 lib/librte_eal/common/eal_common_options.c | 10 +++++-----
 lib/librte_eal/common/eal_private.h        |  1 -
 3 files changed, 6 insertions(+), 9 deletions(-)
lib/librte_eal/common/eal_common_lcore.c +1 -3
index 39efadef1a..1d16fb2156 100644
@@ -139,9 +139,7 @@ rte_eal_cpu_init(void)
 		socket_id = eal_cpu_socket_id(lcore_id);
 		lcore_to_socket_id[lcore_id] = socket_id;
 
-		/* in 1:1 mapping, record related cpu detected state */
-		lcore_config[lcore_id].detected = eal_cpu_detected(lcore_id);
-		if (lcore_config[lcore_id].detected == 0) {
+		if (eal_cpu_detected(lcore_id) == 0) {
 			config->lcore_role[lcore_id] = ROLE_OFF;
 			lcore_config[lcore_id].core_index = -1;
 			continue;
lib/librte_eal/common/eal_common_options.c +5 -5
index a7f9c5f9bd..68f7d1cd73 100644
@@ -373,7 +373,7 @@ eal_parse_service_coremask(const char *coremask)
 					return -1;
 				}
 
-				if (!lcore_config[idx].detected) {
+				if (eal_cpu_detected(idx) == 0) {
 					RTE_LOG(ERR, EAL,
 						"lcore %u unavailable\n", idx);
 					return -1;
@@ -429,7 +429,7 @@ update_lcore_config(int *cores)
 
 	for (i = 0; i < RTE_MAX_LCORE; i++) {
 		if (cores[i] != -1) {
-			if (!lcore_config[i].detected) {
+			if (eal_cpu_detected(i) == 0) {
 				RTE_LOG(ERR, EAL, "lcore %u unavailable\n", i);
 				ret = -1;
 				continue;
@@ -785,7 +785,7 @@ convert_to_cpuset(rte_cpuset_t *cpusetp,
 		if (!set[idx])
 			continue;
 
-		if (!lcore_config[idx].detected) {
+		if (eal_cpu_detected(idx) == 0) {
 			RTE_LOG(ERR, EAL, "core %u "
 				"unavailable\n", idx);
 			return -1;
@@ -1138,7 +1138,7 @@ available_cores(void)
 
 	/* find the first available cpu */
 	for (idx = 0; idx < RTE_MAX_LCORE; idx++) {
-		if (!lcore_config[idx].detected)
+		if (eal_cpu_detected(idx) == 0)
 			continue;
 		break;
 	}
@@ -1152,7 +1152,7 @@ available_cores(void)
 	sequence = 0;
 
 	for (idx++ ; idx < RTE_MAX_LCORE; idx++) {
-		if (!lcore_config[idx].detected)
+		if (eal_cpu_detected(idx) == 0)
 			continue;
 
 		if (idx == previous + 1) {
lib/librte_eal/common/eal_private.h +0 -1
index 8a9d493f0c..ddcfbe2e44 100644
@@ -29,7 +29,6 @@ struct lcore_config {
 	unsigned int core_id;      /**< core number on socket for this lcore */
 	int core_index;            /**< relative index, starting from 0 */
 	uint8_t core_role;         /**< role of core eg: OFF, RTE, SERVICE */
-	uint8_t detected;          /**< true if lcore was detected */
 
 	rte_cpuset_t cpuset;       /**< cpu set which the lcore affinity to */
 };
Signature · 1 line
2.23.0
#3 lore raw

PATCH 3/4 eal: display all detected cores at startup

In reply to David Marchand

Add debug logs to have a trace of unused cores for -c/-l options on systems with more cores than RTE_MAX_LCORE.

---
 lib/librte_eal/common/eal_common_lcore.c | 8 ++++++++
 1 file changed, 8 insertions(+)
lib/librte_eal/common/eal_common_lcore.c +8 -0
index 1d16fb2156..5404922a87 100644
@@ -159,6 +159,14 @@ rte_eal_cpu_init(void)
 				lcore_config[lcore_id].socket_id);
 		count++;
 	}
+	for (; lcore_id < CPU_SETSIZE; lcore_id++) {
+		if (eal_cpu_detected(lcore_id) == 0)
+			continue;
+		RTE_LOG(DEBUG, EAL, "Skipped lcore %u as core %u on socket %u\n",
+			lcore_id, eal_cpu_core_id(lcore_id),
+			eal_cpu_socket_id(lcore_id));
+	}
+
 	/* Set the count of enabled logical cores of the EAL configuration */
 	config->lcore_count = count;
 	RTE_LOG(DEBUG, EAL,
Signature · 1 line
2.23.0
#4 lore raw

PATCH 4/4 eal: remove limitation on cpuset with --lcores

In reply to David Marchand

Contrary to the -c/-l options, where a logical core runs on the same physical core in a 1:1 fashion (example: lcore 0 runs on core 0, lcore 16 runs on core 16), the --lcores option makes it possible to select the physical cores on which runs a logical core.

However the current parsing code still limits the cpuset to the [0, RTE_MAX_LCORE] range.

Example, before the patch, on a 24 cores system with RTE_MAX_LCORE == 16:

$ ./master/app/testpmd --no-huge --no-pci -m 512 --log-level *:debug \
 --lcores 0@16,1@17 -- -i --total-num-mbufs 2048
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 5 on socket 0
EAL: Detected lcore 6 as core 6 on socket 0
EAL: Detected lcore 7 as core 8 on socket 0
EAL: Detected lcore 8 as core 9 on socket 0
EAL: Detected lcore 9 as core 10 on socket 0
EAL: Detected lcore 10 as core 11 on socket 0
EAL: Detected lcore 11 as core 12 on socket 0
EAL: Detected lcore 12 as core 13 on socket 0
EAL: Detected lcore 13 as core 14 on socket 0
EAL: Detected lcore 14 as core 0 on socket 0
EAL: Detected lcore 15 as core 1 on socket 0
EAL: Skipped lcore 16 as core 2 on socket 0
EAL: Skipped lcore 17 as core 3 on socket 0
EAL: Skipped lcore 18 as core 4 on socket 0
EAL: Skipped lcore 19 as core 5 on socket 0
EAL: Skipped lcore 20 as core 6 on socket 0
EAL: Skipped lcore 21 as core 8 on socket 0
EAL: Skipped lcore 22 as core 9 on socket 0
EAL: Skipped lcore 23 as core 10 on socket 0
EAL: Skipped lcore 24 as core 11 on socket 0
EAL: Skipped lcore 25 as core 12 on socket 0
EAL: Skipped lcore 26 as core 13 on socket 0
EAL: Skipped lcore 27 as core 14 on socket 0
EAL: Support maximum 16 logical core(s) by configuration.
EAL: Detected 16 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: invalid parameter for --lcores

We can remove this limitation by using a cpuset_t (which is a more natural type since this is what gets passed to pthread_setaffinity* in the end).

After the patch:

$ ./master/app/testpmd --no-huge --no-pci -m 512 --log-level *:debug \
 --lcores 0@16,1@17 -- -i --total-num-mbufs 2048
[...]
EAL: Master lcore 0 is ready (tid=7f94217bbc00;cpuset=[16])
EAL: lcore 1 is ready (tid=7f941f491700;cpuset=[17])
---
 drivers/bus/fslmc/portal/dpaa2_hw_dpio.c   | 31 ++++++-----
 lib/librte_eal/common/eal_common_options.c | 63 +++++++++++-----------
 lib/librte_eal/common/eal_common_thread.c  |  4 +-
 3 files changed, 50 insertions(+), 48 deletions(-)
drivers/bus/fslmc/portal/dpaa2_hw_dpio.c +18 -13
index 3ca3ae4f51..739ce434ba 100644
@@ -365,20 +365,25 @@ dpaa2_check_lcore_cpuset(void)
 		dpaa2_cpu[lcore_id] = 0xffffffff;
 
 	for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
-		for (i = 0; i < RTE_MAX_LCORE; i++) {
-			rte_cpuset_t cpuset = rte_lcore_cpuset(lcore_id);
-
-			if (CPU_ISSET(i, &cpuset)) {
-				RTE_LOG(DEBUG, EAL, "lcore id = %u cpu=%u\n",
-					lcore_id, i);
-				if (dpaa2_cpu[lcore_id] != 0xffffffff) {
-					DPAA2_BUS_ERR(
-				    "ERR:lcore map to multi-cpu not supported");
-					ret = -1;
-				} else  {
-					dpaa2_cpu[lcore_id] = i;
-				}
+		rte_cpuset_t cpuset = rte_lcore_cpuset(lcore_id);
+
+		for (i = 0; i < CPU_SETSIZE; i++) {
+			if (!CPU_ISSET(i, &cpuset))
+				continue;
+			if (i >= RTE_MAX_LCORE) {
+				DPAA2_BUS_ERR("ERR:lcore map to core %u (>= %u) not supported",
+					i, RTE_MAX_LCORE);
+				ret = -1;
+				continue;
 			}
+			RTE_LOG(DEBUG, EAL, "lcore id = %u cpu=%u\n",
+				lcore_id, i);
+			if (dpaa2_cpu[lcore_id] != 0xffffffff) {
+				DPAA2_BUS_ERR("ERR:lcore map to multi-cpu not supported");
+				ret = -1;
+				continue;
+			}
+			dpaa2_cpu[lcore_id] = i;
 		}
 	}
 	return ret;
lib/librte_eal/common/eal_common_options.c +30 -33

159 lines not on this page · raw message

lib/librte_eal/common/eal_common_thread.c +2 -2
index f9a8cf14d2..78581753c0 100644
@@ -61,7 +61,7 @@ eal_cpuset_socket_id(rte_cpuset_t *cpusetp)
 			break;
 		}
 
-	} while (++cpu < RTE_MAX_LCORE);
+	} while (++cpu < CPU_SETSIZE);
 
 	return socket_id;
 }
@@ -118,7 +118,7 @@ eal_thread_dump_affinity(char *str, unsigned size)
 
 	rte_thread_get_affinity(&cpuset);
 
-	for (cpu = 0; cpu < RTE_MAX_LCORE; cpu++) {
+	for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
 		if (!CPU_ISSET(cpu, &cpuset))
 			continue;
 
Signature · 1 line
2.23.0
#5 lore raw
In reply to David Marchand
Quoted 8 lines · David Marchand

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

The --lcores does not need to care about the underlying cores, remove this limitation.

Any comment?
Thanks.

Signature · 1 line
David Marchand
#6 lore raw

Re: [dpdk-dev] [PATCH 0/4] Extend --lcores to run on cores >RTE_MAX_LCORE

Quoted 16 lines

From: dev [mailto:[email protected]] On Behalf Of David Marchand
Sent: Tuesday, January 14, 2020 1:59 PM

Quoted 2 lines · David Marchand

We are currently stuck with no option but recompile a DPDK if the

system

Quoted 2 lines

has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and

your

Quoted 4 lines

testpmd has been built and packaged with RTE_MAX_LCORE == 128.

The --lcores does not need to care about the underlying cores, remove this limitation.

Quoted 7 lines

Any comment?
Thanks.

--
David Marchand

I haven't reviewed the patch, but recall that the Mempool library sets up caches per lcore using hardcoded loops going from 0 to RTE_MAX_LCORE, regardless how many lcores are present and assigned to DPDK.

Making rte_max_lcore dynamic (and possibly auto-detected by the EAL) instead of defined at compile time might also be an improvement for systems with fewer cores than RTE_MAX_LCORE. But there are probably a lot of considerations regarding multi-process applications.

Med venlig hilsen / kind regards
- Morten Brørup

#7 lore raw
In reply to David Marchand
Quoted 4 lines · David Marchand

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

Just for clarification, in this case the DPDK application will work but will only use RTE_MAX_LCORE cores, right? You need to compile to use all available cores.

Are cores more than RTE_MAX_LCORE usable after this patch?

Quoted 4 lines

The --lcores does not need to care about the underlying cores, remove this limitation.

+1 to remove limit in --lcores, but I didn't make it work with this patchset, I may be missing something, I tried by setting the "RTE_MAX_LCORE=32", in this case should '--lcores "(30-40)@(10-20)"' work?

#8 lore raw
Quoted 9 lines · Ferruh
Quoted 4 lines · David Marchand

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

Just for clarification, in this case the DPDK application will work but will only use RTE_MAX_LCORE cores, right? You need to compile to use all available cores.

EAL starts up to RTE_MAX_LCORE lcores (as in EAL threads), that's true before and after this patch.

By default (using the -c or -l options), a lcore X runs on the physical core X. With the --lcores option, you can select on which physical cores a lcore runs.

But, before this series, the code limits the physical cores to the same [0, RTE_MAX_LCORE[ range.

Quoted 2 lines

Are cores more than RTE_MAX_LCORE usable after this patch?

Yes, see below.

Quoted 9 lines
Quoted 5 lines

The --lcores does not need to care about the underlying cores, remove this limitation.

+1 to remove limit in --lcores, but I didn't make it work with this patchset, I may be missing something, I tried by setting the "RTE_MAX_LCORE=32", in this case should '--lcores "(30-40)@(10-20)"' work?

Quoting the parser code:
* The format pattern: --lcores='<lcores[@cpus]>[<,lcores[@cpus]>...]'
* lcores, cpus could be a single digit/range or a group.
* '(' and ')' are necessary if it's a group.
* If not supply '@cpus', the value of cpus uses the same as lcores.
* e.g. '1,2@(5-7),(3-5)@(0,2),(0,6),7-8' means start 9 EAL thread as below

 *   lcore 0 runs on cpuset 0x41 (cpu 0,6)
 *   lcore 1 runs on cpuset 0x2 (cpu 1)
 *   lcore 2 runs on cpuset 0xe0 (cpu 5,6,7)
 *   lcore 3,4,5 runs on cpuset 0x5 (cpu 0,2)
 *   lcore 6 runs on cpuset 0x41 (cpu 0,6)
 *   lcore 7 runs on cpuset 0x80 (cpu 7)
 *   lcore 8 runs on cpuset 0x100 (cpu 8)

With --lcores "(30-40)@(10-20)", you asked to start lcores 30 to 40 to run on the cpuset 10 to 20.
So it will be refused, since the max lcore is 32.

If your intention was to start lcore 10 on physical core 30, lcore 11 on core 31 etc... then you can try --lcores 10@30,11@31,12@32...

Signature · 1 line
David Marchand
#9 lore raw
In reply to David Marchand

02/12/2019 16:35, David Marchand:

Quoted 5 lines

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

Quoted 8 lines

The --lcores does not need to care about the underlying cores, remove this limitation.

David Marchand (4):
eal/windows: fix cpuset macro name
eal: do not cache lcore detection state
eal: display all detected cores at startup
eal: remove limitation on cpuset with --lcores

The patches look good but it is very hard to review parsing code (last patch). We will better experience corner cases after merging.

Applied for -rc1, thanks

#10 lore raw

Hi,

21/01/2020 01:24, Thomas Monjalon:

Quoted 15 lines

02/12/2019 16:35, David Marchand:

Quoted 5 lines

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

Quoted 8 lines

The --lcores does not need to care about the underlying cores, remove this limitation.

David Marchand (4):
eal/windows: fix cpuset macro name
eal: do not cache lcore detection state
eal: display all detected cores at startup
eal: remove limitation on cpuset with --lcores

Quoted 4 lines

The patches look good but it is very hard to review parsing code (last patch). We will better experience corner cases after merging.

Applied for -rc1, thanks

This patch was merged in 20.02.
We don't have any feedback about issues so it's probably working fine.

It is solving a problem for running DPDK on machines having a lot of cores. Now the difficult question: is it a new feature or a fix? Should we backport this patchset?

#11 lore raw

[AMD Official Use Only - Internal Distribution Only]

Thanks Thomas for bringing this up.
I consider this is not a new feature, but rather a fix to address the issue with statically assigned maximum lcore limit on high-density CPU platform such as AMD Epyc. As I see a lot of DPDK adopters are still using LTS 18.11 & 19.11, and they have 1~2 yrs of lifetime left, we like to backport this to LTS 18.11 & 19.11 at least.

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Friday, February 21, 2020 12:04 AM
To: David Marchand <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Song, Keesang <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [dpdk-dev] [PATCH 0/4] Extend --lcores to run on cores > RTE_MAX_LCORE

[CAUTION: External Email]

Hi,

21/01/2020 01:24, Thomas Monjalon:

Quoted 15 lines

02/12/2019 16:35, David Marchand:

Quoted 5 lines

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

Quoted 8 lines

The --lcores does not need to care about the underlying cores, remove this limitation.

David Marchand (4):
eal/windows: fix cpuset macro name
eal: do not cache lcore detection state
eal: display all detected cores at startup
eal: remove limitation on cpuset with --lcores

Quoted 4 lines

The patches look good but it is very hard to review parsing code (last patch). We will better experience corner cases after merging.

Applied for -rc1, thanks

This patch was merged in 20.02.
We don't have any feedback about issues so it's probably working fine.

It is solving a problem for running DPDK on machines having a lot of cores. Now the difficult question: is it a new feature or a fix? Should we backport this patchset?

#12 lore raw
Quoted 2 lines · Keesang

[AMD Official Use Only - Internal Distribution Only]

Please, get this header removed.
This is a public mailing list.

Quoted 3 lines

Thanks Thomas for bringing this up.
I consider this is not a new feature, but rather a fix to address the issue with statically assigned maximum lcore limit on high-density CPU platform such as AMD Epyc. As I see a lot of DPDK adopters are still using LTS 18.11 & 19.11, and they have 1~2 yrs of lifetime left, we like to backport this to LTS 18.11 & 19.11 at least.

It is not a fix.

The use of static arrays is a design choice that goes back to the early days in dpdk.
The addition of --lcores came in after this, but it was introduced for a different use case than placing lcores on any physical core. And there was no claim that a core > RTE_MAX_LCORE would be usable.

When testing on a new hardware, it is normal to observe some limitations. Running DPDK on those platforms should be possible: "should be" because I do not have access to this hardware and saw neither tests reports nor performance numbers.
Before this patch, the limitation is that on Epyc, cores > RTE_MAX_LCORE are not usable.

Now, this change is quite constrained.
If we backport it, I don't expect issues in the main dpdk components (based on code review and ovs tests with a RTE_MAX_LCORE set to 16 on a 24 cores system).
There might be issues in some examples or not widely used library which uses a physical core id instead of a lcore id.

This is the same recurring question "do we allow new features in a stable branch?".

--
David Marchand

#13 lore raw
Quoted 41 lines · David Marchand
Quoted 2 lines · Keesang

[AMD Official Use Only - Internal Distribution Only]

Please, get this header removed.
This is a public mailing list.

Quoted 7 lines

Thanks Thomas for bringing this up.
I consider this is not a new feature, but rather a fix to address the issue with statically assigned maximum lcore limit on high-density CPU platform such as AMD Epyc.
As I see a lot of DPDK adopters are still using LTS 18.11 & 19.11, and they have 1~2 yrs of lifetime left, we like to backport this to LTS 18.11 & 19.11 at least.

It is not a fix.

The use of static arrays is a design choice that goes back to the early days in dpdk.
The addition of --lcores came in after this, but it was introduced for a different use case than placing lcores on any physical core. And there was no claim that a core > RTE_MAX_LCORE would be usable.

When testing on a new hardware, it is normal to observe some limitations. Running DPDK on those platforms should be possible: "should be" because I do not have access to this hardware and saw neither tests reports nor performance numbers.
Before this patch, the limitation is that on Epyc, cores > RTE_MAX_LCORE are not usable.

Now, this change is quite constrained.
If we backport it, I don't expect issues in the main dpdk components (based on code review and ovs tests with a RTE_MAX_LCORE set to 16 on a 24 cores system).
There might be issues in some examples or not widely used library which uses a physical core id instead of a lcore id.

Quoted 2 lines

This is the same recurring question "do we allow new features in a stable branch?".

Usually, the answer is 'no'. But we do allow some "new" things to be backported (pci ids, etc) that might be required to enable older functionality. Additionally, I'm sure if some feature were required to mitigate a CVE, we'd rather favor backporting it.

I guess we could pose a litmus test:

1. Is the problem this feature solves so widespread that it needs to be addressed ASAP?
2. Is there a known workaround to the problem this is solving?
3. How intrusive is the feature?
4. Is it shown to be stable in the mainline (number of fixes, testing,

     etc)?

5. Is it constrained enough that we know we can support it with even higher priority than other things?

Probably other questions that will need to be asked.

And even in that list of question, I'm not sure I'd be able to advocate backporting this in the upstream branches - it hasn't had much testing. It's unstable. It's "difficult" to use. It is not widespread that people have so many cores. The workaround is much simpler than supporting this (recompile).

Quoted 3 lines

--
David Marchand

#14 lore raw
Quoted 62 lines · On Fri, 21 Feb 2020 09:48:58 -0500 Aaron Conole <[email protected]> wrote:
Quoted 41 lines · David Marchand
Quoted 2 lines · Keesang

[AMD Official Use Only - Internal Distribution Only]

Please, get this header removed.
This is a public mailing list.

Quoted 7 lines

Thanks Thomas for bringing this up.
I consider this is not a new feature, but rather a fix to address the issue with statically assigned maximum lcore limit on high-density CPU platform such as AMD Epyc.
As I see a lot of DPDK adopters are still using LTS 18.11 & 19.11, and they have 1~2 yrs of lifetime left, we like to backport this to LTS 18.11 & 19.11 at least.

It is not a fix.

The use of static arrays is a design choice that goes back to the early days in dpdk.
The addition of --lcores came in after this, but it was introduced for a different use case than placing lcores on any physical core. And there was no claim that a core > RTE_MAX_LCORE would be usable.

When testing on a new hardware, it is normal to observe some limitations. Running DPDK on those platforms should be possible: "should be" because I do not have access to this hardware and saw neither tests reports nor performance numbers.
Before this patch, the limitation is that on Epyc, cores > RTE_MAX_LCORE are not usable.

Now, this change is quite constrained.
If we backport it, I don't expect issues in the main dpdk components (based on code review and ovs tests with a RTE_MAX_LCORE set to 16 on a 24 cores system).
There might be issues in some examples or not widely used library which uses a physical core id instead of a lcore id.

Quoted 2 lines

This is the same recurring question "do we allow new features in a stable branch?".

Usually, the answer is 'no'. But we do allow some "new" things to be backported (pci ids, etc) that might be required to enable older functionality. Additionally, I'm sure if some feature were required to mitigate a CVE, we'd rather favor backporting it.

I guess we could pose a litmus test:

1. Is the problem this feature solves so widespread that it needs to be addressed ASAP?
2. Is there a known workaround to the problem this is solving?
3. How intrusive is the feature?
4. Is it shown to be stable in the mainline (number of fixes, testing,

     etc)?

5. Is it constrained enough that we know we can support it with even higher priority than other things?

Quoted 12 lines

Probably other questions that will need to be asked.

And even in that list of question, I'm not sure I'd be able to advocate backporting this in the upstream branches - it hasn't had much testing. It's unstable. It's "difficult" to use. It is not widespread that people have so many cores. The workaround is much simpler than supporting this (recompile).

Quoted 3 lines

--
David Marchand

RTE_MAX_LCORES is exposed in API/ABI to application. Many applications use that to size internal data structures. Having rte_lcore_id() potentially return a larger value would cause out of bounds access (and crash) in that application.

#15 lore raw
In reply to Thomas Monjalon

[AMD Official Use Only - Internal Distribution Only]

Hi Thomas & David,

We haven't got the final status on this patch, and I don't see this change even from the latest LTS 20.04 repo. So I'd like to confirm whether this patch has been safely submitted to the main upstream.
Can you check the status of that commit?

https://patchwork.dpdk.org/patch/63507/

Thanks for your support in advance,
Keesang

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Friday, February 21, 2020 12:04 AM
To: David Marchand <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Song, Keesang <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [dpdk-dev] [PATCH 0/4] Extend --lcores to run on cores > RTE_MAX_LCORE

[CAUTION: External Email]

Hi,

21/01/2020 01:24, Thomas Monjalon:

Quoted 15 lines

02/12/2019 16:35, David Marchand:

Quoted 5 lines

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

Quoted 8 lines

The --lcores does not need to care about the underlying cores, remove this limitation.

David Marchand (4):
eal/windows: fix cpuset macro name
eal: do not cache lcore detection state
eal: display all detected cores at startup
eal: remove limitation on cpuset with --lcores

Quoted 4 lines

The patches look good but it is very hard to review parsing code (last patch). We will better experience corner cases after merging.

Applied for -rc1, thanks

This patch was merged in 20.02.
We don't have any feedback about issues so it's probably working fine.

It is solving a problem for running DPDK on machines having a lot of cores. Now the difficult question: is it a new feature or a fix? Should we backport this patchset?

#16 lore raw

[AMD Public Use]

Hi Thomas & David,

We haven't got the final status on this patch, and I don't see this change even from the latest LTS 20.04 repo. So I'd like to confirm whether this patch has been safely submitted to the main upstream.
Can you check the status of that commit?

https://patchwork.dpdk.org/patch/63507/

Thanks for your support in advance,
Keesang

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Friday, February 21, 2020 12:04 AM
To: David Marchand <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Song, Keesang <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [dpdk-dev] [PATCH 0/4] Extend --lcores to run on cores > RTE_MAX_LCORE

[CAUTION: External Email]

Hi,

21/01/2020 01:24, Thomas Monjalon:

Quoted 15 lines

02/12/2019 16:35, David Marchand:

Quoted 5 lines

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

Quoted 8 lines

The --lcores does not need to care about the underlying cores, remove this limitation.

David Marchand (4):
eal/windows: fix cpuset macro name
eal: do not cache lcore detection state
eal: display all detected cores at startup
eal: remove limitation on cpuset with --lcores

Quoted 4 lines

The patches look good but it is very hard to review parsing code (last patch). We will better experience corner cases after merging.

Applied for -rc1, thanks

This patch was merged in 20.02.
We don't have any feedback about issues so it's probably working fine.

It is solving a problem for running DPDK on machines having a lot of cores. Now the difficult question: is it a new feature or a fix? Should we backport this patchset?

#17 lore raw

29/05/2020 05:05, Song, Keesang:

Quoted 7 lines

Hi Thomas & David,

We haven't got the final status on this patch, and I don't see this change even from the latest LTS 20.04 repo. So I'd like to confirm whether this patch has been safely submitted to the main upstream.
Can you check the status of that commit?

https://patchwork.dpdk.org/patch/63507/

As you can see below, there is a pending question:
"is it a new feature or a fix?"

Kevin and Luca are the arbiters for the backports in 18.11 and 19.11.

Quoted 33 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Friday, February 21, 2020 12:04 AM

Hi,

21/01/2020 01:24, Thomas Monjalon:

Quoted 15 lines

02/12/2019 16:35, David Marchand:

Quoted 5 lines

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

Quoted 8 lines

The --lcores does not need to care about the underlying cores, remove this limitation.

David Marchand (4):
eal/windows: fix cpuset macro name
eal: do not cache lcore detection state
eal: display all detected cores at startup
eal: remove limitation on cpuset with --lcores

Quoted 4 lines

The patches look good but it is very hard to review parsing code (last patch). We will better experience corner cases after merging.

Applied for -rc1, thanks

This patch was merged in 20.02.
We don't have any feedback about issues so it's probably working fine.

It is solving a problem for running DPDK on machines having a lot of cores. Now the difficult question: is it a new feature or a fix? Should we backport this patchset?

#18 lore raw

[AMD Public Use]

Thanks Thomas for the response.
For a correction, the patchwork has not been submitted for the current LTS release, 19.11.2, but was merged into 20.02 and onward.
The reason I brought this again was to address LTS users and many other application based on the LTS releases(18.11 & 19.11).
Since I found many of our customers and users are still relying on the latest LTS version, I'm seeking an aid for adding this change into at least 19.11, LTS branch.
We could argue that this is actually not a bug, in a way, it's inconvenient for Openstack or cloud deployed DPDK application since it's often inapt to change the base config and recompile the max lcore limit. If backporting is still not a preferred way(pushing this patchwork into 19.11), then can we instead consider changing only the default value of ' CONFIG_RTE_MAX_LCORE' to 256 in common_base file?

# Compile Environment Abstraction Layer
#
CONFIG_RTE_MAX_LCORE=128 --> 256

I'd appreciate if anyone can advise me know what we can do about this to move forward.

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Monday, June 1, 2020 2:23 PM
To: Song, Keesang <[email protected]>
Cc: David Marchand <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [dpdk-dev] [PATCH 0/4] Extend --lcores to run on cores > RTE_MAX_LCORE

[CAUTION: External Email]

29/05/2020 05:05, Song, Keesang:

Quoted 11 lines

Hi Thomas & David,

We haven't got the final status on this patch, and I don't see this change even from the latest LTS 20.04 repo. So I'd like to confirm whether this patch has been safely submitted to the main upstream.
Can you check the status of that commit?

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatc
hwork.dpdk.org%2Fpatch%2F63507%2F&amp;data=02%7C01%7CKeesang.Song%40am d.com%7Cd71ea9aca917447dfb3e08d80671f34c%7C3dd8961fe4884e608e11a82d994 e183d%7C0%7C0%7C637266433776198364&amp;sdata=1EgxevCILVMMLgyQC%2FzaWYJ XJ%2BOijs0Rtym1TA0VS28%3D&amp;reserved=0

As you can see below, there is a pending question:
"is it a new feature or a fix?"

Kevin and Luca are the arbiters for the backports in 18.11 and 19.11.

Quoted 33 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Friday, February 21, 2020 12:04 AM

Hi,

21/01/2020 01:24, Thomas Monjalon:

Quoted 15 lines

02/12/2019 16:35, David Marchand:

Quoted 5 lines

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

Quoted 8 lines

The --lcores does not need to care about the underlying cores, remove this limitation.

David Marchand (4):
eal/windows: fix cpuset macro name
eal: do not cache lcore detection state
eal: display all detected cores at startup
eal: remove limitation on cpuset with --lcores

Quoted 4 lines

The patches look good but it is very hard to review parsing code (last patch). We will better experience corner cases after merging.

Applied for -rc1, thanks

This patch was merged in 20.02.
We don't have any feedback about issues so it's probably working fine.

It is solving a problem for running DPDK on machines having a lot of cores. Now the difficult question: is it a new feature or a fix? Should we backport this patchset?

#19 lore raw

[AMD Public Use]

Hi Kevin and Luca,

We are still waiting for the response.
Can you help on this for the backports in 18.11 and 19.11?
It would work for our customers even with changing the default value of ' CONFIG_RTE_MAX_LCORE' to 256 in common_base file in 18.11 and 19.11.

Thanks,
Keesang

-----Original Message-----
From: Song, Keesang
Sent: Monday, June 1, 2020 3:54 PM
To: Thomas Monjalon <[email protected]>
Cc: David Marchand <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Aman Kumar <[email protected]>; Grimm, Jon <[email protected]>
Subject: RE: [dpdk-dev] [PATCH 0/4] Extend --lcores to run on cores > RTE_MAX_LCORE

[AMD Public Use]

Thanks Thomas for the response.
For a correction, the patchwork has not been submitted for the current LTS release, 19.11.2, but was merged into 20.02 and onward.
The reason I brought this again was to address LTS users and many other application based on the LTS releases(18.11 & 19.11).
Since I found many of our customers and users are still relying on the latest LTS version, I'm seeking an aid for adding this change into at least 19.11, LTS branch.
We could argue that this is actually not a bug, in a way, it's inconvenient for Openstack or cloud deployed DPDK application since it's often inapt to change the base config and recompile the max lcore limit. If backporting is still not a preferred way(pushing this patchwork into 19.11), then can we instead consider changing only the default value of ' CONFIG_RTE_MAX_LCORE' to 256 in common_base file?

# Compile Environment Abstraction Layer
#
CONFIG_RTE_MAX_LCORE=128 --> 256

I'd appreciate if anyone can advise me know what we can do about this to move forward.

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Monday, June 1, 2020 2:23 PM
To: Song, Keesang <[email protected]>
Cc: David Marchand <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [dpdk-dev] [PATCH 0/4] Extend --lcores to run on cores > RTE_MAX_LCORE

[CAUTION: External Email]

29/05/2020 05:05, Song, Keesang:

Quoted 11 lines

Hi Thomas & David,

We haven't got the final status on this patch, and I don't see this change even from the latest LTS 20.04 repo. So I'd like to confirm whether this patch has been safely submitted to the main upstream.
Can you check the status of that commit?

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatc
hwork.dpdk.org%2Fpatch%2F63507%2F&amp;data=02%7C01%7CKeesang.Song%40am d.com%7Cd71ea9aca917447dfb3e08d80671f34c%7C3dd8961fe4884e608e11a82d994 e183d%7C0%7C0%7C637266433776198364&amp;sdata=1EgxevCILVMMLgyQC%2FzaWYJ XJ%2BOijs0Rtym1TA0VS28%3D&amp;reserved=0

As you can see below, there is a pending question:
"is it a new feature or a fix?"

Kevin and Luca are the arbiters for the backports in 18.11 and 19.11.

Quoted 33 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Friday, February 21, 2020 12:04 AM

Hi,

21/01/2020 01:24, Thomas Monjalon:

Quoted 15 lines

02/12/2019 16:35, David Marchand:

Quoted 5 lines

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

Quoted 8 lines

The --lcores does not need to care about the underlying cores, remove this limitation.

David Marchand (4):
eal/windows: fix cpuset macro name
eal: do not cache lcore detection state
eal: display all detected cores at startup
eal: remove limitation on cpuset with --lcores

Quoted 4 lines

The patches look good but it is very hard to review parsing code (last patch). We will better experience corner cases after merging.

Applied for -rc1, thanks

This patch was merged in 20.02.
We don't have any feedback about issues so it's probably working fine.

It is solving a problem for running DPDK on machines having a lot of cores. Now the difficult question: is it a new feature or a fix? Should we backport this patchset?

#20 lore raw
Quoted 10 lines · Keesang

[AMD Public Use]

Hi Kevin and Luca,

We are still waiting for the response.
Can you help on this for the backports in 18.11 and 19.11?
It would work for our customers even with changing the default value of ' CONFIG_RTE_MAX_LCORE' to 256 in common_base file in 18.11 and 19.11.

Thanks,
Keesang

How to send patches for inclusion in LTS releases if not already backported is documented here:

https://core.dpdk.org/contribute/#send

If you do the work to backport and test, on the surface it seems fine to have those in 19.11.4

Quoted 82 lines

-----Original Message-----
From: Song, Keesang
Sent: Monday, June 1, 2020 3:54 PM
To: Thomas Monjalon <[email protected]>
Cc: David Marchand <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Aman Kumar <[email protected]>; Grimm, Jon <[email protected]>
Subject: RE: [dpdk-dev] [PATCH 0/4] Extend --lcores to run on cores > RTE_MAX_LCORE

[AMD Public Use]

Thanks Thomas for the response.
For a correction, the patchwork has not been submitted for the current LTS release, 19.11.2, but was merged into 20.02 and onward.
The reason I brought this again was to address LTS users and many other application based on the LTS releases(18.11 & 19.11).
Since I found many of our customers and users are still relying on the latest LTS version, I'm seeking an aid for adding this change into at least 19.11, LTS branch.
We could argue that this is actually not a bug, in a way, it's inconvenient for Openstack or cloud deployed DPDK application since it's often inapt to change the base config and recompile the max lcore limit. If backporting is still not a preferred way(pushing this patchwork into 19.11), then can we instead consider changing only the default value of ' CONFIG_RTE_MAX_LCORE' to 256 in common_base file?

# Compile Environment Abstraction Layer
#
CONFIG_RTE_MAX_LCORE=128 --> 256

I'd appreciate if anyone can advise me know what we can do about this to move forward.

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Monday, June 1, 2020 2:23 PM
To: Song, Keesang <[email protected]>
Cc: David Marchand <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [dpdk-dev] [PATCH 0/4] Extend --lcores to run on cores > RTE_MAX_LCORE

[CAUTION: External Email]

29/05/2020 05:05, Song, Keesang:

Quoted 11 lines

Hi Thomas & David,

We haven't got the final status on this patch, and I don't see this change even from the latest LTS 20.04 repo. So I'd like to confirm whether this patch has been safely submitted to the main upstream.
Can you check the status of that commit?

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatc
hwork.dpdk.org%2Fpatch%2F63507%2F&amp;data=02%7C01%7CKeesang.Song%40am d.com%7Cd71ea9aca917447dfb3e08d80671f34c%7C3dd8961fe4884e608e11a82d994 e183d%7C0%7C0%7C637266433776198364&amp;sdata=1EgxevCILVMMLgyQC%2FzaWYJ XJ%2BOijs0Rtym1TA0VS28%3D&amp;reserved=0

As you can see below, there is a pending question:
"is it a new feature or a fix?"

Kevin and Luca are the arbiters for the backports in 18.11 and 19.11.

Quoted 32 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Friday, February 21, 2020 12:04 AM

Hi,

21/01/2020 01:24, Thomas Monjalon:

Quoted 14 lines

02/12/2019 16:35, David Marchand:

Quoted 5 lines

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

Quoted 7 lines

The --lcores does not need to care about the underlying cores, remove this limitation.
David Marchand (4):
eal/windows: fix cpuset macro name
eal: do not cache lcore detection state
eal: display all detected cores at startup
eal: remove limitation on cpuset with --lcores

Quoted 4 lines

The patches look good but it is very hard to review parsing code (last patch). We will better experience corner cases after merging.

Applied for -rc1, thanks

This patch was merged in 20.02.
We don't have any feedback about issues so it's probably working fine.

It is solving a problem for running DPDK on machines having a lot of cores. Now the difficult question: is it a new feature or a fix? Should we backport this patchset?

#21 lore raw
Quoted 5 lines · Luca Boccassi
Quoted 4 lines · Keesang

[AMD Public Use]

Hi Kevin and Luca,

Hi Keesang

Quoted 7 lines
Quoted 6 lines

We are still waiting for the response.
Can you help on this for the backports in 18.11 and 19.11?
It would work for our customers even with changing the default value of ' CONFIG_RTE_MAX_LCORE' to 256 in common_base file in 18.11 and 19.11.

Thanks,
Keesang

Quoted 8 lines

How to send patches for inclusion in LTS releases if not already backported is documented here:

https://core.dpdk.org/contribute/#send

If you do the work to backport and test, on the surface it seems fine to have those in 19.11.4

Looks ok to me too. I can take this in 18.11, but it will need to be in 19.11 first. Please send a backport for 18.11 and test that it's working as expected with the 18.11 branch.

thanks,
Kevin.

Quoted 83 lines
Quoted 82 lines

-----Original Message-----
From: Song, Keesang
Sent: Monday, June 1, 2020 3:54 PM
To: Thomas Monjalon <[email protected]>
Cc: David Marchand <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Aman Kumar <[email protected]>; Grimm, Jon <[email protected]>
Subject: RE: [dpdk-dev] [PATCH 0/4] Extend --lcores to run on cores > RTE_MAX_LCORE

[AMD Public Use]

Thanks Thomas for the response.
For a correction, the patchwork has not been submitted for the current LTS release, 19.11.2, but was merged into 20.02 and onward.
The reason I brought this again was to address LTS users and many other application based on the LTS releases(18.11 & 19.11).
Since I found many of our customers and users are still relying on the latest LTS version, I'm seeking an aid for adding this change into at least 19.11, LTS branch.
We could argue that this is actually not a bug, in a way, it's inconvenient for Openstack or cloud deployed DPDK application since it's often inapt to change the base config and recompile the max lcore limit. If backporting is still not a preferred way(pushing this patchwork into 19.11), then can we instead consider changing only the default value of ' CONFIG_RTE_MAX_LCORE' to 256 in common_base file?

# Compile Environment Abstraction Layer
#
CONFIG_RTE_MAX_LCORE=128 --> 256

I'd appreciate if anyone can advise me know what we can do about this to move forward.

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Monday, June 1, 2020 2:23 PM
To: Song, Keesang <[email protected]>
Cc: David Marchand <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [dpdk-dev] [PATCH 0/4] Extend --lcores to run on cores > RTE_MAX_LCORE

[CAUTION: External Email]

29/05/2020 05:05, Song, Keesang:

Quoted 11 lines

Hi Thomas & David,

We haven't got the final status on this patch, and I don't see this change even from the latest LTS 20.04 repo. So I'd like to confirm whether this patch has been safely submitted to the main upstream.
Can you check the status of that commit?

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatc
hwork.dpdk.org%2Fpatch%2F63507%2F&amp;data=02%7C01%7CKeesang.Song%40am d.com%7Cd71ea9aca917447dfb3e08d80671f34c%7C3dd8961fe4884e608e11a82d994 e183d%7C0%7C0%7C637266433776198364&amp;sdata=1EgxevCILVMMLgyQC%2FzaWYJ XJ%2BOijs0Rtym1TA0VS28%3D&amp;reserved=0

As you can see below, there is a pending question:
"is it a new feature or a fix?"

Kevin and Luca are the arbiters for the backports in 18.11 and 19.11.

Quoted 32 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Friday, February 21, 2020 12:04 AM

Hi,

21/01/2020 01:24, Thomas Monjalon:

Quoted 14 lines

02/12/2019 16:35, David Marchand:

Quoted 5 lines

We are currently stuck with no option but recompile a DPDK if the system has more cores than RTE_MAX_LCORE.
A bit of a pity when you get a system with more than 200+ cores and your testpmd has been built and packaged with RTE_MAX_LCORE == 128.

Quoted 7 lines

The --lcores does not need to care about the underlying cores, remove this limitation.
David Marchand (4):
eal/windows: fix cpuset macro name
eal: do not cache lcore detection state
eal: display all detected cores at startup
eal: remove limitation on cpuset with --lcores

Quoted 4 lines

The patches look good but it is very hard to review parsing code (last patch). We will better experience corner cases after merging.

Applied for -rc1, thanks

This patch was merged in 20.02.
We don't have any feedback about issues so it's probably working fine.

It is solving a problem for running DPDK on machines having a lot of cores. Now the difficult question: is it a new feature or a fix? Should we backport this patchset?