Skip to contentSkip to conversation

mempool: fix mempool obj alignment for non x86

PATCH Superseded [email protected] 9 participants 18 messages Dec 19, 2019 – Jan 20, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mempool: fix mempool obj alignment for non x86

From: Jerin Jacob <[email protected]>

The exiting optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (Mirco) architecture may have different memory alignment constraint for better performance and it not same as the existing optimize_object_size() function. Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

Introducing arch_mem_object_align() function to abstract the differences in different (mirco) architectures and avoid wasting memory for mempool object alignment for the architecture the existing optimize_object_size() is not valid.

Additional details:
https://www.mail-archive.com/[email protected]/msg149157.html

Fixes af75078fece3 ("first public release")
---
 doc/guides/prog_guide/mempool_lib.rst |  6 +++---
 lib/librte_mempool/rte_mempool.c      | 17 +++++++++++++----
 2 files changed, 16 insertions(+), 7 deletions(-)
doc/guides/prog_guide/mempool_lib.rst +3 -3
index 3bb84b0a6..eea7a2906 100644
@@ -27,10 +27,10 @@ In debug mode (CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled),
 statistics about get from/put in the pool are stored in the mempool structure.
 Statistics are per-lcore to avoid concurrent access to statistics counters.
 
-Memory Alignment Constraints
-----------------------------
+Memory Alignment Constraints on X86 architecture
+------------------------------------------------
 
-Depending on hardware memory configuration, performance can be greatly improved by adding a specific padding between objects.
+Depending on hardware memory configuration on X86 architecture, performance can be greatly improved by adding a specific padding between objects.
 The objective is to ensure that the beginning of each object starts on a different channel and rank in memory so that all channels are equally loaded.
 
 This is particularly true for packet buffers when doing L3 forwarding or flow classification.
lib/librte_mempool/rte_mempool.c +13 -4
index 78d8eb941..871894525 100644
@@ -45,6 +45,7 @@ EAL_REGISTER_TAILQ(rte_mempool_tailq)
 #define CALC_CACHE_FLUSHTHRESH(c)	\
 	((typeof(c))((c) * CACHE_FLUSHTHRESH_MULTIPLIER))
 
+#if defined(RTE_ARCH_X86)
 /*
  * return the greatest common divisor between a and b (fast algorithm)
  *
@@ -74,12 +75,13 @@ static unsigned get_gcd(unsigned a, unsigned b)
 }
 
 /*
- * Depending on memory configuration, objects addresses are spread
+ * Depending on memory configuration on x86 arch, objects addresses are spread
  * between channels and ranks in RAM: the pool allocator will add
  * padding between objects. This function return the new size of the
  * object.
  */
-static unsigned optimize_object_size(unsigned obj_size)
+static unsigned
+arch_mem_object_align(unsigned obj_size)
 {
 	unsigned nrank, nchan;
 	unsigned new_obj_size;
@@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned obj_size)
 		new_obj_size++;
 	return new_obj_size * RTE_MEMPOOL_ALIGN;
 }
+#else
+static unsigned
+arch_mem_object_align(unsigned obj_size)
+{
+	return obj_size;
+}
+#endif
 
 struct pagesz_walk_arg {
 	int socket_id;
@@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size, uint32_t flags,
 	 */
 	if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
 		unsigned new_size;
-		new_size = optimize_object_size(sz->header_size + sz->elt_size +
-			sz->trailer_size);
+		new_size = arch_mem_object_align
+			    (sz->header_size + sz->elt_size + sz->trailer_size);
 		sz->trailer_size = new_size - sz->header_size - sz->elt_size;
 	}
 
Signature · 1 line
2.24.1
Reviewed-by on v1 1/1
#1 lore raw

Hi Jerin,

It got two coding style warnings, otherwise,

WARNING:UNSPECIFIED_INT: Prefer 'unsigned int' to bare use of 'unsigned' #144: FILE: lib/librte_mempool/rte_mempool.c:84:
+arch_mem_object_align(unsigned obj_size)

WARNING:UNSPECIFIED_INT: Prefer 'unsigned int' to bare use of 'unsigned' #154: FILE: lib/librte_mempool/rte_mempool.c:106:
+arch_mem_object_align(unsigned obj_size)

#2 lore raw
Quoted 5 lines · Gavin Hu

Hi Jerin,

It got two coding style warnings, otherwise,

Thanks Gavin for review. Since the existing code is using "unsigned" in that file, I thought of not change by this patch.
If someone thinks, It is better to change then I can send v2 by fixing "unsigned" to "unsigned int" across the file as a first patch in the series.

Quoted 8 lines

WARNING:UNSPECIFIED_INT: Prefer 'unsigned int' to bare use of 'unsigned' #144: FILE: lib/librte_mempool/rte_mempool.c:84:
+arch_mem_object_align(unsigned obj_size)

WARNING:UNSPECIFIED_INT: Prefer 'unsigned int' to bare use of 'unsigned' #154: FILE: lib/librte_mempool/rte_mempool.c:106:
+arch_mem_object_align(unsigned obj_size)

#3 lore raw

Re: [dpdk-dev] [PATCH] mempool: fix mempool obj alignment for nonx86

Quoted 8 lines

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Jerin Jacob
Sent: Friday, December 20, 2019 4:45 AM

Quoted 3 lines · Gavin Hu

Hi Jerin,

Quoted 9 lines
Quoted 2 lines

It got two coding style warnings, otherwise,

Thanks Gavin for review. Since the existing code is using "unsigned" in that file, I thought of not change by this patch.
If someone thinks, It is better to change then I can send v2 by fixing "unsigned" to "unsigned int" across the file as a first patch in the series.

The use of the type "unsigned" is a general issue with older DPDK libraries. Anyone touching any of these libraries will get these warnings. They should all be fixed. I'm not saying you should do it; I'm only suggesting that someone should create a dedicated patch to fix them all.

Quoted 10 lines
Quoted 2 lines

WARNING:UNSPECIFIED_INT: Prefer 'unsigned int' to bare use of

'unsigned'

Quoted 4 lines

#144: FILE: lib/librte_mempool/rte_mempool.c:84:
+arch_mem_object_align(unsigned obj_size)

WARNING:UNSPECIFIED_INT: Prefer 'unsigned int' to bare use of

'unsigned'

Quoted 2 lines

#154: FILE: lib/librte_mempool/rte_mempool.c:106:
+arch_mem_object_align(unsigned obj_size)

#4 lore raw
In reply to [email protected]

<snip>

Quoted 11 lines

From: Jerin Jacob <[email protected]>

The exiting optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (Mirco) architecture may have different memory alignment constraint for better performance and it not same as the existing optimize_object_size() function. Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

If I understand correctly, address interleaving is the characteristic of the memory controller and not the CPU.
For ex: different SoCs using the same Arm architecture might have different memory controllers. So, the solution should not be architecture specific, but SoC specific.

Quoted 73 lines

Introducing arch_mem_object_align() function to abstract the differences in different (mirco) architectures and avoid wasting memory for mempool object alignment for the architecture the existing optimize_object_size() is not valid.

Additional details:
https://www.mail-archive.com/[email protected]/msg149157.html

Fixes af75078fece3 ("first public release")
---
 doc/guides/prog_guide/mempool_lib.rst |  6 +++---
 lib/librte_mempool/rte_mempool.c      | 17 +++++++++++++----
 2 files changed, 16 insertions(+), 7 deletions(-)
doc/guides/prog_guide/mempool_lib.rst +0 -0
index 3bb84b0a6..eea7a2906 100644
@@ -27,10 +27,10 @@ In debug mode

(CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled), statistics about get from/put in the pool are stored in the mempool structure.
Statistics are per-lcore to avoid concurrent access to statistics counters.

doc/guides/prog_guide/mempool_lib.rst +2 -3
-Memory Alignment Constraints
-----------------------------
+Memory Alignment Constraints on X86 architecture
+------------------------------------------------
 
-Depending on hardware memory configuration, performance can be greatly

improved by adding a specific padding between objects.
+Depending on hardware memory configuration on X86 architecture, performance can be greatly improved by adding a specific padding between objects.
The objective is to ensure that the beginning of each object starts on a
different channel and rank in memory so that all channels are equally loaded.

This is particularly true for packet buffers when doing L3 forwarding or flow
classification.

lib/librte_mempool/rte_mempool.c +5 -2
index 78d8eb941..871894525 100644
@@ -45,6 +45,7 @@ EAL_REGISTER_TAILQ(rte_mempool_tailq)
 #define CALC_CACHE_FLUSHTHRESH(c)	\
 	((typeof(c))((c) * CACHE_FLUSHTHRESH_MULTIPLIER))
 
+#if defined(RTE_ARCH_X86)
 /*
  * return the greatest common divisor between a and b (fast algorithm)
  *
@@ -74,12 +75,13 @@ static unsigned get_gcd(unsigned a, unsigned b)  }
 
 /*
- * Depending on memory configuration, objects addresses are spread
+ * Depending on memory configuration on x86 arch, objects addresses are
+ spread
  * between channels and ranks in RAM: the pool allocator will add
  * padding between objects. This function return the new size of the
  * object.
  */
-static unsigned optimize_object_size(unsigned obj_size)
+static unsigned
+arch_mem_object_align(unsigned obj_size)
 {
 	unsigned nrank, nchan;
 	unsigned new_obj_size;

lib/librte_mempool/rte_mempool.c static unsigned optimize_object_size(unsigned · line 104

Quoted 4 lines
lib/librte_mempool/rte_mempool.c +1 -0
@@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned
 obj_size)
 		new_obj_size++;
 	return new_obj_size * RTE_MEMPOOL_ALIGN;  }
+#else

This applies to add Arm (PPC as well) SoCs which might have different schemes depending on the memory controller. IMO, this should not be architecture specific.

Quoted 23 lines
lib/librte_mempool/rte_mempool.c +5 -1
+static unsigned
+arch_mem_object_align(unsigned obj_size) {
+	return obj_size;
+}
+#endif
 
 struct pagesz_walk_arg {
 	int socket_id;
@@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size,
 uint32_t flags,
 	 */
 	if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
 		unsigned new_size;
-		new_size = optimize_object_size(sz->header_size + sz-
Quoted 1 line

elt_size +

lib/librte_mempool/rte_mempool.c +2 -2
-			sz->trailer_size);
+		new_size = arch_mem_object_align
+			    (sz->header_size + sz->elt_size + sz->trailer_size);
 		sz->trailer_size = new_size - sz->header_size - sz->elt_size;
 	}
 
--

2.24.1

#5 lore raw
Quoted 9 lines · Honnappa Nagarahalli

<snip>

Quoted 6 lines

From: Jerin Jacob <[email protected]>

The exiting optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Quoted 7 lines
Quoted 5 lines

Different (Mirco) architecture may have different memory alignment constraint for better performance and it not same as the existing optimize_object_size() function. Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

If I understand correctly, address interleaving is the characteristic of the memory controller and not the CPU.
For ex: different SoCs using the same Arm architecture might have different memory controllers. So, the solution should not be architecture specific, but SoC specific.

Yes. See below.

Quoted 4 lines
Quoted 4 lines
lib/librte_mempool/rte_mempool.c +2 -1
-static unsigned optimize_object_size(unsigned obj_size)
+static unsigned
+arch_mem_object_align(unsigned obj_size)
 {
Quoted 8 lines

lib/librte_mempool/rte_mempool.c static unsigned optimize_object_size(unsigned · line 104

Quoted 7 lines
      unsigned nrank, nchan;
      unsigned new_obj_size;
lib/librte_mempool/rte_mempool.c +1 -0
@@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned
 obj_size)
              new_obj_size++;
      return new_obj_size * RTE_MEMPOOL_ALIGN;  }
+#else

This applies to add Arm (PPC as well) SoCs which might have different schemes depending on the memory controller. IMO, this should not be architecture specific.

I agree in principle.
I will summarize the
https://www.mail-archive.com/[email protected]/msg149157.html feedback:

1) For x86 arch, it is architecture-specific
2) For power PC arch, It is architecture-specific
3) For the ARM case, it will be the memory controller specific.
4) For the ARM case, The memory controller is not using the existing x86 arch formula.
5) If it is memory/arch-specific, Can userspace code find the optimal alignment? In the case of octeontx2/arm64,
the memory controller does XOR on PA address which userspace code doesn't have much control.

This patch address the known case of (1), (2), (4) and (5). (2) can be added to this framework when POWER9 folks want it.

We can extend this patch to address (3) if there is a case. Without the actual requirement(If some can share the formula of alignment which is the memory
controller specific and it does not come under (4))) then we can create extra layer for the memory controller and abstraction to probe it.
Again there is no standard way of probing the memory controller in userspace and we need platform #define, which won't work for distribution build.
So solution needs to be arch-specific and then fine-tune to memory controller if possible.

I can work on creating an extra layer of code if some can provide the details of the memory controller and probing mechanism or this patch be extended to support such case if it arises in future.

Thoughts?

Quoted 25 lines
Quoted 24 lines
lib/librte_mempool/rte_mempool.c +5 -1
+static unsigned
+arch_mem_object_align(unsigned obj_size) {
+     return obj_size;
+}
+#endif
 
 struct pagesz_walk_arg {
      int socket_id;
@@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size,
 uint32_t flags,
       */
      if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
              unsigned new_size;
-             new_size = optimize_object_size(sz->header_size + sz-
Quoted 1 line

elt_size +

lib/librte_mempool/rte_mempool.c +2 -2
-                     sz->trailer_size);
+             new_size = arch_mem_object_align
+                         (sz->header_size + sz->elt_size + sz->trailer_size);
              sz->trailer_size = new_size - sz->header_size - sz->elt_size;
      }
 
--

2.24.1

#6 lore raw

<snip>

Quoted 59 lines
Quoted 11 lines
Quoted 10 lines

From: Jerin Jacob <[email protected]>

The exiting optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (Mirco) architecture may have different memory alignment constraint for better performance and it not same as the existing optimize_object_size() function. Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

If I understand correctly, address interleaving is the characteristic of the

memory controller and not the CPU.

Quoted 1 line

For ex: different SoCs using the same Arm architecture might have different

memory controllers. So, the solution should not be architecture specific, but SoC specific.

Yes. See below.

Quoted 4 lines
Quoted 4 lines
lib/librte_mempool/rte_mempool.c +2 -1
-static unsigned optimize_object_size(unsigned obj_size)
+static unsigned
+arch_mem_object_align(unsigned obj_size)
 {
Quoted 8 lines

lib/librte_mempool/rte_mempool.c static unsigned optimize_object_size(unsigned · line 104

Quoted 7 lines
      unsigned nrank, nchan;
      unsigned new_obj_size;
lib/librte_mempool/rte_mempool.c +1 -0
@@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned
 obj_size)
              new_obj_size++;
      return new_obj_size * RTE_MEMPOOL_ALIGN;  }
+#else

This applies to add Arm (PPC as well) SoCs which might have different

schemes depending on the memory controller. IMO, this should not be architecture specific.

I agree in principle.
I will summarize the
https://www.mail-archive.com/[email protected]/msg149157.html feedback:

1) For x86 arch, it is architecture-specific
2) For power PC arch, It is architecture-specific
3) For the ARM case, it will be the memory controller specific.
4) For the ARM case, The memory controller is not using the existing x86 arch formula.
5) If it is memory/arch-specific, Can userspace code find the optimal alignment? In the case of octeontx2/arm64, the memory controller does XOR on PA address which userspace code doesn't have much control.

This patch address the known case of (1), (2), (4) and (5). (2) can be added to this framework when POWER9 folks want it.

We can extend this patch to address (3) if there is a case. Without the actual requirement(If some can share the formula of alignment which is the memory controller specific and it does not come under (4))) then we can create extra layer for the memory controller and abstraction to probe it. Again there is no standard way of probing the memory controller in userspace and we need platform #define, which won't work for distribution build.
So solution needs to be arch-specific and then fine-tune to memory controller if possible.

Quoted 2 lines

I can work on creating an extra layer of code if some can provide the details of the memory controller and probing mechanism or this patch be extended

Inputs for BlueField, DPAAx, ThunderX2 would be helpful.

Quoted 3 lines

to support such case if it arises in future.

Thoughts?

How much memory will this save for your platform? Is it affecting performance?

Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
lib/librte_mempool/rte_mempool.c +5 -1
+static unsigned
+arch_mem_object_align(unsigned obj_size) {
+     return obj_size;
+}
+#endif
 
 struct pagesz_walk_arg {
      int socket_id;
@@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size,
 uint32_t flags,
       */
      if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
              unsigned new_size;
-             new_size = optimize_object_size(sz->header_size + sz-
Quoted 1 line

elt_size +

lib/librte_mempool/rte_mempool.c +3 -2
-                     sz->trailer_size);
+             new_size = arch_mem_object_align
+                         (sz->header_size + sz->elt_size +
+ sz->trailer_size);
              sz->trailer_size = new_size - sz->header_size - sz->elt_size;
      }
 
--

2.24.1

#7 lore raw
Quoted 61 lines · Honnappa Nagarahalli

<snip>

Quoted 59 lines
Quoted 11 lines
Quoted 10 lines

From: Jerin Jacob <[email protected]>

The exiting optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (Mirco) architecture may have different memory alignment constraint for better performance and it not same as the existing optimize_object_size() function. Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

If I understand correctly, address interleaving is the characteristic of the

memory controller and not the CPU.

Quoted 1 line

For ex: different SoCs using the same Arm architecture might have different

memory controllers. So, the solution should not be architecture specific, but SoC specific.

Yes. See below.

Quoted 4 lines
Quoted 4 lines
lib/librte_mempool/rte_mempool.c +2 -1
-static unsigned optimize_object_size(unsigned obj_size)
+static unsigned
+arch_mem_object_align(unsigned obj_size)
 {
Quoted 8 lines

lib/librte_mempool/rte_mempool.c static unsigned optimize_object_size(unsigned · line 104

Quoted 7 lines
      unsigned nrank, nchan;
      unsigned new_obj_size;
lib/librte_mempool/rte_mempool.c +1 -0
@@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned
 obj_size)
              new_obj_size++;
      return new_obj_size * RTE_MEMPOOL_ALIGN;  }
+#else

This applies to add Arm (PPC as well) SoCs which might have different

schemes depending on the memory controller. IMO, this should not be architecture specific.

I agree in principle.
I will summarize the
https://www.mail-archive.com/[email protected]/msg149157.html feedback:

1) For x86 arch, it is architecture-specific
2) For power PC arch, It is architecture-specific
3) For the ARM case, it will be the memory controller specific.
4) For the ARM case, The memory controller is not using the existing x86 arch formula.
5) If it is memory/arch-specific, Can userspace code find the optimal alignment? In the case of octeontx2/arm64, the memory controller does XOR on PA address which userspace code doesn't have much control.

This patch address the known case of (1), (2), (4) and (5). (2) can be added to this framework when POWER9 folks want it.

We can extend this patch to address (3) if there is a case. Without the actual requirement(If some can share the formula of alignment which is the memory controller specific and it does not come under (4))) then we can create extra layer for the memory controller and abstraction to probe it. Again there is no standard way of probing the memory controller in userspace and we need platform #define, which won't work for distribution build.
So solution needs to be arch-specific and then fine-tune to memory controller if possible.

Quoted 3 lines
Quoted 2 lines

I can work on creating an extra layer of code if some can provide the details of the memory controller and probing mechanism or this patch be extended

Inputs for BlueField, DPAAx, ThunderX2 would be helpful.

Yes. Probably memory controller used in n1sdp SoC also.

Quoted 5 lines
Quoted 4 lines

to support such case if it arises in future.

Thoughts?

How much memory will this save for your platform? Is it affecting performance?

No performance difference.

The existing code adding the tailer for each objs. Additional space/Trailer space will be function of number of objects in mempool and its obj_size, its alignment and selected rte_memory_get_nchannel() and rte_memory_get_nrank()

I will wait for inputs from Bluefield, DPAAx, ThunderX2 and n1sdp(if any) for any rework on the patch.

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
lib/librte_mempool/rte_mempool.c +5 -1
+static unsigned
+arch_mem_object_align(unsigned obj_size) {
+     return obj_size;
+}
+#endif
 
 struct pagesz_walk_arg {
      int socket_id;
@@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size,
 uint32_t flags,
       */
      if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
              unsigned new_size;
-             new_size = optimize_object_size(sz->header_size + sz-
Quoted 1 line

elt_size +

lib/librte_mempool/rte_mempool.c +3 -2
-                     sz->trailer_size);
+             new_size = arch_mem_object_align
+                         (sz->header_size + sz->elt_size +
+ sz->trailer_size);
              sz->trailer_size = new_size - sz->header_size - sz->elt_size;
      }
 
--

2.24.1

#8 lore raw

Hi,

Quoted 6 lines · Jerin Jacob
Quoted 4 lines · Honnappa Nagarahalli

<snip>

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

From: Jerin Jacob <[email protected]>

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

The exiting optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (Mirco) architecture may have different memory alignment constraint for better performance and it not same as the existing optimize_object_size() function. Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

typo: Mirco -> Micro
Maybe the whole sentence can be reworded a bit (I think a word is missing).

Quoted 49 lines
Quoted 49 lines
Quoted 49 lines
Quoted 1 line

If I understand correctly, address interleaving is the characteristic of the

memory controller and not the CPU.

Quoted 1 line

For ex: different SoCs using the same Arm architecture might have different

memory controllers. So, the solution should not be architecture specific, but SoC specific.

Yes. See below.

Quoted 4 lines
Quoted 4 lines
lib/librte_mempool/rte_mempool.c +2 -1
-static unsigned optimize_object_size(unsigned obj_size)
+static unsigned
+arch_mem_object_align(unsigned obj_size)
 {
Quoted 8 lines

lib/librte_mempool/rte_mempool.c static unsigned optimize_object_size(unsigned · line 104

Quoted 7 lines
      unsigned nrank, nchan;
      unsigned new_obj_size;
lib/librte_mempool/rte_mempool.c +1 -0
@@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned
 obj_size)
              new_obj_size++;
      return new_obj_size * RTE_MEMPOOL_ALIGN;  }
+#else

This applies to add Arm (PPC as well) SoCs which might have different

schemes depending on the memory controller. IMO, this should not be architecture specific.

I agree in principle.
I will summarize the
https://www.mail-archive.com/[email protected]/msg149157.html feedback:

1) For x86 arch, it is architecture-specific
2) For power PC arch, It is architecture-specific
3) For the ARM case, it will be the memory controller specific.
4) For the ARM case, The memory controller is not using the existing x86 arch formula.
5) If it is memory/arch-specific, Can userspace code find the optimal alignment? In the case of octeontx2/arm64, the memory controller does XOR on PA address which userspace code doesn't have much control.

This patch address the known case of (1), (2), (4) and (5). (2) can be added to this framework when POWER9 folks want it.

We can extend this patch to address (3) if there is a case. Without the actual requirement(If some can share the formula of alignment which is the memory controller specific and it does not come under (4))) then we can create extra layer for the memory controller and abstraction to probe it. Again there is no standard way of probing the memory controller in userspace and we need platform #define, which won't work for distribution build.
So solution needs to be arch-specific and then fine-tune to memory controller if possible.

Quoted 11 lines
Quoted 3 lines
Quoted 2 lines

I can work on creating an extra layer of code if some can provide the details of the memory controller and probing mechanism or this patch be extended

Inputs for BlueField, DPAAx, ThunderX2 would be helpful.

Yes. Probably memory controller used in n1sdp SoC also.

Quoted 5 lines
Quoted 4 lines

to support such case if it arises in future.

Thoughts?

How much memory will this save for your platform? Is it affecting performance?

Currently, I think Arm-based architectures use the default (nchan=4, nrank=1). The worst case is for object whose size (including mempool header) is 2 cache lines, where it is optimized to 3 cache lines (+50%).

Examples for cache lines size = 64:
orig optimized

  64    -> 64           +0%
  128   -> 192          +50%
  192   -> 192          +0%
  256   -> 320          +25%
  320   -> 320          +0%
  384   -> 448          +16%

...
2304 -> 2368 +2.7% (~mbuf size)

Quoted 9 lines

No performance difference.

The existing code adding the tailer for each objs. Additional space/Trailer space will be function of number of objects in mempool and its obj_size, its alignment and selected rte_memory_get_nchannel() and rte_memory_get_nrank()

I will wait for inputs from Bluefield, DPAAx, ThunderX2 and n1sdp(if any) for any rework on the patch.

If there is no performance impact on other supporter Arm-based architectures, I think it is a step in a right direction.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

lib/librte_mempool/rte_mempool.c static unsigned optimize_object_size(unsigned obj_size) · line 110

Quoted 5 lines
lib/librte_mempool/rte_mempool.c +5 -0
+static unsigned
+arch_mem_object_align(unsigned obj_size) {
+     return obj_size;
+}
+#endif

I'd prefer "unsigned int" for new code.
Also, the opening brace should be on a separate line.

The documentation of the MEMPOOL_F_NO_SPREAD flag in the .h could be slightly modified, as you did for the comment above arch_mem_object_align().

Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
lib/librte_mempool/rte_mempool.c +0 -1
 struct pagesz_walk_arg {
      int socket_id;
@@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size,
 uint32_t flags,
       */
      if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
              unsigned new_size;
-             new_size = optimize_object_size(sz->header_size + sz-
Quoted 1 line

elt_size +

lib/librte_mempool/rte_mempool.c +3 -2
-                     sz->trailer_size);
+             new_size = arch_mem_object_align
+                         (sz->header_size + sz->elt_size +
+ sz->trailer_size);
              sz->trailer_size = new_size - sz->header_size - sz->elt_size;
      }
 
--

2.24.1

#9 lore raw

PATCH v2 1/1 mempool: fix mempool obj alignment for non x86

In reply to [email protected]

From: Jerin Jacob <[email protected]>

The existing optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (micro) architecture may have different memory alignment constraint for better performance and it not the same as the existing optimize_object_size().

Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

Introducing arch_mem_object_align() function to abstract the difference between different (micro) architectures to avoid wasting memory for mempool object alignment for the architecture that it is not required to do so.

Details on the amount of memory saving:

Currently, arm64 based architectures use the default (nchan=4, nrank=1). The worst case is for an object whose size (including mempool header) is 2 cache lines, where it is optimized to 3 cache lines (+50%).

Examples for cache lines size = 64:
orig optimized

  64    -> 64           +0%
  128   -> 192          +50%
  192   -> 192          +0%
  256   -> 320          +25%
  320   -> 320          +0%
  384   -> 448          +16%

...
2304 -> 2368 +2.7% (~mbuf size)

Additional details:
https://www.mail-archive.com/[email protected]/msg149157.html

Fixes af75078fece3 ("first public release")

---
v2:
- Changed the return type of arch_mem_object_align() to "unsigned int" from "unsigned" to fix the checkpatch issues (Olivier Matz)
- Updated the comments for MEMPOOL_F_NO_SPREAD (Olivier Matz)
- Update the git comments to share the memory saving details.

 doc/guides/prog_guide/mempool_lib.rst |  6 +++---
 lib/librte_mempool/rte_mempool.c      | 17 +++++++++++++----
 lib/librte_mempool/rte_mempool.h      |  6 +++++-
 3 files changed, 21 insertions(+), 8 deletions(-)
doc/guides/prog_guide/mempool_lib.rst +3 -3
index 3bb84b0a6..eea7a2906 100644
@@ -27,10 +27,10 @@ In debug mode (CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled),
 statistics about get from/put in the pool are stored in the mempool structure.
 Statistics are per-lcore to avoid concurrent access to statistics counters.
 
-Memory Alignment Constraints
-----------------------------
+Memory Alignment Constraints on X86 architecture
+------------------------------------------------
 
-Depending on hardware memory configuration, performance can be greatly improved by adding a specific padding between objects.
+Depending on hardware memory configuration on X86 architecture, performance can be greatly improved by adding a specific padding between objects.
 The objective is to ensure that the beginning of each object starts on a different channel and rank in memory so that all channels are equally loaded.
 
 This is particularly true for packet buffers when doing L3 forwarding or flow classification.
lib/librte_mempool/rte_mempool.c +13 -4
index 78d8eb941..1909998e8 100644
@@ -45,6 +45,7 @@ EAL_REGISTER_TAILQ(rte_mempool_tailq)
 #define CALC_CACHE_FLUSHTHRESH(c)	\
 	((typeof(c))((c) * CACHE_FLUSHTHRESH_MULTIPLIER))
 
+#if defined(RTE_ARCH_X86)
 /*
  * return the greatest common divisor between a and b (fast algorithm)
  *
@@ -74,12 +75,13 @@ static unsigned get_gcd(unsigned a, unsigned b)
 }
 
 /*
- * Depending on memory configuration, objects addresses are spread
+ * Depending on memory configuration on x86 arch, objects addresses are spread
  * between channels and ranks in RAM: the pool allocator will add
  * padding between objects. This function return the new size of the
  * object.
  */
-static unsigned optimize_object_size(unsigned obj_size)
+static unsigned int
+arch_mem_object_align(unsigned int obj_size)
 {
 	unsigned nrank, nchan;
 	unsigned new_obj_size;
@@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned obj_size)
 		new_obj_size++;
 	return new_obj_size * RTE_MEMPOOL_ALIGN;
 }
+#else
+static unsigned int
+arch_mem_object_align(unsigned int obj_size)
+{
+	return obj_size;
+}
+#endif
 
 struct pagesz_walk_arg {
 	int socket_id;
@@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size, uint32_t flags,
 	 */
 	if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
 		unsigned new_size;
-		new_size = optimize_object_size(sz->header_size + sz->elt_size +
-			sz->trailer_size);
+		new_size = arch_mem_object_align
+			    (sz->header_size + sz->elt_size + sz->trailer_size);
 		sz->trailer_size = new_size - sz->header_size - sz->elt_size;
 	}
 
lib/librte_mempool/rte_mempool.h +5 -1
index f81152af9..9a4411110 100644
@@ -260,7 +260,11 @@ struct rte_mempool {
 #endif
 }  __rte_cache_aligned;
 
-#define MEMPOOL_F_NO_SPREAD      0x0001 /**< Do not spread among memory channels. */
+#define MEMPOOL_F_NO_SPREAD      0x0001
+/**< Do not spread among memory channels. It is a hint to the library,
+ * library honor this hint only when, if it is required by the
+ * (micro) architecture.
+ */
 #define MEMPOOL_F_NO_CACHE_ALIGN 0x0002 /**< Do not align objs on cache lines.*/
 #define MEMPOOL_F_SP_PUT         0x0004 /**< Default put is "single-producer".*/
 #define MEMPOOL_F_SC_GET         0x0008 /**< Default get is "single-consumer".*/
Signature · 1 line
2.24.1
#10 lore raw

lib/librte_mempool/rte_mempool.h struct rte_mempool { · line 267

Quoted 6 lines · On Sat, 11 Jan 2020 19:04:10 +0530 <[email protected]> wrote:
lib/librte_mempool/rte_mempool.h +5 -1
-#define MEMPOOL_F_NO_SPREAD      0x0001 /**< Do not spread among memory channels. */
+#define MEMPOOL_F_NO_SPREAD      0x0001
+/**< Do not spread among memory channels. It is a hint to the library,
+ * library honor this hint only when, if it is required by the
+ * (micro) architecture.
+ */

That text is awkward for me to read.
There maybe other reasons in future for mempool to ignore the flag.

I prefer the minor change original comment as:

Quoted 1 line

-#define MEMPOOL_F_NO_SPREAD 0x0001 /**< Spreading among memory channels not required. */

#11 lore raw

PATCH v3 1/1 mempool: fix mempool obj alignment for non x86

In reply to [email protected]

From: Jerin Jacob <[email protected]>

The existing optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (micro) architecture may have different memory alignment constraint for better performance and it not the same as the existing optimize_object_size().

Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

Introducing arch_mem_object_align() function to abstract the difference between different (micro) architectures to avoid wasting memory for mempool object alignment for the architecture that it is not required to do so.

Details on the amount of memory saving:

Currently, arm64 based architectures use the default (nchan=4, nrank=1). The worst case is for an object whose size (including mempool header) is 2 cache lines, where it is optimized to 3 cache lines (+50%).

Examples for cache lines size = 64:
orig optimized

  64    -> 64           +0%
  128   -> 192          +50%
  192   -> 192          +0%
  256   -> 320          +25%
  320   -> 320          +0%
  384   -> 448          +16%

...
2304 -> 2368 +2.7% (~mbuf size)

Additional details:
https://www.mail-archive.com/[email protected]/msg149157.html

Fixes af75078fece3 ("first public release")

---
v3:
- Change comment for MEMPOOL_F_NO_SPREAD flag as
" Spreading among memory channels not required." (Stephen Hemminger)

v2:
- Changed the return type of arch_mem_object_align() to "unsigned int" from "unsigned" to fix the checkpatch issues (Olivier Matz)
- Updated the comments for MEMPOOL_F_NO_SPREAD (Olivier Matz)
- Update the git comments to share the memory saving details.

 doc/guides/prog_guide/mempool_lib.rst |  6 +++---
 lib/librte_mempool/rte_mempool.c      | 17 +++++++++++++----
 lib/librte_mempool/rte_mempool.h      |  3 ++-
 3 files changed, 18 insertions(+), 8 deletions(-)
doc/guides/prog_guide/mempool_lib.rst +3 -3
index 3bb84b0a6..eea7a2906 100644
@@ -27,10 +27,10 @@ In debug mode (CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled),
 statistics about get from/put in the pool are stored in the mempool structure.
 Statistics are per-lcore to avoid concurrent access to statistics counters.
 
-Memory Alignment Constraints
-----------------------------
+Memory Alignment Constraints on X86 architecture
+------------------------------------------------
 
-Depending on hardware memory configuration, performance can be greatly improved by adding a specific padding between objects.
+Depending on hardware memory configuration on X86 architecture, performance can be greatly improved by adding a specific padding between objects.
 The objective is to ensure that the beginning of each object starts on a different channel and rank in memory so that all channels are equally loaded.
 
 This is particularly true for packet buffers when doing L3 forwarding or flow classification.
lib/librte_mempool/rte_mempool.c +13 -4
index 78d8eb941..1909998e8 100644
@@ -45,6 +45,7 @@ EAL_REGISTER_TAILQ(rte_mempool_tailq)
 #define CALC_CACHE_FLUSHTHRESH(c)	\
 	((typeof(c))((c) * CACHE_FLUSHTHRESH_MULTIPLIER))
 
+#if defined(RTE_ARCH_X86)
 /*
  * return the greatest common divisor between a and b (fast algorithm)
  *
@@ -74,12 +75,13 @@ static unsigned get_gcd(unsigned a, unsigned b)
 }
 
 /*
- * Depending on memory configuration, objects addresses are spread
+ * Depending on memory configuration on x86 arch, objects addresses are spread
  * between channels and ranks in RAM: the pool allocator will add
  * padding between objects. This function return the new size of the
  * object.
  */
-static unsigned optimize_object_size(unsigned obj_size)
+static unsigned int
+arch_mem_object_align(unsigned int obj_size)
 {
 	unsigned nrank, nchan;
 	unsigned new_obj_size;
@@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned obj_size)
 		new_obj_size++;
 	return new_obj_size * RTE_MEMPOOL_ALIGN;
 }
+#else
+static unsigned int
+arch_mem_object_align(unsigned int obj_size)
+{
+	return obj_size;
+}
+#endif
 
 struct pagesz_walk_arg {
 	int socket_id;
@@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size, uint32_t flags,
 	 */
 	if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
 		unsigned new_size;
-		new_size = optimize_object_size(sz->header_size + sz->elt_size +
-			sz->trailer_size);
+		new_size = arch_mem_object_align
+			    (sz->header_size + sz->elt_size + sz->trailer_size);
 		sz->trailer_size = new_size - sz->header_size - sz->elt_size;
 	}
 
lib/librte_mempool/rte_mempool.h +2 -1
index f81152af9..f48c94def 100644
@@ -260,7 +260,8 @@ struct rte_mempool {
 #endif
 }  __rte_cache_aligned;
 
-#define MEMPOOL_F_NO_SPREAD      0x0001 /**< Do not spread among memory channels. */
+#define MEMPOOL_F_NO_SPREAD      0x0001
+/**< Spreading among memory channels not required. */
 #define MEMPOOL_F_NO_CACHE_ALIGN 0x0002 /**< Do not align objs on cache lines.*/
 #define MEMPOOL_F_SP_PUT         0x0004 /**< Default put is "single-producer".*/
 #define MEMPOOL_F_SC_GET         0x0008 /**< Default get is "single-consumer".*/
Signature · 1 line
2.24.1
#12 lore raw
Quoted 32 lines · On Mon, Jan 13, 2020 at 7:49 AM <[email protected]> wrote:

From: Jerin Jacob <[email protected]>

The existing optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (micro) architecture may have different memory alignment constraint for better performance and it not the same as the existing optimize_object_size().

Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

Introducing arch_mem_object_align() function to abstract the difference between different (micro) architectures to avoid wasting memory for mempool object alignment for the architecture that it is not required to do so.

Details on the amount of memory saving:

Currently, arm64 based architectures use the default (nchan=4, nrank=1). The worst case is for an object whose size (including mempool header) is 2 cache lines, where it is optimized to 3 cache lines (+50%).

Examples for cache lines size = 64:
orig optimized

  64    -> 64           +0%
  128   -> 192          +50%
  192   -> 192          +0%
  256   -> 320          +25%
  320   -> 320          +0%
  384   -> 448          +16%
Quoted 7 lines

...
2304 -> 2368 +2.7% (~mbuf size)

Additional details:
https://www.mail-archive.com/[email protected]/msg149157.html

Fixes af75078fece3 ("first public release")

Weird to flag this as a problem in this sha1.
x86 was the only architecture supported at the time. Either we mark the introduction of new architectures as the point of backport, or we remove this tag and just let Cc: [email protected]

Quoted 1 line

It seems more an optimisation than a fix to me, but in any case, the stable maintainers will be the judges.

Quoted 24 lines

---
v3:
- Change comment for MEMPOOL_F_NO_SPREAD flag as
" Spreading among memory channels not required." (Stephen Hemminger)

v2:
- Changed the return type of arch_mem_object_align() to "unsigned int" from "unsigned" to fix the checkpatch issues (Olivier Matz)
- Updated the comments for MEMPOOL_F_NO_SPREAD (Olivier Matz)
- Update the git comments to share the memory saving details.

 doc/guides/prog_guide/mempool_lib.rst |  6 +++---
 lib/librte_mempool/rte_mempool.c      | 17 +++++++++++++----
 lib/librte_mempool/rte_mempool.h      |  3 ++-
 3 files changed, 18 insertions(+), 8 deletions(-)
doc/guides/prog_guide/mempool_lib.rst +0 -0
index 3bb84b0a6..eea7a2906 100644

doc/guides/prog_guide/mempool_lib.rst In debug mode (CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled), · line 31

Quoted 7 lines
doc/guides/prog_guide/mempool_lib.rst +2 -2
@@ -27,10 +27,10 @@ In debug mode (CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled),
 statistics about get from/put in the pool are stored in the mempool structure.
 Statistics are per-lcore to avoid concurrent access to statistics counters.
 
-Memory Alignment Constraints
-----------------------------
+Memory Alignment Constraints on X86 architecture
+------------------------------------------------

Nit: afaics in the docs, x86 is preferred to X86.

Quoted 6 lines
doc/guides/prog_guide/mempool_lib.rst +1 -1
-Depending on hardware memory configuration, performance can be greatly improved by adding a specific padding between objects.
+Depending on hardware memory configuration on X86 architecture, performance can be greatly improved by adding a specific padding between objects.
 The objective is to ensure that the beginning of each object starts on a different channel and rank in memory so that all channels are equally loaded.
 
 This is particularly true for packet buffers when doing L3 forwarding or flow classification.
Signature · 1 line
David Marchand
#13 lore raw
Quoted 66 lines

-----Original Message-----
From: David Marchand <[email protected]>
Sent: Monday, January 13, 2020 3:17 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: dev <[email protected]>; Thomas Monjalon <[email protected]>; Olivier Matz <[email protected]>; Andrew Rybchenko <[email protected]>; Bruce Richardson <[email protected]>; Ananyev, Konstantin <[email protected]>; Hemant Agrawal <[email protected]>; Shahaf Shuler <[email protected]>; Honnappa Nagarahalli <[email protected]>; Gavin Hu <[email protected]>; [email protected]; David Christensen <[email protected]>; Burakov, Anatoly <[email protected]>; dpdk stable <[email protected]>; Kevin Traynor <[email protected]>; Luca Boccassi <[email protected]>
Subject: [EXT] Re: [dpdk-stable] [dpdk-dev] [PATCH v3] mempool: fix mempool obj alignment for non x86

External Email

----------------------------------------------------------------------

Quoted 33 lines · On Mon, Jan 13, 2020 at 7:49 AM <[email protected]> wrote:

From: Jerin Jacob <[email protected]>

The existing optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (micro) architecture may have different memory alignment constraint for better performance and it not the same as the existing optimize_object_size().

Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

Introducing arch_mem_object_align() function to abstract the difference between different (micro) architectures to avoid wasting memory for mempool object alignment for the architecture that it is not required to do so.

Details on the amount of memory saving:

Currently, arm64 based architectures use the default (nchan=4, nrank=1). The worst case is for an object whose size (including mempool
header) is 2 cache lines, where it is optimized to 3 cache lines (+50%).

Examples for cache lines size = 64:
orig optimized

  64    -> 64           +0%
  128   -> 192          +50%
  192   -> 192          +0%
  256   -> 320          +25%
  320   -> 320          +0%
  384   -> 448          +16%
Quoted 6 lines

...
2304 -> 2368 +2.7% (~mbuf size)

Additional details:
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mail-2Darchiv
e.com_dev-

40dpdk.org_msg149157.html&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&

Quoted 1 line

r=1DGob4H4rxz6H8uITozGOCa0s5f4wCNtTa4UUKvcsvI&m=VKkiHhyflsqwipCoE MtdUR

Quoted 1 line

SXuHSq2neWGqTRmxVfjr8&s=y-LYGZ-

Quoted 8 lines

2MsAfrGo3r5aADQnr2mUcsP7LxXT5XEmTuwE&e=

Quoted 2 lines
Fixes af75078fece3 ("first public release")

Weird to flag this as a problem in this sha1.
x86 was the only architecture supported at the time.
Either we mark the introduction of new architectures as the point of backport, or we remove this tag and just let Cc: [email protected]

While committing the maintainer can take either one of the decision. No issues/opinion on this from my side.

Quoted 5 lines
Quoted 2 lines

It seems more an optimisation than a fix to me, but in any case, the stable maintainers will be the judges.

OK. No issues.

Quoted 55 lines

doc/guides/prog_guide/mempool_lib.rst In debug mode

Quoted 27 lines

---
v3:
- Change comment for MEMPOOL_F_NO_SPREAD flag as " Spreading among memory channels not required." (Stephen Hemminger)

v2:
- Changed the return type of arch_mem_object_align() to "unsigned int" from "unsigned" to fix the checkpatch issues (Olivier Matz)
- Updated the comments for MEMPOOL_F_NO_SPREAD (Olivier Matz)
- Update the git comments to share the memory saving details.

 doc/guides/prog_guide/mempool_lib.rst |  6 +++---
 lib/librte_mempool/rte_mempool.c      | 17 +++++++++++++----
 lib/librte_mempool/rte_mempool.h      |  3 ++-
 3 files changed, 18 insertions(+), 8 deletions(-)
doc/guides/prog_guide/mempool_lib.rst +0 -0
index 3bb84b0a6..eea7a2906 100644
@@ -27,10 +27,10 @@ In debug mode

(CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is

Quoted 1 line

enabled), statistics about get from/put in the pool are stored in the mempool

structure.

doc/guides/prog_guide/mempool_lib.rst In debug mode (CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled), · line 31

Quoted 6 lines

Statistics are per-lcore to avoid concurrent access to statistics counters.

doc/guides/prog_guide/mempool_lib.rst +2 -2
-Memory Alignment Constraints
-----------------------------
+Memory Alignment Constraints on X86 architecture
+------------------------------------------------

Nit: afaics in the docs, x86 is preferred to X86.

Quoted 2 lines

-Depending on hardware memory configuration, performance can be greatly

improved by adding a specific padding between objects.

Quoted 1 line

+Depending on hardware memory configuration on X86 architecture,

performance can be greatly improved by adding a specific padding between objects.

Quoted 1 line

The objective is to ensure that the beginning of each object starts on a

different channel and rank in memory so that all channels are equally loaded.

Quoted 2 lines

This is particularly true for packet buffers when doing L3 forwarding or flow

classification.

--
David Marchand

#14 lore raw

PATCH v4 1/1 mempool: remove memory wastage on non x86

In reply to [email protected]

From: Jerin Jacob <[email protected]>

The existing optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (micro) architecture may have different memory alignment constraint for better performance and it not the same as the existing optimize_object_size().

Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

Introducing arch_mem_object_align() function to abstract the difference between different (micro) architectures to avoid wasting memory for mempool object alignment for the architecture that it is not required to do so.

Details on the amount of memory saving:

Currently, arm64 based architectures use the default (nchan=4, nrank=1). The worst case is for an object whose size (including mempool header) is 2 cache lines, where it is optimized to 3 cache lines (+50%).

Examples for cache lines size = 64:
orig optimized

  64    -> 64           +0%
  128   -> 192          +50%
  192   -> 192          +0%
  256   -> 320          +25%
  320   -> 320          +0%
  384   -> 448          +16%

...
2304 -> 2368 +2.7% (~mbuf size)

Additional details:
https://www.mail-archive.com/[email protected]/msg149157.html

---
v4:

- s/X86/x86 in the doc/guides/prog_guide/mempool_lib.rst file (David)
- Remove Fixes and CC: [email protected] as well, as this is just a improvement.(David)
- Udpate the patch heading to remove checkpatch warning due to above change.

v3:
- Change comment for MEMPOOL_F_NO_SPREAD flag as
" Spreading among memory channels not required." (Stephen Hemminger)

v2:
- Changed the return type of arch_mem_object_align() to "unsigned int" from "unsigned" to fix the checkpatch issues (Olivier Matz)
- Updated the comments for MEMPOOL_F_NO_SPREAD (Olivier Matz)
- Update the git comments to share the memory saving details.

 doc/guides/prog_guide/mempool_lib.rst |  6 +++---
 lib/librte_mempool/rte_mempool.c      | 17 +++++++++++++----
 lib/librte_mempool/rte_mempool.h      |  3 ++-
 3 files changed, 18 insertions(+), 8 deletions(-)
doc/guides/prog_guide/mempool_lib.rst +3 -3
index 3bb84b0a6..f8b430d65 100644
@@ -27,10 +27,10 @@ In debug mode (CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled),
 statistics about get from/put in the pool are stored in the mempool structure.
 Statistics are per-lcore to avoid concurrent access to statistics counters.
 
-Memory Alignment Constraints
-----------------------------
+Memory Alignment Constraints on x86 architecture
+------------------------------------------------
 
-Depending on hardware memory configuration, performance can be greatly improved by adding a specific padding between objects.
+Depending on hardware memory configuration on X86 architecture, performance can be greatly improved by adding a specific padding between objects.
 The objective is to ensure that the beginning of each object starts on a different channel and rank in memory so that all channels are equally loaded.
 
 This is particularly true for packet buffers when doing L3 forwarding or flow classification.
lib/librte_mempool/rte_mempool.c +13 -4
index 78d8eb941..1909998e8 100644
@@ -45,6 +45,7 @@ EAL_REGISTER_TAILQ(rte_mempool_tailq)
 #define CALC_CACHE_FLUSHTHRESH(c)	\
 	((typeof(c))((c) * CACHE_FLUSHTHRESH_MULTIPLIER))
 
+#if defined(RTE_ARCH_X86)
 /*
  * return the greatest common divisor between a and b (fast algorithm)
  *
@@ -74,12 +75,13 @@ static unsigned get_gcd(unsigned a, unsigned b)
 }
 
 /*
- * Depending on memory configuration, objects addresses are spread
+ * Depending on memory configuration on x86 arch, objects addresses are spread
  * between channels and ranks in RAM: the pool allocator will add
  * padding between objects. This function return the new size of the
  * object.
  */
-static unsigned optimize_object_size(unsigned obj_size)
+static unsigned int
+arch_mem_object_align(unsigned int obj_size)
 {
 	unsigned nrank, nchan;
 	unsigned new_obj_size;
@@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned obj_size)
 		new_obj_size++;
 	return new_obj_size * RTE_MEMPOOL_ALIGN;
 }
+#else
+static unsigned int
+arch_mem_object_align(unsigned int obj_size)
+{
+	return obj_size;
+}
+#endif
 
 struct pagesz_walk_arg {
 	int socket_id;
@@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size, uint32_t flags,
 	 */
 	if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
 		unsigned new_size;
-		new_size = optimize_object_size(sz->header_size + sz->elt_size +
-			sz->trailer_size);
+		new_size = arch_mem_object_align
+			    (sz->header_size + sz->elt_size + sz->trailer_size);
 		sz->trailer_size = new_size - sz->header_size - sz->elt_size;
 	}
 
lib/librte_mempool/rte_mempool.h +2 -1
index f81152af9..f48c94def 100644
@@ -260,7 +260,8 @@ struct rte_mempool {
 #endif
 }  __rte_cache_aligned;
 
-#define MEMPOOL_F_NO_SPREAD      0x0001 /**< Do not spread among memory channels. */
+#define MEMPOOL_F_NO_SPREAD      0x0001
+/**< Spreading among memory channels not required. */
 #define MEMPOOL_F_NO_CACHE_ALIGN 0x0002 /**< Do not align objs on cache lines.*/
 #define MEMPOOL_F_SP_PUT         0x0004 /**< Default put is "single-producer".*/
 #define MEMPOOL_F_SC_GET         0x0008 /**< Default get is "single-consumer".*/
Signature · 1 line
2.24.1
#15 lore raw

Re: [dpdk-dev] [PATCH v4] mempool: remove memory wastage on non x86

Quoted 32 lines · On Wed, Jan 15, 2020 at 2:35 AM <[email protected]> wrote:

From: Jerin Jacob <[email protected]>

The existing optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (micro) architecture may have different memory alignment constraint for better performance and it not the same as the existing optimize_object_size().

Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

Introducing arch_mem_object_align() function to abstract the difference between different (micro) architectures to avoid wasting memory for mempool object alignment for the architecture that it is not required to do so.

Details on the amount of memory saving:

Currently, arm64 based architectures use the default (nchan=4, nrank=1). The worst case is for an object whose size (including mempool header) is 2 cache lines, where it is optimized to 3 cache lines (+50%).

Examples for cache lines size = 64:
orig optimized

  64    -> 64           +0%
  128   -> 192          +50%
  192   -> 192          +0%
  256   -> 320          +25%
  320   -> 320          +0%
  384   -> 448          +16%
Quoted 8 lines

...
2304 -> 2368 +2.7% (~mbuf size)

Additional details:
https://www.mail-archive.com/[email protected]/msg149157.html

Ping for merge.

Quoted 112 lines

---
v4:

- s/X86/x86 in the doc/guides/prog_guide/mempool_lib.rst file (David)
- Remove Fixes and CC: [email protected] as well, as this is just a improvement.(David)
- Udpate the patch heading to remove checkpatch warning due to above change.

v3:
- Change comment for MEMPOOL_F_NO_SPREAD flag as
" Spreading among memory channels not required." (Stephen Hemminger)

v2:
- Changed the return type of arch_mem_object_align() to "unsigned int" from "unsigned" to fix the checkpatch issues (Olivier Matz)
- Updated the comments for MEMPOOL_F_NO_SPREAD (Olivier Matz)
- Update the git comments to share the memory saving details.

 doc/guides/prog_guide/mempool_lib.rst |  6 +++---
 lib/librte_mempool/rte_mempool.c      | 17 +++++++++++++----
 lib/librte_mempool/rte_mempool.h      |  3 ++-
 3 files changed, 18 insertions(+), 8 deletions(-)
doc/guides/prog_guide/mempool_lib.rst +3 -3
index 3bb84b0a6..f8b430d65 100644
@@ -27,10 +27,10 @@ In debug mode (CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled),
 statistics about get from/put in the pool are stored in the mempool structure.
 Statistics are per-lcore to avoid concurrent access to statistics counters.
 
-Memory Alignment Constraints
-----------------------------
+Memory Alignment Constraints on x86 architecture
+------------------------------------------------
 
-Depending on hardware memory configuration, performance can be greatly improved by adding a specific padding between objects.
+Depending on hardware memory configuration on X86 architecture, performance can be greatly improved by adding a specific padding between objects.
 The objective is to ensure that the beginning of each object starts on a different channel and rank in memory so that all channels are equally loaded.
 
 This is particularly true for packet buffers when doing L3 forwarding or flow classification.
lib/librte_mempool/rte_mempool.c +13 -4
index 78d8eb941..1909998e8 100644
@@ -45,6 +45,7 @@ EAL_REGISTER_TAILQ(rte_mempool_tailq)
 #define CALC_CACHE_FLUSHTHRESH(c)      \
        ((typeof(c))((c) * CACHE_FLUSHTHRESH_MULTIPLIER))
 
+#if defined(RTE_ARCH_X86)
 /*
  * return the greatest common divisor between a and b (fast algorithm)
  *
@@ -74,12 +75,13 @@ static unsigned get_gcd(unsigned a, unsigned b)
 }
 
 /*
- * Depending on memory configuration, objects addresses are spread
+ * Depending on memory configuration on x86 arch, objects addresses are spread
  * between channels and ranks in RAM: the pool allocator will add
  * padding between objects. This function return the new size of the
  * object.
  */
-static unsigned optimize_object_size(unsigned obj_size)
+static unsigned int
+arch_mem_object_align(unsigned int obj_size)
 {
        unsigned nrank, nchan;
        unsigned new_obj_size;
@@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned obj_size)
                new_obj_size++;
        return new_obj_size * RTE_MEMPOOL_ALIGN;
 }
+#else
+static unsigned int
+arch_mem_object_align(unsigned int obj_size)
+{
+       return obj_size;
+}
+#endif
 
 struct pagesz_walk_arg {
        int socket_id;
@@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size, uint32_t flags,
         */
        if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
                unsigned new_size;
-               new_size = optimize_object_size(sz->header_size + sz->elt_size +
-                       sz->trailer_size);
+               new_size = arch_mem_object_align
+                           (sz->header_size + sz->elt_size + sz->trailer_size);
                sz->trailer_size = new_size - sz->header_size - sz->elt_size;
        }
 
lib/librte_mempool/rte_mempool.h +2 -2
index f81152af9..f48c94def 100644
@@ -260,7 +260,8 @@ struct rte_mempool {
 #endif
 }  __rte_cache_aligned;
 
-#define MEMPOOL_F_NO_SPREAD      0x0001 /**< Do not spread among memory channels. */
+#define MEMPOOL_F_NO_SPREAD      0x0001
+/**< Spreading among memory channels not required. */
 #define MEMPOOL_F_NO_CACHE_ALIGN 0x0002 /**< Do not align objs on cache lines.*/
 #define MEMPOOL_F_SP_PUT         0x0004 /**< Default put is "single-producer".*/
 #define MEMPOOL_F_SC_GET         0x0008 /**< Default get is "single-consumer".*/
--

2.24.1

Acked-by
#16 lore raw

Re: [dpdk-dev] [PATCH v4] mempool: remove memory wastage on non x86

Quoted 36 lines · Jerin Jacob
Quoted 35 lines · On Wed, Jan 15, 2020 at 2:35 AM <[email protected]> wrote:

From: Jerin Jacob <[email protected]>

The existing optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (micro) architecture may have different memory alignment constraint for better performance and it not the same as the existing optimize_object_size().

Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

Introducing arch_mem_object_align() function to abstract the difference between different (micro) architectures to avoid wasting memory for mempool object alignment for the architecture that it is not required to do so.

Details on the amount of memory saving:

Currently, arm64 based architectures use the default (nchan=4, nrank=1). The worst case is for an object whose size (including mempool header) is 2 cache lines, where it is optimized to 3 cache lines (+50%).

Examples for cache lines size = 64:
orig optimized

  64    -> 64           +0%
  128   -> 192          +50%
  192   -> 192          +0%
  256   -> 320          +25%
  320   -> 320          +0%
  384   -> 448          +16%

...
2304 -> 2368 +2.7% (~mbuf size)

Quoted 7 lines
Quoted 5 lines

Additional details:
https://www.mail-archive.com/[email protected]/msg149157.html

Ping for merge.

#17 lore raw

Re: [dpdk-dev] [PATCH v4] mempool: remove memory wastage on non x86

20/01/2020 13:24, Olivier Matz:

Quoted 40 lines · Jerin Jacob
Quoted 39 lines · On Wed, Jan 15, 2020 at 2:35 AM <[email protected]> wrote:
Quoted 38 lines

From: Jerin Jacob <[email protected]>

The existing optimize_object_size() function address the memory object alignment constraint on x86 for better performance.

Different (micro) architecture may have different memory alignment constraint for better performance and it not the same as the existing optimize_object_size().

Some use, XOR(kind of CRC) scheme to enable DRAM channel distribution based on the address and some may have a different formula.

Introducing arch_mem_object_align() function to abstract the difference between different (micro) architectures to avoid wasting memory for mempool object alignment for the architecture that it is not required to do so.

Details on the amount of memory saving:

Currently, arm64 based architectures use the default (nchan=4, nrank=1). The worst case is for an object whose size (including mempool header) is 2 cache lines, where it is optimized to 3 cache lines (+50%).

Examples for cache lines size = 64:
orig optimized

  64    -> 64           +0%
  128   -> 192          +50%
  192   -> 192          +0%
  256   -> 320          +25%
  320   -> 320          +0%
  384   -> 448          +16%

...
2304 -> 2368 +2.7% (~mbuf size)

Additional details:
https://www.mail-archive.com/[email protected]/msg149157.html

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

Ping for merge.

Applied, thanks