Skip to contentSkip to conversation

mempool: Add sanity check when secondary link in less mempools than primary

PATCH Superseded Jean Tourrilhes 2 participants 3 messages Oct 12, 2016 – Oct 14, 2016

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mempool: Add sanity check when secondary link in less mempools than primary

mempool: Add sanity check when secondary link in less mempools than primary

If the primary and secondary process were build using different build systems, the list of constructors included by the linker in each binary might be different. Mempools are registered via constructors, so the linker magic will directly impact which tailqs are registered with the primary and the secondary.

DPDK currently assumes that the secondary has a superset of the mempools registered at the primary, and they are in the same order (same index in primary and secondary). In some build scenario, the secondary might not initialise any mempools at all. This would result in an obscure segfault when trying to use the mempool. Instead, fail early with a more explicit error message.

---
 lib/librte_mempool/rte_mempool.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
lib/librte_mempool/rte_mempool.c +10 -0
index 2e28e2e..4fe9158 100644
@@ -1275,6 +1275,16 @@ rte_mempool_lookup(const char *name)
 		return NULL;
 	}
 
+	/* Sanity check : secondary may have initialised less mempools
+	 * than primary due to linker and constructor magic. Note that
+	 * this does not address the case where the constructor order
+	 * is different between primary and secondary and where the index
+	 * points to the wrong ops. Jean II */
+	if(mp->ops_index >= (int32_t) rte_mempool_ops_table.num_ops) {
+		/* Do not dump mempool list, it will segfault. */
+		rte_panic("Cannot find ops for mempool, ops_index %d, num_ops %d - maybe due to build process or linker configuration\n", mp->ops_index, rte_mempool_ops_table.num_ops);
+	}
+
 	return mp;
 }
 
#1 lore raw

Hi Jean,

Quoted 32 lines · Jean Tourrilhes

mempool: Add sanity check when secondary link in less mempools than primary

If the primary and secondary process were build using different build systems, the list of constructors included by the linker in each binary might be different. Mempools are registered via constructors, so the linker magic will directly impact which tailqs are registered with the primary and the secondary.

DPDK currently assumes that the secondary has a superset of the mempools registered at the primary, and they are in the same order (same index in primary and secondary). In some build scenario, the secondary might not initialise any mempools at all. This would result in an obscure segfault when trying to use the mempool. Instead, fail early with a more explicit error message.

---
 lib/librte_mempool/rte_mempool.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
lib/librte_mempool/rte_mempool.c +3 -0
index 2e28e2e..4fe9158 100644
@@ -1275,6 +1275,16 @@ rte_mempool_lookup(const char *name)
 		return NULL;
 	}
 
+	/* Sanity check : secondary may have initialised less mempools
+	 * than primary due to linker and constructor magic. Note that
+	 * this does not address the case where the constructor order

lib/librte_mempool/rte_mempool.c rte_mempool_lookup(const char *name) · line 1290

Quoted 10 lines
lib/librte_mempool/rte_mempool.c +7 -0
+	 * is different between primary and secondary and where the index
+	 * points to the wrong ops. Jean II */
+	if(mp->ops_index >= (int32_t) rte_mempool_ops_table.num_ops) {
+		/* Do not dump mempool list, it will segfault. */
+		rte_panic("Cannot find ops for mempool, ops_index %d, num_ops %d - maybe due to build process or linker configuration\n", mp->ops_index, rte_mempool_ops_table.num_ops);
+	}
+
 	return mp;
 }
 

I'm not really fan of this. I think the configuration and build system of primary and secondaries should be the same to avoid this kind of issues. Some other issues may happen if the configuration is different, for instance the size of structures may be different.

There is already a lot of mess due to primary/secondary at many places in the code, I'm not sure adding more is really desirable.

Regards,
Olivier

#2 lore raw
Quoted 5 lines · Olivier Matz

Hi Jean,

I'm not really fan of this. I think the configuration and build system of primary and secondaries should be the same to avoid this kind of issues.

You are not going to convert all existing applications to the
DPDK build system. I believe that restricting the build system is irrealistic, it would restrict DPDK secondary only to toy examples.
Note that libdpdk.a is tricky to use outside the DPDK build
system and require some quirks even for primary applications (see Snort DPDK patches). I would say that DPDK is not very friendly to foreign applications and their build system in general.

Quoted 2 lines

Some other issues may happen if the configuration is different, for instance the size of structures may be different.

Impossible, because then libdpdk.a would not work. Remember we
are talking of using the exact same libdpdk.a in primary and secondary, and therefore any structure used in libdpdk.a has to match. And the structures used in the app has to match libdpdk.a as well.

Quoted 2 lines

There is already a lot of mess due to primary/secondary at many places in the code, I'm not sure adding more is really desirable.

Yes, one solution is obviously to get rid of secondary entirely. Personally, I believe it's pretty close to working, the number
of issues I found is manageable. I have a complex application (Snort) working that way without any issues. If DPDK wants to support secondary, you might as well make it work for everybody.
We could discuss better solutions to those issues. For
example, the tailq subsystem has a better solution. But, I'm not going to waste time if secondary is deprecated.

Quoted 2 lines

Regards,
Olivier

Regards,

Jean