acl: avoid vector spills in the RVV search loops

Add __rte_flatten, which inlines every call a function makes into it, where the callee's body is available.

__rte_always_inline marks the callee and inlines it at every call site. Here the requirement is local to one caller, and the callees are shared with paths that do not want inlining. Flatten applies at the boundary that needs it.

A spill slot has a fixed offset on a fixed-width vector ISA. With RVV its size depends on VLEN and is known only at run time, and once the frame holds such a slot the rest of it is addressed through vlenb. Inlining such calls is then worth the code growth.

Signed-off-by Sun Yuechi <[email protected]>

---
 lib/eal/include/rte_common.h | 13 +++++++++++++
 1 file changed, 13 insertions(+)

lib/eal/include/rte_common.h +13 -0

index fed373f412..e32b6b13d6 100644
@@ -547,6 +547,19 @@ static void
 __attribute__((destructor(RTE_PRIO(prio)), used)) func(void)
 #define __rte_noinline __attribute__((noinline))
 #endif
 
+/**
+ * Inline every call made by a function into it, where the callee's body
+ * is available, whatever the callee's own inline annotation.
+ *
+ * Useful where a call inside a hot loop is costly in itself,
+ * for example where values are held in registers that no call preserves.
+ */
+#ifdef RTE_TOOLCHAIN_MSVC
+#define __rte_flatten
+#else
+#define __rte_flatten __attribute__((flatten))
+#endif
+
 /**
  * Hint function in the hot path

*
--
2.56.0

At -O2 the ACL match checks are not inlined into the RVV search loops, so a vector value live across those calls is spilled and the remaining locals are addressed through vlenb. -O3 inlines them already and is unchanged.

Add __rte_flatten, and apply it to the two search functions.

Sun Yuechi (2):
eal: add flatten function attribute
acl: inline the match checks into the RVV search loops

 lib/acl/acl_run_rvv.h        |  8 +++++---
 lib/eal/include/rte_common.h | 13 +++++++++++++
 2 files changed, 18 insertions(+), 3 deletions(-)

--
2.56.0

The compiler is doing what we asked. Why workaround it.

At -O2 the ACL match checks are not inlined into the RVV search loops, so
a vector value live across those calls is spilled and the remaining locals
are addressed through vlenb.  -O3 inlines them already and is unchanged.

Add __rte_flatten, and apply it to the two search functions.

Sun Yuechi (2):
  eal: add flatten function attribute
  acl: inline the match checks into the RVV search loops

 lib/acl/acl_run_rvv.h        |  8 +++++--- lib/eal/include/rte_common.h | 13 +++++++++++++ 2 files changed, 18 insertions(+), 3 deletions(-)

--
2.56.0

The default DPDK build is unaffected.

Some distros package DPDK with buildtype=plain and -O2. Where ACL is used on RISC-V in such a build, the gap between -O2 and -O3 is very large. A gap is expected, but not one this large. The optimization level is the integrator's choice, and a library with hand-written vector paths should not lose this much because of it, so I would rather fix it upstream.