ring: make ring implementation non-inlined
Subsystems (4)
RFC 1/1 ring: make ring implementation non-inlined
As was discussed here:
http://mails.dpdk.org/archives/dev/2020-February/158586.html
this RFC aimed to hide ring internals into .c and make all ring functions non-inlined. In theory that might help to maintain ABI stability in future.
This is just a POC to measure the impact of proposed idea, proper implementation would definetly need some extra effort. On IA box (SKX) ring_perf_autotest shows ~20-30 cycles extra for enqueue+dequeue pair. On some more realistic code, I suspect the impact it might be a bit higher.
For MP/MC bulk transfers degradation seems quite small, though for SP/SC and/or small transfers it is more then noticable (see exact numbers below).
From my perspective we'd probably keep it inlined for now to avoid any non-anticipated perfomance degradations. Though intersted to see perf results and opinions from other interested parties.
Intel(R) Xeon(R) Platinum 8160 CPU @ 2.10GHzring_perf_autotest (without patch/with patch)
### Testing single element enq/deq ###
legacy APIs: SP/SC: single: 8.75/43.23
legacy APIs: MP/MC: single: 56.18/80.44
### Testing burst enq/deq ###
legacy APIs: SP/SC: burst (size: 8): 37.36/53.37 legacy APIs: SP/SC: burst (size: 32): 93.97/117.30 legacy APIs: MP/MC: burst (size: 8): 78.23/91.45 legacy APIs: MP/MC: burst (size: 32): 131.59/152.49
### Testing bulk enq/deq ###
legacy APIs: SP/SC: bulk (size: 8): 37.29/54.48 legacy APIs: SP/SC: bulk (size: 32): 92.68/113.01 legacy APIs: MP/MC: bulk (size: 8): 78.40/93.50 legacy APIs: MP/MC: bulk (size: 32): 131.49/154.25
### Testing empty bulk deq ###
legacy APIs: SP/SC: bulk (size: 8): 4.00/16.86
legacy APIs: MP/MC: bulk (size: 8): 7.01/15.55
### Testing using two hyperthreads ###
legacy APIs: SP/SC: bulk (size: 8): 10.64/17.56
legacy APIs: MP/MC: bulk (size: 8): 15.30/16.69
legacy APIs: SP/SC: bulk (size: 32): 5.84/7.09
legacy APIs: MP/MC: bulk (size: 32): 6.34/7.54
### Testing using two physical cores ###
legacy APIs: SP/SC: bulk (size: 8): 24.34/42.40
legacy APIs: MP/MC: bulk (size: 8): 70.34/71.82
legacy APIs: SP/SC: bulk (size: 32): 12.67/14.68
legacy APIs: MP/MC: bulk (size: 32): 22.41/17.93
### Testing single element enq/deq ###
elem APIs: element size 16B: SP/SC: single: 10.65/41.96 elem APIs: element size 16B: MP/MC: single: 44.33/81.36
### Testing burst enq/deq ###
elem APIs: element size 16B: SP/SC: burst (size: 8): 39.20/58.52 elem APIs: element size 16B: SP/SC: burst (size: 32): 123.19/142.79 elem APIs: element size 16B: MP/MC: burst (size: 8): 80.72/101.36 elem APIs: element size 16B: MP/MC: burst (size: 32): 169.21/185.38
### Testing bulk enq/deq ###
elem APIs: element size 16B: SP/SC: bulk (size: 8): 41.64/58.46 elem APIs: element size 16B: SP/SC: bulk (size: 32): 122.74/142.52 elem APIs: element size 16B: MP/MC: bulk (size: 8): 80.60/103.14 elem APIs: element size 16B: MP/MC: bulk (size: 32): 169.39/186.67
### Testing empty bulk deq ###
elem APIs: element size 16B: SP/SC: bulk (size: 8): 5.01/17.17 elem APIs: element size 16B: MP/MC: bulk (size: 8): 6.01/14.80
### Testing using two hyperthreads ###
elem APIs: element size 16B: SP/SC: bulk (size: 8): 12.02/17.18 elem APIs: element size 16B: MP/MC: bulk (size: 8): 16.81/21.14 elem APIs: element size 16B: SP/SC: bulk (size: 32): 7.87/9.01 elem APIs: element size 16B: MP/MC: bulk (size: 32): 8.22/10.57
### Testing using two physical cores ###
elem APIs: element size 16B: SP/SC: bulk (size: 8): 27.00/51.94 elem APIs: element size 16B: MP/MC: bulk (size: 8): 78.24/74.48 elem APIs: element size 16B: SP/SC: bulk (size: 32): 15.41/16.14 elem APIs: element size 16B: MP/MC: bulk (size: 32): 18.72/21.64
--- app/proc-info/main.c | 20 +- app/test-pmd/cmdline.c | 2 +- app/test/commands.c | 2 +- app/test/test_pdump.c | 6 +- app/test/test_ring_stress.h | 2 +- config/common_base | 2 +- drivers/crypto/ccp/ccp_pmd_ops.c | 2 +- drivers/crypto/meson.build | 1 - drivers/net/ring/rte_eth_ring.c | 17 +- lib/librte_eventdev/rte_event_ring.c | 6 +- lib/librte_eventdev/rte_event_ring.h | 17 +- lib/librte_pdump/rte_pdump.c | 3 +- lib/librte_port/rte_port_ring.c | 29 +- lib/librte_ring/Makefile | 6 +- lib/librte_ring/meson.build | 6 +- lib/librte_ring/ring_elem.c | 919 +++++++++++++++++++++++++++ lib/librte_ring/ring_impl.c | 750 ++++++++++++++++++++++ lib/librte_ring/ring_impl.h | 64 ++ lib/librte_ring/rte_ring.c | 3 +- lib/librte_ring/rte_ring.h | 412 ++---------- lib/librte_ring/rte_ring_elem.h | 519 ++------------- lib/librte_ring/rte_ring_version.map | 46 ++ 22 files changed, 1938 insertions(+), 896 deletions(-) create mode 100644 lib/librte_ring/ring_elem.c create mode 100644 lib/librte_ring/ring_impl.c create mode 100644 lib/librte_ring/ring_impl.h
app/proc-info/main.c +1 -19
@@ -1137,25 +1137,7 @@ show_ring(char *name)if (name !=NULL ) {struct rte_ring *ptr =rte_ring_lookup (name);if (ptr !=NULL ) { -printf (" - Name (%s) on socket (%d)\n" -" - flags:\n" -"\t -- Single Producer Enqueue (%u)\n" -"\t -- Single Consmer Dequeue (%u)\n" , - ptr->name, - ptr->memzone->socket_id, - ptr->flags &RING_F_SP_ENQ , - ptr->flags &RING_F_SC_DEQ ); -printf (" - size (%u) mask (0x%x) capacity (%u)\n" , - ptr->size, - ptr->mask, - ptr->capacity); -printf (" - count (%u) free count (%u)\n" , -rte_ring_count (ptr), -rte_ring_free_count (ptr)); -printf (" - full (%d) empty (%d)\n" , -rte_ring_full (ptr), -rte_ring_empty (ptr)); - +rte_ring_dump (stdout, ptr);STATS_BDR_STR (50 ,"" );return ; }
app/test-pmd/cmdline.c +1 -1
@@ -9562,7 +9562,7 @@ dump_struct_sizes(void)#define DUMP_SIZE (t) printf("sizeof(" #t ") = %u\n", (unsigned)sizeof(t));DUMP_SIZE (struct rte_mbuf );DUMP_SIZE (struct rte_mempool ); -DUMP_SIZE (struct rte_ring ); +//DUMP_SIZE(struct rte_ring); #undef DUMP_SIZE }
app/test/commands.c +1 -1
@@ -107,7 +107,7 @@ dump_struct_sizes(void)#define DUMP_SIZE (t) printf("sizeof(" #t ") = %u\n", (unsigned)sizeof(t));DUMP_SIZE (struct rte_mbuf );DUMP_SIZE (struct rte_mempool ); -DUMP_SIZE (struct rte_ring ); +//DUMP_SIZE(struct rte_ring); #undef DUMP_SIZE }
app/test/test_pdump.c +1 -5
@@ -57,8 +57,7 @@ run_pdump_client_tests(void)if (ret <0 )return -1 ; mp->flags =0x0000 ; - ring_client =rte_ring_create ("SR0" ,RING_SIZE ,rte_socket_id (), -RING_F_SP_ENQ |RING_F_SC_DEQ ); + ring_client =rte_ring_create ("SR0" ,RING_SIZE ,rte_socket_id (),0 );if (ring_client ==NULL ) {printf ("rte_ring_create SR0 failed" );return -1 ;
@@ -71,9 +70,6 @@ run_pdump_client_tests(void) }rte_eth_dev_probing_finish (eth_dev); - ring_client->prod.single =0 ; - ring_client->cons.single =0 ; -printf ("\n***** flags = RTE_PDUMP_FLAG_TX *****\n" );for (itr =0 ; itr <NUM_ITR ; itr++) {
app/test/test_ring_stress.h +1 -1
@@ -350,7 +350,7 @@ mt1_init(struct rte_ring **rng, void **data, uint32_t num)/* alloc ring */ nr =2 * num; sz =rte_ring_get_memsize (nr); - r = rte_zmalloc(NULL, sz, alignof(*r)); + r =rte_zmalloc (NULL , sz,RTE_CACHE_LINE_SIZE );if (r ==NULL ) {printf ("%s: alloca(%zu) for FIFO with %u elems failed" , __func__, sz, nr);
config/common_base +1 -1
@@ -670,7 +670,7 @@ CONFIG_RTE_LIBRTE_PMD_ZUC=n # Compile PMD for Crypto Scheduler device # -CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER=y +CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER=n # # Compile PMD for NULL Crypto device
drivers/crypto/ccp/ccp_pmd_ops.c +1 -1
@@ -666,7 +666,7 @@ ccp_pmd_qp_create_batch_info_ring(struct ccp_qp *qp, r =rte_ring_lookup (qp->name);if (r) { -if (r->size >= ring_size) { +if (rte_ring_get_size (r) >= ring_size) {CCP_LOG_INFO ("Reusing ring %s for processed packets" , qp->name);
drivers/crypto/meson.build +0 -1
@@ -16,7 +16,6 @@ drivers = ['aesni_gcm', 'octeontx2', 'openssl', 'qat', - 'scheduler', 'snow3g', 'virtio', 'zuc']
drivers/net/ring/rte_eth_ring.c +13 -4
@@ -41,6 +41,8 @@ struct ring_queue {struct rte_ring *rng;rte_atomic64_t rx_pkts;rte_atomic64_t tx_pkts; +uint32_t rx_sync_type; +uint32_t tx_sync_type; };struct pmd_internals {
@@ -74,7 +76,7 @@ eth_ring_rx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)struct ring_queue *r = q;const uint16_t nb_rx = (uint16_t )rte_ring_dequeue_burst (r->rng, ptrs, nb_bufs,NULL ); -if (r->rng->flags &RING_F_SC_DEQ ) +if (r->rx_sync_type ==RTE_RING_SYNC_ST ) r->rx_pkts.cnt += nb_rx;else rte_atomic64_add (&(r->rx_pkts), nb_rx);
@@ -88,7 +90,7 @@ eth_ring_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)struct ring_queue *r = q;const uint16_t nb_tx = (uint16_t )rte_ring_enqueue_burst (r->rng, ptrs, nb_bufs,NULL ); -if (r->rng->flags &RING_F_SP_ENQ ) +if (r->tx_sync_type ==RTE_RING_SYNC_ST ) r->tx_pkts.cnt += nb_tx;else rte_atomic64_add (&(r->tx_pkts), nb_tx);
@@ -306,10 +308,14 @@ do_eth_dev_ring_create(const char *name, internals->max_tx_queues = nb_tx_queues;for (i =0 ; i < nb_rx_queues; i++) { internals->rx_ring_queues[i].rng = rx_queues[i]; + internals->rx_ring_queues[i].rx_sync_type = +rte_ring_get_cons_sync_type (rx_queues[i]); data->rx_queues[i] = &internals->rx_ring_queues[i]; }for (i =0 ; i < nb_tx_queues; i++) { internals->tx_ring_queues[i].rng = tx_queues[i]; + internals->tx_ring_queues[i].tx_sync_type = +rte_ring_get_prod_sync_type (rx_queues[i]); data->tx_queues[i] = &internals->tx_ring_queues[i]; }
@@ -403,8 +409,11 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],int rte_eth_from_ring (struct rte_ring *r) { -return rte_eth_from_rings (r->name, &r,1 , &r,1 , - r->memzone ? r->memzone->socket_id :SOCKET_ID_ANY ); +const struct rte_memzone *mz; + + mz =rte_ring_get_memzone (r); +return rte_eth_from_rings (rte_ring_get_name (r), &r,1 , &r,1 , + mz ? mz->socket_id :SOCKET_ID_ANY ); } static int
lib/librte_eventdev/rte_event_ring.c +1 -5
@@ -16,12 +16,8 @@ int rte_event_ring_init(struct rte_event_ring *r, const char *name, unsigned int count, unsigned int flags) { -/* compilation-time checks */ - RTE_BUILD_BUG_ON((sizeof(struct rte_event_ring) & - RTE_CACHE_LINE_MASK) != 0); - /* init the ring structure */ -return rte_ring_init (&r->r, name, count, flags); + return rte_ring_init(r, name, count, flags); }/* create the ring */
lib/librte_eventdev/rte_event_ring.h +8 -9
@@ -32,9 +32,8 @@* used inside software eventdev implementations and by applications * directly as needed. */ -struct rte_event_ring { -struct rte_ring r; -}; + +#define rte_event_ring rte_ring/** * Returns the number of events in the ring
@@ -47,7 +46,7 @@ struct rte_event_ring { static __rte_always_inline unsigned intrte_event_ring_count (const struct rte_event_ring *r) { -return rte_ring_count (&r->r); +return rte_ring_count (r); }/**
@@ -62,7 +61,7 @@ rte_event_ring_count(const struct rte_event_ring *r) static __rte_always_inline unsigned intrte_event_ring_free_count (const struct rte_event_ring *r) { -return rte_ring_free_count (&r->r); +return rte_ring_free_count (r); }/**
@@ -93,7 +92,7 @@ rte_event_ring_enqueue_burst(struct rte_event_ring *r,unsigned int num;uint32_t space; - num =rte_ring_enqueue_burst_elem (&r->r, events, + num =rte_ring_enqueue_burst_elem (r, events,sizeof (struct rte_event ), n, &space);
@@ -129,7 +128,7 @@ rte_event_ring_dequeue_burst(struct rte_event_ring *r,unsigned int num;uint32_t remaining; - num =rte_ring_dequeue_burst_elem (&r->r, events, + num =rte_ring_dequeue_burst_elem (r, events,sizeof (struct rte_event ), n, &remaining);
@@ -250,7 +249,7 @@ rte_event_ring_free(struct rte_event_ring *r);static inline unsigned int rte_event_ring_get_size (const struct rte_event_ring *r) { -return rte_ring_get_size (&r->r); +return rte_ring_get_size (r); }/**
@@ -264,6 +263,6 @@ rte_event_ring_get_size(const struct rte_event_ring *r)static inline unsigned int rte_event_ring_get_capacity (const struct rte_event_ring *r) { -return rte_ring_get_capacity (&r->r); +return rte_ring_get_capacity (r); }#endif
lib/librte_pdump/rte_pdump.c +2 -1
@@ -380,7 +380,8 @@ pdump_validate_ring_mp(struct rte_ring *ring, struct rte_mempool *mp) rte_errno =EINVAL ;return -1 ; } -if (ring->prod.single || ring->cons.single) { +if (rte_ring_get_prod_sync_type (ring) ==RTE_RING_SYNC_ST || +rte_ring_get_cons_sync_type (ring) ==RTE_RING_SYNC_ST ) {PDUMP_LOG (ERR ,"ring with either SP or SC settings" " is not valid for pdump, should have MP and MC settings\n" ); rte_errno =EINVAL ;
lib/librte_port/rte_port_ring.c +19 -10
@@ -40,12 +40,15 @@ rte_port_ring_reader_create_internal(void *params, int socket_id,struct rte_port_ring_reader_params *conf = params;struct rte_port_ring_reader *port; +uint32_t ring_sync; + + ring_sync =rte_ring_get_cons_sync_type (conf->ring);/* Check input parameters */ if ((conf ==NULL ) || (conf->ring ==NULL ) || - (conf->ring->cons.single && is_multi) || - (!(conf->ring->cons.single) && !is_multi)) { + (ring_sync ==RTE_RING_SYNC_ST && is_multi) || + (ring_sync !=RTE_RING_SYNC_ST && !is_multi)) {RTE_LOG (ERR ,PORT ,"%s: Invalid Parameters\n" , __func__);return NULL ; }
@@ -167,13 +170,16 @@ rte_port_ring_writer_create_internal(void *params, int socket_id,struct rte_port_ring_writer_params *conf = params;struct rte_port_ring_writer *port; +uint32_t ring_sync; + + ring_sync =rte_ring_get_prod_sync_type (conf->ring);/* Check input parameters */ if ((conf ==NULL ) || - (conf->ring ==NULL ) || - (conf->ring->prod.single && is_multi) || - (!(conf->ring->prod.single) && !is_multi) || - (conf->tx_burst_sz >RTE_PORT_IN_BURST_SIZE_MAX )) { + (conf->ring ==NULL ) || + (ring_sync ==RTE_RING_SYNC_ST && is_multi) || + (ring_sync !=RTE_RING_SYNC_ST && !is_multi) || + (conf->tx_burst_sz >RTE_PORT_IN_BURST_SIZE_MAX )) {RTE_LOG (ERR ,PORT ,"%s: Invalid Parameters\n" , __func__);return NULL ; }
@@ -436,13 +442,16 @@ rte_port_ring_writer_nodrop_create_internal(void *params, int socket_id,struct rte_port_ring_writer_nodrop_params *conf = params;struct rte_port_ring_writer_nodrop *port; +uint32_t ring_sync; + + ring_sync =rte_ring_get_prod_sync_type (conf->ring);/* Check input parameters */ if ((conf ==NULL ) || - (conf->ring ==NULL ) || - (conf->ring->prod.single && is_multi) || - (!(conf->ring->prod.single) && !is_multi) || - (conf->tx_burst_sz >RTE_PORT_IN_BURST_SIZE_MAX )) { + (conf->ring ==NULL ) || + (ring_sync ==RTE_RING_SYNC_ST && is_multi) || + (ring_sync !=RTE_RING_SYNC_ST && !is_multi) || + (conf->tx_burst_sz >RTE_PORT_IN_BURST_SIZE_MAX )) {RTE_LOG (ERR ,PORT ,"%s: Invalid Parameters\n" , __func__);return NULL ; }
lib/librte_ring/Makefile +3 -3
@@ -13,11 +13,11 @@ EXPORT_MAP := rte_ring_version.map # all source are stored in SRCS-y SRCS-$(CONFIG_RTE_LIBRTE_RING) := rte_ring.c +SRCS-$(CONFIG_RTE_LIBRTE_RING) += ring_impl.c +SRCS-$(CONFIG_RTE_LIBRTE_RING) += ring_elem.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_RING)-include := rte_ring.h \ - rte_ring_elem.h \ - rte_ring_generic.h \ - rte_ring_c11_mem.h + rte_ring_elem.h include $(RTE_SDK)/mk/rte.lib.mk
lib/librte_ring/meson.build +2 -4
@@ -1,11 +1,9 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -sources = files('rte_ring.c') +sources = files('rte_ring.c', 'ring_impl.c', 'ring_elem.c') headers = files('rte_ring.h', - 'rte_ring_elem.h', - 'rte_ring_c11_mem.h', - 'rte_ring_generic.h') + 'rte_ring_elem.h') # rte_ring_create_elem and rte_ring_get_memsize_elem are experimental allow_experimental_apis = true
lib/librte_ring/ring_elem.c new file +919 -0
919 lines not on this page · raw message
lib/librte_ring/ring_impl.c new file +750 -0
750 lines not on this page · raw message
lib/librte_ring/ring_impl.h new file +64 -0
64 lines not on this page · raw message
lib/librte_ring/rte_ring.c +1 -2
@@ -32,8 +32,7 @@#include <rte_spinlock.h> #include <rte_tailq.h> -#include "rte_ring.h" -#include "rte_ring_elem.h" +#include "ring_impl.h" TAILQ_HEAD (rte_ring_list, rte_tailq_entry);
lib/librte_ring/rte_ring.h +67 -345
591 lines not on this page · raw message
lib/librte_ring/rte_ring_elem.h +36 -483
361 lines not on this page · raw message
lib/librte_ring/rte_ring_version.map +46 -0
46 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.17.1
Part of the diff is not shown here. View the raw message.
Quoted 79 lines · On Fri, 20 Mar 2020 16:41:38 +0000 Konstantin Ananyev…
As was discussed here:
http://mails.dpdk.org/archives/dev/2020-February/158586.html
this RFC aimed to hide ring internals into .c and make all ring functions non-inlined. In theory that might help to maintain ABI stability in future.
This is just a POC to measure the impact of proposed idea, proper implementation would definetly need some extra effort. On IA box (SKX)ring_perf_autotestshows ~20-30 cycles extra for enqueue+dequeue pair. On some more realistic code, I suspect the impact it might be a bit higher.
For MP/MC bulk transfers degradation seems quite small, though for SP/SC and/or small transfers it is more then noticable (see exact numbers below).
From my perspective we'd probably keep it inlined for now to avoid any non-anticipated perfomance degradations. Though intersted to see perf results and opinions from other interested parties.Intel(R) Xeon(R) Platinum 8160 CPU @ 2.10GHz
ring_perf_autotest(without patch/with patch)### Testing single element enq/deq ###
legacy APIs: SP/SC: single: 8.75/43.23
legacy APIs: MP/MC: single: 56.18/80.44### Testing burst enq/deq ###
legacy APIs: SP/SC: burst (size: 8): 37.36/53.37 legacy APIs: SP/SC: burst (size: 32): 93.97/117.30 legacy APIs: MP/MC: burst (size: 8): 78.23/91.45 legacy APIs: MP/MC: burst (size: 32): 131.59/152.49### Testing bulk enq/deq ###
legacy APIs: SP/SC: bulk (size: 8): 37.29/54.48 legacy APIs: SP/SC: bulk (size: 32): 92.68/113.01 legacy APIs: MP/MC: bulk (size: 8): 78.40/93.50 legacy APIs: MP/MC: bulk (size: 32): 131.49/154.25### Testing empty bulk deq ###
legacy APIs: SP/SC: bulk (size: 8): 4.00/16.86
legacy APIs: MP/MC: bulk (size: 8): 7.01/15.55### Testing using two hyperthreads ###
legacy APIs: SP/SC: bulk (size: 8): 10.64/17.56
legacy APIs: MP/MC: bulk (size: 8): 15.30/16.69
legacy APIs: SP/SC: bulk (size: 32): 5.84/7.09
legacy APIs: MP/MC: bulk (size: 32): 6.34/7.54### Testing using two physical cores ###
legacy APIs: SP/SC: bulk (size: 8): 24.34/42.40
legacy APIs: MP/MC: bulk (size: 8): 70.34/71.82
legacy APIs: SP/SC: bulk (size: 32): 12.67/14.68
legacy APIs: MP/MC: bulk (size: 32): 22.41/17.93### Testing single element enq/deq ###
elem APIs: element size 16B: SP/SC: single: 10.65/41.96 elem APIs: element size 16B: MP/MC: single: 44.33/81.36### Testing burst enq/deq ###
elem APIs: element size 16B: SP/SC: burst (size: 8): 39.20/58.52 elem APIs: element size 16B: SP/SC: burst (size: 32): 123.19/142.79 elem APIs: element size 16B: MP/MC: burst (size: 8): 80.72/101.36 elem APIs: element size 16B: MP/MC: burst (size: 32): 169.21/185.38### Testing bulk enq/deq ###
elem APIs: element size 16B: SP/SC: bulk (size: 8): 41.64/58.46 elem APIs: element size 16B: SP/SC: bulk (size: 32): 122.74/142.52 elem APIs: element size 16B: MP/MC: bulk (size: 8): 80.60/103.14 elem APIs: element size 16B: MP/MC: bulk (size: 32): 169.39/186.67### Testing empty bulk deq ###
elem APIs: element size 16B: SP/SC: bulk (size: 8): 5.01/17.17 elem APIs: element size 16B: MP/MC: bulk (size: 8): 6.01/14.80### Testing using two hyperthreads ###
elem APIs: element size 16B: SP/SC: bulk (size: 8): 12.02/17.18 elem APIs: element size 16B: MP/MC: bulk (size: 8): 16.81/21.14 elem APIs: element size 16B: SP/SC: bulk (size: 32): 7.87/9.01 elem APIs: element size 16B: MP/MC: bulk (size: 32): 8.22/10.57
Quoted 7 lines
### Testing using two physical cores ###
elem APIs: element size 16B: SP/SC: bulk (size: 8): 27.00/51.94 elem APIs: element size 16B: MP/MC: bulk (size: 8): 78.24/74.48 elem APIs: element size 16B: SP/SC: bulk (size: 32): 15.41/16.14 elem APIs: element size 16B: MP/MC: bulk (size: 32): 18.72/21.64Signed-off-by Konstantin Ananyev <[email protected]>
What is impact with LTO? I suspect compiler might have a chance to get speed back with LTO.
Quoted 97 lines
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Friday, March 20, 2020 5:55 PM
To: Ananyev, Konstantin <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC] ring: make ring implementation non-inlinedQuoted 85 lines · On Fri, 20 Mar 2020 16:41:38 +0000 Konstantin Ananyev…
As was discussed here:
http://mails.dpdk.org/archives/dev/2020-February/158586.html
this RFC aimed to hide ring internals into .c and make all ring functions non-inlined. In theory that might help to maintain ABI stability in future.
This is just a POC to measure the impact of proposed idea, proper implementation would definetly need some extra effort. On IA box (SKX)ring_perf_autotestshows ~20-30 cycles extra for enqueue+dequeue pair. On some more realistic code, I suspect the impact it might be a bit higher.
For MP/MC bulk transfers degradation seems quite small, though for SP/SC and/or small transfers it is more then noticable (see exact numbers below).
From my perspective we'd probably keep it inlined for now to avoid any non-anticipated perfomance degradations. Though intersted to see perf results and opinions from other interested parties.Intel(R) Xeon(R) Platinum 8160 CPU @ 2.10GHz
ring_perf_autotest(without patch/with patch)### Testing single element enq/deq ###
legacy APIs: SP/SC: single: 8.75/43.23
legacy APIs: MP/MC: single: 56.18/80.44### Testing burst enq/deq ###
legacy APIs: SP/SC: burst (size: 8): 37.36/53.37 legacy APIs: SP/SC: burst (size: 32): 93.97/117.30 legacy APIs: MP/MC: burst (size: 8): 78.23/91.45 legacy APIs: MP/MC: burst (size: 32): 131.59/152.49### Testing bulk enq/deq ###
legacy APIs: SP/SC: bulk (size: 8): 37.29/54.48 legacy APIs: SP/SC: bulk (size: 32): 92.68/113.01 legacy APIs: MP/MC: bulk (size: 8): 78.40/93.50 legacy APIs: MP/MC: bulk (size: 32): 131.49/154.25### Testing empty bulk deq ###
legacy APIs: SP/SC: bulk (size: 8): 4.00/16.86
legacy APIs: MP/MC: bulk (size: 8): 7.01/15.55### Testing using two hyperthreads ###
legacy APIs: SP/SC: bulk (size: 8): 10.64/17.56
legacy APIs: MP/MC: bulk (size: 8): 15.30/16.69
legacy APIs: SP/SC: bulk (size: 32): 5.84/7.09
legacy APIs: MP/MC: bulk (size: 32): 6.34/7.54### Testing using two physical cores ###
legacy APIs: SP/SC: bulk (size: 8): 24.34/42.40
legacy APIs: MP/MC: bulk (size: 8): 70.34/71.82
legacy APIs: SP/SC: bulk (size: 32): 12.67/14.68
legacy APIs: MP/MC: bulk (size: 32): 22.41/17.93### Testing single element enq/deq ###
elem APIs: element size 16B: SP/SC: single: 10.65/41.96 elem APIs: element size 16B: MP/MC: single: 44.33/81.36### Testing burst enq/deq ###
elem APIs: element size 16B: SP/SC: burst (size: 8): 39.20/58.52 elem APIs: element size 16B: SP/SC: burst (size: 32): 123.19/142.79 elem APIs: element size 16B: MP/MC: burst (size: 8): 80.72/101.36 elem APIs: element size 16B: MP/MC: burst (size: 32): 169.21/185.38### Testing bulk enq/deq ###
elem APIs: element size 16B: SP/SC: bulk (size: 8): 41.64/58.46 elem APIs: element size 16B: SP/SC: bulk (size: 32): 122.74/142.52 elem APIs: element size 16B: MP/MC: bulk (size: 8): 80.60/103.14 elem APIs: element size 16B: MP/MC: bulk (size: 32): 169.39/186.67### Testing empty bulk deq ###
elem APIs: element size 16B: SP/SC: bulk (size: 8): 5.01/17.17 elem APIs: element size 16B: MP/MC: bulk (size: 8): 6.01/14.80### Testing using two hyperthreads ###
elem APIs: element size 16B: SP/SC: bulk (size: 8): 12.02/17.18 elem APIs: element size 16B: MP/MC: bulk (size: 8): 16.81/21.14 elem APIs: element size 16B: SP/SC: bulk (size: 32): 7.87/9.01 elem APIs: element size 16B: MP/MC: bulk (size: 32): 8.22/10.57### Testing using two physical cores ###
elem APIs: element size 16B: SP/SC: bulk (size: 8): 27.00/51.94 elem APIs: element size 16B: MP/MC: bulk (size: 8): 78.24/74.48 elem APIs: element size 16B: SP/SC: bulk (size: 32): 15.41/16.14 elem APIs: element size 16B: MP/MC: bulk (size: 32): 18.72/21.64
Quoted 4 lines
Quoted 1 line
Signed-off-by Konstantin Ananyev <[email protected]>What is impact with LTO? I suspect compiler might have a chance to get speed back with LTO.
Might be, but LTO is not enabled by default. So don't see much point in digging any further here.
Quoted 10 lines · Konstantin Ananyev
As was discussed here:
http://mails.dpdk.org/archives/dev/2020-February/158586.html
this RFC aimed to hide ring internals into .c and make all ring functions non-inlined. In theory that might help to maintain ABI stability in future.
This is just a POC to measure the impact of proposed idea, proper implementation would definetly need some extra effort. On IA box (SKX)ring_perf_autotestshows ~20-30 cycles extra for enqueue+dequeue pair. On some more realistic code, I suspect
Quoted 8 lines
the impact it might be a bit higher.
For MP/MC bulk transfers degradation seems quite small, though for SP/SC and/or small transfers it is more then noticable (see exact numbers below).
From my perspective we'd probably keep it inlined for now to avoid any non-anticipated perfomance degradations. Though intersted to see perf results and opinions from other interested parties.
+1
My reasoning is a bit different, DPDK is using in embedded boxes too where performance has
more weight than ABI stuff. I think we need to focus first on slow path APIs ABI stuff.
I spend a few cycles to apply this patch +
http://mails.dpdk.org/archives/dev/2020-February/158586.html
on top of the tree, there are a lot of conflicts. If I get a mergeable patch then I will test it on an arm64 box.
Quoted 3 lines
Intel(R) Xeon(R) Platinum 8160 CPU @ 2.10GHz
ring_perf_autotest(without patch/with patch)
Hi Jerin,
Quoted 29 lines
Quoted 10 lines · Konstantin Ananyev
As was discussed here:
http://mails.dpdk.org/archives/dev/2020-February/158586.html
this RFC aimed to hide ring internals into .c and make all ring functions non-inlined. In theory that might help to maintain ABI stability in future.
This is just a POC to measure the impact of proposed idea, proper implementation would definetly need some extra effort. On IA box (SKX)ring_perf_autotestshows ~20-30 cycles extra for enqueue+dequeue pair. On some more realistic code, I suspectQuoted 8 lines
the impact it might be a bit higher.
For MP/MC bulk transfers degradation seems quite small, though for SP/SC and/or small transfers it is more then noticable (see exact numbers below).
From my perspective we'd probably keep it inlined for now to avoid any non-anticipated perfomance degradations. Though intersted to see perf results and opinions from other interested parties.+1
My reasoning is a bit different, DPDK is using in embedded boxes too where performance has
more weight than ABI stuff. I think we need to focus first on slow path APIs ABI stuff.
Quoted 4 lines
I spend a few cycles to apply this patch +
http://mails.dpdk.org/archives/dev/2020-February/158586.html
on top of the tree, there are a lot of conflicts. If I get a mergeable patch then I will test it on an arm64 box.
You don’t need to apply previous patch series. They are completely unrelated.
Just apply that one (http://patches.dpdk.org/patch/66982/)
on top of dpdk.org master.
It should be applied cleanly (at least it does for me). Konstantin
Quoted 8 lines
Quoted 8 lines
Intel(R) Xeon(R) Platinum 8160 CPU @ 2.10GHz
ring_perf_autotest(without patch/with patch)
Quoted 24 lines
From: dev [mailto:[email protected]] On Behalf Of Jerin Jacob
Sent: Wednesday, March 25, 2020 10:09 PMQuoted 10 lines · Konstantin Ananyev
As was discussed here:
http://mails.dpdk.org/archives/dev/2020-February/158586.html
this RFC aimed to hide ring internals into .c and make all ring functions non-inlined. In theory that might help to maintain ABI stability in future.
This is just a POC to measure the impact of proposed idea, proper implementation would definetly need some extra effort. On IA box (SKX)ring_perf_autotestshows ~20-30 cycles extra for enqueue+dequeue pair. On some more realistic code, I suspectQuoted 8 lines
the impact it might be a bit higher.
For MP/MC bulk transfers degradation seems quite small, though for SP/SC and/or small transfers it is more then noticable (see exact numbers below).
From my perspective we'd probably keep it inlined for now to avoid any non-anticipated perfomance degradations. Though intersted to see perf results and opinions from other interested parties.
Quoted 5 lines
+1
My reasoning is a bit different, DPDK is using in embedded boxes too where performance has
more weight than ABI stuff.
As a network appliance vendor I can confirm that we certainly care more about performance than ABI stability. ABI stability is irrelevant for us; and API instability is a non-recurring engineering cost each time we choose to switch to a new DPDK version, which we only do if we cannot avoid it, e.g. due to new drivers, security fixes or new features that we want to use.
For us, the trend pointed in the wrong direction when DPDK switched the preference towards runtime configurability and deprecated compile time configurability. I do understand the reasoning behind it, and the impact is minimal, so we accept it.
However, if DPDK starts sacrificing performance of the core libraries for the benefits of the GNU/Linux distributors, network appliance vendors may put more effort into sticking with old DPDK versions instead of updating.
Quoted 15 lines
I think we need to focus first on slow
path APIs ABI stuff.I spend a few cycles to apply this patch +
http://mails.dpdk.org/archives/dev/2020-February/158586.html
on top of the tree, there are a lot of conflicts. If I get a mergeable patch then I will test it on an arm64 box.Quoted 3 lines
Intel(R) Xeon(R) Platinum 8160 CPU @ 2.10GHz
ring_perf_autotest(without patch/with patch)
26/03/2020 09:04, Morten Brørup:
Quoted 22 lines
From: Jerin Jacob
Quoted 21 lines · On Fri, Mar 20, 2020 Konstantin Ananyev wrote:
Quoted 10 lines
As was discussed here:
http://mails.dpdk.org/archives/dev/2020-February/158586.html
this RFC aimed to hide ring internals into .c and make all ring functions non-inlined. In theory that might help to maintain ABI stability in future.
This is just a POC to measure the impact of proposed idea, proper implementation would definetly need some extra effort. On IA box (SKX)ring_perf_autotestshows ~20-30 cycles extra for enqueue+dequeue pair. On some more realistic code, I suspectQuoted 8 lines
the impact it might be a bit higher.
For MP/MC bulk transfers degradation seems quite small, though for SP/SC and/or small transfers it is more then noticable (see exact numbers below).
From my perspective we'd probably keep it inlined for now to avoid any non-anticipated perfomance degradations. Though intersted to see perf results and opinions from other interested parties.+1
Konstantin, thank you for doing some measures
Quoted 11 lines
Quoted 2 lines
My reasoning is a bit different, DPDK is using in embedded boxes too where performance has more weight than ABI stuff.
As a network appliance vendor I can confirm that we certainly care more about performance than ABI stability.
ABI stability is irrelevant for us;
and API instability is a non-recurring engineering cost each time we choose to switch to a new DPDK version, which we only do if we cannot avoid it, e.g. due to new drivers, security fixes or new features that we want to use.
Quoted 4 lines
For us, the trend pointed in the wrong direction when DPDK switched the preference towards runtime configurability and deprecated compile time configurability. I do understand the reasoning behind it, and the impact is minimal, so we accept it.
The code can be optimized by removing some instructions with #ifdef. But the complexity of managing #ifdef enabling/disabling, depending on the platform and the use case, would be huge. We try to have a reasonable code "always enabled" which performs well in all cases. This is a design choice which makes DPDK a library, not a pool of code to cherry-pick.
Quoted 4 lines
However, if DPDK starts sacrificing performance of the core libraries for the benefits of the GNU/Linux distributors, network appliance vendors may put more effort into sticking with old DPDK versions instead of updating.
The initial choice regarding ABI compatibility was "do not care". Recently, the decision was done to care about ABI compatibility as priority number 2. The priority number 1 remains the performance. That's a reason for allowing some ABI breakages in some specific releases announced in advance.
Quoted 2 lines
Quoted 2 lines
I think we need to focus first on slow
path APIs ABI stuff.
Yes we should not degrade fast path performance for the sake of avoiding uncertain future ABI issues.
Morten, Jerin, thank you for the feedback.
<snip>
Quoted 63 lines
Subject: Re: [dpdk-dev] [RFC] ring: make ring implementation non-inlined
26/03/2020 09:04, Morten Brørup:
Quoted 22 lines
From: Jerin Jacob
Quoted 21 lines · On Fri, Mar 20, 2020 Konstantin Ananyev wrote:
Quoted 10 lines
As was discussed here:
http://mails.dpdk.org/archives/dev/2020-February/158586.html
this RFC aimed to hide ring internals into .c and make all ring functions non-inlined. In theory that might help to maintain ABI stability in future.
This is just a POC to measure the impact of proposed idea, proper implementation would definetly need some extra effort. On IA box (SKX)ring_perf_autotestshows ~20-30 cycles extra for enqueue+dequeue pair. On some more realistic code, I suspectQuoted 8 lines
the impact it might be a bit higher.
For MP/MC bulk transfers degradation seems quite small, though for SP/SC and/or small transfers it is more then noticable (see exact numbers below).
From my perspective we'd probably keep it inlined for now to avoid any non-anticipated perfomance degradations.
Though intersted to see perf results and opinions from other interested parties.+1
Konstantin, thank you for doing some measures
Quoted 11 lines
Quoted 2 lines
My reasoning is a bit different, DPDK is using in embedded boxes too where performance has more weight than ABI stuff.
As a network appliance vendor I can confirm that we certainly care more about performance than ABI stability.
ABI stability is irrelevant for us;
and API instability is a non-recurring engineering cost each time we choose to switch to a new DPDK version, which we only do if we cannot avoid it, e.g. due to new drivers, security fixes or new features that we want to use.Quoted 4 lines
For us, the trend pointed in the wrong direction when DPDK switched the preference towards runtime configurability and deprecated compile time configurability. I do understand the reasoning behind it, and the impact is minimal, so we accept it.
The code can be optimized by removing some instructions with #ifdef. But the complexity of managing #ifdef enabling/disabling, depending on the platform and the use case, would be huge.
We try to have a reasonable code "always enabled" which performs well in all cases. This is a design choice which makes DPDK a library, not a pool of code to cherry-pick.Quoted 4 lines
However, if DPDK starts sacrificing performance of the core libraries for the benefits of the GNU/Linux distributors, network appliance vendors may put more effort into sticking with old DPDK versions instead of updating.
The initial choice regarding ABI compatibility was "do not care". Recently, the decision was done to care about ABI compatibility as priority number 2. The priority number 1 remains the performance.
That's a reason for allowing some ABI breakages in some specific releases announced in advance.
Quoted 6 lines
Quoted 1 line
Quoted 1 line
I think we need to focus first on slow path APIs ABI stuff.
Yes we should not degrade fast path performance for the sake of avoiding uncertain future ABI issues.
Morten, Jerin, thank you for the feedback.
I think we have a consensus here not to make any changes to inline functions for now. Should we mark this as 'Deferred or Rejected'?
Quoted 1 line
Quoted 90 lines
From: dev [mailto:[email protected]] On Behalf Of Honnappa Nagarahalli
Sent: Wednesday, July 1, 2020 1:16 AM<snip>
Quoted 1 line
Subject: Re: [dpdk-dev] [RFC] ring: make ring implementation non-
inlined
Quoted 9 lines
26/03/2020 09:04, Morten Brørup:
Quoted 7 lines
From: Jerin Jacob
Quoted 6 lines · On Fri, Mar 20, 2020 Konstantin Ananyev wrote:
Quoted 5 lines
As was discussed here:
http://mails.dpdk.org/archives/dev/2020-February/158586.html
this RFC aimed to hide ring internals into .c and make all ring functions non-inlined. In theory that might help to maintainABI
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
stability in future.
This is just a POC to measure the impact of proposed idea,proper
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
implementation would definetly need some extra effort. On IA box (SKX)
ring_perf_autotestshows ~20-30 cycles extrafor
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
enqueue+dequeue pair. On some more realistic code, I suspect the impact it might be a bit higher.
For MP/MC bulk transfers degradation seems quite small, thoughfor
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
SP/SC and/or small transfers it is more then noticable (see
exact
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
numbers below).
From my perspective we'd probably keep it inlined for now toavoid
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
any non-anticipated perfomance degradations. Though intersted to see perf results and opinions from other interested parties.
+1
Konstantin, thank you for doing some measures
Quoted 1 line
Quoted 1 line
My reasoning is a bit different, DPDK is using in embedded boxes
too
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
where performance has more weight than ABI stuff.
As a network appliance vendor I can confirm that we certainly care more about performance than ABI stability.
ABI stability is irrelevant for us;
and API instability is a non-recurring engineering cost each timewe
Quoted 1 line
Quoted 1 line
choose to switch to a new DPDK version, which we only do if we
cannot
Quoted 1 line
Quoted 1 line
avoid it, e.g. due to new drivers, security fixes or new features
that
Quoted 4 lines
Quoted 4 lines
we want to use.
For us, the trend pointed in the wrong direction when DPDK switched the preference towards runtime configurability and deprecated
compile
Quoted 1 line
Quoted 1 line
time configurability. I do understand the reasoning behind it, and
the
Quoted 4 lines
Quoted 1 line
impact is minimal, so we accept it.
The code can be optimized by removing some instructions with #ifdef. But the complexity of managing #ifdef enabling/disabling, depending
on the
Quoted 2 lines
platform and the use case, would be huge.
We try to have a reasonable code "always enabled" which performs wellin all
Quoted 1 line
cases. This is a design choice which makes DPDK a library, not a pool
of code
Quoted 3 lines
to cherry-pick.
Quoted 1 line
However, if DPDK starts sacrificing performance of the core
libraries
Quoted 6 lines
Quoted 3 lines
for the benefits of the GNU/Linux distributors, network appliance vendors may put more effort into sticking with old DPDK versions instead of updating.
The initial choice regarding ABI compatibility was "do not care". Recently, the decision was done to care about ABI compatibility as
priority
Quoted 2 lines
number 2. The priority number 1 remains the performance. That's a reason for allowing some ABI breakages in some specific
releases
Quoted 4 lines
announced in advance.
Quoted 1 line
Quoted 1 line
I think we need to focus first on slow path APIs ABI stuff.
Quoted 8 lines
Quoted 1 line
Yes we should not degrade fast path performance for the sake of
avoiding
Quoted 3 lines
uncertain future ABI issues.
Morten, Jerin, thank you for the feedback.
I think we have a consensus here not to make any changes to inline functions for now.
Should we mark this as 'Deferred or Rejected'?
Rejected.
There is no need for this modification now, and no actual use cases for it in the road map. In other words: This modification has no use cases; it is purely academic. Many other suggestions have been rejected for the reason that they have no current use cases.
As Thomas mentioned, DPDK has transitioned towards being a library, rather than a pool of code to cherry-pick from. I have learned to live with this.
Being a library doesn't mean that functions cannot be exposed as inline code in the library header files. DPDK is mainly a high performance library with a tradition of exposing many of its internals in its API, and we should keep it this way. We certainly don't want an opaque API hiding all of its internals, passing around void pointers.
However, it was still an interesting experiment to investigate the performance cost.
Quoted 105 lines
Quoted 91 lines
From: dev [mailto:[email protected]] On Behalf Of Honnappa Nagarahalli
Sent: Wednesday, July 1, 2020 1:16 AM<snip>
Quoted 1 line
Subject: Re: [dpdk-dev] [RFC] ring: make ring implementation non-
inlined
Quoted 9 lines
26/03/2020 09:04, Morten Brørup:
Quoted 7 lines
From: Jerin Jacob
Quoted 6 lines · On Fri, Mar 20, 2020 Konstantin Ananyev wrote:
Quoted 5 lines
As was discussed here:
http://mails.dpdk.org/archives/dev/2020-February/158586.html
this RFC aimed to hide ring internals into .c and make all ring functions non-inlined. In theory that might help to maintainABI
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
stability in future.
This is just a POC to measure the impact of proposed idea,proper
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
implementation would definetly need some extra effort. On IA box (SKX)
ring_perf_autotestshows ~20-30 cycles extrafor
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
enqueue+dequeue pair. On some more realistic code, I suspect the impact it might be a bit higher.
For MP/MC bulk transfers degradation seems quite small, thoughfor
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
SP/SC and/or small transfers it is more then noticable (see
exact
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
numbers below).
From my perspective we'd probably keep it inlined for now toavoid
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
any non-anticipated perfomance degradations. Though intersted to see perf results and opinions from other interested parties.
+1
Konstantin, thank you for doing some measures
Quoted 1 line
Quoted 1 line
My reasoning is a bit different, DPDK is using in embedded boxes
too
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
where performance has more weight than ABI stuff.
As a network appliance vendor I can confirm that we certainly care more about performance than ABI stability.
ABI stability is irrelevant for us;
and API instability is a non-recurring engineering cost each timewe
Quoted 1 line
Quoted 1 line
choose to switch to a new DPDK version, which we only do if we
cannot
Quoted 1 line
Quoted 1 line
avoid it, e.g. due to new drivers, security fixes or new features
that
Quoted 4 lines
Quoted 4 lines
we want to use.
For us, the trend pointed in the wrong direction when DPDK switched the preference towards runtime configurability and deprecated
compile
Quoted 1 line
Quoted 1 line
time configurability. I do understand the reasoning behind it, and
the
Quoted 4 lines
Quoted 1 line
impact is minimal, so we accept it.
The code can be optimized by removing some instructions with #ifdef. But the complexity of managing #ifdef enabling/disabling, depending
on the
Quoted 2 lines
platform and the use case, would be huge.
We try to have a reasonable code "always enabled" which performs wellin all
Quoted 1 line
cases. This is a design choice which makes DPDK a library, not a pool
of code
Quoted 3 lines
to cherry-pick.
Quoted 1 line
However, if DPDK starts sacrificing performance of the core
libraries
Quoted 6 lines
Quoted 3 lines
for the benefits of the GNU/Linux distributors, network appliance vendors may put more effort into sticking with old DPDK versions instead of updating.
The initial choice regarding ABI compatibility was "do not care". Recently, the decision was done to care about ABI compatibility as
priority
Quoted 2 lines
number 2. The priority number 1 remains the performance. That's a reason for allowing some ABI breakages in some specific
releases
Quoted 4 lines
announced in advance.
Quoted 1 line
Quoted 1 line
I think we need to focus first on slow path APIs ABI stuff.
Quoted 8 lines
Quoted 1 line
Yes we should not degrade fast path performance for the sake of
avoiding
Quoted 3 lines
uncertain future ABI issues.
Morten, Jerin, thank you for the feedback.
I think we have a consensus here not to make any changes to inline functions for now.
Should we mark this as 'Deferred or Rejected'?Rejected.
There is no need for this modification now, and no actual use cases for it in the road map. In other words: This modification has no use cases; it is purely academic. Many other suggestions have been rejected for the reason that they have no current use cases.
Quoted 8 lines
As Thomas mentioned, DPDK has transitioned towards being a library, rather than a pool of code to cherry-pick from. I have learned to live with this.
Being a library doesn't mean that functions cannot be exposed as inline code in the library header files. DPDK is mainly a high performance library with a tradition of exposing many of its internals in its API, and we should keep it this way. We certainly don't want an opaque API hiding all of its internals, passing around void pointers.
However, it was still an interesting experiment to investigate the performance cost.
Yes, please reject it.
Konstantin
<snip>
Quoted 117 lines
Quoted 99 lines
Quoted 99 lines
Quoted 2 lines
Subject: Re: [dpdk-dev] [RFC] ring: make ring implementation non-
inlined
Quoted 10 lines
26/03/2020 09:04, Morten Brørup:
Quoted 8 lines
From: Jerin Jacob
Quoted 7 lines · On Fri, Mar 20, 2020 Konstantin Ananyev wrote:
Quoted 6 lines
As was discussed here:
http://mails.dpdk.org/archives/dev/2020-February/158586.html
this RFC aimed to hide ring internals into .c and make all ring functions non-inlined. In theory that might help to maintainABI
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
stability in future.
This is just a POC to measure the impact of proposed idea,proper
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
implementation would definetly need some extra effort. On IA box (SKX)
ring_perf_autotestshows ~20-30 cycles extrafor
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
enqueue+dequeue pair. On some more realistic code, I suspect the impact it might be a bit higher.
For MP/MC bulk transfers degradation seems quite small, thoughfor
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
SP/SC and/or small transfers it is more then noticable (see
exact
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
numbers below).
From my perspective we'd probably keep it inlined for now toavoid
Quoted 11 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
any non-anticipated perfomance degradations. Though intersted to see perf results and opinions from other interested parties.
+1
Konstantin, thank you for doing some measures
Quoted 2 lines
Quoted 2 lines
My reasoning is a bit different, DPDK is using in embedded boxes
too
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
where performance has more weight than ABI stuff.
As a network appliance vendor I can confirm that we certainly care more about performance than ABI stability.
ABI stability is irrelevant for us; and API instability is a non-recurring engineering cost each timewe
Quoted 1 line
Quoted 1 line
choose to switch to a new DPDK version, which we only do if we
cannot
Quoted 2 lines
Quoted 2 lines
avoid it, e.g. due to new drivers, security fixes or new features
that
Quoted 5 lines
Quoted 5 lines
we want to use.
For us, the trend pointed in the wrong direction when DPDK switched the preference towards runtime configurability and deprecated
compile
Quoted 2 lines
Quoted 2 lines
time configurability. I do understand the reasoning behind it, and
the
Quoted 5 lines
Quoted 1 line
impact is minimal, so we accept it.
The code can be optimized by removing some instructions with #ifdef. But the complexity of managing #ifdef enabling/disabling, depending
on the
Quoted 3 lines
platform and the use case, would be huge.
We try to have a reasonable code "always enabled" which performs wellin all
Quoted 2 lines
cases. This is a design choice which makes DPDK a library, not a pool
of code
Quoted 3 lines
to cherry-pick.
Quoted 1 line
However, if DPDK starts sacrificing performance of the core
libraries
Quoted 6 lines
Quoted 3 lines
for the benefits of the GNU/Linux distributors, network appliance vendors may put more effort into sticking with old DPDK versions instead of updating.
The initial choice regarding ABI compatibility was "do not care". Recently, the decision was done to care about ABI compatibility as
priority
Quoted 2 lines
number 2. The priority number 1 remains the performance. That's a reason for allowing some ABI breakages in some specific
releases
Quoted 5 lines
announced in advance.
Quoted 1 line
Quoted 1 line
I think we need to focus first on slow path APIs ABI stuff.
Yes we should not degrade fast path performance for the sake of
avoiding
Quoted 3 lines
uncertain future ABI issues.
Morten, Jerin, thank you for the feedback.
Quoted 8 lines
Quoted 3 lines
I think we have a consensus here not to make any changes to inline functions for now.
Should we mark this as 'Deferred or Rejected'?Rejected.
There is no need for this modification now, and no actual use cases for it in the road map. In other words: This modification has no use cases; it
is purely academic. Many other suggestions have been rejected for the reason that they have no current use cases.
Quoted 3 lines
As Thomas mentioned, DPDK has transitioned towards being a library, rather than a pool of code to cherry-pick from. I have learned to live with
this.
Quoted 4 lines
Being a library doesn't mean that functions cannot be exposed as inline code in the library header files. DPDK is mainly a high performance library with a tradition of exposing many of its internals in its
Quoted 7 lines
API, and we should keep it this way. We certainly don't want an opaque API hiding all of its internals, passing around void pointers.
Quoted 2 lines
However, it was still an interesting experiment to investigate the
performance cost.
Yes, please reject it.
I just tried to mark it rejected in patchwork, I do not have the permissions (probably you are the owner of the patch). Can you please mark it?
Quoted 2 lines
Konstantin
Quoted 2 lines · Honnappa Nagarahalli
Quoted 1 line
Yes, please reject it.
I just tried to mark it rejected in patchwork, I do not have the permissions (probably you are the owner of the patch). Can you please mark it?
Indeed, Konstantin should be able to do it.
I just flagged it Rejected.
There are still some ring patches waiting in pw, could you and Konstantin agree on/flag them?
Thanks.
Signature · 1 line
David Marchand