event/slb: introducing software load balancer eventdev PMD
cover event/slb: introducing software load balancer eventdev PMD
From: Pravin Pathak <[email protected]>
This series introduces the Software Load Balancer (SLB) Eventdev PMD (event_slb) supporting parallel, atomic and ordered queues. It is a distributed software scheduler that scales well.
Motivation
----------
Existing software eventdev PMDs face trade-offs between centralized bottlenecks and feature coverage:
- event_sw relies on a dedicated centralized scheduler consuming an extra CPU core. The single scheduling core also limits maximum throughput and it does not scale.
- event_dsw distributes scheduling across worker cores without a central scheduler core, but does not support ordered queues and incurs per-port control ring messaging overhead during atomic flow migrations. It also does not support service priority.
Design Overview
---------------event_slb is a distributed (RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED), maintenance-free (RTE_EVENT_DEV_CAP_MAINTENANCE_FREE) software eventdev PMD that supports Atomic, Ordered, Parallel, and Single-Link queues:
1. Queue-Centric Ring Architecture:
- Parallel and Single-Link queues map directly to rte_event_ring instances (MP/MC for parallel, MP/SC for single-link), allowing zero-copy burst enqueue and dequeue without history tracking.
- Ordered queues use the ring slot index at dequeue time as an implicit sequence number into a per-queue reorder buffer, draining contiguous completed events in order without global sequence counters. This makes ordered queue implementaion highly efficient performing close to parallel queues.
- Atomic queues use a main MP/SC queue ring alongside per-port SP/SC rings and a 2048-entry flow-to-port affinity table. All cores served by the atomic queue are used for scheduling.
2. Lock-Free / Zero-Pause Atomic Flow Migration:
- When an atomic queue refills per-port rings and detects persistent congestion on a destination port, SLB selects the lightest flow on that port (outside the migration cooldown window) and waits for queues in-flight events to drain naturally before switching the flow's port affinity and schedule hold-buffer entries.
- Hold buffer is used for buffering events towards migrating ports while allowing serving other ports.
- Implementaion avoids cross-port pause/unpause control rings while preserving strict per-flow ordering.
3. Credit-Based Backpressure & Priority Scheduling:
- A global atomic credit pool with per-port local credit caches bounds total in-flight events (nb_events_limit) and enforces new_event_threshold with minimal atomic contention.
- Ports support two strict service priority levels with round-robin scheduling within each priority level.
4. Observability & Testing:
- Device and per-port/per-queue extended statistics (xstats), detailed rte_event_dev_dump() diagnostics, and a unit self-test suite (eventdev_selftest_slb) covering parallel, atomic, ordered, single-link, priority, backpressure, stop-flush, and multi-core atomic migration scenarios.
5. Future Plan:
- Supporting Event priority
- Optimizing atomic flow migration
Pravin Pathak (8):
event/slb: introduce software load balancer PMD event/slb: add support for ordered queues
event/slb: add support for atomic queues
event/slb: add dynamic flow migration for atomic queues event/slb: add extended statistics and diagnostics event/slb: add self-test and documentation
event/slb: update resets and credit defaults event/slb: improve port linking, config checks and atomic refill
MAINTAINERS | 6 + app/test/test_eventdev.c | 7 + doc/guides/eventdevs/features/slb.ini | 25 + doc/guides/eventdevs/index.rst | 1 + doc/guides/eventdevs/slb.rst | 61 ++ drivers/event/meson.build | 1 + drivers/event/slb/meson.build | 21 + drivers/event/slb/slb_evdev.c | 904 ++++++++++++++++++ drivers/event/slb/slb_evdev.h | 263 ++++++ drivers/event/slb/slb_event.c | 1219 +++++++++++++++++++++++++ drivers/event/slb/slb_selftest.c | 522 +++++++++++ drivers/event/slb/slb_xstats.c | 347 +++++++ 12 files changed, 3377 insertions(+) create mode 100644 doc/guides/eventdevs/features/slb.ini create mode 100644 doc/guides/eventdevs/slb.rst create mode 100644 drivers/event/slb/meson.build create mode 100644 drivers/event/slb/slb_evdev.c create mode 100644 drivers/event/slb/slb_evdev.h create mode 100644 drivers/event/slb/slb_event.c create mode 100644 drivers/event/slb/slb_selftest.c create mode 100644 drivers/event/slb/slb_xstats.c
Signature · 1 line
2.53.0
PATCH 1/8 event/slb: introduce software load balancer PMD
From: Pravin Pathak <[email protected]>
Add basic skeleton and initial features for the Software Load Balancer (SLB) eventdev PMD.
This initial commit provides:
- Virtual device driver registration (event_slb) and device probing.
- Device configuration and capability reporting for parallel queues, burst mode, multi-queue ports, and runtime port linking.
- Parallel queue support (multi-producer/multi-consumer rings) and single-link queue support (multi-producer/single-consumer rings).
- Event port configuration, linking/unlinking, and multi-priority queue scheduling.
- Central in-flight event credit management with batched loans (SLB_MIN_CREDIT_LOAN = 512).
- High-performance event enqueue and dequeue burst processing.
- Meson build system integration and MAINTAINERS entry.
--- MAINTAINERS | 4 + drivers/event/meson.build | 1 + drivers/event/slb/meson.build | 15 + drivers/event/slb/slb_evdev.c | 585 ++++++++++++++++++++++++++++++++++ drivers/event/slb/slb_evdev.h | 176 ++++++++++ drivers/event/slb/slb_event.c | 279 ++++++++++++++++ 6 files changed, 1060 insertions(+) create mode 100644 drivers/event/slb/meson.build create mode 100644 drivers/event/slb/slb_evdev.c create mode 100644 drivers/event/slb/slb_evdev.h create mode 100644 drivers/event/slb/slb_event.c
MAINTAINERS +4 -0
@@ -1533,6 +1533,10 @@ M: Mattias Rönnblom <[email protected]> F: drivers/event/dsw/ F: doc/guides/eventdevs/dsw.rst +Software Load Balancer (SLB) Eventdev PMD +M: Pravin Pathak <[email protected]> +F: drivers/event/slb/ + Software OPDL Eventdev PMD M: Liang Ma <[email protected]> M: Peter Mccarthy <[email protected]>
drivers/event/meson.build +1 -0
@@ -15,5 +15,6 @@ drivers = [ 'opdl', 'skeleton', 'sw', + 'slb', ] std_deps = ['eventdev', 'kvargs']
drivers/event/slb/meson.build new file +15 -0
@@ -0,0 +1,15 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2026 Pravin Pathak <[email protected]> + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +sources = files( + 'slb_evdev.c', + 'slb_event.c', +) + +deps += ['bus_vdev', 'ring']
drivers/event/slb/slb_evdev.c new file +585 -0
585 lines not on this page · raw message
drivers/event/slb/slb_evdev.h new file +176 -0
176 lines not on this page · raw message
drivers/event/slb/slb_event.c new file +279 -0
279 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH 2/8 event/slb: add support for ordered queues
In reply to Pravin PathakFrom: Pravin Pathak <[email protected]>
Add support for RTE_SCHED_TYPE_ORDERED queues to provide strict packet reordering guarantees across worker ports.
A reorder buffer is allocated for each ordered queue to hold forwarded events until preceding sequence slots have completed. On dequeue from an ordered queue, sequence numbers are atomically reserved and recorded in the port's history tracker.
When worker ports enqueue forwarded or released events, the corresponding reorder buffer entries are marked ready or released. Contiguous ready events are drained in order to destination queues via slb_reorder_drain.
--- drivers/event/slb/slb_evdev.c | 45 ++- drivers/event/slb/slb_evdev.h | 39 +++ drivers/event/slb/slb_event.c | 529 +++++++++++++++++++++++++++++----- 3 files changed, 539 insertions(+), 74 deletions(-)
drivers/event/slb/slb_evdev.c +44 -1
@@ -17,6 +17,25 @@#define EVENTDEV_NAME_SLB_PMD event_slb +/* Reset history tracking table for a port. +* Token 0 is reserved for parallel and single-link events without history. +* Tokens 1 to 255 (255 total) are available for ordered and atomic queue events. +*/ +static void +slb_port_reset_history (struct slb_port *port) +{ + port->history[0 ].active = false; + port->history[0 ].next_free =SLB_PORT_HISTORY_FREE_END ; +for (int i =1 ; i <SLB_PORT_HISTORY_SIZE ; i++) { + port->history[i].active = false; + port->history[i].next_free = i +1 ; + } + port->history[SLB_PORT_HISTORY_SIZE -1 ].next_free = +SLB_PORT_HISTORY_FREE_END ; + port->history_head =1 ; + port->history_avail =SLB_PORT_HISTORY_MAX_TOKENS ; +} +/* Setup Eventdev port. Reset all mapped queues */ static int slb_port_setup(struct rte_eventdev *dev, uint8_t port_id,
@@ -54,6 +73,7 @@ slb_port_setup(struct rte_eventdev *dev, uint8_t port_id, .credits_drained = false, .total_dequeues = 0, .empty_dequeues = 0, + .pending_releases = 0, .single_linked_q = NULL, .single_linked_ring = NULL, };
@@ -71,6 +91,8 @@ slb_port_setup(struct rte_eventdev *dev, uint8_t port_id,memset (port->q_entries[p],0 ,sizeof (port->q_entries[p])); } +slb_port_reset_history (port); +return 0 ; }
@@ -99,6 +121,10 @@ slb_queue_release(struct rte_eventdev *dev, uint8_t queue_id)struct slb_evdev *slb =slb_pmd_priv (dev);struct slb_queue *queue = &slb->queues[queue_id]; +if (queue->reorder_buffer) { +rte_free (queue->reorder_buffer); + queue->reorder_buffer =NULL ; + }if (queue->q_ring) {rte_event_ring_free (queue->q_ring); queue->q_ring =NULL ;
@@ -128,7 +154,8 @@ slb_queue_setup(struct rte_eventdev *dev, uint8_t queue_id, queue->is_single_link = true; ring =rte_event_ring_create (ring_name, q_size, dev->data->socket_id,RING_F_SC_DEQ |RING_F_EXACT_SZ ); - }else if (conf->schedule_type ==RTE_SCHED_TYPE_PARALLEL ) { + }else if (conf->schedule_type ==RTE_SCHED_TYPE_PARALLEL || + conf->schedule_type ==RTE_SCHED_TYPE_ORDERED ) {/* MP -> MC ring for parallel queues */ queue->is_single_link = false; ring = rte_event_ring_create(ring_name, q_size,
@@ -147,6 +174,21 @@ slb_queue_setup(struct rte_eventdev *dev, uint8_t queue_id, queue->serving_ports_mask =0 ; queue->num_serving_ports =0 ; +/* Ordered queue reorder buffer allocation */ +if (conf->schedule_type ==RTE_SCHED_TYPE_ORDERED ) { + queue->reorder_buffer =rte_zmalloc_socket ("slb_reorder_buf" , +sizeof (struct slb_reorder_entry ) * ring->r.size, +RTE_CACHE_LINE_SIZE , dev->data->socket_id); +if (queue->reorder_buffer ==NULL ) { +rte_event_ring_free (ring); +return -ENOMEM ; + } + queue->reorder_drain_idx =0 ; +rte_spinlock_init (&queue->reorder_lock); + }else { + queue->reorder_buffer =NULL ; + } +return 0 ; }
@@ -337,6 +379,7 @@ slb_info_get(struct rte_eventdev *dev __rte_unused, .max_profiles_per_port = 1, .event_dev_cap =RTE_EVENT_DEV_CAP_BURST_MODE |RTE_EVENT_DEV_CAP_PARALLEL | +RTE_EVENT_DEV_CAP_ORDERED |RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED |RTE_EVENT_DEV_CAP_IMPLICIT_RELEASE_DISABLE | RTE_EVENT_DEV_CAP_NONSEQ_MODE |
drivers/event/slb/slb_evdev.h +39 -0
@@ -9,6 +9,7 @@#include <rte_bitset.h> #include <rte_event_ring.h> #include <rte_eventdev.h> +#include <rte_spinlock.h> #include <rte_stdatomic.h> #include <eventdev_pmd.h>
@@ -58,9 +59,37 @@ struct slb_port_q_entry {uint8_t pad[6 ]; }; +#define SLB_PORT_HISTORY_SIZE 256 +#define SLB_PORT_HISTORY_MAX_TOKENS (SLB_PORT_HISTORY_SIZE - 1) +#define SLB_PORT_HISTORY_FREE_END 0xFFFF + +struct slb_history_entry { +uint32_t flow_id; +uint16_t seq_slot; +uint16_t next_free; +uint8_t orig_queue_id; +uint8_t enq_port_id; +uint8_t sched_type; +bool active; +}; + +enum slb_reorder_state { +SLB_REORDER_EMPTY =0 , +SLB_REORDER_READY =1 , +SLB_REORDER_RELEASED =2 +}; + +struct slb_reorder_entry { +struct rte_event event; +RTE_ATOMIC (uint8_t ) state; +}; +struct __rte_cache_aligned slb_port {/* --- Cacheline 0: Hot Rx & Tx scheduling control (64 bytes) --- */ uint16_t id; +uint16_t history_head; +uint16_t history_avail; +uint16_t pending_releases;int32_t inflight_credits;int32_t new_event_threshold;RTE_ATOMIC (uint64_t ) active_queues;
@@ -78,6 +107,10 @@ struct __rte_cache_aligned slb_port { alignas(RTE_CACHE_LINE_SIZE )struct slb_port_q_entry q_entries[SLB_MAX_PRIORITIES ][SLB_PORT_MAX_Q_MAP_PER_PRIO ]; +/* --- Cachelines 7+: Per-port history buffer --- */ + alignas(RTE_CACHE_LINE_SIZE ) +struct slb_history_entry history[SLB_PORT_HISTORY_SIZE ]; +/* --- Cold Section: Statistics counters and config --- */ alignas(RTE_CACHE_LINE_SIZE )uint16_t dequeue_depth;
@@ -101,6 +134,10 @@ struct slb_queue {uint16_t num_serving_ports;uint64_t serving_ports_mask; alignas(RTE_CACHE_LINE_SIZE )struct rte_event_ring *q_ring; +/* For ordered queues (dynamically allocated) */ +struct slb_reorder_entry *reorder_buffer; +uint32_t reorder_drain_idx; +rte_spinlock_t reorder_lock; };/* Limited by the size of 'serving_ports_mask' and 'active_queues' bitmasks */
@@ -135,6 +172,8 @@ uint16_t slb_event_dequeue_burst(void *port, struct rte_event *events,void slb_port_return_credits (struct slb_evdev *slb,struct slb_port *port,int32_t credits); +void slb_reorder_drain (struct slb_evdev *slb,struct slb_queue *queue); +void slb_dump (struct rte_eventdev *dev,FILE *f); static inline struct slb_evdev *
drivers/event/slb/slb_event.c +456 -73
621 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH 3/8 event/slb: add support for atomic queues
In reply to Pravin PathakFrom: Pravin Pathak <[email protected]>
Add support for RTE_SCHED_TYPE_ATOMIC queues to provide per-flow event serialization across worker ports.
Flow affinity is maintained using a 2048-entry flow map that pins each flow ID to a serving worker port upon first observation. Events for each atomic queue are held in a main ring and scheduled into dedicated per-port rings via slb_atomic_refill using counting sort grouping.
A hold buffer retains events when destination port rings are temporarily full to preserve flow FIFO ordering. Worker ports dequeue atomic events from their private rings and record active flows in port history to track returns on release or forward operations.
--- drivers/event/slb/slb_evdev.c | 90 +++++++++- drivers/event/slb/slb_evdev.h | 37 ++++ drivers/event/slb/slb_event.c | 312 +++++++++++++++++++++++++++++++--- 3 files changed, 411 insertions(+), 28 deletions(-)
drivers/event/slb/slb_evdev.c +88 -2
@@ -121,6 +121,10 @@ slb_queue_release(struct rte_eventdev *dev, uint8_t queue_id)struct slb_evdev *slb =slb_pmd_priv (dev);struct slb_queue *queue = &slb->queues[queue_id]; +if (queue->atomic_state) { +rte_free (queue->atomic_state); + queue->atomic_state =NULL ; + }if (queue->reorder_buffer) {rte_free (queue->reorder_buffer); queue->reorder_buffer =NULL ;
@@ -129,6 +133,12 @@ slb_queue_release(struct rte_eventdev *dev, uint8_t queue_id)rte_event_ring_free (queue->q_ring); queue->q_ring =NULL ; } +for (int i =0 ; i <SLB_MAX_PORTS ; i++) { +if (queue->per_port_rings[i]) { +rte_event_ring_free (queue->per_port_rings[i]); + queue->per_port_rings[i] =NULL ; + } + } }/* Setup Eventdev queue. Eventdev queues are implemented as DPDK rings */
@@ -154,9 +164,14 @@ slb_queue_setup(struct rte_eventdev *dev, uint8_t queue_id, queue->is_single_link = true; ring =rte_event_ring_create (ring_name, q_size, dev->data->socket_id,RING_F_SC_DEQ |RING_F_EXACT_SZ ); + }else if (conf->schedule_type ==RTE_SCHED_TYPE_ATOMIC ) { +/* MP -> SC ring for atomic queues main ring */ + queue->is_single_link = false; + ring =rte_event_ring_create (ring_name, q_size, + dev->data->socket_id,RING_F_SC_DEQ |RING_F_EXACT_SZ ); }else if (conf->schedule_type ==RTE_SCHED_TYPE_PARALLEL || conf->schedule_type ==RTE_SCHED_TYPE_ORDERED ) { -/* MP -> MC ring for parallel queues */ +/* MP -> MC ring for parallel or ordered queues */ queue->is_single_link = false; ring =rte_event_ring_create (ring_name, q_size, dev->data->socket_id,RING_F_EXACT_SZ );
@@ -174,6 +189,31 @@ slb_queue_setup(struct rte_eventdev *dev, uint8_t queue_id, queue->serving_ports_mask =0 ; queue->num_serving_ports =0 ; +/* Atomic queue allocation */ +if (conf->schedule_type ==RTE_SCHED_TYPE_ATOMIC ) { +for (int i =0 ; i <SLB_MAX_PORTS ; i++) + queue->per_port_rings[i] =NULL ; + + queue->atomic_state =rte_zmalloc_socket ("slb_atomic_state" , +sizeof (struct slb_atomic_state ), +RTE_CACHE_LINE_SIZE , dev->data->socket_id); +if (queue->atomic_state ==NULL ) { +rte_event_ring_free (ring); +return -ENOMEM ; + } +for (int i =0 ; i <SLB_ATOMIC_FLOWS ; i++) +rte_atomic_store_explicit ( + &queue->atomic_state->flow_map.flows[i].port_id, +-1 , rte_memory_order_relaxed); + queue->atomic_state->next_port_rr =0 ; +rte_spinlock_init (&queue->atomic_state->refill_lock); + queue->atomic_state->hold_count =0 ; +memset (queue->atomic_state->port_stats,0 , +sizeof (queue->atomic_state->port_stats)); + }else { + queue->atomic_state =NULL ; + } +/* Ordered queue reorder buffer allocation */ if (conf->schedule_type ==RTE_SCHED_TYPE_ORDERED ) { queue->reorder_buffer = rte_zmalloc_socket("slb_reorder_buf",
@@ -244,6 +284,26 @@ slb_port_link_one(struct slb_port *port, struct slb_queue *queue, port->single_linked_q = queue; port->single_linked_ring = queue->q_ring; }else { +if (queue->schedule_type ==RTE_SCHED_TYPE_ATOMIC && + queue->per_port_rings[port->id] ==NULL ) { +char ring_name[RTE_RING_NAMESIZE ]; +struct rte_event_ring *ring; + +snprintf (ring_name,sizeof (ring_name), +"slb%d_q%u_p%u" , port->slb->data->dev_id, + qid, port->id); + ring =rte_event_ring_create ( + ring_name, +SLB_MAX_PORT_DEQUEUE_DEPTH *4 , + port->slb->data->socket_id, +RING_F_SP_ENQ |RING_F_SC_DEQ ); +if (ring ==NULL ) { + rte_errno =ENOMEM ; +return -ENOMEM ; + } + queue->per_port_rings[port->id] = ring; + } +if (has_prio) prio = (app_prio <RTE_EVENT_DEV_PRIORITY_NORMAL ) ?0 :1 ;else
@@ -262,12 +322,15 @@ slb_port_link_one(struct slb_port *port, struct slb_queue *queue, }if (!found) {uint16_t slot = port->num_q[prio]; +struct rte_event_ring *dq_ring = + (queue->schedule_type ==RTE_SCHED_TYPE_ATOMIC ) ? + queue->per_port_rings[port->id] : queue->q_ring;if (slot >=SLB_PORT_MAX_Q_MAP_PER_PRIO ) { rte_errno =ENOSPC ;return -ENOSPC ; } - port->q_entries[prio][slot].dq_ring = queue->q_ring; + port->q_entries[prio][slot].dq_ring = dq_ring; port->q_entries[prio][slot].queue = queue; port->q_entries[prio][slot].queue_id = qid; port->q_entries[prio][slot].schedule_type =
@@ -311,6 +374,13 @@ slb_port_unlink_one(struct slb_port *port, struct slb_queue *queue, uint8_t qid) rte_errno =EINVAL ;return -EINVAL ; } + +if (queue->schedule_type ==RTE_SCHED_TYPE_ATOMIC && + queue->per_port_rings[port->id] !=NULL ) { +rte_event_ring_free (queue->per_port_rings[port->id]); + queue->per_port_rings[port->id] =NULL ; + } +rte_atomic_fetch_and_explicit (&port->active_queues, ~(1ULL << qid), rte_memory_order_release);
@@ -379,6 +449,7 @@ slb_info_get(struct rte_eventdev *dev __rte_unused, .max_profiles_per_port = 1, .event_dev_cap =RTE_EVENT_DEV_CAP_BURST_MODE |RTE_EVENT_DEV_CAP_PARALLEL | +RTE_EVENT_DEV_CAP_ATOMIC |RTE_EVENT_DEV_CAP_ORDERED |RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED | RTE_EVENT_DEV_CAP_IMPLICIT_RELEASE_DISABLE |
@@ -474,6 +545,21 @@ slb_stop(struct rte_eventdev *dev)for (uint8_t qid =0 ; qid < slb->num_queues; qid++) {struct slb_queue *q = &slb->queues[qid]; + +if (q->schedule_type ==RTE_SCHED_TYPE_ATOMIC && q->atomic_state !=NULL ) { +for (int f =0 ; f <SLB_ATOMIC_FLOWS ; f++) +rte_atomic_store_explicit ( + &q->atomic_state->flow_map.flows[f].port_id, +-1 , rte_memory_order_relaxed); + +for (uint16_t pid =0 ; pid < slb->num_ports; pid++) { +if (q->per_port_rings[pid] !=NULL ) +slb_drain_ring (dev_id, + q->per_port_rings[pid], + flush, flush_arg); + } + } +if (q->q_ring !=NULL )slb_drain_ring (dev_id, q->q_ring, flush, flush_arg); }
drivers/event/slb/slb_evdev.h +37 -0
@@ -84,6 +84,39 @@ struct slb_reorder_entry {RTE_ATOMIC (uint8_t ) state; }; +#define SLB_ATOMIC_FLOWS 2048 +#define SLB_ATOMIC_FLOWS_MASK (SLB_ATOMIC_FLOWS - 1) +#define SLB_HOLD_BUFFER_SIZE (SLB_MAX_PORT_DEQUEUE_DEPTH * 64) + +/* Per-flow entry for atomic scheduling affinity */ +struct slb_flow { +RTE_ATOMIC (int32_t ) port_id;/* Port this flow is pinned to */ +}; + +/* Flow-to-port affinity mapping table */ +struct slb_flow_map { +struct slb_flow flows[SLB_ATOMIC_FLOWS ]; +}; + +/* Per-port counters for an atomic queue */ +struct __rte_cache_aligned slb_atomic_port_stats { +uint32_t flow_count;/* Number of active flows mapped to this port */ +uint64_t q_enqueued;/* Events scheduled into this port's atomic port-Q */ +uint64_t q_returned;/* Events returned (FORWARD/RELEASE) by this port */ +}; + +/* Dynamically allocated state for atomic queues */ +struct slb_atomic_state { +struct slb_flow_map flow_map; +uint32_t next_port_rr; +rte_spinlock_t refill_lock; +uint16_t hold_count; + alignas(RTE_CACHE_LINE_SIZE ) +struct slb_atomic_port_stats port_stats[SLB_MAX_PORTS ]; +uint8_t hold_ports[SLB_HOLD_BUFFER_SIZE ]; +struct rte_event hold_buffer[SLB_HOLD_BUFFER_SIZE ]; +}; +struct __rte_cache_aligned slb_port {/* --- Cacheline 0: Hot Rx & Tx scheduling control (64 bytes) --- */ uint16_t id;
@@ -134,6 +167,10 @@ struct slb_queue {uint16_t num_serving_ports;uint64_t serving_ports_mask; alignas(RTE_CACHE_LINE_SIZE )struct rte_event_ring *q_ring; +/* Per-port rings for atomic queues */ +struct rte_event_ring *per_port_rings[SLB_MAX_PORTS ]; +/* For atomic queues (dynamically allocated) */ +struct slb_atomic_state *atomic_state;/* For ordered queues (dynamically allocated) */ struct slb_reorder_entry *reorder_buffer;uint32_t reorder_drain_idx;
drivers/event/slb/slb_event.c +286 -26
369 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH 4/8 event/slb: add dynamic flow migration for atomic queues
In reply to Pravin PathakFrom: Pravin Pathak <[email protected]>
Add dynamic flow migration to automatically rebalance flows across worker ports when an atomic queue's destination port becomes congested.
When consecutive enqueue attempts to a worker's private atomic ring fail and exceed SLB_MIGRATION_LOAD_THRESHOLD, migration is initiated. The PMD inspects the overloaded ring and selects the lightest active flow not currently in migration cooldown.
To preserve strict per-flow event ordering, migration completes only after all previously scheduled events for the source port have been returned (q_enqueued == q_returned). The flow affinity is then remapped to the least-loaded worker port, and any matching events retained in the hold buffer are reassigned.
--- drivers/event/slb/slb_evdev.c | 16 ++- drivers/event/slb/slb_evdev.h | 20 ++- drivers/event/slb/slb_event.c | 252 ++++++++++++++++++++++++++++++++-- 3 files changed, 273 insertions(+), 15 deletions(-)
drivers/event/slb/slb_evdev.c +15 -1
@@ -74,8 +74,13 @@ slb_port_setup(struct rte_eventdev *dev, uint8_t port_id, .total_dequeues = 0, .empty_dequeues = 0, .pending_releases = 0, + .is_loaded = false, .single_linked_q = NULL, .single_linked_ring = NULL, + .flow_migrations = 0, + .migrations_in = 0, + .migrations_out = 0, + .last_migrated_time = 0, }; port->remap_input_q = false;
@@ -208,8 +213,12 @@ slb_queue_setup(struct rte_eventdev *dev, uint8_t queue_id, queue->atomic_state->next_port_rr =0 ;rte_spinlock_init (&queue->atomic_state->refill_lock); queue->atomic_state->hold_count =0 ; + queue->atomic_state->migration.in_progress = false; + queue->atomic_state->migrations_count =0 ;memset (queue->atomic_state->port_stats,0 ,sizeof (queue->atomic_state->port_stats)); +memset (queue->atomic_state->flow_last_migrated,0 , +sizeof (queue->atomic_state->flow_last_migrated)); }else { queue->atomic_state =NULL ; }
@@ -547,12 +556,16 @@ slb_stop(struct rte_eventdev *dev)struct slb_queue *q = &slb->queues[qid];if (q->schedule_type ==RTE_SCHED_TYPE_ATOMIC && q->atomic_state !=NULL ) { -for (int f =0 ; f <SLB_ATOMIC_FLOWS ; f++) +for (int f =0 ; f <SLB_ATOMIC_FLOWS ; f++) {rte_atomic_store_explicit ( &q->atomic_state->flow_map.flows[f].port_id,-1 , rte_memory_order_relaxed); + q->atomic_state->flow_last_migrated[f] =0 ; + } + q->atomic_state->migration.in_progress = false;for (uint16_t pid =0 ; pid < slb->num_ports; pid++) { + q->atomic_state->port_stats[pid].load_counter =0 ;if (q->per_port_rings[pid] !=NULL ) slb_drain_ring(dev_id, q->per_port_rings[pid],
@@ -568,6 +581,7 @@ slb_stop(struct rte_eventdev *dev)struct slb_port *port = &slb->ports[i]; port->inflight_credits =0 ; port->credits_drained = false; + port->is_loaded = false;for (int p =0 ; p <SLB_MAX_PRIORITIES ; p++) port->next_q[p] =0 ; }
drivers/event/slb/slb_evdev.h +19 -1
@@ -87,6 +87,8 @@ struct slb_reorder_entry {#define SLB_ATOMIC_FLOWS 2048#define SLB_ATOMIC_FLOWS_MASK (SLB_ATOMIC_FLOWS - 1)#define SLB_HOLD_BUFFER_SIZE (SLB_MAX_PORT_DEQUEUE_DEPTH * 64) +#define SLB_MIGRATION_LOAD_THRESHOLD 16 +#define SLB_MIGRATION_COOLDOWN_MS 500/* Per-flow entry for atomic scheduling affinity */ struct slb_flow {
@@ -100,17 +102,28 @@ struct slb_flow_map {/* Per-port counters for an atomic queue */ struct __rte_cache_aligned slb_atomic_port_stats { +uint32_t load_counter;/* Incremented when full, decremented when space available */ uint32_t flow_count;/* Number of active flows mapped to this port */ uint64_t q_enqueued;/* Events scheduled into this port's atomic port-Q */ uint64_t q_returned;/* Events returned (FORWARD/RELEASE) by this port */ }; +/* Active migration state for an atomic queue */ +struct slb_migration_state { +bool in_progress; +uint8_t port_id;/* Source port being drained */ +uint32_t flow_idx;/* Flow hash index being migrated */ +}; +/* Dynamically allocated state for atomic queues */ struct slb_atomic_state {struct slb_flow_map flow_map;uint32_t next_port_rr;rte_spinlock_t refill_lock;uint16_t hold_count; +struct slb_migration_state migration; +uint64_t migrations_count; +uint64_t flow_last_migrated[SLB_ATOMIC_FLOWS ]; alignas(RTE_CACHE_LINE_SIZE )struct slb_atomic_port_stats port_stats[SLB_MAX_PORTS ];uint8_t hold_ports[SLB_HOLD_BUFFER_SIZE ];
@@ -132,7 +145,8 @@ struct __rte_cache_aligned slb_port {bool implicit_release;bool credits_drained;bool remap_input_q; -uint8_t pad_port[4 ]; +bool is_loaded;/* Port marked loaded for atomic flow migration */ +uint8_t pad_port[3 ];struct rte_event_ring *single_linked_ring;struct slb_evdev *slb;
@@ -149,6 +163,10 @@ struct __rte_cache_aligned slb_port {uint16_t dequeue_depth;uint16_t enqueue_depth;struct slb_queue *single_linked_q; +uint64_t flow_migrations; +uint64_t migrations_in; +uint64_t migrations_out; +uint64_t last_migrated_time;uint64_t enqueue_calls;uint64_t new_enqueued;uint64_t forward_enqueued;
drivers/event/slb/slb_event.c +239 -13
305 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH 5/8 event/slb: add extended statistics and diagnostics
In reply to Pravin PathakFrom: Pravin Pathak <[email protected]>
Implement the eventdev extended statistics (xstats) interface and enhance PMD diagnostics in the dump callback.
Device and per-port statistics are exposed through xstats callbacks, including enqueue/dequeue counts, credit loan counts, and migration tallies. The slb_dump callback is extended to display detailed per-port and per-queue internal state, atomic migration status, and queue depth.
--- drivers/event/slb/meson.build | 5 + drivers/event/slb/slb_evdev.c | 94 ++++++++- drivers/event/slb/slb_evdev.h | 11 ++ drivers/event/slb/slb_xstats.c | 347 +++++++++++++++++++++++++++++++++ 4 files changed, 450 insertions(+), 7 deletions(-) create mode 100644 drivers/event/slb/slb_xstats.c
drivers/event/slb/meson.build +5 -0
@@ -10,6 +10,11 @@ endif sources = files( 'slb_evdev.c', 'slb_event.c', + 'slb_xstats.c', ) deps += ['bus_vdev', 'ring'] + +if cc.has_argument('-Wno-format-nonliteral') + cflags += '-Wno-format-nonliteral' +endif
drivers/event/slb/slb_evdev.c +87 -7
141 lines not on this page · raw message
drivers/event/slb/slb_evdev.h +11 -0
@@ -231,6 +231,17 @@ void slb_reorder_drain(struct slb_evdev *slb, struct slb_queue *queue);void slb_dump (struct rte_eventdev *dev,FILE *f); +int slb_xstats_get_names (const struct rte_eventdev *dev, +enum rte_event_dev_xstats_mode mode, +uint8_t queue_port_id, +struct rte_event_dev_xstats_name *xstats_names, +uint64_t *ids,unsigned int capacity); +int slb_xstats_get (const struct rte_eventdev *dev, +enum rte_event_dev_xstats_mode mode,uint8_t queue_port_id, +const uint64_t ids[],uint64_t values[],unsigned int n); +uint64_t slb_xstats_get_by_name (const struct rte_eventdev *dev, +const char *name,uint64_t *id); +static inline struct slb_evdev *slb_pmd_priv (const struct rte_eventdev *eventdev) {
drivers/event/slb/slb_xstats.c new file +347 -0
347 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH 6/8 event/slb: add self-test and documentation
In reply to Pravin PathakFrom: Pravin Pathak <[email protected]>
Add driver self-test coverage and documentation for the Software Load Balancer PMD.
The self-test verifies parallel queues, atomic queue flow ordering, ordered queues with forward sequence restoration, single-link queues, and device stop/restart event draining. Register the self-test in app/test as a fast test.
Add PMD user guide, feature matrix table, release notes entry for DPDK 26.11, and update MAINTAINERS.
--- MAINTAINERS | 2 + app/test/test_eventdev.c | 7 + doc/guides/eventdevs/features/slb.ini | 26 ++ doc/guides/eventdevs/index.rst | 1 + doc/guides/eventdevs/slb.rst | 68 ++++ drivers/event/slb/meson.build | 1 + drivers/event/slb/slb_evdev.c | 1 + drivers/event/slb/slb_evdev.h | 4 + drivers/event/slb/slb_selftest.c | 522 ++++++++++++++++++++++++++ 9 files changed, 632 insertions(+) create mode 100644 doc/guides/eventdevs/features/slb.ini create mode 100644 doc/guides/eventdevs/slb.rst create mode 100644 drivers/event/slb/slb_selftest.c
MAINTAINERS +2 -0
@@ -1536,6 +1536,8 @@ F: doc/guides/eventdevs/dsw.rst Software Load Balancer (SLB) Eventdev PMD M: Pravin Pathak <[email protected]> F: drivers/event/slb/ +F: doc/guides/eventdevs/slb.rst +F: doc/guides/eventdevs/features/slb.ini Software OPDL Eventdev PMD M: Liang Ma <[email protected]>
app/test/test_eventdev.c +7 -0
@@ -1527,12 +1527,19 @@ test_eventdev_selftest_cn20k(void) return test_eventdev_selftest_impl("event_cn20k", ""); } +static int +test_eventdev_selftest_slb (void ) +{ +return test_eventdev_selftest_impl ("event_slb" ,"" ); +} +#endif /* !RTE_EXEC_ENV_WINDOWS */ REGISTER_FAST_TEST (eventdev_common_autotest,NOHUGE_OK ,ASAN_OK , test_eventdev_common);#ifndef RTE_EXEC_ENV_WINDOWS REGISTER_FAST_TEST (eventdev_selftest_sw,NOHUGE_OK ,ASAN_OK , test_eventdev_selftest_sw); +REGISTER_FAST_TEST (eventdev_selftest_slb,NOHUGE_OK ,ASAN_OK , test_eventdev_selftest_slb);REGISTER_DRIVER_TEST (eventdev_selftest_octeontx, test_eventdev_selftest_octeontx);REGISTER_DRIVER_TEST (eventdev_selftest_dpaa2, test_eventdev_selftest_dpaa2);REGISTER_DRIVER_TEST (eventdev_selftest_dlb2, test_eventdev_selftest_dlb2);
doc/guides/eventdevs/features/slb.ini new file +26 -0
@@ -0,0 +1,26 @@ +; +; Supported features of the 'software load balancer' (event_slb) eventdev driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Scheduling Features] +atomic_scheduling = Y +ordered_scheduling = Y +parallel_scheduling = Y +distributed_sched = Y +burst_mode = Y +implicit_release_disable = Y +nonseq_mode = Y +runtime_port_link = Y +multiple_queue_port = Y +carry_flow_id = Y +maintenance_free = Y +independent_enq = Y + +[Eth Rx adapter Features] + +[Eth Tx adapter Features] + +[Crypto adapter Features] + +[Timer adapter Features]
doc/guides/eventdevs/index.rst +1 -0
@@ -20,3 +20,4 @@ application through the eventdev API. sw octeontx opdl + slb
doc/guides/eventdevs/slb.rst new file +68 -0
68 lines not on this page · raw message
drivers/event/slb/meson.build +1 -0
@@ -11,6 +11,7 @@ sources = files( 'slb_evdev.c', 'slb_event.c', 'slb_xstats.c', + 'slb_selftest.c', ) deps += ['bus_vdev', 'ring']
drivers/event/slb/slb_evdev.c +1 -0
@@ -758,6 +758,7 @@ static struct eventdev_ops slb_evdev_ops = { .xstats_get = slb_xstats_get, .xstats_get_names = slb_xstats_get_names, .xstats_get_by_name = slb_xstats_get_by_name, + .dev_selftest = test_slb_eventdev, }; static int
drivers/event/slb/slb_evdev.h +4 -0
@@ -242,6 +242,8 @@ int slb_xstats_get(const struct rte_eventdev *dev,uint64_t slb_xstats_get_by_name (const struct rte_eventdev *dev,const char *name,uint64_t *id); +int test_slb_eventdev (void ); +static inline struct slb_evdev *slb_pmd_priv (const struct rte_eventdev *eventdev) {
@@ -250,6 +252,8 @@ slb_pmd_priv(const struct rte_eventdev *eventdev)extern int event_slb_logtype;#define RTE_LOGTYPE_EVENT_SLB event_slb_logtype +#define SLB_LOG_LINE (level, ...) \ + RTE_LOG_LINE(level, EVENT_SLB, __VA_ARGS__)#define SLB_LOG_DP_LINE (level, fmt, ...) \ RTE_LOG_DP_LINE(level, EVENT_SLB, "%s() line %u: " fmt, \ __func__, __LINE__, ## __VA_ARGS__)
drivers/event/slb/slb_selftest.c new file +522 -0
522 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH 7/8 event/slb: update resets and credit defaults
In reply to Pravin PathakFrom: Pravin Pathak <[email protected]>
- Force RTE_SCHED_TYPE_ATOMIC on events dequeued from single-link queues and skip atomic/ordered state allocations for single-link queues.
- Remove RTE_EVENT_DEV_CAP_RUNTIME_PORT_LINK advertisement from driver info, feature matrix, and release notes.
- Reset atomic queue flow-to-port mappings, hold buffers, migration state, per-port stats, ordered reorder buffers, and port history when the device is stopped in slb_stop().
- Default port new_event_threshold in slb_port_def_conf() to 1/4 of SLB_IN_RING_SIZE (clamped to nb_events_limit when configured) and lower SLB_MIN_CREDIT_LOAN from 512 to 256 so more ports can borrow credits concurrently.
--- doc/guides/eventdevs/features/slb.ini | 1 - doc/guides/eventdevs/slb.rst | 31 ++++------ drivers/event/slb/slb_evdev.c | 81 +++++++++++++++++++++------ drivers/event/slb/slb_evdev.h | 5 +- drivers/event/slb/slb_event.c | 23 ++++++-- drivers/event/slb/slb_selftest.c | 12 ++++ 6 files changed, 110 insertions(+), 43 deletions(-)
doc/guides/eventdevs/features/slb.ini +0 -1
@@ -11,7 +11,6 @@ distributed_sched = Y burst_mode = Y implicit_release_disable = Y nonseq_mode = Y -runtime_port_link = Y multiple_queue_port = Y carry_flow_id = Y maintenance_free = Y
doc/guides/eventdevs/slb.rst +12 -19
@@ -4,9 +4,10 @@ Software Load Balancer (SLB) Eventdev Driver ============================================ -The Software Load Balancer (SLB) event device is a poll-mode eventdev driver -which distributes events among multiple worker ports and cores without requiring -dedicated scheduling service cores or hardware coprocessors. +The Software Load Balancer (SLB) is a software event device and a poll-mode Eventdev driver. +It uses a distributed scheduler, allowing it to scale well. SLB does not require dedicated +cores for scheduling. It supports Atomic, Parallel, and Ordered queues with dynamic atomic +flow load balancing. SLB also requires no internal core communication and is maintenance-free. Features --------
@@ -19,25 +20,17 @@ Queues * **Atomic**: Flow-pinned scheduling guaranteeing strict per-flow packet ordering. Includes dynamic load tracking and automatic flow migration to rebalance overloaded ports while preserving in-flight sequence integrity. -* **Ordered**: Ticket-based sequence tracking and reorder buffer providing end-to-end - packet order restoration across parallel processing workers. -* **Single-Link**: Dedicated direct queue mapping from a producer port to a single - consumer port using optimized single-consumer rings. +* **Ordered**: Sequence number-based tracking and reordering. +* **Single-Link**: Implemented using optimized multi-producer single-consumer rings. Ports & Scheduling Capabilities ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -* **Distributed Scheduling**: No dedicated service core required; scheduling work is - performed inline during enqueue and dequeue operations. -* **Prefix Sum Burst Enqueue**: Interleaved multi-queue bursts are grouped and scattered - via prefix-sum offsets, dispatching single batched bursts to queue rings to eliminate - atomic Compare-And-Swap contention on multi-producer rings. +* **Distributed Scheduling**: No dedicated service core required. * **Dynamic Flow Migration**: Atomic queues monitor port dequeue loads and migrate - eligible victim flows from overloaded ports to underloaded ports with cooling cycles. + Eligible flows from overloaded ports to underloaded ports with cooling cycles. * **Credit Pool Management**: Centralized event credit pool with per-port credit caching to enforce device-wide in-flight event limits (``max_num_events``). -* **Runtime Port Linking**: Supports dynamic linking and unlinking of ports and queues - both when stopped and at runtime. * **Maintenance Free**: Operates without requiring periodic maintenance calls (``RTE_EVENT_DEV_CAP_MAINTENANCE_FREE``).
@@ -62,7 +55,7 @@ application code or from the EAL command line: Limitations ----------- -* **Flow Hash Table Size**: Atomic queues use an internal flow mapping table - dimensioned to 2048 concurrent active flows per queue (``SLB_ATOMIC_FLOWS``). -* **Maximum Ports and Queues**: Configured for up to 64 ports (``SLB_MAX_PORTS``) - and 64 queues (``SLB_MAX_QUEUES``). +* **Maximum Ports and Queues**: Supports up to 64 ports and 64 queues. +* **Priorities and linking**: Supports two levels of service priority to link up to + eight queues per port at each priority. SLB currently does not support event priority. +* **Queues Types**: Does not support all-type queues.
drivers/event/slb/slb_evdev.c +65 -16
172 lines not on this page · raw message
drivers/event/slb/slb_evdev.h +3 -2
@@ -28,7 +28,7 @@#define SLB_MAX_FLOWS_MASK (SLB_MAX_FLOWS - 1)/* Avoid making small 'loans' from the central in-flight event credit pool */ -#define SLB_MIN_CREDIT_LOAN (512) +#define SLB_MIN_CREDIT_LOAN (256)#define SLB_PORT_MAX_CREDITS (2 * SLB_MIN_CREDIT_LOAN)#define SLB_PORT_MIN_CREDITS (SLB_MIN_CREDIT_LOAN)
@@ -56,7 +56,8 @@ struct slb_port_q_entry {struct slb_queue *queue;uint8_t queue_id;uint8_t schedule_type; -uint8_t pad[6 ]; +bool is_single_link; +uint8_t pad[5 ]; };#define SLB_PORT_HISTORY_SIZE 256
drivers/event/slb/slb_event.c +18 -5
@@ -976,8 +976,10 @@ slb_port_dequeue(struct slb_port *port, struct rte_event events[],return 0 ; dequeued =rte_event_ring_dequeue_burst (port->single_linked_ring, events, num,NULL ); -for (uint16_t j =0 ; j < dequeued; j++) +for (uint16_t j =0 ; j < dequeued; j++) { + events[j].sched_type =RTE_SCHED_TYPE_ATOMIC ; events[j].impl_opaque =0 ; + }return dequeued; }
@@ -1015,8 +1017,11 @@ slb_port_dequeue(struct slb_port *port, struct rte_event events[],continue ;uint8_t sched_type = qent->schedule_type; -bool is_ordered = (sched_type ==RTE_SCHED_TYPE_ORDERED ); -bool is_atomic = (sched_type ==RTE_SCHED_TYPE_ATOMIC ); +bool is_single_link = qent->is_single_link; +bool is_ordered = !is_single_link && + (sched_type ==RTE_SCHED_TYPE_ORDERED ); +bool is_atomic = !is_single_link && + (sched_type ==RTE_SCHED_TYPE_ATOMIC );if (is_ordered || is_atomic) {if (unlikely (port->history_avail ==0 ))
@@ -1050,8 +1055,16 @@ slb_port_dequeue(struct slb_port *port, struct rte_event events[], }else { dequeued =rte_event_ring_dequeue_burst ( ring_to_dq, events, num,NULL ); -for (uint16_t j =0 ; j < dequeued; j++) - events[j].impl_opaque =0 ; +if (is_single_link) { +for (uint16_t j =0 ; j < dequeued; j++) { + events[j].sched_type = +RTE_SCHED_TYPE_ATOMIC ; + events[j].impl_opaque =0 ; + } + }else { +for (uint16_t j =0 ; j < dequeued; j++) + events[j].impl_opaque =0 ; + } }if (dequeued >0 ) {
drivers/event/slb/slb_selftest.c +12 -0
@@ -362,6 +362,12 @@ test_ordered_forward(void) cons_rx, rx[i].u64);return -1 ; } +if (rx[i].sched_type !=RTE_SCHED_TYPE_ATOMIC ) { +SLB_LOG_LINE (ERR , +"Single-link dequeue sched_type expected %u, got %u" , +RTE_SCHED_TYPE_ATOMIC , rx[i].sched_type); +return -1 ; + } cons_rx++; rx[i].op =RTE_EVENT_OP_RELEASE ; }
@@ -416,6 +422,12 @@ test_single_link(void)for (i =0 ; i <16 ; i++) {if (rx[i].u64 != (uint64_t )(5000 + i))return -1 ; +if (rx[i].sched_type !=RTE_SCHED_TYPE_ATOMIC ) { +SLB_LOG_LINE (ERR , +"Single-link dequeue sched_type expected %u, got %u" , +RTE_SCHED_TYPE_ATOMIC , rx[i].sched_type); +return -1 ; + } rx[i].op =RTE_EVENT_OP_RELEASE ; }rte_event_enqueue_burst (g_dev_id,PORT_CONS , rx,16 );
Signature · 1 line
2.53.0
PATCH 8/8 event/slb: improve port linking, config checks and atomic refill
In reply to Pravin PathakFrom: Pravin Pathak <[email protected]>
- Validate duplicate/conflicting queue-port links and slot capacity at the start of slb_port_link_one(), and fix priority levels to 2.
- Validate nb_event_ports and nb_event_queues limits in slb_configure(), avoid rounding max_inflight to a power of 2, and precompute migration_cooldown_cycles once during configuration.
- Verify all queues are configured and linked to at least one port in slb_start().
- Use relaxed stores when clearing drained reorder buffer slots and avoid duplicate acquire load on the first burst element in slb_reorder_drain().
- Ensure hold buffer capacity before dequeuing in slb_atomic_refill() and continue refilling uncongested ports while hold buffer space remains.
- Enforce 1-to-1 single-link port and single-link queue pairing in slb_port_link_one(), disallowing linking a single-link queue to a regular port or a regular queue to a single-linked port.
- Preserve event sched_type on single-link dequeue and only clear impl_opaque to 0.
--- drivers/event/slb/slb_evdev.c | 200 +++++++++++++-------- drivers/event/slb/slb_evdev.h | 41 +---- drivers/event/slb/slb_event.c | 290 ++++++++++++++++++------------- drivers/event/slb/slb_selftest.c | 12 -- 4 files changed, 306 insertions(+), 237 deletions(-)
drivers/event/slb/slb_evdev.c +123 -77
@@ -84,10 +84,6 @@ slb_port_setup(struct rte_eventdev *dev, uint8_t port_id, .last_migrated_time = 0, }; - port->remap_input_q = false; -rte_atomic_store_explicit (&port->active_queues,0 , - rte_memory_order_relaxed); - dev->data->ports[port_id] = port;/* Reset all queues mapped to this port */
@@ -199,9 +195,7 @@ slb_queue_setup(struct rte_eventdev *dev, uint8_t queue_id,return -ENOMEM ; queue->q_ring = ring; -/* Single-link queue uses atomic schedule type */ - queue->schedule_type = queue->is_single_link ? -RTE_SCHED_TYPE_ATOMIC : conf->schedule_type; + queue->schedule_type = conf->schedule_type; queue->serving_ports_mask =0 ; queue->num_serving_ports =0 ;
@@ -270,16 +264,20 @@ slb_queue_def_conf(struct rte_eventdev *dev __rte_unused,static void slb_port_unlink_queue_from_prio (struct slb_port *port,int prio,int slot) { +/* Shift the queue entries to overwriting the unlinked queue slot */ for (int k = slot; k < port->num_q[prio] -1 ; k++) port->q_entries[prio][k] = port->q_entries[prio][k +1 ]; +/* Decrement mapped queue count and Clear the last slot after shifting */ port->num_q[prio]--;memset (&port->q_entries[prio][port->num_q[prio]],0 ,sizeof (struct slb_port_q_entry )); +/* Adjust the next queue index if it points beyond the new count */ if (port->next_q[prio] >= port->num_q[prio]) port->next_q[prio] =0 ; } +/* Find the slot of a queue in a specific priority level, returning -1 if not found */ static int slb_port_find_queue_slot (struct slb_port *port,int prio,uint8_t qid) {
@@ -290,24 +288,67 @@ slb_port_find_queue_slot(struct slb_port *port, int prio, uint8_t qid)return -1 ; } +/* Link a queue to a port at a specific priority level */ static int slb_port_link_one (struct slb_port *port,struct slb_queue *queue,uint8_t qid,uint8_t app_prio,bool has_prio) { +struct rte_event_ring *dq_ring; +uint16_t slot;int prio; -bool found = false;if (port->is_single_linked) { -if (port->single_linked_q !=NULL && - port->single_linked_q != queue) { +/* Only a single-link queue can be linked to a single-linked port */ +if (!queue->is_single_link) { + rte_errno =EINVAL ; +return -EINVAL ; + } +/* Only one queue can be linked to a single-linked port */ +if (port->single_linked_q !=NULL ) { +if (port->single_linked_q == queue) +return 0 ; + rte_errno =EINVAL ; +return -EINVAL ; + } +/* A single-link queue can only be linked to one port */ +if (queue->serving_ports_mask !=0 ) { rte_errno =EINVAL ;return -EINVAL ; } port->single_linked_q = queue; port->single_linked_ring = queue->q_ring; }else { -if (!queue->is_single_link && - queue->schedule_type ==RTE_SCHED_TYPE_ATOMIC && +/* A single-link queue cannot be linked to a regular port */ +if (queue->is_single_link) { + rte_errno =EINVAL ; +return -EINVAL ; + } + +/* Convert application priority to internal priority level (0 or 1) */ +if (has_prio) + prio = (app_prio <RTE_EVENT_DEV_PRIORITY_NORMAL ) ?0 :1 ; +else + prio =SLB_MAX_PRIORITIES -1 ; + +/* Check if queue is already mapped to this port at requested priority */ +for (int pr =0 ; pr <SLB_MAX_PRIORITIES ; pr++) { +if (slb_port_find_queue_slot (port, pr, qid) >=0 ) { +if (pr == prio) +return 0 ; + rte_errno =EINVAL ; +return -EINVAL ; + } + } + +/* Ensure a free mapping slot is available at the requested priority */ + slot = port->num_q[prio]; +if (slot >=SLB_PORT_MAX_Q_MAP_PER_PRIO ) { + rte_errno =ENOSPC ; +return -ENOSPC ; + } + +/* Allocate per-port ring when linking an atomic queue */ +if (queue->schedule_type ==RTE_SCHED_TYPE_ATOMIC && queue->per_port_rings[port->id] ==NULL ) {char ring_name[RTE_RING_NAMESIZE ];struct rte_event_ring *ring;
@@ -327,53 +368,23 @@ slb_port_link_one(struct slb_port *port, struct slb_queue *queue, queue->per_port_rings[port->id] = ring; } -if (has_prio) - prio = (app_prio <RTE_EVENT_DEV_PRIORITY_NORMAL ) ?0 :1 ; -else - prio =SLB_MAX_PRIORITIES -1 ; - -for (int pr =0 ; pr <SLB_MAX_PRIORITIES ; pr++) { -int s =slb_port_find_queue_slot (port, pr, qid); - -if (s >=0 ) { -if (pr == prio) - found = true; -else -slb_port_unlink_queue_from_prio (port, pr, s); -break ; - } - } -if (!found) { -uint16_t slot = port->num_q[prio]; -struct rte_event_ring *dq_ring = - (!queue->is_single_link && - queue->schedule_type ==RTE_SCHED_TYPE_ATOMIC ) ? - queue->per_port_rings[port->id] : queue->q_ring; - -if (slot >=SLB_PORT_MAX_Q_MAP_PER_PRIO ) { - rte_errno =ENOSPC ; -return -ENOSPC ; - } - port->q_entries[prio][slot].dq_ring = dq_ring; - port->q_entries[prio][slot].queue = queue; - port->q_entries[prio][slot].queue_id = qid; - port->q_entries[prio][slot].schedule_type = - queue->schedule_type; - port->q_entries[prio][slot].is_single_link = - queue->is_single_link; - port->num_q[prio]++; - } + dq_ring = (queue->schedule_type ==RTE_SCHED_TYPE_ATOMIC ) ? + queue->per_port_rings[port->id] : queue->q_ring; -rte_atomic_fetch_or_explicit (&port->active_queues, - (1ULL << qid), - rte_memory_order_release); + port->q_entries[prio][slot].dq_ring = dq_ring; + port->q_entries[prio][slot].queue = queue; + port->q_entries[prio][slot].queue_id = qid; + port->q_entries[prio][slot].schedule_type = queue->schedule_type; + port->num_q[prio]++; } +/* Add newly linked port to the queue's serving ports mask */ queue->serving_ports_mask |= (1ULL << port->id); queue->num_serving_ports =rte_popcount64 (queue->serving_ports_mask);return 0 ; } +/* Unlink a queue from a port */ static int slb_port_unlink_one (struct slb_port *port,struct slb_queue *queue,uint8_t qid) {
@@ -386,7 +397,7 @@ slb_port_unlink_one(struct slb_port *port, struct slb_queue *queue, uint8_t qid) } port->single_linked_q =NULL ; port->single_linked_ring =NULL ; - }else { + }else {/* Unlink queue by finding its slot */ for (int prio =0 ; prio <SLB_MAX_PRIORITIES ; prio++) {int s =slb_port_find_queue_slot (port, prio, qid);
@@ -396,21 +407,17 @@ slb_port_unlink_one(struct slb_port *port, struct slb_queue *queue, uint8_t qid)break ; } } -if (!found) { +if (!found) {/* No link found */ rte_errno =EINVAL ;return -EINVAL ; } +/* For atomic queues, free the per-port ring if it exists */ if (queue->schedule_type ==RTE_SCHED_TYPE_ATOMIC && queue->per_port_rings[port->id] !=NULL ) {rte_event_ring_free (queue->per_port_rings[port->id]); queue->per_port_rings[port->id] =NULL ; } - -rte_atomic_fetch_and_explicit (&port->active_queues, - ~(1ULL << qid), - rte_memory_order_release); - port->remap_input_q = true; } queue->serving_ports_mask &= ~(1ULL << port->id);
@@ -418,6 +425,7 @@ slb_port_unlink_one(struct slb_port *port, struct slb_queue *queue, uint8_t qid)return 0 ; } +/* Link all queues to a port at specific priorities */ static int slb_port_link (struct rte_eventdev *dev,void *p,const uint8_t queues[],const uint8_t priorities[],uint16_t num)
@@ -439,6 +447,7 @@ slb_port_link(struct rte_eventdev *dev, void *p, const uint8_t queues[],return num; } +/* Unlink all queues from a port */ static int slb_port_unlink (struct rte_eventdev *dev,void *p,uint8_t queues[],uint16_t num)
@@ -458,6 +467,7 @@ slb_port_unlink(struct rte_eventdev *dev, void *p, uint8_t queues[],return num; } +/* Get information about the event device */ static void slb_info_get(struct rte_eventdev *dev __rte_unused,struct rte_event_dev_info *info)
@@ -467,7 +477,7 @@ slb_info_get(struct rte_eventdev *dev __rte_unused, .max_event_queues = SLB_MAX_QUEUES, .max_event_queue_flows = SLB_MAX_FLOWS, .max_event_queue_priority_levels = SLB_MAX_PRIORITIES, - .max_event_priority_levels = SLB_MAX_PRIORITIES, + .max_event_priority_levels = 1, .max_event_ports = SLB_MAX_PORTS, .max_event_port_dequeue_depth = SLB_MAX_PORT_DEQUEUE_DEPTH, .max_event_port_enqueue_depth = SLB_MAX_PORT_ENQUEUE_DEPTH,
@@ -487,13 +497,13 @@ slb_info_get(struct rte_eventdev *dev __rte_unused, }; } +/* Configure the event device */ static int slb_configure (const struct rte_eventdev *dev) {struct slb_evdev *slb =slb_pmd_priv (dev);const struct rte_event_dev_config *conf = &dev->data->dev_conf;int32_t min_max_in_flight; -uint32_t q_size;if (conf->nb_events_limit <=0 || conf->nb_events_limit >SLB_MAX_EVENTS ) {
@@ -503,36 +513,60 @@ slb_configure(const struct rte_eventdev *dev)return -EINVAL ; } +/* Validate max event ports and queues that SLB can support */ +if (conf->nb_event_ports ==0 || + conf->nb_event_ports >SLB_MAX_PORTS ) { +SLB_LOG_DP_LINE (INFO , +"nb_event_ports (%u) invalid, max is %d" , + conf->nb_event_ports,SLB_MAX_PORTS ); +return -EINVAL ; + } + +if (conf->nb_event_queues ==0 || + conf->nb_event_queues >SLB_MAX_QUEUES ) { +SLB_LOG_DP_LINE (INFO , +"nb_event_queues (%u) invalid, max is %d" , + conf->nb_event_queues,SLB_MAX_QUEUES ); +return -EINVAL ; + } + slb->num_ports = conf->nb_event_ports; slb->num_queues = conf->nb_event_queues; +/* Compute minimum credits required to avoid deadlock */ min_max_in_flight = conf->nb_event_ports *SLB_PORT_MAX_CREDITS ;if (min_max_in_flight >SLB_MAX_EVENTS ) min_max_in_flight =SLB_MAX_EVENTS ; - q_size =rte_align32pow2 ((uint32_t )RTE_MAX (conf->nb_events_limit, - min_max_in_flight)); -if (q_size >SLB_MAX_EVENTS ) - q_size =SLB_MAX_EVENTS ; - - slb->max_inflight = (int32_t )q_size; +/* Set credit pool size to max of nb_events_limit and minimum required to avoid deadlock */ + slb->max_inflight =RTE_MAX (conf->nb_events_limit, min_max_in_flight); + slb->migration_cooldown_cycles = + (rte_get_timer_hz () *SLB_MIGRATION_COOLDOWN_MS ) /1000 ;return 0 ; } +/* Start the event device */ static int slb_start (struct rte_eventdev *dev) {struct slb_evdev *slb =slb_pmd_priv (dev); +/* Make sure all queues are set up and have at least one linked port */ for (int i =0 ; i < slb->num_queues; i++) { -if (slb->queues[i].serving_ports_mask ==0 ) -SLB_LOG_DP_LINE (INFO , -"Queue %d has no linked ports on start" , i); +if (slb->queues[i].q_ring ==NULL || + slb->queues[i].serving_ports_mask ==0 ) { +SLB_LOG_DP_LINE (ERR , +"Queue %d is not configured or has no linked ports" , + i); +return -ENOLINK ; + } } +/* No events in flight */ rte_atomic_store_explicit (&slb->credits_on_loan,0 , rte_memory_order_relaxed); +/* Reset port states */ for (int i =0 ; i < slb->num_ports; i++) {struct slb_port *port = &slb->ports[i]; port->inflight_credits =0 ;
@@ -544,6 +578,7 @@ slb_start(struct rte_eventdev *dev)return 0 ; } +/* Empty the event queue */ static void slb_drain_ring (uint8_t dev_id,struct rte_event_ring *ring,eventdev_stop_flush_t flush,void *flush_arg)
@@ -553,6 +588,7 @@ slb_drain_ring(uint8_t dev_id, struct rte_event_ring *ring,if (ring ==NULL )return ; +/* Dequeue all events from the ring and call user callback if provided */ while (rte_event_ring_dequeue_burst (ring, &ev,1 ,NULL )) {if (flush !=NULL )flush (dev_id, ev, flush_arg);
@@ -560,6 +596,7 @@ slb_drain_ring(uint8_t dev_id, struct rte_event_ring *ring,rte_ring_reset (&ring->r); } +/* Stop the event device */ static void slb_stop (struct rte_eventdev *dev) {
@@ -575,6 +612,7 @@ slb_stop(struct rte_eventdev *dev) q->atomic_state != NULL) {struct slb_atomic_state *astate = q->atomic_state; +/* Call user callback for each held event */ for (uint16_t h =0 ; h < astate->hold_count; h++) {if (flush !=NULL ) flush(dev_id, astate->hold_buffer[h],
@@ -584,6 +622,7 @@ slb_stop(struct rte_eventdev *dev) astate->next_port_rr =0 ; astate->migration.in_progress = false; +/* Reset flow to port mapping */ for (int f =0 ; f <SLB_ATOMIC_FLOWS ; f++) { rte_atomic_store_explicit( &astate->flow_map.flows[f].port_id,
@@ -594,6 +633,7 @@ slb_stop(struct rte_eventdev *dev)memset (astate->port_stats,0 ,sizeof (astate->port_stats)); +/* Drain all events from per-port rings */ for (uint16_t pid =0 ; pid < slb->num_ports; pid++) {if (q->per_port_rings[pid] !=NULL ) slb_drain_ring(dev_id,
@@ -606,11 +646,12 @@ slb_stop(struct rte_eventdev *dev)uint8_t st =rte_atomic_load_explicit ( &q->reorder_buffer[s].state, rte_memory_order_relaxed); - +/* Flush all ready events from the reorder buffer. */ if (st ==SLB_REORDER_READY && flush !=NULL )flush (dev_id, q->reorder_buffer[s].event, flush_arg); +/* Mark freed slot as empty */ rte_atomic_store_explicit( &q->reorder_buffer[s].state, SLB_REORDER_EMPTY,
@@ -623,6 +664,7 @@ slb_stop(struct rte_eventdev *dev)slb_drain_ring (dev_id, q->q_ring, flush, flush_arg); } +/* Reset each port's state, history lists and queue mappings */ for (int i =0 ; i < slb->num_ports; i++) {struct slb_port *port = &slb->ports[i]; port->inflight_credits =0 ;
@@ -635,21 +677,25 @@ slb_stop(struct rte_eventdev *dev) port->next_q[p] =0 ; } +/* Reset the credit pool to all available */ rte_atomic_store_explicit (&slb->credits_on_loan,0 , rte_memory_order_relaxed); } +/* Close the event device */ static int slb_close (struct rte_eventdev *dev) {struct slb_evdev *slb =slb_pmd_priv (dev); +/* Release all resources acquired by the queues */ for (int i =0 ; i < slb->num_queues; i++)slb_queue_release (dev, i);return 0 ; } +/* Dump the event device current state and statistics */ void slb_dump (struct rte_eventdev *dev,FILE *f) {
@@ -689,14 +735,12 @@ slb_dump(struct rte_eventdev *dev, FILE *f)fprintf (f,"\n PORT STATISTICS & QUEUE LINKAGES:\n" );for (i =0 ; i < slb->num_ports; i++) {struct slb_port *p = &slb->ports[i]; -uint64_t aq =rte_atomic_load_explicit (&p->active_queues, - rte_memory_order_acquire);int linked_count =0 ;fprintf (f, -" Port %2u (credits=%3d, active=0x%08" PRIx64 -", loaded=%d, mig_in=%" PRIu64", mig_out=%" PRIu64"): " , - i, p->inflight_credits, aq, p->is_loaded, +" Port %2u (credits=%3d, loaded=%d" +", mig_in=%" PRIu64", mig_out=%" PRIu64"): " , + i, p->inflight_credits, p->is_loaded, p->migrations_in, p->migrations_out);if (p->is_single_linked) {
@@ -743,7 +787,8 @@ slb_dump(struct rte_eventdev *dev, FILE *f) i, count, ring_size, q->serving_ports_mask, q->num_serving_ports, q->schedule_type, q->is_single_link); -if (q->schedule_type ==RTE_SCHED_TYPE_ATOMIC ) { +if (!q->is_single_link && + q->schedule_type ==RTE_SCHED_TYPE_ATOMIC ) {if (q->atomic_state) {fprintf (f, " Atomic State: total_migrations=%" PRIu64
@@ -774,7 +819,8 @@ slb_dump(struct rte_eventdev *dev, FILE *f) } } } - }else if (q->schedule_type ==RTE_SCHED_TYPE_ORDERED ) { + }else if (!q->is_single_link && + q->schedule_type ==RTE_SCHED_TYPE_ORDERED ) {fprintf (f," Ordered reorder_drain_idx=%u\n" , q->reorder_drain_idx); }
drivers/event/slb/slb_evdev.h +9 -32
91 lines not on this page · raw message
drivers/event/slb/slb_event.c +174 -116
937 lines not on this page · raw message
drivers/event/slb/slb_selftest.c +0 -12
@@ -362,12 +362,6 @@ test_ordered_forward(void) cons_rx, rx[i].u64);return -1 ; } -if (rx[i].sched_type !=RTE_SCHED_TYPE_ATOMIC ) { -SLB_LOG_LINE (ERR , -"Single-link dequeue sched_type expected %u, got %u" , -RTE_SCHED_TYPE_ATOMIC , rx[i].sched_type); -return -1 ; - } cons_rx++; rx[i].op =RTE_EVENT_OP_RELEASE ; }
@@ -422,12 +416,6 @@ test_single_link(void)for (i =0 ; i <16 ; i++) {if (rx[i].u64 != (uint64_t )(5000 + i))return -1 ; -if (rx[i].sched_type !=RTE_SCHED_TYPE_ATOMIC ) { -SLB_LOG_LINE (ERR , -"Single-link dequeue sched_type expected %u, got %u" , -RTE_SCHED_TYPE_ATOMIC , rx[i].sched_type); -return -1 ; - } rx[i].op =RTE_EVENT_OP_RELEASE ; }rte_event_enqueue_burst (g_dev_id,PORT_CONS , rx,16 );
Signature · 1 line
2.53.0
Quoted 19 lines · Pravin Pathak
From: Pravin Pathak <[email protected]>
This series introduces the Software Load Balancer (SLB) Eventdev PMD (
event_slb) supporting parallel, atomic and ordered queues. It is a distributed software scheduler that scales well.Motivation
----------
Existing software eventdev PMDs face trade-offs between centralized bottlenecks and feature coverage:
-event_swrelies on a dedicated centralized scheduler consuming an extra CPU core. The single scheduling core also limits maximum throughput and it does not scale.
-event_dswdistributes scheduling across worker cores without a central scheduler core, but does not support ordered queues and incurs per-port control ring messaging overhead during atomic flow migrations. It also does not support service priority.
Quoted 5 lines
Design Overview
---------------event_slbis a distributed (RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED), maintenance-free (RTE_EVENT_DEV_CAP_MAINTENANCE_FREE) software eventdev PMD that supports Atomic, Ordered, Parallel, and Single-Link queues:
@[email protected] - drivers/event/dsw/ @[email protected] - drivers/event/opdl/ NO MAINTAINER for drivers/event/sw/
Hi Pravin,
Creating new flavours. of evendev SW drivers is a maintenance burden, If this is close to DSW then please introduce mode in DSW driver and have function pointer switches for fastpath,
Also claim should be backed by performance test values. Use testeventdev test cases and share the numbers to prove one way or another.
In summary, IMO, it better to take NEW SW PMD when
a)It operates completely different than DSW and SW and it is NOT possible to update existing SW PMDs
b)Able to show better performance than existing drivers on the specific cases.
Quoted 78 lines
1. Queue-Centric Ring Architecture:
- Parallel and Single-Link queues map directly torte_event_ringinstances (MP/MC for parallel, MP/SC for single-link), allowing zero-copy burst enqueue and dequeue without history tracking.
- Ordered queues use the ring slot index at dequeue time as an implicit sequence number into a per-queue reorder buffer, draining contiguous completed events in order without global sequence counters. This makes ordered queue implementaion highly efficient performing close to parallel queues.
- Atomic queues use a main MP/SC queue ring alongside per-port SP/SC rings and a 2048-entry flow-to-port affinity table. All cores served by the atomic queue are used for scheduling.2. Lock-Free / Zero-Pause Atomic Flow Migration:
- When an atomic queue refills per-port rings and detects persistent congestion on a destination port, SLB selects the lightest flow on that port (outside the migration cooldown window) and waits for queues in-flight events to drain naturally before switching the flow's port affinity and schedule hold-buffer entries.
- Hold buffer is used for buffering events towards migrating ports while allowing serving other ports.
- Implementaion avoids cross-port pause/unpause control rings while preserving strict per-flow ordering.3. Credit-Based Backpressure & Priority Scheduling:
- A global atomic credit pool with per-port local credit caches bounds total in-flight events (nb_events_limit) and enforcesnew_event_thresholdwith minimal atomic contention.
- Ports support two strict service priority levels with round-robin scheduling within each priority level.4. Observability & Testing:
- Device and per-port/per-queue extended statistics (xstats), detailedrte_event_dev_dump()diagnostics, and a unit self-test suite (eventdev_selftest_slb) covering parallel, atomic, ordered, single-link, priority, backpressure, stop-flush, and multi-core atomic migration scenarios.5. Future Plan:
- Supporting Event priority
- Optimizing atomic flow migrationPravin Pathak (8):
event/slb: introduce software load balancer PMD event/slb: add support for ordered queues
event/slb: add support for atomic queues
event/slb: add dynamic flow migration for atomic queues event/slb: add extended statistics and diagnostics event/slb: add self-test and documentation
event/slb: update resets and credit defaults event/slb: improve port linking, config checks and atomic refillMAINTAINERS | 6 + app/test/test_eventdev.c | 7 + doc/guides/eventdevs/features/slb.ini | 25 + doc/guides/eventdevs/index.rst | 1 + doc/guides/eventdevs/slb.rst | 61 ++ drivers/event/meson.build | 1 + drivers/event/slb/meson.build | 21 + drivers/event/slb/slb_evdev.c | 904 ++++++++++++++++++ drivers/event/slb/slb_evdev.h | 263 ++++++ drivers/event/slb/slb_event.c | 1219 +++++++++++++++++++++++++ drivers/event/slb/slb_selftest.c | 522 +++++++++++ drivers/event/slb/slb_xstats.c | 347 +++++++ 12 files changed, 3377 insertions(+) create mode 100644 doc/guides/eventdevs/features/slb.ini create mode 100644 doc/guides/eventdevs/slb.rst create mode 100644 drivers/event/slb/meson.build create mode 100644 drivers/event/slb/slb_evdev.c create mode 100644 drivers/event/slb/slb_evdev.h create mode 100644 drivers/event/slb/slb_event.c create mode 100644 drivers/event/slb/slb_selftest.c create mode 100644 drivers/event/slb/slb_xstats.c--
2.53.0
Quoted 10 lines · On Sat, 7 Nov 2026 12:59:18 +0530 Jerin Jacob <[email protected]> wrote:
Quoted 5 lines
Design Overview
---------------event_slbis a distributed (RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED), maintenance-free (RTE_EVENT_DEV_CAP_MAINTENANCE_FREE) software eventdev PMD that supports Atomic, Ordered, Parallel, and Single-Link queues:@[email protected] -
drivers/event/dsw/@[email protected] -drivers/event/opdl/NO MAINTAINER fordrivers/event/sw/
Quoted 7 lines
Hi Pravin,
Creating new flavours. of evendev SW drivers is a maintenance burden, If this is close to DSW then please introduce mode in DSW driver and have function pointer switches for fastpath,
Also claim should be backed by performance test values. Use testeventdev test cases and share the numbers to prove one way or another.
I ran AI review with Fable and the current AGENTS guidelines. Short answer: it agrees with Jerin
Review: [PATCH v1 0/8] event/slb: introducing software load balancer eventdev PMD
Applied cleanly on 26.11-rc2 (a13c105). Builds per-commit with -Dwerror=true; bisect is clean. Line numbers below are in the post-apply tree.
Cover / series-level
--------------------
Jerin's question (new PMD vs. a DSW mode) is the right one, and the code supports it. The credit pool (slb_port_acquire_credits, slb_port_return_credits, credits_on_loan, the max_inflight formula in slb_configure, the RTE_VERIFY on pending_releases), the log macros, and all of slb_xstats.c are DSW code with the prefix changed; the comment at slb_evdev.h:30 is verbatim dsw_evdev.h:62. What is new is the queue-side scheduling: atomic queues use a main MP/SC ring plus per-port SP/SC rings filled by whichever port finds its ring empty, under a per-queue spinlock; ordered queues use the ring slot index as the sequence number into a per-queue reorder buffer, drained under a second per-queue spinlock. Both are serialised per queue, so "distributed" means "whichever core takes the lock", and the atomic refill does an O(SLB_ATOMIC_FLOWS)+O(ring) scan on every migration trigger. The scaling claim needs testeventdev numbers against sw and dsw, as Jerin asked.
Patches 7 and 8 are fix-ups to patches 1-6 (patch 8 also deletes code added by patch 6). Squash them; the bullet-list commit messages describe the diff, not the change.
The cover letter says the self-test covers priority, backpressure, and multi-core atomic migration. slb_selftest.c is single-threaded, 4 ports, no priorities, no backpressure, and never triggers a migration. None of the concurrency paths below are exercised.
No release notes. Patch 6 says it adds a 26.11 entry and patch 7 says it edits it; neither touches doc/guides/rel_notes/. A new driver after rc2 is 27.03 material anyway.
Patch 1: introduce software load balancer PMD
---------------------------------------------
Error:
- slb_evdev.c:781, slb_dump(): NULL dereference.
uint32_t count = rte_event_ring_count(q->q_ring);
The loop runs i < slb->num_queues, which is set by configure, but q_ring is NULL for any queue not yet set up. rte_event_dev_dump() after configure and before queue_setup segfaults. Guard with if (q->q_ring == NULL) continue; (same loop at :782).
- slb_evdev.c:197, slb_queue_setup(): double free. q_ring is assigned before the atomic/ordered allocations:
queue->q_ring = ring;
and the failure paths at :212 and :239 do
rte_event_ring_free(ring);
return -ENOMEM; leaving queue->q_ring dangling. The next slb_queue_release() (close, or re-setup) frees it again. Set queue->q_ring = NULL on those paths, or assign q_ring only after all allocations succeed.
Warning:
- slb_evdev.c:167, slb_queue_setup() on an already-linked queue frees q_ring and any per_port_rings via slb_queue_release(), but the linked ports' q_entries[][].dq_ring (set at :374) and single_linked_ring (:319) still point at the freed rings. The API allows queue_setup on a stopped device without unlinking first, so the next dequeue is a use-after-free. Either return -EBUSY when serving_ports_mask != 0, or re-resolve the entries.
- slb_event.c:661, new_event_threshold is compared against credits_on_loan, which includes every port's idle cache:
if (unlikely(rte_atomic_load_explicit(&slb->credits_on_loan,
rte_memory_order_relaxed) >
source_port->new_event_threshold))
return 0; This is the DSW check, but patch 7 raises SLB_MIN_CREDIT_LOAN to 256 (DSW: 64), so a port that only dequeues and releases holds 256..511 credits forever (slb_port_return_credits keeps SLB_PORT_MIN_CREDITS). With K terminating ports the floor on credits_on_loan is 256*K regardless of events in flight. The default threshold from slb_port_def_conf is min(4096, nb_events_limit): nb_events_limit=1024 with 4 consumer ports, or the default with 16 workers, rejects every NEW enqueue once the caches fill, and nothing ever lowers them. Either go back to the DSW quantum, or let the port use its own cache before applying the global check (acquire first, check the pool only when borrowing).
Info:
- slb_configure() logs its -EINVAL causes with SLB_LOG_DP_LINE(INFO, ...). Control path; use SLB_LOG_LINE(ERR, ...).
- drivers/event/meson.build:18: 'slb' added after 'sw'; list is alphabetical.
Patch 2: add support for ordered queues
---------------------------------------
Error:
- slb_event.c:993 / :1021, reorder slot reuse. The sequence number is the ring slot:
*start_slot = cons_head & r->r.mask;
h->seq_slot = (start_slot + j) & ring_mask; but the ring slot is freed at dequeue (__rte_ring_update_tail, :998) while the reorder slot is freed only when slb_reorder_drain passes it. The two are only in step if the producer can never advance r.size positions while a slot is outstanding. Credits do not guarantee that: RELEASE returns the credit immediately (slb_event.c:673, and :1139 for implicit release) even though the slot stays RELEASED until drained. So: worker A dequeues slot S and is slow; drain stops at S; other workers dequeue and release (or implicit-release) r.size more events, all returning credits; the producer wraps to slot S; worker B dequeues the new event with seq_slot S. B's forward writes reorder_buffer[S] and drain delivers it past A's event; when A then forwards, its event lands in a slot drain has already passed and is delivered r.size events late, and if A releases while B's event is still READY the state is overwritten and B's event is lost with its credit. For max_inflight=4096 r.size is 8192, i.e. one stalled worker plus a few ms of traffic. Fix: hold the credit for an ordered event until its slot is drained (return it in slb_reorder_drain when a RELEASED slot is reclaimed, and skip ordered releases in the :673/:1139 returns). Then occupied slots + ring entries <= max_inflight < r.size and the ring cannot wrap onto a live slot.
- slb_event.c:114, lost wakeup in slb_reorder_drain():
if (!rte_spinlock_trylock(&queue->reorder_lock))
return;
The holder exits when it reads EMPTY at drain_idx (:125). A second
port that marks that very slot READY and then fails the trylock
returns too. The slot stays READY until some later forward or
release on the same ordered queue happens to call drain; nothing
in the dequeue path does. At the end of a burst the last events
are stranded in the reorder buffer. Re-check after unlock:
rte_spinlock_unlock(&queue->reorder_lock);
rte_atomic_thread_fence(rte_memory_order_seq_cst);
if (rte_atomic_load_explicit(&buf[drain_idx].state,
rte_memory_order_acquire) !=
SLB_REORDER_EMPTY)
goto retry; /* back to the trylock */Warning:
- slb_event.c:987-998, slb_ordered_ring_dequeue_burst() calls __rte_ring_move_cons_head(), __rte_ring_dequeue_elems() and __rte_ring_update_tail() directly. No other driver uses the ring private API; it is not a stable contract. If the slot index is the design, add an exported helper to lib/ring (or rte_event_ring) that returns the consumer head, in its own patch. Same for slb_event.c:198, which assumes the data array follows struct rte_ring at (r + 1).
Patch 3: add support for atomic queues
--------------------------------------
Warning:
- slb_event.c:826, :845, :1172 write q_returned from the consuming port with a plain increment:
orig_q->atomic_state->port_stats[ep].q_returned++;
and slb_event.c:406 (patch 4) reads it from whichever core holds the refill lock, with a plain load, to decide the flow mapping can be switched. Single writer, but cross-core and load-bearing (the enq==ret test is what makes migration order-preserving). Make q_returned RTE_ATOMIC(uint64_t), store with rte_memory_order_release, load with rte_memory_order_acquire so the application's last access to the flow on the old port is ordered before the new port can see events for it.
Info:
- slb_event.c:166:
return (flow_id * 2654435761U) & SLB_ATOMIC_FLOWS_MASK;
The multiplier is odd, so the low 11 bits of the product are a permutation of the low 11 bits of flow_id; flow_id and flow_id+2048 always collide. Fibonacci hashing takes the high
bits: (flow_id * 2654435761U) >> (32 - 11).
- slb_history_entry.flow_id is written at slb_event.c:1024 and never read.
Patch 4: add dynamic flow migration for atomic queues
-----------------------------------------------------
Warning:
- struct slb_port fields is_loaded (slb_evdev.h:146), last_migrated_time, flow_migrations, migrations_in, migrations_out (:164-167) are per port but written under the refill_lock of whichever atomic queue is migrating (slb_event.c:269, :327-328, :337-339, :343, :420) and read by every other atomic queue's refill (:249, :290, :364). Two atomic queues sharing ports race on them: lost increments, and both queues can pick the same port as migration source. Either make them per (queue, port) inside slb_atomic_state (port_stats is already there), or make them atomic. is_loaded also sits in the port's hot cacheline next to inflight_credits and history_head, so a remote refill write bounces that line under the owning core.
Info:
- The hold buffer is per queue, not per port. Once one linked port stops draining, SLB_HOLD_BUFFER_SIZE - 3*128 held events later slb_atomic_refill returns at :501 without touching the main ring and every other port on that queue starves. Worth stating in the Limitations section of slb.rst.
Patch 5: add extended statistics and diagnostics
------------------------------------------------
Warning:
- slb_xstats.c:105 exposes port_%u_queue_%u_dequeued from slb_port.queue_dequeued[], which nothing increments. Either count it in slb_port_dequeue() or drop the stat.
Info:
- Leftovers: slb_port.total_dequeues, enqueue_calls (printed by slb_dump, never incremented), credits_drained (only ever cleared), flow_migrations (incremented, never read); SLB_MAX_FLOWS_BITS and SLB_MAX_FLOWS_MASK unused.
Patch 6: add self-test and documentation
----------------------------------------
Warning:
- doc/guides/eventdevs/features/slb.ini:14 has runtime_port_link = Y but slb_info_get() does not set RTE_EVENT_DEV_CAP_RUNTIME_PORT_LINK. Patch 7 says it removed this from the feature matrix; it removed it from the .rst only.
- slb.rst (as of patch 7) says "link up to eight queues per port at each priority"; SLB_PORT_MAX_Q_MAP_PER_PRIO is SLB_MAX_QUEUES (64).
- Self-test does not use TEST_ASSERT / unit_test_suite_runner and never runs more than one lcore. At minimum add a multi-core ordered test with one deliberately slow worker and an atomic test that forces a migration (SLB_MIGRATION_LOAD_THRESHOLD is 16 full per-port rings); both of the Errors above would show up.
Quoted 39 lines · Jerin Jacob
Quoted 19 lines · Pravin Pathak
From: Pravin Pathak <[email protected]>
This series introduces the Software Load Balancer (SLB) Eventdev PMD (
event_slb) supporting parallel, atomic and ordered queues. It is a distributed software scheduler that scales well.Motivation
----------
Existing software eventdev PMDs face trade-offs between centralized bottlenecks and feature coverage:
-event_swrelies on a dedicated centralized scheduler consuming an extra CPU core. The single scheduling core also limits maximum throughput and it does not scale.
-event_dswdistributes scheduling across worker cores without a central scheduler core, but does not support ordered queues and incurs per-port control ring messaging overhead during atomic flow migrations. It also does not support service priority.Quoted 5 lines
Design Overview
---------------event_slbis a distributed (RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED), maintenance-free (RTE_EVENT_DEV_CAP_MAINTENANCE_FREE) software eventdev PMD that supports Atomic, Ordered, Parallel, and Single-Link queues:@[email protected] -
drivers/event/dsw/@[email protected] -drivers/event/opdl/NO MAINTAINER fordrivers/event/sw/Hi Pravin,
Creating new flavours. of evendev SW drivers is a maintenance burden, If this is close to DSW then please introduce mode in DSW driver and have function pointer switches for fastpath,
Also claim should be backed by performance test values. Use testeventdev test cases and share the numbers to prove one way or another.
Quoted 5 lines
In summary, IMO, it better to take NEW SW PMD when
a)It operates completely different than DSW and SW and it is NOT possible to update existing SW PMDs
b)Able to show better performance than existing drivers on the specific cases.
This driver is architecturally completely different from DSW. And it addresses the gap of scalable software eventdev that can support ordered queues and service priority. The skeleton is borrowed from DSW instead of writing a new one.
But the underlying architecture is completely different. SLB uses queues instead of
directly enquing to destination ports. DSW is an enqueue time scheduler whereas SLB is a dequeue time scheduler. Very basic mechanism is different. SLB does atomic flow migration is based on the port queue associated with the atomic queue being full. It does not compute CPU load but checks if core is able to serve the queue at mapped priority and if not, flows are migrated. SLB is maintenance free unlike DSW. Flow migration happens without any core-to-core communication required. SLB architecture allows extension to support event priorities and configurable number of service priorities. I will make patches with AI review fixes. AI comparison is matching the structure and not underlying architecture. So it shows similarity.
Quoted 79 lines
Quoted 79 lines
1. Queue-Centric Ring Architecture:
- Parallel and Single-Link queues map directly torte_event_ringinstances (MP/MC for parallel, MP/SC for single-link), allowing zero-copy burst enqueue and dequeue without history tracking.
- Ordered queues use the ring slot index at dequeue time as an implicit sequence number into a per-queue reorder buffer, draining contiguous completed events in order without global sequence counters. This makes ordered queue implementaion highly efficient performing close to parallel queues.
- Atomic queues use a main MP/SC queue ring alongside per-port SP/SC rings and a 2048-entry flow-to-port affinity table. All cores served by the atomic queue are used for scheduling.2. Lock-Free / Zero-Pause Atomic Flow Migration:
- When an atomic queue refills per-port rings and detects persistent congestion on a destination port, SLB selects the lightest flow on that port (outside the migration cooldown window) and waits for queues in-flight events to drain naturally before switching the flow's port affinity and schedule hold-buffer entries.
- Hold buffer is used for buffering events towards migrating ports while allowing serving other ports.
- Implementaion avoids cross-port pause/unpause control rings while preserving strict per-flow ordering.3. Credit-Based Backpressure & Priority Scheduling:
- A global atomic credit pool with per-port local credit caches bounds total in-flight events (nb_events_limit) and enforcesnew_event_thresholdwith minimal atomic contention.
- Ports support two strict service priority levels with round-robin scheduling within each priority level.4. Observability & Testing:
- Device and per-port/per-queue extended statistics (xstats), detailedrte_event_dev_dump()diagnostics, and a unit self-test suite (eventdev_selftest_slb) covering parallel, atomic, ordered, single-link, priority, backpressure, stop-flush, and multi-core atomic migration scenarios.5. Future Plan:
- Supporting Event priority
- Optimizing atomic flow migrationPravin Pathak (8):
event/slb: introduce software load balancer PMD event/slb: add support for ordered queues
event/slb: add support for atomic queues
event/slb: add dynamic flow migration for atomic queues event/slb: add extended statistics and diagnostics event/slb: add self-test and documentation
event/slb: update resets and credit defaults event/slb: improve port linking, config checks and atomic refillMAINTAINERS | 6 + app/test/test_eventdev.c | 7 + doc/guides/eventdevs/features/slb.ini | 25 + doc/guides/eventdevs/index.rst | 1 + doc/guides/eventdevs/slb.rst | 61 ++ drivers/event/meson.build | 1 + drivers/event/slb/meson.build | 21 + drivers/event/slb/slb_evdev.c | 904 ++++++++++++++++++ drivers/event/slb/slb_evdev.h | 263 ++++++ drivers/event/slb/slb_event.c | 1219 +++++++++++++++++++++++++ drivers/event/slb/slb_selftest.c | 522 +++++++++++ drivers/event/slb/slb_xstats.c | 347 +++++++ 12 files changed, 3377 insertions(+) create mode 100644 doc/guides/eventdevs/features/slb.ini create mode 100644 doc/guides/eventdevs/slb.rst create mode 100644 drivers/event/slb/meson.build create mode 100644 drivers/event/slb/slb_evdev.c create mode 100644 drivers/event/slb/slb_evdev.h create mode 100644 drivers/event/slb/slb_event.c create mode 100644 drivers/event/slb/slb_selftest.c create mode 100644 drivers/event/slb/slb_xstats.c--
2.53.0
Quoted 46 lines · Pravin Pathak
Quoted 39 lines · Jerin Jacob
Quoted 19 lines · Pravin Pathak
From: Pravin Pathak <[email protected]>
This series introduces the Software Load Balancer (SLB) Eventdev PMD (
event_slb) supporting parallel, atomic and ordered queues. It is a distributed software scheduler that scales well.Motivation
----------
Existing software eventdev PMDs face trade-offs between centralized bottlenecks and feature coverage:
-event_swrelies on a dedicated centralized scheduler consuming an extra CPU core. The single scheduling core also limits maximum throughput and it does not scale.
-event_dswdistributes scheduling across worker cores without a central scheduler core, but does not support ordered queues and incurs per-port control ring messaging overhead during atomic flow migrations. It also does not support service priority.Quoted 5 lines
Design Overview
---------------event_slbis a distributed (RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED), maintenance-free (RTE_EVENT_DEV_CAP_MAINTENANCE_FREE) software eventdev PMD that supports Atomic, Ordered, Parallel, and Single-Link queues:@[email protected] -
drivers/event/dsw/@[email protected] -drivers/event/opdl/NO MAINTAINER fordrivers/event/sw/Hi Pravin,
Creating new flavours. of evendev SW drivers is a maintenance burden, If this is close to DSW then please introduce mode in DSW driver and have function pointer switches for fastpath,
Also claim should be backed by performance test values. Use testeventdev test cases and share the numbers to prove one way or another.Quoted 5 lines
In summary, IMO, it better to take NEW SW PMD when
a)It operates completely different than DSW and SW and it is NOT possible to update existing SW PMDs
b)Able to show better performance than existing drivers on the specific cases.
Quoted 16 lines
This driver is architecturally completely different from DSW. And it addresses the gap of scalable software eventdev that can support ordered queues and service priority. The skeleton is borrowed from DSW instead of writing a new one.
But the underlying architecture is completely different. SLB uses queues instead of
directly enquing to destination ports. DSW is an enqueue time scheduler whereas SLB is a dequeue time scheduler. Very basic mechanism is different. SLB does atomic flow migration is based on the port queue associated with the atomic queue being full. It does not compute CPU load but checks if core is able to serve the queue at mapped priority and if not, flows are migrated. SLB is maintenance free unlike DSW. Flow migration happens without any core-to-core communication required. SLB architecture allows extension to support event priorities and configurable number of service priorities. I will make patches with AI review fixes. AI comparison is matching the structure and not underlying architecture. So it shows similarity.
Lets get the performance numbers for SLB vs DSW based on same test case so that we can decide.
Performance and scalability will be the aspect to decide the usage of YET ANOTHER SW driver.
Quoted 79 lines
Quoted 79 lines
Quoted 79 lines
1. Queue-Centric Ring Architecture:
- Parallel and Single-Link queues map directly torte_event_ringinstances (MP/MC for parallel, MP/SC for single-link), allowing zero-copy burst enqueue and dequeue without history tracking.
- Ordered queues use the ring slot index at dequeue time as an implicit sequence number into a per-queue reorder buffer, draining contiguous completed events in order without global sequence counters. This makes ordered queue implementaion highly efficient performing close to parallel queues.
- Atomic queues use a main MP/SC queue ring alongside per-port SP/SC rings and a 2048-entry flow-to-port affinity table. All cores served by the atomic queue are used for scheduling.2. Lock-Free / Zero-Pause Atomic Flow Migration:
- When an atomic queue refills per-port rings and detects persistent congestion on a destination port, SLB selects the lightest flow on that port (outside the migration cooldown window) and waits for queues in-flight events to drain naturally before switching the flow's port affinity and schedule hold-buffer entries.
- Hold buffer is used for buffering events towards migrating ports while allowing serving other ports.
- Implementaion avoids cross-port pause/unpause control rings while preserving strict per-flow ordering.3. Credit-Based Backpressure & Priority Scheduling:
- A global atomic credit pool with per-port local credit caches bounds total in-flight events (nb_events_limit) and enforcesnew_event_thresholdwith minimal atomic contention.
- Ports support two strict service priority levels with round-robin scheduling within each priority level.4. Observability & Testing:
- Device and per-port/per-queue extended statistics (xstats), detailedrte_event_dev_dump()diagnostics, and a unit self-test suite (eventdev_selftest_slb) covering parallel, atomic, ordered, single-link, priority, backpressure, stop-flush, and multi-core atomic migration scenarios.5. Future Plan:
- Supporting Event priority
- Optimizing atomic flow migrationPravin Pathak (8):
event/slb: introduce software load balancer PMD event/slb: add support for ordered queues
event/slb: add support for atomic queues
event/slb: add dynamic flow migration for atomic queues event/slb: add extended statistics and diagnostics event/slb: add self-test and documentation
event/slb: update resets and credit defaults event/slb: improve port linking, config checks and atomic refillMAINTAINERS | 6 + app/test/test_eventdev.c | 7 + doc/guides/eventdevs/features/slb.ini | 25 + doc/guides/eventdevs/index.rst | 1 + doc/guides/eventdevs/slb.rst | 61 ++ drivers/event/meson.build | 1 + drivers/event/slb/meson.build | 21 + drivers/event/slb/slb_evdev.c | 904 ++++++++++++++++++ drivers/event/slb/slb_evdev.h | 263 ++++++ drivers/event/slb/slb_event.c | 1219 +++++++++++++++++++++++++ drivers/event/slb/slb_selftest.c | 522 +++++++++++ drivers/event/slb/slb_xstats.c | 347 +++++++ 12 files changed, 3377 insertions(+) create mode 100644 doc/guides/eventdevs/features/slb.ini create mode 100644 doc/guides/eventdevs/slb.rst create mode 100644 drivers/event/slb/meson.build create mode 100644 drivers/event/slb/slb_evdev.c create mode 100644 drivers/event/slb/slb_evdev.h create mode 100644 drivers/event/slb/slb_event.c create mode 100644 drivers/event/slb/slb_selftest.c create mode 100644 drivers/event/slb/slb_xstats.c--
2.53.0
Quoted 4 lines · Jerin Jacob
Lets get the performance numbers for SLB vs DSW based on same test case so that we can decide.
Performance and scalability will be the aspect to decide the usage of YET ANOTHER SW driver.
Hi Pravin,
A few points on top of Stephen's review, plus some early numbers since Jerin asked for SLB vs DSW data.
+1 to Stephen's two Errors in patch 2 (reorder slot reuse when RELEASE returns credits before the slot drains, and the lost wakeup in slb_reorder_drain()) and to the credit-threshold issue in patch 1. Those look like the ones that can lose or strand events.
Additional comments:
1. slb_event.c, slb_event_enqueue_burst_generic(): the return value of rte_event_ring_enqueue_burst() is ignored on every path and the function always returns events_len. Correctness then depends entirely on credits guaranteeing ring space. If that ever fails (e.g. the credit-threshold or reorder issues above, or application misuse), events are silently dropped and their credits leak. At minimum, check the return value and report a short enqueue. Because the burst is sorted by destination queue before enqueue, a clean prefix count isn't possible after a partial failure, so this probably needs a space check up front or an explicit design statement that rings can never fill.
2. RTE_VERIFY(num_forward + num_release <= cur_pending) in the enqueue fast path aborts the whole process on application misuse (forwarding more events than were dequeued on that port). Please fail the enqueue and set rte_errno instead.
3. slb_configure(): max_inflight = RTE_MAX(nb_events_limit, nb_ports * SLB_PORT_MAX_CREDITS), so the device can hold more events than the application configured via nb_events_limit. If that is intended (to avoid credit-cache deadlock), please document it in slb.rst.
4. slb_queue_setup() ignores queue_conf->priority; priority comes only from the link priority, yet dev_info reports max_event_queue_priority_levels = 2. Either honour queue priority or report 1 and document that priority is per link.
Performance
-----------
Preliminary data from dpdk-test-eventdev perf_queue, series applied on v26.11-rc2, release build, Xeon 6505P (2x12C), physical cores only, turbo on, default device settings, worker_deq_depth 16, 1024 flows. Mpps, 1 synthetic producer, workers on socket 0:
stlist driver 1w 2w 4w 8w
p slb 79.7 71.5 44.5 34.0 p dsw 50.1 46.3 45.1 41.9 a slb 54.3 65.3 57.0 43.9 a dsw 50.3 47.3 46.7 40.5 a,a,a slb 20.8 16.9 22.9 26.7 a,a,a dsw 20.8 18.4 20.1 18.2 o slb 61.7 29.5 36.6 27.4 o sw 7.7 10.0 11.7 13.7 o,a slb 36.0 19.5 28.3 28.9 o,a sw 3.8 5.7 7.5 7.2
(event_dsw does not support ordered queues, so ordered rows are against event_sw.)
Observations:
- SLB is ahead at low worker counts but loses throughput faster as workers are added on single-stage parallel/atomic queues, which is consistent with all workers contending on one MP/MC ring per queue.
- Ordered queues are well ahead of event_sw (the only other SW PMD with ordered support), but SLB ordered throughput halves going from 1 to 2 workers (61.7 -> 29.5 Mpps, reproduced), so the "close to parallel" claim only holds for a single worker (parallel: 79.7 -> 71.5). This looks like the serialized reorder drain.
- With workers spanning both sockets (16/20 workers) all drivers drop sharply; SLB falls to 5-10 Mpps vs DSW 3-14 depending on stlist.
- Note that with multiple producers perf_queue creates a separate queue set per producer, which splits SLB's per-queue contention; SLB scales very well in that mode (4 producers, 8 workers, atomic:
~235 Mpps vs ~158 for DSW at worker_deq_depth 16), but that does not reflect a pipeline where all traffic shares one atomic queue.
- order_queue passes with up to 16 workers. In that test SLB slows down as workers are added: with 8 workers it sustained ~1.0 M events/s vs ~2.5 M events/s for event_sw (steady-state rate from the progress counter, 20M events); with 1-2 workers SLB was faster than event_sw (rough end-to-end timings, ~3.2 vs ~1.6-2.1 M events/s).
These are untuned runs. In the 1-producer table the SLB/DSW atomic and a,a,a rows, DSW parallel and SLB ordered were run twice and agreed within ~5%; the other rows are single runs. Treat them as indicative. It would help if v2 included your own perf_queue/pipeline numbers with the exact command lines, ideally including a shared-queue case and workers on more than one socket.
Thanks,
Tirthendu