event/dlb Intel DLB PMD
cover event/dlb Intel DLB PMD
Hello Jerin and the DPDK community.
The following patchset adds support for a new eventdev PMD. The DLB PMD adds support for the Intel Dynamic Load Balancer (DLB) hardware. The DLB is a PCIe device that provides load-balanced, prioritized scheduling of core-to-core communication. The device consists of queues and arbiters that connect producer and consumer cores, and implements load-balanced queueing features including:
- Lock-free multi-producer/multi-consumer operation.
- Multiple priority levels for varying traffic types.
- 'Direct' traffic (i.e. multi-producer/single-consumer)
- Simple unordered load-balanced distribution.
- Atomic lock-free load balancing across multiple consumers.
- Queue element reordering feature allowing ordered load-balanced distribution.
The DLB hardware supports both load balanced and directed ports and queues. Unlike other eventdev devices already in the repo, not all DLB ports and queues are equally capable. In particular, directed ports are limited to a single link, and must be connected to a directed queue.
Additionally, even though LDB ports may link multiple queues, the number of queues that may be linked is limited by hardware. Another difference is that DLB does not have a straightforward way of carrying the flow_id in the queue elements (QE) that the hardware operates on.
Due to the above restrictions, we would like to extend the eventdev interface in manner that allows our PMD to take full advantage of the DLB hardware, while also adding useful functionality for non-Intel PMDs. Our proposed changes are contained in the first two patches in this patch set.
While reviewing the code, please be aware that this PMD has full control over the DLB hardware. Intel will be extending the DLB PMD in the future (not as part of this first series) with a mode that we refer to as the bifurcated PMD. The bifurcated PMD communicates with a kernel driver to configure the device, ports, and queues, and memory maps device MMIO so data path operations occur purely in user-space.
The framework to support both the PF PMD and bifurcated PMD exists in this patch set, and is why the iface.[ch] layer is present.
New for V2
=========
1) Correct ABI break that was present in V1.
2) Address some of the review comments received from Mattias. I will address the remaining items identified by Mattias in the next patch delivery.
3) General code cleanup based on internal code reviews
Known Issues:
1) some core dlb enqueue functions appear prematurely in a unrelated patch. This will be addressed in the next update to this patch series, after the first round of reviews has been completed.
Thank You and Best Regards,
Tim McDaniel
McDaniel, Timothy (27):
eventdev: dlb upstream prerequisites
eventdev: do not pass disable_implicit_release bit to trace macro event/dlb: add shared code version 10.7.9
event/dlb: add make and meson build infrastructure event/dlb: add DLB documentation
event/dlb: add dynamic logging
event/dlb: add private data structures and constants event/dlb: add definitions shared with LKM or shared code event/dlb: add inline functions used in multiple files event/dlb: add PFPMD-specific interface layer to shared code event/dlb: add flexible PMD to device interfaces event/dlb: add the PMD's public interfaces
event/dlb: add xstats support
event/dlb: add PMD self-tests
event/dlb: add probe
event/dlb: add infos_get and configure
event/dlb: add queue_def_conf and port_def_conf event/dlb: add queue setup
event/dlb: add port_setup
event/dlb: add port_link
event/dlb: add queue_release and port_release event/dlb: add port_unlink and port_unlinks_in_progress event/dlb: add eventdev_start
event/dlb: add timeout_ticks, dump, xstats, and selftest event/dlb: add enqueue and its burst variants event/dlb: add dequeue, dequeue_burst, and variants event/dlb: add eventdev_stop and eventdev_close
app/test-eventdev/evt_common.h | 1 + app/test-eventdev/test_order_atq.c | 4 + app/test-eventdev/test_order_common.c | 6 +- app/test-eventdev/test_order_queue.c | 4 + app/test-eventdev/test_perf_atq.c | 1 + app/test-eventdev/test_perf_queue.c | 1 + app/test-eventdev/test_pipeline_atq.c | 1 + app/test-eventdev/test_pipeline_queue.c | 1 + app/test/test_eventdev.c | 4 +- config/common_base | 19 +- config/rte_config.h | 8 +- doc/guides/eventdevs/dlb.rst | 497 + drivers/event/Makefile | 7 + drivers/event/dlb/Makefile | 35 + drivers/event/dlb/dlb.c | 4232 +++++++++ drivers/event/dlb/dlb_iface.c | 105 + drivers/event/dlb/dlb_iface.h | 92 + drivers/event/dlb/dlb_inline_fns.h | 80 + drivers/event/dlb/dlb_log.h | 24 + drivers/event/dlb/dlb_priv.h | 564 ++ drivers/event/dlb/dlb_selftest.c | 1564 ++++ drivers/event/dlb/dlb_user.h | 1083 +++ drivers/event/dlb/dlb_xstats.c | 1249 +++ drivers/event/dlb/meson.build | 15 + drivers/event/dlb/pf/base/dlb_hw_types.h | 360 + drivers/event/dlb/pf/base/dlb_mbox.h | 645 ++ drivers/event/dlb/pf/base/dlb_osdep.h | 348 + drivers/event/dlb/pf/base/dlb_osdep_bitmap.h | 442 + drivers/event/dlb/pf/base/dlb_osdep_list.h | 131 + drivers/event/dlb/pf/base/dlb_osdep_types.h | 31 + drivers/event/dlb/pf/base/dlb_regs.h | 2646 ++++++ drivers/event/dlb/pf/base/dlb_resource.c | 9700 ++++++++++++++++++++ drivers/event/dlb/pf/base/dlb_resource.h | 1625 ++++ drivers/event/dlb/pf/base/dlb_user.h | 1084 +++ drivers/event/dlb/pf/dlb_main.c | 609 ++ drivers/event/dlb/pf/dlb_main.h | 54 + drivers/event/dlb/pf/dlb_pf.c | 776 ++ drivers/event/dlb/rte_pmd_dlb.c | 38 + drivers/event/dlb/rte_pmd_dlb.h | 69 + drivers/event/dlb/rte_pmd_dlb_event_version.map | 6 + drivers/event/dpaa2/dpaa2_eventdev.c | 2 +- drivers/event/meson.build | 4 + drivers/event/octeontx/ssovf_evdev.c | 2 +- drivers/event/skeleton/skeleton_eventdev.c | 2 +- drivers/event/sw/sw_evdev.c | 5 +- drivers/event/sw/sw_evdev_selftest.c | 9 +- .../eventdev_pipeline/pipeline_worker_generic.c | 8 +- examples/eventdev_pipeline/pipeline_worker_tx.c | 3 + examples/l2fwd-event/l2fwd_event_generic.c | 5 +- examples/l2fwd-event/l2fwd_event_internal_port.c | 5 +- examples/l3fwd/l3fwd_event_generic.c | 5 +- examples/l3fwd/l3fwd_event_internal_port.c | 5 +- lib/librte_eal/x86/include/rte_cpuflags.h | 1 + lib/librte_eal/x86/rte_cpuflags.c | 1 + lib/librte_eventdev/meson.build | 1 + lib/librte_eventdev/rte_event_crypto_adapter.c | 2 +- lib/librte_eventdev/rte_event_eth_tx_adapter.c | 7 +- lib/librte_eventdev/rte_eventdev.c | 201 +- lib/librte_eventdev/rte_eventdev.h | 198 + lib/librte_eventdev/rte_eventdev_pmd_pci.h | 54 + lib/librte_eventdev/rte_eventdev_trace.h | 9 +- lib/librte_eventdev/rte_eventdev_version.map | 13 +- mk/rte.app.mk | 1 + 63 files changed, 28658 insertions(+), 46 deletions(-) create mode 100644 doc/guides/eventdevs/dlb.rst create mode 100644 drivers/event/dlb/Makefile create mode 100644 drivers/event/dlb/dlb.c create mode 100644 drivers/event/dlb/dlb_iface.c create mode 100644 drivers/event/dlb/dlb_iface.h create mode 100644 drivers/event/dlb/dlb_inline_fns.h create mode 100644 drivers/event/dlb/dlb_log.h create mode 100644 drivers/event/dlb/dlb_priv.h create mode 100644 drivers/event/dlb/dlb_selftest.c create mode 100644 drivers/event/dlb/dlb_user.h create mode 100644 drivers/event/dlb/dlb_xstats.c create mode 100644 drivers/event/dlb/meson.build create mode 100644 drivers/event/dlb/pf/base/dlb_hw_types.h create mode 100644 drivers/event/dlb/pf/base/dlb_mbox.h create mode 100644 drivers/event/dlb/pf/base/dlb_osdep.h create mode 100644 drivers/event/dlb/pf/base/dlb_osdep_bitmap.h create mode 100644 drivers/event/dlb/pf/base/dlb_osdep_list.h create mode 100644 drivers/event/dlb/pf/base/dlb_osdep_types.h create mode 100644 drivers/event/dlb/pf/base/dlb_regs.h create mode 100644 drivers/event/dlb/pf/base/dlb_resource.c create mode 100644 drivers/event/dlb/pf/base/dlb_resource.h create mode 100644 drivers/event/dlb/pf/base/dlb_user.h create mode 100644 drivers/event/dlb/pf/dlb_main.c create mode 100644 drivers/event/dlb/pf/dlb_main.h create mode 100644 drivers/event/dlb/pf/dlb_pf.c create mode 100644 drivers/event/dlb/rte_pmd_dlb.c create mode 100644 drivers/event/dlb/rte_pmd_dlb.h create mode 100644 drivers/event/dlb/rte_pmd_dlb_event_version.map
Signature · 1 line
1.7.10
PATCH 1/27 eventdev: dlb upstream prerequisites
From: "McDaniel, Timothy" <[email protected]>
The DLB hardware does not conform exactly to the eventdev interface.
1) It has a limit on the number of queues that may be linked to a port.
2) Some ports a further restricted to a maximum of 1 linked queue.
3) It does not (currently) have the ability to carry the flow_id as part of the event (QE) payload.
Due to the above, we would like to propose the following enhancements.
1) Add new fields to the rte_event_dev_info struct. These fields allow the device to advertize its capabilities so that applications can take the appropriate actions based on those capabilities.
struct rte_event_dev_info {
uint32_t max_event_port_links;
/**< Maximum number of queues that can be linked to a single event
* port by this device.
*/
uint8_t max_single_link_event_port_queue_pairs;
/**< Maximum number of event ports and queues that are optimized for
* (and only capable of) single-link configurations supported by this
* device. These ports and queues are not accounted for in
* max_event_ports or max_event_queues.
*/
}2) Add a new field to the rte_event_dev_config struct. This field allows the application to specify how many of its ports are limited to a single link, or will be used in single link mode.
/** Event device configuration structure */
struct rte_event_dev_config {
uint8_t nb_single_link_event_port_queues;
/**< Number of event ports and queues that will be singly-linked to
* each other. These are a subset of the overall event ports and
* queues; this value cannot exceed *nb_event_ports* or
* *nb_event_queues*. If the device has ports and queues that are
* optimized for single-link usage, this field is a hint for how many
* to allocate; otherwise, regular event ports and queues can be used.
*/
}3) Replace the dedicated implicit_release_disabled field with a bit field of explicit port capabilities. The implicit_release_disable functionality is assiged to one bit, and a port-is-single-link-only attribute is assigned to other, with the remaining bits available for future assignment.
* Event port configuration bitmap flags */
#define RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL (1ULL << 0)
/**< Configure the port not to release outstanding events in
* rte_event_dev_dequeue_burst(). If set, all events received through
* the port must be explicitly released with RTE_EVENT_OP_RELEASE or
* RTE_EVENT_OP_FORWARD. Must be unset if the device is not
* RTE_EVENT_DEV_CAP_IMPLICIT_RELEASE_DISABLE capable.
*/
#define RTE_EVENT_PORT_CFG_SINGLE_LINK (1ULL << 1)
/**< This event port links only to a single event queue.
*
* @see rte_event_port_setup(), rte_event_port_link()
*/
#define RTE_EVENT_PORT_ATTR_IMPLICIT_RELEASE_DISABLE 3
/**
* The implicit release disable attribute of the port
*/
struct rte_event_port_conf {
uint32_t event_port_cfg; /**< Port cfg flags(EVENT_PORT_CFG_) */
}4) Add UMWAIT/UMONITOR bit to rte_cpuflags
5) Added a new API that is useful for probing PCI devices.
/**
* @internal
* Wrapper for use by pci drivers as a .probe function to attach to a event
* interface. Same as rte_event_pmd_pci_probe, except caller can specify
* the name.
*/
static inline int
rte_event_pmd_pci_probe_named(struct rte_pci_driver *pci_drv,
struct rte_pci_device *pci_dev, size_t private_data_size, eventdev_pmd_pci_callback_t devinit, const char *name);
--- app/test-eventdev/evt_common.h | 1 + app/test-eventdev/test_order_atq.c | 4 + app/test-eventdev/test_order_common.c | 6 +- app/test-eventdev/test_order_queue.c | 4 + app/test-eventdev/test_perf_atq.c | 1 + app/test-eventdev/test_perf_queue.c | 1 + app/test-eventdev/test_pipeline_atq.c | 1 + app/test-eventdev/test_pipeline_queue.c | 1 + app/test/test_eventdev.c | 4 +- drivers/event/dpaa2/dpaa2_eventdev.c | 2 +- drivers/event/octeontx/ssovf_evdev.c | 2 +- drivers/event/skeleton/skeleton_eventdev.c | 2 +- drivers/event/sw/sw_evdev.c | 5 +- drivers/event/sw/sw_evdev_selftest.c | 9 +- .../eventdev_pipeline/pipeline_worker_generic.c | 8 +- examples/eventdev_pipeline/pipeline_worker_tx.c | 3 + examples/l2fwd-event/l2fwd_event_generic.c | 5 +- examples/l2fwd-event/l2fwd_event_internal_port.c | 5 +- examples/l3fwd/l3fwd_event_generic.c | 5 +- examples/l3fwd/l3fwd_event_internal_port.c | 5 +- lib/librte_eal/x86/include/rte_cpuflags.h | 1 + lib/librte_eal/x86/rte_cpuflags.c | 1 + lib/librte_eventdev/meson.build | 1 + lib/librte_eventdev/rte_event_eth_tx_adapter.c | 2 +- lib/librte_eventdev/rte_eventdev.c | 198 ++++++++++++++++++-- lib/librte_eventdev/rte_eventdev.h | 198 ++++++++++++++++++++ lib/librte_eventdev/rte_eventdev_pmd_pci.h | 54 ++++++ lib/librte_eventdev/rte_eventdev_version.map | 13 +- 28 files changed, 507 insertions(+), 35 deletions(-)
app/test-eventdev/evt_common.h +1 -0
@@ -169,6 +169,7 @@ struct evt_options { .dequeue_timeout_ns = opt->deq_tmo_nsec, .nb_event_queues = nb_queues, .nb_event_ports = nb_ports, + .nb_single_link_event_port_queues = 0, .nb_events_limit = info.max_num_events, .nb_event_queue_flows = opt->nb_flows, .nb_event_port_dequeue_depth =
app/test-eventdev/test_order_atq.c +4 -0
@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +if (ev.sub_event_type ==0 ) {/* stage 0 from producer */ order_atq_process_stage_0 (&ev); while (rte_event_enqueue_burst(dev_id, port, &ev, 1)
@@ -68,6 +70,8 @@ }for (i =0 ; i < nb_rx; i++) { + ev[i].flow_id = ev[i].mbuf->udata64; +if (ev[i].sub_event_type ==0 ) {/*stage 0 */ order_atq_process_stage_0 (&ev[i]); }else if (ev[i].sub_event_type ==1 ) {/* stage 1 */
app/test-eventdev/test_order_common.c +5 -1
@@ -49,6 +49,7 @@const uint32_t flow = (uintptr_t )m % nb_flows;/* Maintain seq number per flow */ m->seqn = producer_flow_seq[flow]++; + m->udata64 = flow; ev.flow_id = flow; ev.mbuf = m;
@@ -318,10 +319,11 @@ opt->wkr_deq_dep = dev_info.max_event_port_dequeue_depth;/* port configuration */ -const struct rte_event_port_conf p_conf = { +struct rte_event_port_conf p_conf = { .dequeue_depth = opt->wkr_deq_dep, .enqueue_depth = dev_info.max_event_port_dequeue_depth, .new_event_threshold = dev_info.max_num_events, + .event_port_cfg =0 , };/* setup one port per worker, linking to all queues */
@@ -351,6 +353,8 @@ p->queue_id =0 ; p->t = t; + p_conf.new_event_threshold /=2 ; + ret =rte_event_port_setup (opt->dev_id, port, &p_conf);if (ret) {evt_err ("failed to setup producer port %d" , port);
app/test-eventdev/test_order_queue.c +4 -0
@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +if (ev.queue_id ==0 ) {/* from ordered queue */ order_queue_process_stage_0 (&ev); while (rte_event_enqueue_burst(dev_id, port, &ev, 1)
@@ -68,6 +70,8 @@ }for (i =0 ; i < nb_rx; i++) { + ev[i].flow_id = ev[i].mbuf->udata64; +if (ev[i].queue_id ==0 ) {/* from ordered queue */ order_queue_process_stage_0 (&ev[i]); }else if (ev[i].queue_id ==1 ) {/* from atomic queue */
app/test-eventdev/test_perf_atq.c +1 -0
@@ -204,6 +204,7 @@ .dequeue_depth = opt->wkr_deq_dep, .enqueue_depth = dev_info.max_event_port_dequeue_depth, .new_event_threshold = dev_info.max_num_events, + .event_port_cfg = 0, }; ret = perf_event_dev_port_setup(test, opt, 1 /* stride */, nb_queues,
app/test-eventdev/test_perf_queue.c +1 -0
@@ -219,6 +219,7 @@ .dequeue_depth = opt->wkr_deq_dep, .enqueue_depth = dev_info.max_event_port_dequeue_depth, .new_event_threshold = dev_info.max_num_events, + .event_port_cfg = 0, }; ret = perf_event_dev_port_setup(test, opt, nb_stages /* stride */,
app/test-eventdev/test_pipeline_atq.c +1 -0
@@ -356,6 +356,7 @@ .dequeue_depth = opt->wkr_deq_dep, .enqueue_depth = info.max_event_port_dequeue_depth, .new_event_threshold = info.max_num_events, + .event_port_cfg = 0, };if (!t->internal_port)
app/test-eventdev/test_pipeline_queue.c +1 -0
@@ -379,6 +379,7 @@ .dequeue_depth = opt->wkr_deq_dep, .enqueue_depth = info.max_event_port_dequeue_depth, .new_event_threshold = info.max_num_events, + .event_port_cfg = 0, };if (!t->internal_port) {
app/test/test_eventdev.c +2 -2
@@ -559,10 +559,10 @@if (!(info.event_dev_cap &RTE_EVENT_DEV_CAP_IMPLICIT_RELEASE_DISABLE )) { pconf.enqueue_depth = info.max_event_port_enqueue_depth; - pconf.disable_implicit_release =1 ; + pconf.event_port_cfg =RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL ; ret =rte_event_port_setup (TEST_DEV_ID ,0 , &pconf);TEST_ASSERT (ret == -EINVAL ,"Expected -EINVAL, %d" , ret); - pconf.disable_implicit_release =0 ; + pconf.event_port_cfg =0 ; } ret = rte_event_port_setup(TEST_DEV_ID, info.max_event_ports,
drivers/event/dpaa2/dpaa2_eventdev.c +1 -1
@@ -537,7 +537,7 @@ static void dpaa2_eventdev_process_atomic(struct qbman_swp *swp,DPAA2_EVENT_MAX_PORT_DEQUEUE_DEPTH ; port_conf->enqueue_depth =DPAA2_EVENT_MAX_PORT_ENQUEUE_DEPTH ; - port_conf->disable_implicit_release =0 ; + port_conf->event_port_cfg =0 ; } static int
drivers/event/octeontx/ssovf_evdev.c +1 -1
@@ -224,7 +224,7 @@ struct ssovf_mbox_convert_ns_getworks_iter { port_conf->new_event_threshold = edev->max_num_events; port_conf->dequeue_depth =1 ; port_conf->enqueue_depth =1 ; - port_conf->disable_implicit_release =0 ; + port_conf->event_port_cfg =0 ; } static void
drivers/event/skeleton/skeleton_eventdev.c +1 -1
@@ -209,7 +209,7 @@ port_conf->new_event_threshold =32 *1024 ; port_conf->dequeue_depth =16 ; port_conf->enqueue_depth =16 ; - port_conf->disable_implicit_release =0 ; + port_conf->event_port_cfg =0 ; } static void
drivers/event/sw/sw_evdev.c +3 -2
@@ -175,7 +175,8 @@ } p->inflight_max = conf->new_event_threshold; - p->implicit_release = !conf->disable_implicit_release; + p->implicit_release = !(conf->event_port_cfg & +RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL );/* check if ring exists, same as rx_worker above */ snprintf(buf, sizeof(buf), "sw%d_p%u, %s", dev->data->dev_id,
@@ -508,7 +509,7 @@ port_conf->new_event_threshold =1024 ; port_conf->dequeue_depth =16 ; port_conf->enqueue_depth =16 ; - port_conf->disable_implicit_release =0 ; + port_conf->event_port_cfg =0 ; } static int
drivers/event/sw/sw_evdev_selftest.c +5 -4
@@ -172,7 +172,7 @@ struct test { .new_event_threshold = 1024, .dequeue_depth = 32, .enqueue_depth = 64, - .disable_implicit_release = 0, + .event_port_cfg = 0, };if (num_ports >MAX_PORTS )return -1 ;
@@ -1227,7 +1227,7 @@ struct test_event_dev_stats { .new_event_threshold = 128, .dequeue_depth = 32, .enqueue_depth = 64, - .disable_implicit_release = 0, + .event_port_cfg = 0, };if (rte_event_port_setup (evdev,0 , &port_conf) <0 ) {printf ("%d Error setting up port\n" , __LINE__);
@@ -1317,7 +1317,7 @@ struct test_event_dev_stats { .new_event_threshold = 128, .dequeue_depth = 32, .enqueue_depth = 64, - .disable_implicit_release = 0, + .event_port_cfg = 0, };if (rte_event_port_setup (evdev,0 , &port_conf) <0 ) {printf ("%d Error setting up port\n" , __LINE__);
@@ -3079,7 +3079,8 @@ struct test_event_dev_stats {* only be initialized once - and this needs to be set for multiple runs */ conf.new_event_threshold =512 ; - conf.disable_implicit_release = disable_implicit_release; + conf.event_port_cfg = disable_implicit_release ? +RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL :0 ;if (rte_event_port_setup (evdev,0 , &conf) <0 ) {printf ("Error setting up RX port\n" );
examples/eventdev_pipeline/pipeline_worker_generic.c +6 -2
@@ -129,6 +129,7 @@struct rte_event_dev_config config = { .nb_event_queues = nb_queues, .nb_event_ports = nb_ports, + .nb_single_link_event_port_queues =1 , .nb_events_limit =4096 , .nb_event_queue_flows =1024 , .nb_event_port_dequeue_depth =128 ,
@@ -138,12 +139,13 @@ .dequeue_depth = cdata.worker_cq_depth, .enqueue_depth = 64, .new_event_threshold = 4096, + .event_port_cfg = 0, };struct rte_event_queue_conf wkr_q_conf = { .schedule_type = cdata.queue_type, .priority =RTE_EVENT_DEV_PRIORITY_NORMAL , .nb_atomic_flows =1024 , - .nb_atomic_order_sequences =1024 , + .nb_atomic_order_sequences =1024 , };struct rte_event_queue_conf tx_q_conf = { .priority =RTE_EVENT_DEV_PRIORITY_HIGHEST ,
@@ -167,7 +169,8 @@ disable_implicit_release = (dev_info.event_dev_cap &RTE_EVENT_DEV_CAP_IMPLICIT_RELEASE_DISABLE ); - wkr_p_conf.disable_implicit_release = disable_implicit_release; + wkr_p_conf.event_port_cfg = disable_implicit_release ? +RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL :0 ;if (dev_info.max_num_events < config.nb_events_limit) config.nb_events_limit = dev_info.max_num_events;
@@ -417,6 +420,7 @@ .dequeue_depth = cdata.worker_cq_depth, .enqueue_depth = 64, .new_event_threshold = 4096, + .event_port_cfg = 0, };if (adptr_p_conf.new_event_threshold > dev_info.max_num_events)
examples/eventdev_pipeline/pipeline_worker_tx.c +3 -0
@@ -436,6 +436,7 @@struct rte_event_dev_config config = { .nb_event_queues = nb_queues, .nb_event_ports = nb_ports, + .nb_single_link_event_port_queues =0 , .nb_events_limit =4096 , .nb_event_queue_flows =1024 , .nb_event_port_dequeue_depth =128 ,
@@ -445,6 +446,7 @@ .dequeue_depth = cdata.worker_cq_depth, .enqueue_depth = 64, .new_event_threshold = 4096, + .event_port_cfg = 0, };struct rte_event_queue_conf wkr_q_conf = { .schedule_type = cdata.queue_type,
@@ -746,6 +748,7 @@ struct rx_adptr_services { .dequeue_depth = cdata.worker_cq_depth, .enqueue_depth = 64, .new_event_threshold = 4096, + .event_port_cfg = 0, };init_ports (nb_ports);
examples/l2fwd-event/l2fwd_event_generic.c +3 -2
@@ -126,8 +126,9 @@if (def_p_conf.enqueue_depth < event_p_conf.enqueue_depth) event_p_conf.enqueue_depth = def_p_conf.enqueue_depth; - event_p_conf.disable_implicit_release = - evt_rsrc->disable_implicit_release; + event_p_conf.event_port_cfg =0 ; +if (evt_rsrc->disable_implicit_release) + event_p_conf.event_port_cfg |=RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL ; evt_rsrc->deq_depth = def_p_conf.dequeue_depth; for (event_p_id = 0; event_p_id < evt_rsrc->evp.nb_ports;
examples/l2fwd-event/l2fwd_event_internal_port.c +3 -2
@@ -123,8 +123,9 @@if (def_p_conf.enqueue_depth < event_p_conf.enqueue_depth) event_p_conf.enqueue_depth = def_p_conf.enqueue_depth; - event_p_conf.disable_implicit_release = - evt_rsrc->disable_implicit_release; + event_p_conf.event_port_cfg =0 ; +if (evt_rsrc->disable_implicit_release) + event_p_conf.event_port_cfg |=RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL ;for (event_p_id =0 ; event_p_id < evt_rsrc->evp.nb_ports; event_p_id++) {
examples/l3fwd/l3fwd_event_generic.c +3 -2
@@ -115,8 +115,9 @@if (def_p_conf.enqueue_depth < event_p_conf.enqueue_depth) event_p_conf.enqueue_depth = def_p_conf.enqueue_depth; - event_p_conf.disable_implicit_release = - evt_rsrc->disable_implicit_release; + event_p_conf.event_port_cfg =0 ; +if (evt_rsrc->disable_implicit_release) + event_p_conf.event_port_cfg |=RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL ; evt_rsrc->deq_depth = def_p_conf.dequeue_depth; for (event_p_id = 0; event_p_id < evt_rsrc->evp.nb_ports;
examples/l3fwd/l3fwd_event_internal_port.c +3 -2
@@ -113,8 +113,9 @@if (def_p_conf.enqueue_depth < event_p_conf.enqueue_depth) event_p_conf.enqueue_depth = def_p_conf.enqueue_depth; - event_p_conf.disable_implicit_release = - evt_rsrc->disable_implicit_release; + event_p_conf.event_port_cfg =0 ; +if (evt_rsrc->disable_implicit_release) + event_p_conf.event_port_cfg |=RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL ;for (event_p_id =0 ; event_p_id < evt_rsrc->evp.nb_ports; event_p_id++) {
lib/librte_eal/x86/include/rte_cpuflags.h +1 -0
@@ -130,6 +130,7 @@ enum rte_cpu_flag_t {RTE_CPUFLAG_CLDEMOTE ,/**< Cache Line Demote */ RTE_CPUFLAG_MOVDIRI ,/**< Direct Store Instructions */ RTE_CPUFLAG_MOVDIR64B ,/**< Direct Store Instructions 64B */ +RTE_CPUFLAG_UMWAIT ,/**< UMONITOR/UMWAIT */ RTE_CPUFLAG_AVX512VP2INTERSECT, /**< AVX512 Two Register Intersection *//* The last item */
lib/librte_eal/x86/rte_cpuflags.c +1 -0
@@ -137,6 +137,7 @@ struct feature_entry { FEAT_DEF(CLDEMOTE, 0x00000007, 0, RTE_REG_ECX, 25) FEAT_DEF(MOVDIRI, 0x00000007, 0, RTE_REG_ECX, 27) FEAT_DEF(MOVDIR64B, 0x00000007, 0, RTE_REG_ECX, 28) + FEAT_DEF(UMWAIT, 0x00000007, 0, RTE_REG_ECX, 5) FEAT_DEF(AVX512VP2INTERSECT, 0x00000007, 0, RTE_REG_EDX, 8) };
lib/librte_eventdev/meson.build +1 -0
@@ -7,6 +7,7 @@ else cflags += '-DBSD' endif +use_function_versioning = true sources = files('rte_eventdev.c', 'rte_event_ring.c', 'eventdev_trace_points.c',
lib/librte_eventdev/rte_event_eth_tx_adapter.c +1 -1
@@ -286,7 +286,7 @@ static int txa_service_queue_del(uint8_t id,return ret; } - pc->disable_implicit_release =0 ; + pc->event_port_cfg =0 ; ret =rte_event_port_setup (dev_id, port_id, pc);if (ret) { RTE_EDEV_LOG_ERR("failed to setup event port %u\n",
lib/librte_eventdev/rte_eventdev.c +187 -11
278 lines not on this page · raw message
lib/librte_eventdev/rte_eventdev.h +198 -0
258 lines not on this page · raw message
lib/librte_eventdev/rte_eventdev_pmd_pci.h +54 -0
60 lines not on this page · raw message
lib/librte_eventdev/rte_eventdev_version.map +12 -1
@@ -94,13 +94,21 @@ DPDK_20.0 { local: *; }; +DPDK_21 { + global: + + rte_event_dev_info_get; + rte_event_dev_configure; + rte_event_port_default_conf_get; + rte_event_port_setup; +} DPDK_20.0; + EXPERIMENTAL { global: # added in 20.05 __rte_eventdev_trace_configure; __rte_eventdev_trace_queue_setup; - __rte_eventdev_trace_port_setup; __rte_eventdev_trace_port_link; __rte_eventdev_trace_port_unlink; __rte_eventdev_trace_start;
@@ -134,4 +142,7 @@ EXPERIMENTAL { __rte_eventdev_trace_crypto_adapter_queue_pair_del; __rte_eventdev_trace_crypto_adapter_start; __rte_eventdev_trace_crypto_adapter_stop; + + # changed in 20.08 + __rte_eventdev_trace_port_setup; };
Signature · 1 line
1.7.10
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 32 lines
(quoted diff) +28
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity is limited, + * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higher priority + * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the event dev + * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit* + * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events received through
Review struct rte_event_port_conf { · line 920
Quoted 10 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);
Hi Timothy,
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " struct rte_event_port_conf"
# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/
Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?
(quoted diff) +0 -0
b/app/test-eventdev/test_order_atq.c
index 3366cfc..8246b96 100644
app/test-eventdev/test_order_atq.c +2 -0
@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +
# Since RC1 is near, I am not sure how to accommodate the API changes now and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Saturday, June 27, 2020 2:45 AM
To: McDaniel, Timothy <[email protected]>; Ray Kinsella <[email protected]>; Neil Horman <[email protected]>
Cc: Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 32 lines
(quoted diff) +28
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity is limited, + * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higher priority + * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the event dev + * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit* + * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events received through
Review struct rte_event_port_conf { · line 920
Quoted 10 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);
Hi Timothy,
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " struct rte_event_port_conf"
# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/
Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?
(quoted diff) +0 -0
b/app/test-eventdev/test_order_atq.c
index 3366cfc..8246b96 100644
app/test-eventdev/test_order_atq.c +2 -0
@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +
# Since RC1 is near, I am not sure how to accommodate the API changes now and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.
Hello Jerin,
Thank you for the review comments.
With regard to your comments regarding the fast path flow_id change, the Intel DLB hardware
is not capable of transferring the flow_id as part of the event itself. We therefore require a mechanism to accomplish this. What we have done to work around this is to require the application to embed the flow_id within the data payload. The new flag, #define RTE_EVENT_DEV_CAP_CARRY_FLOW_ID (1ULL << 9), can be used by applications to determine if they need to embed the flow_id, or if its automatically propagated and present in the received event.
What we should have done is to wrap the assignment with a conditional.
if (!(device_capability_flags & RTE_EVENT_DEV_CAP_CARRY_FLOW_ID))
ev.flow_id = ev.mbuf->udata64;
This would minimize/eliminate any performance impact due to the processor's branch prediction logic.
The assignment then becomes in essence a NOOP for all event devices that are capable of carrying the flow_id as part of the event payload itself.
Thanks,
Tim
Thanks,
Tim
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 98 lines · Timothy
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Saturday, June 27, 2020 2:45 AM
To: McDaniel, Timothy <[email protected]>; Ray Kinsella <[email protected]>; Neil Horman <[email protected]>
Cc: Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 32 lines
(quoted diff) +28
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity is limited, + * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higher priority + * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the event dev + * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit* + * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events received throughReview struct rte_event_port_conf { · line 920
Quoted 10 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);Hi Timothy,
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?app/test-eventdev/test_order_atq.c +2
-0@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the API changesnow and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.Hello Jerin,
Thank you for the review comments.
With regard to your comments regarding the fast path
flow_idchange, the Intel DLB hardware
is not capable of transferring theflow_idas part of the event itself. We therefore require a mechanism to accomplish this. What we have done to work around this is to require the application to embed theflow_idwithin the data payload. The new flag, #defineRTE_EVENT_DEV_CAP_CARRY_FLOW_ID(1ULL << 9), can be used by applications to determine if they need to embed theflow_id, or if its automatically propagated and present in the received event.
Quoted 4 lines
What we should have done is to wrap the assignment with a conditional.
if (!(device_capability_flags &
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID))ev.flow_id = ev.mbuf->udata64;
Two problems with this approach,
1) we are assuming mbuf udata64 field is available for DLB driver
2) It won't work with another adapter, eventdev has no dependency with mbuf
Question:
1) In the case of DLB hardware, on dequeue(), what HW returns? is it only event pointer and not have any other metadata like schedule_type etc.
Quoted 11 lines
This would minimize/eliminate any performance impact due to the processor's branch prediction logic.
The assignment then becomes in essence a NOOP for all event devices that are capable of carrying theflow_idas part of the event payload itself.Thanks,
TimThanks,
Tim
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 120 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, June 29, 2020 11:21 PM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil Horman <[email protected]>; Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 5 lines · Timothy
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Saturday, June 27, 2020 2:45 AM
To: McDaniel, Timothy <[email protected]>; Ray Kinsella<[email protected]>; Neil Horman <[email protected]>
Quoted 1 line
Cc Jerin Jacob <[email protected]>; Mattias Rönnblom<[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Quoted 15 lines
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 13 lines
(quoted diff) +13
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity is limited, + * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higher priority + * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the event dev + * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit*Quoted 8 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODEcapable.
Quoted 6 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+ */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODEcapable.
Quoted 52 lines
Quoted 5 lines
(quoted diff) +1
-0+ */ uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events received throughReview struct rte_event_port_conf { · line 920
Quoted 10 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);Hi Timothy,
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?app/test-eventdev/test_order_atq.c +2
-0@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the API changesnow and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.Quoted 5 lines
Hello Jerin,
Thank you for the review comments.
With regard to your comments regarding the fast path
flow_idchange, the IntelDLB hardware
Quoted 1 line
is not capable of transferring the
flow_idas part of the event itself. Wetherefore require a mechanism
Quoted 1 line
to accomplish this. What we have done to work around this is to require the
application to embed the
flow_idQuoted 1 line
within the data payload. The new flag, #define
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID(1ULL << 9), can be usedQuoted 1 line
by applications to determine if they need to embed the
flow_id, or if itsautomatically propagated and present in the
Quoted 2 lines
received event.
Quoted 9 lines
Quoted 4 lines
What we should have done is to wrap the assignment with a conditional.
if (!(device_capability_flags &
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID))ev.flow_id = ev.mbuf->udata64;Two problems with this approach,
1) we are assuming mbuf udata64 field is available for DLB driver
2) It won't work with another adapter, eventdev has no dependency with mbuf
This snippet is not intended to suggest that udata64 always be used to store the flow ID, but as an example of how an application could do it. Some applications won’t need to carry the flow ID through; others can select an unused field in the event data (e.g. hash.rss or udata64 if using mbufs), or (worst-case) re-generate the flow ID in pipeline stages that require it.
Quoted 5 lines
Question:
1) In the case of DLB hardware, ondequeue(), what HW returns? is it only event pointer and not have any other metadata likeschedule_typeetc.
The DLB device provides a 16B “queue entry” that consists of:
* 8B event data
* Queue ID
* Priority
* Scheduling type
* 19 bits of carried-through data
* Assorted error/debug/reserved bits that are set by the device (not carried-through)
For the carried-through 19b, we use 12b for event_type and sub_event_type.
Quoted 14 lines
Quoted 3 lines
This would minimize/eliminate any performance impact due to the processor's
branch prediction logic.
Quoted 1 line
The assignment then becomes in essence a NOOP for all event devices that are
capable of carrying the
flow_idas part of the event payload itself.Quoted 8 lines
Thanks,
TimThanks,
Tim
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 126 lines · Timothy
Quoted 126 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, June 29, 2020 11:21 PM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil Horman <[email protected]>; Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 5 lines · Timothy
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Saturday, June 27, 2020 2:45 AM
To: McDaniel, Timothy <[email protected]>; Ray Kinsella<[email protected]>; Neil Horman <[email protected]>
Quoted 1 line
Cc Jerin Jacob <[email protected]>; Mattias Rönnblom<[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Quoted 15 lines
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 13 lines
(quoted diff) +13
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity is limited, + * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higher priority + * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the event dev + * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit*Quoted 8 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODEcapable.
Quoted 6 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+ */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODEcapable.
Quoted 52 lines
Quoted 5 lines
(quoted diff) +1
-0+ */ uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events received throughReview struct rte_event_port_conf { · line 920
Quoted 10 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);Hi Timothy,
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?app/test-eventdev/test_order_atq.c +2
-0@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the API changesnow and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.Quoted 5 lines
Hello Jerin,
Thank you for the review comments.
With regard to your comments regarding the fast path
flow_idchange, the IntelDLB hardware
Quoted 1 line
is not capable of transferring the
flow_idas part of the event itself. Wetherefore require a mechanism
Quoted 1 line
to accomplish this. What we have done to work around this is to require the
application to embed the
flow_idQuoted 1 line
within the data payload. The new flag, #define
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID(1ULL << 9), can be usedQuoted 1 line
by applications to determine if they need to embed the
flow_id, or if itsautomatically propagated and present in the
Quoted 6 lines
received event.
What we should have done is to wrap the assignment with a conditional.
if (!(device_capability_flags &
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID))ev.flow_id = ev.mbuf->udata64;
Quoted 6 lines
Quoted 4 lines
Two problems with this approach,
1) we are assuming mbuf udata64 field is available for DLB driver
2) It won't work with another adapter, eventdev has no dependency with mbufThis snippet is not intended to suggest that udata64 always be used to store the flow ID, but as an example of how an application could do it. Some applications won’t need to carry the flow ID through; others can select an unused field in the event data (e.g. hash.rss or udata64 if using mbufs), or (worst-case) re-generate the flow ID in pipeline stages that require it.
OK.
Quoted 9 lines
Quoted 6 lines
Question:
1) In the case of DLB hardware, ondequeue(), what HW returns? is it only event pointer and not have any other metadata likeschedule_typeetc.The DLB device provides a 16B “queue entry” that consists of:
Quoted 8 lines
* 8B event data * Queue ID * Priority * Scheduling type * 19 bits of carried-through data * Assorted error/debug/reserved bits that are set by the device (not carried-through)For the carried-through 19b, we use 12b for
event_typeandsub_event_type.
I can only think of TWO options to help
1) Since event pointer always cache aligned, You could grab LSB 6bits(2^6 = 64B ) and 7 bits from (19b - 12b) carried through structure
2) Have separate mempool driver using existing drivers, ie "event pointer" + or - some offset have any amount of custom data.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
This would minimize/eliminate any performance impact due to the processor's
branch prediction logic.
I think, If we need to change common fastpath, better we need to make it template to create code for compile-time to have absolute zero overhead
and use runtime.
See app/test-eventdev/test_order_atq.c: function: worker_wrapper() _create_ worker at compile time based on runtime capability.
Quoted 10 lines
Quoted 10 lines
Quoted 1 line
The assignment then becomes in essence a NOOP for all event devices that are
capable of carrying the
flow_idas part of the event payload itself.Quoted 8 lines
Thanks,
TimThanks,
Tim
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 168 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, June 30, 2020 10:58 AM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil Horman <[email protected]>; Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 105 lines · Timothy
Quoted 105 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, June 29, 2020 11:21 PM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil Horman <[email protected]>; Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 5 lines · Timothy
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Saturday, June 27, 2020 2:45 AM
To: McDaniel, Timothy <[email protected]>; Ray Kinsella<[email protected]>; Neil Horman <[email protected]>
Quoted 1 line
Cc Jerin Jacob <[email protected]>; Mattias Rönnblom<[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Quoted 15 lines
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 13 lines
(quoted diff) +13
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity is limited, + * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higher priority + * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the event dev + * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit*Quoted 8 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODEcapable.
Quoted 6 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+ */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODEcapable.
Quoted 52 lines
Quoted 5 lines
(quoted diff) +1
-0+ */ uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events received throughReview struct rte_event_port_conf { · line 920
Quoted 10 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);Hi Timothy,
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?app/test-eventdev/test_order_atq.c +2
-0@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the API changesnow and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Hello Jerin,
Thank you for the review comments.
With regard to your comments regarding the fast path
flow_idchange, theIntel
Quoted 16 lines
Quoted 16 lines
DLB hardware
Quoted 1 line
is not capable of transferring the
flow_idas part of the event itself. Wetherefore require a mechanism
Quoted 1 line
to accomplish this. What we have done to work around this is to require the
application to embed the
flow_idQuoted 1 line
within the data payload. The new flag, #define
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID(1ULL << 9), can be usedQuoted 1 line
by applications to determine if they need to embed the
flow_id, or if itsautomatically propagated and present in the
Quoted 6 lines
received event.
What we should have done is to wrap the assignment with a conditional.
if (!(device_capability_flags &
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID))ev.flow_id = ev.mbuf->udata64;Quoted 6 lines
Quoted 4 lines
Two problems with this approach,
1) we are assuming mbuf udata64 field is available for DLB driver
2) It won't work with another adapter, eventdev has no dependency with mbufThis snippet is not intended to suggest that udata64 always be used to store the
flow ID, but as an example of how an application could do it. Some applications won’t need to carry the flow ID through; others can select an unused field in the event data (e.g. hash.rss or udata64 if using mbufs), or (worst-case) re-generate the flow ID in pipeline stages that require it.
OK.
Quoted 7 lines
Quoted 6 lines
Question:
1) In the case of DLB hardware, ondequeue(), what HW returns? is it only event pointer and not have any other metadata likeschedule_typeetc.Quoted 8 lines
The DLB device provides a 16B “queue entry” that consists of:
* 8B event data * Queue ID * Priority * Scheduling type * 19 bits of carried-through data * Assorted error/debug/reserved bits that are set by the device (not carried-through)
Quoted 1 line
Quoted 9 lines
Quoted 1 line
For the carried-through 19b, we use 12b for
event_typeandsub_event_type.I can only think of TWO options to help
1) Since event pointer always cache aligned, You could grab LSB 6bits(2^6 = 64B ) and 7 bits from (19b - 12b) carried through structure
2) Have separate mempool driver using existing drivers, ie "event pointer" + or - some offset have any amount of custom data.
We can't guarantee that the event will contain a pointer -- it's possible that 8B is inline data (i.e. struct rte_event's u64 field).
It's really an application decision -- for example an app could allocate space in the 'mbuf private data' to store the flow ID, if the event device lacks that carry-flow-ID capability and the other mbuf fields can't be used for whatever reason. We modified the tests, sample apps to show how this might be done, not necessarily how it must be done.
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
This would minimize/eliminate any performance impact due to the
processor's
Quoted 9 lines
Quoted 1 line
Quoted 1 line
branch prediction logic.
I think, If we need to change common fastpath, better we need to make it template to create code for compile-time to have absolute zero overhead
and use runtime.
Seeapp/test-eventdev/test_order_atq.c: function:worker_wrapper()_create_ worker at compile time based on runtime capability.
Yes, that would be perfect. Thanks for the example!
Quoted 13 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
The assignment then becomes in essence a NOOP for all event devices that
are
Quoted 9 lines
Quoted 9 lines
capable of carrying the
flow_idas part of the event payload itself.Quoted 8 lines
Thanks,
TimThanks,
Tim
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 269 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of McDaniel, Timothy
Sent: Wednesday, July 1, 2020 12:57 AM
To: Jerin Jacob <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 5 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, June 30, 2020 10:58 AM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil HormanQuoted 2 lines
Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads,
Gage
Quoted 1 line
<[email protected]>; Van Haaren, Harry
Quoted 1 line
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream
prerequisites
Quoted 9 lines
Quoted 6 lines · Timothy
Quoted 6 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, June 29, 2020 11:21 PM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil HormanQuoted 2 lines
Quoted 2 lines
Quoted 2 lines
Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>;
Eads, Gage
Quoted 1 line
Quoted 1 line
Quoted 1 line
<[email protected]>; Van Haaren, Harry
Quoted 1 line
Quoted 1 line
Quoted 1 line
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream
prerequisites
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines · Timothy
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Saturday, June 27, 2020 2:45 AM
To: McDaniel, Timothy <[email protected]>; RayKinsella
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
<[email protected]>; Neil Horman <[email protected]>
Quoted 1 line
Cc Jerin Jacob <[email protected]>; Mattias Rönnblom<[email protected]>; dpdk-dev <[email protected]>;
Eads, Gage
Quoted 1 line
Quoted 1 line
Quoted 1 line
<[email protected]>; Van Haaren, Harry
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream
prerequisites
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Review
Quoted 6 lines
(quoted diff) +5
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues onthis port.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ * Use for *closed system* event dev where event capacity
is limited,
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can makehigher priority
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets intothe event dev
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ * can have a lower threshold so as not to overwhelm the
device,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ * while ports used for worker pools can have a higher
threshold.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * This value cannot exceed the *nb_events_limit* + * which was previously supplied to
rte_event_dev_configure().Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this eventport.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ * This value cannot exceed the
*nb_event_port_dequeue_depth*
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not
RTE_EVENT_DEV_CAP_BURST_MODEQuoted 4 lines
Quoted 4 lines
Quoted 4 lines
capable.
Quoted 3 lines
Quoted 3 lines
(quoted diff) +3
-0+ */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this eventport.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ * This value cannot exceed the
*nb_event_port_enqueue_depth*
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not
RTE_EVENT_DEV_CAP_BURST_MODEQuoted 4 lines
Quoted 4 lines
Quoted 4 lines
capable.
Quoted 3 lines
Quoted 3 lines
+ */
uint8_t disable_implicit_release; /**< Configure the port not to release outstanding eventsin
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
*
rte_event_dev_dequeue_burst(). If true, all eventsreceived through
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review struct rte_event_port_conf { · line 911
Quoted 2 lines
(quoted diff) +0
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t
port_id,Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Review struct rte_event_port_conf { · line 915
Quoted 4 lines
(quoted diff) +2
-0struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t
port_id,Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Review struct rte_event_port_conf { · line 919
Quoted 4 lines
(quoted diff) +4
-0+ struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t
port_id,Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Review struct rte_event_port_conf { · line 920
Quoted 1 line
(quoted diff) +1
-0+ struct rte_event_port_conf *port_conf);Hi Timothy,
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned,
only
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"# We have a similar case with ethdev and it deferred to next
release v20.11
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
3A__patches.dpdk.org_patch_69113_&d=DwIGaQ&c=nKjWec2b6R0mO yPaz7xtfQ&r=1cjuAHrGh745jHNmj2fD85sUMIJ2IPIDsIJzo6FN6Z0&m=lL7 dDlN7ICIpENvIB7_El27UclXA2tJLdOwbsirg1Dw&s=CNmSXDDn28U- OjEAaZgJI_A2fDmMKM6zb12sIE9L-Io&e=
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Regarding the API changes:
# The slow path changes general looks good to me. I will reviewthe
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
next level in the coming days
# The following fast path changes bothers to me. Could you sharemore
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
details on below change?
app/test-eventdev/test_order_atq.c +0
-0@@ -34,6 +34,8 @@continue ; }Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-eventdev/test_order_atq.c
Quoted 3 lines
app/test-eventdev/test_order_atq.c +2
-0+ ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the APIchanges
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
now and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecationnotice for
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.Hello Jerin,
Thank you for the review comments.
With regard to your comments regarding the fast path
flow_idchange, the
Quoted 3 lines
Intel
Quoted 2 lines
Quoted 2 lines
DLB hardware
Quoted 1 line
is not capable of transferring the
flow_idas part of the eventitself. We
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
therefore require a mechanism
Quoted 1 line
to accomplish this. What we have done to work around this is to
require the
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
application to embed the
flow_idQuoted 1 line
within the data payload. The new flag, #define
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID(1ULL << 9), can be usedQuoted 1 line
by applications to determine if they need to embed the
flow_id,or if its
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
automatically propagated and present in the
Quoted 3 lines
received event.
What we should have done is to wrap the assignment with a
conditional.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
if (!(device_capability_flags &
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID))Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
ev.flow_id = ev.mbuf->udata64;Two problems with this approach,
1) we are assuming mbuf udata64 field is available for DLB driver
2) It won't work with another adapter, eventdev has nodependency with mbuf
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
This snippet is not intended to suggest that udata64 always be used
to store the
Quoted 1 line
flow ID, but as an example of how an application could do it. Some
applications
Quoted 1 line
won’t need to carry the flow ID through; others can select an unused
field in the
Quoted 1 line
event data (e.g. hash.rss or udata64 if using mbufs), or (worst-case)
re-generate
Quoted 6 lines
the flow ID in pipeline stages that require it.
OK.
Quoted 3 lines
Quoted 3 lines
Question:
1) In the case of DLB hardware, ondequeue(), what HW returns? isit
Quoted 1 line
Quoted 1 line
Quoted 1 line
only event pointer and not have any other metadata like
schedule_typeQuoted 11 lines
Quoted 11 lines
Quoted 2 lines
etc.
The DLB device provides a 16B “queue entry” that consists of:
* 8B event data * Queue ID * Priority * Scheduling type * 19 bits of carried-through data * Assorted error/debug/reserved bits that are set by the device(not carried-
Quoted 3 lines
through)
Quoted 2 lines
For the carried-through 19b, we use 12b for
event_typeand
sub_event_type.Quoted 8 lines
I can only think of TWO options to help
1) Since event pointer always cache aligned, You could grab LSB 6bits(2^6 = 64B ) and 7 bits from (19b - 12b) carried through structure
2) Have separate mempool driver using existing drivers, ie "event pointer" + or - some offset have any amount of custom data.We can't guarantee that the event will contain a pointer -- it's possible that 8B is inline data (i.e. struct rte_event's u64 field).
It's really an application decision -- for example an app could allocate space in the 'mbuf private data' to store the flow ID, if the event device lacks that carry-flow-ID capability and the other mbuf fields can't be used for whatever reason.
We modified the tests, sample apps to show how this might be done, not necessarily how it must be done.Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
This would minimize/eliminate any performance impact due to
the
Quoted 4 lines
processor's
Quoted 1 line
Quoted 1 line
branch prediction logic.
I think, If we need to change common fastpath, better we need to
make
Quoted 8 lines
Quoted 6 lines
it template to create code for compile-time to have absolute zero overhead
and use runtime.
Seeapp/test-eventdev/test_order_atq.c: function:worker_wrapper()_create_ worker at compile time based on runtime capability.Yes, that would be perfect. Thanks for the example!
Just to add instead of having if and else using a jumptbl would be much cleaner Ex.
const pipeline_atq_worker_t pipeline_atq_worker_single_stage[2][2][2] = {
[0][0] = pipeline_atq_worker_single_stage_fwd,
[0][1] = pipeline_atq_worker_single_stage_tx,
[1][0] = pipeline_atq_worker_single_stage_burst_fwd,
[1][1] = pipeline_atq_worker_single_stage_burst_tx,
};
return (pipeline_atq_worker_single_stage[burst][internal_port])(arg);Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
The assignment then becomes in essence a NOOP for all event
devices that
Quoted 10 lines
are
Quoted 9 lines
Quoted 9 lines
capable of carrying the
flow_idas part of the event payload itself.Quoted 8 lines
Thanks,
TimThanks,
Tim
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 300 lines
-----Original Message-----
From: Pavan Nikhilesh Bhagavatula <[email protected]>
Sent: Tuesday, June 30, 2020 3:40 PM
To: McDaniel, Timothy <[email protected]>; Jerin Jacob <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: RE: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 267 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of McDaniel, Timothy
Sent: Wednesday, July 1, 2020 12:57 AM
To: Jerin Jacob <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 5 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, June 30, 2020 10:58 AM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil HormanQuoted 2 lines
Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads,
Gage
Quoted 1 line
<[email protected]>; Van Haaren, Harry
Quoted 1 line
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream
prerequisites
Quoted 9 lines
Quoted 6 lines · Timothy
Quoted 6 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, June 29, 2020 11:21 PM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil HormanQuoted 2 lines
Quoted 2 lines
Quoted 2 lines
Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>;
Eads, Gage
Quoted 1 line
Quoted 1 line
Quoted 1 line
<[email protected]>; Van Haaren, Harry
Quoted 1 line
Quoted 1 line
Quoted 1 line
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream
prerequisites
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines · Timothy
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Saturday, June 27, 2020 2:45 AM
To: McDaniel, Timothy <[email protected]>; RayKinsella
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
<[email protected]>; Neil Horman <[email protected]>
Quoted 1 line
Cc Jerin Jacob <[email protected]>; Mattias Rönnblom<[email protected]>; dpdk-dev <[email protected]>;
Eads, Gage
Quoted 1 line
Quoted 1 line
Quoted 1 line
<[email protected]>; Van Haaren, Harry
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream
prerequisites
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Review
Quoted 6 lines
(quoted diff) +5
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues onthis port.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ * Use for *closed system* event dev where event capacity
is limited,
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can makehigher priority
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets intothe event dev
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ * can have a lower threshold so as not to overwhelm the
device,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ * while ports used for worker pools can have a higher
threshold.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * This value cannot exceed the *nb_events_limit* + * which was previously supplied to
rte_event_dev_configure().Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this eventport.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ * This value cannot exceed the
*nb_event_port_dequeue_depth*
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not
RTE_EVENT_DEV_CAP_BURST_MODEQuoted 4 lines
Quoted 4 lines
Quoted 4 lines
capable.
Quoted 3 lines
Quoted 3 lines
(quoted diff) +3
-0+ */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this eventport.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ * This value cannot exceed the
*nb_event_port_enqueue_depth*
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not
RTE_EVENT_DEV_CAP_BURST_MODEQuoted 4 lines
Quoted 4 lines
Quoted 4 lines
capable.
Quoted 3 lines
Quoted 3 lines
+ */
uint8_t disable_implicit_release; /**< Configure the port not to release outstanding eventsin
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
*
rte_event_dev_dequeue_burst(). If true, all eventsreceived through
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review struct rte_event_port_conf { · line 911
Quoted 2 lines
(quoted diff) +0
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t
port_id,Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Review struct rte_event_port_conf { · line 915
Quoted 4 lines
(quoted diff) +2
-0struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t
port_id,Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Review struct rte_event_port_conf { · line 919
Quoted 4 lines
(quoted diff) +4
-0+ struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t
port_id,Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Review struct rte_event_port_conf { · line 920
Quoted 1 line
(quoted diff) +1
-0+ struct rte_event_port_conf *port_conf);Hi Timothy,
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned,
only
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"# We have a similar case with ethdev and it deferred to next
release v20.11
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
3A__patches.dpdk.org_patch_69113_&d=DwIGaQ&c=nKjWec2b6R0mO yPaz7xtfQ&r=1cjuAHrGh745jHNmj2fD85sUMIJ2IPIDsIJzo6FN6Z0&m=lL7 dDlN7ICIpENvIB7_El27UclXA2tJLdOwbsirg1Dw&s=CNmSXDDn28U- OjEAaZgJI_A2fDmMKM6zb12sIE9L-Io&e=
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Regarding the API changes:
# The slow path changes general looks good to me. I will reviewthe
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
next level in the coming days
# The following fast path changes bothers to me. Could you sharemore
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
details on below change?
app/test-eventdev/test_order_atq.c +0
-0@@ -34,6 +34,8 @@continue ; }Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-eventdev/test_order_atq.c
Quoted 3 lines
app/test-eventdev/test_order_atq.c +2
-0+ ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the APIchanges
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
now and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecationnotice for
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.Hello Jerin,
Thank you for the review comments.
With regard to your comments regarding the fast path
flow_idchange, the
Quoted 3 lines
Intel
Quoted 2 lines
Quoted 2 lines
DLB hardware
Quoted 1 line
is not capable of transferring the
flow_idas part of the eventitself. We
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
therefore require a mechanism
Quoted 1 line
to accomplish this. What we have done to work around this is to
require the
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
application to embed the
flow_idQuoted 1 line
within the data payload. The new flag, #define
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID(1ULL << 9), can be usedQuoted 1 line
by applications to determine if they need to embed the
flow_id,or if its
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
automatically propagated and present in the
Quoted 3 lines
received event.
What we should have done is to wrap the assignment with a
conditional.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
if (!(device_capability_flags &
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID))Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
ev.flow_id = ev.mbuf->udata64;Two problems with this approach,
1) we are assuming mbuf udata64 field is available for DLB driver
2) It won't work with another adapter, eventdev has nodependency with mbuf
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
This snippet is not intended to suggest that udata64 always be used
to store the
Quoted 1 line
flow ID, but as an example of how an application could do it. Some
applications
Quoted 1 line
won’t need to carry the flow ID through; others can select an unused
field in the
Quoted 1 line
event data (e.g. hash.rss or udata64 if using mbufs), or (worst-case)
re-generate
Quoted 6 lines
the flow ID in pipeline stages that require it.
OK.
Quoted 3 lines
Quoted 3 lines
Question:
1) In the case of DLB hardware, ondequeue(), what HW returns? isit
Quoted 1 line
Quoted 1 line
Quoted 1 line
only event pointer and not have any other metadata like
schedule_typeQuoted 11 lines
Quoted 11 lines
Quoted 2 lines
etc.
The DLB device provides a 16B “queue entry” that consists of:
* 8B event data * Queue ID * Priority * Scheduling type * 19 bits of carried-through data * Assorted error/debug/reserved bits that are set by the device(not carried-
Quoted 3 lines
through)
Quoted 2 lines
For the carried-through 19b, we use 12b for
event_typeand
sub_event_type.Quoted 8 lines
I can only think of TWO options to help
1) Since event pointer always cache aligned, You could grab LSB 6bits(2^6 = 64B ) and 7 bits from (19b - 12b) carried through structure
2) Have separate mempool driver using existing drivers, ie "event pointer" + or - some offset have any amount of custom data.We can't guarantee that the event will contain a pointer -- it's possible that 8B is inline data (i.e. struct rte_event's u64 field).
It's really an application decision -- for example an app could allocate space in the 'mbuf private data' to store the flow ID, if the event device lacks that carry-flow-ID capability and the other mbuf fields can't be used for whatever reason.
We modified the tests, sample apps to show how this might be done, not necessarily how it must be done.Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
This would minimize/eliminate any performance impact due to
the
Quoted 4 lines
processor's
Quoted 1 line
Quoted 1 line
branch prediction logic.
I think, If we need to change common fastpath, better we need to
make
Quoted 8 lines
Quoted 6 lines
it template to create code for compile-time to have absolute zero overhead
and use runtime.
Seeapp/test-eventdev/test_order_atq.c: function:worker_wrapper()_create_ worker at compile time based on runtime capability.Yes, that would be perfect. Thanks for the example!
Just to add instead of having if and else using a jumptbl would be much cleaner Ex.
constpipeline_atq_worker_tpipeline_atq_worker_single_stage[2][2][2]= { [0][0] = pipeline_atq_worker_single_stage_fwd, [0][1] = pipeline_atq_worker_single_stage_tx, [1][0] = pipeline_atq_worker_single_stage_burst_fwd, [1][1] = pipeline_atq_worker_single_stage_burst_tx, }; return
Quoted 2 lines
(pipeline_atq_worker_single_stage[burst][internal_port])(arg);
Thank you for the suggestion.
Quoted 15 lines
Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
The assignment then becomes in essence a NOOP for all event
devices that
Quoted 10 lines
are
Quoted 9 lines
Quoted 9 lines
capable of carrying the
flow_idas part of the event payload itself.Quoted 8 lines
Thanks,
TimThanks,
Tim
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
In reply to McDaniel, TimothyQuoted 179 lines · Timothy
Quoted 169 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, June 30, 2020 10:58 AM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil Horman <[email protected]>; Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 105 lines · Timothy
Quoted 105 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, June 29, 2020 11:21 PM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil Horman <[email protected]>; Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 5 lines · Timothy
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Saturday, June 27, 2020 2:45 AM
To: McDaniel, Timothy <[email protected]>; Ray Kinsella<[email protected]>; Neil Horman <[email protected]>
Quoted 1 line
Cc Jerin Jacob <[email protected]>; Mattias Rönnblom<[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Quoted 15 lines
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 13 lines
(quoted diff) +13
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity is limited, + * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higher priority + * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the event dev + * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit*Quoted 8 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODEcapable.
Quoted 6 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+ */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODEcapable.
Quoted 52 lines
Quoted 5 lines
(quoted diff) +1
-0+ */ uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events received throughReview struct rte_event_port_conf { · line 920
Quoted 10 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);Hi Timothy,
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?app/test-eventdev/test_order_atq.c +2
-0@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the API changesnow and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Hello Jerin,
Thank you for the review comments.
With regard to your comments regarding the fast path
flow_idchange, theIntel
Quoted 16 lines
Quoted 16 lines
DLB hardware
Quoted 1 line
is not capable of transferring the
flow_idas part of the event itself. Wetherefore require a mechanism
Quoted 1 line
to accomplish this. What we have done to work around this is to require the
application to embed the
flow_idQuoted 1 line
within the data payload. The new flag, #define
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID(1ULL << 9), can be usedQuoted 1 line
by applications to determine if they need to embed the
flow_id, or if itsautomatically propagated and present in the
Quoted 6 lines
received event.
What we should have done is to wrap the assignment with a conditional.
if (!(device_capability_flags &
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID))ev.flow_id = ev.mbuf->udata64;Quoted 6 lines
Quoted 4 lines
Two problems with this approach,
1) we are assuming mbuf udata64 field is available for DLB driver
2) It won't work with another adapter, eventdev has no dependency with mbufThis snippet is not intended to suggest that udata64 always be used to store the
flow ID, but as an example of how an application could do it. Some applications won’t need to carry the flow ID through; others can select an unused field in the event data (e.g. hash.rss or udata64 if using mbufs), or (worst-case) re-generate the flow ID in pipeline stages that require it.
OK.
Quoted 7 lines
Quoted 6 lines
Question:
1) In the case of DLB hardware, ondequeue(), what HW returns? is it only event pointer and not have any other metadata likeschedule_typeetc.Quoted 8 lines
The DLB device provides a 16B “queue entry” that consists of:
* 8B event data * Queue ID * Priority * Scheduling type * 19 bits of carried-through data * Assorted error/debug/reserved bits that are set by the device (not carried-through)
Quoted 1 line
Quoted 9 lines
Quoted 1 line
For the carried-through 19b, we use 12b for
event_typeandsub_event_type.I can only think of TWO options to help
1) Since event pointer always cache aligned, You could grab LSB 6bits(2^6 = 64B ) and 7 bits from (19b - 12b) carried through structure
2) Have separate mempool driver using existing drivers, ie "event pointer" + or - some offset have any amount of custom data.
Quoted 4 lines
We can't guarantee that the event will contain a pointer -- it's possible that 8B is inline data (i.e. struct rte_event's u64 field).
It's really an application decision -- for example an app could allocate space in the 'mbuf private data' to store the flow ID, if the event device lacks that carry-flow-ID capability and the other mbuf fields can't be used for whatever reason. We modified the tests, sample apps to show how this might be done, not necessarily how it must be done.
Yeah. If HW has limitation we can't do much. It is OK to change eventdev spec to support new HW limitations. aka, RTE_EVENT_DEV_CAP_CARRY_FLOW_ID is OK.
Please update existing drivers has this RTE_EVENT_DEV_CAP_CARRY_FLOW_ID capability which is missing in the patch(I believe)
Quoted 17 lines
Quoted 7 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
This would minimize/eliminate any performance impact due to the
processor's
Quoted 9 lines
Quoted 1 line
Quoted 1 line
branch prediction logic.
I think, If we need to change common fastpath, better we need to make it template to create code for compile-time to have absolute zero overhead
and use runtime.
Seeapp/test-eventdev/test_order_atq.c: function:worker_wrapper()_create_ worker at compile time based on runtime capability.
Quoted 1 line
Yes, that would be perfect. Thanks for the example!
Where ever you are making fastpath change, Please follow this scheme and send the next version.
In order to have clean and reusable code, you could have template function and with "if" and it can opt-out in _compile_ time. i.e
no_inline generic_worker(..., _const_ uint64_t flags) {
..
..
if (! flags & CAP_CARRY_FLOW_ID)
....
}
worker_with_out_carry_flow_id()
{
generic_worker(.., CAP_CARRY_FLOW_ID)
}
normal_worker()
{
generic_worker(.., 0)
}No other controversial top-level comments with this patch series. Once we sorted out the ABI issues then I can review and merge.
Quoted 14 lines
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
The assignment then becomes in essence a NOOP for all event devices that
are
Quoted 9 lines
Quoted 9 lines
capable of carrying the
flow_idas part of the event payload itself.Quoted 8 lines
Thanks,
TimThanks,
Tim
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 272 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, June 30, 2020 11:50 PM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil Horman <[email protected]>; Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 11 lines · Timothy
Quoted 11 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, June 30, 2020 10:58 AM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil Horman <[email protected]>; Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 8 lines
Quoted 8 lines
Quoted 6 lines · Timothy
Quoted 6 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, June 29, 2020 11:21 PM
To: McDaniel, Timothy <[email protected]>
Cc: Ray Kinsella <[email protected]>; Neil HormanQuoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads,
Gage
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
<[email protected]>; Van Haaren, Harry
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream
prerequisites
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 5 lines · Timothy
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Saturday, June 27, 2020 2:45 AM
To: McDaniel, Timothy <[email protected]>; Ray Kinsella<[email protected]>; Neil Horman <[email protected]>
Quoted 1 line
Cc Jerin Jacob <[email protected]>; Mattias Rönnblom<[email protected]>; dpdk-dev <[email protected]>; Eads,
Gage
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
<[email protected]>; Van Haaren, Harry
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream
prerequisites
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Review
Quoted 6 lines
(quoted diff) +5
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on thisport.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ * Use for *closed system* event dev where event capacity is
limited,
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higherpriority
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the eventdev
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Review
Quoted 11 lines
(quoted diff) +11
-0+ * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit* + * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not
RTE_EVENT_DEV_CAP_BURST_MODEQuoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
capable.
Quoted 6 lines
Quoted 6 lines
(quoted diff) +6
-0+ */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not
RTE_EVENT_DEV_CAP_BURST_MODEQuoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
capable.
Quoted 4 lines
Quoted 4 lines
+ */
uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events receivedthrough
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Review struct rte_event_port_conf { · line 920
Quoted 11 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);Hi Timothy,
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"# We have a similar case with ethdev and it deferred to next release
v20.11
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
http://patches.dpdk.org/patch/69113/
Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?app/test-eventdev/test_order_atq.c +2
-0@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the API changesnow and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Hello Jerin,
Thank you for the review comments.
With regard to your comments regarding the fast path
flow_idchange,the
Quoted 5 lines
Quoted 5 lines
Intel
Quoted 4 lines
Quoted 4 lines
DLB hardware
Quoted 1 line
is not capable of transferring the
flow_idas part of the event itself. Wetherefore require a mechanism
Quoted 1 line
to accomplish this. What we have done to work around this is to require
the
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
application to embed the
flow_idQuoted 1 line
within the data payload. The new flag, #define
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID(1ULL << 9), can be usedQuoted 1 line
by applications to determine if they need to embed the
flow_id, or if itsautomatically propagated and present in the
Quoted 2 lines
received event.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
What we should have done is to wrap the assignment with a conditional.
if (!(device_capability_flags &
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID))ev.flow_id = ev.mbuf->udata64;Two problems with this approach,
1) we are assuming mbuf udata64 field is available for DLB driver
2) It won't work with another adapter, eventdev has no dependency withmbuf
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
This snippet is not intended to suggest that udata64 always be used to store
the
Quoted 1 line
Quoted 1 line
flow ID, but as an example of how an application could do it. Some
applications
Quoted 1 line
Quoted 1 line
won’t need to carry the flow ID through; others can select an unused field in
the
Quoted 1 line
Quoted 1 line
event data (e.g. hash.rss or udata64 if using mbufs), or (worst-case) re-
generate
Quoted 10 lines
Quoted 10 lines
the flow ID in pipeline stages that require it.
OK.
Quoted 7 lines
Quoted 6 lines
Question:
1) In the case of DLB hardware, ondequeue(), what HW returns? is it only event pointer and not have any other metadata likeschedule_typeetc.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
The DLB device provides a 16B “queue entry” that consists of:
* 8B event data * Queue ID * Priority * Scheduling type * 19 bits of carried-through data * Assorted error/debug/reserved bits that are set by the device (notcarried-
Quoted 3 lines
Quoted 3 lines
through)
Quoted 2 lines
For the carried-through 19b, we use 12b for
event_typeand
sub_event_type.Quoted 10 lines
Quoted 8 lines
I can only think of TWO options to help
1) Since event pointer always cache aligned, You could grab LSB 6bits(2^6 = 64B ) and 7 bits from (19b - 12b) carried through structure
2) Have separate mempool driver using existing drivers, ie "event pointer" + or - some offset have any amount of custom data.We can't guarantee that the event will contain a pointer -- it's possible that 8B
is inline data (i.e. struct rte_event's u64 field).
Quoted 2 lines
It's really an application decision -- for example an app could allocate space in
the 'mbuf private data' to store the flow ID, if the event device lacks that carry- flow-ID capability and the other mbuf fields can't be used for whatever reason.
Quoted 1 line
We modified the tests, sample apps to show how this might be done, not
necessarily how it must be done.
Yeah. If HW has limitation we can't do much. It is OK to change eventdev spec to support new HW limitations. aka,
RTE_EVENT_DEV_CAP_CARRY_FLOW_IDis OK.
Please update existing drivers has thisRTE_EVENT_DEV_CAP_CARRY_FLOW_IDcapability which is missing in the patch(I believe)Quoted 17 lines
Quoted 7 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
This would minimize/eliminate any performance impact due to the
processor's
Quoted 9 lines
Quoted 1 line
Quoted 1 line
branch prediction logic.
I think, If we need to change common fastpath, better we need to make it template to create code for compile-time to have absolute zero overhead
and use runtime.
Seeapp/test-eventdev/test_order_atq.c: function:worker_wrapper()_create_ worker at compile time based on runtime capability.Quoted 1 line
Yes, that would be perfect. Thanks for the example!
Where ever you are making fastpath change, Please follow this scheme and send the next version.
In order to have clean and reusable code, you could have template function and with "if" and it can opt-out in _compile_ time. i.e
no_inlinegeneric_worker(..., _const_ uint64_t flags){
..
..if (! flags &
CAP_CARRY_FLOW_ID)
....}
worker_with_out_carry_flow_id()
{
generic_worker(.., CAP_CARRY_FLOW_ID)
}
Quoted 8 lines
normal_worker(){ generic_worker(.., 0) }No other controversial top-level comments with this patch series. Once we sorted out the ABI issues then I can review and merge.
Thanks Jerin. I'll get these changes into the v3 patch set.
Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
The assignment then becomes in essence a NOOP for all event devices
that
Quoted 10 lines
Quoted 10 lines
are
Quoted 9 lines
Quoted 9 lines
capable of carrying the
flow_idas part of the event payload itself.Quoted 8 lines
Thanks,
TimThanks,
Tim
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
In reply to Jerin JacobQuoted 45 lines · Jerin Jacob
Quoted 32 lines
(quoted diff) +28
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity is limited, + * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higher priority + * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the event dev + * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit* + * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events received throughReview struct rte_event_port_conf { · line 920
Quoted 10 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);Hi Timothy,
Quoted 5 lines
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"
So the answer is (as always): depends
If the structure is being use in inline functions is when you run into trouble
- as knowledge of the structure is embedded in the linked application.
However if the structure is _strictly_ being used as a non-inlined function parameter, It can be safe to version in this way.
So just to be clear, it is still the function that is actually being versioned here.
Quoted 3 lines
# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/
Yes - I spent a why looking at this one, but I am struggling to recall,
why when I looked it we didn't suggest function versioning as a potential solution in this case.
Looking back at it now, looks like it would have been ok.
Quoted 26 lines
Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?app/test-eventdev/test_order_atq.c +2
-0@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the API changesnow and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 55 lines · Ray
Quoted 45 lines · Jerin Jacob
Quoted 32 lines
(quoted diff) +28
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity is limited, + * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higher priority + * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the event dev + * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit* + * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events received throughReview struct rte_event_port_conf { · line 920
Quoted 10 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);Hi Timothy,
Quoted 5 lines
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"
Quoted 7 lines
So the answer is (as always): depends
If the structure is being use in inline functions is when you run into trouble
- as knowledge of the structure is embedded in the linked application.However if the structure is _strictly_ being used as a non-inlined function parameter, It can be safe to version in this way.
But based on the optimization applied when building the consumer code matters. Right?
i.e compiler can "inline" it, based on the optimization even the source code explicitly mentions it.
Quoted 11 lines
So just to be clear, it is still the function that is actually being versioned here.
Quoted 3 lines
# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/Yes - I spent a why looking at this one, but I am struggling to recall,
why when I looked it we didn't suggest function versioning as a potential solution in this case.Looking back at it now, looks like it would have been ok.
Ok.
Quoted 27 lines
Quoted 27 lines
Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?app/test-eventdev/test_order_atq.c +2
-0@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the API changesnow and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 61 lines · Jerin Jacob
Quoted 60 lines · Ray
Quoted 45 lines · Jerin Jacob
Quoted 32 lines
(quoted diff) +28
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity is limited, + * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higher priority + * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the event dev + * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit* + * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events received throughReview struct rte_event_port_conf { · line 920
Quoted 10 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);Hi Timothy,
Quoted 5 lines
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"So the answer is (as always): depends
If the structure is being use in inline functions is when you run into trouble
- as knowledge of the structure is embedded in the linked application.
Quoted 7 lines
Quoted 2 lines
However if the structure is _strictly_ being used as a non-inlined function parameter, It can be safe to version in this way.
But based on the optimization applied when building the consumer code matters. Right?
i.e compiler can "inline" it, based on the optimization even the source code explicitly mentions it.
Well a compiler will typically only inline within the confines of a given object file, or binary, if LTO is enabled.
If a function symbol is exported from library however, it won't be inlined in a linked application. The compiler doesn't have enough information to inline it.
All the compiler will know about it is it's offset in memory, and it's signature.
Quoted 43 lines
Quoted 6 lines
So just to be clear, it is still the function that is actually being versioned here.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/Yes - I spent a why looking at this one, but I am struggling to recall,
why when I looked it we didn't suggest function versioning as a potential solution in this case.Looking back at it now, looks like it would have been ok.
Ok.
Quoted 27 lines
Quoted 27 lines
Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?app/test-eventdev/test_order_atq.c +2
-0@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the API changesnow and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 73 lines · Ray
Quoted 61 lines · Jerin Jacob
Quoted 60 lines · Ray
Quoted 45 lines · Jerin Jacob
Quoted 32 lines
(quoted diff) +28
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity is limited, + * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higher priority + * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the event dev + * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit* + * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events received throughReview struct rte_event_port_conf { · line 920
Quoted 10 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);Hi Timothy,
Quoted 5 lines
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"So the answer is (as always): depends
If the structure is being use in inline functions is when you run into trouble
- as knowledge of the structure is embedded in the linked application.Quoted 7 lines
Quoted 2 lines
However if the structure is _strictly_ being used as a non-inlined function parameter, It can be safe to version in this way.
But based on the optimization applied when building the consumer code matters. Right?
i.e compiler can "inline" it, based on the optimization even the source code explicitly mentions it.
Quoted 5 lines
Well a compiler will typically only inline within the confines of a given object file, or binary, if LTO is enabled.
If a function symbol is exported from library however, it won't be inlined in a linked application.
Yes, With respect to that function.
But the application can use struct rte_event_port_conf in their code and it can be part of other structures.
Right?
Quoted 46 lines
The compiler doesn't have enough information to inline it.
All the compiler will know about it is it's offset in memory, and it's signature.Quoted 43 lines
Quoted 6 lines
So just to be clear, it is still the function that is actually being versioned here.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/Yes - I spent a why looking at this one, but I am struggling to recall,
why when I looked it we didn't suggest function versioning as a potential solution in this case.Looking back at it now, looks like it would have been ok.
Ok.
Quoted 27 lines
Quoted 27 lines
Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?app/test-eventdev/test_order_atq.c +2
-0@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the API changesnow and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 78 lines · Jerin Jacob
Quoted 77 lines · Ray
Quoted 61 lines · Jerin Jacob
Quoted 60 lines · Ray
Quoted 45 lines · Jerin Jacob
Quoted 32 lines
(quoted diff) +28
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity is limited, + * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higher priority + * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the event dev + * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit* + * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODE capable. + */ uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events received throughReview struct rte_event_port_conf { · line 920
Quoted 10 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);Hi Timothy,
Quoted 5 lines
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"So the answer is (as always): depends
If the structure is being use in inline functions is when you run into trouble
- as knowledge of the structure is embedded in the linked application.Quoted 7 lines
Quoted 2 lines
However if the structure is _strictly_ being used as a non-inlined function parameter, It can be safe to version in this way.
But based on the optimization applied when building the consumer code matters. Right?
i.e compiler can "inline" it, based on the optimization even the source code explicitly mentions it.Well a compiler will typically only inline within the confines of a given object file, or binary, if LTO is enabled.
Quoted 6 lines
Quoted 1 line
If a function symbol is exported from library however, it won't be inlined in a linked application.
Yes, With respect to that function.
But the application can use structrte_event_port_confin their code and it can be part of other structures.
Right?
Tim, it looks like you might be inadvertently breaking other symbols also. For example ...
int
rte_event_crypto_adapter_create(uint8_t id, uint8_t dev_id,
struct rte_event_port_conf *port_config, enum rte_event_crypto_adapter_mode mode);
int
rte_event_port_setup(uint8_t dev_id, uint8_t port_id,
const struct rte_event_port_conf *port_conf);
These and others symbols are also using rte_event_port_conf and would need to be updated to use the v20 struct, if we aren't to break them .
Quoted 48 lines
Quoted 48 lines
The compiler doesn't have enough information to inline it.
All the compiler will know about it is it's offset in memory, and it's signature.Quoted 43 lines
Quoted 6 lines
So just to be clear, it is still the function that is actually being versioned here.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/Yes - I spent a why looking at this one, but I am struggling to recall,
why when I looked it we didn't suggest function versioning as a potential solution in this case.Looking back at it now, looks like it would have been ok.
Ok.
Quoted 27 lines
Quoted 27 lines
Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?app/test-eventdev/test_order_atq.c +2
-0@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the API changesnow and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.
Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Quoted 115 lines
-----Original Message-----
From: Kinsella, Ray <[email protected]>
Sent: Thursday, July 2, 2020 10:21 AM
To: Jerin Jacob <[email protected]>
Cc: McDaniel, Timothy <[email protected]>; Neil Horman <[email protected]>; Jerin Jacob <[email protected]>; Mattias Rönnblom <[email protected]>; dpdk-dev <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisitesQuoted 9 lines · Jerin Jacob
Quoted 8 lines · Ray
Quoted 4 lines · Jerin Jacob
Quoted 3 lines · Ray
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines · Jerin Jacob
Quoted 6 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+ +/** Event port configuration structure */ +struct rte_event_port_conf_v20 { + int32_t new_event_threshold; + /**< A backpressure threshold for new event enqueues on this port. + * Use for *closed system* event dev where event capacity islimited,
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * and cannot exceed the capacity of the event dev. + * Configuring ports with different thresholds can make higherpriority
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ * traffic less likely to be backpressured. + * For example, a port used to inject NIC Rx packets into the eventdev
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Review
Quoted 11 lines
(quoted diff) +11
-0+ * can have a lower threshold so as not to overwhelm the device, + * while ports used for worker pools can have a higher threshold. + * This value cannot exceed the *nb_events_limit* + * which was previously supplied to rte_event_dev_configure(). + * This should be set to '-1' for *open system*. + */ + uint16_t dequeue_depth; + /**< Configure number of bulk dequeues for this event port. + * This value cannot exceed the *nb_event_port_dequeue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODEcapable.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+ */ + uint16_t enqueue_depth; + /**< Configure number of bulk enqueues for this event port. + * This value cannot exceed the *nb_event_port_enqueue_depth* + * which previously supplied to rte_event_dev_configure(). + * Ignored when device is not RTE_EVENT_DEV_CAP_BURST_MODEcapable.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
+ */
uint8_t disable_implicit_release; /**< Configure the port not to release outstanding events in * rte_event_dev_dequeue_burst(). If true, all events receivedthrough
Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
Review struct rte_event_port_conf { · line 920
Quoted 11 lines
(quoted diff) +7
-0@@ -733,6 +911,14 @@ struct rte_event_port_conf { rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, struct rte_event_port_conf *port_conf); +int +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf_v20 *port_conf); + +int +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, + struct rte_event_port_conf *port_conf);Hi Timothy,
+ ABI Maintainers (Ray, Neil)
# As per my understanding, the structures can not be versioned, only function can be versioned.
i.e we can not make any change to " structrte_event_port_conf"Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
So the answer is (as always): depends
If the structure is being use in inline functions is when you run into trouble
- as knowledge of the structure is embedded in the linked application.However if the structure is _strictly_ being used as a non-inlined function
parameter,
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
It can be safe to version in this way.
But based on the optimization applied when building the consumer code matters. Right?
i.e compiler can "inline" it, based on the optimization even the source code explicitly mentions it.Well a compiler will typically only inline within the confines of a given object
file, or
Quoted 4 lines
Quoted 4 lines
binary, if LTO is enabled.
If a function symbol is exported from library however, it won't be inlined in a
linked application.
Quoted 5 lines
Yes, With respect to that function.
But the application can use structrte_event_port_confin their code and it can be part of other structures.
Right?Tim, it looks like you might be inadvertently breaking other symbols also. For example ...
int
rte_event_crypto_adapter_create(uint8_t id,uint8_tdev_id,
structrte_event_port_conf*port_config, enumrte_event_crypto_adapter_modemode);
Quoted 8 lines
int
rte_event_port_setup(uint8_tdev_id,uint8_tport_id,const struct rte_event_port_conf *port_conf);These and others symbols are also using
rte_event_port_confand would need to be updated to use the v20 struct,
if we aren't to break them .
Yes, we just discovered that after successfully running the ABI checker. I will address those in the v3 patch set. Thanks.
Quoted 50 lines
Quoted 9 lines
Quoted 9 lines
The compiler doesn't have enough information to inline it.
All the compiler will know about it is it's offset in memory, and it's signature.Quoted 4 lines
Quoted 4 lines
So just to be clear, it is still the function that is actually being versioned
here.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
# We have a similar case with ethdev and it deferred to next release v20.11
http://patches.dpdk.org/patch/69113/Yes - I spent a why looking at this one, but I am struggling to recall, why when I looked it we didn't suggest function versioning as a potential
solution in this case.
Quoted 32 lines
Quoted 32 lines
Quoted 32 lines
Quoted 2 lines
Looking back at it now, looks like it would have been ok.
Ok.
Quoted 27 lines
Quoted 27 lines
Regarding the API changes:
# The slow path changes general looks good to me. I will review the next level in the coming days
# The following fast path changes bothers to me. Could you share more details on below change?app/test-eventdev/test_order_atq.c +2
-0@@ -34,6 +34,8 @@continue ; } + ev.flow_id = ev.mbuf->udata64; +# Since RC1 is near, I am not sure how to accommodate the API changesnow and sort out ABI stuffs.
# Other concern is eventdev spec get bloated with versioning files just for ONE release as 20.11 will be OK to change the ABI. # While we discuss the API change, Please send deprecation notice for ABI change for 20.11,
so that there is no ambiguity of this patch for the 20.11 release.
PATCH 2/27 eventdev: do not pass disable_implicit_release bit to trace macro
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
Previous implementation traced u8 disable_impl_release, which is now a bit field contained in the new u32 event_port_cfg field.
--- lib/librte_eventdev/rte_event_crypto_adapter.c | 2 +- lib/librte_eventdev/rte_event_eth_tx_adapter.c | 5 +++-- lib/librte_eventdev/rte_eventdev.c | 3 +-- lib/librte_eventdev/rte_eventdev_trace.h | 9 +++++---- 4 files changed, 10 insertions(+), 9 deletions(-)
lib/librte_eventdev/rte_event_crypto_adapter.c +1 -1
@@ -269,7 +269,7 @@ struct crypto_queue_pair_info { event_crypto_adapter[id] = adapter;rte_eventdev_trace_crypto_adapter_create (id, dev_id, adapter, conf_arg, - mode); + mode);return 0 ; }
lib/librte_eventdev/rte_event_eth_tx_adapter.c +3 -2
@@ -943,8 +943,9 @@ static int txa_service_queue_del(uint8_t id, txa_dev_id_array[id] =TXA_INVALID_DEV_ID ;return ret; } +rte_eventdev_trace_eth_tx_adapter_create (id, dev_id,NULL , port_conf, - ret); + ret); txa_dev_id_array[id] = dev_id;return 0 ; }
@@ -987,7 +988,7 @@ static int txa_service_queue_del(uint8_t id, }rte_eventdev_trace_eth_tx_adapter_create (id, dev_id, conf_cb, conf_arg, - ret); + ret); txa_dev_id_array[id] = dev_id;return 0 ; }
lib/librte_eventdev/rte_eventdev.c +1 -2
@@ -937,6 +937,7 @@ diag =rte_event_port_unlink (dev_id, port_id,NULL ,0 );rte_eventdev_trace_port_setup (dev_id, port_id, port_conf, diag); +if (diag <0 )return diag;
@@ -1550,10 +1551,8 @@ struct rte_eventdev * eventdev->data = eventdev_data;if (rte_eal_process_type () ==RTE_PROC_PRIMARY ) { -strlcpy (eventdev->data->name, name,RTE_EVENTDEV_NAME_MAX_LEN ); - eventdev->data->dev_id = dev_id; eventdev->data->socket_id = socket_id; eventdev->data->dev_started =0 ;
lib/librte_eventdev/rte_eventdev_trace.h +5 -4
@@ -53,13 +53,14 @@RTE_TRACE_POINT ( rte_eventdev_trace_port_setup, RTE_TRACE_POINT_ARGS(uint8_t dev_id, uint8_t port_id, - const struct rte_event_port_conf *port_conf, int rc), + const struct rte_event_port_conf *port_conf, + int rc), rte_trace_point_emit_u8(dev_id);rte_trace_point_emit_u8 (port_id);rte_trace_point_emit_i32 (port_conf->new_event_threshold);rte_trace_point_emit_u16 (port_conf->dequeue_depth);rte_trace_point_emit_u16 (port_conf->enqueue_depth); -rte_trace_point_emit_u8 (port_conf->disable_implicit_release); +rte_trace_point_emit_u32 (port_conf->event_port_cfg);rte_trace_point_emit_int (rc); )
@@ -165,7 +166,7 @@rte_trace_point_emit_i32 (port_conf->new_event_threshold);rte_trace_point_emit_u16 (port_conf->dequeue_depth);rte_trace_point_emit_u16 (port_conf->enqueue_depth); -rte_trace_point_emit_u8 (port_conf->disable_implicit_release); +rte_trace_point_emit_u32 (port_conf->event_port_cfg);rte_trace_point_emit_ptr (conf_cb);rte_trace_point_emit_int (rc); )
@@ -257,7 +258,7 @@rte_trace_point_emit_i32 (port_conf->new_event_threshold);rte_trace_point_emit_u16 (port_conf->dequeue_depth);rte_trace_point_emit_u16 (port_conf->enqueue_depth); -rte_trace_point_emit_u8 (port_conf->disable_implicit_release); +rte_trace_point_emit_u32 (port_conf->event_port_cfg); ) RTE_TRACE_POINT(
Signature · 1 line
1.7.10
PATCH 3/27 event/dlb: add shared code version 10.7.9
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
The DLB shared code is auto generated by Intel, and is being committed here so that it can be built in the DPDK environment. The shared code should not be modified. The shared code must be present in order to successfully build the DLB PMD.
Changes since v1 patch series
1) convert C99 comment to standard C
2) remove TODO and FIXME comments
--- drivers/event/dlb/pf/base/dlb_hw_types.h | 360 + drivers/event/dlb/pf/base/dlb_mbox.h | 645 ++ drivers/event/dlb/pf/base/dlb_osdep.h | 348 + drivers/event/dlb/pf/base/dlb_osdep_bitmap.h | 442 ++ drivers/event/dlb/pf/base/dlb_osdep_list.h | 131 + drivers/event/dlb/pf/base/dlb_osdep_types.h | 31 + drivers/event/dlb/pf/base/dlb_regs.h | 2646 +++++++ drivers/event/dlb/pf/base/dlb_resource.c | 9700 ++++++++++++++++++++++++++ drivers/event/dlb/pf/base/dlb_resource.h | 1625 +++++ drivers/event/dlb/pf/base/dlb_user.h | 1084 +++ 10 files changed, 17012 insertions(+) create mode 100644 drivers/event/dlb/pf/base/dlb_hw_types.h create mode 100644 drivers/event/dlb/pf/base/dlb_mbox.h create mode 100644 drivers/event/dlb/pf/base/dlb_osdep.h create mode 100644 drivers/event/dlb/pf/base/dlb_osdep_bitmap.h create mode 100644 drivers/event/dlb/pf/base/dlb_osdep_list.h create mode 100644 drivers/event/dlb/pf/base/dlb_osdep_types.h create mode 100644 drivers/event/dlb/pf/base/dlb_regs.h create mode 100644 drivers/event/dlb/pf/base/dlb_resource.c create mode 100644 drivers/event/dlb/pf/base/dlb_resource.h create mode 100644 drivers/event/dlb/pf/base/dlb_user.h
drivers/event/dlb/pf/base/dlb_hw_types.h new file +360 -0
@@ -0,0 +1,360 @@ +/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-3-Clause) +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef __DLB_HW_TYPES_H +#define __DLB_HW_TYPES_H + +#include "dlb_user.h" +#include "dlb_osdep_types.h" +#include "dlb_osdep_list.h" + +#define DLB_MAX_NUM_VFS 16 +#define DLB_MAX_NUM_DOMAINS 32 +#define DLB_MAX_NUM_LDB_QUEUES 128 +#define DLB_MAX_NUM_LDB_PORTS 64 +#define DLB_MAX_NUM_DIR_PORTS 128 +#define DLB_MAX_NUM_LDB_CREDITS 16384 +#define DLB_MAX_NUM_DIR_CREDITS 4096 +#define DLB_MAX_NUM_LDB_CREDIT_POOLS 64 +#define DLB_MAX_NUM_DIR_CREDIT_POOLS 64 +#define DLB_MAX_NUM_HIST_LIST_ENTRIES 5120 +#define DLB_MAX_NUM_AQOS_ENTRIES 2048 +#define DLB_MAX_NUM_TOTAL_OUTSTANDING_COMPLETIONS 4096 +#define DLB_MAX_NUM_QIDS_PER_LDB_CQ 8 +#define DLB_MAX_NUM_SEQUENCE_NUMBER_GROUPS 4 +#define DLB_MAX_NUM_SEQUENCE_NUMBER_MODES 6 +#define DLB_QID_PRIORITIES 8 +#define DLB_NUM_ARB_WEIGHTS 8 +#define DLB_MAX_WEIGHT 255 +#define DLB_MAX_PORT_CREDIT_QUANTUM 1023 +#define DLB_MAX_CQ_COMP_CHECK_LOOPS 409600 +#define DLB_MAX_QID_EMPTY_CHECK_LOOPS (32 * 64 * 1024 * (800 / 30)) +#define DLB_HZ 800000000 + +/* Used for DLB A-stepping workaround for hardware write buffer lock up issue */ +#define DLB_A_STEP_MAX_PORTS 128 + +#define DLB_PF_DEV_ID 0x270B +#define DLB_VF_DEV_ID 0x270C + +/* Interrupt related macros */ +#define DLB_PF_NUM_NON_CQ_INTERRUPT_VECTORS 8 +#define DLB_PF_NUM_CQ_INTERRUPT_VECTORS 64 +#define DLB_PF_TOTAL_NUM_INTERRUPT_VECTORS \ + (DLB_PF_NUM_NON_CQ_INTERRUPT_VECTORS + \ + DLB_PF_NUM_CQ_INTERRUPT_VECTORS) +#define DLB_PF_NUM_COMPRESSED_MODE_VECTORS \ + (DLB_PF_NUM_NON_CQ_INTERRUPT_VECTORS + 1) +#define DLB_PF_NUM_PACKED_MODE_VECTORS DLB_PF_TOTAL_NUM_INTERRUPT_VECTORS +#define DLB_PF_COMPRESSED_MODE_CQ_VECTOR_ID DLB_PF_NUM_NON_CQ_INTERRUPT_VECTORS + +#define DLB_VF_NUM_NON_CQ_INTERRUPT_VECTORS 1 +#define DLB_VF_NUM_CQ_INTERRUPT_VECTORS 31 +#define DLB_VF_BASE_CQ_VECTOR_ID 0 +#define DLB_VF_LAST_CQ_VECTOR_ID 30 +#define DLB_VF_MBOX_VECTOR_ID 31 +#define DLB_VF_TOTAL_NUM_INTERRUPT_VECTORS \ + (DLB_VF_NUM_NON_CQ_INTERRUPT_VECTORS + \ + DLB_VF_NUM_CQ_INTERRUPT_VECTORS) + +#define DLB_PF_NUM_ALARM_INTERRUPT_VECTORS 4 +/* DLB ALARM interrupts */ +#define DLB_INT_ALARM 0 +/* VF to PF Mailbox Service Request */ +#define DLB_INT_VF_TO_PF_MBOX 1 +/* HCW Ingress Errors */ +#define DLB_INT_INGRESS_ERROR 3 + +#define DLB_ALARM_HW_SOURCE_SYS 0 +#define DLB_ALARM_HW_SOURCE_DLB 1 + +#define DLB_ALARM_HW_UNIT_CHP 1 +#define DLB_ALARM_HW_UNIT_LSP 3 + +#define DLB_ALARM_HW_CHP_AID_OUT_OF_CREDITS 6 +#define DLB_ALARM_HW_CHP_AID_ILLEGAL_ENQ 7 +#define DLB_ALARM_HW_LSP_AID_EXCESS_TOKEN_POPS 15 +#define DLB_ALARM_SYS_AID_ILLEGAL_HCW 0 +#define DLB_ALARM_SYS_AID_ILLEGAL_QID 3 +#define DLB_ALARM_SYS_AID_DISABLED_QID 4 +#define DLB_ALARM_SYS_AID_ILLEGAL_CQID 6 + +/* Hardware-defined base addresses */ +#define DLB_LDB_PP_BASE 0x2100000 +#define DLB_LDB_PP_STRIDE 0x1000 +#define DLB_LDB_PP_BOUND \ + (DLB_LDB_PP_BASE + DLB_LDB_PP_STRIDE * DLB_MAX_NUM_LDB_PORTS) +#define DLB_DIR_PP_BASE 0x2000000 +#define DLB_DIR_PP_STRIDE 0x1000 +#define DLB_DIR_PP_BOUND \ + (DLB_DIR_PP_BASE + DLB_DIR_PP_STRIDE * DLB_MAX_NUM_DIR_PORTS) + +struct dlb_resource_id { +u32 phys_id; +u32 virt_id; +u8 vf_owned; +u8 vf_id; +}; + +struct dlb_freelist { +u32 base; +u32 bound; +u32 offset; +}; + +static inline u32 dlb_freelist_count (struct dlb_freelist *list) +{ +return (list->bound - list->base) - list->offset; +} + +struct dlb_hcw { +u64 data; +/* Word 3 */ +u16 opaque; +u8 qid; +u8 sched_type:2 ; +u8 priority:3 ; +u8 msg_type:3 ; +/* Word 4 */ +u16 lock_id; +u8 meas_lat:1 ; +u8 rsvd1:2 ; +u8 no_dec:1 ; +u8 cmp_id:4 ; +u8 cq_token:1 ; +u8 qe_comp:1 ; +u8 qe_frag:1 ; +u8 qe_valid:1 ; +u8 int_arm:1 ; +u8 error:1 ; +u8 rsvd:2 ; +}; + +struct dlb_ldb_queue { +struct dlb_list_entry domain_list; +struct dlb_list_entry func_list; +struct dlb_resource_id id; +struct dlb_resource_id domain_id; +u32 num_qid_inflights; +struct dlb_freelist aqed_freelist; +u8 sn_cfg_valid; +u32 sn_group; +u32 sn_slot; +u32 num_mappings; +u8 num_pending_additions; +u8 owned; +u8 configured; +}; + +/* Directed ports and queues are paired by nature, so the driver tracks them +* with a single data structure. +*/ +struct dlb_dir_pq_pair { +struct dlb_list_entry domain_list; +struct dlb_list_entry func_list; +struct dlb_resource_id id; +struct dlb_resource_id domain_id; +u8 ldb_pool_used; +u8 dir_pool_used; +u8 queue_configured; +u8 port_configured; +u8 owned; +u8 enabled; +u32 ref_cnt; +}; + +enum dlb_qid_map_state { +/* The slot doesn't contain a valid queue mapping */ +DLB_QUEUE_UNMAPPED , +/* The slot contains a valid queue mapping */ +DLB_QUEUE_MAPPED , +/* The driver is mapping a queue into this slot */ +DLB_QUEUE_MAP_IN_PROGRESS , +/* The driver is unmapping a queue from this slot */ +DLB_QUEUE_UNMAP_IN_PROGRESS , +/* The driver is unmapping a queue from this slot, and once complete +* will replace it with another mapping. +*/ +DLB_QUEUE_UNMAP_IN_PROGRESS_PENDING_MAP , +}; + +struct dlb_ldb_port_qid_map { +u16 qid; +u8 priority; +u16 pending_qid; +u8 pending_priority; +enum dlb_qid_map_state state; +}; + +struct dlb_ldb_port { +struct dlb_list_entry domain_list; +struct dlb_list_entry func_list; +struct dlb_resource_id id; +struct dlb_resource_id domain_id; +u8 ldb_pool_used; +u8 dir_pool_used; +u8 init_tkn_cnt; +u32 hist_list_entry_base; +u32 hist_list_entry_limit; +/* The qid_map represents the hardware QID mapping state. */ +struct dlb_ldb_port_qid_map qid_map[DLB_MAX_NUM_QIDS_PER_LDB_CQ ]; +u32 ref_cnt; +u8 num_pending_removals; +u8 num_mappings; +u8 owned; +u8 enabled; +u8 configured; +}; + +struct dlb_credit_pool { +struct dlb_list_entry domain_list; +struct dlb_list_entry func_list; +struct dlb_resource_id id; +struct dlb_resource_id domain_id; +u32 total_credits; +u32 avail_credits; +u8 owned; +u8 configured; +}; + +struct dlb_sn_group { +u32 mode; +u32 sequence_numbers_per_queue; +u32 slot_use_bitmap; +u32 id; +}; + +static inline bool dlb_sn_group_full (struct dlb_sn_group *group) +{ +u32 mask[6 ] = { +0xffffffff ,/* 32 SNs per queue */ +0x0000ffff ,/* 64 SNs per queue */ +0x000000ff ,/* 128 SNs per queue */ +0x0000000f ,/* 256 SNs per queue */ +0x00000003 ,/* 512 SNs per queue */ +0x00000001 };/* 1024 SNs per queue */ + +return group->slot_use_bitmap == mask[group->mode]; +} + +static inline int dlb_sn_group_alloc_slot (struct dlb_sn_group *group) +{ +int bound[6 ] = {32 ,16 ,8 ,4 ,2 ,1 }; +int i; + +for (i =0 ; i < bound[group->mode]; i++) { +if (!(group->slot_use_bitmap & (1 << i))) { + group->slot_use_bitmap |=1 << i; +return i; + } + } + +return -1 ; +} + +static inline void dlb_sn_group_free_slot (struct dlb_sn_group *group,int slot) +{ + group->slot_use_bitmap &= ~(1 << slot); +} + +static inline int dlb_sn_group_used_slots (struct dlb_sn_group *group) +{ +int i, cnt =0 ; + +for (i =0 ; i <32 ; i++) + cnt += !!(group->slot_use_bitmap & (1 << i)); + +return cnt; +} + +struct dlb_domain { +struct dlb_function_resources *parent_func; +struct dlb_list_entry func_list; +struct dlb_list_head used_ldb_queues; +struct dlb_list_head used_ldb_ports; +struct dlb_list_head used_dir_pq_pairs; +struct dlb_list_head used_ldb_credit_pools; +struct dlb_list_head used_dir_credit_pools; +struct dlb_list_head avail_ldb_queues; +struct dlb_list_head avail_ldb_ports; +struct dlb_list_head avail_dir_pq_pairs; +struct dlb_list_head avail_ldb_credit_pools; +struct dlb_list_head avail_dir_credit_pools; +u32 total_hist_list_entries; +u32 avail_hist_list_entries; +u32 hist_list_entry_base; +u32 hist_list_entry_offset; +struct dlb_freelist qed_freelist; +struct dlb_freelist dqed_freelist; +struct dlb_freelist aqed_freelist; +struct dlb_resource_id id; +int num_pending_removals; +int num_pending_additions; +u8 configured; +u8 started; +}; + +struct dlb_bitmap ; + +struct dlb_function_resources { +u32 num_avail_domains; +struct dlb_list_head avail_domains; +struct dlb_list_head used_domains; +u32 num_avail_ldb_queues; +struct dlb_list_head avail_ldb_queues; +u32 num_avail_ldb_ports; +struct dlb_list_head avail_ldb_ports; +u32 num_avail_dir_pq_pairs; +struct dlb_list_head avail_dir_pq_pairs; +struct dlb_bitmap *avail_hist_list_entries; +struct dlb_bitmap *avail_qed_freelist_entries; +struct dlb_bitmap *avail_dqed_freelist_entries; +struct dlb_bitmap *avail_aqed_freelist_entries; +u32 num_avail_ldb_credit_pools; +struct dlb_list_head avail_ldb_credit_pools; +u32 num_avail_dir_credit_pools; +struct dlb_list_head avail_dir_credit_pools; +u32 num_enabled_ldb_ports;/* (PF only) */ +u8 locked;/* (VF only) */ +}; + +/* After initialization, each resource in dlb_hw_resources is located in one of +* the following lists: +* -- The PF's available resources list. These are unconfigured resources owned +* by the PF and not allocated to a DLB scheduling domain. +* -- A VF's available resources list. These are VF-owned unconfigured +* resources not allocated to a DLB scheduling domain. +* -- A domain's available resources list. These are domain-owned unconfigured +* resources. +* -- A domain's used resources list. These are are domain-owned configured +* resources. +* +* A resource moves to a new list when a VF or domain is created or destroyed, +* or when the resource is configured. +*/ +struct dlb_hw_resources { +struct dlb_ldb_queue ldb_queues[DLB_MAX_NUM_LDB_QUEUES ]; +struct dlb_ldb_port ldb_ports[DLB_MAX_NUM_LDB_PORTS ]; +struct dlb_dir_pq_pair dir_pq_pairs[DLB_MAX_NUM_DIR_PORTS ]; +struct dlb_credit_pool ldb_credit_pools[DLB_MAX_NUM_LDB_CREDIT_POOLS ]; +struct dlb_credit_pool dir_credit_pools[DLB_MAX_NUM_DIR_CREDIT_POOLS ]; +struct dlb_sn_group sn_groups[DLB_MAX_NUM_SEQUENCE_NUMBER_GROUPS ]; +}; + +struct dlb_hw { +/* BAR 0 address */ +void *csr_kva; +unsigned long csr_phys_addr; +/* BAR 2 address */ +void *func_kva; +unsigned long func_phys_addr; + +/* Resource tracking */ +struct dlb_hw_resources rsrcs; +struct dlb_function_resources pf; +struct dlb_function_resources vf[DLB_MAX_NUM_VFS ]; +struct dlb_domain domains[DLB_MAX_NUM_DOMAINS ]; +}; + +#endif /* __DLB_HW_TYPES_H */
drivers/event/dlb/pf/base/dlb_mbox.h new file +645 -0
645 lines not on this page · raw message
drivers/event/dlb/pf/base/dlb_osdep.h new file +348 -0
348 lines not on this page · raw message
drivers/event/dlb/pf/base/dlb_osdep_bitmap.h new file +442 -0
442 lines not on this page · raw message
drivers/event/dlb/pf/base/dlb_osdep_list.h new file +131 -0
131 lines not on this page · raw message
drivers/event/dlb/pf/base/dlb_osdep_types.h new file +31 -0
@@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef __DLB_OSDEP_TYPES_H +#define __DLB_OSDEP_TYPES_H + +#include <linux/types.h> + +#include <inttypes.h> +#include <ctype.h> +#include <stdint.h> +#include <stdbool.h> +#include <string.h> +#include <unistd.h> +#include <errno.h> + +/* Types for user mode PF PMD */ +typedef uint8_t u8 ; +typedef int8_t s8 ; +typedef uint16_t u16 ; +typedef int16_t s16 ; +typedef uint32_t u32 ; +typedef int32_t s32 ; +typedef uint64_t u64 ; + +#define __iomem + +/* END types for user mode PF PMD */ + +#endif /* __DLB_OSDEP_TYPES_H */
drivers/event/dlb/pf/base/dlb_regs.h new file +2646 -0
1,043 lines not on this page · raw message
drivers/event/dlb/pf/base/dlb_resource.c new file +9700 -0
9,700 lines not on this page · raw message
drivers/event/dlb/pf/base/dlb_resource.h new file +1625 -0
1,625 lines not on this page · raw message
drivers/event/dlb/pf/base/dlb_user.h new file +1084 -0
1,084 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
1.7.10
Part of the diff is not shown here. View the raw message.
PATCH 4/27 event/dlb: add make and meson build infrastructure
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- config/common_base | 19 ++++++++++++++++++- config/rte_config.h | 8 +++++++- drivers/event/Makefile | 7 +++++++ drivers/event/dlb/Makefile | 27 +++++++++++++++++++++++++++ drivers/event/dlb/meson.build | 7 +++++++ drivers/event/meson.build | 4 ++++ mk/rte.app.mk | 1 + 7 files changed, 71 insertions(+), 2 deletions(-) create mode 100644 drivers/event/dlb/Makefile create mode 100644 drivers/event/dlb/meson.build
config/common_base +18 -1
@@ -741,7 +741,7 @@ CONFIG_RTE_LIBRTE_PMD_ZLIB=n CONFIG_RTE_LIBRTE_EVENTDEV=y CONFIG_RTE_LIBRTE_EVENTDEV_DEBUG=n CONFIG_RTE_EVENT_MAX_DEVS=16 -CONFIG_RTE_EVENT_MAX_QUEUES_PER_DEV=64 +CONFIG_RTE_EVENT_MAX_QUEUES_PER_DEV=255 CONFIG_RTE_EVENT_TIMER_ADAPTER_NUM_MAX=32 CONFIG_RTE_EVENT_ETH_INTR_RING_SIZE=1024 CONFIG_RTE_EVENT_CRYPTO_ADAPTER_MAX_INSTANCE=32
@@ -789,6 +789,23 @@ CONFIG_RTE_LIBRTE_PMD_DPAA_EVENTDEV=n CONFIG_RTE_LIBRTE_PMD_DPAA2_EVENTDEV=n # +# Compile PMD for dlb event device +# +CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV=y +# Specify an upper bound on cycles between each CQ access when hard polling +CONFIG_RTE_LIBRTE_PMD_DLB_POLL_INTERVAL=1000 +# Specify control state to use when executing UMWAIT instruction. +# If 0, then use CO.2, which has a slower wakeup time, but greater +# power savings. +# If 1, then use C0.1, which has a faster wakeup time, but smaller +# power savings. +CONFIG_RTE_LIBRTE_PMD_DLB_UMWAIT_CTL_STATE=0 +# Suppress statistics calculation +CONFIG_RTE_LIBRTE_PMD_DLB_QUELL_STATS=n +# Specify software credit quanta +CONFIG_RTE_LIBRTE_PMD_DLB_SW_CREDIT_QUANTA=32 + +# # Compile raw device support # EXPERIMENTAL: API may change without prior notice #
config/rte_config.h +7 -1
@@ -72,7 +72,7 @@/* eventdev defines */ #define RTE_EVENT_MAX_DEVS 16 -#define RTE_EVENT_MAX_QUEUES_PER_DEV 64 +#define RTE_EVENT_MAX_QUEUES_PER_DEV 255#define RTE_EVENT_TIMER_ADAPTER_NUM_MAX 32#define RTE_EVENT_ETH_INTR_RING_SIZE 1024#define RTE_EVENT_CRYPTO_ADAPTER_MAX_INSTANCE 32
@@ -133,4 +133,10 @@/* QEDE PMD defines */ #define RTE_LIBRTE_QEDE_FW "" +/* DLB PMD defines */ +#define RTE_LIBRTE_PMD_DLB_POLL_INTERVAL 1000 +#define RTE_LIBRTE_PMD_DLB_UMWAIT_CTL_STATE 0 +#undef RTE_LIBRTE_PMD_DLB_QUELL_STATS +#define RTE_LIBRTE_PMD_DLB_SW_CREDIT_QUANTA 32 +#endif /* _RTE_CONFIG_H_ */
drivers/event/Makefile +7 -0
@@ -16,5 +16,12 @@ ifeq ($(CONFIG_RTE_EAL_VFIO)$(CONFIG_RTE_LIBRTE_FSLMC_BUS),yy) DIRS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_EVENTDEV) += dpaa2 endif DIRS-$(CONFIG_RTE_LIBRTE_PMD_OPDL_EVENTDEV) += opdl +# DLB limited to linux x86 architectures +ifneq ($(filter y,$(CONFIG_RTE_ARCH_X86_64) \ + $(CONFIG_RTE_ARCH_X86)),) +ifeq ($(CONFIG_RTE_EXEC_ENV_LINUX),y) +DIRS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += dlb +endif +endif include $(RTE_SDK)/mk/rte.subdir.mk
drivers/event/dlb/Makefile new file +27 -0
@@ -0,0 +1,27 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2016-2020 Intel Corporation. + +include $(RTE_SDK)/mk/rte.vars.mk + +# library name +LIB = librte_pmd_dlb_eventdev.a + +# build flags +CFLAGS += -O3 -pthread +CFLAGS += $(WERROR_FLAGS) +# rte_mp_ APIs are still marked experimental +LDLIBS += -lrte_eal -lrte_eventdev -lrte_mbuf -lrte_kvargs -lrte_ring +LDLIBS += -lrte_mempool -lpthread -lrte_pci -lrte_bus_pci + +# library version +LIBABIVER := 1 + +# versioning export map +EXPORT_MAP := rte_pmd_dlb_event_version.map + +# library source files +SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += pf/base/dlb_resource.c + +# export include files + +include $(RTE_SDK)/mk/rte.lib.mk
drivers/event/dlb/meson.build new file +7 -0
@@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019-2020 Intel Corporation + +sources = files('pf/base/dlb_resource.c' +) + +deps += ['mbuf', 'mempool', 'ring', 'pci', 'bus_pci']
drivers/event/meson.build +4 -0
@@ -6,6 +6,10 @@ if not (toolchain == 'gcc' and cc.version().version_compare('<4.8.6') and dpdk_conf.has('RTE_ARCH_ARM64')) drivers += 'octeontx' endif +if ((dpdk_conf.has('RTE_ARCH_X86_64') or dpdk_conf.has('RTE_ARCH_X86')) and + is_linux) + drivers += 'dlb' +endif std_deps = ['eventdev', 'kvargs'] config_flag_fmt = 'RTE_LIBRTE_@0@_EVENTDEV_PMD' driver_name_fmt = 'rte_pmd_@0@_event'
mk/rte.app.mk +1 -0
@@ -325,6 +325,7 @@ endif # CONFIG_RTE_LIBRTE_COMPRESSDEV ifeq ($(CONFIG_RTE_LIBRTE_EVENTDEV),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SKELETON_EVENTDEV) += -lrte_pmd_skeleton_event _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV) += -lrte_pmd_sw_event +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += -lrte_pmd_dlb_eventdev _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_DSW_EVENTDEV) += -lrte_pmd_dsw_event _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += -lrte_pmd_octeontx_ssovf ifeq ($(CONFIG_RTE_LIBRTE_DPAA_BUS),y)
Signature · 1 line
1.7.10
PATCH 5/27 event/dlb: add DLB documentation
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- doc/guides/eventdevs/dlb.rst | 497 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 497 insertions(+) create mode 100644 doc/guides/eventdevs/dlb.rst
doc/guides/eventdevs/dlb.rst new file +497 -0
@@ -0,0 +1,497 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Intel Corporation. + +Driver for the Intel® Dynamic Load Balancer (DLB) +================================================== + +The DPDK dlb poll mode driver supports the Intel® Dynamic Load Balancer. + +.. note:: + + This PMD is disabled by default in the build configuration files, owing to + an external dependency on the `Netlink Protocol Library Suite + <http://www.infradead.org/~tgr/libnl/>`_ (libnl-3 and libnl-genl-3) which + must be installed on the board. Once the Netlink libraries are installed, + the PMD can be enabled by setting CONFIG_RTE_LIBRTE_PMD_DLB_QM=y and + recompiling the DPDK. + +Prerequisites +------------- + +- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup + the basic DPDK environment. + +- Learn about the DLB device and its capabilities at `Intel Support + <http://www.intel.com/support>`_. FIXME: Add real link when documentation + becomes available. + +- The DLB kernel module. If it is not included in the machine's OS + distribution, download it from <FIXME: Add 01.org link when available> and + follow the build instructions. + +Configuration +------------- + +The DLB eventdev supports two modes of operation: + +* Bifurcated mode: the PMD is created as a vdev device and depends on the Linux + DLB kernel driver for device access. The bifurcated PMD's configuration + accesses are performed through the kernel driver, and (performance-critical) + datapath functions execute entirely in user-space. + + This mode supports both PF and VF devices, but is supported on Linux only. + +* PF PMD mode: the PF PMD is a user-space PMD that uses VFIO to gain direct + device access. To use this operation mode, the PCIe PF device must be bound + to a DPDK-compatible VFIO driver, such as vfio-pci. The PF PMD does not work + with PCIe VFs, but is portable to all environments (Linux, FreeBSD, etc.) + that DPDK supports. (Note: PF PMD testing has been limited to Linux at this + time.) + +The vdev device can be created from the application code or from the EAL +command line like so: + +* Call ``rte_vdev_init("dlb1_event")`` from the application. + +* Use ``--vdev="dlb1_event"`` in the EAL options, which will call + rte_vdev_init() internally. + +Example: + +.. code-block:: console + + ./your_eventdev_application --vdev="dlb1_event" + +Note: The dlb vdev can be instatiated with the name "event_dlb" as well. + +Eventdev API Notes +------------------ + +The DLB provides the functions of a DPDK event device; specifically, it +supports atomic, ordered, and parallel scheduling events from queues to ports. +However, the DLB hardware is not a perfect match to the eventdev API. Some DLB +features are abstracted by the PMD (e.g. directed ports), some are only +accessible as vdev command-line parameters, and certain eventdev features are +not supported (e.g. the event flow ID is not maintained during scheduling). + +In general the dlb PMD is designed for ease-of-use and doesn't require a +detailed understanding of the hardware, but these details are important when +writing high-performance code. This section describes the places where the +eventdev API and DLB misalign. + +Wait (timeout_ticks) Parameter +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The eventdev API rte_event_dequeue_burst(..) can wait for an event to +arrive. Three different forms of waiting are supported by the dlb PMD: +polling, blocking on a hardware interrupt, and waiting using umonitor/umwait. +Which form of wait to use can be specified using the hybrid timeout data +structure below.The application should use the appropriate bybrid timeout +struct below and cast it to uint32_t or uint64_t, as appropriate. + +If RTE_EVENT_DEV_CFG_PER_DEQUEUE_TIMEOUT is set, then the timeout_ticks +parameter supplied to rte_event_dequeue_burst(..) is used to control if and how +to wait, and dequeue_timeout_ns is ignored. + +If RTE_EVENT_DEV_CFG_PER_DEQUEUE_TIMEOUT is not set, then dequeue_timeout_ns +supplied to the rte_event_dev_configure API is used to control if and how to +wait, and the timeout_ticks value is ignored. + +The application should use the appropriate bybrid timeout struct below and cast +it to uint32_t (for rte_event_dev_configure) or uint64_t (for +rte_event_dequeue_burst), as appropriate. + +Hybrid timeout_ticks +^^^^^^^^^^^^^^^^^^^^ +#. If poll_ticks is not 0 and neither interrupt_wait or umonitor_wait are set, + then we will busy poll for up to poll_ticks time. +#. If the interrupt_wait bit is set and the CQ is empty, then enter kernel + to wait for an interrupt after busy polling for poll_ticks time. There + is no guarantee how much time we spend in the API when using interrupt_wait. +#. If umonitor_wait is set, then repeatedly issue a umwait instruction + until the requested number of events have been dequeued, or until + poll_ticks has expired. + +Note: It is invalid to set both interrupt_wait and umonitor_wait. + +The hybrid timeout data structures are currently located in +drivers/event/dlb/dlb_timeout.h: + +.. code-block:: c + + struct rte_hybrid_timeout_ticks_64 { + RTE_STD_C11 + union { + uint64_t val64; + struct { + uint64_t poll_ticks:62; + uint64_t umonitor_wait:1; + uint64_t interrupt_wait:1; + }; + }; + }; + struct rte_hybrid_timeout_ns_32 { + RTE_STD_C11 + union { + uint32_t val32; + struct { + uint32_t poll_ns:30; + uint32_t umonitor_wait:1; + uint32_t interrupt_wait:1; + }; + }; + }; + +VAS Configuration +~~~~~~~~~~~~~~~~~ + +A VAS is a scheduling domain, of which there are 32 in the DLB. (Producer +ports in one VAS cannot enqueue events to a different VAS, except through the +`Data Mover`_.) When a VAS is configured, it allocates load-balanced and +directed queues, ports, credits, and other hardware resources. Some VAS +resource allocations are user-controlled -- the number of queues, for example +-- and others, like credit pools (one directed and one load-balanced pool per +VAS), are not. + +The dlb PMD creates a single VAS per DLB device. Supporting multiple VASes +per DLB device is a planned feature, where each VAS will be represented as a +separate event device. + +The DLB is a closed system eventdev, and as such the ``nb_events_limit`` device +setup argument and the per-port ``new_event_threshold`` argument apply as +defined in the eventdev header file. The limit is applied to all enqueues, +regardless of whether it will consume a directed or load-balanced credit. + +Load-balanced and Directed Ports +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DLB ports come in two flavors: load-balanced and directed. The eventdev API +does not have the same concept, but it has a similar one: ports and queues that +are singly-linked (i.e. linked to a single queue or port, respectively). + +The ``rte_event_dev_info_get()`` function reports the number of available +event ports and queues (among other things). For the DLB PMD, max_event_ports +and max_event_queues report the number of available load-balanced ports and +queues, and max_single_link_event_port_queue_pairs reports the number of +available directed ports and queues. + +When a VAS is created in ``rte_event_dev_configure()``, the user specifies +``nb_event_ports`` and ``nb_single_link_event_port_queues``, which control the +total number of ports (load-balanced and directed) and the number of directed +ports. Hence, the number of requested load-balanced ports is ``nb_event_ports +- nb_single_link_event_ports``. The ``nb_event_queues`` field specifies the +total number of queues (load-balanced and directed). The number of directed +queues comes from ``nb_single_link_event_port_queues``, since directed ports +and queues come in pairs. + +When a port is setup, the ``RTE_EVENT_PORT_CFG_SINGLE_LINK`` flag determines +whether it should be configured as a directed (the flag is set) or a +load-balanced (the flag is unset) port. Similarly, the +``RTE_EVENT_QUEUE_CFG_SINGLE_LINK`` queue configuration flag controls +whether it is a directed or load-balanced queue. + +Load-balanced ports can only be linked to load-balanced queues, and directed +ports can only be linked to directed queues. Furthermore, directed ports can +only be linked to a single directed queue (and vice versa), and that link +cannot change after the eventdev is started. + +The eventdev API doesn't have a directed scheduling type. To support directed +traffic, the dlb PMD detects when an event is being sent to a directed queue +and overrides its scheduling type. Note that the originally selected scheduling +type (atomic, ordered, or parallel) is not preserved, and an event's sched_type +will be set to ``RTE_SCHED_TYPE_ATOMIC`` when it is dequeued from a directed +port. + +Flow ID +~~~~~~~ + +The flow ID field is not preserved in the event when it is scheduled in the +DLB, because the DLB hardware control word format does not have sufficient +space to preserve every event field. As a result, the flow ID specified with +the enqueued event will not be in the dequeued event. If this field is +required, the application should pass it through an out-of-band path (for +example in the mbuf's udata64 field, if the event points to an mbuf) or +reconstruct the flow ID after receiving the event. + +Also, the DLB hardware control word supports a 16-bit flow ID. Since struct +rte_event's flow_id field is 20 bits, the DLB PMD drops the most significant +four bits from the event's flow ID. + +Hardware Credits +~~~~~~~~~~~~~~~~ + +DLB uses a hardware credit scheme to prevent software from overflowing hardware +event storage, with each unit of storage represented by a credit. A port spends +a credit to enqueue an event, and hardware refills the ports with credits as the +events are scheduled to ports. Refills come from credit pools, and each port is +a member of a load-balanced credit pool and a directed credit pool. The +load-balanced credits are used to enqueue to load-balanced queues, and directed +credits are used for directed queues. + +An dlb eventdev contains one load-balanced and one directed credit pool. These +pools' sizes are controlled by the nb_events_limit field in struct +rte_event_dev_config. The load-balanced pool is sized to contain +nb_events_limit credits, and the directed pool is sized to contain +nb_events_limit/4 credits. The directed pool size can be overriden with the +num_dir_credits vdev argument, like so: + + .. code-block:: console + + --vdev=dlb1_event,num_dir_credits=<value> + +This can be used if the default allocation is too low or too high for the +specific application needs. The PMD also supports a vdev arg that limits the +max_num_events reported by rte_event_dev_info_get(): + + .. code-block:: console + + --vdev=dlb1_event,max_num_events=<value> + +By default, max_num_events is reported as the total available load-balanced +credits. If multiple DLB-based applications are being used, it may be desirable +to control how many load-balanced credits each application uses, particularly +when application(s) are written to configure nb_events_limit equal to the +reported max_num_events. + +Each port is a member of both credit pools. A port's credit allocation is +defined by its low watermark, high watermark, and refill quanta. These three +parameters are calculated by the dlb PMD like so: + +- The load-balanced high watermark is set to the port's enqueue_depth. + The directed high watermark is set to the minimum of the enqueue_depth and + the directed pool size divided by the total number of ports. +- The refill quanta is set to half the high watermark. +- The low watermark is set to the minimum of 8 and the refill quanta. + +When the eventdev is started, each port is pre-allocated a high watermark's +worth of credits. For example, if an eventdev contains four ports with enqueue +depths of 32 and a load-balanced credit pool size of 4096, each port will start +with 32 load-balanced credits, and there will be 3968 credits available to +replenish the ports. Thus, a single port is not capable of enqueueing up to the +nb_events_limit (without any events being dequeued), since the other ports are +retaining their initial credit allocation; in short, all ports must enqueue in +order to reach the limit. + +If a port attempts to enqueue and has no credits available, the enqueue +operation will fail and the application must retry the enqueue. Credits are +replenished asynchronously by the DLB hardware. + +Software Credits +~~~~~~~~~~~~~~~~ + +The DLB is a "closed system" event dev, and the DLB PMD layers a software +credit scheme on top of the hardware credit scheme in order to comply with +the per-port backpressure described in the eventdev API. + +The DLB's hardware scheme is local to a queue/pipeline stage: a port spends a +credit when it enqueues to a queue, and credits are later replenished after the +events are dequeued and released. + +In the software credit scheme, a credit is consumed when a new (.op = +RTE_EVENT_OP_NEW) event is injected into the system, and the credit is +replenished when the event is released from the system (either explicitly with +RTE_EVENT_OP_RELEASE or implicitly in dequeue_burst()). + +In this model, an event is "in the system" from its first enqueue into eventdev +until it is last dequeued. If the event goes through multiple event queues, it +is still considered "in the system" while a worker thread is processing it. + +A port will fail to enqueue if the number of events in the system exceeds its +``new_event_threshold`` (specified at port setup time). A port will also fail +to enqueue if it lacks enough hardware credits to enqueue; load-balanced +credits are used to enqueue to a load-balanced queue, and directed credits are +used to enqueue to a directed queue. + +The out-of-credit situations are typically transient, and an eventdev +application using the DLB ought to retry its enqueues if they fail. +If enqueue fails, DLB PMD sets rte_errno as follows: + +- -ENOSPC: Credit exhaustion (either hardware or software) +- -EINVAL: Invalid argument, such as port ID, queue ID, or sched_type. + +Depending on the pipeline the application has constructed, it's possible to +enter a credit deadlock scenario wherein the worker thread lacks the credit +to enqueue an event, and it must dequeue an event before it can recover the +credit. If the worker thread retries its enqueue indefinitely, it will not +make forward progress. Such deadlock is possible if the application has event +"loops", in which an event in dequeued from queue A and later enqueued back to +queue A. + +Due to this, workers should stop retrying after a time, release the events it +is attempting to enqueue, and dequeue more events. It is important that the +worker release the events and don't simply set them aside to retry the enqueue +again later, because the port has limited history list size (by default, twice +the port's dequeue_depth). + +Priority +~~~~~~~~ + +The DLB supports event priority and per-port queue service priority, as +described in the eventdev header file. The DLB does not support 'global' event +queue priority established at queue creation time. + +DLB supports 8 event and queue service priority levels. For both priority +types, the PMD uses the upper three bits of the priority field to determine the +DLB priority, discarding the 5 least significant bits. The 5 least significant +event priority bits are not preserved when an event is enqueued. + +Load-Balanced Queues +~~~~~~~~~~~~~~~~~~~~ + +A load-balanced queue can support atomic and ordered scheduling, or atomic and +unordered scheduling, but not atomic and unordered and ordered scheduling. A +queue's scheduling types are controlled by the event queue configuration. + +If the user sets the ``RTE_EVENT_QUEUE_CFG_ALL_TYPES`` flag, the +``nb_atomic_order_sequences`` determines the supported scheduling types. +With non-zero ``nb_atomic_order_sequences``, the queue is configured for atomic +and ordered scheduling. In this case, ``RTE_SCHED_TYPE_PARALLEL`` scheduling is +supported by scheduling those events as ordered events. Note that when the +event is dequeued, its sched_type will be ``RTE_SCHED_TYPE_ORDERED``. Else if +``nb_atomic_order_sequences`` is zero, the queue is configured for atomic and +unordered scheduling. In this case, ``RTE_SCHED_TYPE_ORDERED`` is unsupported. + +If the ``RTE_EVENT_QUEUE_CFG_ALL_TYPES`` flag is not set, schedule_type +dictates the queue's scheduling type. + +The ``nb_atomic_order_sequences`` queue configuration field sets the ordered +queue's reorder buffer size. DLB has 4 groups of ordered queues, where each +group is configured to contain either 1 queue with 1024 reorder entries, 2 +queues with 512 reorder entries, and so on down to 32 queues with 32 entries. + +When a load-balanced queue is created, the PMD will configure a new sequence +number group on-demand if num_sequence_numbers does not match a pre-existing +group with available reorder buffer entries. If all sequence number groups are +in use, no new group will be created and queue configuration will fail. (Note +that when the PMD is used with a virtual DLB device, it cannot change the +sequence number configuration.) + +The queue's ``nb_atomic_flows`` parameter is ignored by the DLB PMD, because +the DLB doesn't limit the number of flows a queue can track. In the DLB, all +load-balanced queues can use the full 16-bit flow ID range. + +Reconfiguration +~~~~~~~~~~~~~~~ + +The Eventdev API allows one to reconfigure a device, its ports, and its queues +by first stopping the device, calling the configuration function(s), then +restarting the device. The DLB doesn't support configuring an individual queue +or port without first reconfiguring the entire device, however, so there are +certain reconfiguration sequences that are valid in the eventdev API but not +supported by the PMD. + +Specifically, the PMD supports the following configuration sequence: +1. Configure and start the device +2. Stop the device +3. (Optional) Reconfigure the device +4. (Optional) If step 3 is run: + + a. Setup queue(s). The reconfigured queue(s) lose their previous port links. + b. The reconfigured port(s) lose their previous queue links. + +5. (Optional, only if steps 4a and 4b are run) Link port(s) to queue(s) +6. Restart the device. If the device is reconfigured in step 3 but one or more + of its ports or queues are not, the PMD will apply their previous + configuration (including port->queue links) at this time. + +The PMD does not support the following configuration sequences: +1. Configure and start the device +2. Stop the device +3. Setup queue or setup port +4. Start the device + +This sequence is not supported because the event device must be reconfigured +before its ports or queues can be. + +Ordered Fragments +~~~~~~~~~~~~~~~~~ + +The DLB has a fourth enqueue type: partial enqueue. When a thread is processing +an ordered event, it can perform up to 16 "partial" enqueues, which allows a +single received ordered event to result in multiple reordered events. + +For example, consider the case where three events (A, then B, then C) are +enqueued with ordered scheduling and are received by three different ports. +The ports that receive A and C forward events A' and C', while the port that +receives B generates three partial enqueues -- B1', B2', and B3' -- followed by +a release operation. The DLB will reorder the events in the following order: + +A', B1', B2', B3', C' + +This functionality is not available explicitly through the eventdev API, but +the dlb PMD provides it through an additional (DLB-specific) event operation, +RTE_EVENT_DLB_OP_FRAG. + +Deferred Scheduling +~~~~~~~~~~~~~~~~~~~ + +The DLB PMD's default behavior for managing a CQ is to "pop" the CQ once per +dequeued event before returning from rte_event_dequeue_burst(). This frees the +corresponding entries in the CQ, which enables the DLB to schedule more events +to it. + +To support applications seeking finer-grained scheduling control -- for example +deferring scheduling to get the best possible priority scheduling and +load-balancing -- the PMD supports a deferred scheduling mode. In this mode, +the CQ entry is not popped until the *subsequent* rte_event_dequeue_burst() +call. This mode only applies to load-balanced event ports with dequeue depth of +1. + +To enable deferred scheduling, use the defer_sched vdev argument like so: + + .. code-block:: console + + --vdev=dlb1_event,defer_sched=on + +Atomic Inflights Allocation +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the last stage prior to scheduling an atomic event to a CQ, DLB holds the +inflight event in a temporary buffer that is divided among load-balanced +queues. If a queue's atomic buffer storage fills up, this can result in +head-of-line-blocking. For example: +- An LDB queue allocated N atomic buffer entries +- All N entries are filled with events from flow X, which is pinned to CQ 0. + +Until CQ 0 releases 1+ events, no other atomic flows for that LDB queue can be +scheduled. The likelihood of this case depends on the eventdev configuration, +traffic behavior, event processing latency, potential for a worker to be +interrupted or otherwise delayed, etc. + +By default, the PMD allocates 16 buffer entries for each load-balanced queue, +which provides an even division across all 128 queues but potentially wastes +buffer space (e.g. if not all queues are used, or aren't used for atomic +scheduling). + +The PMD provides a dev arg to override the default per-queue allocation. To +increase a vdev's per-queue atomic-inflight allocation to (for example) 64: + + .. code-block:: console + + --vdev=dlb1_event,atm_inflights=64 + +Atomic Inflights Allocation +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the last stage prior to scheduling an atomic event to a CQ, DLB holds the +inflight event in a temporary buffer that is divided among load-balanced +queues. If a queue's atomic buffer storage fills up, this can result in +head-of-line-blocking. For example: +- An LDB queue allocated N atomic buffer entries +- All N entries are filled with events from flow X, which is pinned to CQ 0. + +Until CQ 0 releases 1+ events, no other atomic flows for that LDB queue can be +scheduled. The likelihood of this case depends on the eventdev configuration, +traffic behavior, event processing latency, potential for a worker to be +interrupted or otherwise delayed, etc. + +By default, the PMD allocates 16 buffer entries for each load-balanced queue, +which provides an even division across all 128 queues but potentially wastes +buffer space (e.g. if not all queues are used, or aren't used for atomic +scheduling). + +The PMD provides a dev arg to override the default per-queue allocation. To +increase a vdev's per-queue atomic-inflight allocation to (for example) 64: + + .. code-block:: console + + --vdev=dlb1_event,atm_inflights=64
Signature · 1 line
1.7.10
Re: [dpdk-dev] [PATCH 05/27] event/dlb: add DLB documentation
Hi Tim,
Quoted 26 lines
doc/guides/eventdevs/dlb.rst | 497 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 497 insertions(+) create mode 100644 doc/guides/eventdevs/dlb.rstdoc/guides/eventdevs/dlb.rst new renamed from doc/guides/eventdevs/dlb.rst +0
-0file mode 100644 index 0000000..21e48fe
doc/guides/eventdevs/dlb.rst new file +15
-0@@ -0,0 +1,497 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Intel Corporation. + +Driver for the Intel® Dynamic Load Balancer (DLB) +================================================== + +The DPDK dlb poll mode driver supports the Intel® Dynamic Load Balancer. + +.. note:: + + This PMD is disabled by default in the build configuration files, owing to + an external dependency on the `Netlink Protocol Library Suite + <http://www.infradead.org/~tgr/libnl/>`_ (libnl-3 and libnl-genl-3) which + must be installed on the board. Once the Netlink libraries are installed, + the PMD can be enabled by setting CONFIG_RTE_LIBRTE_PMD_DLB_QM=yand
doc/guides/eventdevs/dlb.rst
Quoted 2 lines
doc/guides/eventdevs/dlb.rst new file +2
-0+ recompiling the DPDK. +
This description appears to be out-of-date.
doc/guides/eventdevs/dlb.rst line 26
Quoted 11 lines
doc/guides/eventdevs/dlb.rst new file +11
-0+Prerequisites +------------- + +- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to +setup + the basic DPDK environment. + +- Learn about the DLB device and its capabilities at `Intel Support + <http://www.intel.com/support>`_. FIXME: Add real link when +documentation + becomes available.
Leftover FIXME
doc/guides/eventdevs/dlb.rst line 31
Quoted 6 lines
doc/guides/eventdevs/dlb.rst new file +6
-0+ +- The DLB kernel module. If it is not included in the machine's OS + distribution, download it from <FIXME: Add 01.org link when +available> and + follow the build instructions. +
Leftover FIXME
<snip>
Quoted 17 lines
doc/guides/eventdevs/dlb.rst new file +17
-0+The hybrid timeout data structures are currently located in +drivers/event/dlb/dlb_timeout.h: + +.. code-block:: c + + struct rte_hybrid_timeout_ticks_64 { + RTE_STD_C11 + union { + uint64_t val64; + struct { + uint64_t poll_ticks:62; + uint64_t umonitor_wait:1; + uint64_t interrupt_wait:1; + }; + }; + }; + struct rte_hybrid_timeout_ns_32 {
doc/guides/eventdevs/dlb.rst line 142
Quoted 10 lines
doc/guides/eventdevs/dlb.rst new file +10
-0+ RTE_STD_C11 + union { + uint32_t val32; + struct { + uint32_t poll_ns:30; + uint32_t umonitor_wait:1; + uint32_t interrupt_wait:1; + }; + }; + };
Is this description correct? dlb_timeout.h isn't introduced in this patchset.
doc/guides/eventdevs/dlb.rst
Quoted 7 lines
doc/guides/eventdevs/dlb.rst new file +7
-0+ +VAS Configuration +~~~~~~~~~~~~~~~~~ + +A VAS is a scheduling domain, of which there are 32 in the DLB. +(Producer ports in one VAS cannot enqueue events to a different VAS, +except through the `Data Mover`_.) When a VAS is configured, it
I believe this cross-VAS comment is out-of-date.
doc/guides/eventdevs/dlb.rst
Quoted 9 lines
doc/guides/eventdevs/dlb.rst new file +9
-0+allocates load-balanced and directed queues, ports, credits, and other +hardware resources. Some VAS resource allocations are user-controlled +-- the number of queues, for example +-- and others, like credit pools (one directed and one load-balanced +pool per VAS), are not. + +The dlb PMD creates a single VAS per DLB device. Supporting multiple +VASes per DLB device is a planned feature, where each VAS will be +represented as a separate event device.
Is this comment up-to-date? Patch 16 ("event/dlb: add infos_get and configure") indicates that multiple event devices are supported.
<snip>
doc/guides/eventdevs/dlb.rst
Quoted 13 lines
doc/guides/eventdevs/dlb.rst new file +10
-0+Hardware Credits +~~~~~~~~~~~~~~~~ + +DLB uses a hardware credit scheme to prevent software from overflowing +hardware event storage, with each unit of storage represented by a +credit. A port spends a credit to enqueue an event, and hardware +refills the ports with credits as the events are scheduled to ports. +Refills come from credit pools, and each port is a member of a +load-balanced credit pool and a directed credit pool. The load-balanced +credits are used to enqueue to load-balanced queues, and directed creditsare used for directed queues.
doc/guides/eventdevs/dlb.rst new file +2
-0+ +An dlb eventdev contains one load-balanced and one directed credit
"An dlb" -> "A dlb"
Quoted 24 lines
doc/guides/eventdevs/dlb.rst new file +24
-0+pool. These pools' sizes are controlled by the nb_events_limit field in +struct rte_event_dev_config. The load-balanced pool is sized to contain +nb_events_limit credits, and the directed pool is sized to contain +nb_events_limit/4 credits. The directed pool size can be overriden with +the num_dir_credits vdev argument, like so: + + .. code-block:: console + + --vdev=dlb1_event,num_dir_credits=<value> + +This can be used if the default allocation is too low or too high for +the specific application needs. The PMD also supports a vdev arg that +limits the max_num_events reported by rte_event_dev_info_get(): + + .. code-block:: console + + --vdev=dlb1_event,max_num_events=<value> + +By default, max_num_events is reported as the total available +load-balanced credits. If multiple DLB-based applications are being +used, it may be desirable to control how many load-balanced credits +each application uses, particularly when application(s) are written to +configure nb_events_limit equal to the reported max_num_events. +
doc/guides/eventdevs/dlb.rst line 263
Quoted 10 lines
doc/guides/eventdevs/dlb.rst new file +10
-0+Each port is a member of both credit pools. A port's credit allocation +is defined by its low watermark, high watermark, and refill quanta. +These three parameters are calculated by the dlb PMD like so: + +- The load-balanced high watermark is set to the port's enqueue_depth. + The directed high watermark is set to the minimum of the +enqueue_depth and + the directed pool size divided by the total number of ports. +- The refill quanta is set to half the high watermark. +- The low watermark is set to the minimum of 8 and the refill quanta.
From patch 19 ("event/dlb: add port_setup"), this should be 16 instead of 8:
+ cfg.ldb_credit_quantum = cfg.ldb_credit_high_watermark / 2;
+ cfg.ldb_credit_low_watermark = RTE_MIN(16, cfg.ldb_credit_quantum);
+
+ cfg.dir_credit_quantum = cfg.dir_credit_high_watermark / 2;
+ cfg.dir_credit_low_watermark = RTE_MIN(16, cfg.dir_credit_quantum);
Quoted 15 lines
doc/guides/eventdevs/dlb.rst new file +9
-0+ +When the eventdev is started, each port is pre-allocated a high +watermark's worth of credits. For example, if an eventdev contains four +ports with enqueue depths of 32 and a load-balanced credit pool size of +4096, each port will start with 32 load-balanced credits, and there +will be 3968 credits available to replenish the ports. Thus, a single +port is not capable of enqueueing up to the nb_events_limit (without +any events being dequeued), since the other ports are retaining their +initial credit allocation; in short, all ports must enqueue in order to reachthe limit.
doc/guides/eventdevs/dlb.rst new file +5
-0+ +If a port attempts to enqueue and has no credits available, the enqueue +operation will fail and the application must retry the enqueue. Credits +are replenished asynchronously by the DLB hardware. +
<snip>
Quoted 16 lines
doc/guides/eventdevs/dlb.rst new file +7
-0+ +Ordered Fragments +~~~~~~~~~~~~~~~~~ + +The DLB has a fourth enqueue type: partial enqueue. When a thread is +processing an ordered event, it can perform up to 16 "partial" +enqueues, which allows a single received ordered event to result in multiplereordered events.
doc/guides/eventdevs/dlb.rst new file +3
-0+ +For example, consider the case where three events (A, then B, then C) +are enqueued with ordered scheduling and are received by three differentports.
doc/guides/eventdevs/dlb.rst new file +3
-0+The ports that receive A and C forward events A' and C', while the port +that receives B generates three partial enqueues -- B1', B2', and B3' +-- followed by a release operation. The DLB will reorder the events in thefollowing order:
doc/guides/eventdevs/dlb.rst
Quoted 6 lines
doc/guides/eventdevs/dlb.rst new file +6
-0+ +A', B1', B2', B3', C' + +This functionality is not available explicitly through the eventdev +API, but the dlb PMD provides it through an additional (DLB-specific) +event operation, RTE_EVENT_DLB_OP_FRAG.
I don't believe this OP type appears in this patchset.
Quoted 45 lines
doc/guides/eventdevs/dlb.rst new file +44
-0+ +Deferred Scheduling +~~~~~~~~~~~~~~~~~~~ + +The DLB PMD's default behavior for managing a CQ is to "pop" the CQ +once per dequeued event before returning from +rte_event_dequeue_burst(). This frees the corresponding entries in the +CQ, which enables the DLB to schedule more events to it. + +To support applications seeking finer-grained scheduling control -- for +example deferring scheduling to get the best possible priority +scheduling and load-balancing -- the PMD supports a deferred scheduling +mode. In this mode, the CQ entry is not popped until the *subsequent* +rte_event_dequeue_burst() call. This mode only applies to load-balanced +event ports with dequeue depth of 1. + +To enable deferred scheduling, use the defer_sched vdev argument like so: + + .. code-block:: console + + --vdev=dlb1_event,defer_sched=on + +Atomic Inflights Allocation +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the last stage prior to scheduling an atomic event to a CQ, DLB +holds the inflight event in a temporary buffer that is divided among +load-balanced queues. If a queue's atomic buffer storage fills up, this +can result in head-of-line-blocking. For example: +- An LDB queue allocated N atomic buffer entries +- All N entries are filled with events from flow X, which is pinned to CQ 0. + +Until CQ 0 releases 1+ events, no other atomic flows for that LDB queue +can be scheduled. The likelihood of this case depends on the eventdev +configuration, traffic behavior, event processing latency, potential +for a worker to be interrupted or otherwise delayed, etc. + +By default, the PMD allocates 16 buffer entries for each load-balanced +queue, which provides an even division across all 128 queues but +potentially wastes buffer space (e.g. if not all queues are used, or +aren't used for atomic scheduling). + +The PMD provides a dev arg to override the default per-queue +allocation. To increase a vdev's per-queue atomic-inflight allocation to (forexample) 64:
doc/guides/eventdevs/dlb.rst line 471
Quoted 5 lines
doc/guides/eventdevs/dlb.rst new file +5
-0+ + .. code-block:: console + + --vdev=dlb1_event,atm_inflights=64 +
This section is duplicated below.
Quoted 28 lines
doc/guides/eventdevs/dlb.rst new file +22
-0+Atomic Inflights Allocation +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the last stage prior to scheduling an atomic event to a CQ, DLB +holds the inflight event in a temporary buffer that is divided among +load-balanced queues. If a queue's atomic buffer storage fills up, this +can result in head-of-line-blocking. For example: +- An LDB queue allocated N atomic buffer entries +- All N entries are filled with events from flow X, which is pinned to CQ 0. + +Until CQ 0 releases 1+ events, no other atomic flows for that LDB queue +can be scheduled. The likelihood of this case depends on the eventdev +configuration, traffic behavior, event processing latency, potential +for a worker to be interrupted or otherwise delayed, etc. + +By default, the PMD allocates 16 buffer entries for each load-balanced +queue, which provides an even division across all 128 queues but +potentially wastes buffer space (e.g. if not all queues are used, or +aren't used for atomic scheduling). + +The PMD provides a dev arg to override the default per-queue +allocation. To increase a vdev's per-queue atomic-inflight allocation to (forexample) 64:
doc/guides/eventdevs/dlb.rst new file +4
-1+ + .. code-block:: console + + --vdev=dlb1_event,atm_inflights=64 --
Quoted 1 line
1.7.10
Thanks,
Gage
PATCH 6/27 event/dlb: add dynamic logging
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb_log.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 drivers/event/dlb/dlb_log.h
drivers/event/dlb/dlb_log.h new file +24 -0
@@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2020 Intel Corporation +*/ + +#ifndef _DLB_EVDEV_LOG_H_ +#define _DLB_EVDEV_LOG_H_ + +extern int eventdev_dlb_log_level; + +/* Dynamic logging */ +#define DLB_LOG_IMPL(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, eventdev_dlb_log_level, "%s" fmt "\n", \ + __func__, ##args) + +#define DLB_LOG_INFO(fmt, args...) \ + DLB_LOG_IMPL(INFO, fmt, ## args) + +#define DLB_LOG_DBG(fmt, args...) \ + DLB_LOG_IMPL(DEBUG, fmt, ## args) + +#define DLB_LOG_ERR(fmt, args...) \ + DLB_LOG_IMPL(ERR, fmt, ## args) + +#endif /* _DLB_EVDEV_LOG_H_ */
Signature · 1 line
1.7.10
PATCH 7/27 event/dlb: add private data structures and constants
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb_priv.h | 564 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 564 insertions(+) create mode 100644 drivers/event/dlb/dlb_priv.h
drivers/event/dlb/dlb_priv.h new file +564 -0
@@ -0,0 +1,564 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef _DLB_PRIV_H_ +#define _DLB_PRIV_H_ + +#include <emmintrin.h> +#include <rte_eventdev.h> +#include <stdbool.h> +#include "rte_config.h" +#include "dlb_user.h" +#include "dlb_log.h" +#include "rte_pmd_dlb.h" + +#ifndef RTE_LIBRTE_PMD_DLB_QUELL_STATS +#define DLB_INC_STAT (_stat, _incr_val) ((_stat) += _incr_val) +#else +#define DLB_INC_STAT (_stat, _incr_val) +#endif + +#define EVDEV_DLB_NAME_PMD_STR "dlb_event" + +/* command line arg strings */ +#define NUMA_NODE_ARG "numa_node" +#define DLB_MAX_NUM_EVENTS "max_num_events" +#define DLB_NUM_DIR_CREDITS "num_dir_credits" +#define DEV_ID_ARG "dev_id" +#define DLB_DEFER_SCHED_ARG "defer_sched" +#define DLB_NUM_ATM_INFLIGHTS_ARG "atm_inflights" + +/* Begin HW related defines and structs */ + +#define DLB_MAX_NUM_DOMAINS 32 +#define DLB_MAX_NUM_VFS 16 +#define DLB_MAX_NUM_LDB_QUEUES 128 +#define DLB_MAX_NUM_LDB_PORTS 64 +#define DLB_MAX_NUM_DIR_PORTS 128 +#define DLB_MAX_NUM_DIR_QUEUES 128 +#define DLB_MAX_NUM_FLOWS (64 * 1024) +#define DLB_MAX_NUM_LDB_CREDITS 16384 +#define DLB_MAX_NUM_DIR_CREDITS 4096 +#define DLB_MAX_NUM_LDB_CREDIT_POOLS 64 +#define DLB_MAX_NUM_DIR_CREDIT_POOLS 64 +#define DLB_MAX_NUM_HIST_LIST_ENTRIES 5120 +#define DLB_MAX_NUM_ATM_INFLIGHTS 2048 +#define DLB_MAX_NUM_QIDS_PER_LDB_CQ 8 +#define DLB_QID_PRIORITIES 8 +#define DLB_MAX_DEVICE_PATH 32 +#define DLB_MIN_DEQUEUE_TIMEOUT_NS 1 +#define DLB_NUM_SN_GROUPS 4 +#define DLB_MAX_LDB_SN_ALLOC 1024 +/* Note: "- 1" here to support the timeout range check in eventdev_autotest */ +#define DLB_MAX_DEQUEUE_TIMEOUT_NS (UINT32_MAX - 1) +#define DLB_DEF_UNORDERED_QID_INFLIGHTS 2048 + +/* 5120 total hist list entries and 64 total ldb ports, which +* makes for 5120/64 == 80 hist list entries per port. However, CQ +* depth must be a power of 2 and must also be >= HIST LIST entries. +* As a result we just limit the maximum dequeue depth to 64. +*/ +#define DLB_MIN_LDB_CQ_DEPTH 1 +#define DLB_MIN_DIR_CQ_DEPTH 8 +#define DLB_MAX_CQ_DEPTH 64 +#define DLB_NUM_HIST_LIST_ENTRIES_PER_LDB_PORT \ + DLB_MAX_CQ_DEPTH + +/* +* Static per queue/port provisioning values +*/ +#define DLB_NUM_ATOMIC_INFLIGHTS_PER_QUEUE 16 + +#define PP_BASE (is_dir) ((is_dir) ? DLB_DIR_PP_BASE : DLB_LDB_PP_BASE) + +#define PAGE_SIZE (sysconf(_SC_PAGESIZE)) + +#define DLB_NUM_QES_PER_CACHE_LINE 4 + +#define DLB_MAX_ENQUEUE_DEPTH 64 +#define DLB_MIN_ENQUEUE_DEPTH 4 + +#define DLB_NAME_SIZE 64 + +/* Use the upper 3 bits of the event priority to select the DLB priority */ +#define EV_TO_DLB_PRIO (x) ((x) >> 5) +#define DLB_TO_EV_PRIO (x) ((x) << 5) + +enum dlb_hw_port_type { +DLB_LDB , +DLB_DIR , + +/* NUM_DLB_PORT_TYPES must be last */ +NUM_DLB_PORT_TYPES +}; + +#define PORT_TYPE (p) ((p)->is_directed ? DLB_DIR : DLB_LDB) + +/* Do not change - must match hardware! */ +enum dlb_hw_sched_type { +DLB_SCHED_ATOMIC =0 , +DLB_SCHED_UNORDERED , +DLB_SCHED_ORDERED , +DLB_SCHED_DIRECTED , + +/* DLB_NUM_HW_SCHED_TYPES must be last */ +DLB_NUM_HW_SCHED_TYPES +}; + +/* vdev args */ +struct dlb_devargs { +int socket_id; +int max_num_events; +int num_dir_credits_override; +int dev_id; +int defer_sched; +int num_atm_inflights; +}; + +struct dlb_hw_rsrcs { +int32_t nb_events_limit; +uint32_t num_queues;/**> Total queues (lb + dir) */ +uint32_t num_ldb_queues;/**> Number of available ldb queues */ +uint32_t num_ldb_ports;/**< Number of load balanced ports */ +uint32_t num_dir_ports;/**< Number of directed ports */ +uint32_t num_ldb_credits;/**< Number of load balanced credits */ +uint32_t num_dir_credits;/**< Number of directed credits */ +uint32_t reorder_window_size;/**< Size of reorder window */ +}; + +struct dlb_hw_resource_info { +/**> Max resources that can be provided */ +struct dlb_hw_rsrcs hw_rsrc_max; +int num_sched_domains; +uint32_t socket_id; +/**> EAL flags passed to this DLB instance, allowing the application to +* identify the pmd backend indicating hardware or software. +*/ +const char *eal_flags; +}; + +/* hw-specific format - do not change */ + +struct dlb_event_type { +uint8_t major:4 ; +uint8_t unused:4 ; +uint8_t sub; +}; + +union dlb_opaque_data { +uint16_t opaque_data; +struct dlb_event_type event_type; +}; + +struct dlb_msg_info { +uint8_t qid; +uint8_t sched_type:2 ; +uint8_t priority:3 ; +uint8_t msg_type:3 ; +}; + +#define DLB_NEW_CMD_BYTE 0x08 +#define DLB_FWD_CMD_BYTE 0x0A +#define DLB_COMP_CMD_BYTE 0x02 +#define DLB_NOOP_CMD_BYTE 0x00 +#define DLB_POP_CMD_BYTE 0x01 + +/* hw-specific format - do not change */ +struct dlb_enqueue_qe { +uint64_t data; +/* Word 3 */ +union dlb_opaque_data u; +uint8_t qid; +uint8_t sched_type:2 ; +uint8_t priority:3 ; +uint8_t msg_type:3 ; +/* Word 4 */ +uint16_t lock_id; +uint8_t meas_lat:1 ; +uint8_t rsvd1:2 ; +uint8_t no_dec:1 ; +uint8_t cmp_id:4 ; +union { +uint8_t cmd_byte; +struct { +uint8_t cq_token:1 ; +uint8_t qe_comp:1 ; +uint8_t qe_frag:1 ; +uint8_t qe_valid:1 ; +uint8_t int_arm:1 ; +uint8_t error:1 ; +uint8_t rsvd:2 ; + }; + }; +}; + +/* hw-specific format - do not change */ +struct dlb_cq_pop_qe { +uint64_t data; +union dlb_opaque_data u; +uint8_t qid; +uint8_t sched_type:2 ; +uint8_t priority:3 ; +uint8_t msg_type:3 ; +uint16_t tokens:10 ; +uint16_t rsvd2:6 ; +uint8_t meas_lat:1 ; +uint8_t rsvd1:2 ; +uint8_t no_dec:1 ; +uint8_t cmp_id:4 ; +union { +uint8_t cmd_byte; +struct { +uint8_t cq_token:1 ; +uint8_t qe_comp:1 ; +uint8_t qe_frag:1 ; +uint8_t qe_valid:1 ; +uint8_t int_arm:1 ; +uint8_t error:1 ; +uint8_t rsvd:2 ; + }; + }; +}; + +/* hw-specific format - do not change */ +struct dlb_dequeue_qe { +uint64_t data; +union dlb_opaque_data u; +uint8_t qid; +uint8_t sched_type:2 ; +uint8_t priority:3 ; +uint8_t msg_type:3 ; +uint16_t pp_id:10 ; +uint16_t rsvd0:6 ; +uint8_t debug; +uint8_t cq_gen:1 ; +uint8_t qid_depth:1 ; +uint8_t rsvd1:3 ; +uint8_t error:1 ; +uint8_t rsvd2:2 ; +}; + +union dlb_port_config { +struct dlb_create_ldb_port_args ldb; +struct dlb_create_dir_port_args dir; +}; + +enum DLB_PORT_STATE { +PORT_CLOSED , +PORT_STARTED , +PORT_STOPPED +}; + +enum dlb_configuration_state { +/* The resource has not been configured */ +DLB_NOT_CONFIGURED , +/* The resource was configured, but the device was stopped */ +DLB_PREV_CONFIGURED , +/* The resource is currently configured */ +DLB_CONFIGURED +}; + +struct dlb_port { +uint32_t id; +bool is_directed; +bool gen_bit; +uint16_t dir_credits; +uint32_t dequeue_depth; +enum dlb_token_pop_mode token_pop_mode; +union dlb_port_config cfg; +int pp_mmio_base; +uint16_t cached_ldb_credits; +uint16_t ldb_pushcount_at_credit_expiry; +uint16_t ldb_credits; +uint16_t cached_dir_credits; +uint16_t dir_pushcount_at_credit_expiry; +bool int_armed; +bool use_rsvd_token_scheme; +uint8_t cq_rsvd_token_deficit; +uint16_t owed_tokens; +int16_t issued_releases; +int16_t token_pop_thresh; +int cq_depth; +uint16_t cq_idx; +uint16_t cq_idx_unmasked; +uint16_t cq_depth_mask; +uint16_t gen_bit_shift; +enum DLB_PORT_STATE state; +enum dlb_configuration_state config_state; +int num_mapped_qids; +uint8_t *qid_mappings; +struct dlb_enqueue_qe *qe4;/* Cache line's worth of QEs (4) */ +struct dlb_cq_pop_qe *consume_qe; +struct dlb_eventdev *dlb;/* back ptr */ +struct dlb_eventdev_port *ev_port;/* back ptr */ +}; + +/* Per-process per-port mmio and memory pointers */ +struct process_local_port_data { +uint64_t *pp_addr; +uint16_t *ldb_popcount; +uint16_t *dir_popcount; +struct dlb_dequeue_qe *cq_base; +bool mmaped; +}; + +struct dlb_port_low_level_io_functions { +void (*pp_enqueue_four)(void *qe4,void *pp_addr); +}; + +struct dlb_config { +int configured; +int reserved; +uint32_t ldb_credit_pool_id; +uint32_t dir_credit_pool_id; +uint32_t num_ldb_credits; +uint32_t num_dir_credits; +struct dlb_create_sched_domain_args resources; +}; + +struct dlb_hw_dev { +char device_name[DLB_NAME_SIZE ]; +char device_path[DLB_MAX_DEVICE_PATH ]; +int device_path_id; +char domain_device_path[DLB_MAX_DEVICE_PATH ]; +struct dlb_config cfg; +struct dlb_hw_resource_info info; +void *pf_dev;/* opaque pointer to PF PMD dev (struct dlb_dev) */ +int device_id; +uint32_t domain_id; +int domain_id_valid; +rte_spinlock_t resource_lock;/* for MP support */ +}; __rte_cache_aligned + +/* End HW related defines and structs */ + +/* Begin DLB PMD Eventdev related defines and structs */ + +#define DLB_MAX_NUM_QUEUES \ + (DLB_MAX_NUM_DIR_QUEUES + DLB_MAX_NUM_LDB_QUEUES) + +#define DLB_MAX_NUM_PORTS (DLB_MAX_NUM_DIR_PORTS + DLB_MAX_NUM_LDB_PORTS) +#define DLB_MAX_INPUT_QUEUE_DEPTH 256 + +/* Used for parsing dir ports/queues. */ + +/** Structure to hold the queue to port link establishment attributes */ + +struct dlb_event_queue_link { +uint8_t queue_id; +uint8_t priority; +bool mapped; +bool valid; +}; + +struct dlb_traffic_stats { +uint64_t rx_ok; +uint64_t rx_drop; +uint64_t rx_interrupt_wait; +uint64_t rx_umonitor_umwait; +uint64_t tx_ok; +uint64_t total_polls; +uint64_t zero_polls; +uint64_t tx_nospc_ldb_hw_credits; +uint64_t tx_nospc_dir_hw_credits; +uint64_t tx_nospc_inflight_max; +uint64_t tx_nospc_new_event_limit; +uint64_t tx_nospc_inflight_credits; +}; + +struct dlb_port_stats { +struct dlb_traffic_stats traffic; +uint64_t tx_op_cnt[4 ];/* indexed by rte_event.op */ +uint64_t tx_implicit_rel; +uint64_t tx_sched_cnt[DLB_NUM_HW_SCHED_TYPES ]; +uint64_t tx_invalid; +uint64_t rx_sched_cnt[DLB_NUM_HW_SCHED_TYPES ]; +uint64_t rx_sched_invalid; +uint64_t enq_ok[DLB_MAX_NUM_QUEUES ];/* per-queue enq_ok */ +}; + +struct dlb_eventdev_port { +struct dlb_port qm_port;/* hw specific data structure */ +struct rte_event_port_conf conf;/* user-supplied configuration */ +uint16_t inflight_credits;/* num credits this port has right now */ +uint16_t credit_update_quanta; +struct dlb_eventdev *dlb;/* backlink optimization */ + struct dlb_port_stats stats __rte_cache_aligned; +struct dlb_event_queue_link link[DLB_MAX_NUM_QIDS_PER_LDB_CQ ]; +int num_links; +uint32_t id; +/* num releases yet to be completed on this port. +* Only applies to load-balanced ports. +*/ +uint16_t outstanding_releases; +uint16_t inflight_max;/* app requested max inflights for this port */ +/* setup_done is set when the event port is setup */ +bool setup_done; +/* enq_configured is set when the qm port is created */ +bool enq_configured; +uint8_t implicit_release;/* release events before dequeueing */ +} __rte_cache_aligned; + +struct dlb_queue { +uint32_t num_qid_inflights;/* User config */ +uint32_t num_atm_inflights;/* User config */ +enum dlb_configuration_state config_state; +int sched_type;/* LB queue only */ +uint32_t id; +bool is_directed; +}; + +struct dlb_eventdev_queue { +struct dlb_queue qm_queue; +struct rte_event_queue_conf conf;/* User config */ +uint64_t enq_ok; +uint32_t id; +bool setup_done; +uint8_t num_links; +}; + +enum dlb_run_state { +DLB_RUN_STATE_STOPPED =0 , +DLB_RUN_STATE_STOPPING , +DLB_RUN_STATE_STARTING , +DLB_RUN_STATE_STARTED +}; + +#define DLB_IS_VDEV true +#define DLB_NOT_VDEV false + +struct dlb_eventdev { +struct dlb_eventdev_port ev_ports[DLB_MAX_NUM_PORTS ]; +struct dlb_eventdev_queue ev_queues[DLB_MAX_NUM_QUEUES ]; +uint8_t qm_ldb_to_ev_queue_id[DLB_MAX_NUM_QUEUES ]; +uint8_t qm_dir_to_ev_queue_id[DLB_MAX_NUM_QUEUES ]; + +/* store num stats and offset of the stats for each queue */ +uint16_t xstats_count_per_qid[DLB_MAX_NUM_QUEUES ]; +uint16_t xstats_offset_for_qid[DLB_MAX_NUM_QUEUES ]; + +/* store num stats and offset of the stats for each port */ +uint16_t xstats_count_per_port[DLB_MAX_NUM_PORTS ]; +uint16_t xstats_offset_for_port[DLB_MAX_NUM_PORTS ]; +struct dlb_get_num_resources_args hw_rsrc_query_results; +uint32_t xstats_count_mode_queue; +struct dlb_hw_dev qm_instance;/* strictly hw related */ +uint64_t global_dequeue_wait_ticks; +struct dlb_xstats_entry *xstats; +struct rte_eventdev *event_dev;/* backlink to dev */ +uint32_t xstats_count_mode_port; +uint32_t xstats_count_mode_dev; +uint32_t xstats_count; +rte_atomic32_t inflights; +uint32_t new_event_limit; +int max_num_events_override; +int num_dir_credits_override; +volatile enum dlb_run_state run_state; +uint16_t num_dir_queues;/* total num of evdev dir queues requested */ +uint16_t num_dir_credits; +uint16_t num_ldb_credits; +uint16_t num_queues;/* total queues */ +uint16_t num_ldb_queues;/* total num of evdev ldb queues requested */ +uint16_t num_ports;/* total num of evdev ports requested */ +uint16_t num_ldb_ports;/* total num of ldb ports requested */ +uint16_t num_dir_ports;/* total num of dir ports requested */ +bool is_vdev; +bool umwait_allowed; +bool global_dequeue_wait;/* Not using per dequeue wait if true */ +bool defer_sched; +unsigned int num_atm_inflights_per_queue; +enum dlb_cq_poll_modes poll_mode; +uint8_t revision; +bool configured; +}; + +/* End Eventdev related defines and structs */ + +/* Forwards for non-inlined functions */ + +void dlb_free_qe_mem (struct dlb_port *port); + +int dlb_init_qe_mem (struct dlb_port *port,char *mz_name); + +int dlb_init_send_qe (struct dlb_port *port,char *mz_name); + +int dlb_init_partial_qe (struct dlb_port *port,char *mz_name); + +int dlb_init_fwd_qe (struct dlb_port *port,char *mz_name); + +int dlb_init_consume_qe (struct dlb_port *port,char *mz_name); + +int dlb_init_complete_qe (struct dlb_port *port,char *mz_name); + +int dlb_init_noop_qe (struct dlb_port *port,char *mz_name); + +int dlb_uninit (const char *name); + +void dlb_eventdev_dump (struct rte_eventdev *dev,FILE *f); + +int dlb_xstats_init (struct dlb_eventdev *dlb); + +void dlb_xstats_uninit (struct dlb_eventdev *dlb); + +int dlb_eventdev_xstats_get (const struct rte_eventdev *dev, +enum rte_event_dev_xstats_mode mode, +uint8_t queue_port_id,const unsigned int ids[], +uint64_t values[],unsigned int n); + +int dlb_eventdev_xstats_get_names (const struct rte_eventdev *dev, +enum rte_event_dev_xstats_mode mode, +uint8_t queue_port_id, +struct rte_event_dev_xstats_name *xstat_names, +unsigned int *ids,unsigned int size); + +uint64_t dlb_eventdev_xstats_get_by_name (const struct rte_eventdev *dev, +const char *name,unsigned int *id); + +int dlb_eventdev_xstats_reset (struct rte_eventdev *dev, +enum rte_event_dev_xstats_mode mode, +int16_t queue_port_id, +const uint32_t ids[], +uint32_t nb_ids); + +int test_dlb_eventdev (void ); + +int dlb_primary_eventdev_probe (struct rte_eventdev *dev, +const char *name, +struct dlb_devargs *dlb_args, +bool is_vdev); + +int dlb_secondary_eventdev_probe (struct rte_eventdev *dev, +const char *name, +bool is_vdev); +uint32_t dlb_get_queue_depth (struct dlb_eventdev *dlb, +struct dlb_eventdev_queue *queue); + +int set_numa_node(const char *key __rte_unused, const char *value, +void *opaque); + +int set_dir_ports(const char *key __rte_unused, + const char *value __rte_unused, + void *opaque __rte_unused); + +int set_dir_queues(const char *key __rte_unused, + const char *value __rte_unused, + void *opaque __rte_unused); + +int set_max_num_events(const char *key __rte_unused, const char *value, +void *opaque); + +int set_num_dir_credits(const char *key __rte_unused, const char *value, +void *opaque); + +void dlb_drain (struct rte_eventdev *dev); + +void dlb_entry_points_init (struct rte_eventdev *dev); + +int dlb_parse_params (const char *params, +const char *name, +struct dlb_devargs *dlb_args); + +int dlb_string_to_int (int *result,const char *str); + +#endif /* _DLB_PRIV_H_ */
Signature · 1 line
1.7.10
PATCH 8/27 event/dlb: add definitions shared with LKM or shared code
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb_user.h | 1083 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 1083 insertions(+) create mode 100644 drivers/event/dlb/dlb_user.h
drivers/event/dlb/dlb_user.h new file +1083 -0
@@ -0,0 +1,1083 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef __DLB_USER_H +#define __DLB_USER_H + +#define DLB_MAX_NAME_LEN 64 + +#include <linux/types.h> + +enum dlb_error { +DLB_ST_SUCCESS =0 , +DLB_ST_NAME_EXISTS , +DLB_ST_DOMAIN_UNAVAILABLE , +DLB_ST_LDB_PORTS_UNAVAILABLE , +DLB_ST_DIR_PORTS_UNAVAILABLE , +DLB_ST_LDB_QUEUES_UNAVAILABLE , +DLB_ST_LDB_CREDITS_UNAVAILABLE , +DLB_ST_DIR_CREDITS_UNAVAILABLE , +DLB_ST_LDB_CREDIT_POOLS_UNAVAILABLE , +DLB_ST_DIR_CREDIT_POOLS_UNAVAILABLE , +DLB_ST_SEQUENCE_NUMBERS_UNAVAILABLE , +DLB_ST_INVALID_DOMAIN_ID , +DLB_ST_INVALID_QID_INFLIGHT_ALLOCATION , +DLB_ST_ATOMIC_INFLIGHTS_UNAVAILABLE , +DLB_ST_HIST_LIST_ENTRIES_UNAVAILABLE , +DLB_ST_INVALID_LDB_CREDIT_POOL_ID , +DLB_ST_INVALID_DIR_CREDIT_POOL_ID , +DLB_ST_INVALID_POP_COUNT_VIRT_ADDR , +DLB_ST_INVALID_LDB_QUEUE_ID , +DLB_ST_INVALID_CQ_DEPTH , +DLB_ST_INVALID_CQ_VIRT_ADDR , +DLB_ST_INVALID_PORT_ID , +DLB_ST_INVALID_QID , +DLB_ST_INVALID_PRIORITY , +DLB_ST_NO_QID_SLOTS_AVAILABLE , +DLB_ST_QED_FREELIST_ENTRIES_UNAVAILABLE , +DLB_ST_DQED_FREELIST_ENTRIES_UNAVAILABLE , +DLB_ST_INVALID_DIR_QUEUE_ID , +DLB_ST_DIR_QUEUES_UNAVAILABLE , +DLB_ST_INVALID_LDB_CREDIT_LOW_WATERMARK , +DLB_ST_INVALID_LDB_CREDIT_QUANTUM , +DLB_ST_INVALID_DIR_CREDIT_LOW_WATERMARK , +DLB_ST_INVALID_DIR_CREDIT_QUANTUM , +DLB_ST_DOMAIN_NOT_CONFIGURED , +DLB_ST_PID_ALREADY_ATTACHED , +DLB_ST_PID_NOT_ATTACHED , +DLB_ST_INTERNAL_ERROR , +DLB_ST_DOMAIN_IN_USE , +DLB_ST_IOMMU_MAPPING_ERROR , +DLB_ST_FAIL_TO_PIN_MEMORY_PAGE , +DLB_ST_UNABLE_TO_PIN_POPCOUNT_PAGES , +DLB_ST_UNABLE_TO_PIN_CQ_PAGES , +DLB_ST_DISCONTIGUOUS_CQ_MEMORY , +DLB_ST_DISCONTIGUOUS_POP_COUNT_MEMORY , +DLB_ST_DOMAIN_STARTED , +DLB_ST_LARGE_POOL_NOT_SPECIFIED , +DLB_ST_SMALL_POOL_NOT_SPECIFIED , +DLB_ST_NEITHER_POOL_SPECIFIED , +DLB_ST_DOMAIN_NOT_STARTED , +DLB_ST_INVALID_MEASUREMENT_DURATION , +DLB_ST_INVALID_PERF_METRIC_GROUP_ID , +DLB_ST_LDB_PORT_REQUIRED_FOR_LDB_QUEUES , +DLB_ST_DOMAIN_RESET_FAILED , +DLB_ST_MBOX_ERROR , +DLB_ST_INVALID_HIST_LIST_DEPTH , +DLB_ST_NO_MEMORY , +}; + +static const char dlb_error_strings[][128 ] = { +"DLB_ST_SUCCESS" , +"DLB_ST_NAME_EXISTS" , +"DLB_ST_DOMAIN_UNAVAILABLE" , +"DLB_ST_LDB_PORTS_UNAVAILABLE" , +"DLB_ST_DIR_PORTS_UNAVAILABLE" , +"DLB_ST_LDB_QUEUES_UNAVAILABLE" , +"DLB_ST_LDB_CREDITS_UNAVAILABLE" , +"DLB_ST_DIR_CREDITS_UNAVAILABLE" , +"DLB_ST_LDB_CREDIT_POOLS_UNAVAILABLE" , +"DLB_ST_DIR_CREDIT_POOLS_UNAVAILABLE" , +"DLB_ST_SEQUENCE_NUMBERS_UNAVAILABLE" , +"DLB_ST_INVALID_DOMAIN_ID" , +"DLB_ST_INVALID_QID_INFLIGHT_ALLOCATION" , +"DLB_ST_ATOMIC_INFLIGHTS_UNAVAILABLE" , +"DLB_ST_HIST_LIST_ENTRIES_UNAVAILABLE" , +"DLB_ST_INVALID_LDB_CREDIT_POOL_ID" , +"DLB_ST_INVALID_DIR_CREDIT_POOL_ID" , +"DLB_ST_INVALID_POP_COUNT_VIRT_ADDR" , +"DLB_ST_INVALID_LDB_QUEUE_ID" , +"DLB_ST_INVALID_CQ_DEPTH" , +"DLB_ST_INVALID_CQ_VIRT_ADDR" , +"DLB_ST_INVALID_PORT_ID" , +"DLB_ST_INVALID_QID" , +"DLB_ST_INVALID_PRIORITY" , +"DLB_ST_NO_QID_SLOTS_AVAILABLE" , +"DLB_ST_QED_FREELIST_ENTRIES_UNAVAILABLE" , +"DLB_ST_DQED_FREELIST_ENTRIES_UNAVAILABLE" , +"DLB_ST_INVALID_DIR_QUEUE_ID" , +"DLB_ST_DIR_QUEUES_UNAVAILABLE" , +"DLB_ST_INVALID_LDB_CREDIT_LOW_WATERMARK" , +"DLB_ST_INVALID_LDB_CREDIT_QUANTUM" , +"DLB_ST_INVALID_DIR_CREDIT_LOW_WATERMARK" , +"DLB_ST_INVALID_DIR_CREDIT_QUANTUM" , +"DLB_ST_DOMAIN_NOT_CONFIGURED" , +"DLB_ST_PID_ALREADY_ATTACHED" , +"DLB_ST_PID_NOT_ATTACHED" , +"DLB_ST_INTERNAL_ERROR" , +"DLB_ST_DOMAIN_IN_USE" , +"DLB_ST_IOMMU_MAPPING_ERROR" , +"DLB_ST_FAIL_TO_PIN_MEMORY_PAGE" , +"DLB_ST_UNABLE_TO_PIN_POPCOUNT_PAGES" , +"DLB_ST_UNABLE_TO_PIN_CQ_PAGES" , +"DLB_ST_DISCONTIGUOUS_CQ_MEMORY" , +"DLB_ST_DISCONTIGUOUS_POP_COUNT_MEMORY" , +"DLB_ST_DOMAIN_STARTED" , +"DLB_ST_LARGE_POOL_NOT_SPECIFIED" , +"DLB_ST_SMALL_POOL_NOT_SPECIFIED" , +"DLB_ST_NEITHER_POOL_SPECIFIED" , +"DLB_ST_DOMAIN_NOT_STARTED" , +"DLB_ST_INVALID_MEASUREMENT_DURATION" , +"DLB_ST_INVALID_PERF_METRIC_GROUP_ID" , +"DLB_ST_LDB_PORT_REQUIRED_FOR_LDB_QUEUES" , +"DLB_ST_DOMAIN_RESET_FAILED" , +"DLB_ST_MBOX_ERROR" , +"DLB_ST_INVALID_HIST_LIST_DEPTH" , +"DLB_ST_NO_MEMORY" , +}; + +struct dlb_cmd_response { +__u32 status;/* Interpret using enum dlb_error */ +__u32 id; +}; + +/******************************/ +/* 'dlb' device file commands */ +/******************************/ + +#define DLB_DEVICE_VERSION (x) (((x) >> 8) & 0xFF) +#define DLB_DEVICE_REVISION (x) ((x) & 0xFF) + +enum dlb_revisions { +DLB_REV_A0 =0 , +DLB_REV_A1 =1 , +DLB_REV_A2 =2 , +DLB_REV_A3 =3 , +DLB_REV_B0 =4 , +}; + +/* +* DLB_CMD_GET_DEVICE_VERSION: Query the DLB device version. +* +* This ioctl interface is the same in all driver versions and is always +* the first ioctl. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id[7:0]: Device revision. +* response.id[15:8]: Device version. +*/ + +struct dlb_get_device_version_args { +/* Output parameters */ +__u64 response; +}; + +#define DLB_VERSION_MAJOR_NUMBER 10 +#define DLB_VERSION_MINOR_NUMBER 7 +#define DLB_VERSION_REVISION_NUMBER 9 +#define DLB_VERSION (DLB_VERSION_MAJOR_NUMBER << 24 | \ + DLB_VERSION_MINOR_NUMBER << 16 | \ + DLB_VERSION_REVISION_NUMBER) + +#define DLB_VERSION_GET_MAJOR_NUMBER (x) (((x) >> 24) & 0xFF) +#define DLB_VERSION_GET_MINOR_NUMBER (x) (((x) >> 16) & 0xFF) +#define DLB_VERSION_GET_REVISION_NUMBER (x) ((x) & 0xFFFF) + +static inline __u8 dlb_version_incompatible (__u32 version) +{ +__u8 inc; + + inc =DLB_VERSION_GET_MAJOR_NUMBER (version) !=DLB_VERSION_MAJOR_NUMBER ; + inc |= (int )DLB_VERSION_GET_MINOR_NUMBER (version) < +DLB_VERSION_MINOR_NUMBER ; + +return inc; +} + +/* +* DLB_CMD_GET_DRIVER_VERSION: Query the DLB driver version. The major number +* is changed when there is an ABI-breaking change, the minor number is +* changed if the API is changed in a backwards-compatible way, and the +* revision number is changed for fixes that don't affect the API. +* +* If the kernel driver's API version major number and the header's +* DLB_VERSION_MAJOR_NUMBER differ, the two are incompatible, or if the +* major numbers match but the kernel driver's minor number is less than +* the header file's, they are incompatible. The DLB_VERSION_INCOMPATIBLE +* macro should be used to check for compatibility. +* +* This ioctl interface is the same in all driver versions. Applications +* should check the driver version before performing any other ioctl +* operations. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: Driver API version. Use the DLB_VERSION_GET_MAJOR_NUMBER, +* DLB_VERSION_GET_MINOR_NUMBER, and +* DLB_VERSION_GET_REVISION_NUMBER macros to interpret the field. +*/ + +struct dlb_get_driver_version_args { +/* Output parameters */ +__u64 response; +}; + +/* +* DLB_CMD_CREATE_SCHED_DOMAIN: Create a DLB scheduling domain and reserve the +* resources (queues, ports, etc.) that it contains. +* +* Input parameters: +* - num_ldb_queues: Number of load-balanced queues. +* - num_ldb_ports: Number of load-balanced ports. +* - num_dir_ports: Number of directed ports. A directed port has one directed +* queue, so no num_dir_queues argument is necessary. +* - num_atomic_inflights: This specifies the amount of temporary atomic QE +* storage for the domain. This storage is divided among the domain's +* load-balanced queues that are configured for atomic scheduling. +* - num_hist_list_entries: Amount of history list storage. This is divided +* among the domain's CQs. +* - num_ldb_credits: Amount of load-balanced QE storage (QED). QEs occupy this +* space until they are scheduled to a load-balanced CQ. One credit +* represents the storage for one QE. +* - num_dir_credits: Amount of directed QE storage (DQED). QEs occupy this +* space until they are scheduled to a directed CQ. One credit represents +* the storage for one QE. +* - num_ldb_credit_pools: Number of pools into which the load-balanced credits +* are placed. +* - num_dir_credit_pools: Number of pools into which the directed credits are +* placed. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: domain ID. +*/ +struct dlb_create_sched_domain_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 num_ldb_queues; +__u32 num_ldb_ports; +__u32 num_dir_ports; +__u32 num_atomic_inflights; +__u32 num_hist_list_entries; +__u32 num_ldb_credits; +__u32 num_dir_credits; +__u32 num_ldb_credit_pools; +__u32 num_dir_credit_pools; +}; + +/* +* DLB_CMD_GET_NUM_RESOURCES: Return the number of available resources +* (queues, ports, etc.) that this device owns. +* +* Output parameters: +* - num_domains: Number of available scheduling domains. +* - num_ldb_queues: Number of available load-balanced queues. +* - num_ldb_ports: Number of available load-balanced ports. +* - num_dir_ports: Number of available directed ports. There is one directed +* queue for every directed port. +* - num_atomic_inflights: Amount of available temporary atomic QE storage. +* - max_contiguous_atomic_inflights: When a domain is created, the temporary +* atomic QE storage is allocated in a contiguous chunk. This return value +* is the longest available contiguous range of atomic QE storage. +* - num_hist_list_entries: Amount of history list storage. +* - max_contiguous_hist_list_entries: History list storage is allocated in +* a contiguous chunk, and this return value is the longest available +* contiguous range of history list entries. +* - num_ldb_credits: Amount of available load-balanced QE storage. +* - max_contiguous_ldb_credits: QED storage is allocated in a contiguous +* chunk, and this return value is the longest available contiguous range +* of load-balanced credit storage. +* - num_dir_credits: Amount of available directed QE storage. +* - max_contiguous_dir_credits: DQED storage is allocated in a contiguous +* chunk, and this return value is the longest available contiguous range +* of directed credit storage. +* - num_ldb_credit_pools: Number of available load-balanced credit pools. +* - num_dir_credit_pools: Number of available directed credit pools. +* - padding0: Reserved for future use. +*/ +struct dlb_get_num_resources_args { +/* Output parameters */ +__u32 num_sched_domains; +__u32 num_ldb_queues; +__u32 num_ldb_ports; +__u32 num_dir_ports; +__u32 num_atomic_inflights; +__u32 max_contiguous_atomic_inflights; +__u32 num_hist_list_entries; +__u32 max_contiguous_hist_list_entries; +__u32 num_ldb_credits; +__u32 max_contiguous_ldb_credits; +__u32 num_dir_credits; +__u32 max_contiguous_dir_credits; +__u32 num_ldb_credit_pools; +__u32 num_dir_credit_pools; +__u32 padding0; +}; + +/* +* DLB_CMD_SET_SN_ALLOCATION: Configure a sequence number group +* +* Input parameters: +* - group: Sequence number group ID. +* - num: Number of sequence numbers per queue. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +*/ +struct dlb_set_sn_allocation_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 group; +__u32 num; +}; + +/* +* DLB_CMD_GET_SN_ALLOCATION: Get a sequence number group's configuration +* +* Input parameters: +* - group: Sequence number group ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: Specified group's number of sequence numbers per queue. +*/ +struct dlb_get_sn_allocation_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 group; +__u32 padding0; +}; + +enum dlb_cq_poll_modes { +DLB_CQ_POLL_MODE_STD , +DLB_CQ_POLL_MODE_SPARSE , + +/* NUM_DLB_CQ_POLL_MODE must be last */ +NUM_DLB_CQ_POLL_MODE , +}; + +/* +* DLB_CMD_QUERY_CQ_POLL_MODE: Query the CQ poll mode the kernel driver is using +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: CQ poll mode (see enum dlb_cq_poll_modes). +*/ +struct dlb_query_cq_poll_mode_args { +/* Output parameters */ +__u64 response; +}; + +/* +* DLB_CMD_GET_SN_OCCUPANCY: Get a sequence number group's occupancy +* +* Each sequence number group has one or more slots, depending on its +* configuration. I.e.: +* - If configured for 1024 sequence numbers per queue, the group has 1 slot +* - If configured for 512 sequence numbers per queue, the group has 2 slots +* ... +* - If configured for 32 sequence numbers per queue, the group has 32 slots +* +* This ioctl returns the group's number of in-use slots. If its occupancy is +* 0, the group's sequence number allocation can be reconfigured. +* +* Input parameters: +* - group: Sequence number group ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: Specified group's number of used slots. +*/ +struct dlb_get_sn_occupancy_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 group; +__u32 padding0; +}; + +enum dlb_user_interface_commands { +DLB_CMD_GET_DEVICE_VERSION , +DLB_CMD_CREATE_SCHED_DOMAIN , +DLB_CMD_GET_NUM_RESOURCES , +DLB_CMD_GET_DRIVER_VERSION , +DLB_CMD_SAMPLE_PERF_COUNTERS , +DLB_CMD_SET_SN_ALLOCATION , +DLB_CMD_GET_SN_ALLOCATION , +DLB_CMD_MEASURE_SCHED_COUNTS , +DLB_CMD_QUERY_CQ_POLL_MODE , +DLB_CMD_GET_SN_OCCUPANCY , + +/* NUM_DLB_CMD must be last */ +NUM_DLB_CMD , +}; + +/*******************************/ +/* 'domain' device file alerts */ +/*******************************/ + +/* Scheduling domain device files can be read to receive domain-specific +* notifications, for alerts such as hardware errors. +* +* Each alert is encoded in a 16B message. The first 8B contains the alert ID, +* and the second 8B is optional and contains additional information. +* Applications should cast read data to a struct dlb_domain_alert, and +* interpret the struct's alert_id according to dlb_domain_alert_id. The read +* length must be 16B, or the function will return -EINVAL. +* +* Reads are destructive, and in the case of multiple file descriptors for the +* same domain device file, an alert will be read by only one of the file +* descriptors. +* +* The driver stores alerts in a fixed-size alert ring until they are read. If +* the alert ring fills completely, subsequent alerts will be dropped. It is +* recommended that DLB applications dedicate a thread to perform blocking +* reads on the device file. +*/ +enum dlb_domain_alert_id { +/* A destination domain queue that this domain connected to has +* unregistered, and can no longer be sent to. The aux alert data +* contains the queue ID. +*/ +DLB_DOMAIN_ALERT_REMOTE_QUEUE_UNREGISTER , +/* A producer port in this domain attempted to send a QE without a +* credit. aux_alert_data[7:0] contains the port ID, and +* aux_alert_data[15:8] contains a flag indicating whether the port is +* load-balanced (1) or directed (0). +*/ +DLB_DOMAIN_ALERT_PP_OUT_OF_CREDITS , +/* Software issued an illegal enqueue for a port in this domain. An +* illegal enqueue could be: +* - Illegal (excess) completion +* - Illegal fragment +* - Illegal enqueue command +* aux_alert_data[7:0] contains the port ID, and aux_alert_data[15:8] +* contains a flag indicating whether the port is load-balanced (1) or +* directed (0). +*/ +DLB_DOMAIN_ALERT_PP_ILLEGAL_ENQ , +/* Software issued excess CQ token pops for a port in this domain. +* aux_alert_data[7:0] contains the port ID, and aux_alert_data[15:8] +* contains a flag indicating whether the port is load-balanced (1) or +* directed (0). +*/ +DLB_DOMAIN_ALERT_PP_EXCESS_TOKEN_POPS , +/* A enqueue contained either an invalid command encoding or a REL, +* REL_T, RLS, FWD, FWD_T, FRAG, or FRAG_T from a directed port. +* +* aux_alert_data[7:0] contains the port ID, and aux_alert_data[15:8] +* contains a flag indicating whether the port is load-balanced (1) or +* directed (0). +*/ +DLB_DOMAIN_ALERT_ILLEGAL_HCW , +/* The QID must be valid and less than 128. +* +* aux_alert_data[7:0] contains the port ID, and aux_alert_data[15:8] +* contains a flag indicating whether the port is load-balanced (1) or +* directed (0). +*/ +DLB_DOMAIN_ALERT_ILLEGAL_QID , +/* An enqueue went to a disabled QID. +* +* aux_alert_data[7:0] contains the port ID, and aux_alert_data[15:8] +* contains a flag indicating whether the port is load-balanced (1) or +* directed (0). +*/ +DLB_DOMAIN_ALERT_DISABLED_QID , +/* The device containing this domain was reset. All applications using +* the device need to exit for the driver to complete the reset +* procedure. +* +* aux_alert_data doesn't contain any information for this alert. +*/ +DLB_DOMAIN_ALERT_DEVICE_RESET , +/* User-space has enqueued an alert. +* +* aux_alert_data contains user-provided data. +*/ +DLB_DOMAIN_ALERT_USER , + +/* Number of DLB domain alerts */ +NUM_DLB_DOMAIN_ALERTS +}; + +static const char dlb_domain_alert_strings[][128 ] = { +"DLB_DOMAIN_ALERT_REMOTE_QUEUE_UNREGISTER" , +"DLB_DOMAIN_ALERT_PP_OUT_OF_CREDITS" , +"DLB_DOMAIN_ALERT_PP_ILLEGAL_ENQ" , +"DLB_DOMAIN_ALERT_PP_EXCESS_TOKEN_POPS" , +"DLB_DOMAIN_ALERT_ILLEGAL_HCW" , +"DLB_DOMAIN_ALERT_ILLEGAL_QID" , +"DLB_DOMAIN_ALERT_DISABLED_QID" , +"DLB_DOMAIN_ALERT_DEVICE_RESET" , +"DLB_DOMAIN_ALERT_USER" , +}; + +struct dlb_domain_alert { +__u64 alert_id; +__u64 aux_alert_data; +}; + +/*********************************/ +/* 'domain' device file commands */ +/*********************************/ + +/* +* DLB_DOMAIN_CMD_CREATE_LDB_POOL: Configure a load-balanced credit pool. +* Input parameters: +* - num_ldb_credits: Number of load-balanced credits (QED space) for this +* pool. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: pool ID. +*/ +struct dlb_create_ldb_pool_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 num_ldb_credits; +__u32 padding0; +}; + +/* +* DLB_DOMAIN_CMD_CREATE_DIR_POOL: Configure a directed credit pool. +* Input parameters: +* - num_dir_credits: Number of directed credits (DQED space) for this pool. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: Pool ID. +*/ +struct dlb_create_dir_pool_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 num_dir_credits; +__u32 padding0; +}; + +/* +* DLB_DOMAIN_CMD_CREATE_LDB_QUEUE: Configure a load-balanced queue. +* Input parameters: +* - num_atomic_inflights: This specifies the amount of temporary atomic QE +* storage for this queue. If zero, the queue will not support atomic +* scheduling. +* - num_sequence_numbers: This specifies the number of sequence numbers used +* by this queue. If zero, the queue will not support ordered scheduling. +* If non-zero, the queue will not support unordered scheduling. +* - num_qid_inflights: The maximum number of QEs that can be inflight +* (scheduled to a CQ but not completed) at any time. If +* num_sequence_numbers is non-zero, num_qid_inflights must be set equal +* to num_sequence_numbers. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: Queue ID. +*/ +struct dlb_create_ldb_queue_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 num_sequence_numbers; +__u32 num_qid_inflights; +__u32 num_atomic_inflights; +__u32 padding0; +}; + +/* +* DLB_DOMAIN_CMD_CREATE_DIR_QUEUE: Configure a directed queue. +* Input parameters: +* - port_id: Port ID. If the corresponding directed port is already created, +* specify its ID here. Else this argument must be 0xFFFFFFFF to indicate +* that the queue is being created before the port. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: Queue ID. +*/ +struct dlb_create_dir_queue_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__s32 port_id; +__u32 padding0; +}; + +/* +* DLB_DOMAIN_CMD_CREATE_LDB_PORT: Configure a load-balanced port. +* Input parameters: +* - ldb_credit_pool_id: Load-balanced credit pool this port will belong to. +* - dir_credit_pool_id: Directed credit pool this port will belong to. +* - ldb_credit_high_watermark: Number of load-balanced credits from the pool +* that this port will own. +* +* If this port's scheduling domain doesn't have any load-balanced queues, +* this argument is ignored and the port is given no load-balanced +* credits. +* - dir_credit_high_watermark: Number of directed credits from the pool that +* this port will own. +* +* If this port's scheduling domain doesn't have any directed queues, +* this argument is ignored and the port is given no directed credits. +* - ldb_credit_low_watermark: Load-balanced credit low watermark. When the +* port's credits reach this watermark, they become eligible to be +* refilled by the DLB as credits until the high watermark +* (num_ldb_credits) is reached. +* +* If this port's scheduling domain doesn't have any load-balanced queues, +* this argument is ignored and the port is given no load-balanced +* credits. +* - dir_credit_low_watermark: Directed credit low watermark. When the port's +* credits reach this watermark, they become eligible to be refilled by +* the DLB as credits until the high watermark (num_dir_credits) is +* reached. +* +* If this port's scheduling domain doesn't have any directed queues, +* this argument is ignored and the port is given no directed credits. +* - ldb_credit_quantum: Number of load-balanced credits for the DLB to refill +* per refill operation. +* +* If this port's scheduling domain doesn't have any load-balanced queues, +* this argument is ignored and the port is given no load-balanced +* credits. +* - dir_credit_quantum: Number of directed credits for the DLB to refill per +* refill operation. +* +* If this port's scheduling domain doesn't have any directed queues, +* this argument is ignored and the port is given no directed credits. +* - padding0: Reserved for future use. +* - cq_depth: Depth of the port's CQ. Must be a power-of-two between 8 and +* 1024, inclusive. +* - cq_depth_threshold: CQ depth interrupt threshold. A value of N means that +* the CQ interrupt won't fire until there are N or more outstanding CQ +* tokens. +* - cq_history_list_size: Number of history list entries. This must be greater +* than or equal to cq_depth. +* - padding1: Reserved for future use. +* - padding2: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: port ID. +*/ +struct dlb_create_ldb_port_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 ldb_credit_pool_id; +__u32 dir_credit_pool_id; +__u16 ldb_credit_high_watermark; +__u16 ldb_credit_low_watermark; +__u16 ldb_credit_quantum; +__u16 dir_credit_high_watermark; +__u16 dir_credit_low_watermark; +__u16 dir_credit_quantum; +__u16 padding0; +__u16 cq_depth; +__u16 cq_depth_threshold; +__u16 cq_history_list_size; +__u32 padding1; +}; + +/* +* DLB_DOMAIN_CMD_CREATE_DIR_PORT: Configure a directed port. +* Input parameters: +* - ldb_credit_pool_id: Load-balanced credit pool this port will belong to. +* - dir_credit_pool_id: Directed credit pool this port will belong to. +* - ldb_credit_high_watermark: Number of load-balanced credits from the pool +* that this port will own. +* +* If this port's scheduling domain doesn't have any load-balanced queues, +* this argument is ignored and the port is given no load-balanced +* credits. +* - dir_credit_high_watermark: Number of directed credits from the pool that +* this port will own. +* - ldb_credit_low_watermark: Load-balanced credit low watermark. When the +* port's credits reach this watermark, they become eligible to be +* refilled by the DLB as credits until the high watermark +* (num_ldb_credits) is reached. +* +* If this port's scheduling domain doesn't have any load-balanced queues, +* this argument is ignored and the port is given no load-balanced +* credits. +* - dir_credit_low_watermark: Directed credit low watermark. When the port's +* credits reach this watermark, they become eligible to be refilled by +* the DLB as credits until the high watermark (num_dir_credits) is +* reached. +* - ldb_credit_quantum: Number of load-balanced credits for the DLB to refill +* per refill operation. +* +* If this port's scheduling domain doesn't have any load-balanced queues, +* this argument is ignored and the port is given no load-balanced +* credits. +* - dir_credit_quantum: Number of directed credits for the DLB to refill per +* refill operation. +* - cq_depth: Depth of the port's CQ. Must be a power-of-two between 8 and +* 1024, inclusive. +* - cq_depth_threshold: CQ depth interrupt threshold. A value of N means that +* the CQ interrupt won't fire until there are N or more outstanding CQ +* tokens. +* - qid: Queue ID. If the corresponding directed queue is already created, +* specify its ID here. Else this argument must be 0xFFFFFFFF to indicate +* that the port is being created before the queue. +* - padding1: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: Port ID. +*/ +struct dlb_create_dir_port_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 ldb_credit_pool_id; +__u32 dir_credit_pool_id; +__u16 ldb_credit_high_watermark; +__u16 ldb_credit_low_watermark; +__u16 ldb_credit_quantum; +__u16 dir_credit_high_watermark; +__u16 dir_credit_low_watermark; +__u16 dir_credit_quantum; +__u16 cq_depth; +__u16 cq_depth_threshold; +__s32 queue_id; +__u32 padding1; +}; + +/* +* DLB_DOMAIN_CMD_START_DOMAIN: Mark the end of the domain configuration. This +* must be called before passing QEs into the device, and no configuration +* ioctls can be issued once the domain has started. Sending QEs into the +* device before calling this ioctl will result in undefined behavior. +* Input parameters: +* - (None) +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +*/ +struct dlb_start_domain_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +}; + +/* +* DLB_DOMAIN_CMD_MAP_QID: Map a load-balanced queue to a load-balanced port. +* Input parameters: +* - port_id: Load-balanced port ID. +* - qid: Load-balanced queue ID. +* - priority: Queue->port service priority. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +*/ +struct dlb_map_qid_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 port_id; +__u32 qid; +__u32 priority; +__u32 padding0; +}; + +/* +* DLB_DOMAIN_CMD_UNMAP_QID: Unmap a load-balanced queue to a load-balanced +* port. +* Input parameters: +* - port_id: Load-balanced port ID. +* - qid: Load-balanced queue ID. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +*/ +struct dlb_unmap_qid_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 port_id; +__u32 qid; +}; + +/* +* DLB_DOMAIN_CMD_ENABLE_LDB_PORT: Enable scheduling to a load-balanced port. +* Input parameters: +* - port_id: Load-balanced port ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +*/ +struct dlb_enable_ldb_port_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 port_id; +__u32 padding0; +}; + +/* +* DLB_DOMAIN_CMD_ENABLE_DIR_PORT: Enable scheduling to a directed port. +* Input parameters: +* - port_id: Directed port ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +*/ +struct dlb_enable_dir_port_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 port_id; +}; + +/* +* DLB_DOMAIN_CMD_DISABLE_LDB_PORT: Disable scheduling to a load-balanced port. +* Input parameters: +* - port_id: Load-balanced port ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +*/ +struct dlb_disable_ldb_port_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 port_id; +__u32 padding0; +}; + +/* +* DLB_DOMAIN_CMD_DISABLE_DIR_PORT: Disable scheduling to a directed port. +* Input parameters: +* - port_id: Directed port ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +*/ +struct dlb_disable_dir_port_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 port_id; +__u32 padding0; +}; + +/* +* DLB_DOMAIN_CMD_BLOCK_ON_CQ_INTERRUPT: Block on a CQ interrupt until a QE +* arrives for the specified port. If a QE is already present, the ioctl +* will immediately return. +* +* Note: Only one thread can block on a CQ's interrupt at a time. Doing +* otherwise can result in hung threads. +* +* Input parameters: +* - port_id: Port ID. +* - is_ldb: True if the port is load-balanced, false otherwise. +* - arm: Tell the driver to arm the interrupt. +* - cq_gen: Current CQ generation bit. +* - padding0: Reserved for future use. +* - cq_va: VA of the CQ entry where the next QE will be placed. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +*/ +struct dlb_block_on_cq_interrupt_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 port_id; +__u8 is_ldb; +__u8 arm; +__u8 cq_gen; +__u8 padding0; +__u64 cq_va; +}; + +/* +* DLB_DOMAIN_CMD_ENQUEUE_DOMAIN_ALERT: Enqueue a domain alert that will be +* read by one reader thread. +* +* Input parameters: +* - aux_alert_data: user-defined auxiliary data. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +*/ +struct dlb_enqueue_domain_alert_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u64 aux_alert_data; +}; + +/* +* DLB_DOMAIN_CMD_GET_LDB_QUEUE_DEPTH: Get a load-balanced queue's depth. +* Input parameters: +* - queue_id: The load-balanced queue ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: queue depth. +*/ +struct dlb_get_ldb_queue_depth_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 queue_id; +__u32 padding0; +}; + +/* +* DLB_DOMAIN_CMD_GET_DIR_QUEUE_DEPTH: Get a directed queue's depth. +* Input parameters: +* - queue_id: The directed queue ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: queue depth. +*/ +struct dlb_get_dir_queue_depth_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 queue_id; +__u32 padding0; +}; + +/* +* DLB_DOMAIN_CMD_PENDING_PORT_UNMAPS: Get number of queue unmap operations in +* progress for a load-balanced port. +* +* Note: This is a snapshot; the number of unmap operations in progress +* is subject to change at any time. +* +* Input parameters: +* - port_id: Load-balanced port ID. +* +* Output parameters: +* - response: pointer to a struct dlb_cmd_response. +* response.status: Detailed error code. In certain cases, such as if the +* response pointer is invalid, the driver won't set status. +* response.id: number of unmaps in progress. +*/ +struct dlb_pending_port_unmaps_args { +/* Output parameters */ +__u64 response; +/* Input parameters */ +__u32 port_id; +__u32 padding0; +}; + +enum dlb_domain_user_interface_commands { +DLB_DOMAIN_CMD_CREATE_LDB_POOL , +DLB_DOMAIN_CMD_CREATE_DIR_POOL , +DLB_DOMAIN_CMD_CREATE_LDB_QUEUE , +DLB_DOMAIN_CMD_CREATE_DIR_QUEUE , +DLB_DOMAIN_CMD_CREATE_LDB_PORT , +DLB_DOMAIN_CMD_CREATE_DIR_PORT , +DLB_DOMAIN_CMD_START_DOMAIN , +DLB_DOMAIN_CMD_MAP_QID , +DLB_DOMAIN_CMD_UNMAP_QID , +DLB_DOMAIN_CMD_ENABLE_LDB_PORT , +DLB_DOMAIN_CMD_ENABLE_DIR_PORT , +DLB_DOMAIN_CMD_DISABLE_LDB_PORT , +DLB_DOMAIN_CMD_DISABLE_DIR_PORT , +DLB_DOMAIN_CMD_BLOCK_ON_CQ_INTERRUPT , +DLB_DOMAIN_CMD_ENQUEUE_DOMAIN_ALERT , +DLB_DOMAIN_CMD_GET_LDB_QUEUE_DEPTH , +DLB_DOMAIN_CMD_GET_DIR_QUEUE_DEPTH , +DLB_DOMAIN_CMD_PENDING_PORT_UNMAPS , + +/* NUM_DLB_DOMAIN_CMD must be last */ +NUM_DLB_DOMAIN_CMD , +}; + +/* +* Base addresses for memory mapping the consumer queue (CQ) and popcount (PC) +* memory space, and producer port (PP) MMIO space. The CQ, PC, and PP +* addresses are per-port. Every address is page-separated (e.g. LDB PP 0 is at +* 0x2100000 and LDB PP 1 is at 0x2101000). +*/ +#define DLB_LDB_CQ_BASE 0x3000000 +#define DLB_LDB_CQ_MAX_SIZE 65536 +#define DLB_LDB_CQ_OFFS (id) (DLB_LDB_CQ_BASE + (id) * DLB_LDB_CQ_MAX_SIZE) + +#define DLB_DIR_CQ_BASE 0x3800000 +#define DLB_DIR_CQ_MAX_SIZE 65536 +#define DLB_DIR_CQ_OFFS (id) (DLB_DIR_CQ_BASE + (id) * DLB_DIR_CQ_MAX_SIZE) + +#define DLB_LDB_PC_BASE 0x2300000 +#define DLB_LDB_PC_MAX_SIZE 4096 +#define DLB_LDB_PC_OFFS (id) (DLB_LDB_PC_BASE + (id) * DLB_LDB_PC_MAX_SIZE) + +#define DLB_DIR_PC_BASE 0x2200000 +#define DLB_DIR_PC_MAX_SIZE 4096 +#define DLB_DIR_PC_OFFS (id) (DLB_DIR_PC_BASE + (id) * DLB_DIR_PC_MAX_SIZE) + +#define DLB_LDB_PP_BASE 0x2100000 +#define DLB_LDB_PP_MAX_SIZE 4096 +#define DLB_LDB_PP_OFFS (id) (DLB_LDB_PP_BASE + (id) * DLB_LDB_PP_MAX_SIZE) + +#define DLB_DIR_PP_BASE 0x2000000 +#define DLB_DIR_PP_MAX_SIZE 4096 +#define DLB_DIR_PP_OFFS (id) (DLB_DIR_PP_BASE + (id) * DLB_DIR_PP_MAX_SIZE) + +#endif /* __DLB_USER_H */
Signature · 1 line
1.7.10
PATCH 9/27 event/dlb: add inline functions used in multiple files
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb_inline_fns.h | 80 ++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 drivers/event/dlb/dlb_inline_fns.h
drivers/event/dlb/dlb_inline_fns.h new file +80 -0
@@ -0,0 +1,80 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#include "rte_memcpy.h" +#include "rte_io.h" + +/* Inline functions required in more than one source file. +*/ + +static inline struct dlb_eventdev * +dlb_pmd_priv (const struct rte_eventdev *eventdev) +{ +return eventdev->data->dev_private; +} + +static inline void +dlb_umonitor (volatile void *addr) +{ + asmvolatile (".byte 0xf3, 0x0f, 0xae, 0xf7\t\n" + : + :"D" (addr)); +} + +static inline void +dlb_umwait (int state,uint64_t timeout) +{ +uint32_t eax = timeout &UINT32_MAX ; +uint32_t edx = timeout >>32 ; + + asmvolatile (".byte 0xf2, 0x0f, 0xae, 0xf7\t\n" + : + :"D" (state),"a" (eax),"d" (edx)); +} + +static inline void +dlb_movntdq (void *qe4,void *pp_addr) +{ +/* Move entire 64B cache line of QEs, 128 bits (16B) at a time. */ +long long *_qe = (long long *)qe4; +__v2di src_data0 = (__v2di ){_qe[0 ], _qe[1 ]}; +__v2di src_data1 = (__v2di ){_qe[2 ], _qe[3 ]}; +__v2di src_data2 = (__v2di ){_qe[4 ], _qe[5 ]}; +__v2di src_data3 = (__v2di ){_qe[6 ], _qe[7 ]}; + +__builtin_ia32_movntdq ((__v2di *)pp_addr +0 , (__v2di )src_data0); +rte_wmb (); +__builtin_ia32_movntdq ((__v2di *)pp_addr +1 , (__v2di )src_data1); +rte_wmb (); +__builtin_ia32_movntdq ((__v2di *)pp_addr +2 , (__v2di )src_data2); +rte_wmb (); +__builtin_ia32_movntdq ((__v2di *)pp_addr +3 , (__v2di )src_data3); +rte_wmb (); +} + +static inline void +dlb_movntdq_single (void *qe4,void *pp_addr) +{ +long long *_qe = (long long *)qe4; +__v2di src_data0 = (__v2di ){_qe[0 ], _qe[1 ]}; + +__builtin_ia32_movntdq ((__v2di *)pp_addr, (__v2di )src_data0); +} + +static inline void +dlb_cldemote (void *addr) +{ +/* Load addr into RSI, then demote the cache line of the address +* contained in that register. +*/ + asmvolatile (".byte 0x0f, 0x1c, 0x06" ::"S" (addr)); +} + +static inline void +dlb_movdir64b (void *qe4,void *pp_addr) +{ + asmvolatile (".byte 0x66, 0x0f, 0x38, 0xf8, 0x02" + : + :"a" (pp_addr),"d" (qe4)); +}
Signature · 1 line
1.7.10
Re: [dpdk-dev] [PATCH 09/27] event/dlb: add inline functions used in multiple files
Quoted 85 lines · Tim McDaniel
From: "McDaniel, Timothy" <[email protected]>
Signed-off-by McDaniel, Timothy <[email protected]>--- drivers/event/dlb/dlb_inline_fns.h | 80 ++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 drivers/event/dlb/dlb_inline_fns.hdrivers/event/dlb/dlb_inline_fns.h new file +70
-0@@ -0,0 +1,80 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#include "rte_memcpy.h" +#include "rte_io.h" + +/* Inline functions required in more than one source file. +*/ + +static inline struct dlb_eventdev * +dlb_pmd_priv (const struct rte_eventdev *eventdev) +{ +return eventdev->data->dev_private; +} + +static inline void +dlb_umonitor (volatile void *addr) +{ + asmvolatile (".byte 0xf3, 0x0f, 0xae, 0xf7\t\n" + : + :"D" (addr)); +} + +static inline void +dlb_umwait (int state,uint64_t timeout) +{ +uint32_t eax = timeout &UINT32_MAX ; +uint32_t edx = timeout >>32 ; + + asmvolatile (".byte 0xf2, 0x0f, 0xae, 0xf7\t\n" + : + :"D" (state),"a" (eax),"d" (edx)); +} + +static inline void +dlb_movntdq (void *qe4,void *pp_addr) +{ +/* Move entire 64B cache line of QEs, 128 bits (16B) at a time. */ +long long *_qe = (long long *)qe4; +__v2di src_data0 = (__v2di ){_qe[0 ], _qe[1 ]}; +__v2di src_data1 = (__v2di ){_qe[2 ], _qe[3 ]}; +__v2di src_data2 = (__v2di ){_qe[4 ], _qe[5 ]}; +__v2di src_data3 = (__v2di ){_qe[6 ], _qe[7 ]}; + +__builtin_ia32_movntdq ((__v2di *)pp_addr +0 , (__v2di )src_data0); +rte_wmb (); +__builtin_ia32_movntdq ((__v2di *)pp_addr +1 , (__v2di )src_data1); +rte_wmb (); +__builtin_ia32_movntdq ((__v2di *)pp_addr +2 , (__v2di )src_data2); +rte_wmb (); +__builtin_ia32_movntdq ((__v2di *)pp_addr +3 , (__v2di )src_data3); +rte_wmb (); +} + +static inline void +dlb_movntdq_single (void *qe4,void *pp_addr) +{ +long long *_qe = (long long *)qe4; +__v2di src_data0 = (__v2di ){_qe[0 ], _qe[1 ]}; + +__builtin_ia32_movntdq ((__v2di *)pp_addr, (__v2di )src_data0); +} + +static inline void +dlb_cldemote (void *addr) +{ +/* Load addr into RSI, then demote the cache line of the address +* contained in that register. +*/
drivers/event/dlb/dlb_inline_fns.h line 80
Quoted 10 lines
drivers/event/dlb/dlb_inline_fns.h new file +10
-0+ asmvolatile (".byte 0x0f, 0x1c, 0x06" ::"S" (addr)); +} + +static inline void +dlb_movdir64b (void *qe4,void *pp_addr) +{ + asmvolatile (".byte 0x66, 0x0f, 0x38, 0xf8, 0x02" + : + :"a" (pp_addr),"d" (qe4)); +}
Although these are mostly x86-specific, a number of these functions may be needed in other places in DPDK also, so they should perhaps be put in a common location.
Also, for the use of these, can you try and align the parameter names and order as far as possible with those of the equivalent x86 intrinsics documented at
https://software.intel.com/sites/landingpage/IntrinsicsGuide/. This will make it easier in the future to switch to use the proper intrinsics rather than our own versions. For example, _movdir64b intrinsic has parameters "void *dst, const void *src", which is reversed to what you have here.
Thanks,
/Bruce
Re: [dpdk-dev] [PATCH 09/27] event/dlb: add inline functions used in multiple files
Quoted 120 lines
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Tuesday, July 7, 2020 7:03 AM
To: McDaniel, Timothy <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 09/27] event/dlb: add inline functions used in multiple filesQuoted 5 lines · Tim McDaniel
From: "McDaniel, Timothy" <[email protected]>
Signed-off-by McDaniel, Timothy <[email protected]>--- drivers/event/dlb/dlb_inline_fns.h | 80++++++++++++++++++++++++++++++++++++
Review
Quoted 4 lines
1 file changed, 80 insertions(+) create mode 100644 drivers/event/dlb/dlb_inline_fns.h(quoted diff) +0
-0
b/drivers/event/dlb/dlb_inline_fns.hQuoted 76 lines
new file mode 100644
index 0000000..86f85aadrivers/event/dlb/dlb_inline_fns.h new file +71
-0@@ -0,0 +1,80 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#include "rte_memcpy.h" +#include "rte_io.h" + +/* Inline functions required in more than one source file. +*/ + +static inline struct dlb_eventdev * +dlb_pmd_priv (const struct rte_eventdev *eventdev) +{ +return eventdev->data->dev_private; +} + +static inline void +dlb_umonitor (volatile void *addr) +{ + asmvolatile (".byte 0xf3, 0x0f, 0xae, 0xf7\t\n" + : + :"D" (addr)); +} + +static inline void +dlb_umwait (int state,uint64_t timeout) +{ +uint32_t eax = timeout &UINT32_MAX ; +uint32_t edx = timeout >>32 ; + + asmvolatile (".byte 0xf2, 0x0f, 0xae, 0xf7\t\n" + : + :"D" (state),"a" (eax),"d" (edx)); +} + +static inline void +dlb_movntdq (void *qe4,void *pp_addr) +{ +/* Move entire 64B cache line of QEs, 128 bits (16B) at a time. */ +long long *_qe = (long long *)qe4; +__v2di src_data0 = (__v2di ){_qe[0 ], _qe[1 ]}; +__v2di src_data1 = (__v2di ){_qe[2 ], _qe[3 ]}; +__v2di src_data2 = (__v2di ){_qe[4 ], _qe[5 ]}; +__v2di src_data3 = (__v2di ){_qe[6 ], _qe[7 ]}; + +__builtin_ia32_movntdq ((__v2di *)pp_addr +0 , (__v2di )src_data0); +rte_wmb (); +__builtin_ia32_movntdq ((__v2di *)pp_addr +1 , (__v2di )src_data1); +rte_wmb (); +__builtin_ia32_movntdq ((__v2di *)pp_addr +2 , (__v2di )src_data2); +rte_wmb (); +__builtin_ia32_movntdq ((__v2di *)pp_addr +3 , (__v2di )src_data3); +rte_wmb (); +} + +static inline void +dlb_movntdq_single (void *qe4,void *pp_addr) +{ +long long *_qe = (long long *)qe4; +__v2di src_data0 = (__v2di ){_qe[0 ], _qe[1 ]}; + +__builtin_ia32_movntdq ((__v2di *)pp_addr, (__v2di )src_data0); +} + +static inline void +dlb_cldemote (void *addr) +{ +/* Load addr into RSI, then demote the cache line of the address +* contained in that register. +*/ + asmvolatile (".byte 0x0f, 0x1c, 0x06" ::"S" (addr));drivers/event/dlb/dlb_inline_fns.h
Quoted 10 lines
drivers/event/dlb/dlb_inline_fns.h new file +9
-1+} + +static inline void +dlb_movdir64b (void *qe4,void *pp_addr) +{ + asmvolatile (".byte 0x66, 0x0f, 0x38, 0xf8, 0x02" + : + :"a" (pp_addr),"d" (qe4)); +} --Although these are mostly x86-specific, a number of these functions may be needed in other places in DPDK also, so they should perhaps be put in a common location.
Also, for the use of these, can you try and align the parameter names and order as far as possible with those of the equivalent x86 intrinsics documented at
https://software.intel.com/sites/landingpage/IntrinsicsGuide/. This will make it easier in the future to switch to use the proper intrinsics rather than our own versions. For example,_movdir64bintrinsic has parameters "void *dst, const void *src", which is reversed to what you have here.
Quoted 2 lines
Thanks,
/Bruce
Thanks for the suggestions, Bruce. I will make those changes.
PATCH 10/27 event/dlb: add PFPMD-specific interface layer to shared code
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/Makefile | 2 + drivers/event/dlb/meson.build | 4 +- drivers/event/dlb/pf/dlb_main.c | 609 ++++++++++++++++++++++++++++++ drivers/event/dlb/pf/dlb_main.h | 54 +++ drivers/event/dlb/pf/dlb_pf.c | 776 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 1444 insertions(+), 1 deletion(-) create mode 100644 drivers/event/dlb/pf/dlb_main.c create mode 100644 drivers/event/dlb/pf/dlb_main.h create mode 100644 drivers/event/dlb/pf/dlb_pf.c
drivers/event/dlb/Makefile +2 -0
@@ -20,6 +20,8 @@ LIBABIVER := 1 EXPORT_MAP := rte_pmd_dlb_event_version.map # library source files +SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += pf/dlb_pf.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += pf/dlb_main.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += pf/base/dlb_resource.c # export include files
drivers/event/dlb/meson.build +3 -1
@@ -1,7 +1,9 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019-2020 Intel Corporation -sources = files('pf/base/dlb_resource.c' +sources = files('pf/dlb_pf.c', + 'pf/dlb_main.c', + 'pf/base/dlb_resource.c' ) deps += ['mbuf', 'mempool', 'ring', 'pci', 'bus_pci']
drivers/event/dlb/pf/dlb_main.c new file +609 -0
609 lines not on this page · raw message
drivers/event/dlb/pf/dlb_main.h new file +54 -0
54 lines not on this page · raw message
drivers/event/dlb/pf/dlb_pf.c new file +776 -0
776 lines not on this page · raw message
Signature · 1 line
1.7.10
PATCH 11/27 event/dlb: add flexible PMD to device interfaces
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
The PMD uses function pointers so that it can be executed as a bifurcated PMD accessing hardware via a linux kernel mode driver, or as a PF PMD where it has complete control of the hardware device.
Note that this interface is not used in the data path.
--- drivers/event/dlb/Makefile | 1 + drivers/event/dlb/dlb_iface.c | 105 +++++++++++++++++++++++++++++++++++++++++ drivers/event/dlb/dlb_iface.h | 92 ++++++++++++++++++++++++++++++++++++ drivers/event/dlb/meson.build | 3 +- 4 files changed, 200 insertions(+), 1 deletion(-) create mode 100644 drivers/event/dlb/dlb_iface.c create mode 100644 drivers/event/dlb/dlb_iface.h
drivers/event/dlb/Makefile +1 -0
@@ -20,6 +20,7 @@ LIBABIVER := 1 EXPORT_MAP := rte_pmd_dlb_event_version.map # library source files +SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += dlb_iface.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += pf/dlb_pf.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += pf/dlb_main.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += pf/base/dlb_resource.c
drivers/event/dlb/dlb_iface.c new file +105 -0
105 lines not on this page · raw message
drivers/event/dlb/dlb_iface.h new file +92 -0
92 lines not on this page · raw message
drivers/event/dlb/meson.build +2 -1
@@ -1,7 +1,8 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019-2020 Intel Corporation -sources = files('pf/dlb_pf.c', +sources = files('dlb_iface.c', + 'pf/dlb_pf.c', 'pf/dlb_main.c', 'pf/base/dlb_resource.c' )
Signature · 1 line
1.7.10
PATCH 12/27 event/dlb: add the PMD's public interfaces
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/Makefile | 2 + drivers/event/dlb/meson.build | 4 +- drivers/event/dlb/rte_pmd_dlb.c | 38 +++++++++++++ drivers/event/dlb/rte_pmd_dlb.h | 69 +++++++++++++++++++++++ drivers/event/dlb/rte_pmd_dlb_event_version.map | 6 ++ 5 files changed, 118 insertions(+), 1 deletion(-) create mode 100644 drivers/event/dlb/rte_pmd_dlb.c create mode 100644 drivers/event/dlb/rte_pmd_dlb.h create mode 100644 drivers/event/dlb/rte_pmd_dlb_event_version.map
drivers/event/dlb/Makefile +2 -0
@@ -20,11 +20,13 @@ LIBABIVER := 1 EXPORT_MAP := rte_pmd_dlb_event_version.map # library source files +SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += rte_pmd_dlb.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += dlb_iface.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += pf/dlb_pf.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += pf/dlb_main.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += pf/base/dlb_resource.c # export include files +SYMLINK-y-include += rte_pmd_dlb.h include $(RTE_SDK)/mk/rte.lib.mk
drivers/event/dlb/meson.build +3 -1
@@ -1,10 +1,12 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019-2020 Intel Corporation -sources = files('dlb_iface.c', +sources = files('rte_pmd_dlb.c', + 'dlb_iface.c', 'pf/dlb_pf.c', 'pf/dlb_main.c', 'pf/base/dlb_resource.c' ) deps += ['mbuf', 'mempool', 'ring', 'pci', 'bus_pci'] +install_headers('rte_pmd_dlb.h')
drivers/event/dlb/rte_pmd_dlb.c new file +38 -0
@@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2020 Intel Corporation +*/ + +#include "rte_eventdev.h" +#include "rte_eventdev_pmd.h" +#include "rte_pmd_dlb.h" +#include "dlb_priv.h" +#include "dlb_inline_fns.h" + +int +rte_pmd_dlb_set_token_pop_mode (uint8_t dev_id, +uint8_t port_id, +enum dlb_token_pop_mode mode) +{ +struct dlb_eventdev *dlb; +struct rte_eventdev *dev; + +RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET (dev_id, -EINVAL ); + dev = &rte_eventdevs[dev_id]; + + dlb =dlb_pmd_priv (dev); + +if (mode >=NUM_TOKEN_POP_MODES ) +return -EINVAL ; + +/* The event device must be configured, but not yet started */ +if (!dlb->configured || dlb->run_state !=DLB_RUN_STATE_STOPPED ) +return -EINVAL ; + +/* The token pop mode must be set before configuring the port */ +if (port_id >= dlb->num_ports || dlb->ev_ports[port_id].setup_done) +return -EINVAL ; + + dlb->ev_ports[port_id].qm_port.token_pop_mode = mode; + +return 0 ; +}
drivers/event/dlb/rte_pmd_dlb.h new file +69 -0
69 lines not on this page · raw message
drivers/event/dlb/rte_pmd_dlb_event_version.map new file +6 -0
@@ -0,0 +1,6 @@ +DPDK_20.08 { + global: + + rte_pmd_dlb_set_token_pop_mode; + local: *; +};
Signature · 1 line
1.7.10
PATCH 13/27 event/dlb: add xstats support
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/Makefile | 1 + drivers/event/dlb/dlb_xstats.c | 1249 ++++++++++++++++++++++++++++++++++++++++ drivers/event/dlb/meson.build | 3 +- 3 files changed, 1252 insertions(+), 1 deletion(-) create mode 100644 drivers/event/dlb/dlb_xstats.c
drivers/event/dlb/Makefile +1 -0
@@ -20,6 +20,7 @@ LIBABIVER := 1 EXPORT_MAP := rte_pmd_dlb_event_version.map # library source files +SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += dlb_xstats.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += rte_pmd_dlb.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += dlb_iface.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += pf/dlb_pf.c
drivers/event/dlb/dlb_xstats.c new file +1249 -0
1,249 lines not on this page · raw message
drivers/event/dlb/meson.build +2 -1
@@ -1,7 +1,8 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019-2020 Intel Corporation -sources = files('rte_pmd_dlb.c', +sources = files('dlb_xstats.c', + 'rte_pmd_dlb.c', 'dlb_iface.c', 'pf/dlb_pf.c', 'pf/dlb_main.c',
Signature · 1 line
1.7.10
PATCH 14/27 event/dlb: add PMD self-tests
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/Makefile | 1 + drivers/event/dlb/dlb_selftest.c | 1564 ++++++++++++++++++++++++++++++++++++++ drivers/event/dlb/meson.build | 3 +- 3 files changed, 1567 insertions(+), 1 deletion(-) create mode 100644 drivers/event/dlb/dlb_selftest.c
drivers/event/dlb/Makefile +1 -0
@@ -20,6 +20,7 @@ LIBABIVER := 1 EXPORT_MAP := rte_pmd_dlb_event_version.map # library source files +SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += dlb_selftest.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += dlb_xstats.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += rte_pmd_dlb.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += dlb_iface.c
drivers/event/dlb/dlb_selftest.c new file +1564 -0
1,564 lines not on this page · raw message
drivers/event/dlb/meson.build +2 -1
@@ -1,7 +1,8 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019-2020 Intel Corporation -sources = files('dlb_xstats.c', +sources = files('dlb_selftest.c', + 'dlb_xstats.c', 'rte_pmd_dlb.c', 'dlb_iface.c', 'pf/dlb_pf.c',
Signature · 1 line
1.7.10
Re: [dpdk-dev] [PATCH 14/27] event/dlb: add PMD self-tests
Hi Tim,
drivers/event/dlb/dlb_selftest.c line 22
Quoted 11 lines
drivers/event/dlb/dlb_selftest.c new file +11
-0+#include <rte_memory.h> +#include <rte_memzone.h> +#include <rte_launch.h> +#include <rte_eal.h> +#include <rte_per_lcore.h> +#include <rte_lcore.h> +#include <rte_debug.h> +#include <rte_ethdev.h> +#include <rte_cycles.h> +#include <rte_eventdev.h> +#include <rte_pause.h>
Minor nit: I did a test build, looks like rte_ethdev.h, rte_per_lcore.h, and rte_pause.h are not required, but rte_mbuf.h and rte_mempool.h are.
drivers/event/dlb/dlb_selftest.c line 1521
Quoted 4 lines
drivers/event/dlb/dlb_selftest.c new file +4
-0+int +test_dlb_eventdev (void ) +{ +const char *dlb_eventdev_name ="event_dlb" ;
"event_dlb" -> "dlb_event"
This self-test also needs to be added to app/test/test_eventdev.c.
Thanks,
Gage
Re: [dpdk-dev] [PATCH 14/27] event/dlb: add PMD self-tests
Quoted 32 lines
-----Original Message-----
From: Eads, Gage <[email protected]>
Sent: Friday, July 10, 2020 3:43 PM
To: McDaniel, Timothy <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; Van Haaren, Harry <[email protected]>
Subject: RE: [PATCH 14/27] event/dlb: add PMD self-testsHi Tim,
drivers/event/dlb/dlb_selftest.c line 22
Quoted 11 lines
drivers/event/dlb/dlb_selftest.c new file +11
-0+#include <rte_memory.h> +#include <rte_memzone.h> +#include <rte_launch.h> +#include <rte_eal.h> +#include <rte_per_lcore.h> +#include <rte_lcore.h> +#include <rte_debug.h> +#include <rte_ethdev.h> +#include <rte_cycles.h> +#include <rte_eventdev.h> +#include <rte_pause.h> Minor nit: I did a test build, looks like
rte_ethdev.h,rte_per_lcore.h, andrte_pause.hare not required, butrte_mbuf.handrte_mempool.hare.drivers/event/dlb/dlb_selftest.c line 1521
Quoted 4 lines
drivers/event/dlb/dlb_selftest.c new file +4
-0+int +test_dlb_eventdev (void ) +{ +const char *dlb_eventdev_name ="event_dlb" ;"
event_dlb" -> "dlb_event"
Quoted 4 lines
This self-test also needs to be added to
app/test/test_eventdev.c.Thanks,
Gage
Hi Gage,
I have implemented all of the suggested changes. They will be available in patchset 3.
Thanks,
Tim
PATCH 15/27 event/dlb: add probe
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/Makefile | 1 + drivers/event/dlb/dlb.c | 517 +++++++++++++++++++++++++++++++++++++++++ drivers/event/dlb/meson.build | 3 +- 3 files changed, 520 insertions(+), 1 deletion(-) create mode 100644 drivers/event/dlb/dlb.c
drivers/event/dlb/Makefile +1 -0
@@ -20,6 +20,7 @@ LIBABIVER := 1 EXPORT_MAP := rte_pmd_dlb_event_version.map # library source files +SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += dlb.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += dlb_selftest.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += dlb_xstats.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_DLB_EVENTDEV) += rte_pmd_dlb.c
drivers/event/dlb/dlb.c new file +517 -0
517 lines not on this page · raw message
drivers/event/dlb/meson.build +2 -1
@@ -1,7 +1,8 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019-2020 Intel Corporation -sources = files('dlb_selftest.c', +sources = files('dlb.c', + 'dlb_selftest.c', 'dlb_xstats.c', 'rte_pmd_dlb.c', 'dlb_iface.c',
Signature · 1 line
1.7.10
Re: [dpdk-dev] [PATCH 15/27] event/dlb: add probe
Hi Tim,
<snip>
Quoted 9 lines
drivers/event/dlb/dlb.c new file +8
-0+/* declared extern in header, for access from other .c files */ int +eventdev_dlb_log_level; + +RTE_INIT (evdev_dlb_init_log) +{ + eventdev_dlb_log_level =rte_log_register ("pmd.event.dlb" ); +if (eventdev_dlb_log_level >=0 ) +rte_log_set_level (eventdev_dlb_log_level,RTE_LOG_NOTICE );}
There's a new macro RTE_LOG_REGISTER() that's designed to replace this boilerplate code. Examples here: http://mails.dpdk.org/archives/dev/2020-July/172273.html
Thanks,
Gage
PATCH 16/27 event/dlb: add infos_get and configure
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb.c | 401 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 401 insertions(+)
drivers/event/dlb/dlb.c +401 -0
@@ -218,6 +218,395 @@ int dlb_string_to_int(int *result, const char *str) DLB_MAX_NUM_DIR_CREDITS);return -EINVAL ; } +return 0 ; +} + +/* VDEV-only notes: +* This function first unmaps all memory mappings and closes the +* domain's file descriptor, which causes the driver to reset the +* scheduling domain. Once that completes (when close() returns), we +* can safely free the dynamically allocated memory used by the +* scheduling domain. +* +* PF-only notes: +* We will maintain a use count and use that to determine when +* a reset is required. In PF mode, we never mmap, or munmap +* device memory, and we own the entire physical PCI device. +*/ + +static void +dlb_hw_reset_sched_domain (const struct rte_eventdev *dev,bool reconfig) +{ +struct dlb_eventdev *dlb =dlb_pmd_priv (dev); +enum dlb_configuration_state config_state; +int i, j; + +/* Close and reset the domain */ +dlb_iface_domain_close (dlb); + +/* Free all dynamically allocated port memory */ +for (i =0 ; i < dlb->num_ports; i++) +dlb_free_qe_mem (&dlb->ev_ports[i].qm_port); + +/* If reconfiguring, mark the device's queues and ports as "previously +* configured." If the user doesn't reconfigure them, the PMD will +* reapply their previous configuration when the device is started. +*/ + config_state = (reconfig) ?DLB_PREV_CONFIGURED :DLB_NOT_CONFIGURED ; + +for (i =0 ; i < dlb->num_ports; i++) { + dlb->ev_ports[i].qm_port.config_state = config_state; +/* Reset setup_done so ports can be reconfigured */ + dlb->ev_ports[i].setup_done = false; +for (j =0 ; j <DLB_MAX_NUM_QIDS_PER_LDB_CQ ; j++) + dlb->ev_ports[i].link[j].mapped = false; + } + +for (i =0 ; i < dlb->num_queues; i++) + dlb->ev_queues[i].qm_queue.config_state = config_state; + +for (i =0 ; i <DLB_MAX_NUM_QUEUES ; i++) + dlb->ev_queues[i].setup_done = false; + + dlb->num_ports =0 ; + dlb->num_ldb_ports =0 ; + dlb->num_dir_ports =0 ; + dlb->num_queues =0 ; + dlb->num_ldb_queues =0 ; + dlb->num_dir_queues =0 ; + dlb->configured = false; +} + +static int +dlb_ldb_credit_pool_create (struct dlb_hw_dev *handle) +{ +struct dlb_create_ldb_pool_args cfg; +struct dlb_cmd_response response; +int ret; + +if (handle ==NULL ) +return -EINVAL ; + +if (!handle->cfg.resources.num_ldb_credits) { + handle->cfg.ldb_credit_pool_id =0 ; + handle->cfg.num_ldb_credits =0 ; +return 0 ; + } + + cfg.response = (uintptr_t )&response; + cfg.num_ldb_credits = handle->cfg.resources.num_ldb_credits; + + ret =dlb_iface_ldb_credit_pool_create (handle, + &cfg); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: ldb_credit_pool_create ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); + } + + handle->cfg.ldb_credit_pool_id = response.id; + handle->cfg.num_ldb_credits = cfg.num_ldb_credits; + +return ret; +} + +static int +dlb_dir_credit_pool_create (struct dlb_hw_dev *handle) +{ +struct dlb_create_dir_pool_args cfg; +struct dlb_cmd_response response; +int ret; + +if (handle ==NULL ) +return -EINVAL ; + +if (!handle->cfg.resources.num_dir_credits) { + handle->cfg.dir_credit_pool_id =0 ; + handle->cfg.num_dir_credits =0 ; +return 0 ; + } + + cfg.response = (uintptr_t )&response; + cfg.num_dir_credits = handle->cfg.resources.num_dir_credits; + + ret =dlb_iface_dir_credit_pool_create (handle, + &cfg); +if (ret <0 ) +DLB_LOG_ERR ("dlb: dir_credit_pool_create ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); + + handle->cfg.dir_credit_pool_id = response.id; + handle->cfg.num_dir_credits = cfg.num_dir_credits; + +return ret; +} + +static int +dlb_hw_create_sched_domain (struct dlb_hw_dev *handle, +struct dlb_eventdev *dlb, +const struct dlb_hw_rsrcs *resources_asked) +{ +int ret =0 ; +struct dlb_create_sched_domain_args *config_params; +struct dlb_cmd_response response; + +if (resources_asked ==NULL ) { +DLB_LOG_ERR ("dlb: dlb_create NULL parameter\n" ); + ret =EINVAL ; + goto error_exit; + } + +/* Map generic qm resources to dlb resources */ + config_params = &handle->cfg.resources; + + config_params->response = (uintptr_t )&response; + +/* DIR ports and queues */ + + config_params->num_dir_ports = + resources_asked->num_dir_ports; + + config_params->num_dir_credits = + resources_asked->num_dir_credits; + +/* LDB ports and queues */ + + config_params->num_ldb_queues = + resources_asked->num_ldb_queues; + + config_params->num_ldb_ports = + resources_asked->num_ldb_ports; + + config_params->num_ldb_credits = + resources_asked->num_ldb_credits; + + config_params->num_atomic_inflights = + dlb->num_atm_inflights_per_queue * + config_params->num_ldb_queues; + + config_params->num_hist_list_entries = config_params->num_ldb_ports * +DLB_NUM_HIST_LIST_ENTRIES_PER_LDB_PORT ; + +/* dlb limited to 1 credit pool per queue type */ + config_params->num_ldb_credit_pools =1 ; + config_params->num_dir_credit_pools =1 ; + +DLB_LOG_DBG ("sched domain create - ldb_qs=%d, ldb_ports=%d, dir_ports=%d, atomic_inflights=%d, hist_list_entries=%d, ldb_credits=%d, dir_credits=%d, ldb_cred_pools=%d, dir-credit_pools=%d\n" , + config_params->num_ldb_queues, + config_params->num_ldb_ports, + config_params->num_dir_ports, + config_params->num_atomic_inflights, + config_params->num_hist_list_entries, + config_params->num_ldb_credits, + config_params->num_dir_credits, + config_params->num_ldb_credit_pools, + config_params->num_dir_credit_pools); + +/* Configure the QM */ + + ret =dlb_iface_sched_domain_create (handle, config_params); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: domain create failed, device_id = %d, (driver ret = %d, extra status: %s)\n" , + handle->device_id, + ret, + dlb_error_strings[response.status]); + goto error_exit; + } + + handle->domain_id = response.id; + handle->domain_id_valid =1 ; + + config_params->response =0 ; + + ret =dlb_ldb_credit_pool_create (handle); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: create ldb credit pool failed\n" ); + goto error_exit2; + } + + ret =dlb_dir_credit_pool_create (handle); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: create dir credit pool failed\n" ); + goto error_exit2; + } + + handle->cfg.configured = true; + +return 0 ; + +error_exit2: +dlb_iface_domain_close (dlb); + +error_exit: + +return ret; +} + +/* End HW specific */ +static void +dlb_eventdev_info_get (struct rte_eventdev *dev, +struct rte_event_dev_info *dev_info) +{ +struct dlb_eventdev *dlb =dlb_pmd_priv (dev); +int ret; + + ret =dlb_hw_query_resources (dlb); +if (ret) { +const struct rte_eventdev_data *data = dev->data; + +DLB_LOG_ERR ("get resources err=%d, devid=%d\n" , + ret, data->dev_id); +/* fn is void, so fall through and return values set up in +* probe +*/ + } + +/* Add num resources currently owned by this domain. +* These would become available if the scheduling domain were reset due +* to the application recalling eventdev_configure to *reconfigure* the +* domain. +*/ + evdev_dlb_default_info.max_event_ports += dlb->num_ldb_ports; + evdev_dlb_default_info.max_event_queues += dlb->num_ldb_queues; + evdev_dlb_default_info.max_num_events += dlb->num_ldb_credits; + + +/* In DLB A-stepping hardware, applications are limited to 128 +* configured ports (load-balanced or directed). The reported number of +* available ports must reflect this. +*/ +if (dlb->revision <DLB_REV_B0 ) { +int used_ports; + + used_ports =DLB_MAX_NUM_LDB_PORTS +DLB_MAX_NUM_DIR_PORTS - + dlb->hw_rsrc_query_results.num_ldb_ports - + dlb->hw_rsrc_query_results.num_dir_ports; + + evdev_dlb_default_info.max_event_ports = +RTE_MIN (evdev_dlb_default_info.max_event_ports, +128 - used_ports); + } + + evdev_dlb_default_info.max_event_queues = +RTE_MIN (evdev_dlb_default_info.max_event_queues, +RTE_EVENT_MAX_QUEUES_PER_DEV ); + + evdev_dlb_default_info.max_num_events = +RTE_MIN (evdev_dlb_default_info.max_num_events, + dlb->max_num_events_override); + + *dev_info = evdev_dlb_default_info; +} + +/* Note: 1 QM instance per QM device, QM instance/device == event device */ +static int +dlb_eventdev_configure (const struct rte_eventdev *dev) +{ +struct dlb_eventdev *dlb =dlb_pmd_priv (dev); +struct dlb_hw_dev *handle = &dlb->qm_instance; +struct dlb_hw_rsrcs *rsrcs = &handle->info.hw_rsrc_max; +const struct rte_eventdev_data *data = dev->data; +const struct rte_event_dev_config *config = &data->dev_conf; +int ret; + +/* If this eventdev is already configured, we must release the current +* scheduling domain before attempting to configure a new one. +*/ +if (dlb->configured) { +dlb_hw_reset_sched_domain (dev, true); + + ret =dlb_hw_query_resources (dlb); +if (ret) { +DLB_LOG_ERR ("get resources err=%d, devid=%d\n" , + ret, data->dev_id); +return ret; + } + } + +if (config->nb_event_queues > rsrcs->num_queues) { +DLB_LOG_ERR ("nb_event_queues parameter (%d) exceeds the QM device's capabilities (%d).\n" , + config->nb_event_queues, + rsrcs->num_queues); +return -EINVAL ; + } +if (config->nb_event_ports > (rsrcs->num_ldb_ports + + rsrcs->num_dir_ports)) { +DLB_LOG_ERR ("nb_event_ports parameter (%d) exceeds the QM device's capabilities (%d).\n" , + config->nb_event_ports, + (rsrcs->num_ldb_ports + rsrcs->num_dir_ports)); +return -EINVAL ; + } +if (config->nb_events_limit > rsrcs->nb_events_limit) { +DLB_LOG_ERR ("nb_events_limit parameter (%d) exceeds the QM device's capabilities (%d).\n" , + config->nb_events_limit, + rsrcs->nb_events_limit); +return -EINVAL ; + } + +if (config->event_dev_cfg &RTE_EVENT_DEV_CFG_PER_DEQUEUE_TIMEOUT ) { + dlb->global_dequeue_wait = false; + }else { +uint32_t timeout32; + + dlb->global_dequeue_wait = true; + + timeout32 = config->dequeue_timeout_ns; + +/* PF PMD does not support interrupts, and +* UMONITOR wait is temporarily disabled. +*/ + + dlb->global_dequeue_wait_ticks = + timeout32 * (rte_get_timer_hz () /1E9 ); + } + +/* Does this platform support umonitor/umwait? */ +if (rte_cpu_get_flag_enabled (RTE_CPUFLAG_UMWAIT )) { +if (RTE_LIBRTE_PMD_DLB_UMWAIT_CTL_STATE !=0 && +RTE_LIBRTE_PMD_DLB_UMWAIT_CTL_STATE !=1 ) { +DLB_LOG_ERR ("invalid value (%d) for RTE_LIBRTE_PMD_DLB_UMWAIT_CTL_STATE must be 0 or 1.\n" , +RTE_LIBRTE_PMD_DLB_UMWAIT_CTL_STATE ); +return -EINVAL ; + } + dlb->umwait_allowed = true; + } + + rsrcs->num_dir_ports = config->nb_single_link_event_port_queues; + rsrcs->num_ldb_ports = config->nb_event_ports - rsrcs->num_dir_ports; +/* 1 dir queue per dir port */ + rsrcs->num_ldb_queues = config->nb_event_queues - rsrcs->num_dir_ports; + +/* Scale down nb_events_limit by 4 for directed credits, since there +* are 4x as many load-balanced credits. +*/ + rsrcs->num_ldb_credits =0 ; + rsrcs->num_dir_credits =0 ; + +if (rsrcs->num_ldb_queues) + rsrcs->num_ldb_credits = config->nb_events_limit; +if (rsrcs->num_dir_ports) + rsrcs->num_dir_credits = config->nb_events_limit /4 ; +if (dlb->num_dir_credits_override !=-1 ) + rsrcs->num_dir_credits = dlb->num_dir_credits_override; + +if (dlb_hw_create_sched_domain (handle, dlb, rsrcs) <0 ) { +DLB_LOG_ERR ("dlb_hw_create_sched_domain failed\n" ); +return -ENODEV ; + } + + dlb->new_event_limit = config->nb_events_limit; +rte_atomic32_set (&dlb->inflights,0 ); + +/* Save number of ports/queues for this event dev */ + dlb->num_ports = config->nb_event_ports; + dlb->num_queues = config->nb_event_queues; + dlb->num_dir_ports = rsrcs->num_dir_ports; + dlb->num_ldb_ports = dlb->num_ports - dlb->num_dir_ports; + dlb->num_ldb_queues = dlb->num_queues - dlb->num_dir_ports; + dlb->num_dir_queues = dlb->num_dir_ports; + dlb->num_ldb_credits = rsrcs->num_ldb_credits; + dlb->num_dir_credits = rsrcs->num_dir_credits; + + dlb->configured = true;return 0 ; }
@@ -292,6 +681,18 @@ int dlb_string_to_int(int *result, const char *str)return 0 ; } +void +dlb_entry_points_init (struct rte_eventdev *dev) +{ +static struct rte_eventdev_ops dlb_eventdev_entry_ops = { + .dev_infos_get = dlb_eventdev_info_get, + .dev_configure = dlb_eventdev_configure, + }; + +/* Expose PMD's eventdev interface */ + dev->dev_ops = &dlb_eventdev_entry_ops; + +}static void dlb_qm_mmio_fn_init (void )
Signature · 1 line
1.7.10
PATCH 17/27 event/dlb: add queue_def_conf and port_def_conf
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+)
drivers/event/dlb/dlb.c +31 -0
@@ -611,6 +611,35 @@ int dlb_string_to_int(int *result, const char *str)return 0 ; } +static void +dlb_eventdev_port_default_conf_get (struct rte_eventdev *dev, +uint8_t port_id, +struct rte_event_port_conf *port_conf) +{ +RTE_SET_USED (port_id); +struct dlb_eventdev *dlb; + + dlb =dlb_pmd_priv (dev); + + port_conf->new_event_threshold = dlb->new_event_limit; + port_conf->dequeue_depth =32 ; + port_conf->enqueue_depth =DLB_MAX_ENQUEUE_DEPTH ; + port_conf->event_port_cfg =0 ; +} + +static void +dlb_eventdev_queue_default_conf_get (struct rte_eventdev *dev, +uint8_t queue_id, +struct rte_event_queue_conf *queue_conf) +{ +RTE_SET_USED (dev); +RTE_SET_USED (queue_id); + queue_conf->nb_atomic_flows =1024 ; + queue_conf->nb_atomic_order_sequences =32 ; + queue_conf->event_queue_cfg =0 ; + queue_conf->priority =0 ; +} + static int set_dev_id(const char *key __rte_unused, const char *value,
@@ -687,6 +716,8 @@ int dlb_string_to_int(int *result, const char *str)static struct rte_eventdev_ops dlb_eventdev_entry_ops = { .dev_infos_get = dlb_eventdev_info_get, .dev_configure = dlb_eventdev_configure, + .queue_def_conf = dlb_eventdev_queue_default_conf_get, + .port_def_conf = dlb_eventdev_port_default_conf_get, };/* Expose PMD's eventdev interface */
Signature · 1 line
1.7.10
PATCH 18/27 event/dlb: add queue setup
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb.c | 295 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 295 insertions(+)
drivers/event/dlb/dlb.c +295 -0
@@ -221,6 +221,65 @@ int dlb_string_to_int(int *result, const char *str)return 0 ; } +static int32_t +dlb_hw_create_ldb_queue (struct dlb_eventdev *dlb, +struct dlb_queue *queue, +const struct rte_event_queue_conf *evq_conf) +{ +struct dlb_hw_dev *handle = &dlb->qm_instance; +struct dlb_create_ldb_queue_args cfg; +struct dlb_cmd_response response; +int32_t ret; +uint32_t qm_qid; +int sched_type =-1 ; + +if (evq_conf ==NULL ) +return -EINVAL ; + +if (evq_conf->event_queue_cfg &RTE_EVENT_QUEUE_CFG_ALL_TYPES ) { +if (evq_conf->nb_atomic_order_sequences !=0 ) + sched_type =RTE_SCHED_TYPE_ORDERED ; +else + sched_type =RTE_SCHED_TYPE_PARALLEL ; + }else { + sched_type = evq_conf->schedule_type; + } + + cfg.response = (uintptr_t )&response; + cfg.num_atomic_inflights = dlb->num_atm_inflights_per_queue; + cfg.num_sequence_numbers = evq_conf->nb_atomic_order_sequences; + cfg.num_qid_inflights = evq_conf->nb_atomic_order_sequences; + +if (sched_type !=RTE_SCHED_TYPE_ORDERED ) { + cfg.num_sequence_numbers =0 ; + cfg.num_qid_inflights =DLB_DEF_UNORDERED_QID_INFLIGHTS ; + } + + ret =dlb_iface_ldb_queue_create (handle, &cfg); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: create LB event queue error, ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); +return -EINVAL ; + } + + qm_qid = response.id; + +/* Save off queue config for debug, resource lookups, and reconfig */ + queue->num_qid_inflights = cfg.num_qid_inflights; + queue->num_atm_inflights = cfg.num_atomic_inflights; + + queue->sched_type = sched_type; + queue->config_state =DLB_CONFIGURED ; + +DLB_LOG_DBG ("Created LB event queue %d, nb_inflights=%d, nb_seq=%d, qid inflights=%d\n" , + qm_qid, + cfg.num_atomic_inflights, + cfg.num_sequence_numbers, + cfg.num_qid_inflights); + +return qm_qid; +} +/* VDEV-only notes: * This function first unmaps all memory mappings and closes the * domain's file descriptor, which causes the driver to reset the
@@ -442,6 +501,7 @@ int dlb_string_to_int(int *result, const char *str) }/* End HW specific */ +static void dlb_eventdev_info_get (struct rte_eventdev *dev,struct rte_event_dev_info *dev_info)
@@ -640,6 +700,240 @@ int dlb_string_to_int(int *result, const char *str) queue_conf->priority =0 ; } +static int32_t +dlb_get_sn_allocation (struct dlb_eventdev *dlb,int group) +{ +struct dlb_hw_dev *handle = &dlb->qm_instance; +struct dlb_get_sn_allocation_args cfg; +struct dlb_cmd_response response; +int ret; + + cfg.group = group; + cfg.response = (uintptr_t )&response; + + ret =dlb_iface_get_sn_allocation (handle, &cfg); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: get_sn_allocation ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); +return ret; + } + +return response.id; +} + +static int +dlb_set_sn_allocation (struct dlb_eventdev *dlb,int group,int num) +{ +struct dlb_hw_dev *handle = &dlb->qm_instance; +struct dlb_set_sn_allocation_args cfg; +struct dlb_cmd_response response; +int ret; + + cfg.num = num; + cfg.group = group; + cfg.response = (uintptr_t )&response; + + ret =dlb_iface_set_sn_allocation (handle, &cfg); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: set_sn_allocation ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); +return ret; + } + +return ret; +} + +static int32_t +dlb_get_sn_occupancy (struct dlb_eventdev *dlb,int group) +{ +struct dlb_hw_dev *handle = &dlb->qm_instance; +struct dlb_get_sn_occupancy_args cfg; +struct dlb_cmd_response response; +int ret; + + cfg.group = group; + cfg.response = (uintptr_t )&response; + + ret =dlb_iface_get_sn_occupancy (handle, &cfg); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: get_sn_occupancy ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); +return ret; + } + +return response.id; +} + +/* Query the current sequence number allocations and, if they conflict with the +* requested LDB queue configuration, attempt to re-allocate sequence numbers. +* This is best-effort; if it fails, the PMD will attempt to configure the +* load-balanced queue and return an error. +*/ +static void +dlb_program_sn_allocation (struct dlb_eventdev *dlb, +const struct rte_event_queue_conf *queue_conf) +{ +int grp_occupancy[DLB_NUM_SN_GROUPS ]; +int grp_alloc[DLB_NUM_SN_GROUPS ]; +int i, sequence_numbers; + + sequence_numbers = (int )queue_conf->nb_atomic_order_sequences; + +for (i =0 ; i <DLB_NUM_SN_GROUPS ; i++) { +int total_slots; + + grp_alloc[i] =dlb_get_sn_allocation (dlb, i); +if (grp_alloc[i] <0 ) +return ; + + total_slots =DLB_MAX_LDB_SN_ALLOC / grp_alloc[i]; + + grp_occupancy[i] =dlb_get_sn_occupancy (dlb, i); +if (grp_occupancy[i] <0 ) +return ; + +/* DLB has at least one available slot for the requested +* sequence numbers, so no further configuration required. +*/ +if (grp_alloc[i] == sequence_numbers && + grp_occupancy[i] < total_slots) +return ; + } + +/* None of the sequence number groups are configured for the requested +* sequence numbers, so we have to reconfigure one of them. This is +* only possible if a group is not in use. +*/ +for (i =0 ; i <DLB_NUM_SN_GROUPS ; i++) { +if (grp_occupancy[i] ==0 ) +break ; + } + +if (i ==DLB_NUM_SN_GROUPS ) { +printf ("[%s()] No groups with %d sequence_numbers are available or have free slots\n" , + __func__, sequence_numbers); +return ; + } + +/* Attempt to configure slot i with the requested number of sequence +* numbers. Ignore the return value -- if this fails, the error will be +* caught during subsequent queue configuration. +*/ +dlb_set_sn_allocation (dlb, i, sequence_numbers); +} + +static int +dlb_eventdev_ldb_queue_setup (struct rte_eventdev *dev, +struct dlb_eventdev_queue *ev_queue, +const struct rte_event_queue_conf *queue_conf) +{ +struct dlb_eventdev *dlb =dlb_pmd_priv (dev); +int32_t qm_qid; + +if (queue_conf->nb_atomic_order_sequences) +dlb_program_sn_allocation (dlb, queue_conf); + + qm_qid =dlb_hw_create_ldb_queue (dlb, + &ev_queue->qm_queue, + queue_conf); +if (qm_qid <0 ) { +DLB_LOG_ERR ("Failed to create the load-balanced queue\n" ); + +return qm_qid; + } + + dlb->qm_ldb_to_ev_queue_id[qm_qid] = ev_queue->id; + + ev_queue->qm_queue.id = qm_qid; + +return 0 ; +} + +static int dlb_num_dir_queues_setup (struct dlb_eventdev *dlb) +{ +int i, num =0 ; + +for (i =0 ; i < dlb->num_queues; i++) { +if (dlb->ev_queues[i].setup_done && + dlb->ev_queues[i].qm_queue.is_directed) + num++; + } + +return num; +} + +static void +dlb_queue_link_teardown (struct dlb_eventdev *dlb, +struct dlb_eventdev_queue *ev_queue) +{ +struct dlb_eventdev_port *ev_port; +int i, j; + +for (i =0 ; i < dlb->num_ports; i++) { + ev_port = &dlb->ev_ports[i]; + +for (j =0 ; j <DLB_MAX_NUM_QIDS_PER_LDB_CQ ; j++) { +if (!ev_port->link[j].valid || + ev_port->link[j].queue_id != ev_queue->id) +continue ; + + ev_port->link[j].valid = false; + ev_port->num_links--; + } + } + + ev_queue->num_links =0 ; +} + +static int +dlb_eventdev_queue_setup (struct rte_eventdev *dev, +uint8_t ev_qid, +const struct rte_event_queue_conf *queue_conf) +{ +struct dlb_eventdev *dlb =dlb_pmd_priv (dev); +struct dlb_eventdev_queue *ev_queue; +int ret; + +if (!queue_conf) +return -EINVAL ; + +if (ev_qid >= dlb->num_queues) +return -EINVAL ; + + ev_queue = &dlb->ev_queues[ev_qid]; + + ev_queue->qm_queue.is_directed = queue_conf->event_queue_cfg & +RTE_EVENT_QUEUE_CFG_SINGLE_LINK ; + ev_queue->id = ev_qid; + ev_queue->conf = *queue_conf; + +if (!ev_queue->qm_queue.is_directed) { + ret =dlb_eventdev_ldb_queue_setup (dev, ev_queue, queue_conf); + }else { +/* The directed queue isn't setup until link time, at which +* point we know its directed port ID. Directed queue setup +* will only fail if this queue is already setup or there are +* no directed queues left to configure. +*/ + ret =0 ; + + ev_queue->qm_queue.config_state =DLB_NOT_CONFIGURED ; + +if (ev_queue->setup_done || +dlb_num_dir_queues_setup (dlb) == dlb->num_dir_queues) + ret = -EINVAL ; + } + +/* Tear down pre-existing port->queue links */ +if (!ret && dlb->run_state ==DLB_RUN_STATE_STOPPED ) +dlb_queue_link_teardown (dlb, ev_queue); + +if (!ret) + ev_queue->setup_done = true; + +return ret; +} + static int set_dev_id(const char *key __rte_unused, const char *value,
@@ -717,6 +1011,7 @@ int dlb_string_to_int(int *result, const char *str) .dev_infos_get = dlb_eventdev_info_get, .dev_configure = dlb_eventdev_configure, .queue_def_conf = dlb_eventdev_queue_default_conf_get, + .queue_setup = dlb_eventdev_queue_setup, .port_def_conf = dlb_eventdev_port_default_conf_get, };
Signature · 1 line
1.7.10
PATCH 19/27 event/dlb: add port_setup
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb.c | 1235 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 1235 insertions(+)
drivers/event/dlb/dlb.c +1235 -0
@@ -71,6 +71,29 @@/* These functions will vary based on processor capabilities */ static struct dlb_port_low_level_io_functions qm_mmio_fns; +struct process_local_port_data +dlb_port[DLB_MAX_NUM_PORTS ][NUM_DLB_PORT_TYPES ]; + +static __rte_always_inline uint16_t +dlb_read_pc (struct process_local_port_data *port_data,bool ldb) +{ +volatile uint16_t *popcount; + +if (ldb) + popcount = port_data->ldb_popcount; +else + popcount = port_data->dir_popcount; + +return *popcount; +} + +static __rte_always_inline void +dlb_pp_write (struct dlb_enqueue_qe *qe4, +struct process_local_port_data *port_data) +{ + qm_mmio_fns.pp_enqueue_four (qe4, port_data->pp_addr); +} +static int dlb_hw_query_resources (struct dlb_eventdev *dlb) {
@@ -169,6 +192,46 @@ int dlb_string_to_int(int *result, const char *str)return 0 ; } +static inline int +dlb_consume_qe_immediate (struct dlb_port *qm_port,int num) +{ +struct process_local_port_data *port_data; +struct dlb_cq_pop_qe *qe; + +RTE_ASSERT (qm_port->config_state ==DLB_CONFIGURED ); + +if (qm_port->use_rsvd_token_scheme) { +/* Check if there's a deficit of reserved tokens, and return +* early if there are no (unreserved) tokens to consume. +*/ +if (num <= qm_port->cq_rsvd_token_deficit) { + qm_port->cq_rsvd_token_deficit -= num; + qm_port->owed_tokens =0 ; +return 0 ; + } + num -= qm_port->cq_rsvd_token_deficit; + qm_port->cq_rsvd_token_deficit =0 ; + } + + qe = qm_port->consume_qe; + + qe->tokens = num -1 ; + qe->int_arm =0 ; + +/* No store fence needed since no pointer is being sent, and CQ token +* pops can be safely reordered with other HCWs. +*/ + port_data = &dlb_port[qm_port->id][PORT_TYPE (qm_port)]; + +dlb_movntdq_single (qe, port_data->pp_addr); + +DLB_LOG_DBG ("dlb: consume immediate - %d QEs\n" , num); + + qm_port->owed_tokens =0 ; + +return 0 ; +} + int set_max_num_events(const char *key __rte_unused, const char *value,
@@ -221,6 +284,449 @@ int dlb_string_to_int(int *result, const char *str)return 0 ; } +static inline uint16_t +dlb_event_enqueue_delayed (void *event_port, +const struct rte_event events[]); + +static inline uint16_t +dlb_event_enqueue_burst_delayed (void *event_port, +const struct rte_event events[], +uint16_t num); + +static inline uint16_t +dlb_event_enqueue_new_burst_delayed (void *event_port, +const struct rte_event events[], +uint16_t num); + +static inline uint16_t +dlb_event_enqueue_forward_burst_delayed (void *event_port, +const struct rte_event events[], +uint16_t num); + +int +dlb_init_qe_mem (struct dlb_port *qm_port,char *mz_name) +{ +int ret, sz; + + sz =DLB_NUM_QES_PER_CACHE_LINE *sizeof (struct dlb_enqueue_qe ); + + qm_port->qe4 =rte_malloc (mz_name, sz,RTE_CACHE_LINE_SIZE ); + +if (qm_port->qe4 ==NULL ) { +DLB_LOG_ERR ("dlb: no qe4 memory\n" ); + ret = -ENOMEM ; + goto error_exit; + } + +memset (qm_port->qe4,0 , sz); + + ret =dlb_init_consume_qe (qm_port, mz_name); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: dlb_init_consume_qe ret=%d\n" , + ret); + goto error_exit; + } + +return 0 ; + +error_exit: + +dlb_free_qe_mem (qm_port); + +return ret; +} + +int +dlb_init_consume_qe (struct dlb_port *qm_port,char *mz_name) +{ +struct dlb_cq_pop_qe *qe; + + qe =rte_malloc (mz_name, +DLB_NUM_QES_PER_CACHE_LINE * +sizeof (struct dlb_cq_pop_qe ), +RTE_CACHE_LINE_SIZE ); + +if (qe ==NULL ) { +DLB_LOG_ERR ("dlb: no memory for consume_qe\n" ); +return -ENOMEM ; + } + + qm_port->consume_qe = qe; + +memset (qe,0 ,DLB_NUM_QES_PER_CACHE_LINE * +sizeof (struct dlb_cq_pop_qe )); + + qe->qe_valid =0 ; + qe->qe_frag =0 ; + qe->qe_comp =0 ; + qe->cq_token =1 ; +/* Tokens value is 0-based; i.e. '0' returns 1 token, '1' returns 2, +* and so on. +*/ + qe->tokens =0 ;/* set at run time */ + qe->meas_lat =0 ; + qe->no_dec =0 ; +/* Completion IDs are disabled */ + qe->cmp_id =0 ; + +return 0 ; +} + +static int +dlb_hw_create_ldb_port (struct dlb_eventdev *dlb, +struct dlb_eventdev_port *ev_port, +uint32_t dequeue_depth, +uint32_t cq_depth, +uint32_t enqueue_depth, +uint16_t rsvd_tokens, +bool use_rsvd_token_scheme) +{ +struct dlb_hw_dev *handle = &dlb->qm_instance; +struct dlb_create_ldb_port_args cfg = {0 }; +struct dlb_cmd_response response = {0 }; +int ret; +struct dlb_port *qm_port =NULL ; +char mz_name[RTE_MEMZONE_NAMESIZE ]; +uint32_t qm_port_id; + +if (handle ==NULL ) +return -EINVAL ; + +if (cq_depth <DLB_MIN_LDB_CQ_DEPTH || + cq_depth >DLB_MAX_INPUT_QUEUE_DEPTH ) { +DLB_LOG_ERR ("dlb: invalid cq_depth, must be %d-%d\n" , +DLB_MIN_LDB_CQ_DEPTH ,DLB_MAX_INPUT_QUEUE_DEPTH ); +return -EINVAL ; + } + +if (enqueue_depth <DLB_MIN_ENQUEUE_DEPTH ) { +DLB_LOG_ERR ("dlb: invalid enqueue_depth, must be at least %d\n" , +DLB_MIN_ENQUEUE_DEPTH ); +return -EINVAL ; + } + +rte_spinlock_lock (&handle->resource_lock); + + cfg.response = (uintptr_t )&response; + +/* We round up to the next power of 2 if necessary */ + cfg.cq_depth =rte_align32pow2 (cq_depth); + cfg.cq_depth_threshold = rsvd_tokens; + + cfg.cq_history_list_size =DLB_NUM_HIST_LIST_ENTRIES_PER_LDB_PORT ; + +/* User controls the LDB high watermark via enqueue depth. The DIR high +* watermark is equal, unless the directed credit pool is too small. +*/ + cfg.ldb_credit_high_watermark = enqueue_depth; + +/* If there are no directed ports, the kernel driver will ignore this +* port's directed credit settings. Don't use enqueue_depth if it would +* require more directed credits than are available. +*/ + cfg.dir_credit_high_watermark = +RTE_MIN (enqueue_depth, + handle->cfg.num_dir_credits / dlb->num_ports); + + cfg.ldb_credit_quantum = cfg.ldb_credit_high_watermark /2 ; + cfg.ldb_credit_low_watermark =RTE_MIN (16 , cfg.ldb_credit_quantum); + + cfg.dir_credit_quantum = cfg.dir_credit_high_watermark /2 ; + cfg.dir_credit_low_watermark =RTE_MIN (16 , cfg.dir_credit_quantum); + +/* Per QM values */ + + cfg.ldb_credit_pool_id = handle->cfg.ldb_credit_pool_id; + cfg.dir_credit_pool_id = handle->cfg.dir_credit_pool_id; + + ret =dlb_iface_ldb_port_create (handle, &cfg, dlb->poll_mode); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: dlb_ldb_port_create error, ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); + goto error_exit; + } + + qm_port_id = response.id; + +DLB_LOG_DBG ("dlb: ev_port %d uses qm LB port %d <<<<<\n" , + ev_port->id, qm_port_id); + + qm_port = &ev_port->qm_port; + qm_port->ev_port = ev_port;/* back ptr */ + qm_port->dlb = dlb;/* back ptr */ + +/* +* Allocate and init local qe struct(s). +* Note: MOVDIR64 requires the enqueue QE (qe4) to be aligned. +*/ + +snprintf (mz_name,sizeof (mz_name),"%s_ldb_port%d" , + handle->device_name, + ev_port->id); + + ret =dlb_init_qe_mem (qm_port, mz_name); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: init_qe_mem failed, ret=%d\n" , ret); + goto error_exit; + } + + qm_port->pp_mmio_base =DLB_LDB_PP_BASE +PAGE_SIZE * qm_port_id; + qm_port->id = qm_port_id; + +/* The credit window is one high water mark of QEs */ + qm_port->ldb_pushcount_at_credit_expiry =0 ; + qm_port->cached_ldb_credits = cfg.ldb_credit_high_watermark; +/* The credit window is one high water mark of QEs */ + qm_port->dir_pushcount_at_credit_expiry =0 ; + qm_port->cached_dir_credits = cfg.dir_credit_high_watermark; + qm_port->cq_depth = cfg.cq_depth; +/* CQs with depth < 8 use an 8-entry queue, but withhold credits so +* the effective depth is smaller. +*/ + qm_port->cq_depth = cfg.cq_depth <=8 ?8 : cfg.cq_depth; + qm_port->cq_idx =0 ; + qm_port->cq_idx_unmasked =0 ; +if (dlb->poll_mode ==DLB_CQ_POLL_MODE_SPARSE ) + qm_port->cq_depth_mask = (qm_port->cq_depth *4 ) -1 ; +else + qm_port->cq_depth_mask = qm_port->cq_depth -1 ; + + qm_port->gen_bit_shift =__builtin_popcount (qm_port->cq_depth_mask); +/* starting value of gen bit - it toggles at wrap time */ + qm_port->gen_bit =1 ; + + qm_port->use_rsvd_token_scheme = use_rsvd_token_scheme; + qm_port->cq_rsvd_token_deficit = rsvd_tokens; + qm_port->int_armed = false; + +/* Save off for later use in info and lookup APIs. */ + qm_port->qid_mappings = &dlb->qm_ldb_to_ev_queue_id[0 ]; + +/* When using the reserved token scheme, token_pop_thresh is +* initially 2 * dequeue_depth. Once the tokens are reserved, +* the enqueue code re-assigns it to dequeue_depth. +*/ + qm_port->dequeue_depth = dequeue_depth; + qm_port->token_pop_thresh = cq_depth; + +/* When the deferred scheduling vdev arg is selected, use deferred pop +* for all single-entry CQs. +*/ +if (cfg.cq_depth ==1 || (cfg.cq_depth ==2 && use_rsvd_token_scheme)) { +if (dlb->defer_sched) + qm_port->token_pop_mode =DEFERRED_POP ; + } + +/* The default enqueue functions do not include delayed-pop support for +* performance reasons. +*/ +if (qm_port->token_pop_mode ==DELAYED_POP ) { + dlb->event_dev->enqueue = dlb_event_enqueue_delayed; + dlb->event_dev->enqueue_burst = + dlb_event_enqueue_burst_delayed; + dlb->event_dev->enqueue_new_burst = + dlb_event_enqueue_new_burst_delayed; + dlb->event_dev->enqueue_forward_burst = + dlb_event_enqueue_forward_burst_delayed; + } + + qm_port->owed_tokens =0 ; + qm_port->issued_releases =0 ; + +/* Save config message too. */ +rte_memcpy (&qm_port->cfg.ldb, &cfg,sizeof (cfg)); + +/* update state */ + qm_port->state =PORT_STARTED ;/* enabled at create time */ + qm_port->config_state =DLB_CONFIGURED ; + + qm_port->dir_credits = cfg.dir_credit_high_watermark; + qm_port->ldb_credits = cfg.ldb_credit_high_watermark; + +DLB_LOG_DBG ("dlb: created ldb port %d, depth = %d, ldb credits=%d, dir credits=%d\n" , + qm_port_id, + cq_depth, + qm_port->ldb_credits, + qm_port->dir_credits); + +rte_spinlock_unlock (&handle->resource_lock); + +return 0 ; + +error_exit: +if (qm_port) { +dlb_free_qe_mem (qm_port); + qm_port->pp_mmio_base =0 ; + } + +rte_spinlock_unlock (&handle->resource_lock); + +DLB_LOG_ERR ("dlb: create ldb port failed!\n" ); + +return ret; +} + +static int +dlb_hw_create_dir_port (struct dlb_eventdev *dlb, +struct dlb_eventdev_port *ev_port, +uint32_t dequeue_depth, +uint32_t cq_depth, +uint32_t enqueue_depth, +uint16_t rsvd_tokens, +bool use_rsvd_token_scheme) +{ +struct dlb_hw_dev *handle = &dlb->qm_instance; +struct dlb_create_dir_port_args cfg = {0 }; +struct dlb_cmd_response response = {0 }; +int ret; +struct dlb_port *qm_port =NULL ; +char mz_name[RTE_MEMZONE_NAMESIZE ]; +uint32_t qm_port_id; + +if (dlb ==NULL || handle ==NULL ) +return -EINVAL ; + +if (cq_depth <DLB_MIN_DIR_CQ_DEPTH || + cq_depth >DLB_MAX_INPUT_QUEUE_DEPTH ) { +DLB_LOG_ERR ("dlb: invalid cq_depth, must be %d-%d\n" , +DLB_MIN_DIR_CQ_DEPTH ,DLB_MAX_INPUT_QUEUE_DEPTH ); +return -EINVAL ; + } + +rte_spinlock_lock (&handle->resource_lock); + +/* Directed queues are configured at link time. */ + cfg.queue_id =-1 ; + + cfg.response = (uintptr_t )&response; + +/* We round up to the next power of 2 if necessary */ + cfg.cq_depth =rte_align32pow2 (cq_depth); + cfg.cq_depth_threshold = rsvd_tokens; + +/* User controls the LDB high watermark via enqueue depth. The DIR high +* watermark is equal, unless the directed credit pool is too small. +*/ + cfg.ldb_credit_high_watermark = enqueue_depth; + +/* Don't use enqueue_depth if it would require more directed credits +* than are available. +*/ + cfg.dir_credit_high_watermark = +RTE_MIN (enqueue_depth, + handle->cfg.num_dir_credits / dlb->num_ports); + + cfg.ldb_credit_quantum = cfg.ldb_credit_high_watermark /2 ; + cfg.ldb_credit_low_watermark =RTE_MIN (16 , cfg.ldb_credit_quantum); + + cfg.dir_credit_quantum = cfg.dir_credit_high_watermark /2 ; + cfg.dir_credit_low_watermark =RTE_MIN (16 , cfg.dir_credit_quantum); + +/* Per QM values */ + + cfg.ldb_credit_pool_id = handle->cfg.ldb_credit_pool_id; + cfg.dir_credit_pool_id = handle->cfg.dir_credit_pool_id; + + ret =dlb_iface_dir_port_create (handle, &cfg, dlb->poll_mode); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: dlb_dir_port_create error, ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); + goto error_exit; + } + + qm_port_id = response.id; + +DLB_LOG_DBG ("dlb: ev_port %d uses qm DIR port %d <<<<<\n" , + ev_port->id, qm_port_id); + + qm_port = &ev_port->qm_port; + qm_port->ev_port = ev_port;/* back ptr */ + qm_port->dlb = dlb;/* back ptr */ + +/* +* Init local qe struct(s). +* Note: MOVDIR64 requires the enqueue QE to be aligned +*/ + +snprintf (mz_name,sizeof (mz_name),"%s_dir_port%d" , + handle->device_name, + ev_port->id); + + ret =dlb_init_qe_mem (qm_port, mz_name); + +if (ret <0 ) { +DLB_LOG_ERR ("dlb: init_qe_mem failed, ret=%d\n" , ret); + goto error_exit; + } + + qm_port->pp_mmio_base =DLB_DIR_PP_BASE +PAGE_SIZE * qm_port_id; + qm_port->id = qm_port_id; + +/* The credit window is one high water mark of QEs */ + qm_port->ldb_pushcount_at_credit_expiry =0 ; + qm_port->cached_ldb_credits = cfg.ldb_credit_high_watermark; +/* The credit window is one high water mark of QEs */ + qm_port->dir_pushcount_at_credit_expiry =0 ; + qm_port->cached_dir_credits = cfg.dir_credit_high_watermark; + qm_port->cq_depth = cfg.cq_depth; + qm_port->cq_idx =0 ; + qm_port->cq_idx_unmasked =0 ; +if (dlb->poll_mode ==DLB_CQ_POLL_MODE_SPARSE ) + qm_port->cq_depth_mask = (cfg.cq_depth *4 ) -1 ; +else + qm_port->cq_depth_mask = cfg.cq_depth -1 ; + + qm_port->gen_bit_shift =__builtin_popcount (qm_port->cq_depth_mask); +/* starting value of gen bit - it toggles at wrap time */ + qm_port->gen_bit =1 ; + + qm_port->use_rsvd_token_scheme = use_rsvd_token_scheme; + qm_port->cq_rsvd_token_deficit = rsvd_tokens; + qm_port->int_armed = false; + +/* Save off for later use in info and lookup APIs. */ + qm_port->qid_mappings = &dlb->qm_dir_to_ev_queue_id[0 ]; + + qm_port->dequeue_depth = dequeue_depth; + +/* Directed ports are auto-pop, by default. */ + qm_port->token_pop_mode =AUTO_POP ; + qm_port->owed_tokens =0 ; + qm_port->issued_releases =0 ; + +/* Save config message too. */ +rte_memcpy (&qm_port->cfg.dir, &cfg,sizeof (cfg)); + +/* update state */ + qm_port->state =PORT_STARTED ;/* enabled at create time */ + qm_port->config_state =DLB_CONFIGURED ; + + qm_port->dir_credits = cfg.dir_credit_high_watermark; + qm_port->ldb_credits = cfg.ldb_credit_high_watermark; + +DLB_LOG_DBG ("dlb: created dir port %d, depth = %d cr=%d,%d\n" , + qm_port_id, + cq_depth, + cfg.dir_credit_high_watermark, + cfg.ldb_credit_high_watermark); + +rte_spinlock_unlock (&handle->resource_lock); + +return 0 ; + +error_exit: +if (qm_port) { + qm_port->pp_mmio_base =0 ; +dlb_free_qe_mem (qm_port); + } + +rte_spinlock_unlock (&handle->resource_lock); + +DLB_LOG_ERR ("dlb: create dir port failed!\n" ); + +return ret; +} + static int32_t dlb_hw_create_ldb_queue(struct dlb_eventdev *dlb, struct dlb_queue *queue,
@@ -280,6 +786,15 @@ int dlb_string_to_int(int *result, const char *str)return qm_qid; } +static inline void +dlb_hw_do_enqueue (struct dlb_port *qm_port, +struct process_local_port_data *port_data) +{ +DLB_LOG_DBG ("dlb: Flushing QE(s) to DLB\n" ); + +dlb_pp_write (qm_port->qe4, port_data); +} +/* VDEV-only notes: * This function first unmaps all memory mappings and closes the * domain's file descriptor, which causes the driver to reset the
@@ -557,6 +1072,598 @@ int dlb_string_to_int(int *result, const char *str) *dev_info = evdev_dlb_default_info; } +static inline int +dlb_check_enqueue_sw_credits (struct dlb_eventdev *dlb, +struct dlb_eventdev_port *ev_port) +{ +uint32_t sw_inflights =rte_atomic32_read (&dlb->inflights); +const int num =1 ; + +if (unlikely (ev_port->inflight_max < sw_inflights)) { +DLB_INC_STAT (ev_port->stats.traffic.tx_nospc_inflight_max,1 ); + rte_errno = -ENOSPC ; +return 1 ; + } + +if (ev_port->inflight_credits < num) { +/* check if event enqueue brings ev_port over max threshold */ +uint32_t credit_update_quanta = ev_port->credit_update_quanta; + +if (sw_inflights + credit_update_quanta > + dlb->new_event_limit) { +DLB_INC_STAT ( + ev_port->stats.traffic.tx_nospc_new_event_limit, +1 ); + rte_errno = -ENOSPC ; +return 1 ; + } + +rte_atomic32_add (&dlb->inflights, credit_update_quanta); + ev_port->inflight_credits += (credit_update_quanta); + +if (ev_port->inflight_credits < num) { +DLB_INC_STAT ( + ev_port->stats.traffic.tx_nospc_inflight_credits, +1 ); + rte_errno = -ENOSPC ; +return 1 ; + } + } + +return 0 ; +} + +static inline void +dlb_replenish_sw_credits (struct dlb_eventdev *dlb, +struct dlb_eventdev_port *ev_port) +{ +uint16_t quanta = ev_port->credit_update_quanta; + +if (ev_port->inflight_credits >= quanta *2 ) { +rte_atomic32_sub (&dlb->inflights, quanta); + ev_port->inflight_credits -= quanta; + } +} + +static inline int +dlb_check_enqueue_hw_ldb_credits (struct dlb_port *qm_port, +struct process_local_port_data *port_data) +{ +if (unlikely (qm_port->cached_ldb_credits ==0 )) { +uint16_t pc; + + pc =dlb_read_pc (port_data, true); + + qm_port->cached_ldb_credits = pc - + qm_port->ldb_pushcount_at_credit_expiry; +if (unlikely (qm_port->cached_ldb_credits ==0 )) { +DLB_INC_STAT ( + qm_port->ev_port->stats.traffic.tx_nospc_ldb_hw_credits, +1 ); + +DLB_LOG_DBG ("ldb credits exhausted\n" ); +return 1 ; + } + qm_port->ldb_pushcount_at_credit_expiry += + qm_port->cached_ldb_credits; + } + +return 0 ; +} + +static inline int +dlb_check_enqueue_hw_dir_credits (struct dlb_port *qm_port, +struct process_local_port_data *port_data) +{ +if (unlikely (qm_port->cached_dir_credits ==0 )) { +uint16_t pc; + + pc =dlb_read_pc (port_data, false); + + qm_port->cached_dir_credits = pc - + qm_port->dir_pushcount_at_credit_expiry; + +if (unlikely (qm_port->cached_dir_credits ==0 )) { +DLB_INC_STAT ( + qm_port->ev_port->stats.traffic.tx_nospc_dir_hw_credits, +1 ); + +DLB_LOG_DBG ("dir credits exhausted\n" ); +return 1 ; + } + qm_port->dir_pushcount_at_credit_expiry += + qm_port->cached_dir_credits; + } + +return 0 ; +} + +static inline int +dlb_event_enqueue_prep (struct dlb_eventdev_port *ev_port, +struct dlb_port *qm_port, +const struct rte_event ev[], +struct process_local_port_data *port_data, +uint8_t *sched_type, +uint8_t *queue_id) +{ +struct dlb_eventdev *dlb = ev_port->dlb; +struct dlb_eventdev_queue *ev_queue; +uint16_t *cached_credits =NULL ; +struct dlb_queue *qm_queue; + + ev_queue = &dlb->ev_queues[ev->queue_id]; + qm_queue = &ev_queue->qm_queue; + *queue_id = qm_queue->id; + +/* Ignore sched_type and hardware credits on release events */ +if (ev->op ==RTE_EVENT_OP_RELEASE ) + goto op_check; + +if (!qm_queue->is_directed) { +/* Load balanced destination queue */ + +if (dlb_check_enqueue_hw_ldb_credits (qm_port, port_data)) { + rte_errno = -ENOSPC ; +return 1 ; + } + cached_credits = &qm_port->cached_ldb_credits; + +switch (ev->sched_type) { +case RTE_SCHED_TYPE_ORDERED : +DLB_LOG_DBG ("dlb: put_qe: RTE_SCHED_TYPE_ORDERED\n" ); +if (qm_queue->sched_type !=RTE_SCHED_TYPE_ORDERED ) { +DLB_LOG_ERR ("dlb: tried to send ordered event to unordered queue %d\n" , + *queue_id); + rte_errno = -EINVAL ; +return 1 ; + } + *sched_type =DLB_SCHED_ORDERED ; +break ; +case RTE_SCHED_TYPE_ATOMIC : +DLB_LOG_DBG ("dlb: put_qe: RTE_SCHED_TYPE_ATOMIC\n" ); + *sched_type =DLB_SCHED_ATOMIC ; +break ; +case RTE_SCHED_TYPE_PARALLEL : +DLB_LOG_DBG ("dlb: put_qe: RTE_SCHED_TYPE_PARALLEL\n" ); +if (qm_queue->sched_type ==RTE_SCHED_TYPE_ORDERED ) + *sched_type =DLB_SCHED_ORDERED ; +else + *sched_type =DLB_SCHED_UNORDERED ; +break ; +default : +DLB_LOG_ERR ("Unsupported LDB sched type in put_qe\n" ); +DLB_INC_STAT (ev_port->stats.tx_invalid,1 ); + rte_errno = -EINVAL ; +return 1 ; + } + }else { +/* Directed destination queue */ + +if (dlb_check_enqueue_hw_dir_credits (qm_port, port_data)) { + rte_errno = -ENOSPC ; +return 1 ; + } + cached_credits = &qm_port->cached_dir_credits; + +DLB_LOG_DBG ("dlb: put_qe: RTE_SCHED_TYPE_DIRECTED\n" ); + + *sched_type =DLB_SCHED_DIRECTED ; + } + +op_check: +switch (ev->op) { +case RTE_EVENT_OP_NEW : +/* Check that a sw credit is available */ +if (dlb_check_enqueue_sw_credits (dlb, ev_port)) { + rte_errno = -ENOSPC ; +return 1 ; + } + ev_port->inflight_credits--; + (*cached_credits)--; +break ; +case RTE_EVENT_OP_FORWARD : +/* Check for outstanding_releases underflow. If this occurs, +* the application is not using the EVENT_OPs correctly; for +* example, forwarding or releasing events that were not +* dequeued. +*/ +RTE_ASSERT (ev_port->outstanding_releases >0 ); + ev_port->outstanding_releases--; + qm_port->issued_releases++; + (*cached_credits)--; +break ; +case RTE_EVENT_OP_RELEASE : + ev_port->inflight_credits++; +/* Check for outstanding_releases underflow. If this occurs, +* the application is not using the EVENT_OPs correctly; for +* example, forwarding or releasing events that were not +* dequeued. +*/ +RTE_ASSERT (ev_port->outstanding_releases >0 ); + ev_port->outstanding_releases--; + qm_port->issued_releases++; +/* Replenish s/w credits if enough are cached */ +dlb_replenish_sw_credits (dlb, ev_port); +break ; + } + +DLB_INC_STAT (ev_port->stats.tx_op_cnt[ev->op],1 ); +DLB_INC_STAT (ev_port->stats.traffic.tx_ok,1 ); + +#ifndef RTE_LIBRTE_PMD_DLB_QUELL_STATS +if (ev->op !=RTE_EVENT_OP_RELEASE ) { +DLB_INC_STAT (ev_port->stats.enq_ok[ev->queue_id],1 ); +DLB_INC_STAT (ev_port->stats.tx_sched_cnt[*sched_type],1 ); + } +#endif + +return 0 ; +} + +static uint8_t cmd_byte_map[NUM_DLB_PORT_TYPES ][DLB_NUM_HW_SCHED_TYPES ] = { + { +/* Load-balanced cmd bytes */ + [RTE_EVENT_OP_NEW ] =DLB_NEW_CMD_BYTE , + [RTE_EVENT_OP_FORWARD ] =DLB_FWD_CMD_BYTE , + [RTE_EVENT_OP_RELEASE ] =DLB_COMP_CMD_BYTE , + }, + { +/* Directed cmd bytes */ + [RTE_EVENT_OP_NEW ] =DLB_NEW_CMD_BYTE , + [RTE_EVENT_OP_FORWARD ] =DLB_NEW_CMD_BYTE , + [RTE_EVENT_OP_RELEASE ] =DLB_NOOP_CMD_BYTE , + }, +}; + +static inline void +dlb_event_build_hcws (struct dlb_port *qm_port, +const struct rte_event ev[], +int num, +uint8_t *sched_type, +uint8_t *queue_id) +{ +struct dlb_enqueue_qe *qe; +uint16_t sched_word[4 ]; +__m128i sse_qe[2 ]; +int i; + + qe = qm_port->qe4; + + sse_qe[0 ] =_mm_setzero_si128 (); + sse_qe[1 ] =_mm_setzero_si128 (); + +switch (num) { +case 4 : +/* Construct the metadata portion of two HCWs in one 128b SSE +* register. HCW metadata is constructed in the SSE registers +* like so: +* sse_qe[0][63:0]: qe[0]'s metadata +* sse_qe[0][127:64]: qe[1]'s metadata +* sse_qe[1][63:0]: qe[2]'s metadata +* sse_qe[1][127:64]: qe[3]'s metadata +*/ + +/* Convert the event operation into a command byte and store it +* in the metadata: +* sse_qe[0][63:56] = cmd_byte_map[is_directed][ev[0].op] +* sse_qe[0][127:120] = cmd_byte_map[is_directed][ev[1].op] +* sse_qe[1][63:56] = cmd_byte_map[is_directed][ev[2].op] +* sse_qe[1][127:120] = cmd_byte_map[is_directed][ev[3].op] +*/ +#define DLB_QE_CMD_BYTE 7 + sse_qe[0 ] =_mm_insert_epi8 (sse_qe[0 ], + cmd_byte_map[qm_port->is_directed][ev[0 ].op], +DLB_QE_CMD_BYTE ); + sse_qe[0 ] =_mm_insert_epi8 (sse_qe[0 ], + cmd_byte_map[qm_port->is_directed][ev[1 ].op], +DLB_QE_CMD_BYTE +8 ); + sse_qe[1 ] =_mm_insert_epi8 (sse_qe[1 ], + cmd_byte_map[qm_port->is_directed][ev[2 ].op], +DLB_QE_CMD_BYTE ); + sse_qe[1 ] =_mm_insert_epi8 (sse_qe[1 ], + cmd_byte_map[qm_port->is_directed][ev[3 ].op], +DLB_QE_CMD_BYTE +8 ); + +/* Store priority, scheduling type, and queue ID in the sched +* word array because these values are re-used when the +* destination is a directed queue. +*/ + sched_word[0 ] =EV_TO_DLB_PRIO (ev[0 ].priority) <<10 | + sched_type[0 ] <<8 | + queue_id[0 ]; + sched_word[1 ] =EV_TO_DLB_PRIO (ev[1 ].priority) <<10 | + sched_type[1 ] <<8 | + queue_id[1 ]; + sched_word[2 ] =EV_TO_DLB_PRIO (ev[2 ].priority) <<10 | + sched_type[2 ] <<8 | + queue_id[2 ]; + sched_word[3 ] =EV_TO_DLB_PRIO (ev[3 ].priority) <<10 | + sched_type[3 ] <<8 | + queue_id[3 ]; + +/* Store the event priority, scheduling type, and queue ID in +* the metadata: +* sse_qe[0][31:16] = sched_word[0] +* sse_qe[0][95:80] = sched_word[1] +* sse_qe[1][31:16] = sched_word[2] +* sse_qe[1][95:80] = sched_word[3] +*/ +#define DLB_QE_QID_SCHED_WORD 1 + sse_qe[0 ] =_mm_insert_epi16 (sse_qe[0 ], + sched_word[0 ], +DLB_QE_QID_SCHED_WORD ); + sse_qe[0 ] =_mm_insert_epi16 (sse_qe[0 ], + sched_word[1 ], +DLB_QE_QID_SCHED_WORD +4 ); + sse_qe[1 ] =_mm_insert_epi16 (sse_qe[1 ], + sched_word[2 ], +DLB_QE_QID_SCHED_WORD ); + sse_qe[1 ] =_mm_insert_epi16 (sse_qe[1 ], + sched_word[3 ], +DLB_QE_QID_SCHED_WORD +4 ); + +/* If the destination is a load-balanced queue, store the lock +* ID. If it is a directed queue, DLB places this field in +* bytes 10-11 of the received QE, so we format it accordingly: +* sse_qe[0][47:32] = dir queue ? sched_word[0] : flow_id[0] +* sse_qe[0][111:96] = dir queue ? sched_word[1] : flow_id[1] +* sse_qe[1][47:32] = dir queue ? sched_word[2] : flow_id[2] +* sse_qe[1][111:96] = dir queue ? sched_word[3] : flow_id[3] +*/ +#define DLB_QE_LOCK_ID_WORD 2 + sse_qe[0 ] =_mm_insert_epi16 (sse_qe[0 ], + (sched_type[0 ] ==DLB_SCHED_DIRECTED ) ? + sched_word[0 ] : ev[0 ].flow_id, +DLB_QE_LOCK_ID_WORD ); + sse_qe[0 ] =_mm_insert_epi16 (sse_qe[0 ], + (sched_type[1 ] ==DLB_SCHED_DIRECTED ) ? + sched_word[1 ] : ev[1 ].flow_id, +DLB_QE_LOCK_ID_WORD +4 ); + sse_qe[1 ] =_mm_insert_epi16 (sse_qe[1 ], + (sched_type[2 ] ==DLB_SCHED_DIRECTED ) ? + sched_word[2 ] : ev[2 ].flow_id, +DLB_QE_LOCK_ID_WORD ); + sse_qe[1 ] =_mm_insert_epi16 (sse_qe[1 ], + (sched_type[3 ] ==DLB_SCHED_DIRECTED ) ? + sched_word[3 ] : ev[3 ].flow_id, +DLB_QE_LOCK_ID_WORD +4 ); + +/* Store the event type and sub event type in the metadata: +* sse_qe[0][15:0] = flow_id[0] +* sse_qe[0][79:64] = flow_id[1] +* sse_qe[1][15:0] = flow_id[2] +* sse_qe[1][79:64] = flow_id[3] +*/ +#define DLB_QE_EV_TYPE_WORD 0 + sse_qe[0 ] =_mm_insert_epi16 (sse_qe[0 ], + ev[0 ].sub_event_type <<8 | + ev[0 ].event_type, +DLB_QE_EV_TYPE_WORD ); + sse_qe[0 ] =_mm_insert_epi16 (sse_qe[0 ], + ev[1 ].sub_event_type <<8 | + ev[1 ].event_type, +DLB_QE_EV_TYPE_WORD +4 ); + sse_qe[1 ] =_mm_insert_epi16 (sse_qe[1 ], + ev[2 ].sub_event_type <<8 | + ev[2 ].event_type, +DLB_QE_EV_TYPE_WORD ); + sse_qe[1 ] =_mm_insert_epi16 (sse_qe[1 ], + ev[3 ].sub_event_type <<8 | + ev[3 ].event_type, +DLB_QE_EV_TYPE_WORD +4 ); + +/* Store the metadata to memory (use the double-precision +* _mm_storeh_pd because there is no integer function for +* storing the upper 64b): +* qe[0] metadata = sse_qe[0][63:0] +* qe[1] metadata = sse_qe[0][127:64] +* qe[2] metadata = sse_qe[1][63:0] +* qe[3] metadata = sse_qe[1][127:64] +*/ +_mm_storel_epi64 ((__m128i *)&qe[0 ].u.opaque_data, sse_qe[0 ]); +_mm_storeh_pd ((double *)&qe[1 ].u.opaque_data, + (__m128d ) sse_qe[0 ]); +_mm_storel_epi64 ((__m128i *)&qe[2 ].u.opaque_data, sse_qe[1 ]); +_mm_storeh_pd ((double *)&qe[3 ].u.opaque_data, + (__m128d ) sse_qe[1 ]); + + qe[0 ].data = ev[0 ].u64; + qe[1 ].data = ev[1 ].u64; + qe[2 ].data = ev[2 ].u64; + qe[3 ].data = ev[3 ].u64; + +break ; +case 3 : +case 2 : +case 1 : +/* At least one QE will be valid, so only zero out three */ + qe[1 ].cmd_byte =0 ; + qe[2 ].cmd_byte =0 ; + qe[3 ].cmd_byte =0 ; + +for (i =0 ; i < num; i++) { + qe[i].cmd_byte = + cmd_byte_map[qm_port->is_directed][ev[i].op]; + qe[i].sched_type = sched_type[i]; + qe[i].data = ev[i].u64; + qe[i].qid = queue_id[i]; + qe[i].priority =EV_TO_DLB_PRIO (ev[i].priority); + qe[i].lock_id = ev[i].flow_id; +if (sched_type[i] ==DLB_SCHED_DIRECTED ) { +struct dlb_msg_info *info = + (struct dlb_msg_info *)&qe[i].lock_id; + + info->qid = queue_id[i]; + info->sched_type =DLB_SCHED_DIRECTED ; + info->priority = qe[i].priority; + } + qe[i].u.event_type.major = ev[i].event_type; + qe[i].u.event_type.sub = ev[i].sub_event_type; + } +break ; + } +} + +static inline void +dlb_construct_token_pop_qe (struct dlb_port *qm_port,int idx) +{ +struct dlb_cq_pop_qe *qe = (void *)qm_port->qe4; +int num = qm_port->owed_tokens; + +if (qm_port->use_rsvd_token_scheme) { +/* Check if there's a deficit of reserved tokens, and return +* early if there are no (unreserved) tokens to consume. +*/ +if (num <= qm_port->cq_rsvd_token_deficit) { + qm_port->cq_rsvd_token_deficit -= num; + qm_port->owed_tokens =0 ; +return ; + } + num -= qm_port->cq_rsvd_token_deficit; + qm_port->cq_rsvd_token_deficit =0 ; + } + + qe[idx].cmd_byte =DLB_POP_CMD_BYTE ; + qe[idx].tokens = num -1 ; + + qm_port->owed_tokens =0 ; +} + +static inline uint16_t +__dlb_event_enqueue_burst (void *event_port, +const struct rte_event events[], +uint16_t num, +bool use_delayed) +{ +struct dlb_eventdev_port *ev_port = event_port; +struct dlb_port *qm_port = &ev_port->qm_port; +struct process_local_port_data *port_data; +int i, cnt; + +RTE_ASSERT (ev_port->enq_configured); +RTE_ASSERT (events !=NULL ); + + rte_errno =0 ; + cnt =0 ; + + port_data = &dlb_port[qm_port->id][PORT_TYPE (qm_port)]; + +if (!port_data->mmaped) +dlb_iface_port_mmap (qm_port); + +/* Since MOVDIR64B is weakly-ordered, use an SFENCE to ensure that +* application writes complete before enqueueing the release HCW. +*/ +rte_wmb (); + +for (i =0 ; i < num; i +=DLB_NUM_QES_PER_CACHE_LINE ) { +uint8_t sched_types[DLB_NUM_QES_PER_CACHE_LINE ]; +uint8_t queue_ids[DLB_NUM_QES_PER_CACHE_LINE ]; +int j =0 ; + +for (; j <DLB_NUM_QES_PER_CACHE_LINE && (i + j) < num; j++) { +const struct rte_event *ev = &events[i + j]; + +if (dlb_event_enqueue_prep (ev_port, qm_port, ev, + port_data, &sched_types[j], + &queue_ids[j])) +break ; + } + +if (j ==0 ) +break ; + +dlb_event_build_hcws (qm_port, &events[i], j, + sched_types, queue_ids); + +/* The delayed-pop code causes an unnecessary performance +* penalty when it is not in use. The use_delayed argument +* allows the compiler to create a version of this function +* with these checks factored out that the PMD can call +* when delayed-pop is not in use. +*/ +if (use_delayed && + qm_port->token_pop_mode ==DELAYED_POP && j <4 && + qm_port->issued_releases >= qm_port->token_pop_thresh -1 ) { + +dlb_construct_token_pop_qe (qm_port, j); + +/* Reset the releases counter for the next QE batch */ + qm_port->issued_releases -= qm_port->token_pop_thresh; + +/* When using delayed token pop mode, the initial token +* threshold is the full CQ depth. After the first +* token pop, we need to reset it to the dequeue_depth. +*/ + qm_port->token_pop_thresh = qm_port->dequeue_depth; + } + +dlb_hw_do_enqueue (qm_port, port_data); + + cnt += j; + +if (j <DLB_NUM_QES_PER_CACHE_LINE ) +break ; + } + +if (use_delayed && qm_port->token_pop_mode ==DELAYED_POP && + qm_port->issued_releases >= qm_port->token_pop_thresh -1 ) { +dlb_consume_qe_immediate (qm_port, qm_port->owed_tokens); + qm_port->issued_releases -= qm_port->token_pop_thresh; + qm_port->token_pop_thresh = qm_port->dequeue_depth; + } + +RTE_ASSERT (!((cnt ==0 && rte_errno != -ENOSPC ))); + +return cnt; +} + +static inline uint16_t +dlb_event_enqueue_burst (void *event_port, +const struct rte_event events[], +uint16_t num) +{ +return __dlb_event_enqueue_burst (event_port, events, num, false); +} + +static inline uint16_t +dlb_event_enqueue_burst_delayed (void *event_port, +const struct rte_event events[], +uint16_t num) +{ +return __dlb_event_enqueue_burst (event_port, events, num, true); +} + +static inline uint16_t +dlb_event_enqueue (void *event_port, +const struct rte_event events[]) +{ +return __dlb_event_enqueue_burst (event_port, events,1 , false); +} + +static inline uint16_t +dlb_event_enqueue_delayed (void *event_port, +const struct rte_event events[]) +{ +return __dlb_event_enqueue_burst (event_port, events,1 , true); +} + +static uint16_t +dlb_event_enqueue_new_burst_delayed (void *event_port, +const struct rte_event events[], +uint16_t num) +{ +return __dlb_event_enqueue_burst (event_port, events, num, true); +} + +static uint16_t +dlb_event_enqueue_forward_burst_delayed (void *event_port, +const struct rte_event events[], +uint16_t num) +{ +return __dlb_event_enqueue_burst (event_port, events, num, true); +} +/* Note: 1 QM instance per QM device, QM instance/device == event device */ static int dlb_eventdev_configure (const struct rte_eventdev *dev)
@@ -934,6 +2041,133 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb)return ret; } +static void +dlb_port_link_teardown (struct dlb_eventdev *dlb, +struct dlb_eventdev_port *ev_port) +{ +struct dlb_eventdev_queue *ev_queue; +int i; + +for (i =0 ; i <DLB_MAX_NUM_QIDS_PER_LDB_CQ ; i++) { +if (!ev_port->link[i].valid) +continue ; + + ev_queue = &dlb->ev_queues[ev_port->link[i].queue_id]; + + ev_port->link[i].valid = false; + ev_port->num_links--; + ev_queue->num_links--; + } +} + +static int +dlb_eventdev_port_setup (struct rte_eventdev *dev, +uint8_t ev_port_id, +const struct rte_event_port_conf *port_conf) +{ +struct dlb_eventdev *dlb; +struct dlb_eventdev_port *ev_port; +bool use_rsvd_token_scheme; +uint32_t adj_cq_depth; +uint16_t rsvd_tokens; +int ret; + +if (dev ==NULL || port_conf ==NULL ) { +DLB_LOG_ERR ("Null parameter\n" ); +return -EINVAL ; + } + + dlb =dlb_pmd_priv (dev); + +if (ev_port_id >=DLB_MAX_NUM_PORTS ) +return -EINVAL ; + +if (port_conf->dequeue_depth > + evdev_dlb_default_info.max_event_port_dequeue_depth || + port_conf->enqueue_depth > + evdev_dlb_default_info.max_event_port_enqueue_depth) +return -EINVAL ; + + ev_port = &dlb->ev_ports[ev_port_id]; +/* configured? */ +if (ev_port->setup_done) { +DLB_LOG_ERR ("evport %d is already configured\n" , ev_port_id); +return -EINVAL ; + } + +/* The reserved token interrupt arming scheme requires that one or more +* CQ tokens be reserved by the PMD. This limits the amount of CQ space +* usable by the DLB, so in order to give an *effective* CQ depth equal +* to the user-requested value, we double CQ depth and reserve half of +* its tokens. If the user requests the max CQ depth (256) then we +* cannot double it, so we reserve one token and give an effective +* depth of 255 entries. +*/ + use_rsvd_token_scheme = true; + rsvd_tokens =1 ; + adj_cq_depth = port_conf->dequeue_depth; + +if (use_rsvd_token_scheme && adj_cq_depth <256 ) { + rsvd_tokens = adj_cq_depth; + adj_cq_depth *=2 ; + } + + ev_port->qm_port.is_directed = port_conf->event_port_cfg & +RTE_EVENT_PORT_CFG_SINGLE_LINK ; + +if (!ev_port->qm_port.is_directed) { + ret =dlb_hw_create_ldb_port (dlb, + ev_port, + port_conf->dequeue_depth, + adj_cq_depth, + port_conf->enqueue_depth, + rsvd_tokens, + use_rsvd_token_scheme); +if (ret <0 ) { +DLB_LOG_ERR ("Failed to create the lB port ve portId=%d\n" , + ev_port_id); +return ret; + } + }else { + ret =dlb_hw_create_dir_port (dlb, + ev_port, + port_conf->dequeue_depth, + adj_cq_depth, + port_conf->enqueue_depth, + rsvd_tokens, + use_rsvd_token_scheme); +if (ret <0 ) { +DLB_LOG_ERR ("Failed to create the DIR port\n" ); +return ret; + } + } + +/* Save off port config for reconfig */ + dlb->ev_ports[ev_port_id].conf = *port_conf; + + dlb->ev_ports[ev_port_id].id = ev_port_id; + dlb->ev_ports[ev_port_id].enq_configured = true; + dlb->ev_ports[ev_port_id].setup_done = true; + dlb->ev_ports[ev_port_id].inflight_max = + port_conf->new_event_threshold; + dlb->ev_ports[ev_port_id].implicit_release = + !(port_conf->event_port_cfg & +RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL ); + dlb->ev_ports[ev_port_id].outstanding_releases =0 ; + dlb->ev_ports[ev_port_id].inflight_credits =0 ; + dlb->ev_ports[ev_port_id].credit_update_quanta = +RTE_LIBRTE_PMD_DLB_SW_CREDIT_QUANTA ; + dlb->ev_ports[ev_port_id].dlb = dlb;/* reverse link */ + +/* Tear down pre-existing port->queue links */ +if (dlb->run_state ==DLB_RUN_STATE_STOPPED ) +dlb_port_link_teardown (dlb, &dlb->ev_ports[ev_port_id]); + + dev->data->ports[ev_port_id] = &dlb->ev_ports[ev_port_id]; + +return 0 ; +} + static int set_dev_id(const char *key __rte_unused, const char *value,
@@ -1013,6 +2247,7 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb) .queue_def_conf = dlb_eventdev_queue_default_conf_get, .queue_setup = dlb_eventdev_queue_setup, .port_def_conf = dlb_eventdev_port_default_conf_get, + .port_setup = dlb_eventdev_port_setup, };/* Expose PMD's eventdev interface */
Signature · 1 line
1.7.10
PATCH 20/27 event/dlb: add port_link
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb.c | 303 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 303 insertions(+)
drivers/event/dlb/dlb.c +303 -0
@@ -786,6 +786,29 @@ int dlb_string_to_int(int *result, const char *str)return qm_qid; } +static int32_t +dlb_hw_create_dir_queue (struct dlb_eventdev *dlb,int32_t qm_port_id) +{ +struct dlb_hw_dev *handle = &dlb->qm_instance; +struct dlb_create_dir_queue_args cfg; +struct dlb_cmd_response response; +int32_t ret; + + cfg.response = (uintptr_t )&response; + +/* The directed port is always configured before its queue */ + cfg.port_id = qm_port_id; + + ret =dlb_iface_dir_queue_create (handle, &cfg); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: create DIR event queue error, ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); +return -EINVAL ; + } + +return response.id; +} +static inline void dlb_hw_do_enqueue (struct dlb_port *qm_port,struct process_local_port_data *port_data)
@@ -795,6 +818,42 @@ int dlb_string_to_int(int *result, const char *str)dlb_pp_write (qm_port->qe4, port_data); } +static int16_t +dlb_hw_map_ldb_qid_to_port (struct dlb_hw_dev *handle, +uint32_t qm_port_id, +uint16_t qm_qid, +uint8_t priority) +{ +struct dlb_map_qid_args cfg; +struct dlb_cmd_response response; +int32_t ret; + +if (handle ==NULL ) +return -EINVAL ; + +/* Build message */ + cfg.response = (uintptr_t )&response; + cfg.port_id = qm_port_id; + cfg.qid = qm_qid; + cfg.priority =EV_TO_DLB_PRIO (priority); + + ret =dlb_iface_map_qid (handle, &cfg); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: map qid error, ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); +DLB_LOG_ERR ("dlb: device_id=%d grp=%d, qm_port=%d, qm_qid=%d prio=%d\n" , + handle->device_id, + handle->domain_id, cfg.port_id, + cfg.qid, + cfg.priority); + }else { +DLB_LOG_DBG ("dlb: mapped queue %d to qm_port %d\n" , + qm_qid, qm_port_id); + } + +return ret; +} +/* VDEV-only notes: * This function first unmaps all memory mappings and closes the * domain's file descriptor, which causes the driver to reset the
@@ -1778,6 +1837,249 @@ int dlb_string_to_int(int *result, const char *str)return 0 ; } +static int +dlb_event_queue_join_ldb (struct dlb_eventdev *dlb, +struct dlb_eventdev_port *ev_port, +struct dlb_eventdev_queue *ev_queue, +uint8_t priority) +{ +int first_avail =-1 ; +int ret, i; + +for (i =0 ; i <DLB_MAX_NUM_QIDS_PER_LDB_CQ ; i++) { +if (ev_port->link[i].valid) { +if (ev_port->link[i].queue_id == ev_queue->id && + ev_port->link[i].priority == priority) { +if (ev_port->link[i].mapped) +return 0 ;/* already mapped */ + first_avail = i; + } + }else { +if (first_avail ==-1 ) + first_avail = i; + } + } +if (first_avail ==-1 ) { +DLB_LOG_ERR ("dlb: qm_port %d has no available QID slots.\n" , + ev_port->qm_port.id); +return -EINVAL ; + } + + ret =dlb_hw_map_ldb_qid_to_port (&dlb->qm_instance, + ev_port->qm_port.id, + ev_queue->qm_queue.id, + priority); + +if (!ret) + ev_port->link[first_avail].mapped = true; + +return ret; +} + +static int +dlb_eventdev_dir_queue_setup (struct dlb_eventdev *dlb, +struct dlb_eventdev_queue *ev_queue, +struct dlb_eventdev_port *ev_port) +{ +int32_t qm_qid; + + qm_qid =dlb_hw_create_dir_queue (dlb, ev_port->qm_port.id); + +if (qm_qid <0 ) { +DLB_LOG_ERR ("Failed to create the DIR queue\n" ); +return qm_qid; + } + + dlb->qm_dir_to_ev_queue_id[qm_qid] = ev_queue->id; + + ev_queue->qm_queue.id = qm_qid; + +return 0 ; +} + +static int +dlb_do_port_link (struct rte_eventdev *dev, +struct dlb_eventdev_queue *ev_queue, +struct dlb_eventdev_port *ev_port, +uint8_t prio) +{ +struct dlb_eventdev *dlb =dlb_pmd_priv (dev); +int err; + +/* Don't link until start time. */ +if (dlb->run_state ==DLB_RUN_STATE_STOPPED ) +return 0 ; + +if (ev_queue->qm_queue.is_directed) + err =dlb_eventdev_dir_queue_setup (dlb, ev_queue, ev_port); +else + err =dlb_event_queue_join_ldb (dlb, ev_port, ev_queue, prio); + +if (err) { +DLB_LOG_ERR ("port link failure for %s ev_q %d, ev_port %d\n" , + ev_queue->qm_queue.is_directed ?"DIR" :"LDB" , + ev_queue->id, ev_port->id); + + rte_errno = err; +return -1 ; + } + +return 0 ; +} + +static int +dlb_validate_port_link (struct dlb_eventdev_port *ev_port, +uint8_t queue_id, +bool link_exists, +int index) +{ +struct dlb_eventdev *dlb = ev_port->dlb; +struct dlb_eventdev_queue *ev_queue; +bool port_is_dir, queue_is_dir; + +if (queue_id > dlb->num_queues) { + rte_errno = -EINVAL ; +return -1 ; + } + + ev_queue = &dlb->ev_queues[queue_id]; + +if (!ev_queue->setup_done && + ev_queue->qm_queue.config_state !=DLB_PREV_CONFIGURED ) { + rte_errno = -EINVAL ; +return -1 ; + } + + port_is_dir = ev_port->qm_port.is_directed; + queue_is_dir = ev_queue->qm_queue.is_directed; + +if (port_is_dir != queue_is_dir) { +DLB_LOG_ERR ("%s queue %u can't link to %s port %u\n" , + queue_is_dir ?"DIR" :"LDB" , ev_queue->id, + port_is_dir ?"DIR" :"LDB" , ev_port->id); + + rte_errno = -EINVAL ; +return -1 ; + } + +/* Check if there is space for the requested link */ +if (!link_exists && index ==-1 ) { +DLB_LOG_ERR ("no space for new link\n" ); + rte_errno = -ENOSPC ; +return -1 ; + } + +/* Check if the directed port is already linked */ +if (ev_port->qm_port.is_directed && ev_port->num_links >0 && + !link_exists) { +DLB_LOG_ERR ("Can't link DIR port %d to >1 queues\n" , + ev_port->id); + rte_errno = -EINVAL ; +return -1 ; + } + +/* Check if the directed queue is already linked */ +if (ev_queue->qm_queue.is_directed && ev_queue->num_links >0 && + !link_exists) { +DLB_LOG_ERR ("Can't link DIR queue %d to >1 ports\n" , + ev_queue->id); + rte_errno = -EINVAL ; +return -1 ; + } + +return 0 ; +} + +static int +dlb_eventdev_port_link (struct rte_eventdev *dev,void *event_port, +const uint8_t queues[],const uint8_t priorities[], +uint16_t nb_links) + +{ +struct dlb_eventdev_port *ev_port = event_port; +struct dlb_eventdev *dlb; +int i, j; + +RTE_SET_USED (dev); + +if (!ev_port) { +DLB_LOG_ERR ("dlb: evport not setup\n" ); + rte_errno = -EINVAL ; +return 0 ; + } + +if (!ev_port->setup_done && + ev_port->qm_port.config_state !=DLB_PREV_CONFIGURED ) { +DLB_LOG_ERR ("dlb: evport not setup\n" ); + rte_errno = -EINVAL ; +return 0 ; + } + +/* Note: rte_event_port_link() ensures the PMD won't receive a NULL +* queues pointer. +*/ +if (nb_links ==0 ) { +DLB_LOG_DBG ("dlb: nb_links is 0\n" ); +return 0 ;/* Ignore and return success */ + } + + dlb = ev_port->dlb; + +DLB_LOG_DBG ("Linking %u queues to %s port %d\n" , + nb_links, + ev_port->qm_port.is_directed ?"DIR" :"LDB" , + ev_port->id); + +for (i =0 ; i < nb_links; i++) { +struct dlb_eventdev_queue *ev_queue; +uint8_t queue_id, prio; +bool found = false; +int index =-1 ; + + queue_id = queues[i]; + prio = priorities[i]; + +/* Check if the link already exists. */ +for (j =0 ; j <DLB_MAX_NUM_QIDS_PER_LDB_CQ ; j++) +if (ev_port->link[j].valid) { +if (ev_port->link[j].queue_id == queue_id) { + found = true; + index = j; +break ; + } + }else { +if (index ==-1 ) + index = j; + } + +/* could not link */ +if (index ==-1 ) +break ; + +/* Check if already linked at the requested priority */ +if (found && ev_port->link[j].priority == prio) +continue ; + +if (dlb_validate_port_link (ev_port, queue_id, found, index)) +break ;/* return index of offending queue */ + + ev_queue = &dlb->ev_queues[queue_id]; + +if (dlb_do_port_link (dev, ev_queue, ev_port, prio)) +break ;/* return index of offending queue */ + + ev_queue->num_links++; + + ev_port->link[index].queue_id = queue_id; + ev_port->link[index].priority = prio; + ev_port->link[index].valid = true; +/* Entry already exists? If so, then must be prio change */ +if (!found) + ev_port->num_links++; + } +return i; +} + static void dlb_eventdev_port_default_conf_get(struct rte_eventdev *dev, uint8_t port_id,
@@ -2248,6 +2550,7 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb) .queue_setup = dlb_eventdev_queue_setup, .port_def_conf = dlb_eventdev_port_default_conf_get, .port_setup = dlb_eventdev_port_setup, + .port_link = dlb_eventdev_port_link, };/* Expose PMD's eventdev interface */
Signature · 1 line
1.7.10
PATCH 21/27 event/dlb: add queue_release and port_release
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
drivers/event/dlb/dlb.c +25 -0
@@ -2470,6 +2470,29 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb)return 0 ; } +static void +dlb_eventdev_port_release (void *port) +{ +RTE_SET_USED (port); + +/* This function intentionally left blank. dlb does not support +* reconfiguring individual queues or ports -- the entire device +* must be reconfigured. +*/ +} + +static void +dlb_eventdev_queue_release (struct rte_eventdev *dev,uint8_t id) +{ +RTE_SET_USED (dev); +RTE_SET_USED (id); + +/* This function intentionally left blank. dlb does not support +* reconfiguring individual queues or ports -- the entire device +* must be reconfigured. +*/ +} + static int set_dev_id(const char *key __rte_unused, const char *value,
@@ -2548,8 +2571,10 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb) .dev_configure = dlb_eventdev_configure, .queue_def_conf = dlb_eventdev_queue_default_conf_get, .queue_setup = dlb_eventdev_queue_setup, + .queue_release = dlb_eventdev_queue_release, .port_def_conf = dlb_eventdev_port_default_conf_get, .port_setup = dlb_eventdev_port_setup, + .port_release = dlb_eventdev_port_release, .port_link = dlb_eventdev_port_link, };
Signature · 1 line
1.7.10
PATCH 22/27 event/dlb: add port_unlink and port_unlinks_in_progress
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb.c | 166 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 166 insertions(+)
drivers/event/dlb/dlb.c +166 -0
@@ -854,6 +854,30 @@ int dlb_string_to_int(int *result, const char *str)return ret; } +static int16_t +dlb_hw_unmap_ldb_qid_from_port (struct dlb_hw_dev *handle, +uint32_t qm_port_id, +uint16_t qm_qid) +{ +struct dlb_unmap_qid_args cfg; +struct dlb_cmd_response response; +int32_t ret; + +if (handle ==NULL ) +return -EINVAL ; + + cfg.response = (uintptr_t )&response; + cfg.port_id = qm_port_id; + cfg.qid = qm_qid; + + ret =dlb_iface_unmap_qid (handle, &cfg); +if (ret <0 ) +DLB_LOG_ERR ("dlb: unmap qid error, ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); + +return ret; +} +/* VDEV-only notes: * This function first unmaps all memory mappings and closes the * domain's file descriptor, which causes the driver to reset the
@@ -1898,6 +1922,42 @@ int dlb_string_to_int(int *result, const char *str) }static int +dlb_event_queue_detach_ldb (struct dlb_eventdev *dlb, +struct dlb_eventdev_port *ev_port, +struct dlb_eventdev_queue *ev_queue) +{ +int ret, i; + +/* Don't unlink until start time. */ +if (dlb->run_state ==DLB_RUN_STATE_STOPPED ) +return 0 ; + +for (i =0 ; i <DLB_MAX_NUM_QIDS_PER_LDB_CQ ; i++) { +if (ev_port->link[i].valid && + ev_port->link[i].queue_id == ev_queue->id) +break ;/* found */ + } + +/* This is expected with eventdev API! +* It blindly attemmpts to unmap all queues. +*/ +if (i ==DLB_MAX_NUM_QIDS_PER_LDB_CQ ) { +DLB_LOG_DBG ("dlb: ignoring LB QID %d not mapped for qm_port %d.\n" , + ev_queue->qm_queue.id, + ev_port->qm_port.id); +return 0 ; + } + + ret =dlb_hw_unmap_ldb_qid_from_port (&dlb->qm_instance, + ev_port->qm_port.id, + ev_queue->qm_queue.id); +if (!ret) + ev_port->link[i].mapped = false; + +return ret; +} + +static int dlb_do_port_link(struct rte_eventdev *dev, struct dlb_eventdev_queue *ev_queue, struct dlb_eventdev_port *ev_port,
@@ -2470,6 +2530,109 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb)return 0 ; } +static int +dlb_eventdev_port_unlink (struct rte_eventdev *dev,void *event_port, +uint8_t queues[],uint16_t nb_unlinks) +{ +struct dlb_eventdev_port *ev_port = event_port; +struct dlb_eventdev *dlb; +int i; + +RTE_SET_USED (dev); + +if (!ev_port->setup_done) { +DLB_LOG_ERR ("dlb: evport %d is not configured\n" , + ev_port->id); + rte_errno = -EINVAL ; +return 0 ; + } + +if (!queues || nb_unlinks ==0 ) { +DLB_LOG_DBG ("dlb: queues is NULL or nb_unlinks is 0\n" ); +return 0 ;/* Ignore and return success */ + } + +if (ev_port->qm_port.is_directed) { +DLB_LOG_DBG ("dlb: ignore unlink from dir port %d\n" , + ev_port->id); + rte_errno =0 ; +return nb_unlinks;/* as if success */ + } + + dlb = ev_port->dlb; + +for (i =0 ; i < nb_unlinks; i++) { +struct dlb_eventdev_queue *ev_queue; +int ret, j; + +if (queues[i] >= dlb->num_queues) { +DLB_LOG_ERR ("dlb: invalid queue id %d\n" , queues[i]); + rte_errno = -EINVAL ; +return i;/* return index of offending queue */ + } + + ev_queue = &dlb->ev_queues[queues[i]]; + +/* Does a link exist? */ +for (j =0 ; j <DLB_MAX_NUM_QIDS_PER_LDB_CQ ; j++) +if (ev_port->link[j].queue_id == queues[i] && + ev_port->link[j].valid) +break ; + +if (j ==DLB_MAX_NUM_QIDS_PER_LDB_CQ ) +continue ; + + ret =dlb_event_queue_detach_ldb (dlb, ev_port, ev_queue); +if (ret) { +DLB_LOG_ERR ("unlink err=%d for port %d queue %d\n" , + ret, ev_port->id, queues[i]); + rte_errno = -ENOENT ; +return i;/* return index of offending queue */ + } + + ev_port->link[j].valid = false; + ev_port->num_links--; + ev_queue->num_links--; + } + +return nb_unlinks; +} + +static int +dlb_eventdev_port_unlinks_in_progress (struct rte_eventdev *dev, +void *event_port) +{ +struct dlb_eventdev_port *ev_port = event_port; +struct dlb_eventdev *dlb; +struct dlb_hw_dev *handle; +struct dlb_pending_port_unmaps_args cfg; +struct dlb_cmd_response response; +int ret; + +RTE_SET_USED (dev); + +if (!ev_port->setup_done) { +DLB_LOG_ERR ("dlb: evport %d is not configured\n" , + ev_port->id); + rte_errno = -EINVAL ; +return 0 ; + } + + cfg.port_id = ev_port->qm_port.id; + cfg.response = (uintptr_t )&response; + dlb = ev_port->dlb; + handle = &dlb->qm_instance; + ret =dlb_iface_pending_port_unmaps (handle, &cfg); + +if (ret <0 ) { +DLB_LOG_ERR ("dlb: num_unlinks_in_progress ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); +return ret; + } + +return response.id; +} +static void dlb_eventdev_port_release (void *port) {
@@ -2576,6 +2739,9 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb) .port_setup = dlb_eventdev_port_setup, .port_release = dlb_eventdev_port_release, .port_link = dlb_eventdev_port_link, + .port_unlink = dlb_eventdev_port_unlink, + .port_unlinks_in_progress = + dlb_eventdev_port_unlinks_in_progress, };/* Expose PMD's eventdev interface */
Signature · 1 line
1.7.10
PATCH 23/27 event/dlb: add eventdev_start
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb.c | 142 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 142 insertions(+)
drivers/event/dlb/dlb.c +142 -0
@@ -1155,6 +1155,72 @@ int dlb_string_to_int(int *result, const char *str) *dev_info = evdev_dlb_default_info; } +static int +dlb_eventdev_reapply_configuration (struct rte_eventdev *dev); + +static int +dlb_eventdev_apply_port_links (struct rte_eventdev *dev); + +static int +dlb_eventdev_start (struct rte_eventdev *dev) +{ +struct dlb_eventdev *dlb =dlb_pmd_priv (dev); +struct dlb_hw_dev *handle = &dlb->qm_instance; +struct dlb_start_domain_args cfg; +struct dlb_cmd_response response; +int ret, i; + +rte_spinlock_lock (&dlb->qm_instance.resource_lock); +if (dlb->run_state !=DLB_RUN_STATE_STOPPED ) { +DLB_LOG_ERR ("bad state %d for dev_start\n" , + (int )dlb->run_state); +rte_spinlock_unlock (&dlb->qm_instance.resource_lock); +return -EINVAL ; + } + dlb->run_state =DLB_RUN_STATE_STARTING ; +rte_spinlock_unlock (&dlb->qm_instance.resource_lock); + +/* If the device was configured more than once, some event ports and/or +* queues may need to be reconfigured. +*/ + ret =dlb_eventdev_reapply_configuration (dev); +if (ret) +return ret; + +/* The DLB PMD delays port links until the device is started. */ + ret =dlb_eventdev_apply_port_links (dev); +if (ret) +return ret; + + cfg.response = (uintptr_t )&response; + +for (i =0 ; i < dlb->num_ports; i++) { +if (!dlb->ev_ports[i].setup_done) { +DLB_LOG_ERR ("dlb: port %d not setup" , i); +return -ESTALE ; + } + } + +for (i =0 ; i < dlb->num_queues; i++) { +if (dlb->ev_queues[i].num_links ==0 ) { +DLB_LOG_ERR ("dlb: queue %d is not linked" , i); +return -ENOLINK ; + } + } + + ret =dlb_iface_sched_domain_start (handle, &cfg); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: sched_domain_start ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); +return ret; + } + + dlb->run_state =DLB_RUN_STATE_STARTED ; +DLB_LOG_DBG ("dlb: sched_domain_start completed OK\n" ); + +return 0 ; +} +static inline int dlb_check_enqueue_sw_credits (struct dlb_eventdev *dlb,struct dlb_eventdev_port *ev_port)
@@ -2051,6 +2117,40 @@ int dlb_string_to_int(int *result, const char *str) }static int +dlb_eventdev_apply_port_links (struct rte_eventdev *dev) +{ +struct dlb_eventdev *dlb =dlb_pmd_priv (dev); +int i; + +/* Perform requested port->queue links */ +for (i =0 ; i < dlb->num_ports; i++) { +struct dlb_eventdev_port *ev_port = &dlb->ev_ports[i]; +int j; + +for (j =0 ; j <DLB_MAX_NUM_QIDS_PER_LDB_CQ ; j++) { +struct dlb_eventdev_queue *ev_queue; +uint8_t prio, queue_id; + +if (!ev_port->link[j].valid) +continue ; + + prio = ev_port->link[j].priority; + queue_id = ev_port->link[j].queue_id; + +if (dlb_validate_port_link (ev_port, queue_id, true, j)) +return -EINVAL ; + + ev_queue = &dlb->ev_queues[queue_id]; + +if (dlb_do_port_link (dev, ev_queue, ev_port, prio)) +return -EINVAL ; + } + } + +return 0 ; +} + +static int dlb_eventdev_port_link (struct rte_eventdev *dev,void *event_port,const uint8_t queues[],const uint8_t priorities[],uint16_t nb_links)
@@ -2531,6 +2631,47 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb) }static int +dlb_eventdev_reapply_configuration (struct rte_eventdev *dev) +{ +struct dlb_eventdev *dlb =dlb_pmd_priv (dev); +int ret, i; + +/* If an event queue or port was previously configured, but hasn't been +* reconfigured, reapply its original configuration. +*/ +for (i =0 ; i < dlb->num_queues; i++) { +struct dlb_eventdev_queue *ev_queue; + + ev_queue = &dlb->ev_queues[i]; + +if (ev_queue->qm_queue.config_state !=DLB_PREV_CONFIGURED ) +continue ; + + ret =dlb_eventdev_queue_setup (dev, i, &ev_queue->conf); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: failed to reconfigure queue %d" , i); +return ret; + } + } + +for (i =0 ; i < dlb->num_ports; i++) { +struct dlb_eventdev_port *ev_port = &dlb->ev_ports[i]; + +if (ev_port->qm_port.config_state !=DLB_PREV_CONFIGURED ) +continue ; + + ret =dlb_eventdev_port_setup (dev, i, &ev_port->conf); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: failed to reconfigure ev_port %d" , + i); +return ret; + } + } + +return 0 ; +} + +static int dlb_eventdev_port_unlink (struct rte_eventdev *dev,void *event_port,uint8_t queues[],uint16_t nb_unlinks) {
@@ -2732,6 +2873,7 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb)static struct rte_eventdev_ops dlb_eventdev_entry_ops = { .dev_infos_get = dlb_eventdev_info_get, .dev_configure = dlb_eventdev_configure, + .dev_start = dlb_eventdev_start, .queue_def_conf = dlb_eventdev_queue_default_conf_get, .queue_setup = dlb_eventdev_queue_setup, .queue_release = dlb_eventdev_queue_release,
Signature · 1 line
1.7.10
PATCH 24/27 event/dlb: add timeout_ticks, dump, xstats, and selftest
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
drivers/event/dlb/dlb.c +20 -0
@@ -1813,6 +1813,19 @@ int dlb_string_to_int(int *result, const char *str)return __dlb_event_enqueue_burst (event_port, events, num, true); } +static int +dlb_eventdev_timeout_ticks (struct rte_eventdev *dev,uint64_t ns, +uint64_t *timeout_ticks) +{ +RTE_SET_USED (dev); +uint64_t cycles_per_ns; + + cycles_per_ns =rte_get_timer_hz () /1E9 ; + *timeout_ticks = ns * cycles_per_ns; + +return 0 ; +} +/* Note: 1 QM instance per QM device, QM instance/device == event device */ static int dlb_eventdev_configure (const struct rte_eventdev *dev)
@@ -2884,6 +2897,13 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb) .port_unlink = dlb_eventdev_port_unlink, .port_unlinks_in_progress = dlb_eventdev_port_unlinks_in_progress, + .timeout_ticks = dlb_eventdev_timeout_ticks, + .dump = dlb_eventdev_dump, + .xstats_get = dlb_eventdev_xstats_get, + .xstats_get_names = dlb_eventdev_xstats_get_names, + .xstats_get_by_name = dlb_eventdev_xstats_get_by_name, + .xstats_reset = dlb_eventdev_xstats_reset, + .dev_selftest = test_dlb_eventdev, };/* Expose PMD's eventdev interface */
Signature · 1 line
1.7.10
PATCH 25/27 event/dlb: add enqueue and its burst variants
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
drivers/event/dlb/dlb.c +20 -0
@@ -1798,6 +1798,14 @@ int dlb_string_to_int(int *result, const char *str) }static uint16_t +dlb_event_enqueue_new_burst (void *event_port, +const struct rte_event events[], +uint16_t num) +{ +return __dlb_event_enqueue_burst (event_port, events, num, false); +} + +static uint16_t dlb_event_enqueue_new_burst_delayed (void *event_port,const struct rte_event events[],uint16_t num)
@@ -1806,6 +1814,14 @@ int dlb_string_to_int(int *result, const char *str) }static uint16_t +dlb_event_enqueue_forward_burst (void *event_port, +const struct rte_event events[], +uint16_t num) +{ +return __dlb_event_enqueue_burst (event_port, events, num, false); +} + +static uint16_t dlb_event_enqueue_forward_burst_delayed (void *event_port,const struct rte_event events[],uint16_t num)
@@ -2908,6 +2924,10 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb)/* Expose PMD's eventdev interface */ dev->dev_ops = &dlb_eventdev_entry_ops; + dev->enqueue = dlb_event_enqueue; + dev->enqueue_burst = dlb_event_enqueue_burst; + dev->enqueue_new_burst = dlb_event_enqueue_new_burst; + dev->enqueue_forward_burst = dlb_event_enqueue_forward_burst; }
Signature · 1 line
1.7.10
PATCH 26/27 event/dlb: add dequeue, dequeue_burst, and variants
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb.c | 814 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 814 insertions(+)
drivers/event/dlb/dlb.c +814 -0
@@ -818,6 +818,633 @@ int dlb_string_to_int(int *result, const char *str)dlb_pp_write (qm_port->qe4, port_data); } +static __rte_always_inline int +dlb_recv_qe (struct dlb_port *qm_port,struct dlb_dequeue_qe *qe, +uint8_t *offset) +{ +uint8_t xor_mask[2 ][4 ] = { {0x0F ,0x0E ,0x0C ,0x08 }, + {0x00 ,0x01 ,0x03 ,0x07 } }; +uint8_t and_mask[4 ] = {0x0F ,0x0E ,0x0C ,0x08 }; +volatile struct dlb_dequeue_qe *cq_addr; +__m128i *qes = (__m128i *)qe; +uint64_t *cache_line_base; +uint8_t gen_bits; + + cq_addr = dlb_port[qm_port->id][PORT_TYPE (qm_port)].cq_base; + cq_addr = &cq_addr[qm_port->cq_idx]; + + cache_line_base = (void *)(((uintptr_t )cq_addr) & ~0x3F ); + *offset = ((uintptr_t )cq_addr &0x30 ) >>4 ; + +/* Load the next CQ cache line from memory. Pack these reads as tight +* as possible to reduce the chance that DLB invalidates the line while +* the CPU is reading it. Read the cache line backwards to ensure that +* if QE[N] (N > 0) is valid, then QEs[0:N-1] are too. +* +* (Valid QEs start at &qe[offset]) +*/ + qes[3 ] =_mm_load_si128 ((__m128i *)&cache_line_base[6 ]); + qes[2 ] =_mm_load_si128 ((__m128i *)&cache_line_base[4 ]); + qes[1 ] =_mm_load_si128 ((__m128i *)&cache_line_base[2 ]); + qes[0 ] =_mm_load_si128 ((__m128i *)&cache_line_base[0 ]); + +/* Evict the cache line ASAP */ +dlb_cldemote (cache_line_base); + +/* Extract and combine the gen bits */ + gen_bits = ((_mm_extract_epi8 (qes[0 ],15 ) &0x1 ) <<0 ) | + ((_mm_extract_epi8 (qes[1 ],15 ) &0x1 ) <<1 ) | + ((_mm_extract_epi8 (qes[2 ],15 ) &0x1 ) <<2 ) | + ((_mm_extract_epi8 (qes[3 ],15 ) &0x1 ) <<3 ); + +/* XOR the combined bits such that a 1 represents a valid QE */ + gen_bits ^= xor_mask[qm_port->gen_bit][*offset]; + +/* Mask off gen bits we don't care about */ + gen_bits &= and_mask[*offset]; + +return __builtin_popcount (gen_bits); +} + +static __rte_always_inline int +dlb_recv_qe_sparse (struct dlb_port *qm_port,struct dlb_dequeue_qe *qe) +{ +volatile struct dlb_dequeue_qe *cq_addr; +uint8_t xor_mask[2 ] = {0x0F ,0x00 }; +const uint8_t and_mask =0x0F ; +__m128i *qes = (__m128i *)qe; +uint8_t gen_bits, gen_bit; +uintptr_t addr[4 ]; +uint16_t idx; + + cq_addr = dlb_port[qm_port->id][PORT_TYPE (qm_port)].cq_base; + + idx = qm_port->cq_idx; + +/* Load the next 4 QEs */ + addr[0 ] = (uintptr_t )&cq_addr[idx]; + addr[1 ] = (uintptr_t )&cq_addr[(idx +4 ) & qm_port->cq_depth_mask]; + addr[2 ] = (uintptr_t )&cq_addr[(idx +8 ) & qm_port->cq_depth_mask]; + addr[3 ] = (uintptr_t )&cq_addr[(idx +12 ) & qm_port->cq_depth_mask]; + +/* Prefetch next batch of QEs (all CQs occupy minimum 8 cache lines) */ +rte_prefetch0 (&cq_addr[(idx +16 ) & qm_port->cq_depth_mask]); +rte_prefetch0 (&cq_addr[(idx +20 ) & qm_port->cq_depth_mask]); +rte_prefetch0 (&cq_addr[(idx +24 ) & qm_port->cq_depth_mask]); +rte_prefetch0 (&cq_addr[(idx +28 ) & qm_port->cq_depth_mask]); + +/* Correct the xor_mask for wrap-around QEs */ + gen_bit = qm_port->gen_bit; + xor_mask[gen_bit] ^= !!((idx +4 ) > qm_port->cq_depth_mask) <<1 ; + xor_mask[gen_bit] ^= !!((idx +8 ) > qm_port->cq_depth_mask) <<2 ; + xor_mask[gen_bit] ^= !!((idx +12 ) > qm_port->cq_depth_mask) <<3 ; + +/* Read the cache lines backwards to ensure that if QE[N] (N > 0) is +* valid, then QEs[0:N-1] are too. +*/ + qes[3 ] =_mm_load_si128 ((__m128i *)(void *)addr[3 ]); +rte_compiler_barrier (); + qes[2 ] =_mm_load_si128 ((__m128i *)(void *)addr[2 ]); +rte_compiler_barrier (); + qes[1 ] =_mm_load_si128 ((__m128i *)(void *)addr[1 ]); +rte_compiler_barrier (); + qes[0 ] =_mm_load_si128 ((__m128i *)(void *)addr[0 ]); + +/* Extract and combine the gen bits */ + gen_bits = ((_mm_extract_epi8 (qes[0 ],15 ) &0x1 ) <<0 ) | + ((_mm_extract_epi8 (qes[1 ],15 ) &0x1 ) <<1 ) | + ((_mm_extract_epi8 (qes[2 ],15 ) &0x1 ) <<2 ) | + ((_mm_extract_epi8 (qes[3 ],15 ) &0x1 ) <<3 ); + +/* XOR the combined bits such that a 1 represents a valid QE */ + gen_bits ^= xor_mask[gen_bit]; + +/* Mask off gen bits we don't care about */ + gen_bits &= and_mask; + +return __builtin_popcount (gen_bits); +} + +static inline void +dlb_inc_cq_idx (struct dlb_port *qm_port,int cnt) +{ +uint16_t idx = qm_port->cq_idx_unmasked + cnt; + + qm_port->cq_idx_unmasked = idx; + qm_port->cq_idx = idx & qm_port->cq_depth_mask; + qm_port->gen_bit = (~(idx >> qm_port->gen_bit_shift)) &0x1 ; +} + +static inline bool +dlb_cq_is_empty (struct dlb_port *qm_port) +{ +volatile struct dlb_dequeue_qe *qe_ptr; +struct dlb_dequeue_qe qe; + + qe_ptr = dlb_port[qm_port->id][PORT_TYPE (qm_port)].cq_base; + qe = qe_ptr[qm_port->cq_idx]; + +return (qe.cq_gen != qm_port->gen_bit); +} + +static inline void dlb_send_rsvd_token_int_arm (struct dlb_port *qm_port) +{ +struct dlb_cq_pop_qe *qe = qm_port->consume_qe; +struct process_local_port_data *port_data; + + qe->tokens =0 ; + +rte_memcpy (&qm_port->qe4[0 ], qe,sizeof (struct dlb_cq_pop_qe )); + +/* Set int_arm to true in qm_port->qe4[0] so consume_qe isn't +* affected. +*/ + qm_port->qe4[0 ].int_arm =1 ; + +/* Fill remaining 3 entries (1,2, and 3) with noop QEs */ + +memset (&qm_port->qe4[1 ], +0 , + (DLB_NUM_QES_PER_CACHE_LINE -1 ) * +sizeof (struct dlb_cq_pop_qe )); + +/* No store fence needed since no pointer is being sent, and CQ token +* pops can be safely reordered with other HCWs. +*/ + port_data = &dlb_port[qm_port->id][PORT_TYPE (qm_port)]; + +dlb_pp_write (qm_port->qe4, port_data); + +DLB_LOG_DBG ("dlb: send rsvd token int arm\n" ); +} + +static inline void dlb_issue_int_arm_hcw (struct dlb_port *qm_port) +{ +dlb_send_rsvd_token_int_arm (qm_port); + + qm_port->cq_rsvd_token_deficit =1 ; + qm_port->int_armed = true; + +/* The delayed token return needs to account for this token pop HCW */ + qm_port->issued_releases--; +} + +static inline int +dlb_process_dequeue_qes (struct dlb_eventdev_port *ev_port, +struct dlb_port *qm_port, +struct rte_event *events, +struct dlb_dequeue_qe *qes, +int cnt) +{ +uint8_t *qid_mappings = qm_port->qid_mappings; +int i, num; + +RTE_SET_USED (ev_port);/* avoids unused variable error */ + +for (i =0 , num =0 ; i < cnt; i++) { +struct dlb_dequeue_qe *qe = &qes[i]; +int sched_type_map[4 ] = { + [DLB_SCHED_ATOMIC ] =RTE_SCHED_TYPE_ATOMIC , + [DLB_SCHED_UNORDERED ] =RTE_SCHED_TYPE_PARALLEL , + [DLB_SCHED_ORDERED ] =RTE_SCHED_TYPE_ORDERED , + [DLB_SCHED_DIRECTED ] =RTE_SCHED_TYPE_ATOMIC , + }; + +DLB_LOG_DBG ("dequeue success, data = 0x%llx, qid=%d, event_type=%d, subevent=%d\npp_id = %d, sched_type = %d, qid = %d, err=%d\n" , + (long long )qe->data, qe->qid, + qe->u.event_type.major, + qe->u.event_type.sub, + qe->pp_id, qe->sched_type, qe->qid, qe->error); + +/* Fill in event information. +* Note that flow_id must be embedded in the data by +* the app, such as the mbuf RSS hash field if the data +* buffer is a mbuf. +*/ +if (unlikely (qe->error)) { +DLB_LOG_ERR ("QE error bit ON\n" ); +DLB_INC_STAT (ev_port->stats.traffic.rx_drop,1 ); +dlb_consume_qe_immediate (qm_port,1 ); +continue ;/* Ignore */ + } + + events[num].u64 = qe->data; + events[num].queue_id = qid_mappings[qe->qid]; + events[num].priority =DLB_TO_EV_PRIO ((uint8_t )qe->priority); + events[num].event_type = qe->u.event_type.major; + events[num].sub_event_type = qe->u.event_type.sub; + events[num].sched_type = sched_type_map[qe->sched_type]; +DLB_INC_STAT (ev_port->stats.rx_sched_cnt[qe->sched_type],1 ); + +DLB_INC_STAT (ev_port->stats.traffic.rx_ok,1 ); + + num++; + } + +return num; +} + +static inline int +dlb_process_dequeue_four_qes (struct dlb_eventdev_port *ev_port, +struct dlb_port *qm_port, +struct rte_event *events, +struct dlb_dequeue_qe *qes) +{ +int sched_type_map[] = { + [DLB_SCHED_ATOMIC ] =RTE_SCHED_TYPE_ATOMIC , + [DLB_SCHED_UNORDERED ] =RTE_SCHED_TYPE_PARALLEL , + [DLB_SCHED_ORDERED ] =RTE_SCHED_TYPE_ORDERED , + [DLB_SCHED_DIRECTED ] =RTE_SCHED_TYPE_ATOMIC , + }; +const int num_events =DLB_NUM_QES_PER_CACHE_LINE ; +uint8_t *qid_mappings = qm_port->qid_mappings; +__m128i sse_evt[2 ]; +int i; + +RTE_SET_USED (ev_port); + +/* In the unlikely case that any of the QE error bits are set, process +* them one at a time. +*/ +if (unlikely (qes[0 ].error || qes[1 ].error || + qes[2 ].error || qes[3 ].error)) +return dlb_process_dequeue_qes (ev_port, qm_port, events, + qes, num_events); + +for (i =0 ; i <DLB_NUM_QES_PER_CACHE_LINE ; i++) { +DLB_LOG_DBG ("dequeue success, data = 0x%llx, qid=%d, event_type=%d, subevent=%d\npp_id = %d, sched_type = %d, qid = %d, err=%d\n" , + (long long )qes[i].data, qes[i].qid, + qes[i].u.event_type.major, + qes[i].u.event_type.sub, + qes[i].pp_id, qes[i].sched_type, qes[i].qid, + qes[i].error); + } + + events[0 ].u64 = qes[0 ].data; + events[1 ].u64 = qes[1 ].data; + events[2 ].u64 = qes[2 ].data; + events[3 ].u64 = qes[3 ].data; + +/* Construct the metadata portion of two struct rte_events +* in one 128b SSE register. Event metadata is constructed in the SSE +* registers like so: +* sse_evt[0][63:0]: event[0]'s metadata +* sse_evt[0][127:64]: event[1]'s metadata +* sse_evt[1][63:0]: event[2]'s metadata +* sse_evt[1][127:64]: event[3]'s metadata +*/ + sse_evt[0 ] =_mm_setzero_si128 (); + sse_evt[1 ] =_mm_setzero_si128 (); + +/* Convert the hardware queue ID to an event queue ID and store it in +* the metadata: +* sse_evt[0][47:40] = qid_mappings[qes[0].qid] +* sse_evt[0][111:104] = qid_mappings[qes[1].qid] +* sse_evt[1][47:40] = qid_mappings[qes[2].qid] +* sse_evt[1][111:104] = qid_mappings[qes[3].qid] +*/ +#define RTE_EVENT_QUEUE_ID_BYTE 5 + sse_evt[0 ] =_mm_insert_epi8 (sse_evt[0 ], + qid_mappings[qes[0 ].qid], +RTE_EVENT_QUEUE_ID_BYTE ); + sse_evt[0 ] =_mm_insert_epi8 (sse_evt[0 ], + qid_mappings[qes[1 ].qid], +RTE_EVENT_QUEUE_ID_BYTE +8 ); + sse_evt[1 ] =_mm_insert_epi8 (sse_evt[1 ], + qid_mappings[qes[2 ].qid], +RTE_EVENT_QUEUE_ID_BYTE ); + sse_evt[1 ] =_mm_insert_epi8 (sse_evt[1 ], + qid_mappings[qes[3 ].qid], +RTE_EVENT_QUEUE_ID_BYTE +8 ); + +/* Convert the hardware priority to an event priority and store it in +* the metadata: +* sse_evt[0][55:48] = DLB_TO_EV_PRIO(qes[0].priority) +* sse_evt[0][119:112] = DLB_TO_EV_PRIO(qes[1].priority) +* sse_evt[1][55:48] = DLB_TO_EV_PRIO(qes[2].priority) +* sse_evt[1][119:112] = DLB_TO_EV_PRIO(qes[3].priority) +*/ +#define RTE_EVENT_PRIO_BYTE 6 + sse_evt[0 ] =_mm_insert_epi8 (sse_evt[0 ], +DLB_TO_EV_PRIO ((uint8_t )qes[0 ].priority), +RTE_EVENT_PRIO_BYTE ); + sse_evt[0 ] =_mm_insert_epi8 (sse_evt[0 ], +DLB_TO_EV_PRIO ((uint8_t )qes[1 ].priority), +RTE_EVENT_PRIO_BYTE +8 ); + sse_evt[1 ] =_mm_insert_epi8 (sse_evt[1 ], +DLB_TO_EV_PRIO ((uint8_t )qes[2 ].priority), +RTE_EVENT_PRIO_BYTE ); + sse_evt[1 ] =_mm_insert_epi8 (sse_evt[1 ], +DLB_TO_EV_PRIO ((uint8_t )qes[3 ].priority), +RTE_EVENT_PRIO_BYTE +8 ); + +/* Write the event type and sub event type to the event metadata. Leave +* flow ID unspecified, since the hardware does not maintain it during +* scheduling: +* sse_evt[0][31:0] = qes[0].u.event_type.major << 28 | +* qes[0].u.event_type.sub << 20; +* sse_evt[0][95:64] = qes[1].u.event_type.major << 28 | +* qes[1].u.event_type.sub << 20; +* sse_evt[1][31:0] = qes[2].u.event_type.major << 28 | +* qes[2].u.event_type.sub << 20; +* sse_evt[1][95:64] = qes[3].u.event_type.major << 28 | +* qes[3].u.event_type.sub << 20; +*/ +#define RTE_EVENT_EV_TYPE_DW 0 +#define RTE_EVENT_EV_TYPE_SHIFT 28 +#define RTE_EVENT_SUB_EV_TYPE_SHIFT 20 + sse_evt[0 ] =_mm_insert_epi32 (sse_evt[0 ], + qes[0 ].u.event_type.major <<RTE_EVENT_EV_TYPE_SHIFT | + qes[0 ].u.event_type.sub <<RTE_EVENT_SUB_EV_TYPE_SHIFT , +RTE_EVENT_EV_TYPE_DW ); + sse_evt[0 ] =_mm_insert_epi32 (sse_evt[0 ], + qes[1 ].u.event_type.major <<RTE_EVENT_EV_TYPE_SHIFT | + qes[1 ].u.event_type.sub <<RTE_EVENT_SUB_EV_TYPE_SHIFT , +RTE_EVENT_EV_TYPE_DW +2 ); + sse_evt[1 ] =_mm_insert_epi32 (sse_evt[1 ], + qes[2 ].u.event_type.major <<RTE_EVENT_EV_TYPE_SHIFT | + qes[2 ].u.event_type.sub <<RTE_EVENT_SUB_EV_TYPE_SHIFT , +RTE_EVENT_EV_TYPE_DW ); + sse_evt[1 ] =_mm_insert_epi32 (sse_evt[1 ], + qes[3 ].u.event_type.major <<RTE_EVENT_EV_TYPE_SHIFT | + qes[3 ].u.event_type.sub <<RTE_EVENT_SUB_EV_TYPE_SHIFT , +RTE_EVENT_EV_TYPE_DW +2 ); + +/* Write the sched type to the event metadata. 'op' and 'rsvd' are not +* set: +* sse_evt[0][39:32] = sched_type_map[qes[0].sched_type] << 6 +* sse_evt[0][103:96] = sched_type_map[qes[1].sched_type] << 6 +* sse_evt[1][39:32] = sched_type_map[qes[2].sched_type] << 6 +* sse_evt[1][103:96] = sched_type_map[qes[3].sched_type] << 6 +*/ +#define RTE_EVENT_SCHED_TYPE_BYTE 4 +#define RTE_EVENT_SCHED_TYPE_SHIFT 6 + sse_evt[0 ] =_mm_insert_epi8 (sse_evt[0 ], + sched_type_map[qes[0 ].sched_type] <<RTE_EVENT_SCHED_TYPE_SHIFT , +RTE_EVENT_SCHED_TYPE_BYTE ); + sse_evt[0 ] =_mm_insert_epi8 (sse_evt[0 ], + sched_type_map[qes[1 ].sched_type] <<RTE_EVENT_SCHED_TYPE_SHIFT , +RTE_EVENT_SCHED_TYPE_BYTE +8 ); + sse_evt[1 ] =_mm_insert_epi8 (sse_evt[1 ], + sched_type_map[qes[2 ].sched_type] <<RTE_EVENT_SCHED_TYPE_SHIFT , +RTE_EVENT_SCHED_TYPE_BYTE ); + sse_evt[1 ] =_mm_insert_epi8 (sse_evt[1 ], + sched_type_map[qes[3 ].sched_type] <<RTE_EVENT_SCHED_TYPE_SHIFT , +RTE_EVENT_SCHED_TYPE_BYTE +8 ); + +/* Store the metadata to the event (use the double-precision +* _mm_storeh_pd because there is no integer function for storing the +* upper 64b): +* events[0].event = sse_evt[0][63:0] +* events[1].event = sse_evt[0][127:64] +* events[2].event = sse_evt[1][63:0] +* events[3].event = sse_evt[1][127:64] +*/ +_mm_storel_epi64 ((__m128i *)&events[0 ].event, sse_evt[0 ]); +_mm_storeh_pd ((double *)&events[1 ].event, (__m128d ) sse_evt[0 ]); +_mm_storel_epi64 ((__m128i *)&events[2 ].event, sse_evt[1 ]); +_mm_storeh_pd ((double *)&events[3 ].event, (__m128d ) sse_evt[1 ]); + +DLB_INC_STAT (ev_port->stats.rx_sched_cnt[qes[0 ].sched_type],1 ); +DLB_INC_STAT (ev_port->stats.rx_sched_cnt[qes[1 ].sched_type],1 ); +DLB_INC_STAT (ev_port->stats.rx_sched_cnt[qes[2 ].sched_type],1 ); +DLB_INC_STAT (ev_port->stats.rx_sched_cnt[qes[3 ].sched_type],1 ); + +DLB_INC_STAT (ev_port->stats.traffic.rx_ok, num_events); + +return num_events; +} + +static inline int +dlb_dequeue_wait (struct dlb_eventdev *dlb, +struct dlb_eventdev_port *ev_port, +struct dlb_port *qm_port, +uint64_t timeout, +uint64_t start_ticks) +{ +struct process_local_port_data *port_data; +uint64_t elapsed_ticks; + + port_data = &dlb_port[qm_port->id][PORT_TYPE (qm_port)]; + + elapsed_ticks =rte_get_timer_cycles () - start_ticks; + +/* Wait/poll time expired */ +if (elapsed_ticks >= timeout) { +/* Interrupts not supported by PF PMD */ +return 1 ; + }else if (dlb->umwait_allowed) { +volatile struct dlb_dequeue_qe *cq_base; + + cq_base = port_data->cq_base; + +/* Block on cache line write to CQ. Note: it's +* safe to access the per-process cq_base +* address here, since the PMD has already +* attempted at least one CQ dequeue. +*/ +dlb_umonitor (&cq_base[qm_port->cq_idx]); + +/* Avoid race condition. Check if still empty */ +if (dlb_cq_is_empty (qm_port)) { +dlb_umwait (RTE_LIBRTE_PMD_DLB_UMWAIT_CTL_STATE , + timeout + start_ticks); +DLB_INC_STAT (ev_port->stats.traffic.rx_umonitor_umwait, +1 ); + } + }else { +uint64_t poll_interval =RTE_LIBRTE_PMD_DLB_POLL_INTERVAL ; +uint64_t curr_ticks =rte_get_timer_cycles (); +uint64_t init_ticks = curr_ticks; + +while ((curr_ticks - start_ticks < timeout) && + (curr_ticks - init_ticks < poll_interval)) + curr_ticks =rte_get_timer_cycles (); + } + +return 0 ; +} + +static inline int16_t +dlb_hw_dequeue (struct dlb_eventdev *dlb, +struct dlb_eventdev_port *ev_port, +struct rte_event *events, +uint16_t max_num, +uint64_t dequeue_timeout_ticks) +{ +uint64_t timeout; +uint64_t start_ticks =0ULL ; +struct dlb_port *qm_port; +int num =0 ; + + qm_port = &ev_port->qm_port; + +/* If configured for per dequeue wait, then use wait value provided +* to this API. Otherwise we must use the global +* value from eventdev config time. +*/ +if (!dlb->global_dequeue_wait) + timeout = dequeue_timeout_ticks; +else + timeout = dlb->global_dequeue_wait_ticks; + +if (timeout) + start_ticks =rte_get_timer_cycles (); + +while (num < max_num) { +struct dlb_dequeue_qe qes[DLB_NUM_QES_PER_CACHE_LINE ]; +uint8_t offset; +int num_avail; + +/* Copy up to 4 QEs from the current cache line into qes */ + num_avail =dlb_recv_qe (qm_port, qes, &offset); + +/* But don't process more than the user requested */ + num_avail =RTE_MIN (num_avail, max_num - num); + +dlb_inc_cq_idx (qm_port, num_avail); + +if (num_avail ==DLB_NUM_QES_PER_CACHE_LINE ) + num +=dlb_process_dequeue_four_qes (ev_port, + qm_port, + &events[num], + &qes[offset]); +else if (num_avail) + num +=dlb_process_dequeue_qes (ev_port, + qm_port, + &events[num], + &qes[offset], + num_avail); +else if ((timeout ==0 ) || (num >0 )) +/* Not waiting in any form, or 1+ events received? */ +break ; +else if (dlb_dequeue_wait (dlb, ev_port, qm_port, + timeout, start_ticks)) +break ; + } + + qm_port->owed_tokens += num; + +if (num && qm_port->token_pop_mode ==AUTO_POP ) +dlb_consume_qe_immediate (qm_port, num); + + ev_port->outstanding_releases += num; + +return num; +} + +static inline int +dlb_process_dequeue_qe(struct dlb_eventdev_port *ev_port __rte_unused, +struct dlb_port *qm_port, +struct rte_event *event, +struct dlb_dequeue_qe *qe) +{ +int sched_type_map[4 ] = { + [DLB_SCHED_ATOMIC ] =RTE_SCHED_TYPE_ATOMIC , + [DLB_SCHED_UNORDERED ] =RTE_SCHED_TYPE_PARALLEL , + [DLB_SCHED_ORDERED ] =RTE_SCHED_TYPE_ORDERED , + [DLB_SCHED_DIRECTED ] =RTE_SCHED_TYPE_ATOMIC , + }; +uint8_t *qid_mappings = qm_port->qid_mappings; + +DLB_LOG_DBG ("dequeue success, data = 0x%llx, qid=%d, event_type=%d, subevent=%d\npp_id = %d, sched_type = %d, qid = %d, err=%d\n" , + (long long )qe->data, qe->qid, + qe->u.event_type.major, + qe->u.event_type.sub, + qe->pp_id, qe->sched_type, qe->qid, qe->error); + +/* Fill in event information. +* Note that flow_id must be embedded in the data by +* the app, such as the mbuf RSS hash field if the data +* buffer is a mbuf. +*/ +if (unlikely (qe->error)) { +DLB_LOG_ERR ("QE error bit ON\n" ); +DLB_INC_STAT (ev_port->stats.traffic.rx_drop,1 ); +dlb_consume_qe_immediate (qm_port,1 ); +return 0 ; + } + + event->u64 = qe->data; + event->queue_id = qid_mappings[qe->qid]; + event->priority =DLB_TO_EV_PRIO ((uint8_t )qe->priority); + event->event_type = qe->u.event_type.major; + event->sub_event_type = qe->u.event_type.sub; + event->sched_type = sched_type_map[qe->sched_type]; +DLB_INC_STAT (ev_port->stats.rx_sched_cnt[qe->sched_type],1 ); + +DLB_INC_STAT (ev_port->stats.traffic.rx_ok,1 ); + +return 1 ; +} + +static inline int16_t +dlb_hw_dequeue_sparse (struct dlb_eventdev *dlb, +struct dlb_eventdev_port *ev_port, +struct rte_event *events, +uint16_t max_num, +uint64_t dequeue_timeout_ticks) +{ +uint64_t timeout; +uint64_t start_ticks =0ULL ; +struct dlb_port *qm_port; +int num =0 ; + + qm_port = &ev_port->qm_port; + +/* If configured for per dequeue wait, then use wait value provided +* to this API. Otherwise we must use the global +* value from eventdev config time. +*/ +if (!dlb->global_dequeue_wait) + timeout = dequeue_timeout_ticks; +else + timeout = dlb->global_dequeue_wait_ticks; + +if (timeout) + start_ticks =rte_get_timer_cycles (); + +while (num < max_num) { +struct dlb_dequeue_qe qes[DLB_NUM_QES_PER_CACHE_LINE ]; +int num_avail; + +/* Copy up to 4 QEs from the current cache line into qes */ + num_avail =dlb_recv_qe_sparse (qm_port, qes); + +/* But don't process more than the user requested */ + num_avail =RTE_MIN (num_avail, max_num - num); + +dlb_inc_cq_idx (qm_port, num_avail <<2 ); + +if (num_avail ==DLB_NUM_QES_PER_CACHE_LINE ) + num +=dlb_process_dequeue_four_qes (ev_port, + qm_port, + &events[num], + &qes[0 ]); +else if (num_avail) + num +=dlb_process_dequeue_qes (ev_port, + qm_port, + &events[num], + &qes[0 ], + num_avail); +else if ((timeout ==0 ) || (num >0 )) +/* Not waiting in any form, or 1+ events received? */ +break ; +else if (dlb_dequeue_wait (dlb, ev_port, qm_port, + timeout, start_ticks)) +break ; + } + + qm_port->owed_tokens += num; + +if (num && qm_port->token_pop_mode ==AUTO_POP ) +dlb_consume_qe_immediate (qm_port, num); + + ev_port->outstanding_releases += num; + +return num; +} + static int16_t dlb_hw_map_ldb_qid_to_port(struct dlb_hw_dev *handle, uint32_t qm_port_id,
@@ -1956,6 +2583,183 @@ int dlb_string_to_int(int *result, const char *str)return 0 ; } +static void +dlb_event_release (struct dlb_eventdev *dlb, +uint8_t port_id, +int n) +{ +struct process_local_port_data *port_data; +struct dlb_eventdev_port *ev_port; +struct dlb_port *qm_port; +int i, cnt; + +if (port_id > dlb->num_ports) { +DLB_LOG_ERR ("Invalid port id %d in dlb-event_release\n" , + port_id); + rte_errno = -EINVAL ; +return ; + } + + ev_port = &dlb->ev_ports[port_id]; + qm_port = &ev_port->qm_port; + port_data = &dlb_port[qm_port->id][PORT_TYPE (qm_port)]; + + cnt =0 ; + +if (qm_port->is_directed) { + cnt = n; + goto sw_credit_update; + } + +/* Since MOVDIR64B is weakly-ordered, use an SFENCE to ensure that +* application writes complete before enqueueing the release HCW. +*/ +rte_wmb (); + +for (i =0 ; i < n; i +=DLB_NUM_QES_PER_CACHE_LINE ) { +int j; + +/* Zero-out QEs */ + qm_port->qe4[0 ].cmd_byte =0 ; + qm_port->qe4[1 ].cmd_byte =0 ; + qm_port->qe4[2 ].cmd_byte =0 ; + qm_port->qe4[3 ].cmd_byte =0 ; + +for (j =0 ; j <DLB_NUM_QES_PER_CACHE_LINE && (i + j) < n; j++) + qm_port->qe4[j].cmd_byte =DLB_COMP_CMD_BYTE ; + + qm_port->issued_releases += j; + +if (j ==0 ) +break ; + +if (qm_port->token_pop_mode ==DELAYED_POP && j <4 && + qm_port->issued_releases >= qm_port->token_pop_thresh -1 ) { +dlb_construct_token_pop_qe (qm_port, j); + +/* Reset the releases counter for the next QE batch */ + qm_port->issued_releases -= qm_port->token_pop_thresh; + +/* When using delayed token pop mode, the initial token +* threshold is the full CQ depth. After the first +* token pop, we need to reset it to the dequeue_depth. +*/ + qm_port->token_pop_thresh = qm_port->dequeue_depth; + } + +dlb_hw_do_enqueue (qm_port, port_data); + + cnt += j; + } + +if (qm_port->token_pop_mode ==DELAYED_POP && + qm_port->issued_releases >= qm_port->token_pop_thresh -1 ) { +dlb_consume_qe_immediate (qm_port, qm_port->owed_tokens); + qm_port->issued_releases -= qm_port->token_pop_thresh; + qm_port->token_pop_thresh = qm_port->dequeue_depth; + } + +sw_credit_update: +/* each release returns one credit */ +if (!ev_port->outstanding_releases) +rte_panic ("%s: Outstanding releases underflowed.\n" , __func__); + ev_port->outstanding_releases -= cnt; + ev_port->inflight_credits += cnt; + +/* Replenish s/w credits if enough releases are performed */ +dlb_replenish_sw_credits (dlb, ev_port); +} + +static uint16_t +dlb_event_dequeue_burst (void *event_port,struct rte_event *ev,uint16_t num, +uint64_t wait) +{ +struct dlb_eventdev_port *ev_port = event_port; +struct dlb_port *qm_port = &ev_port->qm_port; +struct dlb_eventdev *dlb = ev_port->dlb; +struct process_local_port_data *port_data; +uint16_t cnt; + + rte_errno =0 ; + +RTE_ASSERT (ev_port->setup_done); +RTE_ASSERT (ev !=NULL ); + + port_data = &dlb_port[qm_port->id][PORT_TYPE (qm_port)]; + +if (!port_data->mmaped) +dlb_iface_port_mmap (qm_port); + +if (ev_port->implicit_release && ev_port->outstanding_releases >0 ) { +uint16_t out_rels = ev_port->outstanding_releases; + +dlb_event_release (dlb, ev_port->id, out_rels); + +DLB_INC_STAT (ev_port->stats.tx_implicit_rel, out_rels); + } + +if (qm_port->token_pop_mode ==DEFERRED_POP && + qm_port->owed_tokens) +dlb_consume_qe_immediate (qm_port, qm_port->owed_tokens); + + cnt =dlb_hw_dequeue (dlb, ev_port, ev, num, wait); + +DLB_INC_STAT (ev_port->stats.traffic.total_polls,1 ); +DLB_INC_STAT (ev_port->stats.traffic.zero_polls, ((cnt ==0 ) ?1 :0 )); +return cnt; +} + +static uint16_t +dlb_event_dequeue (void *event_port,struct rte_event *ev,uint64_t wait) +{ +return dlb_event_dequeue_burst (event_port, ev,1 , wait); +} + +static uint16_t +dlb_event_dequeue_burst_sparse (void *event_port,struct rte_event *ev, +uint16_t num,uint64_t wait) +{ +struct dlb_eventdev_port *ev_port = event_port; +struct dlb_port *qm_port = &ev_port->qm_port; +struct dlb_eventdev *dlb = ev_port->dlb; +struct process_local_port_data *port_data; +uint16_t cnt; + + rte_errno =0 ; + +RTE_ASSERT (ev_port->setup_done); +RTE_ASSERT (ev !=NULL ); + + port_data = &dlb_port[qm_port->id][PORT_TYPE (qm_port)]; + +if (!port_data->mmaped) +dlb_iface_port_mmap (qm_port); + +if (ev_port->implicit_release && ev_port->outstanding_releases >0 ) { +uint16_t out_rels = ev_port->outstanding_releases; + +dlb_event_release (dlb, ev_port->id, out_rels); + +DLB_INC_STAT (ev_port->stats.tx_implicit_rel, out_rels); + } + +if (qm_port->token_pop_mode ==DEFERRED_POP && + qm_port->owed_tokens) +dlb_consume_qe_immediate (qm_port, qm_port->owed_tokens); + + cnt =dlb_hw_dequeue_sparse (dlb, ev_port, ev, num, wait); + +DLB_INC_STAT (ev_port->stats.traffic.total_polls,1 ); +DLB_INC_STAT (ev_port->stats.traffic.zero_polls, ((cnt ==0 ) ?1 :0 )); +return cnt; +} + +static uint16_t +dlb_event_dequeue_sparse (void *event_port,struct rte_event *ev,uint64_t wait) +{ +return dlb_event_dequeue_burst_sparse (event_port, ev,1 , wait); +} + static int dlb_event_queue_join_ldb(struct dlb_eventdev *dlb, struct dlb_eventdev_port *ev_port,
@@ -2899,6 +3703,8 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb)void dlb_entry_points_init (struct rte_eventdev *dev) { +struct dlb_eventdev *dlb; +static struct rte_eventdev_ops dlb_eventdev_entry_ops = { .dev_infos_get = dlb_eventdev_info_get, .dev_configure = dlb_eventdev_configure,
@@ -2928,7 +3734,15 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb) dev->enqueue_burst = dlb_event_enqueue_burst; dev->enqueue_new_burst = dlb_event_enqueue_new_burst; dev->enqueue_forward_burst = dlb_event_enqueue_forward_burst; + dev->dequeue = dlb_event_dequeue; + dev->dequeue_burst = dlb_event_dequeue_burst; + + dlb = dev->data->dev_private; +if (dlb->poll_mode ==DLB_CQ_POLL_MODE_SPARSE ) { + dev->dequeue = dlb_event_dequeue_sparse; + dev->dequeue_burst = dlb_event_dequeue_burst_sparse; + } } static void
Signature · 1 line
1.7.10
PATCH 27/27 event/dlb: add eventdev_stop and eventdev_close
In reply to Tim McDanielFrom: "McDaniel, Timothy" <[email protected]>
--- drivers/event/dlb/dlb.c | 263 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 263 insertions(+)
drivers/event/dlb/dlb.c +263 -0
@@ -303,6 +303,23 @@ int dlb_string_to_int(int *result, const char *str)const struct rte_event events[], uint16_t num); +void +dlb_free_qe_mem (struct dlb_port *qm_port) +{ +if (qm_port ==NULL ) +return ; + +if (qm_port->qe4) { +rte_free (qm_port->qe4); + qm_port->qe4 =NULL ; + } + +if (qm_port->consume_qe) { +rte_free (qm_port->consume_qe); + qm_port->consume_qe =NULL ; + } +} +int dlb_init_qe_mem (struct dlb_port *qm_port,char *mz_name) {
@@ -1848,6 +1865,250 @@ static inline void dlb_issue_int_arm_hcw(struct dlb_port *qm_port)return 0 ; } +static void +dlb_flush_port (struct rte_eventdev *dev,int port_id) +{ +struct dlb_eventdev *dlb =dlb_pmd_priv (dev); +eventdev_stop_flush_t flush; +struct rte_event ev; +uint8_t dev_id; +void *arg; +int i; + + flush = dev->dev_ops->dev_stop_flush; + dev_id = dev->data->dev_id; + arg = dev->data->dev_stop_flush_arg; + +while (rte_event_dequeue_burst (dev_id, port_id, &ev,1 ,0 )) { +if (flush) +flush (dev_id, ev, arg); + +if (dlb->ev_ports[port_id].qm_port.is_directed) +continue ; + + ev.op =RTE_EVENT_OP_RELEASE ; + +rte_event_enqueue_burst (dev_id, port_id, &ev,1 ); + } + +/* Enqueue any additional outstanding releases */ + ev.op =RTE_EVENT_OP_RELEASE ; + +for (i = dlb->ev_ports[port_id].outstanding_releases; i >0 ; i--) +rte_event_enqueue_burst (dev_id, port_id, &ev,1 ); +} + +static uint32_t +dlb_get_ldb_queue_depth (struct dlb_eventdev *dlb, +struct dlb_eventdev_queue *queue) +{ +struct dlb_hw_dev *handle = &dlb->qm_instance; +struct dlb_get_ldb_queue_depth_args cfg; +struct dlb_cmd_response response; +int ret; + + cfg.queue_id = queue->qm_queue.id; + cfg.response = (uintptr_t )&response; + + ret =dlb_iface_get_ldb_queue_depth (handle, &cfg); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: get_ldb_queue_depth ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); +return ret; + } + +return response.id; +} + +static uint32_t +dlb_get_dir_queue_depth (struct dlb_eventdev *dlb, +struct dlb_eventdev_queue *queue) +{ +struct dlb_hw_dev *handle = &dlb->qm_instance; +struct dlb_get_dir_queue_depth_args cfg; +struct dlb_cmd_response response; +int ret; + + cfg.queue_id = queue->qm_queue.id; + cfg.response = (uintptr_t )&response; + + ret =dlb_iface_get_dir_queue_depth (handle, &cfg); +if (ret <0 ) { +DLB_LOG_ERR ("dlb: get_dir_queue_depth ret=%d (driver status: %s)\n" , + ret, dlb_error_strings[response.status]); +return ret; + } + +return response.id; +} + +uint32_t +dlb_get_queue_depth (struct dlb_eventdev *dlb, +struct dlb_eventdev_queue *queue) +{ +if (queue->qm_queue.is_directed) +return dlb_get_dir_queue_depth (dlb, queue); +else +return dlb_get_ldb_queue_depth (dlb, queue); +} + +static bool +dlb_queue_is_empty (struct dlb_eventdev *dlb, +struct dlb_eventdev_queue *queue) +{ +return dlb_get_queue_depth (dlb, queue) ==0 ; +} + +static bool +dlb_linked_queues_empty (struct dlb_eventdev *dlb) +{ +int i; + +for (i =0 ; i < dlb->num_queues; i++) { +if (dlb->ev_queues[i].num_links ==0 ) +continue ; +if (!dlb_queue_is_empty (dlb, &dlb->ev_queues[i])) +return false; + } + +return true; +} + +static bool +dlb_queues_empty (struct dlb_eventdev *dlb) +{ +int i; + +for (i =0 ; i < dlb->num_queues; i++) { +if (!dlb_queue_is_empty (dlb, &dlb->ev_queues[i])) +return false; + } + +return true; +} + +void +dlb_drain (struct rte_eventdev *dev) +{ +struct dlb_eventdev *dlb =dlb_pmd_priv (dev); +struct dlb_eventdev_port *ev_port =NULL ; +uint8_t dev_id; +int i; + + dev_id = dev->data->dev_id; + +while (!dlb_linked_queues_empty (dlb)) { +/* Flush all the ev_ports, which will drain all their connected +* queues. +*/ +for (i =0 ; i < dlb->num_ports; i++) +dlb_flush_port (dev, i); + } + +/* The queues are empty, but there may be events left in the ports. */ +for (i =0 ; i < dlb->num_ports; i++) +dlb_flush_port (dev, i); + +/* If the domain's queues are empty, we're done. */ +if (dlb_queues_empty (dlb)) +return ; + +/* Else, there must be at least one unlinked load-balanced queue. +* Select a load-balanced port with which to drain the unlinked +* queue(s). +*/ +for (i =0 ; i < dlb->num_ports; i++) { + ev_port = &dlb->ev_ports[i]; + +if (!ev_port->qm_port.is_directed) +break ; + } + +if (i == dlb->num_ports) { +DLB_LOG_ERR ("internal error: no LDB ev_ports\n" ); +return ; + } + + rte_errno =0 ; +rte_event_port_unlink (dev_id, ev_port->id,NULL ,0 ); + +if (rte_errno) { +DLB_LOG_ERR ("internal error: failed to unlink ev_port %d\n" , + ev_port->id); +return ; + } + +for (i =0 ; i < dlb->num_queues; i++) { +uint8_t qid, prio; +int ret; + +if (dlb_queue_is_empty (dlb, &dlb->ev_queues[i])) +continue ; + + qid = i; + prio =0 ; + +/* Link the ev_port to the queue */ + ret =rte_event_port_link (dev_id, ev_port->id, &qid, &prio,1 ); +if (ret !=1 ) { +DLB_LOG_ERR ("internal error: failed to link ev_port %d to queue %d\n" , + ev_port->id, qid); +return ; + } + +/* Flush the queue */ +while (!dlb_queue_is_empty (dlb, &dlb->ev_queues[i])) +dlb_flush_port (dev, ev_port->id); + +/* Drain any extant events in the ev_port. */ +dlb_flush_port (dev, ev_port->id); + +/* Unlink the ev_port from the queue */ + ret =rte_event_port_unlink (dev_id, ev_port->id, &qid,1 ); +if (ret !=1 ) { +DLB_LOG_ERR ("internal error: failed to unlink ev_port %d to queue %d\n" , + ev_port->id, qid); +return ; + } + } +} + +static void +dlb_eventdev_stop (struct rte_eventdev *dev) +{ +struct dlb_eventdev *dlb =dlb_pmd_priv (dev); + +rte_spinlock_lock (&dlb->qm_instance.resource_lock); + +if (dlb->run_state ==DLB_RUN_STATE_STOPPED ) { +DLB_LOG_DBG ("Internal error: already stopped\n" ); +rte_spinlock_unlock (&dlb->qm_instance.resource_lock); +return ; + }else if (dlb->run_state !=DLB_RUN_STATE_STARTED ) { +DLB_LOG_ERR ("Internal error: bad state %d for dev_stop\n" , + (int )dlb->run_state); +rte_spinlock_unlock (&dlb->qm_instance.resource_lock); +return ; + } + + dlb->run_state =DLB_RUN_STATE_STOPPING ; + +rte_spinlock_unlock (&dlb->qm_instance.resource_lock); + +dlb_drain (dev); + + dlb->run_state =DLB_RUN_STATE_STOPPED ; +} + +static int +dlb_eventdev_close (struct rte_eventdev *dev) +{ +dlb_hw_reset_sched_domain (dev, false); + +return 0 ; +} + +static inline int dlb_check_enqueue_sw_credits (struct dlb_eventdev *dlb,struct dlb_eventdev_port *ev_port)
@@ -3709,6 +3970,8 @@ static int dlb_num_dir_queues_setup(struct dlb_eventdev *dlb) .dev_infos_get = dlb_eventdev_info_get, .dev_configure = dlb_eventdev_configure, .dev_start = dlb_eventdev_start, + .dev_stop = dlb_eventdev_stop, + .dev_close = dlb_eventdev_close, .queue_def_conf = dlb_eventdev_queue_default_conf_get, .queue_setup = dlb_eventdev_queue_setup, .queue_release = dlb_eventdev_queue_release,
Signature · 1 line
1.7.10