Add DLB2 PMD
cover Add DLB2 PMD
The following patch series adds support for a new eventdev PMD. The DLB2 PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2) hardware.
The DLB2 is a PCIe device that provides load-balanced, prioritized scheduling of core-to-core communication. The device consists of queues and arbiters that connect producer and consumer cores, and implements load-balanced queueing features including:
- Lock-free multi-producer/multi-consumer operation.
- Multiple priority levels for varying traffic types.
- 'Direct' traffic (i.e. multi-producer/single-consumer)
- Simple unordered load-balanced distribution.
- Atomic lock-free load balancing across multiple consumers.
- Queue element reordering feature allowing ordered load-balanced distribution.
The DLB2 hardware supports both load balanced and directed ports and queues. Unlike other eventdev devices already in the repo, not all DLB2 ports and queues are equally capable. In particular, directed ports are limited to a single link, and must be connected to a directed queue. Additionally, even though LDB ports may link multiple queues, the number of queues that may be linked is limited by hardware.
While reviewing the code, please be aware that this PMD has full control over the DLB2 hardware. Intel will be extending the DLB2 PMD in the future (not as part of this first series) with a mode that we refer to as the bifurcated PMD. The bifurcated PMD communicates with a kernel driver to configure the device, ports, and queues, and memory maps device MMIO so datapath operations occur purely in user-space. Note that the DLB2 hardware is a successor of the DLB hardware, and as such is structured similarly, both in terms of code layout and implementation.
The framework to support both the PF PMD and bifurcated PMD exists in this patchset, and is why the iface.[ch] layer is present.
Depends-on: patch-77466 ("eventdev: add PCI probe named convenience function")
Depends-on: series-12160 ("Eventdev ABI changes")
Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")
Timothy McDaniel (22):
event/dlb2: add meson build infrastructure event/dlb2: add dynamic logging
event/dlb2: add private data structures and constants event/dlb2: add definitions shared with LKM or shared code event/dlb2: add inline functions
event/dlb2: add probe
event/dlb2: add xstats
event/dlb2: add infos get and configure event/dlb2: add queue and port default conf event/dlb2: add queue setup
event/dlb2: add port setup
event/dlb2: add port link
event/dlb2: add port unlink and port unlinks in progress event/dlb2: add eventdev start
event/dlb2: add enqueue and its burst variants event/dlb2: add dequeue and its burst variants event/dlb2: add eventdev stop and close event/dlb2: add PMD's token pop public interface event/dlb2: add PMD self-tests
event/dlb2: add queue and port release event/dlb2: add timeout ticks entry point
doc: add new DLB2 eventdev driver to relnotes
app/test/test_eventdev.c | 9 + config/rte_config.h | 7 + doc/guides/rel_notes/release_20_11.rst | 5 + drivers/event/dlb2/dlb2.c | 4046 ++++++++++++++ drivers/event/dlb2/dlb2_iface.c | 88 + drivers/event/dlb2/dlb2_iface.h | 75 + drivers/event/dlb2/dlb2_inline_fns.h | 85 + drivers/event/dlb2/dlb2_log.h | 25 + drivers/event/dlb2/dlb2_priv.h | 619 +++ drivers/event/dlb2/dlb2_selftest.c | 1570 ++++++ drivers/event/dlb2/dlb2_user.h | 883 +++ drivers/event/dlb2/dlb2_xstats.c | 1269 +++++ drivers/event/dlb2/meson.build | 16 + drivers/event/dlb2/pf/base/dlb2_hw_types.h | 367 ++ drivers/event/dlb2/pf/base/dlb2_mbox.h | 596 ++ drivers/event/dlb2/pf/base/dlb2_osdep.h | 248 + drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h | 447 ++ drivers/event/dlb2/pf/base/dlb2_osdep_list.h | 131 + drivers/event/dlb2/pf/base/dlb2_osdep_types.h | 31 + drivers/event/dlb2/pf/base/dlb2_regs.h | 2527 +++++++++ drivers/event/dlb2/pf/base/dlb2_resource.c | 6023 +++++++++++++++++++++ drivers/event/dlb2/pf/base/dlb2_resource.h | 1913 +++++++ drivers/event/dlb2/pf/dlb2_main.c | 692 +++ drivers/event/dlb2/pf/dlb2_main.h | 107 + drivers/event/dlb2/pf/dlb2_pf.c | 734 +++ drivers/event/dlb2/rte_pmd_dlb2.c | 39 + drivers/event/dlb2/rte_pmd_dlb2.h | 59 + drivers/event/dlb2/rte_pmd_dlb2_event_version.map | 9 + drivers/event/meson.build | 4 + 29 files changed, 22624 insertions(+) create mode 100644 drivers/event/dlb2/dlb2.c create mode 100644 drivers/event/dlb2/dlb2_iface.c create mode 100644 drivers/event/dlb2/dlb2_iface.h create mode 100644 drivers/event/dlb2/dlb2_inline_fns.h create mode 100644 drivers/event/dlb2/dlb2_log.h create mode 100644 drivers/event/dlb2/dlb2_priv.h create mode 100644 drivers/event/dlb2/dlb2_selftest.c create mode 100644 drivers/event/dlb2/dlb2_user.h create mode 100644 drivers/event/dlb2/dlb2_xstats.c create mode 100644 drivers/event/dlb2/meson.build create mode 100644 drivers/event/dlb2/pf/base/dlb2_hw_types.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_mbox.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_list.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_types.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_regs.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.c create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.h create mode 100644 drivers/event/dlb2/pf/dlb2_main.c create mode 100644 drivers/event/dlb2/pf/dlb2_main.h create mode 100644 drivers/event/dlb2/pf/dlb2_pf.c create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.c create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.h create mode 100644 drivers/event/dlb2/rte_pmd_dlb2_event_version.map
Signature · 1 line
2.6.4
PATCH 1/22 event/dlb2: add meson build infrastructure
Adds the meson build infrastructure, which includes compile-time constants in rte_config.h. DLB2 is only supported on Linux X86 platforms at this time.
--- config/rte_config.h | 7 +++++++ drivers/event/dlb2/meson.build | 7 +++++++ drivers/event/dlb2/rte_pmd_dlb2_event_version.map | 3 +++ drivers/event/meson.build | 4 ++++ 4 files changed, 21 insertions(+) create mode 100644 drivers/event/dlb2/meson.build create mode 100644 drivers/event/dlb2/rte_pmd_dlb2_event_version.map
config/rte_config.h +7 -0
@@ -131,4 +131,11 @@/* QEDE PMD defines */ #define RTE_LIBRTE_QEDE_FW "" +/* DLB2 defines */ +#define RTE_LIBRTE_PMD_DLB2_POLL_INTERVAL 1000 +#define RTE_LIBRTE_PMD_DLB2_UMWAIT_CTL_STATE 0 +#undef RTE_LIBRTE_PMD_DLB2_QUELL_STATS +#define RTE_LIBRTE_PMD_DLB2_SW_CREDIT_QUANTA 32 +#define RTE_PMD_DLB2_DEFAULT_DEPTH_THRESH 256 +#endif /* _RTE_CONFIG_H_ */
drivers/event/dlb2/meson.build new file +7 -0
@@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019-2020 Intel Corporation + +sources = files( +) + +deps += ['mbuf', 'mempool', 'ring', 'bus_vdev', 'pci', 'bus_pci']
drivers/event/dlb2/rte_pmd_dlb2_event_version.map new file +3 -0
@@ -0,0 +1,3 @@ +DPDK_21.0 { + local: *; +};
drivers/event/meson.build +4 -0
@@ -10,6 +10,10 @@ if not (toolchain == 'gcc' and cc.version().version_compare('<4.8.6') and dpdk_conf.has('RTE_ARCH_ARM64')) drivers += 'octeontx' endif +if ((dpdk_conf.has('RTE_ARCH_X86_64') or dpdk_conf.has('RTE_ARCH_X86')) and + is_linux) + drivers += 'dlb2' +endif std_deps = ['eventdev', 'kvargs'] config_flag_fmt = 'RTE_LIBRTE_@0@_EVENTDEV_PMD' driver_name_fmt = 'rte_pmd_@0@_event'
Signature · 1 line
2.6.4
PATCH 2/22 event/dlb2: add dynamic logging
In reply to Timothy McDanielThis commit adds base support for dynamic logging. The default log level is NOTICE. Dynamic logging is used exclusively throughout this patchset.
--- drivers/event/dlb2/dlb2_log.h | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 drivers/event/dlb2/dlb2_log.h
drivers/event/dlb2/dlb2_log.h new file +25 -0
@@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef _DLB2_EVDEV_LOG_H_ +#define _DLB2_EVDEV_LOG_H_ + +extern int eventdev_dlb2_log_level; + +/* Dynamic logging */ +#define DLB2_LOG_IMPL(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, eventdev_dlb2_log_level, "%s" fmt "\n", \ + __func__, ##args) + +#define DLB2_LOG_INFO(fmt, args...) \ + DLB2_LOG_IMPL(INFO, fmt, ## args) + +#define DLB2_LOG_ERR(fmt, args...) \ + DLB2_LOG_IMPL(ERR, fmt, ## args) + +/* remove debug logs at compile time unless actually debugging */ +#define DLB2_LOG_DBG(fmt, args...) \ + RTE_LOG_DP(DEBUG, PMD, fmt, ## args) + +#endif /* _DLB2_EVDEV_LOG_H_ */
Signature · 1 line
2.6.4
PATCH 3/22 event/dlb2: add private data structures and constants
In reply to Timothy McDanielThe header file dlb2_priv.h is used internally by the PMD. It include constants, macros for device resources, structure definitions for hardware interfaces and software state, and various forward-declarations. The header file rte_pmd_dlb2.h will be exported in a subsequent patch, but is included here due to a data structure dependency.
--- drivers/event/dlb2/dlb2_priv.h | 614 ++++++++++++++++++++++++++++++++++++++ drivers/event/dlb2/rte_pmd_dlb2.h | 59 ++++ 2 files changed, 673 insertions(+) create mode 100644 drivers/event/dlb2/dlb2_priv.h create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.h
drivers/event/dlb2/dlb2_priv.h new file +614 -0
@@ -0,0 +1,614 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef _DLB2_PRIV_H_ +#define _DLB2_PRIV_H_ + +#include <emmintrin.h> +#include <stdbool.h> + +#include <rte_eventdev.h> + +#include "rte_config.h" +#include "dlb2_user.h" +#include "dlb2_log.h" +#include "rte_pmd_dlb2.h" + +#ifndef RTE_LIBRTE_PMD_DLB2_QUELL_STATS +#define DLB2_INC_STAT (_stat, _incr_val) ((_stat) += _incr_val) +#else +#define DLB2_INC_STAT (_stat, _incr_val) +#endif + +#define EVDEV_DLB2_NAME_PMD dlb2_event + +/* command line arg strings */ +#define NUMA_NODE_ARG "numa_node" +#define DLB2_MAX_NUM_EVENTS "max_num_events" +#define DLB2_NUM_DIR_CREDITS "num_dir_credits" +#define DEV_ID_ARG "dev_id" +#define DLB2_DEFER_SCHED_ARG "defer_sched" +#define DLB2_QID_DEPTH_THRESH_ARG "qid_depth_thresh" +#define DLB2_COS_ARG "cos" + +/* Begin HW related defines and structs */ + +#define DLB2_MAX_NUM_DOMAINS 32 +#define DLB2_MAX_NUM_VFS 16 +#define DLB2_MAX_NUM_LDB_QUEUES 32 +#define DLB2_MAX_NUM_LDB_PORTS 64 +#define DLB2_MAX_NUM_DIR_PORTS 64 +#define DLB2_MAX_NUM_DIR_QUEUES 64 +#define DLB2_MAX_NUM_FLOWS (64 * 1024) +#define DLB2_MAX_NUM_LDB_CREDITS (8 * 1024) +#define DLB2_MAX_NUM_DIR_CREDITS (2 * 1024) +#define DLB2_MAX_NUM_LDB_CREDIT_POOLS 64 +#define DLB2_MAX_NUM_DIR_CREDIT_POOLS 64 +#define DLB2_MAX_NUM_HIST_LIST_ENTRIES 2048 +#define DLB2_MAX_NUM_AQOS_ENTRIES 2048 +#define DLB2_MAX_NUM_QIDS_PER_LDB_CQ 8 +#define DLB2_QID_PRIORITIES 8 +#define DLB2_MAX_DEVICE_PATH 32 +#define DLB2_MIN_DEQUEUE_TIMEOUT_NS 1 +/* Note: "- 1" here to support the timeout range check in eventdev_autotest */ +#define DLB2_MAX_DEQUEUE_TIMEOUT_NS (UINT32_MAX - 1) +#define DLB2_SW_CREDIT_BATCH_SZ 32 +#define DLB2_NUM_SN_GROUPS 2 +#define DLB2_MAX_LDB_SN_ALLOC 1024 +#define DLB2_MAX_QUEUE_DEPTH_THRESHOLD 8191 + +/* 2048 total hist list entries and 64 total ldb ports, which +* makes for 2048/64 == 32 hist list entries per port. However, CQ +* depth must be a power of 2 and must also be >= HIST LIST entries. +* As a result we just limit the maximum dequeue depth to 32. +*/ +#define DLB2_MIN_CQ_DEPTH 1 +#define DLB2_MAX_CQ_DEPTH 32 +#define DLB2_MIN_HARDWARE_CQ_DEPTH 8 +#define DLB2_NUM_HIST_LIST_ENTRIES_PER_LDB_PORT \ + DLB2_MAX_CQ_DEPTH + +/* +* Static per queue/port provisioning values +*/ +#define DLB2_NUM_ATOMIC_INFLIGHTS_PER_QUEUE 64 + +#define CQ_BASE (is_dir) ((is_dir) ? DLB2_DIR_CQ_BASE : DLB2_LDB_CQ_BASE) +#define CQ_SIZE (is_dir) ((is_dir) ? DLB2_DIR_CQ_MAX_SIZE : \ + DLB2_LDB_CQ_MAX_SIZE) +#define PP_BASE (is_dir) ((is_dir) ? DLB2_DIR_PP_BASE : DLB2_LDB_PP_BASE) + +#define PAGE_SIZE (sysconf(_SC_PAGESIZE)) + +#define DLB2_NUM_QES_PER_CACHE_LINE 4 + +#define DLB2_MAX_ENQUEUE_DEPTH 64 +#define DLB2_MIN_ENQUEUE_DEPTH 4 + +#define DLB2_NAME_SIZE 64 + +#define DLB2_1K 1024 +#define DLB2_2K (2 * DLB2_1K) +#define DLB2_4K (4 * DLB2_1K) +#define DLB2_16K (16 * DLB2_1K) +#define DLB2_32K (32 * DLB2_1K) +#define DLB2_1MB (DLB2_1K * DLB2_1K) +#define DLB2_16MB (16 * DLB2_1MB) + +/* Use the upper 3 bits of the event priority to select the DLB2 priority */ +#define EV_TO_DLB2_PRIO (x) ((x) >> 5) +#define DLB2_TO_EV_PRIO (x) ((x) << 5) + +enum dlb2_hw_port_types { +DLB2_LDB_PORT , +DLB2_DIR_PORT , +DLB2_NUM_PORT_TYPES /* Must be last */ +}; + +enum dlb2_hw_queue_types { +DLB2_LDB_QUEUE , +DLB2_DIR_QUEUE , +DLB2_NUM_QUEUE_TYPES /* Must be last */ +}; + +#define PORT_TYPE (p) ((p)->is_directed ? DLB2_DIR_PORT : DLB2_LDB_PORT) + +/* Do not change - must match hardware! */ +enum dlb2_hw_sched_type { +DLB2_SCHED_ATOMIC =0 , +DLB2_SCHED_UNORDERED , +DLB2_SCHED_ORDERED , +DLB2_SCHED_DIRECTED , +/* DLB2_NUM_HW_SCHED_TYPES must be last */ +DLB2_NUM_HW_SCHED_TYPES +}; + +/* TODO - these structs have been converted from qm - some fields may not be +* required +*/ + +struct dlb2_hw_rsrcs { +int32_t nb_events_limit; +uint32_t num_queues;/**> Total queues (lb + dir) */ +uint32_t num_ldb_queues;/**> Number of available ldb queues */ +uint32_t num_ldb_ports;/**< Number of load balanced ports */ +uint32_t num_dir_ports;/**< Number of directed ports */ +uint32_t num_ldb_credits;/**< Number of load balanced credits */ +uint32_t num_dir_credits;/**< Number of directed credits */ +uint32_t reorder_window_size;/**< Size of reorder window */ +}; + +struct dlb2_hw_resource_info { +/**> Max resources that can be provided */ +struct dlb2_hw_rsrcs hw_rsrc_max; +int num_sched_domains; +uint32_t socket_id; +/**> EAL flags passed to this QM instance, allowing the application to +* identify the pmd backend indicating hardware or software. +*/ +const char *eal_flags; +}; + +enum DLB2_ENQUEUE_TYPE { +/**> +* New : Used to inject a new packet into the QM. +*/ +DLB2_ENQ_NEW , +/**> +* Forward : Enqueues a packet, and +* - if atomic: release any lock it holds in the QM +* - if ordered: release the packet for egress re-ordering +*/ +DLB2_ENQ_FWD , +/**> +* Enqueue Drop : Release an inflight packet. Must be called with +* event == NULL. Used to drop a packet. +* +* Note that all packets dequeued from a load-balanced port must be +* released, either with DLB2_ENQ_DROP or DLB2_ENQ_FWD. +*/ +DLB2_ENQ_DROP , + +/* marker for array sizing etc. */ + _DLB2_NB_ENQ_TYPES +}; + +/* hw-specific format - do not change */ + +struct dlb2_event_type { +uint8_t major:4 ; +uint8_t unused:4 ; +uint8_t sub; +}; + +union dlb2_opaque_data { +uint16_t opaque_data; +struct dlb2_event_type event_type; +}; + +struct dlb2_msg_info { +uint8_t qid; +uint8_t sched_type:2 ; +uint8_t priority:3 ; +uint8_t msg_type:3 ; +}; + +#define DLB2_NEW_CMD_BYTE 0x08 +#define DLB2_FWD_CMD_BYTE 0x0A +#define DLB2_COMP_CMD_BYTE 0x02 +#define DLB2_POP_CMD_BYTE 0x01 +#define DLB2_NOOP_CMD_BYTE 0x00 + +/* hw-specific format - do not change */ +struct dlb2_enqueue_qe { +uint64_t data; +/* Word 3 */ +union dlb2_opaque_data u; +uint8_t qid; +uint8_t sched_type:2 ; +uint8_t priority:3 ; +uint8_t msg_type:3 ; +/* Word 4 */ +uint16_t lock_id; +uint8_t meas_lat:1 ; +uint8_t rsvd1:2 ; +uint8_t no_dec:1 ; +uint8_t cmp_id:4 ; +union { +uint8_t cmd_byte; +struct { +uint8_t cq_token:1 ; +uint8_t qe_comp:1 ; +uint8_t qe_frag:1 ; +uint8_t qe_valid:1 ; +uint8_t rsvd3:1 ; +uint8_t error:1 ; +uint8_t rsvd:2 ; + }; + }; +}; + +/* hw-specific format - do not change */ +struct dlb2_cq_pop_qe { +uint64_t data; +union dlb2_opaque_data u; +uint8_t qid; +uint8_t sched_type:2 ; +uint8_t priority:3 ; +uint8_t msg_type:3 ; +uint16_t tokens:10 ; +uint16_t rsvd2:6 ; +uint8_t meas_lat:1 ; +uint8_t rsvd1:2 ; +uint8_t no_dec:1 ; +uint8_t cmp_id:4 ; +union { +uint8_t cmd_byte; +struct { +uint8_t cq_token:1 ; +uint8_t qe_comp:1 ; +uint8_t qe_frag:1 ; +uint8_t qe_valid:1 ; +uint8_t rsvd3:1 ; +uint8_t error:1 ; +uint8_t rsvd:2 ; + }; + }; +}; + +/* hw-specific format - do not change */ +struct dlb2_dequeue_qe { +uint64_t data; +union dlb2_opaque_data u; +uint8_t qid; +uint8_t sched_type:2 ; +uint8_t priority:3 ; +uint8_t msg_type:3 ; +uint16_t flow_id:16 ;/* was pp_id in v1 */ +uint8_t debug; +uint8_t cq_gen:1 ; +uint8_t qid_depth:2 ;/* 2 bits in v2 */ +uint8_t rsvd1:2 ; +uint8_t error:1 ; +uint8_t rsvd2:2 ; +}; + +union dlb2_port_config { +struct dlb2_create_ldb_port_args ldb; +struct dlb2_create_dir_port_args dir; +}; + +enum DLB2_PORT_STATE { +PORT_CLOSED , +PORT_STARTED , +PORT_STOPPED +}; + +enum dlb2_configuration_state { +/* The resource has not been configured */ +DLB2_NOT_CONFIGURED , +/* The resource was configured, but the device was stopped */ +DLB2_PREV_CONFIGURED , +/* The resource is currently configured */ +DLB2_CONFIGURED +}; + +struct dlb2_port { +uint32_t id; +bool is_directed; +bool gen_bit; +uint16_t dir_credits; +uint32_t dequeue_depth; +enum dlb2_token_pop_mode token_pop_mode; +union dlb2_port_config cfg; +uint32_t *credit_pool[DLB2_NUM_QUEUE_TYPES ];/* use __atomic builtins */ +uint16_t cached_ldb_credits; +uint16_t ldb_credits; +uint16_t cached_dir_credits; +bool int_armed; +uint16_t owed_tokens; +int16_t issued_releases; +int16_t token_pop_thresh; +int cq_depth; +uint16_t cq_idx; +uint16_t cq_idx_unmasked; +uint16_t cq_depth_mask; +uint16_t gen_bit_shift; +enum DLB2_PORT_STATE state; +enum dlb2_configuration_state config_state; +int num_mapped_qids; +uint8_t *qid_mappings; +struct dlb2_enqueue_qe *qe4;/* Cache line's worth of QEs (4) */ +struct dlb2_enqueue_qe *int_arm_qe; +struct dlb2_cq_pop_qe *consume_qe; +struct dlb2_eventdev *dlb2;/* back ptr */ +struct dlb2_eventdev_port *ev_port;/* back ptr */ +}; + +/* Per-process per-port mmio and memory pointers */ +struct process_local_port_data { +uint64_t *pp_addr; +struct dlb2_dequeue_qe *cq_base; +bool mmaped; +}; + +struct dlb2_eventdev ; + +struct dlb2_port_low_level_io_functions { +void (*pp_enqueue_four)(void *qe4, +void *pp_addr); +}; + +/* TODO - Optimize memory use and layout */ +struct dlb2_config { +int configured; +int reserved; +uint32_t num_ldb_credits; +uint32_t num_dir_credits; +struct dlb2_create_sched_domain_args resources; +}; + +enum dlb2_cos { +DLB2_COS_DEFAULT =-1 , +DLB2_COS_0 =0 , +DLB2_COS_1 , +DLB2_COS_2 , +DLB2_COS_3 +}; + +struct dlb2_hw_dev { +char device_name[DLB2_NAME_SIZE ]; +char device_path[DLB2_MAX_DEVICE_PATH ]; +int device_path_id; +struct dlb2_config cfg; +struct dlb2_hw_resource_info info; +void *pf_dev;/* opaque pointer to PF PMD dev (struct dlb2_dev) */ +int device_id; +uint32_t domain_id; +int domain_id_valid; +enum dlb2_cos cos_id; +rte_spinlock_t resource_lock;/* for MP support */ +}; __rte_cache_aligned + +/* End HW related defines and structs */ + +/* Begin DLB2 PMD Eventdev related defines and structs */ + +#define DLB2_MAX_NUM_QUEUES \ + (DLB2_MAX_NUM_DIR_QUEUES + DLB2_MAX_NUM_LDB_QUEUES) + +#define DLB2_MAX_NUM_PORTS (DLB2_MAX_NUM_DIR_PORTS + DLB2_MAX_NUM_LDB_PORTS) +#define DLB2_MAX_INPUT_QUEUE_DEPTH 256 + +/* Used for parsing dir ports/queues. */ + +/* Note: eventdev currently is limited to 64 ports, but DLB hardware has 128 +* directed ports/queues. +*/ +struct dlb2_dir_resource_list { +int entries; +uint32_t id[DLB2_MAX_NUM_DIR_PORTS ]; +}; + +/** Structure to hold the queue to port link establishment attributes */ + +struct dlb2_event_queue_link { +uint8_t queue_id; +uint8_t priority; +bool mapped; +bool valid; +}; + +struct dlb2_traffic_stats { +uint64_t rx_ok; +uint64_t rx_drop; +uint64_t rx_interrupt_wait; +uint64_t rx_umonitor_umwait; +uint64_t tx_ok; +uint64_t total_polls; +uint64_t zero_polls; +uint64_t tx_nospc_ldb_hw_credits; +uint64_t tx_nospc_dir_hw_credits; +uint64_t tx_nospc_inflight_max; +uint64_t tx_nospc_new_event_limit; +uint64_t tx_nospc_inflight_credits; +}; + +/* DLB2 HW sets the 2bit qid_depth in rx QEs based on the programmable depth +* threshold. The global default value in config/common_base (or rte_config.h) +* can be overridden on a per-qid basis using a vdev command line parameter. +* 3: depth > threshold +* 2: threshold >= depth > 3/4 threshold +* 1: 3/4 threshold >= depth > 1/2 threshold +* 0: depth <= 1/2 threshold. +*/ +#define DLB2_QID_DEPTH_LE50 0 +#define DLB2_QID_DEPTH_GT50_LE75 1 +#define DLB2_QID_DEPTH_GT75_LE100 2 +#define DLB2_QID_DEPTH_GT100 3 +#define DLB2_NUM_QID_DEPTH_STAT_VALS 4/* 2 bits */ + +struct dlb2_queue_stats { +uint64_t enq_ok; +uint64_t qid_depth[DLB2_NUM_QID_DEPTH_STAT_VALS ]; +}; + +struct dlb2_port_stats { +struct dlb2_traffic_stats traffic; +uint64_t tx_op_cnt[4 ];/* indexed by rte_event.op */ +uint64_t tx_implicit_rel; +uint64_t tx_sched_cnt[DLB2_NUM_HW_SCHED_TYPES ]; +uint64_t tx_invalid; +uint64_t rx_sched_cnt[DLB2_NUM_HW_SCHED_TYPES ]; +uint64_t rx_sched_invalid; +struct dlb2_queue_stats queue[DLB2_MAX_NUM_QUEUES ]; +}; + +struct dlb2_eventdev_port { +struct dlb2_port qm_port;/* hw specific data structure */ +struct rte_event_port_conf conf;/* user-supplied configuration */ +uint16_t inflight_credits;/* num credits this port has right now */ +uint16_t credit_update_quanta; +struct dlb2_eventdev *dlb2;/* backlink optimization */ + struct dlb2_port_stats stats __rte_cache_aligned; +struct dlb2_event_queue_link link[DLB2_MAX_NUM_QIDS_PER_LDB_CQ ]; +int num_links; +uint32_t id;/* port id */ +/* num releases yet to be completed on this port. +* Only applies to load-balanced ports. +*/ +uint16_t outstanding_releases; +uint16_t inflight_max;/* app requested max inflights for this port */ +/* setup_done is set when the event port is setup */ +bool setup_done; +/* enq_configured is set when the qm port is created */ +bool enq_configured; +uint8_t implicit_release;/* release events before dequeueing */ +} __rte_cache_aligned; + +struct dlb2_queue { +uint32_t num_qid_inflights;/* User config */ +uint32_t num_atm_inflights;/* User config */ +enum dlb2_configuration_state config_state; +int sched_type;/* LB queue only */ +uint32_t id; +bool is_directed; +}; + +struct dlb2_eventdev_queue { +struct dlb2_queue qm_queue; +struct rte_event_queue_conf conf;/* User config */ +int depth_threshold;/* use default if 0 */ +uint32_t id; +bool setup_done; +uint8_t num_links; +}; + +struct dlb2_device_stats { +/* Device specific */ +}; + +enum dlb2_run_state { +DLB2_RUN_STATE_STOPPED =0 , +DLB2_RUN_STATE_STOPPING , +DLB2_RUN_STATE_STARTING , +DLB2_RUN_STATE_STARTED +}; + +struct dlb2_eventdev { +struct dlb2_eventdev_port ev_ports[DLB2_MAX_NUM_PORTS ]; +struct dlb2_eventdev_queue ev_queues[DLB2_MAX_NUM_QUEUES ]; +uint8_t qm_ldb_to_ev_queue_id[DLB2_MAX_NUM_QUEUES ]; +uint8_t qm_dir_to_ev_queue_id[DLB2_MAX_NUM_QUEUES ]; +/* store num stats and offset of the stats for each queue */ +uint16_t xstats_count_per_qid[DLB2_MAX_NUM_QUEUES ]; +uint16_t xstats_offset_for_qid[DLB2_MAX_NUM_QUEUES ]; +/* store num stats and offset of the stats for each port */ +uint16_t xstats_count_per_port[DLB2_MAX_NUM_PORTS ]; +uint16_t xstats_offset_for_port[DLB2_MAX_NUM_PORTS ]; +struct dlb2_get_num_resources_args hw_rsrc_query_results; +uint32_t xstats_count_mode_queue; +struct dlb2_hw_dev qm_instance;/* strictly hw related */ +uint64_t global_dequeue_wait_ticks; +struct dlb2_xstats_entry *xstats; +struct rte_eventdev *event_dev;/* backlink to dev */ +uint32_t xstats_count_mode_dev; +uint32_t xstats_count_mode_port; +uint32_t xstats_count; +uint32_t inflights;/* use __atomic builtins */ +uint32_t new_event_limit; +int max_num_events_override; +int num_dir_credits_override; +volatile enum dlb2_run_state run_state; +uint16_t num_dir_queues;/* total num of evdev dir queues requested */ +uint16_t num_dir_credits; +uint16_t num_ldb_credits; +uint16_t num_queues;/* total queues */ +uint16_t num_ldb_queues;/* total num of evdev ldb queues requested */ +uint16_t num_ports;/* total num of evdev ports requested */ +uint16_t num_ldb_ports;/* total num of ldb ports requested */ +uint16_t num_dir_ports;/* total num of dir ports requested */ +bool umwait_allowed; +bool global_dequeue_wait;/* Not using per dequeue wait if true */ +bool defer_sched; +enum dlb2_cq_poll_modes poll_mode; +uint8_t revision; +bool configured; +uint16_t max_ldb_credits; +uint16_t max_dir_credits; + +/* force hw credit pool counters into exclusive cache lines */ + +/* use __atomic builtins */ /* shared hw cred */ + uint32_t ldb_credit_pool __rte_cache_aligned; +/* use __atomic builtins */ /* shared hw cred */ + uint32_t dir_credit_pool __rte_cache_aligned; + +/* Device stats */ + struct dlb2_device_stats stats __rte_cache_aligned; +}; + +/* used for collecting and passing around the dev args */ +struct dlb2_qid_depth_thresholds { +int val[DLB2_MAX_NUM_QUEUES ]; +}; +struct dlb2_devargs { +int socket_id; +int max_num_events; +int num_dir_credits_override; +int dev_id; +int defer_sched; +struct dlb2_qid_depth_thresholds qid_depth_thresholds; +enum dlb2_cos cos_id; +}; + +/* End Eventdev related defines and structs */ + +/* Forwards for non-inlined functions */ + +int dlb2_uninit (const char *name); + +void dlb2_eventdev_dump (struct rte_eventdev *dev,FILE *f); + +int dlb2_xstats_init (struct dlb2_eventdev *dlb2); + +void dlb2_xstats_uninit (struct dlb2_eventdev *dlb2); + +int dlb2_eventdev_xstats_get (const struct rte_eventdev *dev, +enum rte_event_dev_xstats_mode mode,uint8_t queue_port_id, +const unsigned int ids[],uint64_t values[],unsigned int n); + +int dlb2_eventdev_xstats_get_names (const struct rte_eventdev *dev, +enum rte_event_dev_xstats_mode mode,uint8_t queue_port_id, +struct rte_event_dev_xstats_name *xstat_names, +unsigned int *ids,unsigned int size); + +uint64_t dlb2_eventdev_xstats_get_by_name (const struct rte_eventdev *dev, +const char *name,unsigned int *id); + +int dlb2_eventdev_xstats_reset (struct rte_eventdev *dev, +enum rte_event_dev_xstats_mode mode, +int16_t queue_port_id, +const uint32_t ids[], +uint32_t nb_ids); + +int test_dlb2_eventdev (void ); + +int dlb2_primary_eventdev_probe (struct rte_eventdev *dev, +const char *name, +struct dlb2_devargs *dlb2_args); + +int dlb2_secondary_eventdev_probe (struct rte_eventdev *dev, +const char *name); + +uint32_t dlb2_get_queue_depth (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *queue); + +/* arg parsing helper functions */ +int dlb2_parse_params (const char *params, +const char *name, +struct dlb2_devargs *dlb2_args); + +int dlb2_string_to_int (int *result,const char *str); +#endif /* _DLB2_PRIV_H_ */
drivers/event/dlb2/rte_pmd_dlb2.h new file +59 -0
59 lines not on this page · raw message
Signature · 1 line
2.6.4
PATCH 4/22 event/dlb2: add definitions shared with LKM or shared code
In reply to Timothy McDanielAdd headers containing structs and constants shared between the PMD and the shared code. The term shared code refers to the code that implements the hardware interface. The shared code is introduced in the probe patch, and then is extended as additional eventdev PMD entry points are added to the patchset. In the case of the bifurcated PMD (to be introduced in the future), the shared code is contained in the Linux kernel module itself.
--- drivers/event/dlb2/dlb2_user.h | 883 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 883 insertions(+) create mode 100644 drivers/event/dlb2/dlb2_user.h
drivers/event/dlb2/dlb2_user.h new file +883 -0
@@ -0,0 +1,883 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef __DLB2_USER_H +#define __DLB2_USER_H + +#define DLB2_MAX_NAME_LEN 64 + +#include <linux/types.h> + +enum dlb2_error { +DLB2_ST_SUCCESS =0 , +DLB2_ST_NAME_EXISTS , +DLB2_ST_DOMAIN_UNAVAILABLE , +DLB2_ST_LDB_PORTS_UNAVAILABLE , +DLB2_ST_DIR_PORTS_UNAVAILABLE , +DLB2_ST_LDB_QUEUES_UNAVAILABLE , +DLB2_ST_LDB_CREDITS_UNAVAILABLE , +DLB2_ST_DIR_CREDITS_UNAVAILABLE , +DLB2_ST_SEQUENCE_NUMBERS_UNAVAILABLE , +DLB2_ST_INVALID_DOMAIN_ID , +DLB2_ST_INVALID_QID_INFLIGHT_ALLOCATION , +DLB2_ST_ATOMIC_INFLIGHTS_UNAVAILABLE , +DLB2_ST_HIST_LIST_ENTRIES_UNAVAILABLE , +DLB2_ST_INVALID_LDB_QUEUE_ID , +DLB2_ST_INVALID_CQ_DEPTH , +DLB2_ST_INVALID_CQ_VIRT_ADDR , +DLB2_ST_INVALID_PORT_ID , +DLB2_ST_INVALID_QID , +DLB2_ST_INVALID_PRIORITY , +DLB2_ST_NO_QID_SLOTS_AVAILABLE , +DLB2_ST_INVALID_DIR_QUEUE_ID , +DLB2_ST_DIR_QUEUES_UNAVAILABLE , +DLB2_ST_DOMAIN_NOT_CONFIGURED , +DLB2_ST_INTERNAL_ERROR , +DLB2_ST_DOMAIN_IN_USE , +DLB2_ST_DOMAIN_NOT_FOUND , +DLB2_ST_QUEUE_NOT_FOUND , +DLB2_ST_DOMAIN_STARTED , +DLB2_ST_DOMAIN_NOT_STARTED , +DLB2_ST_LDB_PORT_REQUIRED_FOR_LDB_QUEUES , +DLB2_ST_DOMAIN_RESET_FAILED , +DLB2_ST_MBOX_ERROR , +DLB2_ST_INVALID_HIST_LIST_DEPTH , +DLB2_ST_NO_MEMORY , +DLB2_ST_INVALID_LOCK_ID_COMP_LEVEL , +DLB2_ST_INVALID_COS_ID , +}; + +static const char dlb2_error_strings[][128 ] = { +"DLB2_ST_SUCCESS" , +"DLB2_ST_NAME_EXISTS" , +"DLB2_ST_DOMAIN_UNAVAILABLE" , +"DLB2_ST_LDB_PORTS_UNAVAILABLE" , +"DLB2_ST_DIR_PORTS_UNAVAILABLE" , +"DLB2_ST_LDB_QUEUES_UNAVAILABLE" , +"DLB2_ST_LDB_CREDITS_UNAVAILABLE" , +"DLB2_ST_DIR_CREDITS_UNAVAILABLE" , +"DLB2_ST_SEQUENCE_NUMBERS_UNAVAILABLE" , +"DLB2_ST_INVALID_DOMAIN_ID" , +"DLB2_ST_INVALID_QID_INFLIGHT_ALLOCATION" , +"DLB2_ST_ATOMIC_INFLIGHTS_UNAVAILABLE" , +"DLB2_ST_HIST_LIST_ENTRIES_UNAVAILABLE" , +"DLB2_ST_INVALID_LDB_QUEUE_ID" , +"DLB2_ST_INVALID_CQ_DEPTH" , +"DLB2_ST_INVALID_CQ_VIRT_ADDR" , +"DLB2_ST_INVALID_PORT_ID" , +"DLB2_ST_INVALID_QID" , +"DLB2_ST_INVALID_PRIORITY" , +"DLB2_ST_NO_QID_SLOTS_AVAILABLE" , +"DLB2_ST_INVALID_DIR_QUEUE_ID" , +"DLB2_ST_DIR_QUEUES_UNAVAILABLE" , +"DLB2_ST_DOMAIN_NOT_CONFIGURED" , +"DLB2_ST_INTERNAL_ERROR" , +"DLB2_ST_DOMAIN_IN_USE" , +"DLB2_ST_DOMAIN_NOT_FOUND" , +"DLB2_ST_QUEUE_NOT_FOUND" , +"DLB2_ST_DOMAIN_STARTED" , +"DLB2_ST_DOMAIN_NOT_STARTED" , +"DLB2_ST_LDB_PORT_REQUIRED_FOR_LDB_QUEUES" , +"DLB2_ST_DOMAIN_RESET_FAILED" , +"DLB2_ST_MBOX_ERROR" , +"DLB2_ST_INVALID_HIST_LIST_DEPTH" , +"DLB2_ST_NO_MEMORY" , +"DLB2_ST_INVALID_LOCK_ID_COMP_LEVEL" , +"DLB2_ST_INVALID_COS_ID" , +}; + +struct dlb2_cmd_response { +__u32 status;/* Interpret using enum dlb2_error */ +__u32 id; +}; + +/********************************/ +/* 'dlb2' device file commands */ +/********************************/ + +#define DLB2_DEVICE_VERSION (x) (((x) >> 8) & 0xFF) +#define DLB2_DEVICE_REVISION (x) ((x) & 0xFF) + +enum dlb2_revisions { +DLB2_REV_A0 =0 , +}; + +/* +* DLB2_CMD_GET_DEVICE_VERSION: Query the DLB device version. +* +* This ioctl interface is the same in all driver versions and is always +* the first ioctl. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id[7:0]: Device revision. +* - response.id[15:8]: Device version. +*/ + +struct dlb2_get_device_version_args { +/* Output parameters */ +struct dlb2_cmd_response response; +}; + +/* +* DLB2_CMD_CREATE_SCHED_DOMAIN: Create a DLB 2.0 scheduling domain and reserve +* its hardware resources. This command returns the newly created domain +* ID and a file descriptor for accessing the domain. +* +* Input parameters: +* - num_ldb_queues: Number of load-balanced queues. +* - num_ldb_ports: Number of load-balanced ports that can be allocated from +* from any class-of-service with available ports. +* - num_cos_ldb_ports[4]: Number of load-balanced ports from +* classes-of-service 0-3. +* - num_dir_ports: Number of directed ports. A directed port has one directed +* queue, so no num_dir_queues argument is necessary. +* - num_atomic_inflights: This specifies the amount of temporary atomic QE +* storage for the domain. This storage is divided among the domain's +* load-balanced queues that are configured for atomic scheduling. +* - num_hist_list_entries: Amount of history list storage. This is divided +* among the domain's CQs. +* - num_ldb_credits: Amount of load-balanced QE storage (QED). QEs occupy this +* space until they are scheduled to a load-balanced CQ. One credit +* represents the storage for one QE. +* - num_dir_credits: Amount of directed QE storage (DQED). QEs occupy this +* space until they are scheduled to a directed CQ. One credit represents +* the storage for one QE. +* - cos_strict: If set, return an error if there are insufficient ports in +* class-of-service N to satisfy the num_ldb_ports_cosN argument. If +* unset, attempt to fulfill num_ldb_ports_cosN arguments from other +* classes-of-service if class N does not contain enough free ports. +* - padding1: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: domain ID. +* - domain_fd: file descriptor for performing the domain's ioctl operations +* - padding0: Reserved for future use. +*/ +struct dlb2_create_sched_domain_args { +/* Output parameters */ +struct dlb2_cmd_response response; +__u32 domain_fd; +__u32 padding0; +/* Input parameters */ +__u32 num_ldb_queues; +__u32 num_ldb_ports; +__u32 num_cos_ldb_ports[4 ]; +__u32 num_dir_ports; +__u32 num_atomic_inflights; +__u32 num_hist_list_entries; +__u32 num_ldb_credits; +__u32 num_dir_credits; +__u8 cos_strict; +__u8 padding1[3 ]; +}; + +/* +* DLB2_CMD_GET_NUM_RESOURCES: Return the number of available resources +* (queues, ports, etc.) that this device owns. +* +* Output parameters: +* - num_domains: Number of available scheduling domains. +* - num_ldb_queues: Number of available load-balanced queues. +* - num_ldb_ports: Total number of available load-balanced ports. +* - num_cos_ldb_ports[4]: Number of available load-balanced ports from +* classes-of-service 0-3. +* - num_dir_ports: Number of available directed ports. There is one directed +* queue for every directed port. +* - num_atomic_inflights: Amount of available temporary atomic QE storage. +* - num_hist_list_entries: Amount of history list storage. +* - max_contiguous_hist_list_entries: History list storage is allocated in +* a contiguous chunk, and this return value is the longest available +* contiguous range of history list entries. +* - num_ldb_credits: Amount of available load-balanced QE storage. +* - num_dir_credits: Amount of available directed QE storage. +*/ +struct dlb2_get_num_resources_args { +/* Output parameters */ +__u32 num_sched_domains; +__u32 num_ldb_queues; +__u32 num_ldb_ports; +__u32 num_cos_ldb_ports[4 ]; +__u32 num_dir_ports; +__u32 num_atomic_inflights; +__u32 num_hist_list_entries; +__u32 max_contiguous_hist_list_entries; +__u32 num_ldb_credits; +__u32 num_dir_credits; +}; + +/* +* DLB2_CMD_SET_SN_ALLOCATION: Configure a sequence number group (PF only) +* +* Input parameters: +* - group: Sequence number group ID. +* - num: Number of sequence numbers per queue. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +*/ +struct dlb2_set_sn_allocation_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 group; +__u32 num; +}; + +/* +* DLB2_CMD_GET_SN_ALLOCATION: Get a sequence number group's configuration +* +* Input parameters: +* - group: Sequence number group ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: Specified group's number of sequence numbers per queue. +*/ +struct dlb2_get_sn_allocation_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 group; +__u32 padding0; +}; + +/* +* DLB2_CMD_SET_COS_BW: Set a bandwidth allocation percentage for a +* load-balanced port class-of-service (PF only). +* +* Input parameters: +* - cos_id: class-of-service ID, between 0 and 3 (inclusive). +* - bandwidth: class-of-service bandwidth percentage. Total bandwidth +* percentages across all 4 classes cannot exceed 100%. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +*/ +struct dlb2_set_cos_bw_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 cos_id; +__u32 bandwidth; +}; + +/* +* DLB2_CMD_GET_COS_BW: Get the bandwidth allocation percentage for a +* load-balanced port class-of-service. +* +* Input parameters: +* - cos_id: class-of-service ID, between 0 and 3 (inclusive). +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: Specified class's bandwidth percentage. +*/ +struct dlb2_get_cos_bw_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 cos_id; +__u32 padding0; +}; + +/* +* DLB2_CMD_GET_SN_OCCUPANCY: Get a sequence number group's occupancy +* +* Each sequence number group has one or more slots, depending on its +* configuration. I.e.: +* - If configured for 1024 sequence numbers per queue, the group has 1 slot +* - If configured for 512 sequence numbers per queue, the group has 2 slots +* ... +* - If configured for 32 sequence numbers per queue, the group has 32 slots +* +* This ioctl returns the group's number of in-use slots. If its occupancy is +* 0, the group's sequence number allocation can be reconfigured. +* +* Input parameters: +* - group: Sequence number group ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: Specified group's number of used slots. +*/ +struct dlb2_get_sn_occupancy_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 group; +__u32 padding0; +}; + +enum dlb2_cq_poll_modes { +DLB2_CQ_POLL_MODE_STD , +DLB2_CQ_POLL_MODE_SPARSE , + +/* NUM_DLB2_CQ_POLL_MODE must be last */ +NUM_DLB2_CQ_POLL_MODE , +}; + +/* +* DLB2_CMD_QUERY_CQ_POLL_MODE: Query the CQ poll mode setting +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: CQ poll mode (see enum dlb2_cq_poll_modes). +*/ +struct dlb2_query_cq_poll_mode_args { +/* Output parameters */ +struct dlb2_cmd_response response; +}; + +enum dlb2_user_interface_commands { +DLB2_CMD_GET_DEVICE_VERSION , +DLB2_CMD_CREATE_SCHED_DOMAIN , +DLB2_CMD_GET_NUM_RESOURCES , +DLB2_CMD_SET_SN_ALLOCATION , +DLB2_CMD_GET_SN_ALLOCATION , +DLB2_CMD_SET_COS_BW , +DLB2_CMD_GET_COS_BW , +DLB2_CMD_GET_SN_OCCUPANCY , +DLB2_CMD_QUERY_CQ_POLL_MODE , + +/* NUM_DLB2_CMD must be last */ +NUM_DLB2_CMD , +}; + +/*******************************/ +/* 'domain' device file alerts */ +/*******************************/ + +/* +* Scheduling domain device files can be read to receive domain-specific +* notifications, for alerts such as hardware errors or device reset. +* +* Each alert is encoded in a 16B message. The first 8B contains the alert ID, +* and the second 8B is optional and contains additional information. +* Applications should cast read data to a struct dlb2_domain_alert, and +* interpret the struct's alert_id according to dlb2_domain_alert_id. The read +* length must be 16B, or the function will return -EINVAL. +* +* Reads are destructive, and in the case of multiple file descriptors for the +* same domain device file, an alert will be read by only one of the file +* descriptors. +* +* The driver stores alerts in a fixed-size alert ring until they are read. If +* the alert ring fills completely, subsequent alerts will be dropped. It is +* recommended that DLB2 applications dedicate a thread to perform blocking +* reads on the device file. +*/ +enum dlb2_domain_alert_id { +/* +* Software issued an illegal enqueue for a port in this domain. An +* illegal enqueue could be: +* - Illegal (excess) completion +* - Illegal fragment +* - Insufficient credits +* aux_alert_data[7:0] contains the port ID, and aux_alert_data[15:8] +* contains a flag indicating whether the port is load-balanced (1) or +* directed (0). +*/ +DLB2_DOMAIN_ALERT_PP_ILLEGAL_ENQ , +/* +* Software issued excess CQ token pops for a port in this domain. +* aux_alert_data[7:0] contains the port ID, and aux_alert_data[15:8] +* contains a flag indicating whether the port is load-balanced (1) or +* directed (0). +*/ +DLB2_DOMAIN_ALERT_PP_EXCESS_TOKEN_POPS , +/* +* A enqueue contained either an invalid command encoding or a REL, +* REL_T, RLS, FWD, FWD_T, FRAG, or FRAG_T from a directed port. +* +* aux_alert_data[7:0] contains the port ID, and aux_alert_data[15:8] +* contains a flag indicating whether the port is load-balanced (1) or +* directed (0). +*/ +DLB2_DOMAIN_ALERT_ILLEGAL_HCW , +/* +* The QID must be valid and less than 128. +* +* aux_alert_data[7:0] contains the port ID, and aux_alert_data[15:8] +* contains a flag indicating whether the port is load-balanced (1) or +* directed (0). +*/ +DLB2_DOMAIN_ALERT_ILLEGAL_QID , +/* +* An enqueue went to a disabled QID. +* +* aux_alert_data[7:0] contains the port ID, and aux_alert_data[15:8] +* contains a flag indicating whether the port is load-balanced (1) or +* directed (0). +*/ +DLB2_DOMAIN_ALERT_DISABLED_QID , +/* +* The device containing this domain was reset. All applications using +* the device need to exit for the driver to complete the reset +* procedure. +* +* aux_alert_data doesn't contain any information for this alert. +*/ +DLB2_DOMAIN_ALERT_DEVICE_RESET , +/* +* User-space has enqueued an alert. +* +* aux_alert_data contains user-provided data. +*/ +DLB2_DOMAIN_ALERT_USER , +/* +* The watchdog timer fired for the specified port. This occurs if its +* CQ was not serviced for a large amount of time, likely indicating a +* hung thread. +* aux_alert_data[7:0] contains the port ID, and aux_alert_data[15:8] +* contains a flag indicating whether the port is load-balanced (1) or +* directed (0). +*/ +DLB2_DOMAIN_ALERT_CQ_WATCHDOG_TIMEOUT , + +/* Number of DLB2 domain alerts */ +NUM_DLB2_DOMAIN_ALERTS +}; + +static const char dlb2_domain_alert_strings[][128 ] = { +"DLB2_DOMAIN_ALERT_PP_ILLEGAL_ENQ" , +"DLB2_DOMAIN_ALERT_PP_EXCESS_TOKEN_POPS" , +"DLB2_DOMAIN_ALERT_ILLEGAL_HCW" , +"DLB2_DOMAIN_ALERT_ILLEGAL_QID" , +"DLB2_DOMAIN_ALERT_DISABLED_QID" , +"DLB2_DOMAIN_ALERT_DEVICE_RESET" , +"DLB2_DOMAIN_ALERT_USER" , +"DLB2_DOMAIN_ALERT_CQ_WATCHDOG_TIMEOUT" , +}; + +struct dlb2_domain_alert { +__u64 alert_id; +__u64 aux_alert_data; +}; + +/*********************************/ +/* 'domain' device file commands */ +/*********************************/ + +/* +* DLB2_DOMAIN_CMD_CREATE_LDB_QUEUE: Configure a load-balanced queue. +* Input parameters: +* - num_atomic_inflights: This specifies the amount of temporary atomic QE +* storage for this queue. If zero, the queue will not support atomic +* scheduling. +* - num_sequence_numbers: This specifies the number of sequence numbers used +* by this queue. If zero, the queue will not support ordered scheduling. +* If non-zero, the queue will not support unordered scheduling. +* - num_qid_inflights: The maximum number of QEs that can be inflight +* (scheduled to a CQ but not completed) at any time. If +* num_sequence_numbers is non-zero, num_qid_inflights must be set equal +* to num_sequence_numbers. +* - lock_id_comp_level: Lock ID compression level. Specifies the number of +* unique lock IDs the queue should compress down to. Valid compression +* levels: 0, 64, 128, 256, 512, 1k, 2k, 4k, 64k. If lock_id_comp_level is +* 0, the queue won't compress its lock IDs. +* - depth_threshold: DLB sets two bits in the received QE to indicate the +* depth of the queue relative to the threshold before scheduling the +* QE to a CQ: +* - 2’b11: depth > threshold +* - 2’b10: threshold >= depth > 0.75 * threshold +* - 2’b01: 0.75 * threshold >= depth > 0.5 * threshold +* - 2’b00: depth <= 0.5 * threshold +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: Queue ID. +*/ +struct dlb2_create_ldb_queue_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 num_sequence_numbers; +__u32 num_qid_inflights; +__u32 num_atomic_inflights; +__u32 lock_id_comp_level; +__u32 depth_threshold; +__u32 padding0; +}; + +/* +* DLB2_DOMAIN_CMD_CREATE_DIR_QUEUE: Configure a directed queue. +* Input parameters: +* - port_id: Port ID. If the corresponding directed port is already created, +* specify its ID here. Else this argument must be 0xFFFFFFFF to indicate +* that the queue is being created before the port. +* - depth_threshold: DLB sets two bits in the received QE to indicate the +* depth of the queue relative to the threshold before scheduling the +* QE to a CQ: +* - 2’b11: depth > threshold +* - 2’b10: threshold >= depth > 0.75 * threshold +* - 2’b01: 0.75 * threshold >= depth > 0.5 * threshold +* - 2’b00: depth <= 0.5 * threshold +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: Queue ID. +*/ +struct dlb2_create_dir_queue_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__s32 port_id; +__u32 depth_threshold; +}; + +/* +* DLB2_DOMAIN_CMD_CREATE_LDB_PORT: Configure a load-balanced port. +* Input parameters: +* - cq_depth: Depth of the port's CQ. Must be a power-of-two between 8 and +* 1024, inclusive. +* - cq_depth_threshold: CQ depth interrupt threshold. A value of N means that +* the CQ interrupt won't fire until there are N or more outstanding CQ +* tokens. +* - num_hist_list_entries: Number of history list entries. This must be +* greater than or equal cq_depth. +* - cos_id: class-of-service to allocate this port from. Must be between 0 and +* 3, inclusive. +* - cos_strict: If set, return an error if there are no available ports in the +* requested class-of-service. Else, allocate the port from a different +* class-of-service if the requested class has no available ports. +* +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: port ID. +*/ + +struct dlb2_create_ldb_port_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u16 cq_depth; +__u16 cq_depth_threshold; +__u16 cq_history_list_size; +__u8 cos_id; +__u8 cos_strict; +}; + +/* +* DLB2_DOMAIN_CMD_CREATE_DIR_PORT: Configure a directed port. +* Input parameters: +* - cq_depth: Depth of the port's CQ. Must be a power-of-two between 8 and +* 1024, inclusive. +* - cq_depth_threshold: CQ depth interrupt threshold. A value of N means that +* the CQ interrupt won't fire until there are N or more outstanding CQ +* tokens. +* - qid: Queue ID. If the corresponding directed queue is already created, +* specify its ID here. Else this argument must be 0xFFFFFFFF to indicate +* that the port is being created before the queue. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: Port ID. +*/ +struct dlb2_create_dir_port_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u16 cq_depth; +__u16 cq_depth_threshold; +__s32 queue_id; +}; + +/* +* DLB2_DOMAIN_CMD_START_DOMAIN: Mark the end of the domain configuration. This +* must be called before passing QEs into the device, and no configuration +* ioctls can be issued once the domain has started. Sending QEs into the +* device before calling this ioctl will result in undefined behavior. +* Input parameters: +* - (None) +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +*/ +struct dlb2_start_domain_args { +/* Output parameters */ +struct dlb2_cmd_response response; +}; + +/* +* DLB2_DOMAIN_CMD_MAP_QID: Map a load-balanced queue to a load-balanced port. +* Input parameters: +* - port_id: Load-balanced port ID. +* - qid: Load-balanced queue ID. +* - priority: Queue->port service priority. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +*/ +struct dlb2_map_qid_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 port_id; +__u32 qid; +__u32 priority; +__u32 padding0; +}; + +/* +* DLB2_DOMAIN_CMD_UNMAP_QID: Unmap a load-balanced queue to a load-balanced +* port. +* Input parameters: +* - port_id: Load-balanced port ID. +* - qid: Load-balanced queue ID. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +*/ +struct dlb2_unmap_qid_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 port_id; +__u32 qid; +}; + +/* +* DLB2_DOMAIN_CMD_ENABLE_LDB_PORT: Enable scheduling to a load-balanced port. +* Input parameters: +* - port_id: Load-balanced port ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +*/ +struct dlb2_enable_ldb_port_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 port_id; +__u32 padding0; +}; + +/* +* DLB2_DOMAIN_CMD_ENABLE_DIR_PORT: Enable scheduling to a directed port. +* Input parameters: +* - port_id: Directed port ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +*/ +struct dlb2_enable_dir_port_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 port_id; +}; + +/* +* DLB2_DOMAIN_CMD_DISABLE_LDB_PORT: Disable scheduling to a load-balanced +* port. +* Input parameters: +* - port_id: Load-balanced port ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +*/ +struct dlb2_disable_ldb_port_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 port_id; +__u32 padding0; +}; + +/* +* DLB2_DOMAIN_CMD_DISABLE_DIR_PORT: Disable scheduling to a directed port. +* Input parameters: +* - port_id: Directed port ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +*/ +struct dlb2_disable_dir_port_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 port_id; +__u32 padding0; +}; + +/* +* DLB2_DOMAIN_CMD_ENQUEUE_DOMAIN_ALERT: Enqueue a domain alert that will be +* read by one reader thread. +* +* Input parameters: +* - aux_alert_data: user-defined auxiliary data. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +*/ +struct dlb2_enqueue_domain_alert_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u64 aux_alert_data; +}; + +/* +* DLB2_DOMAIN_CMD_GET_LDB_QUEUE_DEPTH: Get a load-balanced queue's depth. +* Input parameters: +* - queue_id: The load-balanced queue ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: queue depth. +*/ +struct dlb2_get_ldb_queue_depth_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 queue_id; +__u32 padding0; +}; + +/* +* DLB2_DOMAIN_CMD_DIR_QUEUE_DEPTH: Get a directed queue's depth. +* Input parameters: +* - queue_id: The directed queue ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: queue depth. +*/ +struct dlb2_get_dir_queue_depth_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 queue_id; +__u32 padding0; +}; + +/* +* DLB2_DOMAIN_CMD_PENDING_PORT_UNMAPS: Get number of queue unmap operations in +* progress for a load-balanced port. +* +* Note: This is a snapshot; the number of unmap operations in progress +* is subject to change at any time. +* +* Input parameters: +* - port_id: Load-balanced port ID. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: number of unmaps in progress. +*/ +struct dlb2_pending_port_unmaps_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 port_id; +__u32 padding0; +}; + +/* +* DLB2_CMD_GET_LDB_PORT_PP_FD: Get file descriptor to mmap a load-balanced +* port's producer port (PP). +* DLB2_CMD_GET_LDB_PORT_CQ_FD: Get file descriptor to mmap a load-balanced +* port's consumer queue (CQ). +* +* The load-balanced port must have been previously created with the ioctl +* DLB2_CMD_CREATE_LDB_PORT. The fd is used to mmap the PP/CQ region. +* +* DLB2_CMD_GET_DIR_PORT_PP_FD: Get file descriptor to mmap a directed port's +* producer port (PP). +* DLB2_CMD_GET_DIR_PORT_CQ_FD: Get file descriptor to mmap a directed port's +* consumer queue (CQ). +* +* The directed port must have been previously created with the ioctl +* DLB2_CMD_CREATE_DIR_PORT. The fd is used to mmap PP/CQ region. +* +* Input parameters: +* - port_id: port ID. +* - padding0: Reserved for future use. +* +* Output parameters: +* - response.status: Detailed error code. In certain cases, such as if the +* ioctl request arg is invalid, the driver won't set status. +* - response.id: fd. +*/ +struct dlb2_get_port_fd_args { +/* Output parameters */ +struct dlb2_cmd_response response; +/* Input parameters */ +__u32 port_id; +__u32 padding0; +}; + +enum dlb2_domain_user_interface_commands { +DLB2_DOMAIN_CMD_CREATE_LDB_QUEUE , +DLB2_DOMAIN_CMD_CREATE_DIR_QUEUE , +DLB2_DOMAIN_CMD_CREATE_LDB_PORT , +DLB2_DOMAIN_CMD_CREATE_DIR_PORT , +DLB2_DOMAIN_CMD_START_DOMAIN , +DLB2_DOMAIN_CMD_MAP_QID , +DLB2_DOMAIN_CMD_UNMAP_QID , +DLB2_DOMAIN_CMD_ENABLE_LDB_PORT , +DLB2_DOMAIN_CMD_ENABLE_DIR_PORT , +DLB2_DOMAIN_CMD_DISABLE_LDB_PORT , +DLB2_DOMAIN_CMD_DISABLE_DIR_PORT , +DLB2_DOMAIN_CMD_BLOCK_ON_CQ_INTERRUPT , +DLB2_DOMAIN_CMD_ENQUEUE_DOMAIN_ALERT , +DLB2_DOMAIN_CMD_GET_LDB_QUEUE_DEPTH , +DLB2_DOMAIN_CMD_GET_DIR_QUEUE_DEPTH , +DLB2_DOMAIN_CMD_PENDING_PORT_UNMAPS , +DLB2_DOMAIN_CMD_GET_LDB_PORT_PP_FD , +DLB2_DOMAIN_CMD_GET_LDB_PORT_CQ_FD , +DLB2_DOMAIN_CMD_GET_DIR_PORT_PP_FD , +DLB2_DOMAIN_CMD_GET_DIR_PORT_CQ_FD , + +/* NUM_DLB2_DOMAIN_CMD must be last */ +NUM_DLB2_DOMAIN_CMD , +}; + +/* +* Mapping sizes for memory mapping the consumer queue (CQ) memory space, and +* producer port (PP) MMIO space. +*/ +#define DLB2_CQ_SIZE 65536 +#define DLB2_PP_SIZE 4096 + + +#endif /* __DLB2_USER_H */
Signature · 1 line
2.6.4
PATCH 5/22 event/dlb2: add inline functions
In reply to Timothy McDanielAdd miscellaneous inline functions that may be called from multiple files. These functions include inline assembly of new x86 instructions, such as movdir64b, since they are not available as builtin functions in the minimum supported GCC version.
--- drivers/event/dlb2/dlb2_inline_fns.h | 85 ++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 drivers/event/dlb2/dlb2_inline_fns.h
drivers/event/dlb2/dlb2_inline_fns.h new file +85 -0
@@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef _DLB2_INLINE_FNS_H_ +#define _DLB2_INLINE_FNS_H_ + +/* Inline functions required in more than one source file. +*/ + +static inline struct dlb2_eventdev * +dlb2_pmd_priv (const struct rte_eventdev *eventdev) +{ +return eventdev->data->dev_private; +} + +static inline void +dlb2_umonitor (volatile void *addr) +{ +/* TODO: Change to proper assembly when compiler support available */ + asmvolatile (".byte 0xf3, 0x0f, 0xae, 0xf7\t\n" + : + :"D" (addr)); +} + +static inline void +dlb2_umwait (int state,uint64_t timeout) +{ +uint32_t eax = timeout &UINT32_MAX ; +uint32_t edx = timeout >>32 ; + +/* TODO: Change to proper assembly when compiler support available */ + asmvolatile (".byte 0xf2, 0x0f, 0xae, 0xf7\t\n" + : + :"D" (state),"a" (eax),"d" (edx)); +} + +static inline void +dlb2_movntdq (void *qe4,void *pp_addr) +{ +/* Move entire 64B cache line of QEs, 128 bits (16B) at a time. */ +long long *_qe = (long long *)qe4; +__v2di src_data0 = (__v2di ){_qe[0 ], _qe[1 ]}; +__v2di src_data1 = (__v2di ){_qe[2 ], _qe[3 ]}; +__v2di src_data2 = (__v2di ){_qe[4 ], _qe[5 ]}; +__v2di src_data3 = (__v2di ){_qe[6 ], _qe[7 ]}; + +__builtin_ia32_movntdq ((__v2di *)pp_addr +0 , (__v2di )src_data0); +rte_wmb (); +__builtin_ia32_movntdq ((__v2di *)pp_addr +1 , (__v2di )src_data1); +rte_wmb (); +__builtin_ia32_movntdq ((__v2di *)pp_addr +2 , (__v2di )src_data2); +rte_wmb (); +__builtin_ia32_movntdq ((__v2di *)pp_addr +3 , (__v2di )src_data3); +rte_wmb (); +} + +static inline void +dlb2_movntdq_single (void *qe4,void *pp_addr) +{ +long long *_qe = (long long *)qe4; +__v2di src_data0 = (__v2di ){_qe[0 ], _qe[1 ]}; + +__builtin_ia32_movntdq ((__v2di *)pp_addr, (__v2di )src_data0); +} + +static inline void +dlb2_cldemote (void *addr) +{ +/* Load addr into RSI, then demote the cache line of the address +* contained in that register. +*/ + asmvolatile (".byte 0x0f, 0x1c, 0x06" ::"S" (addr)); +} + +static inline void +dlb2_movdir64b (void *qe4,void *pp_addr) +{ +/* TODO: Change to proper assembly when compiler support available */ + asmvolatile (".byte 0x66, 0x0f, 0x38, 0xf8, 0x02" + : + :"a" (pp_addr),"d" (qe4)); +} + +#endif /* _DLB2_INLINE_FNS_H_ */
Signature · 1 line
2.6.4
PATCH 7/22 event/dlb2: add xstats
In reply to Timothy McDanielAdd support for DLB2 xstats. Perform initialization and add standard xstats entry points.
--- drivers/event/dlb2/dlb2.c | 35 +- drivers/event/dlb2/dlb2_xstats.c | 1269 ++++++++++++++++++++++++++++++++++++++ drivers/event/dlb2/meson.build | 1 + 3 files changed, 1302 insertions(+), 3 deletions(-) create mode 100644 drivers/event/dlb2/dlb2_xstats.c
drivers/event/dlb2/dlb2.c +32 -3
@@ -77,6 +77,21 @@ static struct dlb2_port_low_level_io_functions qm_mmio_fns;struct process_local_port_data dlb2_port[DLB2_MAX_NUM_PORTS ][DLB2_NUM_PORT_TYPES ]; +/* +* DUMMY - added so that xstats path will compile/link. +* Will be replaced by real version in a subsequent +* patch. +*/ +uint32_t +dlb2_get_queue_depth (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *queue) +{ +RTE_SET_USED (dlb2); +RTE_SET_USED (queue); + +return 0 ; +} +/* override defaults with value(s) provided on command line */ static void dlb2_init_queue_depth_thresholds(struct dlb2_eventdev *dlb2,
@@ -353,9 +368,16 @@ set_qid_depth_thresh(const char *key __rte_unused,static void dlb2_entry_points_init (struct rte_eventdev *dev) { -RTE_SET_USED (dev); - -/* Eventdev PMD entry points */ +/* Expose PMD's eventdev interface */ +static struct rte_eventdev_ops dlb2_eventdev_entry_ops = { + .dump = dlb2_eventdev_dump, + .xstats_get = dlb2_eventdev_xstats_get, + .xstats_get_names = dlb2_eventdev_xstats_get_names, + .xstats_get_by_name = dlb2_eventdev_xstats_get_by_name, + .xstats_reset = dlb2_eventdev_xstats_reset, + }; + + dev->dev_ops = &dlb2_eventdev_entry_ops; }int
@@ -411,6 +433,13 @@ dlb2_primary_eventdev_probe(struct rte_eventdev *dev,return err; } +/* Complete xtstats runtime initialization */ + err =dlb2_xstats_init (dlb2); +if (err) { +DLB2_LOG_ERR ("dlb2: failed to init xstats, err=%d\n" , err); +return err; + } +/* Initialize each port's token pop mode */ for (i =0 ; i <DLB2_MAX_NUM_PORTS ; i++) dlb2->ev_ports[i].qm_port.token_pop_mode =AUTO_POP ;
drivers/event/dlb2/dlb2_xstats.c new file +1269 -0
1,269 lines not on this page · raw message
drivers/event/dlb2/meson.build +1 -0
@@ -3,6 +3,7 @@ sources = files('dlb2.c', 'dlb2_iface.c', + 'dlb2_xstats.c', 'pf/dlb2_main.c', 'pf/dlb2_pf.c', 'pf/base/dlb2_resource.c'
Signature · 1 line
2.6.4
PATCH 8/22 event/dlb2: add infos get and configure
In reply to Timothy McDanielAdd support for configuring the DLB hardware.
--- drivers/event/dlb2/dlb2.c | 334 +++ drivers/event/dlb2/dlb2_iface.c | 7 +- drivers/event/dlb2/dlb2_iface.h | 5 + drivers/event/dlb2/pf/base/dlb2_resource.c | 3234 ++++++++++++++++++++++++++++ drivers/event/dlb2/pf/dlb2_main.c | 14 + drivers/event/dlb2/pf/dlb2_pf.c | 44 + 6 files changed, 3637 insertions(+), 1 deletion(-)
drivers/event/dlb2/dlb2.c +334 -0
@@ -92,6 +92,28 @@ dlb2_get_queue_depth(struct dlb2_eventdev *dlb2,return 0 ; } +static void +dlb2_free_qe_mem (struct dlb2_port *qm_port) +{ +if (qm_port ==NULL ) +return ; + +if (qm_port->qe4) { +rte_free (qm_port->qe4); + qm_port->qe4 =NULL ; + } + +if (qm_port->int_arm_qe) { +rte_free (qm_port->int_arm_qe); + qm_port->int_arm_qe =NULL ; + } + +if (qm_port->consume_qe) { +rte_free (qm_port->consume_qe); + qm_port->consume_qe =NULL ; + } +} +/* override defaults with value(s) provided on command line */ static void dlb2_init_queue_depth_thresholds(struct dlb2_eventdev *dlb2,
@@ -366,10 +388,322 @@ set_qid_depth_thresh(const char *key __rte_unused, }static void +dlb2_eventdev_info_get (struct rte_eventdev *dev, +struct rte_event_dev_info *dev_info) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +int ret; + + ret =dlb2_hw_query_resources (dlb2); +if (ret) { +const struct rte_eventdev_data *data = dev->data; + +DLB2_LOG_ERR ("get resources err=%d, devid=%d\n" , + ret, data->dev_id); +/* fn is void, so fall through and return values set up in +* probe +*/ + } + +/* Add num resources currently owned by this domain. +* These would become available if the scheduling domain were reset due +* to the application recalling eventdev_configure to *reconfigure* the +* domain. +*/ + evdev_dlb2_default_info.max_event_ports += dlb2->num_ldb_ports; + evdev_dlb2_default_info.max_event_queues += dlb2->num_ldb_queues; + evdev_dlb2_default_info.max_num_events += dlb2->max_ldb_credits; + + evdev_dlb2_default_info.max_event_queues = +RTE_MIN (evdev_dlb2_default_info.max_event_queues, +RTE_EVENT_MAX_QUEUES_PER_DEV ); + + evdev_dlb2_default_info.max_num_events = +RTE_MIN (evdev_dlb2_default_info.max_num_events, + dlb2->max_num_events_override); + + *dev_info = evdev_dlb2_default_info; +} + +static int +dlb2_hw_create_sched_domain (struct dlb2_hw_dev *handle, +const struct dlb2_hw_rsrcs *resources_asked) +{ +int ret =0 ; +struct dlb2_create_sched_domain_args *config_params; + +if (resources_asked ==NULL ) { +DLB2_LOG_ERR ("dlb2: dlb2_create NULL parameter\n" ); + ret =EINVAL ; + goto error_exit; + } + +/* Map generic qm resources to dlb2 resources */ + config_params = &handle->cfg.resources; + +/* DIR ports and queues */ + + config_params->num_dir_ports = + resources_asked->num_dir_ports; + + config_params->num_dir_credits = + resources_asked->num_dir_credits; + +/* LDB queues */ + + config_params->num_ldb_queues = + resources_asked->num_ldb_queues; + +/* LDB ports */ + + config_params->cos_strict =0 ;/* Best effort */ + config_params->num_cos_ldb_ports[0 ] =0 ; + config_params->num_cos_ldb_ports[1 ] =0 ; + config_params->num_cos_ldb_ports[2 ] =0 ; + config_params->num_cos_ldb_ports[3 ] =0 ; + +switch (handle->cos_id) { +case DLB2_COS_0 : + config_params->num_ldb_ports =0 ;/* no don't care ports */ + config_params->num_cos_ldb_ports[0 ] = + resources_asked->num_ldb_ports; +break ; +case DLB2_COS_1 : + config_params->num_ldb_ports =0 ;/* no don't care ports */ + config_params->num_cos_ldb_ports[1 ] = + resources_asked->num_ldb_ports; +break ; +case DLB2_COS_2 : + config_params->num_ldb_ports =0 ;/* no don't care ports */ + config_params->num_cos_ldb_ports[2 ] = + resources_asked->num_ldb_ports; +break ; +case DLB2_COS_3 : + config_params->num_ldb_ports =0 ;/* no don't care ports */ + config_params->num_cos_ldb_ports[3 ] = + resources_asked->num_ldb_ports; +break ; +case DLB2_COS_DEFAULT : +/* all ldb ports are don't care ports from a cos perspective */ + config_params->num_ldb_ports = + resources_asked->num_ldb_ports; +break ; + } + + config_params->num_ldb_credits = + resources_asked->num_ldb_credits; + + config_params->num_atomic_inflights = +DLB2_NUM_ATOMIC_INFLIGHTS_PER_QUEUE * + config_params->num_ldb_queues; + + config_params->num_hist_list_entries = resources_asked->num_ldb_ports * +DLB2_NUM_HIST_LIST_ENTRIES_PER_LDB_PORT ; + +DLB2_LOG_DBG ("sched domain create - ldb_qs=%d, ldb_ports=%d, dir_ports=%d, atomic_inflights=%d, hist_list_entries=%d, ldb_credits=%d, dir_credits=%d\n" , + config_params->num_ldb_queues, + resources_asked->num_ldb_ports, + config_params->num_dir_ports, + config_params->num_atomic_inflights, + config_params->num_hist_list_entries, + config_params->num_ldb_credits, + config_params->num_dir_credits); + +/* Configure the QM */ + + ret =dlb2_iface_sched_domain_create (handle, config_params); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: domain create failed, device_id = %d, (driver ret = %d, extra status: %s)\n" , + handle->device_id, + ret, + dlb2_error_strings + [config_params->response.status]); + + goto error_exit; + } + + handle->domain_id = config_params->response.id; + handle->domain_id_valid =1 ; + handle->cfg.configured = true; + +error_exit: + +return ret; +} + +static void +dlb2_hw_reset_sched_domain (const struct rte_eventdev *dev,bool reconfig) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +enum dlb2_configuration_state config_state; +int i, j; + +dlb2_iface_domain_reset (dlb2); + +/* Free all dynamically allocated port memory */ +for (i =0 ; i < dlb2->num_ports; i++) +dlb2_free_qe_mem (&dlb2->ev_ports[i].qm_port); + +/* If reconfiguring, mark the device's queues and ports as "previously +* configured." If the user doesn't reconfigure them, the PMD will +* reapply their previous configuration when the device is started. +*/ + config_state = (reconfig) ?DLB2_PREV_CONFIGURED : +DLB2_NOT_CONFIGURED ; + +for (i =0 ; i < dlb2->num_ports; i++) { + dlb2->ev_ports[i].qm_port.config_state = config_state; +/* Reset setup_done so ports can be reconfigured */ + dlb2->ev_ports[i].setup_done = false; +for (j =0 ; j <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; j++) + dlb2->ev_ports[i].link[j].mapped = false; + } + +for (i =0 ; i < dlb2->num_queues; i++) + dlb2->ev_queues[i].qm_queue.config_state = config_state; + +for (i =0 ; i <DLB2_MAX_NUM_QUEUES ; i++) + dlb2->ev_queues[i].setup_done = false; + + dlb2->num_ports =0 ; + dlb2->num_ldb_ports =0 ; + dlb2->num_dir_ports =0 ; + dlb2->num_queues =0 ; + dlb2->num_ldb_queues =0 ; + dlb2->num_dir_queues =0 ; + dlb2->configured = false; +} + +/* Note: 1 QM instance per QM device, QM instance/device == event device */ +static int +dlb2_eventdev_configure (const struct rte_eventdev *dev) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_hw_rsrcs *rsrcs = &handle->info.hw_rsrc_max; +const struct rte_eventdev_data *data = dev->data; +const struct rte_event_dev_config *config = &data->dev_conf; +int ret; + +/* If this eventdev is already configured, we must release the current +* scheduling domain before attempting to configure a new one. +*/ +if (dlb2->configured) { +dlb2_hw_reset_sched_domain (dev, true); + + ret =dlb2_hw_query_resources (dlb2); +if (ret) { +DLB2_LOG_ERR ("get resources err=%d, devid=%d\n" , + ret, data->dev_id); +return ret; + } + } + +if (config->nb_event_queues > rsrcs->num_queues) { +DLB2_LOG_ERR ("nb_event_queues parameter (%d) exceeds the QM device's capabilities (%d).\n" , + config->nb_event_queues, + rsrcs->num_queues); +return -EINVAL ; + } +if (config->nb_event_ports > (rsrcs->num_ldb_ports + + rsrcs->num_dir_ports)) { +DLB2_LOG_ERR ("nb_event_ports parameter (%d) exceeds the QM device's capabilities (%d).\n" , + config->nb_event_ports, + (rsrcs->num_ldb_ports + rsrcs->num_dir_ports)); +return -EINVAL ; + } +if (config->nb_events_limit > rsrcs->nb_events_limit) { +DLB2_LOG_ERR ("nb_events_limit parameter (%d) exceeds the QM device's capabilities (%d).\n" , + config->nb_events_limit, + rsrcs->nb_events_limit); +return -EINVAL ; + } + +if (config->event_dev_cfg &RTE_EVENT_DEV_CFG_PER_DEQUEUE_TIMEOUT ) { + dlb2->global_dequeue_wait = false; + }else { +uint32_t timeout32; + + dlb2->global_dequeue_wait = true; + +/* Craziness here is due to size mismatch in eventdev lib. +* TODO: Submit patch so dequeue API and config use same bit +* width timeout value and same units or time, instead of one +* being 32b ns and the other being 64b ticks. +*/ + + timeout32 = config->dequeue_timeout_ns; + + dlb2->global_dequeue_wait_ticks = + timeout32 * (rte_get_timer_hz () /1E9 ); + } + +/* Does this platform support umonitor/umwait? */ +if (rte_cpu_get_flag_enabled (RTE_CPUFLAG_UMWAIT )) { +if (RTE_LIBRTE_PMD_DLB2_UMWAIT_CTL_STATE !=0 && +RTE_LIBRTE_PMD_DLB2_UMWAIT_CTL_STATE !=1 ) { +DLB2_LOG_ERR ("invalid value (%d) for RTE_LIBRTE_PMD_DLB2_UMWAIT_CTL_STATE, must be 0 or 1.\n" , +RTE_LIBRTE_PMD_DLB2_UMWAIT_CTL_STATE ); +return -EINVAL ; + } + dlb2->umwait_allowed = true; + } + +/* FIXME: DLB should revert to load-balanced ports if dir are not +* available +*/ + + rsrcs->num_dir_ports = config->nb_single_link_event_port_queues; + rsrcs->num_ldb_ports = config->nb_event_ports - rsrcs->num_dir_ports; +/* 1 dir queue per dir port */ + rsrcs->num_ldb_queues = config->nb_event_queues - rsrcs->num_dir_ports; + +/* Scale down nb_events_limit by 4 for directed credits, since there +* are 4x as many load-balanced credits. +*/ + rsrcs->num_ldb_credits =0 ; + rsrcs->num_dir_credits =0 ; + +if (rsrcs->num_ldb_queues) + rsrcs->num_ldb_credits = config->nb_events_limit; +if (rsrcs->num_dir_ports) + rsrcs->num_dir_credits = config->nb_events_limit /4 ; +if (dlb2->num_dir_credits_override !=-1 ) + rsrcs->num_dir_credits = dlb2->num_dir_credits_override; + +if (dlb2_hw_create_sched_domain (handle, rsrcs) <0 ) { +DLB2_LOG_ERR ("dlb2_hw_create_sched_domain failed\n" ); +return -ENODEV ; + } + + dlb2->new_event_limit = config->nb_events_limit; +__atomic_store_n (&dlb2->inflights,0 , __ATOMIC_SEQ_CST); + + +/* Save number of ports/queues for this event dev */ + dlb2->num_ports = config->nb_event_ports; + dlb2->num_queues = config->nb_event_queues; + dlb2->num_dir_ports = rsrcs->num_dir_ports; + dlb2->num_ldb_ports = dlb2->num_ports - dlb2->num_dir_ports; + dlb2->num_ldb_queues = dlb2->num_queues - dlb2->num_dir_ports; + dlb2->num_dir_queues = dlb2->num_dir_ports; + dlb2->ldb_credit_pool = rsrcs->num_ldb_credits; + dlb2->max_ldb_credits = rsrcs->num_ldb_credits; + dlb2->dir_credit_pool = rsrcs->num_dir_credits; + dlb2->max_dir_credits = rsrcs->num_dir_credits; + + dlb2->configured = true; + +return 0 ; +} + +static void dlb2_entry_points_init (struct rte_eventdev *dev) {/* Expose PMD's eventdev interface */ static struct rte_eventdev_ops dlb2_eventdev_entry_ops = { + .dev_infos_get = dlb2_eventdev_info_get, + .dev_configure = dlb2_eventdev_configure, .dump = dlb2_eventdev_dump, .xstats_get = dlb2_eventdev_xstats_get, .xstats_get_names = dlb2_eventdev_xstats_get_names,
drivers/event/dlb2/dlb2_iface.c +6 -1
@@ -39,4 +39,9 @@ int (*dlb2_iface_get_cq_poll_mode)(struct dlb2_hw_dev *handle, enum dlb2_cq_poll_modes *mode);int (*dlb2_iface_get_num_resources)(struct dlb2_hw_dev *handle, -struct dlb2_get_num_resources_args *rsrcs); +struct dlb2_get_num_resources_args *rsrcs); + +int (*dlb2_iface_sched_domain_create)(struct dlb2_hw_dev *handle, +struct dlb2_create_sched_domain_args *args); + +void (*dlb2_iface_domain_reset)(struct dlb2_eventdev *dlb2);
drivers/event/dlb2/dlb2_iface.h +5 -0
@@ -26,4 +26,9 @@ extern int (*dlb2_iface_get_cq_poll_mode)(struct dlb2_hw_dev *handle,extern int (*dlb2_iface_get_num_resources)(struct dlb2_hw_dev *handle,struct dlb2_get_num_resources_args *rsrcs); +extern int (*dlb2_iface_sched_domain_create)(struct dlb2_hw_dev *handle, +struct dlb2_create_sched_domain_args *args); + +extern void (*dlb2_iface_domain_reset)(struct dlb2_eventdev *dlb2); +#endif /* _DLB2_IFACE_H_ */
drivers/event/dlb2/pf/base/dlb2_resource.c +3234 -0
2,631 lines not on this page · raw message
drivers/event/dlb2/pf/dlb2_pf.c +44 -0
44 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.6.4
Part of the diff is not shown here. View the raw message.
PATCH 6/22 event/dlb2: add probe
In reply to Timothy McDanielThe DLB2 hardware is a PCI device. This commit adds support for probe and other initialization. The dlb2_iface.[ch] files implement a flexible interface that supports both the PF PMD and the bifurcated PMD. The bifurcated PMD will be released in a future patch set. Note that the flexible interface is only used for configuration, and is not used in the data path. The shared code is added in pf/base.
--- drivers/event/dlb2/dlb2.c | 557 ++++++ drivers/event/dlb2/dlb2_iface.c | 42 + drivers/event/dlb2/dlb2_iface.h | 29 + drivers/event/dlb2/dlb2_priv.h | 5 + drivers/event/dlb2/meson.build | 6 +- drivers/event/dlb2/pf/base/dlb2_hw_types.h | 367 ++++ drivers/event/dlb2/pf/base/dlb2_mbox.h | 596 ++++++ drivers/event/dlb2/pf/base/dlb2_osdep.h | 248 +++ drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h | 447 +++++ drivers/event/dlb2/pf/base/dlb2_osdep_list.h | 131 ++ drivers/event/dlb2/pf/base/dlb2_osdep_types.h | 31 + drivers/event/dlb2/pf/base/dlb2_regs.h | 2527 ++++++++++++++++++++++++ drivers/event/dlb2/pf/base/dlb2_resource.c | 274 +++ drivers/event/dlb2/pf/base/dlb2_resource.h | 1913 ++++++++++++++++++ drivers/event/dlb2/pf/dlb2_main.c | 620 ++++++ drivers/event/dlb2/pf/dlb2_main.h | 107 + drivers/event/dlb2/pf/dlb2_pf.c | 251 +++ 17 files changed, 8150 insertions(+), 1 deletion(-) create mode 100644 drivers/event/dlb2/dlb2.c create mode 100644 drivers/event/dlb2/dlb2_iface.c create mode 100644 drivers/event/dlb2/dlb2_iface.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_hw_types.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_mbox.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_list.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_types.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_regs.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.c create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.h create mode 100644 drivers/event/dlb2/pf/dlb2_main.c create mode 100644 drivers/event/dlb2/pf/dlb2_main.h create mode 100644 drivers/event/dlb2/pf/dlb2_pf.c
drivers/event/dlb2/dlb2.c new file +557 -0
@@ -0,0 +1,557 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#include <assert.h> +#include <errno.h> +#include <nmmintrin.h> +#include <pthread.h> +#include <stdint.h> +#include <stdbool.h> +#include <stdio.h> +#include <string.h> +#include <sys/mman.h> +#include <sys/fcntl.h> + +#include <rte_common.h> +#include <rte_config.h> +#include <rte_cycles.h> +#include <rte_debug.h> +#include <rte_dev.h> +#include <rte_errno.h> +#include <rte_eventdev.h> +#include <rte_eventdev_pmd.h> +#include <rte_kvargs.h> +#include <rte_io.h> +#include <rte_log.h> +#include <rte_malloc.h> +#include <rte_mbuf.h> +#include <rte_prefetch.h> +#include <rte_ring.h> +#include <rte_string_fns.h> + +#include "dlb2_priv.h" +#include "dlb2_iface.h" +#include "dlb2_inline_fns.h" + +#if !definedRTE_ARCH_X86_64 +#error "This implementation only supports RTE_ARCH_X86_64 architecture." +#endif + +/* +* Resources exposed to eventdev. Some values overridden at runtime using +* values returned by the DLB kernel driver. +*/ +#if (RTE_EVENT_MAX_QUEUES_PER_DEV >UINT8_MAX ) +#error "RTE_EVENT_MAX_QUEUES_PER_DEV cannot fit in member max_event_queues" +#endif +static struct rte_event_dev_info evdev_dlb2_default_info = { + .driver_name ="" ,/* probe will set */ + .min_dequeue_timeout_ns =DLB2_MIN_DEQUEUE_TIMEOUT_NS , + .max_dequeue_timeout_ns =DLB2_MAX_DEQUEUE_TIMEOUT_NS , +#if (RTE_EVENT_MAX_QUEUES_PER_DEV < DLB2_MAX_NUM_LDB_QUEUES) + .max_event_queues =RTE_EVENT_MAX_QUEUES_PER_DEV , +#else + .max_event_queues =DLB2_MAX_NUM_LDB_QUEUES , +#endif + .max_event_queue_flows =DLB2_MAX_NUM_FLOWS , + .max_event_queue_priority_levels =DLB2_QID_PRIORITIES , + .max_event_priority_levels =DLB2_QID_PRIORITIES , + .max_event_ports =DLB2_MAX_NUM_LDB_PORTS , + .max_event_port_dequeue_depth =DLB2_MAX_CQ_DEPTH , + .max_event_port_enqueue_depth =DLB2_MAX_ENQUEUE_DEPTH , + .max_event_port_links =DLB2_MAX_NUM_QIDS_PER_LDB_CQ , + .max_num_events =DLB2_MAX_NUM_LDB_CREDITS , + .max_single_link_event_port_queue_pairs =DLB2_MAX_NUM_DIR_PORTS , + .event_dev_cap = (RTE_EVENT_DEV_CAP_QUEUE_QOS | +RTE_EVENT_DEV_CAP_EVENT_QOS | +RTE_EVENT_DEV_CAP_BURST_MODE | +RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED | +RTE_EVENT_DEV_CAP_IMPLICIT_RELEASE_DISABLE | +RTE_EVENT_DEV_CAP_QUEUE_ALL_TYPES ), +}; + +/* These functions will vary based on processor capabilities */ +static struct dlb2_port_low_level_io_functions qm_mmio_fns; + +struct process_local_port_data +dlb2_port[DLB2_MAX_NUM_PORTS ][DLB2_NUM_PORT_TYPES ]; + +/* override defaults with value(s) provided on command line */ +static void +dlb2_init_queue_depth_thresholds (struct dlb2_eventdev *dlb2, +int *qid_depth_thresholds) +{ +int q; + +for (q =0 ; q <DLB2_MAX_NUM_QUEUES ; q++) { +if (qid_depth_thresholds[q] !=0 ) + dlb2->ev_queues[q].depth_threshold = + qid_depth_thresholds[q]; + } +} + +static int +dlb2_hw_query_resources (struct dlb2_eventdev *dlb2) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_hw_resource_info *dlb2_info = &handle->info; +int ret; + +/* Query driver resources provisioned for this VF */ + + ret =dlb2_iface_get_num_resources (handle, + &dlb2->hw_rsrc_query_results); +if (ret) { +DLB2_LOG_ERR ("ioctl get dlb2 num resources, err=%d\n" , + ret); +return ret; + } + +/* Complete filling in device resource info returned to evdev app, +* overriding any default values. +* The capabilities (CAPs) were set at compile time. +*/ + + evdev_dlb2_default_info.max_event_queues = + dlb2->hw_rsrc_query_results.num_ldb_queues; + + evdev_dlb2_default_info.max_event_ports = + dlb2->hw_rsrc_query_results.num_ldb_ports; + + evdev_dlb2_default_info.max_num_events = + dlb2->hw_rsrc_query_results.num_ldb_credits; + +/* Save off values used when creating the scheduling domain. */ + + handle->info.num_sched_domains = + dlb2->hw_rsrc_query_results.num_sched_domains; + + handle->info.hw_rsrc_max.nb_events_limit = + dlb2->hw_rsrc_query_results.num_ldb_credits; + + handle->info.hw_rsrc_max.num_queues = + dlb2->hw_rsrc_query_results.num_ldb_queues + + dlb2->hw_rsrc_query_results.num_dir_ports; + + handle->info.hw_rsrc_max.num_ldb_queues = + dlb2->hw_rsrc_query_results.num_ldb_queues; + + handle->info.hw_rsrc_max.num_ldb_ports = + dlb2->hw_rsrc_query_results.num_ldb_ports; + + handle->info.hw_rsrc_max.num_dir_ports = + dlb2->hw_rsrc_query_results.num_dir_ports; + + handle->info.hw_rsrc_max.reorder_window_size = + dlb2->hw_rsrc_query_results.num_hist_list_entries; + +rte_memcpy (dlb2_info, &handle->info.hw_rsrc_max,sizeof (*dlb2_info)); + +return 0 ; +} + +static void +dlb2_qm_mmio_fn_init (void ) +{ +/* Process-local function pointers for performing low level port i/o */ + +if (rte_cpu_get_flag_enabled (RTE_CPUFLAG_MOVDIR64B )) + qm_mmio_fns.pp_enqueue_four = dlb2_movdir64b; +else + qm_mmio_fns.pp_enqueue_four = dlb2_movntdq; +} + +#define RTE_BASE_10 10 + +int dlb2_string_to_int (int *result,const char *str) +{ +long ret; + +if (str ==NULL || result ==NULL ) +return -EINVAL ; + + errno =0 ; + ret =strtol (str,NULL ,RTE_BASE_10 ); +if (errno) +return -errno; + +/* long int and int may be different width for some architectures */ +if (ret <INT_MIN || ret >INT_MAX ) +return -EINVAL ; + + *result = ret; +return 0 ; +} + +static int +set_numa_node(const char *key __rte_unused, const char *value, void *opaque) +{ +int *socket_id = opaque; +int ret; + + ret =dlb2_string_to_int (socket_id, value); +if (ret <0 ) +return ret; + +if (*socket_id >RTE_MAX_NUMA_NODES ) +return -EINVAL ; +return 0 ; +} + +static int +set_max_num_events(const char *key __rte_unused, +const char *value, +void *opaque) +{ +int *max_num_events = opaque; +int ret; + +if (value ==NULL || opaque ==NULL ) { +DLB2_LOG_ERR ("NULL pointer\n" ); +return -EINVAL ; + } + + ret =dlb2_string_to_int (max_num_events, value); +if (ret <0 ) +return ret; + + +if (*max_num_events <0 || *max_num_events > +DLB2_MAX_NUM_LDB_CREDITS ) { +DLB2_LOG_ERR ("dlb2: max_num_events must be between 0 and %d\n" , +DLB2_MAX_NUM_LDB_CREDITS ); +return -EINVAL ; + } + +return 0 ; +} + +static int +set_num_dir_credits(const char *key __rte_unused, +const char *value, +void *opaque) +{ +int *num_dir_credits = opaque; +int ret; + +if (value ==NULL || opaque ==NULL ) { +DLB2_LOG_ERR ("NULL pointer\n" ); +return -EINVAL ; + } + + ret =dlb2_string_to_int (num_dir_credits, value); +if (ret <0 ) +return ret; + +if (*num_dir_credits <0 || + *num_dir_credits >DLB2_MAX_NUM_DIR_CREDITS ) { +DLB2_LOG_ERR ("dlb2: num_dir_credits must be between 0 and %d\n" , +DLB2_MAX_NUM_DIR_CREDITS ); +return -EINVAL ; + } + +return 0 ; +} + +static int +set_dev_id(const char *key __rte_unused, +const char *value, +void *opaque) +{ +int *dev_id = opaque; +int ret; + +if (value ==NULL || opaque ==NULL ) { +DLB2_LOG_ERR ("NULL pointer\n" ); +return -EINVAL ; + } + + ret =dlb2_string_to_int (dev_id, value); +if (ret <0 ) +return ret; + +return 0 ; +} + +static int +set_cos(const char *key __rte_unused, +const char *value, +void *opaque) +{ +enum dlb2_cos *cos_id = opaque; +int x =0 ; +int ret; + +if (value ==NULL || opaque ==NULL ) { +DLB2_LOG_ERR ("NULL pointer\n" ); +return -EINVAL ; + } + + ret =dlb2_string_to_int (&x, value); +if (ret <0 ) +return ret; + +if (x !=DLB2_COS_DEFAULT && (x <DLB2_COS_0 || x >DLB2_COS_3 )) { +DLB2_LOG_ERR ("COS %d out of range, must be 0-3\n" , x); +return -EINVAL ; + } + + *cos_id = x; + +return 0 ; +} + + +static int +set_qid_depth_thresh(const char *key __rte_unused, +const char *value, +void *opaque) +{ +struct dlb2_qid_depth_thresholds *qid_thresh = opaque; +int first, last, thresh, i; + +if (value ==NULL || opaque ==NULL ) { +DLB2_LOG_ERR ("NULL pointer\n" ); +return -EINVAL ; + } + +/* command line override may take one of the following 3 forms: +* qid_depth_thresh=all:<threshold_value> ... all queues +* qid_depth_thresh=qidA-qidB:<threshold_value> ... a range of queues +* qid_depth_thresh=qid:<threshold_value> ... just one queue +*/ +if (sscanf (value,"all:%d" , &thresh) ==1 ) { + first =0 ; + last =DLB2_MAX_NUM_QUEUES -1 ; + }else if (sscanf (value,"%d-%d:%d" , &first, &last, &thresh) ==3 ) { +/* we have everything we need */ + }else if (sscanf (value,"%d:%d" , &first, &thresh) ==2 ) { + last = first; + }else { +DLB2_LOG_ERR ("Error parsing qid depth vdev arg. Should be all:val, qid-qid:val, or qid:val\n" ); +return -EINVAL ; + } + +if (first > last || first <0 || last >=DLB2_MAX_NUM_QUEUES ) { +DLB2_LOG_ERR ("Error parsing qid depth vdev arg, invalid qid value\n" ); +return -EINVAL ; + } + +if (thresh <0 || thresh >DLB2_MAX_QUEUE_DEPTH_THRESHOLD ) { +DLB2_LOG_ERR ("Error parsing qid depth vdev arg, threshold > %d\n" , +DLB2_MAX_QUEUE_DEPTH_THRESHOLD ); +return -EINVAL ; + } + +for (i = first; i <= last; i++) + qid_thresh->val[i] = thresh;/* indexed by qid */ + +return 0 ; +} + +static void +dlb2_entry_points_init (struct rte_eventdev *dev) +{ +RTE_SET_USED (dev); + +/* Eventdev PMD entry points */ +} + +int +dlb2_primary_eventdev_probe (struct rte_eventdev *dev, +const char *name, +struct dlb2_devargs *dlb2_args) +{ +struct dlb2_eventdev *dlb2; +int err, i; + + dlb2 = dev->data->dev_private; + + dlb2->event_dev = dev;/* backlink */ + + evdev_dlb2_default_info.driver_name = name; + + dlb2->max_num_events_override = dlb2_args->max_num_events; + dlb2->num_dir_credits_override = dlb2_args->num_dir_credits_override; + dlb2->qm_instance.device_path_id = dlb2_args->dev_id; + dlb2->qm_instance.cos_id = dlb2_args->cos_id; + +/* Open the interface. +* For vdev mode, this means open the dlb2 kernel module. +*/ + err =dlb2_iface_open (&dlb2->qm_instance, name); +if (err <0 ) { +DLB2_LOG_ERR ("could not open event hardware device, err=%d\n" , + err); +return err; + } + + err =dlb2_iface_get_device_version (&dlb2->qm_instance, + &dlb2->revision); +if (err <0 ) { +DLB2_LOG_ERR ("dlb2: failed to get the device version, err=%d\n" , + err); +return err; + } + + err =dlb2_hw_query_resources (dlb2); +if (err) { +DLB2_LOG_ERR ("get resources err=%d for %s\n" , + err, name); +return err; + } + +dlb2_iface_hardware_init (&dlb2->qm_instance); + + err =dlb2_iface_get_cq_poll_mode (&dlb2->qm_instance, &dlb2->poll_mode); +if (err <0 ) { +DLB2_LOG_ERR ("dlb2: failed to get the poll mode, err=%d\n" , + err); +return err; + } + +/* Initialize each port's token pop mode */ +for (i =0 ; i <DLB2_MAX_NUM_PORTS ; i++) + dlb2->ev_ports[i].qm_port.token_pop_mode =AUTO_POP ; + +rte_spinlock_init (&dlb2->qm_instance.resource_lock); + +dlb2_qm_mmio_fn_init (); + +dlb2_iface_low_level_io_init (); + +dlb2_entry_points_init (dev); + +dlb2_init_queue_depth_thresholds (dlb2, + dlb2_args->qid_depth_thresholds.val); + +return 0 ; +} + +int +dlb2_secondary_eventdev_probe (struct rte_eventdev *dev, +const char *name) +{ +struct dlb2_eventdev *dlb2; +int err; + + dlb2 = dev->data->dev_private; + + evdev_dlb2_default_info.driver_name = name; + + err =dlb2_iface_open (&dlb2->qm_instance, name); +if (err <0 ) { +DLB2_LOG_ERR ("could not open event hardware device, err=%d\n" , + err); +return err; + } + + err =dlb2_hw_query_resources (dlb2); +if (err) { +DLB2_LOG_ERR ("get resources err=%d for %s\n" , + err, name); +return err; + } + +dlb2_qm_mmio_fn_init (); + +dlb2_iface_low_level_io_init (); + +dlb2_entry_points_init (dev); + +return 0 ; +} + +int +dlb2_parse_params (const char *params, +const char *name, +struct dlb2_devargs *dlb2_args) +{ +int ret =0 ; +static const char *const args[] = {NUMA_NODE_ARG , +DLB2_MAX_NUM_EVENTS , +DLB2_NUM_DIR_CREDITS , +DEV_ID_ARG , +DLB2_QID_DEPTH_THRESH_ARG , +DLB2_COS_ARG , +NULL }; + +if (params !=NULL && params[0 ] !='\0' ) { +struct rte_kvargs *kvlist =rte_kvargs_parse (params, args); + +if (!kvlist) { +RTE_LOG (INFO ,PMD , +"Ignoring unsupported parameters when creating device '%s'\n" , + name); + }else { +int ret =rte_kvargs_process (kvlist,NUMA_NODE_ARG , + set_numa_node, + &dlb2_args->socket_id); +if (ret !=0 ) { +DLB2_LOG_ERR ("%s: Error parsing numa node parameter" , + name); +rte_kvargs_free (kvlist); +return ret; + } + + ret =rte_kvargs_process (kvlist,DLB2_MAX_NUM_EVENTS , + set_max_num_events, + &dlb2_args->max_num_events); +if (ret !=0 ) { +DLB2_LOG_ERR ("%s: Error parsing max_num_events parameter" , + name); +rte_kvargs_free (kvlist); +return ret; + } + + ret =rte_kvargs_process (kvlist, +DLB2_NUM_DIR_CREDITS , + set_num_dir_credits, + &dlb2_args->num_dir_credits_override); +if (ret !=0 ) { +DLB2_LOG_ERR ("%s: Error parsing num_dir_credits parameter" , + name); +rte_kvargs_free (kvlist); +return ret; + } + + ret =rte_kvargs_process (kvlist,DEV_ID_ARG , + set_dev_id, + &dlb2_args->dev_id); +if (ret !=0 ) { +DLB2_LOG_ERR ("%s: Error parsing dev_id parameter" , + name); +rte_kvargs_free (kvlist); +return ret; + } + + ret =rte_kvargs_process ( + kvlist, +DLB2_QID_DEPTH_THRESH_ARG , + set_qid_depth_thresh, + &dlb2_args->qid_depth_thresholds); +if (ret !=0 ) { +DLB2_LOG_ERR ("%s: Error parsing qid_depth_thresh parameter" , + name); +rte_kvargs_free (kvlist); +return ret; + } + + ret =rte_kvargs_process (kvlist,DLB2_COS_ARG , + set_cos, + &dlb2_args->cos_id); +if (ret !=0 ) { +DLB2_LOG_ERR ("%s: Error parsing cos parameter" , + name); +rte_kvargs_free (kvlist); +return ret; + } + +rte_kvargs_free (kvlist); + } + } +return ret; +} +RTE_LOG_REGISTER (eventdev_dlb2_log_level, pmd.event.dlb2,NOTICE ); +
drivers/event/dlb2/dlb2_iface.c new file +42 -0
42 lines not on this page · raw message
drivers/event/dlb2/dlb2_iface.h new file +29 -0
@@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef _DLB2_IFACE_H_ +#define _DLB2_IFACE_H_ + +/* DLB2 PMD Internal interface function pointers. +* If VDEV (bifurcated PMD), these will resolve to functions that issue ioctls +* serviced by DLB kernel module. +* If PCI (PF PMD), these will be implemented locally in user mode. +*/ + +extern void (*dlb2_iface_low_level_io_init)(void ); + +extern int (*dlb2_iface_open)(struct dlb2_hw_dev *handle,const char *name); + +extern int (*dlb2_iface_get_device_version)(struct dlb2_hw_dev *handle, +uint8_t *revision); + +extern void (*dlb2_iface_hardware_init)(struct dlb2_hw_dev *handle); + +extern int (*dlb2_iface_get_cq_poll_mode)(struct dlb2_hw_dev *handle, +enum dlb2_cq_poll_modes *mode); + +extern int (*dlb2_iface_get_num_resources)(struct dlb2_hw_dev *handle, +struct dlb2_get_num_resources_args *rsrcs); + +#endif /* _DLB2_IFACE_H_ */
drivers/event/dlb2/dlb2_priv.h +5 -0
@@ -611,4 +611,9 @@ int dlb2_parse_params(const char *params, struct dlb2_devargs *dlb2_args);int dlb2_string_to_int (int *result,const char *str); + +/* Extern globals */ + +extern struct process_local_port_data dlb2_port[][DLB2_NUM_PORT_TYPES ]; +#endif /* _DLB2_PRIV_H_ */
drivers/event/dlb2/meson.build +5 -1
@@ -1,7 +1,11 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019-2020 Intel Corporation -sources = files( +sources = files('dlb2.c', + 'dlb2_iface.c', + 'pf/dlb2_main.c', + 'pf/dlb2_pf.c', + 'pf/base/dlb2_resource.c' ) deps += ['mbuf', 'mempool', 'ring', 'bus_vdev', 'pci', 'bus_pci']
drivers/event/dlb2/pf/base/dlb2_hw_types.h new file +367 -0
367 lines not on this page · raw message
drivers/event/dlb2/pf/base/dlb2_mbox.h new file +596 -0
596 lines not on this page · raw message
drivers/event/dlb2/pf/base/dlb2_osdep.h new file +248 -0
248 lines not on this page · raw message
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +447 -0
447 lines not on this page · raw message
drivers/event/dlb2/pf/base/dlb2_osdep_list.h new file +131 -0
131 lines not on this page · raw message
drivers/event/dlb2/pf/base/dlb2_osdep_types.h new file +31 -0
@@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef __DLB2_OSDEP_TYPES_H +#define __DLB2_OSDEP_TYPES_H + +#include <linux/types.h> + +#include <inttypes.h> +#include <ctype.h> +#include <stdint.h> +#include <stdbool.h> +#include <string.h> +#include <unistd.h> +#include <errno.h> + +/* Types for user mode PF PMD */ +typedef uint8_t u8 ; +typedef int8_t s8 ; +typedef uint16_t u16 ; +typedef int16_t s16 ; +typedef uint32_t u32 ; +typedef int32_t s32 ; +typedef uint64_t u64 ; + +#define __iomem + +/* END types for user mode PF PMD */ + +#endif /* __DLB2_OSDEP_TYPES_H */
drivers/event/dlb2/pf/base/dlb2_regs.h new file +2527 -0
531 lines not on this page · raw message
drivers/event/dlb2/pf/base/dlb2_resource.c new file +274 -0
274 lines not on this page · raw message
drivers/event/dlb2/pf/base/dlb2_resource.h new file +1913 -0
1,913 lines not on this page · raw message
drivers/event/dlb2/pf/dlb2_main.c new file +620 -0
620 lines not on this page · raw message
drivers/event/dlb2/pf/dlb2_main.h new file +107 -0
107 lines not on this page · raw message
drivers/event/dlb2/pf/dlb2_pf.c new file +251 -0
251 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.6.4
Part of the diff is not shown here. View the raw message.
PATCH 9/22 event/dlb2: add queue and port default conf
In reply to Timothy McDanielAdd support for getting the queue and port default configuration.
--- drivers/event/dlb2/dlb2.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)
drivers/event/dlb2/dlb2.c +33 -0
@@ -698,12 +698,45 @@ dlb2_eventdev_configure(const struct rte_eventdev *dev) }static void +dlb2_eventdev_port_default_conf_get (struct rte_eventdev *dev, +uint8_t port_id, +struct rte_event_port_conf *port_conf) +{ +RTE_SET_USED (port_id); +struct dlb2_eventdev *dlb2; + + dlb2 =dlb2_pmd_priv (dev); + +/* FIXME: Arbitrary values */ + port_conf->new_event_threshold = dlb2->new_event_limit; + port_conf->dequeue_depth =32 ; + port_conf->enqueue_depth =DLB2_MAX_ENQUEUE_DEPTH ; + port_conf->event_port_cfg =0 ; +} + +static void +dlb2_eventdev_queue_default_conf_get (struct rte_eventdev *dev, +uint8_t queue_id, +struct rte_event_queue_conf *queue_conf) +{ +RTE_SET_USED (dev); +RTE_SET_USED (queue_id); +/* FIXME: Arbitrary values */ + queue_conf->nb_atomic_flows =1024 ; + queue_conf->nb_atomic_order_sequences =64 ; + queue_conf->event_queue_cfg =0 ; + queue_conf->priority =0 ; +} + +static void dlb2_entry_points_init (struct rte_eventdev *dev) {/* Expose PMD's eventdev interface */ static struct rte_eventdev_ops dlb2_eventdev_entry_ops = { .dev_infos_get = dlb2_eventdev_info_get, .dev_configure = dlb2_eventdev_configure, + .queue_def_conf = dlb2_eventdev_queue_default_conf_get, + .port_def_conf = dlb2_eventdev_port_default_conf_get, .dump = dlb2_eventdev_dump, .xstats_get = dlb2_eventdev_xstats_get, .xstats_get_names = dlb2_eventdev_xstats_get_names,
Signature · 1 line
2.6.4
PATCH 10/22 event/dlb2: add queue setup
In reply to Timothy McDanielLoad balanced (ldb) queues are setup here. Directed queues are not set up until link time, at which point we know the directed port ID. Directed queue setup will only fail if this queue is already setup or there are no directed queues left to configure.
--- drivers/event/dlb2/dlb2.c | 316 ++++++++++++++++++++ drivers/event/dlb2/dlb2_iface.c | 12 + drivers/event/dlb2/dlb2_iface.h | 12 + drivers/event/dlb2/pf/base/dlb2_resource.c | 464 +++++++++++++++++++++++++++++ drivers/event/dlb2/pf/dlb2_main.c | 10 + drivers/event/dlb2/pf/dlb2_pf.c | 82 +++++ 6 files changed, 896 insertions(+)
drivers/event/dlb2/dlb2.c +316 -0
@@ -728,6 +728,321 @@ dlb2_eventdev_queue_default_conf_get(struct rte_eventdev *dev, queue_conf->priority =0 ; } +static int32_t +dlb2_get_sn_allocation (struct dlb2_eventdev *dlb2,int group) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_get_sn_allocation_args cfg; +int ret; + + cfg.group = group; + + ret =dlb2_iface_get_sn_allocation (handle, &cfg); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: get_sn_allocation ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); +return ret; + } + +return cfg.response.id; +} + +static int +dlb2_set_sn_allocation (struct dlb2_eventdev *dlb2,int group,int num) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_set_sn_allocation_args cfg; +int ret; + + cfg.num = num; + cfg.group = group; + + ret =dlb2_iface_set_sn_allocation (handle, &cfg); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: set_sn_allocation ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); +return ret; + } + +return ret; +} + +static int32_t +dlb2_get_sn_occupancy (struct dlb2_eventdev *dlb2,int group) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_get_sn_occupancy_args cfg; +int ret; + + cfg.group = group; + + ret =dlb2_iface_get_sn_occupancy (handle, &cfg); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: get_sn_occupancy ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); +return ret; + } + +return cfg.response.id; +} + +/* Query the current sequence number allocations and, if they conflict with the +* requested LDB queue configuration, attempt to re-allocate sequence numbers. +* This is best-effort; if it fails, the PMD will attempt to configure the +* load-balanced queue and return an error. +*/ +static void +dlb2_program_sn_allocation (struct dlb2_eventdev *dlb2, +const struct rte_event_queue_conf *queue_conf) +{ +int grp_occupancy[DLB2_NUM_SN_GROUPS ]; +int grp_alloc[DLB2_NUM_SN_GROUPS ]; +int i, sequence_numbers; + + sequence_numbers = (int )queue_conf->nb_atomic_order_sequences; + +for (i =0 ; i <DLB2_NUM_SN_GROUPS ; i++) { +int total_slots; + + grp_alloc[i] =dlb2_get_sn_allocation (dlb2, i); +if (grp_alloc[i] <0 ) +return ; + + total_slots =DLB2_MAX_LDB_SN_ALLOC / grp_alloc[i]; + + grp_occupancy[i] =dlb2_get_sn_occupancy (dlb2, i); +if (grp_occupancy[i] <0 ) +return ; + +/* DLB has at least one available slot for the requested +* sequence numbers, so no further configuration required. +*/ +if (grp_alloc[i] == sequence_numbers && + grp_occupancy[i] < total_slots) +return ; + } + +/* None of the sequence number groups are configured for the requested +* sequence numbers, so we have to reconfigure one of them. This is +* only possible if a group is not in use. +*/ +for (i =0 ; i <DLB2_NUM_SN_GROUPS ; i++) { +if (grp_occupancy[i] ==0 ) +break ; + } + +if (i ==DLB2_NUM_SN_GROUPS ) { +printf ("[%s()] No groups with %d sequence_numbers are available or have free slots\n" , + __func__, sequence_numbers); +return ; + } + +/* Attempt to configure slot i with the requested number of sequence +* numbers. Ignore the return value -- if this fails, the error will be +* caught during subsequent queue configuration. +*/ +dlb2_set_sn_allocation (dlb2, i, sequence_numbers); +} + +static int32_t +dlb2_hw_create_ldb_queue (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *ev_queue, +const struct rte_event_queue_conf *evq_conf) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_queue *queue = &ev_queue->qm_queue; +struct dlb2_create_ldb_queue_args cfg; +int32_t ret; +uint32_t qm_qid; +int sched_type =-1 ; + +if (evq_conf ==NULL ) +return -EINVAL ; + +if (evq_conf->event_queue_cfg &RTE_EVENT_QUEUE_CFG_ALL_TYPES ) { +if (evq_conf->nb_atomic_order_sequences !=0 ) + sched_type =RTE_SCHED_TYPE_ORDERED ; +else + sched_type =RTE_SCHED_TYPE_PARALLEL ; + }else { + sched_type = evq_conf->schedule_type; + } + + cfg.num_atomic_inflights =DLB2_NUM_ATOMIC_INFLIGHTS_PER_QUEUE ; + cfg.num_sequence_numbers = evq_conf->nb_atomic_order_sequences; + cfg.num_qid_inflights = evq_conf->nb_atomic_order_sequences; + +if (sched_type !=RTE_SCHED_TYPE_ORDERED ) { + cfg.num_sequence_numbers =0 ; + cfg.num_qid_inflights =2048 ; + } + +/* App should set this to the number of hardware flows they want, not +* the overall number of flows they're going to use. E.g. if app is +* using 64 flows and sets compression to 64, best-case they'll get +* 64 unique hashed flows in hardware. +*/ +switch (evq_conf->nb_atomic_flows) { +/* Valid DLB2 compression levels */ +case 64 : +case 128 : +case 256 : +case 512 : +case (1 *1024 ):/* 1K */ +case (2 *1024 ):/* 2K */ +case (4 *1024 ):/* 4K */ +case (64 *1024 ):/* 64K */ + cfg.lock_id_comp_level = evq_conf->nb_atomic_flows; +break ; +default : +/* Invalid compression level */ + cfg.lock_id_comp_level =0 ;/* no compression */ + } + +if (ev_queue->depth_threshold ==0 ) { + cfg.depth_threshold =RTE_PMD_DLB2_DEFAULT_DEPTH_THRESH ; + ev_queue->depth_threshold =RTE_PMD_DLB2_DEFAULT_DEPTH_THRESH ; + }else { + cfg.depth_threshold = ev_queue->depth_threshold; + } + + ret =dlb2_iface_ldb_queue_create (handle, &cfg); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: create LB event queue error, ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); +return -EINVAL ; + } + + qm_qid = cfg.response.id; + +/* Save off queue config for debug, resource lookups, and reconfig */ + queue->num_qid_inflights = cfg.num_qid_inflights; + queue->num_atm_inflights = cfg.num_atomic_inflights; + + queue->sched_type = sched_type; + queue->config_state =DLB2_CONFIGURED ; + +DLB2_LOG_DBG ("Created LB event queue %d, nb_inflights=%d, nb_seq=%d, qid inflights=%d\n" , + qm_qid, + cfg.num_atomic_inflights, + cfg.num_sequence_numbers, + cfg.num_qid_inflights); + +return qm_qid; +} + +static int +dlb2_eventdev_ldb_queue_setup (struct rte_eventdev *dev, +struct dlb2_eventdev_queue *ev_queue, +const struct rte_event_queue_conf *queue_conf) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +int32_t qm_qid; + +if (queue_conf->nb_atomic_order_sequences) +dlb2_program_sn_allocation (dlb2, queue_conf); + + qm_qid =dlb2_hw_create_ldb_queue (dlb2, + ev_queue, + queue_conf); +if (qm_qid <0 ) { +DLB2_LOG_ERR ("Failed to create the load-balanced queue\n" ); + +return qm_qid; + } + + dlb2->qm_ldb_to_ev_queue_id[qm_qid] = ev_queue->id; + + ev_queue->qm_queue.id = qm_qid; + +return 0 ; +} + +static int dlb2_num_dir_queues_setup (struct dlb2_eventdev *dlb2) +{ +int i, num =0 ; + +for (i =0 ; i < dlb2->num_queues; i++) { +if (dlb2->ev_queues[i].setup_done && + dlb2->ev_queues[i].qm_queue.is_directed) + num++; + } + +return num; +} + +static void +dlb2_queue_link_teardown (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *ev_queue) +{ +struct dlb2_eventdev_port *ev_port; +int i, j; + +for (i =0 ; i < dlb2->num_ports; i++) { + ev_port = &dlb2->ev_ports[i]; + +for (j =0 ; j <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; j++) { +if (!ev_port->link[j].valid || + ev_port->link[j].queue_id != ev_queue->id) +continue ; + + ev_port->link[j].valid = false; + ev_port->num_links--; + } + } + + ev_queue->num_links =0 ; +} + +static int +dlb2_eventdev_queue_setup (struct rte_eventdev *dev, +uint8_t ev_qid, +const struct rte_event_queue_conf *queue_conf) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +struct dlb2_eventdev_queue *ev_queue; +int ret; + +if (!queue_conf) +return -EINVAL ; + +if (ev_qid >= dlb2->num_queues) +return -EINVAL ; + + ev_queue = &dlb2->ev_queues[ev_qid]; + + ev_queue->qm_queue.is_directed = queue_conf->event_queue_cfg & +RTE_EVENT_QUEUE_CFG_SINGLE_LINK ; + ev_queue->id = ev_qid; + ev_queue->conf = *queue_conf; + +if (!ev_queue->qm_queue.is_directed) { + ret =dlb2_eventdev_ldb_queue_setup (dev, ev_queue, queue_conf); + }else { +/* The directed queue isn't setup until link time, at which +* point we know its directed port ID. Directed queue setup +* will only fail if this queue is already setup or there are +* no directed queues left to configure. +*/ + ret =0 ; + + ev_queue->qm_queue.config_state =DLB2_NOT_CONFIGURED ; + +if (ev_queue->setup_done || +dlb2_num_dir_queues_setup (dlb2) == dlb2->num_dir_queues) + ret = -EINVAL ; + } + +/* Tear down pre-existing port->queue links */ +if (!ret && dlb2->run_state ==DLB2_RUN_STATE_STOPPED ) +dlb2_queue_link_teardown (dlb2, ev_queue); + +if (!ret) + ev_queue->setup_done = true; + +return ret; +} +static void dlb2_entry_points_init (struct rte_eventdev *dev) {
@@ -736,6 +1051,7 @@ dlb2_entry_points_init(struct rte_eventdev *dev) .dev_infos_get = dlb2_eventdev_info_get, .dev_configure = dlb2_eventdev_configure, .queue_def_conf = dlb2_eventdev_queue_default_conf_get, + .queue_setup = dlb2_eventdev_queue_setup, .port_def_conf = dlb2_eventdev_port_default_conf_get, .dump = dlb2_eventdev_dump, .xstats_get = dlb2_eventdev_xstats_get,
drivers/event/dlb2/dlb2_iface.c +12 -0
@@ -45,3 +45,15 @@ int (*dlb2_iface_sched_domain_create)(struct dlb2_hw_dev *handle, struct dlb2_create_sched_domain_args *args);void (*dlb2_iface_domain_reset)(struct dlb2_eventdev *dlb2); + +int (*dlb2_iface_ldb_queue_create)(struct dlb2_hw_dev *handle, +struct dlb2_create_ldb_queue_args *cfg); + +int (*dlb2_iface_get_sn_allocation)(struct dlb2_hw_dev *handle, +struct dlb2_get_sn_allocation_args *args); + +int (*dlb2_iface_set_sn_allocation)(struct dlb2_hw_dev *handle, +struct dlb2_set_sn_allocation_args *args); + +int (*dlb2_iface_get_sn_occupancy)(struct dlb2_hw_dev *handle, +struct dlb2_get_sn_occupancy_args *args);
drivers/event/dlb2/dlb2_iface.h +12 -0
@@ -31,4 +31,16 @@ extern int (*dlb2_iface_sched_domain_create)(struct dlb2_hw_dev *handle,extern void (*dlb2_iface_domain_reset)(struct dlb2_eventdev *dlb2); +extern int (*dlb2_iface_ldb_queue_create)(struct dlb2_hw_dev *handle, +struct dlb2_create_ldb_queue_args *cfg); + +extern int (*dlb2_iface_get_sn_allocation)(struct dlb2_hw_dev *handle, +struct dlb2_get_sn_allocation_args *args); + +extern int (*dlb2_iface_set_sn_allocation)(struct dlb2_hw_dev *handle, +struct dlb2_set_sn_allocation_args *args); + +extern int (*dlb2_iface_get_sn_occupancy)(struct dlb2_hw_dev *handle, +struct dlb2_get_sn_occupancy_args *args); +#endif /* _DLB2_IFACE_H_ */
drivers/event/dlb2/pf/base/dlb2_resource.c +464 -0
467 lines not on this page · raw message
drivers/event/dlb2/pf/dlb2_main.c +10 -0
@@ -632,3 +632,13 @@ dlb2_pf_reset_domain(struct dlb2_hw *hw, u32 id) {return dlb2_reset_domain (hw, id,NOT_VF_REQ ,PF_ID_ZERO ); } + +int +dlb2_pf_create_ldb_queue (struct dlb2_hw *hw, +u32 id, +struct dlb2_create_ldb_queue_args *args, +struct dlb2_cmd_response *resp) +{ +return dlb2_hw_create_ldb_queue (hw, id, args, resp,NOT_VF_REQ , +PF_ID_ZERO ); +}
drivers/event/dlb2/pf/dlb2_pf.c +82 -0
94 lines not on this page · raw message
Signature · 1 line
2.6.4
PATCH 12/22 event/dlb2: add port link
In reply to Timothy McDanielAdd port link entry point. Directed queues are identified and created at this stage. Their setup deferred until link-time, at which point we know the directed port ID. Directed queue setup will only fail if this queue is already setup or there are no directed queues left to configure.
--- drivers/event/dlb2/dlb2.c | 305 ++++++++++++++ drivers/event/dlb2/dlb2_iface.c | 6 + drivers/event/dlb2/dlb2_iface.h | 6 + drivers/event/dlb2/pf/base/dlb2_resource.c | 633 +++++++++++++++++++++++++++++ drivers/event/dlb2/pf/dlb2_main.c | 10 + drivers/event/dlb2/pf/dlb2_pf.c | 50 +++ 6 files changed, 1010 insertions(+)
drivers/event/dlb2/dlb2.c +305 -0
@@ -1569,6 +1569,310 @@ dlb2_eventdev_port_setup(struct rte_eventdev *dev,return 0 ; } +static int16_t +dlb2_hw_map_ldb_qid_to_port (struct dlb2_hw_dev *handle, +uint32_t qm_port_id, +uint16_t qm_qid, +uint8_t priority) +{ +struct dlb2_map_qid_args cfg; +int32_t ret; + +if (handle ==NULL ) +return -EINVAL ; + +/* Build message */ + cfg.port_id = qm_port_id; + cfg.qid = qm_qid; + cfg.priority =EV_TO_DLB2_PRIO (priority); + + ret =dlb2_iface_map_qid (handle, &cfg); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: map qid error, ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); +DLB2_LOG_ERR ("dlb2: device_id=%d grp=%d, qm_port=%d, qm_qid=%d prio=%d\n" , + handle->device_id, + handle->domain_id, cfg.port_id, + cfg.qid, + cfg.priority); + }else { +DLB2_LOG_DBG ("dlb2: mapped queue %d to qm_port %d\n" , + qm_qid, qm_port_id); + } + +return ret; +} + +static int +dlb2_event_queue_join_ldb (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port, +struct dlb2_eventdev_queue *ev_queue, +uint8_t priority) +{ +int first_avail =-1 ; +int ret, i; + +for (i =0 ; i <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; i++) { +if (ev_port->link[i].valid) { +if (ev_port->link[i].queue_id == ev_queue->id && + ev_port->link[i].priority == priority) { +if (ev_port->link[i].mapped) +return 0 ;/* already mapped */ + first_avail = i; + } + }else if (first_avail ==-1 ) { + first_avail = i; + } + } +if (first_avail ==-1 ) { +DLB2_LOG_ERR ("dlb2: qm_port %d has no available QID slots.\n" , + ev_port->qm_port.id); +return -EINVAL ; + } + + ret =dlb2_hw_map_ldb_qid_to_port (&dlb2->qm_instance, + ev_port->qm_port.id, + ev_queue->qm_queue.id, + priority); + +if (!ret) + ev_port->link[first_avail].mapped = true; + +return ret; +} + +static int32_t +dlb2_hw_create_dir_queue (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *ev_queue, +int32_t qm_port_id) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_create_dir_queue_args cfg; +int32_t ret; + +/* The directed port is always configured before its queue */ + cfg.port_id = qm_port_id; + +if (ev_queue->depth_threshold ==0 ) { + cfg.depth_threshold =RTE_PMD_DLB2_DEFAULT_DEPTH_THRESH ; + ev_queue->depth_threshold =RTE_PMD_DLB2_DEFAULT_DEPTH_THRESH ; + }else { + cfg.depth_threshold = ev_queue->depth_threshold; + } + + ret =dlb2_iface_dir_queue_create (handle, &cfg); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: create DIR event queue error, ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); +return -EINVAL ; + } + +return cfg.response.id; +} + +static int +dlb2_eventdev_dir_queue_setup (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *ev_queue, +struct dlb2_eventdev_port *ev_port) +{ +int32_t qm_qid; + + qm_qid =dlb2_hw_create_dir_queue (dlb2, ev_queue, ev_port->qm_port.id); + +if (qm_qid <0 ) { +DLB2_LOG_ERR ("Failed to create the DIR queue\n" ); +return qm_qid; + } + + dlb2->qm_dir_to_ev_queue_id[qm_qid] = ev_queue->id; + + ev_queue->qm_queue.id = qm_qid; + +return 0 ; +} + +static int +dlb2_do_port_link (struct rte_eventdev *dev, +struct dlb2_eventdev_queue *ev_queue, +struct dlb2_eventdev_port *ev_port, +uint8_t prio) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +int err; + +/* Don't link until start time. */ +if (dlb2->run_state ==DLB2_RUN_STATE_STOPPED ) +return 0 ; + +if (ev_queue->qm_queue.is_directed) + err =dlb2_eventdev_dir_queue_setup (dlb2, ev_queue, ev_port); +else + err =dlb2_event_queue_join_ldb (dlb2, ev_port, ev_queue, prio); + +if (err) { +DLB2_LOG_ERR ("port link failure for %s ev_q %d, ev_port %d\n" , + ev_queue->qm_queue.is_directed ?"DIR" :"LDB" , + ev_queue->id, ev_port->id); + + rte_errno = err; +return -1 ; + } + +return 0 ; +} + +static int +dlb2_validate_port_link (struct dlb2_eventdev_port *ev_port, +uint8_t queue_id, +bool link_exists, +int index) +{ +struct dlb2_eventdev *dlb2 = ev_port->dlb2; +struct dlb2_eventdev_queue *ev_queue; +bool port_is_dir, queue_is_dir; + +if (queue_id > dlb2->num_queues) { + rte_errno = -EINVAL ; +return -1 ; + } + + ev_queue = &dlb2->ev_queues[queue_id]; + +if (!ev_queue->setup_done && + ev_queue->qm_queue.config_state !=DLB2_PREV_CONFIGURED ) { + rte_errno = -EINVAL ; +return -1 ; + } + + port_is_dir = ev_port->qm_port.is_directed; + queue_is_dir = ev_queue->qm_queue.is_directed; + +if (port_is_dir != queue_is_dir) { +DLB2_LOG_ERR ("%s queue %u can't link to %s port %u\n" , + queue_is_dir ?"DIR" :"LDB" , ev_queue->id, + port_is_dir ?"DIR" :"LDB" , ev_port->id); + + rte_errno = -EINVAL ; +return -1 ; + } + +/* Check if there is space for the requested link */ +if (!link_exists && index ==-1 ) { +DLB2_LOG_ERR ("no space for new link\n" ); + rte_errno = -ENOSPC ; +return -1 ; + } + +/* Check if the directed port is already linked */ +if (ev_port->qm_port.is_directed && ev_port->num_links >0 && + !link_exists) { +DLB2_LOG_ERR ("Can't link DIR port %d to >1 queues\n" , + ev_port->id); + rte_errno = -EINVAL ; +return -1 ; + } + +/* Check if the directed queue is already linked */ +if (ev_queue->qm_queue.is_directed && ev_queue->num_links >0 && + !link_exists) { +DLB2_LOG_ERR ("Can't link DIR queue %d to >1 ports\n" , + ev_queue->id); + rte_errno = -EINVAL ; +return -1 ; + } + +return 0 ; +} + +static int +dlb2_eventdev_port_link (struct rte_eventdev *dev,void *event_port, +const uint8_t queues[],const uint8_t priorities[], +uint16_t nb_links) + +{ +struct dlb2_eventdev_port *ev_port = event_port; +struct dlb2_eventdev *dlb2; +int i, j; + +RTE_SET_USED (dev); + +if (!ev_port) { +DLB2_LOG_ERR ("dlb2: evport not setup\n" ); + rte_errno = -EINVAL ; +return 0 ; + } + +if (!ev_port->setup_done && + ev_port->qm_port.config_state !=DLB2_PREV_CONFIGURED ) { +DLB2_LOG_ERR ("dlb2: evport not setup\n" ); + rte_errno = -EINVAL ; +return 0 ; + } + +/* Note: rte_event_port_link() ensures the PMD won't receive a NULL +* queues pointer. +*/ +if (nb_links ==0 ) { +DLB2_LOG_DBG ("dlb2: nb_links is 0\n" ); +return 0 ;/* Ignore and return success */ + } + + dlb2 = ev_port->dlb2; + +DLB2_LOG_DBG ("Linking %u queues to %s port %d\n" , + nb_links, + ev_port->qm_port.is_directed ?"DIR" :"LDB" , + ev_port->id); + +for (i =0 ; i < nb_links; i++) { +struct dlb2_eventdev_queue *ev_queue; +uint8_t queue_id, prio; +bool found = false; +int index =-1 ; + + queue_id = queues[i]; + prio = priorities[i]; + +/* Check if the link already exists. */ +for (j =0 ; j <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; j++) +if (ev_port->link[j].valid) { +if (ev_port->link[j].queue_id == queue_id) { + found = true; + index = j; +break ; + } + }else if (index ==-1 ) { + index = j; + } + +/* could not link */ +if (index ==-1 ) +break ; + +/* Check if already linked at the requested priority */ +if (found && ev_port->link[j].priority == prio) +continue ; + +if (dlb2_validate_port_link (ev_port, queue_id, found, index)) +break ;/* return index of offending queue */ + + ev_queue = &dlb2->ev_queues[queue_id]; + +if (dlb2_do_port_link (dev, ev_queue, ev_port, prio)) +break ;/* return index of offending queue */ + + ev_queue->num_links++; + + ev_port->link[index].queue_id = queue_id; + ev_port->link[index].priority = prio; + ev_port->link[index].valid = true; +/* Entry already exists? If so, then must be prio change */ +if (!found) + ev_port->num_links++; + } +return i; +} +static void dlb2_entry_points_init (struct rte_eventdev *dev) {
@@ -1580,6 +1884,7 @@ dlb2_entry_points_init(struct rte_eventdev *dev) .queue_setup = dlb2_eventdev_queue_setup, .port_def_conf = dlb2_eventdev_port_default_conf_get, .port_setup = dlb2_eventdev_port_setup, + .port_link = dlb2_eventdev_port_link, .dump = dlb2_eventdev_dump, .xstats_get = dlb2_eventdev_xstats_get, .xstats_get_names = dlb2_eventdev_xstats_get_names,
drivers/event/dlb2/dlb2_iface.c +6 -0
@@ -66,3 +66,9 @@ int (*dlb2_iface_dir_port_create)(struct dlb2_hw_dev *handle,struct dlb2_create_dir_port_args *cfg, enum dlb2_cq_poll_modes poll_mode); +int (*dlb2_iface_dir_queue_create)(struct dlb2_hw_dev *handle, +struct dlb2_create_dir_queue_args *cfg); + +int (*dlb2_iface_map_qid)(struct dlb2_hw_dev *handle, +struct dlb2_map_qid_args *cfg); +
drivers/event/dlb2/dlb2_iface.h +6 -0
@@ -51,4 +51,10 @@ extern int (*dlb2_iface_dir_port_create)(struct dlb2_hw_dev *handle,struct dlb2_create_dir_port_args *cfg, enum dlb2_cq_poll_modes poll_mode); +extern int (*dlb2_iface_dir_queue_create)(struct dlb2_hw_dev *handle, +struct dlb2_create_dir_queue_args *cfg); + +extern int (*dlb2_iface_map_qid)(struct dlb2_hw_dev *handle, +struct dlb2_map_qid_args *cfg); +#endif /* _DLB2_IFACE_H_ */
drivers/event/dlb2/pf/base/dlb2_resource.c +633 -0
636 lines not on this page · raw message
drivers/event/dlb2/pf/dlb2_main.c +10 -0
@@ -670,3 +670,13 @@ dlb2_pf_create_dir_port(struct dlb2_hw *hw,NOT_VF_REQ , PF_ID_ZERO); } + +int +dlb2_pf_create_dir_queue (struct dlb2_hw *hw, +u32 id, +struct dlb2_create_dir_queue_args *args, +struct dlb2_cmd_response *resp) +{ +return dlb2_hw_create_dir_queue (hw, id, args, resp,NOT_VF_REQ , +PF_ID_ZERO ); +}
drivers/event/dlb2/pf/dlb2_pf.c +50 -0
63 lines not on this page · raw message
Signature · 1 line
2.6.4
PATCH 11/22 event/dlb2: add port setup
In reply to Timothy McDanielConfigure the load balanded (ldb) or directed (dir) port. The consumer queue (CQ) and producer port (PP) are also set up here.
--- drivers/event/dlb2/dlb2.c | 527 +++++++++++++++++ drivers/event/dlb2/dlb2_iface.c | 9 + drivers/event/dlb2/dlb2_iface.h | 8 + drivers/event/dlb2/pf/base/dlb2_resource.c | 921 +++++++++++++++++++++++++++++ drivers/event/dlb2/pf/dlb2_main.c | 28 + drivers/event/dlb2/pf/dlb2_pf.c | 179 ++++++ 6 files changed, 1672 insertions(+)
drivers/event/dlb2/dlb2.c +527 -0
@@ -1043,6 +1043,532 @@ dlb2_eventdev_queue_setup(struct rte_eventdev *dev,return ret; } +static int +dlb2_init_consume_qe (struct dlb2_port *qm_port,char *mz_name) +{ +struct dlb2_cq_pop_qe *qe; + + qe =rte_malloc (mz_name, +DLB2_NUM_QES_PER_CACHE_LINE * +sizeof (struct dlb2_cq_pop_qe ), +RTE_CACHE_LINE_SIZE ); + +if (qe ==NULL ) { +DLB2_LOG_ERR ("dlb2: no memory for consume_qe\n" ); +return -ENOMEM ; + } + qm_port->consume_qe = qe; + +memset (qe,0 ,DLB2_NUM_QES_PER_CACHE_LINE * +sizeof (struct dlb2_cq_pop_qe )); + + qe->qe_valid =0 ; + qe->qe_frag =0 ; + qe->qe_comp =0 ; + qe->cq_token =1 ; +/* Tokens value is 0-based; i.e. '0' returns 1 token, '1' returns 2, +* and so on. +*/ + qe->tokens =0 ;/* set at run time */ + qe->meas_lat =0 ; + qe->no_dec =0 ; +/* Completion IDs are disabled */ + qe->cmp_id =0 ; + +return 0 ; +} + +static int +dlb2_init_int_arm_qe (struct dlb2_port *qm_port,char *mz_name) +{ +struct dlb2_enqueue_qe *qe; + + qe =rte_malloc (mz_name, +DLB2_NUM_QES_PER_CACHE_LINE * +sizeof (struct dlb2_enqueue_qe ), +RTE_CACHE_LINE_SIZE ); + +if (qe ==NULL ) { +DLB2_LOG_ERR ("dlb2: no memory for complete_qe\n" ); +return -ENOMEM ; + } + qm_port->int_arm_qe = qe; + +memset (qe,0 ,DLB2_NUM_QES_PER_CACHE_LINE * +sizeof (struct dlb2_enqueue_qe )); + +/* V2 - INT ARM is CQ_TOKEN + FRAG */ + qe->qe_valid =0 ; + qe->qe_frag =1 ; + qe->qe_comp =0 ; + qe->cq_token =1 ; + qe->meas_lat =0 ; + qe->no_dec =0 ; +/* Completion IDs are disabled */ + qe->cmp_id =0 ; + +return 0 ; +} + +static int +dlb2_init_qe_mem (struct dlb2_port *qm_port,char *mz_name) +{ +int ret, sz; + + sz =DLB2_NUM_QES_PER_CACHE_LINE *sizeof (struct dlb2_enqueue_qe ); + + qm_port->qe4 =rte_malloc (mz_name, sz,RTE_CACHE_LINE_SIZE ); + +if (qm_port->qe4 ==NULL ) { +DLB2_LOG_ERR ("dlb2: no qe4 memory\n" ); + ret = -ENOMEM ; + goto error_exit; + } + +memset (qm_port->qe4,0 , sz); + + ret =dlb2_init_int_arm_qe (qm_port, mz_name); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: dlb2_init_int_arm_qe ret=%d\n" , + ret); + goto error_exit; + } + + ret =dlb2_init_consume_qe (qm_port, mz_name); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: dlb2_init_consume_qe ret=%d\n" , + ret); + goto error_exit; + } + +return 0 ; + +error_exit: + +dlb2_free_qe_mem (qm_port); + +return ret; +} + +static int +dlb2_hw_create_ldb_port (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port, +uint32_t dequeue_depth, +uint32_t enqueue_depth) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_create_ldb_port_args cfg = {0 }; +int ret; +struct dlb2_port *qm_port =NULL ; +char mz_name[RTE_MEMZONE_NAMESIZE ]; +uint32_t qm_port_id; +uint16_t ldb_credit_high_watermark; +uint16_t dir_credit_high_watermark; + +if (handle ==NULL ) +return -EINVAL ; + +if (dequeue_depth <DLB2_MIN_CQ_DEPTH || + dequeue_depth >DLB2_MAX_INPUT_QUEUE_DEPTH ) { +DLB2_LOG_ERR ("dlb2: invalid dequeue_depth, must be %d-%d\n" , +DLB2_MIN_CQ_DEPTH ,DLB2_MAX_INPUT_QUEUE_DEPTH ); +return -EINVAL ; + } + +if (enqueue_depth <DLB2_MIN_ENQUEUE_DEPTH ) { +DLB2_LOG_ERR ("dlb2: invalid enqueue_depth, must be at least %d\n" , +DLB2_MIN_ENQUEUE_DEPTH ); +return -EINVAL ; + } + +rte_spinlock_lock (&handle->resource_lock); + +/* TODO - additional parameter validation */ +/* We round up to the next power of 2 if necessary */ + cfg.cq_depth =rte_align32pow2 (dequeue_depth); + cfg.cq_depth_threshold =1 ; + + cfg.cq_history_list_size =DLB2_NUM_HIST_LIST_ENTRIES_PER_LDB_PORT ; + +if (handle->cos_id ==DLB2_COS_DEFAULT ) + cfg.cos_id =0 ; +else + cfg.cos_id = handle->cos_id; + + cfg.cos_strict =0 ; + +/* User controls the LDB high watermark via enqueue depth. The DIR high +* watermark is equal, unless the directed credit pool is too small. +*/ + ldb_credit_high_watermark = enqueue_depth; + +/* If there are no directed ports, the kernel driver will ignore this +* port's directed credit settings. Don't use enqueue_depth if it would +* require more directed credits than are available. +*/ + dir_credit_high_watermark = +RTE_MIN (enqueue_depth, + handle->cfg.num_dir_credits / dlb2->num_ports); + +/* Per QM values */ + +/* DEBUG +* DLB2_LOG_ERR("create ldb port - grp=%d, devId=%d\n", +* handle->cfg.domain_id, handle->device_id); +*/ + + ret =dlb2_iface_ldb_port_create (handle, &cfg, dlb2->poll_mode); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: dlb2_ldb_port_create error, ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); + goto error_exit; + } + + qm_port_id = cfg.response.id; + +DLB2_LOG_DBG ("dlb2: ev_port %d uses qm LB port %d <<<<<\n" , + ev_port->id, qm_port_id); + + qm_port = &ev_port->qm_port; + qm_port->ev_port = ev_port;/* back ptr */ + qm_port->dlb2 = dlb2;/* back ptr */ +/* +* Allocate and init local qe struct(s). +* Note: MOVDIR64 requires the enqueue QE (qe4) to be aligned. +*/ + +snprintf (mz_name,sizeof (mz_name),"%s_ldb_port%d" , + handle->device_name, + ev_port->id); + + ret =dlb2_init_qe_mem (qm_port, mz_name); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: init_qe_mem failed, ret=%d\n" , ret); + goto error_exit; + } + + qm_port->id = qm_port_id; + + qm_port->cached_ldb_credits =0 ; + qm_port->cached_dir_credits =0 ; +/* CQs with depth < 8 use an 8-entry queue, but withhold credits so +* the effective depth is smaller. +*/ + qm_port->cq_depth = cfg.cq_depth <=8 ?8 : cfg.cq_depth; + qm_port->cq_idx =0 ; + qm_port->cq_idx_unmasked =0 ; + +if (dlb2->poll_mode ==DLB2_CQ_POLL_MODE_SPARSE ) + qm_port->cq_depth_mask = (qm_port->cq_depth *4 ) -1 ; +else + qm_port->cq_depth_mask = qm_port->cq_depth -1 ; + + qm_port->gen_bit_shift =__builtin_popcount (qm_port->cq_depth_mask); +/* starting value of gen bit - it toggles at wrap time */ + qm_port->gen_bit =1 ; + + qm_port->int_armed = false; + +/* Save off for later use in info and lookup APIs. */ + qm_port->qid_mappings = &dlb2->qm_ldb_to_ev_queue_id[0 ]; + + qm_port->dequeue_depth = dequeue_depth; + qm_port->token_pop_thresh = dequeue_depth; + + qm_port->owed_tokens =0 ; + qm_port->issued_releases =0 ; + +/* Save config message too. */ +rte_memcpy (&qm_port->cfg.ldb, &cfg,sizeof (cfg)); + +/* update state */ + qm_port->state =PORT_STARTED ;/* enabled at create time */ + qm_port->config_state =DLB2_CONFIGURED ; + + qm_port->dir_credits = dir_credit_high_watermark; + qm_port->ldb_credits = ldb_credit_high_watermark; + qm_port->credit_pool[DLB2_DIR_QUEUE ] = &dlb2->dir_credit_pool; + qm_port->credit_pool[DLB2_LDB_QUEUE ] = &dlb2->ldb_credit_pool; + +DLB2_LOG_DBG ("dlb2: created ldb port %d, depth = %d, ldb credits=%d, dir credits=%d\n" , + qm_port_id, + dequeue_depth, + qm_port->ldb_credits, + qm_port->dir_credits); + +rte_spinlock_unlock (&handle->resource_lock); + +return 0 ; + +error_exit: + +if (qm_port) +dlb2_free_qe_mem (qm_port); + +rte_spinlock_unlock (&handle->resource_lock); + +DLB2_LOG_ERR ("dlb2: create ldb port failed!\n" ); + +return ret; +} + +static void +dlb2_port_link_teardown (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port) +{ +struct dlb2_eventdev_queue *ev_queue; +int i; + +for (i =0 ; i <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; i++) { +if (!ev_port->link[i].valid) +continue ; + + ev_queue = &dlb2->ev_queues[ev_port->link[i].queue_id]; + + ev_port->link[i].valid = false; + ev_port->num_links--; + ev_queue->num_links--; + } +} + +static int +dlb2_hw_create_dir_port (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port, +uint32_t dequeue_depth, +uint32_t enqueue_depth) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_create_dir_port_args cfg = {0 }; +int ret; +struct dlb2_port *qm_port =NULL ; +char mz_name[RTE_MEMZONE_NAMESIZE ]; +uint32_t qm_port_id; +uint16_t ldb_credit_high_watermark; +uint16_t dir_credit_high_watermark; + +if (dlb2 ==NULL || handle ==NULL ) +return -EINVAL ; + +if (dequeue_depth <DLB2_MIN_CQ_DEPTH || + dequeue_depth >DLB2_MAX_INPUT_QUEUE_DEPTH ) { +DLB2_LOG_ERR ("dlb2: invalid dequeue_depth, must be %d-%d\n" , +DLB2_MIN_CQ_DEPTH ,DLB2_MAX_INPUT_QUEUE_DEPTH ); +return -EINVAL ; + } + +rte_spinlock_lock (&handle->resource_lock); + +/* Directed queues are configured at link time. */ + cfg.queue_id =-1 ; + +/* We round up to the next power of 2 if necessary */ + cfg.cq_depth =rte_align32pow2 (dequeue_depth); + cfg.cq_depth_threshold =1 ; + +/* User controls the LDB high watermark via enqueue depth. The DIR high +* watermark is equal, unless the directed credit pool is too small. +*/ + ldb_credit_high_watermark = enqueue_depth; + +/* Don't use enqueue_depth if it would require more directed credits +* than are available. +*/ + dir_credit_high_watermark = +RTE_MIN (enqueue_depth, + handle->cfg.num_dir_credits / dlb2->num_ports); + +/* Per QM values */ + + ret =dlb2_iface_dir_port_create (handle, &cfg, dlb2->poll_mode); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: dlb2_dir_port_create error, ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); + goto error_exit; + } + + qm_port_id = cfg.response.id; + +DLB2_LOG_DBG ("dlb2: ev_port %d uses qm DIR port %d <<<<<\n" , + ev_port->id, qm_port_id); + + qm_port = &ev_port->qm_port; + qm_port->ev_port = ev_port;/* back ptr */ + qm_port->dlb2 = dlb2;/* back ptr */ + +/* +* Init local qe struct(s). +* Note: MOVDIR64 requires the enqueue QE to be aligned +*/ + +snprintf (mz_name,sizeof (mz_name),"%s_dir_port%d" , + handle->device_name, + ev_port->id); + + ret =dlb2_init_qe_mem (qm_port, mz_name); + +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: init_qe_mem failed, ret=%d\n" , ret); + goto error_exit; + } + + qm_port->id = qm_port_id; + + qm_port->cached_ldb_credits =0 ; + qm_port->cached_dir_credits =0 ; +/* CQs with depth < 8 use an 8-entry queue, but withhold credits so +* the effective depth is smaller. +*/ + qm_port->cq_depth = cfg.cq_depth <=8 ?8 : cfg.cq_depth; + qm_port->cq_idx =0 ; + qm_port->cq_idx_unmasked =0 ; + +if (dlb2->poll_mode ==DLB2_CQ_POLL_MODE_SPARSE ) + qm_port->cq_depth_mask = (cfg.cq_depth *4 ) -1 ; +else + qm_port->cq_depth_mask = cfg.cq_depth -1 ; + + qm_port->gen_bit_shift =__builtin_popcount (qm_port->cq_depth_mask); +/* starting value of gen bit - it toggles at wrap time */ + qm_port->gen_bit =1 ; + + qm_port->int_armed = false; + +/* Save off for later use in info and lookup APIs. */ + qm_port->qid_mappings = &dlb2->qm_dir_to_ev_queue_id[0 ]; + + qm_port->dequeue_depth = dequeue_depth; + +/* Directed ports are auto-pop, by default. */ + qm_port->token_pop_mode =AUTO_POP ; + qm_port->owed_tokens =0 ; + qm_port->issued_releases =0 ; + +/* Save config message too. */ +rte_memcpy (&qm_port->cfg.dir, &cfg,sizeof (cfg)); + +/* update state */ + qm_port->state =PORT_STARTED ;/* enabled at create time */ + qm_port->config_state =DLB2_CONFIGURED ; + + qm_port->dir_credits = dir_credit_high_watermark; + qm_port->ldb_credits = ldb_credit_high_watermark; + qm_port->credit_pool[DLB2_DIR_QUEUE ] = &dlb2->dir_credit_pool; + qm_port->credit_pool[DLB2_LDB_QUEUE ] = &dlb2->ldb_credit_pool; + +DLB2_LOG_DBG ("dlb2: created dir port %d, depth = %d cr=%d,%d\n" , + qm_port_id, + dequeue_depth, + dir_credit_high_watermark, + ldb_credit_high_watermark); + +rte_spinlock_unlock (&handle->resource_lock); + +return 0 ; + +error_exit: + +if (qm_port) +dlb2_free_qe_mem (qm_port); + +rte_spinlock_unlock (&handle->resource_lock); + +DLB2_LOG_ERR ("dlb2: create dir port failed!\n" ); + +return ret; +} + +static int +dlb2_eventdev_port_setup (struct rte_eventdev *dev, +uint8_t ev_port_id, +const struct rte_event_port_conf *port_conf) +{ +struct dlb2_eventdev *dlb2; +struct dlb2_eventdev_port *ev_port; +int ret; + +if (dev ==NULL || port_conf ==NULL ) { +DLB2_LOG_ERR ("Null parameter\n" ); +return -EINVAL ; + } + + dlb2 =dlb2_pmd_priv (dev); + +if (ev_port_id >=DLB2_MAX_NUM_PORTS ) +return -EINVAL ; + +if (port_conf->dequeue_depth > + evdev_dlb2_default_info.max_event_port_dequeue_depth || + port_conf->enqueue_depth > + evdev_dlb2_default_info.max_event_port_enqueue_depth) +return -EINVAL ; + + ev_port = &dlb2->ev_ports[ev_port_id]; +/* configured? */ +if (ev_port->setup_done) { +DLB2_LOG_ERR ("evport %d is already configured\n" , ev_port_id); +return -EINVAL ; + } + +/* The reserved token interrupt arming scheme requires that one or more +* CQ tokens be reserved by the PMD. This limits the amount of CQ space +* usable by the DLB, so in order to give an *effective* CQ depth equal +* to the user-requested value, we double CQ depth and reserve half of +* its tokens. If the user requests the max CQ depth (256) then we +* cannot double it, so we reserve one token and give an effective +* depth of 255 entries. +*/ + + ev_port->qm_port.is_directed = port_conf->event_port_cfg & +RTE_EVENT_PORT_CFG_SINGLE_LINK ; + +if (!ev_port->qm_port.is_directed) { + ret =dlb2_hw_create_ldb_port (dlb2, + ev_port, + port_conf->dequeue_depth, + port_conf->enqueue_depth); +if (ret <0 ) { +DLB2_LOG_ERR ("Failed to create the lB port ve portId=%d\n" , + ev_port_id); + +return ret; + } + }else { + ret =dlb2_hw_create_dir_port (dlb2, + ev_port, + port_conf->dequeue_depth, + port_conf->enqueue_depth); +if (ret <0 ) { +DLB2_LOG_ERR ("Failed to create the DIR port\n" ); +return ret; + } + } + +/* Save off port config for reconfig */ + dlb2->ev_ports[ev_port_id].conf = *port_conf; + + dlb2->ev_ports[ev_port_id].id = ev_port_id; + dlb2->ev_ports[ev_port_id].enq_configured = true; + dlb2->ev_ports[ev_port_id].setup_done = true; + dlb2->ev_ports[ev_port_id].inflight_max = + port_conf->new_event_threshold; + dlb2->ev_ports[ev_port_id].implicit_release = + !(port_conf->event_port_cfg & +RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL ); + dlb2->ev_ports[ev_port_id].outstanding_releases =0 ; + dlb2->ev_ports[ev_port_id].inflight_credits =0 ; + dlb2->ev_ports[ev_port_id].credit_update_quanta = +RTE_LIBRTE_PMD_DLB2_SW_CREDIT_QUANTA ; + dlb2->ev_ports[ev_port_id].dlb2 = dlb2;/* reverse link */ + +/* Tear down pre-existing port->queue links */ +if (dlb2->run_state ==DLB2_RUN_STATE_STOPPED ) +dlb2_port_link_teardown (dlb2, &dlb2->ev_ports[ev_port_id]); + + dev->data->ports[ev_port_id] = &dlb2->ev_ports[ev_port_id]; + +return 0 ; +} +static void dlb2_entry_points_init (struct rte_eventdev *dev) {
@@ -1053,6 +1579,7 @@ dlb2_entry_points_init(struct rte_eventdev *dev) .queue_def_conf = dlb2_eventdev_queue_default_conf_get, .queue_setup = dlb2_eventdev_queue_setup, .port_def_conf = dlb2_eventdev_port_default_conf_get, + .port_setup = dlb2_eventdev_port_setup, .dump = dlb2_eventdev_dump, .xstats_get = dlb2_eventdev_xstats_get, .xstats_get_names = dlb2_eventdev_xstats_get_names,
drivers/event/dlb2/dlb2_iface.c +9 -0
@@ -57,3 +57,12 @@ int (*dlb2_iface_set_sn_allocation)(struct dlb2_hw_dev *handle,int (*dlb2_iface_get_sn_occupancy)(struct dlb2_hw_dev *handle,struct dlb2_get_sn_occupancy_args *args); + +int (*dlb2_iface_ldb_port_create)(struct dlb2_hw_dev *handle, +struct dlb2_create_ldb_port_args *cfg, +enum dlb2_cq_poll_modes poll_mode); + +int (*dlb2_iface_dir_port_create)(struct dlb2_hw_dev *handle, +struct dlb2_create_dir_port_args *cfg, +enum dlb2_cq_poll_modes poll_mode); +
drivers/event/dlb2/dlb2_iface.h +8 -0
@@ -43,4 +43,12 @@ extern int (*dlb2_iface_set_sn_allocation)(struct dlb2_hw_dev *handle,extern int (*dlb2_iface_get_sn_occupancy)(struct dlb2_hw_dev *handle,struct dlb2_get_sn_occupancy_args *args); +extern int (*dlb2_iface_ldb_port_create)(struct dlb2_hw_dev *handle, +struct dlb2_create_ldb_port_args *cfg, +enum dlb2_cq_poll_modes poll_mode); + +extern int (*dlb2_iface_dir_port_create)(struct dlb2_hw_dev *handle, +struct dlb2_create_dir_port_args *cfg, +enum dlb2_cq_poll_modes poll_mode); +#endif /* _DLB2_IFACE_H_ */
drivers/event/dlb2/pf/base/dlb2_resource.c +921 -0
924 lines not on this page · raw message
drivers/event/dlb2/pf/dlb2_main.c +28 -0
@@ -642,3 +642,31 @@ dlb2_pf_create_ldb_queue(struct dlb2_hw *hw,return dlb2_hw_create_ldb_queue (hw, id, args, resp,NOT_VF_REQ ,PF_ID_ZERO ); } + +int +dlb2_pf_create_ldb_port (struct dlb2_hw *hw, +u32 id, +struct dlb2_create_ldb_port_args *args, +uintptr_t cq_dma_base, +struct dlb2_cmd_response *resp) +{ +return dlb2_hw_create_ldb_port (hw, id, args, + cq_dma_base, + resp, +NOT_VF_REQ , +PF_ID_ZERO ); +} + +int +dlb2_pf_create_dir_port (struct dlb2_hw *hw, +u32 id, +struct dlb2_create_dir_port_args *args, +uintptr_t cq_dma_base, +struct dlb2_cmd_response *resp) +{ +return dlb2_hw_create_dir_port (hw, id, args, + cq_dma_base, + resp, +NOT_VF_REQ , +PF_ID_ZERO ); +}
drivers/event/dlb2/pf/dlb2_pf.c +179 -0
191 lines not on this page · raw message
Signature · 1 line
2.6.4
PATCH 13/22 event/dlb2: add port unlink and port unlinks in progress
In reply to Timothy McDanielAdd supports for the port unlink(s) eventdev entry points. The unlink operation is an asynchronous operation executed by a control thread, and the unlinks-in-progress function reads a counter shared with the control thread.
--- drivers/event/dlb2/dlb2.c | 163 +++++++++++++++++ drivers/event/dlb2/dlb2_iface.c | 5 + drivers/event/dlb2/dlb2_iface.h | 8 +- drivers/event/dlb2/pf/base/dlb2_resource.c | 283 +++++++++++++++++++++++++++++ drivers/event/dlb2/pf/dlb2_pf.c | 51 ++++++ 5 files changed, 509 insertions(+), 1 deletion(-)
drivers/event/dlb2/dlb2.c +163 -0
@@ -1873,6 +1873,166 @@ dlb2_eventdev_port_link(struct rte_eventdev *dev, void *event_port,return i; } +static int16_t +dlb2_hw_unmap_ldb_qid_from_port (struct dlb2_hw_dev *handle, +uint32_t qm_port_id, +uint16_t qm_qid) +{ +struct dlb2_unmap_qid_args cfg; +int32_t ret; + +if (handle ==NULL ) +return -EINVAL ; + + cfg.port_id = qm_port_id; + cfg.qid = qm_qid; + + ret =dlb2_iface_unmap_qid (handle, &cfg); +if (ret <0 ) +DLB2_LOG_ERR ("dlb2: unmap qid error, ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); + +return ret; +} + +static int +dlb2_event_queue_detach_ldb (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port, +struct dlb2_eventdev_queue *ev_queue) +{ +int ret, i; + +/* Don't unlink until start time. */ +if (dlb2->run_state ==DLB2_RUN_STATE_STOPPED ) +return 0 ; + +for (i =0 ; i <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; i++) { +if (ev_port->link[i].valid && + ev_port->link[i].queue_id == ev_queue->id) +break ;/* found */ + } + +/* This is expected with eventdev API! +* It blindly attemmpts to unmap all queues. +*/ +if (i ==DLB2_MAX_NUM_QIDS_PER_LDB_CQ ) { +DLB2_LOG_DBG ("dlb2: ignoring LB QID %d not mapped for qm_port %d.\n" , + ev_queue->qm_queue.id, + ev_port->qm_port.id); +return 0 ; + } + + ret =dlb2_hw_unmap_ldb_qid_from_port (&dlb2->qm_instance, + ev_port->qm_port.id, + ev_queue->qm_queue.id); +if (!ret) + ev_port->link[i].mapped = false; + +return ret; +} + +static int +dlb2_eventdev_port_unlink (struct rte_eventdev *dev,void *event_port, +uint8_t queues[],uint16_t nb_unlinks) +{ +struct dlb2_eventdev_port *ev_port = event_port; +struct dlb2_eventdev *dlb2; +int i; + +RTE_SET_USED (dev); + +if (!ev_port->setup_done) { +DLB2_LOG_ERR ("dlb2: evport %d is not configured\n" , + ev_port->id); + rte_errno = -EINVAL ; +return 0 ; + } + +if (!queues || nb_unlinks ==0 ) { +DLB2_LOG_DBG ("dlb2: queues is NULL or nb_unlinks is 0\n" ); +return 0 ;/* Ignore and return success */ + } + +/* FIXME: How to handle unlink on a directed port? */ +if (ev_port->qm_port.is_directed) { +DLB2_LOG_DBG ("dlb2: ignore unlink from dir port %d\n" , + ev_port->id); + rte_errno =0 ; +return nb_unlinks;/* as if success */ + } + + dlb2 = ev_port->dlb2; + +for (i =0 ; i < nb_unlinks; i++) { +struct dlb2_eventdev_queue *ev_queue; +int ret, j; + +if (queues[i] >= dlb2->num_queues) { +DLB2_LOG_ERR ("dlb2: invalid queue id %d\n" , queues[i]); + rte_errno = -EINVAL ; +return i;/* return index of offending queue */ + } + + ev_queue = &dlb2->ev_queues[queues[i]]; + +/* Does a link exist? */ +for (j =0 ; j <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; j++) +if (ev_port->link[j].queue_id == queues[i] && + ev_port->link[j].valid) +break ; + +if (j ==DLB2_MAX_NUM_QIDS_PER_LDB_CQ ) +continue ; + + ret =dlb2_event_queue_detach_ldb (dlb2, ev_port, ev_queue); +if (ret) { +DLB2_LOG_ERR ("unlink err=%d for port %d queue %d\n" , + ret, ev_port->id, queues[i]); + rte_errno = -ENOENT ; +return i;/* return index of offending queue */ + } + + ev_port->link[j].valid = false; + ev_port->num_links--; + ev_queue->num_links--; + } + +return nb_unlinks; +} + +static int +dlb2_eventdev_port_unlinks_in_progress (struct rte_eventdev *dev, +void *event_port) +{ +struct dlb2_eventdev_port *ev_port = event_port; +struct dlb2_eventdev *dlb2; +struct dlb2_hw_dev *handle; +struct dlb2_pending_port_unmaps_args cfg; +int ret; + +RTE_SET_USED (dev); + +if (!ev_port->setup_done) { +DLB2_LOG_ERR ("dlb2: evport %d is not configured\n" , + ev_port->id); + rte_errno = -EINVAL ; +return 0 ; + } + + cfg.port_id = ev_port->qm_port.id; + dlb2 = ev_port->dlb2; + handle = &dlb2->qm_instance; + ret =dlb2_iface_pending_port_unmaps (handle, &cfg); + +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: num_unlinks_in_progress ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); +return ret; + } + +return cfg.response.id; +} +static void dlb2_entry_points_init (struct rte_eventdev *dev) {
@@ -1885,6 +2045,9 @@ dlb2_entry_points_init(struct rte_eventdev *dev) .port_def_conf = dlb2_eventdev_port_default_conf_get, .port_setup = dlb2_eventdev_port_setup, .port_link = dlb2_eventdev_port_link, + .port_unlink = dlb2_eventdev_port_unlink, + .port_unlinks_in_progress = + dlb2_eventdev_port_unlinks_in_progress, .dump = dlb2_eventdev_dump, .xstats_get = dlb2_eventdev_xstats_get, .xstats_get_names = dlb2_eventdev_xstats_get_names,
drivers/event/dlb2/dlb2_iface.c +5 -0
@@ -72,3 +72,8 @@ int (*dlb2_iface_dir_queue_create)(struct dlb2_hw_dev *handle,int (*dlb2_iface_map_qid)(struct dlb2_hw_dev *handle,struct dlb2_map_qid_args *cfg); +int (*dlb2_iface_unmap_qid)(struct dlb2_hw_dev *handle, +struct dlb2_unmap_qid_args *cfg); + +int (*dlb2_iface_pending_port_unmaps)(struct dlb2_hw_dev *handle, +struct dlb2_pending_port_unmaps_args *args);
drivers/event/dlb2/dlb2_iface.h +7 -1
@@ -52,9 +52,15 @@ extern int (*dlb2_iface_dir_port_create)(struct dlb2_hw_dev *handle, enum dlb2_cq_poll_modes poll_mode);extern int (*dlb2_iface_dir_queue_create)(struct dlb2_hw_dev *handle, -struct dlb2_create_dir_queue_args *cfg); +struct dlb2_create_dir_queue_args *cfg);extern int (*dlb2_iface_map_qid)(struct dlb2_hw_dev *handle,struct dlb2_map_qid_args *cfg); +extern int (*dlb2_iface_unmap_qid)(struct dlb2_hw_dev *handle, +struct dlb2_unmap_qid_args *cfg); + +extern int (*dlb2_iface_pending_port_unmaps)(struct dlb2_hw_dev *handle, +struct dlb2_pending_port_unmaps_args *args); +#endif /* _DLB2_IFACE_H_ */
drivers/event/dlb2/pf/base/dlb2_resource.c +283 -0
286 lines not on this page · raw message
drivers/event/dlb2/pf/dlb2_pf.c +51 -0
63 lines not on this page · raw message
Signature · 1 line
2.6.4
PATCH 14/22 event/dlb2: add eventdev start
In reply to Timothy McDanielAdd support for the eventdev start entry point.
--- drivers/event/dlb2/dlb2.c | 133 +++++++++++++++++++++++++++++ drivers/event/dlb2/dlb2_iface.c | 3 + drivers/event/dlb2/dlb2_iface.h | 3 + drivers/event/dlb2/pf/base/dlb2_resource.c | 123 ++++++++++++++++++++++++++ drivers/event/dlb2/pf/dlb2_main.c | 10 +++ drivers/event/dlb2/pf/dlb2_pf.c | 25 ++++++ 6 files changed, 297 insertions(+)
drivers/event/dlb2/dlb2.c +133 -0
@@ -2033,6 +2033,138 @@ dlb2_eventdev_port_unlinks_in_progress(struct rte_eventdev *dev,return cfg.response.id; } +static int +dlb2_eventdev_reapply_configuration (struct rte_eventdev *dev) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +int ret, i; + +/* If an event queue or port was previously configured, but hasn't been +* reconfigured, reapply its original configuration. +*/ +for (i =0 ; i < dlb2->num_queues; i++) { +struct dlb2_eventdev_queue *ev_queue; + + ev_queue = &dlb2->ev_queues[i]; + +if (ev_queue->qm_queue.config_state !=DLB2_PREV_CONFIGURED ) +continue ; + + ret =dlb2_eventdev_queue_setup (dev, i, &ev_queue->conf); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: failed to reconfigure queue %d" , i); +return ret; + } + } + +for (i =0 ; i < dlb2->num_ports; i++) { +struct dlb2_eventdev_port *ev_port = &dlb2->ev_ports[i]; + +if (ev_port->qm_port.config_state !=DLB2_PREV_CONFIGURED ) +continue ; + + ret =dlb2_eventdev_port_setup (dev, i, &ev_port->conf); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: failed to reconfigure ev_port %d" , + i); +return ret; + } + } + +return 0 ; +} + +static int +dlb2_eventdev_apply_port_links (struct rte_eventdev *dev) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +int i; + +/* Perform requested port->queue links */ +for (i =0 ; i < dlb2->num_ports; i++) { +struct dlb2_eventdev_port *ev_port = &dlb2->ev_ports[i]; +int j; + +for (j =0 ; j <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; j++) { +struct dlb2_eventdev_queue *ev_queue; +uint8_t prio, queue_id; + +if (!ev_port->link[j].valid) +continue ; + + prio = ev_port->link[j].priority; + queue_id = ev_port->link[j].queue_id; + +if (dlb2_validate_port_link (ev_port, queue_id, true, j)) +return -EINVAL ; + + ev_queue = &dlb2->ev_queues[queue_id]; + +if (dlb2_do_port_link (dev, ev_queue, ev_port, prio)) +return -EINVAL ; + } + } + +return 0 ; +} + +static int +dlb2_eventdev_start (struct rte_eventdev *dev) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_start_domain_args cfg; +int ret, i; + +rte_spinlock_lock (&dlb2->qm_instance.resource_lock); +if (dlb2->run_state !=DLB2_RUN_STATE_STOPPED ) { +DLB2_LOG_ERR ("bad state %d for dev_start\n" , + (int )dlb2->run_state); +rte_spinlock_unlock (&dlb2->qm_instance.resource_lock); +return -EINVAL ; + } + dlb2->run_state =DLB2_RUN_STATE_STARTING ; +rte_spinlock_unlock (&dlb2->qm_instance.resource_lock); + +/* If the device was configured more than once, some event ports and/or +* queues may need to be reconfigured. +*/ + ret =dlb2_eventdev_reapply_configuration (dev); +if (ret) +return ret; + +/* The DLB PMD delays port links until the device is started. */ + ret =dlb2_eventdev_apply_port_links (dev); +if (ret) +return ret; + +for (i =0 ; i < dlb2->num_ports; i++) { +if (!dlb2->ev_ports[i].setup_done) { +DLB2_LOG_ERR ("dlb2: port %d not setup" , i); +return -ESTALE ; + } + } + +for (i =0 ; i < dlb2->num_queues; i++) { +if (dlb2->ev_queues[i].num_links ==0 ) { +DLB2_LOG_ERR ("dlb2: queue %d is not linked" , i); +return -ENOLINK ; + } + } + + ret =dlb2_iface_sched_domain_start (handle, &cfg); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: sched_domain_start ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); +return ret; + } + + dlb2->run_state =DLB2_RUN_STATE_STARTED ; +DLB2_LOG_DBG ("dlb2: sched_domain_start completed OK\n" ); + +return 0 ; +} +static void dlb2_entry_points_init (struct rte_eventdev *dev) {
@@ -2040,6 +2172,7 @@ dlb2_entry_points_init(struct rte_eventdev *dev)static struct rte_eventdev_ops dlb2_eventdev_entry_ops = { .dev_infos_get = dlb2_eventdev_info_get, .dev_configure = dlb2_eventdev_configure, + .dev_start = dlb2_eventdev_start, .queue_def_conf = dlb2_eventdev_queue_default_conf_get, .queue_setup = dlb2_eventdev_queue_setup, .port_def_conf = dlb2_eventdev_port_default_conf_get,
drivers/event/dlb2/dlb2_iface.c +3 -0
@@ -77,3 +77,6 @@ int (*dlb2_iface_unmap_qid)(struct dlb2_hw_dev *handle,int (*dlb2_iface_pending_port_unmaps)(struct dlb2_hw_dev *handle,struct dlb2_pending_port_unmaps_args *args); + +int (*dlb2_iface_sched_domain_start)(struct dlb2_hw_dev *handle, +struct dlb2_start_domain_args *cfg);
drivers/event/dlb2/dlb2_iface.h +3 -0
@@ -63,4 +63,7 @@ extern int (*dlb2_iface_unmap_qid)(struct dlb2_hw_dev *handle,extern int (*dlb2_iface_pending_port_unmaps)(struct dlb2_hw_dev *handle,struct dlb2_pending_port_unmaps_args *args); +extern int (*dlb2_iface_sched_domain_start)(struct dlb2_hw_dev *handle, +struct dlb2_start_domain_args *cfg); +#endif /* _DLB2_IFACE_H_ */
drivers/event/dlb2/pf/base/dlb2_resource.c +123 -0
126 lines not on this page · raw message
drivers/event/dlb2/pf/dlb2_main.c +10 -0
@@ -680,3 +680,13 @@ dlb2_pf_create_dir_queue(struct dlb2_hw *hw,return dlb2_hw_create_dir_queue (hw, id, args, resp,NOT_VF_REQ ,PF_ID_ZERO ); } + +int +dlb2_pf_start_domain (struct dlb2_hw *hw, +u32 id, +struct dlb2_start_domain_args *args, +struct dlb2_cmd_response *resp) +{ +return dlb2_hw_start_domain (hw, id, args, resp,NOT_VF_REQ , +PF_ID_ZERO ); +}
drivers/event/dlb2/pf/dlb2_pf.c +25 -0
@@ -508,6 +508,30 @@ dlb2_pf_pending_port_unmaps(struct dlb2_hw_dev *handle,return ret; } + +static int +dlb2_pf_sched_domain_start (struct dlb2_hw_dev *handle, +struct dlb2_start_domain_args *cfg) +{ +struct dlb2_dev *dlb2_dev = (struct dlb2_dev *)handle->pf_dev; +struct dlb2_cmd_response response = {0 }; +int ret; + +DLB2_INFO (dev->dlb2_device,"Entering %s()\n" , __func__); + + ret =dlb2_pf_start_domain (&dlb2_dev->hw, + handle->domain_id, + cfg, + &response); + + cfg->response = response; + +DLB2_INFO (dev->dlb2_device,"Exiting %s() with ret=%d\n" , + __func__, ret); + +return ret; +} +static void dlb2_pf_iface_fn_ptrs_init (void ) {
@@ -526,6 +550,7 @@ dlb2_pf_iface_fn_ptrs_init(void) dlb2_iface_dir_port_create = dlb2_pf_dir_port_create; dlb2_iface_map_qid = dlb2_pf_map_qid; dlb2_iface_unmap_qid = dlb2_pf_unmap_qid; + dlb2_iface_sched_domain_start = dlb2_pf_sched_domain_start; dlb2_iface_pending_port_unmaps = dlb2_pf_pending_port_unmaps; dlb2_iface_get_sn_allocation = dlb2_pf_get_sn_allocation; dlb2_iface_set_sn_allocation = dlb2_pf_set_sn_allocation;
Signature · 1 line
2.6.4
PATCH 15/22 event/dlb2: add enqueue and its burst variants
In reply to Timothy McDanielAdd support for enqueue and its variants.
--- drivers/event/dlb2/dlb2.c | 592 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 592 insertions(+)
drivers/event/dlb2/dlb2.c +592 -0
@@ -2165,6 +2165,592 @@ dlb2_eventdev_start(struct rte_eventdev *dev)return 0 ; } +static uint8_t cmd_byte_map[DLB2_NUM_PORT_TYPES ][DLB2_NUM_HW_SCHED_TYPES ] = { + { +/* Load-balanced cmd bytes */ + [RTE_EVENT_OP_NEW ] =DLB2_NEW_CMD_BYTE , + [RTE_EVENT_OP_FORWARD ] =DLB2_FWD_CMD_BYTE , + [RTE_EVENT_OP_RELEASE ] =DLB2_COMP_CMD_BYTE , + }, + { +/* Directed cmd bytes */ + [RTE_EVENT_OP_NEW ] =DLB2_NEW_CMD_BYTE , + [RTE_EVENT_OP_FORWARD ] =DLB2_NEW_CMD_BYTE , + [RTE_EVENT_OP_RELEASE ] =DLB2_NOOP_CMD_BYTE , + }, +}; + +static inline uint32_t +dlb2_port_credits_get (struct dlb2_port *qm_port, +enum dlb2_hw_queue_types type) +{ +uint32_t credits = *qm_port->credit_pool[type]; +uint32_t batch_size =DLB2_SW_CREDIT_BATCH_SZ ; + +if (unlikely (credits < batch_size)) + batch_size = credits; + +if (likely (credits && +__atomic_compare_exchange_n ( + qm_port->credit_pool[type], + &credits, credits - batch_size, false, + __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST))) +return batch_size; +else +return 0 ; +} + +static inline void +dlb2_replenish_sw_credits (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port) +{ +uint16_t quanta = ev_port->credit_update_quanta; + +if (ev_port->inflight_credits >= quanta *2 ) { +/* Replenish credits, saving one quanta for enqueues */ +uint16_t val = ev_port->inflight_credits - quanta; + +__atomic_fetch_sub (&dlb2->inflights, val, __ATOMIC_SEQ_CST); + ev_port->inflight_credits -= val; + } +} + +static inline int +dlb2_check_enqueue_sw_credits (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port) +{ +uint32_t sw_inflights =__atomic_load_n (&dlb2->inflights, + __ATOMIC_SEQ_CST); +const int num =1 ; + +if (unlikely (ev_port->inflight_max < sw_inflights)) { +DLB2_INC_STAT (ev_port->stats.traffic.tx_nospc_inflight_max,1 ); + rte_errno = -ENOSPC ; +return 1 ; + } + +if (ev_port->inflight_credits < num) { +/* check if event enqueue brings ev_port over max threshold */ +uint32_t credit_update_quanta = ev_port->credit_update_quanta; + +if (sw_inflights + credit_update_quanta > + dlb2->new_event_limit) { +DLB2_INC_STAT ( + ev_port->stats.traffic.tx_nospc_new_event_limit, +1 ); + rte_errno = -ENOSPC ; +return 1 ; + } + +__atomic_fetch_add (&dlb2->inflights, credit_update_quanta, + __ATOMIC_SEQ_CST); + ev_port->inflight_credits += (credit_update_quanta); + +if (ev_port->inflight_credits < num) { +DLB2_INC_STAT ( + ev_port->stats.traffic.tx_nospc_inflight_credits, +1 ); + rte_errno = -ENOSPC ; +return 1 ; + } + } + +return 0 ; +} + +static inline int +dlb2_check_enqueue_hw_ldb_credits (struct dlb2_port *qm_port) +{ +if (unlikely (qm_port->cached_ldb_credits ==0 )) { + qm_port->cached_ldb_credits = +dlb2_port_credits_get (qm_port, +DLB2_LDB_QUEUE ); +if (unlikely (qm_port->cached_ldb_credits ==0 )) { +DLB2_INC_STAT ( + qm_port->ev_port->stats.traffic.tx_nospc_ldb_hw_credits, +1 ); +DLB2_LOG_DBG ("ldb credits exhausted\n" ); +return 1 ;/* credits exhausted */ + } + } + +return 0 ; +} + +static inline int +dlb2_check_enqueue_hw_dir_credits (struct dlb2_port *qm_port) +{ +if (unlikely (qm_port->cached_dir_credits ==0 )) { + qm_port->cached_dir_credits = +dlb2_port_credits_get (qm_port, +DLB2_DIR_QUEUE ); +if (unlikely (qm_port->cached_dir_credits ==0 )) { +DLB2_INC_STAT ( + qm_port->ev_port->stats.traffic.tx_nospc_dir_hw_credits, +1 ); +DLB2_LOG_DBG ("dir credits exhausted\n" ); +return 1 ;/* credits exhausted */ + } + } + +return 0 ; +} + +static __rte_always_inline void +dlb2_pp_write (struct dlb2_enqueue_qe *qe4, +struct process_local_port_data *port_data) +{ + qm_mmio_fns.pp_enqueue_four (qe4, port_data->pp_addr); +} + +static inline int +dlb2_consume_qe_immediate (struct dlb2_port *qm_port,int num) +{ +struct process_local_port_data *port_data; +struct dlb2_cq_pop_qe *qe; + +RTE_ASSERT (qm_port->config_state ==DLB2_CONFIGURED ); + + qe = qm_port->consume_qe; + + qe->tokens = num -1 ; + +/* No store fence needed since no pointer is being sent, and CQ token +* pops can be safely reordered with other HCWs. +*/ + port_data = &dlb2_port[qm_port->id][PORT_TYPE (qm_port)]; + +dlb2_movntdq_single (qe, port_data->pp_addr); + +DLB2_LOG_DBG ("dlb2: consume immediate - %d QEs\n" , num); + + qm_port->owed_tokens =0 ; + +return 0 ; +} + +static inline void +dlb2_hw_do_enqueue (struct dlb2_port *qm_port, +bool do_sfence, +struct process_local_port_data *port_data) +{ +/* Since MOVDIR64B is weakly-ordered, use an SFENCE to ensure that +* application writes complete before enqueueing the QE. +*/ +if (do_sfence) +rte_wmb (); + +dlb2_pp_write (qm_port->qe4, port_data); +} + +static inline void +dlb2_construct_token_pop_qe (struct dlb2_port *qm_port,int idx) +{ +struct dlb2_cq_pop_qe *qe = (void *)qm_port->qe4; +int num = qm_port->owed_tokens; + + qe[idx].cmd_byte =DLB2_POP_CMD_BYTE ; + qe[idx].tokens = num -1 ; + + qm_port->owed_tokens =0 ; +} + +static inline void +dlb2_event_build_hcws (struct dlb2_port *qm_port, +const struct rte_event ev[], +int num, +uint8_t *sched_type, +uint8_t *queue_id) +{ +struct dlb2_enqueue_qe *qe; +uint16_t sched_word[4 ]; +__m128i sse_qe[2 ]; +int i; + + qe = qm_port->qe4; + + sse_qe[0 ] =_mm_setzero_si128 (); + sse_qe[1 ] =_mm_setzero_si128 (); + +switch (num) { +case 4 : +/* Construct the metadata portion of two HCWs in one 128b SSE +* register. HCW metadata is constructed in the SSE registers +* like so: +* sse_qe[0][63:0]: qe[0]'s metadata +* sse_qe[0][127:64]: qe[1]'s metadata +* sse_qe[1][63:0]: qe[2]'s metadata +* sse_qe[1][127:64]: qe[3]'s metadata +*/ + +/* Convert the event operation into a command byte and store it +* in the metadata: +* sse_qe[0][63:56] = cmd_byte_map[is_directed][ev[0].op] +* sse_qe[0][127:120] = cmd_byte_map[is_directed][ev[1].op] +* sse_qe[1][63:56] = cmd_byte_map[is_directed][ev[2].op] +* sse_qe[1][127:120] = cmd_byte_map[is_directed][ev[3].op] +*/ +#define DLB2_QE_CMD_BYTE 7 + sse_qe[0 ] =_mm_insert_epi8 (sse_qe[0 ], + cmd_byte_map[qm_port->is_directed][ev[0 ].op], +DLB2_QE_CMD_BYTE ); + sse_qe[0 ] =_mm_insert_epi8 (sse_qe[0 ], + cmd_byte_map[qm_port->is_directed][ev[1 ].op], +DLB2_QE_CMD_BYTE +8 ); + sse_qe[1 ] =_mm_insert_epi8 (sse_qe[1 ], + cmd_byte_map[qm_port->is_directed][ev[2 ].op], +DLB2_QE_CMD_BYTE ); + sse_qe[1 ] =_mm_insert_epi8 (sse_qe[1 ], + cmd_byte_map[qm_port->is_directed][ev[3 ].op], +DLB2_QE_CMD_BYTE +8 ); + +/* Store priority, scheduling type, and queue ID in the sched +* word array because these values are re-used when the +* destination is a directed queue. +*/ + sched_word[0 ] =EV_TO_DLB2_PRIO (ev[0 ].priority) <<10 | + sched_type[0 ] <<8 | + queue_id[0 ]; + sched_word[1 ] =EV_TO_DLB2_PRIO (ev[1 ].priority) <<10 | + sched_type[1 ] <<8 | + queue_id[1 ]; + sched_word[2 ] =EV_TO_DLB2_PRIO (ev[2 ].priority) <<10 | + sched_type[2 ] <<8 | + queue_id[2 ]; + sched_word[3 ] =EV_TO_DLB2_PRIO (ev[3 ].priority) <<10 | + sched_type[3 ] <<8 | + queue_id[3 ]; + +/* Store the event priority, scheduling type, and queue ID in +* the metadata: +* sse_qe[0][31:16] = sched_word[0] +* sse_qe[0][95:80] = sched_word[1] +* sse_qe[1][31:16] = sched_word[2] +* sse_qe[1][95:80] = sched_word[3] +*/ +#define DLB2_QE_QID_SCHED_WORD 1 + sse_qe[0 ] =_mm_insert_epi16 (sse_qe[0 ], + sched_word[0 ], +DLB2_QE_QID_SCHED_WORD ); + sse_qe[0 ] =_mm_insert_epi16 (sse_qe[0 ], + sched_word[1 ], +DLB2_QE_QID_SCHED_WORD +4 ); + sse_qe[1 ] =_mm_insert_epi16 (sse_qe[1 ], + sched_word[2 ], +DLB2_QE_QID_SCHED_WORD ); + sse_qe[1 ] =_mm_insert_epi16 (sse_qe[1 ], + sched_word[3 ], +DLB2_QE_QID_SCHED_WORD +4 ); + +/* If the destination is a load-balanced queue, store the lock +* ID. If it is a directed queue, DLB places this field in +* bytes 10-11 of the received QE, so we format it accordingly: +* sse_qe[0][47:32] = dir queue ? sched_word[0] : flow_id[0] +* sse_qe[0][111:96] = dir queue ? sched_word[1] : flow_id[1] +* sse_qe[1][47:32] = dir queue ? sched_word[2] : flow_id[2] +* sse_qe[1][111:96] = dir queue ? sched_word[3] : flow_id[3] +*/ +#define DLB2_QE_LOCK_ID_WORD 2 + sse_qe[0 ] =_mm_insert_epi16 (sse_qe[0 ], + (sched_type[0 ] ==DLB2_SCHED_DIRECTED ) ? + sched_word[0 ] : ev[0 ].flow_id, +DLB2_QE_LOCK_ID_WORD ); + sse_qe[0 ] =_mm_insert_epi16 (sse_qe[0 ], + (sched_type[1 ] ==DLB2_SCHED_DIRECTED ) ? + sched_word[1 ] : ev[1 ].flow_id, +DLB2_QE_LOCK_ID_WORD +4 ); + sse_qe[1 ] =_mm_insert_epi16 (sse_qe[1 ], + (sched_type[2 ] ==DLB2_SCHED_DIRECTED ) ? + sched_word[2 ] : ev[2 ].flow_id, +DLB2_QE_LOCK_ID_WORD ); + sse_qe[1 ] =_mm_insert_epi16 (sse_qe[1 ], + (sched_type[3 ] ==DLB2_SCHED_DIRECTED ) ? + sched_word[3 ] : ev[3 ].flow_id, +DLB2_QE_LOCK_ID_WORD +4 ); + +/* Store the event type and sub event type in the metadata: +* sse_qe[0][15:0] = flow_id[0] +* sse_qe[0][79:64] = flow_id[1] +* sse_qe[1][15:0] = flow_id[2] +* sse_qe[1][79:64] = flow_id[3] +*/ +#define DLB2_QE_EV_TYPE_WORD 0 + sse_qe[0 ] =_mm_insert_epi16 (sse_qe[0 ], + ev[0 ].sub_event_type <<8 | + ev[0 ].event_type, +DLB2_QE_EV_TYPE_WORD ); + sse_qe[0 ] =_mm_insert_epi16 (sse_qe[0 ], + ev[1 ].sub_event_type <<8 | + ev[1 ].event_type, +DLB2_QE_EV_TYPE_WORD +4 ); + sse_qe[1 ] =_mm_insert_epi16 (sse_qe[1 ], + ev[2 ].sub_event_type <<8 | + ev[2 ].event_type, +DLB2_QE_EV_TYPE_WORD ); + sse_qe[1 ] =_mm_insert_epi16 (sse_qe[1 ], + ev[3 ].sub_event_type <<8 | + ev[3 ].event_type, +DLB2_QE_EV_TYPE_WORD +4 ); + +/* Store the metadata to memory (use the double-precision +* _mm_storeh_pd because there is no integer function for +* storing the upper 64b): +* qe[0] metadata = sse_qe[0][63:0] +* qe[1] metadata = sse_qe[0][127:64] +* qe[2] metadata = sse_qe[1][63:0] +* qe[3] metadata = sse_qe[1][127:64] +*/ +_mm_storel_epi64 ((__m128i *)&qe[0 ].u.opaque_data, sse_qe[0 ]); +_mm_storeh_pd ((double *)&qe[1 ].u.opaque_data, + (__m128d )sse_qe[0 ]); +_mm_storel_epi64 ((__m128i *)&qe[2 ].u.opaque_data, sse_qe[1 ]); +_mm_storeh_pd ((double *)&qe[3 ].u.opaque_data, + (__m128d )sse_qe[1 ]); + + qe[0 ].data = ev[0 ].u64; + qe[1 ].data = ev[1 ].u64; + qe[2 ].data = ev[2 ].u64; + qe[3 ].data = ev[3 ].u64; + +break ; +case 3 : +case 2 : +case 1 : +/* At least one QE will be valid, so only zero out three */ + qe[1 ].cmd_byte =0 ; + qe[2 ].cmd_byte =0 ; + qe[3 ].cmd_byte =0 ; + +for (i =0 ; i < num; i++) { + qe[i].cmd_byte = + cmd_byte_map[qm_port->is_directed][ev[i].op]; + qe[i].sched_type = sched_type[i]; + qe[i].data = ev[i].u64; + qe[i].qid = queue_id[i]; + qe[i].priority =EV_TO_DLB2_PRIO (ev[i].priority); + qe[i].lock_id = ev[i].flow_id; +if (sched_type[i] ==DLB2_SCHED_DIRECTED ) { +struct dlb2_msg_info *info = + (struct dlb2_msg_info *)&qe[i].lock_id; + + info->qid = queue_id[i]; + info->sched_type =DLB2_SCHED_DIRECTED ; + info->priority = qe[i].priority; + } + qe[i].u.event_type.major = ev[i].event_type; + qe[i].u.event_type.sub = ev[i].sub_event_type; + } +break ; + } +} + +static inline int +dlb2_event_enqueue_prep (struct dlb2_eventdev_port *ev_port, +struct dlb2_port *qm_port, +const struct rte_event ev[], +uint8_t *sched_type, +uint8_t *queue_id) +{ +struct dlb2_eventdev *dlb2 = ev_port->dlb2; +struct dlb2_eventdev_queue *ev_queue; +uint16_t *cached_credits =NULL ; +struct dlb2_queue *qm_queue; + + ev_queue = &dlb2->ev_queues[ev->queue_id]; + qm_queue = &ev_queue->qm_queue; + *queue_id = qm_queue->id; + +/* Ignore sched_type and hardware credits on release events */ +if (ev->op ==RTE_EVENT_OP_RELEASE ) + goto op_check; + +if (!qm_queue->is_directed) { +/* Load balanced destination queue */ + +if (dlb2_check_enqueue_hw_ldb_credits (qm_port)) { + rte_errno = -ENOSPC ; +return 1 ; + } + cached_credits = &qm_port->cached_ldb_credits; + +switch (ev->sched_type) { +case RTE_SCHED_TYPE_ORDERED : +DLB2_LOG_DBG ("dlb2: put_qe: RTE_SCHED_TYPE_ORDERED\n" ); +if (qm_queue->sched_type !=RTE_SCHED_TYPE_ORDERED ) { +DLB2_LOG_ERR ("dlb2: tried to send ordered event to unordered queue %d\n" , + *queue_id); + rte_errno = -EINVAL ; +return 1 ; + } + *sched_type =DLB2_SCHED_ORDERED ; +break ; +case RTE_SCHED_TYPE_ATOMIC : +DLB2_LOG_DBG ("dlb2: put_qe: RTE_SCHED_TYPE_ATOMIC\n" ); + *sched_type =DLB2_SCHED_ATOMIC ; +break ; +case RTE_SCHED_TYPE_PARALLEL : +DLB2_LOG_DBG ("dlb2: put_qe: RTE_SCHED_TYPE_PARALLEL\n" ); +if (qm_queue->sched_type ==RTE_SCHED_TYPE_ORDERED ) + *sched_type =DLB2_SCHED_ORDERED ; +else + *sched_type =DLB2_SCHED_UNORDERED ; +break ; +default : +DLB2_LOG_ERR ("Unsupported LDB sched type in put_qe\n" ); +DLB2_INC_STAT (ev_port->stats.tx_invalid,1 ); + rte_errno = -EINVAL ; +return 1 ; + } + }else { +/* Directed destination queue */ + +if (dlb2_check_enqueue_hw_dir_credits (qm_port)) { + rte_errno = -ENOSPC ; +return 1 ; + } + cached_credits = &qm_port->cached_dir_credits; + +DLB2_LOG_DBG ("dlb2: put_qe: RTE_SCHED_TYPE_DIRECTED\n" ); + + *sched_type =DLB2_SCHED_DIRECTED ; + } + +op_check: +switch (ev->op) { +case RTE_EVENT_OP_NEW : +/* Check that a sw credit is available */ +if (dlb2_check_enqueue_sw_credits (dlb2, ev_port)) { + rte_errno = -ENOSPC ; +return 1 ; + } + ev_port->inflight_credits--; + (*cached_credits)--; +break ; +case RTE_EVENT_OP_FORWARD : +/* Check for outstanding_releases underflow. If this occurs, +* the application is not using the EVENT_OPs correctly; for +* example, forwarding or releasing events that were not +* dequeued. +*/ +RTE_ASSERT (ev_port->outstanding_releases >0 ); + ev_port->outstanding_releases--; + qm_port->issued_releases++; + (*cached_credits)--; +break ; +case RTE_EVENT_OP_RELEASE : + ev_port->inflight_credits++; +/* Check for outstanding_releases underflow. If this occurs, +* the application is not using the EVENT_OPs correctly; for +* example, forwarding or releasing events that were not +* dequeued. +*/ +RTE_ASSERT (ev_port->outstanding_releases >0 ); + ev_port->outstanding_releases--; + qm_port->issued_releases++; + +/* Replenish s/w credits if enough are cached */ +dlb2_replenish_sw_credits (dlb2, ev_port); +break ; + } + +DLB2_INC_STAT (ev_port->stats.tx_op_cnt[ev->op],1 ); +DLB2_INC_STAT (ev_port->stats.traffic.tx_ok,1 ); + +#ifndef RTE_LIBRTE_PMD_DLB2_QUELL_STATS +if (ev->op !=RTE_EVENT_OP_RELEASE ) { +DLB2_INC_STAT (ev_port->stats.queue[ev->queue_id].enq_ok,1 ); +DLB2_INC_STAT (ev_port->stats.tx_sched_cnt[*sched_type],1 ); + } +#endif + +return 0 ; +} + +static inline uint16_t +dlb2_event_enqueue_burst (void *event_port, +const struct rte_event events[], +uint16_t num) +{ +struct dlb2_eventdev_port *ev_port = event_port; +struct dlb2_port *qm_port = &ev_port->qm_port; +struct process_local_port_data *port_data; +int i, cnt; + +RTE_ASSERT (ev_port->enq_configured); +RTE_ASSERT (events !=NULL ); + + cnt =0 ; + + port_data = &dlb2_port[qm_port->id][PORT_TYPE (qm_port)]; + +for (i =0 ; i < num; i +=DLB2_NUM_QES_PER_CACHE_LINE ) { +uint8_t sched_types[DLB2_NUM_QES_PER_CACHE_LINE ]; +uint8_t queue_ids[DLB2_NUM_QES_PER_CACHE_LINE ]; +int j =0 ; + +for (; j <DLB2_NUM_QES_PER_CACHE_LINE && (i + j) < num; j++) { +const struct rte_event *ev = &events[i + j]; + +if (dlb2_event_enqueue_prep (ev_port, qm_port, ev, + &sched_types[j], + &queue_ids[j])) +break ; + } + +if (j ==0 ) +break ; + +dlb2_event_build_hcws (qm_port, &events[i], j, + sched_types, queue_ids); + +if (qm_port->token_pop_mode ==DELAYED_POP && j <4 && + qm_port->issued_releases >= qm_port->token_pop_thresh -1 ) { +dlb2_construct_token_pop_qe (qm_port, j); + +/* Reset the releases counter for the next QE batch */ + qm_port->issued_releases -= qm_port->token_pop_thresh; + } + +dlb2_hw_do_enqueue (qm_port, i ==0 , port_data); + + cnt += j; + +if (j <DLB2_NUM_QES_PER_CACHE_LINE ) +break ; + } + +if (qm_port->token_pop_mode ==DELAYED_POP && + qm_port->issued_releases >= qm_port->token_pop_thresh -1 ) { +dlb2_consume_qe_immediate (qm_port, qm_port->owed_tokens); + qm_port->issued_releases -= qm_port->token_pop_thresh; + } + +return cnt; +} + +static inline uint16_t +dlb2_event_enqueue (void *event_port, +const struct rte_event events[]) +{ +return dlb2_event_enqueue_burst (event_port, events,1 ); +} + +static uint16_t +dlb2_event_enqueue_new_burst (void *event_port, +const struct rte_event events[], +uint16_t num) +{ +return dlb2_event_enqueue_burst (event_port, events, num); +} + +static uint16_t +dlb2_event_enqueue_forward_burst (void *event_port, +const struct rte_event events[], +uint16_t num) +{ +return dlb2_event_enqueue_burst (event_port, events, num); +} +static void dlb2_entry_points_init (struct rte_eventdev *dev) {
@@ -2188,7 +2774,13 @@ dlb2_entry_points_init(struct rte_eventdev *dev) .xstats_reset = dlb2_eventdev_xstats_reset, }; +/* Expose PMD's eventdev interface */ + dev->dev_ops = &dlb2_eventdev_entry_ops; + dev->enqueue = dlb2_event_enqueue; + dev->enqueue_burst = dlb2_event_enqueue_burst; + dev->enqueue_new_burst = dlb2_event_enqueue_new_burst; + dev->enqueue_forward_burst = dlb2_event_enqueue_forward_burst; }int
Signature · 1 line
2.6.4
PATCH 16/22 event/dlb2: add dequeue and its burst variants
In reply to Timothy McDanielAdd support for dequeue, dequeue_burst, ...
--- drivers/event/dlb2/dlb2.c | 786 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 786 insertions(+)
drivers/event/dlb2/dlb2.c +786 -0
@@ -2751,9 +2751,786 @@ dlb2_event_enqueue_forward_burst(void *event_port,return dlb2_event_enqueue_burst (event_port, events, num); } +static inline void +dlb2_port_credits_inc (struct dlb2_port *qm_port,int num) +{ +uint32_t batch_size =DLB2_SW_CREDIT_BATCH_SZ ; + +/* increment port credits, and return to pool if exceeds threshold */ +if (!qm_port->is_directed) { + qm_port->cached_ldb_credits += num; +if (qm_port->cached_ldb_credits >=2 * batch_size) { +__atomic_fetch_add ( + qm_port->credit_pool[DLB2_LDB_QUEUE ], + batch_size, __ATOMIC_SEQ_CST); + qm_port->cached_ldb_credits -= batch_size; + } + }else { + qm_port->cached_dir_credits += num; +if (qm_port->cached_dir_credits >=2 * batch_size) { +__atomic_fetch_add ( + qm_port->credit_pool[DLB2_DIR_QUEUE ], + batch_size, __ATOMIC_SEQ_CST); + qm_port->cached_dir_credits -= batch_size; + } + } +} + +static inline bool +dlb2_cq_is_empty (struct dlb2_port *qm_port) +{ +volatile struct dlb2_dequeue_qe *qe_ptr; +struct dlb2_dequeue_qe qe; + + qe_ptr = dlb2_port[qm_port->id][PORT_TYPE (qm_port)].cq_base; + qe = qe_ptr[qm_port->cq_idx]; + +return (qe.cq_gen != qm_port->gen_bit); +} + +static inline int +dlb2_dequeue_wait (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port, +struct dlb2_port *qm_port, +uint64_t timeout, +uint64_t start_ticks) +{ +struct process_local_port_data *port_data; +uint64_t elapsed_ticks; + + port_data = &dlb2_port[qm_port->id][PORT_TYPE (qm_port)]; + + elapsed_ticks =rte_get_timer_cycles () - start_ticks; + +/* Wait/poll time expired */ +if (elapsed_ticks >= timeout) { +return 1 ; + }else if (dlb2->umwait_allowed) { +volatile struct dlb2_dequeue_qe *cq_base; + + cq_base = port_data->cq_base; + +/* Block on cache line write to CQ. Note: it's +* safe to access the per-process cq_base +* address here, since the PMD has already +* attempted at least one CQ dequeue. +*/ +dlb2_umonitor (&cq_base[qm_port->cq_idx]); + +/* Avoid race condition. Check if still empty */ +if (dlb2_cq_is_empty (qm_port)) { +dlb2_umwait (RTE_LIBRTE_PMD_DLB2_UMWAIT_CTL_STATE , + timeout + start_ticks); +DLB2_INC_STAT ( + ev_port->stats.traffic.rx_umonitor_umwait,1 ); + } + }else { +uint64_t poll_interval =RTE_LIBRTE_PMD_DLB2_POLL_INTERVAL ; +uint64_t curr_ticks =rte_get_timer_cycles (); +uint64_t init_ticks = curr_ticks; + +while ((curr_ticks - start_ticks < timeout) && + (curr_ticks - init_ticks < poll_interval)) + curr_ticks =rte_get_timer_cycles (); + } + +return 0 ; +} + +static inline int +dlb2_process_dequeue_qes (struct dlb2_eventdev_port *ev_port, +struct dlb2_port *qm_port, +struct rte_event *events, +struct dlb2_dequeue_qe *qes, +int cnt) +{ +uint8_t *qid_mappings = qm_port->qid_mappings; +int i, num, evq_id; + +RTE_SET_USED (ev_port);/* avoids unused variable error if stats off */ + +for (i =0 , num =0 ; i < cnt; i++) { +struct dlb2_dequeue_qe *qe = &qes[i]; +int sched_type_map[DLB2_NUM_HW_SCHED_TYPES ] = { + [DLB2_SCHED_ATOMIC ] =RTE_SCHED_TYPE_ATOMIC , + [DLB2_SCHED_UNORDERED ] =RTE_SCHED_TYPE_PARALLEL , + [DLB2_SCHED_ORDERED ] =RTE_SCHED_TYPE_ORDERED , + [DLB2_SCHED_DIRECTED ] =RTE_SCHED_TYPE_ATOMIC , + }; + +/* Fill in event information. +* Note that flow_id must be embedded in the data by +* the app, such as the mbuf RSS hash field if the data +* buffer is a mbuf. +*/ +if (unlikely (qe->error)) { +DLB2_LOG_ERR ("QE error bit ON\n" ); +DLB2_INC_STAT (ev_port->stats.traffic.rx_drop,1 ); +dlb2_consume_qe_immediate (qm_port,1 ); +continue ;/* Ignore */ + } + + events[num].u64 = qe->data; + events[num].flow_id = qe->flow_id; + events[num].priority =DLB2_TO_EV_PRIO ((uint8_t )qe->priority); + events[num].event_type = qe->u.event_type.major; + events[num].sub_event_type = qe->u.event_type.sub; + events[num].sched_type = sched_type_map[qe->sched_type]; + events[num].impl_opaque = qe->qid_depth; + +/* qid not preserved for directed queues */ +if (qm_port->is_directed) + evq_id = ev_port->link[0 ].queue_id; +else + evq_id = qid_mappings[qe->qid]; + + events[num].queue_id = evq_id; +DLB2_INC_STAT ( + ev_port->stats.queue[evq_id].qid_depth[qe->qid_depth], +1 ); +DLB2_INC_STAT (ev_port->stats.rx_sched_cnt[qe->sched_type],1 ); +DLB2_INC_STAT (ev_port->stats.traffic.rx_ok,1 ); + num++; + } + +return num; +} + +static inline int +dlb2_process_dequeue_four_qes (struct dlb2_eventdev_port *ev_port, +struct dlb2_port *qm_port, +struct rte_event *events, +struct dlb2_dequeue_qe *qes) +{ +int sched_type_map[] = { + [DLB2_SCHED_ATOMIC ] =RTE_SCHED_TYPE_ATOMIC , + [DLB2_SCHED_UNORDERED ] =RTE_SCHED_TYPE_PARALLEL , + [DLB2_SCHED_ORDERED ] =RTE_SCHED_TYPE_ORDERED , + [DLB2_SCHED_DIRECTED ] =RTE_SCHED_TYPE_ATOMIC , + }; +const int num_events =DLB2_NUM_QES_PER_CACHE_LINE ; +uint8_t *qid_mappings = qm_port->qid_mappings; +__m128i sse_evt[2 ]; + +RTE_SET_USED (ev_port);/* avoids unused variable error, if stats off */ + +/* In the unlikely case that any of the QE error bits are set, process +* them one at a time. +*/ +if (unlikely (qes[0 ].error || qes[1 ].error || + qes[2 ].error || qes[3 ].error)) +return dlb2_process_dequeue_qes (ev_port, qm_port, events, + qes, num_events); + + events[0 ].u64 = qes[0 ].data; + events[1 ].u64 = qes[1 ].data; + events[2 ].u64 = qes[2 ].data; + events[3 ].u64 = qes[3 ].data; + +/* Construct the metadata portion of two struct rte_events +* in one 128b SSE register. Event metadata is constructed in the SSE +* registers like so: +* sse_evt[0][63:0]: event[0]'s metadata +* sse_evt[0][127:64]: event[1]'s metadata +* sse_evt[1][63:0]: event[2]'s metadata +* sse_evt[1][127:64]: event[3]'s metadata +*/ + sse_evt[0 ] =_mm_setzero_si128 (); + sse_evt[1 ] =_mm_setzero_si128 (); + +/* Convert the hardware queue ID to an event queue ID and store it in +* the metadata: +* sse_evt[0][47:40] = qid_mappings[qes[0].qid] +* sse_evt[0][111:104] = qid_mappings[qes[1].qid] +* sse_evt[1][47:40] = qid_mappings[qes[2].qid] +* sse_evt[1][111:104] = qid_mappings[qes[3].qid] +*/ +#define RTE_EVENT_QUEUE_ID_BYTE 5 + sse_evt[0 ] =_mm_insert_epi8 (sse_evt[0 ], + qid_mappings[qes[0 ].qid], +RTE_EVENT_QUEUE_ID_BYTE ); + sse_evt[0 ] =_mm_insert_epi8 (sse_evt[0 ], + qid_mappings[qes[1 ].qid], +RTE_EVENT_QUEUE_ID_BYTE +8 ); + sse_evt[1 ] =_mm_insert_epi8 (sse_evt[1 ], + qid_mappings[qes[2 ].qid], +RTE_EVENT_QUEUE_ID_BYTE ); + sse_evt[1 ] =_mm_insert_epi8 (sse_evt[1 ], + qid_mappings[qes[3 ].qid], +RTE_EVENT_QUEUE_ID_BYTE +8 ); + +/* Convert the hardware priority to an event priority and store it in +* the metadata, while also returning the queue depth status +* value captured by the hardware, storing it in impl_opaque, which can +* be read by the application but not modified +* sse_evt[0][55:48] = DLB2_TO_EV_PRIO(qes[0].priority) +* sse_evt[0][63:56] = qes[0].qid_depth +* sse_evt[0][119:112] = DLB2_TO_EV_PRIO(qes[1].priority) +* sse_evt[0][127:120] = qes[1].qid_depth +* sse_evt[1][55:48] = DLB2_TO_EV_PRIO(qes[2].priority) +* sse_evt[1][63:56] = qes[2].qid_depth +* sse_evt[1][119:112] = DLB2_TO_EV_PRIO(qes[3].priority) +* sse_evt[1][127:120] = qes[3].qid_depth +*/ +#define RTE_EVENT_PRIO_IMPL_OPAQUE_WORD 3 +#define RTE_BYTE_SHIFT 8 + sse_evt[0 ] = +_mm_insert_epi16 (sse_evt[0 ], +DLB2_TO_EV_PRIO ((uint8_t )qes[0 ].priority) | + (qes[0 ].qid_depth <<RTE_BYTE_SHIFT ), +RTE_EVENT_PRIO_IMPL_OPAQUE_WORD ); + sse_evt[0 ] = +_mm_insert_epi16 (sse_evt[0 ], +DLB2_TO_EV_PRIO ((uint8_t )qes[1 ].priority) | + (qes[1 ].qid_depth <<RTE_BYTE_SHIFT ), +RTE_EVENT_PRIO_IMPL_OPAQUE_WORD +4 ); + sse_evt[1 ] = +_mm_insert_epi16 (sse_evt[1 ], +DLB2_TO_EV_PRIO ((uint8_t )qes[2 ].priority) | + (qes[2 ].qid_depth <<RTE_BYTE_SHIFT ), +RTE_EVENT_PRIO_IMPL_OPAQUE_WORD ); + sse_evt[1 ] = +_mm_insert_epi16 (sse_evt[1 ], +DLB2_TO_EV_PRIO ((uint8_t )qes[3 ].priority) | + (qes[3 ].qid_depth <<RTE_BYTE_SHIFT ), +RTE_EVENT_PRIO_IMPL_OPAQUE_WORD +4 ); + +/* Write the event type, sub event type, and flow_id to the event +* metadata. +* sse_evt[0][31:0] = qes[0].flow_id | +* qes[0].u.event_type.major << 28 | +* qes[0].u.event_type.sub << 20; +* sse_evt[0][95:64] = qes[1].flow_id | +* qes[1].u.event_type.major << 28 | +* qes[1].u.event_type.sub << 20; +* sse_evt[1][31:0] = qes[2].flow_id | +* qes[2].u.event_type.major << 28 | +* qes[2].u.event_type.sub << 20; +* sse_evt[1][95:64] = qes[3].flow_id | +* qes[3].u.event_type.major << 28 | +* qes[3].u.event_type.sub << 20; +*/ +#define RTE_EVENT_EV_TYPE_DW 0 +#define RTE_EVENT_EV_TYPE_SHIFT 28 +#define RTE_EVENT_SUB_EV_TYPE_SHIFT 20 + sse_evt[0 ] =_mm_insert_epi32 (sse_evt[0 ], + qes[0 ].flow_id | + qes[0 ].u.event_type.major <<RTE_EVENT_EV_TYPE_SHIFT | + qes[0 ].u.event_type.sub <<RTE_EVENT_SUB_EV_TYPE_SHIFT , +RTE_EVENT_EV_TYPE_DW ); + sse_evt[0 ] =_mm_insert_epi32 (sse_evt[0 ], + qes[1 ].flow_id | + qes[1 ].u.event_type.major <<RTE_EVENT_EV_TYPE_SHIFT | + qes[1 ].u.event_type.sub <<RTE_EVENT_SUB_EV_TYPE_SHIFT , +RTE_EVENT_EV_TYPE_DW +2 ); + sse_evt[1 ] =_mm_insert_epi32 (sse_evt[1 ], + qes[2 ].flow_id | + qes[2 ].u.event_type.major <<RTE_EVENT_EV_TYPE_SHIFT | + qes[2 ].u.event_type.sub <<RTE_EVENT_SUB_EV_TYPE_SHIFT , +RTE_EVENT_EV_TYPE_DW ); + sse_evt[1 ] =_mm_insert_epi32 (sse_evt[1 ], + qes[3 ].flow_id | + qes[3 ].u.event_type.major <<RTE_EVENT_EV_TYPE_SHIFT | + qes[3 ].u.event_type.sub <<RTE_EVENT_SUB_EV_TYPE_SHIFT , +RTE_EVENT_EV_TYPE_DW +2 ); + +/* Write the sched type to the event metadata. 'op' and 'rsvd' are not +* set: +* sse_evt[0][39:32] = sched_type_map[qes[0].sched_type] << 6 +* sse_evt[0][103:96] = sched_type_map[qes[1].sched_type] << 6 +* sse_evt[1][39:32] = sched_type_map[qes[2].sched_type] << 6 +* sse_evt[1][103:96] = sched_type_map[qes[3].sched_type] << 6 +*/ +#define RTE_EVENT_SCHED_TYPE_BYTE 4 +#define RTE_EVENT_SCHED_TYPE_SHIFT 6 + sse_evt[0 ] =_mm_insert_epi8 (sse_evt[0 ], + sched_type_map[qes[0 ].sched_type] <<RTE_EVENT_SCHED_TYPE_SHIFT , +RTE_EVENT_SCHED_TYPE_BYTE ); + sse_evt[0 ] =_mm_insert_epi8 (sse_evt[0 ], + sched_type_map[qes[1 ].sched_type] <<RTE_EVENT_SCHED_TYPE_SHIFT , +RTE_EVENT_SCHED_TYPE_BYTE +8 ); + sse_evt[1 ] =_mm_insert_epi8 (sse_evt[1 ], + sched_type_map[qes[2 ].sched_type] <<RTE_EVENT_SCHED_TYPE_SHIFT , +RTE_EVENT_SCHED_TYPE_BYTE ); + sse_evt[1 ] =_mm_insert_epi8 (sse_evt[1 ], + sched_type_map[qes[3 ].sched_type] <<RTE_EVENT_SCHED_TYPE_SHIFT , +RTE_EVENT_SCHED_TYPE_BYTE +8 ); + +/* Store the metadata to the event (use the double-precision +* _mm_storeh_pd because there is no integer function for storing the +* upper 64b): +* events[0].event = sse_evt[0][63:0] +* events[1].event = sse_evt[0][127:64] +* events[2].event = sse_evt[1][63:0] +* events[3].event = sse_evt[1][127:64] +*/ +_mm_storel_epi64 ((__m128i *)&events[0 ].event, sse_evt[0 ]); +_mm_storeh_pd ((double *)&events[1 ].event, (__m128d ) sse_evt[0 ]); +_mm_storel_epi64 ((__m128i *)&events[2 ].event, sse_evt[1 ]); +_mm_storeh_pd ((double *)&events[3 ].event, (__m128d ) sse_evt[1 ]); + +DLB2_INC_STAT (ev_port->stats.rx_sched_cnt[qes[0 ].sched_type],1 ); +DLB2_INC_STAT (ev_port->stats.rx_sched_cnt[qes[1 ].sched_type],1 ); +DLB2_INC_STAT (ev_port->stats.rx_sched_cnt[qes[2 ].sched_type],1 ); +DLB2_INC_STAT (ev_port->stats.rx_sched_cnt[qes[3 ].sched_type],1 ); + +DLB2_INC_STAT ( + ev_port->stats.queue[events[0 ].queue_id]. + qid_depth[qes[0 ].qid_depth], +1 ); +DLB2_INC_STAT ( + ev_port->stats.queue[events[1 ].queue_id]. + qid_depth[qes[1 ].qid_depth], +1 ); +DLB2_INC_STAT ( + ev_port->stats.queue[events[2 ].queue_id]. + qid_depth[qes[2 ].qid_depth], +1 ); +DLB2_INC_STAT ( + ev_port->stats.queue[events[3 ].queue_id]. + qid_depth[qes[3 ].qid_depth], +1 ); + +DLB2_INC_STAT (ev_port->stats.traffic.rx_ok, num_events); + +return num_events; +} + +static __rte_always_inline int +dlb2_recv_qe_sparse (struct dlb2_port *qm_port,struct dlb2_dequeue_qe *qe) +{ +volatile struct dlb2_dequeue_qe *cq_addr; +uint8_t xor_mask[2 ] = {0x0F ,0x00 }; +const uint8_t and_mask =0x0F ; +__m128i *qes = (__m128i *)qe; +uint8_t gen_bits, gen_bit; +uintptr_t addr[4 ]; +uint16_t idx; + + cq_addr = dlb2_port[qm_port->id][PORT_TYPE (qm_port)].cq_base; + + idx = qm_port->cq_idx; + +/* Load the next 4 QEs */ + addr[0 ] = (uintptr_t )&cq_addr[idx]; + addr[1 ] = (uintptr_t )&cq_addr[(idx +4 ) & qm_port->cq_depth_mask]; + addr[2 ] = (uintptr_t )&cq_addr[(idx +8 ) & qm_port->cq_depth_mask]; + addr[3 ] = (uintptr_t )&cq_addr[(idx +12 ) & qm_port->cq_depth_mask]; + +/* Prefetch next batch of QEs (all CQs occupy minimum 8 cache lines) */ +rte_prefetch0 (&cq_addr[(idx +16 ) & qm_port->cq_depth_mask]); +rte_prefetch0 (&cq_addr[(idx +20 ) & qm_port->cq_depth_mask]); +rte_prefetch0 (&cq_addr[(idx +24 ) & qm_port->cq_depth_mask]); +rte_prefetch0 (&cq_addr[(idx +28 ) & qm_port->cq_depth_mask]); + +/* Correct the xor_mask for wrap-around QEs */ + gen_bit = qm_port->gen_bit; + xor_mask[gen_bit] ^= !!((idx +4 ) > qm_port->cq_depth_mask) <<1 ; + xor_mask[gen_bit] ^= !!((idx +8 ) > qm_port->cq_depth_mask) <<2 ; + xor_mask[gen_bit] ^= !!((idx +12 ) > qm_port->cq_depth_mask) <<3 ; + +/* Read the cache lines backwards to ensure that if QE[N] (N > 0) is +* valid, then QEs[0:N-1] are too. +*/ + qes[3 ] =_mm_load_si128 ((__m128i *)(void *)addr[3 ]); +rte_compiler_barrier (); + qes[2 ] =_mm_load_si128 ((__m128i *)(void *)addr[2 ]); +rte_compiler_barrier (); + qes[1 ] =_mm_load_si128 ((__m128i *)(void *)addr[1 ]); +rte_compiler_barrier (); + qes[0 ] =_mm_load_si128 ((__m128i *)(void *)addr[0 ]); + +/* Extract and combine the gen bits */ + gen_bits = ((_mm_extract_epi8 (qes[0 ],15 ) &0x1 ) <<0 ) | + ((_mm_extract_epi8 (qes[1 ],15 ) &0x1 ) <<1 ) | + ((_mm_extract_epi8 (qes[2 ],15 ) &0x1 ) <<2 ) | + ((_mm_extract_epi8 (qes[3 ],15 ) &0x1 ) <<3 ); + +/* XOR the combined bits such that a 1 represents a valid QE */ + gen_bits ^= xor_mask[gen_bit]; + +/* Mask off gen bits we don't care about */ + gen_bits &= and_mask; + +return __builtin_popcount (gen_bits); +} + +static inline void +dlb2_inc_cq_idx (struct dlb2_port *qm_port,int cnt) +{ +uint16_t idx = qm_port->cq_idx_unmasked + cnt; + + qm_port->cq_idx_unmasked = idx; + qm_port->cq_idx = idx & qm_port->cq_depth_mask; + qm_port->gen_bit = (~(idx >> qm_port->gen_bit_shift)) &0x1 ; +} + +static int +dlb2_event_release (struct dlb2_eventdev *dlb2, +uint8_t port_id, +int n) +{ +struct process_local_port_data *port_data; +struct dlb2_eventdev_port *ev_port; +struct dlb2_port *qm_port; +int i, cnt; + +if (port_id > dlb2->num_ports) { +DLB2_LOG_ERR ("Invalid port id %d in dlb2-event_release\n" , + port_id); + rte_errno = -EINVAL ; +return rte_errno; + } + + ev_port = &dlb2->ev_ports[port_id]; + qm_port = &ev_port->qm_port; + port_data = &dlb2_port[qm_port->id][PORT_TYPE (qm_port)]; + + cnt =0 ; + +if (qm_port->is_directed) { + cnt = n; + goto sw_credit_update; + } + +for (i =0 ; i < n; i +=DLB2_NUM_QES_PER_CACHE_LINE ) { +int j; + +/* Zero-out QEs */ + qm_port->qe4[0 ].cmd_byte =0 ; + qm_port->qe4[1 ].cmd_byte =0 ; + qm_port->qe4[2 ].cmd_byte =0 ; + qm_port->qe4[3 ].cmd_byte =0 ; + +for (j =0 ; j <DLB2_NUM_QES_PER_CACHE_LINE && (i + j) < n; j++) + qm_port->qe4[j].cmd_byte =DLB2_COMP_CMD_BYTE ; + + qm_port->issued_releases += j; + +if (j ==0 ) +break ; + +if (qm_port->token_pop_mode ==DELAYED_POP && j <4 && + qm_port->issued_releases >= qm_port->token_pop_thresh -1 ) { +dlb2_construct_token_pop_qe (qm_port, j); + +/* Reset the releases counter for the next QE batch */ + qm_port->issued_releases -= qm_port->token_pop_thresh; + } + +dlb2_hw_do_enqueue (qm_port, i ==0 , port_data); + + cnt += j; + } + +if (qm_port->token_pop_mode ==DELAYED_POP && + qm_port->issued_releases >= qm_port->token_pop_thresh -1 ) { +dlb2_consume_qe_immediate (qm_port, qm_port->owed_tokens); + qm_port->issued_releases -= qm_port->token_pop_thresh; + } + +sw_credit_update: +/* each release returns one credit */ +if (!ev_port->outstanding_releases) { +DLB2_LOG_ERR ("Unrecoverable application error. Outstanding releases underflowed.\n" ); + rte_errno = -ENOTRECOVERABLE ; +return rte_errno; + } + + ev_port->outstanding_releases -= cnt; + ev_port->inflight_credits += cnt; + +/* Replenish s/w credits if enough releases are performed */ +dlb2_replenish_sw_credits (dlb2, ev_port); +return 0 ; +} + +static inline int16_t +dlb2_hw_dequeue_sparse (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port, +struct rte_event *events, +uint16_t max_num, +uint64_t dequeue_timeout_ticks) +{ +uint64_t timeout; +uint64_t start_ticks =0ULL ; +struct dlb2_port *qm_port; +int num =0 ; + + qm_port = &ev_port->qm_port; + +/* We have a special implementation for waiting. Wait can be: +* 1) no waiting at all +* 2) busy poll only +* 3) wait for interrupt. If wakeup and poll time +* has expired, then return to caller +* 4) umonitor/umwait repeatedly up to poll time +*/ + +/* If configured for per dequeue wait, then use wait value provided +* to this API. Otherwise we must use the global +* value from eventdev config time. +*/ +if (!dlb2->global_dequeue_wait) + timeout = dequeue_timeout_ticks; +else + timeout = dlb2->global_dequeue_wait_ticks; + + start_ticks =rte_get_timer_cycles (); + +while (num < max_num) { +struct dlb2_dequeue_qe qes[DLB2_NUM_QES_PER_CACHE_LINE ]; +int num_avail; + +/* Copy up to 4 QEs from the current cache line into qes */ + num_avail =dlb2_recv_qe_sparse (qm_port, qes); + +/* But don't process more than the user requested */ + num_avail =RTE_MIN (num_avail, max_num - num); + +dlb2_inc_cq_idx (qm_port, num_avail <<2 ); + +if (num_avail ==DLB2_NUM_QES_PER_CACHE_LINE ) + num +=dlb2_process_dequeue_four_qes (ev_port, + qm_port, + &events[num], + &qes[0 ]); +else if (num_avail) + num +=dlb2_process_dequeue_qes (ev_port, + qm_port, + &events[num], + &qes[0 ], + num_avail); +else if ((timeout ==0 ) || (num >0 )) +/* Not waiting in any form, or 1+ events received? */ +break ; +else if (dlb2_dequeue_wait (dlb2, ev_port, qm_port, + timeout, start_ticks)) +break ; + } + + qm_port->owed_tokens += num; + +if (num) { +if (qm_port->token_pop_mode ==AUTO_POP ) +dlb2_consume_qe_immediate (qm_port, num); + + ev_port->outstanding_releases += num; + +dlb2_port_credits_inc (qm_port, num); + } + +return num; +} + +static __rte_always_inline int +dlb2_recv_qe (struct dlb2_port *qm_port,struct dlb2_dequeue_qe *qe, +uint8_t *offset) +{ +uint8_t xor_mask[2 ][4 ] = { {0x0F ,0x0E ,0x0C ,0x08 }, + {0x00 ,0x01 ,0x03 ,0x07 } }; +uint8_t and_mask[4 ] = {0x0F ,0x0E ,0x0C ,0x08 }; +volatile struct dlb2_dequeue_qe *cq_addr; +__m128i *qes = (__m128i *)qe; +uint64_t *cache_line_base; +uint8_t gen_bits; + + cq_addr = dlb2_port[qm_port->id][PORT_TYPE (qm_port)].cq_base; + cq_addr = &cq_addr[qm_port->cq_idx]; + + cache_line_base = (void *)(((uintptr_t )cq_addr) & ~0x3F ); + *offset = ((uintptr_t )cq_addr &0x30 ) >>4 ; + +/* Load the next CQ cache line from memory. Pack these reads as tight +* as possible to reduce the chance that DLB invalidates the line while +* the CPU is reading it. Read the cache line backwards to ensure that +* if QE[N] (N > 0) is valid, then QEs[0:N-1] are too. +* +* (Valid QEs start at &qe[offset]) +*/ + qes[3 ] =_mm_load_si128 ((__m128i *)&cache_line_base[6 ]); + qes[2 ] =_mm_load_si128 ((__m128i *)&cache_line_base[4 ]); + qes[1 ] =_mm_load_si128 ((__m128i *)&cache_line_base[2 ]); + qes[0 ] =_mm_load_si128 ((__m128i *)&cache_line_base[0 ]); + +/* Evict the cache line ASAP */ +dlb2_cldemote (cache_line_base); + +/* Extract and combine the gen bits */ + gen_bits = ((_mm_extract_epi8 (qes[0 ],15 ) &0x1 ) <<0 ) | + ((_mm_extract_epi8 (qes[1 ],15 ) &0x1 ) <<1 ) | + ((_mm_extract_epi8 (qes[2 ],15 ) &0x1 ) <<2 ) | + ((_mm_extract_epi8 (qes[3 ],15 ) &0x1 ) <<3 ); + +/* XOR the combined bits such that a 1 represents a valid QE */ + gen_bits ^= xor_mask[qm_port->gen_bit][*offset]; + +/* Mask off gen bits we don't care about */ + gen_bits &= and_mask[*offset]; + +return __builtin_popcount (gen_bits); +} + +static inline int16_t +dlb2_hw_dequeue (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port, +struct rte_event *events, +uint16_t max_num, +uint64_t dequeue_timeout_ticks) +{ +uint64_t timeout; +uint64_t start_ticks =0ULL ; +struct dlb2_port *qm_port; +int num =0 ; + + qm_port = &ev_port->qm_port; + +/* We have a special implementation for waiting. Wait can be: +* 1) no waiting at all +* 2) busy poll only +* 3) wait for interrupt. If wakeup and poll time +* has expired, then return to caller +* 4) umonitor/umwait repeatedly up to poll time +*/ + +/* If configured for per dequeue wait, then use wait value provided +* to this API. Otherwise we must use the global +* value from eventdev config time. +*/ +if (!dlb2->global_dequeue_wait) + timeout = dequeue_timeout_ticks; +else + timeout = dlb2->global_dequeue_wait_ticks; + + start_ticks =rte_get_timer_cycles (); + +while (num < max_num) { +struct dlb2_dequeue_qe qes[DLB2_NUM_QES_PER_CACHE_LINE ]; +uint8_t offset; +int num_avail; + +/* Copy up to 4 QEs from the current cache line into qes */ + num_avail =dlb2_recv_qe (qm_port, qes, &offset); + +/* But don't process more than the user requested */ + num_avail =RTE_MIN (num_avail, max_num - num); + +dlb2_inc_cq_idx (qm_port, num_avail); + +if (num_avail ==DLB2_NUM_QES_PER_CACHE_LINE ) + num +=dlb2_process_dequeue_four_qes (ev_port, + qm_port, + &events[num], + &qes[offset]); +else if (num_avail) + num +=dlb2_process_dequeue_qes (ev_port, + qm_port, + &events[num], + &qes[offset], + num_avail); +else if ((timeout ==0 ) || (num >0 )) +/* Not waiting in any form, or 1+ events received? */ +break ; +else if (dlb2_dequeue_wait (dlb2, ev_port, qm_port, + timeout, start_ticks)) +break ; + } + + qm_port->owed_tokens += num; + +if (num) { +if (qm_port->token_pop_mode ==AUTO_POP ) +dlb2_consume_qe_immediate (qm_port, num); + + ev_port->outstanding_releases += num; + +dlb2_port_credits_inc (qm_port, num); + } + +return num; +} + +static uint16_t +dlb2_event_dequeue_burst (void *event_port,struct rte_event *ev,uint16_t num, +uint64_t wait) +{ +struct dlb2_eventdev_port *ev_port = event_port; +struct dlb2_port *qm_port = &ev_port->qm_port; +struct dlb2_eventdev *dlb2 = ev_port->dlb2; +uint16_t cnt; + +RTE_ASSERT (ev_port->setup_done); +RTE_ASSERT (ev !=NULL ); + +if (ev_port->implicit_release && ev_port->outstanding_releases >0 ) { +uint16_t out_rels = ev_port->outstanding_releases; + +if (dlb2_event_release (dlb2, ev_port->id, out_rels)) +return 0 ;/* rte_errno is set */ + +DLB2_INC_STAT (ev_port->stats.tx_implicit_rel, out_rels); + } + +if (qm_port->token_pop_mode ==DEFERRED_POP && qm_port->owed_tokens) +dlb2_consume_qe_immediate (qm_port, qm_port->owed_tokens); + + cnt =dlb2_hw_dequeue (dlb2, ev_port, ev, num, wait); + +DLB2_INC_STAT (ev_port->stats.traffic.total_polls,1 ); +DLB2_INC_STAT (ev_port->stats.traffic.zero_polls, ((cnt ==0 ) ?1 :0 )); + +return cnt; +} + +static uint16_t +dlb2_event_dequeue (void *event_port,struct rte_event *ev,uint64_t wait) +{ +return dlb2_event_dequeue_burst (event_port, ev,1 , wait); +} + +static uint16_t +dlb2_event_dequeue_burst_sparse (void *event_port,struct rte_event *ev, +uint16_t num,uint64_t wait) +{ +struct dlb2_eventdev_port *ev_port = event_port; +struct dlb2_port *qm_port = &ev_port->qm_port; +struct dlb2_eventdev *dlb2 = ev_port->dlb2; +uint16_t cnt; + +RTE_ASSERT (ev_port->setup_done); +RTE_ASSERT (ev !=NULL ); + +if (ev_port->implicit_release && ev_port->outstanding_releases >0 ) { +uint16_t out_rels = ev_port->outstanding_releases; + +if (dlb2_event_release (dlb2, ev_port->id, out_rels)) +return 0 ;/* rte_errno is set */ + +DLB2_INC_STAT (ev_port->stats.tx_implicit_rel, out_rels); + } + +if (qm_port->token_pop_mode ==DEFERRED_POP && qm_port->owed_tokens) +dlb2_consume_qe_immediate (qm_port, qm_port->owed_tokens); + + cnt =dlb2_hw_dequeue_sparse (dlb2, ev_port, ev, num, wait); + +DLB2_INC_STAT (ev_port->stats.traffic.total_polls,1 ); +DLB2_INC_STAT (ev_port->stats.traffic.zero_polls, ((cnt ==0 ) ?1 :0 )); +return cnt; +} + +static uint16_t +dlb2_event_dequeue_sparse (void *event_port,struct rte_event *ev, +uint64_t wait) +{ +return dlb2_event_dequeue_burst_sparse (event_port, ev,1 , wait); +} +static void dlb2_entry_points_init (struct rte_eventdev *dev) { +struct dlb2_eventdev *dlb2; +/* Expose PMD's eventdev interface */ static struct rte_eventdev_ops dlb2_eventdev_entry_ops = { .dev_infos_get = dlb2_eventdev_info_get,
@@ -2781,6 +3558,15 @@ dlb2_entry_points_init(struct rte_eventdev *dev) dev->enqueue_burst = dlb2_event_enqueue_burst; dev->enqueue_new_burst = dlb2_event_enqueue_new_burst; dev->enqueue_forward_burst = dlb2_event_enqueue_forward_burst; + + dlb2 = dev->data->dev_private; +if (dlb2->poll_mode ==DLB2_CQ_POLL_MODE_SPARSE ) { + dev->dequeue = dlb2_event_dequeue_sparse; + dev->dequeue_burst = dlb2_event_dequeue_burst_sparse; + }else { + dev->dequeue = dlb2_event_dequeue; + dev->dequeue_burst = dlb2_event_dequeue_burst; + } }int
Signature · 1 line
2.6.4
PATCH 17/22 event/dlb2: add eventdev stop and close
In reply to Timothy McDanielAdd support for eventdev stop and close entry points.
--- drivers/event/dlb2/dlb2.c | 261 +++++++++++++++++++++++++++-- drivers/event/dlb2/dlb2_iface.c | 6 + drivers/event/dlb2/dlb2_iface.h | 6 + drivers/event/dlb2/pf/base/dlb2_resource.c | 91 ++++++++++ drivers/event/dlb2/pf/dlb2_pf.c | 52 ++++++ 5 files changed, 401 insertions(+), 15 deletions(-)
drivers/event/dlb2/dlb2.c +246 -15
@@ -77,21 +77,6 @@ static struct dlb2_port_low_level_io_functions qm_mmio_fns;struct process_local_port_data dlb2_port[DLB2_MAX_NUM_PORTS ][DLB2_NUM_PORT_TYPES ]; -/* -* DUMMY - added so that xstats path will compile/link. -* Will be replaced by real version in a subsequent -* patch. -*/ -uint32_t -dlb2_get_queue_depth (struct dlb2_eventdev *dlb2, -struct dlb2_eventdev_queue *queue) -{ -RTE_SET_USED (dlb2); -RTE_SET_USED (queue); - -return 0 ; -} -static void dlb2_free_qe_mem (struct dlb2_port *qm_port) {
@@ -3527,6 +3512,250 @@ dlb2_event_dequeue_sparse(void *event_port, struct rte_event *ev, }static void +dlb2_flush_port (struct rte_eventdev *dev,int port_id) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +eventdev_stop_flush_t flush; +struct rte_event ev; +uint8_t dev_id; +void *arg; +int i; + + flush = dev->dev_ops->dev_stop_flush; + dev_id = dev->data->dev_id; + arg = dev->data->dev_stop_flush_arg; + +while (rte_event_dequeue_burst (dev_id, port_id, &ev,1 ,0 )) { +if (flush) +flush (dev_id, ev, arg); + +if (dlb2->ev_ports[port_id].qm_port.is_directed) +continue ; + + ev.op =RTE_EVENT_OP_RELEASE ; + +rte_event_enqueue_burst (dev_id, port_id, &ev,1 ); + } + +/* Enqueue any additional outstanding releases */ + ev.op =RTE_EVENT_OP_RELEASE ; + +for (i = dlb2->ev_ports[port_id].outstanding_releases; i >0 ; i--) +rte_event_enqueue_burst (dev_id, port_id, &ev,1 ); +} + +static uint32_t +dlb2_get_ldb_queue_depth (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *queue) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_get_ldb_queue_depth_args cfg; +int ret; + + cfg.queue_id = queue->qm_queue.id; + + ret =dlb2_iface_get_ldb_queue_depth (handle, &cfg); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: get_ldb_queue_depth ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); +return ret; + } + +return cfg.response.id; +} + +static uint32_t +dlb2_get_dir_queue_depth (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *queue) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_get_dir_queue_depth_args cfg; +int ret; + + cfg.queue_id = queue->qm_queue.id; + + ret =dlb2_iface_get_dir_queue_depth (handle, &cfg); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: get_dir_queue_depth ret=%d (driver status: %s)\n" , + ret, dlb2_error_strings[cfg.response.status]); +return ret; + } + +return cfg.response.id; +} + +uint32_t +dlb2_get_queue_depth (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *queue) +{ +if (queue->qm_queue.is_directed) +return dlb2_get_dir_queue_depth (dlb2, queue); +else +return dlb2_get_ldb_queue_depth (dlb2, queue); +} + +static bool +dlb2_queue_is_empty (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *queue) +{ +return dlb2_get_queue_depth (dlb2, queue) ==0 ; +} + +static bool +dlb2_linked_queues_empty (struct dlb2_eventdev *dlb2) +{ +int i; + +for (i =0 ; i < dlb2->num_queues; i++) { +if (dlb2->ev_queues[i].num_links ==0 ) +continue ; +if (!dlb2_queue_is_empty (dlb2, &dlb2->ev_queues[i])) +return false; + } + +return true; +} + +static bool +dlb2_queues_empty (struct dlb2_eventdev *dlb2) +{ +int i; + +for (i =0 ; i < dlb2->num_queues; i++) { +if (!dlb2_queue_is_empty (dlb2, &dlb2->ev_queues[i])) +return false; + } + +return true; +} + +static void +dlb2_drain (struct rte_eventdev *dev) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +struct dlb2_eventdev_port *ev_port =NULL ; +uint8_t dev_id; +int i; + + dev_id = dev->data->dev_id; + +while (!dlb2_linked_queues_empty (dlb2)) { +/* Flush all the ev_ports, which will drain all their connected +* queues. +*/ +for (i =0 ; i < dlb2->num_ports; i++) +dlb2_flush_port (dev, i); + } + +/* The queues are empty, but there may be events left in the ports. */ +for (i =0 ; i < dlb2->num_ports; i++) +dlb2_flush_port (dev, i); + +/* If the domain's queues are empty, we're done. */ +if (dlb2_queues_empty (dlb2)) +return ; + +/* Else, there must be at least one unlinked load-balanced queue. +* Select a load-balanced port with which to drain the unlinked +* queue(s). +*/ +for (i =0 ; i < dlb2->num_ports; i++) { + ev_port = &dlb2->ev_ports[i]; + +if (!ev_port->qm_port.is_directed) +break ; + } + +if (i == dlb2->num_ports) { +DLB2_LOG_ERR ("internal error: no LDB ev_ports\n" ); +return ; + } + + rte_errno =0 ; +rte_event_port_unlink (dev_id, ev_port->id,NULL ,0 ); + +if (rte_errno) { +DLB2_LOG_ERR ("internal error: failed to unlink ev_port %d\n" , + ev_port->id); +return ; + } + +for (i =0 ; i < dlb2->num_queues; i++) { +uint8_t qid, prio; +int ret; + +if (dlb2_queue_is_empty (dlb2, &dlb2->ev_queues[i])) +continue ; + + qid = i; + prio =0 ; + +/* Link the ev_port to the queue */ + ret =rte_event_port_link (dev_id, ev_port->id, &qid, &prio,1 ); +if (ret !=1 ) { +DLB2_LOG_ERR ("internal error: failed to link ev_port %d to queue %d\n" , + ev_port->id, qid); +return ; + } + +/* Flush the queue */ +while (!dlb2_queue_is_empty (dlb2, &dlb2->ev_queues[i])) +dlb2_flush_port (dev, ev_port->id); + +/* Drain any extant events in the ev_port. */ +dlb2_flush_port (dev, ev_port->id); + +/* Unlink the ev_port from the queue */ + ret =rte_event_port_unlink (dev_id, ev_port->id, &qid,1 ); +if (ret !=1 ) { +DLB2_LOG_ERR ("internal error: failed to unlink ev_port %d to queue %d\n" , + ev_port->id, qid); +return ; + } + } +} + +static void +dlb2_eventdev_stop (struct rte_eventdev *dev) +{ +/* FIXME: Handle the case that app threads are waiting in +* rte_event_dequeue_burst() (either blocked on interrupt or in umwait, +* or waiting with a timeout). Looking into proposing a new function to +* wake blocked threads that the app must call before stopping a device. +*/ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); + +rte_spinlock_lock (&dlb2->qm_instance.resource_lock); + +if (dlb2->run_state ==DLB2_RUN_STATE_STOPPED ) { +DLB2_LOG_DBG ("Internal error: already stopped\n" ); +rte_spinlock_unlock (&dlb2->qm_instance.resource_lock); +return ; + }else if (dlb2->run_state !=DLB2_RUN_STATE_STARTED ) { +DLB2_LOG_ERR ("Internal error: bad state %d for dev_stop\n" , + (int )dlb2->run_state); +rte_spinlock_unlock (&dlb2->qm_instance.resource_lock); +return ; + } + + dlb2->run_state =DLB2_RUN_STATE_STOPPING ; + +rte_spinlock_unlock (&dlb2->qm_instance.resource_lock); + +dlb2_drain (dev); + + dlb2->run_state =DLB2_RUN_STATE_STOPPED ; +} + +static int +dlb2_eventdev_close (struct rte_eventdev *dev) +{ +dlb2_hw_reset_sched_domain (dev, false); + +return 0 ; +} + +static void dlb2_entry_points_init (struct rte_eventdev *dev) {struct dlb2_eventdev *dlb2;
@@ -3536,6 +3765,8 @@ dlb2_entry_points_init(struct rte_eventdev *dev) .dev_infos_get = dlb2_eventdev_info_get, .dev_configure = dlb2_eventdev_configure, .dev_start = dlb2_eventdev_start, + .dev_stop = dlb2_eventdev_stop, + .dev_close = dlb2_eventdev_close, .queue_def_conf = dlb2_eventdev_queue_default_conf_get, .queue_setup = dlb2_eventdev_queue_setup, .port_def_conf = dlb2_eventdev_port_default_conf_get,
drivers/event/dlb2/dlb2_iface.c +6 -0
@@ -80,3 +80,9 @@ int (*dlb2_iface_pending_port_unmaps)(struct dlb2_hw_dev *handle,int (*dlb2_iface_sched_domain_start)(struct dlb2_hw_dev *handle,struct dlb2_start_domain_args *cfg); + +int (*dlb2_iface_get_ldb_queue_depth)(struct dlb2_hw_dev *handle, +struct dlb2_get_ldb_queue_depth_args *args); + +int (*dlb2_iface_get_dir_queue_depth)(struct dlb2_hw_dev *handle, +struct dlb2_get_dir_queue_depth_args *args);
drivers/event/dlb2/dlb2_iface.h +6 -0
@@ -66,4 +66,10 @@ extern int (*dlb2_iface_pending_port_unmaps)(struct dlb2_hw_dev *handle,extern int (*dlb2_iface_sched_domain_start)(struct dlb2_hw_dev *handle,struct dlb2_start_domain_args *cfg); +extern int (*dlb2_iface_get_ldb_queue_depth)(struct dlb2_hw_dev *handle, +struct dlb2_get_ldb_queue_depth_args *args); + +extern int (*dlb2_iface_get_dir_queue_depth)(struct dlb2_hw_dev *handle, +struct dlb2_get_dir_queue_depth_args *args); +#endif /* _DLB2_IFACE_H_ */
drivers/event/dlb2/pf/base/dlb2_resource.c +91 -0
94 lines not on this page · raw message
drivers/event/dlb2/pf/dlb2_pf.c +52 -0
64 lines not on this page · raw message
Signature · 1 line
2.6.4
PATCH 18/22 event/dlb2: add PMD's token pop public interface
In reply to Timothy McDanielThe PMD uses a public interface to allow applications to control the token pop mode. Supported token pop modes are as follows, and they impact core scheduling affinity for ldb ports.
AUTO_POP: Pop the CQ tokens immediately after dequeueing.DELAYED_POP: Pop CQ tokens after (dequeue_depth - 1) events
are released. Supported on load-balanced ports only.DEFERRED_POP: Pop the CQ tokens during next dequeue operation.
--- drivers/event/dlb2/meson.build | 5 ++- drivers/event/dlb2/rte_pmd_dlb2.c | 39 +++++++++++++++++++++++ drivers/event/dlb2/rte_pmd_dlb2_event_version.map | 6 ++++ 3 files changed, 49 insertions(+), 1 deletion(-) create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.c
drivers/event/dlb2/meson.build +4 -1
@@ -1,3 +1,4 @@ + # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019-2020 Intel Corporation
@@ -6,7 +7,9 @@ sources = files('dlb2.c', 'dlb2_xstats.c', 'pf/dlb2_main.c', 'pf/dlb2_pf.c', - 'pf/base/dlb2_resource.c' + 'pf/base/dlb2_resource.c', + 'rte_pmd_dlb2.c' ) deps += ['mbuf', 'mempool', 'ring', 'bus_vdev', 'pci', 'bus_pci'] +install_headers('rte_pmd_dlb2.h')
drivers/event/dlb2/rte_pmd_dlb2.c new file +39 -0
@@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2020 Intel Corporation +*/ + +#include <rte_eventdev.h> +#include <rte_eventdev_pmd.h> + +#include "rte_pmd_dlb2.h" +#include "dlb2_priv.h" +#include "dlb2_inline_fns.h" + +int +rte_pmd_dlb2_set_token_pop_mode (uint8_t dev_id, +uint8_t port_id, +enum dlb2_token_pop_mode mode) +{ +struct dlb2_eventdev *dlb2; +struct rte_eventdev *dev; + +RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET (dev_id, -EINVAL ); + dev = &rte_eventdevs[dev_id]; + + dlb2 =dlb2_pmd_priv (dev); + +if (mode >=NUM_TOKEN_POP_MODES ) +return -EINVAL ; + +/* The event device must be configured, but not yet started */ +if (!dlb2->configured || dlb2->run_state !=DLB2_RUN_STATE_STOPPED ) +return -EINVAL ; + +/* The token pop mode must be set before configuring the port */ +if (port_id >= dlb2->num_ports || dlb2->ev_ports[port_id].setup_done) +return -EINVAL ; + + dlb2->ev_ports[port_id].qm_port.token_pop_mode = mode; + +return 0 ; +}
drivers/event/dlb2/rte_pmd_dlb2_event_version.map +6 -0
@@ -1,3 +1,9 @@ DPDK_21.0 { local: *; }; + +EXPERIMENTAL { + global: + + rte_pmd_dlb2_set_token_pop_mode; +};
Signature · 1 line
2.6.4
PATCH 20/22 event/dlb2: add queue and port release
In reply to Timothy McDanielThese entry points are NO-OPS. DLB does not support reconfiguring individual queues or ports. The entire device must be reconfigured.
--- drivers/event/dlb2/dlb2.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
drivers/event/dlb2/dlb2.c +25 -0
@@ -3756,6 +3756,29 @@ dlb2_eventdev_close(struct rte_eventdev *dev) }static void +dlb2_eventdev_queue_release (struct rte_eventdev *dev,uint8_t id) +{ +RTE_SET_USED (dev); +RTE_SET_USED (id); + +/* This function intentionally left blank. dlb2 does not support +* reconfiguring individual queues or ports -- the entire device +* must be reconfigured. +*/ +} + +static void +dlb2_eventdev_port_release (void *port) +{ +RTE_SET_USED (port); + +/* This function intentionally left blank. dlb2 does not support +* reconfiguring individual queues or ports -- the entire device +* must be reconfigured. +*/ +} + +static void dlb2_entry_points_init (struct rte_eventdev *dev) {struct dlb2_eventdev *dlb2;
@@ -3769,8 +3792,10 @@ dlb2_entry_points_init(struct rte_eventdev *dev) .dev_close = dlb2_eventdev_close, .queue_def_conf = dlb2_eventdev_queue_default_conf_get, .queue_setup = dlb2_eventdev_queue_setup, + .queue_release = dlb2_eventdev_queue_release, .port_def_conf = dlb2_eventdev_port_default_conf_get, .port_setup = dlb2_eventdev_port_setup, + .port_release = dlb2_eventdev_port_release, .port_link = dlb2_eventdev_port_link, .port_unlink = dlb2_eventdev_port_unlink, .port_unlinks_in_progress =
Signature · 1 line
2.6.4
PATCH 19/22 event/dlb2: add PMD self-tests
In reply to Timothy McDanielAdd a variety of self-tests for both ldb and directed ports/queues, as well as configure, start, stop, link, etc...
--- app/test/test_eventdev.c | 9 + drivers/event/dlb2/dlb2.c | 1 + drivers/event/dlb2/dlb2_selftest.c | 1570 ++++++++++++++++++++++++++++++++++++ drivers/event/dlb2/meson.build | 3 +- 4 files changed, 1582 insertions(+), 1 deletion(-) create mode 100644 drivers/event/dlb2/dlb2_selftest.c
app/test/test_eventdev.c +9 -0
@@ -1030,6 +1030,13 @@ test_eventdev_selftest_dpaa2(void)return test_eventdev_selftest_impl ("event_dpaa2" ,"" ); } +static int +test_eventdev_selftest_dlb2 (void ) +{ +return test_eventdev_selftest_impl ("dlb2_event" ,"" ); +} + +REGISTER_TEST_COMMAND (eventdev_common_autotest, test_eventdev_common);REGISTER_TEST_COMMAND (eventdev_selftest_sw, test_eventdev_selftest_sw); REGISTER_TEST_COMMAND(eventdev_selftest_octeontx,
@@ -1037,3 +1044,5 @@ REGISTER_TEST_COMMAND(eventdev_selftest_octeontx,REGISTER_TEST_COMMAND (eventdev_selftest_octeontx2, test_eventdev_selftest_octeontx2);REGISTER_TEST_COMMAND (eventdev_selftest_dpaa2, test_eventdev_selftest_dpaa2); +REGISTER_TEST_COMMAND (eventdev_selftest_dlb2, test_eventdev_selftest_dlb2); +
drivers/event/dlb2/dlb2.c +1 -0
@@ -3780,6 +3780,7 @@ dlb2_entry_points_init(struct rte_eventdev *dev) .xstats_get_names = dlb2_eventdev_xstats_get_names, .xstats_get_by_name = dlb2_eventdev_xstats_get_by_name, .xstats_reset = dlb2_eventdev_xstats_reset, + .dev_selftest = test_dlb2_eventdev, };/* Expose PMD's eventdev interface */
drivers/event/dlb2/dlb2_selftest.c new file +1570 -0
1,570 lines not on this page · raw message
drivers/event/dlb2/meson.build +2 -1
@@ -8,7 +8,8 @@ sources = files('dlb2.c', 'pf/dlb2_main.c', 'pf/dlb2_pf.c', 'pf/base/dlb2_resource.c', - 'rte_pmd_dlb2.c' + 'rte_pmd_dlb2.c', + 'dlb2_selftest.c' ) deps += ['mbuf', 'mempool', 'ring', 'bus_vdev', 'pci', 'bus_pci']
Signature · 1 line
2.6.4
PATCH 21/22 event/dlb2: add timeout ticks entry point
In reply to Timothy McDanielAdds the timeout ticks conversion function.
--- drivers/event/dlb2/dlb2.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
drivers/event/dlb2/dlb2.c +14 -0
@@ -3778,6 +3778,19 @@ dlb2_eventdev_port_release(void *port)*/ } +static int +dlb2_eventdev_timeout_ticks (struct rte_eventdev *dev,uint64_t ns, +uint64_t *timeout_ticks) +{ +RTE_SET_USED (dev); +uint64_t cycles_per_ns; + + cycles_per_ns =rte_get_timer_hz () /1E9 ; + *timeout_ticks = ns * cycles_per_ns; + +return 0 ; +} +static void dlb2_entry_points_init (struct rte_eventdev *dev) {
@@ -3800,6 +3813,7 @@ dlb2_entry_points_init(struct rte_eventdev *dev) .port_unlink = dlb2_eventdev_port_unlink, .port_unlinks_in_progress = dlb2_eventdev_port_unlinks_in_progress, + .timeout_ticks = dlb2_eventdev_timeout_ticks, .dump = dlb2_eventdev_dump, .xstats_get = dlb2_eventdev_xstats_get, .xstats_get_names = dlb2_eventdev_xstats_get_names,
Signature · 1 line
2.6.4
PATCH 22/22 doc: add new DLB2 eventdev driver to relnotes
In reply to Timothy McDanielAdded announcement of availabililty for the new driver for Intel Dynamic Load Balancer V2.0 hardware.
--- doc/guides/rel_notes/release_20_11.rst | 5 +++++ 1 file changed, 5 insertions(+)
doc/guides/rel_notes/release_20_11.rst +5 -0
@@ -55,6 +55,11 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Added a new driver for the Intel Dynamic Load Balancer v2.0 device.** + + Added the new ``dlb2`` eventdev driver for the Intel DLB V2.0 device. See the + :doc:`../eventdevs/dlb2` eventdev guide for more details on this new driver. + Removed Items -------------
Signature · 1 line
2.6.4
Re: [dpdk-dev] [PATCH 07/22] event/dlb2: add xstats
In reply to Timothy McDaniel<snip>
Quoted 12 lines
drivers/event/dlb2/dlb2_xstats.c new file +12
-0+dlb2_eventdev_dump(struct rte_eventdev *dev, FILE *f) { + struct dlb2_eventdev *dlb2; +struct dlb2_hw_dev *handle; +int i; + +if (!f) { +printf ("Invalid file pointer\n" ); +return ; + } + +if (!dev) { +fprintf (f,"Invalid event device\n" );
drivers/event/dlb2/dlb2_xstats.c line 1018
Quoted 10 lines
drivers/event/dlb2/dlb2_xstats.c new file +10
-0+return ; + } + + dlb2 =dlb2_pmd_priv (dev); + +if (!dlb2) { +fprintf (f,"DLB2 Event device cannot be dumped!\n" ); +return ; + } +
Not sure if this is enforced. The DPDK coding style discourages using ! on pointers ( see section 1. 8.1 at https://doc.dpdk.org/guides/contributing/coding_style.html).
Quoted 1,380 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Timothy McDaniel
Sent: Friday, September 11, 2020 4:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [dpdk-dev] [PATCH 07/22] event/dlb2: add xstatsAdd support for DLB2 xstats. Perform initialization and add standard xstats entry points.
Signed-off-by Timothy McDaniel <[email protected]>--- drivers/event/dlb2/dlb2.c | 35 +- drivers/event/dlb2/dlb2_xstats.c | 1269 ++++++++++++++++++++++++++++++++++++++ drivers/event/dlb2/meson.build | 1 + 3 files changed, 1302 insertions(+), 3 deletions(-) create mode 100644
drivers/event/dlb2/dlb2_xstats.cdrivers/event/dlb2/dlb2.c index renamed from drivers/event/dlb2/dlb2.c +0
-07ff7dac..0d6fea4 100644
drivers/event/dlb2/dlb2.c +0
-0@@ -77,6 +77,21 @@ static struct dlb2_port_low_level_io_functions
qm_mmio_fns; structprocess_local_port_datadlb2_port[DLB2_MAX_NUM_PORTS][DLB2_NUM_PORT_TYPES];drivers/event/dlb2/dlb2.c +14
-0+/* +* DUMMY - added so that xstats path will compile/link. +* Will be replaced by real version in a subsequent +* patch. +*/ +uint32_t +dlb2_get_queue_depth (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *queue) { +RTE_SET_USED (dlb2); +RTE_SET_USED (queue); + +return 0 ; +} + /* override defaults with value(s) provided on command line */ static voiddlb2_init_queue_depth_thresholds(struct
dlb2_eventdev*dlb2, @@ -353,9 +368,16 @@ set_qid_depth_thresh(const char *key__rte_unused, static voiddlb2_entry_points_init(struct rte_eventdev *dev){drivers/event/dlb2/dlb2.c +17
-3-RTE_SET_USED (dev); - -/* Eventdev PMD entry points */ +/* Expose PMD's eventdev interface */ +static struct rte_eventdev_ops dlb2_eventdev_entry_ops = { + .dump = dlb2_eventdev_dump, + .xstats_get = dlb2_eventdev_xstats_get, + .xstats_get_names = dlb2_eventdev_xstats_get_names, + .xstats_get_by_name = dlb2_eventdev_xstats_get_by_name, + .xstats_reset = dlb2_eventdev_xstats_reset, + }; + + dev->dev_ops = &dlb2_eventdev_entry_ops; }int @@ -411,6 +433,13 @@ dlb2_primary_eventdev_probe(struct rte_eventdev *dev, return err; } +/* Complete xtstats runtime initialization */ + err =dlb2_xstats_init (dlb2); +if (err) { +DLB2_LOG_ERR ("dlb2: failed to init xstats, err=%d\n" , err); +return err; + } +/* Initialize each port's token pop mode */ for (i =0 ; i <DLB2_MAX_NUM_PORTS ; i++)dlb2->ev_ports[i].qm_port.token_pop_mode =
AUTO_POP; diff
--gita/drivers/event/dlb2/dlb2_xstats.cb/drivers/event/dlb2/dlb2_xstats.cnew file mode 100644
index 0000000..9a69d78drivers/event/dlb2/dlb2_xstats.c new file +42
-0@@ -0,0 +1,1269 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation */ + +#include <stdint.h> +#include <stdbool.h> +#include <stdio.h> +#include <sys/mman.h> +#include <sys/fcntl.h> +#include <errno.h> +#include <assert.h> +#include <unistd.h> +#include <string.h> + +#include <rte_debug.h> +#include <rte_log.h> +#include <rte_dev.h> +#include <rte_mbuf.h> +#include <rte_ring.h> +#include <rte_errno.h> +#include <rte_kvargs.h> +#include <rte_malloc.h> +#include <rte_cycles.h> +#include <rte_io.h> +#include <rte_eventdev.h> +#include <rte_eventdev_pmd.h> +#include <rte_eventdev_pmd_vdev.h> + +#include "dlb2_priv.h" +#include "dlb2_inline_fns.h" + +enum dlb2_xstats_type { +/* common to device and port */ + rx_ok,/**< Receive an event */ + rx_drop,/**< Error bit set in received QE */ + rx_interrupt_wait,/**< Wait on an interrupt */ + rx_umonitor_umwait,/**< Block using umwait */ + tx_ok,/**< Transmit an event */ + total_polls,/**< Call dequeue_burst */ + zero_polls,/**< Call dequeue burst and return 0 */ + tx_nospc_ldb_hw_credits,/**< Insufficient LDB h/w credits */ + tx_nospc_dir_hw_credits,/**< Insufficient DIR h/w credits */ + tx_nospc_inflight_max,/**< Reach the new_event_threshold */
drivers/event/dlb2/dlb2_xstats.c new file +9
-0+ tx_nospc_new_event_limit,/**< Insufficient s/w credits */ + tx_nospc_inflight_credits,/**< Port has too few s/w credits */ +/* device specific */ + nb_events_limit, + inflight_events, + ldb_pool_size, + dir_pool_size, +/* port specific */ + tx_new, /**< Send an OP_NEW event*/
drivers/event/dlb2/dlb2_xstats.c new file +4
-0+ tx_fwd,/**< Send an OP_FORWARD event */ + tx_rel,/**< Send an OP_RELEASE event */ + tx_implicit_rel,/**< Issue an implicit event release */ + tx_sched_ordered, /**< Send a SCHED_TYPE_ORDEREDevent */
drivers/event/dlb2/dlb2_xstats.c new file +1
-0+ tx_sched_unordered, /**< Send a SCHED_TYPE_PARALLELevent */
drivers/event/dlb2/dlb2_xstats.c new file +1
-0+ tx_sched_atomic, /**< Send a SCHED_TYPE_ATOMICevent */
drivers/event/dlb2/dlb2_xstats.c new file +4
-0+ tx_sched_directed,/**< Send a directed event */ + tx_invalid,/**< Send an event with an invalid op */ + outstanding_releases,/**< # of releases a port owes */ + max_outstanding_releases, /**< max # of releases a port can owe*/
drivers/event/dlb2/dlb2_xstats.c new file +13
-0+ rx_sched_ordered,/**< Dequeue an ordered event */ + rx_sched_unordered,/**< Dequeue an unordered event */ + rx_sched_atomic,/**< Dequeue an atomic event */ + rx_sched_directed,/**< Dequeue an directed event */ + rx_sched_invalid,/**< Dequeue event sched type invalid */ +/* common to port and queue */ + is_configured,/**< Port is configured */ + is_load_balanced,/**< Port is LDB */ + hw_id,/**< Hardware ID */ +/* queue specific */ + num_links,/**< Number of ports linked */ + sched_type,/**< Queue sched type */ + enq_ok,/**< # events enqueued to the queue */
drivers/event/dlb2/dlb2_xstats.c new file +6
-0+ current_depth,/**< Current queue depth */ + depth_threshold,/**< Programmed depth threshold */ + depth_le50_threshold, +/**< Depth LE to 50% of the configured hardware threshold */ + depth_gt50_le75_threshold, +/**< Depth GT 50%, but LE to 75% of the configured hardware threshold */
drivers/event/dlb2/dlb2_xstats.c new file +158
-0+ depth_gt75_le100_threshold, +/**< Depth GT 75%. but LE to the configured hardware threshold */ + depth_gt100_threshold + /**< Depth GT 100% of the configured hw threshold */ }; + +typedef uint64_t (*dlb2_xstats_fn )(struct dlb2_eventdev *dlb2, +uint16_t obj_idx,/* port or queue id */ +enum dlb2_xstats_type stat,int extra_arg); + +enum dlb2_xstats_fn_type { +DLB2_XSTATS_FN_DEV , +DLB2_XSTATS_FN_PORT , +DLB2_XSTATS_FN_QUEUE +}; + +struct dlb2_xstats_entry { +struct rte_event_dev_xstats_name name; +uint64_t reset_value;/* an offset to be taken away to emulate resets */ +enum dlb2_xstats_fn_type fn_id; +enum dlb2_xstats_type stat; +enum rte_event_dev_xstats_mode mode; +int extra_arg; +uint16_t obj_idx; +uint8_t reset_allowed;/* when set, this value can be reset */ }; + +/* Some device stats are simply a summation of the corresponding port +values */ static uint64_t dlb2_device_traffic_stat_get (struct +dlb2_eventdev *dlb2, +int which_stat) +{ +int i; +uint64_t val =0 ; + +for (i =0 ; i <DLB2_MAX_NUM_PORTS ; i++) { +struct dlb2_eventdev_port *port = &dlb2->ev_ports[i]; + +if (!port->setup_done) +continue ; + +switch (which_stat) { +case rx_ok: + val += port->stats.traffic.rx_ok; +break ; +case rx_drop: + val += port->stats.traffic.rx_drop; +break ; +case rx_interrupt_wait: + val += port->stats.traffic.rx_interrupt_wait; +break ; +case rx_umonitor_umwait: + val += port->stats.traffic.rx_umonitor_umwait; +break ; +case tx_ok: + val += port->stats.traffic.tx_ok; +break ; +case total_polls: + val += port->stats.traffic.total_polls; +break ; +case zero_polls: + val += port->stats.traffic.zero_polls; +break ; +case tx_nospc_ldb_hw_credits: + val += port->stats.traffic.tx_nospc_ldb_hw_credits; +break ; +case tx_nospc_dir_hw_credits: + val += port->stats.traffic.tx_nospc_dir_hw_credits; +break ; +case tx_nospc_inflight_max: + val += port->stats.traffic.tx_nospc_inflight_max; +break ; +case tx_nospc_new_event_limit: + val += port->stats.traffic.tx_nospc_new_event_limit; +break ; +case tx_nospc_inflight_credits: + val += port->stats.traffic.tx_nospc_inflight_credits; +break ; +default : +return -1 ; + } + } +return val; +} + +static uint64_t +get_dev_stat(struct dlb2_eventdev *dlb2, uint16_t obj_idx __rte_unused, + enum dlb2_xstats_type type, int extra_arg __rte_unused) { +switch (type) { +case rx_ok: +case rx_drop: +case rx_interrupt_wait: +case rx_umonitor_umwait: +case tx_ok: +case total_polls: +case zero_polls: +case tx_nospc_ldb_hw_credits: +case tx_nospc_dir_hw_credits: +case tx_nospc_inflight_max: +case tx_nospc_new_event_limit: +case tx_nospc_inflight_credits: +return dlb2_device_traffic_stat_get (dlb2, type); +case nb_events_limit: +return dlb2->new_event_limit; +case inflight_events: +return __atomic_load_n (&dlb2->inflights, __ATOMIC_SEQ_CST); +case ldb_pool_size: +return dlb2->num_ldb_credits; +case dir_pool_size: +return dlb2->num_dir_credits; +default :return -1 ; + } +} + +static uint64_t +get_port_stat (struct dlb2_eventdev *dlb2,uint16_t obj_idx, + enum dlb2_xstats_type type, int extra_arg __rte_unused) { +struct dlb2_eventdev_port *ev_port = &dlb2->ev_ports[obj_idx]; + +switch (type) { +case rx_ok:return ev_port->stats.traffic.rx_ok; + +case rx_drop:return ev_port->stats.traffic.rx_drop; + +case rx_interrupt_wait:return +ev_port->stats.traffic.rx_interrupt_wait; + +case rx_umonitor_umwait: +return ev_port->stats.traffic.rx_umonitor_umwait; + +case tx_ok:return ev_port->stats.traffic.tx_ok; + +case total_polls:return ev_port->stats.traffic.total_polls; + +case zero_polls:return ev_port->stats.traffic.zero_polls; + +case tx_nospc_ldb_hw_credits: +return ev_port->stats.traffic.tx_nospc_ldb_hw_credits; + +case tx_nospc_dir_hw_credits: +return ev_port->stats.traffic.tx_nospc_dir_hw_credits; + +case tx_nospc_inflight_max: +return ev_port->stats.traffic.tx_nospc_inflight_max; + +case tx_nospc_new_event_limit: +return ev_port->stats.traffic.tx_nospc_new_event_limit; + +case tx_nospc_inflight_credits: +return ev_port->stats.traffic.tx_nospc_inflight_credits; + +case is_configured:return ev_port->setup_done; + +case is_load_balanced:return !ev_port->qm_port.is_directed; + +case hw_id:return ev_port->qm_port.id; + +case tx_new:return ev_port->stats.tx_op_cnt[RTE_EVENT_OP_NEW ]; + + case tx_fwd: return ev_port-Quoted 1 line
stats.tx_op_cnt[RTE_EVENT_OP_FORWARD];
drivers/event/dlb2/dlb2_xstats.c new file +10
-0+ +case tx_rel:return ev_port->stats.tx_op_cnt[RTE_EVENT_OP_RELEASE ]; + +case tx_implicit_rel:return ev_port->stats.tx_implicit_rel; + +case tx_sched_ordered: +return ev_port->stats.tx_sched_cnt[DLB2_SCHED_ORDERED ]; + +case tx_sched_unordered: + return ev_port-Quoted 1 line
stats.tx_sched_cnt[DLB2_SCHED_UNORDERED];
drivers/event/dlb2/dlb2_xstats.c new file +19
-0+ +case tx_sched_atomic: +return ev_port->stats.tx_sched_cnt[DLB2_SCHED_ATOMIC ]; + +case tx_sched_directed: +return ev_port->stats.tx_sched_cnt[DLB2_SCHED_DIRECTED ]; + +case tx_invalid:return ev_port->stats.tx_invalid; + +case outstanding_releases:return ev_port->outstanding_releases; + +case max_outstanding_releases: +return DLB2_NUM_HIST_LIST_ENTRIES_PER_LDB_PORT ; + +case rx_sched_ordered: +return ev_port->stats.rx_sched_cnt[DLB2_SCHED_ORDERED ]; + +case rx_sched_unordered: + return ev_port-Quoted 1 line
stats.rx_sched_cnt[DLB2_SCHED_UNORDERED];
drivers/event/dlb2/dlb2_xstats.c new file +303
-0+ +case rx_sched_atomic: +return ev_port->stats.rx_sched_cnt[DLB2_SCHED_ATOMIC ]; + +case rx_sched_directed: +return ev_port->stats.rx_sched_cnt[DLB2_SCHED_DIRECTED ]; + +case rx_sched_invalid:return ev_port->stats.rx_sched_invalid; + +default :return -1 ; + } +} + +static uint64_t +dlb2_get_threshold_stat (struct dlb2_eventdev *dlb2,int qid,int stat) +{ +int port =0 ; +uint64_t tally =0 ; + +for (port =0 ; port <DLB2_MAX_NUM_PORTS ; port++) + tally += dlb2->ev_ports[port].stats.queue[qid].qid_depth[stat]; + +return tally; +} + +static uint64_t +dlb2_get_enq_ok_stat (struct dlb2_eventdev *dlb2,int qid) { +int port =0 ; +uint64_t enq_ok_tally =0 ; + +for (port =0 ; port <DLB2_MAX_NUM_PORTS ; port++) + enq_ok_tally += dlb2->ev_ports[port].stats.queue[qid].enq_ok; + +return enq_ok_tally; +} + +static uint64_t +get_queue_stat (struct dlb2_eventdev *dlb2,uint16_t obj_idx, + enum dlb2_xstats_type type, int extra_arg __rte_unused) { +struct dlb2_eventdev_queue *ev_queue = + &dlb2->ev_queues[obj_idx]; + +switch (type) { +case is_configured:return ev_queue->setup_done; + +case is_load_balanced:return !ev_queue->qm_queue.is_directed; + +case hw_id:return ev_queue->qm_queue.id; + +case num_links:return ev_queue->num_links; + +case sched_type:return ev_queue->qm_queue.sched_type; + +case enq_ok:return dlb2_get_enq_ok_stat (dlb2, obj_idx); + +case current_depth:return dlb2_get_queue_depth (dlb2, ev_queue); + +case depth_threshold:return ev_queue->depth_threshold; + +case depth_le50_threshold: +return dlb2_get_threshold_stat (dlb2, ev_queue->id, +DLB2_QID_DEPTH_LE50 ); + +case depth_gt50_le75_threshold: +return dlb2_get_threshold_stat (dlb2, ev_queue->id, +DLB2_QID_DEPTH_GT50_LE75 ); + +case depth_gt75_le100_threshold: +return dlb2_get_threshold_stat (dlb2, ev_queue->id, +DLB2_QID_DEPTH_GT75_LE100 ); + +case depth_gt100_threshold: +return dlb2_get_threshold_stat (dlb2, ev_queue->id, +DLB2_QID_DEPTH_GT100 ); + +default :return -1 ; + } +} + +int +dlb2_xstats_init (struct dlb2_eventdev *dlb2) { +/* +* define the stats names and types. Used to build up the device +* xstats array +* There are multiple set of stats: +* - device-level, +* - per-port, +* - per-qid, +* +* For each of these sets, we have three parallel arrays, one for the +* names, the other for the stat type parameter to be passed in the fn +* call to get that stat. The third array allows resetting or not. +* All these arrays must be kept in sync +*/ +static const char *const dev_stats[] = { +"rx_ok" , +"rx_drop" , +"rx_interrupt_wait" , +"rx_umonitor_umwait" , +"tx_ok" , +"total_polls" , +"zero_polls" , +"tx_nospc_ldb_hw_credits" , +"tx_nospc_dir_hw_credits" , +"tx_nospc_inflight_max" , +"tx_nospc_new_event_limit" , +"tx_nospc_inflight_credits" , +"nb_events_limit" , +"inflight_events" , +"ldb_pool_size" , +"dir_pool_size" , + }; +static const enum dlb2_xstats_type dev_types[] = { + rx_ok, + rx_drop, + rx_interrupt_wait, + rx_umonitor_umwait, + tx_ok, + total_polls, + zero_polls, + tx_nospc_ldb_hw_credits, + tx_nospc_dir_hw_credits, + tx_nospc_inflight_max, + tx_nospc_new_event_limit, + tx_nospc_inflight_credits, + nb_events_limit, + inflight_events, + ldb_pool_size, + dir_pool_size, + }; +/* Note: generated device stats are not allowed to be reset. */ +static const uint8_t dev_reset_allowed[] = { +0 ,/* rx_ok */ +0 ,/* rx_drop */ +0 ,/* rx_interrupt_wait */ +0 ,/* rx_umonitor_umwait */ +0 ,/* tx_ok */ +0 ,/* total_polls */ +0 ,/* zero_polls */ +0 ,/* tx_nospc_ldb_hw_credits */ +0 ,/* tx_nospc_dir_hw_credits */ +0 ,/* tx_nospc_inflight_max */ +0 ,/* tx_nospc_new_event_limit */ +0 ,/* tx_nospc_inflight_credits */ +0 ,/* nb_events_limit */ +0 ,/* inflight_events */ +0 ,/* ldb_pool_size */ +0 ,/* dir_pool_size */ + }; +static const char *const port_stats[] = { +"is_configured" , +"is_load_balanced" , +"hw_id" , +"rx_ok" , +"rx_drop" , +"rx_interrupt_wait" , +"rx_umonitor_umwait" , +"tx_ok" , +"total_polls" , +"zero_polls" , +"tx_nospc_ldb_hw_credits" , +"tx_nospc_dir_hw_credits" , +"tx_nospc_inflight_max" , +"tx_nospc_new_event_limit" , +"tx_nospc_inflight_credits" , +"tx_new" , +"tx_fwd" , +"tx_rel" , +"tx_implicit_rel" , +"tx_sched_ordered" , +"tx_sched_unordered" , +"tx_sched_atomic" , +"tx_sched_directed" , +"tx_invalid" , +"outstanding_releases" , +"max_outstanding_releases" , +"rx_sched_ordered" , +"rx_sched_unordered" , +"rx_sched_atomic" , +"rx_sched_directed" , +"rx_sched_invalid" + }; +static const enum dlb2_xstats_type port_types[] = { + is_configured, + is_load_balanced, + hw_id, + rx_ok, + rx_drop, + rx_interrupt_wait, + rx_umonitor_umwait, + tx_ok, + total_polls, + zero_polls, + tx_nospc_ldb_hw_credits, + tx_nospc_dir_hw_credits, + tx_nospc_inflight_max, + tx_nospc_new_event_limit, + tx_nospc_inflight_credits, + tx_new, + tx_fwd, + tx_rel, + tx_implicit_rel, + tx_sched_ordered, + tx_sched_unordered, + tx_sched_atomic, + tx_sched_directed, + tx_invalid, + outstanding_releases, + max_outstanding_releases, + rx_sched_ordered, + rx_sched_unordered, + rx_sched_atomic, + rx_sched_directed, + rx_sched_invalid + }; +static const uint8_t port_reset_allowed[] = { +0 ,/* is_configured */ +0 ,/* is_load_balanced */ +0 ,/* hw_id */ +1 ,/* rx_ok */ +1 ,/* rx_drop */ +1 ,/* rx_interrupt_wait */ +1 ,/* rx_umonitor_umwait */ +1 ,/* tx_ok */ +1 ,/* total_polls */ +1 ,/* zero_polls */ +1 ,/* tx_nospc_ldb_hw_credits */ +1 ,/* tx_nospc_dir_hw_credits */ +1 ,/* tx_nospc_inflight_max */ +1 ,/* tx_nospc_new_event_limit */ +1 ,/* tx_nospc_inflight_credits */ +1 ,/* tx_new */ +1 ,/* tx_fwd */ +1 ,/* tx_rel */ +1 ,/* tx_implicit_rel */ +1 ,/* tx_sched_ordered */ +1 ,/* tx_sched_unordered */ +1 ,/* tx_sched_atomic */ +1 ,/* tx_sched_directed */ +1 ,/* tx_invalid */ +0 ,/* outstanding_releases */ +0 ,/* max_outstanding_releases */ +1 ,/* rx_sched_ordered */ +1 ,/* rx_sched_unordered */ +1 ,/* rx_sched_atomic */ +1 ,/* rx_sched_directed */ +1 /* rx_sched_invalid */ + }; + +/* QID specific stats */ +static const char *const qid_stats[] = { +"is_configured" , +"is_load_balanced" , +"hw_id" , +"num_links" , +"sched_type" , +"enq_ok" , +"current_depth" , +"depth_threshold" , +"depth_le50_threshold" , +"depth_gt50_le75_threshold" , +"depth_gt75_le100_threshold" , +"depth_gt100_threshold" , + }; +static const enum dlb2_xstats_type qid_types[] = { + is_configured, + is_load_balanced, + hw_id, + num_links, + sched_type, + enq_ok, + current_depth, + depth_threshold, + depth_le50_threshold, + depth_gt50_le75_threshold, + depth_gt75_le100_threshold, + depth_gt100_threshold, + }; +static const uint8_t qid_reset_allowed[] = { +0 ,/* is_configured */ +0 ,/* is_load_balanced */ +0 ,/* hw_id */ +0 ,/* num_links */ +0 ,/* sched_type */ +1 ,/* enq_ok */ +0 ,/* current_depth */ +0 ,/* depth_threshold */ +1 ,/* depth_le50_threshold */ +1 ,/* depth_gt50_le75_threshold */ +1 ,/* depth_gt75_le100_threshold */ +1 ,/* depth_gt100_threshold */ + }; + +/* ---- end of stat definitions ---- */ + +/* check sizes, since a missed comma can lead to strings being +* joined by the compiler. +*/ +RTE_BUILD_BUG_ON (RTE_DIM (dev_stats) !=RTE_DIM (dev_types)); +RTE_BUILD_BUG_ON (RTE_DIM (port_stats) !=RTE_DIM (port_types)); +RTE_BUILD_BUG_ON (RTE_DIM (qid_stats) !=RTE_DIM (qid_types)); + + RTE_BUILD_BUG_ON(RTE_DIM(dev_stats) !=
RTE_DIM(dev_reset_allowed));drivers/event/dlb2/dlb2_xstats.c new file +1
-0+ RTE_BUILD_BUG_ON(RTE_DIM(port_stats) !=
RTE_DIM(port_reset_allowed));drivers/event/dlb2/dlb2_xstats.c new file +1
-0+ RTE_BUILD_BUG_ON(RTE_DIM(qid_stats) !=
RTE_DIM(qid_reset_allowed));drivers/event/dlb2/dlb2_xstats.c new file +46
-0+ +/* other vars */ +const unsigned int count =RTE_DIM (dev_stats) + +DLB2_MAX_NUM_PORTS *RTE_DIM (port_stats) + +DLB2_MAX_NUM_QUEUES *RTE_DIM (qid_stats); +unsigned int i, port, qid, stat_id =0 ; + + dlb2->xstats =rte_zmalloc_socket (NULL , +sizeof (dlb2->xstats[0 ]) * count,0 , + dlb2->qm_instance.info.socket_id); +if (dlb2->xstats ==NULL ) +return -ENOMEM ; + +#define sname dlb2->xstats[stat_id].name.name +for (i =0 ; i <RTE_DIM (dev_stats); i++, stat_id++) { + dlb2->xstats[stat_id] = (struct dlb2_xstats_entry ) { + .fn_id =DLB2_XSTATS_FN_DEV , + .stat = dev_types[i], + .mode =RTE_EVENT_DEV_XSTATS_DEVICE , + .reset_allowed = dev_reset_allowed[i], + }; +snprintf (sname,sizeof (sname),"dev_%s" , dev_stats[i]); + } + dlb2->xstats_count_mode_dev = stat_id; + +for (port =0 ; port <DLB2_MAX_NUM_PORTS ; port++) { + dlb2->xstats_offset_for_port[port] = stat_id; + +uint32_t count_offset = stat_id; + +for (i =0 ; i <RTE_DIM (port_stats); i++, stat_id++) { + dlb2->xstats[stat_id] = (struct dlb2_xstats_entry ){ + .fn_id =DLB2_XSTATS_FN_PORT , + .obj_idx = port, + .stat = port_types[i], + .mode =RTE_EVENT_DEV_XSTATS_PORT , + .reset_allowed = port_reset_allowed[i], + }; +snprintf (sname,sizeof (sname),"port_%u_%s" , + port, port_stats[i]); + } + + dlb2->xstats_count_per_port[port] = stat_id - count_offset; + } + + dlb2->xstats_count_mode_port = stat_id - dlb2-Quoted 1 line
xstats_count_mode_dev;drivers/event/dlb2/dlb2_xstats.c new file +23
-0+ +for (qid =0 ; qid <DLB2_MAX_NUM_QUEUES ; qid++) { +uint32_t count_offset = stat_id; + + dlb2->xstats_offset_for_qid[qid] = stat_id; + +for (i =0 ; i <RTE_DIM (qid_stats); i++, stat_id++) { + dlb2->xstats[stat_id] = (struct dlb2_xstats_entry ){ + .fn_id =DLB2_XSTATS_FN_QUEUE , + .obj_idx = qid, + .stat = qid_types[i], + .mode =RTE_EVENT_DEV_XSTATS_QUEUE , + .reset_allowed = qid_reset_allowed[i], + }; +snprintf (sname,sizeof (sname),"qid_%u_%s" , + qid, qid_stats[i]); + } + + dlb2->xstats_count_per_qid[qid] = stat_id - count_offset; + } + + dlb2->xstats_count_mode_queue = stat_id - + (dlb2->xstats_count_mode_dev + dlb2-Quoted 1 line
xstats_count_mode_port); #undefdrivers/event/dlb2/dlb2_xstats.c new file +16
-0+sname + + dlb2->xstats_count = stat_id; + +return 0 ; +} + +void +dlb2_xstats_uninit (struct dlb2_eventdev *dlb2) { +rte_free (dlb2->xstats); + dlb2->xstats_count =0 ; +} + +int +dlb2_eventdev_xstats_get_names (const struct rte_eventdev *dev, + enum rte_event_dev_xstats_mode mode, uint8_t
queue_port_id,drivers/event/dlb2/dlb2_xstats.c new file +20
-0+struct rte_event_dev_xstats_name *xstats_names, + unsigned int *ids, unsigned int size) { + const struct dlb2_eventdev *dlb2 = dlb2_pmd_priv(dev); +unsigned int i; +unsigned int xidx =0 ; + +RTE_SET_USED (mode); +RTE_SET_USED (queue_port_id); + +uint32_t xstats_mode_count =0 ; +uint32_t start_offset =0 ; + +switch (mode) { +case RTE_EVENT_DEV_XSTATS_DEVICE : + xstats_mode_count = dlb2->xstats_count_mode_dev; +break ; +case RTE_EVENT_DEV_XSTATS_PORT : +if (queue_port_id >=DLB2_MAX_NUM_PORTS ) +break ; + xstats_mode_count = dlb2-Quoted 1 line
xstats_count_per_port[queue_port_id];
drivers/event/dlb2/dlb2_xstats.c new file +8
-0+ start_offset = dlb2->xstats_offset_for_port[queue_port_id]; +break ; +case RTE_EVENT_DEV_XSTATS_QUEUE : +#if (DLB2_MAX_NUM_QUEUES <=255 )/* max 8 bit value */ +if (queue_port_id >=DLB2_MAX_NUM_QUEUES ) +break ; +#endif + xstats_mode_count = dlb2-Quoted 1 line
xstats_count_per_qid[queue_port_id];
drivers/event/dlb2/dlb2_xstats.c new file +46
-0+ start_offset = dlb2->xstats_offset_for_qid[queue_port_id]; +break ; +default : +return -EINVAL ; + }; + +if (xstats_mode_count > size || !ids || !xstats_names) +return xstats_mode_count; + +for (i =0 ; i < dlb2->xstats_count && xidx < size; i++) { +if (dlb2->xstats[i].mode != mode) +continue ; + +if (mode !=RTE_EVENT_DEV_XSTATS_DEVICE && + queue_port_id != dlb2->xstats[i].obj_idx) +continue ; + + xstats_names[xidx] = dlb2->xstats[i].name; +if (ids) + ids[xidx] = start_offset + xidx; + xidx++; + } +return xidx; +} + +static int +dlb2_xstats_update (struct dlb2_eventdev *dlb2, +enum rte_event_dev_xstats_mode mode, +uint8_t queue_port_id,const unsigned int ids[], +uint64_t values[],unsigned int n,const uint32_t reset) { +unsigned int i; +unsigned int xidx =0 ; + +RTE_SET_USED (mode); +RTE_SET_USED (queue_port_id); + +uint32_t xstats_mode_count =0 ; + +switch (mode) { +case RTE_EVENT_DEV_XSTATS_DEVICE : + xstats_mode_count = dlb2->xstats_count_mode_dev; +break ; +case RTE_EVENT_DEV_XSTATS_PORT : +if (queue_port_id >=DLB2_MAX_NUM_PORTS ) + goto invalid_value; + xstats_mode_count = dlb2-Quoted 1 line
xstats_count_per_port[queue_port_id];
drivers/event/dlb2/dlb2_xstats.c new file +7
-0+break ; +case RTE_EVENT_DEV_XSTATS_QUEUE : +#if (DLB2_MAX_NUM_QUEUES <=255 )/* max 8 bit value */ +if (queue_port_id >=DLB2_MAX_NUM_QUEUES ) + goto invalid_value; +#endif + xstats_mode_count = dlb2-Quoted 1 line
xstats_count_per_qid[queue_port_id];
drivers/event/dlb2/dlb2_xstats.c new file +28
-0+break ; +default : + goto invalid_value; + }; + +for (i =0 ; i < n && xidx < xstats_mode_count; i++) { +struct dlb2_xstats_entry *xs = &dlb2->xstats[ids[i]]; +dlb2_xstats_fn fn; + +if (ids[i] > dlb2->xstats_count || xs->mode != mode) +continue ; + +if (mode !=RTE_EVENT_DEV_XSTATS_DEVICE && + queue_port_id != xs->obj_idx) +continue ; + +switch (xs->fn_id) { +case DLB2_XSTATS_FN_DEV : + fn = get_dev_stat; +break ; +case DLB2_XSTATS_FN_PORT : + fn = get_port_stat; +break ; +case DLB2_XSTATS_FN_QUEUE : + fn = get_queue_stat; +break ; +default : + DLB2_LOG_ERR("Unexpected xstat fn_id %d\n", xs-Quoted 1 line
fn_id);drivers/event/dlb2/dlb2_xstats.c new file +24
-0+ goto invalid_value; + } + +uint64_t val =fn (dlb2, xs->obj_idx, xs->stat, + xs->extra_arg) - xs->reset_value; + +if (values) + values[xidx] = val; + +if (xs->reset_allowed && reset) + xs->reset_value += val; + + xidx++; + } + +return xidx; + +invalid_value: +return -EINVAL ; +} + +int +dlb2_eventdev_xstats_get (const struct rte_eventdev *dev, + enum rte_event_dev_xstats_mode mode, uint8_t
queue_port_id,drivers/event/dlb2/dlb2_xstats.c new file +73
-0+ const unsigned int ids[], uint64_t values[], unsigned int n) { + struct dlb2_eventdev *dlb2 = dlb2_pmd_priv(dev); +const uint32_t reset =0 ; + +return dlb2_xstats_update (dlb2, mode, queue_port_id, ids, values, n, + reset); +} + +uint64_t +dlb2_eventdev_xstats_get_by_name (const struct rte_eventdev *dev, +const char *name,unsigned int *id) { +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +unsigned int i; +dlb2_xstats_fn fn; + +for (i =0 ; i < dlb2->xstats_count; i++) { +struct dlb2_xstats_entry *xs = &dlb2->xstats[i]; + +if (strncmp (xs->name.name, name, +RTE_EVENT_DEV_XSTATS_NAME_SIZE ) ==0 ){ +if (id !=NULL ) + *id = i; + +switch (xs->fn_id) { +case DLB2_XSTATS_FN_DEV : + fn = get_dev_stat; +break ; +case DLB2_XSTATS_FN_PORT : + fn = get_port_stat; +break ; +case DLB2_XSTATS_FN_QUEUE : + fn = get_queue_stat; +break ; +default : +DLB2_LOG_ERR ("Unexpected xstat fn_id %d\n" , + xs->fn_id); +return (uint64_t )-1 ; + } + +return fn (dlb2, xs->obj_idx, xs->stat, + xs->extra_arg) - xs->reset_value; + } + } +if (id !=NULL ) + *id = (uint32_t )-1 ; +return (uint64_t )-1 ; +} + +static void +dlb2_xstats_reset_range (struct dlb2_eventdev *dlb2,uint32_t start, +uint32_t num) +{ +uint32_t i; +dlb2_xstats_fn fn; + +for (i = start; i < start + num; i++) { +struct dlb2_xstats_entry *xs = &dlb2->xstats[i]; + +if (!xs->reset_allowed) +continue ; + +switch (xs->fn_id) { +case DLB2_XSTATS_FN_DEV : + fn = get_dev_stat; +break ; +case DLB2_XSTATS_FN_PORT : + fn = get_port_stat; +break ; +case DLB2_XSTATS_FN_QUEUE : + fn = get_queue_stat; +break ; +default : + DLB2_LOG_ERR("Unexpected xstat fn_id %d\n", xs-Quoted 1 line
fn_id);drivers/event/dlb2/dlb2_xstats.c new file +39
-0+return ; + } + +uint64_t val =fn (dlb2, xs->obj_idx, xs->stat, xs->extra_arg); + xs->reset_value = val; + } +} + +static int +dlb2_xstats_reset_queue (struct dlb2_eventdev *dlb2,uint8_t queue_id, +const uint32_t ids[],uint32_t nb_ids) { +const uint32_t reset =1 ; + +if (ids) { +uint32_t nb_reset =dlb2_xstats_update (dlb2, +RTE_EVENT_DEV_XSTATS_QUEUE , + queue_id, ids,NULL , nb_ids, + reset); +return nb_reset == nb_ids ?0 : -EINVAL ; + } + +if (ids ==NULL ) +dlb2_xstats_reset_range (dlb2, + dlb2->xstats_offset_for_qid[queue_id], + dlb2->xstats_count_per_qid[queue_id]); + +return 0 ; +} + +static int +dlb2_xstats_reset_port (struct dlb2_eventdev *dlb2,uint8_t port_id, +const uint32_t ids[],uint32_t nb_ids) { +const uint32_t reset =1 ; +int offset = dlb2->xstats_offset_for_port[port_id]; +int nb_stat = dlb2->xstats_count_per_port[port_id]; + +if (ids) { + uint32_t nb_reset = dlb2_xstats_update(dlb2, + RTE_EVENT_DEV_XSTATS_PORT,
port_id,drivers/event/dlb2/dlb2_xstats.c new file +129
-0+ ids,NULL , nb_ids, + reset); +return nb_reset == nb_ids ?0 : -EINVAL ; + } + +dlb2_xstats_reset_range (dlb2, offset, nb_stat); +return 0 ; +} + +static int +dlb2_xstats_reset_dev (struct dlb2_eventdev *dlb2,const uint32_t ids[], +uint32_t nb_ids) +{ +uint32_t i; + +if (ids) { +for (i =0 ; i < nb_ids; i++) { +uint32_t id = ids[i]; + +if (id >= dlb2->xstats_count_mode_dev) +return -EINVAL ; +dlb2_xstats_reset_range (dlb2, id,1 ); + } + }else { +for (i =0 ; i < dlb2->xstats_count_mode_dev; i++) +dlb2_xstats_reset_range (dlb2, i,1 ); + } + +return 0 ; +} + +int +dlb2_eventdev_xstats_reset (struct rte_eventdev *dev, +enum rte_event_dev_xstats_mode mode, +int16_t queue_port_id, +const uint32_t ids[], +uint32_t nb_ids) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +uint32_t i; + +/* handle -1 for queue_port_id here, looping over all ports/queues */ +switch (mode) { +case RTE_EVENT_DEV_XSTATS_DEVICE : +if (dlb2_xstats_reset_dev (dlb2, ids, nb_ids)) +return -EINVAL ; +break ; +case RTE_EVENT_DEV_XSTATS_PORT : +if (queue_port_id ==-1 ) { +for (i =0 ; i <DLB2_MAX_NUM_PORTS ; i++) { +if (dlb2_xstats_reset_port (dlb2, i, + ids, nb_ids)) +return -EINVAL ; + } + }else if (queue_port_id <DLB2_MAX_NUM_PORTS ) { +if (dlb2_xstats_reset_port (dlb2, queue_port_id, + ids, nb_ids)) +return -EINVAL ; + } +break ; +case RTE_EVENT_DEV_XSTATS_QUEUE : +if (queue_port_id ==-1 ) { +for (i =0 ; i <DLB2_MAX_NUM_QUEUES ; i++) { +if (dlb2_xstats_reset_queue (dlb2, i, + ids, nb_ids)) +return -EINVAL ; + } + }else if (queue_port_id <DLB2_MAX_NUM_QUEUES ) { +if (dlb2_xstats_reset_queue (dlb2, queue_port_id, + ids, nb_ids)) +return -EINVAL ; + } +break ; + }; + +return 0 ; +} + +void +dlb2_eventdev_dump (struct rte_eventdev *dev,FILE *f) { +struct dlb2_eventdev *dlb2; +struct dlb2_hw_dev *handle; +int i; + +if (!f) { +printf ("Invalid file pointer\n" ); +return ; + } + +if (!dev) { +fprintf (f,"Invalid event device\n" ); +return ; + } + + dlb2 =dlb2_pmd_priv (dev); + +if (!dlb2) { +fprintf (f,"DLB2 Event device cannot be dumped!\n" ); +return ; + } + +if (!dlb2->configured) +fprintf (f,"DLB2 Event device is not configured\n" ); + + handle = &dlb2->qm_instance; + +fprintf (f,"================\n" ); +fprintf (f,"DLB2 Device Dump\n" ); +fprintf (f,"================\n" ); + +fprintf (f,"Processor supports umonitor/umwait instructions = %s\n" , + dlb2->umwait_allowed ?"yes" :"no" ); + +/* Generic top level device information */ + +fprintf (f,"device is configured and run state =" ); +if (dlb2->run_state ==DLB2_RUN_STATE_STOPPED ) +fprintf (f,"STOPPED\n" ); +else if (dlb2->run_state ==DLB2_RUN_STATE_STOPPING ) +fprintf (f,"STOPPING\n" ); +else if (dlb2->run_state ==DLB2_RUN_STATE_STARTING ) +fprintf (f,"STARTING\n" ); +else if (dlb2->run_state ==DLB2_RUN_STATE_STARTED ) +fprintf (f,"STARTED\n" ); +else +fprintf (f,"UNEXPECTED\n" ); + + fprintf(f, + "dev ID=%d, dom ID=%u, name=%s, path=%s, sock=%u,evdev=%p\n",
drivers/event/dlb2/dlb2_xstats.c new file +201
-0+ handle->device_id, handle->domain_id, handle->device_name, + handle->device_path, handle->info.socket_id, dlb2->event_dev); + +fprintf (f,"num dir ports=%u, num dir queues=%u\n" , + dlb2->num_dir_ports, dlb2->num_dir_queues); + +fprintf (f,"num ldb ports=%u, num ldb queues=%u\n" , + dlb2->num_ldb_ports, dlb2->num_ldb_queues); + +fprintf (f,"num atomic inflights=%u, hist list entries=%u\n" , + handle->cfg.resources.num_atomic_inflights, + handle->cfg.resources.num_hist_list_entries); + +fprintf (f,"results from most recent hw resource query:\n" ); + +fprintf (f,"\tnum_sched_domains = %u\n" , + dlb2->hw_rsrc_query_results.num_sched_domains); + +fprintf (f,"\tnum_ldb_queues = %u\n" , + dlb2->hw_rsrc_query_results.num_ldb_queues); + +fprintf (f,"\tnum_ldb_ports = %u\n" , + dlb2->hw_rsrc_query_results.num_ldb_ports); + +fprintf (f,"\tnum_dir_ports = %u\n" , + dlb2->hw_rsrc_query_results.num_dir_ports); + +fprintf (f,"\tnum_atomic_inflights = %u\n" , + dlb2->hw_rsrc_query_results.num_atomic_inflights); + +fprintf (f,"\tnum_hist_list_entries = %u\n" , + dlb2->hw_rsrc_query_results.num_hist_list_entries); + +fprintf (f,"\tmax_contiguous_hist_list_entries = %u\n" , + dlb2->hw_rsrc_query_results.max_contiguous_hist_list_entries); + +fprintf (f,"\tnum_ldb_credits = %u\n" , + dlb2->hw_rsrc_query_results.num_ldb_credits); + +fprintf (f,"\tnum_dir_credits = %u\n" , + dlb2->hw_rsrc_query_results.num_dir_credits); + +/* Port level information */ + +for (i =0 ; i < dlb2->num_ports; i++) { +struct dlb2_eventdev_port *p = &dlb2->ev_ports[i]; +int j; + +if (!p->enq_configured) +fprintf (f,"Port_%d is not configured\n" , i); + +fprintf (f,"Port_%d\n" , i); +fprintf (f,"=======\n" ); + +fprintf (f,"\tevport_%u is configured, setup done=%d\n" , + p->id, p->setup_done); + +fprintf (f,"\tconfig state=%d, port state=%d\n" , + p->qm_port.config_state, p->qm_port.state); + +fprintf (f,"\tport is %s\n" , + p->qm_port.is_directed ?"directed" :"load balanced" ); + +fprintf (f,"\toutstanding releases=%u\n" , + p->outstanding_releases); + +fprintf (f,"\tinflight max=%u, inflight credits=%u\n" , + p->inflight_max, p->inflight_credits); + +fprintf (f,"\tcredit update quanta=%u, implicit release =%u\n" , + p->credit_update_quanta, p->implicit_release); + +fprintf (f,"\tnum_links=%d, queues -> " , p->num_links); + +for (j =0 ; j <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; j++) { +if (p->link[j].valid) +fprintf (f,"id=%u prio=%u " , + p->link[j].queue_id, + p->link[j].priority); + } +fprintf (f,"\n" ); + +fprintf (f,"\thardware port id=%u\n" , p->qm_port.id); + +fprintf (f,"\tcached_ldb_credits=%u\n" , + p->qm_port.cached_ldb_credits); + +fprintf (f,"\tldb_credits = %u\n" , + p->qm_port.ldb_credits); + +fprintf (f,"\tcached_dir_credits = %u\n" , + p->qm_port.cached_dir_credits); + +fprintf (f,"\tdir_credits = %u\n" , + p->qm_port.dir_credits); + +fprintf (f,"\tgenbit=%d, cq_idx=%d, cq_depth=%d\n" , + p->qm_port.gen_bit, + p->qm_port.cq_idx, + p->qm_port.cq_depth); + +fprintf (f,"\tinterrupt armed=%d\n" , + p->qm_port.int_armed); + +fprintf (f,"\tPort statistics\n" ); + +fprintf (f,"\t\trx_ok %" PRIu64"\n" , + p->stats.traffic.rx_ok); + +fprintf (f,"\t\trx_drop %" PRIu64"\n" , + p->stats.traffic.rx_drop); + +fprintf (f,"\t\trx_interrupt_wait %" PRIu64"\n" , + p->stats.traffic.rx_interrupt_wait); + +fprintf (f,"\t\trx_umonitor_umwait %" PRIu64"\n" , + p->stats.traffic.rx_umonitor_umwait); + +fprintf (f,"\t\ttx_ok %" PRIu64"\n" , + p->stats.traffic.tx_ok); + +fprintf (f,"\t\ttotal_polls %" PRIu64"\n" , + p->stats.traffic.total_polls); + +fprintf (f,"\t\tzero_polls %" PRIu64"\n" , + p->stats.traffic.zero_polls); + +fprintf (f,"\t\ttx_nospc_ldb_hw_credits %" PRIu64"\n" , + p->stats.traffic.tx_nospc_ldb_hw_credits); + +fprintf (f,"\t\ttx_nospc_dir_hw_credits %" PRIu64"\n" , + p->stats.traffic.tx_nospc_dir_hw_credits); + +fprintf (f,"\t\ttx_nospc_inflight_max %" PRIu64"\n" , + p->stats.traffic.tx_nospc_inflight_max); + +fprintf (f,"\t\ttx_nospc_new_event_limit %" PRIu64"\n" , + p->stats.traffic.tx_nospc_new_event_limit); + +fprintf (f,"\t\ttx_nospc_inflight_credits %" PRIu64"\n" , + p->stats.traffic.tx_nospc_inflight_credits); + +fprintf (f,"\t\ttx_new %" PRIu64"\n" , + p->stats.tx_op_cnt[RTE_EVENT_OP_NEW ]); + +fprintf (f,"\t\ttx_fwd %" PRIu64"\n" , + p->stats.tx_op_cnt[RTE_EVENT_OP_FORWARD ]); + +fprintf (f,"\t\ttx_rel %" PRIu64"\n" , + p->stats.tx_op_cnt[RTE_EVENT_OP_RELEASE ]); + +fprintf (f,"\t\ttx_implicit_rel %" PRIu64"\n" , + p->stats.tx_implicit_rel); + +fprintf (f,"\t\ttx_sched_ordered %" PRIu64"\n" , + p->stats.tx_sched_cnt[DLB2_SCHED_ORDERED ]); + +fprintf (f,"\t\ttx_sched_unordered %" PRIu64"\n" , + p->stats.tx_sched_cnt[DLB2_SCHED_UNORDERED ]); + +fprintf (f,"\t\ttx_sched_atomic %" PRIu64"\n" , + p->stats.tx_sched_cnt[DLB2_SCHED_ATOMIC ]); + +fprintf (f,"\t\ttx_sched_directed %" PRIu64"\n" , + p->stats.tx_sched_cnt[DLB2_SCHED_DIRECTED ]); + +fprintf (f,"\t\ttx_invalid %" PRIu64"\n" , + p->stats.tx_invalid); + +fprintf (f,"\t\trx_sched_ordered %" PRIu64"\n" , + p->stats.rx_sched_cnt[DLB2_SCHED_ORDERED ]); + +fprintf (f,"\t\trx_sched_unordered %" PRIu64"\n" , + p->stats.rx_sched_cnt[DLB2_SCHED_UNORDERED ]); + +fprintf (f,"\t\trx_sched_atomic %" PRIu64"\n" , + p->stats.rx_sched_cnt[DLB2_SCHED_ATOMIC ]); + +fprintf (f,"\t\trx_sched_directed %" PRIu64"\n" , + p->stats.rx_sched_cnt[DLB2_SCHED_DIRECTED ]); + +fprintf (f,"\t\trx_sched_invalid %" PRIu64"\n" , + p->stats.rx_sched_invalid); + } + +/* Queue level information */ + +for (i =0 ; i < dlb2->num_queues; i++) { +struct dlb2_eventdev_queue *q = &dlb2->ev_queues[i]; +int j, k; + +if (!q->setup_done) +fprintf (f,"Queue_%d is not configured\n" , i); + +fprintf (f,"Queue_%d\n" , i); +fprintf (f,"========\n" ); + +fprintf (f,"\tevqueue_%u is set up\n" , q->id); + + fprintf(f, "\tqueue is %s\n", + q->qm_queue.is_directed ? "directed" : "loadbalanced");
drivers/event/dlb2/dlb2_xstats.c new file +7
-0+ +fprintf (f,"\tnum_links=%d, ports -> " , q->num_links); + +for (j =0 ; j < dlb2->num_ports; j++) { +struct dlb2_eventdev_port *p = &dlb2->ev_ports[j]; + + for (k = 0; k < DLB2_MAX_NUM_QIDS_PER_LDB_CQ;k++) {
drivers/event/dlb2/dlb2_xstats.c new file +13
-0+if (p->link[k].valid && + p->link[k].queue_id == q->id) +fprintf (f,"id=%u prio=%u " , + p->id, p->link[k].priority); + } + } +fprintf (f,"\n" ); + +fprintf (f,"\tcurrent depth: %u events\n" , +dlb2_get_queue_depth (dlb2, q)); + + fprintf(f, "\tnum qid inflights=%u, sched_type=%d\n", + q->qm_queue.num_qid_inflights, q-Quoted 1 line
qm_queue.sched_type);
drivers/event/dlb2/dlb2_xstats.c new file +2
-0+ } +}drivers/event/dlb2/meson.build +1
-1@@ -3,6 +3,7 @@ sources = files('dlb2.c', 'dlb2_iface.c', + 'dlb2_xstats.c', 'pf/dlb2_main.c', 'pf/dlb2_pf.c', 'pf/base/dlb2_resource.c' --2.6.4
Re: [dpdk-dev] [PATCH 07/22] event/dlb2: add xstats
Quoted 30 lines
-----Original Message-----
From: Chen, Mike Ximing <[email protected]>
Sent: Thursday, September 17, 2020 3:58 PM
To: McDaniel, Timothy <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: RE: [dpdk-dev] [PATCH 07/22] event/dlb2: add xstats<snip>
Quoted 18 lines
drivers/event/dlb2/dlb2_xstats.c new file +18
-0+dlb2_eventdev_dump(struct rte_eventdev *dev, FILE *f) { + struct dlb2_eventdev *dlb2; +struct dlb2_hw_dev *handle; +int i; + +if (!f) { +printf ("Invalid file pointer\n" ); +return ; + } + +if (!dev) { +fprintf (f,"Invalid event device\n" ); +return ; + } + + dlb2 =dlb2_pmd_priv (dev); + +if (!dlb2) {
Quoted 9 lines
drivers/event/dlb2/dlb2_xstats.c line 1018
Quoted 4 lines
drivers/event/dlb2/dlb2_xstats.c new file +4
-0+fprintf (f,"DLB2 Event device cannot be dumped!\n" ); +return ; + } +Not sure if this is enforced. The DPDK coding style discourages using ! on pointers ( see section 1. 8.1 at
https://doc.dpdk.org/guides/contributing/coding_style.html).
I see !ptr used in many other dpdk components, and it is not flagged by checkpatch either.
Re: [dpdk-dev] [PATCH 07/22] event/dlb2: add xstats
Quoted 51 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Thursday, September 17, 2020 5:26 PM
To: Chen, Mike Ximing <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: RE: [dpdk-dev] [PATCH 07/22] event/dlb2: add xstatsQuoted 28 lines
-----Original Message-----
From: Chen, Mike Ximing <[email protected]>
Sent: Thursday, September 17, 2020 3:58 PM
To: McDaniel, Timothy <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: RE: [dpdk-dev] [PATCH 07/22] event/dlb2: add xstats<snip>
Quoted 18 lines
drivers/event/dlb2/dlb2_xstats.c new file +18
-0+dlb2_eventdev_dump(struct rte_eventdev *dev, FILE *f) { + struct dlb2_eventdev *dlb2; +struct dlb2_hw_dev *handle; +int i; + +if (!f) { +printf ("Invalid file pointer\n" ); +return ; + } + +if (!dev) { +fprintf (f,"Invalid event device\n" ); +return ; + } + + dlb2 =dlb2_pmd_priv (dev); + +if (!dlb2) {Quoted 9 lines
drivers/event/dlb2/dlb2_xstats.c line 1018
Quoted 4 lines
drivers/event/dlb2/dlb2_xstats.c new file +4
-0+fprintf (f,"DLB2 Event device cannot be dumped!\n" ); +return ; + } +Not sure if this is enforced. The DPDK coding style discourages using ! on pointers ( see section 1. 8.1 at
https://doc.dpdk.org/guides/contributing/coding_style.html).
Quoted 3 lines
I see !ptr used in many other dpdk components, and it is not flagged by checkpatch either.
Yes, I do see !ptr in dpdk functions. I guess the rule is not enforced.
Re: [dpdk-dev] [PATCH 07/22] event/dlb2: add xstats
Quoted 51 lines · Mike Ximing
Quoted 51 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Thursday, September 17, 2020 5:26 PM
To: Chen, Mike Ximing <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: RE: [dpdk-dev] [PATCH 07/22] event/dlb2: add xstatsQuoted 37 lines
-----Original Message-----
From: Chen, Mike Ximing <[email protected]>
Sent: Thursday, September 17, 2020 3:58 PM
To: McDaniel, Timothy <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: RE: [dpdk-dev] [PATCH 07/22] event/dlb2: add xstats<snip>
Quoted 12 lines
drivers/event/dlb2/dlb2_xstats.c new file +12
-0+dlb2_eventdev_dump(struct rte_eventdev *dev, FILE *f) { + struct dlb2_eventdev *dlb2; +struct dlb2_hw_dev *handle; +int i; + +if (!f) { +printf ("Invalid file pointer\n" ); +return ; + } + +if (!dev) { +fprintf (f,"Invalid event device\n" );drivers/event/dlb2/dlb2_xstats.c line 1018
Quoted 10 lines
drivers/event/dlb2/dlb2_xstats.c new file +10
-0+return ; + } + + dlb2 =dlb2_pmd_priv (dev); + +if (!dlb2) { +fprintf (f,"DLB2 Event device cannot be dumped!\n" ); +return ; + } +Not sure if this is enforced. The DPDK coding style discourages using ! on pointers ( see section 1. 8.1 at
https://doc.dpdk.org/guides/contributing/coding_style.html).
Quoted 4 lines
Quoted 3 lines
I see !ptr used in many other dpdk components, and it is not flagged by checkpatch either.
Yes, I do see !ptr in dpdk functions. I guess the rule is not enforced.
Not strictly enforced, no, but for new code we should always endeavour to follow the standards. If a new revision of this patch will be done, please have all pointer comparisons checking for NULL explicitly in that version.
Thanks,
/Bruce
Quoted 33 lines · Timothy McDaniel
The following patch series adds support for a new eventdev PMD. The DLB2 PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2) hardware.
The DLB2 is a PCIe device that provides load-balanced, prioritized scheduling of core-to-core communication. The device consists of queues and arbiters that connect producer and consumer cores, and implements load-balanced queueing features including:
- Lock-free multi-producer/multi-consumer operation.
- Multiple priority levels for varying traffic types.
- 'Direct' traffic (i.e. multi-producer/single-consumer)
- Simple unordered load-balanced distribution.
- Atomic lock-free load balancing across multiple consumers.
- Queue element reordering feature allowing ordered load-balanced distribution.The DLB2 hardware supports both load balanced and directed ports and queues. Unlike other eventdev devices already in the repo, not all DLB2 ports and queues are equally capable. In particular, directed ports are limited to a single link, and must be connected to a directed queue. Additionally, even though LDB ports may link multiple queues, the number of queues that may be linked is limited by hardware.
While reviewing the code, please be aware that this PMD has full control over the DLB2 hardware. Intel will be extending the DLB2 PMD in the future (not as part of this first series) with a mode that we refer to as the bifurcated PMD. The bifurcated PMD communicates with a kernel driver to configure the device, ports, and queues, and memory maps device MMIO so datapath operations occur purely in user-space. Note that the DLB2 hardware is a successor of the DLB hardware, and as such is structured similarly, both in terms of code layout and implementation.
Quoted 8 lines
The framework to support both the PF PMD and bifurcated PMD exists in this patchset, and is why the iface.[ch] layer is present.
Depends-on: patch-77466 ("eventdev: add PCI probe named convenience function")
Depends-on: series-12160 ("Eventdev ABI changes")
Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")Timothy McDaniel (22):
There is two version of driver(DLB and DLB2)[1]
Both are needed?
Is two separate set of HW?
http://patches.dpdk.org/project/dpdk/list/?submitter=826
Quoted 82 lines
event/dlb2: add meson build infrastructure event/dlb2: add dynamic logging
event/dlb2: add private data structures and constants event/dlb2: add definitions shared with LKM or shared code event/dlb2: add inline functions
event/dlb2: add probe
event/dlb2: add xstats
event/dlb2: add infos get and configure event/dlb2: add queue and port default conf event/dlb2: add queue setup
event/dlb2: add port setup
event/dlb2: add port link
event/dlb2: add port unlink and port unlinks in progress event/dlb2: add eventdev start
event/dlb2: add enqueue and its burst variants event/dlb2: add dequeue and its burst variants event/dlb2: add eventdev stop and close event/dlb2: add PMD's token pop public interface event/dlb2: add PMD self-tests
event/dlb2: add queue and port release event/dlb2: add timeout ticks entry point
doc: add new DLB2 eventdev driver to relnotesapp/test/test_eventdev.c | 9 + config/rte_config.h | 7 + doc/guides/rel_notes/release_20_11.rst | 5 + drivers/event/dlb2/dlb2.c | 4046 ++++++++++++++ drivers/event/dlb2/dlb2_iface.c | 88 + drivers/event/dlb2/dlb2_iface.h | 75 + drivers/event/dlb2/dlb2_inline_fns.h | 85 + drivers/event/dlb2/dlb2_log.h | 25 + drivers/event/dlb2/dlb2_priv.h | 619 +++ drivers/event/dlb2/dlb2_selftest.c | 1570 ++++++ drivers/event/dlb2/dlb2_user.h | 883 +++ drivers/event/dlb2/dlb2_xstats.c | 1269 +++++ drivers/event/dlb2/meson.build | 16 + drivers/event/dlb2/pf/base/dlb2_hw_types.h | 367 ++ drivers/event/dlb2/pf/base/dlb2_mbox.h | 596 ++ drivers/event/dlb2/pf/base/dlb2_osdep.h | 248 + drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h | 447 ++ drivers/event/dlb2/pf/base/dlb2_osdep_list.h | 131 + drivers/event/dlb2/pf/base/dlb2_osdep_types.h | 31 + drivers/event/dlb2/pf/base/dlb2_regs.h | 2527 +++++++++ drivers/event/dlb2/pf/base/dlb2_resource.c | 6023 +++++++++++++++++++++ drivers/event/dlb2/pf/base/dlb2_resource.h | 1913 +++++++ drivers/event/dlb2/pf/dlb2_main.c | 692 +++ drivers/event/dlb2/pf/dlb2_main.h | 107 + drivers/event/dlb2/pf/dlb2_pf.c | 734 +++ drivers/event/dlb2/rte_pmd_dlb2.c | 39 + drivers/event/dlb2/rte_pmd_dlb2.h | 59 + drivers/event/dlb2/rte_pmd_dlb2_event_version.map | 9 + drivers/event/meson.build | 4 + 29 files changed, 22624 insertions(+) create mode 100644 drivers/event/dlb2/dlb2.c create mode 100644 drivers/event/dlb2/dlb2_iface.c create mode 100644 drivers/event/dlb2/dlb2_iface.h create mode 100644 drivers/event/dlb2/dlb2_inline_fns.h create mode 100644 drivers/event/dlb2/dlb2_log.h create mode 100644 drivers/event/dlb2/dlb2_priv.h create mode 100644 drivers/event/dlb2/dlb2_selftest.c create mode 100644 drivers/event/dlb2/dlb2_user.h create mode 100644 drivers/event/dlb2/dlb2_xstats.c create mode 100644 drivers/event/dlb2/meson.build create mode 100644 drivers/event/dlb2/pf/base/dlb2_hw_types.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_mbox.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_list.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_types.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_regs.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.c create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.h create mode 100644 drivers/event/dlb2/pf/dlb2_main.c create mode 100644 drivers/event/dlb2/pf/dlb2_main.h create mode 100644 drivers/event/dlb2/pf/dlb2_pf.c create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.c create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.h create mode 100644 drivers/event/dlb2/rte_pmd_dlb2_event_version.map--
2.6.4
Quoted 54 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, September 21, 2020 12:12 PM
To: McDaniel, Timothy <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMDQuoted 33 lines · Timothy McDaniel
The following patch series adds support for a new eventdev PMD. The DLB2 PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2) hardware.
The DLB2 is a PCIe device that provides load-balanced, prioritized scheduling of core-to-core communication. The device consists of queues and arbiters that connect producer and consumer cores, and implements load-balanced queueing features including:
- Lock-free multi-producer/multi-consumer operation.
- Multiple priority levels for varying traffic types.
- 'Direct' traffic (i.e. multi-producer/single-consumer)
- Simple unordered load-balanced distribution.
- Atomic lock-free load balancing across multiple consumers.
- Queue element reordering feature allowing ordered load-balanced distribution.The DLB2 hardware supports both load balanced and directed ports and queues. Unlike other eventdev devices already in the repo, not all DLB2 ports and queues are equally capable. In particular, directed ports are limited to a single link, and must be connected to a directed queue. Additionally, even though LDB ports may link multiple queues, the number of queues that may be linked is limited by hardware.
While reviewing the code, please be aware that this PMD has full control over the DLB2 hardware. Intel will be extending the DLB2 PMD in the future (not as part of this first series) with a mode that we refer to as the bifurcated PMD. The bifurcated PMD communicates with a kernel driver to configure the device, ports, and queues, and memory maps device MMIO so datapath operations occur purely in user-space. Note that the DLB2 hardware is a successor of the DLB hardware, and as such is structured similarly, both in terms of code layout and implementation.
Quoted 4 lines
The framework to support both the PF PMD and bifurcated PMD exists in this patchset, and is why the iface.[ch] layer is present.
Depends-on: patch-77466 ("eventdev: add PCI probe named convenience
function")
Quoted 3 lines
Depends-on: series-12160 ("Eventdev ABI changes")
Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")
Quoted 7 lines
Quoted 1 line
Timothy McDaniel (22):
There is two version of driver(DLB and DLB2)[1]
Both are needed?
Is two separate set of HW?
Yes, the two PMDs, DLB and DLB2 are both needed. They support two different hardware devices.
Quoted 20 lines · Timothy
Quoted 20 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, September 21, 2020 12:12 PM
To: McDaniel, Timothy <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMDQuoted 4 lines · Timothy McDaniel
Timothy McDaniel (22):
Quoted 6 lines
Quoted 4 lines
There is two version of driver(DLB and DLB2)[1]
Both are needed?
Is two separate set of HW?Yes, the two PMDs, DLB and DLB2 are both needed. They support two different hardware devices.
There are plenty of build errors. Please see
http://mails.dpdk.org/archives/test-report/2020-September/152143.html
Also, I would like close the eventdev spec first before we merge this driver as the driver is depended on
http://patches.dpdk.org/patch/77465/
Quoted 2 lines
Quoted 40 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, September 29, 2020 1:41 PM
To: McDaniel, Timothy <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMDQuoted 8 lines · Timothy
Quoted 8 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, September 21, 2020 12:12 PM
To: McDaniel, Timothy <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>;Eads,
Quoted 12 lines
Quoted 12 lines
Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMDQuoted 4 lines · Timothy McDaniel
Timothy McDaniel (22):
Quoted 6 lines
Quoted 4 lines
There is two version of driver(DLB and DLB2)[1] Both are needed?
Is two separate set of HW?Yes, the two PMDs, DLB and DLB2 are both needed. They support two
Quoted 9 lines
different hardware devices.
There are plenty of build errors. Please see
http://mails.dpdk.org/archives/test-report/2020-September/152143.htmlAlso, I would like close the eventdev spec first before we merge this driver as the driver is depended on
http://patches.dpdk.org/patch/77465/
Thank you, Jerin.
I am working on addressing the build breaks and your other review comments. I hope to upload those changes tomorrow, and that will include the performance comparison that you requested.
Quoted 33 lines · Timothy McDaniel
The following patch series adds support for a new eventdev PMD. The DLB2 PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2) hardware.
The DLB2 is a PCIe device that provides load-balanced, prioritized scheduling of core-to-core communication. The device consists of queues and arbiters that connect producer and consumer cores, and implements load-balanced queueing features including:
- Lock-free multi-producer/multi-consumer operation.
- Multiple priority levels for varying traffic types.
- 'Direct' traffic (i.e. multi-producer/single-consumer)
- Simple unordered load-balanced distribution.
- Atomic lock-free load balancing across multiple consumers.
- Queue element reordering feature allowing ordered load-balanced distribution.The DLB2 hardware supports both load balanced and directed ports and queues. Unlike other eventdev devices already in the repo, not all DLB2 ports and queues are equally capable. In particular, directed ports are limited to a single link, and must be connected to a directed queue. Additionally, even though LDB ports may link multiple queues, the number of queues that may be linked is limited by hardware.
While reviewing the code, please be aware that this PMD has full control over the DLB2 hardware. Intel will be extending the DLB2 PMD in the future (not as part of this first series) with a mode that we refer to as the bifurcated PMD. The bifurcated PMD communicates with a kernel driver to configure the device, ports, and queues, and memory maps device MMIO so datapath operations occur purely in user-space. Note that the DLB2 hardware is a successor of the DLB hardware, and as such is structured similarly, both in terms of code layout and implementation.
Quoted 6 lines
The framework to support both the PF PMD and bifurcated PMD exists in this patchset, and is why the iface.[ch] layer is present.
Depends-on: patch-77466 ("eventdev: add PCI probe named convenience function")
Depends-on: series-12160 ("Eventdev ABI changes")
Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")
There is a duplicate for patch-77460 as
https://patches.dpdk.org/patch/76554/ from different authors, and both are open comments to be closed.
@McDaniel, Timothy and @Liang Ma , We need to have this eal patch on master before I merge this
patch series in eventdev-next. Please check.
Quoted 1 line
Quoted 39 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, September 29, 2020 1:41 PM
To: McDaniel, Timothy <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMDQuoted 8 lines · Timothy
Quoted 8 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, September 21, 2020 12:12 PM
To: McDaniel, Timothy <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>;Eads,
Quoted 17 lines
Quoted 16 lines
Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMDQuoted 4 lines · Timothy McDaniel
Timothy McDaniel (22):
There is two version of driver(DLB and DLB2)[1] Both are needed?
Is two separate set of HW?
Quoted 6 lines
Quoted 1 line
Yes, the two PMDs, DLB and DLB2 are both needed. They support two
different hardware devices.
There are plenty of build errors. Please see
http://mails.dpdk.org/archives/test-report/2020-September/152143.html
The build errors appear to be a result of the depends-on patches not having been applied. Also, note that there is a 2-way dependency on the ABI changes patch and the test/apps patch. They both depend on each other.
Quoted 56 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, September 29, 2020 1:46 PM
To: McDaniel, Timothy <[email protected]>; Thomas Monjalon <[email protected]>; David Marchand <[email protected]>; Ma, Liang J <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMDQuoted 33 lines · Timothy McDaniel
The following patch series adds support for a new eventdev PMD. The DLB2 PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2) hardware.
The DLB2 is a PCIe device that provides load-balanced, prioritized scheduling of core-to-core communication. The device consists of queues and arbiters that connect producer and consumer cores, and implements load-balanced queueing features including:
- Lock-free multi-producer/multi-consumer operation.
- Multiple priority levels for varying traffic types.
- 'Direct' traffic (i.e. multi-producer/single-consumer)
- Simple unordered load-balanced distribution.
- Atomic lock-free load balancing across multiple consumers.
- Queue element reordering feature allowing ordered load-balanced distribution.The DLB2 hardware supports both load balanced and directed ports and queues. Unlike other eventdev devices already in the repo, not all DLB2 ports and queues are equally capable. In particular, directed ports are limited to a single link, and must be connected to a directed queue. Additionally, even though LDB ports may link multiple queues, the number of queues that may be linked is limited by hardware.
While reviewing the code, please be aware that this PMD has full control over the DLB2 hardware. Intel will be extending the DLB2 PMD in the future (not as part of this first series) with a mode that we refer to as the bifurcated PMD. The bifurcated PMD communicates with a kernel driver to configure the device, ports, and queues, and memory maps device MMIO so datapath operations occur purely in user-space. Note that the DLB2 hardware is a successor of the DLB hardware, and as such is structured similarly, both in terms of code layout and implementation.
Quoted 4 lines
The framework to support both the PF PMD and bifurcated PMD exists in this patchset, and is why the iface.[ch] layer is present.
Depends-on: patch-77466 ("eventdev: add PCI probe named convenience
function")
Quoted 2 lines
Depends-on: series-12160 ("Eventdev ABI changes")
Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")
Quoted 10 lines
Cc Thomas and DavidThere is a duplicate for patch-77460 as
https://patches.dpdk.org/patch/76554/ from different authors, and both are open comments to be closed.
@McDaniel, Timothy and @Liang Ma , We need to have this eal patch on master before I merge this
patch series in eventdev-next. Please check.
We have discussed this duplication with @Liang Ma, who split out just the
bit definitions into its own patch. DLB/DLB2 have been updated to use his new patch going forward, but will duplicate it here until it is merged.
Re: [dpdk-dev] [PATCH 01/22] event/dlb2: add meson build infrastructure
In reply to Timothy McDanielQuoted 48 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
To: Richardson, Bruce <[email protected]>; Ray Kinsella <[email protected]>; Neil Horman <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 01/22] event/dlb2: add meson build infrastructureAdds the meson build infrastructure, which includes compile-time constants in
rte_config.h. DLB2 is only supported on Linux X86 platforms at this time.Signed-off-by Timothy McDaniel <[email protected]>--- config/rte_config.h | 7 +++++++ drivers/event/dlb2/meson.build | 7 +++++++ drivers/event/dlb2/rte_pmd_dlb2_event_version.map | 3 +++ drivers/event/meson.build | 4 ++++ 4 files changed, 21 insertions(+) create mode 100644 drivers/event/dlb2/meson.build create mode 100644 drivers/event/dlb2/rte_pmd_dlb2_event_version.mapconfig/rte_config.h +7
-0@@ -131,4 +131,11 @@/* QEDE PMD defines */ #define RTE_LIBRTE_QEDE_FW "" +/* DLB2 defines */ +#define RTE_LIBRTE_PMD_DLB2_POLL_INTERVAL 1000 +#define RTE_LIBRTE_PMD_DLB2_UMWAIT_CTL_STATE 0 +#undef RTE_LIBRTE_PMD_DLB2_QUELL_STATS +#define RTE_LIBRTE_PMD_DLB2_SW_CREDIT_QUANTA 32 +#define RTE_PMD_DLB2_DEFAULT_DEPTH_THRESH 256 +#endif /* _RTE_CONFIG_H_ */ drivers/event/dlb2/meson.build new file +0
-0
drivers/event/dlb2/meson.build line 7
Quoted 7 lines
drivers/event/dlb2/meson.build new file +7
-0@@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019-2020 Intel Corporation + +sources = files( +) + +deps += ['mbuf', 'mempool', 'ring', 'bus_vdev', 'pci', 'bus_pci']
I don't believe the code in this series depends on bus_vdev, is that right?
With that resolved (if need be):
Thanks,
Gage
Re: [dpdk-dev] [PATCH 02/22] event/dlb2: add dynamic logging
In reply to Timothy McDanielQuoted 8 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 02/22] event/dlb2: add dynamic logging
Quoted 5 lines
This commit adds base support for dynamic logging. The default log level is NOTICE. Dynamic logging is used exclusively throughout this patchset.
Signed-off-by Timothy McDaniel <[email protected]>
Thanks,
Gage
Re: [dpdk-dev] [PATCH 03/22] event/dlb2: add private data structures and constants
In reply to Timothy McDanielQuoted 36 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 03/22] event/dlb2: add private data structures and constantsThe header file
dlb2_priv.his used internally by the PMD. It include constants, macros for device resources, structure definitions for hardware interfaces and software state, and various forward-declarations. The header filerte_pmd_dlb2.hwill be exported in a subsequent patch, but is included here due to a data structure dependency.Signed-off-by Timothy McDaniel <[email protected]>--- drivers/event/dlb2/dlb2_priv.h | 614 ++++++++++++++++++++++++++++++++++++++ drivers/event/dlb2/rte_pmd_dlb2.h | 59 ++++ 2 files changed, 673 insertions(+) create mode 100644 drivers/event/dlb2/dlb2_priv.h create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.hdrivers/event/dlb2/dlb2_priv.h new file +3
-0@@ -0,0 +1,614 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/
drivers/event/dlb2/dlb2_priv.h line 13
Quoted 10 lines
drivers/event/dlb2/dlb2_priv.h new file +10
-0+ +#ifndef _DLB2_PRIV_H_ +#define _DLB2_PRIV_H_ + +#include <emmintrin.h> +#include <stdbool.h> + +#include <rte_eventdev.h> + +#include "rte_config.h"
This should be included with angle brackets and immediately follow rte_eventdev.h.
[...]
Quoted 11 lines
drivers/event/dlb2/dlb2_priv.h new file +11
-0+/* Do not change - must match hardware! */ +enum dlb2_hw_sched_type { +DLB2_SCHED_ATOMIC =0 , +DLB2_SCHED_UNORDERED , +DLB2_SCHED_ORDERED , +DLB2_SCHED_DIRECTED , +/* DLB2_NUM_HW_SCHED_TYPES must be last */ +DLB2_NUM_HW_SCHED_TYPES +}; + +/* TODO - these structs have been converted from qm - some fields may not
Quoted 3 lines
be
+ * required
+ */
Leftover TODO -- still valid?
drivers/event/dlb2/dlb2_priv.h line 133
Quoted 4 lines
drivers/event/dlb2/dlb2_priv.h new file +4
-0+ +struct dlb2_hw_rsrcs { +int32_t nb_events_limit; +uint32_t num_queues;/**> Total queues (lb + dir) */
I don't believe "**>" is correct doxygen format (angle bracket's facing the wrong direction), but since this is an internal/private header file it won't be included in the doxygen output anyway. I'd recommend either correcting the inconsistency in angle brackets or just dropping the doxygen style here altogether.
Quoted 7 lines
drivers/event/dlb2/dlb2_priv.h new file +7
-0+uint32_t num_ldb_queues;/**> Number of available ldb queues */ +uint32_t num_ldb_ports;/**< Number of load balanced ports */ +uint32_t num_dir_ports;/**< Number of directed ports */ +uint32_t num_ldb_credits;/**< Number of load balanced credits */ +uint32_t num_dir_credits;/**< Number of directed credits */ +uint32_t reorder_window_size;/**< Size of reorder window */ +};
drivers/event/dlb2/dlb2_priv.h line 150
Quoted 10 lines
drivers/event/dlb2/dlb2_priv.h new file +10
-0+ +struct dlb2_hw_resource_info { +/**> Max resources that can be provided */ +struct dlb2_hw_rsrcs hw_rsrc_max; +int num_sched_domains; +uint32_t socket_id; +/**> EAL flags passed to this QM instance, allowing the application to +* identify the pmd backend indicating hardware or software. +*/ +const char *eal_flags;
Doesn't look like eal_flags is used in any of the later patches.
[...]
drivers/event/dlb2/dlb2_priv.h line 286
Quoted 6 lines
drivers/event/dlb2/dlb2_priv.h new file +6
-0+ +enum DLB2_PORT_STATE { +PORT_CLOSED , +PORT_STARTED , +PORT_STOPPED +};
Nit: other enum names in this file are lower-case.
Quoted 45 lines
drivers/event/dlb2/dlb2_priv.h new file +19
-0+ +enum dlb2_configuration_state { +/* The resource has not been configured */ +DLB2_NOT_CONFIGURED , +/* The resource was configured, but the device was stopped */ +DLB2_PREV_CONFIGURED , +/* The resource is currently configured */ +DLB2_CONFIGURED +}; + +struct dlb2_port { +uint32_t id; +bool is_directed; +bool gen_bit; +uint16_t dir_credits; +uint32_t dequeue_depth; +enum dlb2_token_pop_mode token_pop_mode; +union dlb2_port_config cfg; +uint32_t *credit_pool[DLB2_NUM_QUEUE_TYPES ];/* use __atomic builtins */
drivers/event/dlb2/dlb2_priv.h new file +25
-0+uint16_t cached_ldb_credits; +uint16_t ldb_credits; +uint16_t cached_dir_credits; +bool int_armed; +uint16_t owed_tokens; +int16_t issued_releases; +int16_t token_pop_thresh; +int cq_depth; +uint16_t cq_idx; +uint16_t cq_idx_unmasked; +uint16_t cq_depth_mask; +uint16_t gen_bit_shift; +enum DLB2_PORT_STATE state; +enum dlb2_configuration_state config_state; +int num_mapped_qids; +uint8_t *qid_mappings; +struct dlb2_enqueue_qe *qe4;/* Cache line's worth of QEs (4) */ +struct dlb2_enqueue_qe *int_arm_qe; +struct dlb2_cq_pop_qe *consume_qe; +struct dlb2_eventdev *dlb2;/* back ptr */ +struct dlb2_eventdev_port *ev_port;/* back ptr */ +}; + +/* Per-process per-port mmio and memory pointers */ +struct process_local_port_data {
drivers/event/dlb2/dlb2_priv.h line 340
Quoted 10 lines
drivers/event/dlb2/dlb2_priv.h new file +10
-0+uint64_t *pp_addr; +struct dlb2_dequeue_qe *cq_base; +bool mmaped; +}; + +struct dlb2_eventdev ; + +struct dlb2_port_low_level_io_functions { +void (*pp_enqueue_four)(void *qe4, +void *pp_addr);
In the DLB PMD v4 patchset, I noticed the pp_enqueue_four indirection is dropped in favor of directly calling the MOVDIR64 function -- can that be done here as well?
drivers/event/dlb2/dlb2_priv.h line 343
Quoted 3 lines
drivers/event/dlb2/dlb2_priv.h new file +3
-0+}; + +/* TODO - Optimize memory use and layout */
Leftover TODO
Quoted 19 lines
drivers/event/dlb2/dlb2_priv.h new file +19
-0+struct dlb2_config { +int configured; +int reserved; +uint32_t num_ldb_credits; +uint32_t num_dir_credits; +struct dlb2_create_sched_domain_args resources; +}; + +enum dlb2_cos { +DLB2_COS_DEFAULT =-1 , +DLB2_COS_0 =0 , +DLB2_COS_1 , +DLB2_COS_2 , +DLB2_COS_3 +}; + +struct dlb2_hw_dev { +char device_name[DLB2_NAME_SIZE ]; +char device_path[DLB2_MAX_DEVICE_PATH ];
drivers/event/dlb2/dlb2_priv.h line 372
Quoted 10 lines
drivers/event/dlb2/dlb2_priv.h new file +10
-0+int device_path_id; +struct dlb2_config cfg; +struct dlb2_hw_resource_info info; +void *pf_dev;/* opaque pointer to PF PMD dev (struct dlb2_dev) */ +int device_id; +uint32_t domain_id; +int domain_id_valid; +enum dlb2_cos cos_id; +rte_spinlock_t resource_lock;/* for MP support */ +}; __rte_cache_aligned
Attribute needs to come before the semi-colon.
drivers/event/dlb2/dlb2_priv.h line 384
Quoted 13 lines
drivers/event/dlb2/dlb2_priv.h new file +9
-0+ +/* End HW related defines and structs */ + +/* Begin DLB2 PMD Eventdev related defines and structs */ + +#define DLB2_MAX_NUM_QUEUES \ + (DLB2_MAX_NUM_DIR_QUEUES + DLB2_MAX_NUM_LDB_QUEUES) + +#define DLB2_MAX_NUM_PORTS (DLB2_MAX_NUM_DIR_PORTS +
DLB2_MAX_NUM_LDB_PORTS)drivers/event/dlb2/dlb2_priv.h new file +3
-0+#define DLB2_MAX_INPUT_QUEUE_DEPTH 256 + +/* Used for parsing dir ports/queues. */
Does this comment refer to the following code? Seems out of place.
drivers/event/dlb2/dlb2_priv.h line 388
Quoted 4 lines
drivers/event/dlb2/dlb2_priv.h new file +4
-0+ +/* Note: eventdev currently is limited to 64 ports, but DLB hardware has 128 +* directed ports/queues. +*/
Does this refer to RTE_EVENT_MAX_QUEUES_PER_DEV? If so, looks like the change in the DLB PMD to increase that to 255 makes this comment unnecessary.
drivers/event/dlb2/dlb2_priv.h line 392
Quoted 4 lines
drivers/event/dlb2/dlb2_priv.h new file +4
-0+struct dlb2_dir_resource_list { +int entries; +uint32_t id[DLB2_MAX_NUM_DIR_PORTS ]; +};
Doesn't look this structure is used elsewhere in this series.
[...]
drivers/event/dlb2/dlb2_priv.h line 490
Quoted 12 lines
drivers/event/dlb2/dlb2_priv.h new file +12
-0+struct dlb2_eventdev_queue { +struct dlb2_queue qm_queue; +struct rte_event_queue_conf conf;/* User config */ +int depth_threshold;/* use default if 0 */ +uint32_t id; +bool setup_done; +uint8_t num_links; +}; + +struct dlb2_device_stats { +/* Device specific */ +};
Any need for this, since it's empty?
Quoted 57 lines
drivers/event/dlb2/dlb2_priv.h new file +34
-0+ +enum dlb2_run_state { +DLB2_RUN_STATE_STOPPED =0 , +DLB2_RUN_STATE_STOPPING , +DLB2_RUN_STATE_STARTING , +DLB2_RUN_STATE_STARTED +}; + +struct dlb2_eventdev { +struct dlb2_eventdev_port ev_ports[DLB2_MAX_NUM_PORTS ]; +struct dlb2_eventdev_queue ev_queues[DLB2_MAX_NUM_QUEUES ]; +uint8_t qm_ldb_to_ev_queue_id[DLB2_MAX_NUM_QUEUES ]; +uint8_t qm_dir_to_ev_queue_id[DLB2_MAX_NUM_QUEUES ]; +/* store num stats and offset of the stats for each queue */ +uint16_t xstats_count_per_qid[DLB2_MAX_NUM_QUEUES ]; +uint16_t xstats_offset_for_qid[DLB2_MAX_NUM_QUEUES ]; +/* store num stats and offset of the stats for each port */ +uint16_t xstats_count_per_port[DLB2_MAX_NUM_PORTS ]; +uint16_t xstats_offset_for_port[DLB2_MAX_NUM_PORTS ]; +struct dlb2_get_num_resources_args hw_rsrc_query_results; +uint32_t xstats_count_mode_queue; +struct dlb2_hw_dev qm_instance;/* strictly hw related */ +uint64_t global_dequeue_wait_ticks; +struct dlb2_xstats_entry *xstats; +struct rte_eventdev *event_dev;/* backlink to dev */ +uint32_t xstats_count_mode_dev; +uint32_t xstats_count_mode_port; +uint32_t xstats_count; +uint32_t inflights;/* use __atomic builtins */ +uint32_t new_event_limit; +int max_num_events_override; +int num_dir_credits_override; +volatile enum dlb2_run_state run_state; +uint16_t num_dir_queues;/* total num of evdev dir queues requested */
drivers/event/dlb2/dlb2_priv.h new file +4
-0+uint16_t num_dir_credits; +uint16_t num_ldb_credits; +uint16_t num_queues;/* total queues */ +uint16_t num_ldb_queues;/* total num of evdev ldb queues requested */
drivers/event/dlb2/dlb2_priv.h new file +17
-0+uint16_t num_ports;/* total num of evdev ports requested */ +uint16_t num_ldb_ports;/* total num of ldb ports requested */ +uint16_t num_dir_ports;/* total num of dir ports requested */ +bool umwait_allowed; +bool global_dequeue_wait;/* Not using per dequeue wait if true */ +bool defer_sched; +enum dlb2_cq_poll_modes poll_mode; +uint8_t revision; +bool configured; +uint16_t max_ldb_credits; +uint16_t max_dir_credits; + +/* force hw credit pool counters into exclusive cache lines */ + +/* use __atomic builtins */ /* shared hw cred */ + uint32_t ldb_credit_pool __rte_cache_aligned; +/* use __atomic builtins */ /* shared hw cred */
drivers/event/dlb2/dlb2_priv.h line 555
Quoted 10 lines
drivers/event/dlb2/dlb2_priv.h new file +10
-0+ uint32_t dir_credit_pool __rte_cache_aligned; + +/* Device stats */ + struct dlb2_device_stats stats __rte_cache_aligned; +}; + +/* used for collecting and passing around the dev args */ +struct dlb2_qid_depth_thresholds { +int val[DLB2_MAX_NUM_QUEUES ]; +};
Nit: whitespace between the struct definitions, to match the rest of the file.
Quoted 5 lines
drivers/event/dlb2/dlb2_priv.h new file +5
-0+struct dlb2_devargs { +int socket_id; +int max_num_events; +int num_dir_credits_override; +int dev_id;
drivers/event/dlb2/dlb2_priv.h line 570
Quoted 10 lines
drivers/event/dlb2/dlb2_priv.h new file +10
-0+int defer_sched; +struct dlb2_qid_depth_thresholds qid_depth_thresholds; +enum dlb2_cos cos_id; +}; + +/* End Eventdev related defines and structs */ + +/* Forwards for non-inlined functions */ + +int dlb2_uninit (const char *name);
This function is never defined
Quoted 35 lines
drivers/event/dlb2/dlb2_priv.h new file +9
-0+ +void dlb2_eventdev_dump (struct rte_eventdev *dev,FILE *f); + +int dlb2_xstats_init (struct dlb2_eventdev *dlb2); + +void dlb2_xstats_uninit (struct dlb2_eventdev *dlb2); + +int dlb2_eventdev_xstats_get (const struct rte_eventdev *dev, + enum rte_event_dev_xstats_mode mode, uint8_t
queue_port_id,drivers/event/dlb2/dlb2_priv.h new file +4
-0+ const unsigned int ids[], uint64_t values[], unsigned int n); + +int dlb2_eventdev_xstats_get_names (const struct rte_eventdev *dev, + enum rte_event_dev_xstats_mode mode, uint8_t
queue_port_id,drivers/event/dlb2/dlb2_priv.h new file +20
-0+struct rte_event_dev_xstats_name *xstat_names, + unsigned int *ids, unsigned int size); + +uint64_t dlb2_eventdev_xstats_get_by_name (const struct rte_eventdev *dev, +const char *name,unsigned int *id); + +int dlb2_eventdev_xstats_reset (struct rte_eventdev *dev, +enum rte_event_dev_xstats_mode mode, +int16_t queue_port_id, +const uint32_t ids[], +uint32_t nb_ids); + +int test_dlb2_eventdev (void ); + +int dlb2_primary_eventdev_probe (struct rte_eventdev *dev, +const char *name, +struct dlb2_devargs *dlb2_args); + +int dlb2_secondary_eventdev_probe (struct rte_eventdev *dev, +const char *name);
drivers/event/dlb2/dlb2_priv.h line 613
Quoted 10 lines
drivers/event/dlb2/dlb2_priv.h new file +10
-0+ +uint32_t dlb2_get_queue_depth (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *queue); + +/* arg parsing helper functions */ +int dlb2_parse_params (const char *params, +const char *name, +struct dlb2_devargs *dlb2_args); + +int dlb2_string_to_int (int *result,const char *str);
Looks like this is only used in dlb2.c, is the forward declaration needed?
Thanks,
Gage
Re: [dpdk-dev] [PATCH 04/22] event/dlb2: add definitions shared with LKM or shared code
In reply to Timothy McDanielQuoted 20 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 04/22] event/dlb2: add definitions shared with LKM or shared codeAdd headers containing structs and constants shared between the PMD and the shared code. The term shared code refers to the code that implements the hardware interface. The shared code is introduced in the probe patch, and then is extended as additional eventdev PMD entry points are added to the patchset. In the case of the bifurcated PMD (to be introduced in the future), the shared code is contained in the Linux kernel module itself.
Quoted 1 line
Signed-off-by Timothy McDaniel <[email protected]>
The checkpatch warnings are codespell false positives; looks good to me.
Thanks,
Gage
Re: [dpdk-dev] [PATCH 05/22] event/dlb2: add inline functions
In reply to Timothy McDanielQuoted 30 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 05/22] event/dlb2: add inline functionsAdd miscellaneous inline functions that may be called from multiple files. These functions include inline assembly of new x86 instructions, such as movdir64b, since they are not available as builtin functions in the minimum supported GCC version.
Signed-off-by Timothy McDaniel <[email protected]>--- drivers/event/dlb2/dlb2_inline_fns.h | 85 ++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 drivers/event/dlb2/dlb2_inline_fns.hdrivers/event/dlb2/dlb2_inline_fns.h new file +0
-0
drivers/event/dlb2/dlb2_inline_fns.h line 9
Quoted 9 lines
drivers/event/dlb2/dlb2_inline_fns.h new file +9
-0@@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef _DLB2_INLINE_FNS_H_ +#define _DLB2_INLINE_FNS_H_ + +/* Inline functions required in more than one source file. +*/
Nit: this comment can be a one-liner.
drivers/event/dlb2/dlb2_inline_fns.h line 20
Quoted 11 lines
drivers/event/dlb2/dlb2_inline_fns.h new file +11
-0+ +static inline struct dlb2_eventdev * +dlb2_pmd_priv (const struct rte_eventdev *eventdev) +{ +return eventdev->data->dev_private; +} + +static inline void +dlb2_umonitor (volatile void *addr) +{ +/* TODO: Change to proper assembly when compiler support available */
I would drop/reword the 'TODO', here and elsewhere in the file, so it doesn't give the wrong impression that this code is incomplete.
drivers/event/dlb2/dlb2_inline_fns.h line 24
Quoted 4 lines
drivers/event/dlb2/dlb2_inline_fns.h new file +4
-0+ asmvolatile (".byte 0xf3, 0x0f, 0xae, 0xf7\t\n" + : + :"D" (addr)); +}
Thanks,
Gage
Re: [dpdk-dev] [PATCH 03/22] event/dlb2: add private data structures and constants
In reply to Timothy McDanielQuoted 19 lines
+/*!
+ * Configure the token pop mode for a DLB2 port. By default, all ports use
+ *AUTO_POP. This function must be called before callingrte_event_port_setup()
+ * for the port, but after callingrte_event_dev_configure().
+ *
+ * @paramdev_id
+ * The identifier of the event device.
+ * @paramport_id
+ * The identifier of the event port.
+ * @param mode
+ * The token pop mode.
+ *
+ * @return
+ * - 0: Success
+ * - EINVAL: Invaliddev_id,port_id, or mode
+ * - EINVAL: The DLB2 is not configured, is already running, or the port is
+ * already setup
+ */
drivers/event/dlb2/rte_pmd_dlb2.h line 53
Quoted 5 lines
drivers/event/dlb2/rte_pmd_dlb2.h new file +5
-0+ +int +rte_pmd_dlb2_set_token_pop_mode (uint8_t dev_id, +uint8_t port_id, +enum dlb2_token_pop_mode mode);
When I build the full series (plus dependencies), I get this error:
"rte_pmd_dlb2_set_token_pop_mode is not flagged as experimental
but is listed in version map
Please add __rte_experimental to the definition of rte_pmd_dlb2_set_token_pop_mode "
(https://doc.dpdk.org/guides/contributing/abi_policy.html#experimental-apis)
Thanks,
Gage
Re: [dpdk-dev] [PATCH 06/22] event/dlb2: add probe
In reply to Timothy McDanielQuoted 21 lines
drivers/event/dlb2/dlb2.c new file +15
-0@@ -0,0 +1,557 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#include <assert.h> +#include <errno.h> +#include <nmmintrin.h> +#include <pthread.h> +#include <stdint.h> +#include <stdbool.h> +#include <stdio.h> +#include <string.h> +#include <sys/mman.h> +#include <sys/fcntl.h> +
drivers/event/dlb2/dlb2.c line 25
Quoted 10 lines
drivers/event/dlb2/dlb2.c new file +10
-0+#include <rte_common.h> +#include <rte_config.h> +#include <rte_cycles.h> +#include <rte_debug.h> +#include <rte_dev.h> +#include <rte_errno.h> +#include <rte_eventdev.h> +#include <rte_eventdev_pmd.h> +#include <rte_kvargs.h> +#include <rte_io.h>
Nit: flip the order of rte_kvargs.h and rte_io.h
Quoted 4 lines
drivers/event/dlb2/dlb2.c new file +4
-0+#include <rte_log.h> +#include <rte_malloc.h> +#include <rte_mbuf.h> +#include <rte_prefetch.h>
drivers/event/dlb2/dlb2.c line 39
Quoted 10 lines
drivers/event/dlb2/dlb2.c new file +10
-0+#include <rte_ring.h> +#include <rte_string_fns.h> + +#include "dlb2_priv.h" +#include "dlb2_iface.h" +#include "dlb2_inline_fns.h" + +#if !definedRTE_ARCH_X86_64 +#error "This implementation only supports RTE_ARCH_X86_64 architecture." +#endif
In patch #1, meson.build allows RTE_ARCH_X86 -- should that be removed?
Quoted 54 lines
drivers/event/dlb2/dlb2.c new file +7
-0+ +/* +* Resources exposed to eventdev. Some values overridden at runtime using +* values returned by the DLB kernel driver. +*/ +#if (RTE_EVENT_MAX_QUEUES_PER_DEV >UINT8_MAX ) +#error "RTE_EVENT_MAX_QUEUES_PER_DEV cannot fit in member
max_event_queues"drivers/event/dlb2/dlb2.c new file +19
-0+#endif +static struct rte_event_dev_info evdev_dlb2_default_info = { + .driver_name ="" ,/* probe will set */ + .min_dequeue_timeout_ns =DLB2_MIN_DEQUEUE_TIMEOUT_NS , + .max_dequeue_timeout_ns =DLB2_MAX_DEQUEUE_TIMEOUT_NS , +#if (RTE_EVENT_MAX_QUEUES_PER_DEV < DLB2_MAX_NUM_LDB_QUEUES) + .max_event_queues =RTE_EVENT_MAX_QUEUES_PER_DEV , +#else + .max_event_queues =DLB2_MAX_NUM_LDB_QUEUES , +#endif + .max_event_queue_flows =DLB2_MAX_NUM_FLOWS , + .max_event_queue_priority_levels =DLB2_QID_PRIORITIES , + .max_event_priority_levels =DLB2_QID_PRIORITIES , + .max_event_ports =DLB2_MAX_NUM_LDB_PORTS , + .max_event_port_dequeue_depth =DLB2_MAX_CQ_DEPTH , + .max_event_port_enqueue_depth =DLB2_MAX_ENQUEUE_DEPTH , + .max_event_port_links =DLB2_MAX_NUM_QIDS_PER_LDB_CQ , + .max_num_events =DLB2_MAX_NUM_LDB_CREDITS , + .max_single_link_event_port_queue_pairs =
DLB2_MAX_NUM_DIR_PORTS,drivers/event/dlb2/dlb2.c new file +26
-0+ .event_dev_cap = (RTE_EVENT_DEV_CAP_QUEUE_QOS | +RTE_EVENT_DEV_CAP_EVENT_QOS | +RTE_EVENT_DEV_CAP_BURST_MODE | +RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED | +RTE_EVENT_DEV_CAP_IMPLICIT_RELEASE_DISABLE | + RTE_EVENT_DEV_CAP_QUEUE_ALL_TYPES), +}; + +/* These functions will vary based on processor capabilities */ +static struct dlb2_port_low_level_io_functions qm_mmio_fns; + +struct process_local_port_data +dlb2_port[DLB2_MAX_NUM_PORTS ][DLB2_NUM_PORT_TYPES ]; + +/* override defaults with value(s) provided on command line */ +static void +dlb2_init_queue_depth_thresholds (struct dlb2_eventdev *dlb2, +int *qid_depth_thresholds) +{ +int q; + +for (q =0 ; q <DLB2_MAX_NUM_QUEUES ; q++) { +if (qid_depth_thresholds[q] !=0 ) + dlb2->ev_queues[q].depth_threshold = + qid_depth_thresholds[q]; + }
drivers/event/dlb2/dlb2.c line 101
Quoted 10 lines
drivers/event/dlb2/dlb2.c new file +10
-0+} + +static int +dlb2_hw_query_resources (struct dlb2_eventdev *dlb2) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_hw_resource_info *dlb2_info = &handle->info; +int ret; + +/* Query driver resources provisioned for this VF */
"VF" -> "device"?
drivers/event/dlb2/dlb2.c line 107
Quoted 6 lines
drivers/event/dlb2/dlb2.c new file +6
-0+ + ret =dlb2_iface_get_num_resources (handle, + &dlb2->hw_rsrc_query_results); +if (ret) { +DLB2_LOG_ERR ("ioctl get dlb2 num resources, err=%d\n" , + ret);
Nit: this could be a one-liner
Quoted 61 lines
drivers/event/dlb2/dlb2.c new file +42
-0+return ret; + } + +/* Complete filling in device resource info returned to evdev app, +* overriding any default values. +* The capabilities (CAPs) were set at compile time. +*/ + + evdev_dlb2_default_info.max_event_queues = + dlb2->hw_rsrc_query_results.num_ldb_queues; + + evdev_dlb2_default_info.max_event_ports = + dlb2->hw_rsrc_query_results.num_ldb_ports; + + evdev_dlb2_default_info.max_num_events = + dlb2->hw_rsrc_query_results.num_ldb_credits; + +/* Save off values used when creating the scheduling domain. */ + + handle->info.num_sched_domains = + dlb2->hw_rsrc_query_results.num_sched_domains; + + handle->info.hw_rsrc_max.nb_events_limit = + dlb2->hw_rsrc_query_results.num_ldb_credits; + + handle->info.hw_rsrc_max.num_queues = + dlb2->hw_rsrc_query_results.num_ldb_queues + + dlb2->hw_rsrc_query_results.num_dir_ports; + + handle->info.hw_rsrc_max.num_ldb_queues = + dlb2->hw_rsrc_query_results.num_ldb_queues; + + handle->info.hw_rsrc_max.num_ldb_ports = + dlb2->hw_rsrc_query_results.num_ldb_ports; + + handle->info.hw_rsrc_max.num_dir_ports = + dlb2->hw_rsrc_query_results.num_dir_ports; + + handle->info.hw_rsrc_max.reorder_window_size = + dlb2->hw_rsrc_query_results.num_hist_list_entries; + + rte_memcpy(dlb2_info, &handle->info.hw_rsrc_max,
sizeof(*dlb2_info));drivers/event/dlb2/dlb2.c new file +18
-0+ +return 0 ; +} + +static void +dlb2_qm_mmio_fn_init (void ) +{ +/* Process-local function pointers for performing low level port i/o */ + +if (rte_cpu_get_flag_enabled (RTE_CPUFLAG_MOVDIR64B )) + qm_mmio_fns.pp_enqueue_four = dlb2_movdir64b; +else + qm_mmio_fns.pp_enqueue_four = dlb2_movntdq; +} + +#define RTE_BASE_10 10 + +int dlb2_string_to_int (int *result,const char *str)
drivers/event/dlb2/dlb2.c line 177
Quoted 10 lines
drivers/event/dlb2/dlb2.c new file +10
-0+{ +long ret; + +if (str ==NULL || result ==NULL ) +return -EINVAL ; + + errno =0 ; + ret =strtol (str,NULL ,RTE_BASE_10 ); +if (errno) +return -errno;
Looks like strtol will return 0 if 'str' doesn't contain valid numeric characters, but won't set errno in this case. See the "No digits were found" case in the man page's example at the bottom: http://man7.org/linux/man-pages/man3/strtol.3.html
Quoted 33 lines
drivers/event/dlb2/dlb2.c new file +11
-0+ +/* long int and int may be different width for some architectures */ +if (ret <INT_MIN || ret >INT_MAX ) +return -EINVAL ; + + *result = ret; +return 0 ; +} + +static int +set_numa_node(const char *key __rte_unused, const char *value, void*opaque)
drivers/event/dlb2/dlb2.c new file +21
-0+{ +int *socket_id = opaque; +int ret; + + ret =dlb2_string_to_int (socket_id, value); +if (ret <0 ) +return ret; + +if (*socket_id >RTE_MAX_NUMA_NODES ) +return -EINVAL ; +return 0 ; +} + +static int +set_max_num_events(const char *key __rte_unused, +const char *value, +void *opaque) +{ +int *max_num_events = opaque; +int ret; +
drivers/event/dlb2/dlb2.c line 219
Quoted 10 lines
drivers/event/dlb2/dlb2.c new file +10
-0+if (value ==NULL || opaque ==NULL ) { +DLB2_LOG_ERR ("NULL pointer\n" ); +return -EINVAL ; + } + + ret =dlb2_string_to_int (max_num_events, value); +if (ret <0 ) +return ret; + +
Nit: extra newline
Quoted 69 lines
drivers/event/dlb2/dlb2.c new file +3
-0+if (*max_num_events <0 || *max_num_events > +DLB2_MAX_NUM_LDB_CREDITS ) { + DLB2_LOG_ERR("dlb2: max_num_events must be between 0 and%d\n",
drivers/event/dlb2/dlb2.c new file +27
-0+ DLB2_MAX_NUM_LDB_CREDITS); +return -EINVAL ; + } + +return 0 ; +} + +static int +set_num_dir_credits(const char *key __rte_unused, +const char *value, +void *opaque) +{ +int *num_dir_credits = opaque; +int ret; + +if (value ==NULL || opaque ==NULL ) { +DLB2_LOG_ERR ("NULL pointer\n" ); +return -EINVAL ; + } + + ret =dlb2_string_to_int (num_dir_credits, value); +if (ret <0 ) +return ret; + +if (*num_dir_credits <0 || + *num_dir_credits >DLB2_MAX_NUM_DIR_CREDITS ) { + DLB2_LOG_ERR("dlb2: num_dir_credits must be between 0 and%d\n",
drivers/event/dlb2/dlb2.c new file +37
-0+ DLB2_MAX_NUM_DIR_CREDITS); +return -EINVAL ; + } + +return 0 ; +} + +static int +set_dev_id(const char *key __rte_unused, +const char *value, +void *opaque) +{ +int *dev_id = opaque; +int ret; + +if (value ==NULL || opaque ==NULL ) { +DLB2_LOG_ERR ("NULL pointer\n" ); +return -EINVAL ; + } + + ret =dlb2_string_to_int (dev_id, value); +if (ret <0 ) +return ret; + +return 0 ; +} + +static int +set_cos(const char *key __rte_unused, +const char *value, +void *opaque) +{ +enum dlb2_cos *cos_id = opaque; +int x =0 ; +int ret; + +if (value ==NULL || opaque ==NULL ) {
drivers/event/dlb2/dlb2.c line 296
Quoted 10 lines
drivers/event/dlb2/dlb2.c new file +10
-0+DLB2_LOG_ERR ("NULL pointer\n" ); +return -EINVAL ; + } + + ret =dlb2_string_to_int (&x, value); +if (ret <0 ) +return ret; + +if (x !=DLB2_COS_DEFAULT && (x <DLB2_COS_0 || x >DLB2_COS_3 )) { +DLB2_LOG_ERR ("COS %d out of range, must be 0-3\n" , x);
If DLB2_COS_DEFAULT (-1) is a valid value, should this printf reflect that?
Quoted 36 lines
drivers/event/dlb2/dlb2.c new file +36
-0+return -EINVAL ; + } + + *cos_id = x; + +return 0 ; +} + + +static int +set_qid_depth_thresh(const char *key __rte_unused, +const char *value, +void *opaque) +{ +struct dlb2_qid_depth_thresholds *qid_thresh = opaque; +int first, last, thresh, i; + +if (value ==NULL || opaque ==NULL ) { +DLB2_LOG_ERR ("NULL pointer\n" ); +return -EINVAL ; + } + +/* command line override may take one of the following 3 forms: +* qid_depth_thresh=all:<threshold_value> ... all queues +* qid_depth_thresh=qidA-qidB:<threshold_value> ... a range of queues +* qid_depth_thresh=qid:<threshold_value> ... just one queue +*/ +if (sscanf (value,"all:%d" , &thresh) ==1 ) { + first =0 ; + last =DLB2_MAX_NUM_QUEUES -1 ; + }else if (sscanf (value,"%d-%d:%d" , &first, &last, &thresh) ==3 ) { +/* we have everything we need */ + }else if (sscanf (value,"%d:%d" , &first, &thresh) ==2 ) { + last = first; + }else { + DLB2_LOG_ERR("Error parsing qid depth vdev arg. Should be
Quoted 1 line
all:val, qid-qid:val, or qid:val\n");
"vdev arg" -> "devarg"
Quoted 6 lines
drivers/event/dlb2/dlb2.c new file +5
-0+return -EINVAL ; + } + +if (first > last || first <0 || last >=DLB2_MAX_NUM_QUEUES ) { + DLB2_LOG_ERR("Error parsing qid depth vdev arg, invalid qidvalue\n");
Ditto
Quoted 6 lines
drivers/event/dlb2/dlb2.c new file +5
-0+return -EINVAL ; + } + +if (thresh <0 || thresh >DLB2_MAX_QUEUE_DEPTH_THRESHOLD ) { + DLB2_LOG_ERR("Error parsing qid depth vdev arg, threshold >%d\n",
Ditto
Quoted 131 lines
drivers/event/dlb2/dlb2.c new file +34
-0+ DLB2_MAX_QUEUE_DEPTH_THRESHOLD); +return -EINVAL ; + } + +for (i = first; i <= last; i++) + qid_thresh->val[i] = thresh;/* indexed by qid */ + +return 0 ; +} + +static void +dlb2_entry_points_init (struct rte_eventdev *dev) +{ +RTE_SET_USED (dev); + +/* Eventdev PMD entry points */ +} + +int +dlb2_primary_eventdev_probe (struct rte_eventdev *dev, +const char *name, +struct dlb2_devargs *dlb2_args) +{ +struct dlb2_eventdev *dlb2; +int err, i; + + dlb2 = dev->data->dev_private; + + dlb2->event_dev = dev;/* backlink */ + + evdev_dlb2_default_info.driver_name = name; + + dlb2->max_num_events_override = dlb2_args->max_num_events; + dlb2->num_dir_credits_override = dlb2_args-Quoted 1 line
num_dir_credits_override;drivers/event/dlb2/dlb2.c new file +9
-0+ dlb2->qm_instance.device_path_id = dlb2_args->dev_id; + dlb2->qm_instance.cos_id = dlb2_args->cos_id; + +/* Open the interface. +* For vdev mode, this means open the dlb2 kernel module. +*/ + err =dlb2_iface_open (&dlb2->qm_instance, name); +if (err <0 ) { + DLB2_LOG_ERR("could not open event hardware device,err=%d\n",
drivers/event/dlb2/dlb2.c new file +8
-0+ err); +return err; + } + + err =dlb2_iface_get_device_version (&dlb2->qm_instance, + &dlb2->revision); +if (err <0 ) { + DLB2_LOG_ERR("dlb2: failed to get the device version,err=%d\n",
drivers/event/dlb2/dlb2.c new file +14
-0+ err); +return err; + } + + err =dlb2_hw_query_resources (dlb2); +if (err) { +DLB2_LOG_ERR ("get resources err=%d for %s\n" , + err, name); +return err; + } + +dlb2_iface_hardware_init (&dlb2->qm_instance); + + err = dlb2_iface_get_cq_poll_mode(&dlb2->qm_instance, &dlb2-Quoted 1 line
poll_mode);drivers/event/dlb2/dlb2.c new file +38
-0+if (err <0 ) { +DLB2_LOG_ERR ("dlb2: failed to get the poll mode, err=%d\n" , + err); +return err; + } + +/* Initialize each port's token pop mode */ +for (i =0 ; i <DLB2_MAX_NUM_PORTS ; i++) + dlb2->ev_ports[i].qm_port.token_pop_mode =AUTO_POP ; + +rte_spinlock_init (&dlb2->qm_instance.resource_lock); + +dlb2_qm_mmio_fn_init (); + +dlb2_iface_low_level_io_init (); + +dlb2_entry_points_init (dev); + +dlb2_init_queue_depth_thresholds (dlb2, + dlb2_args->qid_depth_thresholds.val); + +return 0 ; +} + +int +dlb2_secondary_eventdev_probe (struct rte_eventdev *dev, +const char *name) +{ +struct dlb2_eventdev *dlb2; +int err; + + dlb2 = dev->data->dev_private; + + evdev_dlb2_default_info.driver_name = name; + + err =dlb2_iface_open (&dlb2->qm_instance, name); +if (err <0 ) { + DLB2_LOG_ERR("could not open event hardware device,err=%d\n",
drivers/event/dlb2/dlb2.c new file +23
-0+ err); +return err; + } + + err =dlb2_hw_query_resources (dlb2); +if (err) { +DLB2_LOG_ERR ("get resources err=%d for %s\n" , + err, name); +return err; + } + +dlb2_qm_mmio_fn_init (); + +dlb2_iface_low_level_io_init (); + +dlb2_entry_points_init (dev); + +return 0 ; +} + +int +dlb2_parse_params(const char *params, + const char *name,
drivers/event/dlb2/dlb2.c line 478
Quoted 10 lines
drivers/event/dlb2/dlb2.c new file +10
-0+ struct dlb2_devargs *dlb2_args) +{ +int ret =0 ; +static const char *const args[] = {NUMA_NODE_ARG , +DLB2_MAX_NUM_EVENTS , +DLB2_NUM_DIR_CREDITS , +DEV_ID_ARG , +DLB2_QID_DEPTH_THRESH_ARG , +DLB2_COS_ARG , +NULL };
Nit: const_args entries alignment
Quoted 77 lines
drivers/event/dlb2/dlb2.c new file +7
-0+ +if (params !=NULL && params[0 ] !='\0' ) { +struct rte_kvargs *kvlist =rte_kvargs_parse (params, args); + +if (!kvlist) { + RTE_LOG(INFO, PMD, + "Ignoring unsupported parameters when creatingdevice '%s'\n",
drivers/event/dlb2/dlb2.c new file +7
-0+ name); + }else { +int ret =rte_kvargs_process (kvlist,NUMA_NODE_ARG , + set_numa_node, + &dlb2_args->socket_id); +if (ret !=0 ) { + DLB2_LOG_ERR("%s: Error parsing numa nodeparameter",
drivers/event/dlb2/dlb2.c new file +6
-0+ name); +rte_kvargs_free (kvlist); +return ret; + } + + ret = rte_kvargs_process(kvlist,
DLB2_MAX_NUM_EVENTS,drivers/event/dlb2/dlb2.c new file +4
-0+ set_max_num_events, + &dlb2_args->max_num_events); +if (ret !=0 ) { + DLB2_LOG_ERR("%s: Error parsing
max_num_eventsparameter",drivers/event/dlb2/dlb2.c new file +11
-0+ name); +rte_kvargs_free (kvlist); +return ret; + } + + ret =rte_kvargs_process (kvlist, +DLB2_NUM_DIR_CREDITS , + set_num_dir_credits, + &dlb2_args->num_dir_credits_override); +if (ret !=0 ) { + DLB2_LOG_ERR("%s: Error parsing
num_dir_creditsparameter",drivers/event/dlb2/dlb2.c new file +10
-0+ name); +rte_kvargs_free (kvlist); +return ret; + } + + ret =rte_kvargs_process (kvlist,DEV_ID_ARG , + set_dev_id, + &dlb2_args->dev_id); +if (ret !=0 ) { + DLB2_LOG_ERR("%s: Error parsing dev_idparameter",
drivers/event/dlb2/dlb2.c new file +12
-0+ name); +rte_kvargs_free (kvlist); +return ret; + } + + ret =rte_kvargs_process ( + kvlist, +DLB2_QID_DEPTH_THRESH_ARG , + set_qid_depth_thresh, + &dlb2_args->qid_depth_thresholds); +if (ret !=0 ) { + DLB2_LOG_ERR("%s: Error parsing
qid_depth_threshparameter",drivers/event/dlb2/dlb2.c new file +10
-0+ name); +rte_kvargs_free (kvlist); +return ret; + } + + ret =rte_kvargs_process (kvlist,DLB2_COS_ARG , + set_cos, + &dlb2_args->cos_id); +if (ret !=0 ) { + DLB2_LOG_ERR("%s: Error parsing cosparameter",
drivers/event/dlb2/dlb2.c new file +2
-0+ name); +rte_kvargs_free (kvlist);
drivers/event/dlb2/dlb2.c line 557
Quoted 10 lines
drivers/event/dlb2/dlb2.c new file +10
-0+return ret; + } + +rte_kvargs_free (kvlist); + } + } +return ret; +} +RTE_LOG_REGISTER (eventdev_dlb2_log_level, pmd.event.dlb2,NOTICE ); +
Nit: whitespace at the end of the file
Quoted 15 lines
drivers/event/dlb2/dlb2_iface.c new file +9
-0@@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#include <stdbool.h> +#include <stdint.h> +#include <rte_debug.h> +#include <rte_bus_pci.h> +#include <rte_log.h>
drivers/event/dlb2/dlb2_iface.c line 19
Quoted 10 lines
drivers/event/dlb2/dlb2_iface.c new file +10
-0+#include <rte_dev.h> +#include <rte_mbuf.h> +#include <rte_ring.h> +#include <rte_errno.h> +#include <rte_kvargs.h> +#include <rte_malloc.h> +#include <rte_cycles.h> +#include <rte_io.h> +#include <rte_eventdev.h> +#include <rte_eventdev_pmd.h>
Doesn't look stdbool.h and the rte_*.h includes are needed here -- I was able to build the full series with them commented out.
[...]
Quoted 92 lines
drivers/event/dlb2/pf/base/dlb2_osdep.h new file +35
-0@@ -0,0 +1,248 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef __DLB2_OSDEP_H +#define __DLB2_OSDEP_H + +#include <string.h> +#include <time.h> +#include <unistd.h> +#include <pthread.h> + +#include <rte_string_fns.h> +#include <rte_cycles.h> +#include <rte_io.h> +#include <rte_log.h> +#include <rte_spinlock.h> +#include "../dlb2_main.h" +#include "dlb2_resource.h" +#include "../../dlb2_log.h" +#include "../../dlb2_user.h" + + +#define DLB2_PCI_REG_READ (addr) rte_read32((void *)addr) +#define DLB2_PCI_REG_WRITE (reg, value) rte_write32(value, (void *)reg) + +/* Read/write register 'reg' in the CSR BAR space */ +#define DLB2_CSR_REG_ADDR (a, reg) ((void *)((uintptr_t)(a)->csr_kva + (reg))) +#define DLB2_CSR_RD (hw, reg) \ + DLB2_PCI_REG_READ(DLB2_CSR_REG_ADDR((hw), (reg))) +#define DLB2_CSR_WR (hw, reg, value) \ + DLB2_PCI_REG_WRITE(DLB2_CSR_REG_ADDR((hw), (reg)), (value)) + +/* Read/write register 'reg' in the func BAR space */ +#define DLB2_FUNC_REG_ADDR (a, reg) ((void *)((uintptr_t)(a)->func_kva +(reg)))
drivers/event/dlb2/pf/base/dlb2_osdep.h new file +38
-0+#define DLB2_FUNC_RD (hw, reg) \ + DLB2_PCI_REG_READ(DLB2_FUNC_REG_ADDR((hw), (reg))) +#define DLB2_FUNC_WR (hw, reg, value) \ + DLB2_PCI_REG_WRITE(DLB2_FUNC_REG_ADDR((hw), (reg)), (value)) + +/* Map to PMDs logging interface */ +#define DLB2_ERR(dev, fmt, args...) \ + DLB2_LOG_ERR(fmt, ## args) + +#define DLB2_INFO(dev, fmt, args...) \ + DLB2_LOG_INFO(fmt, ## args) + +#define DLB2_DEBUG(dev, fmt, args...) \ + DLB2_LOG_DBG(fmt, ## args) + +/** +* os_udelay() - busy-wait for a number of microseconds +* @usecs: delay duration. +*/ +static inline void os_udelay (int usecs) +{ +rte_delay_us (usecs); +} + +/** +* os_msleep() - sleep for a number of milliseconds +* @usecs: delay duration. +*/ +static inline void os_msleep (int msecs) +{ +rte_delay_ms (msecs); +} + +#define DLB2_PP_BASE (__is_ldb) \ + ((__is_ldb) ? DLB2_LDB_PP_BASE : DLB2_DIR_PP_BASE) + +/** +* os_map_producer_port() - map a producer port into the caller's address space
+ * @hw:dlb2_hwhandle for a particular device.
+ * @port_id: port ID
+ * @is_ldb: true for load-balanced port, false for a directed port
+ *
+ * This function maps the requested producer port memory into the caller's
+ * address space.
+ *
+ * Return:
+ * Returns the base address at which the PP memory was mapped, else NULL.
+ */
drivers/event/dlb2/pf/base/dlb2_osdep.h line 91
Quoted 8 lines
drivers/event/dlb2/pf/base/dlb2_osdep.h new file +8
-0+static inline void *os_map_producer_port (struct dlb2_hw *hw, +u8 port_id, +bool is_ldb) +{ +uint64_t addr; +uint64_t pp_dma_base; + +
Nit: extra whitespace
[...]
Quoted 194 lines
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +62
-0@@ -0,0 +1,447 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef __DLB2_OSDEP_BITMAP_H +#define __DLB2_OSDEP_BITMAP_H + +#include <stdint.h> +#include <stdbool.h> +#include <stdio.h> +#include <unistd.h> +#include <rte_bitmap.h> +#include <rte_string_fns.h> +#include <rte_malloc.h> +#include <rte_errno.h> +#include "../dlb2_main.h" + +/*************************/ +/*** Bitmap operations ***/ +/*************************/ +struct dlb2_bitmap { +struct rte_bitmap *map; +unsigned int len; +}; + +/** +* dlb2_bitmap_alloc() - alloc a bitmap data structure +* @bitmap: pointer to dlb2_bitmap structure pointer. +* @len: number of entries in the bitmap. +* +* This function allocates a bitmap and initializes it with length @len. All +* entries are initially zero. +* +* Return: +* Returns 0 upon success, < 0 otherwise. +* +* Errors: +* EINVAL - bitmap is NULL or len is 0. +* ENOMEM - could not allocate memory for the bitmap data structure. +*/ +static inline int dlb2_bitmap_alloc (struct dlb2_bitmap **bitmap, +unsigned int len) +{ +struct dlb2_bitmap *bm; +void *mem; +uint32_t alloc_size; +uint32_t nbits = (uint32_t )len; + +if (!bitmap || nbits ==0 ) +return -EINVAL ; + +/* Allocate DLB2 bitmap control struct */ + bm =rte_malloc ("DLB2_PF" , +sizeof (struct dlb2_bitmap ), +RTE_CACHE_LINE_SIZE ); + +if (!bm) +return -ENOMEM ; + +/* Allocate bitmap memory */ + alloc_size =rte_bitmap_get_memory_footprint (nbits); + mem = rte_malloc("DLB2_PF_BITMAP", alloc_size,
RTE_CACHE_LINE_SIZE);drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +124
-0+if (!mem) { +rte_free (bm); +return -ENOMEM ; + } + + bm->map =rte_bitmap_init (len, mem, alloc_size); +if (!bm->map) { +rte_free (mem); +rte_free (bm); +return -ENOMEM ; + } + + bm->len = len; + + *bitmap = bm; + +return 0 ; +} + +/** +* dlb2_bitmap_free() - free a previously allocated bitmap data structure +* @bitmap: pointer to dlb2_bitmap structure. +* +* This function frees a bitmap that was allocated with dlb2_bitmap_alloc(). +*/ +static inline void dlb2_bitmap_free (struct dlb2_bitmap *bitmap) +{ +if (!bitmap) +return ; + +rte_free (bitmap->map); +rte_free (bitmap); +} + +/** +* dlb2_bitmap_fill() - fill a bitmap with all 1s +* @bitmap: pointer to dlb2_bitmap structure. +* +* This function sets all bitmap values to 1. +* +* Return: +* Returns 0 upon success, < 0 otherwise. +* +* Errors: +* EINVAL - bitmap is NULL or is uninitialized. +*/ +static inline int dlb2_bitmap_fill (struct dlb2_bitmap *bitmap) +{ +unsigned int i; + +if (!bitmap || !bitmap->map) +return -EINVAL ; + +/* TODO - optimize */ +for (i =0 ; i != bitmap->len; i++) +rte_bitmap_set (bitmap->map, i); + +return 0 ; +} + +/** +* dlb2_bitmap_fill() - fill a bitmap with all 0s +* @bitmap: pointer to dlb2_bitmap structure. +* +* This function sets all bitmap values to 0. +* +* Return: +* Returns 0 upon success, < 0 otherwise. +* +* Errors: +* EINVAL - bitmap is NULL or is uninitialized. +*/ +static inline int dlb2_bitmap_zero (struct dlb2_bitmap *bitmap) +{ +if (!bitmap || !bitmap->map) +return -EINVAL ; + +rte_bitmap_reset (bitmap->map); + +return 0 ; +} + +/** +* dlb2_bitmap_set() - set a bitmap entry +* @bitmap: pointer to dlb2_bitmap structure. +* @bit: bit index. +* +* Return: +* Returns 0 upon success, < 0 otherwise. +* +* Errors: +* EINVAL - bitmap is NULL or is uninitialized, or bit is larger than the +* bitmap length. +*/ +static inline int dlb2_bitmap_set (struct dlb2_bitmap *bitmap, +unsigned int bit) +{ +if (!bitmap || !bitmap->map) +return -EINVAL ; + +if (bitmap->len <= bit) +return -EINVAL ; + +rte_bitmap_set (bitmap->map, bit); + +return 0 ; +} + +/** +* dlb2_bitmap_set_range() - set a range of bitmap entries +* @bitmap: pointer to dlb2_bitmap structure. +* @bit: starting bit index. +* @len: length of the range. +* +* Return: +* Returns 0 upon success, < 0 otherwise. +* +* Errors: +* EINVAL - bitmap is NULL or is uninitialized, or the range exceeds the bitmap +* length. +*/ +static inline int dlb2_bitmap_set_range (struct dlb2_bitmap *bitmap, +unsigned int bit, +unsigned int len)
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h line 196
Quoted 10 lines
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +10
-0+{ +unsigned int i; + +if (!bitmap || !bitmap->map) +return -EINVAL ; + +if (bitmap->len <= bit) +return -EINVAL ; + +/* TODO - optimize */
Leftover TODO
Quoted 48 lines
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +48
-0+for (i =0 ; i != len; i++) +rte_bitmap_set (bitmap->map, bit + i); + +return 0 ; +} + +/** +* dlb2_bitmap_clear() - clear a bitmap entry +* @bitmap: pointer to dlb2_bitmap structure. +* @bit: bit index. +* +* Return: +* Returns 0 upon success, < 0 otherwise. +* +* Errors: +* EINVAL - bitmap is NULL or is uninitialized, or bit is larger than the +* bitmap length. +*/ +static inline int dlb2_bitmap_clear (struct dlb2_bitmap *bitmap, +unsigned int bit) +{ +if (!bitmap || !bitmap->map) +return -EINVAL ; + +if (bitmap->len <= bit) +return -EINVAL ; + +rte_bitmap_clear (bitmap->map, bit); + +return 0 ; +} + +/** +* dlb2_bitmap_clear_range() - clear a range of bitmap entries +* @bitmap: pointer to dlb2_bitmap structure. +* @bit: starting bit index. +* @len: length of the range. +* +* Return: +* Returns 0 upon success, < 0 otherwise. +* +* Errors: +* EINVAL - bitmap is NULL or is uninitialized, or the range exceeds the bitmap +* length. +*/ +static inline int dlb2_bitmap_clear_range (struct dlb2_bitmap *bitmap, +unsigned int bit, +unsigned int len)
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h line 254
Quoted 10 lines
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +10
-0+{ +unsigned int i; + +if (!bitmap || !bitmap->map) +return -EINVAL ; + +if (bitmap->len <= bit) +return -EINVAL ; + +/* TODO - optimize */
Leftover TODO
Quoted 22 lines
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +22
-0+for (i =0 ; i != len; i++) +rte_bitmap_clear (bitmap->map, bit + i); + +return 0 ; +} + +/** +* dlb2_bitmap_find_set_bit_range() - find an range of set bits +* @bitmap: pointer to dlb2_bitmap structure. +* @len: length of the range. +* +* This function looks for a range of set bits of length @len. +* +* Return: +* Returns the base bit index upon success, < 0 otherwise. +* +* Errors: +* ENOENT - unable to find a length *len* range of set bits. +* EINVAL - bitmap is NULL or is uninitialized, or len is invalid. +*/ +static inline int dlb2_bitmap_find_set_bit_range (struct dlb2_bitmap *bitmap, +unsigned int len)
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h line 286
Quoted 10 lines
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +10
-0+{ +unsigned int i, j =0 ; + +if (!bitmap || !bitmap->map || len ==0 ) +return -EINVAL ; + +if (bitmap->len < len) +return -ENOENT ; + +/* TODO - optimize */
Leftover TODO
Quoted 26 lines
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +26
-0+for (i =0 ; i != bitmap->len; i++) { +if (rte_bitmap_get (bitmap->map, i)) { +if (++j == len) +return i - j +1 ; + }else { + j =0 ; + } + } + +/* No set bit range of length len? */ +return -ENOENT ; +} + +/** +* dlb2_bitmap_find_set_bit() - find an range of set bits +* @bitmap: pointer to dlb2_bitmap structure. +* +* This function looks for a single set bit. +* +* Return: +* Returns the base bit index upon success, -1 if not found, <-1 otherwise. +* +* Errors: +* EINVAL - bitmap is NULL or is uninitialized, or len is invalid. +*/ +static inline int dlb2_bitmap_find_set_bit (struct dlb2_bitmap *bitmap)
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h line 322
Quoted 10 lines
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +10
-0+{ +unsigned int i; + +if (!bitmap) +return -EINVAL ; + +if (!bitmap->map) +return -EINVAL ; + +/* TODO - optimize */
Leftover TODO
Quoted 22 lines
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +22
-0+for (i =0 ; i != bitmap->len; i++) { +if (rte_bitmap_get (bitmap->map, i)) +return i; + } + +return -ENOENT ; +} + +/** +* dlb2_bitmap_count() - returns the number of set bits +* @bitmap: pointer to dlb2_bitmap structure. +* +* This function looks for a single set bit. +* +* Return: +* Returns the number of set bits upon success, <0 otherwise. +* +* Errors: +* EINVAL - bitmap is NULL or is uninitialized. +*/ +static inline int dlb2_bitmap_count (struct dlb2_bitmap *bitmap) +{
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h line 354
Quoted 10 lines
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +10
-0+int weight =0 ; +unsigned int i; + +if (!bitmap) +return -EINVAL ; + +if (!bitmap->map) +return -EINVAL ; + +/* TODO - optimize */
Leftover TODO
Quoted 21 lines
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +21
-0+for (i =0 ; i != bitmap->len; i++) { +if (rte_bitmap_get (bitmap->map, i)) + weight++; + } +return weight; +} + +/** +* dlb2_bitmap_longest_set_range() - returns longest contiguous range of set +* bits +* @bitmap: pointer to dlb2_bitmap structure. +* +* Return: +* Returns the bitmap's longest contiguous range of of set bits upon success, +* <0 otherwise. +* +* Errors: +* EINVAL - bitmap is NULL or is uninitialized. +*/ +static inline int dlb2_bitmap_longest_set_range (struct dlb2_bitmap *bitmap) +{
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h line 385
Quoted 10 lines
drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h new file +10
-0+int max_len =0 , len =0 ; +unsigned int i; + +if (!bitmap) +return -EINVAL ; + +if (!bitmap->map) +return -EINVAL ; + +/* TODO - optimize */
Leftover TODO
[...]
Quoted 633 lines
drivers/event/dlb2/pf/dlb2_main.c new file +212
-0@@ -0,0 +1,620 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#include <stdint.h> +#include <stdbool.h> +#include <stdio.h> +#include <errno.h> +#include <assert.h> +#include <unistd.h> +#include <string.h> + +#include <rte_malloc.h> +#include <rte_errno.h> + +#include "base/dlb2_resource.h" +#include "base/dlb2_osdep.h" +#include "base/dlb2_regs.h" +#include "dlb2_main.h" +#include "../dlb2_user.h" +#include "../dlb2_priv.h" +#include "../dlb2_iface.h" +#include "../dlb2_inline_fns.h" + +#define PF_ID_ZERO 0/* PF ONLY! */ +#define NO_OWNER_VF 0/* PF ONLY! */ +#define NOT_VF_REQ false/* PF ONLY! */ + +#define DLB2_PCI_CFG_SPACE_SIZE 256 +#define DLB2_PCI_CAP_POINTER 0x34 +#define DLB2_PCI_CAP_NEXT (hdr) (((hdr) >> 8) & 0xFC) +#define DLB2_PCI_CAP_ID (hdr) ((hdr) & 0xFF) +#define DLB2_PCI_EXT_CAP_NEXT (hdr) (((hdr) >> 20) & 0xFFC) +#define DLB2_PCI_EXT_CAP_ID (hdr) ((hdr) & 0xFFFF) +#define DLB2_PCI_EXT_CAP_ID_ERR 1 +#define DLB2_PCI_ERR_UNCOR_MASK 8 +#define DLB2_PCI_ERR_UNC_UNSUP 0x00100000 + +#define DLB2_PCI_EXP_DEVCTL 8 +#define DLB2_PCI_LNKCTL 16 +#define DLB2_PCI_SLTCTL 24 +#define DLB2_PCI_RTCTL 28 +#define DLB2_PCI_EXP_DEVCTL2 40 +#define DLB2_PCI_LNKCTL2 48 +#define DLB2_PCI_SLTCTL2 56 +#define DLB2_PCI_CMD 4 +#define DLB2_PCI_X_CMD 2 +#define DLB2_PCI_EXP_DEVSTA 10 +#define DLB2_PCI_EXP_DEVSTA_TRPND 0x20 +#define DLB2_PCI_EXP_DEVCTL_BCR_FLR 0x8000 + +#define DLB2_PCI_CAP_ID_EXP 0x10 +#define DLB2_PCI_CAP_ID_MSIX 0x11 +#define DLB2_PCI_EXT_CAP_ID_PAS 0x1B +#define DLB2_PCI_EXT_CAP_ID_PRI 0x13 +#define DLB2_PCI_EXT_CAP_ID_ACS 0xD + +#define DLB2_PCI_PRI_CTRL_ENABLE 0x1 +#define DLB2_PCI_PRI_ALLOC_REQ 0xC +#define DLB2_PCI_PRI_CTRL 0x4 +#define DLB2_PCI_MSIX_FLAGS 0x2 +#define DLB2_PCI_MSIX_FLAGS_ENABLE 0x8000 +#define DLB2_PCI_MSIX_FLAGS_MASKALL 0x4000 +#define DLB2_PCI_ERR_ROOT_STATUS 0x30 +#define DLB2_PCI_ERR_COR_STATUS 0x10 +#define DLB2_PCI_ERR_UNCOR_STATUS 0x4 +#define DLB2_PCI_COMMAND_INTX_DISABLE 0x400 +#define DLB2_PCI_ACS_CAP 0x4 +#define DLB2_PCI_ACS_CTRL 0x6 +#define DLB2_PCI_ACS_SV 0x1 +#define DLB2_PCI_ACS_RR 0x4 +#define DLB2_PCI_ACS_CR 0x8 +#define DLB2_PCI_ACS_UF 0x10 +#define DLB2_PCI_ACS_EC 0x20 + +static int +dlb2_pci_find_ext_capability (struct rte_pci_device *pdev,uint32_t id) +{ +uint32_t hdr; +size_t sz; +int pos; + + pos =DLB2_PCI_CFG_SPACE_SIZE ; + sz =sizeof (hdr); + +while (pos >0xFF ) { +if (rte_pci_read_config (pdev, &hdr, sz, pos) != (int )sz) +return -1 ; + +if (DLB2_PCI_EXT_CAP_ID (hdr) == id) +return pos; + + pos =DLB2_PCI_EXT_CAP_NEXT (hdr); + } + +return -1 ; +} + +static int dlb2_pci_find_capability (struct rte_pci_device *pdev,uint32_t id) +{ +uint8_t pos; +int ret; +uint16_t hdr; + + ret =rte_pci_read_config (pdev, &pos,1 ,DLB2_PCI_CAP_POINTER ); + pos &=0xFC ; + +if (ret !=1 ) +return -1 ; + +while (pos >0x3F ) { + ret =rte_pci_read_config (pdev, &hdr,2 , pos); +if (ret !=2 ) +return -1 ; + +if (DLB2_PCI_CAP_ID (hdr) == id) +return pos; + +if (DLB2_PCI_CAP_ID (hdr) ==0xFF ) +return -1 ; + + pos =DLB2_PCI_CAP_NEXT (hdr); + } + +return -1 ; +} + +static int +dlb2_pf_init_driver_state (struct dlb2_dev *dlb2_dev) +{ +int i; + +if (rte_cpu_get_flag_enabled (RTE_CPUFLAG_MOVDIR64B )) + dlb2_dev->enqueue_four = dlb2_movdir64b; +else + dlb2_dev->enqueue_four = dlb2_movntdq; + +/* Initialize software state */ +for (i =0 ; i <DLB2_MAX_NUM_LDB_PORTS ; i++) +dlb2_list_init_head (&dlb2_dev->ldb_port_pages[i].list); + +for (i =0 ; i <DLB2_MAX_NUM_DIR_PORTS ; i++) +dlb2_list_init_head (&dlb2_dev->dir_port_pages[i].list); + +rte_spinlock_init (&dlb2_dev->resource_mutex); +rte_spinlock_init (&dlb2_dev->measurement_lock); + +return 0 ; +} + +static void dlb2_pf_enable_pm (struct dlb2_dev *dlb2_dev) +{ +dlb2_clr_pmcsr_disable (&dlb2_dev->hw); +} + +#define DLB2_READY_RETRY_LIMIT 1000 +static int dlb2_pf_wait_for_device_ready (struct dlb2_dev *dlb2_dev) +{ +u32 retries =0 ; + +/* Allow at least 1s for the device to become active after power-on */ +for (retries =0 ; retries <DLB2_READY_RETRY_LIMIT ; retries++) { +union dlb2_cfg_mstr_cfg_diagnostic_idle_status idle; +union dlb2_cfg_mstr_cfg_pm_status pm_st; +u32 addr; + + addr =DLB2_CFG_MSTR_CFG_PM_STATUS ; + pm_st.val =DLB2_CSR_RD (&dlb2_dev->hw, addr); + addr =DLB2_CFG_MSTR_CFG_DIAGNOSTIC_IDLE_STATUS ; + idle.val =DLB2_CSR_RD (&dlb2_dev->hw, addr); +if (pm_st.field.pmsm ==1 && idle.field.dlb_func_idle ==1 ) +break ; + +rte_delay_ms (1 ); + }; + +if (retries ==DLB2_READY_RETRY_LIMIT ) { +printf ("[%s()] wait for device ready timed out\n" , + __func__); +return -1 ; + } + +return 0 ; +} + +struct dlb2_dev * +dlb2_probe (struct rte_pci_device *pdev) +{ +struct dlb2_dev *dlb2_dev; +int ret =0 ; + +DLB2_INFO (dlb2_dev,"probe\n" ); + + dlb2_dev =rte_malloc ("DLB2_PF" ,sizeof (struct dlb2_dev ), +RTE_CACHE_LINE_SIZE ); + +if (!dlb2_dev) { + ret = -ENOMEM ; + goto dlb2_dev_malloc_fail; + } + +/* PCI Bus driver has already mapped bar space into process. +* Save off our IO register and FUNC addresses. +*/ + +/* BAR 0 */ +if (pdev->mem_resource[0 ].addr ==NULL ) { +DLB2_ERR (dlb2_dev,"probe: BAR 0 addr (csr_kva) is NULL\n" ); + ret = -EINVAL ; + goto pci_mmap_bad_addr; + } + dlb2_dev->hw.func_kva = (void *)(uintptr_t)pdev-Quoted 1 line
mem_resource[0].addr;
drivers/event/dlb2/pf/dlb2_main.c new file +14
-0+ dlb2_dev->hw.func_phys_addr = pdev->mem_resource[0 ].phys_addr; + +DLB2_INFO (dlb2_dev,"DLB2 FUNC VA=%p, PA=%p, len=%p\n" , + (void *)dlb2_dev->hw.func_kva, + (void *)dlb2_dev->hw.func_phys_addr, + (void *)(pdev->mem_resource[0 ].len)); + +/* BAR 2 */ +if (pdev->mem_resource[2 ].addr ==NULL ) { +DLB2_ERR (dlb2_dev,"probe: BAR 2 addr (func_kva) is NULL\n" ); + ret = -EINVAL ; + goto pci_mmap_bad_addr; + } + dlb2_dev->hw.csr_kva = (void *)(uintptr_t)pdev-Quoted 1 line
mem_resource[2].addr;
drivers/event/dlb2/pf/dlb2_main.c new file +83
-0+ dlb2_dev->hw.csr_phys_addr = pdev->mem_resource[2 ].phys_addr; + +DLB2_INFO (dlb2_dev,"DLB2 CSR VA=%p, PA=%p, len=%p\n" , + (void *)dlb2_dev->hw.csr_kva, + (void *)dlb2_dev->hw.csr_phys_addr, + (void *)(pdev->mem_resource[2 ].len)); + + dlb2_dev->pdev = pdev; + +/* PM enable must be done before any other MMIO accesses, and this +* setting is persistent across device reset. +*/ +dlb2_pf_enable_pm (dlb2_dev); + + ret =dlb2_pf_wait_for_device_ready (dlb2_dev); +if (ret) + goto wait_for_device_ready_fail; + + ret =dlb2_pf_reset (dlb2_dev); +if (ret) + goto dlb2_reset_fail; + + ret =dlb2_pf_init_driver_state (dlb2_dev); +if (ret) + goto init_driver_state_fail; + + ret =dlb2_resource_init (&dlb2_dev->hw); +if (ret) + goto resource_init_fail; + +return dlb2_dev; + +resource_init_fail: +dlb2_resource_free (&dlb2_dev->hw); +init_driver_state_fail: +dlb2_reset_fail: +pci_mmap_bad_addr: +wait_for_device_ready_fail: +rte_free (dlb2_dev); +dlb2_dev_malloc_fail: + rte_errno = ret; +return NULL ; +} + +int +dlb2_pf_reset (struct dlb2_dev *dlb2_dev) +{ +int ret =0 ; +int i =0 ; +uint32_t dword[16 ]; +uint16_t cmd; +off_t off; + +uint16_t dev_ctl_word; +uint16_t dev_ctl2_word; +uint16_t lnk_word; +uint16_t lnk_word2; +uint16_t slt_word; +uint16_t slt_word2; +uint16_t rt_ctl_word; +uint32_t pri_reqs_dword; +uint16_t pri_ctrl_word; + +int pcie_cap_offset; +int pri_cap_offset; +int msix_cap_offset; +int err_cap_offset; +int acs_cap_offset; +int wait_count; + +uint16_t devsta_busy_word; +uint16_t devctl_word; + +struct rte_pci_device *pdev = dlb2_dev->pdev; + +/* Save PCI config state */ + +for (i =0 ; i <16 ; i++) { +if (rte_pci_read_config (pdev, &dword[i],4 , i *4 ) !=4 ) +return ret; + } + + pcie_cap_offset = dlb2_pci_find_capability(pdev,
DLB2_PCI_CAP_ID_EXP);drivers/event/dlb2/pf/dlb2_main.c new file +105
-0+ +if (pcie_cap_offset <0 ) { +printf ("[%s()] failed to find the pcie capability\n" , + __func__); +return pcie_cap_offset; + } + + off = pcie_cap_offset +DLB2_PCI_EXP_DEVCTL ; +if (rte_pci_read_config (pdev, &dev_ctl_word,2 , off) !=2 ) + dev_ctl_word =0 ; + + off = pcie_cap_offset +DLB2_PCI_LNKCTL ; +if (rte_pci_read_config (pdev, &lnk_word,2 , off) !=2 ) + lnk_word =0 ; + + off = pcie_cap_offset +DLB2_PCI_SLTCTL ; +if (rte_pci_read_config (pdev, &slt_word,2 , off) !=2 ) + slt_word =0 ; + + off = pcie_cap_offset +DLB2_PCI_RTCTL ; +if (rte_pci_read_config (pdev, &rt_ctl_word,2 , off) !=2 ) + rt_ctl_word =0 ; + + off = pcie_cap_offset +DLB2_PCI_EXP_DEVCTL2 ; +if (rte_pci_read_config (pdev, &dev_ctl2_word,2 , off) !=2 ) + dev_ctl2_word =0 ; + + off = pcie_cap_offset +DLB2_PCI_LNKCTL2 ; +if (rte_pci_read_config (pdev, &lnk_word2,2 , off) !=2 ) + lnk_word2 =0 ; + + off = pcie_cap_offset +DLB2_PCI_SLTCTL2 ; +if (rte_pci_read_config (pdev, &slt_word2,2 , off) !=2 ) + slt_word2 =0 ; + + off =DLB2_PCI_EXT_CAP_ID_PRI ; + pri_cap_offset =dlb2_pci_find_ext_capability (pdev, off); + +if (pri_cap_offset >=0 ) { + off = pri_cap_offset +DLB2_PCI_PRI_ALLOC_REQ ; +if (rte_pci_read_config (pdev, &pri_reqs_dword,4 , off) !=4 ) + pri_reqs_dword =0 ; + } + +/* clear the PCI command register before issuing the FLR */ + + off =DLB2_PCI_CMD ; + cmd =0 ; +if (rte_pci_write_config (pdev, &cmd,2 , off) !=2 ) { +printf ("[%s()] failed to write the pci command\n" , + __func__); +return ret; + } + +/* issue the FLR */ +for (wait_count =0 ; wait_count <4 ; wait_count++) { +int sleep_time; + + off = pcie_cap_offset +DLB2_PCI_EXP_DEVSTA ; + ret =rte_pci_read_config (pdev, &devsta_busy_word,2 , off); +if (ret !=2 ) { +printf ("[%s()] failed to read the pci device status\n" , + __func__); +return ret; + } + +if (!(devsta_busy_word &DLB2_PCI_EXP_DEVSTA_TRPND )) +break ; + + sleep_time = (1 << (wait_count)) *100 ; +rte_delay_ms (sleep_time); + } + +if (wait_count ==4 ) { +printf ("[%s()] wait for pci pending transactions timed out\n" , + __func__); +return -1 ; + } + + off = pcie_cap_offset +DLB2_PCI_EXP_DEVCTL ; + ret =rte_pci_read_config (pdev, &devctl_word,2 , off); +if (ret !=2 ) { +printf ("[%s()] failed to read the pcie device control\n" , + __func__); +return ret; + } + + devctl_word |=DLB2_PCI_EXP_DEVCTL_BCR_FLR ; + + ret =rte_pci_write_config (pdev, &devctl_word,2 , off); +if (ret !=2 ) { +printf ("[%s()] failed to write the pcie device control\n" , + __func__); +return ret; + } + +rte_delay_ms (100 ); + +/* Restore PCI config state */ + +if (pcie_cap_offset >=0 ) { + off = pcie_cap_offset +DLB2_PCI_EXP_DEVCTL ; + ret =rte_pci_write_config (pdev, &dev_ctl_word,2 , off); +if (ret !=2 ) { + printf("[%s()] failed to write the pcie device control atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +8
-0+ __func__, (int)off); +return ret; + } + + off = pcie_cap_offset +DLB2_PCI_LNKCTL ; + ret =rte_pci_write_config (pdev, &lnk_word,2 , off); +if (ret !=2 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +8
-0+ __func__, (int)off); +return ret; + } + + off = pcie_cap_offset +DLB2_PCI_SLTCTL ; + ret =rte_pci_write_config (pdev, &slt_word,2 , off); +if (ret !=2 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +8
-0+ __func__, (int)off); +return ret; + } + + off = pcie_cap_offset +DLB2_PCI_RTCTL ; + ret =rte_pci_write_config (pdev, &rt_ctl_word,2 , off); +if (ret !=2 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +8
-0+ __func__, (int)off); +return ret; + } + + off = pcie_cap_offset +DLB2_PCI_EXP_DEVCTL2 ; + ret =rte_pci_write_config (pdev, &dev_ctl2_word,2 , off); +if (ret !=2 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +8
-0+ __func__, (int)off); +return ret; + } + + off = pcie_cap_offset +DLB2_PCI_LNKCTL2 ; + ret =rte_pci_write_config (pdev, &lnk_word2,2 , off); +if (ret !=2 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +8
-0+ __func__, (int)off); +return ret; + } + + off = pcie_cap_offset +DLB2_PCI_SLTCTL2 ; + ret =rte_pci_write_config (pdev, &slt_word2,2 , off); +if (ret !=2 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +12
-0+ __func__, (int)off); +return ret; + } + } + +if (pri_cap_offset >=0 ) { + pri_ctrl_word =DLB2_PCI_PRI_CTRL_ENABLE ; + + off = pri_cap_offset +DLB2_PCI_PRI_ALLOC_REQ ; + ret =rte_pci_write_config (pdev, &pri_reqs_dword,4 , off); +if (ret !=4 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +8
-0+ __func__, (int)off); +return ret; + } + + off = pri_cap_offset +DLB2_PCI_PRI_CTRL ; + ret =rte_pci_write_config (pdev, &pri_ctrl_word,2 , off); +if (ret !=2 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +18
-0+ __func__, (int)off); +return ret; + } + } + + off =DLB2_PCI_EXT_CAP_ID_ERR ; + err_cap_offset =dlb2_pci_find_ext_capability (pdev, off); + +if (err_cap_offset >=0 ) { +uint32_t tmp; + + off = err_cap_offset +DLB2_PCI_ERR_ROOT_STATUS ; +if (rte_pci_read_config (pdev, &tmp,4 , off) !=4 ) + tmp =0 ; + + ret =rte_pci_write_config (pdev, &tmp,4 , off); +if (ret !=4 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +11
-0+ __func__, (int)off); +return ret; + } + + off = err_cap_offset +DLB2_PCI_ERR_COR_STATUS ; +if (rte_pci_read_config (pdev, &tmp,4 , off) !=4 ) + tmp =0 ; + + ret =rte_pci_write_config (pdev, &tmp,4 , off); +if (ret !=4 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +11
-0+ __func__, (int)off); +return ret; + } + + off = err_cap_offset +DLB2_PCI_ERR_UNCOR_STATUS ; +if (rte_pci_read_config (pdev, &tmp,4 , off) !=4 ) + tmp =0 ; + + ret =rte_pci_write_config (pdev, &tmp,4 , off); +if (ret !=4 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +10
-0+ __func__, (int)off); +return ret; + } + } + +for (i =16 ; i >0 ; i--) { + off = (i -1 ) *4 ; + ret =rte_pci_write_config (pdev, &dword[i -1 ],4 , off); +if (ret !=4 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +60
-0+ __func__, (int)off); +return ret; + } + } + + off =DLB2_PCI_CMD ; +if (rte_pci_read_config (pdev, &cmd,2 , off) ==2 ) { + cmd &= ~DLB2_PCI_COMMAND_INTX_DISABLE ; +if (rte_pci_write_config (pdev, &cmd,2 , off) !=2 ) { +printf ("[%s()] failed to write the pci command\n" , + __func__); +return ret; + } + } + + msix_cap_offset =dlb2_pci_find_capability (pdev, +DLB2_PCI_CAP_ID_MSIX ); +if (msix_cap_offset >=0 ) { + off = msix_cap_offset +DLB2_PCI_MSIX_FLAGS ; +if (rte_pci_read_config (pdev, &cmd,2 , off) ==2 ) { + cmd |=DLB2_PCI_MSIX_FLAGS_ENABLE ; + cmd |=DLB2_PCI_MSIX_FLAGS_MASKALL ; +if (rte_pci_write_config (pdev, &cmd,2 , off) !=2 ) { +printf ("[%s()] failed to write msix flags\n" , + __func__); +return ret; + } + } + + off = msix_cap_offset +DLB2_PCI_MSIX_FLAGS ; +if (rte_pci_read_config (pdev, &cmd,2 , off) ==2 ) { + cmd &= ~DLB2_PCI_MSIX_FLAGS_MASKALL ; +if (rte_pci_write_config (pdev, &cmd,2 , off) !=2 ) { +printf ("[%s()] failed to write msix flags\n" , + __func__); +return ret; + } + } + } + + off =DLB2_PCI_EXT_CAP_ID_ACS ; + acs_cap_offset =dlb2_pci_find_ext_capability (pdev, off); + +if (acs_cap_offset >=0 ) { +uint16_t acs_cap, acs_ctrl, acs_mask; + off = acs_cap_offset +DLB2_PCI_ACS_CAP ; +if (rte_pci_read_config (pdev, &acs_cap,2 , off) !=2 ) + acs_cap =0 ; + + off = acs_cap_offset +DLB2_PCI_ACS_CTRL ; +if (rte_pci_read_config (pdev, &acs_ctrl,2 , off) !=2 ) + acs_ctrl =0 ; + + acs_mask =DLB2_PCI_ACS_SV |DLB2_PCI_ACS_RR ; + acs_mask |= (DLB2_PCI_ACS_CR |DLB2_PCI_ACS_UF ); + acs_ctrl |= (acs_cap & acs_mask); + + ret =rte_pci_write_config (pdev, &acs_ctrl,2 , off); +if (ret !=2 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c new file +16
-0+ __func__, (int)off); +return ret; + } + + off = acs_cap_offset +DLB2_PCI_ACS_CTRL ; +if (rte_pci_read_config (pdev, &acs_ctrl,2 , off) !=2 ) + acs_ctrl =0 ; + + acs_mask =DLB2_PCI_ACS_RR |DLB2_PCI_ACS_CR ; + acs_mask |=DLB2_PCI_ACS_EC ; + acs_ctrl &= ~acs_mask; + + off = acs_cap_offset +DLB2_PCI_ACS_CTRL ; + ret =rte_pci_write_config (pdev, &acs_ctrl,2 , off); +if (ret !=2 ) { + printf("[%s()] failed to write the pcie config space atoffset %d\n",
drivers/event/dlb2/pf/dlb2_main.c line 615
Quoted 7 lines
drivers/event/dlb2/pf/dlb2_main.c new file +7
-0+ __func__, (int)off); +return ret; + } + } + +return 0 ; +}
Since this PCIe function-level reset code is more PCIe-specific than the other functions in this file, what do you think about putting code this PCIe function-level reset code in its own file, e.g. dlb2_pci.c? Just a thought.
drivers/event/dlb2/pf/dlb2_main.c line 620
Quoted 5 lines
drivers/event/dlb2/pf/dlb2_main.c new file +5
-0+ +/**********************************/ +/****** Device configuration ******/ +/**********************************/ +
Nit: this comment block would make more sense in patch 8 when the first device configuration functions are added.
Quoted 56 lines
drivers/event/dlb2/pf/dlb2_main.h new file +36
-0@@ -0,0 +1,107 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef __DLB2_MAIN_H +#define __DLB2_MAIN_H + +#include <rte_debug.h> +#include <rte_log.h> +#include <rte_spinlock.h> +#include <rte_pci.h> +#include <rte_bus_pci.h> + +#ifndef PAGE_SIZE +#define PAGE_SIZE (sysconf(_SC_PAGESIZE)) +#endif + +#include "base/dlb2_hw_types.h" +#include "../dlb2_user.h" + +#define DLB2_DEFAULT_UNREGISTER_TIMEOUT_S 5 + +struct dlb2_dev; + +struct dlb2_port_memory { + struct dlb2_list_head list; + void *cq_base; + bool valid; +}; + +struct dlb2_dev { + struct rte_pci_device *pdev; + struct dlb2_hw hw; + /* struct list_head list; */ + struct device *dlb2_device; + struct dlb2_port_memoryldb_port_pages[DLB2_MAX_NUM_LDB_PORTS];
drivers/event/dlb2/pf/dlb2_main.h new file +1
-0+struct dlb2_port_memory dir_port_pages[DLB2_MAX_NUM_DIR_PORTS];
drivers/event/dlb2/pf/dlb2_main.h new file +1
-0+/* The enqueue_four function enqueues four HCWs (one cache-line worth)
drivers/event/dlb2/pf/dlb2_main.h new file +8
-0+* to the DLB2, using whichever mechanism is supported by the platform +* on which this driver is running. +*/ +void (*enqueue_four)(void *qe4,void *pp_addr); + +bool domain_reset_failed; +/* The resource mutex serializes access to driver data structures and +* hardware registers.
drivers/event/dlb2/pf/dlb2_main.h line 56
Quoted 10 lines
drivers/event/dlb2/pf/dlb2_main.h new file +10
-0+*/ +rte_spinlock_t resource_mutex; +rte_spinlock_t measurement_lock; +bool worker_launched; +u8 revision; +}; + +struct dlb2_dev *dlb2_probe (struct rte_pci_device *pdev); + +/* The following functions were pf_ops in kernel driver implementation */
Probably not important for the DPDK implementation...and better to avoid mentioning the kernel implementation to avoid any appearance of licensing impropriety.
Quoted 89 lines
drivers/event/dlb2/pf/dlb2_main.h new file +51
-0+int dlb2_pf_reset (struct dlb2_dev *dlb2_dev); +int dlb2_pf_create_sched_domain (struct dlb2_hw *hw, +struct dlb2_create_sched_domain_args *args, +struct dlb2_cmd_response *resp); +int dlb2_pf_create_ldb_queue (struct dlb2_hw *hw, +u32 domain_id, +struct dlb2_create_ldb_queue_args *args, +struct dlb2_cmd_response *resp); +int dlb2_pf_create_dir_queue (struct dlb2_hw *hw, +u32 domain_id, +struct dlb2_create_dir_queue_args *args, +struct dlb2_cmd_response *resp); +int dlb2_pf_create_ldb_port (struct dlb2_hw *hw, +u32 domain_id, +struct dlb2_create_ldb_port_args *args, +uintptr_t cq_dma_base, +struct dlb2_cmd_response *resp); +int dlb2_pf_create_dir_port (struct dlb2_hw *hw, +u32 domain_id, +struct dlb2_create_dir_port_args *args, +uintptr_t cq_dma_base, +struct dlb2_cmd_response *resp); +int dlb2_pf_start_domain (struct dlb2_hw *hw, +u32 domain_id, +struct dlb2_start_domain_args *args, +struct dlb2_cmd_response *resp); +int dlb2_pf_enable_ldb_port (struct dlb2_hw *hw, +u32 domain_id, +struct dlb2_enable_ldb_port_args *args, +struct dlb2_cmd_response *resp); +int dlb2_pf_disable_ldb_port (struct dlb2_hw *hw, +u32 domain_id, +struct dlb2_disable_ldb_port_args *args, +struct dlb2_cmd_response *resp); +int dlb2_pf_enable_dir_port (struct dlb2_hw *hw, +u32 domain_id, +struct dlb2_enable_dir_port_args *args, +struct dlb2_cmd_response *resp); +int dlb2_pf_disable_dir_port (struct dlb2_hw *hw, +u32 domain_id, +struct dlb2_disable_dir_port_args *args, +struct dlb2_cmd_response *resp); +int dlb2_pf_reset_domain (struct dlb2_hw *hw,u32 domain_id); +int dlb2_pf_ldb_port_owned_by_domain (struct dlb2_hw *hw, +u32 domain_id, +u32 port_id); +int dlb2_pf_dir_port_owned_by_domain (struct dlb2_hw *hw, +u32 domain_id, +u32 port_id); + +#endif /* __DLB2_MAIN_H */ drivers/event/dlb2/pf/dlb2_pf.c new file +32
-0@@ -0,0 +1,251 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#include <stdint.h> +#include <stdbool.h> +#include <stdio.h> +#include <sys/mman.h> +#include <sys/fcntl.h> +#include <sys/time.h> +#include <errno.h> +#include <assert.h> +#include <unistd.h> +#include <string.h> +#include <rte_debug.h> +#include <rte_log.h> +#include <rte_dev.h> +#include <rte_devargs.h> +#include <rte_mbuf.h> +#include <rte_ring.h> +#include <rte_errno.h> +#include <rte_kvargs.h> +#include <rte_malloc.h> +#include <rte_cycles.h> +#include <rte_io.h> +#include <rte_pci.h> +#include <rte_bus_pci.h> +#include <rte_eventdev.h> +#include <rte_eventdev_pmd.h> +#include <rte_eventdev_pmd_pci.h> +#include <rte_memory.h> +#include <rte_string_fns.h>
drivers/event/dlb2/pf/dlb2_pf.c line 42
Quoted 10 lines
drivers/event/dlb2/pf/dlb2_pf.c new file +10
-0+ +#include "../dlb2_priv.h" +#include "../dlb2_iface.h" +#include "../dlb2_inline_fns.h" +#include "dlb2_main.h" +#include "base/dlb2_hw_types.h" +#include "base/dlb2_osdep.h" +#include "base/dlb2_resource.h" + +extern struct dlb2_dev *dlb2_probe (struct rte_pci_device *pdev);
Not needed, dlb2_probe is declared in pf/dlb2_main.h
drivers/event/dlb2/pf/dlb2_pf.c line 47
Quoted 5 lines
drivers/event/dlb2/pf/dlb2_pf.c new file +5
-0+ +#if !definedRTE_ARCH_X86_64 +#error "This implementation only supports RTE_ARCH_X86_64 architecture." +#endif +
Seems redundant, this is checked in dlb2.c and meson.build should ensure it's not built for other archs anyway.
[...]
drivers/event/dlb2/pf/dlb2_pf.c line 120
Quoted 4 lines
drivers/event/dlb2/pf/dlb2_pf.c new file +4
-0+static void +dlb2_pf_iface_fn_ptrs_init (void ) +{ +
Nit: extra whitespace
Thanks,
Gage
Re: [dpdk-dev] [PATCH 07/22] event/dlb2: add xstats
In reply to Timothy McDanielQuoted 9 lines
drivers/event/dlb2/dlb2_xstats.c new file +3
-0@@ -0,0 +1,1269 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/
drivers/event/dlb2/dlb2_xstats.c line 13
Quoted 10 lines
drivers/event/dlb2/dlb2_xstats.c new file +10
-0+ +#include <stdint.h> +#include <stdbool.h> +#include <stdio.h> +#include <sys/mman.h> +#include <sys/fcntl.h> +#include <errno.h> +#include <assert.h> +#include <unistd.h> +#include <string.h>
+inttypes.h for the printf format specifiers (PRIu64, etc.)
Quoted 4 lines
drivers/event/dlb2/dlb2_xstats.c new file +4
-0+ +#include <rte_debug.h> +#include <rte_log.h> +#include <rte_dev.h>
drivers/event/dlb2/dlb2_xstats.c line 27
Quoted 10 lines
drivers/event/dlb2/dlb2_xstats.c new file +10
-0+#include <rte_mbuf.h> +#include <rte_ring.h> +#include <rte_errno.h> +#include <rte_kvargs.h> +#include <rte_malloc.h> +#include <rte_cycles.h> +#include <rte_io.h> +#include <rte_eventdev.h> +#include <rte_eventdev_pmd.h> +#include <rte_eventdev_pmd_vdev.h>
Besides rte_malloc.h and rte_eventdev.h, I suspect the rest are unnecessary.
[...]
drivers/event/dlb2/dlb2_xstats.c line 674
Quoted 13 lines
drivers/event/dlb2/dlb2_xstats.c new file +3
-0+int +dlb2_eventdev_xstats_get_names (const struct rte_eventdev *dev, + enum rte_event_dev_xstats_mode mode, uint8_t
queue_port_id,drivers/event/dlb2/dlb2_xstats.c new file +9
-0+struct rte_event_dev_xstats_name *xstats_names, + unsigned int *ids, unsigned int size) +{ + const struct dlb2_eventdev *dlb2 = dlb2_pmd_priv(dev); +unsigned int i; +unsigned int xidx =0 ; + +RTE_SET_USED (mode); +RTE_SET_USED (queue_port_id);
Not needed, these variables are used below.
Quoted 13 lines
drivers/event/dlb2/dlb2_xstats.c new file +12
-0+ +uint32_t xstats_mode_count =0 ; +uint32_t start_offset =0 ; + +switch (mode) { +case RTE_EVENT_DEV_XSTATS_DEVICE : + xstats_mode_count = dlb2->xstats_count_mode_dev; +break ; +case RTE_EVENT_DEV_XSTATS_PORT : +if (queue_port_id >=DLB2_MAX_NUM_PORTS ) +break ; + xstats_mode_count = dlb2-Quoted 1 line
xstats_count_per_port[queue_port_id];
drivers/event/dlb2/dlb2_xstats.c line 690
Quoted 4 lines
drivers/event/dlb2/dlb2_xstats.c new file +4
-0+ start_offset = dlb2->xstats_offset_for_port[queue_port_id]; +break ; +case RTE_EVENT_DEV_XSTATS_QUEUE : +#if (DLB2_MAX_NUM_QUEUES <=255 )/* max 8 bit value */
Looks like this macro is tied to the h/w definition, is it subject to grow?
Quoted 32 lines
drivers/event/dlb2/dlb2_xstats.c new file +4
-0+if (queue_port_id >=DLB2_MAX_NUM_QUEUES ) +break ; +#endif + xstats_mode_count = dlb2-Quoted 1 line
xstats_count_per_qid[queue_port_id];
drivers/event/dlb2/dlb2_xstats.c new file +27
-0+ start_offset = dlb2->xstats_offset_for_qid[queue_port_id]; +break ; +default : +return -EINVAL ; + }; + +if (xstats_mode_count > size || !ids || !xstats_names) +return xstats_mode_count; + +for (i =0 ; i < dlb2->xstats_count && xidx < size; i++) { +if (dlb2->xstats[i].mode != mode) +continue ; + +if (mode !=RTE_EVENT_DEV_XSTATS_DEVICE && + queue_port_id != dlb2->xstats[i].obj_idx) +continue ; + + xstats_names[xidx] = dlb2->xstats[i].name; +if (ids) + ids[xidx] = start_offset + xidx; + xidx++; + } +return xidx; +} + +static int +dlb2_xstats_update(struct dlb2_eventdev *dlb2,
drivers/event/dlb2/dlb2_xstats.c line 731
Quoted 10 lines
drivers/event/dlb2/dlb2_xstats.c new file +10
-0+enum rte_event_dev_xstats_mode mode, + uint8_t queue_port_id, const unsigned int ids[], + uint64_t values[], unsigned int n, const uint32_t reset) +{ + unsigned int i; +unsigned int xidx =0 ; + +RTE_SET_USED (mode); +RTE_SET_USED (queue_port_id); +
Not needed, these variables are used below.
Quoted 11 lines
drivers/event/dlb2/dlb2_xstats.c new file +10
-0+uint32_t xstats_mode_count =0 ; + +switch (mode) { +case RTE_EVENT_DEV_XSTATS_DEVICE : + xstats_mode_count = dlb2->xstats_count_mode_dev; +break ; +case RTE_EVENT_DEV_XSTATS_PORT : +if (queue_port_id >=DLB2_MAX_NUM_PORTS ) + goto invalid_value; + xstats_mode_count = dlb2-Quoted 1 line
xstats_count_per_port[queue_port_id];
drivers/event/dlb2/dlb2_xstats.c line 690
Quoted 3 lines
drivers/event/dlb2/dlb2_xstats.c new file +3
-0+break ; +case RTE_EVENT_DEV_XSTATS_QUEUE : +#if (DLB2_MAX_NUM_QUEUES <=255 )/* max 8 bit value */
(Same comment as above)
[...]
Quoted 11 lines
drivers/event/dlb2/dlb2_xstats.c new file +11
-0+int +dlb2_eventdev_xstats_reset (struct rte_eventdev *dev, +enum rte_event_dev_xstats_mode mode, +int16_t queue_port_id, +const uint32_t ids[], +uint32_t nb_ids) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +uint32_t i; + +/* handle -1 for queue_port_id here, looping over all ports/queues */
drivers/event/dlb2/dlb2_xstats.c line 968
Quoted 10 lines
drivers/event/dlb2/dlb2_xstats.c new file +10
-0+switch (mode) { +case RTE_EVENT_DEV_XSTATS_DEVICE : +if (dlb2_xstats_reset_dev (dlb2, ids, nb_ids)) +return -EINVAL ; +break ; +case RTE_EVENT_DEV_XSTATS_PORT : +if (queue_port_id ==-1 ) { +for (i =0 ; i <DLB2_MAX_NUM_PORTS ; i++) { +if (dlb2_xstats_reset_port (dlb2, i, + ids, nb_ids))
Nit: argument alignment (occurs below as well)
Quoted 15 lines
drivers/event/dlb2/dlb2_xstats.c new file +15
-0+return -EINVAL ; + } + }else if (queue_port_id <DLB2_MAX_NUM_PORTS ) { +if (dlb2_xstats_reset_port (dlb2, queue_port_id, + ids, nb_ids)) +return -EINVAL ; + } +break ; +case RTE_EVENT_DEV_XSTATS_QUEUE : +if (queue_port_id ==-1 ) { +for (i =0 ; i <DLB2_MAX_NUM_QUEUES ; i++) { +if (dlb2_xstats_reset_queue (dlb2, i, + ids, nb_ids)) +return -EINVAL ; + }
drivers/event/dlb2/dlb2_xstats.c line 993
Quoted 10 lines
drivers/event/dlb2/dlb2_xstats.c new file +10
-0+ }else if (queue_port_id <DLB2_MAX_NUM_QUEUES ) { +if (dlb2_xstats_reset_queue (dlb2, queue_port_id, + ids, nb_ids)) +return -EINVAL ; + } +break ; + }; + +return 0 ; +}
Thanks,
Gage
Re: [dpdk-dev] [PATCH 09/22] event/dlb2: add queue and port default conf
In reply to Timothy McDanielQuoted 26 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 09/22] event/dlb2: add queue and port default confAdd support for getting the queue and port default configuration.
Signed-off-by Timothy McDaniel <[email protected]>--- drivers/event/dlb2/dlb2.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)drivers/event/dlb2/dlb2.c +0
-0@@ -698,12 +698,45 @@ dlb2_eventdev_configure(const struct rte_eventdev *dev) } static void
drivers/event/dlb2/dlb2.c dlb2_eventdev_configure(const struct rte_eventdev · line 711
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+dlb2_eventdev_port_default_conf_get(struct rte_eventdev *dev, + uint8_t port_id, + struct rte_event_port_conf *port_conf) +{ +RTE_SET_USED (port_id); +struct dlb2_eventdev *dlb2; + + dlb2 =dlb2_pmd_priv (dev); + +/* FIXME: Arbitrary values */
Leftover FIXME. Hopefully not too arbitrary :p.
Quoted 4 lines
drivers/event/dlb2/dlb2.c +4
-0+ port_conf->new_event_threshold = dlb2->new_event_limit; + port_conf->dequeue_depth =32 ; + port_conf->enqueue_depth =DLB2_MAX_ENQUEUE_DEPTH ; + port_conf->event_port_cfg =0 ;
drivers/event/dlb2/dlb2.c dlb2_eventdev_configure(const struct rte_eventdev *dev) · line 724
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+} + +static void +dlb2_eventdev_queue_default_conf_get (struct rte_eventdev *dev, +uint8_t queue_id, +struct rte_event_queue_conf *queue_conf) +{ +RTE_SET_USED (dev); +RTE_SET_USED (queue_id); +/* FIXME: Arbitrary values */
Ditto
drivers/event/dlb2/dlb2.c dlb2_eventdev_configure(const struct rte_eventdev *dev) · line 729
Quoted 5 lines
drivers/event/dlb2/dlb2.c +5
-0+ queue_conf->nb_atomic_flows =1024 ; + queue_conf->nb_atomic_order_sequences =64 ; + queue_conf->event_queue_cfg =0 ; + queue_conf->priority =0 ; +}
Thanks,
Gage
Re: [dpdk-dev] [PATCH 08/22] event/dlb2: add infos get and configure
In reply to Timothy McDanielQuoted 11 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 08/22] event/dlb2: add infos get and configureAdd support for configuring the DLB hardware.
This is a large patch, even if most of the code is in the low-level/hardware-specific base directory, and could use more explanation in the commit message.
Quoted 30 lines
Signed-off-by Timothy McDaniel <[email protected]>--- drivers/event/dlb2/dlb2.c | 334 +++ drivers/event/dlb2/dlb2_iface.c | 7 +- drivers/event/dlb2/dlb2_iface.h | 5 + drivers/event/dlb2/pf/base/dlb2_resource.c | 3234 ++++++++++++++++++++++++++++ drivers/event/dlb2/pf/dlb2_main.c | 14 + drivers/event/dlb2/pf/dlb2_pf.c | 44 + 6 files changed, 3637 insertions(+), 1 deletion(-)drivers/event/dlb2/dlb2.c +10
-0@@ -92,6 +92,28 @@ dlb2_get_queue_depth(struct dlb2_eventdev *dlb2,return 0 ; } +static void +dlb2_free_qe_mem (struct dlb2_port *qm_port) +{ +if (qm_port ==NULL ) +return ; + +if (qm_port->qe4) { +rte_free (qm_port->qe4); + qm_port->qe4 =NULL ; + }
drivers/event/dlb2/dlb2.c dlb2_get_queue_depth(struct dlb2_eventdev *dlb2, · line 114
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+ +if (qm_port->int_arm_qe) { +rte_free (qm_port->int_arm_qe); + qm_port->int_arm_qe =NULL ; + } + +if (qm_port->consume_qe) { +rte_free (qm_port->consume_qe); + qm_port->consume_qe =NULL ; + }
You can pass a NULL pointer to rte_free() -- it's guaranteed to do nothing in that case.
Quoted 141 lines
drivers/event/dlb2/dlb2.c +21
-0+} +/* override defaults with value(s) provided on command line */ static void dlb2_init_queue_depth_thresholds(struct dlb2_eventdev *dlb2,@@ -366,10 +388,322 @@ set_qid_depth_thresh(const char *key __rte_unused, }static void +dlb2_eventdev_info_get (struct rte_eventdev *dev, +struct rte_event_dev_info *dev_info) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +int ret; + + ret =dlb2_hw_query_resources (dlb2); +if (ret) { +const struct rte_eventdev_data *data = dev->data; + +DLB2_LOG_ERR ("get resources err=%d, devid=%d\n" , + ret, data->dev_id); +/* fn is void, so fall through and return values set up in +* probe +*/ + } + +/* Add num resources currently owned by this domain. +* These would become available if the scheduling domain were reset due
drivers/event/dlb2/dlb2.c +5
-0+* to the application recalling eventdev_configure to *reconfigure* the +* domain. +*/ + evdev_dlb2_default_info.max_event_ports += dlb2->num_ldb_ports; + evdev_dlb2_default_info.max_event_queues += dlb2-Quoted 1 line
num_ldb_queues;drivers/event/dlb2/dlb2.c +86
-0+ evdev_dlb2_default_info.max_num_events += dlb2->max_ldb_credits; + + evdev_dlb2_default_info.max_event_queues = +RTE_MIN (evdev_dlb2_default_info.max_event_queues, +RTE_EVENT_MAX_QUEUES_PER_DEV ); + + evdev_dlb2_default_info.max_num_events = +RTE_MIN (evdev_dlb2_default_info.max_num_events, + dlb2->max_num_events_override); + + *dev_info = evdev_dlb2_default_info; +} + +static int +dlb2_hw_create_sched_domain (struct dlb2_hw_dev *handle, +const struct dlb2_hw_rsrcs *resources_asked) +{ +int ret =0 ; +struct dlb2_create_sched_domain_args *config_params; + +if (resources_asked ==NULL ) { +DLB2_LOG_ERR ("dlb2: dlb2_create NULL parameter\n" ); + ret =EINVAL ; + goto error_exit; + } + +/* Map generic qm resources to dlb2 resources */ + config_params = &handle->cfg.resources; + +/* DIR ports and queues */ + + config_params->num_dir_ports = + resources_asked->num_dir_ports; + + config_params->num_dir_credits = + resources_asked->num_dir_credits; + +/* LDB queues */ + + config_params->num_ldb_queues = + resources_asked->num_ldb_queues; + +/* LDB ports */ + + config_params->cos_strict =0 ;/* Best effort */ + config_params->num_cos_ldb_ports[0 ] =0 ; + config_params->num_cos_ldb_ports[1 ] =0 ; + config_params->num_cos_ldb_ports[2 ] =0 ; + config_params->num_cos_ldb_ports[3 ] =0 ; + +switch (handle->cos_id) { +case DLB2_COS_0 : + config_params->num_ldb_ports =0 ;/* no don't care ports */ + config_params->num_cos_ldb_ports[0 ] = + resources_asked->num_ldb_ports; +break ; +case DLB2_COS_1 : + config_params->num_ldb_ports =0 ;/* no don't care ports */ + config_params->num_cos_ldb_ports[1 ] = + resources_asked->num_ldb_ports; +break ; +case DLB2_COS_2 : + config_params->num_ldb_ports =0 ;/* no don't care ports */ + config_params->num_cos_ldb_ports[2 ] = + resources_asked->num_ldb_ports; +break ; +case DLB2_COS_3 : + config_params->num_ldb_ports =0 ;/* no don't care ports */ + config_params->num_cos_ldb_ports[3 ] = + resources_asked->num_ldb_ports; +break ; +case DLB2_COS_DEFAULT : +/* all ldb ports are don't care ports from a cos perspective */ + config_params->num_ldb_ports = + resources_asked->num_ldb_ports; +break ; + } + + config_params->num_ldb_credits = + resources_asked->num_ldb_credits; + + config_params->num_atomic_inflights = +DLB2_NUM_ATOMIC_INFLIGHTS_PER_QUEUE * + config_params->num_ldb_queues; + + config_params->num_hist_list_entries = resources_asked-Quoted 1 line
num_ldb_ports*drivers/event/dlb2/dlb2.c +3
-0+DLB2_NUM_HIST_LIST_ENTRIES_PER_LDB_PORT ; + + DLB2_LOG_DBG("sched domain create - ldb_qs=%d, ldb_ports=%d,dir_ports=%d, atomic_inflights=%d, hist_list_entries=%d, ldb_credits=%d, dir_credits=%d\n",
drivers/event/dlb2/dlb2.c +13
-0+ config_params->num_ldb_queues, + resources_asked->num_ldb_ports, + config_params->num_dir_ports, + config_params->num_atomic_inflights, + config_params->num_hist_list_entries, + config_params->num_ldb_credits, + config_params->num_dir_credits); + +/* Configure the QM */ + + ret =dlb2_iface_sched_domain_create (handle, config_params); +if (ret <0 ) { + DLB2_LOG_ERR("dlb2: domain create failed, device_id = %d,(driver ret = %d, extra status: %s)\n",
drivers/event/dlb2/dlb2.c set_qid_depth_thresh(const char *key __rte_unused, · line 520
Quoted 4 lines
drivers/event/dlb2/dlb2.c +4
-0+ handle->device_id, + ret, + dlb2_error_strings + [config_params->response.status]);
Perhaps rename/shorten config_params so this array+index can fit on one line, or assign config_params->response.status to a local variable?
Quoted 106 lines
drivers/event/dlb2/dlb2.c +83
-0+ + goto error_exit; + } + + handle->domain_id = config_params->response.id; + handle->domain_id_valid =1 ; + handle->cfg.configured = true; + +error_exit: + +return ret; +} + +static void +dlb2_hw_reset_sched_domain (const struct rte_eventdev *dev,bool reconfig) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +enum dlb2_configuration_state config_state; +int i, j; + +dlb2_iface_domain_reset (dlb2); + +/* Free all dynamically allocated port memory */ +for (i =0 ; i < dlb2->num_ports; i++) +dlb2_free_qe_mem (&dlb2->ev_ports[i].qm_port); + +/* If reconfiguring, mark the device's queues and ports as "previously +* configured." If the user doesn't reconfigure them, the PMD will +* reapply their previous configuration when the device is started. +*/ + config_state = (reconfig) ?DLB2_PREV_CONFIGURED : +DLB2_NOT_CONFIGURED ; + +for (i =0 ; i < dlb2->num_ports; i++) { + dlb2->ev_ports[i].qm_port.config_state = config_state; +/* Reset setup_done so ports can be reconfigured */ + dlb2->ev_ports[i].setup_done = false; +for (j =0 ; j <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; j++) + dlb2->ev_ports[i].link[j].mapped = false; + } + +for (i =0 ; i < dlb2->num_queues; i++) + dlb2->ev_queues[i].qm_queue.config_state = config_state; + +for (i =0 ; i <DLB2_MAX_NUM_QUEUES ; i++) + dlb2->ev_queues[i].setup_done = false; + + dlb2->num_ports =0 ; + dlb2->num_ldb_ports =0 ; + dlb2->num_dir_ports =0 ; + dlb2->num_queues =0 ; + dlb2->num_ldb_queues =0 ; + dlb2->num_dir_queues =0 ; + dlb2->configured = false; +} + +/* Note: 1 QM instance per QM device, QM instance/device == event device */ +static int +dlb2_eventdev_configure (const struct rte_eventdev *dev) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_hw_rsrcs *rsrcs = &handle->info.hw_rsrc_max; +const struct rte_eventdev_data *data = dev->data; +const struct rte_event_dev_config *config = &data->dev_conf; +int ret; + +/* If this eventdev is already configured, we must release the current +* scheduling domain before attempting to configure a new one. +*/ +if (dlb2->configured) { +dlb2_hw_reset_sched_domain (dev, true); + + ret =dlb2_hw_query_resources (dlb2); +if (ret) { +DLB2_LOG_ERR ("get resources err=%d, devid=%d\n" , + ret, data->dev_id); +return ret; + } + } + +if (config->nb_event_queues > rsrcs->num_queues) { + DLB2_LOG_ERR("nb_event_queues parameter (%d) exceeds theQM device's capabilities (%d).\n",
drivers/event/dlb2/dlb2.c +7
-0+ config->nb_event_queues, + rsrcs->num_queues); +return -EINVAL ; + } +if (config->nb_event_ports > (rsrcs->num_ldb_ports + + rsrcs->num_dir_ports)) { + DLB2_LOG_ERR("nb_event_ports parameter (%d) exceeds theQM device's capabilities (%d).\n",
drivers/event/dlb2/dlb2.c +6
-0+ config->nb_event_ports, + (rsrcs->num_ldb_ports + rsrcs->num_dir_ports)); +return -EINVAL ; + } +if (config->nb_events_limit > rsrcs->nb_events_limit) { + DLB2_LOG_ERR("nb_events_limit parameter (%d) exceeds theQM device's capabilities (%d).\n",
drivers/event/dlb2/dlb2.c +6
-0+ config->nb_events_limit, + rsrcs->nb_events_limit); +return -EINVAL ; + } + + if (config->event_dev_cfg &
RTE_EVENT_DEV_CFG_PER_DEQUEUE_TIMEOUT) {
drivers/event/dlb2/dlb2.c set_qid_depth_thresh(const char *key __rte_unused, · line 623
Quoted 1 line
drivers/event/dlb2/dlb2.c +1
-0+ dlb2->global_dequeue_wait = false;
Nit -- coding style discourages braces on single-statement conditionals, even if It's followed by a multi-statement else block.
https://doc.dpdk.org/guides/contributing/coding_style.html#control-statements-and-loops
drivers/event/dlb2/dlb2.c set_qid_depth_thresh(const char *key __rte_unused, · line 632
Quoted 9 lines
drivers/event/dlb2/dlb2.c +9
-0+ }else { +uint32_t timeout32; + + dlb2->global_dequeue_wait = true; + +/* Craziness here is due to size mismatch in eventdev lib. +* TODO: Submit patch so dequeue API and config use same bit +* width timeout value and same units or time, instead of one +* being 32b ns and the other being 64b ticks.
Leftover TODO
Quoted 16 lines
drivers/event/dlb2/dlb2.c +13
-0+*/ + + timeout32 = config->dequeue_timeout_ns; + + dlb2->global_dequeue_wait_ticks = + timeout32 * (rte_get_timer_hz () /1E9 ); + } + +/* Does this platform support umonitor/umwait? */ +if (rte_cpu_get_flag_enabled (RTE_CPUFLAG_UMWAIT )) { +if (RTE_LIBRTE_PMD_DLB2_UMWAIT_CTL_STATE !=0 && +RTE_LIBRTE_PMD_DLB2_UMWAIT_CTL_STATE !=1 ) { + DLB2_LOG_ERR("invalid value (%d) for
RTE_LIBRTE_PMD_DLB2_UMWAIT_CTL_STATE, must be 0 or 1.\n", +RTE_LIBRTE_PMD_DLB2_UMWAIT_CTL_STATE);
drivers/event/dlb2/dlb2.c set_qid_depth_thresh(const char *key __rte_unused, · line 654
Quoted 8 lines
drivers/event/dlb2/dlb2.c +8
-0+return -EINVAL ; + } + dlb2->umwait_allowed = true; + } + +/* FIXME: DLB should revert to load-balanced ports if dir are not +* available +*/
Leftover FIXME
Quoted 20 lines
drivers/event/dlb2/dlb2.c +3
-0+ + rsrcs->num_dir_ports = config->nb_single_link_event_port_queues; + rsrcs->num_ldb_ports = config->nb_event_ports - rsrcs-Quoted 1 line
num_dir_ports;drivers/event/dlb2/dlb2.c +2
-0+/* 1 dir queue per dir port */ + rsrcs->num_ldb_queues = config->nb_event_queues - rsrcs-Quoted 1 line
num_dir_ports;drivers/event/dlb2/dlb2.c +13
-0+ +/* Scale down nb_events_limit by 4 for directed credits, since there +* are 4x as many load-balanced credits. +*/ + rsrcs->num_ldb_credits =0 ; + rsrcs->num_dir_credits =0 ; + +if (rsrcs->num_ldb_queues) + rsrcs->num_ldb_credits = config->nb_events_limit; +if (rsrcs->num_dir_ports) + rsrcs->num_dir_credits = config->nb_events_limit /4 ; +if (dlb2->num_dir_credits_override !=-1 ) + rsrcs->num_dir_credits = dlb2->num_dir_credits_override;
drivers/event/dlb2/dlb2.c set_qid_depth_thresh(const char *key __rte_unused, · line 682
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+ +if (dlb2_hw_create_sched_domain (handle, rsrcs) <0 ) { +DLB2_LOG_ERR ("dlb2_hw_create_sched_domain failed\n" ); +return -ENODEV ; + } + + dlb2->new_event_limit = config->nb_events_limit; +__atomic_store_n (&dlb2->inflights,0 , __ATOMIC_SEQ_CST); + +
Nit: extra whitespace
Quoted 6 lines
drivers/event/dlb2/dlb2.c +6
-0+/* Save number of ports/queues for this event dev */ + dlb2->num_ports = config->nb_event_ports; + dlb2->num_queues = config->nb_event_queues; + dlb2->num_dir_ports = rsrcs->num_dir_ports; + dlb2->num_ldb_ports = dlb2->num_ports - dlb2->num_dir_ports; + dlb2->num_ldb_queues = dlb2->num_queues - dlb2->num_dir_ports;
drivers/event/dlb2/dlb2.c set_qid_depth_thresh(const char *key __rte_unused, · line 698
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+ dlb2->num_dir_queues = dlb2->num_dir_ports; + dlb2->ldb_credit_pool = rsrcs->num_ldb_credits; + dlb2->max_ldb_credits = rsrcs->num_ldb_credits; + dlb2->dir_credit_pool = rsrcs->num_dir_credits; + dlb2->max_dir_credits = rsrcs->num_dir_credits; + + dlb2->configured = true; + +return 0 ; +}
Thanks,
Gage
Re: [dpdk-dev] [PATCH 10/22] event/dlb2: add queue setup
In reply to Timothy McDanielQuoted 4 lines
drivers/event/dlb2/dlb2.c +4
-0+static int +dlb2_eventdev_ldb_queue_setup (struct rte_eventdev *dev, +struct dlb2_eventdev_queue *ev_queue, +const struct rte_event_queue_conf *queue_conf)
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_default_conf_get(struct rte_eventdev *dev, · line 947
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +int32_t qm_qid; + +if (queue_conf->nb_atomic_order_sequences) +dlb2_program_sn_allocation (dlb2, queue_conf); + + qm_qid =dlb2_hw_create_ldb_queue (dlb2, + ev_queue, + queue_conf);
Nit: this can fit on one line
Thanks,
Gage
Re: [dpdk-dev] [PATCH 11/22] event/dlb2: add port setup
In reply to Timothy McDanielQuoted 42 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 11/22] event/dlb2: add port setupConfigure the load balanded (ldb) or directed (dir) port. The consumer queue (CQ) and producer port (PP) are also set up here.
Signed-off-by Timothy McDaniel <[email protected]>--- drivers/event/dlb2/dlb2.c | 527 +++++++++++++++++ drivers/event/dlb2/dlb2_iface.c | 9 + drivers/event/dlb2/dlb2_iface.h | 8 + drivers/event/dlb2/pf/base/dlb2_resource.c | 921 +++++++++++++++++++++++++++++ drivers/event/dlb2/pf/dlb2_main.c | 28 + drivers/event/dlb2/pf/dlb2_pf.c | 179 ++++++ 6 files changed, 1672 insertions(+)drivers/event/dlb2/dlb2.c +8
-0@@ -1043,6 +1043,532 @@ dlb2_eventdev_queue_setup(struct rte_eventdev *dev, return ret; } +static int +dlb2_init_consume_qe (struct dlb2_port *qm_port,char *mz_name) +{ +struct dlb2_cq_pop_qe *qe; + + qe = rte_malloc(mz_name, + DLB2_NUM_QES_PER_CACHE_LINE * + sizeof(struct dlb2_cq_pop_qe),
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_setup(struct rte_eventdev · line 1064
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+ RTE_CACHE_LINE_SIZE); + +if (qe ==NULL ) { +DLB2_LOG_ERR ("dlb2: no memory for consume_qe\n" ); +return -ENOMEM ; + } + qm_port->consume_qe = qe; + +memset (qe,0 ,DLB2_NUM_QES_PER_CACHE_LINE * +sizeof (struct dlb2_cq_pop_qe ));
You can use rte_zmalloc() instead (applies to other init_*_qe functions below too), and no need to zero-init the fields again after the memset.
Quoted 61 lines
drivers/event/dlb2/dlb2.c +60
-0+ + qe->qe_valid =0 ; + qe->qe_frag =0 ; + qe->qe_comp =0 ; + qe->cq_token =1 ; +/* Tokens value is 0-based; i.e. '0' returns 1 token, '1' returns 2, +* and so on. +*/ + qe->tokens =0 ;/* set at run time */ + qe->meas_lat =0 ; + qe->no_dec =0 ; +/* Completion IDs are disabled */ + qe->cmp_id =0 ; + +return 0 ; +} + +static int +dlb2_init_int_arm_qe (struct dlb2_port *qm_port,char *mz_name) +{ +struct dlb2_enqueue_qe *qe; + + qe =rte_malloc (mz_name, +DLB2_NUM_QES_PER_CACHE_LINE * +sizeof (struct dlb2_enqueue_qe ), +RTE_CACHE_LINE_SIZE ); + +if (qe ==NULL ) { +DLB2_LOG_ERR ("dlb2: no memory for complete_qe\n" ); +return -ENOMEM ; + } + qm_port->int_arm_qe = qe; + +memset (qe,0 ,DLB2_NUM_QES_PER_CACHE_LINE * +sizeof (struct dlb2_enqueue_qe )); + +/* V2 - INT ARM is CQ_TOKEN + FRAG */ + qe->qe_valid =0 ; + qe->qe_frag =1 ; + qe->qe_comp =0 ; + qe->cq_token =1 ; + qe->meas_lat =0 ; + qe->no_dec =0 ; +/* Completion IDs are disabled */ + qe->cmp_id =0 ; + +return 0 ; +} + +static int +dlb2_init_qe_mem (struct dlb2_port *qm_port,char *mz_name) +{ +int ret, sz; + + sz =DLB2_NUM_QES_PER_CACHE_LINE *sizeof (struct dlb2_enqueue_qe ); + + qm_port->qe4 =rte_malloc (mz_name, sz,RTE_CACHE_LINE_SIZE ); + +if (qm_port->qe4 ==NULL ) { +DLB2_LOG_ERR ("dlb2: no qe4 memory\n" );
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_setup(struct rte_eventdev *dev, · line 1133
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+ ret = -ENOMEM ; + goto error_exit; + } + +memset (qm_port->qe4,0 , sz); + + ret =dlb2_init_int_arm_qe (qm_port, mz_name); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: dlb2_init_int_arm_qe ret=%d\n" , + ret);
This can fit on one line
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_setup(struct rte_eventdev *dev, · line 1140
Quoted 7 lines
drivers/event/dlb2/dlb2.c +7
-0+ goto error_exit; + } + + ret =dlb2_init_consume_qe (qm_port, mz_name); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: dlb2_init_consume_qe ret=%d\n" , + ret);
This can fit on one line
Quoted 42 lines
drivers/event/dlb2/dlb2.c +34
-0+ goto error_exit; + } + +return 0 ; + +error_exit: + +dlb2_free_qe_mem (qm_port); + +return ret; +} + +static int +dlb2_hw_create_ldb_port (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port, +uint32_t dequeue_depth, +uint32_t enqueue_depth) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_create_ldb_port_args cfg = {0 }; +int ret; +struct dlb2_port *qm_port =NULL ; +char mz_name[RTE_MEMZONE_NAMESIZE ]; +uint32_t qm_port_id; +uint16_t ldb_credit_high_watermark; +uint16_t dir_credit_high_watermark; + +if (handle ==NULL ) +return -EINVAL ; + +if (dequeue_depth <DLB2_MIN_CQ_DEPTH || + dequeue_depth >DLB2_MAX_INPUT_QUEUE_DEPTH ) { + DLB2_LOG_ERR("dlb2: invalid dequeue_depth, must be %d- %d\n", + DLB2_MIN_CQ_DEPTH,
DLB2_MAX_INPUT_QUEUE_DEPTH);drivers/event/dlb2/dlb2.c +5
-0+return -EINVAL ; + } + +if (enqueue_depth <DLB2_MIN_ENQUEUE_DEPTH ) { + DLB2_LOG_ERR("dlb2: invalid enqueue_depth, must be at least%d\n",
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_setup(struct rte_eventdev *dev, · line 1182
Quoted 3 lines
drivers/event/dlb2/dlb2.c +3
-0+ DLB2_MIN_ENQUEUE_DEPTH); +return -EINVAL ; + }
The dequeue and enqueue depth checks are suspiciously inconsistent -- only the dequeue depth is compared against an upper bound. I suspect the enqueue upper bound check is missing because it's already checked in both dlb2_eventdev_port_setup() and rte_event_port_setup()...if that's the case, can the dequeue depth max check be dropped as well?
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_setup(struct rte_eventdev *dev, · line 1186
Quoted 4 lines
drivers/event/dlb2/dlb2.c +4
-0+ +rte_spinlock_lock (&handle->resource_lock); + +/* TODO - additional parameter validation */
Leftover TODO
Quoted 24 lines
drivers/event/dlb2/dlb2.c +5
-0+/* We round up to the next power of 2 if necessary */ + cfg.cq_depth =rte_align32pow2 (dequeue_depth); + cfg.cq_depth_threshold =1 ; + + cfg.cq_history_list_size =
DLB2_NUM_HIST_LIST_ENTRIES_PER_LDB_PORT;drivers/event/dlb2/dlb2.c +9
-0+ +if (handle->cos_id ==DLB2_COS_DEFAULT ) + cfg.cos_id =0 ; +else + cfg.cos_id = handle->cos_id; + + cfg.cos_strict =0 ; + +/* User controls the LDB high watermark via enqueue depth. The DIR high
drivers/event/dlb2/dlb2.c +8
-0+* watermark is equal, unless the directed credit pool is too small. +*/ + ldb_credit_high_watermark = enqueue_depth; + +/* If there are no directed ports, the kernel driver will ignore this +* port's directed credit settings. Don't use enqueue_depth if it would +* require more directed credits than are available. +*/
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_setup(struct rte_eventdev *dev, · line 1218
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+ dir_credit_high_watermark = +RTE_MIN (enqueue_depth, + handle->cfg.num_dir_credits / dlb2->num_ports); + +/* Per QM values */ + +/* DEBUG +* DLB2_LOG_ERR("create ldb port - grp=%d, devId=%d\n", +* handle->cfg.domain_id, handle->device_id); +*/
Leftover debug code
Quoted 16 lines
drivers/event/dlb2/dlb2.c +4
-0+ + ret =dlb2_iface_ldb_port_create (handle, &cfg, dlb2->poll_mode); +if (ret <0 ) { + DLB2_LOG_ERR("dlb2: dlb2_ldb_port_create error, ret=%d(driver status: %s)\n",
drivers/event/dlb2/dlb2.c +11
-0+ ret, dlb2_error_strings[cfg.response.status]); + goto error_exit; + } + + qm_port_id = cfg.response.id; + +DLB2_LOG_DBG ("dlb2: ev_port %d uses qm LB port %d <<<<<\n" , + ev_port->id, qm_port_id); + + qm_port = &ev_port->qm_port; + qm_port->ev_port = ev_port;/* back ptr */
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_setup(struct rte_eventdev *dev, · line 1243
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+ qm_port->dlb2 = dlb2;/* back ptr */ +/* +* Allocate and init local qe struct(s). +* Note: MOVDIR64 requires the enqueue QE (qe4) to be aligned. +*/ + +snprintf (mz_name,sizeof (mz_name),"%s_ldb_port%d" , + handle->device_name, + ev_port->id); +
I see handle->device_name getting read here, in dlb2_hw_create_dir_port(), and also in dlb2_eventdev_dump(), but I don't see it written anywhere?
Quoted 30 lines
drivers/event/dlb2/dlb2.c +23
-0+ ret =dlb2_init_qe_mem (qm_port, mz_name); +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: init_qe_mem failed, ret=%d\n" , ret); + goto error_exit; + } + + qm_port->id = qm_port_id; + + qm_port->cached_ldb_credits =0 ; + qm_port->cached_dir_credits =0 ; +/* CQs with depth < 8 use an 8-entry queue, but withhold credits so +* the effective depth is smaller. +*/ + qm_port->cq_depth = cfg.cq_depth <=8 ?8 : cfg.cq_depth; + qm_port->cq_idx =0 ; + qm_port->cq_idx_unmasked =0 ; + +if (dlb2->poll_mode ==DLB2_CQ_POLL_MODE_SPARSE ) + qm_port->cq_depth_mask = (qm_port->cq_depth *4 ) -1 ; +else + qm_port->cq_depth_mask = qm_port->cq_depth -1 ; + + qm_port->gen_bit_shift = __builtin_popcount(qm_port-Quoted 1 line
cq_depth_mask);drivers/event/dlb2/dlb2.c +6
-0+/* starting value of gen bit - it toggles at wrap time */ + qm_port->gen_bit =1 ; + + qm_port->int_armed = false; + +/* Save off for later use in info and lookup APIs. */
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_setup(struct rte_eventdev *dev, · line 1282
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+ qm_port->qid_mappings = &dlb2->qm_ldb_to_ev_queue_id[0 ]; + + qm_port->dequeue_depth = dequeue_depth; + qm_port->token_pop_thresh = dequeue_depth; + + qm_port->owed_tokens =0 ; + qm_port->issued_releases =0 ; + +/* Save config message too. */ +rte_memcpy (&qm_port->cfg.ldb, &cfg,sizeof (cfg));
I know qm_port->cfg.ldb and cfg are the same type, but just for safety in case that ever changes in the future...probably better to use sizeof() on the destination rather than the source.
Quoted 76 lines
drivers/event/dlb2/dlb2.c +11
-0+ +/* update state */ + qm_port->state =PORT_STARTED ;/* enabled at create time */ + qm_port->config_state =DLB2_CONFIGURED ; + + qm_port->dir_credits = dir_credit_high_watermark; + qm_port->ldb_credits = ldb_credit_high_watermark; + qm_port->credit_pool[DLB2_DIR_QUEUE ] = &dlb2->dir_credit_pool; + qm_port->credit_pool[DLB2_LDB_QUEUE ] = &dlb2->ldb_credit_pool; + + DLB2_LOG_DBG("dlb2: created ldb port %d, depth = %d, ldb credits=%d,dir credits=%d\n",
drivers/event/dlb2/dlb2.c +61
-0+ qm_port_id, + dequeue_depth, + qm_port->ldb_credits, + qm_port->dir_credits); + +rte_spinlock_unlock (&handle->resource_lock); + +return 0 ; + +error_exit: + +if (qm_port) +dlb2_free_qe_mem (qm_port); + +rte_spinlock_unlock (&handle->resource_lock); + +DLB2_LOG_ERR ("dlb2: create ldb port failed!\n" ); + +return ret; +} + +static void +dlb2_port_link_teardown (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port) +{ +struct dlb2_eventdev_queue *ev_queue; +int i; + +for (i =0 ; i <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; i++) { +if (!ev_port->link[i].valid) +continue ; + + ev_queue = &dlb2->ev_queues[ev_port->link[i].queue_id]; + + ev_port->link[i].valid = false; + ev_port->num_links--; + ev_queue->num_links--; + } +} + +static int +dlb2_hw_create_dir_port (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port, +uint32_t dequeue_depth, +uint32_t enqueue_depth) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_create_dir_port_args cfg = {0 }; +int ret; +struct dlb2_port *qm_port =NULL ; +char mz_name[RTE_MEMZONE_NAMESIZE ]; +uint32_t qm_port_id; +uint16_t ldb_credit_high_watermark; +uint16_t dir_credit_high_watermark; + +if (dlb2 ==NULL || handle ==NULL ) +return -EINVAL ; + +if (dequeue_depth <DLB2_MIN_CQ_DEPTH || + dequeue_depth >DLB2_MAX_INPUT_QUEUE_DEPTH ) { + DLB2_LOG_ERR("dlb2: invalid dequeue_depth, must be %d-%d\n",
drivers/event/dlb2/dlb2.c +1
-0+ DLB2_MIN_CQ_DEPTH,
DLB2_MAX_INPUT_QUEUE_DEPTH);
drivers/event/dlb2/dlb2.c
Quoted 2 lines
drivers/event/dlb2/dlb2.c +2
-0+return -EINVAL ; + }
Enqueue depth check needed?
Quoted 39 lines
drivers/event/dlb2/dlb2.c +11
-0+ +rte_spinlock_lock (&handle->resource_lock); + +/* Directed queues are configured at link time. */ + cfg.queue_id =-1 ; + +/* We round up to the next power of 2 if necessary */ + cfg.cq_depth =rte_align32pow2 (dequeue_depth); + cfg.cq_depth_threshold =1 ; + +/* User controls the LDB high watermark via enqueue depth. The DIR high
drivers/event/dlb2/dlb2.c +16
-0+* watermark is equal, unless the directed credit pool is too small. +*/ + ldb_credit_high_watermark = enqueue_depth; + +/* Don't use enqueue_depth if it would require more directed credits +* than are available. +*/ + dir_credit_high_watermark = +RTE_MIN (enqueue_depth, + handle->cfg.num_dir_credits / dlb2->num_ports); + +/* Per QM values */ + + ret =dlb2_iface_dir_port_create (handle, &cfg, dlb2->poll_mode); +if (ret <0 ) { + DLB2_LOG_ERR("dlb2: dlb2_dir_port_create error, ret=%d (driverstatus: %s)\n",
drivers/event/dlb2/dlb2.c +10
-0+ ret, dlb2_error_strings[cfg.response.status]); + goto error_exit; + } + + qm_port_id = cfg.response.id; + +DLB2_LOG_DBG ("dlb2: ev_port %d uses qm DIR port %d <<<<<\n" , + ev_port->id, qm_port_id); + + qm_port = &ev_port->qm_port;
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_setup(struct rte_eventdev *dev, · line 1404
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+ qm_port->ev_port = ev_port;/* back ptr */ + qm_port->dlb2 = dlb2;/* back ptr */ + +/* +* Init local qe struct(s). +* Note: MOVDIR64 requires the enqueue QE to be aligned +*/ + + snprintf(mz_name, sizeof(mz_name), "%s_dir_port%d", + handle->device_name,
(See device_name comment above)
Quoted 34 lines
drivers/event/dlb2/dlb2.c +26
-0+ ev_port->id); + + ret =dlb2_init_qe_mem (qm_port, mz_name); + +if (ret <0 ) { +DLB2_LOG_ERR ("dlb2: init_qe_mem failed, ret=%d\n" , ret); + goto error_exit; + } + + qm_port->id = qm_port_id; + + qm_port->cached_ldb_credits =0 ; + qm_port->cached_dir_credits =0 ; +/* CQs with depth < 8 use an 8-entry queue, but withhold credits so +* the effective depth is smaller. +*/ + qm_port->cq_depth = cfg.cq_depth <=8 ?8 : cfg.cq_depth; + qm_port->cq_idx =0 ; + qm_port->cq_idx_unmasked =0 ; + +if (dlb2->poll_mode ==DLB2_CQ_POLL_MODE_SPARSE ) + qm_port->cq_depth_mask = (cfg.cq_depth *4 ) -1 ; +else + qm_port->cq_depth_mask = cfg.cq_depth -1 ; + + qm_port->gen_bit_shift = __builtin_popcount(qm_port-Quoted 1 line
cq_depth_mask);drivers/event/dlb2/dlb2.c +7
-0+/* starting value of gen bit - it toggles at wrap time */ + qm_port->gen_bit =1 ; + + qm_port->int_armed = false; + +/* Save off for later use in info and lookup APIs. */ + qm_port->qid_mappings = &dlb2->qm_dir_to_ev_queue_id[0 ];
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_setup(struct rte_eventdev *dev, · line 1447
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+ + qm_port->dequeue_depth = dequeue_depth; + +/* Directed ports are auto-pop, by default. */ + qm_port->token_pop_mode =AUTO_POP ; + qm_port->owed_tokens =0 ; + qm_port->issued_releases =0 ; + +/* Save config message too. */ +rte_memcpy (&qm_port->cfg.dir, &cfg,sizeof (cfg));
(See sizeof() comment above)
Quoted 67 lines
drivers/event/dlb2/dlb2.c +61
-0+ +/* update state */ + qm_port->state =PORT_STARTED ;/* enabled at create time */ + qm_port->config_state =DLB2_CONFIGURED ; + + qm_port->dir_credits = dir_credit_high_watermark; + qm_port->ldb_credits = ldb_credit_high_watermark; + qm_port->credit_pool[DLB2_DIR_QUEUE ] = &dlb2->dir_credit_pool; + qm_port->credit_pool[DLB2_LDB_QUEUE ] = &dlb2->ldb_credit_pool; + +DLB2_LOG_DBG ("dlb2: created dir port %d, depth = %d cr=%d,%d\n" , + qm_port_id, + dequeue_depth, + dir_credit_high_watermark, + ldb_credit_high_watermark); + +rte_spinlock_unlock (&handle->resource_lock); + +return 0 ; + +error_exit: + +if (qm_port) +dlb2_free_qe_mem (qm_port); + +rte_spinlock_unlock (&handle->resource_lock); + +DLB2_LOG_ERR ("dlb2: create dir port failed!\n" ); + +return ret; +} + +static int +dlb2_eventdev_port_setup (struct rte_eventdev *dev, +uint8_t ev_port_id, +const struct rte_event_port_conf *port_conf) +{ +struct dlb2_eventdev *dlb2; +struct dlb2_eventdev_port *ev_port; +int ret; + +if (dev ==NULL || port_conf ==NULL ) { +DLB2_LOG_ERR ("Null parameter\n" ); +return -EINVAL ; + } + + dlb2 =dlb2_pmd_priv (dev); + +if (ev_port_id >=DLB2_MAX_NUM_PORTS ) +return -EINVAL ; + +if (port_conf->dequeue_depth > + evdev_dlb2_default_info.max_event_port_dequeue_depth || + port_conf->enqueue_depth > + evdev_dlb2_default_info.max_event_port_enqueue_depth) +return -EINVAL ; + + ev_port = &dlb2->ev_ports[ev_port_id]; +/* configured? */ +if (ev_port->setup_done) { + DLB2_LOG_ERR("evport %d is already configured\n",
ev_port_id);drivers/event/dlb2/dlb2.c +4
-0+return -EINVAL ; + } + +/* The reserved token interrupt arming scheme requires that one or more
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_setup(struct rte_eventdev *dev, · line 1519
Quoted 7 lines
drivers/event/dlb2/dlb2.c +7
-0+* CQ tokens be reserved by the PMD. This limits the amount of CQ space +* usable by the DLB, so in order to give an *effective* CQ depth equal +* to the user-requested value, we double CQ depth and reserve half of +* its tokens. If the user requests the max CQ depth (256) then we +* cannot double it, so we reserve one token and give an effective +* depth of 255 entries. +*/
I don't think this comment applies to the 2.0 device.
Quoted 19 lines
drivers/event/dlb2/dlb2.c +11
-0+ + ev_port->qm_port.is_directed = port_conf->event_port_cfg & +RTE_EVENT_PORT_CFG_SINGLE_LINK ; + +if (!ev_port->qm_port.is_directed) { + ret =dlb2_hw_create_ldb_port (dlb2, + ev_port, + port_conf->dequeue_depth, + port_conf->enqueue_depth); +if (ret <0 ) { + DLB2_LOG_ERR("Failed to create the lB port veportId=%d\n",
drivers/event/dlb2/dlb2.c +7
-0+ ev_port_id); + +return ret; + } + }else { + ret = dlb2_hw_create_dir_port(dlb2, + ev_port,
drivers/event/dlb2/dlb2.c dlb2_eventdev_queue_setup(struct rte_eventdev *dev, · line 1547
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+ port_conf->dequeue_depth, + port_conf->enqueue_depth); +if (ret <0 ) { +DLB2_LOG_ERR ("Failed to create the DIR port\n" ); +return ret; + } + } + +/* Save off port config for reconfig */ + dlb2->ev_ports[ev_port_id].conf = *port_conf;
Nit: 'ev_port' is assigned to &dlb2->ev_ports[ev_port_id] above, use it here and below?
Quoted 23 lines
drivers/event/dlb2/dlb2.c +23
-0+ + dlb2->ev_ports[ev_port_id].id = ev_port_id; + dlb2->ev_ports[ev_port_id].enq_configured = true; + dlb2->ev_ports[ev_port_id].setup_done = true; + dlb2->ev_ports[ev_port_id].inflight_max = + port_conf->new_event_threshold; + dlb2->ev_ports[ev_port_id].implicit_release = + !(port_conf->event_port_cfg & +RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL ); + dlb2->ev_ports[ev_port_id].outstanding_releases =0 ; + dlb2->ev_ports[ev_port_id].inflight_credits =0 ; + dlb2->ev_ports[ev_port_id].credit_update_quanta = +RTE_LIBRTE_PMD_DLB2_SW_CREDIT_QUANTA ; + dlb2->ev_ports[ev_port_id].dlb2 = dlb2;/* reverse link */ + +/* Tear down pre-existing port->queue links */ +if (dlb2->run_state ==DLB2_RUN_STATE_STOPPED ) +dlb2_port_link_teardown (dlb2, &dlb2->ev_ports[ev_port_id]); + + dev->data->ports[ev_port_id] = &dlb2->ev_ports[ev_port_id]; + +return 0 ; +}
[...]
Quoted 9 lines
drivers/event/dlb2/pf/dlb2_pf.c +0
-0@@ -234,6 +234,183 @@ dlb2_pf_set_sn_allocation(struct dlb2_hw_dev *handle, return ret; }
drivers/event/dlb2/pf/dlb2_pf.c dlb2_pf_set_sn_allocation(struct dlb2_hw_dev · line 247
Quoted 10 lines
drivers/event/dlb2/pf/dlb2_pf.c +10
-0+static void * +dlb2_alloc_coherent_aligned (uintptr_t *phys,size_t size,int align) +{ +const struct rte_memzone *mz; +char mz_name[RTE_MEMZONE_NAMESIZE ]; +uint32_t core_id =rte_lcore_id (); +unsigned int socket_id; + +snprintf (mz_name,sizeof (mz_name) -1 ,"%lx" , + (unsigned long )rte_get_timer_cycles ());
For debug purposes, it would be better if this name can trace the mz back to this driver. How about something like event_dlb2_pf_name + ldb/dir_port + port ID?
drivers/event/dlb2/pf/dlb2_pf.c dlb2_pf_set_sn_allocation(struct dlb2_hw_dev *handle, · line 249
Quoted 3 lines
drivers/event/dlb2/pf/dlb2_pf.c +3
-0+if (core_id == (unsigned int )LCORE_ID_ANY ) + core_id =rte_get_master_lcore (); + socket_id =rte_lcore_to_socket_id (core_id);
Should this use the socket ID devarg (and perhaps fall back to the core's socket if unspecified)?
Quoted 36 lines
drivers/event/dlb2/pf/dlb2_pf.c +35
-0+ mz =rte_memzone_reserve_aligned (mz_name, size, socket_id, +RTE_MEMZONE_IOVA_CONTIG , align); +if (!mz) { + DLB2_LOG_DBG("Unable to allocate DMA memory of size %zu bytes - %s\n", + size, rte_strerror(rte_errno)); + *phys =0 ; +return NULL ; + } + *phys = mz->iova; +return mz->addr; +} + +static int +dlb2_pf_ldb_port_create (struct dlb2_hw_dev *handle, +struct dlb2_create_ldb_port_args *cfg, +enum dlb2_cq_poll_modes poll_mode) +{ +struct dlb2_dev *dlb2_dev = (struct dlb2_dev *)handle->pf_dev; +struct dlb2_cmd_response response = {0 }; +struct dlb2_port_memory port_memory; +int ret, cq_alloc_depth; +uint8_t *port_base; +int alloc_sz, qe_sz; +phys_addr_t cq_base; +phys_addr_t pp_base; +int is_dir = false; + +DLB2_INFO (dev->dlb2_device,"Entering %s()\n" , __func__); + +if (poll_mode ==DLB2_CQ_POLL_MODE_STD ) + qe_sz =sizeof (struct dlb2_dequeue_qe ); +else + qe_sz =RTE_CACHE_LINE_SIZE ; + +/* Calculate the port memory required, and round up to the nearest
drivers/event/dlb2/pf/dlb2_pf.c dlb2_pf_set_sn_allocation(struct dlb2_hw_dev *handle, · line 293
Quoted 10 lines
drivers/event/dlb2/pf/dlb2_pf.c +9
-0+* cache line. +*/ + cq_alloc_depth =RTE_MAX (cfg->cq_depth,DLB2_MIN_HARDWARE_CQ_DEPTH ); + alloc_sz = cq_alloc_depth * qe_sz; + alloc_sz =RTE_CACHE_LINE_ROUNDUP (alloc_sz); + + port_base =dlb2_alloc_coherent_aligned (&cq_base, + alloc_sz, +PAGE_SIZE );
This can fit on one line
Quoted 15 lines
drivers/event/dlb2/pf/dlb2_pf.c +7
-0+if (port_base ==NULL ) +return -ENOMEM ; + +/* Lock the page in memory */ + ret =rte_mem_lock_page (port_base); +if (ret <0 ) { + DLB2_LOG_ERR("dlb2 pf pmd could not lock page for devicei/o\n");
drivers/event/dlb2/pf/dlb2_pf.c +7
-0+ goto create_port_err; + } + + +memset (port_base,0 , alloc_sz); + + ret = dlb2_pf_create_ldb_port(&dlb2_dev->hw,
drivers/event/dlb2/pf/dlb2_pf.c dlb2_pf_set_sn_allocation(struct dlb2_hw_dev *handle, · line 317
Quoted 10 lines
drivers/event/dlb2/pf/dlb2_pf.c +10
-0+ handle->domain_id, + cfg, + cq_base, + &response); +if (ret) + goto create_port_err; + + pp_base = (uintptr_t )dlb2_dev->hw.func_kva +PP_BASE (is_dir); + dlb2_port[response.id][DLB2_LDB_PORT ].pp_addr = + (void *)(uintptr_t )(pp_base + (PAGE_SIZE * response.id));
If pp_base is defined as a uintptr_t, I think you can avoid some of the explicit casts.
drivers/event/dlb2/pf/dlb2_pf.c dlb2_pf_set_sn_allocation(struct dlb2_hw_dev *handle, · line 320
Quoted 3 lines
drivers/event/dlb2/pf/dlb2_pf.c +3
-0+ + dlb2_port[response.id][DLB2_LDB_PORT ].cq_base = + (void *)(uintptr_t )(port_base);
Since port_base is a uint8_t*, the uintptr_t cast shouldn't be necessary
drivers/event/dlb2/pf/dlb2_pf.c dlb2_pf_set_sn_allocation(struct dlb2_hw_dev *handle, · line 327
Quoted 7 lines
drivers/event/dlb2/pf/dlb2_pf.c +7
-0+memset (&port_memory,0 ,sizeof (port_memory)); +dlb2_list_init_head (&port_memory.list); + +/* Fill out the per-port memory tracking structure */ + dlb2_dev->ldb_port_pages[response.id].valid = true; +dlb2_list_splice (&port_memory.list, + &dlb2_dev->ldb_port_pages[response.id].list);
Does the list serve any purpose? Looks like port_memory is zero-initialized, then it becomes the sole entry on a per-port list.
drivers/event/dlb2/pf/dlb2_pf.c dlb2_pf_set_sn_allocation(struct dlb2_hw_dev *handle, · line 334
Quoted 7 lines
drivers/event/dlb2/pf/dlb2_pf.c +7
-0+ + cfg->response = response; + +DLB2_INFO (dev->dlb2_device,"Exiting %s() with ret=%d\n" , + __func__, ret); + +create_port_err:
Need to free the memzone in this case.
Quoted 25 lines
drivers/event/dlb2/pf/dlb2_pf.c +25
-0+ +return ret; +} + +static int +dlb2_pf_dir_port_create (struct dlb2_hw_dev *handle, +struct dlb2_create_dir_port_args *cfg, +enum dlb2_cq_poll_modes poll_mode) +{ +struct dlb2_dev *dlb2_dev = (struct dlb2_dev *)handle->pf_dev; +struct dlb2_cmd_response response = {0 }; +struct dlb2_port_memory port_memory; +int ret; +uint8_t *port_base; +int alloc_sz, qe_sz; +phys_addr_t cq_base; +phys_addr_t pp_base; +int is_dir = true; + +DLB2_INFO (dev->dlb2_device,"Entering %s()\n" , __func__); + +if (poll_mode ==DLB2_CQ_POLL_MODE_STD ) + qe_sz =sizeof (struct dlb2_dequeue_qe ); +else + qe_sz =RTE_CACHE_LINE_SIZE ;
drivers/event/dlb2/pf/dlb2_pf.c dlb2_pf_set_sn_allocation(struct dlb2_hw_dev *handle, · line 369
Quoted 10 lines
drivers/event/dlb2/pf/dlb2_pf.c +10
-0+ +/* Calculate the port memory required, and round up to the nearest +* cache line. +*/ + alloc_sz = cfg->cq_depth * qe_sz; + alloc_sz =RTE_CACHE_LINE_ROUNDUP (alloc_sz); + + port_base =dlb2_alloc_coherent_aligned (&cq_base, + alloc_sz, +PAGE_SIZE );
This can fit on one line
Quoted 12 lines
drivers/event/dlb2/pf/dlb2_pf.c +7
-0+if (port_base ==NULL ) +return -ENOMEM ; + +/* Lock the page in memory */ + ret =rte_mem_lock_page (port_base); +if (ret <0 ) { + DLB2_LOG_ERR("dlb2 pf pmd could not lock page for devicei/o\n");
drivers/event/dlb2/pf/dlb2_pf.c +4
-0+ goto create_port_err; + } + +memset (port_base,0 , alloc_sz);
drivers/event/dlb2/pf/dlb2_pf.c dlb2_pf_set_sn_allocation(struct dlb2_hw_dev *handle, · line 390
Quoted 10 lines
drivers/event/dlb2/pf/dlb2_pf.c +10
-0+ + ret =dlb2_pf_create_dir_port (&dlb2_dev->hw, + handle->domain_id, + cfg, + cq_base, + &response); +if (ret) + goto create_port_err; + + pp_base = (uintptr_t )dlb2_dev->hw.func_kva +PP_BASE (is_dir);
(See uintptr_t comment above)
drivers/event/dlb2/pf/dlb2_pf.c dlb2_pf_set_sn_allocation(struct dlb2_hw_dev *handle, · line 395
Quoted 5 lines
drivers/event/dlb2/pf/dlb2_pf.c +5
-0+ dlb2_port[response.id][DLB2_DIR_PORT ].pp_addr = + (void *)(uintptr_t )(pp_base + (PAGE_SIZE * response.id)); + + dlb2_port[response.id][DLB2_DIR_PORT ].cq_base = + (void *)(uintptr_t )(port_base);
(See port_base comment above)
drivers/event/dlb2/pf/dlb2_pf.c dlb2_pf_set_sn_allocation(struct dlb2_hw_dev *handle, · line 403
Quoted 8 lines
drivers/event/dlb2/pf/dlb2_pf.c +8
-0+memset (&port_memory,0 ,sizeof (port_memory)); +dlb2_list_init_head (&port_memory.list); + +/* Fill out the per-port memory tracking structure */ + dlb2_dev->dir_port_pages[response.id].valid = true; +dlb2_list_splice (&port_memory.list, + &dlb2_dev->dir_port_pages[response.id].list); +
(See list comment above)
drivers/event/dlb2/pf/dlb2_pf.c dlb2_pf_set_sn_allocation(struct dlb2_hw_dev *handle, · line 334
Quoted 6 lines
drivers/event/dlb2/pf/dlb2_pf.c +6
-0+ cfg->response = response; + +DLB2_INFO (dev->dlb2_device,"Exiting %s() with ret=%d\n" , + __func__, ret); + +create_port_err:
Need to free the memzone in this case
Quoted 17 lines
drivers/event/dlb2/pf/dlb2_pf.c +6
-1+ +return ret; +} +static void dlb2_pf_iface_fn_ptrs_init (void ) {@@ -247,6 +424,8 @@ dlb2_pf_iface_fn_ptrs_init(void) dlb2_iface_get_cq_poll_mode = dlb2_pf_get_cq_poll_mode; dlb2_iface_sched_domain_create = dlb2_pf_sched_domain_create; dlb2_iface_ldb_queue_create = dlb2_pf_ldb_queue_create; + dlb2_iface_ldb_port_create = dlb2_pf_ldb_port_create; + dlb2_iface_dir_port_create = dlb2_pf_dir_port_create; dlb2_iface_get_sn_allocation = dlb2_pf_get_sn_allocation; dlb2_iface_set_sn_allocation = dlb2_pf_set_sn_allocation; dlb2_iface_get_sn_occupancy = dlb2_pf_get_sn_occupancy; --
Quoted 1 line
2.6.4
I don't see the port memzones getting freed anywhere, e.g. if the event device is reset. Looks like a possible memory leak.
Thanks,
Gage
Re: [dpdk-dev] [PATCH 12/22] event/dlb2: add port link
In reply to Timothy McDanielQuoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+static int +dlb2_event_queue_join_ldb (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_port *ev_port, +struct dlb2_eventdev_queue *ev_queue, +uint8_t priority) +{ +int first_avail =-1 ; +int ret, i; + +for (i =0 ; i <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; i++) {
drivers/event/dlb2/dlb2.c dlb2_eventdev_port_setup(struct rte_eventdev *dev, · line 1625
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+if (ev_port->link[i].valid) { +if (ev_port->link[i].queue_id == ev_queue->id && + ev_port->link[i].priority == priority) { +if (ev_port->link[i].mapped) +return 0 ;/* already mapped */ + first_avail = i; + } + }else if (first_avail ==-1 ) { + first_avail = i; + }
Nit: braces discouraged on a single-statement conditionals
Quoted 35 lines
drivers/event/dlb2/dlb2.c +32
-0+ } +if (first_avail ==-1 ) { +DLB2_LOG_ERR ("dlb2: qm_port %d has no available QID slots.\n" , + ev_port->qm_port.id); +return -EINVAL ; + } + + ret =dlb2_hw_map_ldb_qid_to_port (&dlb2->qm_instance, + ev_port->qm_port.id, + ev_queue->qm_queue.id, + priority); + +if (!ret) + ev_port->link[first_avail].mapped = true; + +return ret; +} + +static int32_t +dlb2_hw_create_dir_queue (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *ev_queue, +int32_t qm_port_id) +{ +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_create_dir_queue_args cfg; +int32_t ret; + +/* The directed port is always configured before its queue */ + cfg.port_id = qm_port_id; + +if (ev_queue->depth_threshold ==0 ) { + cfg.depth_threshold =
RTE_PMD_DLB2_DEFAULT_DEPTH_THRESH;(quoted diff) +1
-0+ ev_queue->depth_threshold =
RTE_PMD_DLB2_DEFAULT_DEPTH_THRESH;
drivers/event/dlb2/dlb2.c dlb2_eventdev_port_setup(struct rte_eventdev *dev, · line 1661
Quoted 3 lines
drivers/event/dlb2/dlb2.c +3
-0+ }else { + cfg.depth_threshold = ev_queue->depth_threshold; + }
Nit: braces discouraged on a single-statement conditionals
Quoted 179 lines
drivers/event/dlb2/dlb2.c +4
-0+ + ret =dlb2_iface_dir_queue_create (handle, &cfg); +if (ret <0 ) { + DLB2_LOG_ERR("dlb2: create DIR event queue error, ret=%d(driver status: %s)\n",
drivers/event/dlb2/dlb2.c +15
-0+ ret, dlb2_error_strings[cfg.response.status]); +return -EINVAL ; + } + +return cfg.response.id; +} + +static int +dlb2_eventdev_dir_queue_setup (struct dlb2_eventdev *dlb2, +struct dlb2_eventdev_queue *ev_queue, +struct dlb2_eventdev_port *ev_port) +{ +int32_t qm_qid; + + qm_qid = dlb2_hw_create_dir_queue(dlb2, ev_queue, ev_port-Quoted 1 line
qm_port.id);
drivers/event/dlb2/dlb2.c +28
-0+ +if (qm_qid <0 ) { +DLB2_LOG_ERR ("Failed to create the DIR queue\n" ); +return qm_qid; + } + + dlb2->qm_dir_to_ev_queue_id[qm_qid] = ev_queue->id; + + ev_queue->qm_queue.id = qm_qid; + +return 0 ; +} + +static int +dlb2_do_port_link (struct rte_eventdev *dev, +struct dlb2_eventdev_queue *ev_queue, +struct dlb2_eventdev_port *ev_port, +uint8_t prio) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +int err; + +/* Don't link until start time. */ +if (dlb2->run_state ==DLB2_RUN_STATE_STOPPED ) +return 0 ; + +if (ev_queue->qm_queue.is_directed) + err = dlb2_eventdev_dir_queue_setup(dlb2, ev_queue,
ev_port);(quoted diff) +2
-0+ else + err = dlb2_event_queue_join_ldb(dlb2, ev_port, ev_queue,prio);
drivers/event/dlb2/dlb2.c +126
-0+ +if (err) { +DLB2_LOG_ERR ("port link failure for %s ev_q %d, ev_port %d\n" , + ev_queue->qm_queue.is_directed ?"DIR" :"LDB" , + ev_queue->id, ev_port->id); + + rte_errno = err; +return -1 ; + } + +return 0 ; +} + +static int +dlb2_validate_port_link (struct dlb2_eventdev_port *ev_port, +uint8_t queue_id, +bool link_exists, +int index) +{ +struct dlb2_eventdev *dlb2 = ev_port->dlb2; +struct dlb2_eventdev_queue *ev_queue; +bool port_is_dir, queue_is_dir; + +if (queue_id > dlb2->num_queues) { + rte_errno = -EINVAL ; +return -1 ; + } + + ev_queue = &dlb2->ev_queues[queue_id]; + +if (!ev_queue->setup_done && + ev_queue->qm_queue.config_state !=DLB2_PREV_CONFIGURED ) { + rte_errno = -EINVAL ; +return -1 ; + } + + port_is_dir = ev_port->qm_port.is_directed; + queue_is_dir = ev_queue->qm_queue.is_directed; + +if (port_is_dir != queue_is_dir) { +DLB2_LOG_ERR ("%s queue %u can't link to %s port %u\n" , + queue_is_dir ?"DIR" :"LDB" , ev_queue->id, + port_is_dir ?"DIR" :"LDB" , ev_port->id); + + rte_errno = -EINVAL ; +return -1 ; + } + +/* Check if there is space for the requested link */ +if (!link_exists && index ==-1 ) { +DLB2_LOG_ERR ("no space for new link\n" ); + rte_errno = -ENOSPC ; +return -1 ; + } + +/* Check if the directed port is already linked */ +if (ev_port->qm_port.is_directed && ev_port->num_links >0 && + !link_exists) { +DLB2_LOG_ERR ("Can't link DIR port %d to >1 queues\n" , + ev_port->id); + rte_errno = -EINVAL ; +return -1 ; + } + +/* Check if the directed queue is already linked */ +if (ev_queue->qm_queue.is_directed && ev_queue->num_links >0 && + !link_exists) { +DLB2_LOG_ERR ("Can't link DIR queue %d to >1 ports\n" , + ev_queue->id); + rte_errno = -EINVAL ; +return -1 ; + } + +return 0 ; +} + +static int +dlb2_eventdev_port_link (struct rte_eventdev *dev,void *event_port, +const uint8_t queues[],const uint8_t priorities[], +uint16_t nb_links) + +{ +struct dlb2_eventdev_port *ev_port = event_port; +struct dlb2_eventdev *dlb2; +int i, j; + +RTE_SET_USED (dev); + +if (!ev_port) { +DLB2_LOG_ERR ("dlb2: evport not setup\n" ); + rte_errno = -EINVAL ; +return 0 ; + } + +if (!ev_port->setup_done && + ev_port->qm_port.config_state !=DLB2_PREV_CONFIGURED ) { +DLB2_LOG_ERR ("dlb2: evport not setup\n" ); + rte_errno = -EINVAL ; +return 0 ; + } + +/* Note: rte_event_port_link() ensures the PMD won't receive a NULL +* queues pointer. +*/ +if (nb_links ==0 ) { +DLB2_LOG_DBG ("dlb2: nb_links is 0\n" ); +return 0 ;/* Ignore and return success */ + } + + dlb2 = ev_port->dlb2; + +DLB2_LOG_DBG ("Linking %u queues to %s port %d\n" , + nb_links, + ev_port->qm_port.is_directed ?"DIR" :"LDB" , + ev_port->id); + +for (i =0 ; i < nb_links; i++) { +struct dlb2_eventdev_queue *ev_queue; +uint8_t queue_id, prio; +bool found = false; +int index =-1 ; + + queue_id = queues[i]; + prio = priorities[i]; + +/* Check if the link already exists. */
drivers/event/dlb2/dlb2.c dlb2_eventdev_port_setup(struct rte_eventdev *dev, · line 1846
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+for (j =0 ; j <DLB2_MAX_NUM_QIDS_PER_LDB_CQ ; j++) +if (ev_port->link[j].valid) { +if (ev_port->link[j].queue_id == queue_id) { + found = true; + index = j; +break ; + } + }else if (index ==-1 ) { + index = j; + }
Nit: braces discouraged on a single-statement conditionals
Thanks,
Gage
Re: [dpdk-dev] [PATCH 13/22] event/dlb2: add port unlink and port unlinks in progress
In reply to Timothy McDanielQuoted 10 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 13/22] event/dlb2: add port unlink and port unlinks in progress
Quoted 6 lines
Add supports for the port unlink(s) eventdev entry points. The unlink operation is an asynchronous operation executed by a control thread, and the unlinks-in-progress function reads a counter shared with the control thread.
Signed-off-by Timothy McDaniel <[email protected]>
Thanks,
Gage
Re: [dpdk-dev] [PATCH 14/22] event/dlb2: add eventdev start
In reply to Timothy McDanielQuoted 10 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 14/22] event/dlb2: add eventdev start
Quoted 2 lines
Add support for the eventdev start entry point.
Code looks fine (one whitespace nit below), but it could use a more detailed commit message -- e.g. describing how the PMD delays port links until eventdev start time, how the configuration will be reapplied at this time if this is a reconfiguration scenario, etc.
[...]
Quoted 6 lines
drivers/event/dlb2/dlb2.c +6
-0+static int +dlb2_eventdev_start (struct rte_eventdev *dev) +{ +struct dlb2_eventdev *dlb2 =dlb2_pmd_priv (dev); +struct dlb2_hw_dev *handle = &dlb2->qm_instance; +struct dlb2_start_domain_args cfg;
drivers/event/dlb2/dlb2.c dlb2_eventdev_port_unlinks_in_progress(struct rte_eventdev *dev, · line 2126
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+int ret, i; + +rte_spinlock_lock (&dlb2->qm_instance.resource_lock); +if (dlb2->run_state !=DLB2_RUN_STATE_STOPPED ) { +DLB2_LOG_ERR ("bad state %d for dev_start\n" , + (int )dlb2->run_state); +rte_spinlock_unlock (&dlb2->qm_instance.resource_lock); +return -EINVAL ; + } + dlb2->run_state =DLB2_RUN_STATE_STARTING ;
Looks like there's a tab between run_state and the equals sign.
Thanks,
Gage
Re: [dpdk-dev] [PATCH 15/22] event/dlb2: add enqueue and its burst variants
In reply to Timothy McDanielQuoted 10 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 15/22] event/dlb2: add enqueue and its burst variants
Quoted 3 lines
Add support for enqueue and its variants.
Signed-off-by Timothy McDaniel <[email protected]>
Thanks,
Gage
Re: [dpdk-dev] [PATCH 16/22] event/dlb2: add dequeue and its burst variants
In reply to Timothy McDanielQuoted 11 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 16/22] event/dlb2: add dequeue and its burst variantsAdd support for dequeue,
dequeue_burst, ...
Please elaborate -- this commit message doesn't mention (e.g.) the use of umonitor/umwait or the "sparse" dequeue function variants. I think the average reviewer needs some more context in order to understand this change.
[...]
drivers/event/dlb2/dlb2.c dlb2_event_enqueue_forward_burst(void *event_port, · line 2850
Quoted 12 lines
drivers/event/dlb2/dlb2.c +12
-0+ +static inline int +dlb2_process_dequeue_qes (struct dlb2_eventdev_port *ev_port, +struct dlb2_port *qm_port, +struct rte_event *events, +struct dlb2_dequeue_qe *qes, +int cnt) +{ +uint8_t *qid_mappings = qm_port->qid_mappings; +int i, num, evq_id; + +RTE_SET_USED (ev_port);/* avoids unused variable error if stats off */
Looks like ev_port is used unconditionally later on: "evq_id = ev_port->link[0].queue_id;"
Quoted 44 lines
drivers/event/dlb2/dlb2.c +6
-0+ + for (i = 0, num = 0; i < cnt; i++) { + struct dlb2_dequeue_qe *qe = &qes[i]; + int sched_type_map[DLB2_NUM_HW_SCHED_TYPES] = { + [DLB2_SCHED_ATOMIC] = RTE_SCHED_TYPE_ATOMIC, + [DLB2_SCHED_UNORDERED] =
RTE_SCHED_TYPE_PARALLEL,drivers/event/dlb2/dlb2.c +33
-0+ [DLB2_SCHED_ORDERED] = RTE_SCHED_TYPE_ORDERED, + [DLB2_SCHED_DIRECTED] = RTE_SCHED_TYPE_ATOMIC, + }; + +/* Fill in event information. +* Note that flow_id must be embedded in the data by +* the app, such as the mbuf RSS hash field if the data +* buffer is a mbuf. +*/ +if (unlikely (qe->error)) { +DLB2_LOG_ERR ("QE error bit ON\n" ); +DLB2_INC_STAT (ev_port->stats.traffic.rx_drop,1 ); +dlb2_consume_qe_immediate (qm_port,1 ); +continue ;/* Ignore */ + } + + events[num].u64 = qe->data; + events[num].flow_id = qe->flow_id; + events[num].priority =DLB2_TO_EV_PRIO ((uint8_t )qe->priority); + events[num].event_type = qe->u.event_type.major; + events[num].sub_event_type = qe->u.event_type.sub; + events[num].sched_type = sched_type_map[qe->sched_type]; + events[num].impl_opaque = qe->qid_depth; + +/* qid not preserved for directed queues */ +if (qm_port->is_directed) + evq_id = ev_port->link[0 ].queue_id; +else + evq_id = qid_mappings[qe->qid]; + + events[num].queue_id = evq_id; + DLB2_INC_STAT( + ev_port->stats.queue[evq_id].qid_depth[qe-Quoted 1 line
qid_depth],(quoted diff) +2
-0+ 1); + DLB2_INC_STAT(ev_port->stats.rx_sched_cnt[qe->sched_type],1);
drivers/event/dlb2/dlb2.c dlb2_event_enqueue_forward_burst(void *event_port, · line 2892
Quoted 1 line
drivers/event/dlb2/dlb2.c +1
-0+DLB2_INC_STAT (ev_port->stats.traffic.rx_ok,1 );
Move this outside the loop and increment by 'num' rather than '1'?
Quoted 13 lines
drivers/event/dlb2/dlb2.c +13
-0+ num++; + } + +return num; +} + +static inline int +dlb2_process_dequeue_four_qes (struct dlb2_eventdev_port *ev_port, +struct dlb2_port *qm_port, +struct rte_event *events, +struct dlb2_dequeue_qe *qes) +{ +int sched_type_map[] = {
drivers/event/dlb2/dlb2.c dlb2_event_enqueue_forward_burst(void *event_port, · line 2915
Quoted 10 lines
drivers/event/dlb2/dlb2.c +10
-0+ [DLB2_SCHED_ATOMIC] = RTE_SCHED_TYPE_ATOMIC, + [DLB2_SCHED_UNORDERED] = RTE_SCHED_TYPE_PARALLEL, + [DLB2_SCHED_ORDERED] = RTE_SCHED_TYPE_ORDERED, + [DLB2_SCHED_DIRECTED] = RTE_SCHED_TYPE_ATOMIC, + }; +const int num_events =DLB2_NUM_QES_PER_CACHE_LINE ; +uint8_t *qid_mappings = qm_port->qid_mappings; +__m128i sse_evt[2 ]; + +RTE_SET_USED (ev_port);/* avoids unused variable error, if stats off */
ev_port gets passed to dlb2_process_dequeue_qes, I don't think this line is necessary.
drivers/event/dlb2/dlb2.c dlb2_event_enqueue_forward_burst(void *event_port, · line 2923
Quoted 8 lines
drivers/event/dlb2/dlb2.c +8
-0+ +/* In the unlikely case that any of the QE error bits are set, process +* them one at a time. +*/ +if (unlikely (qes[0 ].error || qes[1 ].error || + qes[2 ].error || qes[3 ].error)) +return dlb2_process_dequeue_qes (ev_port, qm_port, events, + qes, num_events);
Thanks,
Gage
Re: [dpdk-dev] [PATCH 17/22] event/dlb2: add eventdev stop and close
In reply to Timothy McDanieldrivers/event/dlb2/dlb2.c dlb2_event_dequeue_sparse(void *event_port, struct rte_event *ev, · line 3725
Quoted 8 lines
drivers/event/dlb2/dlb2.c +8
-0+static void +dlb2_eventdev_stop (struct rte_eventdev *dev) +{ +/* FIXME: Handle the case that app threads are waiting in +* rte_event_dequeue_burst() (either blocked on interrupt or in umwait, +* or waiting with a timeout). Looking into proposing a new function to +* wake blocked threads that the app must call before stopping a device. +*/
Leftover FIXME -- otherwise looks good.
Thanks,
Gage
Re: [dpdk-dev] [PATCH 18/22] event/dlb2: add PMD's token pop public interface
In reply to Timothy McDanielQuoted 22 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
To: Ray Kinsella <[email protected]>; Neil Horman <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 18/22] event/dlb2: add PMD's token pop public interfaceThe PMD uses a public interface to allow applications to control the token pop mode. Supported token pop modes are as follows, and they impact core scheduling affinity for ldb ports.
AUTO_POP: Pop the CQ tokens immediately after dequeueing.DELAYED_POP: Pop CQ tokens after (dequeue_depth- 1) events
are released. Supported on load-balanced ports only.DEFERRED_POP: Pop the CQ tokens during next dequeue operation.
Quoted 5 lines
Signed-off-by Timothy McDaniel <[email protected]>--- drivers/event/dlb2/meson.build | 5 ++- drivers/event/dlb2/rte_pmd_dlb2.c | 39 +++++++++++++++++++++++ drivers/event/dlb2/rte_pmd_dlb2_event_version.map | 6 ++++
Should rte_pmd_dlb2.h be listed in doc/api/doxy-api-index.md, so it's included in the doxygen docs?
Besides that, and the build error related to the experimental tag mentioned in another review, this looks good to me.
Thanks,
Gage
Re: [dpdk-dev] [PATCH 19/22] event/dlb2: add PMD self-tests
In reply to Timothy McDanielQuoted 28 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
To: Jerin Jacob <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: [PATCH 19/22] event/dlb2: add PMD self-testsAdd a variety of self-tests for both ldb and directed ports/queues, as well as configure, start, stop, link, etc...
Signed-off-by Timothy McDaniel <[email protected]>--- app/test/test_eventdev.c | 9 + drivers/event/dlb2/dlb2.c | 1 + drivers/event/dlb2/dlb2_selftest.c | 1570 ++++++++++++++++++++++++++++++++++++ drivers/event/dlb2/meson.build | 3 +- 4 files changed, 1582 insertions(+), 1 deletion(-) create mode 100644 drivers/event/dlb2/dlb2_selftest.capp/test/test_eventdev.c +0
-0
app/test/test_eventdev.c test_eventdev_selftest_dpaa2(void) · line 1039
Quoted 10 lines
app/test/test_eventdev.c +7
-0@@ -1030,6 +1030,13 @@ test_eventdev_selftest_dpaa2(void)return test_eventdev_selftest_impl ("event_dpaa2" ,"" ); } +static int +test_eventdev_selftest_dlb2 (void ) +{ +return test_eventdev_selftest_impl ("dlb2_event" ,"" ); +} + +
Nit: extra newline
Quoted 10 lines
app/test/test_eventdev.c +0
-0REGISTER_TEST_COMMAND (eventdev_common_autotest, test_eventdev_common); REGISTER_TEST_COMMAND(eventdev_selftest_sw,
test_eventdev_selftest_sw);
REGISTER_TEST_COMMAND(eventdev_selftest_octeontx,app/test/test_eventdev.c +0
-0@@ -1037,3 +1044,5 @@REGISTER_TEST_COMMAND (eventdev_selftest_octeontx,REGISTER_TEST_COMMAND (eventdev_selftest_octeontx2, test_eventdev_selftest_octeontx2); REGISTER_TEST_COMMAND(eventdev_selftest_dpaa2,
Quoted 4 lines
test_eventdev_selftest_dpaa2);
+REGISTER_TEST_COMMAND(eventdev_selftest_dlb2,test_eventdev_selftest_dlb2);
+
Nit: newline at the end of the file
Quoted 4 lines
drivers/event/dlb2/dlb2.c +0
-0
[...]
drivers/event/dlb2/dlb2.c
Quoted 3 lines
drivers/event/dlb2/dlb2.c +3
-0+/* destruction */ +static inline void +cleanup(struct test *t __rte_unused)
Since 't' is unused, just drop the parameter.
With that and the whitespace issues fixed:
Thanks,
Gage
Re: [dpdk-dev] [PATCH 20/22] event/dlb2: add queue and port release
In reply to Timothy McDanieldrivers/event/dlb2/dlb2.c dlb2_eventdev_close(struct rte_eventdev *dev) · line 3767
Quoted 10 lines
static void
drivers/event/dlb2/dlb2.c +9
-0+dlb2_eventdev_queue_release(struct rte_eventdev *dev, uint8_t id) +{ +RTE_SET_USED (dev); +RTE_SET_USED (id); + +/* This function intentionally left blank. dlb2 does not support +* reconfiguring individual queues or ports -- the entire device +* must be reconfigured. +*/
I think the 2nd sentence in the comment need to be reconsidered (or just removed). The release callbacks are called (for every port and queue) when rte_event_dev_configure() is called a 2nd/3rd/etc. time to reconfigure the device, and the callbacks give the PMD the opportunity clean-up the individual port/queue. Implementing the logic can be useful for entire-device-reconfiguration.
drivers/event/dlb2/dlb2.c dlb2_eventdev_close(struct rte_eventdev *dev) · line 3778
Quoted 11 lines
drivers/event/dlb2/dlb2.c +11
-0+} + +static void +dlb2_eventdev_port_release (void *port) +{ +RTE_SET_USED (port); + +/* This function intentionally left blank. dlb2 does not support +* reconfiguring individual queues or ports -- the entire device +* must be reconfigured. +*/
This might be the appropriate place to free each port's memzone (see my comments in patch 11).
Thanks,
Gage
Re: [dpdk-dev] [PATCH 21/22] event/dlb2: add timeout ticks entry point
In reply to Timothy McDanielQuoted 10 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 21/22] event/dlb2: add timeout ticks entry point
Quoted 3 lines
Adds the timeout ticks conversion function.
Signed-off-by Timothy McDaniel <[email protected]>
Re: [dpdk-dev] [PATCH 22/22] doc: add new DLB2 eventdev driver to relnotes
In reply to Timothy McDanielQuoted 12 lines
-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
To: Mcnamara, John <[email protected]>; Kovacevic, Marko <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 22/22] doc: add new DLB2 eventdev driver to relnotes
Quoted 2 lines
Added announcement of availabililty for the new driver for Intel Dynamic Load Balancer V2.0 hardware.
There's some inconsistency in the version numbering (2.0, V2.0, v2.0) in this series -- I believe 2.0 is the marketing-approved one.
Quoted 12 lines
Signed-off-by Timothy McDaniel <[email protected]>--- doc/guides/rel_notes/release_20_11.rst | 5 +++++ 1 file changed, 5 insertions(+)doc/guides/rel_notes/release_20_11.rst +0
-0
doc/guides/rel_notes/release_20_11.rst New Features · line 61
Quoted 7 lines
doc/guides/rel_notes/release_20_11.rst +4
-0@@ -55,6 +55,11 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Added a new driver for the Intel Dynamic Load Balancer v2.0 device.** + + Added the new ``dlb2`` eventdev driver for the Intel DLB V2.0 device. See the + :doc:`../eventdevs/dlb2` eventdev guide for more details on this new driver.
Looks like the guide didn't make it into this series ;)
Please also add an entry MAINTAINERS entry for the PMD.
Thanks,
Gage
v2 cover Add DLB2 PMD
In reply to Timothy McDanielThe following patch series adds support for a new eventdev PMD. The DLB2 PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2) hardware.
The DLB2 is a PCIe device that provides load-balanced, prioritized scheduling of core-to-core communication. The device consists of queues and arbiters that connect producer and consumer cores, and implements load-balanced queueing features including:
- Lock-free multi-producer/multi-consumer operation.
- Multiple priority levels for varying traffic types.
- 'Direct' traffic (i.e. multi-producer/single-consumer)
- Simple unordered load-balanced distribution.
- Atomic lock-free load balancing across multiple consumers.
- Queue element reordering feature allowing ordered load-balanced distribution.
The DLB2 hardware supports both load balanced and directed ports and queues. Unlike other eventdev devices already in the repo, not all DLB2 ports and queues are equally capable. In particular, directed ports are limited to a single link, and must be connected to a directed queue. Additionally, even though LDB ports may link multiple queues, the number of queues that may be linked is limited by hardware.
While reviewing the code, please be aware that this PMD has full control over the DLB2 hardware. Intel will be extending the DLB2 PMD in the future (not as part of this first series) with a mode that we refer to as the bifurcated PMD. The bifurcated PMD communicates with a kernel driver to configure the device, ports, and queues, and memory maps device MMIO so datapath operations occur purely in user-space. Note that the DLB2 hardware is a successor of the DLB hardware, and as such is structured similarly, both in terms of code layout and implementation.
The framework to support both the PF PMD and bifurcated PMD exists in this patchset, and is why the iface.[ch] layer is present.
Changes since V1:
- implement changes requested in code reviews by Gage Eads and Mike Chen
- fix a memzone leak
- convert to use eal rte-cpuflags patch from Liang Ma
Known Issues:
- the documentation contained in dlb2.rst is not complete, and should be updated to include command line parameters (class of service, etc ...).
Depends-on: patch-79539 ("eal: add new x86 cpuid support for WAITPKG")
Timothy McDaniel (22):
event/dlb2: add documentation and meson build infrastructure event/dlb2: add dynamic logging
event/dlb2: add private data structures and constants event/dlb2: add definitions shared with LKM or shared code event/dlb2: add inline functions
event/dlb2: add probe
event/dlb2: add xstats
event/dlb2: add infos get and configure event/dlb2: add queue and port default conf event/dlb2: add queue setup
event/dlb2: add port setup
event/dlb2: add port link
event/dlb2: add port unlink and port unlinks in progress event/dlb2: add eventdev start
event/dlb2: add enqueue and its burst variants event/dlb2: add dequeue and its burst variants event/dlb2: add eventdev stop and close event/dlb2: add PMD's token pop public interface event/dlb2: add PMD self-tests
event/dlb2: add queue and port release
event/dlb2: add timeout ticks entry point
doc: add new DLB2 eventdev driver to relnotes
MAINTAINERS | 5 + app/test/test_eventdev.c | 7 + config/rte_config.h | 7 + doc/api/doxy-api-index.md | 1 + doc/guides/eventdevs/dlb2.rst | 330 ++ doc/guides/eventdevs/index.rst | 1 + doc/guides/rel_notes/release_20_11.rst | 5 + drivers/event/dlb2/dlb2.c | 3955 ++++++++++++++ drivers/event/dlb2/dlb2_iface.c | 74 + drivers/event/dlb2/dlb2_iface.h | 75 + drivers/event/dlb2/dlb2_inline_fns.h | 81 + drivers/event/dlb2/dlb2_log.h | 25 + drivers/event/dlb2/dlb2_priv.h | 578 ++ drivers/event/dlb2/dlb2_selftest.c | 1570 ++++++ drivers/event/dlb2/dlb2_user.h | 679 +++ drivers/event/dlb2/dlb2_xstats.c | 1240 +++++ drivers/event/dlb2/meson.build | 16 + drivers/event/dlb2/pf/base/dlb2_hw_types.h | 367 ++ drivers/event/dlb2/pf/base/dlb2_mbox.h | 596 ++ drivers/event/dlb2/pf/base/dlb2_osdep.h | 247 + drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h | 440 ++ drivers/event/dlb2/pf/base/dlb2_osdep_list.h | 131 + drivers/event/dlb2/pf/base/dlb2_osdep_types.h | 31 + drivers/event/dlb2/pf/base/dlb2_regs.h | 2527 +++++++++ drivers/event/dlb2/pf/base/dlb2_resource.c | 6024 +++++++++++++++++++++ drivers/event/dlb2/pf/base/dlb2_resource.h | 1913 +++++++ drivers/event/dlb2/pf/dlb2_main.c | 679 +++ drivers/event/dlb2/pf/dlb2_main.h | 104 + drivers/event/dlb2/pf/dlb2_pf.c | 729 +++ drivers/event/dlb2/rte_pmd_dlb2.c | 39 + drivers/event/dlb2/rte_pmd_dlb2.h | 68 + drivers/event/dlb2/rte_pmd_dlb2_event_version.map | 9 + drivers/event/meson.build | 3 + 33 files changed, 22556 insertions(+) create mode 100644 doc/guides/eventdevs/dlb2.rst create mode 100644 drivers/event/dlb2/dlb2.c create mode 100644 drivers/event/dlb2/dlb2_iface.c create mode 100644 drivers/event/dlb2/dlb2_iface.h create mode 100644 drivers/event/dlb2/dlb2_inline_fns.h create mode 100644 drivers/event/dlb2/dlb2_log.h create mode 100644 drivers/event/dlb2/dlb2_priv.h create mode 100644 drivers/event/dlb2/dlb2_selftest.c create mode 100644 drivers/event/dlb2/dlb2_user.h create mode 100644 drivers/event/dlb2/dlb2_xstats.c create mode 100644 drivers/event/dlb2/meson.build create mode 100644 drivers/event/dlb2/pf/base/dlb2_hw_types.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_mbox.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_list.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_types.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_regs.h create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.c create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.h create mode 100644 drivers/event/dlb2/pf/dlb2_main.c create mode 100644 drivers/event/dlb2/pf/dlb2_main.h create mode 100644 drivers/event/dlb2/pf/dlb2_pf.c create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.c create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.h create mode 100644 drivers/event/dlb2/rte_pmd_dlb2_event_version.map
Signature · 1 line
2.6.4
PATCH v2 1/22 event/dlb2: add documentation and meson build infrastructure
Adds the meson build infrastructure, which includes compile-time constants in rte_config.h. DLB2 is only supported on Linux X86 platforms at this time.
--- config/rte_config.h | 7 + doc/guides/eventdevs/dlb2.rst | 330 ++++++++++++++++++++++ doc/guides/eventdevs/index.rst | 1 + drivers/event/dlb2/meson.build | 7 + drivers/event/dlb2/rte_pmd_dlb2_event_version.map | 3 + drivers/event/meson.build | 3 + 6 files changed, 351 insertions(+) create mode 100644 doc/guides/eventdevs/dlb2.rst create mode 100644 drivers/event/dlb2/meson.build create mode 100644 drivers/event/dlb2/rte_pmd_dlb2_event_version.map
config/rte_config.h +7 -0
@@ -131,4 +131,11 @@/* QEDE PMD defines */ #define RTE_LIBRTE_QEDE_FW "" +/* DLB2 defines */ +#define RTE_LIBRTE_PMD_DLB2_POLL_INTERVAL 1000 +#define RTE_LIBRTE_PMD_DLB2_UMWAIT_CTL_STATE 0 +#undef RTE_LIBRTE_PMD_DLB2_QUELL_STATS +#define RTE_LIBRTE_PMD_DLB2_SW_CREDIT_QUANTA 32 +#define RTE_PMD_DLB2_DEFAULT_DEPTH_THRESH 256 +#endif /* _RTE_CONFIG_H_ */
doc/guides/eventdevs/dlb2.rst new file +330 -0
330 lines not on this page · raw message
doc/guides/eventdevs/index.rst +1 -0
@@ -18,3 +18,4 @@ application through the eventdev API. octeontx octeontx2 opdl + dlb2
drivers/event/dlb2/meson.build new file +7 -0
@@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019-2020 Intel Corporation + +sources = files( +) + +deps += ['mbuf', 'mempool', 'ring', 'pci', 'bus_pci']
drivers/event/dlb2/rte_pmd_dlb2_event_version.map new file +3 -0
@@ -0,0 +1,3 @@ +DPDK_21.0 { + local: *; +};
drivers/event/meson.build +3 -0
@@ -10,6 +10,9 @@ if not (toolchain == 'gcc' and cc.version().version_compare('<4.8.6') and dpdk_conf.has('RTE_ARCH_ARM64')) drivers += 'octeontx' endif +if (dpdk_conf.has('RTE_ARCH_X86_64') and is_linux) + drivers += 'dlb2' +endif std_deps = ['eventdev', 'kvargs'] config_flag_fmt = 'RTE_LIBRTE_@0@_EVENTDEV_PMD' driver_name_fmt = 'rte_pmd_@0@_event'
Signature · 1 line
2.6.4
PATCH v2 2/22 event/dlb2: add dynamic logging
In reply to Timothy McDanielThis commit adds base support for dynamic logging. The default log level is NOTICE. Dynamic logging is used exclusively throughout this patchset.
--- drivers/event/dlb2/dlb2_log.h | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 drivers/event/dlb2/dlb2_log.h
drivers/event/dlb2/dlb2_log.h new file +25 -0
@@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2016-2020 Intel Corporation +*/ + +#ifndef _DLB2_EVDEV_LOG_H_ +#define _DLB2_EVDEV_LOG_H_ + +extern int eventdev_dlb2_log_level; + +/* Dynamic logging */ +#define DLB2_LOG_IMPL(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, eventdev_dlb2_log_level, "%s" fmt "\n", \ + __func__, ##args) + +#define DLB2_LOG_INFO(fmt, args...) \ + DLB2_LOG_IMPL(INFO, fmt, ## args) + +#define DLB2_LOG_ERR(fmt, args...) \ + DLB2_LOG_IMPL(ERR, fmt, ## args) + +/* remove debug logs at compile time unless actually debugging */ +#define DLB2_LOG_DBG(fmt, args...) \ + RTE_LOG_DP(DEBUG, PMD, fmt, ## args) + +#endif /* _DLB2_EVDEV_LOG_H_ */
Signature · 1 line
2.6.4