iecm/idpf series cover letter
Subsystems (4)
cover iecm/idpf series cover letter
This series introduces both the Intel Ethernet Common Module and the Intel Data Plane Function. The patches also incorporate extended features and functionality added in the virtchnl.h file.
The format of the series flow is to add the data set, then introduce function stubs, and then introduce pieces in large cohesive subjects or functionality. This is to allow for more in depth understanding and review of the bigger picture as the series is reviewed.
Patch 13 adds the common module to the kernel build system, and patch 14 adds the idpf driver which utilizes the common module.
Patches:
idpf: Introduce idpf driver
iecm: Add framework set of header files
iecm: Add TX/RX header files
iecm: Common module introduction and function stubs
iecm: Add basic netdevice functionality
iecm: Implement mailbox functionality
iecm: Implement virtchnl commands
iecm: Implement vector allocation
iecm: Init and allocate vport
iecm: Deinit vport
iecm: Add splitq TX/RX
iecm: Add singleq TX/RX
iecm: Add ethtool
iecm: Add iecm to the kernel build system
Work done by:
.../networking/device_drivers/intel/idpf.rst | 47 + .../networking/device_drivers/intel/iecm.rst | 91 + MAINTAINERS | 2 + drivers/net/ethernet/intel/Kconfig | 15 + drivers/net/ethernet/intel/Makefile | 2 + drivers/net/ethernet/intel/idpf/Makefile | 14 + drivers/net/ethernet/intel/idpf/idpf_dev.h | 18 + drivers/net/ethernet/intel/idpf/idpf_devids.h | 10 + drivers/net/ethernet/intel/idpf/idpf_main.c | 139 + drivers/net/ethernet/intel/idpf/idpf_reg.c | 152 + drivers/net/ethernet/intel/iecm/Makefile | 21 + .../net/ethernet/intel/iecm/iecm_controlq.c | 673 +++ .../ethernet/intel/iecm/iecm_controlq_setup.c | 177 + .../net/ethernet/intel/iecm/iecm_ethtool.c | 1119 +++++ drivers/net/ethernet/intel/iecm/iecm_lib.c | 1092 +++++ drivers/net/ethernet/intel/iecm/iecm_main.c | 51 + drivers/net/ethernet/intel/iecm/iecm_osdep.c | 28 + .../ethernet/intel/iecm/iecm_singleq_txrx.c | 890 ++++ drivers/net/ethernet/intel/iecm/iecm_txrx.c | 3959 +++++++++++++++++ .../net/ethernet/intel/iecm/iecm_virtchnl.c | 2231 ++++++++++ drivers/net/ethernet/intel/include/iecm.h | 432 ++ .../net/ethernet/intel/include/iecm_alloc.h | 29 + .../ethernet/intel/include/iecm_controlq.h | 95 + .../intel/include/iecm_controlq_api.h | 223 + .../ethernet/intel/include/iecm_lan_pf_regs.h | 114 + .../ethernet/intel/include/iecm_lan_txrx.h | 636 +++ .../net/ethernet/intel/include/iecm_osdep.h | 29 + .../net/ethernet/intel/include/iecm_txrx.h | 610 +++ .../net/ethernet/intel/include/iecm_type.h | 47 + 29 files changed, 12946 insertions(+) create mode 100644 Documentation/networking/device_drivers/intel/idpf.rst create mode 100644 Documentation/networking/device_drivers/intel/iecm.rst create mode 100644 drivers/net/ethernet/intel/idpf/Makefile create mode 100644 drivers/net/ethernet/intel/idpf/idpf_dev.h create mode 100644 drivers/net/ethernet/intel/idpf/idpf_devids.h create mode 100644 drivers/net/ethernet/intel/idpf/idpf_main.c create mode 100644 drivers/net/ethernet/intel/idpf/idpf_reg.c create mode 100644 drivers/net/ethernet/intel/iecm/Makefile create mode 100644 drivers/net/ethernet/intel/iecm/iecm_controlq.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_controlq_setup.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_ethtool.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_lib.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_main.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_osdep.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_singleq_txrx.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_txrx.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_virtchnl.c create mode 100644 drivers/net/ethernet/intel/include/iecm.h create mode 100644 drivers/net/ethernet/intel/include/iecm_alloc.h create mode 100644 drivers/net/ethernet/intel/include/iecm_controlq.h create mode 100644 drivers/net/ethernet/intel/include/iecm_controlq_api.h create mode 100644 drivers/net/ethernet/intel/include/iecm_lan_pf_regs.h create mode 100644 drivers/net/ethernet/intel/include/iecm_lan_txrx.h create mode 100644 drivers/net/ethernet/intel/include/iecm_osdep.h create mode 100644 drivers/net/ethernet/intel/include/iecm_txrx.h create mode 100644 drivers/net/ethernet/intel/include/iecm_type.h
Signature · 1 line
2.21.0
PATCH 1/14 iecm: Add framework set of header files
Introduces the framework of data for the driver common module.
--- drivers/net/ethernet/intel/include/iecm.h | 432 ++++++++++++++++++ .../net/ethernet/intel/include/iecm_alloc.h | 29 ++ .../ethernet/intel/include/iecm_controlq.h | 95 ++++ .../intel/include/iecm_controlq_api.h | 221 +++++++++ .../net/ethernet/intel/include/iecm_osdep.h | 24 + .../net/ethernet/intel/include/iecm_type.h | 47 ++ 6 files changed, 848 insertions(+) create mode 100644 drivers/net/ethernet/intel/include/iecm.h create mode 100644 drivers/net/ethernet/intel/include/iecm_alloc.h create mode 100644 drivers/net/ethernet/intel/include/iecm_controlq.h create mode 100644 drivers/net/ethernet/intel/include/iecm_controlq_api.h create mode 100644 drivers/net/ethernet/intel/include/iecm_osdep.h create mode 100644 drivers/net/ethernet/intel/include/iecm_type.h
drivers/net/ethernet/intel/include/iecm.h new file +432 -0
@@ -0,0 +1,432 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright (C) 2020 Intel Corporation */ + +#ifndef _IECM_H_ +#define _IECM_H_ + +#include <linux/aer.h> +#include <linux/bitmap.h> +#include <linux/compiler.h> +#include <linux/cpumask.h> +#include <linux/delay.h> +#include <linux/dma-mapping.h> +#include <linux/errno.h> +#include <linux/etherdevice.h> +#include <linux/ethtool.h> +#include <linux/if_bridge.h> +#include <linux/if_vlan.h> +#include <linux/interrupt.h> +#include <linux/ip.h> +#include <linux/ipv6.h> +#include <linux/jiffies.h> +#include <linux/kernel.h> +#include <linux/log2.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/netdevice.h> +#include <linux/pci.h> +#include <linux/pkt_sched.h> +#include <linux/rtnetlink.h> +#include <linux/sctp.h> +#include <linux/skbuff.h> +#include <linux/string.h> +#include <linux/timer.h> +#include <linux/types.h> +#include <linux/workqueue.h> +#include <net/tcp.h> +#include <net/ipv6.h> +#include <net/ip6_checksum.h> +#include <linux/prefetch.h> +#include <net/pkt_sched.h> + +#include <net/gre.h> +#include <net/udp_tunnel.h> +#include <linux/avf/virtchnl.h> + +#include "iecm_osdep.h" +#include "iecm_controlq_api.h" +#include "iecm_lan_txrx.h" + +#include "iecm_txrx.h" +#include "iecm_type.h" + +extern const char iecm_drv_ver[]; +extern char iecm_drv_name[]; + +#define IECM_BAR0 0 +#define IECM_NO_FREE_SLOT 0xffff + +/* Default Mailbox settings */ +#define IECM_DFLT_MBX_BUF_SIZE (1 * 1024) +#define IECM_NUM_QCTX_PER_MSG 3 +#define IECM_DFLT_MBX_Q_LEN 64 +#define IECM_DFLT_MBX_ID -1 +/* maximum number of times to try before resetting mailbox */ +#define IECM_MB_MAX_ERR 20 + +#define IECM_MAX_NUM_VPORTS 1 + +/* default message levels */ +#define IECM_DFLT_NETIF_M (NETIF_MSG_DRV | NETIF_MSG_PROBE | NETIF_MSG_LINK) + +/* Forward declaration */ +struct iecm_adapter ; + +enum iecm_state { + __IECM_STARTUP, + __IECM_VER_CHECK, + __IECM_GET_RES, + __IECM_GET_CAPS, + __IECM_GET_DFLT_VPORT_PARAMS, + __IECM_INIT_SW, + __IECM_DOWN, + __IECM_UP, + __IECM_REMOVE, + __IECM_STATE_LAST/* this member MUST be last */ +}; + +enum iecm_flags { +/* Soft reset causes */ + __IECM_SR_Q_CHANGE,/* Soft reset to do queue change */ + __IECM_SR_Q_DESC_CHANGE, + __IECM_SR_Q_SCH_CHANGE,/* Scheduling mode change in queue context */ + __IECM_SR_MTU_CHANGE, + __IECM_SR_TC_CHANGE, +/* Hard reset causes */ + __IECM_HR_FUNC_RESET,/* Hard reset when txrx timeout */ + __IECM_HR_CORE_RESET,/* when reset event is received on virtchannel */ + __IECM_HR_DRV_LOAD,/* Set on driver load for a clean HW */ +/* Generic bits to share a message */ + __IECM_DEL_QUEUES, + __IECM_UP_REQUESTED,/* Set if open to be called explicitly by driver */ +/* Mailbox interrupt event */ + __IECM_MB_INTR_MODE, + __IECM_MB_INTR_TRIGGER, +/* must be last */ + __IECM_FLAGS_NBITS, +}; + +struct iecm_netdev_priv { +struct iecm_vport *vport; +}; + +struct iecm_reset_reg { +u32 rstat; +u32 rstat_m; +}; + +/* product specific register API */ +struct iecm_reg_ops { +void (*ctlq_reg_init)(struct iecm_ctlq_create_info *cq); +void (*vportq_reg_init)(struct iecm_vport *vport); +void (*intr_reg_init)(struct iecm_vport *vport); +void (*mb_intr_reg_init)(struct iecm_adapter *adapter); +void (*reset_reg_init)(struct iecm_reset_reg *reset_reg); +void (*trigger_reset)(struct iecm_adapter *adapter, +enum iecm_flags trig_cause); +}; + +struct iecm_virtchnl_ops { +int (*core_init)(struct iecm_adapter *adapter,int *vport_id); +void (*vport_init)(struct iecm_vport *vport,int vport_id); +enum iecm_status (*vport_queue_ids_init)(struct iecm_vport *vport); +enum iecm_status (*get_caps)(struct iecm_adapter *adapter); +enum iecm_status (*config_queues)(struct iecm_vport *vport); +enum iecm_status (*enable_queues)(struct iecm_vport *vport); +enum iecm_status (*disable_queues)(struct iecm_vport *vport); +enum iecm_status (*irq_map_unmap)(struct iecm_vport *vport,bool map); +enum iecm_status (*enable_vport)(struct iecm_vport *vport); +enum iecm_status (*disable_vport)(struct iecm_vport *vport); +enum iecm_status (*destroy_vport)(struct iecm_vport *vport); +enum iecm_status (*get_ptype)(struct iecm_vport *vport); +enum iecm_status (*get_set_rss_lut)(struct iecm_vport *vport,bool get); +enum iecm_status (*get_set_rss_hash)(struct iecm_vport *vport, +bool get); +enum iecm_status (*adjust_qs)(struct iecm_vport *vport); +enum iecm_status (*recv_mbx_msg)(struct iecm_adapter *adapter, +void *msg,int msg_size, +struct iecm_ctlq_msg *ctlq_msg, +bool *work_done); +enum iecm_status (*alloc_vectors)(struct iecm_adapter *adapter, +u16 num_vectors); +enum iecm_status (*dealloc_vectors)(struct iecm_adapter *adapter); +bool (*is_cap_ena)(struct iecm_adapter *adapter,u64 flag); +}; + +struct iecm_dev_ops { +void (*reg_ops_init)(struct iecm_adapter *adapter); +void (*vc_ops_init)(struct iecm_adapter *adapter); +void (*crc_enable)(u64 *td_cmd); +struct iecm_reg_ops reg_ops; +struct iecm_virtchnl_ops vc_ops; +}; + +/* vport specific data structure */ + +enum iecm_vport_vc_state { +IECM_VC_ENA_VPORT , +IECM_VC_ENA_VPORT_ERR , +IECM_VC_DIS_VPORT , +IECM_VC_DIS_VPORT_ERR , +IECM_VC_DESTROY_VPORT , +IECM_VC_DESTROY_VPORT_ERR , +IECM_VC_CONFIG_TXQ , +IECM_VC_CONFIG_TXQ_ERR , +IECM_VC_CONFIG_RXQ , +IECM_VC_CONFIG_RXQ_ERR , +IECM_VC_CONFIG_Q , +IECM_VC_CONFIG_Q_ERR , +IECM_VC_ENA_QUEUES , +IECM_VC_ENA_QUEUES_ERR , +IECM_VC_DIS_QUEUES , +IECM_VC_DIS_QUEUES_ERR , +IECM_VC_MAP_IRQ , +IECM_VC_MAP_IRQ_ERR , +IECM_VC_UNMAP_IRQ , +IECM_VC_UNMAP_IRQ_ERR , +IECM_VC_ADD_QUEUES , +IECM_VC_ADD_QUEUES_ERR , +IECM_VC_DEL_QUEUES , +IECM_VC_DEL_QUEUES_ERR , +IECM_VC_ALLOC_VECTORS , +IECM_VC_ALLOC_VECTORS_ERR , +IECM_VC_DEALLOC_VECTORS , +IECM_VC_DEALLOC_VECTORS_ERR , +IECM_VC_CREATE_VFS , +IECM_VC_CREATE_VFS_ERR , +IECM_VC_DESTROY_VFS , +IECM_VC_DESTROY_VFS_ERR , +IECM_VC_GET_RSS_HASH , +IECM_VC_GET_RSS_HASH_ERR , +IECM_VC_SET_RSS_HASH , +IECM_VC_SET_RSS_HASH_ERR , +IECM_VC_GET_RSS_LUT , +IECM_VC_GET_RSS_LUT_ERR , +IECM_VC_SET_RSS_LUT , +IECM_VC_SET_RSS_LUT_ERR , +IECM_VC_GET_RSS_KEY , +IECM_VC_GET_RSS_KEY_ERR , +IECM_VC_CONFIG_RSS_KEY , +IECM_VC_CONFIG_RSS_KEY_ERR , +IECM_VC_GET_STATS , +IECM_VC_GET_STATS_ERR , +IECM_VC_NBITS +}; + +enum iecm_vport_flags { + __IECM_VPORT_SW_MARKER, + __IECM_VPORT_FLAGS_NBITS, +}; + +struct iecm_vport { +/* TX */ +unsigned int num_txq; +unsigned int num_complq; +/* It makes more sense for descriptor count to be part of only iecm +* queue structure. But when user changes the count via ethtool, driver +* has to store that value somewhere other than queue structure as the +* queues will be freed and allocated again. +*/ +unsigned int txq_desc_count; +unsigned int complq_desc_count; +unsigned int compln_clean_budget; +unsigned int num_txq_grp; +struct iecm_txq_group *txq_grps; +enum virtchnl_queue_model txq_model; +/* Used only in hotpath to get to the right queue very fast */ +struct iecm_queue **txqs; +wait_queue_head_t sw_marker_wq; + DECLARE_BITMAP(flags, __IECM_VPORT_FLAGS_NBITS); + +/* RX */ +unsigned int num_rxq; +unsigned int num_bufq; +unsigned int rxq_desc_count; +unsigned int bufq_desc_count; +unsigned int num_rxq_grp; +struct iecm_rxq_group *rxq_grps; +enum virtchnl_queue_model rxq_model; +struct iecm_rx_ptype_decoded rx_ptype_lkup[IECM_RX_MAX_PTYPE ]; + +struct iecm_adapter *adapter; +struct net_device *netdev; +u16 vport_type; +u16 vport_id; +u16 idx;/* software index in adapter vports struct */ +struct rtnl_link_stats64 netstats; + +/* handler for hard interrupt */ +irqreturn_t (*irq_q_handler)(int irq,void *data); +struct iecm_q_vector *q_vectors;/* q vector array */ +u16 num_q_vectors; +u16 q_vector_base; +u16 max_mtu; +u8 default_mac_addr[ETH_ALEN ]; +u16 qset_handle; +/* Duplicated in queue structure for performance reasons */ +enum iecm_rx_hsplit rx_hsplit_en; +}; + +/* User defined configuration values */ +struct iecm_user_config_data { +u32 num_req_qs;/* user requested queues through ethtool */ +u32 num_req_txq_desc; +u32 num_req_rxq_desc; +void *req_qs_chunks; +}; + +struct iecm_rss_data { +u64 rss_hash; +u16 rss_key_size; +u8 *rss_key; +u16 rss_lut_size; +u8 *rss_lut; +}; + +struct iecm_adapter { +struct pci_dev *pdev; + +u32 tx_timeout_count; +u32 msg_enable; +enum iecm_state state; + DECLARE_BITMAP(flags, __IECM_FLAGS_NBITS); +struct mutex reset_lock;/* lock to protect reset flows */ +struct iecm_reset_reg reset_reg; +struct iecm_hw hw; + +u16 num_req_msix; +u16 num_msix_entries; +struct msix_entry *msix_entries; +struct virtchnl_alloc_vectors *req_vec_chunks; +struct iecm_q_vector mb_vector; +/* handler for hard interrupt for mailbox*/ +irqreturn_t (*irq_mb_handler)(int irq,void *data); + +/* vport structs */ +struct iecm_vport **vports;/* vports created by the driver */ +u16 num_alloc_vport; +u16 next_vport;/* Next free slot in pf->vport[] - 0-based! */ +struct mutex sw_mutex;/* lock to protect vport alloc flow */ + +struct delayed_work init_task;/* delayed init task */ +struct workqueue_struct *init_wq; +u32 mb_wait_count; +struct delayed_work serv_task;/* delayed service task */ +struct workqueue_struct *serv_wq; +struct delayed_work stats_task;/* delayed statistics task */ +struct workqueue_struct *stats_wq; +struct delayed_work vc_event_task;/* delayed virtchannel event task */ +struct workqueue_struct *vc_event_wq; +/* Store the resources data received from control plane */ +void **vport_params_reqd; +void **vport_params_recvd; +/* User set parameters */ +struct iecm_user_config_data config_data; +void *caps; +wait_queue_head_t vchnl_wq; + DECLARE_BITMAP(vc_state, IECM_VC_NBITS); +struct mutex vc_msg_lock;/* lock to protect vc_msg flow */ +char vc_msg[IECM_DFLT_MBX_BUF_SIZE ]; +struct iecm_rss_data rss_data; +struct iecm_dev_ops dev_ops; +enum virtchnl_link_speed link_speed; +bool link_up; +}; + +/** +* iecm_is_queue_model_split - check if queue model is split +* @q_model: queue model single or split +* +* Returns true if queue model is split else false +*/ +static inline int iecm_is_queue_model_split (enum virtchnl_queue_model q_model) +{ +return (q_model ==VIRTCHNL_QUEUE_MODEL_SPLIT ); +} + +/** +* iecm_is_cap_ena - Determine if HW capability is supported +* @adapter: private data struct +* @flag: Feature flag to check +*/ +static inline bool iecm_is_cap_ena (struct iecm_adapter *adapter,u64 flag) +{ +return adapter->dev_ops.vc_ops.is_cap_ena (adapter, flag); +} + +/** +* iecm_is_feature_ena - Determine if a particular feature is enabled +* @vport: vport to check +* @feature: netdev flag to check +* +* Returns true or flase if a particular feature is enabled. +*/ +static inline bool iecm_is_feature_ena (struct iecm_vport *vport, +netdev_features_t feature) +{ +return vport->netdev->features & feature; +} + +/** +* iecm_rx_offset - Return expected offset into page to access data +* @rx_q: queue we are requesting offset of +* +* Returns the offset value for queue into the data buffer. +*/ +static inline unsigned int +iecm_rx_offset (struct iecm_queue __maybe_unused *rx_q) +{ +return 0 ; +} + +int iecm_probe (struct pci_dev *pdev, +const struct pci_device_id __always_unused *ent, +struct iecm_adapter *adapter); +void iecm_remove (struct pci_dev *pdev); +void iecm_shutdown (struct pci_dev *pdev); +enum iecm_status iecm_vport_adjust_qs (struct iecm_vport *vport); +enum iecm_status +iecm_ctlq_reg_init (struct iecm_ctlq_create_info *cq,int num_q); +enum iecm_status iecm_init_dflt_mbx (struct iecm_adapter *adapter); +void iecm_deinit_dflt_mbx (struct iecm_adapter *adapter); +void iecm_vc_ops_init (struct iecm_adapter *adapter); +int iecm_vc_core_init (struct iecm_adapter *adapter,int *vport_id); +enum iecm_status +iecm_wait_for_event (struct iecm_adapter *adapter, +enum iecm_vport_vc_state state, +enum iecm_vport_vc_state err_check); +enum iecm_status iecm_send_get_caps_msg (struct iecm_adapter *adapter); +enum iecm_status iecm_send_delete_queues_msg (struct iecm_vport *vport); +enum iecm_status +iecm_send_add_queues_msg (struct iecm_vport *vport,u16 num_tx_q, +u16 num_complq,u16 num_rx_q,u16 num_rx_bufq); +enum iecm_status iecm_initiate_soft_reset (struct iecm_vport *vport, +enum iecm_flags reset_cause); +enum iecm_status iecm_send_config_tx_queues_msg (struct iecm_vport *vport); +enum iecm_status iecm_send_config_rx_queues_msg (struct iecm_vport *vport); +enum iecm_status iecm_send_enable_vport_msg (struct iecm_vport *vport); +enum iecm_status iecm_send_disable_vport_msg (struct iecm_vport *vport); +enum iecm_status iecm_send_destroy_vport_msg (struct iecm_vport *vport); +enum iecm_status iecm_send_get_rx_ptype_msg (struct iecm_vport *vport); +enum iecm_status +iecm_send_get_set_rss_key_msg (struct iecm_vport *vport,bool get); +enum iecm_status +iecm_send_get_set_rss_lut_msg (struct iecm_vport *vport,bool get); +enum iecm_status +iecm_send_get_set_rss_hash_msg (struct iecm_vport *vport,bool get); +int iecm_vport_params_buf_alloc (struct iecm_adapter *adapter); +void iecm_vport_params_buf_rel (struct iecm_adapter *adapter); +struct iecm_vport *iecm_netdev_to_vport (struct net_device *netdev); +struct iecm_adapter *iecm_netdev_to_adapter (struct net_device *netdev); +enum iecm_status iecm_send_get_stats_msg (struct iecm_vport *vport); +int iecm_vport_get_vec_ids (u16 *vecids,int num_vecids, +struct virtchnl_vector_chunks *chunks); +enum iecm_status +iecm_recv_mb_msg (struct iecm_adapter *adapter,enum virtchnl_ops op, +void *msg,int msg_size); +enum iecm_status +iecm_send_mb_msg (struct iecm_adapter *adapter,enum virtchnl_ops op, +u16 msg_size,u8 *msg); +void iecm_set_ethtool_ops (struct net_device *netdev); +void iecm_vport_set_hsplit (struct iecm_vport *vport,struct bpf_prog *prog); +#endif /* !_IECM_H_ */
drivers/net/ethernet/intel/include/iecm_alloc.h new file +29 -0
@@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2020, Intel Corporation. */ + +#ifndef _IECM_ALLOC_H_ +#define _IECM_ALLOC_H_ + +/* Memory types */ +enum iecm_memset_type { +IECM_NONDMA_MEM =0 , +IECM_DMA_MEM +}; + +/* Memcpy types */ +enum iecm_memcpy_type { +IECM_NONDMA_TO_NONDMA =0 , +IECM_NONDMA_TO_DMA , +IECM_DMA_TO_DMA , +IECM_DMA_TO_NONDMA +}; + +struct iecm_hw ; +struct iecm_dma_mem ; + +/* prototype for functions used for dynamic memory allocation */ +void *iecm_alloc_dma_mem (struct iecm_hw *hw,struct iecm_dma_mem *mem, +u64 size); +void iecm_free_dma_mem (struct iecm_hw *hw,struct iecm_dma_mem *mem); + +#endif /* _IECM_ALLOC_H_ */
drivers/net/ethernet/intel/include/iecm_controlq.h new file +95 -0
95 lines not on this page · raw message
drivers/net/ethernet/intel/include/iecm_controlq_api.h new file +221 -0
221 lines not on this page · raw message
drivers/net/ethernet/intel/include/iecm_osdep.h new file +24 -0
@@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2020, Intel Corporation. */ + +#ifndef _IECM_OSDEP_H_ +#define _IECM_OSDEP_H_ + +#include <linux/bitops.h> +#include <linux/delay.h> +#include <linux/dma-mapping.h> +#include "iecm_alloc.h" + +#define wr32 (a, reg, value) writel((value), (u8 *)((a)->hw_addr + (reg))) +#define rd32 (a, reg) readl((u8 *)(a)->hw_addr + (reg)) +#define wr64 (a, reg, value) writeq((value), (u8 *)((a)->hw_addr + (reg))) +#define rd64 (a, reg) readq((u8 *)(a)->hw_addr + (reg)) + +struct iecm_dma_mem { +void *va; +dma_addr_t pa; +size_t size; +}; + +#define iecm_wmb () wmb()/* memory barrier */ +#endif /* _IECM_OSDEP_H_ */
drivers/net/ethernet/intel/include/iecm_type.h new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH 2/14 iecm: Add TX/RX header files
In reply to Alice MichaelIntroduces the data for the TX/RX paths for use
by the common module.
--- .../ethernet/intel/include/iecm_lan_pf_regs.h | 120 ++++ .../ethernet/intel/include/iecm_lan_txrx.h | 636 ++++++++++++++++++ .../net/ethernet/intel/include/iecm_txrx.h | 610 +++++++++++++++++ 3 files changed, 1366 insertions(+) create mode 100644 drivers/net/ethernet/intel/include/iecm_lan_pf_regs.h create mode 100644 drivers/net/ethernet/intel/include/iecm_lan_txrx.h create mode 100644 drivers/net/ethernet/intel/include/iecm_txrx.h
drivers/net/ethernet/intel/include/iecm_lan_pf_regs.h new file +120 -0
@@ -0,0 +1,120 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2020, Intel Corporation. */ + +#ifndef _IECM_LAN_PF_REGS_H_ +#define _IECM_LAN_PF_REGS_H_ + +/* Receive queues */ +#define PF_QRX_BASE 0x00000000 +#define PF_QRX_TAIL (_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000))) +#define PF_QRX_BUFFQ_BASE 0x03000000 +#define PF_QRX_BUFFQ_TAIL (_QRX) \ + (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000))) + +/* Transmit queues */ +#define PF_QTX_BASE 0x05000000 +#define PF_QTX_COMM_DBELL (_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000)) + +/* Control(PF Mailbox) Queue */ +#define PF_FW_BASE 0x08400000 + +#define PF_FW_ARQBAL (PF_FW_BASE) +#define PF_FW_ARQBAH (PF_FW_BASE + 0x4) +#define PF_FW_ARQLEN (PF_FW_BASE + 0x8) +#define PF_FW_ARQLEN_ARQLEN_S 0 +#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S) +#define PF_FW_ARQLEN_ARQVFE_S 28 +#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S) +#define PF_FW_ARQLEN_ARQOVFL_S 29 +#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S) +#define PF_FW_ARQLEN_ARQCRIT_S 30 +#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S) +#define PF_FW_ARQLEN_ARQENABLE_S 31 +#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S) +#define PF_FW_ARQH (PF_FW_BASE + 0xC) +#define PF_FW_ARQH_ARQH_S 0 +#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S) +#define PF_FW_ARQT (PF_FW_BASE + 0x10) + +#define PF_FW_ATQBAL (PF_FW_BASE + 0x14) +#define PF_FW_ATQBAH (PF_FW_BASE + 0x18) +#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C) +#define PF_FW_ATQLEN_ATQLEN_S 0 +#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S) +#define PF_FW_ATQLEN_ATQVFE_S 28 +#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S) +#define PF_FW_ATQLEN_ATQOVFL_S 29 +#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S) +#define PF_FW_ATQLEN_ATQCRIT_S 30 +#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S) +#define PF_FW_ATQLEN_ATQENABLE_S 31 +#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S) +#define PF_FW_ATQH (PF_FW_BASE + 0x20) +#define PF_FW_ATQH_ATQH_S 0 +#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S) +#define PF_FW_ATQT (PF_FW_BASE + 0x24) + +/* Interrupts */ +#define PF_GLINT_BASE 0x08900000 +#define PF_GLINT_DYN_CTL (_INT) (PF_GLINT_BASE + ((_INT) * 0x1000)) +#define PF_GLINT_DYN_CTL_INTENA_S 0 +#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S) +#define PF_GLINT_DYN_CTL_CLEARPBA_S 1 +#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S) +#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2 +#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S) +#define PF_GLINT_DYN_CTL_ITR_INDX_S 3 +#define PF_GLINT_DYN_CTL_INTERVAL_S 5 +#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S) +#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24 +#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M \ + BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S) +#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25 +#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S) +#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31 +#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S) +#define PF_GLINT_ITR (_i, _INT) (PF_GLINT_BASE + (((_INT) + \ + (((_i) + 1) * 4)) * 0x1000)) +#define PF_GLINT_ITR_MAX_INDEX 2 +#define PF_GLINT_ITR_INTERVAL_S 0 +#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S) + +/* Generic registers */ +#define PF_INT_DIR_OICR_ENA 0x08406000 +#define PF_INT_DIR_OICR_ENA_S 0 +#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S) +#define PF_INT_DIR_OICR 0x08406004 +#define PF_INT_DIR_OICR_TSYN_EVNT 0 +#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1) +#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2) +#define PF_INT_DIR_OICR_CAUSE 0x08406008 +#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0 +#define PF_INT_DIR_OICR_CAUSE_CAUSE_M \ + MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S) +#define PF_INT_PBA_CLEAR 0x0840600C + +#define PF_FUNC_RID 0x08406010 +#define PF_FUNC_RID_FUNCTION_NUMBER_S 0 +#define PF_FUNC_RID_FUNCTION_NUMBER_M \ + MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S) +#define PF_FUNC_RID_DEVICE_NUMBER_S 3 +#define PF_FUNC_RID_DEVICE_NUMBER_M \ + MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S) +#define PF_FUNC_RID_BUS_NUMBER_S 8 +#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S) + +/* Reset registers */ +#define PFGEN_RTRIG 0x08407000 +#define PFGEN_RTRIG_CORER_S 0 +#define PFGEN_RTRIG_CORER_M BIT(0) +#define PFGEN_RTRIG_LINKR_S 1 +#define PFGEN_RTRIG_LINKR_M BIT(1) +#define PFGEN_RTRIG_IMCR_S 2 +#define PFGEN_RTRIG_IMCR_M BIT(2) +#define PFGEN_RSTAT 0x08407008/* PFR Status */ +#define PFGEN_RSTAT_PFR_STATE_S 0 +#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S) +#define PFGEN_CTRL 0x0840700C +#define PFGEN_CTRL_PFSWR BIT(0) + +#endif
drivers/net/ethernet/intel/include/iecm_lan_txrx.h new file +636 -0
636 lines not on this page · raw message
drivers/net/ethernet/intel/include/iecm_txrx.h new file +610 -0
610 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH 3/14 iecm: Common module introduction and function stubs
In reply to Alice MichaelThis introduces function stubs for the framework of the common module.
--- .../net/ethernet/intel/iecm/iecm_controlq.c | 200 +++ .../ethernet/intel/iecm/iecm_controlq_setup.c | 84 ++ .../net/ethernet/intel/iecm/iecm_ethtool.c | 16 + drivers/net/ethernet/intel/iecm/iecm_lib.c | 406 ++++++ drivers/net/ethernet/intel/iecm/iecm_main.c | 48 + drivers/net/ethernet/intel/iecm/iecm_osdep.c | 15 + .../ethernet/intel/iecm/iecm_singleq_txrx.c | 255 ++++ drivers/net/ethernet/intel/iecm/iecm_txrx.c | 1256 +++++++++++++++++ .../net/ethernet/intel/iecm/iecm_virtchnl.c | 570 ++++++++ 9 files changed, 2850 insertions(+) create mode 100644 drivers/net/ethernet/intel/iecm/iecm_controlq.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_controlq_setup.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_ethtool.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_lib.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_main.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_osdep.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_singleq_txrx.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_txrx.c create mode 100644 drivers/net/ethernet/intel/iecm/iecm_virtchnl.c
drivers/net/ethernet/intel/iecm/iecm_controlq.c new file +200 -0
@@ -0,0 +1,200 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2020, Intel Corporation. */ + +#include "iecm_osdep.h" +#include "iecm_controlq.h" +#include "iecm_type.h" + +/** +* iecm_ctlq_setup_regs - initialize control queue registers +* @cq: pointer to the specific control queue +* @q_create_info: structs containing info for each queue to be initialized +*/ +static void +iecm_ctlq_setup_regs (struct iecm_ctlq_info *cq, +struct iecm_ctlq_create_info *q_create_info) +{ +/* stub */ +} + +/** +* iecm_ctlq_init_regs - Initialize control queue registers +* @hw: pointer to hw struct +* @cq: pointer to the specific Control queue +* @is_rxq: true if receive control queue, false otherwise +* +* Initialize registers. The caller is expected to have already initialized the +* descriptor ring memory and buffer memory +*/ +static enum iecm_status iecm_ctlq_init_regs (struct iecm_hw *hw, +struct iecm_ctlq_info *cq, +bool is_rxq) +{ +/* stub */ +} + +/** +* iecm_ctlq_init_rxq_bufs - populate receive queue descriptors with buf +* @cq: pointer to the specific Control queue +* +* Record the address of the receive queue DMA buffers in the descriptors. +* The buffers must have been previously allocated. +*/ +static void iecm_ctlq_init_rxq_bufs (struct iecm_ctlq_info *cq) +{ +/* stub */ +} + +/** +* iecm_ctlq_shutdown - shutdown the CQ +* @hw: pointer to hw struct +* @cq: pointer to the specific Control queue +* +* The main shutdown routine for any controq queue +*/ +static void iecm_ctlq_shutdown (struct iecm_hw *hw,struct iecm_ctlq_info *cq) +{ +/* stub */ +} + +/** +* iecm_ctlq_add - add one control queue +* @hw: pointer to hardware struct +* @q_info: info for queue to be created +* @cq: (output) double pointer to control queue to be created +* +* Allocate and initialize a control queue and add it to the control queue list. +* The cq parameter will be allocated/initialized and passed back to the caller +* if no errors occur. +* +* Note: iecm_ctlq_init must be called prior to any calls to iecm_ctlq_add +*/ +enum iecm_status iecm_ctlq_add (struct iecm_hw *hw, +struct iecm_ctlq_create_info *qinfo, +struct iecm_ctlq_info **cq) +{ +/* stub */ +} + +/** +* iecm_ctlq_remove - deallocate and remove specified control queue +* @hw: pointer to hardware struct +* @cq: pointer to control queue to be removed +*/ +void iecm_ctlq_remove (struct iecm_hw *hw, +struct iecm_ctlq_info *cq) +{ +/* stub */ +} + +/** +* iecm_ctlq_init - main initialization routine for all control queues +* @hw: pointer to hardware struct +* @num_q: number of queues to initialize +* @q_info: array of structs containing info for each queue to be initialized +* +* This initializes any number and any type of control queues. This is an all +* or nothing routine; if one fails, all previously allocated queues will be +* destroyed. This must be called prior to using the individual add/remove +* APIs. +*/ +enum iecm_status iecm_ctlq_init (struct iecm_hw *hw,u8 num_q, +struct iecm_ctlq_create_info *q_info) +{ +/* stub */ +} + +/** +* iecm_ctlq_deinit - destroy all control queues +* @hw: pointer to hw struct +*/ +enum iecm_status iecm_ctlq_deinit (struct iecm_hw *hw) +{ +/* stub */ +} + +/** +* iecm_ctlq_send - send command to Control Queue (CTQ) +* @hw: pointer to hw struct +* @cq: handle to control queue struct to send on +* @num_q_msg: number of messages to send on control queue +* @q_msg: pointer to array of queue messages to be sent +* +* The caller is expected to allocate DMAable buffers and pass them to the +* send routine via the q_msg struct / control queue specific data struct. +* The control queue will hold a reference to each send message until +* the completion for that message has been cleaned. +*/ +enum iecm_status iecm_ctlq_send (struct iecm_hw *hw, +struct iecm_ctlq_info *cq, +u16 num_q_msg, +struct iecm_ctlq_msg q_msg[]) +{ +/* stub */ +} + +/** +* iecm_ctlq_clean_sq - reclaim send descriptors on HW write back for the +* requested queue +* @hw: pointer to hw struct +* @cq: pointer to the specific Control queue +* @clean_count: (input|output) number of descriptors to clean as input, and +* number of descriptors actually cleaned as output +* @msg_status: (output) pointer to msg pointer array to be populated; needs +* to be allocated by caller +* +* Returns an an array of message pointers associated with the cleaned +* descriptors. The pointers are to the original ctlq_msgs sent on the cleaned +* descriptors. The status will be returned for each; any messages that failed +* to send will have a non-zero status. The caller is expected to free original +* ctlq_msgs and free or reuse the DMA buffers. +*/ +enum iecm_status iecm_ctlq_clean_sq (struct iecm_hw *hw, +struct iecm_ctlq_info *cq, +u16 *clean_count, +struct iecm_ctlq_msg *msg_status[]) +{ +/* stub */ +} + +/** +* iecm_ctlq_post_rx_buffs - post buffers to descriptor ring +* @hw: pointer to hw struct +* @cq: pointer to control queue handle +* @buff_count: (input|output) input is number of buffers caller is trying to +* return; output is number of buffers that were not posted +* @buffs: array of pointers to dma mem structs to be given to hardware +* +* Caller uses this function to return DMA buffers to the descriptor ring after +* consuming them; buff_count will be the number of buffers. +* +* Note: this function needs to be called after a receive call even +* if there are no DMA buffers to be returned, i.e. buff_count = 0, +* buffs = NULL to support direct commands +*/ +enum iecm_status iecm_ctlq_post_rx_buffs (struct iecm_hw *hw, +struct iecm_ctlq_info *cq, +u16 *buff_count, +struct iecm_dma_mem **buffs) +{ +/* stub */ +} + +/** +* iecm_ctlq_recv - receive control queue message call back +* @hw: pointer to hw struct +* @cq: pointer to control queue handle to receive on +* @num_q_msg: (input|output) input number of messages that should be received; +* output number of messages actually received +* @q_msg: (output) array of received control queue messages on this q; +* needs to be pre-allocated by caller for as many messages as requested +* +* Called by interrupt handler or polling mechanism. Caller is expected +* to free buffers +*/ +enum iecm_status iecm_ctlq_recv (struct iecm_hw *hw, +struct iecm_ctlq_info *cq, +u16 *num_q_msg,struct iecm_ctlq_msg *q_msg) +{ +/* stub */ +}
drivers/net/ethernet/intel/iecm/iecm_controlq_setup.c new file +84 -0
84 lines not on this page · raw message
drivers/net/ethernet/intel/iecm/iecm_ethtool.c new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (C) 2020 Intel Corporation */ + +#include "iecm.h" + +/** +* iecm_set_ethtool_ops - Initialize ethtool ops struct +* @netdev: network interface device structure +* +* Sets ethtool ops struct in our netdev so that ethtool can call +* our functions. +*/ +void iecm_set_ethtool_ops (struct net_device *netdev) +{ +/* stub */ +}
drivers/net/ethernet/intel/iecm/iecm_lib.c new file +406 -0
406 lines not on this page · raw message
drivers/net/ethernet/intel/iecm/iecm_main.c new file +48 -0
48 lines not on this page · raw message
drivers/net/ethernet/intel/iecm/iecm_osdep.c new file +15 -0
@@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2020 Intel Corporation. */ + +#include "iecm_osdep.h" +#include "iecm.h" + +void *iecm_alloc_dma_mem (struct iecm_hw *hw,struct iecm_dma_mem *mem,u64 size) +{ +/* stub */ +} + +void iecm_free_dma_mem (struct iecm_hw *hw,struct iecm_dma_mem *mem) +{ +/* stub */ +}
drivers/net/ethernet/intel/iecm/iecm_singleq_txrx.c new file +255 -0
255 lines not on this page · raw message
drivers/net/ethernet/intel/iecm/iecm_txrx.c new file +1256 -0
1,256 lines not on this page · raw message
drivers/net/ethernet/intel/iecm/iecm_virtchnl.c new file +570 -0
570 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH 4/14 iecm: Add basic netdevice functionality
In reply to Alice MichaelThis implements probe, interface up/down, and netdev_ops.
--- drivers/net/ethernet/intel/iecm/iecm_lib.c | 404 +++++++++++++++++- drivers/net/ethernet/intel/iecm/iecm_main.c | 7 +- drivers/net/ethernet/intel/iecm/iecm_txrx.c | 6 +- .../net/ethernet/intel/iecm/iecm_virtchnl.c | 73 +++- 4 files changed, 467 insertions(+), 23 deletions(-)
drivers/net/ethernet/intel/iecm/iecm_lib.c +387 -17
@@ -24,7 +24,17 @@ static void iecm_mb_intr_rel_irq(struct iecm_adapter *adapter)*/ static void iecm_intr_rel (struct iecm_adapter *adapter) { -/* stub */ +if (!adapter->msix_entries) +return ; +clear_bit (__IECM_MB_INTR_MODE, adapter->flags); +clear_bit (__IECM_MB_INTR_TRIGGER, adapter->flags); +iecm_mb_intr_rel_irq (adapter); + +pci_free_irq_vectors (adapter->pdev); +kfree (adapter->msix_entries); + adapter->msix_entries =NULL ; +kfree (adapter->req_vec_chunks); + adapter->req_vec_chunks =NULL ; }/**
@@ -96,7 +106,53 @@ void iecm_intr_distribute(struct iecm_adapter *adapter)*/ static int iecm_intr_req (struct iecm_adapter *adapter) { -/* stub */ +int min_vectors, max_vectors, err =0 ; +unsigned int vector; +int num_vecs; +int v_actual; + + num_vecs = adapter->vports[0 ]->num_q_vectors + +IECM_MAX_NONQ_VEC +IECM_MAX_RDMA_VEC ; + + min_vectors =IECM_MIN_VEC ; +#define IECM_MAX_EVV_MAPPED_VEC 16 + max_vectors =min (num_vecs,IECM_MAX_EVV_MAPPED_VEC ); + + v_actual =pci_alloc_irq_vectors (adapter->pdev, min_vectors, + max_vectors,PCI_IRQ_MSIX ); +if (v_actual <0 ) { +dev_err (&adapter->pdev->dev,"Failed to allocate MSIX vectors: %d\n" , + v_actual); +return v_actual; + } + + adapter->msix_entries =kcalloc (v_actual,sizeof (struct msix_entry ), +GFP_KERNEL ); + +if (!adapter->msix_entries) { +pci_free_irq_vectors (adapter->pdev); +return -ENOMEM ; + } + +for (vector =0 ; vector < v_actual; vector++) { + adapter->msix_entries[vector].entry = vector; + adapter->msix_entries[vector].vector = +pci_irq_vector (adapter->pdev, vector); + } + adapter->num_msix_entries = v_actual; + adapter->num_req_msix = num_vecs; + +iecm_intr_distribute (adapter); + + err =iecm_mb_intr_init (adapter); +if (err) + goto intr_rel; +iecm_mb_irq_enable (adapter); +return err; + +intr_rel: +iecm_intr_rel (adapter); +return err; }/**
@@ -118,7 +174,21 @@ static int iecm_cfg_netdev(struct iecm_vport *vport)*/ static int iecm_cfg_hw (struct iecm_adapter *adapter) { -/* stub */ +struct pci_dev *pdev = adapter->pdev; +struct iecm_hw *hw = &adapter->hw; + + hw->hw_addr_len =pci_resource_len (pdev,0 ); + hw->hw_addr =ioremap (pci_resource_start (pdev,0 ), hw->hw_addr_len); + +if (!hw->hw_addr) +return -EIO ; + + hw->back = adapter; + hw->bus.device =PCI_SLOT (pdev->devfn); + hw->bus.func =PCI_FUNC (pdev->devfn); + hw->bus.bus_id = pdev->bus->number; + +return 0 ; }/**
@@ -132,7 +202,22 @@ static int iecm_cfg_hw(struct iecm_adapter *adapter)*/ static int iecm_get_free_slot (void *array,int size,int curr) { -/* stub */ +int **tmp_array = (int **)array; +int next; + +if (curr < (size -1 ) && !tmp_array[curr +1 ]) { + next = curr +1 ; + }else { +int i =0 ; + +while ((i < size) && (tmp_array[i])) + i++; +if (i == size) + next =IECM_NO_FREE_SLOT ; +else + next = i; + } +return next; }/**
@@ -141,7 +226,9 @@ static int iecm_get_free_slot(void *array, int size, int curr)*/ struct iecm_vport *iecm_netdev_to_vport (struct net_device *netdev) { -/* stub */ +struct iecm_netdev_priv *np =netdev_priv (netdev); + +return np->vport; }/**
@@ -150,7 +237,9 @@ struct iecm_vport *iecm_netdev_to_vport(struct net_device *netdev)*/ struct iecm_adapter *iecm_netdev_to_adapter (struct net_device *netdev) { -/* stub */ +struct iecm_netdev_priv *np =netdev_priv (netdev); + +return np->vport->adapter; }/**
@@ -185,7 +274,22 @@ static int iecm_stop(struct net_device *netdev)*/ int iecm_vport_rel (struct iecm_vport *vport) { -/* stub */ +struct iecm_adapter *adapter; + +if (!vport->adapter) +return -ENODEV ; + adapter = vport->adapter; + +iecm_vport_stop (vport); +iecm_deinit_rss (vport); +unregister_netdev (vport->netdev); +free_netdev (vport->netdev); + vport->netdev =NULL ; +if (adapter->dev_ops.vc_ops.destroy_vport) + adapter->dev_ops.vc_ops.destroy_vport (vport); +kfree (vport); + +return 0 ; }/**
@@ -194,7 +298,24 @@ int iecm_vport_rel(struct iecm_vport *vport)*/ static void iecm_vport_rel_all (struct iecm_adapter *adapter) { -/* stub */ +int err, i; + +if (!adapter->vports) +return ; + +for (i =0 ; i < adapter->num_alloc_vport; i++) { +if (!adapter->vports[i]) +continue ; + + err =iecm_vport_rel (adapter->vports[i]); +if (err) +dev_dbg (&adapter->pdev->dev, +"Failed to release adapter->vport[%d], err %d,\n" , + i, err); +else + adapter->vports[i] =NULL ; + } + adapter->num_alloc_vport =0 ; }/**
@@ -217,7 +338,47 @@ void iecm_vport_set_hsplit(struct iecm_vport *vport, struct bpf_prog *prog)static struct iecm_vport *iecm_vport_alloc (struct iecm_adapter *adapter,int vport_id) { -/* stub */ +struct iecm_vport *vport =NULL ; + +if (adapter->next_vport ==IECM_NO_FREE_SLOT ) +return vport; + +/* Need to protect the allocation of the vports at the adapter level */ +mutex_lock (&adapter->sw_mutex); + + vport =kzalloc (sizeof (*vport),GFP_KERNEL ); +if (!vport) + goto unlock_adapter; + + vport->adapter = adapter; + vport->idx = adapter->next_vport; + vport->compln_clean_budget =IECM_TX_COMPLQ_CLEAN_BUDGET ; + adapter->num_alloc_vport++; + adapter->dev_ops.vc_ops.vport_init (vport, vport_id); + +/* Setup default MSIX irq handler for the vport */ + vport->irq_q_handler = iecm_vport_intr_clean_queues; + vport->q_vector_base =IECM_MAX_NONQ_VEC ; + +/* fill vport slot in the adapter struct */ + adapter->vports[adapter->next_vport] = vport; +if (iecm_cfg_netdev (vport)) + goto cfg_netdev_fail; + +/* prepare adapter->next_vport for next use */ + adapter->next_vport =iecm_get_free_slot (adapter->vports, + adapter->num_alloc_vport, + adapter->next_vport); + + goto unlock_adapter; + +cfg_netdev_fail: + adapter->vports[adapter->next_vport] =NULL ; +kfree (vport); + vport =NULL ; +unlock_adapter: +mutex_unlock (&adapter->sw_mutex); +return vport; }/**
@@ -227,7 +388,22 @@ iecm_vport_alloc(struct iecm_adapter *adapter, int vport_id)*/ static void iecm_service_task (struct work_struct *work) { -/* stub */ +struct iecm_adapter *adapter =container_of (work, + struct iecm_adapter, + serv_task.work); + +if (test_bit (__IECM_MB_INTR_MODE, adapter->flags)) { +if (test_and_clear_bit (__IECM_MB_INTR_TRIGGER, + adapter->flags)) { +iecm_recv_mb_msg (adapter,VIRTCHNL_OP_UNKNOWN ,NULL ,0 ); +iecm_mb_irq_enable (adapter); + } + }else { +iecm_recv_mb_msg (adapter,VIRTCHNL_OP_UNKNOWN ,NULL ,0 ); + } + +queue_delayed_work (adapter->serv_wq, &adapter->serv_task, +msecs_to_jiffies (300 )); }/**
@@ -261,7 +437,41 @@ static int iecm_vport_open(struct iecm_vport *vport)*/ static void iecm_init_task (struct work_struct *work) { -/* stub */ +struct iecm_adapter *adapter =container_of (work, + struct iecm_adapter, + init_task.work); +struct iecm_vport *vport; +struct pci_dev *pdev; +int vport_id, err; + + err = adapter->dev_ops.vc_ops.core_init (adapter, &vport_id); +if (err) +return ; + + pdev = adapter->pdev; + vport =iecm_vport_alloc (adapter, vport_id); +if (!vport) { + err = -EFAULT ; +dev_err (&pdev->dev,"probe failed on vport setup:%d\n" , + err); +return ; + } +/* Start the service task before requesting vectors. This will ensure +* vector information response from mailbox is handled +*/ +queue_delayed_work (adapter->serv_wq, &adapter->serv_task, +msecs_to_jiffies (5 * (pdev->devfn &0x07 ))); + err =iecm_intr_req (adapter); +if (err) { +dev_err (&pdev->dev,"failed to enable interrupt vectors: %d\n" , + err); +iecm_vport_rel (vport); +return ; + } +/* Once state is put into DOWN, driver is ready for dev_open */ + adapter->state = __IECM_DOWN; +if (test_and_clear_bit (__IECM_UP_REQUESTED, adapter->flags)) +iecm_vport_open (vport); }/**
@@ -272,7 +482,40 @@ static void iecm_init_task(struct work_struct *work)*/ static int iecm_api_init (struct iecm_adapter *adapter) { -/* stub */ +struct iecm_reg_ops *reg_ops = &adapter->dev_ops.reg_ops; +struct pci_dev *pdev = adapter->pdev; + +if (!adapter->dev_ops.reg_ops_init) { +dev_err (&pdev->dev,"Invalid device, register API init not defined.\n" ); +return -EINVAL ; + } + adapter->dev_ops.reg_ops_init (adapter); +if (!(reg_ops->ctlq_reg_init && reg_ops->vportq_reg_init && + reg_ops->intr_reg_init && reg_ops->mb_intr_reg_init && + reg_ops->reset_reg_init && reg_ops->trigger_reset)) { +dev_err (&pdev->dev,"Invalid device, missing one or more register functions\n" ); +return -EINVAL ; + } + +if (adapter->dev_ops.vc_ops_init) { +struct iecm_virtchnl_ops *vc_ops; + + adapter->dev_ops.vc_ops_init (adapter); + vc_ops = &adapter->dev_ops.vc_ops; +if (!(vc_ops->core_init && vc_ops->vport_init && + vc_ops->vport_queue_ids_init && vc_ops->get_caps && + vc_ops->config_queues && vc_ops->enable_queues && + vc_ops->disable_queues && vc_ops->irq_map_unmap && + vc_ops->get_set_rss_lut && vc_ops->get_set_rss_hash && + vc_ops->adjust_qs && vc_ops->get_ptype)) { +dev_err (&pdev->dev,"Invalid device, missing one or more virtchnl functions\n" ); +return -EINVAL ; + } + }else { +iecm_vc_ops_init (adapter); + } + +return 0 ; }/**
@@ -284,7 +527,11 @@ static int iecm_api_init(struct iecm_adapter *adapter)*/ void iecm_deinit_task (struct iecm_adapter *adapter) { -/* stub */ +iecm_vport_rel_all (adapter); +cancel_delayed_work_sync (&adapter->serv_task); +iecm_deinit_dflt_mbx (adapter); +iecm_vport_params_buf_rel (adapter); +iecm_intr_rel (adapter); }/**
@@ -306,7 +553,13 @@ iecm_init_hard_reset(struct iecm_adapter *adapter)*/ static void iecm_vc_event_task (struct work_struct *work) { -/* stub */ +struct iecm_adapter *adapter =container_of (work, + struct iecm_adapter, + vc_event_task.work); + +if (test_bit (__IECM_HR_CORE_RESET, adapter->flags) || +test_bit (__IECM_HR_FUNC_RESET, adapter->flags)) +iecm_init_hard_reset (adapter); }/**
@@ -335,7 +588,103 @@ int iecm_probe(struct pci_dev *pdev,const struct pci_device_id __always_unused *ent, struct iecm_adapter *adapter) { - /* stub */ + int err; + + adapter->pdev = pdev; + err =iecm_api_init (adapter); +if (err) { +dev_err (&pdev->dev,"Device API is incorrectly configured\n" ); +return err; + } + + err =pcim_iomap_regions (pdev,BIT (IECM_BAR0 ),pci_name (pdev)); +if (err) { +dev_err (&pdev->dev,"BAR0 I/O map error %d\n" , err); +return err; + } + +/* set up for high or low dma */ + err =dma_set_mask_and_coherent (&pdev->dev,DMA_BIT_MASK (64 )); +if (err) + err =dma_set_mask_and_coherent (&pdev->dev,DMA_BIT_MASK (32 )); +if (err) { +dev_err (&pdev->dev,"DMA configuration failed: 0x%x\n" , err); +return err; + } + +pci_enable_pcie_error_reporting (pdev); +pci_set_master (pdev); +pci_set_drvdata (pdev, adapter); + + adapter->init_wq = +alloc_workqueue ("%s" ,WQ_MEM_RECLAIM ,0 ,KBUILD_MODNAME ); +if (!adapter->init_wq) { +dev_err (&pdev->dev,"Failed to allocate workqueue\n" ); + err = -ENOMEM ; + goto err_wq_alloc; + } + + adapter->serv_wq = +alloc_workqueue ("%s" ,WQ_MEM_RECLAIM ,0 ,KBUILD_MODNAME ); +if (!adapter->serv_wq) { +dev_err (&pdev->dev,"Failed to allocate workqueue\n" ); + err = -ENOMEM ; + goto err_mbx_wq_alloc; + } +/* setup msglvl */ + adapter->msg_enable =netif_msg_init (debug,IECM_DFLT_NETIF_M ); + + adapter->vports =kcalloc (IECM_MAX_NUM_VPORTS , +sizeof (*adapter->vports),GFP_KERNEL ); +if (!adapter->vports) { + err = -ENOMEM ; + goto err_vport_alloc; + } + + err =iecm_vport_params_buf_alloc (adapter); +if (err) { +dev_err (&pdev->dev,"Failed to alloc vport params buffer: %d\n" , + err); + goto err_mb_res; + } + + err =iecm_cfg_hw (adapter); +if (err) { +dev_err (&pdev->dev,"Failed to configure HW structure for adapter: %d\n" , + err); + goto err_cfg_hw; + } + +mutex_init (&adapter->sw_mutex); +mutex_init (&adapter->vc_msg_lock); +mutex_init (&adapter->reset_lock); +init_waitqueue_head (&adapter->vchnl_wq); + +INIT_DELAYED_WORK (&adapter->serv_task, iecm_service_task); +INIT_DELAYED_WORK (&adapter->init_task, iecm_init_task); +INIT_DELAYED_WORK (&adapter->vc_event_task, iecm_vc_event_task); + +mutex_lock (&adapter->reset_lock); +set_bit (__IECM_HR_DRV_LOAD, adapter->flags); + err =iecm_init_hard_reset (adapter); +if (err) { +dev_err (&pdev->dev,"Failed to reset device: %d\n" , err); + goto err_mb_init; + } + +return 0 ; +err_mb_init: +err_cfg_hw: +iecm_vport_params_buf_rel (adapter); +err_mb_res: +kfree (adapter->vports); +err_vport_alloc: +destroy_workqueue (adapter->serv_wq); +err_mbx_wq_alloc: +destroy_workqueue (adapter->init_wq); +err_wq_alloc: +pci_disable_pcie_error_reporting (pdev); +return err; }EXPORT_SYMBOL (iecm_probe);
@@ -345,7 +694,22 @@ EXPORT_SYMBOL(iecm_probe);*/ void iecm_remove (struct pci_dev *pdev) { -/* stub */ +struct iecm_adapter *adapter =pci_get_drvdata (pdev); + +if (!adapter) +return ; + +iecm_deinit_task (adapter); +cancel_delayed_work_sync (&adapter->vc_event_task); +destroy_workqueue (adapter->serv_wq); +destroy_workqueue (adapter->init_wq); +kfree (adapter->vports); +kfree (adapter->vport_params_recvd); +kfree (adapter->vport_params_reqd); +mutex_destroy (&adapter->sw_mutex); +mutex_destroy (&adapter->vc_msg_lock); +mutex_destroy (&adapter->reset_lock); +pci_disable_pcie_error_reporting (pdev); }EXPORT_SYMBOL (iecm_remove);
@@ -355,7 +719,13 @@ EXPORT_SYMBOL(iecm_remove);*/ void iecm_shutdown (struct pci_dev *pdev) { -/* stub */ +struct iecm_adapter *adapter; + + adapter =pci_get_drvdata (pdev); + adapter->state = __IECM_REMOVE; + +if (system_state ==SYSTEM_POWER_OFF ) +pci_set_power_state (pdev, PCI_D3hot); }EXPORT_SYMBOL (iecm_shutdown);
drivers/net/ethernet/intel/iecm/iecm_main.c +5 -2
@@ -31,7 +31,10 @@ MODULE_PARM_DESC(debug, "netif level (0=none,...,16=all)");*/ static int __init iecm_module_init (void ) { -/* stub */ +pr_info ("%s - version %d\n" , iecm_driver_string,LINUX_VERSION_CODE ); +pr_info ("%s\n" , iecm_copyright); + +return 0 ; }module_init (iecm_module_init);
@@ -43,6 +46,6 @@ module_init(iecm_module_init);*/ static void __exit iecm_module_exit(void) { -/* stub */ +pr_info ("module unloaded\n" ); }module_exit (iecm_module_exit);
drivers/net/ethernet/intel/iecm/iecm_txrx.c +5 -1
@@ -989,7 +989,11 @@ static int iecm_rx_splitq_clean(struct iecm_queue *rxq, int budget)irqreturn_t iecm_vport_intr_clean_queues (int __always_unused irq,void *data) { -/* stub */ +struct iecm_q_vector *q_vector = (struct iecm_q_vector *)data; + +napi_schedule (&q_vector->napi); + +return IRQ_HANDLED ; }/**
drivers/net/ethernet/intel/iecm/iecm_virtchnl.c +70 -3
90 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH 5/14 iecm: Implement mailbox functionality
In reply to Alice MichaelImplement mailbox setup, take down, and commands.
--- .../net/ethernet/intel/iecm/iecm_controlq.c | 497 +++++++++++++++++- .../ethernet/intel/iecm/iecm_controlq_setup.c | 105 +++- drivers/net/ethernet/intel/iecm/iecm_lib.c | 71 ++- drivers/net/ethernet/intel/iecm/iecm_osdep.c | 17 +- .../net/ethernet/intel/iecm/iecm_virtchnl.c | 427 ++++++++++++++- 5 files changed, 1088 insertions(+), 29 deletions(-)
drivers/net/ethernet/intel/iecm/iecm_controlq.c +485 -12
@@ -14,7 +14,15 @@ static void iecm_ctlq_setup_regs(struct iecm_ctlq_info *cq, struct iecm_ctlq_create_info *q_create_info) { - /* stub */ +/* set head and tail registers in our local struct */ + cq->reg.head = q_create_info->reg.head; + cq->reg.tail = q_create_info->reg.tail; + cq->reg.len = q_create_info->reg.len; + cq->reg.bah = q_create_info->reg.bah; + cq->reg.bal = q_create_info->reg.bal; + cq->reg.len_mask = q_create_info->reg.len_mask; + cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask; + cq->reg.head_mask = q_create_info->reg.head_mask; }/**
@@ -30,7 +38,32 @@ static enum iecm_status iecm_ctlq_init_regs(struct iecm_hw *hw, struct iecm_ctlq_info *cq, bool is_rxq) { - /* stub */ +u32 reg =0 ; + +if (is_rxq) +/* Update tail to post pre-allocated buffers for rx queues */ +wr32 (hw, cq->reg.tail, (u32)(cq->ring_size -1 )); +else +wr32 (hw, cq->reg.tail,0 ); + +/* For non-Mailbox control queues only TAIL need to be set */ +if (cq->q_id !=-1 ) +return 0 ; + +/* Clear Head for both send or receive */ +wr32 (hw, cq->reg.head,0 ); + +/* set starting point */ +wr32 (hw, cq->reg.bal,IECM_LO_DWORD (cq->desc_ring.pa)); +wr32 (hw, cq->reg.bah,IECM_HI_DWORD (cq->desc_ring.pa)); +wr32 (hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask)); + +/* Check one register to verify that config was applied */ + reg =rd32 (hw, cq->reg.bah); +if (reg !=IECM_HI_DWORD (cq->desc_ring.pa)) +return IECM_ERR_CTLQ_ERROR ; + +return 0 ; }/**
@@ -42,7 +75,30 @@ static enum iecm_status iecm_ctlq_init_regs(struct iecm_hw *hw,*/ static void iecm_ctlq_init_rxq_bufs (struct iecm_ctlq_info *cq) { -/* stub */ +int i =0 ; + +for (i =0 ; i < cq->ring_size; i++) { +struct iecm_ctlq_desc *desc =IECM_CTLQ_DESC (cq, i); +struct iecm_dma_mem *bi = cq->bi.rx_buff[i]; + +/* No buffer to post to descriptor, continue */ +if (!bi) +continue ; + + desc->flags = +cpu_to_le16 (IECM_CTLQ_FLAG_BUF |IECM_CTLQ_FLAG_RD ); + desc->opcode =0 ; + desc->datalen = (__le16 )cpu_to_le16 (bi->size); + desc->ret_val =0 ; + desc->cookie_high =0 ; + desc->cookie_low =0 ; + desc->params.indirect.addr_high = +cpu_to_le32 (IECM_HI_DWORD (bi->pa)); + desc->params.indirect.addr_low = +cpu_to_le32 (IECM_LO_DWORD (bi->pa)); + desc->params.indirect.param0 =0 ; + desc->params.indirect.param1 =0 ; + } }/**
@@ -54,7 +110,20 @@ static void iecm_ctlq_init_rxq_bufs(struct iecm_ctlq_info *cq)*/ static void iecm_ctlq_shutdown (struct iecm_hw *hw,struct iecm_ctlq_info *cq) { -/* stub */ +mutex_lock (&cq->cq_lock); + +if (!cq->ring_size) + goto shutdown_sq_out; + +/* free ring buffers and the ring itself */ +iecm_ctlq_dealloc_ring_res (hw, cq); + +/* Set ring_size to 0 to indicate uninitialized queue */ + cq->ring_size =0 ; + +shutdown_sq_out: +mutex_unlock (&cq->cq_lock); +mutex_destroy (&cq->cq_lock); }/**
@@ -73,7 +142,74 @@ enum iecm_status iecm_ctlq_add(struct iecm_hw *hw,struct iecm_ctlq_create_info *qinfo, struct iecm_ctlq_info **cq) { - /* stub */ + enum iecm_status status = 0; +bool is_rxq = false; + +if (!qinfo->len || !qinfo->buf_size || + qinfo->len >IECM_CTLQ_MAX_RING_SIZE || + qinfo->buf_size >IECM_CTLQ_MAX_BUF_LEN ) +return IECM_ERR_CFG ; + + *cq =kcalloc (1 ,sizeof (struct iecm_ctlq_info ),GFP_KERNEL ); +if (!(*cq)) +return IECM_ERR_NO_MEMORY ; + + (*cq)->cq_type = qinfo->type; + (*cq)->q_id = qinfo->id; + (*cq)->buf_size = qinfo->buf_size; + (*cq)->ring_size = qinfo->len; + + (*cq)->next_to_use =0 ; + (*cq)->next_to_clean =0 ; + (*cq)->next_to_post = (*cq)->ring_size -1 ; + +switch (qinfo->type) { +case IECM_CTLQ_TYPE_MAILBOX_RX : + is_rxq = true; +/* fallthrough */ +case IECM_CTLQ_TYPE_MAILBOX_TX : + status =iecm_ctlq_alloc_ring_res (hw, *cq); +break ; +default : + status =IECM_ERR_PARAM ; +break ; + } + +if (status) + goto init_free_q; + +if (is_rxq) { +iecm_ctlq_init_rxq_bufs (*cq); + }else { +/* Allocate the array of msg pointers for TX queues */ + (*cq)->bi.tx_msg =kcalloc (qinfo->len, +sizeof (struct iecm_ctlq_msg *), +GFP_KERNEL ); +if (!(*cq)->bi.tx_msg) { + status =IECM_ERR_NO_MEMORY ; + goto init_dealloc_q_mem; + } + } + +iecm_ctlq_setup_regs (*cq, qinfo); + + status =iecm_ctlq_init_regs (hw, *cq, is_rxq); +if (status) + goto init_dealloc_q_mem; + +mutex_init (&(*cq)->cq_lock); + +list_add (&(*cq)->cq_list, &hw->cq_list_head); + +return status; + +init_dealloc_q_mem: +/* free ring buffers and the ring itself */ +iecm_ctlq_dealloc_ring_res (hw, *cq); +init_free_q: +kfree (*cq); + +return status; }/**
@@ -84,7 +220,9 @@ enum iecm_status iecm_ctlq_add(struct iecm_hw *hw,void iecm_ctlq_remove (struct iecm_hw *hw,struct iecm_ctlq_info *cq) { -/* stub */ +list_del (&cq->cq_list); +iecm_ctlq_shutdown (hw, cq); +kfree (cq); }/**
@@ -101,7 +239,27 @@ void iecm_ctlq_remove(struct iecm_hw *hw,enum iecm_status iecm_ctlq_init (struct iecm_hw *hw,u8 num_q,struct iecm_ctlq_create_info *q_info) { -/* stub */ +struct iecm_ctlq_info *cq =NULL , *tmp =NULL ; +enum iecm_status ret_code =0 ; +int i =0 ; + +INIT_LIST_HEAD (&hw->cq_list_head); + +for (i =0 ; i < num_q; i++) { +struct iecm_ctlq_create_info *qinfo = q_info + i; + + ret_code =iecm_ctlq_add (hw, qinfo, &cq); +if (ret_code) + goto init_destroy_qs; + } + +return ret_code; + +init_destroy_qs: +list_for_each_entry_safe (cq, tmp, &hw->cq_list_head, cq_list) +iecm_ctlq_remove (hw, cq); + +return ret_code; }/**
@@ -110,7 +268,13 @@ enum iecm_status iecm_ctlq_init(struct iecm_hw *hw, u8 num_q,*/ enum iecm_status iecm_ctlq_deinit (struct iecm_hw *hw) { -/* stub */ +struct iecm_ctlq_info *cq =NULL , *tmp =NULL ; +enum iecm_status ret_code =0 ; + +list_for_each_entry_safe (cq, tmp, &hw->cq_list_head, cq_list) +iecm_ctlq_remove (hw, cq); + +return ret_code; }/**
@@ -130,7 +294,79 @@ enum iecm_status iecm_ctlq_send(struct iecm_hw *hw, u16 num_q_msg, struct iecm_ctlq_msg q_msg[]) { - /* stub */ +enum iecm_status status =0 ; +struct iecm_ctlq_desc *desc; +int num_desc_avail =0 ; +int i =0 ; + +if (!cq || !cq->ring_size) +return IECM_ERR_CTLQ_EMPTY ; + +mutex_lock (&cq->cq_lock); + +/* Ensure there are enough descriptors to send all messages */ + num_desc_avail =IECM_CTLQ_DESC_UNUSED (cq); +if (num_desc_avail ==0 || num_desc_avail < num_q_msg) { + status =IECM_ERR_CTLQ_FULL ; + goto sq_send_command_out; + } + +for (i =0 ; i < num_q_msg; i++) { +struct iecm_ctlq_msg *msg = &q_msg[i]; +u64 msg_cookie; + + desc =IECM_CTLQ_DESC (cq, cq->next_to_use); + + desc->opcode =cpu_to_le16 (msg->opcode); + desc->pfid_vfid = msg->func_id; + + msg_cookie = *(u64 *)&msg->cookie; + desc->cookie_high = +cpu_to_le32 (IECM_HI_DWORD (msg_cookie)); + desc->cookie_low = +cpu_to_le32 (IECM_LO_DWORD (msg_cookie)); + +if (msg->data_len) { +struct iecm_dma_mem *buff = msg->ctx.indirect.payload; + + desc->datalen =cpu_to_le16 (msg->data_len); + desc->flags |=cpu_to_le16 (IECM_CTLQ_FLAG_BUF ); + desc->flags |=cpu_to_le16 (IECM_CTLQ_FLAG_RD ); + +/* Update the address values in the desc with the pa +* value for respective buffer +*/ + desc->params.indirect.addr_high = +cpu_to_le32 (IECM_HI_DWORD (buff->pa)); + desc->params.indirect.addr_low = +cpu_to_le32 (IECM_LO_DWORD (buff->pa)); + +memcpy (&desc->params, msg->ctx.indirect.context, +IECM_INDIRECT_CTX_SIZE ); + }else { +memcpy (&desc->params, msg->ctx.direct, +IECM_DIRECT_CTX_SIZE ); + } + +/* Store buffer info */ + cq->bi.tx_msg[cq->next_to_use] = msg; + + (cq->next_to_use)++; +if (cq->next_to_use == cq->ring_size) + cq->next_to_use =0 ; + } + +/* Force memory write to complete before letting hardware +* know that there are new descriptors to fetch. +*/ +iecm_wmb (); + +wr32 (hw, cq->reg.tail, cq->next_to_use); + +sq_send_command_out: +mutex_unlock (&cq->cq_lock); + +return status; }/**
@@ -154,7 +390,58 @@ enum iecm_status iecm_ctlq_clean_sq(struct iecm_hw *hw,u16 *clean_count, struct iecm_ctlq_msg *msg_status[]) { - /* stub */ + enum iecm_status ret = 0; +struct iecm_ctlq_desc *desc; +u16 i =0 , num_to_clean; +u16 ntc, desc_err; + +if (!cq || !cq->ring_size) +return IECM_ERR_CTLQ_EMPTY ; + +if (*clean_count ==0 ) +return 0 ; +else if (*clean_count > cq->ring_size) +return IECM_ERR_PARAM ; + +mutex_lock (&cq->cq_lock); + + ntc = cq->next_to_clean; + + num_to_clean = *clean_count; + +for (i =0 ; i < num_to_clean; i++) { +/* Fetch next descriptor and check if marked as done */ + desc =IECM_CTLQ_DESC (cq, ntc); +if (!(le16_to_cpu (desc->flags) &IECM_CTLQ_FLAG_DD )) +break ; + + desc_err =le16_to_cpu (desc->ret_val); +if (desc_err) { +/* strip off FW internal code */ + desc_err &=0xff ; + } + + msg_status[i] = cq->bi.tx_msg[ntc]; + msg_status[i]->status = desc_err; + + cq->bi.tx_msg[ntc] =NULL ; + +/* Zero out any stale data */ +memset (desc,0 ,sizeof (*desc)); + + ntc++; +if (ntc == cq->ring_size) + ntc =0 ; + } + + cq->next_to_clean = ntc; + +mutex_unlock (&cq->cq_lock); + +/* Return number of descriptors actually cleaned */ + *clean_count = i; + +return ret; }/**
@@ -177,7 +464,111 @@ enum iecm_status iecm_ctlq_post_rx_buffs(struct iecm_hw *hw,u16 *buff_count, struct iecm_dma_mem **buffs) { - /* stub */ + enum iecm_status status = 0; +struct iecm_ctlq_desc *desc; +u16 ntp = cq->next_to_post; +bool buffs_avail = false; +u16 tbp = ntp +1 ; +int i =0 ; + +if (*buff_count > cq->ring_size) +return IECM_ERR_PARAM ; + +if (*buff_count >0 ) + buffs_avail = true; + +mutex_lock (&cq->cq_lock); + +if (tbp >= cq->ring_size) + tbp =0 ; + +if (tbp == cq->next_to_clean) +/* Nothing to do */ + goto post_buffs_out; + +/* Post buffers for as many as provided or up until the last one used */ +while (ntp != cq->next_to_clean) { + desc =IECM_CTLQ_DESC (cq, ntp); + +if (!cq->bi.rx_buff[ntp]) { +if (!buffs_avail) { +/* If the caller hasn't given us any buffers or +* there are none left, search the ring itself +* for an available buffer to move to this +* entry starting at the next entry in the ring +*/ + tbp = ntp +1 ; + +/* Wrap ring if necessary */ +if (tbp >= cq->ring_size) + tbp =0 ; + +while (tbp != cq->next_to_clean) { +if (cq->bi.rx_buff[tbp]) { + cq->bi.rx_buff[ntp] = + cq->bi.rx_buff[tbp]; + cq->bi.rx_buff[tbp] =NULL ; + +/* Found a buffer, no need to +* search anymore +*/ +break ; + } + +/* Wrap ring if necessary */ + tbp++; +if (tbp >= cq->ring_size) + tbp =0 ; + } + +if (tbp == cq->next_to_clean) + goto post_buffs_out; + }else { +/* Give back pointer to DMA buffer */ + cq->bi.rx_buff[ntp] = buffs[i]; + i++; + +if (i >= *buff_count) + buffs_avail = false; + } + } + + desc->flags = +cpu_to_le16 (IECM_CTLQ_FLAG_BUF |IECM_CTLQ_FLAG_RD ); + +/* Post buffers to descriptor */ + desc->datalen =cpu_to_le16 (cq->bi.rx_buff[ntp]->size); + desc->params.indirect.addr_high = +cpu_to_le32 (IECM_HI_DWORD (cq->bi.rx_buff[ntp]->pa)); + desc->params.indirect.addr_low = +cpu_to_le32 (IECM_LO_DWORD (cq->bi.rx_buff[ntp]->pa)); + + ntp++; +if (ntp == cq->ring_size) + ntp =0 ; + } + +post_buffs_out: +/* Only update tail if buffers were actually posted */ +if (cq->next_to_post != ntp) { +if (ntp) +/* Update next_to_post to ntp - 1 since current ntp +* will not have a buffer +*/ + cq->next_to_post = ntp -1 ; +else +/* Wrap to end of end ring since current ntp is 0 */ + cq->next_to_post = cq->ring_size -1 ; + +wr32 (hw, cq->reg.tail, cq->next_to_post); + } + +mutex_unlock (&cq->cq_lock); + +/* return the number of buffers that were not posted */ + *buff_count = *buff_count - i; + +return status; }/**
@@ -196,5 +587,87 @@ enum iecm_status iecm_ctlq_recv(struct iecm_hw *hw,struct iecm_ctlq_info *cq, u16 *num_q_msg, struct iecm_ctlq_msg *q_msg) { - /* stub */ + u16 num_to_clean, ntc, ret_val, flags; +enum iecm_status ret_code =0 ; +struct iecm_ctlq_desc *desc; +u16 i =0 ; + +if (!cq || !cq->ring_size) +return IECM_ERR_CTLQ_EMPTY ; + +if (*num_q_msg ==0 ) +return 0 ; +else if (*num_q_msg > cq->ring_size) +return IECM_ERR_PARAM ; + +/* take the lock before we start messing with the ring */ +mutex_lock (&cq->cq_lock); + + ntc = cq->next_to_clean; + + num_to_clean = *num_q_msg; + +for (i =0 ; i < num_to_clean; i++) { +u64 msg_cookie; + +/* Fetch next descriptor and check if marked as done */ + desc =IECM_CTLQ_DESC (cq, ntc); + flags =le16_to_cpu (desc->flags); + +if (!(flags &IECM_CTLQ_FLAG_DD )) +break ; + + ret_val =le16_to_cpu (desc->ret_val); + + q_msg[i].vmvf_type = (flags & + (IECM_CTLQ_FLAG_FTYPE_VM | +IECM_CTLQ_FLAG_FTYPE_PF )) >> +IECM_CTLQ_FLAG_FTYPE_S ; + +if (flags &IECM_CTLQ_FLAG_ERR ) + ret_code =IECM_ERR_CTLQ_ERROR ; + + msg_cookie = (u64 )le32_to_cpu (desc->cookie_high) <<32 ; + msg_cookie |= (u64 )le32_to_cpu (desc->cookie_low); +memcpy (&q_msg[i].cookie, &msg_cookie,sizeof (u64)); + + q_msg[i].opcode =le16_to_cpu (desc->opcode); + q_msg[i].data_len =le16_to_cpu (desc->datalen); + q_msg[i].status = ret_val; + +if (desc->datalen) { +memcpy (q_msg[i].ctx.indirect.context, + &desc->params.indirect,IECM_INDIRECT_CTX_SIZE ); + +/* Assign pointer to dma buffer to ctlq_msg array +* to be given to upper layer +*/ + q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc]; + +/* Zero out pointer to DMA buffer info; +* will be repopulated by post buffers API +*/ + cq->bi.rx_buff[ntc] =NULL ; + }else { +memcpy (q_msg[i].ctx.direct, desc->params.raw, +IECM_DIRECT_CTX_SIZE ); + } + +/* Zero out stale data in descriptor */ +memset (desc,0 ,sizeof (struct iecm_ctlq_desc )); + + ntc++; +if (ntc == cq->ring_size) + ntc =0 ; + }; + + cq->next_to_clean = ntc; + +mutex_unlock (&cq->cq_lock); + + *num_q_msg = i; +if (*num_q_msg ==0 ) + ret_code =IECM_ERR_CTLQ_NO_WORK ; + +return ret_code; }
drivers/net/ethernet/intel/iecm/iecm_controlq_setup.c +99 -6
139 lines not on this page · raw message
drivers/net/ethernet/intel/iecm/iecm_lib.c +70 -1
77 lines not on this page · raw message
drivers/net/ethernet/intel/iecm/iecm_osdep.c +15 -2
@@ -6,10 +6,23 @@void *iecm_alloc_dma_mem (struct iecm_hw *hw,struct iecm_dma_mem *mem,u64 size) { -/* stub */ +size_t sz =ALIGN (size,4096 ); +struct iecm_adapter *pf = hw->back; + + mem->va =dma_alloc_coherent (&pf->pdev->dev, sz, + &mem->pa,GFP_KERNEL | __GFP_ZERO); + mem->size = size; + +return mem->va; }void iecm_free_dma_mem (struct iecm_hw *hw,struct iecm_dma_mem *mem) { -/* stub */ +struct iecm_adapter *pf = hw->back; + +dma_free_coherent (&pf->pdev->dev, mem->size, + mem->va, mem->pa); + mem->size =0 ; + mem->va =NULL ; + mem->pa =0 ; }
drivers/net/ethernet/intel/iecm/iecm_virtchnl.c +419 -8
475 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH 6/14 iecm: Implement virtchnl commands
In reply to Alice MichaelImplement various virtchnl commands that enable
communication with hardware.
--- .../net/ethernet/intel/iecm/iecm_virtchnl.c | 1171 ++++++++++++++++- 1 file changed, 1144 insertions(+), 27 deletions(-)
drivers/net/ethernet/intel/iecm/iecm_virtchnl.c +1144 -27
@@ -442,7 +442,13 @@ EXPORT_SYMBOL(iecm_recv_mb_msg);static enum iecm_status iecm_send_ver_msg (struct iecm_adapter *adapter) { -/* stub */ +struct virtchnl_version_info vvi; + + vvi.major =VIRTCHNL_VERSION_MAJOR ; + vvi.minor =VIRTCHNL_VERSION_MINOR ; + +return iecm_send_mb_msg (adapter,VIRTCHNL_OP_VERSION ,sizeof (vvi), + (u8 *)&vvi); }/**
@@ -454,7 +460,19 @@ iecm_send_ver_msg(struct iecm_adapter *adapter)static enum iecm_status iecm_recv_ver_msg (struct iecm_adapter *adapter) { -/* stub */ +enum iecm_status err =0 ; +struct virtchnl_version_info vvi; + + err =iecm_recv_mb_msg (adapter,VIRTCHNL_OP_VERSION , &vvi,sizeof (vvi)); +if (err) + goto error; + +if (vvi.major >VIRTCHNL_VERSION_MAJOR || + (vvi.major ==VIRTCHNL_VERSION_MAJOR && + vvi.minor >VIRTCHNL_VERSION_MINOR )) +dev_warn (&adapter->pdev->dev,"Virtchnl version not matched\n" ); +error: +return err; }/**
@@ -466,7 +484,25 @@ iecm_recv_ver_msg(struct iecm_adapter *adapter)enum iecm_status iecm_send_get_caps_msg (struct iecm_adapter *adapter) { -/* stub */ +struct virtchnl_get_capabilities caps = {0 }; +int buf_size; + + buf_size =sizeof (struct virtchnl_get_capabilities ); + adapter->caps =kzalloc (buf_size,GFP_KERNEL ); +if (!adapter->caps) +return IECM_ERR_NO_MEMORY ; + + caps.cap_flags =VIRTCHNL_CAP_STATELESS_OFFLOADS | +VIRTCHNL_CAP_UDP_SEG_OFFLOAD | +VIRTCHNL_CAP_RSS | +VIRTCHNL_CAP_TCP_RSC | +VIRTCHNL_CAP_HEADER_SPLIT | +VIRTCHNL_CAP_RDMA | +VIRTCHNL_CAP_SRIOV | +VIRTCHNL_CAP_EDT ; + +return iecm_send_mb_msg (adapter,VIRTCHNL_OP_GET_CAPS ,sizeof (caps), + (u8 *)&caps); }EXPORT_SYMBOL (iecm_send_get_caps_msg);
@@ -479,7 +515,8 @@ EXPORT_SYMBOL(iecm_send_get_caps_msg);static enum iecm_status iecm_recv_get_caps_msg (struct iecm_adapter *adapter) { -/* stub */ +return iecm_recv_mb_msg (adapter,VIRTCHNL_OP_GET_CAPS , adapter->caps, +sizeof (struct virtchnl_get_capabilities )); }/**
@@ -493,7 +530,30 @@ iecm_recv_get_caps_msg(struct iecm_adapter *adapter)static enum iecm_status iecm_send_create_vport_msg (struct iecm_adapter *adapter) { -/* stub */ +struct virtchnl_create_vport *vport_msg; +enum iecm_status err =0 ; +int buf_size; + + buf_size =sizeof (struct virtchnl_create_vport ); +if (!adapter->vport_params_reqd[0 ]) { + adapter->vport_params_reqd[0 ] =kzalloc (buf_size,GFP_KERNEL ); +if (!adapter->vport_params_reqd[0 ]) { + err =IECM_ERR_NO_MEMORY ; + goto error; + } + } + + vport_msg = (struct virtchnl_create_vport *) + adapter->vport_params_reqd[0 ]; + vport_msg->vport_type =VIRTCHNL_VPORT_TYPE_DEFAULT ; + vport_msg->txq_model =VIRTCHNL_QUEUE_MODEL_SPLIT ; + vport_msg->rxq_model =VIRTCHNL_QUEUE_MODEL_SPLIT ; +iecm_vport_calc_total_qs (vport_msg,0 ); + + err =iecm_send_mb_msg (adapter,VIRTCHNL_OP_CREATE_VPORT , buf_size, + (u8 *)vport_msg); +error: +return err; }/**
@@ -509,7 +569,26 @@ static enum iecm_status iecm_recv_create_vport_msg(struct iecm_adapter *adapter, int *vport_id) { - /* stub */ + struct virtchnl_create_vport *vport_msg; +enum iecm_status err =0 ; + +if (!adapter->vport_params_recvd[0 ]) { + adapter->vport_params_recvd[0 ] =kzalloc (IECM_DFLT_MBX_BUF_SIZE , +GFP_KERNEL ); +if (!adapter->vport_params_recvd[0 ]) { + err =IECM_ERR_NO_MEMORY ; + goto error; + } + } + + vport_msg = (struct virtchnl_create_vport *) + adapter->vport_params_recvd[0 ]; + + err =iecm_recv_mb_msg (adapter,VIRTCHNL_OP_CREATE_VPORT , vport_msg, +IECM_DFLT_MBX_BUF_SIZE ); + *vport_id = vport_msg->vport_id; +error: +return err; }/**
@@ -560,7 +639,20 @@ EXPORT_SYMBOL(iecm_wait_for_event);enum iecm_status iecm_send_destroy_vport_msg (struct iecm_vport *vport) { -/* stub */ +struct iecm_adapter *adapter = vport->adapter; +struct virtchnl_vport v_id; +enum iecm_status err; + + v_id.vport_id = vport->vport_id; + + err =iecm_send_mb_msg (adapter,VIRTCHNL_OP_DESTROY_VPORT , +sizeof (v_id), (u8 *)&v_id); + +if (!err) + err =iecm_wait_for_event (adapter,IECM_VC_DESTROY_VPORT , +IECM_VC_DESTROY_VPORT_ERR ); + +return err; }/**
@@ -598,7 +690,121 @@ iecm_send_disable_vport_msg(struct iecm_vport *vport)enum iecm_status iecm_send_config_tx_queues_msg (struct iecm_vport *vport) { -/* stub */ +struct virtchnl_config_tx_queues *ctq =NULL ; +struct virtchnl_txq_info_v2 *qi; +enum iecm_status err =0 ; +int totqs, num_msgs; +int i, k =0 ; +int num_qs; + + totqs = vport->num_txq + vport->num_complq; + qi =kcalloc (totqs,sizeof (struct virtchnl_txq_info_v2 ),GFP_KERNEL ); +if (!qi) { + err =IECM_ERR_NO_MEMORY ; + goto error; + } + +/* Populate the queue info buffer with all queue context info */ +for (i =0 ; i < vport->num_txq_grp; i++) { +struct iecm_txq_group *tx_qgrp = &vport->txq_grps[i]; +int j; + +for (j =0 ; j < tx_qgrp->num_txq; j++, k++) { + qi[k].queue_id = tx_qgrp->txqs[j].q_id; + qi[k].model = vport->txq_model; + qi[k].type = tx_qgrp->txqs[j].q_type; + qi[k].ring_len = tx_qgrp->txqs[j].desc_count; + qi[k].dma_ring_addr = tx_qgrp->txqs[j].dma; +if (iecm_is_queue_model_split (vport->txq_model)) { +struct iecm_queue *q = &tx_qgrp->txqs[j]; + + qi[k].tx_compl_queue_id = tx_qgrp->complq->q_id; + qi[k].desc_profile = +VIRTCHNL_TXQ_DESC_PROFILE_NATIVE ; + +if (test_bit (__IECM_Q_FLOW_SCH_EN, q->flags)) + qi[k].sched_mode = +VIRTCHNL_TXQ_SCHED_MODE_FLOW ; +else + qi[k].sched_mode = +VIRTCHNL_TXQ_SCHED_MODE_QUEUE ; + }else { + qi[k].sched_mode = +VIRTCHNL_TXQ_SCHED_MODE_QUEUE ; + qi[k].desc_profile = +VIRTCHNL_TXQ_DESC_PROFILE_BASE ; + } + } + +if (iecm_is_queue_model_split (vport->txq_model)) { + qi[k].queue_id = tx_qgrp->complq->q_id; + qi[k].model = vport->txq_model; + qi[k].type = tx_qgrp->complq->q_type; + qi[k].desc_profile =VIRTCHNL_TXQ_DESC_PROFILE_NATIVE ; + qi[k].ring_len = tx_qgrp->complq->desc_count; + qi[k].dma_ring_addr = tx_qgrp->complq->dma; + k++; + } + } + +if (k != totqs) { + err =IECM_ERR_CFG ; + goto error; + } + +/* Chunk up the queue contexts into multiple messages to avoid +* sending a control queue message buffer that is too large +*/ +if (totqs <IECM_NUM_QCTX_PER_MSG ) + num_qs = totqs; +else + num_qs =IECM_NUM_QCTX_PER_MSG ; + + num_msgs = totqs /IECM_NUM_QCTX_PER_MSG ; +if (totqs %IECM_NUM_QCTX_PER_MSG ) + num_msgs++; + +for (i =0 , k =0 ; i < num_msgs || num_qs; i++) { +int buf_size =sizeof (struct virtchnl_config_tx_queues ) + + (sizeof (struct virtchnl_txq_info_v2 ) * (num_qs -1 )); +if (!ctq || num_qs !=IECM_NUM_QCTX_PER_MSG ) { +kfree (ctq); + ctq =kzalloc (buf_size,GFP_KERNEL ); +if (!ctq) { + err =IECM_ERR_NO_MEMORY ; + goto error; + } + }else { +memset (ctq,0 , buf_size); + } + + ctq->vport_id = vport->vport_id; + ctq->num_qinfo = num_qs; +memcpy (ctq->qinfo, &qi[k], +sizeof (struct virtchnl_txq_info_v2 ) * num_qs); + + err =iecm_send_mb_msg (vport->adapter, +VIRTCHNL_OP_CONFIG_TX_QUEUES , + buf_size, (u8 *)ctq); + +if (!err) + err =iecm_wait_for_event (vport->adapter, +IECM_VC_CONFIG_TXQ , +IECM_VC_CONFIG_TXQ_ERR ); +if (err) + goto mbx_error; + + k += num_qs; + totqs -= num_qs; +if (totqs <IECM_NUM_QCTX_PER_MSG ) + num_qs = totqs; + } + +mbx_error: +kfree (ctq); +error: +kfree (qi); +return err; }/**
@@ -612,7 +818,148 @@ iecm_send_config_tx_queues_msg(struct iecm_vport *vport)enum iecm_status iecm_send_config_rx_queues_msg (struct iecm_vport *vport) { -/* stub */ +struct virtchnl_config_rx_queues *crq =NULL ; +struct virtchnl_rxq_info_v2 *qi; +enum iecm_status err =0 ; +int totqs, num_msgs; +int i, k =0 ; +int num_qs; + + totqs = vport->num_rxq + vport->num_bufq; + qi =kcalloc (totqs,sizeof (struct virtchnl_rxq_info_v2 ),GFP_KERNEL ); +if (!qi) { + err =IECM_ERR_NO_MEMORY ; + goto error; + } + +/* Populate the queue info buffer with all queue context info */ +for (i =0 ; i < vport->num_rxq_grp; i++) { +struct iecm_rxq_group *rx_qgrp = &vport->rxq_grps[i]; +int num_rxq; +int j; + +if (iecm_is_queue_model_split (vport->rxq_model)) + num_rxq = rx_qgrp->splitq.num_rxq_sets; +else + num_rxq = rx_qgrp->singleq.num_rxq; + +for (j =0 ; j < num_rxq; j++, k++) { +struct iecm_queue *rxq; + +if (iecm_is_queue_model_split (vport->rxq_model)) { + rxq = &rx_qgrp->splitq.rxq_sets[j].rxq; + qi[k].rx_bufq1_id = + rxq->rxq_grp->splitq.bufq_sets[0 ].bufq.q_id; + qi[k].rx_bufq2_id = + rxq->rxq_grp->splitq.bufq_sets[1 ].bufq.q_id; + qi[k].hdr_buffer_size = rxq->rx_hbuf_size; + qi[k].rsc_low_watermark = + rxq->rsc_low_watermark; + +if (rxq->rx_hsplit_en) { + qi[k].queue_flags = +VIRTCHNL_RXQ_HDR_SPLIT ; + qi[k].hdr_buffer_size = + rxq->rx_hbuf_size; + } +if (iecm_is_feature_ena (vport,NETIF_F_GRO_HW )) + qi[k].queue_flags |=VIRTCHNL_RXQ_RSC ; + }else { + rxq = &rx_qgrp->singleq.rxqs[j]; + } + + qi[k].queue_id = rxq->q_id; + qi[k].model = vport->rxq_model; + qi[k].type = rxq->q_type; + qi[k].desc_profile =VIRTCHNL_TXQ_DESC_PROFILE_BASE ; + qi[k].desc_size =VIRTCHNL_RXQ_DESC_SIZE_32BYTE ; + qi[k].ring_len = rxq->desc_count; + qi[k].dma_ring_addr = rxq->dma; + qi[k].max_pkt_size = rxq->rx_max_pkt_size; + qi[k].data_buffer_size = rxq->rx_buf_size; + } + +if (iecm_is_queue_model_split (vport->rxq_model)) { +for (j =0 ; j <IECM_BUFQS_PER_RXQ_SET ; j++, k++) { +struct iecm_queue *bufq = + &rx_qgrp->splitq.bufq_sets[j].bufq; + + qi[k].queue_id = bufq->q_id; + qi[k].model = vport->rxq_model; + qi[k].type = bufq->q_type; + qi[k].desc_profile = +VIRTCHNL_TXQ_DESC_PROFILE_NATIVE ; + qi[k].desc_size = +VIRTCHNL_RXQ_DESC_SIZE_32BYTE ; + qi[k].ring_len = bufq->desc_count; + qi[k].dma_ring_addr = bufq->dma; + qi[k].data_buffer_size = bufq->rx_buf_size; + qi[k].buffer_notif_stride = + bufq->rx_buf_stride; + qi[k].rsc_low_watermark = + bufq->rsc_low_watermark; + } + } + } + +if (k != totqs) { + err =IECM_ERR_CFG ; + goto error; + } + +/* Chunk up the queue contexts into multiple messages to avoid +* sending a control queue message buffer that is too large +*/ +if (totqs <IECM_NUM_QCTX_PER_MSG ) + num_qs = totqs; +else + num_qs =IECM_NUM_QCTX_PER_MSG ; + + num_msgs = totqs /IECM_NUM_QCTX_PER_MSG ; +if (totqs %IECM_NUM_QCTX_PER_MSG ) + num_msgs++; + +for (i =0 , k =0 ; i < num_msgs || num_qs; i++) { +int buf_size =sizeof (struct virtchnl_config_rx_queues ) + + (sizeof (struct virtchnl_rxq_info_v2 ) * (num_qs -1 )); +if (!crq || num_qs !=IECM_NUM_QCTX_PER_MSG ) { +kfree (crq); + crq =kzalloc (buf_size,GFP_KERNEL ); +if (!crq) { + err =IECM_ERR_NO_MEMORY ; + goto error; + } + }else { +memset (crq,0 , buf_size); + } + + crq->vport_id = vport->vport_id; + crq->num_qinfo = num_qs; +memcpy (crq->qinfo, &qi[k], +sizeof (struct virtchnl_rxq_info_v2 ) * num_qs); + + err =iecm_send_mb_msg (vport->adapter, +VIRTCHNL_OP_CONFIG_RX_QUEUES , + buf_size, (u8 *)crq); + +if (!err) + err =iecm_wait_for_event (vport->adapter, +IECM_VC_CONFIG_RXQ , +IECM_VC_CONFIG_RXQ_ERR ); +if (err) + goto mbx_error; + + k += num_qs; + totqs -= num_qs; +if (totqs <IECM_NUM_QCTX_PER_MSG ) + num_qs = totqs; + } + +mbx_error: +kfree (crq); +error: +kfree (qi); +return err; }/**
@@ -629,7 +976,118 @@ static enum iecm_status iecm_send_ena_dis_queues_msg(struct iecm_vport *vport, enum virtchnl_ops vc_op) { - /* stub */ + struct virtchnl_del_ena_dis_queues *eq; +struct virtchnl_queue_chunk *qc; +int num_txq, num_rxq, num_q; +int i, j, k =0 , buf_size; +enum iecm_status err =0 ; + +/* validate virtchnl op */ +switch (vc_op) { +case VIRTCHNL_OP_ENABLE_QUEUES_V2 : +case VIRTCHNL_OP_DISABLE_QUEUES_V2 : +break ; +default : + err =IECM_ERR_CFG ; + goto error; + } + + num_txq = vport->num_txq + vport->num_complq; + num_rxq = vport->num_rxq + vport->num_bufq; + num_q = num_txq + num_rxq; + buf_size =sizeof (struct virtchnl_del_ena_dis_queues ) + + (sizeof (struct virtchnl_queue_chunk ) * (num_q -1 )); + eq =kzalloc (buf_size,GFP_KERNEL ); +if (!eq) { + err =IECM_ERR_NO_MEMORY ; + goto error; + } + + eq->vport_id = vport->vport_id; + eq->chunks.num_chunks = num_q; + qc = eq->chunks.chunks; + +for (i =0 ; i < vport->num_txq_grp; i++) { +struct iecm_txq_group *tx_qgrp = &vport->txq_grps[i]; + +for (j =0 ; j < tx_qgrp->num_txq; j++, k++) { + qc[k].type = tx_qgrp->txqs[j].q_type; + qc[k].start_queue_id = tx_qgrp->txqs[j].q_id; + qc[k].num_queues =1 ; + } + } +if (vport->num_txq != k) { + err =IECM_ERR_CFG ; + goto err_cfg; + } + +if (iecm_is_queue_model_split (vport->txq_model)) { +for (i =0 ; i < vport->num_txq_grp; i++, k++) { +struct iecm_txq_group *tx_qgrp = &vport->txq_grps[i]; + + qc[k].type = tx_qgrp->complq->q_type; + qc[k].start_queue_id = tx_qgrp->complq->q_id; + qc[k].num_queues =1 ; + } +if (vport->num_complq != (k - vport->num_txq)) { + err =IECM_ERR_CFG ; + goto err_cfg; + } + } + +for (i =0 ; i < vport->num_rxq_grp; i++) { +struct iecm_rxq_group *rx_qgrp = &vport->rxq_grps[i]; + +if (iecm_is_queue_model_split (vport->rxq_model)) + num_rxq = rx_qgrp->splitq.num_rxq_sets; +else + num_rxq = rx_qgrp->singleq.num_rxq; + +for (j =0 ; j < num_rxq; j++, k++) { +if (iecm_is_queue_model_split (vport->rxq_model)) { + qc[k].start_queue_id = + rx_qgrp->splitq.rxq_sets[j].rxq.q_id; + qc[k].type = + rx_qgrp->splitq.rxq_sets[j].rxq.q_type; + }else { + qc[k].start_queue_id = + rx_qgrp->singleq.rxqs[j].q_id; + qc[k].type = + rx_qgrp->singleq.rxqs[j].q_type; + } + qc[k].num_queues =1 ; + } + } +if (vport->num_rxq != k - (vport->num_txq + vport->num_complq)) { + err =IECM_ERR_CFG ; + goto err_cfg; + } + +if (iecm_is_queue_model_split (vport->rxq_model)) { +for (i =0 ; i < vport->num_rxq_grp; i++) { +struct iecm_rxq_group *rx_qgrp = &vport->rxq_grps[i]; +struct iecm_queue *q; + +for (j =0 ; j <IECM_BUFQS_PER_RXQ_SET ; j++, k++) { + q = &rx_qgrp->splitq.bufq_sets[j].bufq; + qc[k].type = q->q_type; + qc[k].start_queue_id = q->q_id; + qc[k].num_queues =1 ; + } + } +if (vport->num_bufq != k - (vport->num_txq + + vport->num_complq + + vport->num_rxq)) { + err =IECM_ERR_CFG ; + goto err_cfg; + } + } + + err =iecm_send_mb_msg (vport->adapter, vc_op, buf_size, (u8 *)eq); +err_cfg: +kfree (eq); +error: +return err; }/**
@@ -646,7 +1104,107 @@ static enum iecm_status iecm_send_map_unmap_queue_vector_msg(struct iecm_vport *vport, bool map) { - /* stub */ +struct iecm_adapter *adapter = vport->adapter; +struct virtchnl_queue_vector_maps *vqvm; +struct virtchnl_queue_vector *vqv; +int buf_size, num_q, i, j, k =0 ; +enum iecm_status err =0 ; + + num_q = vport->num_txq + vport->num_rxq; + + buf_size =sizeof (struct virtchnl_queue_vector_maps ) + + (sizeof (struct virtchnl_queue_vector ) * (num_q -1 )); + vqvm =kzalloc (buf_size,GFP_KERNEL ); +if (!vqvm) { + err =IECM_ERR_NO_MEMORY ; + goto error; + } + + vqvm->vport_id = vport->vport_id; + vqvm->num_maps = num_q; + vqv = vqvm->qv_maps; + +for (i =0 ; i < vport->num_txq_grp; i++) { +struct iecm_txq_group *tx_qgrp = &vport->txq_grps[i]; + +for (j =0 ; j < tx_qgrp->num_txq; j++, k++) { + vqv[k].queue_type = tx_qgrp->txqs[j].q_type; + vqv[k].queue_id = tx_qgrp->txqs[j].q_id; + +if (iecm_is_queue_model_split (vport->txq_model)) { + vqv[k].vector_id = + tx_qgrp->complq->q_vector->v_idx; + vqv[k].itr_idx = + tx_qgrp->complq->itr.itr_idx; + }else { + vqv[k].vector_id = + tx_qgrp->txqs[j].q_vector->v_idx; + vqv[k].itr_idx = + tx_qgrp->txqs[j].itr.itr_idx; + } + } + } + +if (vport->num_txq != k) { + err =IECM_ERR_CFG ; + goto err_cfg; + } + +for (i =0 ; i < vport->num_rxq_grp; i++) { +struct iecm_rxq_group *rx_qgrp = &vport->rxq_grps[i]; +int num_rxq; + +if (iecm_is_queue_model_split (vport->rxq_model)) + num_rxq = rx_qgrp->splitq.num_rxq_sets; +else + num_rxq = rx_qgrp->singleq.num_rxq; + +for (j =0 ; j < num_rxq; j++, k++) { +struct iecm_queue *rxq; + +if (iecm_is_queue_model_split (vport->rxq_model)) + rxq = &rx_qgrp->splitq.rxq_sets[j].rxq; +else + rxq = &rx_qgrp->singleq.rxqs[j]; + + vqv[k].queue_type = rxq->q_type; + vqv[k].queue_id = rxq->q_id; + vqv[k].vector_id = rxq->q_vector->v_idx; + vqv[k].itr_idx = rxq->itr.itr_idx; + } + } + +if (iecm_is_queue_model_split (vport->txq_model)) { +if (vport->num_rxq != k - vport->num_complq) { + err =IECM_ERR_CFG ; + goto err_cfg; + } + }else { +if (vport->num_rxq != k - vport->num_txq) { + err =IECM_ERR_CFG ; + goto err_cfg; + } + } + +if (map) { + err =iecm_send_mb_msg (adapter, +VIRTCHNL_OP_MAP_QUEUE_VECTOR , + buf_size, (u8 *)vqvm); +if (!err) + err =iecm_wait_for_event (adapter,IECM_VC_MAP_IRQ , +IECM_VC_MAP_IRQ_ERR ); + }else { + err =iecm_send_mb_msg (adapter, +VIRTCHNL_OP_UNMAP_QUEUE_VECTOR , + buf_size, (u8 *)vqvm); +if (!err) + err =iecm_wait_for_event (adapter,IECM_VC_UNMAP_IRQ , +IECM_VC_UNMAP_IRQ_ERR ); + } +err_cfg: +kfree (vqvm); +error: +return err; }/**
@@ -658,7 +1216,16 @@ iecm_send_map_unmap_queue_vector_msg(struct iecm_vport *vport,static enum iecm_status iecm_send_enable_queues_msg (struct iecm_vport *vport) { -/* stub */ +struct iecm_adapter *adapter = vport->adapter; +enum iecm_status err; + + err =iecm_send_ena_dis_queues_msg (vport, +VIRTCHNL_OP_ENABLE_QUEUES_V2 ); +if (!err) + err =iecm_wait_for_event (adapter,IECM_VC_ENA_QUEUES , +IECM_VC_ENA_QUEUES_ERR ); + +return err; }/**
@@ -670,7 +1237,15 @@ iecm_send_enable_queues_msg(struct iecm_vport *vport)static enum iecm_status iecm_send_disable_queues_msg (struct iecm_vport *vport) { -/* stub */ +struct iecm_adapter *adapter = vport->adapter; +enum iecm_status err; + + err =iecm_send_ena_dis_queues_msg (vport, +VIRTCHNL_OP_DISABLE_QUEUES_V2 ); +if (!err) + err =iecm_wait_for_event (adapter,IECM_VC_DIS_QUEUES , +IECM_VC_DIS_QUEUES_ERR ); +return err; }/**
@@ -682,7 +1257,48 @@ iecm_send_disable_queues_msg(struct iecm_vport *vport)enum iecm_status iecm_send_delete_queues_msg (struct iecm_vport *vport) { -/* stub */ +struct iecm_adapter *adapter = vport->adapter; +struct virtchnl_create_vport *vport_params; +struct virtchnl_del_ena_dis_queues *eq; +struct virtchnl_queue_chunks *chunks; +int buf_size, num_chunks; +enum iecm_status err; + +if (vport->adapter->config_data.req_qs_chunks) { +struct virtchnl_add_queues *vc_aq = + (struct virtchnl_add_queues *) + vport->adapter->config_data.req_qs_chunks; + chunks = &vc_aq->chunks; + }else { + vport_params = (struct virtchnl_create_vport *) + vport->adapter->vport_params_recvd[0 ]; + chunks = &vport_params->chunks; + } + + num_chunks = chunks->num_chunks; + buf_size =sizeof (struct virtchnl_del_ena_dis_queues ) + + (sizeof (struct virtchnl_queue_chunk ) * + (num_chunks -1 )); + + eq =kzalloc (buf_size,GFP_KERNEL ); +if (!eq) { + err =IECM_ERR_NO_MEMORY ; +return err; + } + eq->vport_id = vport->vport_id; + eq->chunks.num_chunks = num_chunks; + +memcpy (eq->chunks.chunks, chunks->chunks, num_chunks * +sizeof (struct virtchnl_queue_chunk )); + + err =iecm_send_mb_msg (vport->adapter,VIRTCHNL_OP_DEL_QUEUES , + buf_size, (u8 *)eq); +if (!err) + err =iecm_wait_for_event (adapter,IECM_VC_DEL_QUEUES , +IECM_VC_DEL_QUEUES_ERR ); + +kfree (eq); +return err; }/**
@@ -694,7 +1310,14 @@ iecm_send_delete_queues_msg(struct iecm_vport *vport)static enum iecm_status iecm_send_config_queues_msg (struct iecm_vport *vport) { -/* stub */ +enum iecm_status err; + + err =iecm_send_config_tx_queues_msg (vport); + +if (!err) + err =iecm_send_config_rx_queues_msg (vport); + +return err; }/**
@@ -711,7 +1334,55 @@ enum iecm_status iecm_send_add_queues_msg(struct iecm_vport *vport, u16 num_tx_q, u16 num_complq, u16 num_rx_q, u16 num_rx_bufq) { - /* stub */ + struct iecm_adapter *adapter = vport->adapter; +struct virtchnl_add_queues aq = {0 }; +enum iecm_status err; +int size; + + aq.vport_id = vport->vport_id; + aq.num_tx_q = num_tx_q; + aq.num_tx_complq = num_complq; + aq.num_rx_q = num_rx_q; + aq.num_rx_bufq = num_rx_bufq; + + err =iecm_send_mb_msg (adapter, +VIRTCHNL_OP_ADD_QUEUES , +sizeof (struct virtchnl_add_queues ), (u8 *)&aq); + +if (!err) + err =iecm_wait_for_event (adapter,IECM_VC_ADD_QUEUES , +IECM_VC_ADD_QUEUES_ERR ); + +if (!err) { +struct virtchnl_add_queues *vc_msg = + (struct virtchnl_add_queues *)adapter->vc_msg; + +kfree (adapter->config_data.req_qs_chunks); + adapter->config_data.req_qs_chunks =NULL ; + +/* compare vc_msg num queues with vport num queues */ +if (vc_msg->num_tx_q != num_tx_q || + vc_msg->num_rx_q != num_rx_q || + vc_msg->num_tx_complq != num_complq || + vc_msg->num_rx_bufq != num_rx_bufq) +return IECM_ERR_CFG ; + + size =sizeof (struct virtchnl_add_queues ) + + ((vc_msg->chunks.num_chunks -1 ) * +sizeof (struct virtchnl_queue_chunk )); + adapter->config_data.req_qs_chunks = +kzalloc (size,GFP_KERNEL ); +if (!adapter->config_data.req_qs_chunks) { + err =IECM_ERR_NO_MEMORY ; +mutex_unlock (&adapter->vc_msg_lock); + goto mem_err; + } +memcpy (adapter->config_data.req_qs_chunks, + adapter->vc_msg, size); +mutex_unlock (&adapter->vc_msg_lock); + } +mem_err: +return err; }/**
@@ -723,7 +1394,44 @@ iecm_send_add_queues_msg(struct iecm_vport *vport, u16 num_tx_q,enum iecm_status iecm_send_get_stats_msg (struct iecm_vport *vport) { -/* stub */ +struct iecm_adapter *adapter = vport->adapter; +struct virtchnl_queue_select vqs; +enum iecm_status err; + +/* Don't send get_stats message if one is pending or the +* link is down +*/ +if (test_bit (IECM_VC_GET_STATS , adapter->vc_state) || + adapter->state <= __IECM_DOWN) +return 0 ; + + vqs.vsi_id = vport->vport_id; + + err =iecm_send_mb_msg (adapter,VIRTCHNL_OP_GET_STATS , +sizeof (vqs), (u8 *)&vqs); + +if (!err) + err =iecm_wait_for_event (adapter,IECM_VC_GET_STATS , +IECM_VC_GET_STATS_ERR ); + +if (!err) { +struct virtchnl_eth_stats *stats = + (struct virtchnl_eth_stats *)adapter->vc_msg; + vport->netstats.rx_packets = stats->rx_unicast + + stats->rx_multicast + + stats->rx_broadcast; + vport->netstats.tx_packets = stats->tx_unicast + + stats->tx_multicast + + stats->tx_broadcast; + vport->netstats.rx_bytes = stats->rx_bytes; + vport->netstats.tx_bytes = stats->tx_bytes; + vport->netstats.tx_errors = stats->tx_errors; + vport->netstats.rx_dropped = stats->rx_discards; + vport->netstats.tx_dropped = stats->tx_discards; +mutex_unlock (&adapter->vc_msg_lock); + } + +return err; }/**
@@ -736,7 +1444,36 @@ iecm_send_get_stats_msg(struct iecm_vport *vport)enum iecm_status iecm_send_get_set_rss_hash_msg (struct iecm_vport *vport,bool get) { -/* stub */ +struct iecm_adapter *adapter = vport->adapter; +struct virtchnl_rss_hash rh = {0 }; +enum iecm_status err; + + rh.vport_id = vport->vport_id; + rh.hash = adapter->rss_data.rss_hash; + +if (get) { + err =iecm_send_mb_msg (adapter,VIRTCHNL_OP_GET_RSS_HASH , +sizeof (rh), (u8 *)&rh); +if (!err) { + err =iecm_wait_for_event (adapter,IECM_VC_GET_RSS_HASH , +IECM_VC_GET_RSS_HASH_ERR ); +if (!err) { +memcpy (&rh, adapter->vc_msg,sizeof (rh)); + adapter->rss_data.rss_hash = rh.hash; +/* Leave the buffer clean for next message */ +memset (adapter->vc_msg,0 , +IECM_DFLT_MBX_BUF_SIZE ); +mutex_unlock (&adapter->vc_msg_lock); + } + } + }else { + err =iecm_send_mb_msg (adapter,VIRTCHNL_OP_SET_RSS_HASH , +sizeof (rh), (u8 *)&rh); +if (!err) + err =iecm_wait_for_event (adapter,IECM_VC_SET_RSS_HASH , +IECM_VC_SET_RSS_HASH_ERR ); + } +return err; }/**
@@ -749,7 +1486,72 @@ iecm_send_get_set_rss_hash_msg(struct iecm_vport *vport, bool get)enum iecm_status iecm_send_get_set_rss_lut_msg (struct iecm_vport *vport,bool get) { -/* stub */ +struct iecm_adapter *adapter = vport->adapter; +struct virtchnl_rss_lut_v2 *recv_rl; +struct virtchnl_rss_lut_v2 *rl; +enum iecm_status err; +int i, buf_size; + + buf_size =sizeof (struct virtchnl_rss_lut_v2 ) + + (sizeof (u16) * (adapter->rss_data.rss_lut_size -1 )); + rl =kzalloc (buf_size,GFP_KERNEL ); +if (!rl) { + err =IECM_ERR_NO_MEMORY ; + goto error; + } + +if (!get) { + rl->lut_entries = adapter->rss_data.rss_lut_size; +for (i =0 ; i < adapter->rss_data.rss_lut_size; i++) + rl->lut[i] = adapter->rss_data.rss_lut[i]; + } + rl->vport_id = vport->vport_id; + +if (get) { + err =iecm_send_mb_msg (vport->adapter,VIRTCHNL_OP_GET_RSS_LUT , + buf_size, (u8 *)rl); +if (!err) { + err =iecm_wait_for_event (adapter,IECM_VC_GET_RSS_LUT , +IECM_VC_GET_RSS_LUT_ERR ); +if (err) + goto get_lut_err; + recv_rl = (struct virtchnl_rss_lut_v2 *)adapter->vc_msg; +if (adapter->rss_data.rss_lut_size != + recv_rl->lut_entries) { + adapter->rss_data.rss_lut_size = + recv_rl->lut_entries; +kfree (adapter->rss_data.rss_lut); + adapter->rss_data.rss_lut = +kzalloc (adapter->rss_data.rss_lut_size, +GFP_KERNEL ); +if (!adapter->rss_data.rss_lut) { + adapter->rss_data.rss_lut_size =0 ; + err =IECM_ERR_NO_MEMORY ; +/* Leave the buffer clean */ +memset (adapter->vc_msg,0 , +IECM_DFLT_MBX_BUF_SIZE ); +mutex_unlock (&adapter->vc_msg_lock); + goto mem_alloc_err; + } + } +memcpy (adapter->rss_data.rss_lut, adapter->vc_msg, + adapter->rss_data.rss_lut_size); +/* Leave the buffer clean for next message */ +memset (adapter->vc_msg,0 ,IECM_DFLT_MBX_BUF_SIZE ); +mutex_unlock (&adapter->vc_msg_lock); + } + }else { + err =iecm_send_mb_msg (adapter,VIRTCHNL_OP_SET_RSS_LUT , + buf_size, (u8 *)rl); +if (!err) + err =iecm_wait_for_event (adapter,IECM_VC_SET_RSS_LUT , +IECM_VC_SET_RSS_LUT_ERR ); + } +mem_alloc_err: +get_lut_err: +kfree (rl); +error: +return err; }/**
@@ -762,7 +1564,74 @@ iecm_send_get_set_rss_lut_msg(struct iecm_vport *vport, bool get)enum iecm_status iecm_send_get_set_rss_key_msg (struct iecm_vport *vport,bool get) { -/* stub */ +struct iecm_adapter *adapter = vport->adapter; +struct virtchnl_rss_key *recv_rk; +struct virtchnl_rss_key *rk; +enum iecm_status err; +int i, buf_size; + + buf_size =sizeof (struct virtchnl_rss_key ) + + (sizeof (u8) * (adapter->rss_data.rss_key_size -1 )); + rk =kzalloc (buf_size,GFP_KERNEL ); +if (!rk) { + err =IECM_ERR_NO_MEMORY ; + goto error; + } + +if (!get) { + rk->key_len = adapter->rss_data.rss_key_size; +for (i =0 ; i < adapter->rss_data.rss_key_size; i++) + rk->key[i] = adapter->rss_data.rss_key[i]; + } + rk->vsi_id = vport->vport_id; + +if (get) { + err =iecm_send_mb_msg (adapter,VIRTCHNL_OP_GET_RSS_KEY , + buf_size, (u8 *)rk); +if (!err) { + err =iecm_wait_for_event (adapter,IECM_VC_GET_RSS_KEY , +IECM_VC_GET_RSS_KEY_ERR ); +if (err) + goto get_key_err; + recv_rk = (struct virtchnl_rss_key *)adapter->vc_msg; +if (adapter->rss_data.rss_key_size != + recv_rk->key_len) { + adapter->rss_data.rss_key_size = +min_t (u16,NETDEV_RSS_KEY_LEN , + recv_rk->key_len); +kfree (adapter->rss_data.rss_key); + adapter->rss_data.rss_key = +kzalloc (adapter->rss_data.rss_key_size, +GFP_KERNEL ); +if (!adapter->rss_data.rss_key) { + adapter->rss_data.rss_key_size =0 ; + err =IECM_ERR_NO_MEMORY ; +/* Leave the buffer clean */ +memset (adapter->vc_msg,0 , +IECM_DFLT_MBX_BUF_SIZE ); +mutex_unlock (&adapter->vc_msg_lock); + goto mem_alloc_err; + } + } +memcpy (adapter->rss_data.rss_key, adapter->vc_msg, + adapter->rss_data.rss_key_size); +/* Leave the buffer clean for next message */ +memset (adapter->vc_msg,0 ,IECM_DFLT_MBX_BUF_SIZE ); +mutex_unlock (&adapter->vc_msg_lock); + } + }else { + err =iecm_send_mb_msg (adapter,VIRTCHNL_OP_CONFIG_RSS_KEY , + buf_size, (u8 *)rk); +if (!err) + err =iecm_wait_for_event (adapter, +IECM_VC_CONFIG_RSS_KEY , +IECM_VC_CONFIG_RSS_KEY_ERR ); + } +mem_alloc_err: +get_key_err: +kfree (rk); +error: +return err; }/**
@@ -773,7 +1642,24 @@ iecm_send_get_set_rss_key_msg(struct iecm_vport *vport, bool get)*/ enum iecm_status iecm_send_get_rx_ptype_msg (struct iecm_vport *vport) { -/* stub */ +struct iecm_rx_ptype_decoded *rx_ptype_lkup = vport->rx_ptype_lkup; +int ptype_list[IECM_RX_SUPP_PTYPE ] = {0 ,1 ,11 ,12 ,22 ,23 ,24 ,25 ,26 , +27 ,28 ,88 ,89 ,90 ,91 ,92 ,93 , +94 }; +enum iecm_status err =0 ; +int i; + +for (i =0 ; i <IECM_RX_MAX_PTYPE ; i++) + rx_ptype_lkup[i] = iecm_rx_ptype_lkup[0 ]; + +for (i =0 ; i <IECM_RX_SUPP_PTYPE ; i++) { +int j = ptype_list[i]; + + rx_ptype_lkup[j] = iecm_rx_ptype_lkup[i]; + rx_ptype_lkup[j].ptype = ptype_list[i]; + }; + +return err; }/**
@@ -912,7 +1798,54 @@ void iecm_vport_params_buf_rel(struct iecm_adapter *adapter)*/ int iecm_vc_core_init (struct iecm_adapter *adapter,int *vport_id) { -/* stub */ +switch (adapter->state) { +case __IECM_STARTUP: +if (iecm_send_ver_msg (adapter)) + goto init_failed; + adapter->state = __IECM_VER_CHECK; + goto restart; +case __IECM_VER_CHECK: +if (iecm_recv_ver_msg (adapter)) + goto init_failed; + adapter->state = __IECM_GET_CAPS; +if (adapter->dev_ops.vc_ops.get_caps (adapter)) + goto init_failed; + goto restart; +case __IECM_GET_CAPS: +if (iecm_recv_get_caps_msg (adapter)) + goto init_failed; +if (iecm_send_create_vport_msg (adapter)) + goto init_failed; + adapter->state = __IECM_GET_DFLT_VPORT_PARAMS; + goto restart; +case __IECM_GET_DFLT_VPORT_PARAMS: +if (iecm_recv_create_vport_msg (adapter, vport_id)) + goto init_failed; + adapter->state = __IECM_INIT_SW; +break ; +case __IECM_INIT_SW: +break ; +default : +dev_err (&adapter->pdev->dev,"Device is in bad state: %d\n" , + adapter->state); + goto init_failed; + } + +return 0 ; + +restart: +queue_delayed_work (adapter->init_wq, &adapter->init_task, +msecs_to_jiffies (30 )); +/* Not an error. Using try again to continue with state machine */ +return -EAGAIN ; +init_failed: +if (++adapter->mb_wait_count >IECM_MB_MAX_ERR ) { +dev_err (&adapter->pdev->dev,"Failed to establish mailbox communications with hardware\n" ); +return -EFAULT ; + } + adapter->state = __IECM_STARTUP; +queue_delayed_work (adapter->init_wq, &adapter->init_task,HZ ); +return -EAGAIN ; }EXPORT_SYMBOL (iecm_vc_core_init);
@@ -959,7 +1892,32 @@ iecm_vport_get_queue_ids(u16 *qids, int num_qids,enum virtchnl_queue_type q_type, struct virtchnl_queue_chunks *chunks) { - /* stub */ + int num_chunks = chunks->num_chunks; +struct virtchnl_queue_chunk *chunk; +int num_q_id_filled =0 ; +int start_q_id; +int num_q; +int i; + +while (num_chunks) { + chunk = &chunks->chunks[num_chunks -1 ]; +if (chunk->type == q_type) { + num_q = chunk->num_queues; + start_q_id = chunk->start_queue_id; +for (i =0 ; i < num_q; i++) { +if ((num_q_id_filled + i) < num_qids) { + qids[num_q_id_filled + i] = start_q_id; + start_q_id++; + }else { +break ; + } + } + num_q_id_filled = num_q_id_filled + i; + } + num_chunks--; + } + +return num_q_id_filled; }/**
@@ -976,7 +1934,80 @@ static int __iecm_vport_queue_ids_init(struct iecm_vport *vport, u16 *qids, int num_qids, enum virtchnl_queue_type q_type) { - /* stub */ + struct iecm_queue *q; +int i, j, k =0 ; + +switch (q_type) { +case VIRTCHNL_QUEUE_TYPE_TX : +for (i =0 ; i < vport->num_txq_grp; i++) { +struct iecm_txq_group *tx_qgrp = &vport->txq_grps[i]; + +for (j =0 ; j < tx_qgrp->num_txq; j++) { +if (k < num_qids) { + tx_qgrp->txqs[j].q_id = qids[k]; + tx_qgrp->txqs[j].q_type = +VIRTCHNL_QUEUE_TYPE_TX ; + k++; + }else { +break ; + } + } + } +break ; +case VIRTCHNL_QUEUE_TYPE_RX : +for (i =0 ; i < vport->num_rxq_grp; i++) { +struct iecm_rxq_group *rx_qgrp = &vport->rxq_grps[i]; +int num_rxq; + +if (iecm_is_queue_model_split (vport->rxq_model)) + num_rxq = rx_qgrp->splitq.num_rxq_sets; +else + num_rxq = rx_qgrp->singleq.num_rxq; + +for (j =0 ; j < num_rxq && k < num_qids; j++, k++) { +if (iecm_is_queue_model_split (vport->rxq_model)) + q = &rx_qgrp->splitq.rxq_sets[j].rxq; +else + q = &rx_qgrp->singleq.rxqs[j]; + q->q_id = qids[k]; + q->q_type =VIRTCHNL_QUEUE_TYPE_RX ; + } + } +break ; +case VIRTCHNL_QUEUE_TYPE_TX_COMPLETION : +for (i =0 ; i < vport->num_txq_grp; i++) { +struct iecm_txq_group *tx_qgrp = &vport->txq_grps[i]; + +if (k < num_qids) { + tx_qgrp->complq->q_id = qids[k]; + tx_qgrp->complq->q_type = +VIRTCHNL_QUEUE_TYPE_TX_COMPLETION ; + k++; + }else { +break ; + } + } +break ; +case VIRTCHNL_QUEUE_TYPE_RX_BUFFER : +for (i =0 ; i < vport->num_rxq_grp; i++) { +struct iecm_rxq_group *rx_qgrp = &vport->rxq_grps[i]; + +for (j =0 ; j <IECM_BUFQS_PER_RXQ_SET ; j++) { +if (k < num_qids) { + q = &rx_qgrp->splitq.bufq_sets[j].bufq; + q->q_id = qids[k]; + q->q_type = +VIRTCHNL_QUEUE_TYPE_RX_BUFFER ; + k++; + }else { +break ; + } + } + } +break ; + } + +return k; }/**
@@ -989,7 +2020,76 @@ __iecm_vport_queue_ids_init(struct iecm_vport *vport, u16 *qids,static enum iecm_status iecm_vport_queue_ids_init (struct iecm_vport *vport) { -/* stub */ +struct virtchnl_create_vport *vport_params; +struct virtchnl_queue_chunks *chunks; +enum virtchnl_queue_type q_type; +/* We may never deal with more that 256 same type of queues */ +#define IECM_MAX_QIDS 256 +u16 qids[IECM_MAX_QIDS ]; +int num_ids; + +if (vport->adapter->config_data.num_req_qs) { +struct virtchnl_add_queues *vc_aq = + (struct virtchnl_add_queues *) + vport->adapter->config_data.req_qs_chunks; + chunks = &vc_aq->chunks; + }else { + vport_params = (struct virtchnl_create_vport *) + vport->adapter->vport_params_recvd[0 ]; + chunks = &vport_params->chunks; +/* compare vport_params num queues with vport num queues */ +if (vport_params->num_tx_q != vport->num_txq || + vport_params->num_rx_q != vport->num_rxq || + vport_params->num_tx_complq != vport->num_complq || + vport_params->num_rx_bufq != vport->num_bufq) +return IECM_ERR_CFG ; + } + + num_ids =iecm_vport_get_queue_ids (qids,IECM_MAX_QIDS , +VIRTCHNL_QUEUE_TYPE_TX , + chunks); +if (num_ids != vport->num_txq) +return IECM_ERR_CFG ; + num_ids =__iecm_vport_queue_ids_init (vport, qids, num_ids, +VIRTCHNL_QUEUE_TYPE_TX ); +if (num_ids != vport->num_txq) +return IECM_ERR_CFG ; + num_ids =iecm_vport_get_queue_ids (qids,IECM_MAX_QIDS , +VIRTCHNL_QUEUE_TYPE_RX , + chunks); +if (num_ids != vport->num_rxq) +return IECM_ERR_CFG ; + num_ids =__iecm_vport_queue_ids_init (vport, qids, num_ids, +VIRTCHNL_QUEUE_TYPE_RX ); +if (num_ids != vport->num_rxq) +return IECM_ERR_CFG ; + +if (iecm_is_queue_model_split (vport->txq_model)) { + q_type =VIRTCHNL_QUEUE_TYPE_TX_COMPLETION ; + num_ids =iecm_vport_get_queue_ids (qids,IECM_MAX_QIDS , q_type, + chunks); +if (num_ids != vport->num_complq) +return IECM_ERR_CFG ; + num_ids =__iecm_vport_queue_ids_init (vport, qids, + num_ids, + q_type); +if (num_ids != vport->num_complq) +return IECM_ERR_CFG ; + } + +if (iecm_is_queue_model_split (vport->rxq_model)) { + q_type =VIRTCHNL_QUEUE_TYPE_RX_BUFFER ; + num_ids =iecm_vport_get_queue_ids (qids,IECM_MAX_QIDS , q_type, + chunks); +if (num_ids != vport->num_bufq) +return IECM_ERR_CFG ; + num_ids =__iecm_vport_queue_ids_init (vport, qids, num_ids, + q_type); +if (num_ids != vport->num_bufq) +return IECM_ERR_CFG ; + } + +return 0 ; }/**
@@ -1000,7 +2100,23 @@ enum iecm_status iecm_vport_queue_ids_init(struct iecm_vport *vport)*/ enum iecm_status iecm_vport_adjust_qs (struct iecm_vport *vport) { -/* stub */ +struct virtchnl_create_vport vport_msg; +enum iecm_status err; + + vport_msg.txq_model = vport->txq_model; + vport_msg.rxq_model = vport->rxq_model; +iecm_vport_calc_total_qs (&vport_msg, + vport->adapter->config_data.num_req_qs); + err =iecm_send_add_queues_msg (vport, vport_msg.num_tx_q, + vport_msg.num_tx_complq, + vport_msg.num_rx_q, + vport_msg.num_rx_bufq); +if (err) + goto failure; +iecm_vport_init_num_qs (vport, &vport_msg); +iecm_vport_calc_num_q_groups (vport); +failure: +return err; }/**
@@ -1012,7 +2128,8 @@ enum iecm_status iecm_vport_adjust_qs(struct iecm_vport *vport)*/ static bool iecm_is_capability_ena (struct iecm_adapter *adapter,u64 flag) { -/* stub */ +return ((struct virtchnl_get_capabilities *)adapter->caps)->cap_flags & + flag; }/**
Signature · 1 line
2.21.0
PATCH 7/14 iecm: Implement vector allocation
In reply to Alice MichaelThis allocates PCI vectors and maps to interrupt
routines.
--- drivers/net/ethernet/intel/iecm/iecm_lib.c | 63 +- drivers/net/ethernet/intel/iecm/iecm_txrx.c | 606 +++++++++++++++++- .../net/ethernet/intel/iecm/iecm_virtchnl.c | 24 +- 3 files changed, 669 insertions(+), 24 deletions(-)
drivers/net/ethernet/intel/iecm/iecm_lib.c +56 -7
@@ -15,7 +15,11 @@ extern int debug;*/ static void iecm_mb_intr_rel_irq (struct iecm_adapter *adapter) { -/* stub */ +int irq_num; + + irq_num = adapter->msix_entries[0 ].vector; +synchronize_irq (irq_num); +free_irq (irq_num, adapter); }/**
@@ -44,7 +48,12 @@ static void iecm_intr_rel(struct iecm_adapter *adapter)*/ irqreturn_t iecm_mb_intr_clean (int __always_unused irq,void *data) { -/* stub */ +struct iecm_adapter *adapter = (struct iecm_adapter *)data; + +set_bit (__IECM_MB_INTR_TRIGGER, adapter->flags); +queue_delayed_work (adapter->serv_wq, &adapter->serv_task, +msecs_to_jiffies (0 )); +return IRQ_HANDLED ; }/**
@@ -53,7 +62,12 @@ irqreturn_t iecm_mb_intr_clean(int __always_unused irq, void *data)*/ void iecm_mb_irq_enable (struct iecm_adapter *adapter) { -/* stub */ +struct iecm_hw *hw = &adapter->hw; +struct iecm_intr_reg *intr = &adapter->mb_vector.intr_reg; +u32 val; + + val = intr->dyn_ctl_intena_m | intr->dyn_ctl_itridx_m; +writel_relaxed (val, (u8 *)(hw->hw_addr + intr->dyn_ctl)); }/**
@@ -62,7 +76,22 @@ void iecm_mb_irq_enable(struct iecm_adapter *adapter)*/ int iecm_mb_intr_req_irq (struct iecm_adapter *adapter) { -/* stub */ +struct iecm_q_vector *mb_vector = &adapter->mb_vector; +int irq_num, mb_vidx =0 , err; + + irq_num = adapter->msix_entries[mb_vidx].vector; +snprintf (mb_vector->name,sizeof (mb_vector->name) -1 , +"%s-%s-%d" ,dev_driver_string (&adapter->pdev->dev), +"Mailbox" , mb_vidx); + err =request_irq (irq_num, adapter->irq_mb_handler,0 , + mb_vector->name, adapter); +if (err) { +dev_err (&adapter->pdev->dev, +"Request_irq for mailbox failed, error: %d\n" , err); +return err; + } +set_bit (__IECM_MB_INTR_MODE, adapter->flags); +return 0 ; }/**
@@ -74,7 +103,16 @@ int iecm_mb_intr_req_irq(struct iecm_adapter *adapter)*/ void iecm_get_mb_vec_id (struct iecm_adapter *adapter) { -/* stub */ +struct virtchnl_vector_chunks *vchunks; +struct virtchnl_vector_chunk *chunk; + +if (adapter->req_vec_chunks) { + vchunks = &adapter->req_vec_chunks->vchunks; + chunk = &vchunks->num_vchunk[0 ]; + adapter->mb_vector.v_idx = chunk->start_vector_id; + }else { + adapter->mb_vector.v_idx =0 ; + } }/**
@@ -83,7 +121,13 @@ void iecm_get_mb_vec_id(struct iecm_adapter *adapter)*/ int iecm_mb_intr_init (struct iecm_adapter *adapter) { -/* stub */ +int err =0 ; + +iecm_get_mb_vec_id (adapter); + adapter->dev_ops.reg_ops.mb_intr_reg_init (adapter); + adapter->irq_mb_handler = iecm_mb_intr_clean; + err =iecm_mb_intr_req_irq (adapter); +return err; }/**
@@ -95,7 +139,12 @@ int iecm_mb_intr_init(struct iecm_adapter *adapter)*/ void iecm_intr_distribute (struct iecm_adapter *adapter) { -/* stub */ +struct iecm_vport *vport; + + vport = adapter->vports[0 ]; +if (adapter->num_msix_entries != adapter->num_req_msix) + vport->num_q_vectors = adapter->num_msix_entries - +IECM_MAX_NONQ_VEC -IECM_MIN_RDMA_VEC ; }/**
drivers/net/ethernet/intel/iecm/iecm_txrx.c +590 -16
700 lines not on this page · raw message
drivers/net/ethernet/intel/iecm/iecm_virtchnl.c +23 -1
@@ -1874,7 +1874,29 @@ int iecm_vport_get_vec_ids(u16 *vecids, int num_vecids, struct virtchnl_vector_chunks *chunks) { -/* stub */ +int num_chunks = chunks->num_vector_chunks; +struct virtchnl_vector_chunk *chunk; +int num_vecid_filled =0 ; +int start_vecid; +int num_vec; +int i, j; + +for (j =0 ; j < num_chunks; j++) { + chunk = &chunks->num_vchunk[j]; + num_vec = chunk->num_vectors; + start_vecid = chunk->start_vector_id; +for (i =0 ; i < num_vec; i++) { +if ((num_vecid_filled + i) < num_vecids) { + vecids[num_vecid_filled + i] = start_vecid; + start_vecid++; + }else { +break ; + } + } + num_vecid_filled = num_vecid_filled + i; + } + +return num_vecid_filled; }/**
Signature · 1 line
2.21.0
PATCH 8/14 iecm: Init and allocate vport
In reply to Alice MichaelInitialize vport and allocate queue resources.
--- drivers/net/ethernet/intel/iecm/iecm_lib.c | 87 +- drivers/net/ethernet/intel/iecm/iecm_txrx.c | 797 +++++++++++++++++- .../net/ethernet/intel/iecm/iecm_virtchnl.c | 37 +- 3 files changed, 890 insertions(+), 31 deletions(-)
drivers/net/ethernet/intel/iecm/iecm_lib.c +83 -4
@@ -443,7 +443,15 @@ static void iecm_vport_rel_all(struct iecm_adapter *adapter)*/ void iecm_vport_set_hsplit (struct iecm_vport *vport,struct bpf_prog *prog) { -/* stub */ +if (prog) { + vport->rx_hsplit_en =IECM_RX_NO_HDR_SPLIT ; +return ; + } +if (iecm_is_cap_ena (vport->adapter,VIRTCHNL_CAP_HEADER_SPLIT ) && +iecm_is_queue_model_split (vport->rxq_model)) + vport->rx_hsplit_en =IECM_RX_HDR_SPLIT ; +else + vport->rx_hsplit_en =IECM_RX_NO_HDR_SPLIT ; }/**
@@ -531,7 +539,12 @@ static void iecm_service_task(struct work_struct *work)*/ static void iecm_up_complete (struct iecm_vport *vport) { -/* stub */ +netif_set_real_num_rx_queues (vport->netdev, vport->num_txq); +netif_set_real_num_tx_queues (vport->netdev, vport->num_rxq); +netif_carrier_on (vport->netdev); +netif_tx_start_all_queues (vport->netdev); + + vport->adapter->state = __IECM_UP; }/**
@@ -540,7 +553,71 @@ static void iecm_up_complete(struct iecm_vport *vport)*/ static int iecm_vport_open (struct iecm_vport *vport) { -/* stub */ +struct iecm_adapter *adapter = vport->adapter; +int err; + +if (vport->adapter->state != __IECM_DOWN) +return -EBUSY ; + +/* we do not allow interface up just yet */ +netif_carrier_off (vport->netdev); + +if (adapter->dev_ops.vc_ops.enable_vport) { + err = adapter->dev_ops.vc_ops.enable_vport (vport); +if (err) +return -EAGAIN ; + } + +if (iecm_vport_queues_alloc (vport)) { + err = -ENOMEM ; + goto unroll_queues_alloc; + } + + err =iecm_vport_intr_init (vport); +if (err) + goto unroll_intr_init; + + err = vport->adapter->dev_ops.vc_ops.config_queues (vport); +if (err) + goto unroll_config_queues; + err = vport->adapter->dev_ops.vc_ops.irq_map_unmap (vport, true); +if (err) { +dev_err (&vport->adapter->pdev->dev, +"Call to irq_map_unmap returned %d\n" , err); + goto unroll_config_queues; + } + err = vport->adapter->dev_ops.vc_ops.enable_queues (vport); +if (err) + goto unroll_enable_queues; + + err = vport->adapter->dev_ops.vc_ops.get_ptype (vport); +if (err) + goto unroll_get_ptype; + +if (adapter->rss_data.rss_lut) + err =iecm_config_rss (vport); +else + err =iecm_init_rss (vport); +if (err) + goto unroll_init_rss; +iecm_up_complete (vport); + +netif_info (vport->adapter, hw, vport->netdev,"%s\n" , __func__); + +return 0 ; +unroll_init_rss: +unroll_get_ptype: + vport->adapter->dev_ops.vc_ops.disable_queues (vport); +unroll_enable_queues: + vport->adapter->dev_ops.vc_ops.irq_map_unmap (vport, false); +unroll_config_queues: +iecm_vport_intr_deinit (vport); +unroll_intr_init: +iecm_vport_queues_rel (vport); +unroll_queues_alloc: + adapter->dev_ops.vc_ops.disable_vport (vport); + +return err; }/**
@@ -861,7 +938,9 @@ EXPORT_SYMBOL(iecm_shutdown);*/ static int iecm_open (struct net_device *netdev) { -/* stub */ +struct iecm_netdev_priv *np =netdev_priv (netdev); + +return iecm_vport_open (np->vport); }/**
drivers/net/ethernet/intel/iecm/iecm_txrx.c +772 -25
947 lines not on this page · raw message
drivers/net/ethernet/intel/iecm/iecm_virtchnl.c +35 -2
@@ -664,7 +664,20 @@ iecm_send_destroy_vport_msg(struct iecm_vport *vport)enum iecm_status iecm_send_enable_vport_msg (struct iecm_vport *vport) { -/* stub */ +struct iecm_adapter *adapter = vport->adapter; +struct virtchnl_vport v_id; +enum iecm_status err; + + v_id.vport_id = vport->vport_id; + + err =iecm_send_mb_msg (adapter,VIRTCHNL_OP_ENABLE_VPORT , +sizeof (v_id), (u8 *)&v_id); + +if (!err) + err =iecm_wait_for_event (adapter,IECM_VC_ENA_VPORT , +IECM_VC_ENA_VPORT_ERR ); + +return err; }/**
@@ -1858,7 +1871,27 @@ EXPORT_SYMBOL(iecm_vc_core_init);*/ static void iecm_vport_init (struct iecm_vport *vport,int vport_id) { -/* stub */ +struct virtchnl_create_vport *vport_msg; + + vport_msg = (struct virtchnl_create_vport *) + vport->adapter->vport_params_recvd[0 ]; + vport->txq_model = vport_msg->txq_model; + vport->rxq_model = vport_msg->rxq_model; + vport->vport_type = (u16 )vport_msg->vport_type; + vport->vport_id = vport_msg->vport_id; + vport->adapter->rss_data.rss_key_size =min_t (u16,NETDEV_RSS_KEY_LEN , + vport_msg->rss_key_size); + vport->adapter->rss_data.rss_lut_size = vport_msg->rss_lut_size; +ether_addr_copy (vport->default_mac_addr, vport_msg->default_mac_addr); + vport->max_mtu =IECM_MAX_MTU ; + +iecm_vport_set_hsplit (vport,NULL ); + +init_waitqueue_head (&vport->sw_marker_wq); +iecm_vport_init_num_qs (vport, vport_msg); +iecm_vport_calc_num_q_desc (vport); +iecm_vport_calc_num_q_groups (vport); +iecm_vport_calc_num_q_vec (vport); }/**
Signature · 1 line
2.21.0
PATCH 9/14 iecm: Deinit vport
In reply to Alice MichaelImplement vport take down and release its queue
resources.
--- drivers/net/ethernet/intel/iecm/iecm_lib.c | 29 ++- drivers/net/ethernet/intel/iecm/iecm_txrx.c | 218 ++++++++++++++++-- .../net/ethernet/intel/iecm/iecm_virtchnl.c | 15 +- 3 files changed, 246 insertions(+), 16 deletions(-)
drivers/net/ethernet/intel/iecm/iecm_lib.c +27 -2
@@ -366,7 +366,27 @@ struct iecm_adapter *iecm_netdev_to_adapter(struct net_device *netdev)*/ static void iecm_vport_stop (struct iecm_vport *vport) { -/* stub */ +struct iecm_adapter *adapter = vport->adapter; + +if (adapter->state <= __IECM_DOWN) +return ; + adapter->dev_ops.vc_ops.irq_map_unmap (vport, false); + adapter->dev_ops.vc_ops.disable_queues (vport); +/* Normally we ask for queues in create_vport, but if we're changing +* number of requested queues we do a delete then add instead of +* deleting and reallocating the vport. +*/ +if (test_and_clear_bit (__IECM_DEL_QUEUES, + vport->adapter->flags)) +iecm_send_delete_queues_msg (vport); +netif_carrier_off (vport->netdev); +netif_tx_disable (vport->netdev); + adapter->link_up = false; +iecm_vport_intr_deinit (vport); +iecm_vport_queues_rel (vport); +if (adapter->dev_ops.vc_ops.disable_vport) + adapter->dev_ops.vc_ops.disable_vport (vport); + adapter->state = __IECM_DOWN; }/**
@@ -381,7 +401,11 @@ static void iecm_vport_stop(struct iecm_vport *vport)*/ static int iecm_stop (struct net_device *netdev) { -/* stub */ +struct iecm_netdev_priv *np =netdev_priv (netdev); + +iecm_vport_stop (np->vport); + +return 0 ; }/**
@@ -488,6 +512,7 @@ iecm_vport_alloc(struct iecm_adapter *adapter, int vport_id)/* fill vport slot in the adapter struct */ adapter->vports[adapter->next_vport] = vport; +if (iecm_cfg_netdev (vport)) goto cfg_netdev_fail;
drivers/net/ethernet/intel/iecm/iecm_txrx.c +205 -13
294 lines not on this page · raw message
drivers/net/ethernet/intel/iecm/iecm_virtchnl.c +14 -1
@@ -689,7 +689,20 @@ iecm_send_enable_vport_msg(struct iecm_vport *vport)enum iecm_status iecm_send_disable_vport_msg (struct iecm_vport *vport) { -/* stub */ +struct iecm_adapter *adapter = vport->adapter; +struct virtchnl_vport v_id; +enum iecm_status err; + + v_id.vport_id = vport->vport_id; + + err =iecm_send_mb_msg (adapter,VIRTCHNL_OP_DISABLE_VPORT , +sizeof (v_id), (u8 *)&v_id); + +if (!err) + err =iecm_wait_for_event (adapter,IECM_VC_DIS_VPORT , +IECM_VC_DIS_VPORT_ERR ); + +return err; }/**
Signature · 1 line
2.21.0
PATCH 10/14 iecm: Add splitq TX/RX
In reply to Alice MichaelImplement main TX/RX flows for split queue model.
--- drivers/net/ethernet/intel/iecm/iecm_txrx.c | 1283 ++++++++++++++++++- 1 file changed, 1235 insertions(+), 48 deletions(-)
drivers/net/ethernet/intel/iecm/iecm_txrx.c +1235 -48
@@ -11,7 +11,12 @@static enum iecm_status iecm_buf_lifo_push (struct iecm_buf_lifo *stack,struct iecm_tx_buf *buf) { -/* stub */ +if (stack->top == stack->size) +return IECM_ERR_MAX_LIMIT ; + + stack->bufs[stack->top++] = buf; + +return 0 ; }/**
@@ -20,7 +25,10 @@ static enum iecm_status iecm_buf_lifo_push(struct iecm_buf_lifo *stack,**/ static struct iecm_tx_buf *iecm_buf_lifo_pop (struct iecm_buf_lifo *stack) { -/* stub */ +if (!stack->top) +return NULL ; + +return stack->bufs[--stack->top]; }/**
@@ -31,7 +39,16 @@ static struct iecm_tx_buf *iecm_buf_lifo_pop(struct iecm_buf_lifo *stack)void iecm_get_stats64 (struct net_device *netdev,struct rtnl_link_stats64 *stats) { -/* stub */ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); + +iecm_send_get_stats_msg (vport); + stats->rx_packets = vport->netstats.rx_packets; + stats->tx_packets = vport->netstats.tx_packets; + stats->rx_bytes = vport->netstats.rx_bytes; + stats->tx_bytes = vport->netstats.tx_bytes; + stats->tx_errors = vport->netstats.tx_errors; + stats->rx_dropped = vport->netstats.rx_dropped; + stats->tx_dropped = vport->netstats.tx_dropped; }/**
@@ -1246,7 +1263,16 @@ enum iecm_status iecm_vport_queues_alloc(struct iecm_vport *vport)static struct iecm_queue *iecm_tx_find_q (struct iecm_vport *vport,int q_id) { -/* stub */ +int i; + +for (i =0 ; i < vport->num_txq; i++) { +struct iecm_queue *tx_q = vport->txqs[i]; + +if (tx_q->q_id == q_id) +return tx_q; + } + +return NULL ; }/**
@@ -1255,7 +1281,22 @@ iecm_tx_find_q(struct iecm_vport *vport, int q_id)*/ static void iecm_tx_handle_sw_marker (struct iecm_queue *tx_q) { -/* stub */ +struct iecm_vport *vport = tx_q->vport; +bool drain_complete = true; +int i; + +clear_bit (__IECM_Q_SW_MARKER, tx_q->flags); +/* Hardware must write marker packets to all queues associated with +* completion queues. So check if all queues received marker packets +*/ +for (i =0 ; i < vport->num_txq; i++) { +if (test_bit (__IECM_Q_SW_MARKER, vport->txqs[i]->flags)) + drain_complete = false; + } +if (drain_complete) { +set_bit (__IECM_VPORT_SW_MARKER, vport->flags); +wake_up (&vport->sw_marker_wq); + } }/**
@@ -1270,7 +1311,30 @@ static struct iecm_tx_queue_stats iecm_tx_splitq_clean_buf(struct iecm_queue *tx_q, struct iecm_tx_buf *tx_buf, int napi_budget) { - /* stub */ + struct iecm_tx_queue_stats cleaned = {0}; +struct netdev_queue *nq; + +/* update the statistics for this packet */ + cleaned.bytes = tx_buf->bytecount; + cleaned.packets = tx_buf->gso_segs; + +/* free the skb */ +napi_consume_skb (tx_buf->skb, napi_budget); + nq =netdev_get_tx_queue (tx_q->vport->netdev, tx_q->idx); +netdev_tx_completed_queue (nq, cleaned.packets, + cleaned.bytes); + +/* unmap skb header data */ +dma_unmap_single (tx_q->dev, +dma_unmap_addr (tx_buf, dma), +dma_unmap_len (tx_buf, len), +DMA_TO_DEVICE ); + +/* clear tx_buf data */ + tx_buf->skb =NULL ; +dma_unmap_len_set (tx_buf, len,0 ); + +return cleaned; }/**
@@ -1282,7 +1346,33 @@ iecm_tx_splitq_clean_buf(struct iecm_queue *tx_q, struct iecm_tx_buf *tx_buf,static int iecm_stash_flow_sch_buffers (struct iecm_queue *txq,struct iecm_tx_buf *tx_buf) { -/* stub */ +struct iecm_adapter *adapter = txq->vport->adapter; +struct iecm_tx_buf *shadow_buf; + + shadow_buf =iecm_buf_lifo_pop (&txq->buf_stack); +if (!shadow_buf) { +dev_err (&adapter->pdev->dev, +"No out-of-order TX buffers left!\n" ); +return -ENOMEM ; + } + +/* Store buffer params in shadow buffer */ + shadow_buf->skb = tx_buf->skb; + shadow_buf->bytecount = tx_buf->bytecount; + shadow_buf->gso_segs = tx_buf->gso_segs; + shadow_buf->dma = tx_buf->dma; + shadow_buf->len = tx_buf->len; + shadow_buf->compl_tag = tx_buf->compl_tag; + +/* Add buffer to buf_hash table to be freed +* later +*/ +hash_add (txq->sched_buf_hash, &shadow_buf->hlist, + shadow_buf->compl_tag); + +memset (tx_buf,0 ,sizeof (struct iecm_tx_buf )); + +return 0 ; }/**
@@ -1305,7 +1395,91 @@ static struct iecm_tx_queue_stats iecm_tx_splitq_clean(struct iecm_queue *tx_q, u16 end, int napi_budget, bool descs_only) { - /* stub */ + union iecm_tx_flex_desc *next_pending_desc = NULL; +struct iecm_tx_queue_stats cleaned_stats = {0 }; +union iecm_tx_flex_desc *tx_desc; +s16 ntc = tx_q->next_to_clean; +struct iecm_tx_buf *tx_buf; + + tx_desc =IECM_FLEX_TX_DESC (tx_q, ntc); + next_pending_desc =IECM_FLEX_TX_DESC (tx_q, end); + tx_buf = &tx_q->tx_buf[ntc]; + ntc -= tx_q->desc_count; + +while (tx_desc != next_pending_desc) { +union iecm_tx_flex_desc *eop_desc = + (union iecm_tx_flex_desc *)tx_buf->next_to_watch; + +/* clear next_to_watch to prevent false hangs */ + tx_buf->next_to_watch =NULL ; + +if (descs_only) { +if (iecm_stash_flow_sch_buffers (tx_q, tx_buf)) + goto tx_splitq_clean_out; + +while (tx_desc != eop_desc) { + tx_buf++; + tx_desc++; + ntc++; +if (unlikely (!ntc)) { + ntc -= tx_q->desc_count; + tx_buf = tx_q->tx_buf; + tx_desc =IECM_FLEX_TX_DESC (tx_q,0 ); + } + +if (dma_unmap_len (tx_buf, len)) { +if (iecm_stash_flow_sch_buffers (tx_q, + tx_buf)) + goto tx_splitq_clean_out; + } + } + }else { +struct iecm_tx_queue_stats buf_stats = {0 }; + + buf_stats =iecm_tx_splitq_clean_buf (tx_q, tx_buf, + napi_budget); + +/* update the statistics for this packet */ + cleaned_stats.bytes += buf_stats.bytes; + cleaned_stats.packets += buf_stats.packets; + +/* unmap remaining buffers */ +while (tx_desc != eop_desc) { + tx_buf++; + tx_desc++; + ntc++; +if (unlikely (!ntc)) { + ntc -= tx_q->desc_count; + tx_buf = tx_q->tx_buf; + tx_desc =IECM_FLEX_TX_DESC (tx_q,0 ); + } + +/* unmap any remaining paged data */ +if (dma_unmap_len (tx_buf, len)) { +dma_unmap_page (tx_q->dev, +dma_unmap_addr (tx_buf, dma), +dma_unmap_len (tx_buf, len), +DMA_TO_DEVICE ); +dma_unmap_len_set (tx_buf, len,0 ); + } + } + } + + tx_buf++; + tx_desc++; + ntc++; +if (unlikely (!ntc)) { + ntc -= tx_q->desc_count; + tx_buf = tx_q->tx_buf; + tx_desc =IECM_FLEX_TX_DESC (tx_q,0 ); + } + } + +tx_splitq_clean_out: + ntc += tx_q->desc_count; + tx_q->next_to_clean = ntc; + +return cleaned_stats; }/**
@@ -1315,7 +1489,18 @@ iecm_tx_splitq_clean(struct iecm_queue *tx_q, u16 end, int napi_budget,*/ static inline void iecm_tx_hw_tstamp (struct sk_buff *skb,u8 *desc_ts) { -/* stub */ +struct skb_shared_hwtstamps hwtstamps; +u64 tstamp; + +/* Only report timestamp to stack if requested */ +if (!likely (skb_shinfo (skb)->tx_flags &SKBTX_HW_TSTAMP )) +return ; + + tstamp = (desc_ts[0 ] | (desc_ts[1 ] <<8 ) | (desc_ts[2 ] &0x3F ) <<16 ); + hwtstamps.hwtstamp = +ns_to_ktime (tstamp <<IECM_TW_TIME_STAMP_GRAN_512_DIV_S ); + +skb_tstamp_tx (skb, &hwtstamps); }/**
@@ -1330,7 +1515,39 @@ static struct iecm_tx_queue_stats iecm_tx_clean_flow_sch_bufs(struct iecm_queue *txq, u16 compl_tag, u8 *desc_ts, int budget) { - /* stub */ + struct iecm_tx_queue_stats cleaned_stats = {0}; +struct hlist_node *tmp_buf =NULL ; +struct iecm_tx_buf *tx_buf =NULL ; + +/* Buffer completion */ +hash_for_each_possible_safe (txq->sched_buf_hash, tx_buf, tmp_buf, + hlist, compl_tag) { +if (tx_buf->compl_tag != compl_tag) +continue ; + +if (likely (tx_buf->skb)) { +/* fetch timestamp from completion +* descriptor to report to stack +*/ +iecm_tx_hw_tstamp (tx_buf->skb, desc_ts); + + cleaned_stats =iecm_tx_splitq_clean_buf (txq, tx_buf, + budget); + }else if (dma_unmap_len (tx_buf, len)) { +dma_unmap_page (txq->dev, +dma_unmap_addr (tx_buf, dma), +dma_unmap_len (tx_buf, len), +DMA_TO_DEVICE ); +dma_unmap_len_set (tx_buf, len,0 ); + } + +/* Push shadow buf back onto stack */ +iecm_buf_lifo_push (&txq->buf_stack, tx_buf); + +hash_del (&tx_buf->hlist); + } + +return cleaned_stats; }/**
@@ -1343,7 +1560,109 @@ iecm_tx_clean_flow_sch_bufs(struct iecm_queue *txq, u16 compl_tag,static bool iecm_tx_clean_complq (struct iecm_queue *complq,int budget) { -/* stub */ +struct iecm_splitq_tx_compl_desc *tx_desc; +struct iecm_vport *vport = complq->vport; +s16 ntc = complq->next_to_clean; +unsigned int complq_budget; + + complq_budget = vport->compln_clean_budget; + tx_desc =IECM_SPLITQ_TX_COMPLQ_DESC (complq, ntc); + ntc -= complq->desc_count; + +do { +struct iecm_tx_queue_stats cleaned_stats = {0 }; +bool descs_only = false; +struct iecm_queue *tx_q; +u16 compl_tag, hw_head; +int tx_qid; +u8 ctype;/* completion type */ +u16 gen; + +/* if the descriptor isn't done, no work yet to do */ + gen = (le16_to_cpu (tx_desc->qid_comptype_gen) & +IECM_TXD_COMPLQ_GEN_M ) >>IECM_TXD_COMPLQ_GEN_S ; +if (test_bit (__IECM_Q_GEN_CHK, complq->flags) != gen) +break ; + +/* Find necessary info of TX queue to clean buffers */ + tx_qid = (le16_to_cpu (tx_desc->qid_comptype_gen) & +IECM_TXD_COMPLQ_QID_M ) >>IECM_TXD_COMPLQ_QID_S ; + tx_q =iecm_tx_find_q (vport, tx_qid); +if (!tx_q) { +dev_err (&complq->vport->adapter->pdev->dev, +"TxQ #%d not found\n" , tx_qid); + goto fetch_next_desc; + } + +/* Determine completion type */ + ctype = (le16_to_cpu (tx_desc->qid_comptype_gen) & +IECM_TXD_COMPLQ_COMPL_TYPE_M ) >> +IECM_TXD_COMPLQ_COMPL_TYPE_S ; +switch (ctype) { +case IECM_TXD_COMPLT_RE : + hw_head =le16_to_cpu (tx_desc->q_head_compl_tag.q_head); + + cleaned_stats =iecm_tx_splitq_clean (tx_q, hw_head, + budget, + descs_only); +break ; +case IECM_TXD_COMPLT_RS : +if (test_bit (__IECM_Q_FLOW_SCH_EN, tx_q->flags)) { + compl_tag = +le16_to_cpu (tx_desc->q_head_compl_tag.compl_tag); + + cleaned_stats = +iecm_tx_clean_flow_sch_bufs (tx_q, + compl_tag, + tx_desc->ts, + budget); + }else { + hw_head = +le16_to_cpu (tx_desc->q_head_compl_tag.q_head); + + cleaned_stats =iecm_tx_splitq_clean (tx_q, + hw_head, + budget, + false); + } + +break ; +case IECM_TXD_COMPLT_SW_MARKER : +iecm_tx_handle_sw_marker (tx_q); +break ; +default : +dev_err (&tx_q->vport->adapter->pdev->dev, +"Unknown TX completion type: %d\n" , + ctype); + goto fetch_next_desc; + } + + tx_q->itr.stats.tx.packets += cleaned_stats.packets; + tx_q->itr.stats.tx.bytes += cleaned_stats.bytes; +u64_stats_update_begin (&tx_q->stats_sync); + tx_q->q_stats.tx.packets += cleaned_stats.packets; + tx_q->q_stats.tx.bytes += cleaned_stats.bytes; +u64_stats_update_end (&tx_q->stats_sync); + +fetch_next_desc: + tx_desc++; + ntc++; +if (unlikely (!ntc)) { + ntc -= complq->desc_count; + tx_desc =IECM_SPLITQ_TX_COMPLQ_DESC (complq,0 ); +change_bit (__IECM_Q_GEN_CHK, complq->flags); + } + +prefetch (tx_desc); + +/* update budget accounting */ + complq_budget--; + }while (likely (complq_budget)); + + ntc += complq->desc_count; + complq->next_to_clean = ntc; + +return !!complq_budget; }/**
@@ -1359,7 +1678,12 @@ iecm_tx_splitq_build_ctb(union iecm_tx_flex_desc *desc,struct iecm_tx_splitq_params *parms, u16 td_cmd, u16 size) { - /* stub */ + desc->q.qw1.cmd_dtype = +cpu_to_le16 (parms->dtype &IECM_FLEX_TXD_QW1_DTYPE_M ); + desc->q.qw1.cmd_dtype |= +cpu_to_le16 ((td_cmd <<IECM_FLEX_TXD_QW1_CMD_S ) & +IECM_FLEX_TXD_QW1_CMD_M ); + desc->q.qw1.buf_size =cpu_to_le16 ((u16 )size); }/**
@@ -1375,7 +1699,13 @@ iecm_tx_splitq_build_flow_desc(union iecm_tx_flex_desc *desc,struct iecm_tx_splitq_params *parms, u16 td_cmd, u16 size) { - /* stub */ + desc->flow.qw1.cmd_dtype = cpu_to_le16((u16)parms->dtype | td_cmd); + desc->flow.qw1.rxr_bufsize =cpu_to_le16 ((u16 )size); + desc->flow.qw1.compl_tag =cpu_to_le16 (parms->compl_tag); + + desc->flow.qw1.ts[0 ] = parms->offload.desc_ts &0xff ; + desc->flow.qw1.ts[1 ] = (parms->offload.desc_ts >>8 ) &0xff ; + desc->flow.qw1.ts[2 ] = (parms->offload.desc_ts >>16 ) &0xff ; }/**
@@ -1388,7 +1718,19 @@ iecm_tx_splitq_build_flow_desc(union iecm_tx_flex_desc *desc,static int __iecm_tx_maybe_stop (struct iecm_queue *tx_q,unsigned int size) { -/* stub */ +netif_stop_subqueue (tx_q->vport->netdev, tx_q->idx); + +/* Memory barrier before checking head and tail */ +smp_mb (); + +/* Check again in a case another CPU has just made room available. */ +if (likely (IECM_DESC_UNUSED (tx_q) < size)) +return -EBUSY ; + +/* A reprieve! - use start_subqueue because it doesn't call schedule */ +netif_start_subqueue (tx_q->vport->netdev, tx_q->idx); + +return 0 ; }/**
@@ -1400,7 +1742,10 @@ __iecm_tx_maybe_stop(struct iecm_queue *tx_q, unsigned int size)*/ int iecm_tx_maybe_stop (struct iecm_queue *tx_q,unsigned int size) { -/* stub */ +if (likely (IECM_DESC_UNUSED (tx_q) >= size)) +return 0 ; + +return __iecm_tx_maybe_stop (tx_q, size); }/**
@@ -1412,7 +1757,23 @@ int iecm_tx_maybe_stop(struct iecm_queue *tx_q, unsigned int size)void iecm_tx_buf_hw_update (struct iecm_queue *tx_q,u32 val,struct sk_buff *skb) { -/* stub */ +struct netdev_queue *nq; + + nq =netdev_get_tx_queue (tx_q->vport->netdev, tx_q->idx); + tx_q->next_to_use = val; + +iecm_tx_maybe_stop (tx_q,IECM_TX_DESC_NEEDED ); + +/* Force memory writes to complete before letting h/w +* know there are new descriptors to fetch. (Only +* applicable for weak-ordered memory model archs, +* such as IA-64). +*/ +wmb (); + +/* notify HW of packet */ +if (netif_xmit_stopped (nq) || !netdev_xmit_more ()) +writel_relaxed (val, tx_q->tail); }/**
@@ -1445,7 +1806,7 @@ void iecm_tx_buf_hw_update(struct iecm_queue *tx_q, u32 val,*/ static unsigned int __iecm_tx_desc_count_required (unsigned int size) { -/* stub */ +return ((size *85 ) >>20 ) +IECM_TX_DESCS_FOR_SKB_DATA_PTR ; }/**
@@ -1456,13 +1817,26 @@ static unsigned int __iecm_tx_desc_count_required(unsigned int size)*/ unsigned int iecm_tx_desc_count_required (struct sk_buff *skb) { -/* stub */ +const skb_frag_t *frag = &skb_shinfo (skb)->frags[0 ]; +unsigned int nr_frags =skb_shinfo (skb)->nr_frags; +unsigned int count =0 , size =skb_headlen (skb); + +for (;;) { + count +=__iecm_tx_desc_count_required (size); + +if (!nr_frags--) +break ; + + size =skb_frag_size (frag++); + } + +return count; }/** * iecm_tx_splitq_map - Build the Tx flex descriptor * @tx_q: queue to send buffer on -* @off: pointer to offload params struct +* @parms: pointer to splitq params struct * @first: first buffer info buffer to use * * This function loops over the skb data pointed to by *first
@@ -1471,10 +1845,130 @@ unsigned int iecm_tx_desc_count_required(struct sk_buff *skb)*/ static void iecm_tx_splitq_map (struct iecm_queue *tx_q, -struct iecm_tx_offload_params *off, +struct iecm_tx_splitq_params *parms,struct iecm_tx_buf *first) { -/* stub */ +union iecm_tx_flex_desc *tx_desc; +unsigned int data_len, size; +struct iecm_tx_buf *tx_buf; +u16 i = tx_q->next_to_use; +struct netdev_queue *nq; +struct sk_buff *skb; +skb_frag_t *frag; +u16 td_cmd =0 ; +dma_addr_t dma; + + skb = first->skb; + + td_cmd = parms->offload.td_cmd; + parms->compl_tag = tx_q->tx_buf_key; + + data_len = skb->data_len; + size =skb_headlen (skb); + + tx_desc =IECM_FLEX_TX_DESC (tx_q, i); + + dma =dma_map_single (tx_q->dev, skb->data, size,DMA_TO_DEVICE ); + + tx_buf = first; + +for (frag = &skb_shinfo (skb)->frags[0 ];; frag++) { +unsigned int max_data =IECM_TX_MAX_DESC_DATA_ALIGNED ; + +if (dma_mapping_error (tx_q->dev, dma)) + goto dma_error; + +/* record length, and DMA address */ +dma_unmap_len_set (tx_buf, len, size); +dma_unmap_addr_set (tx_buf, dma, dma); + +/* align size to end of page */ + max_data += -dma & (IECM_TX_MAX_READ_REQ_SIZE -1 ); + +/* buf_addr is in same location for both desc types */ + tx_desc->q.buf_addr =cpu_to_le64 (dma); + +/* account for data chunks larger than the hardware +* can handle +*/ +while (unlikely (size >IECM_TX_MAX_DESC_DATA )) { + parms->splitq_build_ctb (tx_desc, parms, td_cmd, size); + + tx_desc++; + i++; + +if (i == tx_q->desc_count) { + tx_desc =IECM_FLEX_TX_DESC (tx_q,0 ); + i =0 ; + } + + dma += max_data; + size -= max_data; + + max_data =IECM_TX_MAX_DESC_DATA_ALIGNED ; +/* buf_addr is in same location for both desc types */ + tx_desc->q.buf_addr =cpu_to_le64 (dma); + } + +if (likely (!data_len)) +break ; + parms->splitq_build_ctb (tx_desc, parms, td_cmd, size); + tx_desc++; + i++; + +if (i == tx_q->desc_count) { + tx_desc =IECM_FLEX_TX_DESC (tx_q,0 ); + i =0 ; + } + + size =skb_frag_size (frag); + data_len -= size; + + dma =skb_frag_dma_map (tx_q->dev, frag,0 , size, +DMA_TO_DEVICE ); + + tx_buf->compl_tag = parms->compl_tag; + tx_buf = &tx_q->tx_buf[i]; + } + +/* record bytecount for BQL */ + nq =netdev_get_tx_queue (tx_q->vport->netdev, tx_q->idx); +netdev_tx_sent_queue (nq, first->bytecount); + +/* record SW timestamp if HW timestamp is not available */ +skb_tx_timestamp (first->skb); + +/* write last descriptor with RS and EOP bits */ + td_cmd |= parms->eop_cmd; + parms->splitq_build_ctb (tx_desc, parms, td_cmd, size); + i++; +if (i == tx_q->desc_count) + i =0 ; + +/* set next_to_watch value indicating a packet is present */ + first->next_to_watch = tx_desc; + tx_buf->compl_tag = parms->compl_tag++; + +iecm_tx_buf_hw_update (tx_q, i, skb); + +/* Update TXQ Completion Tag key for next buffer */ + tx_q->tx_buf_key = parms->compl_tag; + +return ; + +dma_error: +/* clear dma mappings for failed tx_buf map */ +for (;;) { + tx_buf = &tx_q->tx_buf[i]; +iecm_tx_buf_rel (tx_q, tx_buf); +if (tx_buf == first) +break ; +if (i ==0 ) + i = tx_q->desc_count; + i--; + } + + tx_q->next_to_use = i; }/**
@@ -1490,7 +1984,79 @@ iecm_tx_splitq_map(struct iecm_queue *tx_q,static int iecm_tso (struct iecm_tx_buf *first,struct iecm_tx_offload_params *off) { -/* stub */ +struct sk_buff *skb = first->skb; +union { +struct iphdr *v4; +struct ipv6hdr *v6; +unsigned char *hdr; + } ip; +union { +struct tcphdr *tcp; +struct udphdr *udp; +unsigned char *hdr; + } l4; +u32 paylen, l4_start; +int err; + +if (skb->ip_summed !=CHECKSUM_PARTIAL ) +return 0 ; + +if (!skb_is_gso (skb)) +return 0 ; + + err =skb_cow_head (skb,0 ); +if (err <0 ) +return err; + + ip.hdr =skb_network_header (skb); + l4.hdr =skb_transport_header (skb); + +/* initialize outer IP header fields */ +if (ip.v4->version ==4 ) { + ip.v4->tot_len =0 ; + ip.v4->check =0 ; + }else { + ip.v6->payload_len =0 ; + } + +/* determine offset of transport header */ + l4_start = l4.hdr - skb->data; + +/* remove payload length from checksum */ + paylen = skb->len - l4_start; + +switch (skb_shinfo (skb)->gso_type) { +case SKB_GSO_TCPV4 : +case SKB_GSO_TCPV6 : +csum_replace_by_diff (&l4.tcp->check, + (__force __wsum )htonl (paylen)); + +/* compute length of segmentation header */ + off->tso_hdr_len = (l4.tcp->doff *4 ) + l4_start; +break ; +case SKB_GSO_UDP_L4 : +csum_replace_by_diff (&l4.udp->check, + (__force __wsum )htonl (paylen)); +/* compute length of segmentation header */ + off->tso_hdr_len =sizeof (struct udphdr ) + l4_start; + l4.udp->len = +htons (skb_shinfo (skb)->gso_size + +sizeof (struct udphdr )); +break ; +default : +return -EINVAL ; + } + + off->tso_len = skb->len - off->tso_hdr_len; + off->mss =skb_shinfo (skb)->gso_size; + +/* update gso_segs and bytecount */ + first->gso_segs =skb_shinfo (skb)->gso_segs; + first->bytecount += (first->gso_segs -1 ) * off->tso_hdr_len; + + first->tx_flags |=IECM_TX_FLAGS_TSO ; + +return 0 ; }/**
@@ -1503,7 +2069,84 @@ static int iecm_tso(struct iecm_tx_buf *first,static netdev_tx_t iecm_tx_splitq_frame (struct sk_buff *skb,struct iecm_queue *tx_q) { -/* stub */ +struct iecm_tx_splitq_params tx_parms = {0 }; +struct iecm_tx_buf *first; +unsigned int count; + + count =iecm_tx_desc_count_required (skb); + +/* need: 1 descriptor per page * PAGE_SIZE/IECM_MAX_DATA_PER_TXD, +* + 1 desc for skb_head_len/IECM_MAX_DATA_PER_TXD, +* + 4 desc gap to avoid the cache line where head is, +* + 1 desc for context descriptor, +* otherwise try next time +*/ +if (iecm_tx_maybe_stop (tx_q, count +IECM_TX_DESCS_PER_CACHE_LINE + +IECM_TX_DESCS_FOR_CTX )) { +return NETDEV_TX_BUSY ; + } + +/* record the location of the first descriptor for this packet */ + first = &tx_q->tx_buf[tx_q->next_to_use]; + first->skb = skb; + first->bytecount = max_t(unsigned int, skb->len, ETH_ZLEN); + first->gso_segs =1 ; + first->tx_flags =0 ; + +if (iecm_tso (first, &tx_parms.offload) <0 ) { +/* If tso returns an error, drop the packet */ +dev_kfree_skb_any (skb); +return NETDEV_TX_OK ; + } + +if (first->tx_flags &IECM_TX_FLAGS_TSO ) { +/* If tso is needed, set up context desc */ +union iecm_flex_tx_ctx_desc *ctx_desc; +int i = tx_q->next_to_use; + +/* grab the next descriptor */ + ctx_desc =IECM_FLEX_TX_CTX_DESC (tx_q, i); + i++; + tx_q->next_to_use = (i < tx_q->desc_count) ? i :0 ; + + ctx_desc->tso.qw1.cmd_dtype |= +cpu_to_le16 (IECM_TX_DESC_DTYPE_FLEX_TSO_CTX | +IECM_TX_FLEX_CTX_DESC_CMD_TSO ); + ctx_desc->tso.qw0.flex_tlen = +cpu_to_le32 (tx_parms.offload.tso_len & +IECM_TXD_FLEX_CTX_TLEN_M ); + ctx_desc->tso.qw0.mss_rt = +cpu_to_le16 (tx_parms.offload.mss & +IECM_TXD_FLEX_CTX_MSS_RT_M ); + ctx_desc->tso.qw0.hdr_len = tx_parms.offload.tso_hdr_len; + } + +if (test_bit (__IECM_Q_FLOW_SCH_EN, tx_q->flags)) { +s64 ts_ns = first->skb->skb_mstamp_ns; + + tx_parms.offload.desc_ts = + ts_ns >>IECM_TW_TIME_STAMP_GRAN_512_DIV_S ; + + tx_parms.dtype =IECM_TX_DESC_DTYPE_FLEX_FLOW_SCHE ; + tx_parms.splitq_build_ctb = iecm_tx_splitq_build_flow_desc; + tx_parms.eop_cmd = +IECM_TXD_FLEX_FLOW_CMD_EOP |IECM_TXD_FLEX_FLOW_CMD_RE ; + +if (skb->ip_summed ==CHECKSUM_PARTIAL ) + tx_parms.offload.td_cmd |=IECM_TXD_FLEX_FLOW_CMD_CS_EN ; + + }else { + tx_parms.dtype =IECM_TX_DESC_DTYPE_FLEX_DATA ; + tx_parms.splitq_build_ctb = iecm_tx_splitq_build_ctb; + tx_parms.eop_cmd =IECM_TX_DESC_CMD_EOP |IECM_TX_DESC_CMD_RS ; + +if (skb->ip_summed ==CHECKSUM_PARTIAL ) + tx_parms.offload.td_cmd |=IECM_TX_FLEX_DESC_CMD_CS_EN ; + } + +iecm_tx_splitq_map (tx_q, &tx_parms, first); + +return NETDEV_TX_OK ; }/**
@@ -1516,7 +2159,18 @@ iecm_tx_splitq_frame(struct sk_buff *skb, struct iecm_queue *tx_q)netdev_tx_t iecm_tx_splitq_start (struct sk_buff *skb,struct net_device *netdev) { -/* stub */ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); +struct iecm_queue *tx_q; + + tx_q = vport->txqs[skb->queue_mapping]; + +/* hardware can't handle really short frames, hardware padding works +* beyond this point +*/ +if (skb_put_padto (skb,IECM_TX_MIN_LEN )) +return NETDEV_TX_OK ; + +return iecm_tx_splitq_frame (skb, tx_q); }/**
@@ -1531,7 +2185,18 @@ netdev_tx_t iecm_tx_splitq_start(struct sk_buff *skb,static enum pkt_hash_types iecm_ptype_to_htype (struct iecm_vport *vport,u16 ptype) { -/* stub */ +struct iecm_rx_ptype_decoded decoded = vport->rx_ptype_lkup[ptype]; + +if (!decoded.known) +return PKT_HASH_TYPE_NONE ; +if (decoded.payload_layer ==IECM_RX_PTYPE_PAYLOAD_LAYER_PAY4 ) +return PKT_HASH_TYPE_L4 ; +if (decoded.payload_layer ==IECM_RX_PTYPE_PAYLOAD_LAYER_PAY3 ) +return PKT_HASH_TYPE_L3 ; +if (decoded.outer_ip ==IECM_RX_PTYPE_OUTER_L2 ) +return PKT_HASH_TYPE_L2 ; + +return PKT_HASH_TYPE_NONE ; }/**
@@ -1545,7 +2210,17 @@ static void iecm_rx_hash(struct iecm_queue *rxq, struct sk_buff *skb, struct iecm_flex_rx_desc *rx_desc, u16 ptype) { - /* stub */ + u32 hash; + +if (!iecm_is_feature_ena (rxq->vport,NETIF_F_RXHASH )) +return ; + + hash = rx_desc->status_err1 | + (rx_desc->fflags1 <<8 ) | + (rx_desc->ts_low <<16 ) | + (rx_desc->ff2_mirrid_hash2.hash2 <<24 ); + +skb_set_hash (skb, hash,iecm_ptype_to_htype (rxq->vport, ptype)); }/**
@@ -1561,7 +2236,63 @@ static void iecm_rx_csum(struct iecm_queue *rxq, struct sk_buff *skb, struct iecm_flex_rx_desc *rx_desc, u16 ptype) { - /* stub */ + struct iecm_rx_ptype_decoded decoded; +u8 rx_status_0_qw1, rx_status_0_qw0; +bool ipv4, ipv6; + +/* Start with CHECKSUM_NONE and by default csum_level = 0 */ + skb->ip_summed =CHECKSUM_NONE ; + +/* check if Rx checksum is enabled */ +if (!iecm_is_feature_ena (rxq->vport,NETIF_F_RXCSUM )) +return ; + + rx_status_0_qw1 = rx_desc->status_err0_qw1; +/* check if HW has decoded the packet and checksum */ +if (!(rx_status_0_qw1 &BIT (IECM_RX_FLEX_DESC_STATUS0_L3L4P_S ))) +return ; + + decoded = rxq->vport->rx_ptype_lkup[ptype]; +if (!(decoded.known && decoded.outer_ip)) +return ; + + ipv4 = (decoded.outer_ip ==IECM_RX_PTYPE_OUTER_IP ) && + (decoded.outer_ip_ver ==IECM_RX_PTYPE_OUTER_IPV4 ); + ipv6 = (decoded.outer_ip ==IECM_RX_PTYPE_OUTER_IP ) && + (decoded.outer_ip_ver ==IECM_RX_PTYPE_OUTER_IPV6 ); + +if (ipv4 && (rx_status_0_qw1 & + (BIT (IECM_RX_FLEX_DESC_STATUS0_XSUM_IPE_S ) | +BIT (IECM_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S )))) + goto checksum_fail; + + rx_status_0_qw0 = rx_desc->status_err0_qw0; +if (ipv6 && (rx_status_0_qw0 & + (BIT (IECM_RX_FLEX_DESC_STATUS0_IPV6EXADD_S )))) +return ; + +/* check for L4 errors and handle packets that were not able to be +* checksummed +*/ +if (rx_status_0_qw1 &BIT (IECM_RX_FLEX_DESC_STATUS0_XSUM_L4E_S )) + goto checksum_fail; + +/* Only report checksum unnecessary for ICMP, TCP, UDP, or SCTP */ +switch (decoded.inner_prot) { +case IECM_RX_PTYPE_INNER_PROT_ICMP : +case IECM_RX_PTYPE_INNER_PROT_TCP : +case IECM_RX_PTYPE_INNER_PROT_UDP : +case IECM_RX_PTYPE_INNER_PROT_SCTP : + skb->ip_summed =CHECKSUM_UNNECESSARY ; + rxq->q_stats.rx.basic_csum++; +default : +break ; + } +return ; + +checksum_fail: + rxq->q_stats.rx.csum_err++; +dev_dbg (rxq->dev,"RX Checksum not available\n" ); }/**
@@ -1577,7 +2308,74 @@ iecm_rx_csum(struct iecm_queue *rxq, struct sk_buff *skb,static bool iecm_rx_rsc (struct iecm_queue *rxq,struct sk_buff *skb,struct iecm_flex_rx_desc *rx_desc,u16 ptype) { -/* stub */ +struct iecm_rx_ptype_decoded decoded; +u16 rsc_segments, rsc_payload_len; +struct ipv6hdr *ipv6h; +struct tcphdr *tcph; +struct iphdr *ipv4h; +bool ipv4, ipv6; +u16 hdr_len; + + rsc_payload_len =le32_to_cpu (rx_desc->fmd1_misc.rscseglen); +if (!rsc_payload_len) + goto rsc_err; + + decoded = rxq->vport->rx_ptype_lkup[ptype]; +if (!(decoded.known && decoded.outer_ip)) + goto rsc_err; + + ipv4 = (decoded.outer_ip ==IECM_RX_PTYPE_OUTER_IP ) && + (decoded.outer_ip_ver ==IECM_RX_PTYPE_OUTER_IPV4 ); + ipv6 = (decoded.outer_ip ==IECM_RX_PTYPE_OUTER_IP ) && + (decoded.outer_ip_ver ==IECM_RX_PTYPE_OUTER_IPV6 ); + +if (!(ipv4 ^ ipv6)) + goto rsc_err; + +if (ipv4) + hdr_len =ETH_HLEN +sizeof (struct tcphdr ) + +sizeof (struct iphdr ); +else + hdr_len =ETH_HLEN +sizeof (struct tcphdr ) + +sizeof (struct ipv6hdr ); + + rsc_segments =DIV_ROUND_UP (skb->len - hdr_len, rsc_payload_len); + +NAPI_GRO_CB (skb)->count = rsc_segments; +skb_shinfo (skb)->gso_size = rsc_payload_len; + +skb_reset_network_header (skb); + +if (ipv4) { + ipv4h =ip_hdr (skb); +skb_shinfo (skb)->gso_type =SKB_GSO_TCPV4 ; + +/* Reset and set transport header offset in skb */ +skb_set_transport_header (skb,sizeof (struct iphdr )); + tcph =tcp_hdr (skb); + +/* Compute the TCP pseudo header checksum*/ + tcph->check = + ~tcp_v4_check (skb->len -skb_transport_offset (skb), + ipv4h->saddr, ipv4h->daddr,0 ); + }else { + ipv6h =ipv6_hdr (skb); +skb_shinfo (skb)->gso_type =SKB_GSO_TCPV6 ; +skb_set_transport_header (skb,sizeof (struct ipv6hdr )); + tcph =tcp_hdr (skb); + tcph->check = + ~tcp_v6_check (skb->len -skb_transport_offset (skb), + &ipv6h->saddr, &ipv6h->daddr,0 ); + } + +tcp_gro_complete (skb); + +/* Map rx qid to the skb*/ +skb_record_rx_queue (skb, rxq->q_id); + +return true; +rsc_err: +return false; }/**
@@ -1589,7 +2387,19 @@ static bool iecm_rx_rsc(struct iecm_queue *rxq, struct sk_buff *skb,static void iecm_rx_hwtstamp (struct iecm_flex_rx_desc *rx_desc,struct sk_buff __maybe_unused *skb) { -/* stub */ +u8 ts_lo = rx_desc->ts_low; +u32 ts_hi =0 ; +u64 ts_ns =0 ; + + ts_hi =le32_to_cpu (rx_desc->flex_ts.ts_high); + + ts_ns |= ts_lo | ((u64 )ts_hi <<8 ); + +if (ts_ns) { +memset (skb_hwtstamps (skb),0 , +sizeof (struct skb_shared_hwtstamps )); +skb_hwtstamps (skb)->hwtstamp =ns_to_ktime (ts_ns); + } }/**
@@ -1606,7 +2416,26 @@ static bool iecm_rx_process_skb_fields(struct iecm_queue *rxq, struct sk_buff *skb, struct iecm_flex_rx_desc *rx_desc) { - /* stub */ + bool err = false; +u16 rx_ptype; +bool rsc; + + rx_ptype =le16_to_cpu (rx_desc->ptype_err_fflags0) & +IECM_RXD_FLEX_PTYPE_M ; + +/* modifies the skb - consumes the enet header */ + skb->protocol =eth_type_trans (skb, rxq->vport->netdev); +iecm_rx_csum (rxq, skb, rx_desc, rx_ptype); +/* process RSS/hash */ +iecm_rx_hash (rxq, skb, rx_desc, rx_ptype); + + rsc =le16_to_cpu (rx_desc->hdrlen_flags) &IECM_RXD_FLEX_RSC_M ; +if (rsc) + err =iecm_rx_rsc (rxq, skb, rx_desc, rx_ptype); + +iecm_rx_hwtstamp (rx_desc, skb); + +return err; }/**
@@ -1619,7 +2448,7 @@ iecm_rx_process_skb_fields(struct iecm_queue *rxq, struct sk_buff *skb,*/ void iecm_rx_skb (struct iecm_queue *rxq,struct sk_buff *skb) { -/* stub */ +napi_gro_receive (&rxq->q_vector->napi, skb); }/**
@@ -1628,7 +2457,7 @@ void iecm_rx_skb(struct iecm_queue *rxq, struct sk_buff *skb)*/ static bool iecm_rx_page_is_reserved (struct page *page) { -/* stub */ +return (page_to_nid (page) !=numa_mem_id ()) ||page_is_pfmemalloc (page); }/**
@@ -1644,7 +2473,13 @@ static bool iecm_rx_page_is_reserved(struct page *page)static void iecm_rx_buf_adjust_pg_offset (struct iecm_rx_buf *rx_buf,unsigned int size) { -/* stub */ +#if (PAGE_SIZE <8192 ) +/* flip page offset to other buffer */ + rx_buf->page_offset ^= size; +#else +/* move offset up to the next cache line */ + rx_buf->page_offset += size; +#endif }/**
@@ -1658,7 +2493,34 @@ iecm_rx_buf_adjust_pg_offset(struct iecm_rx_buf *rx_buf, unsigned int size)*/ static bool iecm_rx_can_reuse_page (struct iecm_rx_buf *rx_buf) { -/* stub */ +#if (PAGE_SIZE >=8192 ) +#endif +unsigned int pagecnt_bias = rx_buf->pagecnt_bias; +struct page *page = rx_buf->page; + +/* avoid re-using remote pages */ +if (unlikely (iecm_rx_page_is_reserved (page))) +return false; + +#if (PAGE_SIZE <8192 ) +/* if we are only owner of page we can reuse it */ +if (unlikely ((page_count (page) - pagecnt_bias) >1 )) +return false; +#else +if (rx_buf->page_offset > last_offset) +return false; +#endif /* PAGE_SIZE < 8192) */ + +/* If we have drained the page fragment pool we need to update +* the pagecnt_bias and page count so that we fully restock the +* number of references the driver holds. +*/ +if (unlikely (pagecnt_bias ==1 )) { +page_ref_add (page,USHRT_MAX -1 ); + rx_buf->pagecnt_bias =USHRT_MAX ; + } + +return true; }/**
@@ -1674,7 +2536,17 @@ static bool iecm_rx_can_reuse_page(struct iecm_rx_buf *rx_buf)void iecm_rx_add_frag (struct iecm_rx_buf *rx_buf,struct sk_buff *skb,unsigned int size) { -/* stub */ +#if (PAGE_SIZE >=8192 ) +unsigned int truesize =SKB_DATA_ALIGN (size); +#else +unsigned int truesize =IECM_RX_BUF_2048 ; +#endif + +skb_add_rx_frag (skb,skb_shinfo (skb)->nr_frags, rx_buf->page, + rx_buf->page_offset, size, truesize); + +/* page is being used so we must update the page offset */ +iecm_rx_buf_adjust_pg_offset (rx_buf, truesize); }/**
@@ -1689,7 +2561,22 @@ void iecm_rx_reuse_page(struct iecm_queue *rx_bufq,bool hsplit, struct iecm_rx_buf *old_buf) { - /* stub */ + u16 ntu = rx_bufq->next_to_use; +struct iecm_rx_buf *new_buf; + +if (hsplit) + new_buf = &rx_bufq->rx_buf.hdr_buf[ntu]; +else + new_buf = &rx_bufq->rx_buf.buf[ntu]; + +/* Transfer page from old buffer to new buffer. +* Move each member individually to avoid possible store +* forwarding stalls and unnecessary copy of skb. +*/ + new_buf->dma = old_buf->dma; + new_buf->page = old_buf->page; + new_buf->page_offset = old_buf->page_offset; + new_buf->pagecnt_bias = old_buf->pagecnt_bias; }/**
@@ -1704,7 +2591,15 @@ static void iecm_rx_get_buf_page(struct device *dev, struct iecm_rx_buf *rx_buf, const unsigned int size) { - /* stub */ + prefetch(rx_buf->page); + +/* we are reusing so sync this buffer for CPU use */ +dma_sync_single_range_for_cpu (dev, rx_buf->dma, + rx_buf->page_offset, size, +DMA_FROM_DEVICE ); + +/* We have pulled a buffer for use, so decrement pagecnt_bias */ + rx_buf->pagecnt_bias--; }/**
@@ -1721,7 +2616,52 @@ struct sk_buff * iecm_rx_construct_skb(struct iecm_queue *rxq, struct iecm_rx_buf *rx_buf, unsigned int size) { - /* stub */ + void *va = page_address(rx_buf->page) + rx_buf->page_offset; +unsigned int headlen; +struct sk_buff *skb; + +/* prefetch first cache line of first page */ +prefetch (va); +#if L1_CACHE_BYTES <128 +prefetch ((u8 *)va +L1_CACHE_BYTES ); +#endif /* L1_CACHE_BYTES */ +/* allocate a skb to store the frags */ + skb =__napi_alloc_skb (&rxq->q_vector->napi,IECM_RX_HDR_SIZE , +GFP_ATOMIC | __GFP_NOWARN); +if (unlikely (!skb)) +return NULL ; + +skb_record_rx_queue (skb, rxq->idx); + +/* Determine available headroom for copy */ + headlen = size; +if (headlen >IECM_RX_HDR_SIZE ) + headlen =eth_get_headlen (skb->dev, va,IECM_RX_HDR_SIZE ); + +/* align pull length to size of long to optimize memcpy performance */ +memcpy (__skb_put (skb, headlen), va,ALIGN (headlen,sizeof (long ))); + +/* if we exhaust the linear part then add what is left as a frag */ + size -= headlen; +if (size) { +#if (PAGE_SIZE >=8192 ) +unsigned int truesize =SKB_DATA_ALIGN (size); +#else +unsigned int truesize =IECM_RX_BUF_2048 ; +#endif +skb_add_rx_frag (skb,0 , rx_buf->page, + rx_buf->page_offset + headlen, size, truesize); +/* buffer is used by skb, update page_offset */ +iecm_rx_buf_adjust_pg_offset (rx_buf, truesize); + }else { +/* buffer is unused, reset bias back to rx_buf; data was copied +* onto skb's linear part so there's no need for adjusting +* page offset and we can reuse this buffer as-is +*/ + rx_buf->pagecnt_bias++; + } + +return skb; }/**
@@ -1738,7 +2678,11 @@ iecm_rx_construct_skb(struct iecm_queue *rxq, struct iecm_rx_buf *rx_buf,*/ bool iecm_rx_cleanup_headers (struct sk_buff *skb) { -/* stub */ +/* if eth_skb_pad returns an error the skb was freed */ +if (eth_skb_pad (skb)) +return true; + +return false; }/**
@@ -1751,7 +2695,7 @@ bool iecm_rx_cleanup_headers(struct sk_buff *skb)static bool iecm_rx_splitq_test_staterr (u8 stat_err_field,const u8 stat_err_bits) { -/* stub */ +return !!(stat_err_field & stat_err_bits); }/**
@@ -1764,7 +2708,13 @@ iecm_rx_splitq_test_staterr(u8 stat_err_field, const u8 stat_err_bits)static bool iecm_rx_splitq_is_non_eop (struct iecm_flex_rx_desc *rx_desc) { -/* stub */ +/* if we are the last buffer then there is nothing else to do */ +#define IECM_RXD_EOF BIT(IECM_RX_FLEX_DESC_STATUS0_EOF_S) +if (likely (iecm_rx_splitq_test_staterr (rx_desc->status_err0_qw1, +IECM_RXD_EOF ))) +return false; + +return true; }/**
@@ -1781,7 +2731,24 @@ iecm_rx_splitq_is_non_eop(struct iecm_flex_rx_desc *rx_desc)bool iecm_rx_recycle_buf (struct iecm_queue *rx_bufq,bool hsplit,struct iecm_rx_buf *rx_buf) { -/* stub */ +bool recycled = false; + +if (iecm_rx_can_reuse_page (rx_buf)) { +/* hand second half of page back to the queue */ +iecm_rx_reuse_page (rx_bufq, hsplit, rx_buf); + recycled = true; + }else { +/* we are not reusing the buffer so unmap it */ +dma_unmap_page_attrs (rx_bufq->dev, rx_buf->dma,PAGE_SIZE , +DMA_FROM_DEVICE ,IECM_RX_DMA_ATTR ); +__page_frag_cache_drain (rx_buf->page, rx_buf->pagecnt_bias); + } + +/* clear contents of buffer_info */ + rx_buf->page =NULL ; + rx_buf->skb =NULL ; + +return recycled; }/**
@@ -1797,7 +2764,19 @@ static void iecm_rx_splitq_put_bufs(struct iecm_queue *rx_bufq,struct iecm_rx_buf *hdr_buf, struct iecm_rx_buf *rx_buf) { - /* stub */ + u16 ntu = rx_bufq->next_to_use; +bool recycled = false; + +if (likely (hdr_buf)) + recycled =iecm_rx_recycle_buf (rx_bufq, true, hdr_buf); +if (likely (rx_buf)) + recycled =iecm_rx_recycle_buf (rx_bufq, false, rx_buf); + +/* update, and store next to alloc if the buffer was recycled */ +if (recycled) { + ntu++; + rx_bufq->next_to_use = (ntu < rx_bufq->desc_count) ? ntu :0 ; + } }/**
@@ -1806,7 +2785,14 @@ static void iecm_rx_splitq_put_bufs(struct iecm_queue *rx_bufq,*/ static void iecm_rx_bump_ntc (struct iecm_queue *q) { -/* stub */ +u16 ntc = q->next_to_clean +1 ; +/* fetch, update, and store next to clean */ +if (ntc < q->desc_count) { + q->next_to_clean = ntc; + }else { + q->next_to_clean =0 ; +change_bit (__IECM_Q_GEN_CHK, q->flags); + } }/**
@@ -1823,7 +2809,158 @@ static void iecm_rx_bump_ntc(struct iecm_queue *q)*/ static int iecm_rx_splitq_clean (struct iecm_queue *rxq,int budget) { -/* stub */ +unsigned int total_rx_bytes =0 , total_rx_pkts =0 ; +u16 cleaned_count[IECM_BUFQS_PER_RXQ_SET ] = {0 }; +struct iecm_queue *rx_bufq =NULL ; +struct sk_buff *skb = rxq->skb; +bool failure = false; +int i; + +/* Process Rx packets bounded by budget */ +while (likely (total_rx_pkts < (unsigned int )budget)) { +struct iecm_flex_rx_desc *splitq_flex_rx_desc; +union iecm_rx_desc *rx_desc; +struct iecm_rx_buf *hdr_buf =NULL ; +struct iecm_rx_buf *rx_buf =NULL ; +unsigned int pkt_len =0 ; +unsigned int hdr_len =0 ; +u16 gen_id, buf_id; +u8 stat_err0_qw0; +u8 stat_err_bits; +/* Header buffer overflow only valid for header split */ +bool hbo = false; +int bufq_id; + +/* get the Rx desc from Rx queue based on 'next_to_clean' */ + rx_desc =IECM_RX_DESC (rxq, rxq->next_to_clean); + splitq_flex_rx_desc = (struct iecm_flex_rx_desc *)rx_desc; + +/* This memory barrier is needed to keep us from reading +* any other fields out of the rx_desc +*/ +dma_rmb (); + +/* if the descriptor isn't done, no work yet to do */ + gen_id =le16_to_cpu (splitq_flex_rx_desc->pktlen_gen_bufq_id); + gen_id = (gen_id &IECM_RXD_FLEX_GEN_M ) >>IECM_RXD_FLEX_GEN_S ; +if (test_bit (__IECM_Q_GEN_CHK, rxq->flags) != gen_id) +break ; + + pkt_len =le16_to_cpu (splitq_flex_rx_desc->pktlen_gen_bufq_id) & +IECM_RXD_FLEX_LEN_PBUF_M ; + + hbo =le16_to_cpu (splitq_flex_rx_desc->status_err0_qw1) & +BIT (IECM_RX_FLEX_DESC_STATUS0_HBO_S ); + +if (unlikely (hbo)) { + rxq->q_stats.rx.hsplit_hbo++; + goto bypass_hsplit; + } + + hdr_len = +le16_to_cpu (splitq_flex_rx_desc->hdrlen_flags) & +IECM_RXD_FLEX_LEN_HDR_M ; + +bypass_hsplit: + bufq_id =le16_to_cpu (splitq_flex_rx_desc->pktlen_gen_bufq_id); + bufq_id = (bufq_id &IECM_RXD_FLEX_BUFQ_ID_M ) >> +IECM_RXD_FLEX_BUFQ_ID_S ; +/* retrieve buffer from the rxq */ + rx_bufq = &rxq->rxq_grp->splitq.bufq_sets[bufq_id].bufq; + + buf_id =le16_to_cpu (splitq_flex_rx_desc->fmd0_bufid.buf_id); + +if (pkt_len) { + rx_buf = &rx_bufq->rx_buf.buf[buf_id]; +iecm_rx_get_buf_page (rx_bufq->dev, rx_buf, pkt_len); + } + +if (hdr_len) { + hdr_buf = &rx_bufq->rx_buf.hdr_buf[buf_id]; +iecm_rx_get_buf_page (rx_bufq->dev, hdr_buf, + hdr_len); + + skb =iecm_rx_construct_skb (rxq, hdr_buf, hdr_len); + } + +if (skb && pkt_len) +iecm_rx_add_frag (rx_buf, skb, pkt_len); +else if (pkt_len) + skb =iecm_rx_construct_skb (rxq, rx_buf, pkt_len); + +/* exit if we failed to retrieve a buffer */ +if (!skb) { +/* If we fetched a buffer, but didn't use it +* undo pagecnt_bias decrement +*/ +if (rx_buf) + rx_buf->pagecnt_bias++; +break ; + } + +iecm_rx_splitq_put_bufs (rx_bufq, hdr_buf, rx_buf); +iecm_rx_bump_ntc (rxq); + cleaned_count[bufq_id]++; + +/* skip if it is non EOP desc */ +if (iecm_rx_splitq_is_non_eop (splitq_flex_rx_desc)) +continue ; + + stat_err_bits =BIT (IECM_RX_FLEX_DESC_STATUS0_RXE_S ); + stat_err0_qw0 = splitq_flex_rx_desc->status_err0_qw0; +if (unlikely (iecm_rx_splitq_test_staterr (stat_err0_qw0, + stat_err_bits))) { +dev_kfree_skb_any (skb); + skb =NULL ; +continue ; + } + +/* correct empty headers and pad skb if needed (to make valid +* ethernet frame +*/ +if (iecm_rx_cleanup_headers (skb)) { + skb =NULL ; +continue ; + } + +/* probably a little skewed due to removing CRC */ + total_rx_bytes += skb->len; + +/* protocol */ +if (unlikely (iecm_rx_process_skb_fields (rxq, skb, + splitq_flex_rx_desc))) { +dev_kfree_skb_any (skb); + skb =NULL ; +continue ; + } + +/* send completed skb up the stack */ +iecm_rx_skb (rxq, skb); + skb =NULL ; + +/* update budget accounting */ + total_rx_pkts++; + } +for (i =0 ; i <IECM_BUFQS_PER_RXQ_SET ; i++) { +if (cleaned_count[i]) { + rx_bufq = &rxq->rxq_grp->splitq.bufq_sets[i].bufq; + failure =iecm_rx_buf_hw_alloc_all (rx_bufq, + cleaned_count[i]) || + failure; + } + } + + rxq->skb = skb; +u64_stats_update_begin (&rxq->stats_sync); + rxq->q_stats.rx.packets += total_rx_pkts; + rxq->q_stats.rx.bytes += total_rx_bytes; +u64_stats_update_end (&rxq->stats_sync); + + rxq->itr.stats.rx.packets += total_rx_pkts; + rxq->itr.stats.rx.bytes += total_rx_bytes; + +/* guarantee a trip back through this routine if there was a failure */ +return failure ? budget : (int )total_rx_pkts; }/**
@@ -2379,7 +3516,15 @@ iecm_vport_intr_napi_ena_all(struct iecm_vport *vport)static inline bool iecm_tx_splitq_clean_all (struct iecm_q_vector *q_vec,int budget) { -/* stub */ +bool clean_complete = true; +int i, budget_per_q; + + budget_per_q =max (budget / q_vec->num_txq,1 ); +for (i =0 ; i < q_vec->num_txq; i++) { +if (!iecm_tx_clean_complq (q_vec->tx[i], budget_per_q)) + clean_complete = false; + } +return clean_complete; }/**
@@ -2394,7 +3539,22 @@ static inline bool iecm_rx_splitq_clean_all(struct iecm_q_vector *q_vec, int budget, int *cleaned) { - /* stub */ + bool clean_complete = true; +int pkts_cleaned_per_q; +int i, budget_per_q; + + budget_per_q =max (budget / q_vec->num_rxq,1 ); +for (i =0 ; i < q_vec->num_rxq; i++) { + pkts_cleaned_per_q =iecm_rx_splitq_clean (q_vec->rx[i], + budget_per_q); +/* if we clean as many as budgeted, we must not +* be done +*/ +if (pkts_cleaned_per_q >= budget_per_q) + clean_complete = false; + *cleaned += pkts_cleaned_per_q; + } +return clean_complete; }/**
@@ -2404,7 +3564,34 @@ iecm_rx_splitq_clean_all(struct iecm_q_vector *q_vec, int budget,*/ int iecm_vport_splitq_napi_poll (struct napi_struct *napi,int budget) { -/* stub */ +struct iecm_q_vector *q_vector = + container_of(napi, struct iecm_q_vector, napi); +bool clean_complete; +int work_done =0 ; + + clean_complete =iecm_tx_splitq_clean_all (q_vector, budget); + +/* Handle case where we are called by netpoll with a budget of 0 */ +if (budget <=0 ) +return budget; + +/* We attempt to distribute budget to each Rx queue fairly, but don't +* allow the budget to go below 1 because that would exit polling early. +*/ + clean_complete |=iecm_rx_splitq_clean_all (q_vector, budget, + &work_done); + +/* If work not completed, return budget and polling will return */ +if (!clean_complete) +return budget; + +/* Exit the polling mode, but don't re-enable interrupts if stack might +* poll us due to busy-polling +*/ +if (likely (napi_complete_done (napi, work_done))) +iecm_vport_intr_update_itr_ena_irq (q_vector); + +return min_t (int, work_done, budget -1 ); }/**
Signature · 1 line
2.21.0
PATCH 11/14 iecm: Add singleq TX/RX
In reply to Alice MichaelImplement legacy single queue model for TX/RX flows.
--- .../ethernet/intel/iecm/iecm_singleq_txrx.c | 670 +++++++++++++++++- 1 file changed, 652 insertions(+), 18 deletions(-)
drivers/net/ethernet/intel/iecm/iecm_singleq_txrx.c +652 -18
@@ -17,7 +17,11 @@ static __le64 iecm_tx_singleq_build_ctob(u64 td_cmd, u64 td_offset, unsigned int size, u64 td_tag) { - /* stub */ + return cpu_to_le64(IECM_TX_DESC_DTYPE_DATA | + (td_cmd << IECM_TXD_QW1_CMD_S) | + (td_offset << IECM_TXD_QW1_OFFSET_S) | + ((u64)size << IECM_TXD_QW1_TX_BUF_SZ_S) | + (td_tag << IECM_TXD_QW1_L2TAG1_S)); }/**
@@ -31,7 +35,93 @@ staticint iecm_tx_singleq_csum (struct iecm_tx_buf *first,struct iecm_tx_offload_params *off) { -/* stub */ +u32 l4_len =0 , l3_len =0 , l2_len =0 ; +struct sk_buff *skb = first->skb; +union { +struct iphdr *v4; +struct ipv6hdr *v6; +unsigned char *hdr; + } ip; +union { +struct tcphdr *tcp; +unsigned char *hdr; + } l4; +__be16 frag_off, protocol; +unsigned char *exthdr; +u32 offset, cmd =0 ; +u8 l4_proto =0 ; + +if (skb->ip_summed !=CHECKSUM_PARTIAL ) +return 0 ; + +if (skb->encapsulation) +return -1 ; + + ip.hdr =skb_network_header (skb); + l4.hdr =skb_transport_header (skb); + +/* compute outer L2 header size */ + l2_len = ip.hdr - skb->data; + offset = (l2_len /2 ) <<IECM_TX_DESC_LEN_MACLEN_S ; + +/* Enable IP checksum offloads */ + protocol =vlan_get_protocol (skb); +if (protocol ==htons (ETH_P_IP )) { + l4_proto = ip.v4->protocol; +/* the stack computes the IP header already, the only time we +* need the hardware to recompute it is in the case of TSO. +*/ +if (first->tx_flags &IECM_TX_FLAGS_TSO ) + cmd |=IECM_TX_DESC_CMD_IIPT_IPV4_CSUM ; +else + cmd |=IECM_TX_DESC_CMD_IIPT_IPV4 ; + + }else if (protocol ==htons (ETH_P_IPV6 )) { + cmd |=IECM_TX_DESC_CMD_IIPT_IPV6 ; + exthdr = ip.hdr +sizeof (*ip.v6); + l4_proto = ip.v6->nexthdr; +if (l4.hdr != exthdr) +ipv6_skip_exthdr (skb, exthdr - skb->data, &l4_proto, + &frag_off); + }else { +return -1 ; + } + +/* compute inner L3 header size */ + l3_len = l4.hdr - ip.hdr; + offset |= (l3_len /4 ) <<IECM_TX_DESC_LEN_IPLEN_S ; + +/* Enable L4 checksum offloads */ +switch (l4_proto) { +case IPPROTO_TCP : +/* enable checksum offloads */ + cmd |=IECM_TX_DESC_CMD_L4T_EOFT_TCP ; + l4_len = l4.tcp->doff; + offset |= l4_len <<IECM_TX_DESC_LEN_L4_LEN_S ; +break ; +case IPPROTO_UDP : +/* enable UDP checksum offload */ + cmd |=IECM_TX_DESC_CMD_L4T_EOFT_UDP ; + l4_len = (sizeof (struct udphdr ) >>2 ); + offset |= l4_len <<IECM_TX_DESC_LEN_L4_LEN_S ; +break ; +case IPPROTO_SCTP : +/* enable SCTP checksum offload */ + cmd |=IECM_TX_DESC_CMD_L4T_EOFT_SCTP ; + l4_len =sizeof (struct sctphdr ) >>2 ; + offset |= l4_len <<IECM_TX_DESC_LEN_L4_LEN_S ; +break ; + +default : +if (first->tx_flags &IECM_TX_FLAGS_TSO ) +return -1 ; +skb_checksum_help (skb); +return 0 ; + } + + off->td_cmd |= cmd; + off->hdr_offsets |= offset; +return 1 ; }/**
@@ -48,7 +138,125 @@ static void iecm_tx_singleq_map(struct iecm_queue *tx_q, struct iecm_tx_buf *first, struct iecm_tx_offload_params *offloads) { - /* stub */ + u32 offsets = offloads->hdr_offsets; +struct iecm_base_tx_desc *tx_desc; +u64 td_cmd = offloads->td_cmd; +unsigned int data_len, size; +struct iecm_tx_buf *tx_buf; +u16 i = tx_q->next_to_use; +struct netdev_queue *nq; +struct sk_buff *skb; +skb_frag_t *frag; +dma_addr_t dma; + + skb = first->skb; + + data_len = skb->data_len; + size =skb_headlen (skb); + + tx_desc =IECM_BASE_TX_DESC (tx_q, i); + + dma =dma_map_single (tx_q->dev, skb->data, size,DMA_TO_DEVICE ); + + tx_buf = first; + +/* write each descriptor with CRC bit */ +if (tx_q->vport->adapter->dev_ops.crc_enable) + tx_q->vport->adapter->dev_ops.crc_enable (&td_cmd); + +for (frag = &skb_shinfo (skb)->frags[0 ];; frag++) { +unsigned int max_data =IECM_TX_MAX_DESC_DATA_ALIGNED ; + +if (dma_mapping_error (tx_q->dev, dma)) + goto dma_error; + +/* record length, and DMA address */ +dma_unmap_len_set (tx_buf, len, size); +dma_unmap_addr_set (tx_buf, dma, dma); + +/* align size to end of page */ + max_data += -dma & (IECM_TX_MAX_READ_REQ_SIZE -1 ); + tx_desc->buf_addr =cpu_to_le64 (dma); + +/* account for data chunks larger than the hardware +* can handle +*/ +while (unlikely (size >IECM_TX_MAX_DESC_DATA )) { + tx_desc->qw1 =iecm_tx_singleq_build_ctob (td_cmd, + offsets, + size,0 ); + tx_desc++; + i++; + +if (i == tx_q->desc_count) { + tx_desc =IECM_BASE_TX_DESC (tx_q,0 ); + i =0 ; + } + + dma += max_data; + size -= max_data; + + max_data =IECM_TX_MAX_DESC_DATA_ALIGNED ; + tx_desc->buf_addr =cpu_to_le64 (dma); + } + +if (likely (!data_len)) +break ; + tx_desc->qw1 =iecm_tx_singleq_build_ctob (td_cmd, offsets, + size,0 ); + tx_desc++; + i++; + +if (i == tx_q->desc_count) { + tx_desc =IECM_BASE_TX_DESC (tx_q,0 ); + i =0 ; + } + + size =skb_frag_size (frag); + data_len -= size; + + dma =skb_frag_dma_map (tx_q->dev, frag,0 , size, +DMA_TO_DEVICE ); + + tx_buf = &tx_q->tx_buf[i]; + } + +/* record bytecount for BQL */ + nq =netdev_get_tx_queue (tx_q->vport->netdev, tx_q->idx); +netdev_tx_sent_queue (nq, first->bytecount); + +/* record SW timestamp if HW timestamp is not available */ +skb_tx_timestamp (first->skb); + +/* write last descriptor with RS and EOP bits */ + td_cmd |= (u64)(IECM_TX_DESC_CMD_EOP |IECM_TX_DESC_CMD_RS ); + + tx_desc->qw1 =iecm_tx_singleq_build_ctob (td_cmd, offsets, size,0 ); + + i++; +if (i == tx_q->desc_count) + i =0 ; + +/* set next_to_watch value indicating a packet is present */ + first->next_to_watch = tx_desc; + +iecm_tx_buf_hw_update (tx_q, i, skb); + +return ; + +dma_error: +/* clear dma mappings for failed tx_buf map */ +for (;;) { + tx_buf = &tx_q->tx_buf[i]; +iecm_tx_buf_rel (tx_q, tx_buf); +if (tx_buf == first) +break ; +if (i ==0 ) + i = tx_q->desc_count; + i--; + } + + tx_q->next_to_use = i; }/**
@@ -61,7 +269,42 @@ iecm_tx_singleq_map(struct iecm_queue *tx_q, struct iecm_tx_buf *first,static netdev_tx_t iecm_tx_singleq_frame (struct sk_buff *skb,struct iecm_queue *tx_q) { -/* stub */ +struct iecm_tx_offload_params offload = {0 }; +struct iecm_tx_buf *first; +unsigned int count; +int csum; + + count =iecm_tx_desc_count_required (skb); + +/* need: 1 descriptor per page * PAGE_SIZE/IECM_MAX_DATA_PER_TXD, +* + 1 desc for skb_head_len/IECM_MAX_DATA_PER_TXD, +* + 4 desc gap to avoid the cache line where head is, +* + 1 desc for context descriptor, +* otherwise try next time +*/ +if (iecm_tx_maybe_stop (tx_q, count +IECM_TX_DESCS_PER_CACHE_LINE + +IECM_TX_DESCS_FOR_CTX )) { +return NETDEV_TX_BUSY ; + } + +/* record the location of the first descriptor for this packet */ + first = &tx_q->tx_buf[tx_q->next_to_use]; + first->skb = skb; + first->bytecount = max_t(unsigned int, skb->len, ETH_ZLEN); + first->gso_segs =1 ; + first->tx_flags =0 ; + + csum =iecm_tx_singleq_csum (first, &offload); +if (csum <0 ) + goto out_drop; + +iecm_tx_singleq_map (tx_q, first, &offload); + +return NETDEV_TX_OK ; + +out_drop: +dev_kfree_skb_any (skb); +return NETDEV_TX_OK ; }/**
@@ -74,7 +317,18 @@ iecm_tx_singleq_frame(struct sk_buff *skb, struct iecm_queue *tx_q)netdev_tx_t iecm_tx_singleq_start (struct sk_buff *skb,struct net_device *netdev) { -/* stub */ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); +struct iecm_queue *tx_q; + + tx_q = vport->txqs[skb->queue_mapping]; + +/* hardware can't handle really short frames, hardware padding works +* beyond this point +*/ +if (skb_put_padto (skb,IECM_TX_MIN_LEN )) +return NETDEV_TX_OK ; + +return iecm_tx_singleq_frame (skb, tx_q); }/**
@@ -85,7 +339,98 @@ netdev_tx_t iecm_tx_singleq_start(struct sk_buff *skb,*/ static bool iecm_tx_singleq_clean (struct iecm_queue *tx_q,int napi_budget) { -/* stub */ +unsigned int budget = tx_q->vport->compln_clean_budget; +unsigned int total_bytes =0 , total_pkts =0 ; +struct iecm_base_tx_desc *tx_desc; +s16 ntc = tx_q->next_to_clean; +struct iecm_tx_buf *tx_buf; +struct netdev_queue *nq; + + tx_desc =IECM_BASE_TX_DESC (tx_q, ntc); + tx_buf = &tx_q->tx_buf[ntc]; + ntc -= tx_q->desc_count; + +do { +struct iecm_base_tx_desc *eop_desc = tx_buf->next_to_watch; + +/* if next_to_watch is not set then no work pending */ +if (!eop_desc) +break ; + +/* prevent any other reads prior to eop_desc */ +smp_rmb (); + +/* if the descriptor isn't done, no work yet to do */ +if (!(eop_desc->qw1 & +cpu_to_le64 (IECM_TX_DESC_DTYPE_DESC_DONE ))) +break ; + +/* clear next_to_watch to prevent false hangs */ + tx_buf->next_to_watch =NULL ; + +/* update the statistics for this packet */ + total_bytes += tx_buf->bytecount; + total_pkts += tx_buf->gso_segs; + +/* free the skb */ +napi_consume_skb (tx_buf->skb, napi_budget); + +/* unmap skb header data */ +dma_unmap_single (tx_q->dev, +dma_unmap_addr (tx_buf, dma), +dma_unmap_len (tx_buf, len), +DMA_TO_DEVICE ); + +/* clear tx_buf data */ + tx_buf->skb =NULL ; +dma_unmap_len_set (tx_buf, len,0 ); + +/* unmap remaining buffers */ +while (tx_desc != eop_desc) { + tx_buf++; + tx_desc++; + ntc++; +if (unlikely (!ntc)) { + ntc -= tx_q->desc_count; + tx_buf = tx_q->tx_buf; + tx_desc =IECM_BASE_TX_DESC (tx_q,0 ); + } + +/* unmap any remaining paged data */ +if (dma_unmap_len (tx_buf, len)) { +dma_unmap_page (tx_q->dev, +dma_unmap_addr (tx_buf, dma), +dma_unmap_len (tx_buf, len), +DMA_TO_DEVICE ); +dma_unmap_len_set (tx_buf, len,0 ); + } + } + + tx_buf++; + tx_desc++; + ntc++; +if (unlikely (!ntc)) { + ntc -= tx_q->desc_count; + tx_buf = tx_q->tx_buf; + tx_desc =IECM_BASE_TX_DESC (tx_q,0 ); + } +/* update budget */ + budget--; + }while (likely (budget)); + + ntc += tx_q->desc_count; + tx_q->next_to_clean = ntc; + nq =netdev_get_tx_queue (tx_q->vport->netdev, tx_q->idx); +netdev_tx_completed_queue (nq, total_pkts, total_bytes); + tx_q->itr.stats.tx.packets += total_pkts; + tx_q->itr.stats.tx.bytes += total_bytes; + +u64_stats_update_begin (&tx_q->stats_sync); + tx_q->q_stats.tx.packets += total_pkts; + tx_q->q_stats.tx.bytes += total_bytes; +u64_stats_update_end (&tx_q->stats_sync); + +return !!budget; }/**
@@ -98,7 +443,16 @@ static bool iecm_tx_singleq_clean(struct iecm_queue *tx_q, int napi_budget)static inline bool iecm_tx_singleq_clean_all (struct iecm_q_vector *q_vec,int budget) { -/* stub */ +bool clean_complete = true; +int i, budget_per_q; + + budget_per_q =max (budget / q_vec->num_txq,1 ); +for (i =0 ; i < q_vec->num_txq; i++) { +if (!iecm_tx_singleq_clean (q_vec->tx[i], budget_per_q)) + clean_complete = false; + } + +return clean_complete; }/**
@@ -116,7 +470,8 @@ static bool iecm_rx_singleq_test_staterr(struct iecm_singleq_base_rx_desc *rx_desc, const u64 stat_err_bits) { - /* stub */ +return !!(rx_desc->qword1.status_error_ptype_len & +cpu_to_le64 (stat_err_bits)); }/**
@@ -129,7 +484,15 @@ static bool iecm_rx_singleq_is_non_eop(struct iecm_queue *rxq,struct iecm_singleq_base_rx_desc *rx_desc, struct sk_buff *skb) { - /* stub */ + /* if we are the last buffer then there is nothing else to do */ + #define IECM_RXD_EOF BIT(IECM_RX_BASE_DESC_STATUS_EOF_S) + if (likely(iecm_rx_singleq_test_staterr(rx_desc, IECM_RXD_EOF))) + return false; + +/* place skb in next buffer to be received */ + rxq->rx_buf.buf[rxq->next_to_clean].skb = skb; + +return true; }/**
@@ -145,7 +508,63 @@ static void iecm_rx_singleq_csum(struct iecm_queue *rxq, struct sk_buff *skb, struct iecm_singleq_base_rx_desc *rx_desc, u8 ptype) { - /* stub */ +u64 qw1 =le64_to_cpu (rx_desc->qword1.status_error_ptype_len); +struct iecm_rx_ptype_decoded decoded; +bool ipv4, ipv6; +u32 rx_status; +u8 rx_error; + +/* Start with CHECKSUM_NONE and by default csum_level = 0 */ + skb->ip_summed =CHECKSUM_NONE ; +skb_checksum_none_assert (skb); + +/* check if Rx checksum is enabled */ +if (!(rxq->vport->netdev->features &NETIF_F_RXCSUM )) +return ; + + rx_status = ((qw1 &IECM_RXD_QW1_STATUS_M ) >>IECM_RXD_QW1_STATUS_S ); + rx_error = ((qw1 &IECM_RXD_QW1_ERROR_M ) >>IECM_RXD_QW1_ERROR_S ); + +/* check if HW has decoded the packet and checksum */ +if (!(rx_status &BIT (IECM_RX_BASE_DESC_STATUS_L3L4P_S ))) +return ; + + decoded = rxq->vport->rx_ptype_lkup[ptype]; +if (!(decoded.known && decoded.outer_ip)) +return ; + + ipv4 = (decoded.outer_ip ==IECM_RX_PTYPE_OUTER_IP ) && + (decoded.outer_ip_ver ==IECM_RX_PTYPE_OUTER_IPV4 ); + ipv6 = (decoded.outer_ip ==IECM_RX_PTYPE_OUTER_IP ) && + (decoded.outer_ip_ver ==IECM_RX_PTYPE_OUTER_IPV6 ); + +if (ipv4 && (rx_error & (BIT (IECM_RX_BASE_DESC_ERROR_IPE_S ) | +BIT (IECM_RX_BASE_DESC_ERROR_EIPE_S )))) + goto checksum_fail; +else if (ipv6 && (rx_status & + (BIT (IECM_RX_BASE_DESC_STATUS_IPV6EXADD_S )))) + goto checksum_fail; + +/* check for L4 errors and handle packets that were not able to be +* checksummed due to arrival speed +*/ +if (rx_error &BIT (IECM_RX_BASE_DESC_ERROR_L3L4E_S )) + goto checksum_fail; + +/* Only report checksum unnecessary for ICMP, TCP, UDP, or SCTP */ +switch (decoded.inner_prot) { +case IECM_RX_PTYPE_INNER_PROT_ICMP : +case IECM_RX_PTYPE_INNER_PROT_TCP : +case IECM_RX_PTYPE_INNER_PROT_UDP : +case IECM_RX_PTYPE_INNER_PROT_SCTP : + skb->ip_summed =CHECKSUM_UNNECESSARY ; +default : +break ; + } +return ; + +checksum_fail: +dev_dbg (rxq->dev,"RX Checksum not available\n" ); }/**
@@ -163,7 +582,10 @@ iecm_rx_singleq_process_skb_fields(struct iecm_queue *rxq, struct sk_buff *skb, struct iecm_singleq_base_rx_desc *rx_desc, u8 ptype) { - /* stub */ +/* modifies the skb - consumes the enet header */ + skb->protocol =eth_type_trans (skb, rxq->vport->netdev); + +iecm_rx_singleq_csum (rxq, skb, rx_desc, ptype); }/**
@@ -176,7 +598,44 @@ iecm_rx_singleq_process_skb_fields(struct iecm_queue *rxq, struct sk_buff *skb,bool iecm_rx_singleq_buf_hw_alloc_all (struct iecm_queue *rx_q,u16 cleaned_count) { -/* stub */ +struct iecm_singleq_rx_buf_desc *singleq_rx_desc =NULL ; +u16 nta = rx_q->next_to_alloc; +struct iecm_rx_buf *buf; + +/* do nothing if no valid netdev defined */ +if (!rx_q->vport->netdev || !cleaned_count) +return false; + + singleq_rx_desc =IECM_SINGLEQ_RX_BUF_DESC (rx_q, nta); + buf = &rx_q->rx_buf.buf[nta]; + +do { +if (!iecm_rx_buf_hw_alloc (rx_q, buf)) +break ; + +/* Refresh the desc even if buffer_addrs didn't change +* because each write-back erases this info. +*/ + singleq_rx_desc->pkt_addr = +cpu_to_le64 (buf->dma + buf->page_offset); + singleq_rx_desc->hdr_addr =0 ; + singleq_rx_desc++; + + buf++; + nta++; +if (unlikely (nta == rx_q->desc_count)) { + singleq_rx_desc =IECM_SINGLEQ_RX_BUF_DESC (rx_q,0 ); + buf = rx_q->rx_buf.buf; + nta =0 ; + } + + cleaned_count--; + }while (cleaned_count); + +if (rx_q->next_to_alloc != nta) +iecm_rx_buf_hw_update (rx_q, nta); + +return !!cleaned_count; }/**
@@ -190,7 +649,16 @@ bool iecm_rx_singleq_buf_hw_alloc_all(struct iecm_queue *rx_q,static void iecm_singleq_rx_put_buf (struct iecm_queue *rx_bufq,struct iecm_rx_buf *rx_buf) { -/* stub */ +u16 ntu = rx_bufq->next_to_use; +bool recycled = false; + + recycled =iecm_rx_recycle_buf (rx_bufq, false, rx_buf); + +/* update, and store next to alloc if the buffer was recycled */ +if (recycled) { + ntu++; + rx_bufq->next_to_use = (ntu < rx_bufq->desc_count) ? ntu :0 ; + } }/**
@@ -199,7 +667,12 @@ static void iecm_singleq_rx_put_buf(struct iecm_queue *rx_bufq,*/ static void iecm_singleq_rx_bump_ntc (struct iecm_queue *q) { -/* stub */ +u16 ntc = q->next_to_clean +1 ; +/* fetch, update, and store next to clean */ +if (ntc < q->desc_count) + q->next_to_clean = ntc; +else + q->next_to_clean =0 ; }/**
@@ -214,7 +687,17 @@ static struct sk_buff * iecm_singleq_rx_get_buf_page(struct device *dev, struct iecm_rx_buf *rx_buf, const unsigned int size) { - /* stub */ + prefetch(rx_buf->page); + +/* we are reusing so sync this buffer for CPU use */ +dma_sync_single_range_for_cpu (dev, rx_buf->dma, + rx_buf->page_offset, size, +DMA_FROM_DEVICE ); + +/* We have pulled a buffer for use, so decrement pagecnt_bias */ + rx_buf->pagecnt_bias--; + +return rx_buf->skb; }/**
@@ -226,7 +709,116 @@ iecm_singleq_rx_get_buf_page(struct device *dev, struct iecm_rx_buf *rx_buf,*/ static int iecm_rx_singleq_clean (struct iecm_queue *rx_q,int budget) { -/* stub */ +struct iecm_singleq_base_rx_desc *singleq_base_rx_desc; +unsigned int total_rx_bytes =0 , total_rx_pkts =0 ; +u16 cleaned_count =0 ; +bool failure = false; + +/* Process Rx packets bounded by budget */ +while (likely (total_rx_pkts < (unsigned int )budget)) { +union iecm_rx_desc *rx_desc; +struct sk_buff *skb =NULL ; +struct iecm_rx_buf *rx_buf; +unsigned int size; +u8 rx_ptype; +u64 qword; + +/* get the Rx desc from Rx queue based on 'next_to_clean' */ + rx_desc =IECM_RX_DESC (rx_q, rx_q->next_to_clean); + singleq_base_rx_desc = (struct iecm_singleq_base_rx_desc *) + rx_desc; +/* status_error_ptype_len will always be zero for unused +* descriptors because it's cleared in cleanup, and overlaps +* with hdr_addr which is always zero because packet split +* isn't used, if the hardware wrote DD then the length will be +* non-zero +*/ + qword = +le64_to_cpu (rx_desc->base_wb.qword1.status_error_ptype_len); + +/* This memory barrier is needed to keep us from reading +* any other fields out of the rx_desc +*/ +dma_rmb (); +#define IECM_RXD_DD BIT(IECM_RX_BASE_DESC_STATUS_DD_S) +if (!iecm_rx_singleq_test_staterr (singleq_base_rx_desc, +IECM_RXD_DD )) +break ; + + size = (qword &IECM_RXD_QW1_LEN_PBUF_M ) >> +IECM_RXD_QW1_LEN_PBUF_S ; +if (!size) +break ; + + rx_buf = &rx_q->rx_buf.buf[rx_q->next_to_clean]; + skb =iecm_singleq_rx_get_buf_page (rx_q->dev, rx_buf, size); + +if (skb) +iecm_rx_add_frag (rx_buf, skb, size); +else + skb =iecm_rx_construct_skb (rx_q, rx_buf, size); + +/* exit if we failed to retrieve a buffer */ +if (!skb) { + rx_buf->pagecnt_bias++; +break ; + } + +iecm_singleq_rx_put_buf (rx_q, rx_buf); +iecm_singleq_rx_bump_ntc (rx_q); + + cleaned_count++; + +/* skip if it is non EOP desc */ +if (iecm_rx_singleq_is_non_eop (rx_q, singleq_base_rx_desc, + skb)) +continue ; + +#define IECM_RXD_ERR_S BIT(IECM_RXD_QW1_ERROR_S) +if (unlikely (iecm_rx_singleq_test_staterr (singleq_base_rx_desc, +IECM_RXD_ERR_S ))) { +dev_kfree_skb_any (skb); + skb =NULL ; +continue ; + } + +/* correct empty headers and pad skb if needed (to make valid +* ethernet frame +*/ +if (iecm_rx_cleanup_headers (skb)) { + skb =NULL ; +continue ; + } + +/* probably a little skewed due to removing CRC */ + total_rx_bytes += skb->len; + + rx_ptype = (qword &IECM_RXD_QW1_PTYPE_M ) >> +IECM_RXD_QW1_PTYPE_S ; + +/* protocol */ +iecm_rx_singleq_process_skb_fields (rx_q, skb, + singleq_base_rx_desc, + rx_ptype); + +/* send completed skb up the stack */ +iecm_rx_skb (rx_q, skb); + +/* update budget accounting */ + total_rx_pkts++; + } +if (cleaned_count) + failure =iecm_rx_singleq_buf_hw_alloc_all (rx_q, cleaned_count); + + rx_q->itr.stats.rx.packets += total_rx_pkts; + rx_q->itr.stats.rx.bytes += total_rx_bytes; +u64_stats_update_begin (&rx_q->stats_sync); + rx_q->q_stats.rx.packets += total_rx_pkts; + rx_q->q_stats.rx.bytes += total_rx_bytes; +u64_stats_update_end (&rx_q->stats_sync); + +/* guarantee a trip back through this routine if there was a failure */ +return failure ? budget : (int )total_rx_pkts; }/**
@@ -241,7 +833,22 @@ static inline bool iecm_rx_singleq_clean_all(struct iecm_q_vector *q_vec, int budget, int *cleaned) { - /* stub */ + bool clean_complete = true; +int pkts_cleaned_per_q; +int budget_per_q, i; + + budget_per_q =max (budget / q_vec->num_rxq,1 ); +for (i =0 ; i < q_vec->num_rxq; i++) { + pkts_cleaned_per_q =iecm_rx_singleq_clean (q_vec->rx[0 ], + budget_per_q); + +/* if we clean as many as budgeted, we must not be done */ +if (pkts_cleaned_per_q >= budget_per_q) + clean_complete = false; + *cleaned += pkts_cleaned_per_q; + } + +return clean_complete; }/**
@@ -251,5 +858,32 @@ iecm_rx_singleq_clean_all(struct iecm_q_vector *q_vec, int budget,*/ int iecm_vport_singleq_napi_poll (struct napi_struct *napi,int budget) { -/* stub */ +struct iecm_q_vector *q_vector = + container_of(napi, struct iecm_q_vector, napi); +bool clean_complete; +int work_done =0 ; + + clean_complete =iecm_tx_singleq_clean_all (q_vector, budget); + +/* Handle case where we are called by netpoll with a budget of 0 */ +if (budget <=0 ) +return budget; + +/* We attempt to distribute budget to each Rx queue fairly, but don't +* allow the budget to go below 1 because that would exit polling early. +*/ + clean_complete |=iecm_rx_singleq_clean_all (q_vector, budget, + &work_done); + +/* If work not completed, return budget and polling will return */ +if (!clean_complete) +return budget; + +/* Exit the polling mode, but don't re-enable interrupts if stack might +* poll us due to busy-polling +*/ +if (likely (napi_complete_done (napi, work_done))) +iecm_vport_intr_update_itr_ena_irq (q_vector); + +return min_t (int, work_done, budget -1 ); }
Signature · 1 line
2.21.0
PATCH 12/14 iecm: Add ethtool
In reply to Alice MichaelImplement ethtool interface for the common module.
--- .../net/ethernet/intel/iecm/iecm_ethtool.c | 1107 ++++++++++++++++- drivers/net/ethernet/intel/iecm/iecm_lib.c | 100 +- 2 files changed, 1203 insertions(+), 4 deletions(-)
drivers/net/ethernet/intel/iecm/iecm_ethtool.c +1106 -1
@@ -3,6 +3,1111 @@#include "iecm.h" +/** +* iecm_get_rxnfc - command to get RX flow classification rules +* @netdev: network interface device structure +* @cmd: ethtool rxnfc command +* @rule_locs: pointer to store rule locations +* +* Returns Success if the command is supported. +*/ +static int iecm_get_rxnfc (struct net_device *netdev,struct ethtool_rxnfc *cmd, +u32 *rule_locs) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); +int ret = -EOPNOTSUPP ; + +switch (cmd->cmd) { +case ETHTOOL_GRXRINGS : + cmd->data = vport->num_rxq; + ret =0 ; +break ; +case ETHTOOL_GRXFH : +netdev_info (netdev,"RSS hash info is not available\n" ); +break ; +default : +break ; + } + +return ret; +} + +/** +* iecm_get_rxfh_key_size - get the RSS hash key size +* @netdev: network interface device structure +* +* Returns the table size. +*/ +static u32 iecm_get_rxfh_key_size (struct net_device *netdev) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); + +if (!iecm_is_cap_ena (vport->adapter,VIRTCHNL_CAP_RSS )) { +dev_info (&vport->adapter->pdev->dev,"RSS is not supported on this device\n" ); +return 0 ; + } + +return vport->adapter->rss_data.rss_key_size; +} + +/** +* iecm_get_rxfh_indir_size - get the rx flow hash indirection table size +* @netdev: network interface device structure +* +* Returns the table size. +*/ +static u32 iecm_get_rxfh_indir_size (struct net_device *netdev) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); + +if (!iecm_is_cap_ena (vport->adapter,VIRTCHNL_CAP_RSS )) { +dev_info (&vport->adapter->pdev->dev,"RSS is not supported on this device\n" ); +return 0 ; + } + +return vport->adapter->rss_data.rss_lut_size; +} + +/** +* iecm_find_virtual_qid - Finds the virtual RX qid from the absolute RX qid +* @vport: virtual port structure +* @qid_list: List of the RX qid's +* +* Returns the virtual RX QID. +*/ +static u32 iecm_find_virtual_qid (struct iecm_vport *vport,u16 *qid_list, +u32 abs_rx_qid) +{ +u32 i; + +for (i =0 ; i < vport->num_rxq; i++) +if ((u32 )qid_list[i] == abs_rx_qid) +break ; +return i; +} + +/** +* iecm_get_rxfh - get the rx flow hash indirection table +* @netdev: network interface device structure +* @indir: indirection table +* @key: hash key +* @hfunc: hash function in use +* +* Reads the indirection table directly from the hardware. Always returns 0. +*/ +static int iecm_get_rxfh (struct net_device *netdev,u32 *indir,u8 *key, +u8 *hfunc) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); +struct iecm_adapter *adapter; +u16 i, *qid_list; +u32 abs_qid; + + adapter = vport->adapter; + +if (!iecm_is_cap_ena (adapter,VIRTCHNL_CAP_RSS )) { +dev_info (&vport->adapter->pdev->dev,"RSS is not supported on this device\n" ); +return 0 ; + } + +if (adapter->state != __IECM_UP) +return 0 ; + +if (hfunc) + *hfunc =ETH_RSS_HASH_TOP ; + +if (key) +memcpy (key, adapter->rss_data.rss_key, + adapter->rss_data.rss_key_size); + + qid_list =kcalloc (vport->num_rxq,sizeof (u16),GFP_KERNEL ); +if (!qid_list) +return -ENOMEM ; + +iecm_get_rx_qid_list (vport, qid_list); + +if (indir) +/* Each 32 bits pointed by 'indir' is stored with a lut entry */ +for (i =0 ; i < adapter->rss_data.rss_lut_size; i++) { + abs_qid = (u32 )adapter->rss_data.rss_lut[i]; + indir[i] =iecm_find_virtual_qid (vport, qid_list, + abs_qid); + } + +kfree (qid_list); + +return 0 ; +} + +/** +* iecm_set_rxfh - set the rx flow hash indirection table +* @netdev: network interface device structure +* @indir: indirection table +* @key: hash key +* @hfunc: hash function to use +* +* Returns -EINVAL if the table specifies an invalid queue id, otherwise +* returns 0 after programming the table. +*/ +static int iecm_set_rxfh (struct net_device *netdev,const u32 *indir, +const u8 *key,const u8 hfunc) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); +struct iecm_adapter *adapter; +u16 *qid_list; +u16 lut; + + adapter = vport->adapter; + +if (!iecm_is_cap_ena (adapter,VIRTCHNL_CAP_RSS )) { +dev_info (&adapter->pdev->dev,"RSS is not supported on this device.\n" ); +return 0 ; + } +if (adapter->state != __IECM_UP) +return 0 ; + +if (hfunc !=ETH_RSS_HASH_NO_CHANGE && hfunc !=ETH_RSS_HASH_TOP ) +return -EOPNOTSUPP ; + +if (key) +memcpy (adapter->rss_data.rss_key, key, + adapter->rss_data.rss_key_size); + + qid_list =kcalloc (vport->num_rxq,sizeof (u16),GFP_KERNEL ); +if (!qid_list) +return -ENOMEM ; + +iecm_get_rx_qid_list (vport, qid_list); + +if (indir) { +for (lut =0 ; lut < adapter->rss_data.rss_lut_size; lut++) { +int index = indir[lut]; + +if (index >= vport->num_rxq) { +kfree (qid_list); +return -EINVAL ; + } + adapter->rss_data.rss_lut[lut] = qid_list[index]; + } + }else { +iecm_fill_dflt_rss_lut (vport, qid_list); + } + +kfree (qid_list); + +return iecm_config_rss (vport); +} + +/** +* iecm_get_channels: get the number of channels supported by the device +* @netdev: network interface device structure +* @ch: channel information structure +* +* Report mamimum of TX and RX. Report one extra channel to match our MailBox +* Queue. +*/ +static void iecm_get_channels (struct net_device *netdev, +struct ethtool_channels *ch) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); + +/* Report maximum channels */ + ch->max_combined =IECM_MAX_Q ; + + ch->max_other =IECM_MAX_NONQ ; + ch->other_count =IECM_MAX_NONQ ; + + ch->combined_count =max (vport->num_txq, vport->num_rxq); +} + +/** +* iecm_set_channels: set the new channel count +* @netdev: network interface device structure +* @ch: channel information structure +* +* Negotiate a new number of channels with CP. Returns 0 on success, negative +* on failure. +*/ +static int iecm_set_channels (struct net_device *netdev, +struct ethtool_channels *ch) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); +int num_req_q = ch->combined_count; + +if (num_req_q ==max (vport->num_txq, vport->num_rxq)) +return 0 ; + +/* All of these should have already been checked by ethtool before this +* even gets to us, but just to be sure. +*/ +if (num_req_q <=0 || num_req_q >IECM_MAX_Q ) +return -EINVAL ; + +if (ch->rx_count || ch->tx_count || ch->other_count !=IECM_MAX_NONQ ) +return -EINVAL ; + + vport->adapter->config_data.num_req_qs = num_req_q; + +return iecm_initiate_soft_reset (vport, __IECM_SR_Q_CHANGE); +} + +/** +* iecm_get_ringparam - Get ring parameters +* @netdev: network interface device structure +* @ring: ethtool ringparam structure +* +* Returns current ring parameters. TX and RX rings are reported separately, +* but the number of rings is not reported. +*/ +static void iecm_get_ringparam (struct net_device *netdev, +struct ethtool_ringparam *ring) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); + + ring->rx_max_pending =IECM_MAX_RXQ_DESC ; + ring->tx_max_pending =IECM_MAX_TXQ_DESC ; + ring->rx_pending = vport->rxq_desc_count; + ring->tx_pending = vport->txq_desc_count; +} + +/** +* iecm_set_ringparam - Set ring parameters +* @netdev: network interface device structure +* @ring: ethtool ringparam structure +* +* Sets ring parameters. TX and RX rings are controlled separately, but the +* number of rings is not specified, so all rings get the same settings. +*/ +static int iecm_set_ringparam (struct net_device *netdev, +struct ethtool_ringparam *ring) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); +u32 new_rx_count, new_tx_count; + +if (ring->rx_mini_pending || ring->rx_jumbo_pending) +return -EINVAL ; + + new_tx_count =clamp_t (u32, ring->tx_pending, +IECM_MIN_TXQ_DESC , +IECM_MAX_TXQ_DESC ); + new_tx_count =ALIGN (new_tx_count,IECM_REQ_DESC_MULTIPLE ); + + new_rx_count =clamp_t (u32, ring->rx_pending, +IECM_MIN_RXQ_DESC , +IECM_MAX_RXQ_DESC ); + new_rx_count =ALIGN (new_rx_count,IECM_REQ_DESC_MULTIPLE ); + +/* if nothing to do return success */ +if (new_tx_count == vport->txq_desc_count && + new_rx_count == vport->rxq_desc_count) +return 0 ; + + vport->adapter->config_data.num_req_txq_desc = new_tx_count; + vport->adapter->config_data.num_req_rxq_desc = new_rx_count; + +return iecm_initiate_soft_reset (vport, __IECM_SR_Q_DESC_CHANGE); +} + +/** +* struct iecm_stats - definition for an ethtool statistic +* @stat_string: statistic name to display in ethtool -S output +* @sizeof_stat: the sizeof() the stat, must be no greater than sizeof(u64) +* @stat_offset: offsetof() the stat from a base pointer +* +* This structure defines a statistic to be added to the ethtool stats buffer. +* It defines a statistic as offset from a common base pointer. Stats should +* be defined in constant arrays using the IECM_STAT macro, with every element +* of the array using the same _type for calculating the sizeof_stat and +* stat_offset. +* +* The @sizeof_stat is expected to be sizeof(u8), sizeof(u16), sizeof(u32) or +* sizeof(u64). Other sizes are not expected and will produce a WARN_ONCE from +* the iecm_add_ethtool_stat() helper function. +* +* The @stat_string is interpreted as a format string, allowing formatted +* values to be inserted while looping over multiple structures for a given +* statistics array. Thus, every statistic string in an array should have the +* same type and number of format specifiers, to be formatted by variadic +* arguments to the iecm_add_stat_string() helper function. +*/ +struct iecm_stats { +char stat_string[ETH_GSTRING_LEN ]; +int sizeof_stat; +int stat_offset; +}; + +/* Helper macro to define an iecm_stat structure with proper size and type. +* Use this when defining constant statistics arrays. Note that @_type expects +* only a type name and is used multiple times. +*/ +#define IECM_STAT (_type, _name, _stat) { \ + .stat_string = _name, \ + .sizeof_stat = sizeof_field(_type, _stat), \ + .stat_offset = offsetof(_type, _stat) \ +} + +/* Helper macro for defining some statistics related to queues */ +#define IECM_QUEUE_STAT (_name, _stat) \ + IECM_STAT(struct iecm_queue, _name, _stat) + +/* Stats associated with a Tx queue */ +static const struct iecm_stats iecm_gstrings_tx_queue_stats[] = { +IECM_QUEUE_STAT ("%s-%u.packets" , q_stats.tx.packets), +IECM_QUEUE_STAT ("%s-%u.bytes" , q_stats.tx.bytes), +}; + +/* Stats associated with an Rx queue */ +static const struct iecm_stats iecm_gstrings_rx_queue_stats[] = { +IECM_QUEUE_STAT ("%s-%u.packets" , q_stats.rx.packets), +IECM_QUEUE_STAT ("%s-%u.bytes" , q_stats.rx.bytes), +IECM_QUEUE_STAT ("%s-%u.generic_csum" , q_stats.rx.generic_csum), +IECM_QUEUE_STAT ("%s-%u.basic_csum" , q_stats.rx.basic_csum), +IECM_QUEUE_STAT ("%s-%u.csum_err" , q_stats.rx.csum_err), +IECM_QUEUE_STAT ("%s-%u.hsplit_buf_overflow" , q_stats.rx.hsplit_hbo), +}; + +#define IECM_TX_QUEUE_STATS_LEN ARRAY_SIZE(iecm_gstrings_tx_queue_stats) +#define IECM_RX_QUEUE_STATS_LEN ARRAY_SIZE(iecm_gstrings_rx_queue_stats) + +/* For now we have one and only one private flag and it is only defined +* when we have support for the SKIP_CPU_SYNC DMA attribute. Instead +* of leaving all this code sitting around empty we will strip it unless +* our one private flag is actually available. +*/ +struct iecm_priv_flags { +char flag_string[ETH_GSTRING_LEN ]; +bool read_only; +u32 flag; +}; + +#define IECM_PRIV_FLAG (_name, _flag, _read_only) { \ + .read_only = _read_only, \ + .flag_string = _name, \ + .flag = _flag, \ +} + +static const struct iecm_priv_flags iecm_gstrings_priv_flags[] = { +IECM_PRIV_FLAG ("" ,0 ,0 ), +}; + +#define IECM_PRIV_FLAGS_STR_LEN ARRAY_SIZE(iecm_gstrings_priv_flags) + +/** +* __iecm_add_stat_strings - copy stat strings into ethtool buffer +* @p: ethtool supplied buffer +* @stats: stat definitions array +* @size: size of the stats array +* +* Format and copy the strings described by stats into the buffer pointed at +* by p. +*/ +static void __iecm_add_stat_strings (u8 **p,const struct iecm_stats stats[], +const unsigned int size, ...) +{ +unsigned int i; + +for (i =0 ; i < size; i++) { +va_list args; + +va_start (args, size); +vsnprintf ((char *)*p,ETH_GSTRING_LEN , + stats[i].stat_string, args); + *p +=ETH_GSTRING_LEN ; +va_end (args); + } +} + +/** +* iecm_add_stat_strings - copy stat strings into ethtool buffer +* @p: ethtool supplied buffer +* @stats: stat definitions array +* +* Format and copy the strings described by the const static stats value into +* the buffer pointed@by p. +* +* The parameter @stats is evaluated twice, so parameters with side effects +* should be avoided. Additionally, stats must be an array such that +* ARRAY_SIZE can be called on it. +*/ +#define iecm_add_stat_strings (p, stats, ...) \ + __iecm_add_stat_strings(p, stats, ARRAY_SIZE(stats), ## __VA_ARGS__) + +/** +* iecm_get_stat_strings - Get stat strings +* @netdev: network interface device structure +* @data: buffer for string data +* +* Builds the statistics string table +*/ +static void iecm_get_stat_strings (struct net_device *netdev,u8 *data) +{ +unsigned int i; + +/* It's critical that we always report a constant number of strings and +* that the strings are reported in the same order regardless of how +* many queues are actually in use. +*/ +for (i =0 ; i <IECM_MAX_Q ; i++) +iecm_add_stat_strings (&data, iecm_gstrings_tx_queue_stats, +"tx" , i); +for (i =0 ; i <IECM_MAX_Q ; i++) +iecm_add_stat_strings (&data, iecm_gstrings_rx_queue_stats, +"rx" , i); +} + +/** +* iecm_get_priv_flag_strings - Get private flag strings +* @netdev: network interface device structure +* @data: buffer for string data +* +* Builds the private flags string table +*/ +static void iecm_get_priv_flag_strings (struct net_device *netdev,u8 *data) +{ +unsigned int i; + +for (i =0 ; i <IECM_PRIV_FLAGS_STR_LEN ; i++) { +snprintf ((char *)data,ETH_GSTRING_LEN ,"%s" , + iecm_gstrings_priv_flags[i].flag_string); + data +=ETH_GSTRING_LEN ; + } +} + +/** +* iecm_get_strings - Get string set +* @netdev: network interface device structure +* @sset: id of string set +* @data: buffer for string data +* +* Builds string tables for various string sets +*/ +static void iecm_get_strings (struct net_device *netdev,u32 sset,u8 *data) +{ +switch (sset) { +case ETH_SS_STATS : +iecm_get_stat_strings (netdev, data); +break ; +case ETH_SS_PRIV_FLAGS : +iecm_get_priv_flag_strings (netdev, data); +break ; +default : +break ; + } +} + +/** +* iecm_get_sset_count - Get length of string set +* @netdev: network interface device structure +* @sset: id of string set +* +* Reports size of various string tables. +*/ +static int iecm_get_sset_count (struct net_device *netdev,int sset) +{ +if (sset ==ETH_SS_STATS ) +/* This size reported back here *must* be constant throughout +* the lifecycle of the netdevice, i.e. we must report the +* maximum length even for queues that don't technically exist. +* This is due to the fact that this userspace API uses three +* separate ioctl calls to get stats data but has no way to +* communicate back to userspace when that size has changed, +* which can typically happen as a result of changing number of +* queues. If the number/order of stats change in the middle of +* this call chain it will lead to userspace crashing/accessing +* bad data through buffer under/overflow. +*/ +return (IECM_TX_QUEUE_STATS_LEN *IECM_MAX_Q ) + + (IECM_RX_QUEUE_STATS_LEN *IECM_MAX_Q ); +else if (sset ==ETH_SS_PRIV_FLAGS ) +return IECM_PRIV_FLAGS_STR_LEN ; +else +return -EINVAL ; +} + +/** +* iecm_add_one_ethtool_stat - copy the stat into the supplied buffer +* @data: location to store the stat value +* @pstat: old stat pointer to copy from +* @stat: the stat definition +* +* Copies the stat data defined by the pointer and stat structure pair into +* the memory supplied as data. Used to implement iecm_add_ethtool_stats and +* iecm_add_queue_stats. If the pointer is null, data will be zero'd. +*/ +static void +iecm_add_one_ethtool_stat (u64 *data,void *pstat, +const struct iecm_stats *stat) +{ +char *p; + +if (!pstat) { +/* ensure that the ethtool data buffer is zero'd for any stats +* which don't have a valid pointer. +*/ + *data =0 ; +return ; + } + + p = (char *)pstat + stat->stat_offset; +switch (stat->sizeof_stat) { +case sizeof (u64): + *data = *((u64 *)p); +break ; +case sizeof (u32): + *data = *((u32 *)p); +break ; +case sizeof (u16): + *data = *((u16 *)p); +break ; +case sizeof (u8): + *data = *((u8 *)p); +break ; +default : +WARN_ONCE (1 ,"unexpected stat size for %s" , + stat->stat_string); + *data =0 ; + } +} + +/** +* iecm_add_queue_stats - copy queue statistics into supplied buffer +* @data: ethtool stats buffer +* @q: the queue to copy +* +* Queue statistics must be copied while protected by +* u64_stats_fetch_begin_irq, so we can't directly use iecm_add_ethtool_stats. +* Assumes that queue stats are defined in iecm_gstrings_queue_stats. If the +* queue pointer is null, zero out the queue stat values and update the data +* pointer. Otherwise safely copy the stats from the queue into the supplied +* buffer and update the data pointer when finished. +* +* This function expects to be called while under rcu_read_lock(). +*/ +static void +iecm_add_queue_stats (u64 **data,struct iecm_queue *q) +{ +const struct iecm_stats *stats; +unsigned int start; +unsigned int size; +unsigned int i; + +if (q->q_type ==VIRTCHNL_QUEUE_TYPE_RX ) { + size =IECM_RX_QUEUE_STATS_LEN ; + stats = iecm_gstrings_rx_queue_stats; + }else { + size =IECM_TX_QUEUE_STATS_LEN ; + stats = iecm_gstrings_tx_queue_stats; + } + +/* To avoid invalid statistics values, ensure that we keep retrying +* the copy until we get a consistent value according to +* u64_stats_fetch_retry_irq. But first, make sure our queue is +* non-null before attempting to access its syncp. +*/ +do { + start =u64_stats_fetch_begin_irq (&q->stats_sync); +for (i =0 ; i < size; i++) +iecm_add_one_ethtool_stat (&(*data)[i], q, &stats[i]); + }while (u64_stats_fetch_retry_irq (&q->stats_sync, start)); + +/* Once we successfully copy the stats in, update the data pointer */ + *data += size; +} + +/** +* iecm_add_empty_queue_stats - Add stats for a non-existent queue +* @data: pointer to data buffer +* @qtype: type of data queue +* +* We must report a constant length of stats back to userspace regardless of +* how many queues are actually in use because stats collection happens over +* three separate ioctls and there's no way to notify userspace the size +* changed between those calls. This adds empty to data to the stats since we +* don't have a real queue to refer to for this stats slot. +*/ +static void +iecm_add_empty_queue_stats (u64 **data,enum virtchnl_queue_type qtype) +{ +unsigned int i; +int stats_len; + +if (qtype ==VIRTCHNL_QUEUE_TYPE_RX ) + stats_len =IECM_RX_QUEUE_STATS_LEN ; +else + stats_len =IECM_TX_QUEUE_STATS_LEN ; + +for (i =0 ; i < stats_len; i++) + (*data)[i] =0 ; + *data += stats_len; +} + +/** +* iecm_get_ethtool_stats - report device statistics +* @netdev: network interface device structure +* @stats: ethtool statistics structure +* @data: pointer to data buffer +* +* All statistics are added to the data buffer as an array of u64. +*/ +static void iecm_get_ethtool_stats (struct net_device *netdev, +struct ethtool_stats *stats,u64 *data) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); +enum virtchnl_queue_type qtype; +unsigned int total =0 ; +unsigned int i, j; + +if (vport->adapter->state != __IECM_UP) +return ; + +rcu_read_lock (); +for (i =0 ; i < vport->num_txq_grp; i++) { +struct iecm_txq_group *txq_grp = &vport->txq_grps[i]; + + qtype =VIRTCHNL_QUEUE_TYPE_TX ; + +for (j =0 ; j < txq_grp->num_txq; j++, total++) { +struct iecm_queue *txq = &txq_grp->txqs[j]; + +if (!txq) +iecm_add_empty_queue_stats (&data, qtype); +else +iecm_add_queue_stats (&data, txq); + } + } +/* It is critical we provide a constant number of stats back to +* userspace regardless of how many queues are actually in use because +* there is no way to inform userspace the size has changed between +* ioctl calls. This will fill in any missing stats with zero. +*/ +for (; total <IECM_MAX_Q ; total++) +iecm_add_empty_queue_stats (&data,VIRTCHNL_QUEUE_TYPE_TX ); + total =0 ; + +for (i =0 ; i < vport->num_rxq_grp; i++) { +struct iecm_rxq_group *rxq_grp = &vport->rxq_grps[i]; +int num_rxq; + + qtype =VIRTCHNL_QUEUE_TYPE_RX ; + +if (iecm_is_queue_model_split (vport->rxq_model)) + num_rxq = rxq_grp->splitq.num_rxq_sets; +else + num_rxq = rxq_grp->singleq.num_rxq; + +for (j =0 ; j < num_rxq; j++, total++) { +struct iecm_queue *rxq; + +if (iecm_is_queue_model_split (vport->rxq_model)) + rxq = &rxq_grp->splitq.rxq_sets[j].rxq; +else + rxq = &rxq_grp->singleq.rxqs[j]; +if (!rxq) +iecm_add_empty_queue_stats (&data, qtype); +else +iecm_add_queue_stats (&data, rxq); + } + } +for (; total <IECM_MAX_Q ; total++) +iecm_add_empty_queue_stats (&data,VIRTCHNL_QUEUE_TYPE_RX ); +rcu_read_unlock (); +} + +/** +* iecm_find_rxq - find rxq from q index +* @vport: virtual port associated to queue +* @q_num: q index used to find queue +* +* returns pointer to rx queue +*/ +static struct iecm_queue * +iecm_find_rxq (struct iecm_vport *vport,int q_num) +{ +struct iecm_queue *rxq; +int q_grp, q_idx; + +if (iecm_is_queue_model_split (vport->rxq_model)) { + q_grp = q_num /IECM_DFLT_SPLITQ_RXQ_PER_GROUP ; + q_idx = q_num %IECM_DFLT_SPLITQ_RXQ_PER_GROUP ; + + rxq = &vport->rxq_grps[q_grp].splitq.rxq_sets[q_idx].rxq; + }else { + q_grp = q_num /IECM_DFLT_SINGLEQ_RXQ_PER_GROUP ; + q_idx = q_num %IECM_DFLT_SINGLEQ_RXQ_PER_GROUP ; + + rxq = &vport->rxq_grps[q_grp].singleq.rxqs[q_idx]; + } + +return rxq; +} + +/** +* iecm_find_txq - find txq from q index +* @vport: virtual port associated to queue +* @q_num: q index used to find queue +* +* returns pointer to tx queue +*/ +static struct iecm_queue * +iecm_find_txq (struct iecm_vport *vport,int q_num) +{ +struct iecm_queue *txq; + +if (iecm_is_queue_model_split (vport->txq_model)) { +int q_grp = q_num /IECM_DFLT_SPLITQ_TXQ_PER_GROUP ; + + txq = vport->txq_grps[q_grp].complq; + }else { + txq = vport->txqs[q_num]; + } + +return txq; +} + +/** +* __iecm_get_q_coalesce - get ITR values for specific queue +* @ec: ethtool structure to fill with driver's coalesce settings +* @q: quuee of Rx or Tx +*/ +static void +__iecm_get_q_coalesce (struct ethtool_coalesce *ec,struct iecm_queue *q) +{ +u16 itr_setting; +bool dyn_ena; + + itr_setting =IECM_ITR_SETTING (q->itr.target_itr); + dyn_ena =IECM_ITR_IS_DYNAMIC (q->itr.target_itr); +if (q->q_type ==VIRTCHNL_QUEUE_TYPE_RX ) { + ec->use_adaptive_rx_coalesce = dyn_ena; + ec->rx_coalesce_usecs = itr_setting; + }else { + ec->use_adaptive_tx_coalesce = dyn_ena; + ec->tx_coalesce_usecs = itr_setting; + } +} + +/** +* iecm_get_q_coalesce - get ITR values for specific queue +* @netdev: pointer to the netdev associated with this query +* @ec: coalesce settings to program the device with +* @q_num: update ITR/INTRL (coalesce) settings for this queue number/index +* +* Return 0 on success, and negative on failure +*/ +static int +iecm_get_q_coalesce (struct net_device *netdev,struct ethtool_coalesce *ec, +u32 q_num) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); + +if (vport->adapter->state != __IECM_UP) +return 0 ; + +if (q_num >= vport->num_rxq && q_num >= vport->num_txq) +return -EINVAL ; + +if (q_num < vport->num_rxq) { +struct iecm_queue *rxq =iecm_find_rxq (vport, q_num); + +__iecm_get_q_coalesce (ec, rxq); + } + +if (q_num < vport->num_txq) { +struct iecm_queue *txq =iecm_find_txq (vport, q_num); + +__iecm_get_q_coalesce (ec, txq); + } + +return 0 ; +} + +/** +* iecm_get_coalesce - get ITR values as requested by user +* @netdev: pointer to the netdev associated with this query +* @ec: coalesce settings to be filled +* +* Return 0 on success, and negative on failure +*/ +static int +iecm_get_coalesce (struct net_device *netdev,struct ethtool_coalesce *ec) +{ +/* Return coalesce based on queue number zero */ +return iecm_get_q_coalesce (netdev, ec,0 ); +} + +/** +* iecm_get_per_q_coalesce - get ITR values as requested by user +* @netdev: pointer to the netdev associated with this query +* @q_num: queue for which the itr values has to retrieved +* @ec: coalesce settings to be filled +* +* Return 0 on success, and negative on failure +*/ + +static int +iecm_get_per_q_coalesce (struct net_device *netdev,u32 q_num, +struct ethtool_coalesce *ec) +{ +return iecm_get_q_coalesce (netdev, ec, q_num); +} + +/** +* __iecm_set_q_coalesce - set ITR values for specific queue +* @ec: ethtool structure from user to update ITR settings +* @q: queue for which itr values has to be set +* +* Returns 0 on success, negative otherwise. +*/ +static int +__iecm_set_q_coalesce (struct ethtool_coalesce *ec,struct iecm_queue *q) +{ +const char *q_type_str = (q->q_type ==VIRTCHNL_QUEUE_TYPE_RX ) + ?"Rx" :"Tx" ; +u32 use_adaptive_coalesce, coalesce_usecs; +struct iecm_vport *vport; +u16 itr_setting; + + itr_setting =IECM_ITR_SETTING (q->itr.target_itr); + vport = q->vport; +if (q->q_type ==VIRTCHNL_QUEUE_TYPE_RX ) { + use_adaptive_coalesce = ec->use_adaptive_rx_coalesce; + coalesce_usecs = ec->rx_coalesce_usecs; + }else { + use_adaptive_coalesce = ec->use_adaptive_tx_coalesce; + coalesce_usecs = ec->tx_coalesce_usecs; + } + +if (itr_setting != coalesce_usecs && use_adaptive_coalesce) { +netdev_info (vport->netdev,"%s ITR cannot be changed if adaptive-%s is enabled\n" , + q_type_str, q_type_str); +return -EINVAL ; + } + +if (coalesce_usecs >IECM_ITR_MAX ) { +netdev_info (vport->netdev, +"Invalid value, %d-usecs range is 0-%d\n" , + coalesce_usecs,IECM_ITR_MAX ); +return -EINVAL ; + } + +/* hardware only supports an ITR granularity of 2us */ +if (coalesce_usecs %2 !=0 ) { +netdev_info (vport->netdev, +"Invalid value, %d-usecs must be even\n" , + coalesce_usecs); +return -EINVAL ; + } + + q->itr.target_itr = coalesce_usecs; +if (use_adaptive_coalesce) + q->itr.target_itr |=IECM_ITR_DYNAMIC ; +/* Update of static/dynamic itr will be taken care when interrupt is +* fired +*/ +return 0 ; +} + +/** +* iecm_set_q_coalesce - set ITR values for specific queue +* @vport: vport associated to the queue that need updating +* @ec: coalesce settings to program the device with +* @q_num: update ITR/INTRL (coalesce) settings for this queue number/index +* @is_rxq: is queue type rx +* +* Return 0 on success, and negative on failure +*/ +static int +iecm_set_q_coalesce (struct iecm_vport *vport,struct ethtool_coalesce *ec, +int q_num,bool is_rxq) +{ +if (is_rxq) { +struct iecm_queue *rxq =iecm_find_rxq (vport, q_num); + +if (rxq &&__iecm_set_q_coalesce (ec, rxq)) +return -EINVAL ; + }else { +struct iecm_queue *txq =iecm_find_txq (vport, q_num); + +if (txq &&__iecm_set_q_coalesce (ec, txq)) +return -EINVAL ; + } + +return 0 ; +} + +/** +* iecm_set_coalesce - set ITR values as requested by user +* @netdev: pointer to the netdev associated with this query +* @ec: coalesce settings to program the device with +* +* Return 0 on success, and negative on failure +*/ +static int +iecm_set_coalesce (struct net_device *netdev,struct ethtool_coalesce *ec) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); +int i, err =0 ; + +if (vport->adapter->state != __IECM_UP) +return 0 ; + +for (i =0 ; i < vport->num_txq; i++) { + err =iecm_set_q_coalesce (vport, ec, i, false); +if (err) + goto set_coalesce_err; + } + +for (i =0 ; i < vport->num_rxq; i++) { + err =iecm_set_q_coalesce (vport, ec, i, true); +if (err) + goto set_coalesce_err; + } +set_coalesce_err: +return err; +} + +/** +* iecm_set_per_q_coalesce - set ITR values as requested by user +* @netdev: pointer to the netdev associated with this query +* @q_num: queue for which the itr values has to be set +* @ec: coalesce settings to program the device with +* +* Return 0 on success, and negative on failure +*/ +static int +iecm_set_per_q_coalesce (struct net_device *netdev,u32 q_num, +struct ethtool_coalesce *ec) +{ +struct iecm_vport *vport =iecm_netdev_to_vport (netdev); +int err; + +if (vport->adapter->state != __IECM_UP) +return 0 ; + + err =iecm_set_q_coalesce (vport, ec, q_num, false); +if (!err) + err =iecm_set_q_coalesce (vport, ec, q_num, true); + +return err; +} + +/** +* iecm_get_msglevel - Get debug message level +* @netdev: network interface device structure +* +* Returns current debug message level. +*/ +static u32 iecm_get_msglevel (struct net_device *netdev) +{ +struct iecm_adapter *adapter =iecm_netdev_to_adapter (netdev); + +return adapter->msg_enable; +} + +/** +* iecm_set_msglevel - Set debug message level +* @netdev: network interface device structure +* @data: message level +* +* Set current debug message level. Higher values cause the driver to +* be noisier. +*/ +static void iecm_set_msglevel (struct net_device *netdev,u32 data) +{ +struct iecm_adapter *adapter =iecm_netdev_to_adapter (netdev); + + adapter->msg_enable = data; +} + +/** +* iecm_get_link_ksettings - Get Link Speed and Duplex settings +* @netdev: network interface device structure +* @ecmd: ethtool command +* +* Reports speed/duplex settings. +**/ +static int iecm_get_link_ksettings (struct net_device *netdev, +struct ethtool_link_ksettings *cmd) +{ +struct iecm_netdev_priv *np =netdev_priv (netdev); +struct iecm_adapter *adapter = np->vport->adapter; + +ethtool_link_ksettings_zero_link_mode (cmd, supported); + cmd->base.autoneg =AUTONEG_DISABLE ; + cmd->base.port =PORT_NONE ; +/* Set speed and duplex */ +switch (adapter->link_speed) { +case VIRTCHNL_LINK_SPEED_40GB : + cmd->base.speed =SPEED_40000 ; +break ; +case VIRTCHNL_LINK_SPEED_25GB : +#ifdef SPEED_25000 + cmd->base.speed =SPEED_25000 ; +#else +netdev_info (netdev, +"Speed is 25G, display not supported by this version of ethtool.\n" ); +#endif +break ; +case VIRTCHNL_LINK_SPEED_20GB : + cmd->base.speed =SPEED_20000 ; +break ; +case VIRTCHNL_LINK_SPEED_10GB : + cmd->base.speed =SPEED_10000 ; +break ; +case VIRTCHNL_LINK_SPEED_1GB : + cmd->base.speed =SPEED_1000 ; +break ; +case VIRTCHNL_LINK_SPEED_100MB : + cmd->base.speed =SPEED_100 ; +break ; +default : +break ; + } + cmd->base.duplex =DUPLEX_FULL ; + +return 0 ; +} + +/** +* iecm_get_drvinfo - Get driver info +* @netdev: network interface device structure +* @drvinfo: ethool driver info structure +* +* Returns information about the driver and device for display to the user. +*/ +static void iecm_get_drvinfo (struct net_device *netdev, +struct ethtool_drvinfo *drvinfo) +{ +struct iecm_adapter *adapter =iecm_netdev_to_adapter (netdev); + +strlcpy (drvinfo->driver, iecm_drv_name,32 ); +strlcpy (drvinfo->fw_version,"N/A" ,4 ); +strlcpy (drvinfo->bus_info,pci_name (adapter->pdev),32 ); +} + +static const struct ethtool_ops iecm_ethtool_ops = { + .get_drvinfo = iecm_get_drvinfo, + .get_msglevel = iecm_get_msglevel, + .set_msglevel = iecm_set_msglevel, + .get_coalesce = iecm_get_coalesce, + .set_coalesce = iecm_set_coalesce, + .get_per_queue_coalesce = iecm_get_per_q_coalesce, + .set_per_queue_coalesce = iecm_set_per_q_coalesce, + .get_ethtool_stats = iecm_get_ethtool_stats, + .get_strings = iecm_get_strings, + .get_sset_count = iecm_get_sset_count, + .get_rxnfc = iecm_get_rxnfc, + .get_rxfh_key_size = iecm_get_rxfh_key_size, + .get_rxfh_indir_size = iecm_get_rxfh_indir_size, + .get_rxfh = iecm_get_rxfh, + .set_rxfh = iecm_set_rxfh, + .get_channels = iecm_get_channels, + .set_channels = iecm_set_channels, + .get_ringparam = iecm_get_ringparam, + .set_ringparam = iecm_set_ringparam, + .get_link_ksettings = iecm_get_link_ksettings, +}; +/** * iecm_set_ethtool_ops - Initialize ethtool ops struct * @netdev: network interface device structure
@@ -12,5 +1117,5 @@*/ void iecm_set_ethtool_ops (struct net_device *netdev) { -/* stub */ + netdev->ethtool_ops = &iecm_ethtool_ops; }
drivers/net/ethernet/intel/iecm/iecm_lib.c +97 -3
124 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH 13/14 iecm: Add iecm to the kernel build system
In reply to Alice MichaelThis introduces iecm as a module to the kernel, and adds relevant documentation.
--- .../networking/device_drivers/intel/iecm.rst | 93 +++++++++++++++++++ MAINTAINERS | 1 + drivers/net/ethernet/intel/Kconfig | 7 ++ drivers/net/ethernet/intel/Makefile | 1 + drivers/net/ethernet/intel/iecm/Makefile | 21 +++++ 5 files changed, 123 insertions(+) create mode 100644 Documentation/networking/device_drivers/intel/iecm.rst create mode 100644 drivers/net/ethernet/intel/iecm/Makefile
Documentation/networking/device_drivers/intel/iecm.rst new file +93 -0
@@ -0,0 +1,93 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================== +Intel Ethernet Common Module +======================== + +The Intel Ethernet Common Module is meant to serve as an abstraction layer +between device specific implementation details and common net device driver +flows. This library provides several function hooks which allow a device driver +to specify register addresses, control queue communications, and other device +specific functionality. Some functions are required to be implemented while +others have a default implementation that is used when none is supplied by the +device driver. Doing this, a device driver can be written to take advantage +of existing code while also giving the flexibility to implement device specific +features. + +The common use case for this library is for a network device driver that wants +specify its own device specific details but also leverage the more common +code flows found in network device drivers. + +Sections in this document: + Entry Point + Exit Point + Register Operations API + Virtchnl Operations API + +Entry Point +~~~~~~~~~~~ +The primary entry point to the library is the iecm_probe function. Prior to +calling this, device drivers must have allocated an iecm_adapter struct and +initialized it with the required API functions. The adapter struct, along with +the pci_dev struct and the pci_device_id struct, is provided to iecm_probe +which finalizes device initialization and prepares the device for open. + +The iecm_dev_ops struct within the iecm_adapter struct is the primary vehicle +for passing information from device drivers to the common module. A dependent +module must define and assign a reg_ops_init function which will assign the +respective function pointers to initialize register values (see iecm_reg_ops +struct). These are required to be provided by the dependent device driver as +no suitable default can be assumed for register addresses. + +The vc_ops_init function pointer and the related iecm_virtchnl_ops struct are +optional and should only be necessary for device drivers which use a different +method/timing for communicating across a mailbox to the hardware. Within iecm +is a default interface provided in the case where one is not provided by the +device driver. + +Exit Point +~~~~~~~~~~ +When the device driver is being prepared to be removed through the pci_driver +remove callback, it is required for the device driver to call iecm_remove with +the pci_dev struct provided. This is required to ensure all resources are +properly freed and returned to the operating system. + +Register Operations API +~~~~~~~~~~~~~~~~~~~~~~~ +iecm_reg_ops contains three different function pointers relating to intializing +registers for the specific net device using the library. + +ctlq_reg_init relates specifically to setting up registers related to control +queue/mailbox communications. Registers that should be defined include: head, +tail, len, bah, bal, len_mask, len_ena_mask, and head_mask. + +vportq_reg_init relates to setting up queue registers. The tail registers to +be assigned to the iecm_queue struct for each RX/TX queue. + +intr_reg_init relates to any registers needed to setup interrupts. These +include registers needed to enable the interrupt and change ITR settings. + +If the initialization function finds that one or more required function +pointers were not provided, an error will be issued and the device will be +inoperable. + + +Virtchnl Operations API +~~~~~~~~~~~~~~~~~~~~~~~ +The virtchnl is a conduit between driver and hardware that allows device +drivers to send and receive control messages to/from hardware. This is +optional to be specified as there is a general interface that can be assumed +when using this library. However, if a device deviates in some way to +communicate across the mailbox correctly, this interface is provided to allow +that. + +If vc_ops_init is set in the dev_ops field of the iecm_adapter struct, then it +is assumed the device driver is using providing it's own interface to do +virtchnl communications. While providing vc_ops_init is optional, if it is +provided, it is required that the device driver provide function pointers for +those functions in vc_ops, with expection for the enable_vport, disable_vport, +and destroy_vport functions which may not be required for all devices. + +If the initialization function finds that vc_ops_init was defined but one or +more required function pointers were not provided, an error will be issued and +the device will be inoperable.
MAINTAINERS +1 -0
@@ -8546,6 +8546,7 @@ F: Documentation/networking/device_drivers/intel/fm10k.rst F: Documentation/networking/device_drivers/intel/i40e.rst F: Documentation/networking/device_drivers/intel/iavf.rst F: Documentation/networking/device_drivers/intel/ice.rst +F: Documentation/networking/device_drivers/intel/iecm.rst F: Documentation/networking/device_drivers/intel/igb.rst F: Documentation/networking/device_drivers/intel/igbvf.rst F: Documentation/networking/device_drivers/intel/ixgb.rst
drivers/net/ethernet/intel/Kconfig +7 -0
@@ -344,4 +344,11 @@ config IGC To compile this driver as a module, choose M here. The module will be called igc. +config IECM + tristate "Intel(R) Ethernet Common Module Support" + default n + depends on PCI + ---help--- + To compile this driver as a module, choose M here. The module + will be called iecm. MSI-X interrupt support is required endif # NET_VENDOR_INTEL
drivers/net/ethernet/intel/Makefile +1 -0
@@ -16,3 +16,4 @@ obj-$(CONFIG_IXGB) += ixgb/ obj-$(CONFIG_IAVF) += iavf/ obj-$(CONFIG_FM10K) += fm10k/ obj-$(CONFIG_ICE) += ice/ +obj-$(CONFIG_IECM) += iecm/
drivers/net/ethernet/intel/iecm/Makefile new file +21 -0
@@ -0,0 +1,21 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright (C) 2020 Intel Corporation + +# +# Makefile for the Intel(R) Ethernet Common Module +# + +ccflags-y += -I$(srctree)/drivers/net/ethernet/intel/include + +obj-$(CONFIG_IECM) += iecm.o + +iecm-y := \ + iecm_lib.o \ + iecm_virtchnl.o \ + iecm_txrx.o \ + iecm_singleq_txrx.o \ + iecm_ethtool.o \ + iecm_controlq.o \ + iecm_osdep.o \ + iecm_controlq_setup.o \ + iecm_main.o
Signature · 1 line
2.21.0
PATCH 14/14 idpf: Introduce idpf driver
In reply to Alice MichaelFrom: Alan Brady <[email protected]>
Utilizes the Intel Ethernet Common Module and provides a device specific implementation for data plane devices.
--- .../networking/device_drivers/intel/idpf.rst | 47 ++++++ MAINTAINERS | 1 + drivers/net/ethernet/intel/Kconfig | 8 + drivers/net/ethernet/intel/Makefile | 1 + drivers/net/ethernet/intel/idpf/Makefile | 14 ++ drivers/net/ethernet/intel/idpf/idpf_dev.h | 17 ++ drivers/net/ethernet/intel/idpf/idpf_devids.h | 10 ++ drivers/net/ethernet/intel/idpf/idpf_main.c | 139 ++++++++++++++++ drivers/net/ethernet/intel/idpf/idpf_reg.c | 152 ++++++++++++++++++ 9 files changed, 389 insertions(+) create mode 100644 Documentation/networking/device_drivers/intel/idpf.rst create mode 100644 drivers/net/ethernet/intel/idpf/Makefile create mode 100644 drivers/net/ethernet/intel/idpf/idpf_dev.h create mode 100644 drivers/net/ethernet/intel/idpf/idpf_devids.h create mode 100644 drivers/net/ethernet/intel/idpf/idpf_main.c create mode 100644 drivers/net/ethernet/intel/idpf/idpf_reg.c
Documentation/networking/device_drivers/intel/idpf.rst new file +47 -0
@@ -0,0 +1,47 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================================================================== +Linux Base Driver for the Intel(R) Smart Network Adapter Family Series +================================================================== + +Intel idpf Linux driver. +Copyright(c) 2020 Intel Corporation. + +Contents +======== + +- Enabling the driver +- Support + +The driver in this release supports Intel's Smart Network Adapter Family Series +of products. For more information, visit Intel's support page at +https://support.intel.com. + +Enabling the driver +=================== +The driver is enabled via the standard kernel configuration system, +using the make command:: + + make oldconfig/menuconfig/etc. + +The driver is located in the menu structure at: + + -> Device Drivers + -> Network device support (NETDEVICES [=y]) + -> Ethernet driver support + -> Intel devices + -> Intel(R) Smart Network Adapter Family Series Support + +Support +======= +For general information, go to the Intel support website at: + +https://www.intel.com/support/ + +or the Intel Wired Networking project hosted by Sourceforge at: + +https://sourceforge.net/projects/e1000 + +If an issue is identified with the released source code on a supported kernel +with a supported adapter, email the specific information related to the issue +to e1000-devel at lists.sf.net.
MAINTAINERS +1 -0
@@ -8546,6 +8546,7 @@ F: Documentation/networking/device_drivers/intel/fm10k.rst F: Documentation/networking/device_drivers/intel/i40e.rst F: Documentation/networking/device_drivers/intel/iavf.rst F: Documentation/networking/device_drivers/intel/ice.rst +F: Documentation/networking/device_drivers/intel/idpf.rst F: Documentation/networking/device_drivers/intel/iecm.rst F: Documentation/networking/device_drivers/intel/igb.rst F: Documentation/networking/device_drivers/intel/igbvf.rst
drivers/net/ethernet/intel/Kconfig +8 -0
@@ -351,4 +351,12 @@ config IECM ---help--- To compile this driver as a module, choose M here. The module will be called iecm. MSI-X interrupt support is required + +config IDPF + tristate "Intel(R) Data Plane Function Support" + default n + depends on PCI + ---help--- + To compile this driver as a module, choose M here. The module + will be called idpf. endif # NET_VENDOR_INTEL
drivers/net/ethernet/intel/Makefile +1 -0
@@ -17,3 +17,4 @@ obj-$(CONFIG_IAVF) += iavf/ obj-$(CONFIG_FM10K) += fm10k/ obj-$(CONFIG_ICE) += ice/ obj-$(CONFIG_IECM) += iecm/ +obj-$(CONFIG_IDPF) += idpf/
drivers/net/ethernet/intel/idpf/Makefile new file +14 -0
@@ -0,0 +1,14 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright (C) 2020 Intel Corporation + +# +# Makefile for the Intel(R) Data Plane Function Linux Driver +# + +obj-$(CONFIG_IDPF) += idpf.o + +ccflags-y += -I${src}/../include -I${src} + +idpf-y := \ + idpf_main.o \ + idpf_reg.o
drivers/net/ethernet/intel/idpf/idpf_dev.h new file +17 -0
@@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright (C) 2020 Intel Corporation */ + +#ifndef _IDPF_DEV_H_ +#define _IDPF_DEV_H_ + +#include "iecm.h" + +void idpf_intr_reg_init (struct iecm_vport *vport); +void idpf_mb_intr_reg_init (struct iecm_adapter *adapter); +void idpf_reset_reg_init (struct iecm_reset_reg *reset_reg); +void idpf_trigger_reset (struct iecm_adapter *adapter, +enum iecm_flags trig_cause); +void idpf_vportq_reg_init (struct iecm_vport *vport); +void idpf_ctlq_reg_init (struct iecm_ctlq_create_info *cq); + +#endif /* _IDPF_DEV_H_ */
drivers/net/ethernet/intel/idpf/idpf_devids.h new file +10 -0
@@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright (C) 2020 Intel Corporation */ + +#ifndef _IDPF_DEVIDS_H_ +#define _IDPF_DEVIDS_H_ + +/* Device IDs */ +#define IDPF_DEV_ID_PF 0x1452 + +#endif /* _IDPF_DEVIDS_H_ */
drivers/net/ethernet/intel/idpf/idpf_main.c new file +139 -0
139 lines not on this page · raw message
drivers/net/ethernet/intel/idpf/idpf_reg.c new file +152 -0
152 lines not on this page · raw message
Signature · 1 line
2.21.0
kbuild test robot
build bot[Intel-wired-lan] [iecm/idpf V1 14/14] idpf: Introduce idpf driver
Thank you for the patch! Yet something to improve:
Hi Alice,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on jkirsher-next-queue/dev-queue]
[also build test ERROR on linus/master v5.7-rc5 next-20200515]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Alice-Michael/iecm-idpf-series-cover-letter/20200515-230855 base: https://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/next-queue.git dev-queue config: x86_64-allyesconfig (attached as .config) compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project 9d4cf5bd421fb6467ff5f00e26a37527246dd4d6)
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>, old ones prefixed by <<):
In file included from drivers/net/ethernet/intel/idpf/idpf_main.c:8:
>> drivers/net/ethernet/intel/idpf/idpf_dev.h:7:10: fatal error: 'iecm.h' file not found
#include "iecm.h"
^~~~~~~~
1 error generated.
vim +7 drivers/net/ethernet/intel/idpf/idpf_dev.h
6
> 7 #include "iecm.h"
8 ---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: .config.gz
Type: application/gzip
Size: 73506 bytes
Desc: not available
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20200516/234fe3b1/attachment-0001.bin>