Add E800 live migration driver
Subsystems (5)
v2 cover Add E800 live migration driver
This series adds vfio live migration support for Intel E810 VF devices based on the v2 migration protocol.
v2:
- clarified comments and commit message
---
Lingyu Liu (11):
ice: check VF migration status before sending messages to VF
ice: add migration init field and helper functions
ice: save VF messages as device state
ice: save and restore device state
ice: do not notify VF link state during migration
ice: change VSI id in virtual channel message after migration
ice: save and restore RX queue head
ice: save and restore TX queue head
ice: stop device before saving device states
ice: mask VF advanced capabilities if live migration is activated vfio/ice: implement vfio_pci driver for E800 devices
Xu Ting (1):
ice: Fix missing legacy 32byte RXDID in the supported bitmap
Yahui Cao (3):
ice: add function to get rxq context
vfio: Expose vfio_device_has_container()
vfio/ice: support iommufd vfio compat mode
MAINTAINERS | 7 + drivers/net/ethernet/intel/ice/Makefile | 1 + drivers/net/ethernet/intel/ice/ice.h | 2 + drivers/net/ethernet/intel/ice/ice_common.c | 268 +++++ drivers/net/ethernet/intel/ice/ice_common.h | 5 + .../net/ethernet/intel/ice/ice_migration.c | 778 ++++++++++++++ .../intel/ice/ice_migration_private.h | 31 + drivers/net/ethernet/intel/ice/ice_vf_lib.c | 8 + drivers/net/ethernet/intel/ice/ice_vf_lib.h | 6 + drivers/net/ethernet/intel/ice/ice_virtchnl.c | 238 +++-- drivers/net/ethernet/intel/ice/ice_virtchnl.h | 15 +- .../ethernet/intel/ice/ice_virtchnl_fdir.c | 28 +- drivers/vfio/group.c | 1 + drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/ice/Kconfig | 10 + drivers/vfio/pci/ice/Makefile | 4 + drivers/vfio/pci/ice/ice_vfio_pci.c | 998 ++++++++++++++++++ include/linux/net/intel/ice_migration.h | 53 + include/linux/vfio.h | 1 + 20 files changed, 2347 insertions(+), 111 deletions(-) create mode 100644 drivers/net/ethernet/intel/ice/ice_migration.c create mode 100644 drivers/net/ethernet/intel/ice/ice_migration_private.h create mode 100644 drivers/vfio/pci/ice/Kconfig create mode 100644 drivers/vfio/pci/ice/Makefile create mode 100644 drivers/vfio/pci/ice/ice_vfio_pci.c create mode 100644 include/linux/net/intel/ice_migration.h
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 1/15 ice: Fix missing legacy 32byte RXDID in the supported bitmap
From: Xu Ting <[email protected]>
32byte legacy descriptor format is preassigned.
Commit e753df8fbca5 ("ice: Add support Flex RXD") created a supported RXDIDs bitmap according to DDP package. But it missed the legacy 32byte RXDID since it is not listed in the package. Mark 32byte legacy descriptor format as supported in the supported RXDIDs flags.
--- drivers/net/ethernet/intel/ice/ice_virtchnl.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/ice/ice_virtchnl.c +6 -3
@@ -2617,10 +2617,13 @@ static int ice_vc_query_rxdid(struct ice_vf *vf)/* Read flexiflag registers to determine whether the * corresponding RXDID is configured and supported or not. -* Since Legacy 16byte descriptor format is not supported, -* start from Legacy 32byte descriptor. +* But the legacy 32byte RXDID is not listed in DDP package, +* add it in the bitmap manually and skip check for it in the loop. +* Legacy 16byte descriptor is not supported. */ -for (i =ICE_RXDID_LEGACY_1 ; i <ICE_FLEX_DESC_RXDID_MAX_NUM ; i++) { + rxdid->supported_rxdids |=BIT (ICE_RXDID_LEGACY_1 ); + +for (i =ICE_RXDID_FLEX_NIC ; i <ICE_FLEX_DESC_RXDID_MAX_NUM ; i++) { regval =rd32 (hw,GLFLXP_RXDID_FLAGS (i,0 ));if ((regval >>GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_S ) &GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_M )
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 2/15 ice: add function to get rxq context
In reply to Lingyu LiuFrom: Yahui Cao <[email protected]>
Export rxq context. Read binaries from hw registers. Then use offsets and masks to extract different context fields.
ice_vfio_pci driver that is introduced in coming patches of this series will use the exported rxq context when saving and restoring device state.
--- drivers/net/ethernet/intel/ice/ice_common.c | 268 ++++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_common.h | 5 + 2 files changed, 273 insertions(+)
drivers/net/ethernet/intel/ice/ice_common.c +268 -0
@@ -1428,6 +1428,34 @@ ice_copy_rxq_ctx_to_hw(struct ice_hw *hw, u8 *ice_rxq_ctx, u32 rxq_index)return 0 ; } +/** +* ice_copy_rxq_ctx_from_hw - Copy rxq context register from HW +* @hw: pointer to the hardware structure +* @ice_rxq_ctx: pointer to the rxq context +* @rxq_index: the index of the Rx queue +* +* Copies rxq context from HW register space to dense structure +*/ +static int +ice_copy_rxq_ctx_from_hw (struct ice_hw *hw,u8 *ice_rxq_ctx,u32 rxq_index) +{ +u8 i; + +if (!ice_rxq_ctx || rxq_index >QRX_CTRL_MAX_INDEX ) +return -EINVAL ; + +/* Copy each dword separately from HW */ +for (i =0 ; i <ICE_RXQ_CTX_SIZE_DWORDS ; i++) { +u32 *ctx = (u32 *)(ice_rxq_ctx + (i *sizeof (u32))); + + *ctx =rd32 (hw,QRX_CONTEXT (i, rxq_index)); + +ice_debug (hw,ICE_DBG_QCTX ,"qrxdata[%d]: %08X\n" , i, *ctx); + } + +return 0 ; +} +/* LAN Rx Queue Context */ static const struct ice_ctx_ele ice_rlan_ctx_info[] = {/* Field Width LSB */
@@ -1479,6 +1507,32 @@ ice_write_rxq_ctx(struct ice_hw *hw, struct ice_rlan_ctx *rlan_ctx,return ice_copy_rxq_ctx_to_hw (hw, ctx_buf, rxq_index); } +/** +* ice_read_rxq_ctx - Read rxq context from HW +* @hw: pointer to the hardware structure +* @rlan_ctx: pointer to the rxq context +* @rxq_index: the index of the Rx queue +* +* Read rxq context from HW register space and then converts it from dense +* structure to sparse +*/ +int +ice_read_rxq_ctx (struct ice_hw *hw,struct ice_rlan_ctx *rlan_ctx, +u32 rxq_index) +{ +u8 ctx_buf[ICE_RXQ_CTX_SZ ] = {0 }; +int status; + +if (!rlan_ctx) +return -EINVAL ; + + status =ice_copy_rxq_ctx_from_hw (hw, ctx_buf, rxq_index); +if (status) +return status; + +return ice_get_ctx (ctx_buf, (u8 *)rlan_ctx, ice_rlan_ctx_info); +} +/* LAN Tx Queue Context */ const struct ice_ctx_ele ice_tlan_ctx_info[] = {/* Field Width LSB */
@@ -4469,6 +4523,220 @@ ice_set_ctx(struct ice_hw *hw, u8 *src_ctx, u8 *dest_ctx,return 0 ; } +/** +* ice_read_byte - read context byte into struct +* @src_ctx: the context structure to read from +* @dest_ctx: the context to be written to +* @ce_info: a description of the struct to be filled +*/ +static void +ice_read_byte (u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info) +{ +u8 dest_byte, mask; +u8 *src, *target; +u16 shift_width; + +/* prepare the bits and mask */ + shift_width = ce_info->lsb %8 ; + mask = (u8)(BIT (ce_info->width) -1 ); + +/* shift to correct alignment */ + mask <<= shift_width; + +/* get the current bits from the src bit string */ + src = src_ctx + (ce_info->lsb /8 ); + +memcpy (&dest_byte, src,sizeof (dest_byte)); + + dest_byte &= mask; + + dest_byte >>= shift_width; + +/* get the address from the struct field */ + target = dest_ctx + ce_info->offset; + +/* put it back in the struct */ +memcpy (target, &dest_byte,sizeof (dest_byte)); +} + +/** +* ice_read_word - read context word into struct +* @src_ctx: the context structure to read from +* @dest_ctx: the context to be written to +* @ce_info: a description of the struct to be filled +*/ +static void +ice_read_word (u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info) +{ +u16 dest_word, mask; +u8 *src, *target; +__le16 src_word; +u16 shift_width; + +/* prepare the bits and mask */ + shift_width = ce_info->lsb %8 ; + mask =BIT (ce_info->width) -1 ; + +/* shift to correct alignment */ + mask <<= shift_width; + +/* get the current bits from the src bit string */ + src = src_ctx + (ce_info->lsb /8 ); + +memcpy (&src_word, src,sizeof (src_word)); + +/* the data in the memory is stored as little endian so mask it +* correctly +*/ + src_word &=cpu_to_le16 (mask); + +/* get the data back into host order before shifting */ + dest_word =le16_to_cpu (src_word); + + dest_word >>= shift_width; + +/* get the address from the struct field */ + target = dest_ctx + ce_info->offset; + +/* put it back in the struct */ +memcpy (target, &dest_word,sizeof (dest_word)); +} + +/** +* ice_read_dword - read context dword into struct +* @src_ctx: the context structure to read from +* @dest_ctx: the context to be written to +* @ce_info: a description of the struct to be filled +*/ +static void +ice_read_dword (u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info) +{ +u32 dest_dword, mask; +__le32 src_dword; +u8 *src, *target; +u16 shift_width; + +/* prepare the bits and mask */ + shift_width = ce_info->lsb %8 ; + +/* if the field width is exactly 32 on an x86 machine, then the shift +* operation will not work because the SHL instructions count is masked +* to 5 bits so the shift will do nothing +*/ +if (ce_info->width <32 ) + mask =BIT (ce_info->width) -1 ; +else + mask = (u32 )~0 ; + +/* shift to correct alignment */ + mask <<= shift_width; + +/* get the current bits from the src bit string */ + src = src_ctx + (ce_info->lsb /8 ); + +memcpy (&src_dword, src,sizeof (src_dword)); + +/* the data in the memory is stored as little endian so mask it +* correctly +*/ + src_dword &=cpu_to_le32 (mask); + +/* get the data back into host order before shifting */ + dest_dword =le32_to_cpu (src_dword); + + dest_dword >>= shift_width; + +/* get the address from the struct field */ + target = dest_ctx + ce_info->offset; + +/* put it back in the struct */ +memcpy (target, &dest_dword,sizeof (dest_dword)); +} + +/** +* ice_read_qword - read context qword into struct +* @src_ctx: the context structure to read from +* @dest_ctx: the context to be written to +* @ce_info: a description of the struct to be filled +*/ +static void +ice_read_qword (u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info) +{ +u64 dest_qword, mask; +__le64 src_qword; +u8 *src, *target; +u16 shift_width; + +/* prepare the bits and mask */ + shift_width = ce_info->lsb %8 ; + +/* if the field width is exactly 64 on an x86 machine, then the shift +* operation will not work because the SHL instructions count is masked +* to 6 bits so the shift will do nothing +*/ +if (ce_info->width <64 ) + mask =BIT_ULL (ce_info->width) -1 ; +else + mask = (u64 )~0 ; + +/* shift to correct alignment */ + mask <<= shift_width; + +/* get the current bits from the src bit string */ + src = src_ctx + (ce_info->lsb /8 ); + +memcpy (&src_qword, src,sizeof (src_qword)); + +/* the data in the memory is stored as little endian so mask it +* correctly +*/ + src_qword &=cpu_to_le64 (mask); + +/* get the data back into host order before shifting */ + dest_qword =le64_to_cpu (src_qword); + + dest_qword >>= shift_width; + +/* get the address from the struct field */ + target = dest_ctx + ce_info->offset; + +/* put it back in the struct */ +memcpy (target, &dest_qword,sizeof (dest_qword)); +} + +/** +* ice_get_ctx - extract context bits from a packed structure +* @src_ctx: pointer to a generic packed context structure +* @dest_ctx: pointer to a generic non-packed context structure +* @ce_info: a description of the structure to be read from +*/ +int +ice_get_ctx (u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info) +{ +int i; + +for (i =0 ; ce_info[i].width; i++) { +switch (ce_info[i].size_of) { +case 1 : +ice_read_byte (src_ctx, dest_ctx, &ce_info[i]); +break ; +case 2 : +ice_read_word (src_ctx, dest_ctx, &ce_info[i]); +break ; +case 4 : +ice_read_dword (src_ctx, dest_ctx, &ce_info[i]); +break ; +case 8 : +ice_read_qword (src_ctx, dest_ctx, &ce_info[i]); +break ; +default : +return -EINVAL ; + } + } + +return 0 ; +} +/** * ice_get_lan_q_ctx - get the LAN queue context for the given VSI and TC * @hw: pointer to the HW struct
drivers/net/ethernet/intel/ice/ice_common.h +5 -0
@@ -56,6 +56,9 @@ void ice_set_safe_mode_caps(struct ice_hw *hw);int ice_write_rxq_ctx (struct ice_hw *hw,struct ice_rlan_ctx *rlan_ctx,u32 rxq_index); +int +ice_read_rxq_ctx (struct ice_hw *hw,struct ice_rlan_ctx *rlan_ctx, +u32 rxq_index);int ice_aq_get_rss_lut (struct ice_hw *hw,struct ice_aq_get_set_rss_lut_params *get_params);
@@ -75,6 +78,8 @@ extern const struct ice_ctx_ele ice_tlan_ctx_info[];int ice_set_ctx (struct ice_hw *hw,u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info); +int +ice_get_ctx (u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info);extern struct mutex ice_global_cfg_lock_sw;
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 3/15 ice: check VF migration status before sending messages to VF
In reply to Lingyu LiuAdd a new VF state ICE_VF_STATE_REPLAYING_VC. During live migration, virtual channel commands will be replayed to restore VF device state. Don't send messages back to VF when replaying by checking VF state before PF sends messages to VF.
ice_vc_send_msg_to_vf() implies sending messages but actually it does not while replaying messages. Renaming ice_vc_send_msg_to_vf() to ice_vc_respond_to_vf() to indicate its behavior correctly.
--- drivers/net/ethernet/intel/ice/ice_vf_lib.h | 1 + drivers/net/ethernet/intel/ice/ice_virtchnl.c | 182 ++++++++++-------- drivers/net/ethernet/intel/ice/ice_virtchnl.h | 8 +- .../ethernet/intel/ice/ice_virtchnl_fdir.c | 28 +-- 4 files changed, 120 insertions(+), 99 deletions(-)
drivers/net/ethernet/intel/ice/ice_vf_lib.h +1 -0
@@ -37,6 +37,7 @@ enum ice_vf_states {ICE_VF_STATE_DIS ,ICE_VF_STATE_MC_PROMISC ,ICE_VF_STATE_UC_PROMISC , +ICE_VF_STATE_REPLAYING_VC , ICE_VF_STATES_NBITS };
drivers/net/ethernet/intel/ice/ice_virtchnl.c +101 -81
393 lines not on this page · raw message
drivers/net/ethernet/intel/ice/ice_virtchnl.h +4 -4
@@ -60,8 +60,8 @@ void ice_vc_notify_vf_link_state(struct ice_vf *vf);void ice_vc_notify_link_state (struct ice_pf *pf);void ice_vc_notify_reset (struct ice_pf *pf);int -ice_vc_send_msg_to_vf (struct ice_vf *vf,u32 v_opcode, -enum virtchnl_status_code v_retval,u8 *msg,u16 msglen); +ice_vc_respond_to_vf (struct ice_vf *vf,u32 v_opcode, +enum virtchnl_status_code v_retval,u8 *msg,u16 msglen);bool ice_vc_isvalid_vsi_id (struct ice_vf *vf,u16 vsi_id);void ice_vc_process_vf_msg (struct ice_pf *pf,struct ice_rq_event_info *event,struct ice_mbx_data *mbxdata);
@@ -73,8 +73,8 @@ static inline void ice_vc_notify_link_state(struct ice_pf *pf) { }static inline void ice_vc_notify_reset (struct ice_pf *pf) { }static inline int -ice_vc_send_msg_to_vf (struct ice_vf *vf,u32 v_opcode, -enum virtchnl_status_code v_retval,u8 *msg,u16 msglen) +ice_vc_respond_to_vf (struct ice_vf *vf,u32 v_opcode, +enum virtchnl_status_code v_retval,u8 *msg,u16 msglen) {return -EOPNOTSUPP ; }
drivers/net/ethernet/intel/ice/ice_virtchnl_fdir.c +14 -14
@@ -1571,8 +1571,8 @@ ice_vc_add_fdir_fltr_post(struct ice_vf *vf, struct ice_vf_fdir_ctx *ctx, resp->flow_id = conf->flow_id; vf->fdir.fdir_fltr_cnt[conf->input.flow_type][is_tun]++; - ret =ice_vc_send_msg_to_vf (vf, ctx->v_opcode, v_ret, - (u8 *)resp, len); + ret =ice_vc_respond_to_vf (vf, ctx->v_opcode, v_ret, + (u8 *)resp, len);kfree (resp); dev_dbg(dev, "VF %d: flow_id:0x%X, FDIR %s success!\n",
@@ -1587,8 +1587,8 @@ ice_vc_add_fdir_fltr_post(struct ice_vf *vf, struct ice_vf_fdir_ctx *ctx,ice_vc_fdir_remove_entry (vf, conf, conf->flow_id);devm_kfree (dev, conf); - ret =ice_vc_send_msg_to_vf (vf, ctx->v_opcode, v_ret, - (u8 *)resp, len); + ret =ice_vc_respond_to_vf (vf, ctx->v_opcode, v_ret, + (u8 *)resp, len);kfree (resp);return ret; }
@@ -1635,8 +1635,8 @@ ice_vc_del_fdir_fltr_post(struct ice_vf *vf, struct ice_vf_fdir_ctx *ctx,ice_vc_fdir_remove_entry (vf, conf, conf->flow_id); vf->fdir.fdir_fltr_cnt[conf->input.flow_type][is_tun]--; - ret =ice_vc_send_msg_to_vf (vf, ctx->v_opcode, v_ret, - (u8 *)resp, len); + ret =ice_vc_respond_to_vf (vf, ctx->v_opcode, v_ret, + (u8 *)resp, len);kfree (resp); dev_dbg(dev, "VF %d: flow_id:0x%X, FDIR %s success!\n",
@@ -1652,8 +1652,8 @@ ice_vc_del_fdir_fltr_post(struct ice_vf *vf, struct ice_vf_fdir_ctx *ctx,if (success)devm_kfree (dev, conf); - ret =ice_vc_send_msg_to_vf (vf, ctx->v_opcode, v_ret, - (u8 *)resp, len); + ret =ice_vc_respond_to_vf (vf, ctx->v_opcode, v_ret, + (u8 *)resp, len);kfree (resp);return ret; }
@@ -1850,8 +1850,8 @@ int ice_vc_add_fdir_fltr(struct ice_vf *vf, u8 *msg) v_ret =VIRTCHNL_STATUS_SUCCESS ; stat->status =VIRTCHNL_FDIR_SUCCESS ;devm_kfree (dev, conf); - ret =ice_vc_send_msg_to_vf (vf,VIRTCHNL_OP_ADD_FDIR_FILTER , - v_ret, (u8 *)stat, len); + ret =ice_vc_respond_to_vf (vf,VIRTCHNL_OP_ADD_FDIR_FILTER , + v_ret, (u8 *)stat, len); goto exit; }
@@ -1909,8 +1909,8 @@ int ice_vc_add_fdir_fltr(struct ice_vf *vf, u8 *msg) err_free_conf:devm_kfree (dev, conf); err_exit: - ret =ice_vc_send_msg_to_vf (vf,VIRTCHNL_OP_ADD_FDIR_FILTER , v_ret, - (u8 *)stat, len); + ret =ice_vc_respond_to_vf (vf,VIRTCHNL_OP_ADD_FDIR_FILTER , v_ret, + (u8 *)stat, len);kfree (stat);return ret; }
@@ -1993,8 +1993,8 @@ int ice_vc_del_fdir_fltr(struct ice_vf *vf, u8 *msg) err_del_tmr:ice_vc_fdir_clear_irq_ctx (vf); err_exit: - ret =ice_vc_send_msg_to_vf (vf,VIRTCHNL_OP_DEL_FDIR_FILTER , v_ret, - (u8 *)stat, len); + ret =ice_vc_respond_to_vf (vf,VIRTCHNL_OP_DEL_FDIR_FILTER , v_ret, + (u8 *)stat, len);kfree (stat);return ret; }
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 4/15 ice: add migration init field and helper functions
In reply to Lingyu LiuAdds a function to get ice VF device from pci device. Adds a field in VF structure to indicate migration init state, and functions to init and uninit migration.
This will be used by ice_vfio_pci driver introduced in coming patches from this series.
--- drivers/net/ethernet/intel/ice/Makefile | 1 + drivers/net/ethernet/intel/ice/ice.h | 1 + .../net/ethernet/intel/ice/ice_migration.c | 68 +++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_vf_lib.c | 7 ++ drivers/net/ethernet/intel/ice/ice_vf_lib.h | 1 + include/linux/net/intel/ice_migration.h | 26 +++++++ 6 files changed, 104 insertions(+) create mode 100644 drivers/net/ethernet/intel/ice/ice_migration.c create mode 100644 include/linux/net/intel/ice_migration.h
drivers/net/ethernet/intel/ice/Makefile +1 -0
@@ -49,3 +49,4 @@ ice-$(CONFIG_RFS_ACCEL) += ice_arfs.o ice-$(CONFIG_XDP_SOCKETS) += ice_xsk.o ice-$(CONFIG_ICE_SWITCHDEV) += ice_eswitch.o ice_eswitch_br.o ice-$(CONFIG_GNSS) += ice_gnss.o +ice-$(CONFIG_ICE_VFIO_PCI) += ice_migration.o
drivers/net/ethernet/intel/ice/ice.h +1 -0
@@ -55,6 +55,7 @@#include <net/vxlan.h> #include <net/gtp.h> #include <linux/ppp_defs.h> +#include <linux/net/intel/ice_migration.h> #include "ice_devids.h" #include "ice_type.h" #include "ice_txrx.h"
drivers/net/ethernet/intel/ice/ice_migration.c new file +68 -0
68 lines not on this page · raw message
drivers/net/ethernet/intel/ice/ice_vf_lib.c +7 -0
@@ -56,6 +56,9 @@ static void ice_release_vf(struct kref *ref) { struct ice_vf *vf = container_of(ref, struct ice_vf, refcnt); +if (vf->migration_active) +ice_migration_uninit_vf (vf); + vf->vf_ops->free (vf); }
@@ -260,6 +263,10 @@ static void ice_vf_pre_vsi_rebuild(struct ice_vf *vf)if (vf->vf_ops->irq_close) vf->vf_ops->irq_close (vf); +if (vf->migration_active) { +ice_migration_uninit_vf (vf); +ice_migration_init_vf (vf); + }ice_vf_clear_counters (vf); vf->vf_ops->clear_reset_trigger (vf); }
drivers/net/ethernet/intel/ice/ice_vf_lib.h +1 -0
@@ -134,6 +134,7 @@ struct ice_vf {/* devlink port data */ struct devlink_port devlink_port; +bool migration_active; };/* Flags for controlling behavior of ice_reset_vf */
include/linux/net/intel/ice_migration.h new file +26 -0
@@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright (C) 2018-2023 Intel Corporation */ + +#ifndef _ICE_MIGRATION_H_ +#define _ICE_MIGRATION_H_ + +#if IS_ENABLED (CONFIG_ICE_VFIO_PCI ) +void *ice_migration_get_vf (struct pci_dev *vf_pdev); +void ice_migration_put_vf (void *opaque); +void ice_migration_init_vf (void *opaque); +void ice_migration_uninit_vf (void *opaque); + +#else +static inline void *ice_migration_get_vf (struct pci_dev *vf_pdev) +{ +return NULL ; +} +static inline void ice_migration_put_vf (void *opaque) +{ +return NULL ; +} +static inline void ice_migration_init_vf (void *opaque) { } +static inline void ice_migration_uninit_vf (void *opaque) { } +#endif /* CONFIG_ICE_VFIO_PCI */ + +#endif /* _ICE_MIGRATION_H_ */
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [PATCH iwl-next V2 04/15] ice: add migration init field and helper functions
Quoted 63 lines · Lingyu Liu
Adds a function to get ice VF device from pci device. Adds a field in VF structure to indicate migration init state, and functions to init and uninit migration.
This will be used by
ice_vfio_pcidriver introduced in coming patches from this series.Signed-off-by Lingyu Liu <[email protected]>Signed-off-by Yahui Cao <[email protected]>--- drivers/net/ethernet/intel/ice/Makefile | 1 + drivers/net/ethernet/intel/ice/ice.h | 1 + .../net/ethernet/intel/ice/ice_migration.c | 68 +++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_vf_lib.c | 7 ++ drivers/net/ethernet/intel/ice/ice_vf_lib.h | 1 + include/linux/net/intel/ice_migration.h | 26 +++++++ 6 files changed, 104 insertions(+) create mode 100644 drivers/net/ethernet/intel/ice/ice_migration.c create mode 100644 include/linux/net/intel/ice_migration.hdrivers/net/ethernet/intel/ice/Makefile +1
-0@@ -49,3 +49,4 @@ ice-$(CONFIG_RFS_ACCEL) += ice_arfs.o ice-$(CONFIG_XDP_SOCKETS) += ice_xsk.o ice-$(CONFIG_ICE_SWITCHDEV) += ice_eswitch.o ice_eswitch_br.o ice-$(CONFIG_GNSS) += ice_gnss.o +ice-$(CONFIG_ICE_VFIO_PCI) += ice_migration.odrivers/net/ethernet/intel/ice/ice.h +1
-0@@ -55,6 +55,7 @@#include <net/vxlan.h> #include <net/gtp.h> #include <linux/ppp_defs.h> +#include <linux/net/intel/ice_migration.h> #include "ice_devids.h" #include "ice_type.h" #include "ice_txrx.h" drivers/net/ethernet/intel/ice/ice_migration.c new file +16
-0@@ -0,0 +1,68 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2018-2023 Intel Corporation */ + +#include "ice.h" + +/** +* ice_migration_get_vf - Get ice VF structure pointer by pdev +* @vf_pdev: pointer to ice vfio pci VF pdev structure +* +* Return nonzero for success, NULL for failure. +* +* ice_put_vf() should be called after finishing accessing VF +*/ +void *ice_migration_get_vf (struct pci_dev *vf_pdev) +{ +struct pci_dev *pf_pdev = vf_pdev->physfn;
drivers/net/ethernet/intel/ice/ice_migration.c line 26
Quoted 10 lines
drivers/net/ethernet/intel/ice/ice_migration.c new file +10
-0+int vf_id =pci_iov_vf_id (vf_pdev); +struct ice_pf *pf; + +if (!pf_pdev || vf_id <0 ) +return NULL ; + + pf =pci_get_drvdata (pf_pdev); +return ice_get_vf_by_id (pf, vf_id); +} +EXPORT_SYMBOL (ice_migration_get_vf);
This doesn't look right, you shouldn't need functions like this.
The VF knows itself, and it goes back to the PF safely from the VFIO code. You should not be doing things like 'vf_pdev->physfn'
Loook at how the other drivers are structured.
Jason
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [PATCH iwl-next V2 04/15] ice: add migration init field and helper functions
Quoted 76 lines · Jason Gunthorpe
Quoted 63 lines · Lingyu Liu
Adds a function to get ice VF device from pci device. Adds a field in VF structure to indicate migration init state, and functions to init and uninit migration.
This will be used by
ice_vfio_pcidriver introduced in coming patches from this series.Signed-off-by Lingyu Liu <[email protected]>Signed-off-by Yahui Cao <[email protected]>--- drivers/net/ethernet/intel/ice/Makefile | 1 + drivers/net/ethernet/intel/ice/ice.h | 1 + .../net/ethernet/intel/ice/ice_migration.c | 68 +++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_vf_lib.c | 7 ++ drivers/net/ethernet/intel/ice/ice_vf_lib.h | 1 + include/linux/net/intel/ice_migration.h | 26 +++++++ 6 files changed, 104 insertions(+)create mode 100644
drivers/net/ethernet/intel/ice/ice_migration.ccreate mode 100644include/linux/net/intel/ice_migration.hdrivers/net/ethernet/intel/ice/Makefile +1
-0@@ -49,3 +49,4 @@ ice-$(CONFIG_RFS_ACCEL) += ice_arfs.o ice-$(CONFIG_XDP_SOCKETS) += ice_xsk.o ice-$(CONFIG_ICE_SWITCHDEV) += ice_eswitch.o ice_eswitch_br.o ice-$(CONFIG_GNSS) += ice_gnss.o +ice-$(CONFIG_ICE_VFIO_PCI) += ice_migration.odrivers/net/ethernet/intel/ice/ice.h +1
-0@@ -55,6 +55,7 @@#include <net/vxlan.h> #include <net/gtp.h> #include <linux/ppp_defs.h> +#include <linux/net/intel/ice_migration.h> #include "ice_devids.h" #include "ice_type.h" #include "ice_txrx.h" drivers/net/ethernet/intel/ice/ice_migration.c new file +16
-0@@ -0,0 +1,68 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2018-2023 Intel Corporation */ + +#include "ice.h" + +/** +* ice_migration_get_vf - Get ice VF structure pointer by pdev +* @vf_pdev: pointer to ice vfio pci VF pdev structure +* +* Return nonzero for success, NULL for failure. +* +* ice_put_vf() should be called after finishing accessing VF +*/ +void *ice_migration_get_vf (struct pci_dev *vf_pdev) +{ +struct pci_dev *pf_pdev = vf_pdev->physfn;drivers/net/ethernet/intel/ice/ice_migration.c line 26
Quoted 10 lines
drivers/net/ethernet/intel/ice/ice_migration.c new file +10
-0+int vf_id =pci_iov_vf_id (vf_pdev); +struct ice_pf *pf; + +if (!pf_pdev || vf_id <0 ) +return NULL ; + + pf =pci_get_drvdata (pf_pdev); +return ice_get_vf_by_id (pf, vf_id); +} +EXPORT_SYMBOL (ice_migration_get_vf);This doesn't look right, you shouldn't need functions like this.
Quoted 6 lines
The VF knows itself, and it goes back to the PF safely from the VFIO code. You should not be doing things like '
vf_pdev->physfn'Loook at how the other drivers are structured.
Jason
I'll change the way to go back to the PF, following how other drivers do.
Thanks.
Yahui.
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 5/15 ice: save VF messages as device state
In reply to Lingyu LiuAdd a list in VF structure to save the virtual channel messages sent by VF.
ice_vfio_pci driver introduced in following patches from this series needs the saved virtual channel list.
--- drivers/net/ethernet/intel/ice/ice.h | 1 + .../net/ethernet/intel/ice/ice_migration.c | 94 +++++++++++++++++++ .../intel/ice/ice_migration_private.h | 22 +++++ drivers/net/ethernet/intel/ice/ice_vf_lib.h | 3 + drivers/net/ethernet/intel/ice/ice_virtchnl.c | 3 + 5 files changed, 123 insertions(+) create mode 100644 drivers/net/ethernet/intel/ice/ice_migration_private.h
drivers/net/ethernet/intel/ice/ice.h +1 -0
@@ -77,6 +77,7 @@#include "ice_vsi_vlan_ops.h" #include "ice_gnss.h" #include "ice_irq.h" +#include "ice_migration_private.h" #define ICE_BAR0 0#define ICE_REQ_DESC_MULTIPLE 32
drivers/net/ethernet/intel/ice/ice_migration.c +94 -0
117 lines not on this page · raw message
drivers/net/ethernet/intel/ice/ice_migration_private.h new file +22 -0
@@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright (C) 2018-2023 Intel Corporation */ + +#ifndef _ICE_MIGRATION_PRIVATE_H_ +#define _ICE_MIGRATION_PRIVATE_H_ + +/* This header file is for exposing functions in ice_migration.c to +* files which will be compiled in ice.ko. +* Functions which may be used by other files which will be compiled +* in ice-vfio-pic.ko should be exposed as part of ice_migration.h. +*/ + +#if IS_ENABLED (CONFIG_ICE_VFIO_PCI ) +void ice_migration_save_vf_msg (struct ice_vf *vf, +struct ice_rq_event_info *event); +#else +static inline void +ice_migration_save_vf_msg (struct ice_vf *vf, +struct ice_rq_event_info *event) { } +#endif /* CONFIG_ICE_VFIO_PCI */ + +#endif /* _ICE_MIGRATION_PRIVATE_H_ */
drivers/net/ethernet/intel/ice/ice_vf_lib.h +3 -0
@@ -77,6 +77,7 @@ struct ice_vfs {unsigned long last_printed_mdd_jiffies;/* MDD message rate limit */ }; +#define VIRTCHNL_MSG_MAX 1000/* VF information structure */ struct ice_vf {struct hlist_node entry;
@@ -135,6 +136,8 @@ struct ice_vf {/* devlink port data */ struct devlink_port devlink_port;bool migration_active; +struct list_head virtchnl_msg_list; +u64 virtchnl_msg_num; };/* Flags for controlling behavior of ice_reset_vf */
drivers/net/ethernet/intel/ice/ice_virtchnl.c +3 -0
@@ -4078,6 +4078,9 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event, vf_id, v_opcode, err); } +if (!err && vf->migration_active) +ice_migration_save_vf_msg (vf, event); + finish:mutex_unlock (&vf->cfg_lock);ice_put_vf (vf);
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 6/15 ice: save and restore device state
In reply to Lingyu LiuAdd function to migrate device state.
ice_vfio_pci driver introduced in following patches from this series needs the exported function to save and restore device state.
--- .../net/ethernet/intel/ice/ice_migration.c | 125 ++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_virtchnl.c | 26 +++- drivers/net/ethernet/intel/ice/ice_virtchnl.h | 7 +- include/linux/net/intel/ice_migration.h | 12 ++ 4 files changed, 161 insertions(+), 9 deletions(-)
drivers/net/ethernet/intel/ice/ice_migration.c +125 -0
@@ -160,3 +160,128 @@ void ice_migration_save_vf_msg(struct ice_vf *vf,break ; } } + +/** +* ice_migration_save_devstate - save VF msg to migration buffer +* @opaque: pointer to VF handler in ice vdev +* @buf: pointer to VF msg in migration buffer +* @buf_sz: size of migration buffer +* +* Return 0 for success, negative for error +*/ +int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz) +{ +struct ice_migration_virtchnl_msg_listnode *msg_listnode; +struct ice_migration_virtchnl_msg_slot *last_op; +struct ice_vf *vf = (struct ice_vf *)opaque; +struct device *dev =ice_pf_to_dev (vf->pf); +u64 total_sz =0 ; + +if (vf ==NULL ) +return -EINVAL ; + +list_for_each_entry (msg_listnode, &vf->virtchnl_msg_list, node) { +struct ice_migration_virtchnl_msg_slot *msg_slot; +u64 slot_size; + + msg_slot = &msg_listnode->msg_slot; + slot_size =struct_size (msg_slot, msg_buffer, + msg_slot->msg_len); + total_sz += slot_size; +if (total_sz > buf_sz) { +dev_err (dev,"Insufficient buffer to store virtchnl message for VF %d op: %d, len: %d\n" , + vf->vf_id, msg_slot->opcode, msg_slot->msg_len); +return -ENOBUFS ; + } +dev_dbg (dev,"VF %d copy virtchnl message to migration buffer op: %d, len: %d\n" , + vf->vf_id, msg_slot->opcode, msg_slot->msg_len); +memcpy (buf, msg_slot, slot_size); + buf += slot_size; + } +/* reserve space to mark end of vf messages */ + total_sz +=sizeof (struct ice_migration_virtchnl_msg_slot ); +if (total_sz > buf_sz) { +dev_err (dev,"Insufficient buffer to store virtchnl message for VF %d\n" , + vf->vf_id); +return -ENOBUFS ; + } + +/* use op code unknown to mark end of vc messages */ + last_op = (struct ice_migration_virtchnl_msg_slot *)buf; + last_op->opcode =VIRTCHNL_OP_UNKNOWN ; +return 0 ; +} +EXPORT_SYMBOL (ice_migration_save_devstate); + +/** +* ice_migration_restore_devstate - restore device state at dst +* @opaque: pointer to VF handler in ice vdev +* @buf: pointer to device state buf in migration buffer +* @buf_sz: size of migration buffer +* +* This function uses the device state saved in migration buffer +* to restore device state at dst VM +* +* Return 0 for success, negative for error +*/ +int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz) +{ +struct ice_migration_virtchnl_msg_slot *msg_slot; +struct ice_vf *vf = (struct ice_vf *)opaque; +struct device *dev =ice_pf_to_dev (vf->pf); +struct ice_rq_event_info event; +u64 total_sz =0 ; +u64 op_msglen_sz; +u64 slot_sz; +int ret =0 ; + +if (!buf) +return -EINVAL ; + + msg_slot = (struct ice_migration_virtchnl_msg_slot *)buf; + op_msglen_sz =sizeof (struct ice_migration_virtchnl_msg_slot ); +/* check whether enough space for opcode and msg_len */ +if (total_sz + op_msglen_sz > buf_sz) { +dev_err (dev,"VF %d msg size exceeds buffer size\n" , vf->vf_id); +return -ENOBUFS ; + } + +set_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states); + +while (msg_slot->opcode !=VIRTCHNL_OP_UNKNOWN ) { + slot_sz =struct_size (msg_slot, msg_buffer, msg_slot->msg_len); + total_sz += slot_sz; +/* check whether enough space for whole message */ +if (total_sz > buf_sz) { +dev_err (dev,"VF %d msg size exceeds buffer size\n" , + vf->vf_id); + ret = -ENOBUFS ; +break ; + } +dev_dbg (dev,"VF %d replay virtchnl message op code: %d, msg len: %d\n" , + vf->vf_id, msg_slot->opcode, msg_slot->msg_len); + event.desc.cookie_high = msg_slot->opcode; + event.msg_len = msg_slot->msg_len; + event.desc.retval = vf->vf_id; + event.msg_buf = (unsigned char *)msg_slot->msg_buffer; + ret =ice_vc_process_vf_msg (vf->pf, &event,NULL ); +if (ret) { +dev_err (dev,"failed to replay virtchnl message op code: %d\n" , + msg_slot->opcode); +break ; + } + event.msg_buf =NULL ; + msg_slot = (struct ice_migration_virtchnl_msg_slot *) + ((char *)msg_slot + slot_sz); +/* check whether enough space for opcode and msg_len */ +if (total_sz + op_msglen_sz > buf_sz) { +dev_err (dev,"VF %d msg size exceeds buffer size\n" , + vf->vf_id); + ret = -ENOBUFS ; +break ; + } + } +clear_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states); +return ret; +} +EXPORT_SYMBOL (ice_migration_restore_devstate);
drivers/net/ethernet/intel/ice/ice_virtchnl.c +20 -6
@@ -3895,11 +3895,24 @@ ice_is_malicious_vf(struct ice_vf *vf, struct ice_mbx_data *mbxdata)* @event: pointer to the AQ event * @mbxdata: information used to detect VF attempting mailbox overflow * -* called from the common asq/arq handler to -* process request from VF +* This function will be called from: +* 1. the common asq/arq handler to process request from VF +* +* The return value is ignored, as the command will send the status of the +* request as a response to the VF. This flow sets the mbxdata to +* a non-NULL value and must call ice_is_malicious_vf to determine if this +* VF might be attempting to overflow the PF message queue. +* +* 2. replay virtual channel commamds during live migration +* +* The return value is used to indicate failure to replay vc commands and +* that the migration failed. This flow sets mbxdata to NULL and skips the +* ice_is_malicious_vf checks which are unnecessary during replay. +* +* Return 0 if success, negative for failure. */ -void ice_vc_process_vf_msg (struct ice_pf *pf,struct ice_rq_event_info *event, -struct ice_mbx_data *mbxdata) +int ice_vc_process_vf_msg (struct ice_pf *pf,struct ice_rq_event_info *event, +struct ice_mbx_data *mbxdata) {u32 v_opcode =le32_to_cpu (event->desc.cookie_high);s16 vf_id =le16_to_cpu (event->desc.retval);
@@ -3916,13 +3929,13 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event,if (!vf) {dev_err (dev,"Unable to locate VF for message from VF ID %d, opcode %d, len %d\n" , vf_id, v_opcode, msglen); -return ; +return -EINVAL ; }mutex_lock (&vf->cfg_lock);/* Check if the VF is trying to overflow the mailbox */ -if (ice_is_malicious_vf (vf, mbxdata)) +if (!test_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states) &&ice_is_malicious_vf (vf, mbxdata)) goto finish;/* Check if VF is disabled. */
@@ -4084,4 +4097,5 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event, finish:mutex_unlock (&vf->cfg_lock);ice_put_vf (vf); +return err; }
drivers/net/ethernet/intel/ice/ice_virtchnl.h +4 -3
@@ -63,8 +63,8 @@ int ice_vc_respond_to_vf(struct ice_vf *vf, u32 v_opcode, enum virtchnl_status_code v_retval, u8 *msg, u16 msglen);bool ice_vc_isvalid_vsi_id (struct ice_vf *vf,u16 vsi_id); -void ice_vc_process_vf_msg (struct ice_pf *pf,struct ice_rq_event_info *event, -struct ice_mbx_data *mbxdata); +int ice_vc_process_vf_msg (struct ice_pf *pf,struct ice_rq_event_info *event, +struct ice_mbx_data *mbxdata);#else /* CONFIG_PCI_IOV */ static inline void ice_virtchnl_set_dflt_ops (struct ice_vf *vf) { }static inline void ice_virtchnl_set_repr_ops (struct ice_vf *vf) { }
@@ -84,10 +84,11 @@ static inline bool ice_vc_isvalid_vsi_id(struct ice_vf *vf, u16 vsi_id) return false; } -static inline void +static inline int ice_vc_process_vf_msg (struct ice_pf *pf,struct ice_rq_event_info *event,struct ice_mbx_data *mbxdata) { +return -EOPNOTSUPP ; }#endif /* !CONFIG_PCI_IOV */
include/linux/net/intel/ice_migration.h +12 -0
@@ -9,6 +9,8 @@ void *ice_migration_get_vf(struct pci_dev *vf_pdev);void ice_migration_put_vf (void *opaque);void ice_migration_init_vf (void *opaque);void ice_migration_uninit_vf (void *opaque); +int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz); +int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz);#else static inline void *ice_migration_get_vf (struct pci_dev *vf_pdev)
@@ -21,6 +23,16 @@ static inline void ice_migration_put_vf(void *opaque) }static inline void ice_migration_init_vf (void *opaque) { }static inline void ice_migration_uninit_vf (void *opaque) { } +static inline int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz) +{ +return 0 ; +} + +static inline int ice_migration_restore_devstate (void *opaque,const u8 *buf, +u64 buf_sz) +{ +return 0 ; +}#endif /* CONFIG_ICE_VFIO_PCI */ #endif /* _ICE_MIGRATION_H_ */
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 7/15 ice: do not notify VF link state during migration
In reply to Lingyu LiuPF driver will notify VF link state when replaying virtual channel commands GET_VF_RESOURCE and ENABLE_QUEUES.
Block PF driver to send message to VF if it is migrating.
--- drivers/net/ethernet/intel/ice/ice_virtchnl.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/net/ethernet/intel/ice/ice_virtchnl.c +3 -0
@@ -233,6 +233,9 @@ void ice_vc_notify_vf_link_state(struct ice_vf *vf)struct virtchnl_pf_event pfe = {0 };struct ice_hw *hw = &vf->pf->hw; +if (test_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states)) +return ; + pfe.event =VIRTCHNL_EVENT_LINK_CHANGE ; pfe.severity =PF_EVENT_SEVERITY_INFO ;
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 8/15 ice: change VSI id in virtual channel message after migration
In reply to Lingyu LiuSave the VSI num used in migration src VM in VF structure and change VSI id in virtual channel message payload to the dst VF's VSI id under following two conditions:
1) the VSI id in virtual channel message payload is the same with src VF's VSI id. Or
2) it is replaying virtual channel message.
This is to prevent PF rejects to process VF messages during migration.
--- .../net/ethernet/intel/ice/ice_migration.c | 108 ++++++++++++++++++ .../intel/ice/ice_migration_private.h | 3 + drivers/net/ethernet/intel/ice/ice_vf_lib.c | 1 + drivers/net/ethernet/intel/ice/ice_vf_lib.h | 1 + drivers/net/ethernet/intel/ice/ice_virtchnl.c | 3 + 5 files changed, 116 insertions(+)
drivers/net/ethernet/intel/ice/ice_migration.c +108 -0
@@ -14,6 +14,10 @@ struct ice_migration_virtchnl_msg_listnode {struct ice_migration_virtchnl_msg_slot msg_slot; }; +struct ice_migration_dev_state { +u16 vsi_id; +}__aligned(8) ; +/** * ice_migration_get_vf - Get ice VF structure pointer by pdev * @vf_pdev: pointer to ice vfio pci VF pdev structure
@@ -62,6 +66,7 @@ void ice_migration_init_vf(void *opaque) vf->migration_active = true;INIT_LIST_HEAD (&vf->virtchnl_msg_list); vf->virtchnl_msg_num =0 ; + vf->vm_vsi_num = vf->lan_vsi_num; }EXPORT_SYMBOL (ice_migration_init_vf);
@@ -175,11 +180,24 @@ int ice_migration_save_devstate(void *opaque, u8 *buf, u64 buf_sz)struct ice_migration_virtchnl_msg_slot *last_op;struct ice_vf *vf = (struct ice_vf *)opaque;struct device *dev =ice_pf_to_dev (vf->pf); +struct ice_migration_dev_state *devstate;u64 total_sz =0 ;if (vf ==NULL )return -EINVAL ; +/* reserve space to store device state, saving VSI id in the beginning */ + total_sz +=sizeof (struct ice_migration_dev_state ); +if (total_sz > buf_sz) { +dev_err (dev,"Insufficient buffer to store device state for VF %d\n" , + vf->vf_id); +return -ENOBUFS ; + } + + devstate = (struct ice_migration_dev_state *)buf; + devstate->vsi_id = vf->vm_vsi_num; + buf +=sizeof (*devstate); +list_for_each_entry (msg_listnode, &vf->virtchnl_msg_list, node) {struct ice_migration_virtchnl_msg_slot *msg_slot;u64 slot_size;
@@ -229,6 +247,7 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)struct ice_migration_virtchnl_msg_slot *msg_slot;struct ice_vf *vf = (struct ice_vf *)opaque;struct device *dev =ice_pf_to_dev (vf->pf); +struct ice_migration_dev_state *devstate;struct ice_rq_event_info event;u64 total_sz =0 ;u64 op_msglen_sz;
@@ -238,6 +257,16 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)if (!buf)return -EINVAL ; + total_sz +=sizeof (struct ice_migration_dev_state ); +if (total_sz > buf_sz) { +dev_err (dev,"VF %d msg size exceeds buffer size\n" , vf->vf_id); +return -ENOBUFS ; + } + + devstate = (struct ice_migration_dev_state *)buf; + vf->vm_vsi_num = devstate->vsi_id; +dev_dbg (dev,"VF %d vm vsi num is:%d\n" , vf->vf_id, vf->vm_vsi_num); + buf +=sizeof (*devstate); msg_slot = (struct ice_migration_virtchnl_msg_slot *)buf; op_msglen_sz =sizeof (struct ice_migration_virtchnl_msg_slot );/* check whether enough space for opcode and msg_len */
@@ -285,3 +314,82 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)return ret; }EXPORT_SYMBOL (ice_migration_restore_devstate); + +/** +* ice_migration_fix_msg_vsi - change virtual channel msg VSI id +* +* @vf: pointer to the VF structure +* @v_opcode: virtchnl message operation code +* @msg: pointer to the virtual channel message +* +* After migration, the VSI id of virtual channel message is still +* migration src VSI id. Some virtual channel commands will fail +* due to unmatch VSI id. +* Change virtual channel message payload VSI id to real VSI id. +*/ +void ice_migration_fix_msg_vsi (struct ice_vf *vf,u32 v_opcode,u8 *msg) +{ +if (!vf->migration_active) +return ; + +switch (v_opcode) { +case VIRTCHNL_OP_ADD_ETH_ADDR : +case VIRTCHNL_OP_DEL_ETH_ADDR : +case VIRTCHNL_OP_ENABLE_QUEUES : +case VIRTCHNL_OP_DISABLE_QUEUES : +case VIRTCHNL_OP_CONFIG_RSS_KEY : +case VIRTCHNL_OP_CONFIG_RSS_LUT : +case VIRTCHNL_OP_GET_STATS : +case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE : +case VIRTCHNL_OP_ADD_FDIR_FILTER : +case VIRTCHNL_OP_DEL_FDIR_FILTER : +case VIRTCHNL_OP_ADD_VLAN : +case VIRTCHNL_OP_DEL_VLAN : { +/* Read the beginning two bytes of message for VSI id */ +u16 *vsi_id = (u16 *)msg; + +if (*vsi_id == vf->vm_vsi_num || +test_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states)) + *vsi_id = vf->lan_vsi_num; +break ; + } +case VIRTCHNL_OP_CONFIG_IRQ_MAP : { +struct virtchnl_irq_map_info *irqmap_info; +u16 num_q_vectors_mapped; +int i; + + irqmap_info = (struct virtchnl_irq_map_info *)msg; + num_q_vectors_mapped = irqmap_info->num_vectors; +for (i =0 ; i < num_q_vectors_mapped; i++) { +struct virtchnl_vector_map *map; + + map = &irqmap_info->vecmap[i]; +if (map->vsi_id == vf->vm_vsi_num || +test_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states)) + map->vsi_id = vf->lan_vsi_num; + } +break ; + } +case VIRTCHNL_OP_CONFIG_VSI_QUEUES : { +struct virtchnl_vsi_queue_config_info *qci; + + qci = (struct virtchnl_vsi_queue_config_info *)msg; +if (qci->vsi_id == vf->vm_vsi_num || +test_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states)) { +int i; + + qci->vsi_id = vf->lan_vsi_num; +for (i =0 ; i < qci->num_queue_pairs; i++) { +struct virtchnl_queue_pair_info *qpi; + + qpi = &qci->qpair[i]; + qpi->txq.vsi_id = vf->lan_vsi_num; + qpi->rxq.vsi_id = vf->lan_vsi_num; + } + } +break ; + } +default : +break ; + } +}
drivers/net/ethernet/intel/ice/ice_migration_private.h +3 -0
@@ -13,10 +13,13 @@#if IS_ENABLED (CONFIG_ICE_VFIO_PCI )void ice_migration_save_vf_msg (struct ice_vf *vf,struct ice_rq_event_info *event); +void ice_migration_fix_msg_vsi (struct ice_vf *vf,u32 v_opcode,u8 *msg);#else static inline void ice_migration_save_vf_msg (struct ice_vf *vf,struct ice_rq_event_info *event) { } +static inline void +ice_migration_fix_msg_vsi (struct ice_vf *vf,u32 v_opcode,u8 *msg) { }#endif /* CONFIG_ICE_VFIO_PCI */ #endif /* _ICE_MIGRATION_PRIVATE_H_ */
drivers/net/ethernet/intel/ice/ice_vf_lib.c +1 -0
@@ -1332,6 +1332,7 @@ void ice_vf_set_initialized(struct ice_vf *vf)clear_bit (ICE_VF_STATE_DIS , vf->vf_states);set_bit (ICE_VF_STATE_INIT , vf->vf_states);memset (&vf->vlan_v2_caps,0 ,sizeof (vf->vlan_v2_caps)); + vf->vm_vsi_num = vf->lan_vsi_num; }/**
drivers/net/ethernet/intel/ice/ice_vf_lib.h +1 -0
@@ -138,6 +138,7 @@ struct ice_vf {bool migration_active;struct list_head virtchnl_msg_list;u64 virtchnl_msg_num; +u16 vm_vsi_num; };/* Flags for controlling behavior of ice_reset_vf */
drivers/net/ethernet/intel/ice/ice_virtchnl.c +3 -0
@@ -3974,6 +3974,9 @@ int ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event, goto finish; } +if (vf->migration_active) +ice_migration_fix_msg_vsi (vf, v_opcode, msg); +switch (v_opcode) {case VIRTCHNL_OP_VERSION : err = ops->get_ver_msg (vf, msg);
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 9/15 ice: save and restore RX queue head
In reply to Lingyu LiuSave RX queue head in device migration region and
restore RX queue head at migration dst by writing RX queue context after replaying virtual channel command VIRTCHNL_OP_CONFIG_VSI_QUEUES.
--- .../net/ethernet/intel/ice/ice_migration.c | 98 +++++++++++++++++++ include/linux/net/intel/ice_migration.h | 3 + 2 files changed, 101 insertions(+)
drivers/net/ethernet/intel/ice/ice_migration.c +98 -0
@@ -2,6 +2,7 @@/* Copyright (C) 2018-2023 Intel Corporation */ #include "ice.h" +#include "ice_base.h" struct ice_migration_virtchnl_msg_slot {u32 opcode;
@@ -16,6 +17,8 @@ struct ice_migration_virtchnl_msg_listnode {struct ice_migration_dev_state {u16 vsi_id; +/* next RX desc index to be processed by the device */ +u16 rx_head[IAVF_QRX_TAIL_MAX ]; }__aligned(8) ;/**
@@ -166,6 +169,44 @@ void ice_migration_save_vf_msg(struct ice_vf *vf, } } +static int +ice_migration_save_rx_head (struct ice_vf *vf, +struct ice_migration_dev_state *devstate) +{ +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +struct device *dev; +struct ice_hw *hw; +int i; + + dev =ice_pf_to_dev (pf); + hw = &pf->hw; + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } +ice_for_each_rxq (vsi, i) { +struct ice_rx_ring *rx_ring = vsi->rx_rings[i]; +struct ice_rlan_ctx rlan_ctx = {0 }; +int status; +u16 pf_q; + +if (!test_bit (i, vf->rxq_ena)) +continue ; + + pf_q = rx_ring->reg_idx; + status =ice_read_rxq_ctx (hw, &rlan_ctx, pf_q); +if (status) { +dev_err (dev,"Failed to read RXQ[%d] context, err=%d\n" , + rx_ring->q_index, status); +return -EIO ; + } + devstate->rx_head[i] = rlan_ctx.head; + } +return 0 ; +} +/** * ice_migration_save_devstate - save VF msg to migration buffer * @opaque: pointer to VF handler in ice vdev
@@ -182,6 +223,7 @@ int ice_migration_save_devstate(void *opaque, u8 *buf, u64 buf_sz)struct device *dev =ice_pf_to_dev (vf->pf);struct ice_migration_dev_state *devstate;u64 total_sz =0 ; +int ret;if (vf ==NULL )return -EINVAL ;
@@ -196,6 +238,11 @@ int ice_migration_save_devstate(void *opaque, u8 *buf, u64 buf_sz) devstate = (struct ice_migration_dev_state *)buf; devstate->vsi_id = vf->vm_vsi_num; + ret =ice_migration_save_rx_head (vf, devstate); +if (ret) { +dev_err (dev,"VF %d failed to save rxq head\n" , vf->vf_id); +return ret; + } buf +=sizeof (*devstate);list_for_each_entry (msg_listnode, &vf->virtchnl_msg_list, node) {
@@ -231,6 +278,48 @@ int ice_migration_save_devstate(void *opaque, u8 *buf, u64 buf_sz) }EXPORT_SYMBOL (ice_migration_save_devstate); +static int +ice_migration_restore_rx_head (struct ice_vf *vf, +struct ice_migration_dev_state *devstate) +{ +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +struct device *dev; +int i; + + dev =ice_pf_to_dev (pf); + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } +ice_for_each_rxq (vsi, i) { +struct ice_rx_ring *rx_ring = vsi->rx_rings[i]; +struct ice_rlan_ctx rlan_ctx = {0 }; +int status; +u16 pf_q; + +if (!rx_ring) +return -EINVAL ; + pf_q = rx_ring->reg_idx; + status =ice_read_rxq_ctx (&pf->hw, &rlan_ctx, pf_q); +if (status) { +dev_err (dev,"Failed to read RXQ[%d] context, err=%d\n" , + rx_ring->q_index, status); +return -EIO ; + } + + rlan_ctx.head = devstate->rx_head[i]; + status =ice_write_rxq_ctx (&pf->hw, &rlan_ctx, pf_q); +if (status) { +dev_err (dev,"Failed to set LAN RXQ[%d] context, err=%d\n" , + rx_ring->q_index, status); +return -EIO ; + } + } +return 0 ; +} +/** * ice_migration_restore_devstate - restore device state at dst * @opaque: pointer to VF handler in ice vdev
@@ -299,6 +388,15 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz) msg_slot->opcode);break ; } +if (msg_slot->opcode ==VIRTCHNL_OP_CONFIG_VSI_QUEUES ) { + ret =ice_migration_restore_rx_head (vf, devstate); +if (ret) { +dev_err (dev,"VF %d failed to restore rx head\n" , + vf->vf_id); +break ; + } + } + event.msg_buf =NULL ; msg_slot = (struct ice_migration_virtchnl_msg_slot *) ((char *)msg_slot + slot_sz);
include/linux/net/intel/ice_migration.h +3 -0
@@ -5,6 +5,9 @@#define _ICE_MIGRATION_H_#if IS_ENABLED (CONFIG_ICE_VFIO_PCI ) + +#define IAVF_QRX_TAIL_MAX 256 +void *ice_migration_get_vf (struct pci_dev *vf_pdev);void ice_migration_put_vf (void *opaque);void ice_migration_init_vf (void *opaque);
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 10/15 ice: save and restore TX queue head
In reply to Lingyu LiuSave TX queue head in device migration region and restore TX queue head at migration dst by inserting NOP descriptors packages and kicking doorbell.
By default, VF driver's TX DMA ring context is configured by ice driver. In device resuming stage of live migration, it's ice driver's reponsibility to recover the TX DMA ring context. Due to HW limitation, some of TX DMA ring context can't be recovered by writing mmio registers. Instead, it can be recovered by injecting NOP descriptors into the DMA ring descriptors indirectly. The NOP descriptor injection needs assistance from host kernel driver to read/write TX DMA ring descriptors. As a result, vfio_dma_rw() is used to recover the TX DMA ring context.
--- .../net/ethernet/intel/ice/ice_migration.c | 203 +++++++++++++++++- drivers/net/ethernet/intel/ice/ice_virtchnl.c | 10 + include/linux/net/intel/ice_migration.h | 9 +- 3 files changed, 213 insertions(+), 9 deletions(-)
drivers/net/ethernet/intel/ice/ice_migration.c +196 -7
@@ -3,6 +3,7 @@#include "ice.h" #include "ice_base.h" +#include "ice_txrx_lib.h" struct ice_migration_virtchnl_msg_slot {u32 opcode;
@@ -19,6 +20,8 @@ struct ice_migration_dev_state {u16 vsi_id;/* next RX desc index to be processed by the device */ u16 rx_head[IAVF_QRX_TAIL_MAX ]; +/* next TX desc index to be processed by the device */ +u16 tx_head[IAVF_QRX_TAIL_MAX ]; }__aligned(8) ;/**
@@ -207,10 +210,51 @@ ice_migration_save_rx_head(struct ice_vf *vf,return 0 ; } +static int +ice_migration_save_tx_head (struct ice_vf *vf, +struct ice_migration_dev_state *devstate) +{ +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +struct device *dev; +int i =0 ; + + dev =ice_pf_to_dev (pf); + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } + +ice_for_each_txq (vsi, i) { +u16 tx_head; +u32 reg; + +if (!test_bit (i, vf->txq_ena)) +continue ; + + reg =rd32 (&pf->hw,QTX_COMM_HEAD (vsi->txq_map[i])); + tx_head = (reg &QTX_COMM_HEAD_HEAD_M ) + >>QTX_COMM_HEAD_HEAD_S ; + +if (tx_head ==QTX_COMM_HEAD_HEAD_M || + tx_head == (vsi->tx_rings[i]->count -1 )) +/* when transmitted packet number is 0 or tx_ring +* length, the next packet to be sent is 0. +*/ + tx_head =0 ; +else + tx_head++; + + devstate->tx_head[i] = tx_head; + } +return 0 ; +} +/** * ice_migration_save_devstate - save VF msg to migration buffer -* @opaque: pointer to VF handler in ice vdev -* @buf: pointer to VF msg in migration buffer +* @opaque: pointer to vf handler in ice vdev +* @buf: pointer to vf msg in migration buffer * @buf_sz: size of migration buffer * * Return 0 for success, negative for error
@@ -243,6 +287,11 @@ int ice_migration_save_devstate(void *opaque, u8 *buf, u64 buf_sz)dev_err (dev,"VF %d failed to save rxq head\n" , vf->vf_id);return ret; } + ret =ice_migration_save_tx_head (vf, devstate); +if (ret) { +dev_err (dev,"VF %d failed to save txq head\n" , vf->vf_id); +return ret; + } buf +=sizeof (*devstate);list_for_each_entry (msg_listnode, &vf->virtchnl_msg_list, node) {
@@ -320,18 +369,145 @@ ice_migration_restore_rx_head(struct ice_vf *vf,return 0 ; } +static int +ice_migration_restore_tx_head (struct ice_vf *vf, +struct ice_migration_dev_state *devstate, +struct vfio_device *vdev) +{ +struct ice_tx_desc *tx_desc_dummy, *tx_desc; +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +u16 max_ring_len =0 ; +struct device *dev; +int ret =0 ; +int i =0 ; + + dev =ice_pf_to_dev (vf->pf); + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } + +ice_for_each_txq (vsi, i) { +if (!test_bit (i, vf->txq_ena)) +continue ; + + max_ring_len =max (vsi->tx_rings[i]->count, max_ring_len); + } + +if (max_ring_len ==0 ) +return 0 ; + + tx_desc = (struct ice_tx_desc *)kcalloc + (max_ring_len,sizeof (struct ice_tx_desc ),GFP_KERNEL ); + tx_desc_dummy = (struct ice_tx_desc *)kcalloc + (max_ring_len,sizeof (struct ice_tx_desc ),GFP_KERNEL ); +if (!tx_desc || !tx_desc_dummy) { +dev_err (dev,"VF %d failed to allocate memory for tx descriptors to restore tx head\n" , + vf->vf_id); + ret = -ENOMEM ; + goto err; + } + +for (i =0 ; i < max_ring_len; i++) { +u32 td_cmd; + + td_cmd =ICE_TXD_LAST_DESC_CMD |ICE_TX_DESC_CMD_DUMMY ; + tx_desc_dummy[i].cmd_type_offset_bsz = +ice_build_ctob (td_cmd,0 ,SZ_256 ,0 ); + } + +/* For each tx queue, we restore the tx head following below steps: +* 1. backup original tx ring descriptor memory +* 2. overwrite the tx ring descriptor with dummy packets +* 3. kick doorbell register to trigger descriptor writeback, +* then tx head will move from 0 to tail - 1 and tx head is restored +* to the place we expect. +* 4. restore the tx ring with original tx ring descriptor memory in +* order not to corrupt the ring context. +*/ +ice_for_each_txq (vsi, i) { +struct ice_tx_ring *tx_ring = vsi->tx_rings[i]; +u16 *tx_heads = devstate->tx_head; +u32 tx_head; +int j; + +if (!test_bit (i, vf->txq_ena) || tx_heads[i] ==0 ) +continue ; + +if (tx_heads[i] >= tx_ring->count) { +dev_err (dev,"saved tx ring head exceeds tx ring count\n" ); + ret = -EINVAL ; + goto err; + } + ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc, + tx_ring->count *sizeof (tx_desc[0 ]), false); +if (ret) { +dev_err (dev,"kvm read guest tx ring error: %d\n" , + ret); + goto err; + } + ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc_dummy, + tx_heads[i] *sizeof (tx_desc_dummy[0 ]), true); +if (ret) { +dev_err (dev,"kvm write guest return error: %d\n" , + ret); + goto err; + } + +/* Force memory writes to complete before letting h/w know there +* are new descriptors to fetch. +*/ +wmb (); +writel (tx_heads[i], tx_ring->tail); +/* wait until tx_head equals tx_heads[i] - 1 */ + tx_head =rd32 (&pf->hw,QTX_COMM_HEAD (vsi->txq_map[i])); + tx_head = (tx_head &QTX_COMM_HEAD_HEAD_M ) + >>QTX_COMM_HEAD_HEAD_S ; +for (j =0 ; j <QTX_HEAD_RESTORE_DELAY_MAX && + tx_head != (u32)(tx_heads[i] -1 ); j++) { +usleep_range (QTX_HEAD_RESTORE_DELAY_SLEEP_US_MIN ,QTX_HEAD_RESTORE_DELAY_SLEEP_US_MAX ); + tx_head =rd32 (&pf->hw,QTX_COMM_HEAD (vsi->txq_map[i])); + tx_head = (tx_head &QTX_COMM_HEAD_HEAD_M ) + >>QTX_COMM_HEAD_HEAD_S ; + } +if (j ==10 ) { + ret = -EIO ; +dev_err (dev,"VF %d txq[%d] head restore timeout\n" , + vf->vf_id, i); + goto err; + } + ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc, + tx_ring->count *sizeof (tx_desc[0 ]), true); +if (ret) { +dev_err (dev,"kvm write guest tx ring error: %d\n" , + ret); + goto err; + } + } + +err: +kfree (tx_desc_dummy); +kfree (tx_desc); + +return ret; +} +/** * ice_migration_restore_devstate - restore device state at dst * @opaque: pointer to VF handler in ice vdev * @buf: pointer to device state buf in migration buffer * @buf_sz: size of migration buffer +* @vdev: pointer to vfio device * * This function uses the device state saved in migration buffer * to restore device state at dst VM * * Return 0 for success, negative for error */ -int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz) +int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz, +struct vfio_device *vdev) {struct ice_migration_virtchnl_msg_slot *msg_slot;struct ice_vf *vf = (struct ice_vf *)opaque;
@@ -343,7 +519,7 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)u64 slot_sz;int ret =0 ; -if (!buf) +if (!buf || !vdev)return -EINVAL ; total_sz +=sizeof (struct ice_migration_dev_state );
@@ -374,7 +550,7 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)dev_err (dev,"VF %d msg size exceeds buffer size\n" , vf->vf_id); ret = -ENOBUFS ; -break ; + goto err; }dev_dbg (dev,"VF %d replay virtchnl message op code: %d, msg len: %d\n" , vf->vf_id, msg_slot->opcode, msg_slot->msg_len);
@@ -386,7 +562,7 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)if (ret) {dev_err (dev,"failed to replay virtchnl message op code: %d\n" , msg_slot->opcode); -break ; + goto err; }if (msg_slot->opcode ==VIRTCHNL_OP_CONFIG_VSI_QUEUES ) { ret =ice_migration_restore_rx_head (vf, devstate);
@@ -405,9 +581,22 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)dev_err (dev,"VF %d msg size exceeds buffer size\n" , vf->vf_id); ret = -ENOBUFS ; -break ; + goto err; } } + +/* Since we can't restore tx head directly due to HW limitation, we +* could only restore tx head indirectly by dummy packets injection. +* After virtual channel replay completes, tx rings are enabled. +* Then restore tx head for tx rings by injecting dummy packets. +*/ + ret =ice_migration_restore_tx_head (vf, devstate, vdev); +if (ret) { +dev_err (dev,"failed to restore tx queue head\n" ); + goto err; + } + +err:clear_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states);return ret; }
drivers/net/ethernet/intel/ice/ice_virtchnl.c +10 -0
@@ -1328,6 +1328,16 @@ static int ice_vc_ena_qs_msg(struct ice_vf *vf, u8 *msg)continue ;ice_vf_ena_txq_interrupt (vsi, vf_q_id); +/* set TXQ head value to 0x1FFF to indicate no packet is sent. According +* to CVL HAS Transmit Queue Context Structure, the size of descriptor +* queue is from 8 descriptores (QLEN=0x8) to 8K-32 descriptors +* (QLEN=0x1FE0). So QTX_COMM_HEAD.HEAD rang value from 0x1fe0 to 0x1fff +* is reserved and will never be used by HW. So, use 0x1FFF as a marker. +* This is used by live migration. +*/ +if (vf->migration_active) +wr32 (&vsi->back->hw,QTX_COMM_HEAD (vsi->txq_map[vf_q_id]), +QTX_COMM_HEAD_HEAD_M );set_bit (vf_q_id, vf->txq_ena); }
include/linux/net/intel/ice_migration.h +7 -2
@@ -3,17 +3,22 @@#ifndef _ICE_MIGRATION_H_#define _ICE_MIGRATION_H_ +#include <linux/vfio.h> #if IS_ENABLED (CONFIG_ICE_VFIO_PCI )#define IAVF_QRX_TAIL_MAX 256 +#define QTX_HEAD_RESTORE_DELAY_MAX 100 +#define QTX_HEAD_RESTORE_DELAY_SLEEP_US_MIN 10 +#define QTX_HEAD_RESTORE_DELAY_SLEEP_US_MAX 10void *ice_migration_get_vf (struct pci_dev *vf_pdev);void ice_migration_put_vf (void *opaque);void ice_migration_init_vf (void *opaque);void ice_migration_uninit_vf (void *opaque);int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz); -int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz); +int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz, +struct vfio_device *vdev);#else static inline void *ice_migration_get_vf (struct pci_dev *vf_pdev)
@@ -32,7 +37,7 @@ static inline int ice_migration_save_devstate(void *opaque, u8 *buf, u64 buf_sz) }static inline int ice_migration_restore_devstate (void *opaque,const u8 *buf, -u64 buf_sz) +u64 buf_sz,struct vfio_device *vdev) {return 0 ; }
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Jason Gunthorpe
reviewerRe: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Quoted 73 lines · Lingyu Liu
drivers/net/ethernet/intel/ice/ice_migration.c +68
-0+static int +ice_migration_restore_tx_head (struct ice_vf *vf, +struct ice_migration_dev_state *devstate, +struct vfio_device *vdev) +{ +struct ice_tx_desc *tx_desc_dummy, *tx_desc; +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +u16 max_ring_len =0 ; +struct device *dev; +int ret =0 ; +int i =0 ; + + dev =ice_pf_to_dev (vf->pf); + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } + +ice_for_each_txq (vsi, i) { +if (!test_bit (i, vf->txq_ena)) +continue ; + + max_ring_len =max (vsi->tx_rings[i]->count, max_ring_len); + } + +if (max_ring_len ==0 ) +return 0 ; + + tx_desc = (struct ice_tx_desc *)kcalloc + (max_ring_len,sizeof (struct ice_tx_desc ),GFP_KERNEL ); + tx_desc_dummy = (struct ice_tx_desc *)kcalloc + (max_ring_len,sizeof (struct ice_tx_desc ),GFP_KERNEL ); +if (!tx_desc || !tx_desc_dummy) { +dev_err (dev,"VF %d failed to allocate memory for tx descriptors to restore tx head\n" , + vf->vf_id); + ret = -ENOMEM ; + goto err; + } + +for (i =0 ; i < max_ring_len; i++) { +u32 td_cmd; + + td_cmd =ICE_TXD_LAST_DESC_CMD |ICE_TX_DESC_CMD_DUMMY ; + tx_desc_dummy[i].cmd_type_offset_bsz = +ice_build_ctob (td_cmd,0 ,SZ_256 ,0 ); + } + +/* For each tx queue, we restore the tx head following below steps: +* 1. backup original tx ring descriptor memory +* 2. overwrite the tx ring descriptor with dummy packets +* 3. kick doorbell register to trigger descriptor writeback, +* then tx head will move from 0 to tail - 1 and tx head is restored +* to the place we expect. +* 4. restore the tx ring with original tx ring descriptor memory in +* order not to corrupt the ring context. +*/ +ice_for_each_txq (vsi, i) { +struct ice_tx_ring *tx_ring = vsi->tx_rings[i]; +u16 *tx_heads = devstate->tx_head; +u32 tx_head; +int j; + +if (!test_bit (i, vf->txq_ena) || tx_heads[i] ==0 ) +continue ; + +if (tx_heads[i] >= tx_ring->count) {
drivers/net/ethernet/intel/ice/ice_migration.c ice_migration_restore_rx_head(struct ice_vf *vf, · line 449
Quoted 10 lines
drivers/net/ethernet/intel/ice/ice_migration.c +10
-0+dev_err (dev,"saved tx ring head exceeds tx ring count\n" ); + ret = -EINVAL ; + goto err; + } + ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc, + tx_ring->count *sizeof (tx_desc[0 ]), false); +if (ret) { +dev_err (dev,"kvm read guest tx ring error: %d\n" , + ret); + goto err;
You can't call VFIO functions from a netdev driver. All this code needs to be moved into the varient driver.
This design seems pretty wild to me, it doesn't seem too robust against a hostile VM - eg these DMAs can all fail under guest control, and then what?
We also don't have any guarentees defined for the VFIO protocol about what state the vIOMMU will be in prior to reaching RUNNING.
IDK, all of this looks like it is trying really hard to hackily force HW that was never ment to support live migration to somehow do something that looks like it.
You really need to present an explanation in the VFIO driver comments about how this whole scheme actually works and is secure and functional against a hostile guest.
Jason
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Tian, Kevin
reviewerRe: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Quoted 96 lines
From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, June 21, 2023 10:37 PMReview
Quoted 1 line · Lingyu Liu
(quoted diff) +0
-0
b/drivers/net/ethernet/intel/ice/ice_migration.cQuoted 28 lines
index
2579bc0bd193..c2a83a97af05100644drivers/net/ethernet/intel/ice/ice_migration.c +24
-0+static int +ice_migration_restore_tx_head (struct ice_vf *vf, +struct ice_migration_dev_state *devstate, +struct vfio_device *vdev) +{ +struct ice_tx_desc *tx_desc_dummy, *tx_desc; +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +u16 max_ring_len =0 ; +struct device *dev; +int ret =0 ; +int i =0 ; + + dev =ice_pf_to_dev (vf->pf); + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } + +ice_for_each_txq (vsi, i) { +if (!test_bit (i, vf->txq_ena)) +continue ; +drivers/net/ethernet/intel/ice/ice_migration.c ice_migration_restore_rx_head(struct ice_vf *vf,
Quoted 10 lines
drivers/net/ethernet/intel/ice/ice_migration.c +10
-0+ max_ring_len =max (vsi->tx_rings[i]->count, max_ring_len); + } + +if (max_ring_len ==0 ) +return 0 ; + + tx_desc = (struct ice_tx_desc *)kcalloc + (max_ring_len,sizeof (struct ice_tx_desc ),GFP_KERNEL ); + tx_desc_dummy = (struct ice_tx_desc *)kcalloc + (max_ring_len, sizeof(struct ice_tx_desc),
GFP_KERNEL);drivers/net/ethernet/intel/ice/ice_migration.c
Quoted 2 lines
drivers/net/ethernet/intel/ice/ice_migration.c +2
-0+if (!tx_desc || !tx_desc_dummy) { + dev_err(dev, "VF %d failed to allocate memory for txdescriptors to restore tx head\n",
drivers/net/ethernet/intel/ice/ice_migration.c ice_migration_restore_rx_head(struct ice_vf *vf,
Quoted 9 lines
drivers/net/ethernet/intel/ice/ice_migration.c +9
-0+ vf->vf_id); + ret = -ENOMEM ; + goto err; + } + +for (i =0 ; i < max_ring_len; i++) { +u32 td_cmd; + + td_cmd = ICE_TXD_LAST_DESC_CMD |
ICE_TX_DESC_CMD_DUMMY;Quoted 14 lines
drivers/net/ethernet/intel/ice/ice_migration.c +14
-0+ tx_desc_dummy[i].cmd_type_offset_bsz = +ice_build_ctob (td_cmd,0 ,SZ_256 ,0 ); + } + +/* For each tx queue, we restore the tx head following below steps: +* 1. backup original tx ring descriptor memory +* 2. overwrite the tx ring descriptor with dummy packets +* 3. kick doorbell register to trigger descriptor writeback, +* then tx head will move from 0 to tail - 1 and tx head is restored +* to the place we expect. +* 4. restore the tx ring with original tx ring descriptor memory in +* order not to corrupt the ring context. +*/ +ice_for_each_txq (vsi, i) {drivers/net/ethernet/intel/ice/ice_migration.c ice_migration_restore_rx_head(struct ice_vf *vf,
Quoted 10 lines
drivers/net/ethernet/intel/ice/ice_migration.c +10
-0+struct ice_tx_ring *tx_ring = vsi->tx_rings[i]; +u16 *tx_heads = devstate->tx_head; +u32 tx_head; +int j; + +if (!test_bit (i, vf->txq_ena) || tx_heads[i] ==0 ) +continue ; + +if (tx_heads[i] >= tx_ring->count) { + dev_err(dev, "saved tx ring head exceeds tx ringcount\n");
drivers/net/ethernet/intel/ice/ice_migration.c ice_migration_restore_rx_head(struct ice_vf *vf, · line 449
Quoted 9 lines
drivers/net/ethernet/intel/ice/ice_migration.c +9
-0+ ret = -EINVAL ; + goto err; + } + ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc, + tx_ring->count *sizeof (tx_desc[0 ]), false); +if (ret) { +dev_err (dev,"kvm read guest tx ring error: %d\n" , + ret); + goto err;You can't call VFIO functions from a netdev driver. All this code needs to be moved into the varient driver.
Quoted 3 lines
This design seems pretty wild to me, it doesn't seem too robust against a hostile VM - eg these DMAs can all fail under guest control, and then what?
Yeah that sounds fragile.
at least the range which will be overwritten in the resuming path should be verified in the src side. If inaccessible then the driver should fail the state transition immediately instead of letting it identified in the resuming path which is unrecoverable.
btw I don't know how its spec describes the hw behavior in such situation. If the behavior is undefined when a hostile software deliberately causes DMA failures to TX queue then not restoring the queue head could also be an option to continue the migration in such scenario.
Quoted 3 lines
We also don't have any guarentees defined for the VFIO protocol about what state the vIOMMU will be in prior to reaching RUNNING.
This is a good point. Actually it's not just a gap on vIOMMU. it's kind of a dependency on IOMMUFD no matter the IOAS which the migrated device is currently attached to is GPA or GIOVA. The device state can be restored only after IOMMUFD is fully recovered and the device is re-attached to the IOAS.
Need a way for migration driver to advocate such dependency to the user.
Quoted 9 lines
IDK, all of this looks like it is trying really hard to hackily force HW that was never ment to support live migration to somehow do something that looks like it.
You really need to present an explanation in the VFIO driver comments about how this whole scheme actually works and is secure and functional against a hostile guest.
Agree. And please post the next version to the VFIO community to gain more attention.
Re: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Hi Jason & Kevin,
Quoted 82 lines · Kevin
Quoted 82 lines
From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, June 21, 2023 10:37 PMReview
Quoted 1 line · Lingyu Liu
(quoted diff) +0
-0
b/drivers/net/ethernet/intel/ice/ice_migration.cQuoted 27 lines
index
2579bc0bd193..c2a83a97af05100644drivers/net/ethernet/intel/ice/ice_migration.c +24
-0+static int +ice_migration_restore_tx_head (struct ice_vf *vf, +struct ice_migration_dev_state *devstate, +struct vfio_device *vdev) +{ +struct ice_tx_desc *tx_desc_dummy, *tx_desc; +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +u16 max_ring_len =0 ; +struct device *dev; +int ret =0 ; +int i =0 ; + + dev =ice_pf_to_dev (vf->pf); + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } + +ice_for_each_txq (vsi, i) { +if (!test_bit (i, vf->txq_ena)) +continue ; +drivers/net/ethernet/intel/ice/ice_migration.c ice_migration_restore_rx_head(struct ice_vf *vf,
Quoted 10 lines
drivers/net/ethernet/intel/ice/ice_migration.c +10
-0+ max_ring_len =max (vsi->tx_rings[i]->count, max_ring_len); + } + +if (max_ring_len ==0 ) +return 0 ; + + tx_desc = (struct ice_tx_desc *)kcalloc + (max_ring_len,sizeof (struct ice_tx_desc ),GFP_KERNEL ); + tx_desc_dummy = (struct ice_tx_desc *)kcalloc + (max_ring_len, sizeof(struct ice_tx_desc),
GFP_KERNEL);drivers/net/ethernet/intel/ice/ice_migration.c
Quoted 2 lines
drivers/net/ethernet/intel/ice/ice_migration.c +2
-0+if (!tx_desc || !tx_desc_dummy) { + dev_err(dev, "VF %d failed to allocate memory for txdescriptors to restore tx head\n",
drivers/net/ethernet/intel/ice/ice_migration.c ice_migration_restore_rx_head(struct ice_vf *vf,
Quoted 9 lines
drivers/net/ethernet/intel/ice/ice_migration.c +9
-0+ vf->vf_id); + ret = -ENOMEM ; + goto err; + } + +for (i =0 ; i < max_ring_len; i++) { +u32 td_cmd; + + td_cmd = ICE_TXD_LAST_DESC_CMD |
ICE_TX_DESC_CMD_DUMMY;Quoted 14 lines
drivers/net/ethernet/intel/ice/ice_migration.c +14
-0+ tx_desc_dummy[i].cmd_type_offset_bsz = +ice_build_ctob (td_cmd,0 ,SZ_256 ,0 ); + } + +/* For each tx queue, we restore the tx head following below steps: +* 1. backup original tx ring descriptor memory +* 2. overwrite the tx ring descriptor with dummy packets +* 3. kick doorbell register to trigger descriptor writeback, +* then tx head will move from 0 to tail - 1 and tx head is restored +* to the place we expect. +* 4. restore the tx ring with original tx ring descriptor memory in +* order not to corrupt the ring context. +*/ +ice_for_each_txq (vsi, i) {drivers/net/ethernet/intel/ice/ice_migration.c ice_migration_restore_rx_head(struct ice_vf *vf,
Quoted 10 lines
drivers/net/ethernet/intel/ice/ice_migration.c +10
-0+struct ice_tx_ring *tx_ring = vsi->tx_rings[i]; +u16 *tx_heads = devstate->tx_head; +u32 tx_head; +int j; + +if (!test_bit (i, vf->txq_ena) || tx_heads[i] ==0 ) +continue ; + +if (tx_heads[i] >= tx_ring->count) { + dev_err(dev, "saved tx ring head exceeds tx ringcount\n");
Quoted 11 lines
Quoted 11 lines
drivers/net/ethernet/intel/ice/ice_migration.c ice_migration_restore_rx_head(struct ice_vf *vf, · line 449
Quoted 9 lines
drivers/net/ethernet/intel/ice/ice_migration.c +9
-0+ ret = -EINVAL ; + goto err; + } + ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc, + tx_ring->count *sizeof (tx_desc[0 ]), false); +if (ret) { +dev_err (dev,"kvm read guest tx ring error: %d\n" , + ret); + goto err;You can't call VFIO functions from a netdev driver. All this code needs to be moved into the varient driver.
Will move vfio_dma_rw() into vfio driver and passing callback function into netdev driver
Quoted 11 lines
Quoted 4 lines
This design seems pretty wild to me, it doesn't seem too robust against a hostile VM - eg these DMAs can all fail under guest control, and then what?
Yeah that sounds fragile.
at least the range which will be overwritten in the resuming path should be verified in the src side. If inaccessible then the driver should fail the state transition immediately instead of letting it identified in the resuming path which is unrecoverable.
Quoted 4 lines
btw I don't know how its spec describes the hw behavior in such situation. If the behavior is undefined when a hostile software deliberately causes DMA failures to TX queue then not restoring the queue head could also be an option to continue the migration in such scenario.
Thanks for the advice. Will check the vfio_dma_rw() correctness on the source side and fail the state transition once function return failure.
When a hostile software deliberately causes DMA failure to TX queue, TX queue head will remain to be the original value, which is 0 on destination side cases. In this case, I'll let VM resumes by letting TX HEAD to stay with original value.
Quoted 10 lines
Quoted 3 lines
We also don't have any guarentees defined for the VFIO protocol about what state the vIOMMU will be in prior to reaching RUNNING.
This is a good point. Actually it's not just a gap on vIOMMU. it's kind of a dependency on IOMMUFD no matter the IOAS which the migrated device is currently attached to is GPA or GIOVA. The device state can be restored only after IOMMUFD is fully recovered and the device is re-attached to the IOAS.
Need a way for migration driver to advocate such dependency to the user.
Since this part is new to me, may need further guidance on how to resolve the dependency from you and other community experts.
Thanks.
Quoted 11 lines
Quoted 9 lines
IDK, all of this looks like it is trying really hard to hackily force HW that was never ment to support live migration to somehow do something that looks like it.
You really need to present an explanation in the VFIO driver comments about how this whole scheme actually works and is secure and functional against a hostile guest.
Agree. And please post the next version to the VFIO community to gain more attention.
I'll add more comments about the whole scheme and post next version to VFIO community.
Thank you Jason and Kevin for the valuable feedback.
Thanks.
Yahui.
Jason Gunthorpe
reviewerRe: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Quoted 5 lines · Yahui
Quoted 2 lines
Quoted 2 lines
You can't call VFIO functions from a netdev driver. All this code needs to be moved into the varient driver.
Will move
vfio_dma_rw()into vfio driver and passing callback function into netdev driver
Please make proper layers, you should not need to stitch your driver together with weird function pointers.
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
We also don't have any guarentees defined for the VFIO protocol about what state the vIOMMU will be in prior to reaching RUNNING.
This is a good point. Actually it's not just a gap on vIOMMU. it's kind of a dependency on IOMMUFD no matter the IOAS which the migrated device is currently attached to is GPA or GIOVA. The device state can be restored only after IOMMUFD is fully recovered and the device is re-attached to the IOAS.
Quoted 4 lines
Quoted 1 line
Need a way for migration driver to advocate such dependency to the user.
Since this part is new to me, may need further guidance on how to resolve the dependency from you and other community experts.
Personally I'm quite uncomfortable with a driver that tries to work this way, I'm not sure we should encourage this. Can Intel really be convincing that this is safe and correct?
Jason
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Tian, Kevin
reviewerRe: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Quoted 13 lines
From: Jason Gunthorpe <[email protected]>
Sent: Tuesday, July 4, 2023 5:04 AMQuoted 8 lines · Yahui
Quoted 8 lines
Quoted 2 lines
We also don't have any guarentees defined for the VFIO protocol about what state the vIOMMU will be in prior to reaching RUNNING.
This is a good point. Actually it's not just a gap on vIOMMU. it's kind of a dependency on IOMMUFD no matter the IOAS which the migrated device is currently attached to is GPA or GIOVA. The device state can be restored only after IOMMUFD is fully recovered and the device is re-attached to the IOAS.
Quoted 9 lines
Quoted 4 lines
Quoted 1 line
Need a way for migration driver to advocate such dependency to the user.
Since this part is new to me, may need further guidance on how to resolve the dependency from you and other community experts.
Personally I'm quite uncomfortable with a driver that tries to work this way, I'm not sure we should encourage this. Can Intel really be convincing that this is safe and correct?
I dislike it too. Will discuss with Yahui on the correctness of this approach and any cleaner alternative.
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
In reply to Jason GunthorpeQuoted 92 lines
From: Jason Gunthorpe <[email protected]>
Review
Quoted 1 line · Lingyu Liu
(quoted diff) +0
-0
b/drivers/net/ethernet/intel/ice/ice_migration.cQuoted 30 lines
index
2579bc0bd193..c2a83a97af05100644drivers/net/ethernet/intel/ice/ice_migration.c +26
-0+static int +ice_migration_restore_tx_head (struct ice_vf *vf, +struct ice_migration_dev_state *devstate, +struct vfio_device *vdev) +{ +struct ice_tx_desc *tx_desc_dummy, *tx_desc; +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +u16 max_ring_len =0 ; +struct device *dev; +int ret =0 ; +int i =0 ; + + dev =ice_pf_to_dev (vf->pf); + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } + +ice_for_each_txq (vsi, i) { +if (!test_bit (i, vf->txq_ena)) +continue ; + + max_ring_len =max (vsi->tx_rings[i]->count, max_ring_len); + }drivers/net/ethernet/intel/ice/ice_migration.c ice_migration_restore_rx_head(struct ice_vf *vf,
Quoted 10 lines
drivers/net/ethernet/intel/ice/ice_migration.c +10
-0+ +if (max_ring_len ==0 ) +return 0 ; + + tx_desc = (struct ice_tx_desc *)kcalloc + (max_ring_len,sizeof (struct ice_tx_desc ),GFP_KERNEL ); + tx_desc_dummy = (struct ice_tx_desc *)kcalloc + (max_ring_len,sizeof (struct ice_tx_desc ),GFP_KERNEL ); +if (!tx_desc || !tx_desc_dummy) { + dev_err(dev, "VF %d failed to allocate memory for tx descriptors torestore tx head\n",
Quoted 32 lines
drivers/net/ethernet/intel/ice/ice_migration.c +32
-0+ vf->vf_id); + ret = -ENOMEM ; + goto err; + } + +for (i =0 ; i < max_ring_len; i++) { +u32 td_cmd; + + td_cmd =ICE_TXD_LAST_DESC_CMD |ICE_TX_DESC_CMD_DUMMY ; + tx_desc_dummy[i].cmd_type_offset_bsz = +ice_build_ctob (td_cmd,0 ,SZ_256 ,0 ); + } + +/* For each tx queue, we restore the tx head following below steps: +* 1. backup original tx ring descriptor memory +* 2. overwrite the tx ring descriptor with dummy packets +* 3. kick doorbell register to trigger descriptor writeback, +* then tx head will move from 0 to tail - 1 and tx head is restored +* to the place we expect. +* 4. restore the tx ring with original tx ring descriptor memory in +* order not to corrupt the ring context. +*/ +ice_for_each_txq (vsi, i) { +struct ice_tx_ring *tx_ring = vsi->tx_rings[i]; +u16 *tx_heads = devstate->tx_head; +u32 tx_head; +int j; + +if (!test_bit (i, vf->txq_ena) || tx_heads[i] ==0 ) +continue ; + +if (tx_heads[i] >= tx_ring->count) {drivers/net/ethernet/intel/ice/ice_migration.c ice_migration_restore_rx_head(struct ice_vf *vf, · line 449
Quoted 10 lines
drivers/net/ethernet/intel/ice/ice_migration.c +10
-0+dev_err (dev,"saved tx ring head exceeds tx ring count\n" ); + ret = -EINVAL ; + goto err; + } + ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc, + tx_ring->count *sizeof (tx_desc[0 ]), false); +if (ret) { +dev_err (dev,"kvm read guest tx ring error: %d\n" , + ret); + goto err;You can't call VFIO functions from a netdev driver. All this code needs to be moved into the varient driver.
Quoted 6 lines
This design seems pretty wild to me, it doesn't seem too robust against a hostile VM - eg these DMAs can all fail under guest control, and then what?
We also don't have any guarentees defined for the VFIO protocol about what state the vIOMMU will be in prior to reaching RUNNING.
For QEMU, vIOMMU is supposed to be restored prior to devices per the below patch. But indeed, there is no guarantee that all VMMs will do the same as QEMU does.
https://lore.kernel.org/qemu-devel/[email protected]/
Regards,
Yi Liu
Quoted 9 lines
IDK, all of this looks like it is trying really hard to hackily force HW that was never ment to support live migration to somehow do something that looks like it.
You really need to present an explanation in the VFIO driver comments about how this whole scheme actually works and is secure and functional against a hostile guest.
Jason
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Jason Gunthorpe
reviewerRe: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Quoted 7 lines · Yi L
Quoted 7 lines
You can't call VFIO functions from a netdev driver. All this code needs to be moved into the varient driver.
This design seems pretty wild to me, it doesn't seem too robust against a hostile VM - eg these DMAs can all fail under guest control, and then what?
Quoted 6 lines
Quoted 2 lines
We also don't have any guarentees defined for the VFIO protocol about what state the vIOMMU will be in prior to reaching RUNNING.
For QEMU, vIOMMU is supposed to be restored prior to devices per the below patch. But indeed, there is no guarantee that all VMMs will do the same as QEMU does.
That doesn't seem consistent with how the kernel interface is defined to work, I wonder why it was done?
Since it is 2017, I suppose it has to do with internal qemu devices?
Jason
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Quoted 15 lines
From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, June 28, 2023 8:39 PMQuoted 10 lines · Yi L
Quoted 9 lines
You can't call VFIO functions from a netdev driver. All this code needs to be moved into the varient driver.
This design seems pretty wild to me, it doesn't seem too robust against a hostile VM - eg these DMAs can all fail under guest control, and then what?
We also don't have any guarentees defined for the VFIO protocol about what state the vIOMMU will be in prior to reaching RUNNING.
Quoted 6 lines
Quoted 3 lines
For QEMU, vIOMMU is supposed to be restored prior to devices per the below patch. But indeed, there is no guarantee that all VMMs will do the same as QEMU does.
That doesn't seem consistent with how the kernel interface is defined to work, I wonder why it was done?
Add Peter.
Quoted 2 lines
Since it is 2017, I suppose it has to do with internal qemu devices?
Do you mean the devices emulated by QEMU?
Regards,
Yi Liu
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Tian, Kevin
reviewerRe: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Quoted 25 lines
From: Liu, Yi L <[email protected]>
Sent: Monday, July 3, 2023 8:54 PMQuoted 15 lines
From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, June 28, 2023 8:39 PMQuoted 10 lines · Yi L
Quoted 9 lines
You can't call VFIO functions from a netdev driver. All this code needs to be moved into the varient driver.
This design seems pretty wild to me, it doesn't seem too robust against a hostile VM - eg these DMAs can all fail under guest control, and then what?
We also don't have any guarentees defined for the VFIO protocol about what state the vIOMMU will be in prior to reaching RUNNING.
Quoted 6 lines
Quoted 3 lines
For QEMU, vIOMMU is supposed to be restored prior to devices per the below patch. But indeed, there is no guarantee that all VMMs will do the same as QEMU does.
That doesn't seem consistent with how the kernel interface is defined to work, I wonder why it was done?
Quoted 7 lines
Add Peter.
Quoted 2 lines
Since it is 2017, I suppose it has to do with internal qemu devices?
Do you mean the devices emulated by QEMU?
This sounds counter-intuitive even for emulated devices. vIOMMU is involved only when the device wants to access guest memory. But by definition the RESUMING path should just restore the device to the point where it is stopped. Why would such restore create a dependency on vIOMMU state?
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Quoted 18 lines · Kevin
Quoted 18 lines
From: Liu, Yi L <[email protected]>
Sent: Monday, July 3, 2023 8:54 PMQuoted 15 lines
From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, June 28, 2023 8:39 PMQuoted 10 lines · Yi L
Quoted 9 lines
You can't call VFIO functions from a netdev driver. All this code needs to be moved into the varient driver.
This design seems pretty wild to me, it doesn't seem too robust against a hostile VM - eg these DMAs can all fail under guest control, and then what?
We also don't have any guarentees defined for the VFIO protocol about what state the vIOMMU will be in prior to reaching RUNNING.
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
For QEMU, vIOMMU is supposed to be restored prior to devices per the below patch. But indeed, there is no guarantee that all VMMs will do the same as QEMU does.
That doesn't seem consistent with how the kernel interface is defined to work, I wonder why it was done?
Add Peter.
Yi - I think I replied to your private email, I'm not 100% sure this is the same thing asked? I hope you received the email.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Since it is 2017, I suppose it has to do with internal qemu devices?
Quoted 8 lines
Quoted 2 lines
Do you mean the devices emulated by QEMU?
This sounds counter-intuitive even for emulated devices. vIOMMU is involved only when the device wants to access guest memory. But by definition the RESUMING path should just restore the device to the point where it is stopped. Why would such restore create a dependency on vIOMMU state?
For why QEMU migrates vIOMMU earlier than PCI generic devices, I can try to repeat here: IIRC it's because some device needs dma translations during post_load(), I forgot which device and why, but it can make some sense to me if recovering states of the pci device may need help from a translation.
If someone wants to know the details, one can just make vIOMMU prio lower than pci devices and give it a shot and see what explodes first.
Thanks,
Signature · 6 lines
Peter Xu _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Jason Gunthorpe
reviewerRe: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Quoted 10 lines · Peter Xu
Quoted 5 lines
This sounds counter-intuitive even for emulated devices. vIOMMU is involved only when the device wants to access guest memory. But by definition the RESUMING path should just restore the device to the point where it is stopped. Why would such restore create a dependency on vIOMMU state?
For why QEMU migrates vIOMMU earlier than PCI generic devices, I can try to repeat here: IIRC it's because some device needs dma translations during
post_load(), I forgot which device and why, but it can make some sense to me if recovering states of the pci device may need help from a translation.
Well, it has nothing to do with VFIO given the timeframe, and if internal qemu devices are doing weird things like this it also suggests they have problems with P2P as well.
The VFIO definition we have is that a device may not do any DMA while in resuming/resuming_p2p, it is only allowed to initiate DMA once it fully reaches running.
Obviously we have to setup the vIOMMU before going to running.
The internal qemu devices must follow this definition as well or they are not P2P compatible.
About the only exception I could think is if some internal devices was caching translations for some purpose, but it really should setup that caching at entry to running..
Jason
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Quoted 21 lines · Jason Gunthorpe
Quoted 10 lines · Peter Xu
Quoted 5 lines
This sounds counter-intuitive even for emulated devices. vIOMMU is involved only when the device wants to access guest memory. But by definition the RESUMING path should just restore the device to the point where it is stopped. Why would such restore create a dependency on vIOMMU state?
For why QEMU migrates vIOMMU earlier than PCI generic devices, I can try to repeat here: IIRC it's because some device needs dma translations during
post_load(), I forgot which device and why, but it can make some sense to me if recovering states of the pci device may need help from a translation.Well, it has nothing to do with VFIO given the timeframe, and if internal qemu devices are doing weird things like this it also suggests they have problems with P2P as well.
The VFIO definition we have is that a device may not do any DMA while in resuming/resuming_p2p, it is only allowed to initiate DMA once it fully reaches running.
Quoted 8 lines
Obviously we have to setup the vIOMMU before going to running.
The internal qemu devices must follow this definition as well or they are not P2P compatible.
About the only exception I could think is if some internal devices was caching translations for some purpose, but it really should setup that caching at entry to running..
Yes it's something like that which is a pure translation request, rather than doing a real DMA, as far as I remember. It does sound fine to postpone that into a vm state change handler to me, but worth checking when someone really works on it, and also whether it's feasible - e.g. hopefully only a few devices need to work that around.
It may boil down to why we need to avoid migrating vIOMMU before other devices (which I am still unsure about..), and which way is easier.
Thanks,
Signature · 6 lines
Peter Xu _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Jason Gunthorpe
reviewerRe: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Quoted 2 lines · Peter Xu
It may boil down to why we need to avoid migrating vIOMMU before other devices (which I am still unsure about..), and which way is easier.
The statement is just that the kernel cannot assume anything about vIOMMU ordering and people can't make VFIO migration drivers that assume the vIOMMU is setup before the entry to running.
That qemu enforces this ordering today for pointless reasons is interesting, but it should not leak into becoming the ABI contract of the kernel.
Jason
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [PATCH iwl-next V2 10/15] ice: save and restore TX queue head
Quoted 5 lines · Jason Gunthorpe
Quoted 2 lines · Peter Xu
It may boil down to why we need to avoid migrating vIOMMU before other devices (which I am still unsure about..), and which way is easier.
Quoted 7 lines
The statement is just that the kernel cannot assume anything about vIOMMU ordering and people can't make VFIO migration drivers that assume the vIOMMU is setup before the entry to running.
That qemu enforces this ordering today for pointless reasons is interesting, but it should not leak into becoming the ABI contract of the kernel.
I wouldn't say it's pointless because we did that obviously with a reason. But I agree it should be qemu internal and shouldn't even be exposed unless extremely necessary.
Signature · 6 lines
Peter Xu _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 11/15 ice: stop device before saving device states
In reply to Lingyu LiuStop device by disabling TX queue and RX queue.
ice_vfio_pci driver introduced in following patches from this series needs this function when device state transits from VFIO_DEVICE_STATE_RUNNING to VFIO_DEVICE_STATE_STOP.
--- .../net/ethernet/intel/ice/ice_migration.c | 68 +++++++++++++++++++ include/linux/net/intel/ice_migration.h | 5 ++ 2 files changed, 73 insertions(+)
drivers/net/ethernet/intel/ice/ice_migration.c +68 -0
@@ -2,6 +2,8 @@/* Copyright (C) 2018-2023 Intel Corporation */ #include "ice.h" +#include "ice_lib.h" +#include "ice_fltr.h" #include "ice_base.h" #include "ice_txrx_lib.h"
@@ -172,6 +174,72 @@ void ice_migration_save_vf_msg(struct ice_vf *vf, } } +/** +* ice_migration_suspend_vf - suspend device on src +* @opaque: pointer to VF handler in ice vdev +* @is_dst: false: migration src true: migration dst +* +* Return 0 for success, negative for error +*/ +int ice_migration_suspend_vf (void *opaque,bool is_dst) +{ +struct ice_vf *vf = (struct ice_vf *)opaque; +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +struct device *dev; +int ret; + +if (is_dst) +return 0 ; + + dev =ice_pf_to_dev (pf); +if (vf->virtchnl_msg_num >=VIRTCHNL_MSG_MAX ) { +dev_err (dev,"SR-IOV live migration disabled on VF %d. Migration buffer exceeded\n" , + vf->vf_id); +return -EIO ; + } + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } +/* Prevent VSI from incoming packets by removing all filters before +* stop rx ring and draining the traffic. There are possibilities that +* rx ring head value jitters when rx ring is stopped with large amount +* of packets incoming. In this case, HW mismatches SW on rx ring head +* state. As a result, after restoring rx ring head on the destination +* VM, the missing rx descriptors will never be written back, causing +* packets receiving failure and dropped. +*/ +ice_fltr_remove_all (vsi); +/* MAC based filter rule is disabled at this point. Set MAC to zero +* to keep consistency when using ip link to display MAC address. +*/ +eth_zero_addr (vf->hw_lan_addr); +eth_zero_addr (vf->dev_lan_addr); +/* For the tx side, there is possibility that some descriptors are +* still pending to be transmitted by HW. Since VM is stopped now, +* wait a while to make sure all the transmission is completed. +* For the rx side, head value jittering may happen in case of high +* packet rate. Since all forwarding filters are removed now, wait a +* while to make sure all the reception is completed and rx head no +* longer moves. +*/ +usleep_range (1000 ,2000 ); + ret =ice_vsi_stop_lan_tx_rings (vsi,ICE_NO_RESET , vf->vf_id); +if (ret) { +dev_err (dev,"VF %d failed to stop tx rings\n" , vf->vf_id); +return -EIO ; + } + ret =ice_vsi_stop_all_rx_rings (vsi); +if (ret) { +dev_err (dev,"VF %d failed to stop rx rings\n" , vf->vf_id); +return -EIO ; + } +return 0 ; +} +EXPORT_SYMBOL (ice_migration_suspend_vf); +static int ice_migration_save_rx_head (struct ice_vf *vf,struct ice_migration_dev_state *devstate)
include/linux/net/intel/ice_migration.h +5 -0
@@ -16,6 +16,7 @@ void *ice_migration_get_vf(struct pci_dev *vf_pdev);void ice_migration_put_vf (void *opaque);void ice_migration_init_vf (void *opaque);void ice_migration_uninit_vf (void *opaque); +int ice_migration_suspend_vf (void *opaque,bool mig_dst);int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz);int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz,struct vfio_device *vdev);
@@ -31,6 +32,10 @@ static inline void ice_migration_put_vf(void *opaque) }static inline void ice_migration_init_vf (void *opaque) { }static inline void ice_migration_uninit_vf (void *opaque) { } +static inline int ice_migration_suspend_vf (void *opaque,bool mig_dst) +{ +return 0 ; +}static inline int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz) {return 0 ;
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 12/15 ice: mask VF advanced capabilities if live migration is activated
In reply to Lingyu LiuMask the advanced VF capability flags that are
not supported when VF migration is activated.
--- .../net/ethernet/intel/ice/ice_migration.c | 27 +++++++++++++++++++ .../intel/ice/ice_migration_private.h | 6 +++++ drivers/net/ethernet/intel/ice/ice_virtchnl.c | 2 ++ 3 files changed, 35 insertions(+)
drivers/net/ethernet/intel/ice/ice_migration.c +27 -0
@@ -748,3 +748,30 @@ void ice_migration_fix_msg_vsi(struct ice_vf *vf, u32 v_opcode, u8 *msg)break ; } } + +#define VIRTCHNL_VF_MIGRATION_SUPPORT_FEATURE \ + (VIRTCHNL_VF_OFFLOAD_L2 | \ + VIRTCHNL_VF_OFFLOAD_RSS_PF | \ + VIRTCHNL_VF_OFFLOAD_RSS_AQ | \ + VIRTCHNL_VF_OFFLOAD_RSS_REG | \ + VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 | \ + VIRTCHNL_VF_OFFLOAD_ENCAP | \ + VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM | \ + VIRTCHNL_VF_OFFLOAD_RX_POLLING | \ + VIRTCHNL_VF_OFFLOAD_WB_ON_ITR | \ + VIRTCHNL_VF_CAP_ADV_LINK_SPEED | \ + VIRTCHNL_VF_OFFLOAD_VLAN | \ + VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC | \ + VIRTCHNL_VF_OFFLOAD_USO) + +/** +* ice_migration_supported_caps - get migration supported VF capablities +* +* When migration is activated, some VF capabilities are not supported. +* So unmask those capability flags for VF resources. +*/ +u32 ice_migration_supported_caps (void ) +{ + +return VIRTCHNL_VF_MIGRATION_SUPPORT_FEATURE ; +}
drivers/net/ethernet/intel/ice/ice_migration_private.h +6 -0
@@ -14,12 +14,18 @@void ice_migration_save_vf_msg (struct ice_vf *vf,struct ice_rq_event_info *event);void ice_migration_fix_msg_vsi (struct ice_vf *vf,u32 v_opcode,u8 *msg); +u32 ice_migration_supported_caps (void );#else static inline void ice_migration_save_vf_msg (struct ice_vf *vf,struct ice_rq_event_info *event) { }static inline void ice_migration_fix_msg_vsi (struct ice_vf *vf,u32 v_opcode,u8 *msg) { } +static inline u32 +ice_migration_supported_caps (void ) +{ +return 0xFFFFFFFF ; +}#endif /* CONFIG_ICE_VFIO_PCI */ #endif /* _ICE_MIGRATION_PRIVATE_H_ */
drivers/net/ethernet/intel/ice/ice_virtchnl.c +2 -0
@@ -468,6 +468,8 @@ static int ice_vc_get_vf_res_msg(struct ice_vf *vf, u8 *msg)VIRTCHNL_VF_OFFLOAD_RSS_REG |VIRTCHNL_VF_OFFLOAD_VLAN ; +if (vf->migration_active) + vf->driver_caps &=ice_migration_supported_caps (); vfres->vf_cap_flags =VIRTCHNL_VF_OFFLOAD_L2 ; vsi =ice_get_vf_vsi (vf);if (!vsi) {
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 13/15 vfio/ice: implement vfio_pci driver for E800 devices
In reply to Lingyu LiuAdd a vendor-specific vfio_pci driver for E800 devices.
It uses vfio_pci_core to register to the VFIO subsystem and then implements the E800 specific logic to support VF live migration.
It implements the device state transition flow for live migration.
--- MAINTAINERS | 7 + drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/ice/Kconfig | 10 + drivers/vfio/pci/ice/Makefile | 4 + drivers/vfio/pci/ice/ice_vfio_pci.c | 841 ++++++++++++++++++++++++++++ 6 files changed, 866 insertions(+) create mode 100644 drivers/vfio/pci/ice/Kconfig create mode 100644 drivers/vfio/pci/ice/Makefile create mode 100644 drivers/vfio/pci/ice/ice_vfio_pci.c
MAINTAINERS +7 -0
@@ -22170,6 +22170,13 @@ L: [email protected] S: Maintained F: drivers/vfio/pci/mlx5/ +VFIO ICE PCI DRIVER +M: Yahui Cao <[email protected]> +M: Lingyu Liu <[email protected]> +L: [email protected] +S: Maintained +F: drivers/vfio/pci/ice/ + VFIO PCI DEVICE SPECIFIC DRIVERS R: Jason Gunthorpe <[email protected]> R: Yishai Hadas <[email protected]>
drivers/vfio/pci/Kconfig +2 -0
@@ -59,4 +59,6 @@ source "drivers/vfio/pci/mlx5/Kconfig" source "drivers/vfio/pci/hisilicon/Kconfig" +source "drivers/vfio/pci/ice/Kconfig" + endif
drivers/vfio/pci/Makefile +2 -0
@@ -11,3 +11,5 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o obj-$(CONFIG_MLX5_VFIO_PCI) += mlx5/ obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/ + +obj-$(CONFIG_ICE_VFIO_PCI) += ice/
drivers/vfio/pci/ice/Kconfig new file +10 -0
@@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0-only +config ICE_VFIO_PCI + tristate "VFIO support for Intel(R) Ethernet Connection E800 Series" + depends on ICE + depends on VFIO_PCI_CORE + help + This provides migration support for Intel(R) Ethernet connection E800 + series devices using the VFIO framework. + + If you don't know what to do here, say N.
drivers/vfio/pci/ice/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_ICE_VFIO_PCI) += ice-vfio-pci.o +ice-vfio-pci-y := ice_vfio_pci.o +
drivers/vfio/pci/ice/ice_vfio_pci.c new file +841 -0
841 lines not on this page · raw message
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [PATCH iwl-next V2 13/15] vfio/ice: implement vfio_pci driver for E800 devices
drivers/vfio/pci/ice/ice_vfio_pci.c line 592
Quoted 12 lines · Lingyu Liu
drivers/vfio/pci/ice/ice_vfio_pci.c new file +12
-0+static struct file * +ice_vfio_pci_step_device_state_locked (struct ice_vfio_pci_core_device *ice_vdev, +u32 new,u32 final) +{ +struct device *dev = &ice_vdev->core_device.pdev->dev; +u32 cur = ice_vdev->mig_state; +int ret; + +if (final ==VFIO_DEVICE_STATE_RESUMING ) + ice_vdev->is_dst = true; +else if (final ==VFIO_DEVICE_STATE_STOP ) + ice_vdev->is_dst = false;
Definately not. The kernel should not be guessing which end is which, the protocol already makes it clear.
Quoted 36 lines
drivers/vfio/pci/ice/ice_vfio_pci.c new file +36
-0+ +if (cur ==VFIO_DEVICE_STATE_RUNNING && new ==VFIO_DEVICE_STATE_STOP ) { +if (!ice_vdev->is_dst) +dev_info (dev,"Live migration begins\n" ); +ice_migration_suspend_vf (ice_vdev->vf_handle, ice_vdev->is_dst); +return NULL ; + } + +if (cur ==VFIO_DEVICE_STATE_STOP && new ==VFIO_DEVICE_STATE_STOP_COPY ) { +struct ice_vfio_pci_migration_file *migf; + + migf =ice_vfio_pci_stop_copy (ice_vdev); +if (IS_ERR (migf)) +return ERR_CAST (migf); +get_file (migf->filp); + ice_vdev->saving_migf = migf; +return migf->filp; + } + +if (cur ==VFIO_DEVICE_STATE_STOP_COPY && new ==VFIO_DEVICE_STATE_STOP ) { +ice_vfio_pci_disable_fds (ice_vdev); +dev_info (dev,"Live migration ends\n" ); +return NULL ; + } + +if (cur ==VFIO_DEVICE_STATE_STOP && new ==VFIO_DEVICE_STATE_RESUMING ) { +struct ice_vfio_pci_migration_file *migf; + + migf =ice_vfio_pci_resume (ice_vdev); +if (IS_ERR (migf)) +return ERR_CAST (migf); +get_file (migf->filp); + ice_vdev->resuming_migf = migf; +return migf->filp; + } +
drivers/vfio/pci/ice/ice_vfio_pci.c line 638
Quoted 10 lines
drivers/vfio/pci/ice/ice_vfio_pci.c new file +10
-0+if (cur ==VFIO_DEVICE_STATE_RESUMING && new ==VFIO_DEVICE_STATE_STOP ) { + ret =ice_vfio_pci_load_state (ice_vdev); +if (ret) +return ERR_PTR (ret); +ice_vfio_pci_disable_fds (ice_vdev); +return NULL ; + } + +if (cur ==VFIO_DEVICE_STATE_STOP && new ==VFIO_DEVICE_STATE_RUNNING ) +return NULL ;
Lack of P2P is going to be a problem too
Jason
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [PATCH iwl-next V2 13/15] vfio/ice: implement vfio_pci driver for E800 devices
Quoted 16 lines · Jason Gunthorpe
drivers/vfio/pci/ice/ice_vfio_pci.c line 592
Quoted 12 lines · Lingyu Liu
drivers/vfio/pci/ice/ice_vfio_pci.c new file +12
-0+static struct file * +ice_vfio_pci_step_device_state_locked (struct ice_vfio_pci_core_device *ice_vdev, +u32 new,u32 final) +{ +struct device *dev = &ice_vdev->core_device.pdev->dev; +u32 cur = ice_vdev->mig_state; +int ret; + +if (final ==VFIO_DEVICE_STATE_RESUMING ) + ice_vdev->is_dst = true; +else if (final ==VFIO_DEVICE_STATE_STOP ) + ice_vdev->is_dst = false;Definately not. The kernel should not be guessing which end is which, the protocol already makes it clear.
Hi Jason, thanks for your review. This will be changed in next revision.
Quoted 39 lines
Quoted 39 lines
drivers/vfio/pci/ice/ice_vfio_pci.c new file +39
-0+ +if (cur ==VFIO_DEVICE_STATE_RUNNING && new ==VFIO_DEVICE_STATE_STOP ) { +if (!ice_vdev->is_dst) +dev_info (dev,"Live migration begins\n" ); +ice_migration_suspend_vf (ice_vdev->vf_handle, ice_vdev->is_dst); +return NULL ; + } + +if (cur ==VFIO_DEVICE_STATE_STOP && new ==VFIO_DEVICE_STATE_STOP_COPY ) { +struct ice_vfio_pci_migration_file *migf; + + migf =ice_vfio_pci_stop_copy (ice_vdev); +if (IS_ERR (migf)) +return ERR_CAST (migf); +get_file (migf->filp); + ice_vdev->saving_migf = migf; +return migf->filp; + } + +if (cur ==VFIO_DEVICE_STATE_STOP_COPY && new ==VFIO_DEVICE_STATE_STOP ) { +ice_vfio_pci_disable_fds (ice_vdev); +dev_info (dev,"Live migration ends\n" ); +return NULL ; + } + +if (cur ==VFIO_DEVICE_STATE_STOP && new ==VFIO_DEVICE_STATE_RESUMING ) { +struct ice_vfio_pci_migration_file *migf; + + migf =ice_vfio_pci_resume (ice_vdev); +if (IS_ERR (migf)) +return ERR_CAST (migf); +get_file (migf->filp); + ice_vdev->resuming_migf = migf; +return migf->filp; + } + +if (cur ==VFIO_DEVICE_STATE_RESUMING && new ==VFIO_DEVICE_STATE_STOP ) { + ret =ice_vfio_pci_load_state (ice_vdev); +if (ret)
Quoted 8 lines
drivers/vfio/pci/ice/ice_vfio_pci.c line 638
Quoted 7 lines
drivers/vfio/pci/ice/ice_vfio_pci.c new file +7
-0+return ERR_PTR (ret); +ice_vfio_pci_disable_fds (ice_vdev); +return NULL ; + } + +if (cur ==VFIO_DEVICE_STATE_STOP && new ==VFIO_DEVICE_STATE_RUNNING ) +return NULL ;Lack of P2P is going to be a problem too
Will add P2P support in next revision. Thanks.
Quoted 2 lines
Jason
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 14/15 vfio: Expose vfio_device_has_container()
In reply to Lingyu LiuFrom: Yahui Cao <[email protected]>
Expose vfio_device_has_container() to be used by drivers to know what is the vfio uAPI probed by user application.
Next patch for ice vfio pci driver will use it.
--- drivers/vfio/group.c | 1 + include/linux/vfio.h | 1 + 2 files changed, 2 insertions(+)
drivers/vfio/group.c +1 -0
@@ -744,6 +744,7 @@ bool vfio_device_has_container(struct vfio_device *device) {return device->group->container; } +EXPORT_SYMBOL_GPL (vfio_device_has_container);/** * vfio_file_iommu_group - Return the struct iommu_group for the vfio group file
include/linux/vfio.h +1 -0
@@ -253,6 +253,7 @@ bool vfio_file_is_group(struct file *file);bool vfio_file_enforced_coherent (struct file *file);void vfio_file_set_kvm (struct file *file,struct kvm *kvm);bool vfio_file_has_dev (struct file *file,struct vfio_device *device); +bool vfio_device_has_container (struct vfio_device *device);#define VFIO_PIN_PAGES_MAX_ENTRIES (PAGE_SIZE/sizeof(unsigned long))
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v2 15/15 vfio/ice: support iommufd vfio compat mode
In reply to Lingyu LiuFrom: Yahui Cao <[email protected]>
In iommufd vfio compat mode, vfio_dma_rw() will return failure, since vfio_device_has_container() returns false and device->iommufd_access is NULL.
Currently device->iommufd_access will not be created if vfio device is backed by pci device. To support IOVA access, manually create iommufd_access context by iommufd_access_create/attach() and access IOVA by iommufd_access_rw(). And in order to minimize the iommufd_access's impact, store the iommufd_access context in driver data, create it only before loading the device state and destroy it once finishing loading the device state.
To be compatible with legacy vfio, use vfio_device_has_container() to check the vfio uAPI. If in legacy vfio mode, call vfio_dma_rw() directly, otherwise call iommufd_access_rw().
--- .../net/ethernet/intel/ice/ice_migration.c | 23 +-- drivers/vfio/pci/ice/ice_vfio_pci.c | 171 +++++++++++++++++- include/linux/net/intel/ice_migration.h | 4 +- 3 files changed, 179 insertions(+), 19 deletions(-)
drivers/net/ethernet/intel/ice/ice_migration.c +12 -11
@@ -440,7 +440,7 @@ ice_migration_restore_rx_head(struct ice_vf *vf,static int ice_migration_restore_tx_head (struct ice_vf *vf,struct ice_migration_dev_state *devstate, -struct vfio_device *vdev) +dma_rw_handler_t handler,void *data) {struct ice_tx_desc *tx_desc_dummy, *tx_desc;struct ice_vsi *vsi =ice_get_vf_vsi (vf);
@@ -509,15 +509,15 @@ ice_migration_restore_tx_head(struct ice_vf *vf, ret = -EINVAL ; goto err; } - ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc, - tx_ring->count *sizeof (tx_desc[0 ]), false); + ret =handler (data, tx_ring->dma, (void *)tx_desc, + tx_ring->count *sizeof (tx_desc[0 ]), false);if (ret) {dev_err (dev,"kvm read guest tx ring error: %d\n" , ret); goto err; } - ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc_dummy, - tx_heads[i] *sizeof (tx_desc_dummy[0 ]), true); + ret =handler (data, tx_ring->dma, (void *)tx_desc_dummy, + tx_heads[i] *sizeof (tx_desc_dummy[0 ]), true);if (ret) {dev_err (dev,"kvm write guest return error: %d\n" , ret);
@@ -546,8 +546,8 @@ ice_migration_restore_tx_head(struct ice_vf *vf, vf->vf_id, i); goto err; } - ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc, - tx_ring->count *sizeof (tx_desc[0 ]), true); + ret =handler (data, tx_ring->dma, (void *)tx_desc, + tx_ring->count *sizeof (tx_desc[0 ]), true);if (ret) {dev_err (dev,"kvm write guest tx ring error: %d\n" , ret);
@@ -567,7 +567,8 @@ ice_migration_restore_tx_head(struct ice_vf *vf,* @opaque: pointer to VF handler in ice vdev * @buf: pointer to device state buf in migration buffer * @buf_sz: size of migration buffer -* @vdev: pointer to vfio device +* @handler: dma_rw_handler +* @data: dma_rw_handler data * * This function uses the device state saved in migration buffer * to restore device state at dst VM
@@ -575,7 +576,7 @@ ice_migration_restore_tx_head(struct ice_vf *vf,* Return 0 for success, negative for error */ int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz, -struct vfio_device *vdev) +dma_rw_handler_t handler,void *data) {struct ice_migration_virtchnl_msg_slot *msg_slot;struct ice_vf *vf = (struct ice_vf *)opaque;
@@ -587,7 +588,7 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz,u64 slot_sz;int ret =0 ; -if (!buf || !vdev) +if (!buf)return -EINVAL ; total_sz +=sizeof (struct ice_migration_dev_state );
@@ -658,7 +659,7 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz,* After virtual channel replay completes, tx rings are enabled. * Then restore tx head for tx rings by injecting dummy packets. */ - ret =ice_migration_restore_tx_head (vf, devstate, vdev); + ret =ice_migration_restore_tx_head (vf, devstate, handler, data);if (ret) {dev_err (dev,"failed to restore tx queue head\n" ); goto err;
drivers/vfio/pci/ice/ice_vfio_pci.c +164 -7
209 lines not on this page · raw message
include/linux/net/intel/ice_migration.h +3 -1
@@ -7,6 +7,8 @@#if IS_ENABLED (CONFIG_ICE_VFIO_PCI ) +typedef int (*dma_rw_handler_t )(void *data,dma_addr_t iova,void *buf, +size_t len,bool write);#define IAVF_QRX_TAIL_MAX 256#define QTX_HEAD_RESTORE_DELAY_MAX 100#define QTX_HEAD_RESTORE_DELAY_SLEEP_US_MIN 10
@@ -19,7 +21,7 @@ void ice_migration_uninit_vf(void *opaque);int ice_migration_suspend_vf (void *opaque,bool mig_dst);int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz);int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz, -struct vfio_device *vdev); +dma_rw_handler_t handler,void *data);#else static inline void *ice_migration_get_vf (struct pci_dev *vf_pdev)
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [PATCH iwl-next V2 15/15] vfio/ice: support iommufd vfio compat mode
Quoted 14 lines · Lingyu Liu
From: Yahui Cao <[email protected]>
In iommufd vfio compat mode,
vfio_dma_rw()will return failure, sincevfio_device_has_container()returns false anddevice->iommufd_accessis NULL.Currently
device->iommufd_accesswill not be created if vfio device is backed by pci device. To support IOVA access, manually createiommufd_accesscontext by iommufd_access_create/attach() and access IOVA byiommufd_access_rw(). And in order to minimize the iommufd_access's impact, store theiommufd_accesscontext in driver data, create it only before loading the device state and destroy it once finishing loading the device state.
Quoted 3 lines
To be compatible with legacy vfio, use
vfio_device_has_container()to check the vfio uAPI. If in legacy vfio mode, callvfio_dma_rw()directly, otherwise calliommufd_access_rw().
This is not the right approach, you should create your access by overloading the iommufd ops. Nak on exposing vfio_device_has_container
Quoted 10 lines
drivers/vfio/pci/ice/ice_vfio_pci.c +10
-0+/** +* ice_vfio_pci_emulated_unmap - callback to unmap IOVA +* @data: function handler data +* @iova: I/O virtuall address +* @len: IOVA length +* +* This function is called when application are doing DMA unmap and in some +* cases driver needs to explicitly do some unmap ops if this device does not +* have backed iommu. Nothing is required here since this is pci baseed vfio +* device, which has backed iommu.
drivers/vfio/pci/ice/ice_vfio_pci.c ice_vfio_pci_load_regs(struct ice_vfio_pci_core_device *ice_vdev, · line 205
Quoted 10 lines
drivers/vfio/pci/ice/ice_vfio_pci.c +10
-0+*/ +static void +ice_vfio_pci_emulated_unmap (void *data,unsigned long iova,unsigned long len) +{ +} + +static const struct iommufd_access_ops ice_vfio_user_ops = { + .needs_pin_pages =1 , + .unmap = ice_vfio_pci_emulated_unmap, +};
If you don't call pin pages then you shouldn't set needs_pin_pages?
An empty unmap op is unconditionally wrong.
Quoted 16 lines
+ *
ice_vfio_dma_rw- read/write function for device IOVA address space
+ * @data: function handler data
+ * @iova: I/O virtuall address
+ * @buf: buffer for read/write access
+ * @len: buffer length
+ * @write: true for write, false for read
+ *
+ * Read/write function for device IOVA access. Sincevfio_dma_rw()may fail
+ * at iommufd vfio compatiable mode, we need runtime check what uAPI it is
+ * using and use corresponding access method for IOVA access.
+ *
+ * Return 0 for success, negative value for failure.
+ */drivers/vfio/pci/ice/ice_vfio_pci.c +3
-0+static int ice_vfio_dma_rw (void *data,dma_addr_t iova, +void *buf,size_t len,bool write) +{
drivers/vfio/pci/ice/ice_vfio_pci.c ice_vfio_pci_load_regs(struct ice_vfio_pci_core_device *ice_vdev, · line 233
Quoted 10 lines
drivers/vfio/pci/ice/ice_vfio_pci.c +10
-0+struct ice_vfio_pci_core_device *ice_vdev = + (struct ice_vfio_pci_core_device *)data; +struct vfio_device *vdev = &ice_vdev->core_device.vdev; +unsigned int flags =0 ; + +if (vfio_device_has_container (vdev)) +return vfio_dma_rw (vdev, iova, buf, len, write); + +if (!current->mm) + flags |=IOMMUFD_ACCESS_RW_KTHREAD ;
No, you need to know your own calling context, you can't guess like this.
I suppose this is always called from an ioctl?
include/linux/net/intel/ice_migration.h void ice_migration_uninit_vf(void *opaque); · line 24
Quoted 6 lines
include/linux/net/intel/ice_migration.h +1
-1@@ -19,7 +21,7 @@ void ice_migration_uninit_vf(void *opaque);int ice_migration_suspend_vf (void *opaque,bool mig_dst);int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz);int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz, -struct vfio_device *vdev); +dma_rw_handler_t handler,void *data);
Please remove all the wild function pointers and void * opaques I see in this driver. Use proper types and get your layering right so you dont't have to fake up improper cross-layer calls like this.
Jason
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [PATCH iwl-next V2 15/15] vfio/ice: support iommufd vfio compat mode
Hi Jason,
Quoted 15 lines · Jason Gunthorpe
Quoted 14 lines · Lingyu Liu
From: Yahui Cao <[email protected]>
In iommufd vfio compat mode,
vfio_dma_rw()will return failure, sincevfio_device_has_container()returns false anddevice->iommufd_accessis NULL.Currently
device->iommufd_accesswill not be created if vfio device is backed by pci device. To support IOVA access, manually createiommufd_accesscontext by iommufd_access_create/attach() and access IOVA byiommufd_access_rw(). And in order to minimize the iommufd_access's impact, store theiommufd_accesscontext in driver data, create it only before loading the device state and destroy it once finishing loading the device state.
Quoted 5 lines
Quoted 3 lines
To be compatible with legacy vfio, use
vfio_device_has_container()to check the vfio uAPI. If in legacy vfio mode, callvfio_dma_rw()directly, otherwise calliommufd_access_rw().This is not the right approach, you should create your access by overloading the iommufd ops. Nak on exposing
vfio_device_has_container
Could you explain a little bit more about "create your access by overloading the iommufd ops." ?
Quoted 22 lines
Quoted 10 lines
drivers/vfio/pci/ice/ice_vfio_pci.c +10
-0+/** +* ice_vfio_pci_emulated_unmap - callback to unmap IOVA +* @data: function handler data +* @iova: I/O virtuall address +* @len: IOVA length +* +* This function is called when application are doing DMA unmap and in some +* cases driver needs to explicitly do some unmap ops if this device does not +* have backed iommu. Nothing is required here since this is pci baseed vfio +* device, which has backed iommu. drivers/vfio/pci/ice/ice_vfio_pci.c ice_vfio_pci_load_regs(struct ice_vfio_pci_core_device *ice_vdev, · line 205
Quoted 10 lines
drivers/vfio/pci/ice/ice_vfio_pci.c +10
-0+*/ +static void +ice_vfio_pci_emulated_unmap (void *data,unsigned long iova,unsigned long len) +{ +} + +static const struct iommufd_access_ops ice_vfio_user_ops = { + .needs_pin_pages =1 , + .unmap = ice_vfio_pci_emulated_unmap, +};If you don't call pin pages then you shouldn't set
needs_pin_pages?
Quoted 1 line
An empty unmap op is unconditionally wrong.
Will change in next version.
Quoted 29 lines
Quoted 17 lines
+ *
ice_vfio_dma_rw- read/write function for device IOVA address space
+ * @data: function handler data
+ * @iova: I/O virtuall address
+ * @buf: buffer for read/write access
+ * @len: buffer length
+ * @write: true for write, false for read
+ *
+ * Read/write function for device IOVA access. Sincevfio_dma_rw()may fail
+ * at iommufd vfio compatiable mode, we need runtime check what uAPI it is
+ * using and use corresponding access method for IOVA access.
+ *
+ * Return 0 for success, negative value for failure.
+ */drivers/vfio/pci/ice/ice_vfio_pci.c +3
-0+static int ice_vfio_dma_rw (void *data,dma_addr_t iova, +void *buf,size_t len,bool write) +{drivers/vfio/pci/ice/ice_vfio_pci.c ice_vfio_pci_load_regs(struct ice_vfio_pci_core_device *ice_vdev, · line 233
Quoted 10 lines
drivers/vfio/pci/ice/ice_vfio_pci.c +10
-0+struct ice_vfio_pci_core_device *ice_vdev = + (struct ice_vfio_pci_core_device *)data; +struct vfio_device *vdev = &ice_vdev->core_device.vdev; +unsigned int flags =0 ; + +if (vfio_device_has_container (vdev)) +return vfio_dma_rw (vdev, iova, buf, len, write); + +if (!current->mm) + flags |=IOMMUFD_ACCESS_RW_KTHREAD ;No, you need to know your own calling context, you can't guess like this.
Quoted 1 line
I suppose this is always called from an ioctl?
Yes. This is always called from ioctl. Will remove this calling context check.
Quoted 11 lines
include/linux/net/intel/ice_migration.h void ice_migration_uninit_vf(void *opaque); · line 24
Quoted 6 lines
include/linux/net/intel/ice_migration.h +1
-1@@ -19,7 +21,7 @@ void ice_migration_uninit_vf(void *opaque);int ice_migration_suspend_vf (void *opaque,bool mig_dst);int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz);int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz, -struct vfio_device *vdev); +dma_rw_handler_t handler,void *data);Please remove all the wild function pointers and void * opaques I see in this driver. Use proper types and get your layering right so you dont't have to fake up improper cross-layer calls like this.
Jason
Will refactor the API with proper types.
Thanks.
Yahui.
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan