Add E800 live migration driver
cover Add E800 live migration driver
This series adds vfio live migration support for Intel E810 VF devices based on the v2 migration protocol.
Lingyu Liu (11):
ice: check VF migration status before sending messages to VF
ice: add migration init field and helper functions
ice: save VF messages as device state
ice: save and restore device state
ice: do not notify VF link state during migration
ice: change VSI id in virtual channel message after migration
ice: save and restore RX queue head
ice: save and restore TX queue head
ice: stop device before saving device states
ice: mask VF advanced capabilities if live migration is activated vfio/ice: implement vfio_pci driver for E800 devices
Xu Ting (1):
ice: Fix missing legacy 32byte RXDID in the supported bitmap
Yahui Cao (3):
ice: add function to get rxq context
vfio: Expose vfio_device_has_container()
vfio/ice: support iommufd vfio compat mode
MAINTAINERS | 7 + drivers/net/ethernet/intel/ice/Makefile | 1 + drivers/net/ethernet/intel/ice/ice.h | 2 + drivers/net/ethernet/intel/ice/ice_common.c | 268 +++++ drivers/net/ethernet/intel/ice/ice_common.h | 5 + .../net/ethernet/intel/ice/ice_migration.c | 778 ++++++++++++++ .../intel/ice/ice_migration_private.h | 31 + drivers/net/ethernet/intel/ice/ice_vf_lib.c | 8 + drivers/net/ethernet/intel/ice/ice_vf_lib.h | 6 + drivers/net/ethernet/intel/ice/ice_virtchnl.c | 241 +++-- drivers/net/ethernet/intel/ice/ice_virtchnl.h | 15 +- .../ethernet/intel/ice/ice_virtchnl_fdir.c | 28 +- drivers/vfio/group.c | 1 + drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/ice/Kconfig | 10 + drivers/vfio/pci/ice/Makefile | 4 + drivers/vfio/pci/ice/ice_vfio_pci.c | 998 ++++++++++++++++++ include/linux/net/intel/ice_migration.h | 53 + include/linux/vfio.h | 1 + 20 files changed, 2348 insertions(+), 113 deletions(-) create mode 100644 drivers/net/ethernet/intel/ice/ice_migration.c create mode 100644 drivers/net/ethernet/intel/ice/ice_migration_private.h create mode 100644 drivers/vfio/pci/ice/Kconfig create mode 100644 drivers/vfio/pci/ice/Makefile create mode 100644 drivers/vfio/pci/ice/ice_vfio_pci.c create mode 100644 include/linux/net/intel/ice_migration.h
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 1/15 ice: Fix missing legacy 32byte RXDID in the supported bitmap
From: Xu Ting <[email protected]>
32byte legacy descriptor format is preassigned. Commit e753df8fbca5 ("ice: Add support Flex RXD") created a supported RXDIDs bitmap according to DDP package. But it missed the legacy 32byte RXDID since it is not listed in the package. This patch adds this RXDID to the bitmap.
--- drivers/net/ethernet/intel/ice/ice_virtchnl.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
drivers/net/ethernet/intel/ice/ice_virtchnl.c +7 -5
@@ -2615,12 +2615,14 @@ static int ice_vc_query_rxdid(struct ice_vf *vf) goto err; } -/* Read flexiflag registers to determine whether the -* corresponding RXDID is configured and supported or not. -* Since Legacy 16byte descriptor format is not supported, -* start from Legacy 32byte descriptor. +/* RXDIDs supported by DDP package can be read from the register +* to get the supported RXDID bitmap. But the legacy 32byte RXDID +* is not listed in DDP package, add it in the bitmap manually. +* Legacy 16byte descriptor is not supported. */ -for (i =ICE_RXDID_LEGACY_1 ; i <ICE_FLEX_DESC_RXDID_MAX_NUM ; i++) { + rxdid->supported_rxdids |=BIT (ICE_RXDID_LEGACY_1 ); + +for (i =ICE_RXDID_FLEX_NIC ; i <ICE_FLEX_DESC_RXDID_MAX_NUM ; i++) { regval =rd32 (hw,GLFLXP_RXDID_FLAGS (i,0 ));if ((regval >>GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_S ) &GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_M )
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Przemek Kitszel
maintainerRe: [Intel-wired-lan] [PATCH iwl-next V1 01/15] ice: Fix missing legacy 32byte RXDID in the supported bitmap
Quoted 7 lines · Lingyu Liu
From: Xu Ting <[email protected]>
32byte legacy descriptor format is preassigned. Commit
e753df8fbca5("ice: Add support Flex RXD") created a supported RXDIDs bitmap according to DDP package. But it missed the legacy 32byte RXDID since it is not listed in the package. This patch adds this RXDID to the bitmap.
Please reword into imperative mood, perhaps staring with "Mark 32byte legacy descriptor format as supported int the supported RXDIDs flags", or similar.
Quoted 12 lines
Signed-off-by Xu Ting <[email protected]>Signed-off-by Lingyu Liu <[email protected]>--- drivers/net/ethernet/intel/ice/ice_virtchnl.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)drivers/net/ethernet/intel/ice/ice_virtchnl.c +0
-0
drivers/net/ethernet/intel/ice/ice_virtchnl.c static int ice_vc_query_rxdid(struct ice_vf *vf) · line 2619
Quoted 5 lines
drivers/net/ethernet/intel/ice/ice_virtchnl.c +0
-2@@ -2615,12 +2615,14 @@ static int ice_vc_query_rxdid(struct ice_vf *vf) goto err; } -/* Read flexiflag registers to determine whether the -* corresponding RXDID is configured and supported or not.
This part of the comment was valuable too (esp given all the bit twiddling in the loop). Perhaps you could bring it back?
drivers/net/ethernet/intel/ice/ice_virtchnl.c static int ice_vc_query_rxdid(struct ice_vf *vf) · line 2620
Quoted 5 lines
drivers/net/ethernet/intel/ice/ice_virtchnl.c +3
-2-* Since Legacy 16byte descriptor format is not supported, -* start from Legacy 32byte descriptor. +/* RXDIDs supported by DDP package can be read from the register +* to get the supported RXDID bitmap. But the legacy 32byte RXDID +* is not listed in DDP package, add it in the bitmap manually.
I had to grep defines of ICE_RXDID_FLEX_NIC and ICE_RXDID_LEGACY_1 to ensure that the code is correct (it is), comment above has certainly helped me. Perhaps you could add "(and skip check for it in the loop)" at the end of the above sentence (after "manually")?
drivers/net/ethernet/intel/ice/ice_virtchnl.c static int ice_vc_query_rxdid(struct ice_vf *vf) · line 2628
Quoted 9 lines
drivers/net/ethernet/intel/ice/ice_virtchnl.c +4
-1+* Legacy 16byte descriptor is not supported. */ -for (i =ICE_RXDID_LEGACY_1 ; i <ICE_FLEX_DESC_RXDID_MAX_NUM ; i++) { + rxdid->supported_rxdids |=BIT (ICE_RXDID_LEGACY_1 ); + +for (i =ICE_RXDID_FLEX_NIC ; i <ICE_FLEX_DESC_RXDID_MAX_NUM ; i++) { regval =rd32 (hw,GLFLXP_RXDID_FLAGS (i,0 ));if ((regval >>GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_S ) &GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_M )
Codewise it's ok!
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [PATCH iwl-next V1 01/15] ice: Fix missing legacy 32byte RXDID in the supported bitmap
Quoted 11 lines · Przemek Kitszel
Quoted 7 lines · Lingyu Liu
From: Xu Ting <[email protected]>
32byte legacy descriptor format is preassigned. Commit
e753df8fbca5("ice: Add support Flex RXD") created a supported RXDIDs bitmap according to DDP package. But it missed the legacy 32byte RXDID since it is not listed in the package. This patch adds this RXDID to the bitmap.Please reword into imperative mood, perhaps staring with "Mark 32byte legacy descriptor format as supported int the supported RXDIDs flags", or similar.
Hi Przemek, thanks for your valuable reviews. Update in V2.
Quoted 7 lines
Quoted 7 lines
Signed-off-by Xu Ting <[email protected]>Signed-off-by Lingyu Liu <[email protected]>---
drivers/net/ethernet/intel/ice/ice_virtchnl.c| 12 +++++++-----1 file changed, 7 insertions(+), 5 deletions(-)
Quoted 12 lines
Quoted 9 lines
drivers/net/ethernet/intel/ice/ice_virtchnl.c +0
-0@@ -2615,12 +2615,14 @@ static int ice_vc_query_rxdid(struct ice_vf *vf) goto err; }- /* Read flexiflag registers to determine whether the - * corresponding RXDID is configured and supported or not.
This part of the comment was valuable too (esp given all the bit twiddling in the loop). Perhaps you could bring it back?
Brought back in V2. Thanks.
Quoted 8 lines
Quoted 5 lines
- * Since Legacy 16byte descriptor format is not supported, - * start from Legacy 32byte descriptor.
+ /* RXDIDs supported by DDP package can be read from the register + * to get the supported RXDID bitmap. But the legacy 32byte RXDID + * is not listed in DDP package, add it in the bitmap manually.I had to grep defines of
ICE_RXDID_FLEX_NICandICE_RXDID_LEGACY_1to ensure that the code is correct (it is), comment above has certainly helped me. Perhaps you could add "(and skip check for it in the loop)" at the end of the above sentence (after "manually")?
Thanks for the suggestion. Added in V2.
Quoted 12 lines
Quoted 11 lines
+ * Legacy 16byte descriptor is not supported.
*/
- for (i =ICE_RXDID_LEGACY_1; i <ICE_FLEX_DESC_RXDID_MAX_NUM; i++) {
+rxdid->supported_rxdids|=BIT(ICE_RXDID_LEGACY_1); +
+ for (i =ICE_RXDID_FLEX_NIC; i <ICE_FLEX_DESC_RXDID_MAX_NUM; i++) {
regval =rd32(hw, GLFLXP_RXDID_FLAGS(i, 0)); if ((regval >>GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_S) &GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_M)
Quoted 1 line
Codewise it's ok!
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 2/15 ice: add function to get rxq context
In reply to Lingyu LiuFrom: Yahui Cao <[email protected]>
Export rxq context. Read binaries from hw registers. Then use offsets and masks to extract different context fields.
ice_vfio_pci driver that is introduced in coming patches of this series will use the exported rxq context when saving and restoring device state.
--- drivers/net/ethernet/intel/ice/ice_common.c | 268 ++++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_common.h | 5 + 2 files changed, 273 insertions(+)
drivers/net/ethernet/intel/ice/ice_common.c +268 -0
@@ -1428,6 +1428,34 @@ ice_copy_rxq_ctx_to_hw(struct ice_hw *hw, u8 *ice_rxq_ctx, u32 rxq_index)return 0 ; } +/** +* ice_copy_rxq_ctx_from_hw - Copy rxq context register from HW +* @hw: pointer to the hardware structure +* @ice_rxq_ctx: pointer to the rxq context +* @rxq_index: the index of the Rx queue +* +* Copies rxq context from HW register space to dense structure +*/ +static int +ice_copy_rxq_ctx_from_hw (struct ice_hw *hw,u8 *ice_rxq_ctx,u32 rxq_index) +{ +u8 i; + +if (!ice_rxq_ctx || rxq_index >QRX_CTRL_MAX_INDEX ) +return -EINVAL ; + +/* Copy each dword separately from HW */ +for (i =0 ; i <ICE_RXQ_CTX_SIZE_DWORDS ; i++) { +u32 *ctx = (u32 *)(ice_rxq_ctx + (i *sizeof (u32))); + + *ctx =rd32 (hw,QRX_CONTEXT (i, rxq_index)); + +ice_debug (hw,ICE_DBG_QCTX ,"qrxdata[%d]: %08X\n" , i, *ctx); + } + +return 0 ; +} +/* LAN Rx Queue Context */ static const struct ice_ctx_ele ice_rlan_ctx_info[] = {/* Field Width LSB */
@@ -1479,6 +1507,32 @@ ice_write_rxq_ctx(struct ice_hw *hw, struct ice_rlan_ctx *rlan_ctx,return ice_copy_rxq_ctx_to_hw (hw, ctx_buf, rxq_index); } +/** +* ice_read_rxq_ctx - Read rxq context from HW +* @hw: pointer to the hardware structure +* @rlan_ctx: pointer to the rxq context +* @rxq_index: the index of the Rx queue +* +* Read rxq context from HW register space and then converts it from dense +* structure to sparse +*/ +int +ice_read_rxq_ctx (struct ice_hw *hw,struct ice_rlan_ctx *rlan_ctx, +u32 rxq_index) +{ +u8 ctx_buf[ICE_RXQ_CTX_SZ ] = {0 }; +int status; + +if (!rlan_ctx) +return -EINVAL ; + + status =ice_copy_rxq_ctx_from_hw (hw, ctx_buf, rxq_index); +if (status) +return status; + +return ice_get_ctx (ctx_buf, (u8 *)rlan_ctx, ice_rlan_ctx_info); +} +/* LAN Tx Queue Context */ const struct ice_ctx_ele ice_tlan_ctx_info[] = {/* Field Width LSB */
@@ -4469,6 +4523,220 @@ ice_set_ctx(struct ice_hw *hw, u8 *src_ctx, u8 *dest_ctx,return 0 ; } +/** +* ice_read_byte - read context byte into struct +* @src_ctx: the context structure to read from +* @dest_ctx: the context to be written to +* @ce_info: a description of the struct to be filled +*/ +static void +ice_read_byte (u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info) +{ +u8 dest_byte, mask; +u8 *src, *target; +u16 shift_width; + +/* prepare the bits and mask */ + shift_width = ce_info->lsb %8 ; + mask = (u8)(BIT (ce_info->width) -1 ); + +/* shift to correct alignment */ + mask <<= shift_width; + +/* get the current bits from the src bit string */ + src = src_ctx + (ce_info->lsb /8 ); + +memcpy (&dest_byte, src,sizeof (dest_byte)); + + dest_byte &= mask; + + dest_byte >>= shift_width; + +/* get the address from the struct field */ + target = dest_ctx + ce_info->offset; + +/* put it back in the struct */ +memcpy (target, &dest_byte,sizeof (dest_byte)); +} + +/** +* ice_read_word - read context word into struct +* @src_ctx: the context structure to read from +* @dest_ctx: the context to be written to +* @ce_info: a description of the struct to be filled +*/ +static void +ice_read_word (u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info) +{ +u16 dest_word, mask; +u8 *src, *target; +__le16 src_word; +u16 shift_width; + +/* prepare the bits and mask */ + shift_width = ce_info->lsb %8 ; + mask =BIT (ce_info->width) -1 ; + +/* shift to correct alignment */ + mask <<= shift_width; + +/* get the current bits from the src bit string */ + src = src_ctx + (ce_info->lsb /8 ); + +memcpy (&src_word, src,sizeof (src_word)); + +/* the data in the memory is stored as little endian so mask it +* correctly +*/ + src_word &=cpu_to_le16 (mask); + +/* get the data back into host order before shifting */ + dest_word =le16_to_cpu (src_word); + + dest_word >>= shift_width; + +/* get the address from the struct field */ + target = dest_ctx + ce_info->offset; + +/* put it back in the struct */ +memcpy (target, &dest_word,sizeof (dest_word)); +} + +/** +* ice_read_dword - read context dword into struct +* @src_ctx: the context structure to read from +* @dest_ctx: the context to be written to +* @ce_info: a description of the struct to be filled +*/ +static void +ice_read_dword (u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info) +{ +u32 dest_dword, mask; +__le32 src_dword; +u8 *src, *target; +u16 shift_width; + +/* prepare the bits and mask */ + shift_width = ce_info->lsb %8 ; + +/* if the field width is exactly 32 on an x86 machine, then the shift +* operation will not work because the SHL instructions count is masked +* to 5 bits so the shift will do nothing +*/ +if (ce_info->width <32 ) + mask =BIT (ce_info->width) -1 ; +else + mask = (u32 )~0 ; + +/* shift to correct alignment */ + mask <<= shift_width; + +/* get the current bits from the src bit string */ + src = src_ctx + (ce_info->lsb /8 ); + +memcpy (&src_dword, src,sizeof (src_dword)); + +/* the data in the memory is stored as little endian so mask it +* correctly +*/ + src_dword &=cpu_to_le32 (mask); + +/* get the data back into host order before shifting */ + dest_dword =le32_to_cpu (src_dword); + + dest_dword >>= shift_width; + +/* get the address from the struct field */ + target = dest_ctx + ce_info->offset; + +/* put it back in the struct */ +memcpy (target, &dest_dword,sizeof (dest_dword)); +} + +/** +* ice_read_qword - read context qword into struct +* @src_ctx: the context structure to read from +* @dest_ctx: the context to be written to +* @ce_info: a description of the struct to be filled +*/ +static void +ice_read_qword (u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info) +{ +u64 dest_qword, mask; +__le64 src_qword; +u8 *src, *target; +u16 shift_width; + +/* prepare the bits and mask */ + shift_width = ce_info->lsb %8 ; + +/* if the field width is exactly 64 on an x86 machine, then the shift +* operation will not work because the SHL instructions count is masked +* to 6 bits so the shift will do nothing +*/ +if (ce_info->width <64 ) + mask =BIT_ULL (ce_info->width) -1 ; +else + mask = (u64 )~0 ; + +/* shift to correct alignment */ + mask <<= shift_width; + +/* get the current bits from the src bit string */ + src = src_ctx + (ce_info->lsb /8 ); + +memcpy (&src_qword, src,sizeof (src_qword)); + +/* the data in the memory is stored as little endian so mask it +* correctly +*/ + src_qword &=cpu_to_le64 (mask); + +/* get the data back into host order before shifting */ + dest_qword =le64_to_cpu (src_qword); + + dest_qword >>= shift_width; + +/* get the address from the struct field */ + target = dest_ctx + ce_info->offset; + +/* put it back in the struct */ +memcpy (target, &dest_qword,sizeof (dest_qword)); +} + +/** +* ice_get_ctx - extract context bits from a packed structure +* @src_ctx: pointer to a generic packed context structure +* @dest_ctx: pointer to a generic non-packed context structure +* @ce_info: a description of the structure to be read from +*/ +int +ice_get_ctx (u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info) +{ +int i; + +for (i =0 ; ce_info[i].width; i++) { +switch (ce_info[i].size_of) { +case 1 : +ice_read_byte (src_ctx, dest_ctx, &ce_info[i]); +break ; +case 2 : +ice_read_word (src_ctx, dest_ctx, &ce_info[i]); +break ; +case 4 : +ice_read_dword (src_ctx, dest_ctx, &ce_info[i]); +break ; +case 8 : +ice_read_qword (src_ctx, dest_ctx, &ce_info[i]); +break ; +default : +return -EINVAL ; + } + } + +return 0 ; +} +/** * ice_get_lan_q_ctx - get the LAN queue context for the given VSI and TC * @hw: pointer to the HW struct
drivers/net/ethernet/intel/ice/ice_common.h +5 -0
@@ -56,6 +56,9 @@ void ice_set_safe_mode_caps(struct ice_hw *hw);int ice_write_rxq_ctx (struct ice_hw *hw,struct ice_rlan_ctx *rlan_ctx,u32 rxq_index); +int +ice_read_rxq_ctx (struct ice_hw *hw,struct ice_rlan_ctx *rlan_ctx, +u32 rxq_index);int ice_aq_get_rss_lut (struct ice_hw *hw,struct ice_aq_get_set_rss_lut_params *get_params);
@@ -75,6 +78,8 @@ extern const struct ice_ctx_ele ice_tlan_ctx_info[];int ice_set_ctx (struct ice_hw *hw,u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info); +int +ice_get_ctx (u8 *src_ctx,u8 *dest_ctx,const struct ice_ctx_ele *ce_info);extern struct mutex ice_global_cfg_lock_sw;
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 3/15 ice: check VF migration status before sending messages to VF
In reply to Lingyu LiuAdd a new VF state ICE_VF_STATE_REPLAYING_VC. During live migration, virtual channel commands will be replayed to restore VF device state. Don't send messages back to VF when replaying by checking VF state before PF sends messages to VF.
ice_vc_send_msg_to_vf() implies sending messages but actually it does not while replaying messages. Renaming ice_vc_send_msg_to_vf() to ice_vc_respond_to_vf() to indicate its behavior correctly.
--- drivers/net/ethernet/intel/ice/ice_vf_lib.h | 1 + drivers/net/ethernet/intel/ice/ice_virtchnl.c | 182 ++++++++++-------- drivers/net/ethernet/intel/ice/ice_virtchnl.h | 8 +- .../ethernet/intel/ice/ice_virtchnl_fdir.c | 28 +-- 4 files changed, 120 insertions(+), 99 deletions(-)
drivers/net/ethernet/intel/ice/ice_vf_lib.h +1 -0
@@ -37,6 +37,7 @@ enum ice_vf_states {ICE_VF_STATE_DIS ,ICE_VF_STATE_MC_PROMISC ,ICE_VF_STATE_UC_PROMISC , +ICE_VF_STATE_REPLAYING_VC , ICE_VF_STATES_NBITS };
drivers/net/ethernet/intel/ice/ice_virtchnl.c +101 -81
393 lines not on this page · raw message
drivers/net/ethernet/intel/ice/ice_virtchnl.h +4 -4
@@ -60,8 +60,8 @@ void ice_vc_notify_vf_link_state(struct ice_vf *vf);void ice_vc_notify_link_state (struct ice_pf *pf);void ice_vc_notify_reset (struct ice_pf *pf);int -ice_vc_send_msg_to_vf (struct ice_vf *vf,u32 v_opcode, -enum virtchnl_status_code v_retval,u8 *msg,u16 msglen); +ice_vc_respond_to_vf (struct ice_vf *vf,u32 v_opcode, +enum virtchnl_status_code v_retval,u8 *msg,u16 msglen);bool ice_vc_isvalid_vsi_id (struct ice_vf *vf,u16 vsi_id);void ice_vc_process_vf_msg (struct ice_pf *pf,struct ice_rq_event_info *event,struct ice_mbx_data *mbxdata);
@@ -73,8 +73,8 @@ static inline void ice_vc_notify_link_state(struct ice_pf *pf) { }static inline void ice_vc_notify_reset (struct ice_pf *pf) { }static inline int -ice_vc_send_msg_to_vf (struct ice_vf *vf,u32 v_opcode, -enum virtchnl_status_code v_retval,u8 *msg,u16 msglen) +ice_vc_respond_to_vf (struct ice_vf *vf,u32 v_opcode, +enum virtchnl_status_code v_retval,u8 *msg,u16 msglen) {return -EOPNOTSUPP ; }
drivers/net/ethernet/intel/ice/ice_virtchnl_fdir.c +14 -14
@@ -1571,8 +1571,8 @@ ice_vc_add_fdir_fltr_post(struct ice_vf *vf, struct ice_vf_fdir_ctx *ctx, resp->flow_id = conf->flow_id; vf->fdir.fdir_fltr_cnt[conf->input.flow_type][is_tun]++; - ret =ice_vc_send_msg_to_vf (vf, ctx->v_opcode, v_ret, - (u8 *)resp, len); + ret =ice_vc_respond_to_vf (vf, ctx->v_opcode, v_ret, + (u8 *)resp, len);kfree (resp); dev_dbg(dev, "VF %d: flow_id:0x%X, FDIR %s success!\n",
@@ -1587,8 +1587,8 @@ ice_vc_add_fdir_fltr_post(struct ice_vf *vf, struct ice_vf_fdir_ctx *ctx,ice_vc_fdir_remove_entry (vf, conf, conf->flow_id);devm_kfree (dev, conf); - ret =ice_vc_send_msg_to_vf (vf, ctx->v_opcode, v_ret, - (u8 *)resp, len); + ret =ice_vc_respond_to_vf (vf, ctx->v_opcode, v_ret, + (u8 *)resp, len);kfree (resp);return ret; }
@@ -1635,8 +1635,8 @@ ice_vc_del_fdir_fltr_post(struct ice_vf *vf, struct ice_vf_fdir_ctx *ctx,ice_vc_fdir_remove_entry (vf, conf, conf->flow_id); vf->fdir.fdir_fltr_cnt[conf->input.flow_type][is_tun]--; - ret =ice_vc_send_msg_to_vf (vf, ctx->v_opcode, v_ret, - (u8 *)resp, len); + ret =ice_vc_respond_to_vf (vf, ctx->v_opcode, v_ret, + (u8 *)resp, len);kfree (resp); dev_dbg(dev, "VF %d: flow_id:0x%X, FDIR %s success!\n",
@@ -1652,8 +1652,8 @@ ice_vc_del_fdir_fltr_post(struct ice_vf *vf, struct ice_vf_fdir_ctx *ctx,if (success)devm_kfree (dev, conf); - ret =ice_vc_send_msg_to_vf (vf, ctx->v_opcode, v_ret, - (u8 *)resp, len); + ret =ice_vc_respond_to_vf (vf, ctx->v_opcode, v_ret, + (u8 *)resp, len);kfree (resp);return ret; }
@@ -1850,8 +1850,8 @@ int ice_vc_add_fdir_fltr(struct ice_vf *vf, u8 *msg) v_ret =VIRTCHNL_STATUS_SUCCESS ; stat->status =VIRTCHNL_FDIR_SUCCESS ;devm_kfree (dev, conf); - ret =ice_vc_send_msg_to_vf (vf,VIRTCHNL_OP_ADD_FDIR_FILTER , - v_ret, (u8 *)stat, len); + ret =ice_vc_respond_to_vf (vf,VIRTCHNL_OP_ADD_FDIR_FILTER , + v_ret, (u8 *)stat, len); goto exit; }
@@ -1909,8 +1909,8 @@ int ice_vc_add_fdir_fltr(struct ice_vf *vf, u8 *msg) err_free_conf:devm_kfree (dev, conf); err_exit: - ret =ice_vc_send_msg_to_vf (vf,VIRTCHNL_OP_ADD_FDIR_FILTER , v_ret, - (u8 *)stat, len); + ret =ice_vc_respond_to_vf (vf,VIRTCHNL_OP_ADD_FDIR_FILTER , v_ret, + (u8 *)stat, len);kfree (stat);return ret; }
@@ -1993,8 +1993,8 @@ int ice_vc_del_fdir_fltr(struct ice_vf *vf, u8 *msg) err_del_tmr:ice_vc_fdir_clear_irq_ctx (vf); err_exit: - ret =ice_vc_send_msg_to_vf (vf,VIRTCHNL_OP_DEL_FDIR_FILTER , v_ret, - (u8 *)stat, len); + ret =ice_vc_respond_to_vf (vf,VIRTCHNL_OP_DEL_FDIR_FILTER , v_ret, + (u8 *)stat, len);kfree (stat);return ret; }
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 4/15 ice: add migration init field and helper functions
In reply to Lingyu LiuAdds a function to get ice VF device from pci device. Adds a field in VF structure to indicate migration init state, and functions to init and uninit migration.
This will be used by ice_vfio_pci driver introduced in coming patches from this series.
--- drivers/net/ethernet/intel/ice/Makefile | 1 + drivers/net/ethernet/intel/ice/ice.h | 1 + .../net/ethernet/intel/ice/ice_migration.c | 68 +++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_vf_lib.c | 7 ++ drivers/net/ethernet/intel/ice/ice_vf_lib.h | 1 + include/linux/net/intel/ice_migration.h | 26 +++++++ 6 files changed, 104 insertions(+) create mode 100644 drivers/net/ethernet/intel/ice/ice_migration.c create mode 100644 include/linux/net/intel/ice_migration.h
drivers/net/ethernet/intel/ice/Makefile +1 -0
@@ -49,3 +49,4 @@ ice-$(CONFIG_RFS_ACCEL) += ice_arfs.o ice-$(CONFIG_XDP_SOCKETS) += ice_xsk.o ice-$(CONFIG_ICE_SWITCHDEV) += ice_eswitch.o ice_eswitch_br.o ice-$(CONFIG_GNSS) += ice_gnss.o +ice-$(CONFIG_ICE_VFIO_PCI) += ice_migration.o
drivers/net/ethernet/intel/ice/ice.h +1 -0
@@ -55,6 +55,7 @@#include <net/vxlan.h> #include <net/gtp.h> #include <linux/ppp_defs.h> +#include <linux/net/intel/ice_migration.h> #include "ice_devids.h" #include "ice_type.h" #include "ice_txrx.h"
drivers/net/ethernet/intel/ice/ice_migration.c new file +68 -0
68 lines not on this page · raw message
drivers/net/ethernet/intel/ice/ice_vf_lib.c +7 -0
@@ -56,6 +56,9 @@ static void ice_release_vf(struct kref *ref) { struct ice_vf *vf = container_of(ref, struct ice_vf, refcnt); +if (vf->migration_active) +ice_migration_uninit_vf (vf); + vf->vf_ops->free (vf); }
@@ -260,6 +263,10 @@ static void ice_vf_pre_vsi_rebuild(struct ice_vf *vf)if (vf->vf_ops->irq_close) vf->vf_ops->irq_close (vf); +if (vf->migration_active) { +ice_migration_uninit_vf (vf); +ice_migration_init_vf (vf); + }ice_vf_clear_counters (vf); vf->vf_ops->clear_reset_trigger (vf); }
drivers/net/ethernet/intel/ice/ice_vf_lib.h +1 -0
@@ -134,6 +134,7 @@ struct ice_vf {/* devlink port data */ struct devlink_port devlink_port; +bool migration_active; };/* Flags for controlling behavior of ice_reset_vf */
include/linux/net/intel/ice_migration.h new file +26 -0
@@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright (C) 2018-2023 Intel Corporation */ + +#ifndef _ICE_MIGRATION_H_ +#define _ICE_MIGRATION_H_ + +#if IS_ENABLED (CONFIG_ICE_VFIO_PCI ) +void *ice_migration_get_vf (struct pci_dev *vf_pdev); +void ice_migration_put_vf (void *opaque); +void ice_migration_init_vf (void *opaque); +void ice_migration_uninit_vf (void *opaque); + +#else +static inline void *ice_migration_get_vf (struct pci_dev *vf_pdev) +{ +return NULL ; +} +static inline void ice_migration_put_vf (void *opaque) +{ +return NULL ; +} +static inline void ice_migration_init_vf (void *opaque) { } +static inline void ice_migration_uninit_vf (void *opaque) { } +#endif /* CONFIG_ICE_VFIO_PCI */ + +#endif /* _ICE_MIGRATION_H_ */
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 5/15 ice: save VF messages as device state
In reply to Lingyu LiuAdd a list in VF structure to save the virtual channel messages sent by VF.
ice_vfio_pci driver introduced in following patches from this series needs the saved virtual channel list.
--- drivers/net/ethernet/intel/ice/ice.h | 1 + .../net/ethernet/intel/ice/ice_migration.c | 94 +++++++++++++++++++ .../intel/ice/ice_migration_private.h | 22 +++++ drivers/net/ethernet/intel/ice/ice_vf_lib.h | 3 + drivers/net/ethernet/intel/ice/ice_virtchnl.c | 3 + 5 files changed, 123 insertions(+) create mode 100644 drivers/net/ethernet/intel/ice/ice_migration_private.h
drivers/net/ethernet/intel/ice/ice.h +1 -0
@@ -77,6 +77,7 @@#include "ice_vsi_vlan_ops.h" #include "ice_gnss.h" #include "ice_irq.h" +#include "ice_migration_private.h" #define ICE_BAR0 0#define ICE_REQ_DESC_MULTIPLE 32
drivers/net/ethernet/intel/ice/ice_migration.c +94 -0
117 lines not on this page · raw message
drivers/net/ethernet/intel/ice/ice_migration_private.h new file +22 -0
@@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright (C) 2018-2023 Intel Corporation */ + +#ifndef _ICE_MIGRATION_PRIVATE_H_ +#define _ICE_MIGRATION_PRIVATE_H_ + +/* This header file is for exposing functions in ice_migration.c to +* files which will be compiled in ice.ko. +* Functions which may be used by other files which will be compiled +* in ice-vfio-pic.ko should be exposed as part of ice_migration.h. +*/ + +#if IS_ENABLED (CONFIG_ICE_VFIO_PCI ) +void ice_migration_save_vf_msg (struct ice_vf *vf, +struct ice_rq_event_info *event); +#else +static inline void +ice_migration_save_vf_msg (struct ice_vf *vf, +struct ice_rq_event_info *event) { } +#endif /* CONFIG_ICE_VFIO_PCI */ + +#endif /* _ICE_MIGRATION_PRIVATE_H_ */
drivers/net/ethernet/intel/ice/ice_vf_lib.h +3 -0
@@ -77,6 +77,7 @@ struct ice_vfs {unsigned long last_printed_mdd_jiffies;/* MDD message rate limit */ }; +#define VIRTCHNL_MSG_MAX 1000/* VF information structure */ struct ice_vf {struct hlist_node entry;
@@ -135,6 +136,8 @@ struct ice_vf {/* devlink port data */ struct devlink_port devlink_port;bool migration_active; +struct list_head virtchnl_msg_list; +u64 virtchnl_msg_num; };/* Flags for controlling behavior of ice_reset_vf */
drivers/net/ethernet/intel/ice/ice_virtchnl.c +3 -0
@@ -4077,6 +4077,9 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event, vf_id, v_opcode, err); } +if (!err && vf->migration_active) +ice_migration_save_vf_msg (vf, event); + finish:mutex_unlock (&vf->cfg_lock);ice_put_vf (vf);
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 6/15 ice: save and restore device state
In reply to Lingyu LiuAdd function to migrate device state.
ice_vfio_pci driver introduced in following patches from this series needs the exported function to save and restore device state.
--- .../net/ethernet/intel/ice/ice_migration.c | 125 ++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_virtchnl.c | 26 +++- drivers/net/ethernet/intel/ice/ice_virtchnl.h | 7 +- include/linux/net/intel/ice_migration.h | 12 ++ 4 files changed, 161 insertions(+), 9 deletions(-)
drivers/net/ethernet/intel/ice/ice_migration.c +125 -0
@@ -160,3 +160,128 @@ void ice_migration_save_vf_msg(struct ice_vf *vf,break ; } } + +/** +* ice_migration_save_devstate - save VF msg to migration buffer +* @opaque: pointer to VF handler in ice vdev +* @buf: pointer to VF msg in migration buffer +* @buf_sz: size of migration buffer +* +* Return 0 for success, negative for error +*/ +int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz) +{ +struct ice_migration_virtchnl_msg_listnode *msg_listnode; +struct ice_migration_virtchnl_msg_slot *last_op; +struct ice_vf *vf = (struct ice_vf *)opaque; +struct device *dev =ice_pf_to_dev (vf->pf); +u64 total_sz =0 ; + +if (vf ==NULL ) +return -EINVAL ; + +list_for_each_entry (msg_listnode, &vf->virtchnl_msg_list, node) { +struct ice_migration_virtchnl_msg_slot *msg_slot; +u64 slot_size; + + msg_slot = &msg_listnode->msg_slot; + slot_size =struct_size (msg_slot, msg_buffer, + msg_slot->msg_len); + total_sz += slot_size; +if (total_sz > buf_sz) { +dev_err (dev,"Insufficient buffer to store virtchnl message for VF %d op: %d, len: %d\n" , + vf->vf_id, msg_slot->opcode, msg_slot->msg_len); +return -ENOBUFS ; + } +dev_dbg (dev,"VF %d copy virtchnl message to migration buffer op: %d, len: %d\n" , + vf->vf_id, msg_slot->opcode, msg_slot->msg_len); +memcpy (buf, msg_slot, slot_size); + buf += slot_size; + } +/* reserve space to mark end of vf messages */ + total_sz +=sizeof (struct ice_migration_virtchnl_msg_slot ); +if (total_sz > buf_sz) { +dev_err (dev,"Insufficient buffer to store virtchnl message for VF %d\n" , + vf->vf_id); +return -ENOBUFS ; + } + +/* use op code unknown to mark end of vc messages */ + last_op = (struct ice_migration_virtchnl_msg_slot *)buf; + last_op->opcode =VIRTCHNL_OP_UNKNOWN ; +return 0 ; +} +EXPORT_SYMBOL (ice_migration_save_devstate); + +/** +* ice_migration_restore_devstate - restore device state at dst +* @opaque: pointer to VF handler in ice vdev +* @buf: pointer to device state buf in migration buffer +* @buf_sz: size of migration buffer +* +* This function uses the device state saved in migration buffer +* to restore device state at dst VM +* +* Return 0 for success, negative for error +*/ +int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz) +{ +struct ice_migration_virtchnl_msg_slot *msg_slot; +struct ice_vf *vf = (struct ice_vf *)opaque; +struct device *dev =ice_pf_to_dev (vf->pf); +struct ice_rq_event_info event; +u64 total_sz =0 ; +u64 op_msglen_sz; +u64 slot_sz; +int ret =0 ; + +if (!buf) +return -EINVAL ; + + msg_slot = (struct ice_migration_virtchnl_msg_slot *)buf; + op_msglen_sz =sizeof (struct ice_migration_virtchnl_msg_slot ); +/* check whether enough space for opcode and msg_len */ +if (total_sz + op_msglen_sz > buf_sz) { +dev_err (dev,"VF %d msg size exceeds buffer size\n" , vf->vf_id); +return -ENOBUFS ; + } + +set_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states); + +while (msg_slot->opcode !=VIRTCHNL_OP_UNKNOWN ) { + slot_sz =struct_size (msg_slot, msg_buffer, msg_slot->msg_len); + total_sz += slot_sz; +/* check whether enough space for whole message */ +if (total_sz > buf_sz) { +dev_err (dev,"VF %d msg size exceeds buffer size\n" , + vf->vf_id); + ret = -ENOBUFS ; +break ; + } +dev_dbg (dev,"VF %d replay virtchnl message op code: %d, msg len: %d\n" , + vf->vf_id, msg_slot->opcode, msg_slot->msg_len); + event.desc.cookie_high = msg_slot->opcode; + event.msg_len = msg_slot->msg_len; + event.desc.retval = vf->vf_id; + event.msg_buf = (unsigned char *)msg_slot->msg_buffer; + ret =ice_vc_process_vf_msg (vf->pf, &event,NULL ); +if (ret) { +dev_err (dev,"failed to replay virtchnl message op code: %d\n" , + msg_slot->opcode); +break ; + } + event.msg_buf =NULL ; + msg_slot = (struct ice_migration_virtchnl_msg_slot *) + ((char *)msg_slot + slot_sz); +/* check whether enough space for opcode and msg_len */ +if (total_sz + op_msglen_sz > buf_sz) { +dev_err (dev,"VF %d msg size exceeds buffer size\n" , + vf->vf_id); + ret = -ENOBUFS ; +break ; + } + } +clear_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states); +return ret; +} +EXPORT_SYMBOL (ice_migration_restore_devstate);
drivers/net/ethernet/intel/ice/ice_virtchnl.c +20 -6
@@ -3894,11 +3894,24 @@ ice_is_malicious_vf(struct ice_vf *vf, struct ice_mbx_data *mbxdata)* @event: pointer to the AQ event * @mbxdata: information used to detect VF attempting mailbox overflow * -* called from the common asq/arq handler to -* process request from VF +* This function will be called from: +* 1. the common asq/arq handler to process request from VF +* +* The return value is ignored, as the command will send the status of the +* request as a response to the VF. This flow sets the mbxdata to +* a non-NULL value and must call ice_is_malicious_vf to determine if this +* VF might be attempting to overflow the PF message queue. +* +* 2. replay virtual channel commamds during live migration +* +* The return value is used to indicate failure to replay vc commands and +* that the migration failed. This flow sets mbxdata to NULL and skips the +* ice_is_malicious_vf checks which are unnecessary during replay. +* +* Return 0 if success, negative for failure. */ -void ice_vc_process_vf_msg (struct ice_pf *pf,struct ice_rq_event_info *event, -struct ice_mbx_data *mbxdata) +int ice_vc_process_vf_msg (struct ice_pf *pf,struct ice_rq_event_info *event, +struct ice_mbx_data *mbxdata) {u32 v_opcode =le32_to_cpu (event->desc.cookie_high);s16 vf_id =le16_to_cpu (event->desc.retval);
@@ -3915,13 +3928,13 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event,if (!vf) {dev_err (dev,"Unable to locate VF for message from VF ID %d, opcode %d, len %d\n" , vf_id, v_opcode, msglen); -return ; +return -EINVAL ; }mutex_lock (&vf->cfg_lock);/* Check if the VF is trying to overflow the mailbox */ -if (ice_is_malicious_vf (vf, mbxdata)) +if (!test_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states) &&ice_is_malicious_vf (vf, mbxdata)) goto finish;/* Check if VF is disabled. */
@@ -4083,4 +4096,5 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event, finish:mutex_unlock (&vf->cfg_lock);ice_put_vf (vf); +return err; }
drivers/net/ethernet/intel/ice/ice_virtchnl.h +4 -3
@@ -63,8 +63,8 @@ int ice_vc_respond_to_vf(struct ice_vf *vf, u32 v_opcode, enum virtchnl_status_code v_retval, u8 *msg, u16 msglen);bool ice_vc_isvalid_vsi_id (struct ice_vf *vf,u16 vsi_id); -void ice_vc_process_vf_msg (struct ice_pf *pf,struct ice_rq_event_info *event, -struct ice_mbx_data *mbxdata); +int ice_vc_process_vf_msg (struct ice_pf *pf,struct ice_rq_event_info *event, +struct ice_mbx_data *mbxdata);#else /* CONFIG_PCI_IOV */ static inline void ice_virtchnl_set_dflt_ops (struct ice_vf *vf) { }static inline void ice_virtchnl_set_repr_ops (struct ice_vf *vf) { }
@@ -84,10 +84,11 @@ static inline bool ice_vc_isvalid_vsi_id(struct ice_vf *vf, u16 vsi_id) return false; } -static inline void +static inline int ice_vc_process_vf_msg (struct ice_pf *pf,struct ice_rq_event_info *event,struct ice_mbx_data *mbxdata) { +return -EOPNOTSUPP ; }#endif /* !CONFIG_PCI_IOV */
include/linux/net/intel/ice_migration.h +12 -0
@@ -9,6 +9,8 @@ void *ice_migration_get_vf(struct pci_dev *vf_pdev);void ice_migration_put_vf (void *opaque);void ice_migration_init_vf (void *opaque);void ice_migration_uninit_vf (void *opaque); +int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz); +int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz);#else static inline void *ice_migration_get_vf (struct pci_dev *vf_pdev)
@@ -21,6 +23,16 @@ static inline void ice_migration_put_vf(void *opaque) }static inline void ice_migration_init_vf (void *opaque) { }static inline void ice_migration_uninit_vf (void *opaque) { } +static inline int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz) +{ +return 0 ; +} + +static inline int ice_migration_restore_devstate (void *opaque,const u8 *buf, +u64 buf_sz) +{ +return 0 ; +}#endif /* CONFIG_ICE_VFIO_PCI */ #endif /* _ICE_MIGRATION_H_ */
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 7/15 ice: do not notify VF link state during migration
In reply to Lingyu LiuPF driver will notify VF link state when replaying virtual channel commands GET_VF_RESOURCE and ENABLE_QUEUES.
Block PF driver to send message to VF if it is migrating.
--- drivers/net/ethernet/intel/ice/ice_virtchnl.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/net/ethernet/intel/ice/ice_virtchnl.c +3 -0
@@ -233,6 +233,9 @@ void ice_vc_notify_vf_link_state(struct ice_vf *vf)struct virtchnl_pf_event pfe = {0 };struct ice_hw *hw = &vf->pf->hw; +if (test_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states)) +return ; + pfe.event =VIRTCHNL_EVENT_LINK_CHANGE ; pfe.severity =PF_EVENT_SEVERITY_INFO ;
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 8/15 ice: change VSI id in virtual channel message after migration
In reply to Lingyu LiuSave the VSI num used in migration src VM in VF structure and change VSI id in virtual channel message payload to the dst VF's VSI id under following two conditions:
1) the VSI id in virtual channel message payload is the same with src VF's VSI id. Or
2) it is replaying virtual channel message.
This is to prevent PF rejects to process VF messages during migration.
--- .../net/ethernet/intel/ice/ice_migration.c | 108 ++++++++++++++++++ .../intel/ice/ice_migration_private.h | 3 + drivers/net/ethernet/intel/ice/ice_vf_lib.c | 1 + drivers/net/ethernet/intel/ice/ice_vf_lib.h | 1 + drivers/net/ethernet/intel/ice/ice_virtchnl.c | 3 + 5 files changed, 116 insertions(+)
drivers/net/ethernet/intel/ice/ice_migration.c +108 -0
@@ -14,6 +14,10 @@ struct ice_migration_virtchnl_msg_listnode {struct ice_migration_virtchnl_msg_slot msg_slot; }; +struct ice_migration_dev_state { +u16 vsi_id; +}__aligned(8) ; +/** * ice_migration_get_vf - Get ice VF structure pointer by pdev * @vf_pdev: pointer to ice vfio pci VF pdev structure
@@ -62,6 +66,7 @@ void ice_migration_init_vf(void *opaque) vf->migration_active = true;INIT_LIST_HEAD (&vf->virtchnl_msg_list); vf->virtchnl_msg_num =0 ; + vf->vm_vsi_num = vf->lan_vsi_num; }EXPORT_SYMBOL (ice_migration_init_vf);
@@ -175,11 +180,24 @@ int ice_migration_save_devstate(void *opaque, u8 *buf, u64 buf_sz)struct ice_migration_virtchnl_msg_slot *last_op;struct ice_vf *vf = (struct ice_vf *)opaque;struct device *dev =ice_pf_to_dev (vf->pf); +struct ice_migration_dev_state *devstate;u64 total_sz =0 ;if (vf ==NULL )return -EINVAL ; +/* reserve space to store device state, saving VSI id in the beginning */ + total_sz +=sizeof (struct ice_migration_dev_state ); +if (total_sz > buf_sz) { +dev_err (dev,"Insufficient buffer to store device state for VF %d\n" , + vf->vf_id); +return -ENOBUFS ; + } + + devstate = (struct ice_migration_dev_state *)buf; + devstate->vsi_id = vf->vm_vsi_num; + buf +=sizeof (*devstate); +list_for_each_entry (msg_listnode, &vf->virtchnl_msg_list, node) {struct ice_migration_virtchnl_msg_slot *msg_slot;u64 slot_size;
@@ -229,6 +247,7 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)struct ice_migration_virtchnl_msg_slot *msg_slot;struct ice_vf *vf = (struct ice_vf *)opaque;struct device *dev =ice_pf_to_dev (vf->pf); +struct ice_migration_dev_state *devstate;struct ice_rq_event_info event;u64 total_sz =0 ;u64 op_msglen_sz;
@@ -238,6 +257,16 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)if (!buf)return -EINVAL ; + total_sz +=sizeof (struct ice_migration_dev_state ); +if (total_sz > buf_sz) { +dev_err (dev,"VF %d msg size exceeds buffer size\n" , vf->vf_id); +return -ENOBUFS ; + } + + devstate = (struct ice_migration_dev_state *)buf; + vf->vm_vsi_num = devstate->vsi_id; +dev_dbg (dev,"VF %d vm vsi num is:%d\n" , vf->vf_id, vf->vm_vsi_num); + buf +=sizeof (*devstate); msg_slot = (struct ice_migration_virtchnl_msg_slot *)buf; op_msglen_sz =sizeof (struct ice_migration_virtchnl_msg_slot );/* check whether enough space for opcode and msg_len */
@@ -285,3 +314,82 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)return ret; }EXPORT_SYMBOL (ice_migration_restore_devstate); + +/** +* ice_migration_fix_msg_vsi - change virtual channel msg VSI id +* +* @vf: pointer to the VF structure +* @v_opcode: virtchnl message operation code +* @msg: pointer to the virtual channel message +* +* After migration, the VSI id of virtual channel message is still +* migration src VSI id. Some virtual channel commands will fail +* due to unmatch VSI id. +* Change virtual channel message payload VSI id to real VSI id. +*/ +void ice_migration_fix_msg_vsi (struct ice_vf *vf,u32 v_opcode,u8 *msg) +{ +if (!vf->migration_active) +return ; + +switch (v_opcode) { +case VIRTCHNL_OP_ADD_ETH_ADDR : +case VIRTCHNL_OP_DEL_ETH_ADDR : +case VIRTCHNL_OP_ENABLE_QUEUES : +case VIRTCHNL_OP_DISABLE_QUEUES : +case VIRTCHNL_OP_CONFIG_RSS_KEY : +case VIRTCHNL_OP_CONFIG_RSS_LUT : +case VIRTCHNL_OP_GET_STATS : +case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE : +case VIRTCHNL_OP_ADD_FDIR_FILTER : +case VIRTCHNL_OP_DEL_FDIR_FILTER : +case VIRTCHNL_OP_ADD_VLAN : +case VIRTCHNL_OP_DEL_VLAN : { +/* Read the beginning two bytes of message for VSI id */ +u16 *vsi_id = (u16 *)msg; + +if (*vsi_id == vf->vm_vsi_num || +test_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states)) + *vsi_id = vf->lan_vsi_num; +break ; + } +case VIRTCHNL_OP_CONFIG_IRQ_MAP : { +struct virtchnl_irq_map_info *irqmap_info; +u16 num_q_vectors_mapped; +int i; + + irqmap_info = (struct virtchnl_irq_map_info *)msg; + num_q_vectors_mapped = irqmap_info->num_vectors; +for (i =0 ; i < num_q_vectors_mapped; i++) { +struct virtchnl_vector_map *map; + + map = &irqmap_info->vecmap[i]; +if (map->vsi_id == vf->vm_vsi_num || +test_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states)) + map->vsi_id = vf->lan_vsi_num; + } +break ; + } +case VIRTCHNL_OP_CONFIG_VSI_QUEUES : { +struct virtchnl_vsi_queue_config_info *qci; + + qci = (struct virtchnl_vsi_queue_config_info *)msg; +if (qci->vsi_id == vf->vm_vsi_num || +test_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states)) { +int i; + + qci->vsi_id = vf->lan_vsi_num; +for (i =0 ; i < qci->num_queue_pairs; i++) { +struct virtchnl_queue_pair_info *qpi; + + qpi = &qci->qpair[i]; + qpi->txq.vsi_id = vf->lan_vsi_num; + qpi->rxq.vsi_id = vf->lan_vsi_num; + } + } +break ; + } +default : +break ; + } +}
drivers/net/ethernet/intel/ice/ice_migration_private.h +3 -0
@@ -13,10 +13,13 @@#if IS_ENABLED (CONFIG_ICE_VFIO_PCI )void ice_migration_save_vf_msg (struct ice_vf *vf,struct ice_rq_event_info *event); +void ice_migration_fix_msg_vsi (struct ice_vf *vf,u32 v_opcode,u8 *msg);#else static inline void ice_migration_save_vf_msg (struct ice_vf *vf,struct ice_rq_event_info *event) { } +static inline void +ice_migration_fix_msg_vsi (struct ice_vf *vf,u32 v_opcode,u8 *msg) { }#endif /* CONFIG_ICE_VFIO_PCI */ #endif /* _ICE_MIGRATION_PRIVATE_H_ */
drivers/net/ethernet/intel/ice/ice_vf_lib.c +1 -0
@@ -1332,6 +1332,7 @@ void ice_vf_set_initialized(struct ice_vf *vf)clear_bit (ICE_VF_STATE_DIS , vf->vf_states);set_bit (ICE_VF_STATE_INIT , vf->vf_states);memset (&vf->vlan_v2_caps,0 ,sizeof (vf->vlan_v2_caps)); + vf->vm_vsi_num = vf->lan_vsi_num; }/**
drivers/net/ethernet/intel/ice/ice_vf_lib.h +1 -0
@@ -138,6 +138,7 @@ struct ice_vf {bool migration_active;struct list_head virtchnl_msg_list;u64 virtchnl_msg_num; +u16 vm_vsi_num; };/* Flags for controlling behavior of ice_reset_vf */
drivers/net/ethernet/intel/ice/ice_virtchnl.c +3 -0
@@ -3973,6 +3973,9 @@ int ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event, goto finish; } +if (vf->migration_active) +ice_migration_fix_msg_vsi (vf, v_opcode, msg); +switch (v_opcode) {case VIRTCHNL_OP_VERSION : err = ops->get_ver_msg (vf, msg);
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 9/15 ice: save and restore RX queue head
In reply to Lingyu LiuSave RX queue head in device migration region and
restore RX queue head at migration dst by writing RX queue context after replaying virtual channel command VIRTCHNL_OP_CONFIG_VSI_QUEUES.
--- .../net/ethernet/intel/ice/ice_migration.c | 98 +++++++++++++++++++ include/linux/net/intel/ice_migration.h | 3 + 2 files changed, 101 insertions(+)
drivers/net/ethernet/intel/ice/ice_migration.c +98 -0
@@ -2,6 +2,7 @@/* Copyright (C) 2018-2023 Intel Corporation */ #include "ice.h" +#include "ice_base.h" struct ice_migration_virtchnl_msg_slot {u32 opcode;
@@ -16,6 +17,8 @@ struct ice_migration_virtchnl_msg_listnode {struct ice_migration_dev_state {u16 vsi_id; +/* next RX desc index to be processed by the device */ +u16 rx_head[IAVF_QRX_TAIL_MAX ]; }__aligned(8) ;/**
@@ -166,6 +169,44 @@ void ice_migration_save_vf_msg(struct ice_vf *vf, } } +static int +ice_migration_save_rx_head (struct ice_vf *vf, +struct ice_migration_dev_state *devstate) +{ +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +struct device *dev; +struct ice_hw *hw; +int i; + + dev =ice_pf_to_dev (pf); + hw = &pf->hw; + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } +ice_for_each_rxq (vsi, i) { +struct ice_rx_ring *rx_ring = vsi->rx_rings[i]; +struct ice_rlan_ctx rlan_ctx = {0 }; +int status; +u16 pf_q; + +if (!test_bit (i, vf->rxq_ena)) +continue ; + + pf_q = rx_ring->reg_idx; + status =ice_read_rxq_ctx (hw, &rlan_ctx, pf_q); +if (status) { +dev_err (dev,"Failed to read RXQ[%d] context, err=%d\n" , + rx_ring->q_index, status); +return -EIO ; + } + devstate->rx_head[i] = rlan_ctx.head; + } +return 0 ; +} +/** * ice_migration_save_devstate - save VF msg to migration buffer * @opaque: pointer to VF handler in ice vdev
@@ -182,6 +223,7 @@ int ice_migration_save_devstate(void *opaque, u8 *buf, u64 buf_sz)struct device *dev =ice_pf_to_dev (vf->pf);struct ice_migration_dev_state *devstate;u64 total_sz =0 ; +int ret;if (vf ==NULL )return -EINVAL ;
@@ -196,6 +238,11 @@ int ice_migration_save_devstate(void *opaque, u8 *buf, u64 buf_sz) devstate = (struct ice_migration_dev_state *)buf; devstate->vsi_id = vf->vm_vsi_num; + ret =ice_migration_save_rx_head (vf, devstate); +if (ret) { +dev_err (dev,"VF %d failed to save rxq head\n" , vf->vf_id); +return ret; + } buf +=sizeof (*devstate);list_for_each_entry (msg_listnode, &vf->virtchnl_msg_list, node) {
@@ -231,6 +278,48 @@ int ice_migration_save_devstate(void *opaque, u8 *buf, u64 buf_sz) }EXPORT_SYMBOL (ice_migration_save_devstate); +static int +ice_migration_restore_rx_head (struct ice_vf *vf, +struct ice_migration_dev_state *devstate) +{ +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +struct device *dev; +int i; + + dev =ice_pf_to_dev (pf); + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } +ice_for_each_rxq (vsi, i) { +struct ice_rx_ring *rx_ring = vsi->rx_rings[i]; +struct ice_rlan_ctx rlan_ctx = {0 }; +int status; +u16 pf_q; + +if (!rx_ring) +return -EINVAL ; + pf_q = rx_ring->reg_idx; + status =ice_read_rxq_ctx (&pf->hw, &rlan_ctx, pf_q); +if (status) { +dev_err (dev,"Failed to read RXQ[%d] context, err=%d\n" , + rx_ring->q_index, status); +return -EIO ; + } + + rlan_ctx.head = devstate->rx_head[i]; + status =ice_write_rxq_ctx (&pf->hw, &rlan_ctx, pf_q); +if (status) { +dev_err (dev,"Failed to set LAN RXQ[%d] context, err=%d\n" , + rx_ring->q_index, status); +return -EIO ; + } + } +return 0 ; +} +/** * ice_migration_restore_devstate - restore device state at dst * @opaque: pointer to VF handler in ice vdev
@@ -299,6 +388,15 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz) msg_slot->opcode);break ; } +if (msg_slot->opcode ==VIRTCHNL_OP_CONFIG_VSI_QUEUES ) { + ret =ice_migration_restore_rx_head (vf, devstate); +if (ret) { +dev_err (dev,"VF %d failed to restore rx head\n" , + vf->vf_id); +break ; + } + } + event.msg_buf =NULL ; msg_slot = (struct ice_migration_virtchnl_msg_slot *) ((char *)msg_slot + slot_sz);
include/linux/net/intel/ice_migration.h +3 -0
@@ -5,6 +5,9 @@#define _ICE_MIGRATION_H_#if IS_ENABLED (CONFIG_ICE_VFIO_PCI ) + +#define IAVF_QRX_TAIL_MAX 256 +void *ice_migration_get_vf (struct pci_dev *vf_pdev);void ice_migration_put_vf (void *opaque);void ice_migration_init_vf (void *opaque);
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 10/15 ice: save and restore TX queue head
In reply to Lingyu LiuSave TX queue head in device migration region and restore TX queue head at migration dst by inserting NOP descriptors packages and kicking doorbell.
By default, VF driver's TX DMA ring context is configured by ice driver. In device resuming stage of live migration, it's ice driver's reponsibility to recover the TX DMA ring context. Due to HW limitation, some of TX DMA ring context can't be recovered by writing mmio registers. Instead, it can be recovered by injecting NOP descriptors into the DMA ring descriptors indirectly. The NOP descriptor injection needs assistance from host kernel driver to read/write TX DMA ring descriptors. As a result, vfio_dma_rw() is used to recover the TX DMA ring context.
--- .../net/ethernet/intel/ice/ice_migration.c | 203 +++++++++++++++++- drivers/net/ethernet/intel/ice/ice_virtchnl.c | 10 + include/linux/net/intel/ice_migration.h | 9 +- 3 files changed, 213 insertions(+), 9 deletions(-)
drivers/net/ethernet/intel/ice/ice_migration.c +196 -7
@@ -3,6 +3,7 @@#include "ice.h" #include "ice_base.h" +#include "ice_txrx_lib.h" struct ice_migration_virtchnl_msg_slot {u32 opcode;
@@ -19,6 +20,8 @@ struct ice_migration_dev_state {u16 vsi_id;/* next RX desc index to be processed by the device */ u16 rx_head[IAVF_QRX_TAIL_MAX ]; +/* next TX desc index to be processed by the device */ +u16 tx_head[IAVF_QRX_TAIL_MAX ]; }__aligned(8) ;/**
@@ -207,10 +210,51 @@ ice_migration_save_rx_head(struct ice_vf *vf,return 0 ; } +static int +ice_migration_save_tx_head (struct ice_vf *vf, +struct ice_migration_dev_state *devstate) +{ +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +struct device *dev; +int i =0 ; + + dev =ice_pf_to_dev (pf); + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } + +ice_for_each_txq (vsi, i) { +u16 tx_head; +u32 reg; + +if (!test_bit (i, vf->txq_ena)) +continue ; + + reg =rd32 (&pf->hw,QTX_COMM_HEAD (vsi->txq_map[i])); + tx_head = (reg &QTX_COMM_HEAD_HEAD_M ) + >>QTX_COMM_HEAD_HEAD_S ; + +if (tx_head ==QTX_COMM_HEAD_HEAD_M || + tx_head == (vsi->tx_rings[i]->count -1 )) +/* when transmitted packet number is 0 or tx_ring +* length, the next packet to be sent is 0. +*/ + tx_head =0 ; +else + tx_head++; + + devstate->tx_head[i] = tx_head; + } +return 0 ; +} +/** * ice_migration_save_devstate - save VF msg to migration buffer -* @opaque: pointer to VF handler in ice vdev -* @buf: pointer to VF msg in migration buffer +* @opaque: pointer to vf handler in ice vdev +* @buf: pointer to vf msg in migration buffer * @buf_sz: size of migration buffer * * Return 0 for success, negative for error
@@ -243,6 +287,11 @@ int ice_migration_save_devstate(void *opaque, u8 *buf, u64 buf_sz)dev_err (dev,"VF %d failed to save rxq head\n" , vf->vf_id);return ret; } + ret =ice_migration_save_tx_head (vf, devstate); +if (ret) { +dev_err (dev,"VF %d failed to save txq head\n" , vf->vf_id); +return ret; + } buf +=sizeof (*devstate);list_for_each_entry (msg_listnode, &vf->virtchnl_msg_list, node) {
@@ -320,18 +369,145 @@ ice_migration_restore_rx_head(struct ice_vf *vf,return 0 ; } +static int +ice_migration_restore_tx_head (struct ice_vf *vf, +struct ice_migration_dev_state *devstate, +struct vfio_device *vdev) +{ +struct ice_tx_desc *tx_desc_dummy, *tx_desc; +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +u16 max_ring_len =0 ; +struct device *dev; +int ret =0 ; +int i =0 ; + + dev =ice_pf_to_dev (vf->pf); + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } + +ice_for_each_txq (vsi, i) { +if (!test_bit (i, vf->txq_ena)) +continue ; + + max_ring_len =max (vsi->tx_rings[i]->count, max_ring_len); + } + +if (max_ring_len ==0 ) +return 0 ; + + tx_desc = (struct ice_tx_desc *)kcalloc + (max_ring_len,sizeof (struct ice_tx_desc ),GFP_KERNEL ); + tx_desc_dummy = (struct ice_tx_desc *)kcalloc + (max_ring_len,sizeof (struct ice_tx_desc ),GFP_KERNEL ); +if (!tx_desc || !tx_desc_dummy) { +dev_err (dev,"VF %d failed to allocate memory for tx descriptors to restore tx head\n" , + vf->vf_id); + ret = -ENOMEM ; + goto err; + } + +for (i =0 ; i < max_ring_len; i++) { +u32 td_cmd; + + td_cmd =ICE_TXD_LAST_DESC_CMD |ICE_TX_DESC_CMD_DUMMY ; + tx_desc_dummy[i].cmd_type_offset_bsz = +ice_build_ctob (td_cmd,0 ,SZ_256 ,0 ); + } + +/* For each tx queue, we restore the tx head following below steps: +* 1. backup original tx ring descriptor memory +* 2. overwrite the tx ring descriptor with dummy packets +* 3. kick doorbell register to trigger descriptor writeback, +* then tx head will move from 0 to tail - 1 and tx head is restored +* to the place we expect. +* 4. restore the tx ring with original tx ring descriptor memory in +* order not to corrupt the ring context. +*/ +ice_for_each_txq (vsi, i) { +struct ice_tx_ring *tx_ring = vsi->tx_rings[i]; +u16 *tx_heads = devstate->tx_head; +u32 tx_head; +int j; + +if (!test_bit (i, vf->txq_ena) || tx_heads[i] ==0 ) +continue ; + +if (tx_heads[i] >= tx_ring->count) { +dev_err (dev,"saved tx ring head exceeds tx ring count\n" ); + ret = -EINVAL ; + goto err; + } + ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc, + tx_ring->count *sizeof (tx_desc[0 ]), false); +if (ret) { +dev_err (dev,"kvm read guest tx ring error: %d\n" , + ret); + goto err; + } + ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc_dummy, + tx_heads[i] *sizeof (tx_desc_dummy[0 ]), true); +if (ret) { +dev_err (dev,"kvm write guest return error: %d\n" , + ret); + goto err; + } + +/* Force memory writes to complete before letting h/w know there +* are new descriptors to fetch. +*/ +wmb (); +writel (tx_heads[i], tx_ring->tail); +/* wait until tx_head equals tx_heads[i] - 1 */ + tx_head =rd32 (&pf->hw,QTX_COMM_HEAD (vsi->txq_map[i])); + tx_head = (tx_head &QTX_COMM_HEAD_HEAD_M ) + >>QTX_COMM_HEAD_HEAD_S ; +for (j =0 ; j <QTX_HEAD_RESTORE_DELAY_MAX && + tx_head != (u32)(tx_heads[i] -1 ); j++) { +usleep_range (QTX_HEAD_RESTORE_DELAY_SLEEP_US_MIN ,QTX_HEAD_RESTORE_DELAY_SLEEP_US_MAX ); + tx_head =rd32 (&pf->hw,QTX_COMM_HEAD (vsi->txq_map[i])); + tx_head = (tx_head &QTX_COMM_HEAD_HEAD_M ) + >>QTX_COMM_HEAD_HEAD_S ; + } +if (j ==10 ) { + ret = -EIO ; +dev_err (dev,"VF %d txq[%d] head restore timeout\n" , + vf->vf_id, i); + goto err; + } + ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc, + tx_ring->count *sizeof (tx_desc[0 ]), true); +if (ret) { +dev_err (dev,"kvm write guest tx ring error: %d\n" , + ret); + goto err; + } + } + +err: +kfree (tx_desc_dummy); +kfree (tx_desc); + +return ret; +} +/** * ice_migration_restore_devstate - restore device state at dst * @opaque: pointer to VF handler in ice vdev * @buf: pointer to device state buf in migration buffer * @buf_sz: size of migration buffer +* @vdev: pointer to vfio device * * This function uses the device state saved in migration buffer * to restore device state at dst VM * * Return 0 for success, negative for error */ -int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz) +int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz, +struct vfio_device *vdev) {struct ice_migration_virtchnl_msg_slot *msg_slot;struct ice_vf *vf = (struct ice_vf *)opaque;
@@ -343,7 +519,7 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)u64 slot_sz;int ret =0 ; -if (!buf) +if (!buf || !vdev)return -EINVAL ; total_sz +=sizeof (struct ice_migration_dev_state );
@@ -374,7 +550,7 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)dev_err (dev,"VF %d msg size exceeds buffer size\n" , vf->vf_id); ret = -ENOBUFS ; -break ; + goto err; }dev_dbg (dev,"VF %d replay virtchnl message op code: %d, msg len: %d\n" , vf->vf_id, msg_slot->opcode, msg_slot->msg_len);
@@ -386,7 +562,7 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)if (ret) {dev_err (dev,"failed to replay virtchnl message op code: %d\n" , msg_slot->opcode); -break ; + goto err; }if (msg_slot->opcode ==VIRTCHNL_OP_CONFIG_VSI_QUEUES ) { ret =ice_migration_restore_rx_head (vf, devstate);
@@ -405,9 +581,22 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz)dev_err (dev,"VF %d msg size exceeds buffer size\n" , vf->vf_id); ret = -ENOBUFS ; -break ; + goto err; } } + +/* Since we can't restore tx head directly due to HW limitation, we +* could only restore tx head indirectly by dummy packets injection. +* After virtual channel replay completes, tx rings are enabled. +* Then restore tx head for tx rings by injecting dummy packets. +*/ + ret =ice_migration_restore_tx_head (vf, devstate, vdev); +if (ret) { +dev_err (dev,"failed to restore tx queue head\n" ); + goto err; + } + +err:clear_bit (ICE_VF_STATE_REPLAYING_VC , vf->vf_states);return ret; }
drivers/net/ethernet/intel/ice/ice_virtchnl.c +10 -0
@@ -1328,6 +1328,16 @@ static int ice_vc_ena_qs_msg(struct ice_vf *vf, u8 *msg)continue ;ice_vf_ena_txq_interrupt (vsi, vf_q_id); +/* set TXQ head value to 0x1FFF to indicate no packet is sent. According +* to CVL HAS Transmit Queue Context Structure, the size of descriptor +* queue is from 8 descriptores (QLEN=0x8) to 8K-32 descriptors +* (QLEN=0x1FE0). So QTX_COMM_HEAD.HEAD rang value from 0x1fe0 to 0x1fff +* is reserved and will never be used by HW. So, use 0x1FFF as a marker. +* This is used by live migration. +*/ +if (vf->migration_active) +wr32 (&vsi->back->hw,QTX_COMM_HEAD (vsi->txq_map[vf_q_id]), +QTX_COMM_HEAD_HEAD_M );set_bit (vf_q_id, vf->txq_ena); }
include/linux/net/intel/ice_migration.h +7 -2
@@ -3,17 +3,22 @@#ifndef _ICE_MIGRATION_H_#define _ICE_MIGRATION_H_ +#include <linux/vfio.h> #if IS_ENABLED (CONFIG_ICE_VFIO_PCI )#define IAVF_QRX_TAIL_MAX 256 +#define QTX_HEAD_RESTORE_DELAY_MAX 100 +#define QTX_HEAD_RESTORE_DELAY_SLEEP_US_MIN 10 +#define QTX_HEAD_RESTORE_DELAY_SLEEP_US_MAX 10void *ice_migration_get_vf (struct pci_dev *vf_pdev);void ice_migration_put_vf (void *opaque);void ice_migration_init_vf (void *opaque);void ice_migration_uninit_vf (void *opaque);int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz); -int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz); +int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz, +struct vfio_device *vdev);#else static inline void *ice_migration_get_vf (struct pci_dev *vf_pdev)
@@ -32,7 +37,7 @@ static inline int ice_migration_save_devstate(void *opaque, u8 *buf, u64 buf_sz) }static inline int ice_migration_restore_devstate (void *opaque,const u8 *buf, -u64 buf_sz) +u64 buf_sz,struct vfio_device *vdev) {return 0 ; }
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 11/15 ice: stop device before saving device states
In reply to Lingyu LiuStop device by disabling TX queue and RX queue.
ice_vfio_pci driver introduced in following patches from this series needs this function when device state transits from VFIO_DEVICE_STATE_RUNNING to VFIO_DEVICE_STATE_STOP.
--- .../net/ethernet/intel/ice/ice_migration.c | 68 +++++++++++++++++++ include/linux/net/intel/ice_migration.h | 5 ++ 2 files changed, 73 insertions(+)
drivers/net/ethernet/intel/ice/ice_migration.c +68 -0
@@ -2,6 +2,8 @@/* Copyright (C) 2018-2023 Intel Corporation */ #include "ice.h" +#include "ice_lib.h" +#include "ice_fltr.h" #include "ice_base.h" #include "ice_txrx_lib.h"
@@ -172,6 +174,72 @@ void ice_migration_save_vf_msg(struct ice_vf *vf, } } +/** +* ice_migration_suspend_vf - suspend device on src +* @opaque: pointer to VF handler in ice vdev +* @is_dst: false: migration src true: migration dst +* +* Return 0 for success, negative for error +*/ +int ice_migration_suspend_vf (void *opaque,bool is_dst) +{ +struct ice_vf *vf = (struct ice_vf *)opaque; +struct ice_vsi *vsi =ice_get_vf_vsi (vf); +struct ice_pf *pf = vf->pf; +struct device *dev; +int ret; + +if (is_dst) +return 0 ; + + dev =ice_pf_to_dev (pf); +if (vf->virtchnl_msg_num >=VIRTCHNL_MSG_MAX ) { +dev_err (dev,"SR-IOV live migration disabled on VF %d. Migration buffer exceeded\n" , + vf->vf_id); +return -EIO ; + } + +if (!vsi) { +dev_err (dev,"VF %d VSI is NULL\n" , vf->vf_id); +return -EINVAL ; + } +/* Prevent VSI from incoming packets by removing all filters before +* stop rx ring and draining the traffic. There are possibilities that +* rx ring head value jitters when rx ring is stopped with large amount +* of packets incoming. In this case, HW mismatches SW on rx ring head +* state. As a result, after restoring rx ring head on the destination +* VM, the missing rx descriptors will never be written back, causing +* packets receiving failure and dropped. +*/ +ice_fltr_remove_all (vsi); +/* MAC based filter rule is disabled at this point. Set MAC to zero +* to keep consistency when using ip link to display MAC address. +*/ +eth_zero_addr (vf->hw_lan_addr); +eth_zero_addr (vf->dev_lan_addr); +/* For the tx side, there is possibility that some descriptors are +* still pending to be transmitted by HW. Since VM is stopped now, +* wait a while to make sure all the transmission is completed. +* For the rx side, head value jittering may happen in case of high +* packet rate. Since all forwarding filters are removed now, wait a +* while to make sure all the reception is completed and rx head no +* longer moves. +*/ +usleep_range (1000 ,2000 ); + ret =ice_vsi_stop_lan_tx_rings (vsi,ICE_NO_RESET , vf->vf_id); +if (ret) { +dev_err (dev,"VF %d failed to stop tx rings\n" , vf->vf_id); +return -EIO ; + } + ret =ice_vsi_stop_all_rx_rings (vsi); +if (ret) { +dev_err (dev,"VF %d failed to stop rx rings\n" , vf->vf_id); +return -EIO ; + } +return 0 ; +} +EXPORT_SYMBOL (ice_migration_suspend_vf); +static int ice_migration_save_rx_head (struct ice_vf *vf,struct ice_migration_dev_state *devstate)
include/linux/net/intel/ice_migration.h +5 -0
@@ -16,6 +16,7 @@ void *ice_migration_get_vf(struct pci_dev *vf_pdev);void ice_migration_put_vf (void *opaque);void ice_migration_init_vf (void *opaque);void ice_migration_uninit_vf (void *opaque); +int ice_migration_suspend_vf (void *opaque,bool mig_dst);int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz);int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz,struct vfio_device *vdev);
@@ -31,6 +32,10 @@ static inline void ice_migration_put_vf(void *opaque) }static inline void ice_migration_init_vf (void *opaque) { }static inline void ice_migration_uninit_vf (void *opaque) { } +static inline int ice_migration_suspend_vf (void *opaque,bool mig_dst) +{ +return 0 ; +}static inline int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz) {return 0 ;
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 12/15 ice: mask VF advanced capabilities if live migration is activated
In reply to Lingyu LiuMask the advanced VF capability flags that are
not supported when VF migration is activated.
--- .../net/ethernet/intel/ice/ice_migration.c | 27 +++++++++++++++++++ .../intel/ice/ice_migration_private.h | 6 +++++ drivers/net/ethernet/intel/ice/ice_virtchnl.c | 2 ++ 3 files changed, 35 insertions(+)
drivers/net/ethernet/intel/ice/ice_migration.c +27 -0
@@ -748,3 +748,30 @@ void ice_migration_fix_msg_vsi(struct ice_vf *vf, u32 v_opcode, u8 *msg)break ; } } + +#define VIRTCHNL_VF_MIGRATION_SUPPORT_FEATURE \ + (VIRTCHNL_VF_OFFLOAD_L2 | \ + VIRTCHNL_VF_OFFLOAD_RSS_PF | \ + VIRTCHNL_VF_OFFLOAD_RSS_AQ | \ + VIRTCHNL_VF_OFFLOAD_RSS_REG | \ + VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 | \ + VIRTCHNL_VF_OFFLOAD_ENCAP | \ + VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM | \ + VIRTCHNL_VF_OFFLOAD_RX_POLLING | \ + VIRTCHNL_VF_OFFLOAD_WB_ON_ITR | \ + VIRTCHNL_VF_CAP_ADV_LINK_SPEED | \ + VIRTCHNL_VF_OFFLOAD_VLAN | \ + VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC | \ + VIRTCHNL_VF_OFFLOAD_USO) + +/** +* ice_migration_supported_caps - get migration supported VF capablities +* +* When migration is activated, some VF capabilities are not supported. +* So unmask those capability flags for VF resources. +*/ +u32 ice_migration_supported_caps (void ) +{ + +return VIRTCHNL_VF_MIGRATION_SUPPORT_FEATURE ; +}
drivers/net/ethernet/intel/ice/ice_migration_private.h +6 -0
@@ -14,12 +14,18 @@void ice_migration_save_vf_msg (struct ice_vf *vf,struct ice_rq_event_info *event);void ice_migration_fix_msg_vsi (struct ice_vf *vf,u32 v_opcode,u8 *msg); +u32 ice_migration_supported_caps (void );#else static inline void ice_migration_save_vf_msg (struct ice_vf *vf,struct ice_rq_event_info *event) { }static inline void ice_migration_fix_msg_vsi (struct ice_vf *vf,u32 v_opcode,u8 *msg) { } +static inline u32 +ice_migration_supported_caps (void ) +{ +return 0xFFFFFFFF ; +}#endif /* CONFIG_ICE_VFIO_PCI */ #endif /* _ICE_MIGRATION_PRIVATE_H_ */
drivers/net/ethernet/intel/ice/ice_virtchnl.c +2 -0
@@ -468,6 +468,8 @@ static int ice_vc_get_vf_res_msg(struct ice_vf *vf, u8 *msg)VIRTCHNL_VF_OFFLOAD_RSS_REG |VIRTCHNL_VF_OFFLOAD_VLAN ; +if (vf->migration_active) + vf->driver_caps &=ice_migration_supported_caps (); vfres->vf_cap_flags =VIRTCHNL_VF_OFFLOAD_L2 ; vsi =ice_get_vf_vsi (vf);if (!vsi) {
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 13/15 vfio/ice: implement vfio_pci driver for E800 devices
In reply to Lingyu LiuAdd a vendor-specific vfio_pci driver for E800 devices.
It uses vfio_pci_core to register to the VFIO subsystem and then implements the E800 specific logic to support VF live migration.
It implements the device state transition flow for live migration.
--- MAINTAINERS | 7 + drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/ice/Kconfig | 10 + drivers/vfio/pci/ice/Makefile | 4 + drivers/vfio/pci/ice/ice_vfio_pci.c | 841 ++++++++++++++++++++++++++++ 6 files changed, 866 insertions(+) create mode 100644 drivers/vfio/pci/ice/Kconfig create mode 100644 drivers/vfio/pci/ice/Makefile create mode 100644 drivers/vfio/pci/ice/ice_vfio_pci.c
MAINTAINERS +7 -0
@@ -22170,6 +22170,13 @@ L: [email protected] S: Maintained F: drivers/vfio/pci/mlx5/ +VFIO ICE PCI DRIVER +M: Yahui Cao <[email protected]> +M: Lingyu Liu <[email protected]> +L: [email protected] +S: Maintained +F: drivers/vfio/pci/ice/ + VFIO PCI DEVICE SPECIFIC DRIVERS R: Jason Gunthorpe <[email protected]> R: Yishai Hadas <[email protected]>
drivers/vfio/pci/Kconfig +2 -0
@@ -59,4 +59,6 @@ source "drivers/vfio/pci/mlx5/Kconfig" source "drivers/vfio/pci/hisilicon/Kconfig" +source "drivers/vfio/pci/ice/Kconfig" + endif
drivers/vfio/pci/Makefile +2 -0
@@ -11,3 +11,5 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o obj-$(CONFIG_MLX5_VFIO_PCI) += mlx5/ obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/ + +obj-$(CONFIG_ICE_VFIO_PCI) += ice/
drivers/vfio/pci/ice/Kconfig new file +10 -0
@@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0-only +config ICE_VFIO_PCI + tristate "VFIO support for Intel(R) Ethernet Connection E800 Series" + depends on ICE + depends on VFIO_PCI_CORE + help + This provides migration support for Intel(R) Ethernet connection E800 + series devices using the VFIO framework. + + If you don't know what to do here, say N.
drivers/vfio/pci/ice/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_ICE_VFIO_PCI) += ice-vfio-pci.o +ice-vfio-pci-y := ice_vfio_pci.o +
drivers/vfio/pci/ice/ice_vfio_pci.c new file +841 -0
841 lines not on this page · raw message
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 14/15 vfio: Expose vfio_device_has_container()
In reply to Lingyu LiuFrom: Yahui Cao <[email protected]>
Expose vfio_device_has_container() to be used by drivers to know what is the vfio uAPI probed by user application.
Next patch for ice vfio pci driver will use it.
--- drivers/vfio/group.c | 1 + include/linux/vfio.h | 1 + 2 files changed, 2 insertions(+)
drivers/vfio/group.c +1 -0
@@ -744,6 +744,7 @@ bool vfio_device_has_container(struct vfio_device *device) {return device->group->container; } +EXPORT_SYMBOL_GPL (vfio_device_has_container);/** * vfio_file_iommu_group - Return the struct iommu_group for the vfio group file
include/linux/vfio.h +1 -0
@@ -253,6 +253,7 @@ bool vfio_file_is_group(struct file *file);bool vfio_file_enforced_coherent (struct file *file);void vfio_file_set_kvm (struct file *file,struct kvm *kvm);bool vfio_file_has_dev (struct file *file,struct vfio_device *device); +bool vfio_device_has_container (struct vfio_device *device);#define VFIO_PIN_PAGES_MAX_ENTRIES (PAGE_SIZE/sizeof(unsigned long))
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 15/15 vfio/ice: support iommufd vfio compat mode
In reply to Lingyu LiuFrom: Yahui Cao <[email protected]>
In iommufd vfio compat mode, vfio_dma_rw() will return failure, since vfio_device_has_container() returns false and device->iommufd_access is NULL.
Currently device->iommufd_access will not be created if vfio device is backed by pci device. To support IOVA access, manually create iommufd_access context by iommufd_access_create/attach() and access IOVA by iommufd_access_rw(). And in order to minimize the iommufd_access's impact, store the iommufd_access context in driver data, create it only before loading the device state and destroy it once finishing loading the device state.
To be compatible with legacy vfio, use vfio_device_has_container() to check the vfio uAPI. If in legacy vfio mode, call vfio_dma_rw() directly, otherwise call iommufd_access_rw().
--- .../net/ethernet/intel/ice/ice_migration.c | 23 +-- drivers/vfio/pci/ice/ice_vfio_pci.c | 171 +++++++++++++++++- include/linux/net/intel/ice_migration.h | 4 +- 3 files changed, 179 insertions(+), 19 deletions(-)
drivers/net/ethernet/intel/ice/ice_migration.c +12 -11
@@ -440,7 +440,7 @@ ice_migration_restore_rx_head(struct ice_vf *vf,static int ice_migration_restore_tx_head (struct ice_vf *vf,struct ice_migration_dev_state *devstate, -struct vfio_device *vdev) +dma_rw_handler_t handler,void *data) {struct ice_tx_desc *tx_desc_dummy, *tx_desc;struct ice_vsi *vsi =ice_get_vf_vsi (vf);
@@ -509,15 +509,15 @@ ice_migration_restore_tx_head(struct ice_vf *vf, ret = -EINVAL ; goto err; } - ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc, - tx_ring->count *sizeof (tx_desc[0 ]), false); + ret =handler (data, tx_ring->dma, (void *)tx_desc, + tx_ring->count *sizeof (tx_desc[0 ]), false);if (ret) {dev_err (dev,"kvm read guest tx ring error: %d\n" , ret); goto err; } - ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc_dummy, - tx_heads[i] *sizeof (tx_desc_dummy[0 ]), true); + ret =handler (data, tx_ring->dma, (void *)tx_desc_dummy, + tx_heads[i] *sizeof (tx_desc_dummy[0 ]), true);if (ret) {dev_err (dev,"kvm write guest return error: %d\n" , ret);
@@ -546,8 +546,8 @@ ice_migration_restore_tx_head(struct ice_vf *vf, vf->vf_id, i); goto err; } - ret =vfio_dma_rw (vdev, tx_ring->dma, (void *)tx_desc, - tx_ring->count *sizeof (tx_desc[0 ]), true); + ret =handler (data, tx_ring->dma, (void *)tx_desc, + tx_ring->count *sizeof (tx_desc[0 ]), true);if (ret) {dev_err (dev,"kvm write guest tx ring error: %d\n" , ret);
@@ -567,7 +567,8 @@ ice_migration_restore_tx_head(struct ice_vf *vf,* @opaque: pointer to VF handler in ice vdev * @buf: pointer to device state buf in migration buffer * @buf_sz: size of migration buffer -* @vdev: pointer to vfio device +* @handler: dma_rw_handler +* @data: dma_rw_handler data * * This function uses the device state saved in migration buffer * to restore device state at dst VM
@@ -575,7 +576,7 @@ ice_migration_restore_tx_head(struct ice_vf *vf,* Return 0 for success, negative for error */ int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz, -struct vfio_device *vdev) +dma_rw_handler_t handler,void *data) {struct ice_migration_virtchnl_msg_slot *msg_slot;struct ice_vf *vf = (struct ice_vf *)opaque;
@@ -587,7 +588,7 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz,u64 slot_sz;int ret =0 ; -if (!buf || !vdev) +if (!buf)return -EINVAL ; total_sz +=sizeof (struct ice_migration_dev_state );
@@ -658,7 +659,7 @@ int ice_migration_restore_devstate(void *opaque, const u8 *buf, u64 buf_sz,* After virtual channel replay completes, tx rings are enabled. * Then restore tx head for tx rings by injecting dummy packets. */ - ret =ice_migration_restore_tx_head (vf, devstate, vdev); + ret =ice_migration_restore_tx_head (vf, devstate, handler, data);if (ret) {dev_err (dev,"failed to restore tx queue head\n" ); goto err;
drivers/vfio/pci/ice/ice_vfio_pci.c +164 -7
209 lines not on this page · raw message
include/linux/net/intel/ice_migration.h +3 -1
@@ -7,6 +7,8 @@#if IS_ENABLED (CONFIG_ICE_VFIO_PCI ) +typedef int (*dma_rw_handler_t )(void *data,dma_addr_t iova,void *buf, +size_t len,bool write);#define IAVF_QRX_TAIL_MAX 256#define QTX_HEAD_RESTORE_DELAY_MAX 100#define QTX_HEAD_RESTORE_DELAY_SLEEP_US_MIN 10
@@ -19,7 +21,7 @@ void ice_migration_uninit_vf(void *opaque);int ice_migration_suspend_vf (void *opaque,bool mig_dst);int ice_migration_save_devstate (void *opaque,u8 *buf,u64 buf_sz);int ice_migration_restore_devstate (void *opaque,const u8 *buf,u64 buf_sz, -struct vfio_device *vdev); +dma_rw_handler_t handler,void *data);#else static inline void *ice_migration_get_vf (struct pci_dev *vf_pdev)
Signature · 6 lines
2.25.1 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Dear Lingyu,
Thank you for your patches.
Am 20.06.23 um 11:59 schrieb Lingyu Liu:
Quoted 2 lines
This series adds vfio live migration support for Intel E810 VF devices based on the v2 migration protocol.
[…]
I’d be great, if you documented how to test the new driver.
Kind regards,
Paul
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Quoted 5 lines · Paul Menzel
Dear Lingyu,
Thank you for your patches.
Quoted 8 lines
Am 20.06.23 um 11:59 schrieb Lingyu Liu:
Quoted 2 lines
This series adds vfio live migration support for Intel E810 VF devices based on the v2 migration protocol.
[…]
I’d be great, if you documented how to test the new driver.
Hi Paul, will add the steps to test live migration in next revision. Hope you can help to check then.
Quoted 4 lines
Kind regards,
Paul
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan