vdpa: enable user to set mac, mtu
Subsystems (3)
v3 cover vdpa: enable user to set mac, mtu
Currently user cannot set the mac address and mtu of the vdpa device. This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
If a vendor driver supports such configuration user can set it otherwise user gets unsupported error.
vdpa mac address and mtu are device configuration layout fields. To keep interface generic enough for multiple types of vdpa devices, mac address and mtu setting is implemented as configuration layout config knobs.
This enables to use similar config layout for other virtio devices.
An example of query & set of config layout fields for vdpa_sim_net driver:
Configuration layout fields are set after device is created.
This enables user to change such fields at later point without destroying and recreating the device for new config.
$ vdpa mgmtdev show vdpasim_net: supported_classes net
Add the device:
$ vdpa dev add name bar mgmtdev vdpasim_net
Configure mac address and mtu:
$ vdpa dev config set bar mac 00:11:22:33:44:55 mtu 9000
In above command only mac address or only mtu can also be set.
View the config after setting:
$ vdpa dev config show bar: mac 00:11:22:33:44:55 link up link_announce false mtu 9000 speed 0 duplex 0
Patch summary:
Patch-1 introduced and use helpers for get/set config area Patch-2 implement query device config layout
Patch-3 enanble user to set mac and mtu in config space Patch-4 vdpa_sim_net implements get and set of config layout Patch-5 mlx5 vdpa driver supports user provided mac config Patch-6 mlx5 vdpa driver uses user provided mac during rx flow steering
changelog:v2->v3:
- dropped patches which are merged
- simplified code to handle non transitional devices
v1->v2:
- new patches to fix kdoc comment to add new kdoc section
- new patch to have synchronized access to features and config space
- read whole net config layout instead of individual fields
- added error extack for unmanaged vdpa device
- fixed several endianness issues
- introduced vdpa device ops for get config which is synchronized with other get/set features ops and config ops
- fixed mtu range checking for max
- using NLA_POLICY_ETH_ADDR
- set config moved to device ops instead of mgmtdev ops
- merged build and set to single routine
- ensuring that user has NET_ADMIN capability for configuring network attributes
- using updated interface and callbacks for get/set config
- following new api for config get/set for mgmt tool in mlx5 vdpa driver
- fixes for accessing right SF dma device and bar address
- fix for mtu calculation
- fix for bit access in features
- fix for index restore with suspend/resume operation
Eli Cohen (2):
vdpa/mlx5: Support configuration of MAC vdpa/mlx5: Forward only packets with allowed MAC address
Parav Pandit (4):
vdpa: Introduce and use vdpa device get, set config helpers
vdpa: Introduce query of device config layout
vdpa: Enable user to set mac and mtu of vdpa device
vdpa_sim_net: Enable user to set mac address and mtu
drivers/vdpa/mlx5/net/mlx5_vnet.c | 101 ++++++-- drivers/vdpa/vdpa.c | 337 +++++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.c | 13 ++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 2 + drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 34 +-- drivers/vhost/vdpa.c | 3 +- include/linux/vdpa.h | 38 +-- include/uapi/linux/vdpa.h | 12 + 8 files changed, 490 insertions(+), 50 deletions(-)
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v3 1/6 vdpa: Introduce and use vdpa device get, set config helpers
Subsequent patches enable get and set configuration either via management device or via vdpa device' config ops.
This requires synchronization between multiple callers to get and set config callbacks. Features setting also influence the layout of the configuration fields endianness.
To avoid exposing synchronization primitives to callers, introduce helper for setting the configuration and use it.
---
changelog:v1->v2
- new patch to have synchronized access to features and config space
--- drivers/vdpa/vdpa.c | 36 ++++++++++++++++++++++++++++++++++++ drivers/vhost/vdpa.c | 3 +-- include/linux/vdpa.h | 18 ++++-------------- 3 files changed, 41 insertions(+), 16 deletions(-)
drivers/vdpa/vdpa.c +36 -0
@@ -284,6 +284,42 @@ void vdpa_mgmtdev_unregister(struct vdpa_mgmt_dev *mdev) }EXPORT_SYMBOL_GPL (vdpa_mgmtdev_unregister); +/** +* vdpa_get_config - Get one or more device configuration fields. +* @vdev: vdpa device to operate on +* @offset: starting byte offset of the field +* @buf: buffer pointer to read to +* @len: length of the configuration fields in bytes +*/ +void vdpa_get_config (struct vdpa_device *vdev,unsigned int offset, +void *buf,unsigned int len) +{ +const struct vdpa_config_ops *ops = vdev->config; + +/* +* Config accesses aren't supposed to trigger before features are set. +* If it does happen we assume a legacy guest. +*/ +if (!vdev->features_valid) +vdpa_set_features (vdev,0 ); + ops->get_config (vdev, offset, buf, len); +} +EXPORT_SYMBOL_GPL (vdpa_get_config); + +/** +* vdpa_set_config - Set one or more device configuration fields. +* @vdev: vdpa device to operate on +* @offset: starting byte offset of the field +* @buf: buffer pointer to read from +* @length: length of the configuration fields in bytes +*/ +void vdpa_set_config (struct vdpa_device *vdev,unsigned int offset, +void *buf,unsigned int length) +{ + vdev->config->set_config (vdev, offset, buf, length); +} +EXPORT_SYMBOL_GPL (vdpa_set_config); +static bool mgmtdev_handle_match (const struct vdpa_mgmt_dev *mdev,const char *busname,const char *devname) {
drivers/vhost/vdpa.c +1 -2
@@ -231,7 +231,6 @@ static long vhost_vdpa_set_config(struct vhost_vdpa *v, struct vhost_vdpa_config __user *c) {struct vdpa_device *vdpa = v->vdpa; -const struct vdpa_config_ops *ops = vdpa->config;struct vhost_vdpa_config config;unsigned long size = offsetof(struct vhost_vdpa_config , buf);u8 *buf;
@@ -245,7 +244,7 @@ static long vhost_vdpa_set_config(struct vhost_vdpa *v,if (IS_ERR (buf))return PTR_ERR (buf); - ops->set_config (vdpa, config.off, buf, config.len); +vdpa_set_config (vdpa, config.off, buf, config.len);kvfree (buf);return 0 ;
include/linux/vdpa.h +4 -14
@@ -332,20 +332,10 @@ static inline int vdpa_set_features(struct vdpa_device *vdev, u64 features)return ops->set_features (vdev, features); } - -static inline void vdpa_get_config (struct vdpa_device *vdev,unsigned offset, -void *buf,unsigned int len) -{ -const struct vdpa_config_ops *ops = vdev->config; - -/* -* Config accesses aren't supposed to trigger before features are set. -* If it does happen we assume a legacy guest. -*/ -if (!vdev->features_valid) -vdpa_set_features (vdev,0 ); - ops->get_config (vdev, offset, buf, len); -} +void vdpa_get_config (struct vdpa_device *vdev,unsigned int offset, +void *buf,unsigned int len); +void vdpa_set_config (struct vdpa_device *dev,unsigned int offset, +void *buf,unsigned int length);/** * struct vdpa_mgmtdev_ops - vdpa device ops
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 1/6] vdpa: Introduce and use vdpa device get, set config helpers
Quoted 7 lines · Parav Pandit
Subsequent patches enable get and set configuration either via management device or via vdpa device' config ops.
This requires synchronization between multiple callers to get and set config callbacks. Features setting also influence the layout of the configuration fields endianness.
Quoted 5 lines
To avoid exposing synchronization primitives to callers, introduce helper for setting the configuration and use it.
Signed-off-by Parav Pandit <[email protected]>Reviewed-by Eli Cohen <[email protected]>
Quoted 97 lines
---
changelog:v1->v2
- new patch to have synchronized access to features and config space--- drivers/vdpa/vdpa.c | 36 ++++++++++++++++++++++++++++++++++++ drivers/vhost/vdpa.c | 3 +-- include/linux/vdpa.h | 18 ++++-------------- 3 files changed, 41 insertions(+), 16 deletions(-)drivers/vdpa/vdpa.c +36
-0@@ -284,6 +284,42 @@ void vdpa_mgmtdev_unregister(struct vdpa_mgmt_dev *mdev) }EXPORT_SYMBOL_GPL (vdpa_mgmtdev_unregister); +/** +* vdpa_get_config - Get one or more device configuration fields. +* @vdev: vdpa device to operate on +* @offset: starting byte offset of the field +* @buf: buffer pointer to read to +* @len: length of the configuration fields in bytes +*/ +void vdpa_get_config (struct vdpa_device *vdev,unsigned int offset, +void *buf,unsigned int len) +{ +const struct vdpa_config_ops *ops = vdev->config; + +/* +* Config accesses aren't supposed to trigger before features are set. +* If it does happen we assume a legacy guest. +*/ +if (!vdev->features_valid) +vdpa_set_features (vdev,0 ); + ops->get_config (vdev, offset, buf, len); +} +EXPORT_SYMBOL_GPL (vdpa_get_config); + +/** +* vdpa_set_config - Set one or more device configuration fields. +* @vdev: vdpa device to operate on +* @offset: starting byte offset of the field +* @buf: buffer pointer to read from +* @length: length of the configuration fields in bytes +*/ +void vdpa_set_config (struct vdpa_device *vdev,unsigned int offset, +void *buf,unsigned int length) +{ + vdev->config->set_config (vdev, offset, buf, length); +} +EXPORT_SYMBOL_GPL (vdpa_set_config); +static bool mgmtdev_handle_match (const struct vdpa_mgmt_dev *mdev,const char *busname,const char *devname) {drivers/vhost/vdpa.c +1
-2@@ -231,7 +231,6 @@ static long vhost_vdpa_set_config(struct vhost_vdpa *v, struct vhost_vdpa_config __user *c) {struct vdpa_device *vdpa = v->vdpa; -const struct vdpa_config_ops *ops = vdpa->config;struct vhost_vdpa_config config;unsigned long size = offsetof(struct vhost_vdpa_config , buf);u8 *buf;@@ -245,7 +244,7 @@ static long vhost_vdpa_set_config(struct vhost_vdpa *v,if (IS_ERR (buf))return PTR_ERR (buf); - ops->set_config (vdpa, config.off, buf, config.len); +vdpa_set_config (vdpa, config.off, buf, config.len);kvfree (buf);return 0 ;include/linux/vdpa.h +0
-11@@ -332,20 +332,10 @@ static inline int vdpa_set_features(struct vdpa_device *vdev, u64 features)return ops->set_features (vdev, features); } - -static inline void vdpa_get_config (struct vdpa_device *vdev,unsigned offset, -void *buf,unsigned int len) -{ -const struct vdpa_config_ops *ops = vdev->config; - -/* -* Config accesses aren't supposed to trigger before features are set. -* If it does happen we assume a legacy guest. -*/ -if (!vdev->features_valid)
include/linux/vdpa.h static inline int vdpa_set_features(struct vdpa_device *vdev, u64 features) · line 341
Quoted 10 lines
include/linux/vdpa.h +4
-3-vdpa_set_features (vdev,0 ); - ops->get_config (vdev, offset, buf, len); -} +void vdpa_get_config (struct vdpa_device *vdev,unsigned int offset, +void *buf,unsigned int len); +void vdpa_set_config (struct vdpa_device *dev,unsigned int offset, +void *buf,unsigned int length);/** * struct vdpa_mgmtdev_ops - vdpa device ops
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v3 2/6 vdpa: Introduce query of device config layout
In reply to Parav PanditIntroduce a command to query a device config layout.
An example query of network vdpa device:
$ vdpa dev add name bar mgmtdev vdpasim_net
$ vdpa dev config show bar: mac 00:35:09:19:48:05 link up link_announce false mtu 1500 speed 0 duplex 0
$ vdpa dev config show -jp
{
"config": {
"bar": {
"mac": "00:35:09:19:48:05",
"link ": "up",
"link_announce ": false,
"mtu": 1500,
"speed": 0,
"duplex": 0
}
}
}--- drivers/vdpa/vdpa.c | 212 ++++++++++++++++++++++++++++++++++++++ include/linux/vdpa.h | 2 + include/uapi/linux/vdpa.h | 11 ++ 3 files changed, 225 insertions(+)
drivers/vdpa/vdpa.c +212 -0
@@ -14,6 +14,8 @@#include <uapi/linux/vdpa.h> #include <net/genetlink.h> #include <linux/mod_devicetable.h> +#include <linux/virtio_net.h> +#include <linux/virtio_ids.h> static LIST_HEAD (mdev_head);/* A global mutex that protects vdpa management device and device level operations. */
@@ -60,6 +62,7 @@ static void vdpa_release_dev(struct device *d) ops->free (vdev);ida_simple_remove (&vdpa_index_ida, vdev->index); +mutex_destroy (&vdev->cf_mutex);kfree (vdev); }
@@ -114,6 +117,7 @@ struct vdpa_device *__vdpa_alloc_device(struct device *parent,if (err) goto err_name; +mutex_init (&vdev->cf_mutex);device_initialize (&vdev->dev);return vdev;
@@ -296,6 +300,7 @@ void vdpa_get_config(struct vdpa_device *vdev, unsigned int offset, {const struct vdpa_config_ops *ops = vdev->config; +mutex_lock (&vdev->cf_mutex);/* * Config accesses aren't supposed to trigger before features are set. * If it does happen we assume a legacy guest.
@@ -303,6 +308,7 @@ void vdpa_get_config(struct vdpa_device *vdev, unsigned int offset,if (!vdev->features_valid)vdpa_set_features (vdev,0 ); ops->get_config (vdev, offset, buf, len); +mutex_unlock (&vdev->cf_mutex); }EXPORT_SYMBOL_GPL (vdpa_get_config);
@@ -316,7 +322,9 @@ EXPORT_SYMBOL_GPL(vdpa_get_config);void vdpa_set_config (struct vdpa_device *vdev,unsigned int offset,void *buf,unsigned int length) { +mutex_lock (&vdev->cf_mutex); vdev->config->set_config (vdev, offset, buf, length); +mutex_unlock (&vdev->cf_mutex); }EXPORT_SYMBOL_GPL (vdpa_set_config);
@@ -643,6 +651,204 @@ static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callbareturn msg->len; } +static int vdpa_dev_net_mq_config_fill (struct vdpa_device *vdev, +struct sk_buff *msg,u64 features, +const struct virtio_net_config *config) +{ +u16 val_u16; + +if ((features & (1ULL <<VIRTIO_NET_F_MQ )) ==0 ) +return 0 ; + + val_u16 =le16_to_cpu (config->max_virtqueue_pairs); +return nla_put_u16 (msg,VDPA_ATTR_DEV_NET_CFG_MAX_VQP , val_u16); +} + +static int vdpa_dev_net_rss_config_fill (struct vdpa_device *vdev, +struct sk_buff *msg,u64 features, +const struct virtio_net_config *config) +{ +u16 val_u16; +u16 val_u32; + +if ((features & (1ULL <<VIRTIO_NET_F_RSS )) ==0 ) +return 0 ; + +if (nla_put_u8 (msg,VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN , + config->rss_max_key_size)) +return -EMSGSIZE ; + + val_u16 =le16_to_cpu (config->rss_max_key_size); +if (nla_put_u16 (msg,VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN , val_u16)) +return -EMSGSIZE ; + + val_u32 =le32_to_cpu (config->supported_hash_types); +if (nla_put_u32 (msg,VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES , val_u32)) +return -EMSGSIZE ; +return 0 ; +} + +static int vdpa_dev_net_config_fill (struct vdpa_device *vdev,struct sk_buff *msg) +{ +struct virtio_net_config config = {}; +u64 features; +u32 val_u32; +u16 val_u16; +int err; + +vdpa_get_config (vdev,0 , &config,sizeof (config)); + +if (nla_put (msg,VDPA_ATTR_DEV_NET_CFG_MACADDR ,sizeof (config.mac), + config.mac)) +return -EMSGSIZE ; + + val_u16 =le16_to_cpu (config.status); +if (nla_put_u16 (msg,VDPA_ATTR_DEV_NET_STATUS , val_u16)) +return -EMSGSIZE ; + + val_u16 =le16_to_cpu (config.mtu); +if (nla_put_u16 (msg,VDPA_ATTR_DEV_NET_CFG_MTU , val_u16)) +return -EMSGSIZE ; + + val_u32 =le32_to_cpu (config.speed); +if (nla_put_u32 (msg,VDPA_ATTR_DEV_NET_CFG_SPEED , val_u32)) +return -EMSGSIZE ; + +if (nla_put_u8 (msg,VDPA_ATTR_DEV_NET_CFG_DUPLEX , config.duplex)) +return -EMSGSIZE ; + + features = vdev->config->get_features (vdev); + + err =vdpa_dev_net_mq_config_fill (vdev, msg, features, &config); +if (err) +return err; +return vdpa_dev_net_rss_config_fill (vdev, msg, features, &config); +} + +static int +vdpa_dev_config_fill (struct vdpa_device *vdev,struct sk_buff *msg,u32 portid,u32 seq, +int flags,struct netlink_ext_ack *extack) +{ +u32 device_id; +void *hdr; +int err; + + hdr =genlmsg_put (msg, portid, seq, &vdpa_nl_family, flags, +VDPA_CMD_DEV_CONFIG_GET ); +if (!hdr) +return -EMSGSIZE ; + +if (nla_put_string (msg,VDPA_ATTR_DEV_NAME ,dev_name (&vdev->dev))) { + err = -EMSGSIZE ; + goto msg_err; + } + + device_id = vdev->config->get_device_id (vdev); +if (nla_put_u32 (msg,VDPA_ATTR_DEV_ID , device_id)) { + err = -EMSGSIZE ; + goto msg_err; + } + +switch (device_id) { +case VIRTIO_ID_NET : + err =vdpa_dev_net_config_fill (vdev, msg); +break ; +default : + err = -EOPNOTSUPP ; +break ; + } +if (err) + goto msg_err; + +genlmsg_end (msg, hdr); +return 0 ; + +msg_err: +genlmsg_cancel (msg, hdr); +return err; +} + +static int vdpa_nl_cmd_dev_config_get_doit (struct sk_buff *skb,struct genl_info *info) +{ +struct vdpa_device *vdev; +struct sk_buff *msg; +const char *devname; +struct device *dev; +int err; + +if (!info->attrs[VDPA_ATTR_DEV_NAME ]) +return -EINVAL ; + devname =nla_data (info->attrs[VDPA_ATTR_DEV_NAME ]); + msg =nlmsg_new (NLMSG_DEFAULT_SIZE ,GFP_KERNEL ); +if (!msg) +return -ENOMEM ; + +mutex_lock (&vdpa_dev_mutex); + dev =bus_find_device (&vdpa_bus,NULL , devname, vdpa_name_match); +if (!dev) { +NL_SET_ERR_MSG_MOD (info->extack,"device not found" ); + err = -ENODEV ; + goto dev_err; + } + vdev = container_of(dev, struct vdpa_device, dev); +if (!vdev->mdev) { +NL_SET_ERR_MSG_MOD (info->extack,"unmanaged vdpa device" ); + err = -EINVAL ; + goto mdev_err; + } + err =vdpa_dev_config_fill (vdev, msg, info->snd_portid, info->snd_seq, +0 , info->extack); +if (!err) + err =genlmsg_reply (msg, info); + +mdev_err: +put_device (dev); +dev_err: +mutex_unlock (&vdpa_dev_mutex); +if (err) +nlmsg_free (msg); +return err; +} + +static int vdpa_dev_config_dump (struct device *dev,void *data) +{ + struct vdpa_device *vdev = container_of(dev, struct vdpa_device, dev); +struct vdpa_dev_dump_info *info = data; +int err; + +if (!vdev->mdev) +return 0 ; +if (info->idx < info->start_idx) { + info->idx++; +return 0 ; + } + err =vdpa_dev_config_fill (vdev, info->msg,NETLINK_CB (info->cb->skb).portid, + info->cb->nlh->nlmsg_seq,NLM_F_MULTI , + info->cb->extack); +if (err) +return err; + + info->idx++; +return 0 ; +} + +static int +vdpa_nl_cmd_dev_config_get_dumpit (struct sk_buff *msg,struct netlink_callback *cb) +{ +struct vdpa_dev_dump_info info; + + info.msg = msg; + info.cb = cb; + info.start_idx = cb->args[0 ]; + info.idx =0 ; + +mutex_lock (&vdpa_dev_mutex); +bus_for_each_dev (&vdpa_bus,NULL , &info, vdpa_dev_config_dump); +mutex_unlock (&vdpa_dev_mutex); + cb->args[0 ] = info.idx; +return msg->len; +} +static const struct nla_policy vdpa_nl_policy[VDPA_ATTR_MAX +1 ] = { [VDPA_ATTR_MGMTDEV_BUS_NAME ] = { .type =NLA_NUL_STRING }, [VDPA_ATTR_MGMTDEV_DEV_NAME ] = { .type =NLA_STRING },
@@ -674,6 +880,12 @@ static const struct genl_ops vdpa_nl_ops[] = { .doit = vdpa_nl_cmd_dev_get_doit, .dumpit = vdpa_nl_cmd_dev_get_dumpit, }, + { + .cmd = VDPA_CMD_DEV_CONFIG_GET, + .validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP, + .doit = vdpa_nl_cmd_dev_config_get_doit, + .dumpit = vdpa_nl_cmd_dev_config_get_dumpit, + }, };static struct genl_family vdpa_nl_family__ro_after_init = {
include/linux/vdpa.h +2 -0
@@ -42,6 +42,7 @@ struct vdpa_mgmt_dev;* @dev: underlying device * @dma_dev: the actual device that is performing DMA * @config: the configuration ops for this device. +* @cf_mutex: Protects get and set access to features and configuration layout. * @index: device index * @features_valid: were features initialized? for legacy guests * @nvqs: maximum number of supported virtqueues
@@ -52,6 +53,7 @@ struct vdpa_device {struct device dev;struct device *dma_dev;const struct vdpa_config_ops *config; +struct mutex cf_mutex;/* Protects get/set config and features */ unsigned int index;bool features_valid;int nvqs;
include/uapi/linux/vdpa.h +11 -0
@@ -17,6 +17,7 @@ enum vdpa_command {VDPA_CMD_DEV_NEW ,VDPA_CMD_DEV_DEL ,VDPA_CMD_DEV_GET ,/* can dump */ + VDPA_CMD_DEV_CONFIG_GET, /* can dump */ }; enum vdpa_attr {
@@ -33,6 +34,16 @@ enum vdpa_attr {VDPA_ATTR_DEV_MAX_VQS ,/* u32 */ VDPA_ATTR_DEV_MAX_VQ_SIZE ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_MACADDR ,/* binary */ +VDPA_ATTR_DEV_NET_STATUS ,/* u8 */ +VDPA_ATTR_DEV_NET_CFG_MAX_VQP ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_MTU ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_SPEED ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_DUPLEX ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN ,/* u8 */ +VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES ,/* u32 */ +/* new attributes must be added above here */ VDPA_ATTR_MAX, };
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 77 lines · Parav Pandit
Introduce a command to query a device config layout.
An example query of network vdpa device:
$ vdpa dev add name bar mgmtdev vdpasim_net$ vdpa dev config show bar: mac 00:35:09:19:48:05 link up link_announce false mtu 1500 speed 0 duplex 0$ vdpa dev config show -jp { "config": { "bar": { "mac": "00:35:09:19:48:05", "link ": "up", "link_announce ": false, "mtu": 1500, "speed": 0, "duplex": 0 } } }Signed-off-by Parav Pandit <[email protected]>Signed-off-by Eli Cohen <[email protected]>--- drivers/vdpa/vdpa.c | 212 ++++++++++++++++++++++++++++++++++++++ include/linux/vdpa.h | 2 + include/uapi/linux/vdpa.h | 11 ++ 3 files changed, 225 insertions(+)drivers/vdpa/vdpa.c +6
-0@@ -14,6 +14,8 @@#include <uapi/linux/vdpa.h> #include <net/genetlink.h> #include <linux/mod_devicetable.h> +#include <linux/virtio_net.h> +#include <linux/virtio_ids.h> static LIST_HEAD (mdev_head);/* A global mutex that protects vdpa management device and device level operations. */ @@ -60,6 +62,7 @@ static void vdpa_release_dev(struct device *d) ops->free (vdev);ida_simple_remove (&vdpa_index_ida, vdev->index); +mutex_destroy (&vdev->cf_mutex);kfree (vdev); }@@ -114,6 +117,7 @@ struct vdpa_device *__vdpa_alloc_device(struct device *parent,if (err) goto err_name; +mutex_init (&vdev->cf_mutex);device_initialize (&vdev->dev);return vdev;@@ -296,6 +300,7 @@ void vdpa_get_config(struct vdpa_device *vdev, unsigned int offset, {const struct vdpa_config_ops *ops = vdev->config; +mutex_lock (&vdev->cf_mutex);/* * Config accesses aren't supposed to trigger before features are set. * If it does happen we assume a legacy guest. @@ -303,6 +308,7 @@ void vdpa_get_config(struct vdpa_device *vdev, unsigned int offset,if (!vdev->features_valid)vdpa_set_features (vdev,0 ); ops->get_config (vdev, offset, buf, len); +mutex_unlock (&vdev->cf_mutex); }EXPORT_SYMBOL_GPL (vdpa_get_config);
drivers/vdpa/vdpa.c EXPORT_SYMBOL_GPL(vdpa_get_config); · line 327
Quoted 6 lines
drivers/vdpa/vdpa.c +2
-0@@ -316,7 +322,9 @@ EXPORT_SYMBOL_GPL(vdpa_get_config);void vdpa_set_config (struct vdpa_device *vdev,unsigned int offset,void *buf,unsigned int length) { +mutex_lock (&vdev->cf_mutex); vdev->config->set_config (vdev, offset, buf, length); +mutex_unlock (&vdev->cf_mutex);
I think it's better to use a separate patch to implement the synchronization in set_get()/get_config()
Quoted 51 lines
drivers/vdpa/vdpa.c +44
-0}EXPORT_SYMBOL_GPL (vdpa_set_config);@@ -643,6 +651,204 @@ static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callbareturn msg->len; } +static int vdpa_dev_net_mq_config_fill (struct vdpa_device *vdev, +struct sk_buff *msg,u64 features, +const struct virtio_net_config *config) +{ +u16 val_u16; + +if ((features & (1ULL <<VIRTIO_NET_F_MQ )) ==0 ) +return 0 ; + + val_u16 =le16_to_cpu (config->max_virtqueue_pairs); +return nla_put_u16 (msg,VDPA_ATTR_DEV_NET_CFG_MAX_VQP , val_u16); +} + +static int vdpa_dev_net_rss_config_fill (struct vdpa_device *vdev, +struct sk_buff *msg,u64 features, +const struct virtio_net_config *config) +{ +u16 val_u16; +u16 val_u32; + +if ((features & (1ULL <<VIRTIO_NET_F_RSS )) ==0 ) +return 0 ; + +if (nla_put_u8 (msg,VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN , + config->rss_max_key_size)) +return -EMSGSIZE ; + + val_u16 =le16_to_cpu (config->rss_max_key_size); +if (nla_put_u16 (msg,VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN , val_u16)) +return -EMSGSIZE ; + + val_u32 =le32_to_cpu (config->supported_hash_types); +if (nla_put_u32 (msg,VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES , val_u32)) +return -EMSGSIZE ; +return 0 ; +} + +static int vdpa_dev_net_config_fill (struct vdpa_device *vdev,struct sk_buff *msg) +{ +struct virtio_net_config config = {}; +u64 features; +u32 val_u32; +u16 val_u16; +int err;
drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba · line 707
Quoted 10 lines
drivers/vdpa/vdpa.c +10
-0+ +vdpa_get_config (vdev,0 , &config,sizeof (config)); + +if (nla_put (msg,VDPA_ATTR_DEV_NET_CFG_MACADDR ,sizeof (config.mac), + config.mac)) +return -EMSGSIZE ; + + val_u16 =le16_to_cpu (config.status); +if (nla_put_u16 (msg,VDPA_ATTR_DEV_NET_STATUS , val_u16)) +return -EMSGSIZE ;
Note that status field only exist when VIRITO_NET_F_STATUS is negotiated. And if not, we need assume the link is up.
drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba · line 718
Quoted 11 lines
drivers/vdpa/vdpa.c +11
-0+ + val_u16 =le16_to_cpu (config.mtu); +if (nla_put_u16 (msg,VDPA_ATTR_DEV_NET_CFG_MTU , val_u16)) +return -EMSGSIZE ; + + val_u32 =le32_to_cpu (config.speed); +if (nla_put_u32 (msg,VDPA_ATTR_DEV_NET_CFG_SPEED , val_u32)) +return -EMSGSIZE ; + +if (nla_put_u8 (msg,VDPA_ATTR_DEV_NET_CFG_DUPLEX , config.duplex)) +return -EMSGSIZE ;
The above two only exists when VIRTIO_NET_F_SPEED_DUPLEX is negotiated.
Quoted 184 lines
drivers/vdpa/vdpa.c +139
-0+ + features = vdev->config->get_features (vdev); + + err =vdpa_dev_net_mq_config_fill (vdev, msg, features, &config); +if (err) +return err; +return vdpa_dev_net_rss_config_fill (vdev, msg, features, &config); +} + +static int +vdpa_dev_config_fill (struct vdpa_device *vdev,struct sk_buff *msg,u32 portid,u32 seq, +int flags,struct netlink_ext_ack *extack) +{ +u32 device_id; +void *hdr; +int err; + + hdr =genlmsg_put (msg, portid, seq, &vdpa_nl_family, flags, +VDPA_CMD_DEV_CONFIG_GET ); +if (!hdr) +return -EMSGSIZE ; + +if (nla_put_string (msg,VDPA_ATTR_DEV_NAME ,dev_name (&vdev->dev))) { + err = -EMSGSIZE ; + goto msg_err; + } + + device_id = vdev->config->get_device_id (vdev); +if (nla_put_u32 (msg,VDPA_ATTR_DEV_ID , device_id)) { + err = -EMSGSIZE ; + goto msg_err; + } + +switch (device_id) { +case VIRTIO_ID_NET : + err =vdpa_dev_net_config_fill (vdev, msg); +break ; +default : + err = -EOPNOTSUPP ; +break ; + } +if (err) + goto msg_err; + +genlmsg_end (msg, hdr); +return 0 ; + +msg_err: +genlmsg_cancel (msg, hdr); +return err; +} + +static int vdpa_nl_cmd_dev_config_get_doit (struct sk_buff *skb,struct genl_info *info) +{ +struct vdpa_device *vdev; +struct sk_buff *msg; +const char *devname; +struct device *dev; +int err; + +if (!info->attrs[VDPA_ATTR_DEV_NAME ]) +return -EINVAL ; + devname =nla_data (info->attrs[VDPA_ATTR_DEV_NAME ]); + msg =nlmsg_new (NLMSG_DEFAULT_SIZE ,GFP_KERNEL ); +if (!msg) +return -ENOMEM ; + +mutex_lock (&vdpa_dev_mutex); + dev =bus_find_device (&vdpa_bus,NULL , devname, vdpa_name_match); +if (!dev) { +NL_SET_ERR_MSG_MOD (info->extack,"device not found" ); + err = -ENODEV ; + goto dev_err; + } + vdev = container_of(dev, struct vdpa_device, dev); +if (!vdev->mdev) { +NL_SET_ERR_MSG_MOD (info->extack,"unmanaged vdpa device" ); + err = -EINVAL ; + goto mdev_err; + } + err =vdpa_dev_config_fill (vdev, msg, info->snd_portid, info->snd_seq, +0 , info->extack); +if (!err) + err =genlmsg_reply (msg, info); + +mdev_err: +put_device (dev); +dev_err: +mutex_unlock (&vdpa_dev_mutex); +if (err) +nlmsg_free (msg); +return err; +} + +static int vdpa_dev_config_dump (struct device *dev,void *data) +{ + struct vdpa_device *vdev = container_of(dev, struct vdpa_device, dev); +struct vdpa_dev_dump_info *info = data; +int err; + +if (!vdev->mdev) +return 0 ; +if (info->idx < info->start_idx) { + info->idx++; +return 0 ; + } + err =vdpa_dev_config_fill (vdev, info->msg,NETLINK_CB (info->cb->skb).portid, + info->cb->nlh->nlmsg_seq,NLM_F_MULTI , + info->cb->extack); +if (err) +return err; + + info->idx++; +return 0 ; +} + +static int +vdpa_nl_cmd_dev_config_get_dumpit (struct sk_buff *msg,struct netlink_callback *cb) +{ +struct vdpa_dev_dump_info info; + + info.msg = msg; + info.cb = cb; + info.start_idx = cb->args[0 ]; + info.idx =0 ; + +mutex_lock (&vdpa_dev_mutex); +bus_for_each_dev (&vdpa_bus,NULL , &info, vdpa_dev_config_dump); +mutex_unlock (&vdpa_dev_mutex); + cb->args[0 ] = info.idx; +return msg->len; +} +static const struct nla_policy vdpa_nl_policy[VDPA_ATTR_MAX +1 ] = { [VDPA_ATTR_MGMTDEV_BUS_NAME ] = { .type =NLA_NUL_STRING }, [VDPA_ATTR_MGMTDEV_DEV_NAME ] = { .type =NLA_STRING },@@ -674,6 +880,12 @@ static const struct genl_ops vdpa_nl_ops[] = { .doit = vdpa_nl_cmd_dev_get_doit, .dumpit = vdpa_nl_cmd_dev_get_dumpit, }, + { + .cmd = VDPA_CMD_DEV_CONFIG_GET, + .validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP, + .doit = vdpa_nl_cmd_dev_config_get_doit, + .dumpit = vdpa_nl_cmd_dev_config_get_dumpit, + }, };static struct genl_family vdpa_nl_family__ro_after_init = {include/linux/vdpa.h +2
-0@@ -42,6 +42,7 @@ struct vdpa_mgmt_dev;* @dev: underlying device * @dma_dev: the actual device that is performing DMA * @config: the configuration ops for this device. +* @cf_mutex: Protects get and set access to features and configuration layout. * @index: device index * @features_valid: were features initialized? for legacy guests * @nvqs: maximum number of supported virtqueues @@ -52,6 +53,7 @@ struct vdpa_device {struct device dev;struct device *dma_dev;const struct vdpa_config_ops *config; +struct mutex cf_mutex;/* Protects get/set config and features */ unsigned int index;bool features_valid;int nvqs;include/uapi/linux/vdpa.h +1
-0@@ -17,6 +17,7 @@ enum vdpa_command {VDPA_CMD_DEV_NEW ,VDPA_CMD_DEV_DEL ,VDPA_CMD_DEV_GET ,/* can dump */ + VDPA_CMD_DEV_CONFIG_GET, /* can dump */ }; enum vdpa_attr {@@ -33,6 +34,16 @@ enum vdpa_attr {VDPA_ATTR_DEV_MAX_VQS ,/* u32 */ VDPA_ATTR_DEV_MAX_VQ_SIZE, /* u16 */
include/uapi/linux/vdpa.h enum vdpa_attr { · line 45
Quoted 10 lines
include/uapi/linux/vdpa.h +9
-0+VDPA_ATTR_DEV_NET_CFG_MACADDR ,/* binary */ +VDPA_ATTR_DEV_NET_STATUS ,/* u8 */ +VDPA_ATTR_DEV_NET_CFG_MAX_VQP ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_MTU ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_SPEED ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_DUPLEX ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN ,/* u8 */ +VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN ,/* u16 */ + VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES, /* u32 */
Is it better to use a separate enum for net specific attributes?
Another question (sorry if it has been asked before). Can we simply return the config (binary) to the userspace, then usespace can use the existing uAPI like virtio_net_config plus the feature to explain the config?
Thanks
include/uapi/linux/vdpa.h enum vdpa_attr { · line 49
Quoted 4 lines
include/uapi/linux/vdpa.h +1
-0+/* new attributes must be added above here */ VDPA_ATTR_MAX, };
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 5 lines
From: Jason Wang <[email protected]>
Sent: Tuesday, June 22, 2021 12:50 PM
[..]
Quoted 16 lines
drivers/vdpa/vdpa.c void vdpa_get_config(struct vdpa_device *vdev,
Quoted 8 lines
{ const struct vdpa_config_ops *ops = vdev->config;drivers/vdpa/vdpa.c +1
-0+mutex_lock (&vdev->cf_mutex);/* * Config accesses aren't supposed to trigger before features are set. * If it does happen we assume a legacy guest. @@ -303,6 +308,7 @@ void vdpa_get_config(struct vdpa_device *vdev,unsigned int offset,
Quoted 7 lines
drivers/vdpa/vdpa.c +1
-0if (!vdev->features_valid)vdpa_set_features (vdev,0 ); ops->get_config (vdev, offset, buf, len); +mutex_unlock (&vdev->cf_mutex); }EXPORT_SYMBOL_GPL (vdpa_get_config);
Quoted 12 lines
drivers/vdpa/vdpa.c EXPORT_SYMBOL_GPL(vdpa_get_config); · line 327
Quoted 7 lines
drivers/vdpa/vdpa.c +2
-0@@ -316,7 +322,9 @@ EXPORT_SYMBOL_GPL(vdpa_get_config);void vdpa_set_config (struct vdpa_device *vdev,unsigned int offset,void *buf,unsigned int length) { +mutex_lock (&vdev->cf_mutex); vdev->config->set_config (vdev, offset, buf, length); +mutex_unlock (&vdev->cf_mutex);I think it's better to use a separate patch to implement the synchronization in set_get()/get_config()
Ok. I will split.
Quoted 64 lines
drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct
Quoted 5 lines
drivers/vdpa/vdpa.c +0
-0}EXPORT_SYMBOL_GPL (vdpa_set_config);@@ -643,6 +651,204 @@ static int vdpa_nl_cmd_dev_get_dumpit(struct
sk_buff*msg, structnetlink_callbaQuoted 4 lines
drivers/vdpa/vdpa.c +1
-0return msg->len; } +static int vdpa_dev_net_mq_config_fill(struct vdpa_device *vdev,drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 10 lines
drivers/vdpa/vdpa.c +10
-0+ struct sk_buff *msg, u64 features, + const struct virtio_net_config *config) +{ + u16 val_u16; + +if ((features & (1ULL <<VIRTIO_NET_F_MQ )) ==0 ) +return 0 ; + + val_u16 =le16_to_cpu (config->max_virtqueue_pairs); + return nla_put_u16(msg, VDPA_ATTR_DEV_NET_CFG_MAX_VQP,
val_u16);drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 5 lines
drivers/vdpa/vdpa.c +5
-0+} + +static int vdpa_dev_net_rss_config_fill (struct vdpa_device *vdev, +struct sk_buff *msg,u64 features, + const struct virtio_net_config*config)
drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 8 lines
drivers/vdpa/vdpa.c +8
-0+{ +u16 val_u16; +u16 val_u32; + +if ((features & (1ULL <<VIRTIO_NET_F_RSS )) ==0 ) +return 0 ; + + if (nla_put_u8(msg,
VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN,drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 5 lines
drivers/vdpa/vdpa.c +5
-0+ config->rss_max_key_size)) +return -EMSGSIZE ; + + val_u16 =le16_to_cpu (config->rss_max_key_size); + if (nla_put_u16(msg,
VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN,val_u16))drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 4 lines
drivers/vdpa/vdpa.c +4
-0+return -EMSGSIZE ; + + val_u32 =le32_to_cpu (config->supported_hash_types); + if (nla_put_u32(msg, VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES,
val_u32))drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 5 lines
drivers/vdpa/vdpa.c +5
-0+return -EMSGSIZE ; +return 0 ; +} + +static int vdpa_dev_net_config_fill(struct vdpa_device *vdev, struct
sk_buff*msg)drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 10 lines
drivers/vdpa/vdpa.c +10
-0+{ +struct virtio_net_config config = {}; +u64 features; +u32 val_u32; +u16 val_u16; +int err; + +vdpa_get_config (vdev,0 , &config,sizeof (config)); + + if (nla_put(msg, VDPA_ATTR_DEV_NET_CFG_MACADDR,sizeof(config.mac),
Quoted 11 lines
drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba · line 707
Quoted 6 lines
drivers/vdpa/vdpa.c +6
-0+ config.mac)) +return -EMSGSIZE ; + + val_u16 =le16_to_cpu (config.status); +if (nla_put_u16 (msg,VDPA_ATTR_DEV_NET_STATUS , val_u16)) +return -EMSGSIZE ;Note that status field only exist when
VIRITO_NET_F_STATUSis negotiated. And if not, we need assume the link is up.
Ok. will change.
Quoted 12 lines
drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 11 lines
drivers/vdpa/vdpa.c +10
-0+ + val_u16 =le16_to_cpu (config.mtu); +if (nla_put_u16 (msg,VDPA_ATTR_DEV_NET_CFG_MTU , val_u16)) +return -EMSGSIZE ; + + val_u32 =le32_to_cpu (config.speed); +if (nla_put_u32 (msg,VDPA_ATTR_DEV_NET_CFG_SPEED , val_u32)) +return -EMSGSIZE ; + + if (nla_put_u8(msg, VDPA_ATTR_DEV_NET_CFG_DUPLEX,config.duplex))
Quoted 6 lines
Review static int vdpa_nl_cmd_dev_get_dumpit(struct · line 718
Quoted 1 line
(quoted diff) +1
-0+ return -EMSGSIZE;The above two only exists when
VIRTIO_NET_F_SPEED_DUPLEXis negotiated.
I think I missed fixing this. You or Michael mentioned this in v2. Will fix.
Quoted 199 lines
drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 12 lines
drivers/vdpa/vdpa.c +11
-0+ + features = vdev->config->get_features (vdev); + + err =vdpa_dev_net_mq_config_fill (vdev, msg, features, &config); +if (err) +return err; +return vdpa_dev_net_rss_config_fill (vdev, msg, features, &config); +} + +static int +vdpa_dev_config_fill(struct vdpa_device *vdev, struct sk_buff *msg, u32portid, u32 seq,
Quoted 33 lines
drivers/vdpa/vdpa.c +32
-0+ int flags, struct netlink_ext_ack *extack) +{ + u32 device_id; +void *hdr; +int err; + + hdr =genlmsg_put (msg, portid, seq, &vdpa_nl_family, flags, +VDPA_CMD_DEV_CONFIG_GET ); +if (!hdr) +return -EMSGSIZE ; + +if (nla_put_string (msg,VDPA_ATTR_DEV_NAME ,dev_name (&vdev- dev))) { + err = -EMSGSIZE ; + goto msg_err; + } + + device_id = vdev->config->get_device_id (vdev); +if (nla_put_u32 (msg,VDPA_ATTR_DEV_ID , device_id)) { + err = -EMSGSIZE ; + goto msg_err; + } + +switch (device_id) { +case VIRTIO_ID_NET : + err =vdpa_dev_net_config_fill (vdev, msg); +break ; +default : + err = -EOPNOTSUPP ; +break ; + } +if (err) + goto msg_err;drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 10 lines
drivers/vdpa/vdpa.c +10
-0+ +genlmsg_end (msg, hdr); +return 0 ; + +msg_err: +genlmsg_cancel (msg, hdr); +return err; +} + +static int vdpa_nl_cmd_dev_config_get_doit(struct sk_buff *skb, struct
genl_info*info)Quoted 6 lines
drivers/vdpa/vdpa.c +6
-0+{ +struct vdpa_device *vdev; +struct sk_buff *msg; +const char *devname; +struct device *dev; +int err;drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 10 lines
drivers/vdpa/vdpa.c +10
-0+ +if (!info->attrs[VDPA_ATTR_DEV_NAME ]) +return -EINVAL ; + devname =nla_data (info->attrs[VDPA_ATTR_DEV_NAME ]); + msg =nlmsg_new (NLMSG_DEFAULT_SIZE ,GFP_KERNEL ); +if (!msg) +return -ENOMEM ; + +mutex_lock (&vdpa_dev_mutex); + dev = bus_find_device(&vdpa_bus, NULL, devname,
vdpa_name_match);drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 8 lines
drivers/vdpa/vdpa.c +8
-0+if (!dev) { +NL_SET_ERR_MSG_MOD (info->extack,"device not found" ); + err = -ENODEV ; + goto dev_err; + } + vdev = container_of(dev, struct vdpa_device, dev); +if (!vdev->mdev) { + NL_SET_ERR_MSG_MOD(info->extack, "unmanaged vdpadevice");
Quoted 11 lines
drivers/vdpa/vdpa.c +10
-0+ err = -EINVAL ; + goto mdev_err; + } + err =vdpa_dev_config_fill (vdev, msg, info->snd_portid, info- snd_seq, +0 , info->extack); +if (!err) + err =genlmsg_reply (msg, info); + +mdev_err: +put_device (dev);drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 10 lines
drivers/vdpa/vdpa.c +10
-0+dev_err: +mutex_unlock (&vdpa_dev_mutex); +if (err) +nlmsg_free (msg); +return err; +} + +static int vdpa_dev_config_dump (struct device *dev,void *data) +{ + struct vdpa_device *vdev = container_of(dev, struct vdpa_device,dev);
Quoted 12 lines
drivers/vdpa/vdpa.c +11
-0+struct vdpa_dev_dump_info *info = data; +int err; + +if (!vdev->mdev) +return 0 ; +if (info->idx < info->start_idx) { + info->idx++; +return 0 ; + } + err = vdpa_dev_config_fill(vdev, info->msg, NETLINK_CB(info->cb- skb).portid, + info->cb->nlh->nlmsg_seq, NLM_F_MULTI,drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 10 lines
drivers/vdpa/vdpa.c +10
-0+ info->cb->extack); +if (err) +return err; + + info->idx++; +return 0 ; +} + +static int +vdpa_nl_cmd_dev_config_get_dumpit(struct sk_buff *msg, struct
netlink_callback*cb)drivers/vdpa/vdpa.c static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct netlink_callba
Quoted 10 lines
drivers/vdpa/vdpa.c +10
-0+{ +struct vdpa_dev_dump_info info; + + info.msg = msg; + info.cb = cb; + info.start_idx = cb->args[0 ]; + info.idx =0 ; + +mutex_lock (&vdpa_dev_mutex); + bus_for_each_dev(&vdpa_bus, NULL, &info,
vdpa_dev_config_dump);Quoted 7 lines
drivers/vdpa/vdpa.c +5
-0+mutex_unlock (&vdpa_dev_mutex); + cb->args[0 ] = info.idx; +return msg->len; +} +static const struct nla_policy vdpa_nl_policy[VDPA_ATTR_MAX + 1] = { [VDPA_ATTR_MGMTDEV_BUS_NAME] = { .type = NLA_NUL_STRING},
drivers/vdpa/vdpa.c static const struct genl_ops vdpa_nl_ops[] = { · line 885
Quoted 8 lines
[
VDPA_ATTR_MGMTDEV_DEV_NAME] = { .type =NLA_STRING},drivers/vdpa/vdpa.c +3
-0@@ -674,6 +880,12 @@ static const struct genl_ops vdpa_nl_ops[] = { .doit = vdpa_nl_cmd_dev_get_doit, .dumpit = vdpa_nl_cmd_dev_get_dumpit, }, + { + .cmd = VDPA_CMD_DEV_CONFIG_GET, + .validate = GENL_DONT_VALIDATE_STRICT |
GENL_DONT_VALIDATE_DUMP,Quoted 11 lines
drivers/vdpa/vdpa.c +3
-0+ .doit = vdpa_nl_cmd_dev_config_get_doit, + .dumpit = vdpa_nl_cmd_dev_config_get_dumpit, + }, }; static struct genl_family vdpa_nl_family __ro_after_init = {include/linux/vdpa.h +0
-0include/linux/vdpa.h struct vdpa_mgmt_dev; · line 45
Quoted 4 lines
include/linux/vdpa.h +1
-0@@ -42,6 +42,7 @@ struct vdpa_mgmt_dev;* @dev: underlying device * @dma_dev: the actual device that is performing DMA * @config: the configuration ops for this device. +* @cf_mutex: Protects get and set access to features and configuration layout.
Quoted 27 lines
include/linux/vdpa.h +1
-0* @index: device index * @features_valid: were features initialized? for legacy guests * @nvqs: maximum number of supported virtqueues @@ -52,6 +53,7 @@ struct vdpa_device {struct device dev;struct device *dma_dev;const struct vdpa_config_ops *config; +struct mutex cf_mutex;/* Protects get/set config and features */ unsigned int index;bool features_valid;int nvqs;include/uapi/linux/vdpa.h +1
-0@@ -17,6 +17,7 @@ enum vdpa_command {VDPA_CMD_DEV_NEW ,VDPA_CMD_DEV_DEL ,VDPA_CMD_DEV_GET ,/* can dump */ + VDPA_CMD_DEV_CONFIG_GET, /* can dump */ }; enum vdpa_attr {@@ -33,6 +34,16 @@ enum vdpa_attr {VDPA_ATTR_DEV_MAX_VQS ,/* u32 */ VDPA_ATTR_DEV_MAX_VQ_SIZE, /* u16 */
Quoted 13 lines
include/uapi/linux/vdpa.h enum vdpa_attr { · line 45
Quoted 9 lines
include/uapi/linux/vdpa.h +9
-0+VDPA_ATTR_DEV_NET_CFG_MACADDR ,/* binary */ +VDPA_ATTR_DEV_NET_STATUS ,/* u8 */ +VDPA_ATTR_DEV_NET_CFG_MAX_VQP ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_MTU ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_SPEED ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_DUPLEX ,/* u16 */ +VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN ,/* u8 */ +VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN ,/* u16 */ + VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES, /* u32 */Is it better to use a separate enum for net specific attributes?
Yes, because they are only net specific.
I guess it is related to your below question.
Quoted 4 lines
Another question (sorry if it has been asked before). Can we simply return the config (binary) to the userspace, then usespace can use the existing uAPI like
virtio_net_configplus the feature to explain the config?
We did discuss in v2.
Usually returning the whole blob and parsing is not desired via netlink.
Returning individual fields give the full flexibility to return only the valid fields.
Otherwise we need to implement another bitmask too to tell which fields from the struct are valid and share with user space. Returning individual fields is the widely used approach.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 9 lines · Parav Pandit
Quoted 2 lines
Is it better to use a separate enum for net specific attributes?
Yes, because they are only net specific.
I guess it is related to your below question.Quoted 4 lines
Another question (sorry if it has been asked before). Can we simply return the config (binary) to the userspace, then usespace can use the existing uAPI like
virtio_net_configplus the feature to explain the config?
Quoted 5 lines
We did discuss in v2.
Usually returning the whole blob and parsing is not desired via netlink.
Returning individual fields give the full flexibility to return only the valid fields.
Otherwise we need to implement another bitmask too to tell which fields from the struct are valid and share with user space. Returning individual fields is the widely used approach.
The main concerns are:
1) The blob will be self contained if it was passed with the negotiated features, so we don't need bitmask.
2) Using individual fields means it must duplicate the config fields of every virtio devices
And actually, it's not the binary blob since uapi clearly define the format (e.g struct virtio_net_config), can we find a way to use that? E.g introduce device/net specific command and passing the blob with length and negotiated features.
Thanks
Quoted 1 line
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 21 lines
From: Jason Wang <[email protected]>
Sent: Wednesday, June 23, 2021 9:39 AMQuoted 8 lines · Parav Pandit
Quoted 2 lines
Is it better to use a separate enum for net specific attributes?
Yes, because they are only net specific.
I guess it is related to your below question.Quoted 3 lines
Another question (sorry if it has been asked before). Can we simply return the config (binary) to the userspace, then usespace can use the existing uAPI like
virtio_net_configplus the feature to explain theconfig?
Quoted 4 lines
Quoted 1 line
We did discuss in v2.
Usually returning the whole blob and parsing is not desired via netlink. Returning individual fields give the full flexibility to return only the validfields.
Quoted 1 line
Otherwise we need to implement another bitmask too to tell which fields
Quoted 8 lines
from the struct are valid and share with user space.
Quoted 1 line
Returning individual fields is the widely used approach.
The main concerns are:
1) The blob will be self contained if it was passed with the negotiated features, so we don't need bitmask.
Which fields of the struct are valid is told by additional fields.
Quoted 3 lines
2) Using individual fields means it must duplicate the config fields of every virtio devices
Mostly no. if there are common config fields across two device types, they would be named as VDPA_ATTR_DEV_CFG_*
Net specific will be,
VDPA_ATTR_DEV_NET_CFG_*
Block specific, will be,
VDPA_ATTR_DEV_BLK_CFG_*
Quoted 4 lines
And actually, it's not the binary blob since uapi clearly define the format (e.g struct
virtio_net_config), can we find a way to use that? E.g introduce device/net specific command and passing the blob with length and negotiated features.
Length may change in future, mostly expand. And parsing based on length is not such a clean way.
Parsing fields require knowledge of features as well and application needs to make multiple netlink calls to parse the config space. I prefer to follow rest of the kernel style to return self contained invidividual fields.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 30 lines · Parav Pandit
Quoted 18 lines
From: Jason Wang <[email protected]>
Sent: Wednesday, June 23, 2021 9:39 AMQuoted 8 lines · Parav Pandit
Quoted 2 lines
Is it better to use a separate enum for net specific attributes?
Yes, because they are only net specific.
I guess it is related to your below question.Quoted 3 lines
Another question (sorry if it has been asked before). Can we simply return the config (binary) to the userspace, then usespace can use the existing uAPI like
virtio_net_configplus the feature to explain theconfig?
Quoted 3 lines
We did discuss in v2.
Usually returning the whole blob and parsing is not desired via netlink. Returning individual fields give the full flexibility to return only the validfields.
Quoted 8 lines
Quoted 1 line
Otherwise we need to implement another bitmask too to tell which fields
from the struct are valid and share with user space.
Quoted 1 line
Returning individual fields is the widely used approach.
The main concerns are:
1) The blob will be self contained if it was passed with the negotiated features, so we don't need bitmask.
Which fields of the struct are valid is told by additional fields.
Quoted 3 lines
2) Using individual fields means it must duplicate the config fields of every virtio devices
Quoted 6 lines
Mostly no. if there are common config fields across two device types, they would be named as VDPA_ATTR_DEV_CFG_*
Net specific will be,
VDPA_ATTR_DEV_NET_CFG_*
Block specific, will be,
VDPA_ATTR_DEV_BLK_CFG_*
I meant it looks like VDPA_ATTR_DEV_NET will duplicate all the fields of:
struct virtio_net_config;
And VDPA_ATTR_DEV_BLOCK will duplicate all the fields of
struct virtio_blk_config; which has ~21 fields.
And we had a plenty of other types of virtio devices.
Consider we had a mature set of virtio specific uAPI for config space. It would be a burden if we need an unnecessary translation layer of netlink in the middle:
[vDPA parent (virtio_net_config)] <-> [netlink (VDPA_ATTR_DEV_NET_XX)] <-> [userspace (VDPA_ATTR_DEV_NET_XX)] <-> [ user (virtio_net_config)]
If we make netlink simply a transport, it would be much easier. And we had the chance to unify the logic of build_config() and set_config() in the driver.
Quoted 6 lines
Quoted 5 lines
And actually, it's not the binary blob since uapi clearly define the format (e.g struct
virtio_net_config), can we find a way to use that? E.g introduce device/net specific command and passing the blob with length and negotiated features.Length may change in future, mostly expand. And parsing based on length is not such a clean way.
Length is only for legal checking. The config is self contained with:
1) device id
2) features
Quoted 1 line
Parsing fields require knowledge of features as well and application needs to make multiple netlink calls to parse the config space.
I think we don't care about the performance in this case. It's about three netlink calls:
1) get config
2) get device id
3) get features
For build config, it's only one
1) build config
Quoted 1 line
I prefer to follow rest of the kernel style to return self contained invidividual fields.
But I saw a lot of kernel codes choose to use e.g nla_put() directly with module specific structure.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 56 lines
From: Jason Wang <[email protected]>
Sent: Thursday, June 24, 2021 11:13 AMQuoted 33 lines · Parav Pandit
Quoted 22 lines
From: Jason Wang <[email protected]>
Sent: Wednesday, June 23, 2021 9:39 AMQuoted 9 lines · Parav Pandit
Quoted 2 lines
Is it better to use a separate enum for net specific attributes?
Yes, because they are only net specific.
I guess it is related to your below question.Quoted 4 lines
Another question (sorry if it has been asked before). Can we simply return the config (binary) to the userspace, then usespace can use the existing uAPI like
virtio_net_configplus the feature to explain theconfig?
Quoted 4 lines
We did discuss in v2.
Usually returning the whole blob and parsing is not desired via netlink. Returning individual fields give the full flexibility to return only the validfields.
Quoted 2 lines
Otherwise we need to implement another bitmask too to tell which fields
Quoted 7 lines
from the struct are valid and share with user space.
Quoted 1 line
Returning individual fields is the widely used approach.
The main concerns are:
1) The blob will be self contained if it was passed with the negotiated features, so we don't need bitmask.
Which fields of the struct are valid is told by additional fields.
Quoted 3 lines
2) Using individual fields means it must duplicate the config fields of every virtio devices
Quoted 7 lines
Mostly no. if there are common config fields across two device types, they would be named as
VDPA_ATTR_DEV_CFG_*
Net specific will be,
VDPA_ATTR_DEV_NET_CFG_*
Block specific, will be,
VDPA_ATTR_DEV_BLK_CFG_*I meant it looks like
VDPA_ATTR_DEV_NETwill duplicate all the fields of:struct
virtio_net_config;And
VDPA_ATTR_DEV_BLOCKwill duplicate all the fields ofstruct
virtio_blk_config; which has ~21 fields.
Quoted 10 lines
And we had a plenty of other types of virtio devices.
Consider we had a mature set of virtio specific uAPI for config space. It would be a burden if we need an unnecessary translation layer of netlink in the middle:
[vDPA parent (
virtio_net_config)] <-> [netlink (VDPA_ATTR_DEV_NET_XX)] <-> [userspace (VDPA_ATTR_DEV_NET_XX)]Quoted 1 line
<-> [ user (
virtio_net_config)]
This translation is not there. We show relevant net config fields as VDPA_ATTR_DEV_NET individually. It is not a binary dump which is harder for users to parse and make any use of it.
It is only one level of translation from virtio_net_config (kernel) -> netlink vdpa fields. It is similar to 'struct netdevice' -> rtnl info fields.
Quoted 3 lines
If we make netlink simply a transport, it would be much easier. And we had the chance to unify the logic of
build_config()andset_config()in the driver.
How? We need bit mask to tell that out of 21 fields which fields to update and which not. And that is further mixed with offset and length.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
And actually, it's not the binary blob since uapi clearly define the format (e.g struct
virtio_net_config), can we find a way to use that? E.g introduce device/net specific command and passing the blob with length and negotiated features.
Quoted 6 lines
Quoted 1 line
Length may change in future, mostly expand. And parsing based on length
is not such a clean way.
Length is only for legal checking. The config is self contained with:
Unlikely. When structure size increases later, the parsing will change based on the length. Because older kernel would return shorter length with older iproute2 tool. So user space always have to deal and have nasty parsing/typecasting based on the length.
Quoted 11 lines
1) device id
2) featuresQuoted 1 line
Parsing fields require knowledge of features as well and application needs
to make multiple netlink calls to parse the config space.
I think we don't care about the performance in this case. It's about three netlink calls:
Its not about performance. By the time 1st call is made, features got updated and it is out of sync with config.
Quoted 4 lines
1) get config
2) get device id
3) get features
This requires using features from 3rd netlink output to decode output of 1st netlink output. Which is a bit odd of netlink.
Other netlink nla_put() probably sending whole structure doesn’t need to do it.
Quoted 5 lines
For build config, it's only one
1) build config
Quoted 7 lines
Quoted 1 line
I prefer to follow rest of the kernel style to return self contained
invidividual fields.
But I saw a lot of kernel codes choose to use e.g
nla_put()directly with module specific structure.
It might be self-contained structure that probably has not found the need to expand.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 58 lines · Parav Pandit
Quoted 58 lines
From: Jason Wang <[email protected]>
Sent: Thursday, June 24, 2021 11:13 AMQuoted 31 lines · Parav Pandit
Quoted 19 lines
From: Jason Wang <[email protected]>
Sent: Wednesday, June 23, 2021 9:39 AMQuoted 9 lines · Parav Pandit
Quoted 2 lines
Is it better to use a separate enum for net specific attributes?
Yes, because they are only net specific.
I guess it is related to your below question.Quoted 4 lines
Another question (sorry if it has been asked before). Can we simply return the config (binary) to the userspace, then usespace can use the existing uAPI like
virtio_net_configplus the feature to explain theconfig?
Quoted 4 lines
We did discuss in v2.
Usually returning the whole blob and parsing is not desired via netlink. Returning individual fields give the full flexibility to return only the validfields.
Quoted 8 lines
Quoted 2 lines
Otherwise we need to implement another bitmask too to tell which fields
from the struct are valid and share with user space.
Quoted 1 line
Returning individual fields is the widely used approach.
The main concerns are:
1) The blob will be self contained if it was passed with the negotiated features, so we don't need bitmask.
Which fields of the struct are valid is told by additional fields.
Quoted 3 lines
2) Using individual fields means it must duplicate the config fields of every virtio devices
Quoted 7 lines
Mostly no. if there are common config fields across two device types, they would be named as
VDPA_ATTR_DEV_CFG_*
Net specific will be,
VDPA_ATTR_DEV_NET_CFG_*
Block specific, will be,
VDPA_ATTR_DEV_BLK_CFG_*I meant it looks like
VDPA_ATTR_DEV_NETwill duplicate all the fields of:struct
virtio_net_config;And
VDPA_ATTR_DEV_BLOCKwill duplicate all the fields ofstruct
virtio_blk_config; which has ~21 fields.And we had a plenty of other types of virtio devices.
Consider we had a mature set of virtio specific uAPI for config space. It would be a burden if we need an unnecessary translation layer of netlink in the middle:
Quoted 5 lines
Quoted 3 lines
[vDPA parent (
virtio_net_config)] <-> [netlink (VDPA_ATTR_DEV_NET_XX)] <-> [userspace (VDPA_ATTR_DEV_NET_XX)]Quoted 1 line
<-> [ user (
virtio_net_config)]This translation is not there. We show relevant net config fields as
VDPA_ATTR_DEV_NETindividually. It is not a binary dump which is harder for users to parse and make any use of it.
The is done implicitly, user needs to understand the semantic of virtio_net_config and map the individual fields to the vdpa tool sub-command.
Quoted 3 lines
It is only one level of translation from
virtio_net_config(kernel) -> netlink vdpa fields. It is similar to 'struct netdevice' -> rtnl info fields.
I think not, the problem is that the netdevice is not a part of uAPI but virtio_net_config is.
Quoted 5 lines
Quoted 3 lines
If we make netlink simply a transport, it would be much easier. And we had the chance to unify the logic of
build_config()andset_config()in the driver.How? We need bit mask to tell that out of 21 fields which fields to update and which not. And that is further mixed with offset and length.
So set_config() could be called from userspace, so did build_config(). The only difference is:
1) they're using different transport, ioctl vs netlink
2) build_config() is only expected to be called by the management tool
If qemu works well via set_config ioctl, netlink should work as well.
Btw, what happens if management tool tries to modify the mac of vDPA when the device is already used by the driver?
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
And actually, it's not the binary blob since uapi clearly define the format (e.g struct
virtio_net_config), can we find a way to use that? E.g introduce device/net specific command and passing the blob with length and negotiated features.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Length may change in future, mostly expand. And parsing based on length
is not such a clean way.
Length is only for legal checking. The config is self contained with:
Unlikely. When structure size increases later, the parsing will change based on the length. Because older kernel would return shorter length with older iproute2 tool.
This is fine since the older kernel only support less features. The only possible issue if the old iproute 2 runs on new kernel. With the current proposal, it may cause some config fields can't not be showed.
I think it might be useful to introduce a command to simply dump the config space.
Quoted 1 line
So user space always have to deal and have nasty parsing/typecasting based on the length.
That's how userspace (Qemu) is expected to work now. The userspace should determine the semantic of the fields based on the features.
Differentiate config fields doesn't help much, e.g userspace still need to differ LINK_UP and ANNOUNCE for the status field.
Quoted 14 lines
Quoted 5 lines
1) device id
2) featuresQuoted 7 lines
Quoted 1 line
Parsing fields require knowledge of features as well and application needs
to make multiple netlink calls to parse the config space.
I think we don't care about the performance in this case. It's about three netlink calls:
Its not about performance. By the time 1st call is made, features got updated and it is out of sync with config.
Quoted 7 lines
Quoted 4 lines
1) get config
2) get device id
3) get featuresThis requires using features from 3rd netlink output to decode output of 1st netlink output. Which is a bit odd of netlink.
Other netlinknla_put()probably sending whole structure doesn’t need to do it.
Well, we can pack them all into a single skb isn't it? (probably with a config len).
Quoted 6 lines
Quoted 6 lines
For build config, it's only one
1) build config
Quoted 8 lines
Quoted 7 lines
Quoted 1 line
I prefer to follow rest of the kernel style to return self contained
invidividual fields.
But I saw a lot of kernel codes choose to use e.g
nla_put()directly with module specific structure.It might be self-contained structure that probably has not found the need to expand.
I think it's just a matter of putting the config length with the config data. Note that we've already had .get_config_size() ops for validating inputs through VHOST_SET_CONFIG/VHOST_GET_CONFIG.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 14 lines
From: Jason Wang <[email protected]>
Sent: Thursday, June 24, 2021 12:35 PMQuoted 6 lines
Quoted 6 lines
Consider we had a mature set of virtio specific uAPI for config space. It would be a burden if we need an unnecessary translation layer of netlink in the middle:
[vDPA parent (
virtio_net_config)] <-> [netlink
(VDPA_ATTR_DEV_NET_XX)] <-> [userspace(
VDPA_ATTR_DEV_NET_XX)]Quoted 2 lines
Quoted 1 line
Quoted 1 line
<-> [ user (
virtio_net_config)]This translation is not there. We show relevant net config fields as
Quoted 8 lines
VDPA_ATTR_DEV_NETindividually.Quoted 1 line
It is not a binary dump which is harder for users to parse and make any use
of it.
The is done implicitly, user needs to understand the semantic of
virtio_net_configand map the individual fields to the vdpa tool sub- command.
Mostly not virtio_net_config is for the producer and consumer sw entities. Here user doesn't know about such layout and where its located. User only sets config params that gets set in the config space. (without understanding what is config layout and its location).
Quoted 10 lines
Quoted 4 lines
It is only one level of translation from
virtio_net_config(kernel) -> netlinkvdpa fields.
Quoted 1 line
It is similar to 'struct netdevice' -> rtnl info fields.
I think not, the problem is that the netdevice is not a part of uAPI but
virtio_net_configis.
Virtio_net_config is a UAPI for sw consumption.
That way yes, netlink can also do it, however it requires side channel communicate what is valid.
Quoted 12 lines
Quoted 4 lines
Quoted 4 lines
If we make netlink simply a transport, it would be much easier. And we
had
Quoted 1 line
Quoted 1 line
the chance to unify the logic of
build_config()andset_config()in thedriver.
Quoted 1 line
How? We need bit mask to tell that out of 21 fields which fields to update
and which not.
Quoted 1 line
And that is further mixed with offset and length.
Quoted 8 lines
So
set_config()could be called from userspace, so didbuild_config(). The only difference is:1) they're using different transport, ioctl vs netlink
2)build_config()is only expected to be called by the management toolIf qemu works well via
set_configioctl, netlink should work as well.
mlx5 set_config is noop.vdpa_set_config() need to return an error code. I don'tvp_vdpa.c blindly writes the config as its passthrough.
Parsing which fields to write and which not, using offset and length is a messy code with typecast and compare old values etc.
Quoted 2 lines
Btw, what happens if management tool tries to modify the mac of vDPA when the device is already used by the driver?
At present it allows modifying, but it should be improved in future to fail if device is in use.
Quoted 11 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
And actually, it's not the binary blob since uapi clearly define the format (e.g struct
virtio_net_config), can we find a way to use that? E.g introduce device/net specific command and passing the blob with length and negotiated features.Length may change in future, mostly expand. And parsing based on
length
Quoted 5 lines
Quoted 5 lines
is not such a clean way.
Length is only for legal checking. The config is self contained with:
Quoted 9 lines
Quoted 1 line
Unlikely. When structure size increases later, the parsing will change based
on the length.
Quoted 1 line
Because older kernel would return shorter length with older iproute2 tool.
This is fine since the older kernel only support less features. The only possible issue if the old iproute 2 runs on new kernel. With the current proposal, it may cause some config fields can't not be showed.
Not showing is ok.
But the code is messy to typecast on size.
Quoted 6 lines
I think it might be useful to introduce a command to simply dump the config space.
Quoted 1 line
So user space always have to deal and have nasty parsing/typecasting
based on the length.
Such nasty parsing is not required for netlink interface.
Quoted 7 lines
That's how userspace (Qemu) is expected to work now. The userspace should determine the semantic of the fields based on the features.
Differentiate config fields doesn't help much, e.g userspace still need to differ
LINK_UPand ANNOUNCE for the status field.
Yes, this parsing is from constant size u16 status.
Quoted 2 lines
[..]
Quoted 9 lines
Quoted 2 lines
Its not about performance. By the time 1st call is made, features got
updated and it is out of sync with config.
Quoted 6 lines
Quoted 5 lines
1) get config
2) get device id
3) get featuresThis requires using features from 3rd netlink output to decode output of
Quoted 9 lines
1st netlink output.
Quoted 2 lines
Which is a bit odd of netlink.
Other netlinknla_put()probably sending whole structure doesn’t need todo it.
Well, we can pack them all into a single skb isn't it? (probably with a config len).
You want to pack features and config both in the single nla_put()?
If so, it isn't necessary. There are more examples in kernel that adds individual fields instead of nla_put(blob). I wouldn’t follow those nla_put() callers.
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
For build config, it's only one
1) build config
Quoted 1 line
I prefer to follow rest of the kernel style to return self contained
invidividual fields.
But I saw a lot of kernel codes choose to use e.g
nla_put()directly with module specific structure.
Quoted 7 lines
Quoted 1 line
It might be self-contained structure that probably has not found the need
to expand.
I think it's just a matter of putting the config length with the config data. Note that we've already had .get_config_size() ops for validating inputs through VHOST_SET_CONFIG/VHOST_GET_CONFIG.
This length comes as part of the netlink interface already, no need for extra length.
The whole point is to avoid parsing based on length.
We cannot change the virtio_net_config UAPI in use, but netlink code doesn’t need to be bound to size based typecasting and compare fields during build_config().
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 51 lines · Parav Pandit
Quoted 13 lines
From: Jason Wang <[email protected]>
Sent: Thursday, June 24, 2021 12:35 PMQuoted 6 lines
Quoted 6 lines
Consider we had a mature set of virtio specific uAPI for config space. It would be a burden if we need an unnecessary translation layer of netlink in the middle:
[vDPA parent (
virtio_net_config)] <-> [netlink
(VDPA_ATTR_DEV_NET_XX)] <-> [userspace(
VDPA_ATTR_DEV_NET_XX)]Quoted 2 lines
Quoted 1 line
Quoted 1 line
<-> [ user (
virtio_net_config)]This translation is not there. We show relevant net config fields as
Quoted 8 lines
VDPA_ATTR_DEV_NETindividually.Quoted 1 line
It is not a binary dump which is harder for users to parse and make any use
of it.
The is done implicitly, user needs to understand the semantic of
virtio_net_configand map the individual fields to the vdpa tool sub- command.Mostly not
virtio_net_configis for the producer and consumer sw entities. Here user doesn't know about such layout and where its located. User only sets config params that gets set in the config space. (without understanding what is config layout and its location).Quoted 7 lines
Quoted 2 lines
It is only one level of translation from
virtio_net_config(kernel) -> netlinkvdpa fields.
Quoted 1 line
It is similar to 'struct netdevice' -> rtnl info fields.
I think not, the problem is that the netdevice is not a part of uAPI but
virtio_net_configis.
Virtio_net_configis a UAPI for sw consumption.
That way yes, netlink can also do it, however it requires side channel communicate what is valid.Quoted 15 lines
Quoted 2 lines
Quoted 2 lines
If we make netlink simply a transport, it would be much easier. And we
had
Quoted 1 line
Quoted 1 line
the chance to unify the logic of
build_config()andset_config()in thedriver.
Quoted 1 line
How? We need bit mask to tell that out of 21 fields which fields to update
and which not.
Quoted 1 line
And that is further mixed with offset and length.
So
set_config()could be called from userspace, so didbuild_config(). The only difference is:1) they're using different transport, ioctl vs netlink
2)build_config()is only expected to be called by the management tool
Quoted 6 lines
Quoted 2 lines
If qemu works well via
set_configioctl, netlink should work as well.mlx5
set_configis noop.vdpa_set_config()need to return an error code. I don'tvp_vdpa.cblindly writes the config as its passthrough.
Parsing which fields to write and which not, using offset and length is a messy code with typecast and compare old values etc.
I don't see why it needs typecast, virtio_net_config is also uABI, you can deference the fields directly.
Quoted 4 lines
Quoted 3 lines
Btw, what happens if management tool tries to modify the mac of vDPA when the device is already used by the driver?
At present it allows modifying, but it should be improved in future to fail if device is in use.
This is something we need to fix I think. Or if it's really useful to allowing the attributes to be modified after the device is created.
Why not simply allow the config to be built only at device creation?
Quoted 54 lines
Quoted 12 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
And actually, it's not the binary blob since uapi clearly define the format (e.g struct
virtio_net_config), can we find a way to use that? E.g introduce device/net specific command and passing the blob with length and negotiated features.Length may change in future, mostly expand. And parsing based on
length
Quoted 5 lines
Quoted 5 lines
is not such a clean way.
Length is only for legal checking. The config is self contained with:
Quoted 8 lines
Quoted 1 line
Unlikely. When structure size increases later, the parsing will change based
on the length.
Quoted 1 line
Because older kernel would return shorter length with older iproute2 tool.
This is fine since the older kernel only support less features. The only possible issue if the old iproute 2 runs on new kernel. With the current proposal, it may cause some config fields can't not be showed.
Not showing is ok.
But the code is messy to typecast on size.Quoted 6 lines
I think it might be useful to introduce a command to simply dump the config space.
Quoted 1 line
So user space always have to deal and have nasty parsing/typecasting
based on the length.
Such nasty parsing is not required for netlink interface.
Quoted 6 lines
That's how userspace (Qemu) is expected to work now. The userspace should determine the semantic of the fields based on the features.
Differentiate config fields doesn't help much, e.g userspace still need to differ
LINK_UPand ANNOUNCE for the status field.Yes, this parsing is from constant size u16 status.
Quoted 1 line
[..]
Quoted 13 lines
Quoted 1 line
Its not about performance. By the time 1st call is made, features got
updated and it is out of sync with config.
Quoted 5 lines
Quoted 4 lines
1) get config
2) get device id
3) get featuresThis requires using features from 3rd netlink output to decode output of
1st netlink output.
Quoted 2 lines
Which is a bit odd of netlink.
Other netlinknla_put()probably sending whole structure doesn’t need todo it.
Quoted 6 lines
Quoted 3 lines
Well, we can pack them all into a single skb isn't it? (probably with a config len).
You want to pack features and config both in the single
nla_put()?
If so, it isn't necessary. There are more examples in kernel that adds individual fields instead ofnla_put(blob). I wouldn’t follow thosenla_put()callers.
No, a single skb not single nla_put().
Actually git grep told me a very good example of carrying uABI via netlink, that is the ndt_config:
1) we had ndt_config definition in the uAPI
2) netlink simply carries the structure in neightbl_fill_info():
if (nla_put(skb, NDTA_CONFIG, sizeof(ndc), &ndc))
For virito_net_config, why not simply:
len = ops->get_config_len();
config = kmalloc(len, GFP_KERNEL); ops->get_config(vdev, 0, config, len); nla_put(skb, VIRTIO_CONFIG, config, len); nla_put_le64(skb, VIRTIO_FETURES, features);
For build_config, we can simply do thing reversely. Then everything works via the existing virtio uAPI/ABI.
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 6 lines
For build config, it's only one
1) build config
Quoted 7 lines
Quoted 1 line
I prefer to follow rest of the kernel style to return self contained
invidividual fields.
But I saw a lot of kernel codes choose to use e.g
nla_put()directly with module specific structure.It might be self-contained structure that probably has not found the need
Quoted 8 lines
Quoted 6 lines
to expand.
I think it's just a matter of putting the config length with the config data. Note that we've already had .get_config_size() ops for validating inputs through VHOST_SET_CONFIG/VHOST_GET_CONFIG.
This length comes as part of the netlink interface already, no need for extra length. The whole point is to avoid parsing based on length.
Well, it doesn't do anything difference compared to xxx_is_valid which just calculating the offset implicitly (via the compiler).
Quoted 1 line
We cannot change the
virtio_net_configUAPI in use, but netlink code doesn’t need to be bound to size based typecasting and compare fields duringbuild_config().
The points are:
1) Avoid duplicating the existing uAPIs
2) Avoid unnecessary parsing in the netlink, netlink is just the transport, it's the charge of the vDPA parent to do that
Thanks
_______________________________________________
Virtualization mailing list [email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 70 lines
From: Jason Wang <[email protected]>
Sent: Friday, June 25, 2021 8:59 AMQuoted 30 lines · Parav Pandit
Quoted 14 lines
From: Jason Wang <[email protected]>
Sent: Thursday, June 24, 2021 12:35 PMQuoted 6 lines
Quoted 6 lines
Consider we had a mature set of virtio specific uAPI for config space. It would be a burden if we need an unnecessary translation layer of netlink in the middle:
[vDPA parent (
virtio_net_config)] <-> [netlink
(VDPA_ATTR_DEV_NET_XX)] <-> [userspace(
VDPA_ATTR_DEV_NET_XX)]Quoted 2 lines
Quoted 1 line
Quoted 1 line
<-> [ user (
virtio_net_config)]This translation is not there. We show relevant net config fields as
VDPA_ATTR_DEV_NETindividually.Quoted 8 lines
Quoted 2 lines
It is not a binary dump which is harder for users to parse and make any use
of it.
The is done implicitly, user needs to understand the semantic of
virtio_net_configand map the individual fields to the vdpa tool sub- command.Mostly not
virtio_net_configis for the producer and consumer sw entities. Here user doesn't know about such layout and where its located. User only sets config params that gets set in the config space. (without understanding what is config layout and its location).Quoted 3 lines
Quoted 3 lines
It is only one level of translation from
virtio_net_config(kernel) -> netlinkQuoted 7 lines
Quoted 5 lines
vdpa fields.
Quoted 1 line
It is similar to 'struct netdevice' -> rtnl info fields.
I think not, the problem is that the netdevice is not a part of uAPI but
virtio_net_configis.
Virtio_net_configis a UAPI for sw consumption.
That way yes, netlink can also do it, however it requires side channelcommunicate what is valid.
Quoted 20 lines
Quoted 20 lines
Quoted 4 lines
Quoted 4 lines
If we make netlink simply a transport, it would be much easier. And we
had
Quoted 2 lines
Quoted 2 lines
the chance to unify the logic of
build_config()andset_config()in thedriver.
Quoted 2 lines
How? We need bit mask to tell that out of 21 fields which fields to update
and which not.
Quoted 1 line
And that is further mixed with offset and length.
So
set_config()could be called from userspace, so didbuild_config(). The only difference is:1) they're using different transport, ioctl vs netlink
2)build_config()is only expected to be called by the management toolQuoted 6 lines
Quoted 2 lines
If qemu works well via
set_configioctl, netlink should work as well.mlx5
set_configis noop.vdpa_set_config()need to return an error code. I don'tvp_vdpa.cblindly writes the config as its passthrough.
Parsing which fields to write and which not, using offset and length is a
Quoted 6 lines
messy code with typecast and compare old values etc.
I don't see why it needs typecast,
virtio_net_configis also uABI, you can deference the fields directly.
User wants set only the mac address of the config space. How do user space tell this? Pass the whole virtio_net_config and inform via side channel?
Or vendor driver is expected to compare what fields changed from old config space?
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Btw, what happens if management tool tries to modify the mac of vDPA when the device is already used by the driver?
Quoted 9 lines
Quoted 1 line
At present it allows modifying, but it should be improved in future to fail if
device is in use.
This is something we need to fix I think. Or if it's really useful to allowing the attributes to be modified after the device is created.
Why not simply allow the config to be built only at device creation?
That avoids the problem of modifying fields after bind to vhost. But UAPI issue still remains so lets resolve that first.
Quoted 69 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
And actually, it's not the binary blob since uapi clearly define the format (e.g struct
virtio_net_config), can we find a way to use that? E.g introduce device/net specific command and passing the blob with length and negotiated features.Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Length may change in future, mostly expand. And parsing based on
length
Quoted 6 lines
Quoted 5 lines
is not such a clean way.
Length is only for legal checking. The config is self contained with:
Unlikely. When structure size increases later, the parsing will change
based
Quoted 2 lines
Quoted 2 lines
on the length.
Quoted 1 line
Because older kernel would return shorter length with older iproute2
tool.
Quoted 17 lines
Quoted 5 lines
This is fine since the older kernel only support less features. The only possible issue if the old iproute 2 runs on new kernel. With the current proposal, it may cause some config fields can't not be showed.
Not showing is ok.
But the code is messy to typecast on size.Quoted 6 lines
I think it might be useful to introduce a command to simply dump the config space.
Quoted 1 line
So user space always have to deal and have nasty parsing/typecasting
based on the length.
Such nasty parsing is not required for netlink interface.
Quoted 1 line
Quoted 19 lines
Quoted 5 lines
That's how userspace (Qemu) is expected to work now. The userspace should determine the semantic of the fields based on the features.
Differentiate config fields doesn't help much, e.g userspace still need to differ
LINK_UPand ANNOUNCE for the status field.Yes, this parsing is from constant size u16 status.
Quoted 1 line
[..]
Quoted 10 lines
Quoted 1 line
Its not about performance. By the time 1st call is made, features got
updated and it is out of sync with config.
Quoted 5 lines
Quoted 4 lines
1) get config
2) get device id
3) get featuresThis requires using features from 3rd netlink output to decode output of
1st netlink output.
Quoted 2 lines
Which is a bit odd of netlink.
Other netlinknla_put()probably sending whole structure doesn’t needto
Quoted 8 lines
Quoted 6 lines
do it.
Well, we can pack them all into a single skb isn't it? (probably with a config len).
You want to pack features and config both in the single
nla_put()? If so, it isn't necessary. There are more examples in kernel that addsindividual fields instead of
nla_put(blob).Quoted 1 line
I wouldn’t follow those
nla_put()callers.No, a single skb not single
nla_put().
Quoted 8 lines
Actually git grep told me a very good example of carrying uABI via netlink, that is the
ndt_config:1) we had
ndt_configdefinition in the uAPI
2) netlink simply carries the structure inneightbl_fill_info():if (
nla_put(skb, NDTA_CONFIG, sizeof(ndc), &ndc))
Sure. But the reverse path doesn’t have this that requires side band mask.
My concern is not for existing virtio_net_config layout, but the future increase of it requires size based typecasting on both directions.
Quoted 7 lines
For
virito_net_config, why not simply:len =
ops->get_config_len();
config =kmalloc(len, GFP_KERNEL);ops->get_config(vdev, 0, config, len);nla_put(skb, VIRTIO_CONFIG, config, len);
User space need to parse content based on this length as it can change in future. Length telling how to typecast is want I want to avoid here.
Quoted 6 lines
nla_put_le64(skb, VIRTIO_FETURES, features);For
build_config, we can simply do thing reversely. Then everything works via the existing virtio uAPI/ABI.
In reverse path how do you tell which fields of the config space to set and which to ignore? Shall we use u64 features for it?
Will type of device able to describe their config space via a feature bit?
Quoted 36 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
For build config, it's only one
1) build config
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 1 line
I prefer to follow rest of the kernel style to return self contained
invidividual fields.
But I saw a lot of kernel codes choose to use e.g
nla_put()directly with module specific structure.It might be self-contained structure that probably has not found the
need
Quoted 7 lines
Quoted 6 lines
to expand.
I think it's just a matter of putting the config length with the config data. Note that we've already had .get_config_size() ops for validating inputs through VHOST_SET_CONFIG/VHOST_GET_CONFIG.
This length comes as part of the netlink interface already, no need for extra
length.
Quoted 1 line
The whole point is to avoid parsing based on length.
Well, it doesn't do anything difference compared to
xxx_is_validwhich just calculating the offset implicitly (via the compiler).Quoted 1 line
We cannot change the
virtio_net_configUAPI in use, but netlink codedoesn’t need to be bound to size based typecasting and compare fields during
build_config().
Quoted 6 lines
The points are:
1) Avoid duplicating the existing uAPIs
2) Avoid unnecessary parsing in the netlink, netlink is just the transport, it's the charge of the vDPA parent to do that
All those parsing will move to vendor drivers to validate offset/length to update only specific fields of config space. It is a transport to carry fields which is what we are using for.
I agree there that these config fields are exposed individually in both directions to keep it safe from structure layout increments.
Quoted 2 lines
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 63 lines · Parav Pandit
Quoted 63 lines
From: Jason Wang <[email protected]>
Sent: Friday, June 25, 2021 8:59 AMQuoted 26 lines · Parav Pandit
Quoted 13 lines
From: Jason Wang <[email protected]>
Sent: Thursday, June 24, 2021 12:35 PMQuoted 6 lines
Quoted 6 lines
Consider we had a mature set of virtio specific uAPI for config space. It would be a burden if we need an unnecessary translation layer of netlink in the middle:
[vDPA parent (
virtio_net_config)] <-> [netlink
(VDPA_ATTR_DEV_NET_XX)] <-> [userspace(
VDPA_ATTR_DEV_NET_XX)]Quoted 2 lines
Quoted 1 line
Quoted 1 line
<-> [ user (
virtio_net_config)]This translation is not there. We show relevant net config fields as
VDPA_ATTR_DEV_NETindividually.Quoted 8 lines
Quoted 2 lines
It is not a binary dump which is harder for users to parse and make any use
of it.
The is done implicitly, user needs to understand the semantic of
virtio_net_configand map the individual fields to the vdpa tool sub- command.Mostly not
virtio_net_configis for the producer and consumer sw entities. Here user doesn't know about such layout and where its located. User only sets config params that gets set in the config space. (without understanding what is config layout and its location).Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
It is only one level of translation from
virtio_net_config(kernel) -> netlinkvdpa fields.
Quoted 1 line
It is similar to 'struct netdevice' -> rtnl info fields.
I think not, the problem is that the netdevice is not a part of uAPI but
virtio_net_configis.
Virtio_net_configis a UAPI for sw consumption.
That way yes, netlink can also do it, however it requires side channelcommunicate what is valid.
Quoted 23 lines
Quoted 10 lines
Quoted 2 lines
Quoted 2 lines
If we make netlink simply a transport, it would be much easier. And we
had
Quoted 2 lines
Quoted 2 lines
the chance to unify the logic of
build_config()andset_config()in thedriver.
Quoted 2 lines
How? We need bit mask to tell that out of 21 fields which fields to update
and which not.
Quoted 1 line
And that is further mixed with offset and length.
Quoted 9 lines
So
set_config()could be called from userspace, so didbuild_config(). The only difference is:1) they're using different transport, ioctl vs netlink
2)build_config()is only expected to be called by the management toolIf qemu works well via
set_configioctl, netlink should work as well.mlx5
set_configis noop.vdpa_set_config()need to return an error code. I don'tvp_vdpa.cblindly writes the config as its passthrough.
Parsing which fields to write and which not, using offset and length is a
Quoted 7 lines
Quoted 6 lines
messy code with typecast and compare old values etc.
I don't see why it needs typecast,
virtio_net_configis also uABI, you can deference the fields directly.User wants set only the mac address of the config space. How do user space tell this?
Good question, but we need first answer:
"Do we allow userspace space to modify one specific field of all the config?"
Quoted 1 line
Pass the whole
virtio_net_configand inform via side channel?
That could be a method.
Quoted 1 line
Or vendor driver is expected to compare what fields changed from old config space?
So I think we need solve them all, but netlink is probably the wrong layer, we need to solve them at virtio level and let netlink a transport for them virtio uAPI/ABI.
And we need to figure out if we want to allow the userspace to modify the config after the device is created. If not, simply build the virtio_net_config and pass it to the vDPA parent during device creation. If not, invent new uAPI at virtio level to passing the config fields. Virtio or vDPA core can provide the library to compare the difference.
My feeling is that, if we restrict to only support build the config during the creation, it would simply a lot of things. And I didn't notice a use case that we need to change the config fields in the middle via the management API/tool.
Quoted 89 lines
Quoted 12 lines
Quoted 4 lines
Quoted 3 lines
Btw, what happens if management tool tries to modify the mac of vDPA when the device is already used by the driver?
At present it allows modifying, but it should be improved in future to fail if
device is in use.
This is something we need to fix I think. Or if it's really useful to allowing the attributes to be modified after the device is created.
Why not simply allow the config to be built only at device creation?
That avoids the problem of modifying fields after bind to vhost. But UAPI issue still remains so lets resolve that first.
Quoted 63 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
And actually, it's not the binary blob since uapi clearly define the format (e.g struct
virtio_net_config), can we find a way to use that? E.g introduce device/net specific command and passing the blob with length and negotiated features.Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Length may change in future, mostly expand. And parsing based on
length
Quoted 6 lines
Quoted 5 lines
is not such a clean way.
Length is only for legal checking. The config is self contained with:
Unlikely. When structure size increases later, the parsing will change
based
Quoted 2 lines
Quoted 2 lines
on the length.
Quoted 1 line
Because older kernel would return shorter length with older iproute2
tool.
Quoted 15 lines
Quoted 4 lines
This is fine since the older kernel only support less features. The only possible issue if the old iproute 2 runs on new kernel. With the current proposal, it may cause some config fields can't not be showed.
Not showing is ok.
But the code is messy to typecast on size.Quoted 6 lines
I think it might be useful to introduce a command to simply dump the config space.
Quoted 1 line
So user space always have to deal and have nasty parsing/typecasting
based on the length.
Such nasty parsing is not required for netlink interface.
Quoted 18 lines
Quoted 5 lines
That's how userspace (Qemu) is expected to work now. The userspace should determine the semantic of the fields based on the features.
Differentiate config fields doesn't help much, e.g userspace still need to differ
LINK_UPand ANNOUNCE for the status field.Yes, this parsing is from constant size u16 status. [..]
Quoted 10 lines
Quoted 1 line
Its not about performance. By the time 1st call is made, features got
updated and it is out of sync with config.
Quoted 5 lines
Quoted 4 lines
1) get config
2) get device id
3) get featuresThis requires using features from 3rd netlink output to decode output of
1st netlink output.
Quoted 2 lines
Which is a bit odd of netlink.
Other netlinknla_put()probably sending whole structure doesn’t needto
Quoted 8 lines
Quoted 6 lines
do it.
Well, we can pack them all into a single skb isn't it? (probably with a config len).
You want to pack features and config both in the single
nla_put()? If so, it isn't necessary. There are more examples in kernel that addsindividual fields instead of
nla_put(blob).Quoted 1 line
I wouldn’t follow those
nla_put()callers.No, a single skb not single
nla_put().Quoted 8 lines
Actually git grep told me a very good example of carrying uABI via netlink, that is the
ndt_config:1) we had
ndt_configdefinition in the uAPI
2) netlink simply carries the structure inneightbl_fill_info():if (
nla_put(skb, NDTA_CONFIG, sizeof(ndc), &ndc))Sure. But the reverse path doesn’t have this that requires side band mask.
My concern is not for existingvirtio_net_configlayout, but the future increase of it requires size based typecasting on both directions.
Quoted 8 lines
Quoted 6 lines
For
virito_net_config, why not simply:len =
ops->get_config_len();
config =kmalloc(len, GFP_KERNEL);ops->get_config(vdev, 0, config, len);nla_put(skb, VIRTIO_CONFIG, config, len);User space need to parse content based on this length as it can change in future. Length telling how to typecast is want I want to avoid here.
So there's no real difference, using xxx_is_valid, is just a implicit length checking as what is done via config_len:
if (a_is_valid) {
/* dump a */
} else if (b_is_valid) {
/* dump b */
}
vs.
if (length < offsetof(struct virtio_net_config, next field of a)) { /* dump a*/
}
Actually, Qemu has solved the similar issues via the uAPI:
If the current uAPI is not sufficient, let's tweak it.
Quoted 8 lines
Quoted 7 lines
nla_put_le64(skb, VIRTIO_FETURES, features);For
build_config, we can simply do thing reversely. Then everything works via the existing virtio uAPI/ABI.In reverse path how do you tell which fields of the config space to set and which to ignore?
See my above reply.
Quoted 2 lines
Shall we use u64 features for it?
Will type of device able to describe their config space via a feature bit?
I think not. They're a lot of fields can not be deduced from the features (mtu, queue paris, mac etc).
But I agree the the config fields can not work without the feature bits.
Quoted 34 lines
Quoted 34 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
For build config, it's only one
1) build config
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 1 line
I prefer to follow rest of the kernel style to return self contained
invidividual fields.
But I saw a lot of kernel codes choose to use e.g
nla_put()directly with module specific structure.It might be self-contained structure that probably has not found the
need
Quoted 7 lines
Quoted 6 lines
to expand.
I think it's just a matter of putting the config length with the config data. Note that we've already had .get_config_size() ops for validating inputs through VHOST_SET_CONFIG/VHOST_GET_CONFIG.
This length comes as part of the netlink interface already, no need for extra
length.
Quoted 1 line
The whole point is to avoid parsing based on length.
Well, it doesn't do anything difference compared to
xxx_is_validwhich just calculating the offset implicitly (via the compiler).Quoted 1 line
We cannot change the
virtio_net_configUAPI in use, but netlink codedoesn’t need to be bound to size based typecasting and compare fields during
build_config().
Quoted 7 lines
Quoted 6 lines
The points are:
1) Avoid duplicating the existing uAPIs
2) Avoid unnecessary parsing in the netlink, netlink is just the transport, it's the charge of the vDPA parent to do thatAll those parsing will move to vendor drivers to validate offset/length to update only specific fields of config space.
Or the vDPA or virtio core can provide helpers to compare the difference if it's necessary.
Thanks
Quoted 5 lines
It is a transport to carry fields which is what we are using for.
I agree there that these config fields are exposed individually in both directions to keep it safe from structure layout increments.Quoted 2 lines
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 4 lines
From: Jason Wang <[email protected]>
Sent: Monday, June 28, 2021 10:33 AM
[..]
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
I don't see why it needs typecast,
virtio_net_configis also uABI, you can deference the fields directly.
Quoted 8 lines
Quoted 1 line
User wants set only the mac address of the config space. How do user
space tell this?
Good question, but we need first answer:
"Do we allow userspace space to modify one specific field of all the config?"
Even if we restrict to specify config params at creation time, question still remains open how to pass, either as whole struct + side_based info or as individual fields. More below.
Quoted 5 lines
Quoted 2 lines
Pass the whole
virtio_net_configand inform via side channel?That could be a method.
I prefer the method to pass individual fields which has the clean code approach and full flexibility. Clean code =
1. no typecasting based on length
2. self-describing fields, do not depends on feature bits parsing
3. proof against structure size increases in fully backward/forward compatibility without code changes
Quoted 9 lines
Quoted 3 lines
Or vendor driver is expected to compare what fields changed from old
config space?
So I think we need solve them all, but netlink is probably the wrong layer, we need to solve them at virtio level and let netlink a transport for them virtio uAPI/ABI.
In spirit of using the virtio UAPI structure, we creating other side band fields, that results into code that’s not common to netlink method. Ioctl() interface of QEMU/vhost didn't have any other choice with ioctl().
Quoted 4 lines
And we need to figure out if we want to allow the userspace to modify the config after the device is created. If not, simply build the
virtio_net_configand pass it to the vDPA parent during device creation.
I like this idea to pass fields at creation time.
Quoted 9 lines
If not, invent new uAPI at virtio level to passing the config fields. Virtio or vDPA core can provide the library to compare the difference.
My feeling is that, if we restrict to only support build the config during the creation, it would simply a lot of things. And I didn't notice a use case that we need to change the config fields in the middle via the management API/tool.
Sure yes. Whichever config fields user wants to pass, user space passes it.
Quoted 20 lines
Quoted 7 lines
Quoted 6 lines
For
virito_net_config, why not simply:len =
ops->get_config_len();
config =kmalloc(len, GFP_KERNEL);ops->get_config(vdev, 0, config, len);nla_put(skb, VIRTIO_CONFIG, config, len);User space need to parse content based on this length as it can change in
future.
Quoted 1 line
Length telling how to typecast is want I want to avoid here.
So there's no real difference, using
xxx_is_valid, is just a implicit length checking as what is done viaconfig_len:if (
a_is_valid) {
/* dump a */
} else if (b_is_valid) {
/* dump b */
}
Quoted 4 lines
vs.
if (length <
offsetof(struct virtio_net_config, next field of a)) {
/* dump a*/
+ the feature parsing code, for each field.
Quoted 8 lines
}
Actually, Qemu has solved the similar issues via the uAPI:
https://git.qemu.org/?p=qemu.git;a=blob;f=hw/net/virtio-
net.c;h=bd7958b9f0eed2705e0d6a2feaeaefb5e63bd6a4;hb=HEAD#l92If the current uAPI is not sufficient, let's tweak it.
I am unable to convince my self to build side bitmask for config fields, type casting code in spirit of using existing structure UAPI. This creates messy code for future.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 12 lines · Parav Pandit
Quoted 3 lines
From: Jason Wang <[email protected]>
Sent: Monday, June 28, 2021 10:33 AM[..]
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines
I don't see why it needs typecast,
virtio_net_configis also uABI, you can deference the fields directly.User wants set only the mac address of the config space. How do user
space tell this?
Quoted 6 lines
Quoted 4 lines
Good question, but we need first answer:
"Do we allow userspace space to modify one specific field of all the config?"
Even if we restrict to specify config params at creation time, question still remains open how to pass, either as whole struct +
side_basedinfo or as individual fields. More below.
Right.
Quoted 9 lines
Quoted 4 lines
Quoted 2 lines
Pass the whole
virtio_net_configand inform via side channel?That could be a method.
I prefer the method to pass individual fields which has the clean code approach and full flexibility. Clean code =
1. no typecasting based on length
2. self-describing fields, do not depends on feature bits parsing
3. proof against structure size increases in fully backward/forward compatibility without code changes
So I think I agree. But I think we'd better to that in the virito uAPI (include/uapi/linux/virito_xxx.h)
Quoted 9 lines
Quoted 8 lines
Quoted 2 lines
Or vendor driver is expected to compare what fields changed from old
config space?
So I think we need solve them all, but netlink is probably the wrong layer, we need to solve them at virtio level and let netlink a transport for them virtio uAPI/ABI.
In spirit of using the virtio UAPI structure, we creating other side band fields, that results into code that’s not common to netlink method.
I think maybe we can start from inventing new virtio uAPI and see if it has some contradict with netlink. Or maybe you can give me some example?
Quoted 44 lines
Ioctl()interface of QEMU/vhost didn't have any other choice withioctl().Quoted 3 lines
And we need to figure out if we want to allow the userspace to modify the config after the device is created. If not, simply build the
virtio_net_configand pass it to the vDPA parent during device creation.I like this idea to pass fields at creation time.
Quoted 8 lines
If not, invent new uAPI at virtio level to passing the config fields. Virtio or vDPA core can provide the library to compare the difference.
My feeling is that, if we restrict to only support build the config during the creation, it would simply a lot of things. And I didn't notice a use case that we need to change the config fields in the middle via the management API/tool.
Sure yes. Whichever config fields user wants to pass, user space passes it.
Quoted 19 lines
Quoted 7 lines
Quoted 6 lines
For
virito_net_config, why not simply:len =
ops->get_config_len();
config =kmalloc(len, GFP_KERNEL);ops->get_config(vdev, 0, config, len);nla_put(skb, VIRTIO_CONFIG, config, len);User space need to parse content based on this length as it can change in
future.
Quoted 1 line
Length telling how to typecast is want I want to avoid here.
So there's no real difference, using
xxx_is_valid, is just a implicit length checking as what is done viaconfig_len:if (
a_is_valid) {
/* dump a */
} else if (b_is_valid) {
/* dump b */
}Quoted 4 lines
vs.
if (length <
offsetof(struct virtio_net_config, next field of a)) {
/* dump a*/+ the feature parsing code, for each field.
Quoted 2 lines
}
Quoted 8 lines
Quoted 6 lines
Actually, Qemu has solved the similar issues via the uAPI:
https://git.qemu.org/?p=qemu.git;a=blob;f=hw/net/virtio-
net.c;h=bd7958b9f0eed2705e0d6a2feaeaefb5e63bd6a4;hb=HEAD#l92If the current uAPI is not sufficient, let's tweak it.
I am unable to convince my self to build side bitmask for config fields, type casting code in spirit of using existing structure UAPI. This creates messy code for future.
Just a quick thought, how about simply something like:
struct virtio_net_config_build {
__u8 mac[ETH_ALEN];
__virtio16 max_virtqueue_pairs;
__virtio16 reserved[3];
};
It looks to we don't need the rest of fields in the virtio_net_config to build the config since they are all hardware attributes.
So it looks self-contained and can be transported via netlink.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Hi Jason,
Quoted 19 lines
From: Jason Wang <[email protected]>
Sent: Tuesday, June 29, 2021 9:22 AMQuoted 4 lines
Quoted 3 lines
Quoted 1 line
Pass the whole
virtio_net_configand inform via side channel?That could be a method.
I prefer the method to pass individual fields which has the clean code
approach and full flexibility.
Quoted 5 lines
Clean code =
1. no typecasting based on length
2. self-describing fields, do not depends on feature bits parsing 3. proof against structure size increases in fully backward/forward compatibility without code changesSo I think I agree. But I think we'd better to that in the virito uAPI (
include/uapi/linux/virito_xxx.h)
[..]
Quoted 11 lines
I think maybe we can start from inventing new virtio uAPI and see if it has some contradict with netlink. Or maybe you can give me some example?
Quoted 1 line
I am unable to convince my self to build side bitmask for config fields, type
casting code in spirit of using existing structure UAPI.
Quoted 1 line
This creates messy code for future.
Quoted 7 lines
Just a quick thought, how about simply something like:
struct
virtio_net_config_build{
__u8mac[ETH_ALEN];
__virtio16max_virtqueue_pairs;
__virtio16reserved[3];
};
In this structure we need to add bi field flags to indicate which entry is valid. And when structure layout changes, we end up with similar typecast issues, length checks and more. Most of it is inbuild to the netlink.
So I propose,
(a) we pass config parameters during vdpa device create
$ vdpa dev add name foo mgmtdev pci/0000:03:00.4 mac 00:11:22:33:44:55 maxq 10
This results in adding two onenew netlink optional attributes as VDPA_DEV_NET_MAC. VDPA_ATTR_DEV_MAX_VQ_SIZE is already dfined for max queues. NLA_POLICY_ETH_ADDR takes care to validate length size when passed.
Quoted 2 lines
It looks to we don't need the rest of fields in the
virtio_net_configto build the config since they are all hardware attributes.
Today it is only mac and max queues. Later on we may need to define rss hashing as hw/device advances. And structure size will change.
Hence, I propose to have each as individual attribute that doesn’t need to cast in struct.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 47 lines · Parav Pandit
Hi Jason,
Quoted 16 lines
From: Jason Wang <[email protected]>
Sent: Tuesday, June 29, 2021 9:22 AMQuoted 3 lines
Quoted 2 lines
Quoted 1 line
Pass the whole
virtio_net_configand inform via side channel?That could be a method.
I prefer the method to pass individual fields which has the clean code
approach and full flexibility.
Quoted 5 lines
Clean code =
1. no typecasting based on length
2. self-describing fields, do not depends on feature bits parsing 3. proof against structure size increases in fully backward/forward compatibility without code changesSo I think I agree. But I think we'd better to that in the virito uAPI (
include/uapi/linux/virito_xxx.h)[..]
Quoted 8 lines
I think maybe we can start from inventing new virtio uAPI and see if it has some contradict with netlink. Or maybe you can give me some example?
Quoted 1 line
I am unable to convince my self to build side bitmask for config fields, type
casting code in spirit of using existing structure UAPI.
Quoted 1 line
This creates messy code for future.
Quoted 7 lines
Just a quick thought, how about simply something like:
struct
virtio_net_config_build{
__u8mac[ETH_ALEN];
__virtio16max_virtqueue_pairs;
__virtio16reserved[3];
};In this structure we need to add bi field flags to indicate which entry is valid. And when structure layout changes, we end up with similar typecast issues, length checks and more. Most of it is inbuild to the netlink.
So I propose,
(a) we pass config parameters during vdpa device create$ vdpa dev add name foo mgmtdev pci/0000:03:00.4 mac 00:11:22:33:44:55 maxq 10This results in adding two onenew netlink optional attributes as
VDPA_DEV_NET_MAC.VDPA_ATTR_DEV_MAX_VQ_SIZEis already dfined for max queues.NLA_POLICY_ETH_ADDRtakes care to validate length size when passed.
Quoted 5 lines
Quoted 2 lines
It looks to we don't need the rest of fields in the
virtio_net_configto build the config since they are all hardware attributes.Today it is only mac and max queues. Later on we may need to define rss hashing as hw/device advances. And structure size will change.
Hence, I propose to have each as individual attribute that doesn’t need to cast in struct.
Ok, that should work. If Michael are fine with this, I'm also fine.
Just to clarify, if I understand this correctly, with the individual attribute, there's no need for the bit like xxx_is_valid?
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Hi Jason,
Quoted 9 lines
From: Jason Wang <[email protected]>
Sent: Wednesday, June 30, 2021 10:02 AMQuoted 3 lines
Quoted 2 lines
It looks to we don't need the rest of fields in the
virtio_net_configto build the config since they are all hardware attributes.Today it is only mac and max queues. Later on we may need to define rss
hashing as hw/device advances.
Quoted 2 lines
And structure size will change.
Hence, I propose to have each as individual attribute that doesn’t need to
Quoted 7 lines
cast in struct.
Ok, that should work. If Michael are fine with this, I'm also fine.
Just to clarify, if I understand this correctly, with the individual attribute, there's no need for the bit like
xxx_is_valid?
xxx_is_valid is not present in the get calls.
It is also not present in UAPI set calls.
It is not a UAPI.
It is an internal between vdpa.c and vendor driver to tell which fields to use as there are optional.
If we want to get rid of those valid flags below code will move to vendor driver where we pass nl_attr, during device add callback.
+ if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR]) {
+ macaddr = nla_data(nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR]);
+ memcpy(config.net.mac, macaddr, sizeof(config.net.mac));
+ config.net_mask.mac_valid = true;
+ }
+ if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU]) {
+ config.net.mtu =
+ nla_get_u16(nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU]);
+ config.net_mask.mtu_valid = true;
+ }
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 24 lines · Parav Pandit
Hi Jason,
Quoted 8 lines
From: Jason Wang <[email protected]>
Sent: Wednesday, June 30, 2021 10:02 AMQuoted 3 lines
Quoted 2 lines
It looks to we don't need the rest of fields in the
virtio_net_configto build the config since they are all hardware attributes.Today it is only mac and max queues. Later on we may need to define rss
hashing as hw/device advances.
Quoted 2 lines
And structure size will change.
Hence, I propose to have each as individual attribute that doesn’t need toQuoted 7 lines
cast in struct.
Ok, that should work. If Michael are fine with this, I'm also fine.
Just to clarify, if I understand this correctly, with the individual attribute, there's no need for the bit like
xxx_is_valid?
xxx_is_validis not present in the get calls.
It is also not present in UAPI set calls.
It is not a UAPI.
It is an internal betweenvdpa.cand vendor driver to tell which fields to use as there are optional.
If we want to get rid of those valid flags below code will move to vendor driver where we passnl_attr, during device add callback.
Review
Quoted 10 lines
(quoted diff) +10
-0+ if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR]) { + macaddr = nla_data(nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR]); + memcpy(config.net.mac, macaddr, sizeof(config.net.mac)); + config.net_mask.mac_valid = true; + } + if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU]) { + config.net.mtu = + nla_get_u16(nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU]); + config.net_mask.mtu_valid = true; + }
Have a hard thought on this. I still think re-invent (duplicate) the virtio-net config filed is not a good choice (e.g for block we need to duplicate more than 20 attributes).
We may meet similar issue when provision VF/SF instance at the hardware level. So I think we may need something in the virtio spec in the near future.
So assuming we don't want a single attributes to be modified and we want to let user to specify all the attributes at one time during creation.
Maybe we can tweak virtio_net_config_set a little bit:
struct virtio_net_config_set {
__virtio64 features;
__u8 mac[ETH_ALEN];
__virtio16 max_virtqueue_pairs;
__virtio16 mtu;
__virtio16 reserved[62];
}
So we have:
- both features and config fields, we're self contained
- reserved fields which should be sufficient for the next 10 years, so we don't need to care about the growing.
Or actually it also allows per field modification.
E.g if we don't specify VIRTIO_NET_F_MAC, it means mac field is invalid. So did for qps and mtu.
The advantage is that we can standardize this in the virtio spec which could be used for SF/VF provisioning.
For get, we probably need more work:
struct virtio_net_config_get {
__virtio64 features;
union {
struct virtio_net_config;
__virtio64 reserved[16];
}
}
Or just follow how it is work today, simply pass the config plus the device_features.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 26 lines
From: Jason Wang <[email protected]>
Sent: Thursday, July 1, 2021 9:04 AMQuoted 6 lines
Quoted 2 lines
Just to clarify, if I understand this correctly, with the individual attribute, there's no need for the bit like
xxx_is_valid?
xxx_is_validis not present in the get calls.
It is also not present in UAPI set calls.
It is not a UAPI.
It is an internal betweenvdpa.cand vendor driver to tell which fields to useas there are optional.
Quoted 1 line
If we want to get rid of those valid flags below code will move to vendor
driver where we pass
nl_attr, during device add callback.Review
Quoted 4 lines
(quoted diff) +2
-0+ if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR]) { + macaddr =
nla_data(nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR]);Review
Quoted 6 lines
(quoted diff) +6
-0+ memcpy(config.net.mac, macaddr, sizeof(config.net.mac)); + config.net_mask.mac_valid = true; + } + if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU]) { + config.net.mtu = +nla_get_u16(nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU]);
Quoted 7 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ config.net_mask.mtu_valid = true; + }Have a hard thought on this. I still think re-invent (duplicate) the virtio-net config filed is not a good choice (e.g for block we need to duplicate more than 20 attributes).
We are re-inventing by defining a new structure below.
Instead of doing them as individual netlink attributes, its lumped together in a struct of arbitrary length. :-)
I notice several fields of the vduse device is setup via ioctl, which I think should be setup via this vdpa device add interface.
Also we can always wrap above nl_attr code in a helper API so that drivers to not hand-code it.
Quoted 3 lines
We may meet similar issue when provision VF/SF instance at the hardware level. So I think we may need something in the virtio spec in the near future.
Do you mean in a virtio vf and virtio sf?
If so, probably yes.
Given that we have the ability to transport individual fields, we don't need to attach the U->K UAPI to a undefined and evolving structure.
Quoted 14 lines
So assuming we don't want a single attributes to be modified and we want to let user to specify all the attributes at one time during creation.
Maybe we can tweak
virtio_net_config_seta little bit:struct
virtio_net_config_set{
__virtio64features;
__u8mac[ETH_ALEN];
__virtio16max_virtqueue_pairs;
__virtio16mtu;
__virtio16reserved[62];
}
Quoted 5 lines
So we have:
- both features and config fields, we're self contained
- reserved fields which should be sufficient for the next 10 years, so we don't need to care about the growing.
This is the reverse of netlink which offers to not reserve any arbitrary size structure. Though I agree that it may not grow.
Quoted 8 lines
Or actually it also allows per field modification.
E.g if we don't specify
VIRTIO_NET_F_MAC, it means mac field is invalid. So did for qps and mtu.The advantage is that we can standardize this in the virtio spec which could be used for SF/VF provisioning.
Virtio spec can be still standardized about which fields of config space should be setup. To do so, we don't need to lump them in one structure.
Quoted 11 lines
For get, we probably need more work:
struct
virtio_net_config_get{
__virtio64features;
union {
structvirtio_net_config;
__virtio64reserved[16];
}
}
Quoted 2 lines
Or just follow how it is work today, simply pass the config plus the
device_features.
If we go with individual attribute get and add both sorted out neatly, expandable.
You already explained that there isn't one to one mapping of features to config fields for other device types too. Netlink already enables us to avoid non symmetric u64 reserved[16] in get and u16 reserved[16] in set.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 23 lines · Parav Pandit
Quoted 23 lines
From: Jason Wang <[email protected]>
Sent: Thursday, July 1, 2021 9:04 AMQuoted 6 lines
Quoted 2 lines
Just to clarify, if I understand this correctly, with the individual attribute, there's no need for the bit like
xxx_is_valid?
xxx_is_validis not present in the get calls.
It is also not present in UAPI set calls.
It is not a UAPI.
It is an internal betweenvdpa.cand vendor driver to tell which fields to useas there are optional.
Quoted 1 line
If we want to get rid of those valid flags below code will move to vendor
driver where we pass
nl_attr, during device add callback.Review
Quoted 3 lines
(quoted diff) +2
-0+ if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR]) { + macaddr =
nla_data(nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR]);Review
Quoted 6 lines
(quoted diff) +6
-0+ memcpy(config.net.mac, macaddr, sizeof(config.net.mac)); + config.net_mask.mac_valid = true; + } + if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU]) { + config.net.mtu = +nla_get_u16(nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU]);
Quoted 7 lines
Quoted 6 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ config.net_mask.mtu_valid = true; + }Have a hard thought on this. I still think re-invent (duplicate) the virtio-net config filed is not a good choice (e.g for block we need to duplicate more than 20 attributes).
We are re-inventing by defining a new structure below.
Actually it depends on what attributes is required for building the config.
We can simply reuse the existing virtio_net_config, if most of the fields are required.
struct virtio_net_config_set {
__virtio64 features;
union {
struct virtio_net_config;
__virtio64 reserved[64];
}
};
If only few of the is required, we can just pick them and use another structure.
Actually, I think just pass the whole config with the device_features during device creation is a good choice that can simplify a lot of things.
We can define what is needed and ignore the others in the virtio spec. Then there's no need to worry about any other things. vDPA core can just do santiy test like checking size vs features.
Quoted 1 line
Instead of doing them as individual netlink attributes, its lumped together in a struct of arbitrary length. :-)
I think not? We want to have a fixed length of the structure which never grow.
So the different is:
1) using netlink dedicated fields
if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR])
2) using netlink as transport
if (features & VIRTIO_NET_F_MAC)
Quoted 4 lines
I notice several fields of the vduse device is setup via ioctl, which I think should be setup via this vdpa device add interface.
Also we can always wrap above
nl_attrcode in a helper API so that drivers to not hand-code it.
Then it would be still more like 2) above (wrap netlink back to something like virtio_net_config)?
Quoted 4 lines
Quoted 3 lines
We may meet similar issue when provision VF/SF instance at the hardware level. So I think we may need something in the virtio spec in the near future.
Do you mean in a virtio vf and virtio sf?
Yes.
Quoted 2 lines
If so, probably yes.
Given that we have the ability to transport individual fields, we don't need to attach theU->KUAPI to a undefined and evolving structure.
I don't object but it needs to be done in virtio uAPI instead of netlink, since it's the device ABI.
Quoted 14 lines
Quoted 14 lines
So assuming we don't want a single attributes to be modified and we want to let user to specify all the attributes at one time during creation.
Maybe we can tweak
virtio_net_config_seta little bit:struct
virtio_net_config_set{
__virtio64features;
__u8mac[ETH_ALEN];
__virtio16max_virtqueue_pairs;
__virtio16mtu;
__virtio16reserved[62];
}
Quoted 6 lines
Quoted 5 lines
So we have:
- both features and config fields, we're self contained
- reserved fields which should be sufficient for the next 10 years, so we don't need to care about the growing.This is the reverse of netlink which offers to not reserve any arbitrary size structure.
It's not arbitrary but with fixed length.
Quoted 11 lines
Though I agree that it may not grow.
Quoted 7 lines
Or actually it also allows per field modification.
E.g if we don't specify
VIRTIO_NET_F_MAC, it means mac field is invalid. So did for qps and mtu.The advantage is that we can standardize this in the virtio spec which could be used for SF/VF provisioning.
Virtio spec can be still standardized about which fields of config space should be setup. To do so, we don't need to lump them in one structure.
Yes, agree.
Quoted 11 lines
Quoted 11 lines
For get, we probably need more work:
struct
virtio_net_config_get{
__virtio64features;
union {
structvirtio_net_config;
__virtio64reserved[16];
}
}
Quoted 3 lines
Quoted 2 lines
Or just follow how it is work today, simply pass the config plus the
device_features.If we go with individual attribute get and add both sorted out neatly, expandable.
It may only work for netlink (with some duplication with the existing virtio uAPI). If we can solve it at general virtio layer, it would be better. Otherwise we need to invent them again in the virtio spec.
E.g virito is expected to support something similar to SF, it requires the SF to be created/provisioned via the admin virtqueue in the PF.
In this case, we still need to define what is required it create a virtio "SF". Netlink can't be used in this context.
I think even for the current mlx5e vDPA it would be better, otherwise we may have:
vDPA tool -> [netlink specific vDPA attributes(1)] -> vDPA core -> [vDPA core specific VDPA attributes(2)] -> mlx5e_vDPA -> [mlx5e specific vDPA attributes(3)] -> mlx5e_core
We need to use a single and unified virtio structure in all the (1), (2) and (3).
Quoted 2 lines
You already explained that there isn't one to one mapping of features to config fields for other device types too.
Yes, but features + config is self contained. That is to say, it's sufficient to explain a specific filed if we had device features.
Thanks
Quoted 1 line
Netlink already enables us to avoid non symmetric u64 reserved[16] in get and u16 reserved[16] in set.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 19 lines
From: Jason Wang <[email protected]>
Sent: Thursday, July 1, 2021 1:13 PMActually it depends on what attributes is required for building the config.
We can simply reuse the existing
virtio_net_config, if most of the fields are required.struct
virtio_net_config_set{
__virtio64features;
union {
structvirtio_net_config;
__virtio64reserved[64];
}
};
Quoted 2 lines
If only few of the is required, we can just pick them and use another structure.
The point is we define structure based on current fields. Tomorrow a new RSS or rx scaling scheme appears, and structure size might need change. And it demands us to go back to length based typecasting code.
and to avoid some length check we pick some arbitrary size reserved words.
And I do not know what network research group will come up for new rss algorithm and needed plumbing.
Quoted 3 lines
Actually, I think just pass the whole config with the
device_featuresduring device creation is a good choice that can simplify a lot of things.
Yes. I totally agree to this.
Quoted 4 lines
We can define what is needed and ignore the others in the virtio spec. Then there's no need to worry about any other things. vDPA core can just do santiy test like checking size vs features.
Yes, we are trying to have code that avoids such sanity checks based on structure size, length etc fields. :-)
Quoted 9 lines
Quoted 3 lines
Instead of doing them as individual netlink attributes, its lumped together
in a struct of arbitrary length. :-)
I think not? We want to have a fixed length of the structure which never grow.
I am not sure defining that future now is right choice, at least for me.
Quoted 24 lines
So the different is:
1) using netlink dedicated fields
if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR])
2) using netlink as transport
if (features &
VIRTIO_NET_F_MAC)Quoted 2 lines
I notice several fields of the vduse device is setup via ioctl, which I think
should be setup via this vdpa device add interface.
Quoted 2 lines
Also we can always wrap above
nl_attrcode in a helper API so that driversto not hand-code it.
Then it would be still more like 2) above (wrap netlink back to something like
virtio_net_config)?Quoted 1 line
Quoted 4 lines
Quoted 1 line
Quoted 1 line
We may meet similar issue when provision VF/SF instance at the
hardware
Quoted 1 line
Quoted 1 line
level. So I think we may need something in the virtio spec in the near
future.
Given the device config is not spelled out in the virtio spec, may be we can wait for it to define virtio management interface.
Quoted 3 lines
I don't object but it needs to be done in virtio uAPI instead of netlink, since it's the device ABI.
Device config can surely be part of the virtio uAPI. We need not have put that in UAPI.
More below.
Quoted 5 lines
Quoted 1 line
This is the reverse of netlink which offers to not reserve any arbitrary size
structure.
It's not arbitrary but with fixed length.
Its fixed, but decided arbitrarily large in anticipation that we likely need to grow. And sometimes that fall short when next research comes up with more creative thoughts.
Quoted 5 lines
It may only work for netlink (with some duplication with the existing virtio uAPI). If we can solve it at general virtio layer, it would be better. Otherwise we need to invent them again in the virtio spec.
Virtio spec will likely define what should be config fields to program and its layout. Kernel can always fill up the format that virtio spec demands.
Quoted 9 lines
I think even for the current mlx5e vDPA it would be better, otherwise we may have:
vDPA tool -> [netlink specific vDPA attributes(1)] -> vDPA core -> [vDPA core specific VDPA attributes(2)] ->
mlx5e_vDPA-> [mlx5e specific vDPA attributes(3)] ->mlx5e_coreWe need to use a single and unified virtio structure in all the (1), (2) and (3).
This is where I differ.
Its only vdpa tool -> vdpa core -> vendor_driver
Vdpa tool -> vdpa core = netlink attribute
Vdpa core -> vendor driver = struct_foo. (internal inside the linux kernel)
If tomorrow virtio spec defines struct_foo to be something else, kernel can always upgrade to struct_bar without upgrading UAPI netlink attributes. Netlink attributes addition will be needed only when struct_foo has newer fields.
This will be still forward/backward compatible.
An exact example of this is drivers/net/vxlan.cvxlan_nl2conf().
A vxlan device needs VNI, src ip, dst ip, tos, and more.
Instead of putting all in single structure vxlan_config as UAPI, those optional fields are netlink attributes. And vxlan driver internally fills up the config structure.
I am very much convinced with the above vxlan approach that enables all functionality needed without typecasting code and without defining arbitrary length structs.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 18 lines · Parav Pandit
Quoted 18 lines
From: Jason Wang <[email protected]>
Sent: Thursday, July 1, 2021 1:13 PMActually it depends on what attributes is required for building the config.
We can simply reuse the existing
virtio_net_config, if most of the fields are required.struct
virtio_net_config_set{
__virtio64features;
union {
structvirtio_net_config;
__virtio64reserved[64];
}
};
Quoted 6 lines
Quoted 2 lines
If only few of the is required, we can just pick them and use another structure.
The point is we define structure based on current fields. Tomorrow a new RSS or rx scaling scheme appears, and structure size might need change. And it demands us to go back to length based typecasting code.
and to avoid some length check we pick some arbitrary size reserved words.
And I do not know what network research group will come up for new rss algorithm and needed plumbing.
Yes, but as discussed, we may suffer the similar issue at the device level. E.g we need a command to let PF to "build" the config for a VF or SF.
Quoted 41 lines
Quoted 3 lines
Actually, I think just pass the whole config with the
device_featuresduring device creation is a good choice that can simplify a lot of things.Yes. I totally agree to this.
Quoted 3 lines
We can define what is needed and ignore the others in the virtio spec. Then there's no need to worry about any other things. vDPA core can just do santiy test like checking size vs features.
Yes, we are trying to have code that avoids such sanity checks based on structure size, length etc fields. :-)
Quoted 8 lines
Quoted 2 lines
Instead of doing them as individual netlink attributes, its lumped together
in a struct of arbitrary length. :-)
I think not? We want to have a fixed length of the structure which never grow.
I am not sure defining that future now is right choice, at least for me.
Quoted 21 lines
So the different is:
1) using netlink dedicated fields
if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR])
2) using netlink as transport
if (features &
VIRTIO_NET_F_MAC)Quoted 1 line
I notice several fields of the vduse device is setup via ioctl, which I think
should be setup via this vdpa device add interface.
Quoted 1 line
Also we can always wrap above
nl_attrcode in a helper API so that driversto not hand-code it.
Then it would be still more like 2) above (wrap netlink back to something like
virtio_net_config)?
Quoted 5 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
We may meet similar issue when provision VF/SF instance at the
hardware
Quoted 1 line
Quoted 1 line
level. So I think we may need something in the virtio spec in the near
future.
Given the device config is not spelled out in the virtio spec, may be we can wait for it to define virtio management interface.
Yes.
Quoted 7 lines
Quoted 3 lines
I don't object but it needs to be done in virtio uAPI instead of netlink, since it's the device ABI.
Device config can surely be part of the virtio uAPI. We need not have put that in UAPI.
More below.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
This is the reverse of netlink which offers to not reserve any arbitrary size
structure.
It's not arbitrary but with fixed length.
Its fixed, but decided arbitrarily large in anticipation that we likely need to grow. And sometimes that fall short when next research comes up with more creative thoughts.
How about something like TLVs in the virtio spec then?
Quoted 7 lines
Quoted 5 lines
It may only work for netlink (with some duplication with the existing virtio uAPI). If we can solve it at general virtio layer, it would be better. Otherwise we need to invent them again in the virtio spec.
Virtio spec will likely define what should be config fields to program and its layout. Kernel can always fill up the format that virtio spec demands.
Yes, I wonder if you have the interest to work on the spec to support this.
Quoted 10 lines
Quoted 10 lines
I think even for the current mlx5e vDPA it would be better, otherwise we may have:
vDPA tool -> [netlink specific vDPA attributes(1)] -> vDPA core -> [vDPA core specific VDPA attributes(2)] ->
mlx5e_vDPA-> [mlx5e specific vDPA attributes(3)] ->mlx5e_coreWe need to use a single and unified virtio structure in all the (1), (2) and (3).
Quoted 7 lines
This is where I differ.
Its only vdpa tool -> vdpa core ->vendor_driverVdpa tool -> vdpa core = netlink attribute
Vdpa core -> vendor driver =struct_foo. (internal inside the linux kernel)If tomorrow virtio spec defines
struct_footo be something else, kernel can always upgrade tostruct_barwithout upgrading UAPI netlink attributes.
That's fine. Note that actually have an extra level if vendor_driver is virtio-pci vDPA driver (vp_vdpa).
Then we have
vdpa tool -> vdpa core -> vp_vdpa -> virtio-pci device
So we still need invent commands to configure/build VF/SF config space between vp_vdpa and virtio-pci device. And I think we may suffer the similar issue as we met here (vdpa tool -> vdpa core).
Quoted 10 lines
Netlink attributes addition will be needed only when
struct_foohas newer fields. This will be still forward/backward compatible.An exact example of this is
drivers/net/vxlan.cvxlan_nl2conf().
A vxlan device needs VNI, src ip, dst ip, tos, and more.
Instead of putting all in single structurevxlan_configas UAPI, those optional fields are netlink attributes. And vxlan driver internally fills up the config structure.I am very much convinced with the above vxlan approach that enables all functionality needed without typecasting code and without defining arbitrary length structs.
Right, but we had some small differences here:
1) vxlan doesn't have a existing uAPI
2) vxlan configuration is not used for hardware
Basically, I'm not against this approach, I just wonder if it's better/simpler to solve it at virtio layer because the semantic is defined by the spec not netlink.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 30 lines
From: Jason Wang <[email protected]>
Sent: Monday, July 5, 2021 10:05 AMQuoted 18 lines · Parav Pandit
Quoted 18 lines
From: Jason Wang <[email protected]>
Sent: Thursday, July 1, 2021 1:13 PMActually it depends on what attributes is required for building the config.
We can simply reuse the existing
virtio_net_config, if most of the fields are required.struct
virtio_net_config_set{
__virtio64features;
union {
structvirtio_net_config;
__virtio64reserved[64];
}
};Quoted 3 lines
Quoted 2 lines
If only few of the is required, we can just pick them and use another structure.
The point is we define structure based on current fields. Tomorrow a new
RSS or rx scaling scheme appears, and structure size might need change.
Quoted 2 lines
And it demands us to go back to length based typecasting code. and to avoid some length check we pick some arbitrary size reserved
Quoted 7 lines
words.
Quoted 1 line
And I do not know what network research group will come up for new rss
algorithm and needed plumbing.
Yes, but as discussed, we may suffer the similar issue at the device level. E.g we need a command to let PF to "build" the config for a VF or SF.
I am not sure.
Current scope of a VDPA is, once there is a has PF,VF,SF and you configure or create a vdpa device out of it.
Quoted 4 lines
Quoted 1 line
Given the device config is not spelled out in the virtio spec, may be we can
wait for it to define virtio management interface.
Yes.
Wait is needed only if we want to cast U->K UAPI in a structure which is bound to evolve. And hence I just want to exchange as individual fields.
Quoted 8 lines
Quoted 2 lines
Quoted 1 line
It's not arbitrary but with fixed length.
Its fixed, but decided arbitrarily large in anticipation that we likely need to
grow.
Quoted 1 line
And sometimes that fall short when next research comes up with more
creative thoughts.
How about something like TLVs in the virtio spec then?
Possibly yes.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
It may only work for netlink (with some duplication with the existing virtio uAPI). If we can solve it at general virtio layer, it would be better. Otherwise we need to invent them again in the virtio spec.
Quoted 7 lines
Quoted 1 line
Virtio spec will likely define what should be config fields to program and its
layout.
Quoted 1 line
Kernel can always fill up the format that virtio spec demands.
Yes, I wonder if you have the interest to work on the spec to support this.
I am happy to contribute, I need to ask my supervisor to spend some time in this area. Let me figure out the logistics.
Quoted 24 lines
Quoted 11 lines
Quoted 11 lines
I think even for the current mlx5e vDPA it would be better, otherwise we may have:
vDPA tool -> [netlink specific vDPA attributes(1)] -> vDPA core -> [vDPA core specific VDPA attributes(2)] ->
mlx5e_vDPA-> [mlx5e specific vDPA attributes(3)] ->mlx5e_coreWe need to use a single and unified virtio structure in all the (1), (2) and (3).
Quoted 7 lines
This is where I differ.
Its only vdpa tool -> vdpa core ->vendor_driverVdpa tool -> vdpa core = netlink attribute
Vdpa core -> vendor driver =struct_foo. (internal inside the linux kernel)If tomorrow virtio spec defines
struct_footo be something else, kernel canalways upgrade to
struct_barwithout upgrading UAPI netlink attributes.That's fine. Note that actually have an extra level if
vendor_driveris virtio-pci vDPA driver (vp_vdpa).
Quoted 6 lines
Then we have
vdpa tool -> vdpa core ->
vp_vdpa-> virtio-pci deviceSo we still need invent commands to configure/build VF/SF config space between
vp_vdpaand virtio-pci device.
Yes. This is needed, but again lets keep the two layers separate.
In the example I provided, we will be able to fill the structure and pass this internally between vp_vdpa->virtio pci driver.
Quoted 16 lines
And I think we may suffer the
similar issue as we met here (vdpa tool -> vdpa core).Quoted 1 line
Netlink attributes addition will be needed only when
struct_foohas newerfields.
Quoted 6 lines
This will be still forward/backward compatible.
An exact example of this is
drivers/net/vxlan.cvxlan_nl2conf().
A vxlan device needs VNI, src ip, dst ip, tos, and more. Instead of putting all in single structurevxlan_configas UAPI, thoseoptional fields are netlink attributes.
Quoted 3 lines
And vxlan driver internally fills up the config structure.
I am very much convinced with the above vxlan approach that enables all
Quoted 9 lines
functionality needed without typecasting code and without defining arbitrary length structs.
Right, but we had some small differences here:
1) vxlan doesn't have a existing uAPI
2) vxlan configuration is not used for hardware
True but vxlan example doesn’t prevent to do #2.
Quoted 3 lines
Basically, I'm not against this approach, I just wonder if it's better/simpler to solve it at virtio layer because the semantic is defined by the spec not netlink.
vdpa core will be able to use the virtio spec defined config whenever it occurs.
_______________________________________________
Virtualization mailing list [email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 59 lines · Parav Pandit
Quoted 28 lines
From: Jason Wang <[email protected]>
Sent: Monday, July 5, 2021 10:05 AMQuoted 17 lines · Parav Pandit
Quoted 17 lines
From: Jason Wang <[email protected]>
Sent: Thursday, July 1, 2021 1:13 PMActually it depends on what attributes is required for building the config.
We can simply reuse the existing
virtio_net_config, if most of the fields are required.struct virtio_net_config_set { __virtio64 features; union { struct virtio_net_config; __virtio64 reserved[64]; } };Quoted 3 lines
Quoted 2 lines
If only few of the is required, we can just pick them and use another structure.
The point is we define structure based on current fields. Tomorrow a new
RSS or rx scaling scheme appears, and structure size might need change.
Quoted 2 lines
And it demands us to go back to length based typecasting code. and to avoid some length check we pick some arbitrary size reserved
Quoted 7 lines
words.
Quoted 1 line
And I do not know what network research group will come up for new rss
algorithm and needed plumbing.
Yes, but as discussed, we may suffer the similar issue at the device level. E.g we need a command to let PF to "build" the config for a VF or SF.
I am not sure.
Current scope of a VDPA is, once there is a has PF,VF,SF and you configure or create a vdpa device out of it.Quoted 4 lines
Quoted 1 line
Given the device config is not spelled out in the virtio spec, may be we can
wait for it to define virtio management interface.
Yes.
Wait is needed only if we want to cast
U->KUAPI in a structure which is bound to evolve. And hence I just want to exchange as individual fields.Quoted 8 lines
Quoted 2 lines
Quoted 1 line
It's not arbitrary but with fixed length.
Its fixed, but decided arbitrarily large in anticipation that we likely need to
grow.
Quoted 1 line
And sometimes that fall short when next research comes up with more
creative thoughts.
How about something like TLVs in the virtio spec then?
Possibly yes.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
It may only work for netlink (with some duplication with the existing virtio uAPI). If we can solve it at general virtio layer, it would be better. Otherwise we need to invent them again in the virtio spec.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Virtio spec will likely define what should be config fields to program and its
layout.
Quoted 1 line
Kernel can always fill up the format that virtio spec demands.
Yes, I wonder if you have the interest to work on the spec to support this.
I am happy to contribute, I need to ask my supervisor to spend some time in this area. Let me figure out the logistics.
Good to know that.
Quoted 58 lines
Quoted 23 lines
Quoted 10 lines
Quoted 10 lines
I think even for the current mlx5e vDPA it would be better, otherwise we may have:
vDPA tool -> [netlink specific vDPA attributes(1)] -> vDPA core -> [vDPA core specific VDPA attributes(2)] ->
mlx5e_vDPA-> [mlx5e specific vDPA attributes(3)] ->mlx5e_coreWe need to use a single and unified virtio structure in all the (1), (2) and (3).
Quoted 7 lines
This is where I differ.
Its only vdpa tool -> vdpa core ->vendor_driverVdpa tool -> vdpa core = netlink attribute
Vdpa core -> vendor driver =struct_foo. (internal inside the linux kernel)If tomorrow virtio spec defines
struct_footo be something else, kernel canalways upgrade to
struct_barwithout upgrading UAPI netlink attributes.That's fine. Note that actually have an extra level if
vendor_driveris virtio-pci vDPA driver (vp_vdpa).Quoted 6 lines
Then we have
vdpa tool -> vdpa core ->
vp_vdpa-> virtio-pci deviceSo we still need invent commands to configure/build VF/SF config space between
vp_vdpaand virtio-pci device.Yes. This is needed, but again lets keep the two layers separate.
In the example I provided, we will be able to fill the structure and pass this internally betweenvp_vdpa->virtiopci driver.Quoted 16 lines
And I think we may suffer the
similar issue as we met here (vdpa tool -> vdpa core).Quoted 1 line
Netlink attributes addition will be needed only when
struct_foohas newerfields.
Quoted 6 lines
This will be still forward/backward compatible.
An exact example of this is
drivers/net/vxlan.cvxlan_nl2conf().
A vxlan device needs VNI, src ip, dst ip, tos, and more. Instead of putting all in single structurevxlan_configas UAPI, thoseoptional fields are netlink attributes.
Quoted 3 lines
And vxlan driver internally fills up the config structure.
I am very much convinced with the above vxlan approach that enables all
Quoted 9 lines
functionality needed without typecasting code and without defining arbitrary length structs.
Right, but we had some small differences here:
1) vxlan doesn't have a existing uAPI
2) vxlan configuration is not used for hardware
Quoted 6 lines
True but vxlan example doesn’t prevent to do #2.
Quoted 3 lines
Basically, I'm not against this approach, I just wonder if it's better/simpler to solve it at virtio layer because the semantic is defined by the spec not netlink.
vdpa core will be able to use the virtio spec defined config whenever it occurs.
So I think both of us have strong points. Maybe it's the time for Michael to decide how it will go.
Michael, please share your thoughts here.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerRe: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
In reply to Jason WangQuoted 3 lines · Jason Wang
So I think we need solve them all, but netlink is probably the wrong layer, we need to solve them at virtio level and let netlink a transport for them virtio uAPI/ABI.
I'm not sure I follow. virtio defines VF to driver communication. This is PF to hypervisor. virtio simply does not cover it ATM.
Signature · 6 lines
MST _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 6 lines · Michael S. Tsirkin
Quoted 3 lines · Jason Wang
So I think we need solve them all, but netlink is probably the wrong layer, we need to solve them at virtio level and let netlink a transport for them virtio uAPI/ABI.
I'm not sure I follow. virtio defines VF to driver communication. This is PF to hypervisor. virtio simply does not cover it ATM.
Note that this is not PF to hypervisor but the uAPI from userspace (vDPA tool) to vDPA core.
We had two choices.
1) tweak virtio uAPIs
2) invent virtio specific uAPI in netlink
1) seems better.
Thanks
Quoted 1 line
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerRe: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 13 lines · Jason Wang
Quoted 6 lines · Michael S. Tsirkin
Quoted 3 lines · Jason Wang
So I think we need solve them all, but netlink is probably the wrong layer, we need to solve them at virtio level and let netlink a transport for them virtio uAPI/ABI.
I'm not sure I follow. virtio defines VF to driver communication. This is PF to hypervisor. virtio simply does not cover it ATM.
Note that this is not PF to hypervisor but the uAPI from userspace (vDPA tool) to vDPA core.
Quoted 9 lines
We had two choices.
1) tweak virtio uAPIs
2) invent virtio specific uAPI in netlink1) seems better.
Thanks
Well things like setting mac aren't virtio specific. What are the virtio specific things you have in mind?
Quoted 1 line
Quoted 1 line
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 2/6] vdpa: Introduce query of device config layout
Quoted 17 lines · Michael S. Tsirkin
Quoted 16 lines · Jason Wang
Quoted 6 lines · Michael S. Tsirkin
Quoted 3 lines · Jason Wang
So I think we need solve them all, but netlink is probably the wrong layer, we need to solve them at virtio level and let netlink a transport for them virtio uAPI/ABI.
I'm not sure I follow. virtio defines VF to driver communication. This is PF to hypervisor. virtio simply does not cover it ATM.
Note that this is not PF to hypervisor but the uAPI from userspace (vDPA tool) to vDPA core.
We had two choices.
1) tweak virtio uAPIs
2) invent virtio specific uAPI in netlink
Quoted 6 lines
Quoted 4 lines
1) seems better.
Thanks
Well things like setting mac aren't virtio specific. What are the virtio specific things you have in mind?
E.g max_virtqueue_pairs? Even if mac aren't virtio specific, the idea is to build the device config which is virtio specific.
And we may have other devices like virtio-blk.
Thanks
Quoted 1 line
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v3 3/6 vdpa: Enable user to set mac and mtu of vdpa device
In reply to Parav Pandit$ vdpa dev add name bar mgmtdev vdpasim_net
$ vdpa dev config set bar mac 00:11:22:33:44:55 mtu 9000
$ vdpa dev config show bar: mac 00:11:22:33:44:55 link up link_announce false mtu 9000 speed 0 duplex 0
$ vdpa dev config show -jp
{
"config": {
"bar": {
"mac": "00:11:22:33:44:55",
"link ": "up",
"link_announce ": false,
"mtu": 9000,
"speed": 0,
"duplex": 0
}
}
}---
changelog:v2->v3:
- using new setup_config callback to setup device params via mgmt tool to avoid mixing with existing set_config().
--- drivers/vdpa/vdpa.c | 91 ++++++++++++++++++++++++++++++++++++++- include/linux/vdpa.h | 18 ++++++++ include/uapi/linux/vdpa.h | 1 + 3 files changed, 109 insertions(+), 1 deletion(-)
drivers/vdpa/vdpa.c +90 -1
@@ -14,7 +14,6 @@#include <uapi/linux/vdpa.h> #include <net/genetlink.h> #include <linux/mod_devicetable.h> -#include <linux/virtio_net.h> #include <linux/virtio_ids.h> static LIST_HEAD (mdev_head);
@@ -849,10 +848,94 @@ vdpa_nl_cmd_dev_config_get_dumpit(struct sk_buff *msg, struct netlink_callback *return msg->len; } +static int vdpa_dev_net_config_set (struct vdpa_device *vdev, +struct sk_buff *skb,struct genl_info *info) +{ +struct nlattr **nl_attrs = info->attrs; +struct vdpa_dev_set_config config = {}; +const u8 *macaddr; +int err; + +if (!netlink_capable (skb,CAP_NET_ADMIN )) +return -EPERM ; + +if (!vdev->config->setup_config) +return -EOPNOTSUPP ; + +if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR ]) { + macaddr =nla_data (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR ]); +memcpy (config.net.mac, macaddr,sizeof (config.net.mac)); + config.net_mask.mac_valid = true; + } +if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU ]) { + config.net.mtu = +nla_get_u16 (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU ]); + config.net_mask.mtu_valid = true; + } + +mutex_lock (&vdev->cf_mutex); + err = vdev->config->setup_config (vdev, &config); +mutex_unlock (&vdev->cf_mutex); +return err; +} + +static int vdpa_dev_config_set (struct vdpa_device *vdev,struct sk_buff *skb, +struct genl_info *info) +{ +int err = -EOPNOTSUPP ; +u32 device_id; + +if (!vdev->mdev) +return -EOPNOTSUPP ; + + device_id = vdev->config->get_device_id (vdev); +switch (device_id) { +case VIRTIO_ID_NET : + err =vdpa_dev_net_config_set (vdev, skb, info); +break ; +default : +break ; + } +return err; +} + +static int vdpa_nl_cmd_dev_config_set_doit (struct sk_buff *skb,struct genl_info *info) +{ +struct vdpa_device *vdev; +const char *devname; +struct device *dev; +int err; + +if (!info->attrs[VDPA_ATTR_DEV_NAME ]) +return -EINVAL ; + devname =nla_data (info->attrs[VDPA_ATTR_DEV_NAME ]); + +mutex_lock (&vdpa_dev_mutex); + dev =bus_find_device (&vdpa_bus,NULL , devname, vdpa_name_match); +if (!dev) { +mutex_unlock (&vdpa_dev_mutex); +NL_SET_ERR_MSG_MOD (info->extack,"device not found" ); +return -ENODEV ; + } + vdev = container_of(dev, struct vdpa_device, dev); +if (!vdev->mdev) { +mutex_unlock (&vdpa_dev_mutex); +put_device (dev); +return -EINVAL ; + } + err =vdpa_dev_config_set (vdev, skb, info); +put_device (dev); +mutex_unlock (&vdpa_dev_mutex); +return err; +} +static const struct nla_policy vdpa_nl_policy[VDPA_ATTR_MAX +1 ] = { [VDPA_ATTR_MGMTDEV_BUS_NAME ] = { .type =NLA_NUL_STRING }, [VDPA_ATTR_MGMTDEV_DEV_NAME ] = { .type =NLA_STRING }, [VDPA_ATTR_DEV_NAME ] = { .type =NLA_STRING }, + [VDPA_ATTR_DEV_NET_CFG_MACADDR ] =NLA_POLICY_ETH_ADDR , +/* virtio spec 1.1 section 5.1.4.1 for valid MTU range */ + [VDPA_ATTR_DEV_NET_CFG_MTU ] =NLA_POLICY_MIN (NLA_U16 ,68 ), };static const struct genl_ops vdpa_nl_ops[] = {
@@ -886,6 +969,12 @@ static const struct genl_ops vdpa_nl_ops[] = { .doit = vdpa_nl_cmd_dev_config_get_doit, .dumpit = vdpa_nl_cmd_dev_config_get_dumpit, }, + { + .cmd = VDPA_CMD_DEV_CONFIG_SET, + .validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP, + .doit = vdpa_nl_cmd_dev_config_set_doit, + .flags = GENL_ADMIN_PERM, + }, };static struct genl_family vdpa_nl_family__ro_after_init = {
include/linux/vdpa.h +18 -0
@@ -6,6 +6,8 @@#include <linux/device.h> #include <linux/interrupt.h> #include <linux/vhost_iotlb.h> +#include <linux/virtio_net.h> +#include <linux/if_ether.h> /** * struct vdpa_calllback - vDPA callback definition.
@@ -70,6 +72,17 @@ struct vdpa_iova_range {u64 last; }; +struct vdpa_dev_set_config { +struct { +u8 mac[ETH_ALEN ]; +u16 mtu; + } net; +struct { +u8 mac_valid:1 ; +u8 mtu_valid:1 ; + } net_mask; +}; +/** * struct vdpa_config_ops - operations for configuring a vDPA device. * Note: vDPA device drivers are required to implement all of the
@@ -169,6 +182,9 @@ struct vdpa_iova_range {* @buf: buffer used to write from * @len: the length to write to * configuration space +* @setup_config: Setup configuration space +* @vdev: vdpa device +* #config: configuration to apply to device * @get_generation: Get device config generation (optional) * @vdev: vdpa device * Returns u32: device generation
@@ -241,6 +257,8 @@ struct vdpa_config_ops { void *buf, unsigned int len);void (*set_config)(struct vdpa_device *vdev,unsigned int offset,const void *buf,unsigned int len); +int (*setup_config)(struct vdpa_device *vdev, +const struct vdpa_dev_set_config *config);u32 (*get_generation)(struct vdpa_device *vdev);struct vdpa_iova_range (*get_iova_range)(struct vdpa_device *vdev);
include/uapi/linux/vdpa.h +1 -0
@@ -18,6 +18,7 @@ enum vdpa_command {VDPA_CMD_DEV_DEL ,VDPA_CMD_DEV_GET ,/* can dump */ VDPA_CMD_DEV_CONFIG_GET ,/* can dump */ + VDPA_CMD_DEV_CONFIG_SET, }; enum vdpa_attr {
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v3 3/6] vdpa: Enable user to set mac and mtu of vdpa device
Quoted 50 lines · Parav Pandit
$ vdpa dev add name bar mgmtdev vdpasim_net$ vdpa dev config set bar mac 00:11:22:33:44:55 mtu 9000$ vdpa dev config show bar: mac 00:11:22:33:44:55 link up link_announce false mtu 9000 speed 0 duplex 0$ vdpa dev config show -jp { "config": { "bar": { "mac": "00:11:22:33:44:55", "link ": "up", "link_announce ": false, "mtu": 9000, "speed": 0, "duplex": 0 } } }Signed-off-by Parav Pandit <[email protected]>Reviewed-by Eli Cohen <[email protected]>---
changelog:v2->v3:
- using newsetup_configcallback to setup device params via mgmt tool to avoid mixing with existingset_config().--- drivers/vdpa/vdpa.c | 91 ++++++++++++++++++++++++++++++++++++++- include/linux/vdpa.h | 18 ++++++++ include/uapi/linux/vdpa.h | 1 + 3 files changed, 109 insertions(+), 1 deletion(-)drivers/vdpa/vdpa.c +0
-1@@ -14,7 +14,6 @@#include <uapi/linux/vdpa.h> #include <net/genetlink.h> #include <linux/mod_devicetable.h> -#include <linux/virtio_net.h> #include <linux/virtio_ids.h> static LIST_HEAD (mdev_head);@@ -849,10 +848,94 @@ vdpa_nl_cmd_dev_config_get_dumpit(struct sk_buff *msg, struct netlink_callback *return msg->len; }
drivers/vdpa/vdpa.c vdpa_nl_cmd_dev_config_get_dumpit(struct sk_buff *msg, struct netlink_callback * · line 860
Quoted 10 lines
drivers/vdpa/vdpa.c +10
-0+static int vdpa_dev_net_config_set (struct vdpa_device *vdev, +struct sk_buff *skb,struct genl_info *info) +{ +struct nlattr **nl_attrs = info->attrs; +struct vdpa_dev_set_config config = {}; +const u8 *macaddr; +int err; + +if (!netlink_capable (skb,CAP_NET_ADMIN )) +return -EPERM ;
Interesting, I wonder how cap would be used for other type of devices (e.g block).
Quoted 4 lines
drivers/vdpa/vdpa.c +4
-0+ +if (!vdev->config->setup_config) +return -EOPNOTSUPP ; +
drivers/vdpa/vdpa.c vdpa_nl_cmd_dev_config_get_dumpit(struct sk_buff *msg, struct netlink_callback * · line 874
Quoted 10 lines
drivers/vdpa/vdpa.c +10
-0+if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR ]) { + macaddr =nla_data (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR ]); +memcpy (config.net.mac, macaddr,sizeof (config.net.mac)); + config.net_mask.mac_valid = true; + } +if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU ]) { + config.net.mtu = +nla_get_u16 (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU ]); + config.net_mask.mtu_valid = true; + }
Instead of doing memcpy and pass the whole config structure like this. I wonder if it's better to switch to use:
vdev->config->setup_config(vdev, offsetof(struct virtio_net_config, mtu), &mtu, sizeof(mtu));
Then there's no need for the vdpa_dev_set_config structure which will became structure virtio_net_config gradually.
The setup_config() can fail if the offset is not at the boundary of a specific attribute.
Thanks
Quoted 135 lines
drivers/vdpa/vdpa.c +66
-0+ +mutex_lock (&vdev->cf_mutex); + err = vdev->config->setup_config (vdev, &config); +mutex_unlock (&vdev->cf_mutex); +return err; +} + +static int vdpa_dev_config_set (struct vdpa_device *vdev,struct sk_buff *skb, +struct genl_info *info) +{ +int err = -EOPNOTSUPP ; +u32 device_id; + +if (!vdev->mdev) +return -EOPNOTSUPP ; + + device_id = vdev->config->get_device_id (vdev); +switch (device_id) { +case VIRTIO_ID_NET : + err =vdpa_dev_net_config_set (vdev, skb, info); +break ; +default : +break ; + } +return err; +} + +static int vdpa_nl_cmd_dev_config_set_doit (struct sk_buff *skb,struct genl_info *info) +{ +struct vdpa_device *vdev; +const char *devname; +struct device *dev; +int err; + +if (!info->attrs[VDPA_ATTR_DEV_NAME ]) +return -EINVAL ; + devname =nla_data (info->attrs[VDPA_ATTR_DEV_NAME ]); + +mutex_lock (&vdpa_dev_mutex); + dev =bus_find_device (&vdpa_bus,NULL , devname, vdpa_name_match); +if (!dev) { +mutex_unlock (&vdpa_dev_mutex); +NL_SET_ERR_MSG_MOD (info->extack,"device not found" ); +return -ENODEV ; + } + vdev = container_of(dev, struct vdpa_device, dev); +if (!vdev->mdev) { +mutex_unlock (&vdpa_dev_mutex); +put_device (dev); +return -EINVAL ; + } + err =vdpa_dev_config_set (vdev, skb, info); +put_device (dev); +mutex_unlock (&vdpa_dev_mutex); +return err; +} +static const struct nla_policy vdpa_nl_policy[VDPA_ATTR_MAX +1 ] = { [VDPA_ATTR_MGMTDEV_BUS_NAME ] = { .type =NLA_NUL_STRING }, [VDPA_ATTR_MGMTDEV_DEV_NAME ] = { .type =NLA_STRING }, [VDPA_ATTR_DEV_NAME ] = { .type =NLA_STRING }, + [VDPA_ATTR_DEV_NET_CFG_MACADDR ] =NLA_POLICY_ETH_ADDR , +/* virtio spec 1.1 section 5.1.4.1 for valid MTU range */ + [VDPA_ATTR_DEV_NET_CFG_MTU ] =NLA_POLICY_MIN (NLA_U16 ,68 ), };static const struct genl_ops vdpa_nl_ops[] = {@@ -886,6 +969,12 @@ static const struct genl_ops vdpa_nl_ops[] = { .doit = vdpa_nl_cmd_dev_config_get_doit, .dumpit = vdpa_nl_cmd_dev_config_get_dumpit, }, + { + .cmd = VDPA_CMD_DEV_CONFIG_SET, + .validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP, + .doit = vdpa_nl_cmd_dev_config_set_doit, + .flags = GENL_ADMIN_PERM, + }, };static struct genl_family vdpa_nl_family__ro_after_init = {include/linux/vdpa.h +18
-0@@ -6,6 +6,8 @@#include <linux/device.h> #include <linux/interrupt.h> #include <linux/vhost_iotlb.h> +#include <linux/virtio_net.h> +#include <linux/if_ether.h> /** * struct vdpa_calllback - vDPA callback definition. @@ -70,6 +72,17 @@ struct vdpa_iova_range {u64 last; }; +struct vdpa_dev_set_config { +struct { +u8 mac[ETH_ALEN ]; +u16 mtu; + } net; +struct { +u8 mac_valid:1 ; +u8 mtu_valid:1 ; + } net_mask; +}; +/** * struct vdpa_config_ops - operations for configuring a vDPA device. * Note: vDPA device drivers are required to implement all of the @@ -169,6 +182,9 @@ struct vdpa_iova_range {* @buf: buffer used to write from * @len: the length to write to * configuration space +* @setup_config: Setup configuration space +* @vdev: vdpa device +* #config: configuration to apply to device * @get_generation: Get device config generation (optional) * @vdev: vdpa device * Returns u32: device generation @@ -241,6 +257,8 @@ struct vdpa_config_ops { void *buf, unsigned int len);void (*set_config)(struct vdpa_device *vdev,unsigned int offset,const void *buf,unsigned int len); +int (*setup_config)(struct vdpa_device *vdev, +const struct vdpa_dev_set_config *config);u32 (*get_generation)(struct vdpa_device *vdev);struct vdpa_iova_range (*get_iova_range)(struct vdpa_device *vdev);include/uapi/linux/vdpa.h +0
-0
include/uapi/linux/vdpa.h enum vdpa_command { · line 24
Quoted 7 lines
include/uapi/linux/vdpa.h +1
-0@@ -18,6 +18,7 @@ enum vdpa_command {VDPA_CMD_DEV_DEL ,VDPA_CMD_DEV_GET ,/* can dump */ VDPA_CMD_DEV_CONFIG_GET ,/* can dump */ + VDPA_CMD_DEV_CONFIG_SET, }; enum vdpa_attr {
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v3 3/6] vdpa: Enable user to set mac and mtu of vdpa device
Quoted 95 lines
From: Jason Wang <[email protected]>
Sent: Tuesday, June 22, 2021 1:13 PMQuoted 24 lines · Parav Pandit
$ vdpa dev add name bar mgmtdev vdpasim_net$ vdpa dev config set bar mac 00:11:22:33:44:55 mtu 9000$ vdpa dev config show bar: mac 00:11:22:33:44:55 link up link_announce false mtu 9000 speed 0 duplex 0$ vdpa dev config show -jp { "config": { "bar": { "mac": "00:11:22:33:44:55", "link ": "up", "link_announce ": false, "mtu": 9000, "speed": 0, "duplex": 0 } } }Signed-off-by Parav Pandit <[email protected]>Reviewed-by Eli Cohen <[email protected]>Quoted 7 lines
---
changelog:v2->v3:
- using newsetup_configcallback to setup device params via mgmt tool to avoid mixing with existingset_config().--- drivers/vdpa/vdpa.c | 91++++++++++++++++++++++++++++++++++++++-
drivers/vdpa/vdpa.c vdpa_nl_cmd_dev_config_get_dumpit(struct
Quoted 17 lines
include/linux/vdpa.h | 18 ++++++++ include/uapi/linux/vdpa.h | 1 + 3 files changed, 109 insertions(+), 1 deletion(-)drivers/vdpa/vdpa.c index renamed from drivers/vdpa/vdpa.c +0
-0
1295528244c3..40874bd92126100644drivers/vdpa/vdpa.c +0
-1@@ -14,7 +14,6 @@#include <uapi/linux/vdpa.h> #include <net/genetlink.h> #include <linux/mod_devicetable.h> -#include <linux/virtio_net.h> #include <linux/virtio_ids.h> static LIST_HEAD (mdev_head);@@ -849,10 +848,94 @@ vdpa_nl_cmd_dev_config_get_dumpit(struct
sk_buff*msg, structnetlink_callback*drivers/vdpa/vdpa.c vdpa_nl_cmd_dev_config_get_dumpit(struct sk_buff *msg, struct netlink_callback * · line 860
Quoted 12 lines
drivers/vdpa/vdpa.c +9
-0return msg->len; } +static int vdpa_dev_net_config_set (struct vdpa_device *vdev, +struct sk_buff *skb,struct genl_info *info) { +struct nlattr **nl_attrs = info->attrs; +struct vdpa_dev_set_config config = {}; +const u8 *macaddr; +int err; + +if (!netlink_capable (skb,CAP_NET_ADMIN )) +return -EPERM ;Interesting, I wonder how cap would be used for other type of devices (e.g block).
drivers/vdpa/vdpa.c vdpa_nl_cmd_dev_config_get_dumpit(struct sk_buff *msg, struct netlink_callback *
Quoted 6 lines
drivers/vdpa/vdpa.c +6
-0+ +if (!vdev->config->setup_config) +return -EOPNOTSUPP ; + +if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR ]) { + macaddr =
nla_data(nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR]);drivers/vdpa/vdpa.c vdpa_nl_cmd_dev_config_get_dumpit(struct sk_buff *msg, struct netlink_callback * · line 875
Quoted 6 lines
drivers/vdpa/vdpa.c +6
-0+memcpy (config.net.mac, macaddr,sizeof (config.net.mac)); + config.net_mask.mac_valid = true; + } +if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU ]) { + config.net.mtu = +nla_get_u16(nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU]);drivers/vdpa/vdpa.c vdpa_nl_cmd_dev_config_get_dumpit(struct · line 873
Quoted 2 lines
drivers/vdpa/vdpa.c +2
-0+ config.net_mask.mtu_valid = true; + }Instead of doing memcpy and pass the whole config structure like this. I wonder if it's better to switch to use:
Quoted 3 lines
vdev->config->setup_config(vdev, offsetof(struct virtio_net_config, mtu), &mtu, sizeof(mtu));
Well, we need a way to differentiate that the caller is management tool and not the vhost path.
Instead of passing some flag of the caller to setup_config(), a explicitly defined callback served better.
And secondly we need to return the error status. setup_config() cb is void. This is the minor one.
Quoted 6 lines
Then there's no need for the
vdpa_dev_set_configstructure which will became structurevirtio_net_configgradually.The
setup_config()can fail if the offset is not at the boundary of a specific attribute.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v3 4/6 vdpa_sim_net: Enable user to set mac address and mtu
In reply to Parav PanditEnable user to set the mac address and mtu so that each vdpa device can have its own user specified mac address and mtu. This is done by implementing the management device's configuration layout fields setting callback routine.
Now that user is enabled to set the mac address, remove the module parameter for same.
And example of setting mac addr and mtu:
$ vdpa mgmtdev show
$ vdpa dev add name bar mgmtdev vdpasim_net $ vdpa dev config set bar mac 00:11:22:33:44:55 mtu 9000
View the config after setting:
$ vdpa dev config show bar: mac 00:11:22:33:44:55 link up link_announce false mtu 9000 speed 0 duplex 0
---
changelog:v1->v2:
- using updated interface and callbacks for get/set config
--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 13 +++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 2 ++ drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 34 +++++++++++++++------------- 3 files changed, 33 insertions(+), 16 deletions(-)
drivers/vdpa/vdpa_sim/vdpa_sim.c +13 -0
@@ -497,6 +497,17 @@ static void vdpasim_set_config(struct vdpa_device *vdpa, unsigned int offset, vdpasim->dev_attr.set_config (vdpasim, vdpasim->config); } +static int vdpasim_setup_config (struct vdpa_device *vdpa, +const struct vdpa_dev_set_config *config) +{ +struct vdpasim *vdpasim =vdpa_to_sim (vdpa); + +if (!vdpasim->dev_attr.setup_config) +return -EOPNOTSUPP ; + +return vdpasim->dev_attr.setup_config (vdpasim, config); +} +static u32 vdpasim_get_generation (struct vdpa_device *vdpa) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);
@@ -608,6 +619,7 @@ static const struct vdpa_config_ops vdpasim_config_ops = { .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config, + .setup_config = vdpasim_setup_config, .get_generation = vdpasim_get_generation, .get_iova_range = vdpasim_get_iova_range, .dma_map = vdpasim_dma_map,
@@ -636,6 +648,7 @@ static const struct vdpa_config_ops vdpasim_batch_config_ops = { .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config, + .setup_config = vdpasim_setup_config, .get_generation = vdpasim_get_generation, .get_iova_range = vdpasim_get_iova_range, .set_map = vdpasim_set_map,
drivers/vdpa/vdpa_sim/vdpa_sim.h +2 -0
@@ -45,6 +45,8 @@ struct vdpasim_dev_attr {work_func_t work_fn;void (*get_config)(struct vdpasim *vdpasim,void *config);void (*set_config)(struct vdpasim *vdpasim,const void *config); +int (*setup_config)(struct vdpasim *vdpasim, +const struct vdpa_dev_set_config *config); };/* State of each vdpasim device */
drivers/vdpa/vdpa_sim/vdpa_sim_net.c +18 -16
@@ -29,12 +29,6 @@#define VDPASIM_NET_VQ_NUM 2 -static char *macaddr; -module_param (macaddr, charp,0 ); -MODULE_PARM_DESC (macaddr,"Ethernet MAC address" ); - -static u8 macaddr_buf[ETH_ALEN ]; -static void vdpasim_net_work (struct work_struct *work) { struct vdpasim *vdpasim = container_of(work, struct vdpasim, work);
@@ -112,9 +106,19 @@ static void vdpasim_net_get_config(struct vdpasim *vdpasim, void *config) {struct virtio_net_config *net_config = config; - net_config->mtu =cpu_to_vdpasim16 (vdpasim,1500 ); net_config->status =cpu_to_vdpasim16 (vdpasim,VIRTIO_NET_S_LINK_UP ); -memcpy (net_config->mac, macaddr_buf,ETH_ALEN ); +} + +static int vdpasim_net_setup_config (struct vdpasim *vdpasim, +const struct vdpa_dev_set_config *config) +{ +struct virtio_net_config *vio_config = vdpasim->config; + +if (config->net_mask.mac_valid) +memcpy (vio_config->mac, config->net.mac,ETH_ALEN ); +if (config->net_mask.mtu_valid) + vio_config->mtu =cpu_to_vdpasim16 (vdpasim, config->net.mtu); +return 0 ; }static void vdpasim_net_mgmtdev_release (struct device *dev)
@@ -128,6 +132,7 @@ static struct device vdpasim_net_mgmtdev = {static int vdpasim_net_dev_add (struct vdpa_mgmt_dev *mdev,const char *name) { +struct virtio_net_config *vio_config;struct vdpasim_dev_attr dev_attr = {};struct vdpasim *simdev;int ret;
@@ -139,6 +144,7 @@ static int vdpasim_net_dev_add(struct vdpa_mgmt_dev *mdev, const char *name) dev_attr.nvqs =VDPASIM_NET_VQ_NUM ; dev_attr.config_size =sizeof (struct virtio_net_config ); dev_attr.get_config = vdpasim_net_get_config; + dev_attr.setup_config = vdpasim_net_setup_config; dev_attr.work_fn = vdpasim_net_work; dev_attr.buffer_size =PAGE_SIZE ;
@@ -146,6 +152,10 @@ static int vdpasim_net_dev_add(struct vdpa_mgmt_dev *mdev, const char *name)if (IS_ERR (simdev))return PTR_ERR (simdev); + vio_config = simdev->config; +/* Setup default MTU to be 1500 */ + vio_config->mtu =cpu_to_le16 (1500 ); + ret =_vdpa_register_device (&simdev->vdpa,VDPASIM_NET_VQ_NUM );if (ret) goto reg_err;
@@ -185,14 +195,6 @@ static int __init vdpasim_net_init(void) {int ret; -if (macaddr) { -mac_pton (macaddr, macaddr_buf); -if (!is_valid_ether_addr (macaddr_buf)) -return -EADDRNOTAVAIL ; - }else { -eth_random_addr (macaddr_buf); - } - ret =device_register (&vdpasim_net_mgmtdev);if (ret)return ret;
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v3 5/6 vdpa/mlx5: Support configuration of MAC
In reply to Parav PanditFrom: Eli Cohen <[email protected]>
Add code to accept MAC configuration through vdpa tool. The MAC is written into the config struct and later can be retrieved through get_config().
Examples:
1. Configure MAC:
$ vdpa dev config set vdpa0 mac 00:11:22:33:44:55
2. Show configured params:
$ vdpa dev config show vdpa0: mac 00:11:22:33:44:55 link down link_announce false mtu 0 speed 0 duplex 0
---
changelog:v2->v3:
- following new api for config space setup for mgmt toolv1->v2:
- following new api for config get/set for mgmt tool
--- drivers/vdpa/mlx5/net/mlx5_vnet.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
drivers/vdpa/mlx5/net/mlx5_vnet.c +25 -0
@@ -1831,6 +1831,30 @@ static void mlx5_vdpa_set_config(struct vdpa_device *vdev, unsigned int offset,/* not supported */ } +static int mlx5_vdpa_setup_config (struct vdpa_device *vdev, +const struct vdpa_dev_set_config *config) +{ +struct mlx5_vdpa_dev *mvdev =to_mvdev (vdev); +struct mlx5_vdpa_net *ndev =to_mlx5_vdpa_ndev (mvdev); +int err =0 ; + +mutex_lock (&ndev->reslock); +if (ndev->setup) + err = -EBUSY ; +mutex_unlock (&ndev->reslock); + +if (err) +return err; + +if (config->net_mask.mtu_valid) +return -EOPNOTSUPP ; + +if (config->net_mask.mac_valid) +memcpy (ndev->config.mac, config->net.mac,ETH_ALEN ); + +return 0 ; +} +static u32 mlx5_vdpa_get_generation (struct vdpa_device *vdev) {struct mlx5_vdpa_dev *mvdev =to_mvdev (vdev);
@@ -1909,6 +1933,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { .get_config_size = mlx5_vdpa_get_config_size, .get_config = mlx5_vdpa_get_config, .set_config = mlx5_vdpa_set_config, + .setup_config = mlx5_vdpa_setup_config, .get_generation = mlx5_vdpa_get_generation, .set_map = mlx5_vdpa_set_map, .free = mlx5_vdpa_free,
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v3 6/6 vdpa/mlx5: Forward only packets with allowed MAC address
In reply to Parav PanditFrom: Eli Cohen <[email protected]>
Add rules to forward packets to the net device's TIR only if the destination MAC is equal to the configured MAC. This is required to prevent the netdevice from receiving traffic not destined to its configured MAC.
--- drivers/vdpa/mlx5/net/mlx5_vnet.c | 76 +++++++++++++++++++++++-------- 1 file changed, 58 insertions(+), 18 deletions(-)
drivers/vdpa/mlx5/net/mlx5_vnet.c +58 -18
@@ -148,7 +148,8 @@ struct mlx5_vdpa_net {struct mutex reslock;struct mlx5_flow_table *rxft;struct mlx5_fc *rx_counter; -struct mlx5_flow_handle *rx_rule; +struct mlx5_flow_handle *rx_rule_ucast; +struct mlx5_flow_handle *rx_rule_mcast;bool setup;u16 mtu; };
@@ -1296,21 +1297,33 @@ static int add_fwd_to_tir(struct mlx5_vdpa_net *ndev)struct mlx5_flow_table_attr ft_attr = {};struct mlx5_flow_act flow_act = {};struct mlx5_flow_namespace *ns; +struct mlx5_flow_spec *spec; +void *headers_c; +void *headers_v; +u8 *dmac_c; +u8 *dmac_v;int err; -/* for now, one entry, match all, forward to tir */ - ft_attr.max_fte =1 ; - ft_attr.autogroup.max_num_groups =1 ; + spec =kvzalloc (sizeof (*spec),GFP_KERNEL ); +if (!spec) +return -ENOMEM ; + + spec->match_criteria_enable =MLX5_MATCH_OUTER_HEADERS ; + ft_attr.max_fte =2 ; + ft_attr.autogroup.max_num_groups =2 ; ns =mlx5_get_flow_namespace (ndev->mvdev.mdev,MLX5_FLOW_NAMESPACE_BYPASS );if (!ns) { -mlx5_vdpa_warn (&ndev->mvdev,"get flow namespace\n" ); -return -EOPNOTSUPP ; +mlx5_vdpa_warn (&ndev->mvdev,"failed to get flow namespace\n" ); + err = -EOPNOTSUPP ; + goto err_ns; } ndev->rxft =mlx5_create_auto_grouped_flow_table (ns, &ft_attr); -if (IS_ERR (ndev->rxft)) -return PTR_ERR (ndev->rxft); +if (IS_ERR (ndev->rxft)) { + err =PTR_ERR (ndev->rxft); + goto err_ns; + } ndev->rx_counter =mlx5_fc_create (ndev->mvdev.mdev, false);if (IS_ERR (ndev->rx_counter)) {
@@ -1318,37 +1331,64 @@ static int add_fwd_to_tir(struct mlx5_vdpa_net *ndev) goto err_fc; } + headers_c =MLX5_ADDR_OF (fte_match_param, spec->match_criteria, outer_headers); + dmac_c =MLX5_ADDR_OF (fte_match_param, headers_c, outer_headers.dmac_47_16); +memset (dmac_c,0xff ,ETH_ALEN ); + headers_v =MLX5_ADDR_OF (fte_match_param, spec->match_value, outer_headers); + dmac_v =MLX5_ADDR_OF (fte_match_param, headers_v, outer_headers.dmac_47_16); +ether_addr_copy (dmac_v, ndev->config.mac); + flow_act.action =MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |MLX5_FLOW_CONTEXT_ACTION_COUNT ; dest[0 ].type =MLX5_FLOW_DESTINATION_TYPE_TIR ; dest[0 ].tir_num = ndev->res.tirn; dest[1 ].type =MLX5_FLOW_DESTINATION_TYPE_COUNTER ; dest[1 ].counter_id =mlx5_fc_id (ndev->rx_counter); - ndev->rx_rule =mlx5_add_flow_rules (ndev->rxft,NULL , &flow_act, dest,2 ); -if (IS_ERR (ndev->rx_rule)) { - err =PTR_ERR (ndev->rx_rule); - ndev->rx_rule =NULL ; - goto err_rule; + ndev->rx_rule_ucast =mlx5_add_flow_rules (ndev->rxft, spec, &flow_act, dest,2 ); + +if (IS_ERR (ndev->rx_rule_ucast)) { + err =PTR_ERR (ndev->rx_rule_ucast); + ndev->rx_rule_ucast =NULL ; + goto err_rule_ucast; + } + +memset (dmac_c,0 ,ETH_ALEN ); +memset (dmac_v,0 ,ETH_ALEN ); + dmac_c[0 ] =1 ; + dmac_v[0 ] =1 ; + flow_act.action =MLX5_FLOW_CONTEXT_ACTION_FWD_DEST ; + ndev->rx_rule_mcast =mlx5_add_flow_rules (ndev->rxft, spec, &flow_act, dest,1 ); +if (IS_ERR (ndev->rx_rule_mcast)) { + err =PTR_ERR (ndev->rx_rule_mcast); + ndev->rx_rule_mcast =NULL ; + goto err_rule_mcast; } +kvfree (spec);return 0 ; -err_rule: +err_rule_mcast: +mlx5_del_flow_rules (ndev->rx_rule_ucast); + ndev->rx_rule_ucast =NULL ; +err_rule_ucast:mlx5_fc_destroy (ndev->mvdev.mdev, ndev->rx_counter); err_fc:mlx5_destroy_flow_table (ndev->rxft); +err_ns: +kvfree (spec);return err; }static void remove_fwd_to_tir (struct mlx5_vdpa_net *ndev) { -if (!ndev->rx_rule) +if (!ndev->rx_rule_ucast)return ; -mlx5_del_flow_rules (ndev->rx_rule); +mlx5_del_flow_rules (ndev->rx_rule_mcast); + ndev->rx_rule_mcast =NULL ; +mlx5_del_flow_rules (ndev->rx_rule_ucast); + ndev->rx_rule_ucast =NULL ;mlx5_fc_destroy (ndev->mvdev.mdev, ndev->rx_counter);mlx5_destroy_flow_table (ndev->rxft); - - ndev->rx_rule =NULL ; }static void mlx5_vdpa_kick_vq (struct vdpa_device *vdev,u16 idx)
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerQuoted 5 lines · Parav Pandit
Currently user cannot set the mac address and mtu of the vdpa device. This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
If a vendor driver supports such configuration user can set it otherwise user gets unsupported error.
This makes sense to me overall. People are used to use netlink to set these parameters, and virtio does not necessarily have a way to set all device parameters - they can be RO in the config space.
Quoted 75 lines
vdpa mac address and mtu are device configuration layout fields. To keep interface generic enough for multiple types of vdpa devices, mac address and mtu setting is implemented as configuration layout config knobs.
This enables to use similar config layout for other virtio devices.An example of query & set of config layout fields for
vdpa_sim_netdriver:Configuration layout fields are set after device is created.
This enables user to change such fields at later point without destroying and recreating the device for new config.$ vdpa mgmtdev show vdpasim_net: supported_classes netAdd the device:
$ vdpa dev add name bar mgmtdev vdpasim_netConfigure mac address and mtu:
$ vdpa dev config set bar mac 00:11:22:33:44:55 mtu 9000In above command only mac address or only mtu can also be set.
View the config after setting:
$ vdpa dev config show bar: mac 00:11:22:33:44:55 link up link_announce false mtu 9000 speed 0 duplex 0Patch summary:
Patch-1 introduced and use helpers for get/set config area Patch-2 implement query device config layout
Patch-3 enanble user to set mac and mtu in config space Patch-4vdpa_sim_netimplements get and set of config layout Patch-5 mlx5 vdpa driver supports user provided mac config Patch-6 mlx5 vdpa driver uses user provided mac during rx flow steeringchangelog:
v2->v3:
- dropped patches which are merged
- simplified code to handle non transitional devices
v1->v2:
- new patches to fix kdoc comment to add new kdoc section
- new patch to have synchronized access to features and config space
- read whole net config layout instead of individual fields
- added error extack for unmanaged vdpa device
- fixed several endianness issues
- introduced vdpa device ops for get config which is synchronized with other get/set features ops and config ops
- fixed mtu range checking for max
- usingNLA_POLICY_ETH_ADDR
- set config moved to device ops instead of mgmtdev ops
- merged build and set to single routine
- ensuring that user hasNET_ADMINcapability for configuring network attributes
- using updated interface and callbacks for get/set config
- following new api for config get/set for mgmt tool in mlx5 vdpa driver
- fixes for accessing right SF dma device and bar address
- fix for mtu calculation
- fix for bit access in features
- fix for index restore with suspend/resume operationEli Cohen (2):
vdpa/mlx5: Support configuration of MAC vdpa/mlx5: Forward only packets with allowed MAC addressParav Pandit (4):
vdpa: Introduce and use vdpa device get, set config helpers
vdpa: Introduce query of device config layout
vdpa: Enable user to set mac and mtu of vdpa device
vdpa_sim_net: Enable user to set mac address and mtu
Quoted 12 lines
drivers/vdpa/mlx5/net/mlx5_vnet.c | 101 ++++++-- drivers/vdpa/vdpa.c | 337 +++++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.c | 13 ++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 2 + drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 34 +-- drivers/vhost/vdpa.c | 3 +- include/linux/vdpa.h | 38 +-- include/uapi/linux/vdpa.h | 12 + 8 files changed, 490 insertions(+), 50 deletions(-)Signature · 1 line
2.26.2
_______________________________________________
Virtualization mailing list [email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 11 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Thursday, August 5, 2021 3:28 PMQuoted 5 lines · Parav Pandit
Currently user cannot set the mac address and mtu of the vdpa device. This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
If a vendor driver supports such configuration user can set it otherwise user gets unsupported error.
Quoted 3 lines
This makes sense to me overall. People are used to use netlink to set these parameters, and virtio does not necessarily have a way to set all device parameters - they can be RO in the config space.
Yes. this series enables it to RO when driver doesn't support setting it. When driver supports it, it is RW.
Do I need to rebase + resend?
Please let me know.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerQuoted 15 lines · Parav Pandit
Quoted 11 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Thursday, August 5, 2021 3:28 PMQuoted 5 lines · Parav Pandit
Currently user cannot set the mac address and mtu of the vdpa device. This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
If a vendor driver supports such configuration user can set it otherwise user gets unsupported error.Quoted 3 lines
This makes sense to me overall. People are used to use netlink to set these parameters, and virtio does not necessarily have a way to set all device parameters - they can be RO in the config space.
Quoted 5 lines
Yes. this series enables it to RO when driver doesn't support setting it. When driver supports it, it is RW.
Do I need to rebase + resend?
Please let me know.
Can't hurt.
Signature · 6 lines
MST _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 10 lines · Michael S. Tsirkin
Quoted 5 lines · Parav Pandit
Currently user cannot set the mac address and mtu of the vdpa device. This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
If a vendor driver supports such configuration user can set it otherwise user gets unsupported error.This makes sense to me overall. People are used to use netlink to set these parameters, and virtio does not necessarily have a way to set all device parameters - they can be RO in the config space.
I don't get here, we need to care RO as well (e.g the max_virtqueue_pairs).
And do we really want netlink uAPI for virtio like:
enum vdpa_attr {
@@ -33,6 +34,16 @@ enum vdpa_attr {
VDPA_ATTR_DEV_MAX_VQS, /* u32 */
VDPA_ATTR_DEV_MAX_VQ_SIZE, /* u16 */
+ VDPA_ATTR_DEV_NET_CFG_MACADDR, /* binary */
+ VDPA_ATTR_DEV_NET_STATUS, /* u8 */
+ VDPA_ATTR_DEV_NET_CFG_MAX_VQP, /* u16 */
+ VDPA_ATTR_DEV_NET_CFG_MTU, /* u16 */
+ VDPA_ATTR_DEV_NET_CFG_SPEED, /* u16 */
+ VDPA_ATTR_DEV_NET_CFG_DUPLEX, /* u16 */
+ VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN, /* u8 */
+ VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN, /* u16 */
+ VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES, /* u32 */
+
/* new attributes must be added above here */ VDPA_ATTR_MAX,
};
Or virtio uAPI and make netlink a transport?
I prefer the latter since we will meet the similar issue at the hardware level when we want to create and provision virtio device dynamically.
Thanks
Quoted 2 lines
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerQuoted 8 lines · Jason Wang
Quoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Parav Pandit
Currently user cannot set the mac address and mtu of the vdpa device. This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
If a vendor driver supports such configuration user can set it otherwise user gets unsupported error.
Quoted 7 lines
Quoted 4 lines
This makes sense to me overall. People are used to use netlink to set these parameters, and virtio does not necessarily have a way to set all device parameters - they can be RO in the config space.
I don't get here, we need to care RO as well (e.g the
max_virtqueue_pairs).
Point I tried to make is, a virtio transport will not allow writing max_virtqueue_pairs, but when managing virtio VFs from a PF we do need to set it. Thus virtio devices need a new set of interfaces for managing them, it is not just a virtio transport.
Quoted 9 lines
And do we really want netlink uAPI for virtio like:
enum
vdpa_attr{(quoted diff) +3
-0@@ -33,6 +34,16 @@ enum vdpa_attr { VDPA_ATTR_DEV_MAX_VQS, /* u32 */ VDPA_ATTR_DEV_MAX_VQ_SIZE, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_MACADDR, /* binary */ + VDPA_ATTR_DEV_NET_STATUS, /* u8 */ + VDPA_ATTR_DEV_NET_CFG_MAX_VQP, /* u16 */
Review enum vdpa_attr { · line 48
Quoted 10 lines
(quoted diff) +7
-0+ VDPA_ATTR_DEV_NET_CFG_MTU, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_SPEED, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_DUPLEX, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN, /* u8 */ + VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES, /* u32 */ + /* new attributes must be added above here */ VDPA_ATTR_MAX, };
The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?
Quoted 6 lines
Or virtio uAPI and make netlink a transport?
I prefer the latter since we will meet the similar issue at the hardware level when we want to create and provision virtio device dynamically.
Thanks
Creating devices dynamically exists with e.g. vxlan. That is using IFLA_MTU IFLA_ADDRESS etc.
Quoted 3 lines
Quoted 3 lines
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 20 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Friday, August 6, 2021 2:12 PMQuoted 14 lines
enum
vdpa_attr{(quoted diff) +10
-0@@ -33,6 +34,16 @@ enum vdpa_attr { VDPA_ATTR_DEV_MAX_VQS, /* u32 */ VDPA_ATTR_DEV_MAX_VQ_SIZE, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_MACADDR, /* binary */ + VDPA_ATTR_DEV_NET_STATUS, /* u8 */ + VDPA_ATTR_DEV_NET_CFG_MAX_VQP, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_MTU, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_SPEED, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_DUPLEX, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN, /* u8 */ + VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES, /* u32 */ +
Quoted 8 lines
Review enum vdpa_attr { · line 48
Quoted 3 lines
(quoted diff) +0
-0/* new attributes must be added above here */ VDPA_ATTR_MAX, };The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?
As you know vdpa is the backend device for the front-end netdevice accessed by the ethtool. vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as, max_virtqueue_pairs = 4.
And the MAC address chosen by hypervisor in mac[6].
Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.
How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
I must be misunderstanding your comment about ethtool. Can you please explain?
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 33 lines · Parav Pandit
Quoted 17 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Friday, August 6, 2021 2:12 PMQuoted 13 lines
enum vdpa_attr {(quoted diff) +9
-0@@ -33,6 +34,16 @@ enum vdpa_attr { VDPA_ATTR_DEV_MAX_VQS, /* u32 */ VDPA_ATTR_DEV_MAX_VQ_SIZE, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_MACADDR, /* binary */ + VDPA_ATTR_DEV_NET_STATUS, /* u8 */ + VDPA_ATTR_DEV_NET_CFG_MAX_VQP, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_MTU, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_SPEED, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_DUPLEX, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN, /* u8 */ + VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES, /* u32 */Quoted 8 lines
Review enum vdpa_attr { · line 48
Quoted 4 lines
(quoted diff) +1
-0+ /* new attributes must be added above here */ VDPA_ATTR_MAX, };The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?As you know vdpa is the backend device for the front-end netdevice accessed by the ethtool. vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4.
And the MAC address chosen by hypervisor in mac[6].
Quoted 8 lines
Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
Reread the cover letter:
"
This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
"
It looks to me the mechanism that introduced in the series is not for provisioning but for post-creation configuration?
Quoted 3 lines
I must be misunderstanding your comment about ethtool. Can you please explain?
I guess the meaning is that, if the vDPA is assigned to guest, it's the charge of guest to configure the MTU/MAC/RSS via the existing management interface like ethtool/iproute2 netlink protocol. The control virtqueue is designed for this.
But if it was used for provisioning, it looks like another topic which should be done during the device creation.
Thanks
Quoted 1 line
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Hi Jason,
Quoted 35 lines
From: Jason Wang <[email protected]>
Sent: Monday, August 9, 2021 8:38 AMQuoted 5 lines
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4.
And the MAC address chosen by hypervisor in mac[6].Quoted 8 lines
Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
Reread the cover letter:
"
This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
"
It looks to me the mechanism that introduced in the series is not for provisioning but for post-creation configuration?
Quoted 3 lines
I must be misunderstanding your comment about ethtool. Can you please explain?
Quoted 7 lines
I guess the meaning is that, if the vDPA is assigned to guest, it's the charge of guest to configure the MTU/MAC/RSS via the existing management interface like ethtool/iproute2 netlink protocol. The control virtqueue is designed for this.
But if it was used for provisioning, it looks like another topic which should be done during the device creation.
We already discussed and agreed, that I should change these params as creation time params instead post-creation. We were waiting for Michael to respond if he is ok with either
(a) extendible individual param or
(b) should prefer to see a typecast based structure blob coming through netlink.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 39 lines · Parav Pandit
Hi Jason,
Quoted 37 lines
From: Jason Wang <[email protected]>
Sent: Monday, August 9, 2021 8:38 AMQuoted 5 lines
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4.
And the MAC address chosen by hypervisor in mac[6].Quoted 8 lines
Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
Reread the cover letter:
"
This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
"
It looks to me the mechanism that introduced in the series is not for provisioning but for post-creation configuration?
Quoted 2 lines
I must be misunderstanding your comment about ethtool. Can you please explain?
I guess the meaning is that, if the vDPA is assigned to guest, it's the charge of guest to configure the MTU/MAC/RSS via the existing management interface like ethtool/iproute2 netlink protocol. The control virtqueue is designed for this.
Quoted 6 lines
Quoted 2 lines
But if it was used for provisioning, it looks like another topic which should be done during the device creation.
We already discussed and agreed, that I should change these params as creation time params instead post-creation. We were waiting for Michael to respond if he is ok with either
(a) extendible individual param or
(b) should prefer to see a typecast based structure blob coming through netlink.
Right. So it's for creation time params.
Michael, please advise which way is better.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 85 lines
From: Eli Cohen <[email protected]>
Sent: Monday, August 9, 2021 10:51 AMQuoted 11 lines · Jason Wang
Quoted 9 lines · Parav Pandit
Quoted 9 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Friday, August 6, 2021 2:12 PMReview enum vdpa_attr { · line 36
Quoted 5 lines
enum vdpa_attr {(quoted diff) +1
-0@@ -33,6 +34,16 @@ enum vdpa_attr { VDPA_ATTR_DEV_MAX_VQS, /* u32 */ VDPA_ATTR_DEV_MAX_VQ_SIZE, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_MACADDR, /*binary */
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review enum vdpa_attr { · line 38
Quoted 2 lines
(quoted diff) +2
-0+ VDPA_ATTR_DEV_NET_STATUS, /* u8 */ + VDPA_ATTR_DEV_NET_CFG_MAX_VQP, /*u16 */
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Review enum vdpa_attr { · line 42
Quoted 4 lines
(quoted diff) +4
-0+ VDPA_ATTR_DEV_NET_CFG_MTU, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_SPEED, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_DUPLEX, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN, /* u8*/
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 43
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN, /*u16 */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 44
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES, /*u32 */
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Review enum vdpa_attr { · line 48
Quoted 4 lines
(quoted diff) +1
-0+ /* new attributes must be added above here */ VDPA_ATTR_MAX, };The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?As you know vdpa is the backend device for the front-end netdevice
accessed by the ethtool.
Quoted 21 lines
Quoted 7 lines
vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4.
And the MAC address chosen by hypervisor in mac[6].Quoted 8 lines
Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
Reread the cover letter:
"
Quoted 8 lines
This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
"
It looks to me the mechanism that introduced in the series is not for provisioning but for post-creation configuration?
The difference is subtle. In both cases you provide configuration.
Quoted 11 lines
Quoted 4 lines
I must be misunderstanding your comment about ethtool. Can you please explain?
I guess the meaning is that, if the vDPA is assigned to guest, it's the charge of guest to configure the MTU/MAC/RSS via the existing management interface like ethtool/iproute2 netlink protocol. The control virtqueue is designed for this.
Quoted 3 lines
I was under the impression that we want somehow to control the capablity if the guest to use arbitrary MAC addresses.
If this is is not required than I think control VQ is the mechanism to use.
How does the guest VM identify which unique mac address to set on this virtio net device when this is the only device in the VM? Usually hypervisor knows what mac to set for a VM.
How do you set up the config space of the vdpa device migrating from source to destination hypervisor? Is this done through qemu vhost framework to setup the config space?
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 95 lines
From: Eli Cohen <[email protected]>
Sent: Monday, August 9, 2021 11:28 AMQuoted 89 lines · Parav Pandit
Quoted 89 lines
From: Eli Cohen <[email protected]>
Sent: Monday, August 9, 2021 10:51 AMQuoted 11 lines · Jason Wang
Quoted 9 lines · Parav Pandit
Quoted 9 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Friday, August 6, 2021 2:12 PMReview enum vdpa_attr { · line 36
Quoted 5 lines
enum vdpa_attr {(quoted diff) +1
-0@@ -33,6 +34,16 @@ enum vdpa_attr { VDPA_ATTR_DEV_MAX_VQS, /* u32 */ VDPA_ATTR_DEV_MAX_VQ_SIZE, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_MACADDR, /*binary */
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review enum vdpa_attr { · line 38
Quoted 2 lines
(quoted diff) +2
-0+ VDPA_ATTR_DEV_NET_STATUS, /* u8 */ + VDPA_ATTR_DEV_NET_CFG_MAX_VQP, /*u16 */
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Review enum vdpa_attr { · line 42
Quoted 4 lines
(quoted diff) +4
-0+ VDPA_ATTR_DEV_NET_CFG_MTU, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_SPEED, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_DUPLEX, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN, /* u8*/
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 43
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN, /*u16 */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 44
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES, /*u32 */
Quoted 10 lines
Quoted 10 lines
Quoted 8 lines
Review enum vdpa_attr { · line 48
Quoted 4 lines
(quoted diff) +1
-0+ /* new attributes must be added above here */ VDPA_ATTR_MAX, };The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?As you know vdpa is the backend device for the front-end netdevice
accessed by the ethtool.
Quoted 21 lines
Quoted 7 lines
vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4. And the MAC address chosen by hypervisor in mac[6].Quoted 8 lines
Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
Reread the cover letter:
"
Quoted 8 lines
This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
"
It looks to me the mechanism that introduced in the series is not for provisioning but for post-creation configuration?
The difference is subtle. In both cases you provide configuration.
Quoted 11 lines
Quoted 4 lines
I must be misunderstanding your comment about ethtool. Can you please explain?
I guess the meaning is that, if the vDPA is assigned to guest, it's the charge of guest to configure the MTU/MAC/RSS via the existing management interface like ethtool/iproute2 netlink protocol. The control virtqueue is designed for this.
I was under the impression that we want somehow to control the capablity if the guest to use arbitrary MAC addresses.
If this is is not required than I think control VQ is the mechanism to use.
Quoted 8 lines
Quoted 1 line
How does the guest VM identify which unique mac address to set on this
virtio net device when this is the only device in the VM?
Quoted 1 line
Usually hypervisor knows what mac to set for a VM.
You don't need to know. You could use any MAC you want, if no one else is using it in your subnet, and everything will work fine. The point is do you want to allow the guest to choose its MAC. This has implications of security.
Lets assume for a moment that a guest VM is able to program a MAC of netdevice of virtio_device of net type. How does a VM know that a randomly chosen mac is not used in network when this VM doesn’t have any external connectivity?
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 121 lines
From: Eli Cohen <[email protected]>
Sent: Monday, August 9, 2021 11:38 AMQuoted 15 lines · Parav Pandit
Quoted 15 lines
From: Eli Cohen <[email protected]>
Sent: Monday, August 9, 2021 11:28 AMQuoted 9 lines · Parav Pandit
Quoted 9 lines
From: Eli Cohen <[email protected]>
Sent: Monday, August 9, 2021 10:51 AMQuoted 3 lines · Jason Wang
Quoted 1 line · Parav Pandit
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Friday, August 6, 2021 2:12 PMReview enum vdpa_attr { · line 36
Quoted 5 lines
enum vdpa_attr {(quoted diff) +1
-0@@ -33,6 +34,16 @@ enum vdpa_attr { VDPA_ATTR_DEV_MAX_VQS, /* u32 */ VDPA_ATTR_DEV_MAX_VQ_SIZE, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_MACADDR,/*Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
binary */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 37
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_STATUS, /* u8*/
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 38
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_MAX_VQP,/*Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
u16 */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 39
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_MTU, /*u16 */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 40
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_SPEED, /*u16 */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 41
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_DUPLEX, /*u16 */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 42
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN,/* u8Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
*/
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 43
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN,/*Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
u16 */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 44
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES,/*Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
u32 */
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Review enum vdpa_attr { · line 48
Quoted 4 lines
(quoted diff) +1
-0+ /* new attributes must be added above here */ VDPA_ATTR_MAX, };The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
As you know vdpa is the backend device for the front-end netdevice
accessed by the ethtool.
Quoted 5 lines
Quoted 5 lines
vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs=4.
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
And the MAC address chosen by hypervisor in mac[6].
Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the
defaults?
Quoted 30 lines
Quoted 30 lines
Quoted 30 lines
Quoted 30 lines
Quoted 6 lines
Reread the cover letter:
"
Quoted 8 lines
This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
"
It looks to me the mechanism that introduced in the series is not for provisioning but for post-creation configuration?
The difference is subtle. In both cases you provide configuration.
Quoted 11 lines
Quoted 4 lines
I must be misunderstanding your comment about ethtool. Can you please explain?
I guess the meaning is that, if the vDPA is assigned to guest, it's the charge of guest to configure the MTU/MAC/RSS via the existing management interface like ethtool/iproute2 netlink protocol. The control virtqueue is designed for this.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
I was under the impression that we want somehow to control the capablity if the guest to use arbitrary MAC addresses.
If this is is not required than I think control VQ is the mechanism touse.
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
How does the guest VM identify which unique mac address to set on this
virtio net device when this is the only device in the VM?
Quoted 1 line
Usually hypervisor knows what mac to set for a VM.
You don't need to know. You could use any MAC you want, if no one else is using it in your subnet, and everything will work fine. The point is do you want to allow the guest to choose its MAC. This has
Quoted 8 lines
implications of security.
Quoted 2 lines
Quoted 1 line
Lets assume for a moment that a guest VM is able to program a MAC of
netdevice of
virtio_deviceof net type.Quoted 1 line
How does a VM know that a randomly chosen mac is not used in network
when this VM doesn’t have any external connectivity?
There's no gurantee
Hence a VF users, relies on the hypervisor to setup a unique MAC in the network.
And advance guest VM who has a device capable to modify the MAC (for bonding and other use) can override the mac.
So I don’t see them as mutually exclusive capability.
Quoted 2 lines
but it's being used all over and Linux has a specific API to generate random MAC addresses:
eth_hw_addr_random().
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 127 lines · Parav Pandit
Quoted 127 lines
From: Eli Cohen <[email protected]>
Sent: Monday, August 9, 2021 11:38 AMQuoted 15 lines · Parav Pandit
Quoted 15 lines
From: Eli Cohen <[email protected]>
Sent: Monday, August 9, 2021 11:28 AMQuoted 9 lines · Parav Pandit
Quoted 9 lines
From: Eli Cohen <[email protected]>
Sent: Monday, August 9, 2021 10:51 AMQuoted 3 lines · Jason Wang
Quoted 1 line · Parav Pandit
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Friday, August 6, 2021 2:12 PMReview enum vdpa_attr { · line 36
Quoted 5 lines
enum vdpa_attr {(quoted diff) +1
-0@@ -33,6 +34,16 @@ enum vdpa_attr { VDPA_ATTR_DEV_MAX_VQS, /* u32 */ VDPA_ATTR_DEV_MAX_VQ_SIZE, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_MACADDR,/*Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
binary */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_STATUS, /* u8*/
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 38
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_MAX_VQP,/*Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
u16 */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_MTU, /*u16 */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 40
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_SPEED, /*u16 */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 41
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_DUPLEX, /*u16 */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review enum vdpa_attr { · line 42
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN,/* u8Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
*/
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN,/*Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
u16 */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
(quoted diff) +1
-0+ VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES,/*Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
u32 */
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Review enum vdpa_attr { · line 48
Quoted 4 lines
(quoted diff) +1
-0+ /* new attributes must be added above here */ VDPA_ATTR_MAX, };The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
As you know vdpa is the backend device for the front-end netdevice
accessed by the ethtool.
Quoted 5 lines
Quoted 5 lines
vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs=4.
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
And the MAC address chosen by hypervisor in mac[6].
Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the
defaults?
Quoted 30 lines
Quoted 30 lines
Quoted 30 lines
Quoted 30 lines
Quoted 6 lines
Reread the cover letter:
"
Quoted 8 lines
This patchset enables users to set the mac address and mtu of the vdpa device once the device is created.
"
It looks to me the mechanism that introduced in the series is not for provisioning but for post-creation configuration?
The difference is subtle. In both cases you provide configuration.
Quoted 11 lines
Quoted 4 lines
I must be misunderstanding your comment about ethtool. Can you please explain?
I guess the meaning is that, if the vDPA is assigned to guest, it's the charge of guest to configure the MTU/MAC/RSS via the existing management interface like ethtool/iproute2 netlink protocol. The control virtqueue is designed for this.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
I was under the impression that we want somehow to control the capablity if the guest to use arbitrary MAC addresses.
If this is is not required than I think control VQ is the mechanism touse.
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
How does the guest VM identify which unique mac address to set on this
virtio net device when this is the only device in the VM?
Quoted 1 line
Usually hypervisor knows what mac to set for a VM.
You don't need to know. You could use any MAC you want, if no one else is using it in your subnet, and everything will work fine. The point is do you want to allow the guest to choose its MAC. This has
implications of security.
Quoted 2 lines
Quoted 1 line
Lets assume for a moment that a guest VM is able to program a MAC of
netdevice of
virtio_deviceof net type.Quoted 1 line
How does a VM know that a randomly chosen mac is not used in network
Quoted 8 lines
Quoted 3 lines
when this VM doesn’t have any external connectivity?
There's no gurantee
Hence a VF users, relies on the hypervisor to setup a unique MAC in the network.
And advance guest VM who has a device capable to modify the MAC (for bonding and other use) can override the mac.
So I don’t see them as mutually exclusive capability.
Yes, we probably need both.
CVQ for post creation configuration and netlink API for provisioning.
Quoted 3 lines
Quoted 3 lines
but it's being used all over and Linux has a specific API to generate random MAC addresses:
eth_hw_addr_random().
Yes, but it uses local assignment bit, management may want use others.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerQuoted 5 lines · Jason Wang
Quoted 1 line
So I don’t see them as mutually exclusive capability.
Yes, we probably need both.
CVQ for post creation configuration and netlink API for provisioning.
To note when host wants to change post provisioning it will also use netlink I guess.
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
but it's being used all over and Linux has a specific API to generate random MAC addresses:
eth_hw_addr_random().Yes, but it uses local assignment bit, management may want use others.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerQuoted 43 lines · Parav Pandit
Quoted 20 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Friday, August 6, 2021 2:12 PMQuoted 14 lines
enum
vdpa_attr{(quoted diff) +10
-0@@ -33,6 +34,16 @@ enum vdpa_attr { VDPA_ATTR_DEV_MAX_VQS, /* u32 */ VDPA_ATTR_DEV_MAX_VQ_SIZE, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_MACADDR, /* binary */ + VDPA_ATTR_DEV_NET_STATUS, /* u8 */ + VDPA_ATTR_DEV_NET_CFG_MAX_VQP, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_MTU, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_SPEED, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_DUPLEX, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_RSS_MAX_KEY_LEN, /* u8 */ + VDPA_ATTR_DEV_NET_CFG_RSS_MAX_IT_LEN, /* u16 */ + VDPA_ATTR_DEV_NET_CFG_RSS_HASH_TYPES, /* u32 */ +Quoted 8 lines
Review enum vdpa_attr { · line 48
Quoted 3 lines
(quoted diff) +0
-0/* new attributes must be added above here */ VDPA_ATTR_MAX, };The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?As you know vdpa is the backend device for the front-end netdevice accessed by the ethtool. vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4.
And the MAC address chosen by hypervisor in mac[6].Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.
Quoted 5 lines
How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
I must be misunderstanding your comment about ethtool. Can you please explain?
I am basically saying that we probably want to be able to
change MAC of a VDPA device on the host without desroying and recreating the device as long as it's not in use.
For a VF I can do this on the host:
ip link set eth0 vf 1 mac 00:11:22:33:44:55
ideally same thing would work for vdpa.
Signature · 6 lines
MST _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 30 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, August 9, 2021 3:10 PMQuoted 9 lines · Parav Pandit
Quoted 7 lines
The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?As you know vdpa is the backend device for the front-end netdevice
accessed by the ethtool.
Quoted 16 lines
vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4.
And the MAC address chosen by hypervisor in mac[6].Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
Quoted 7 lines
Quoted 2 lines
I must be misunderstanding your comment about ethtool. Can you please explain?
I am basically saying that we probably want to be able to change MAC of a VDPA device on the host without desroying and recreating the device as long as it's not in use.
Ok. I understood your comment now.
Yes, this was the objective which is why they are present as independent config knob. Jason was suggesting to have them as creation only knobs, which requires recreate.
I don't have strong opinion for either method.
Passing them at creation time is simpler for user.
If user needs the ability to modify and reuse same device with different config, extending such support in future like this patch should possible.
So there are two questions to close.
1. Can we start with config params at vdpa device creation time?
2. Is it ok to have these config params as individual fields at netlink U->K UAPI level? This is the method proposed in this patch series.
(Similar to incrementally growing vxlan ip link command).
Or
They should be packed in a structure between U-> K and deal with typecasting based on size and more? (Jason's input).
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerQuoted 41 lines · Parav Pandit
Quoted 30 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, August 9, 2021 3:10 PMQuoted 9 lines · Parav Pandit
Quoted 7 lines
The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?As you know vdpa is the backend device for the front-end netdevice
accessed by the ethtool.
Quoted 16 lines
vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4.
And the MAC address chosen by hypervisor in mac[6].Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
Quoted 7 lines
Quoted 2 lines
I must be misunderstanding your comment about ethtool. Can you please explain?
I am basically saying that we probably want to be able to change MAC of a VDPA device on the host without desroying and recreating the device as long as it's not in use.
Ok. I understood your comment now.
Yes, this was the objective which is why they are present as independent config knob. Jason was suggesting to have them as creation only knobs, which requires recreate.
Quoted 7 lines
I don't have strong opinion for either method.
Passing them at creation time is simpler for user.
If user needs the ability to modify and reuse same device with different config, extending such support in future like this patch should possible.So there are two questions to close.
1. Can we start with config params at vdpa device creation time?
I'm not sure whether we need both but I'd like to see a full API and I think we all agree host wants ability to tweak mac after device creation even if guest is not allowed to change mac, right?
Quoted 7 lines
2. Is it ok to have these config params as individual fields at netlink
U->KUAPI level? This is the method proposed in this patch series.
(Similar to incrementally growing vxlan ip link command).Or
They should be packed in a structure between U-> K and deal with typecasting based on size and more? (Jason's input).
I'm inclined to say vxlan is closer to a model to follow.
Signature · 6 lines
MST _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 53 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Tuesday, August 17, 2021 2:24 AMQuoted 34 lines · Parav Pandit
Quoted 34 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, August 9, 2021 3:10 PMQuoted 9 lines · Parav Pandit
Quoted 7 lines
The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?As you know vdpa is the backend device for the front-end netdevice
accessed by the ethtool.
Quoted 18 lines
vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4. And the MAC address chosen by hypervisor in mac[6].Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
I must be misunderstanding your comment about ethtool. Can you please explain?
Quoted 5 lines
Quoted 3 lines
I am basically saying that we probably want to be able to change MAC of a VDPA device on the host without desroying and recreating the device as long as it's not in use.
Ok. I understood your comment now.
Yes, this was the objective which is why they are present as independentconfig knob.
Quoted 1 line
Jason was suggesting to have them as creation only knobs, which requires
recreate.
Quoted 5 lines
I don't have strong opinion for either method.
Passing them at creation time is simpler for user.
If user needs the ability to modify and reuse same device with different
Quoted 9 lines
config, extending such support in future like this patch should possible.
Quoted 3 lines
So there are two questions to close.
1. Can we start with config params at vdpa device creation time?I'm not sure whether we need both but I'd like to see a full API and I think we all agree host wants ability to tweak mac after device creation even if guest is not allowed to change mac, right?
Yes.
$ vdpa dev add name foo mgmtdev pci/0000:03:00.0 mac 00:11:22:33:44:55 maxvqs 8 mtu 9000
Above API if we do at creation time. It is likely simpler for user to pass necessary params during creation time.
Quoted 11 lines
Quoted 1 line
2. Is it ok to have these config params as individual fields at netlink
U->KUAPI level?
Quoted 5 lines
This is the method proposed in this patch series. (Similar to incrementally growing vxlan ip link command).
Or
They should be packed in a structure between U-> K and deal withtypecasting based on size and more?
Quoted 1 line
(Jason's input).
I'm inclined to say vxlan is closer to a model to follow.
Ok. thanks for the feedback. We are using the model close to vxlan.
Lets resolve should we have it at creation time, post creation or both?
(a) Creation time
Pros:
- simpler single api for user
- eliminates needs of inventing stats reset in future series
Cons:
- inability to reuse the device with different config
- This may not be of great advantage, and it is probably fine to have creation time params
(b) post creation time:
Pros:
- able to reuse the device with different config for say different VM.
- will require stats reset in future once stats are implemented
Cons:
- more commands for users to config a device, better to have the ability at create time.
Quoted 3 lines
--
MST
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 79 lines · Parav Pandit
Quoted 54 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Tuesday, August 17, 2021 2:24 AMQuoted 34 lines · Parav Pandit
Quoted 34 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, August 9, 2021 3:10 PMQuoted 9 lines · Parav Pandit
Quoted 7 lines
The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?As you know vdpa is the backend device for the front-end netdevice
accessed by the ethtool.
Quoted 18 lines
vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4. And the MAC address chosen by hypervisor in mac[6].Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
I must be misunderstanding your comment about ethtool. Can you please explain?
Quoted 5 lines
Quoted 3 lines
I am basically saying that we probably want to be able to change MAC of a VDPA device on the host without desroying and recreating the device as long as it's not in use.
Ok. I understood your comment now.
Yes, this was the objective which is why they are present as independentconfig knob.
Quoted 1 line
Jason was suggesting to have them as creation only knobs, which requires
recreate.
Quoted 5 lines
I don't have strong opinion for either method.
Passing them at creation time is simpler for user.
If user needs the ability to modify and reuse same device with differentQuoted 9 lines
config, extending such support in future like this patch should possible.
Quoted 3 lines
So there are two questions to close.
1. Can we start with config params at vdpa device creation time?I'm not sure whether we need both but I'd like to see a full API and I think we all agree host wants ability to tweak mac after device creation even if guest is not allowed to change mac, right?
Yes.
$ vdpa dev add name foo mgmtdev pci/0000:03:00.0 mac 00:11:22:33:44:55 maxvqs 8 mtu 9000Above API if we do at creation time. It is likely simpler for user to pass necessary params during creation time.
Quoted 11 lines
Quoted 1 line
2. Is it ok to have these config params as individual fields at netlink
U->KUAPI level?
Quoted 5 lines
This is the method proposed in this patch series. (Similar to incrementally growing vxlan ip link command).
Or
They should be packed in a structure between U-> K and deal withtypecasting based on size and more?
Quoted 1 line
(Jason's input).
I'm inclined to say vxlan is closer to a model to follow.
Quoted 8 lines
Ok. thanks for the feedback. We are using the model close to vxlan. Lets resolve should we have it at creation time, post creation or both?
(a) Creation time
Pros:
- simpler single api for user
- eliminates needs of inventing stats reset in future series Cons:
- inability to reuse the device with different config
This can be solved by destroying the instance and re-creating it with a different params?
Quoted 6 lines
- This may not be of great advantage, and it is probably fine to have creation time params
(b) post creation time:
Pros:
- able to reuse the device with different config for say different VM.
- will require stats reset in future once stats are implemented
Any reason for doing this other than re-creating the device?
Quoted 2 lines
Cons:
- more commands for users to config a device, better to have the ability at create time.
We probably need to support post creation but it should be device specific.
E.g we may support device resize for virtio-blk devices.
But it can be done on top I think.
Thanks
Quoted 5 lines
Quoted 4 lines
--
MST
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 5 lines
From: Jason Wang <[email protected]>
Sent: Wednesday, August 18, 2021 10:02 AM
On Wed, Aug 18, 2021 at 11:15 AM Parav Pandit <[email protected]> wrote:
[..]
Quoted 10 lines
Quoted 9 lines
Quoted 1 line
I'm inclined to say vxlan is closer to a model to follow.
Ok. thanks for the feedback. We are using the model close to vxlan. Lets resolve should we have it at creation time, post creation or both?
(a) Creation time
Pros:
- simpler single api for user
- eliminates needs of inventing stats reset in future series Cons:
- inability to reuse the device with different config
Quoted 3 lines
This can be solved by destroying the instance and re-creating it with a different params?
Yes, which is what I tried be say below.
Quoted 1 line
Quoted 1 line
- This may not be of great advantage, and it is probably fine to have creation time params
^^^^^ here.
Quoted 8 lines
Quoted 5 lines
(b) post creation time:
Pros:
- able to reuse the device with different config for say different VM.
- will require stats reset in future once stats are implementedAny reason for doing this other than re-creating the device?
No. Only reason I can think of is, device reconfig may be faster than recreate. But I weigh user simplicity more at the beginning and optimizations to bring later if required.
Quoted 5 lines
Quoted 2 lines
Cons:
- more commands for users to config a device, better to have the ability atcreate time.
We probably need to support post creation but it should be device specific.
True. Your below device resize is good example of it.
Quoted 4 lines
E.g we may support device resize for virtio-blk devices.
But it can be done on top I think.
I think so too.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 14 lines · Parav Pandit
Quoted 4 lines
From: Jason Wang <[email protected]>
Sent: Wednesday, August 18, 2021 10:02 AM
On Wed, Aug 18, 2021 at 11:15 AM Parav Pandit <[email protected]> wrote:[..]
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
I'm inclined to say vxlan is closer to a model to follow.
Ok. thanks for the feedback. We are using the model close to vxlan. Lets resolve should we have it at creation time, post creation or both?
(a) Creation time
Pros:
- simpler single api for user
- eliminates needs of inventing stats reset in future series Cons:
- inability to reuse the device with different config
Quoted 7 lines
Quoted 3 lines
This can be solved by destroying the instance and re-creating it with a different params?
Yes, which is what I tried be say below.
Quoted 1 line
Quoted 1 line
- This may not be of great advantage, and it is probably fine to have creation time params
^^^^^ here.
Oh right, I miss this since it belongs to the bullets of "cons".
Quoted 10 lines
Quoted 7 lines
Quoted 5 lines
(b) post creation time:
Pros:
- able to reuse the device with different config for say different VM.
- will require stats reset in future once stats are implementedAny reason for doing this other than re-creating the device?
No. Only reason I can think of is, device reconfig may be faster than recreate. But I weigh user simplicity more at the beginning and optimizations to bring later if required.
Right, but it looks to me we should only allow the post-creation changes only if it is allowed by the spec.
E.g if we allow mac to be modified by the guest, it is not expected to be changed from the host.
Quoted 11 lines
Quoted 5 lines
Quoted 2 lines
Cons:
- more commands for users to config a device, better to have the ability atcreate time.
We probably need to support post creation but it should be device specific.
True. Your below device resize is good example of it.
Quoted 3 lines
E.g we may support device resize for virtio-blk devices.
But it can be done on top I think.
I think so too.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerQuoted 92 lines · Jason Wang
Quoted 79 lines · Parav Pandit
Quoted 54 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Tuesday, August 17, 2021 2:24 AMQuoted 34 lines · Parav Pandit
Quoted 34 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, August 9, 2021 3:10 PMQuoted 9 lines · Parav Pandit
Quoted 7 lines
The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?As you know vdpa is the backend device for the front-end netdevice
accessed by the ethtool.
Quoted 18 lines
vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4. And the MAC address chosen by hypervisor in mac[6].Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
I must be misunderstanding your comment about ethtool. Can you please explain?
Quoted 5 lines
Quoted 3 lines
I am basically saying that we probably want to be able to change MAC of a VDPA device on the host without desroying and recreating the device as long as it's not in use.
Ok. I understood your comment now.
Yes, this was the objective which is why they are present as independentconfig knob.
Quoted 1 line
Jason was suggesting to have them as creation only knobs, which requires
recreate.
Quoted 5 lines
I don't have strong opinion for either method.
Passing them at creation time is simpler for user.
If user needs the ability to modify and reuse same device with differentQuoted 9 lines
config, extending such support in future like this patch should possible.
Quoted 3 lines
So there are two questions to close.
1. Can we start with config params at vdpa device creation time?I'm not sure whether we need both but I'd like to see a full API and I think we all agree host wants ability to tweak mac after device creation even if guest is not allowed to change mac, right?
Yes.
$ vdpa dev add name foo mgmtdev pci/0000:03:00.0 mac 00:11:22:33:44:55 maxvqs 8 mtu 9000Above API if we do at creation time. It is likely simpler for user to pass necessary params during creation time.
Quoted 11 lines
Quoted 1 line
2. Is it ok to have these config params as individual fields at netlink
U->KUAPI level?
Quoted 5 lines
This is the method proposed in this patch series. (Similar to incrementally growing vxlan ip link command).
Or
They should be packed in a structure between U-> K and deal withtypecasting based on size and more?
Quoted 1 line
(Jason's input).
I'm inclined to say vxlan is closer to a model to follow.
Quoted 8 lines
Ok. thanks for the feedback. We are using the model close to vxlan. Lets resolve should we have it at creation time, post creation or both?
(a) Creation time
Pros:
- simpler single api for user
- eliminates needs of inventing stats reset in future series Cons:
- inability to reuse the device with different configThis can be solved by destroying the instance and re-creating it with a different params?
Quoted 8 lines
Quoted 6 lines
- This may not be of great advantage, and it is probably fine to have creation time params
(b) post creation time:
Pros:
- able to reuse the device with different config for say different VM.
- will require stats reset in future once stats are implementedAny reason for doing this other than re-creating the device?
Permissions.
Quoted 16 lines
Quoted 2 lines
Cons:
- more commands for users to config a device, better to have the ability at create time.We probably need to support post creation but it should be device specific.
E.g we may support device resize for virtio-blk devices.
But it can be done on top I think.
Thanks
Quoted 5 lines
Quoted 4 lines
--
MST
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 83 lines · Michael S. Tsirkin
Quoted 82 lines · Jason Wang
Quoted 70 lines · Parav Pandit
Quoted 46 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Tuesday, August 17, 2021 2:24 AMQuoted 27 lines · Parav Pandit
Quoted 27 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, August 9, 2021 3:10 PMQuoted 6 lines · Parav Pandit
Quoted 5 lines
The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?As you know vdpa is the backend device for the front-end netdevice
accessed by the ethtool.
Quoted 16 lines
vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4. And the MAC address chosen by hypervisor in mac[6].Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the defaults?
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
I must be misunderstanding your comment about ethtool. Can you please explain?
I am basically saying that we probably want to be able to change MAC of a VDPA device on the host without desroying and recreating the device as long as it's not in use.
Ok. I understood your comment now.
Yes, this was the objective which is why they are present as independentconfig knob.
Quoted 1 line
Jason was suggesting to have them as creation only knobs, which requires
recreate.
Quoted 2 lines
I don't have strong opinion for either method.
Quoted 9 lines
Quoted 2 lines
Passing them at creation time is simpler for user.
If user needs the ability to modify and reuse same device with differentconfig, extending such support in future like this patch should possible.
Quoted 2 lines
So there are two questions to close.
1. Can we start with config params at vdpa device creation time?I'm not sure whether we need both but I'd like to see a full API and I think we all agree host wants ability to tweak mac after device creation even if guest is not allowed to change mac, right?
Yes.
$ vdpa dev add name foo mgmtdev pci/0000:03:00.0 mac 00:11:22:33:44:55 maxvqs 8 mtu 9000Above API if we do at creation time. It is likely simpler for user to pass necessary params during creation time.
Quoted 10 lines
Quoted 1 line
2. Is it ok to have these config params as individual fields at netlink
U->KUAPI level?
Quoted 5 lines
This is the method proposed in this patch series. (Similar to incrementally growing vxlan ip link command).
Or
They should be packed in a structure between U-> K and deal withtypecasting based on size and more?
Quoted 1 line
(Jason's input).
I'm inclined to say vxlan is closer to a model to follow.
Quoted 8 lines
Ok. thanks for the feedback. We are using the model close to vxlan. Lets resolve should we have it at creation time, post creation or both?
(a) Creation time
Pros:
- simpler single api for user
- eliminates needs of inventing stats reset in future series Cons:
- inability to reuse the device with different configThis can be solved by destroying the instance and re-creating it with a different params?
Quoted 8 lines
Quoted 7 lines
Quoted 6 lines
- This may not be of great advantage, and it is probably fine to have creation time params
(b) post creation time:
Pros:
- able to reuse the device with different config for say different VM.
- will require stats reset in future once stats are implementedAny reason for doing this other than re-creating the device?
Permissions.
I would expect that CAP_NET_ADMIN is required for both cases.
Or anything I miss here?
Thanks
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Cons:
- more commands for users to config a device, better to have the ability at create time.We probably need to support post creation but it should be device specific.
Quoted 8 lines
Quoted 8 lines
E.g we may support device resize for virtio-blk devices.
But it can be done on top I think.
Thanks
Quoted 2 lines
Quoted 2 lines
--
MST
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 106 lines
From: Jason Wang <[email protected]>
Sent: Thursday, August 19, 2021 9:52 AMQuoted 2 lines · Michael S. Tsirkin
Quoted 1 line · Jason Wang
On Wed, Aug 18, 2021 at 11:15 AM Parav Pandit <[email protected]>
Quoted 25 lines · wrote:
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Tuesday, August 17, 2021 2:24 AMQuoted 19 lines · Parav Pandit
Quoted 19 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, August 9, 2021 3:10 PMQuoted 7 lines · Parav Pandit
Quoted 5 lines
The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?As you know vdpa is the backend device for the front-end netdevice
accessed by the ethtool.
Quoted 7 lines
vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4. And the MAC address chosen by hypervisor in mac[6].Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the
defaults?
Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
Quoted 10 lines
Quoted 7 lines
Quoted 3 lines
I must be misunderstanding your comment about ethtool. Can you please explain?
I am basically saying that we probably want to be able to change MAC of a VDPA device on the host without desroying and recreating the device as long as it's not in use.
Ok. I understood your comment now.
Yes, this was the objective which is why they are present as independentconfig knob.
Quoted 2 lines
Jason was suggesting to have them as creation only knobs, which requires
recreate.
Quoted 5 lines
I don't have strong opinion for either method.
Passing them at creation time is simpler for user. If user needs the ability to modify and reuse same device with different
config, extending such support in future like this patch should possible.
Quoted 2 lines
So there are two questions to close.
1. Can we start with config params at vdpa device creation time?I'm not sure whether we need both but I'd like to see a full API and I think we all agree host wants ability to tweak mac after device creation even if guest is not allowed to change mac, right?
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Yes.
$ vdpa dev add name foo mgmtdev pci/0000:03:00.0 mac 00:11:22:33:44:55 maxvqs 8 mtu 9000Above API if we do at creation time. It is likely simpler for user to pass
necessary params during creation time.
Quoted 30 lines
Quoted 30 lines
Quoted 12 lines
Quoted 4 lines
Quoted 3 lines
2. Is it ok to have these config params as individual fields at netlink
U->KUAPI level?
Quoted 8 lines
Quoted 5 lines
This is the method proposed in this patch series. (Similar to incrementally growing vxlan ip link command).
Or
They should be packed in a structure between U-> K and deal withtypecasting based on size and more?
Quoted 1 line
(Jason's input).
I'm inclined to say vxlan is closer to a model to follow.
Quoted 8 lines
Ok. thanks for the feedback. We are using the model close to vxlan. Lets resolve should we have it at creation time, post creation or both?
(a) Creation time
Pros:
- simpler single api for user
- eliminates needs of inventing stats reset in future series Cons:
- inability to reuse the device with different configThis can be solved by destroying the instance and re-creating it with a different params?
Quoted 7 lines
- This may not be of great advantage, and it is probably fine to have creation time params
(b) post creation time:
Pros:
- able to reuse the device with different config for say different VM.
- will require stats reset in future once stats are implemented
Quoted 5 lines
Quoted 2 lines
Quoted 1 line
Any reason for doing this other than re-creating the device?
Permissions.
I would expect that
CAP_NET_ADMINis required for both cases.
Correct. Patch-3 in this series has the code for CAP_NET_ADMIN for setting the mac, snippet below. For vdpa net device addition we do not have the check yet.
You/Michael mentioned that QEMU runs without any permissions in some other thread.
Do you mean QEMU can run without these capabilities?
If yes, is it fair ask for non QEMU sw to setup the vdpa device which has the higher capabilities than QEMU and after that QEMU runs with lower capabilities?
+static int vdpa_dev_net_config_set(struct vdpa_device *vdev,
+ struct sk_buff *skb, struct genl_info *info) {
+ struct nlattr **nl_attrs = info->attrs;
+ struct vdpa_dev_set_config config = {};
+ const u8 *macaddr;
+ int err;
+
+ if (!netlink_capable(skb, CAP_NET_ADMIN))
+ return -EPERM;
_______________________________________________
Virtualization mailing list [email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 111 lines · Parav Pandit
Quoted 111 lines
From: Jason Wang <[email protected]>
Sent: Thursday, August 19, 2021 9:52 AMQuoted 2 lines · Michael S. Tsirkin
Quoted 1 line · Jason Wang
On Wed, Aug 18, 2021 at 11:15 AM Parav Pandit <[email protected]>
Quoted 25 lines · wrote:
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Tuesday, August 17, 2021 2:24 AMQuoted 19 lines · Parav Pandit
Quoted 19 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, August 9, 2021 3:10 PMQuoted 7 lines · Parav Pandit
Quoted 5 lines
The point is to try and not reinvent a dedicated vpda interface where a generic one exits.
E.g. for phy things such as mac speed etc, I think most people are using ethtool things right?As you know vdpa is the backend device for the front-end netdevice
accessed by the ethtool.
Quoted 7 lines
vdpa management tool here is composing the vdpa device.
For example creator (hypervisor) of the vdpa devices knows that a guest VM is given 4 vcpus, So hypervisor creates a vdpa devices with config space layout as,
max_virtqueue_pairs= 4. And the MAC address chosen by hypervisor in mac[6].Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Guest VM ethtool can still chose to use less number of channels.
Typically,
ethtool is for guest VM.
vdpa device is in hypevisor.How can hypervisor compose a vdpa device without any tool? How can it tell ethtool, what is supported and what are the
defaults?
Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
Quoted 10 lines
Quoted 7 lines
Quoted 3 lines
I must be misunderstanding your comment about ethtool. Can you please explain?
I am basically saying that we probably want to be able to change MAC of a VDPA device on the host without desroying and recreating the device as long as it's not in use.
Ok. I understood your comment now.
Yes, this was the objective which is why they are present as independentconfig knob.
Quoted 2 lines
Jason was suggesting to have them as creation only knobs, which requires
recreate.
Quoted 5 lines
I don't have strong opinion for either method.
Passing them at creation time is simpler for user. If user needs the ability to modify and reuse same device with different
config, extending such support in future like this patch should possible.
Quoted 2 lines
So there are two questions to close.
1. Can we start with config params at vdpa device creation time?I'm not sure whether we need both but I'd like to see a full API and I think we all agree host wants ability to tweak mac after device creation even if guest is not allowed to change mac, right?
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Yes.
$ vdpa dev add name foo mgmtdev pci/0000:03:00.0 mac 00:11:22:33:44:55 maxvqs 8 mtu 9000Above API if we do at creation time. It is likely simpler for user to pass
necessary params during creation time.
Quoted 32 lines
Quoted 23 lines
Quoted 12 lines
Quoted 4 lines
Quoted 3 lines
2. Is it ok to have these config params as individual fields at netlink
U->KUAPI level?
Quoted 8 lines
Quoted 5 lines
This is the method proposed in this patch series. (Similar to incrementally growing vxlan ip link command).
Or
They should be packed in a structure between U-> K and deal withtypecasting based on size and more?
Quoted 1 line
(Jason's input).
I'm inclined to say vxlan is closer to a model to follow.
Quoted 8 lines
Ok. thanks for the feedback. We are using the model close to vxlan. Lets resolve should we have it at creation time, post creation or both?
(a) Creation time
Pros:
- simpler single api for user
- eliminates needs of inventing stats reset in future series Cons:
- inability to reuse the device with different configThis can be solved by destroying the instance and re-creating it with a different params?
Quoted 8 lines
Quoted 7 lines
- This may not be of great advantage, and it is probably fine to have creation time params
(b) post creation time:
Pros:
- able to reuse the device with different config for say different VM.
- will require stats reset in future once stats are implementedAny reason for doing this other than re-creating the device?
Permissions.
Quoted 7 lines
Quoted 1 line
I would expect that
CAP_NET_ADMINis required for both cases.Correct. Patch-3 in this series has the code for
CAP_NET_ADMINfor setting the mac, snippet below. For vdpa net device addition we do not have the check yet.You/Michael mentioned that QEMU runs without any permissions in some other thread. Do you mean QEMU can run without these capabilities?
Yes.
Quoted 1 line
If yes, is it fair ask for non QEMU sw to setup the vdpa device which has the higher capabilities than QEMU and after that QEMU runs with lower capabilities?
Right, e.g it's the charge of libvirt or other privileged process to those kind of configuration.
So I don't see how it differs from device creation from the view of permission.
Thanks
Review
Quoted 11 lines
(quoted diff) +9
-0+static int vdpa_dev_net_config_set(struct vdpa_device *vdev, + struct sk_buff *skb, struct genl_info *info) { + struct nlattr **nl_attrs = info->attrs; + struct vdpa_dev_set_config config = {}; + const u8 *macaddr; + int err; + + if (!netlink_capable(skb, CAP_NET_ADMIN)) + return -EPERM;
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization