Skip to contentSkip to conversation

Provide master netdev to mana_ib

PATCH In review Konstantin Taranov 5 participants 7 messages Jul 1, 2024 – Jul 4, 2024

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

v2 cover Provide master netdev to mana_ib

From: Konstantin Taranov <[email protected]>

This patch series aims to allow mana_ib to get master netdev from mana. In the netvsc deployment, the VF netdev is enslaved by netvsc and the upper device should be considered for the network state. In the baremetal case, the VF netdev is a master device.

I would appreciate if I could get Acks on it from:
* netvsc maintainers (e.g., Haiyang)
* net maintainers (e.g., Jakub, David, Eric, Paolo)

v1 -> v2:
Leon Romanovsky asked to make a helper in the net/mana and get acks from net maintainers.

Konstantin Taranov (2):
net: mana: introduce helper to get a master netdev RDMA/mana_ib: Set correct device into ib

 drivers/infiniband/hw/mana/device.c           | 16 ++++++++--------
 drivers/net/ethernet/microsoft/mana/mana_en.c | 18 ++++++++++++++++++
 include/net/mana/mana.h                       |  2 ++
 3 files changed, 28 insertions(+), 8 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 1/2 net: mana: introduce helper to get a master netdev

From: Konstantin Taranov <[email protected]>

Add helper to get a master netdevice for a given port. When mana is used with netvsc, the VF netdev is controlled by an upper netvsc device. In a baremetal case, the VF netdev is the master device.

---
 drivers/net/ethernet/microsoft/mana/mana_en.c | 18 ++++++++++++++++++
 include/net/mana/mana.h                       |  2 ++
 2 files changed, 20 insertions(+)
drivers/net/ethernet/microsoft/mana/mana_en.c +18 -0
index d087cf9..b893339 100644
@@ -2948,3 +2948,21 @@ out:
 	gd->gdma_context = NULL;
 	kfree(ac);
 }
+
+/* the caller should hold rcu_read_lock */
+struct net_device *mana_get_master_netdev_rcu(struct mana_context *ac, u32 port_index)
+{
+	struct net_device *ndev;
+
+	if (port_index >= ac->num_ports)
+		return NULL;
+
+	/* When mana is used in netvsc, the upper netdevice should be returned. */
+	if (ac->ports[port_index]->flags & IFF_SLAVE)
+		ndev = netdev_master_upper_dev_get_rcu(ac->ports[port_index]);
+	else
+		ndev = ac->ports[port_index];
+
+	return ndev;
+}
+EXPORT_SYMBOL_NS(mana_get_master_netdev_rcu, NET_MANA);
include/net/mana/mana.h +2 -0
index 46f741e..2d3625c 100644
@@ -797,4 +797,6 @@ void mana_destroy_wq_obj(struct mana_port_context *apc, u32 wq_type,
 int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id,
 		   u32 doorbell_pg_id);
 void mana_uncfg_vport(struct mana_port_context *apc);
+
+struct net_device *mana_get_master_netdev_rcu(struct mana_context *ac, u32 port_index);
 #endif /* _MANA_H */
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH net-next v2 1/2] net: mana: introduce helper to get a master netdev

Quoted 18 lines · Konstantin Taranov

From: Konstantin Taranov <[email protected]>

Add helper to get a master netdevice for a given port. When mana is used with netvsc, the VF netdev is controlled by an upper netvsc device. In a baremetal case, the VF netdev is the master device.

---
  drivers/net/ethernet/microsoft/mana/mana_en.c | 18 ++++++++++++++++++
  include/net/mana/mana.h                       |  2 ++
  2 files changed, 20 insertions(+)
drivers/net/ethernet/microsoft/mana/mana_en.c +0 -0
index d087cf9..b893339 100644

drivers/net/ethernet/microsoft/mana/mana_en.c out: · line 2955

Quoted 8 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +5 -0
@@ -2948,3 +2948,21 @@ out:
  	gd->gdma_context = NULL;
  	kfree(ac);
  }
+
+/* the caller should hold rcu_read_lock */
+struct net_device *mana_get_master_netdev_rcu(struct mana_context *ac, u32 port_index)
+{
+	struct net_device *ndev;

From the comments, this function requires rcu_read_lock. Can the following be added in this function?

RCU_LOCKDEP_WARN(!rcu_read_lock_held(), "no rcu read lock held");

If rcu_read_lock is not held, this function will notify the caller.
This can explicitly ensure that rcu_read_lock should be held before this fuction is called.

Best Regards,

Zhu Yanjun

Quoted 24 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +13 -0
+
+	if (port_index >= ac->num_ports)
+		return NULL;
+
+	/* When mana is used in netvsc, the upper netdevice should be returned. */
+	if (ac->ports[port_index]->flags & IFF_SLAVE)
+		ndev = netdev_master_upper_dev_get_rcu(ac->ports[port_index]);
+	else
+		ndev = ac->ports[port_index];
+
+	return ndev;
+}
+EXPORT_SYMBOL_NS(mana_get_master_netdev_rcu, NET_MANA);
include/net/mana/mana.h +2 -0
index 46f741e..2d3625c 100644
@@ -797,4 +797,6 @@ void mana_destroy_wq_obj(struct mana_port_context *apc, u32 wq_type,
  int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id,
  		   u32 doorbell_pg_id);
  void mana_uncfg_vport(struct mana_port_context *apc);
+
+struct net_device *mana_get_master_netdev_rcu(struct mana_context *ac, u32 port_index);
  #endif /* _MANA_H */
#3 lore raw

Re: [PATCH net-next v2 1/2] net: mana: introduce helper to get a master netdev

Quoted 8 lines

drivers/net/ethernet/microsoft/mana/mana_en.c out: · line 2955

Quoted 4 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +4 -0
+/* the caller should hold rcu_read_lock */ struct net_device
+*mana_get_master_netdev_rcu(struct mana_context *ac, u32 port_index)
+{
+	struct net_device *ndev;

From the comments, this function requires rcu_read_lock. Can the following
be added in this function?

Quoted 5 lines

RCU_LOCKDEP_WARN(!rcu_read_lock_held(), "no rcu read lock held");

If rcu_read_lock is not held, this function will notify the caller. This can explicitly ensure that rcu_read_lock should be held before this fuction is called.

Thanks! Great idea!

Quoted 5 lines

Best Regards,

Zhu Yanjun

#4 lore raw

PATCH v2 2/2 RDMA/mana_ib: Set correct device into ib

In reply to Konstantin Taranov

From: Konstantin Taranov <[email protected]>

Use the mana_get_master_netdev_rcu() helper to get a master netdevice for querying network states. The helper allows the mana_ib transparently support baremetal and netvsc deployment cases.

Fixes 8b184e4f1c32 ("RDMA/mana_ib: Enable RoCE on port 1")
---
 drivers/infiniband/hw/mana/device.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
drivers/infiniband/hw/mana/device.c +8 -8
index b07a8e2..ec849ac 100644
@@ -56,7 +56,7 @@ static int mana_ib_probe(struct auxiliary_device *adev,
 {
 	struct mana_adev *madev = container_of(adev, struct mana_adev, adev);
 	struct gdma_dev *mdev = madev->mdev;
-	struct net_device *upper_ndev;
+	struct net_device *ndev;
 	struct mana_context *mc;
 	struct mana_ib_dev *dev;
 	u8 mac_addr[ETH_ALEN];
@@ -84,17 +84,17 @@ static int mana_ib_probe(struct auxiliary_device *adev,
 	dev->ib_dev.num_comp_vectors = mdev->gdma_context->max_num_queues;
 	dev->ib_dev.dev.parent = mdev->gdma_context->dev;
 
-	rcu_read_lock(); /* required to get upper dev */
-	upper_ndev = netdev_master_upper_dev_get_rcu(mc->ports[0]);
-	if (!upper_ndev) {
+	rcu_read_lock(); /* required to get master dev */
+	ndev = mana_get_master_netdev_rcu(mc, 0);
+	if (!ndev) {
 		rcu_read_unlock();
 		ret = -ENODEV;
-		ibdev_err(&dev->ib_dev, "Failed to get master netdev");
+		ibdev_err(&dev->ib_dev, "Failed to get netdev for IB port 1");
 		goto free_ib_device;
 	}
-	ether_addr_copy(mac_addr, upper_ndev->dev_addr);
-	addrconf_addr_eui48((u8 *)&dev->ib_dev.node_guid, upper_ndev->dev_addr);
-	ret = ib_device_set_netdev(&dev->ib_dev, upper_ndev, 1);
+	ether_addr_copy(mac_addr, ndev->dev_addr);
+	addrconf_addr_eui48((u8 *)&dev->ib_dev.node_guid, ndev->dev_addr);
+	ret = ib_device_set_netdev(&dev->ib_dev, ndev, 1);
 	rcu_read_unlock();
 	if (ret) {
 		ibdev_err(&dev->ib_dev, "Failed to set ib netdev, ret %d", ret);
Signature · 1 line
2.43.0
Acked-by on v2 2/2
#5 lore raw

Re: [PATCH net-next v2 2/2] RDMA/mana_ib: Set correct device into ib

Quoted 7 lines · Konstantin Taranov

From: Konstantin Taranov <[email protected]>

Use the mana_get_master_netdev_rcu() helper to get a master netdevice for querying network states. The helper allows the mana_ib transparently support baremetal and netvsc deployment cases.

Quoted 6 lines
Fixes 8b184e4f1c32 ("RDMA/mana_ib: Enable RoCE on port 1")
---
 drivers/infiniband/hw/mana/device.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

Thanks,

maintainer
#6 lore raw
In reply to Konstantin Taranov
Quoted 8 lines · On Mon, 1 Jul 2024 05:58:54 -0700 Konstantin Taranov wrote:

This patch series aims to allow mana_ib to get master netdev from mana. In the netvsc deployment, the VF netdev is enslaved by netvsc and the upper device should be considered for the network state. In the baremetal case, the VF netdev is a master device.

I would appreciate if I could get Acks on it from:
* netvsc maintainers (e.g., Haiyang)
* net maintainers (e.g., Jakub, David, Eric, Paolo)

Would you like us to take this via net-next?
- it doesn't apply cleanly
- the patches should be squashed, they aren't separate logical changes
- please find a better word than master