Skip to contentSkip to conversation

Revert "RDMA/rxe: Remove VLAN code leftovers from RXE"

PATCH Applied [email protected] 2 participants 3 messages Jan 20, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 Revert "RDMA/rxe: Remove VLAN code leftovers from RXE"

From: Martin Wilck <[email protected]>

This reverts commit b2d2440430c0fdd5e0cad3efd6d1c9e3d3d02e5b.

It's true that creating rxe on top of 802.1q interfaces doesn't work. Thus, commit fd49ddaf7e26 ("RDMA/rxe: prevent rxe creation on top of vlan interface") was absolutely correct.

But b2d2440430c0 was incorrect assuming that with this change, RDMA and VLAN don't work togehter at all. It just has to be set up differently. Rather than creating rxe on top of the VLAN interface, rxe must be created on top of the physical interface. RDMA then works just fine through VLAN interfaces on top of that physical interface, via the "upper device" logic.

I've tested this mainly with NVMe over RDMA and rping, but I don't see why it wouldn't work just as well for other protocols. If there are real issues, I'd like to know.

b2d2440430c0 broke this setup deliberately and should thus be reverted. Also, b2d2440430c0 removed rxe_dma_device() (which is indeed not necessary), but not its declaration in rxe_loc.h.

Fixes b2d2440430c0 ("RDMA/rxe: Remove VLAN code leftovers from RXE")
Cc Zhu Yanjun <[email protected]>
Cc Mohammad Heib <[email protected]>
Cc Nicolas Morey-Chaisemartin <[email protected]>
---
 drivers/infiniband/sw/rxe/rxe_loc.h  | 1 -
 drivers/infiniband/sw/rxe/rxe_net.c  | 6 ++++++
 drivers/infiniband/sw/rxe/rxe_resp.c | 5 +++++
 3 files changed, 11 insertions(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_loc.h +0 -1
index 0d758760b9ae..8adcef54e4b3 100644
@@ -116,7 +116,6 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av,
 				int paylen, struct rxe_pkt_info *pkt);
 int rxe_prepare(struct rxe_pkt_info *pkt, struct sk_buff *skb, u32 *crc);
 const char *rxe_parent_name(struct rxe_dev *rxe, unsigned int port_num);
-struct device *rxe_dma_device(struct rxe_dev *rxe);
 int rxe_mcast_add(struct rxe_dev *rxe, union ib_gid *mgid);
 int rxe_mcast_delete(struct rxe_dev *rxe, union ib_gid *mgid);
 
drivers/infiniband/sw/rxe/rxe_net.c +6 -0
index c4b06ced30a7..943914c2a50c 100644
@@ -8,6 +8,7 @@
 #include <linux/if_arp.h>
 #include <linux/netdevice.h>
 #include <linux/if.h>
+#include <linux/if_vlan.h>
 #include <net/udp_tunnel.h>
 #include <net/sch_generic.h>
 #include <linux/netfilter.h>
@@ -153,9 +154,14 @@ static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb)
 {
 	struct udphdr *udph;
 	struct net_device *ndev = skb->dev;
+	struct net_device *rdev = ndev;
 	struct rxe_dev *rxe = rxe_get_dev_from_net(ndev);
 	struct rxe_pkt_info *pkt = SKB_TO_PKT(skb);
 
+	if (!rxe && is_vlan_dev(rdev)) {
+		rdev = vlan_dev_real_dev(ndev);
+		rxe = rxe_get_dev_from_net(rdev);
+	}
 	if (!rxe)
 		goto drop;
 
drivers/infiniband/sw/rxe/rxe_resp.c +5 -0
index 5a098083a9d2..c7e3b6a4af38 100644
@@ -872,6 +872,11 @@ static enum resp_states do_complete(struct rxe_qp *qp,
 			else
 				wc->network_hdr_type = RDMA_NETWORK_IPV6;
 
+			if (is_vlan_dev(skb->dev)) {
+				wc->wc_flags |= IB_WC_WITH_VLAN;
+				wc->vlan_id = vlan_dev_vlan_id(skb->dev);
+			}
+
 			if (pkt->mask & RXE_IMMDT_MASK) {
 				wc->wc_flags |= IB_WC_WITH_IMM;
 				wc->ex.imm_data = immdt_imm(pkt);
Signature · 1 line
2.29.2
maintainer Applied
#1 lore raw
Quoted 29 lines · On Wed, Jan 20, 2021 at 05:19:13PM +0100, [email protected] wrote:

From: Martin Wilck <[email protected]>

This reverts commit b2d2440430c0fdd5e0cad3efd6d1c9e3d3d02e5b.

It's true that creating rxe on top of 802.1q interfaces doesn't work. Thus, commit fd49ddaf7e26 ("RDMA/rxe: prevent rxe creation on top of vlan interface") was absolutely correct.

But b2d2440430c0 was incorrect assuming that with this change, RDMA and VLAN don't work togehter at all. It just has to be set up differently. Rather than creating rxe on top of the VLAN interface, rxe must be created on top of the physical interface. RDMA then works just fine through VLAN interfaces on top of that physical interface, via the "upper device" logic.

I've tested this mainly with NVMe over RDMA and rping, but I don't see why it wouldn't work just as well for other protocols. If there are real issues, I'd like to know.

b2d2440430c0 broke this setup deliberately and should thus be reverted. Also, b2d2440430c0 removed rxe_dma_device() (which is indeed not necessary), but not its declaration in rxe_loc.h.

Fixes b2d2440430c0 ("RDMA/rxe: Remove VLAN code leftovers from RXE")
Cc Zhu Yanjun <[email protected]>
Cc Mohammad Heib <[email protected]>
Cc Nicolas Morey-Chaisemartin <[email protected]>
Quoted 5 lines
---
 drivers/infiniband/sw/rxe/rxe_loc.h  | 1 -
 drivers/infiniband/sw/rxe/rxe_net.c  | 6 ++++++
 drivers/infiniband/sw/rxe/rxe_resp.c | 5 +++++
 3 files changed, 11 insertions(+), 1 deletion(-)

Applied to for-rc, thanks

Jason

maintainer
#2 lore raw
In reply to [email protected]
Quoted 38 lines · On Wed, Jan 20, 2021 at 05:19:13PM +0100, [email protected] wrote:

From: Martin Wilck <[email protected]>

This reverts commit b2d2440430c0fdd5e0cad3efd6d1c9e3d3d02e5b.

It's true that creating rxe on top of 802.1q interfaces doesn't work. Thus, commit fd49ddaf7e26 ("RDMA/rxe: prevent rxe creation on top of vlan interface") was absolutely correct.

But b2d2440430c0 was incorrect assuming that with this change, RDMA and VLAN don't work togehter at all. It just has to be set up differently. Rather than creating rxe on top of the VLAN interface, rxe must be created on top of the physical interface. RDMA then works just fine through VLAN interfaces on top of that physical interface, via the "upper device" logic.

I've tested this mainly with NVMe over RDMA and rping, but I don't see why it wouldn't work just as well for other protocols. If there are real issues, I'd like to know.

b2d2440430c0 broke this setup deliberately and should thus be reverted. Also, b2d2440430c0 removed rxe_dma_device() (which is indeed not necessary), but not its declaration in rxe_loc.h.

Fixes b2d2440430c0 ("RDMA/rxe: Remove VLAN code leftovers from RXE")
Cc Zhu Yanjun <[email protected]>
Cc Mohammad Heib <[email protected]>
Cc Nicolas Morey-Chaisemartin <[email protected]>
 drivers/infiniband/sw/rxe/rxe_loc.h  | 1 -
 drivers/infiniband/sw/rxe/rxe_net.c  | 6 ++++++
 drivers/infiniband/sw/rxe/rxe_resp.c | 5 +++++
 3 files changed, 11 insertions(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_loc.h +1 -0
index 0d758760b9ae..8adcef54e4b3 100644
+++ b/drivers/infiniband/sw/rxe/rxe_loc.h

drivers/infiniband/sw/rxe/rxe_loc.h struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av, · line 120

Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_loc.h +0 -1
@@ -116,7 +116,6 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av,
 				int paylen, struct rxe_pkt_info *pkt);
 int rxe_prepare(struct rxe_pkt_info *pkt, struct sk_buff *skb, u32 *crc);
 const char *rxe_parent_name(struct rxe_dev *rxe, unsigned int port_num);
-struct device *rxe_dma_device(struct rxe_dev *rxe);
 int rxe_mcast_add(struct rxe_dev *rxe, union ib_gid *mgid);
 int rxe_mcast_delete(struct rxe_dev *rxe, union ib_gid *mgid);

I can't take this hunk as a -rc patch

But I took the rest to for-rc, thanks

Jason