Skip to contentSkip to conversation

rdma_rxe: fix bug rejecting multicast packets

PATCH Applied Bob Pearson 3 participants 6 messages Oct 8, 2020 – Oct 10, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 rdma_rxe: fix bug rejecting multicast packets

- Fix a bug in rxe_rcv that causes all multicast packets to be dropped. Currently rxe_match_dgid is called for each packet to verify that the destination IP address matches one of the entries in the port source GID table. This is incorrect for IP multicast addresses since they do not appear in the GID table.
- Add code to detect multicast addresses.
- Change function name to rxe_chk_dgid which is clearer.

---
 drivers/infiniband/sw/rxe/rxe_recv.c | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)
drivers/infiniband/sw/rxe/rxe_recv.c +16 -3
index a3eed4da1540..b6fee61b2aee 100644
@@ -280,7 +280,17 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb)
 	kfree_skb(skb);
 }
 
-static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
+/**
+ * rxe_chk_dgid - validate destination IP address
+ * @rxe: rxe device that received packet
+ * @skb: the received packet buffer
+ *
+ * Accept any loopback packets
+ * Extract IP address from packet and
+ * Accept if multicast packet
+ * Accept if matches an SGID table entry
+ */
+static int rxe_chk_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
 {
 	struct rxe_pkt_info *pkt = SKB_TO_PKT(skb);
 	const struct ib_gid_attr *gid_attr;
@@ -298,6 +308,9 @@ static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
 		pdgid = (union ib_gid *)&ipv6_hdr(skb)->daddr;
 	}
 
+	if (rdma_is_multicast_addr((struct in6_addr *)pdgid))
+		return 0;
+
 	gid_attr = rdma_find_gid_by_port(&rxe->ib_dev, pdgid,
 					 IB_GID_TYPE_ROCE_UDP_ENCAP,
 					 1, skb->dev);
@@ -322,8 +335,8 @@ void rxe_rcv(struct sk_buff *skb)
 	if (unlikely(skb->len < pkt->offset + RXE_BTH_BYTES))
 		goto drop;
 
-	if (rxe_match_dgid(rxe, skb) < 0) {
-		pr_warn_ratelimited("failed matching dgid\n");
+	if (rxe_chk_dgid(rxe, skb) < 0) {
+		pr_warn_ratelimited("failed checking dgid\n");
 		goto drop;
 	}
 
Signature · 1 line
2.25.1
maintainer Applied
#1 lore raw
Quoted 8 lines · Bob Pearson

- Fix a bug in rxe_rcv that causes all multicast packets to be dropped. Currently rxe_match_dgid is called for each packet to verify that the destination IP address matches one of the entries in the port source GID table. This is incorrect for IP multicast addresses since they do not appear in the GID table.
- Add code to detect multicast addresses.
- Change function name to rxe_chk_dgid which is clearer.

Quoted 4 lines
---
 drivers/infiniband/sw/rxe/rxe_recv.c | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

Looks right, was there 4 patches? I only see three

Applied to for-next

Thanks,
Jason

maintainer
#2 lore raw
In reply to Bob Pearson
Quoted 18 lines · Bob Pearson

- Fix a bug in rxe_rcv that causes all multicast packets to be dropped. Currently rxe_match_dgid is called for each packet to verify that the destination IP address matches one of the entries in the port source GID table. This is incorrect for IP multicast addresses since they do not appear in the GID table.
- Add code to detect multicast addresses.
- Change function name to rxe_chk_dgid which is clearer.

---
  drivers/infiniband/sw/rxe/rxe_recv.c | 19 ++++++++++++++++---
  1 file changed, 16 insertions(+), 3 deletions(-)
drivers/infiniband/sw/rxe/rxe_recv.c +0 -0
index a3eed4da1540..b6fee61b2aee 100644

drivers/infiniband/sw/rxe/rxe_recv.c static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) · line 288

Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_recv.c +6 -1
@@ -280,7 +280,17 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb)
  	kfree_skb(skb);
  }
 
-static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
+/**
+ * rxe_chk_dgid - validate destination IP address
+ * @rxe: rxe device that received packet
+ * @skb: the received packet buffer
+ *
+ * Accept any loopback packets

About loopback packets, will rdma_find_gid_by_port return correct value?

In my tests, to loopback packets, sometimes rdma_find_gid_by_port return incorrect value.

Then the packets will be freed.

Zhu Yanjun

Quoted 29 lines
drivers/infiniband/sw/rxe/rxe_recv.c +10 -2
+ * Extract IP address from packet and
+ * Accept if multicast packet
+ * Accept if matches an SGID table entry
+ */
+static int rxe_chk_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
  {
  	struct rxe_pkt_info *pkt = SKB_TO_PKT(skb);
  	const struct ib_gid_attr *gid_attr;
@@ -298,6 +308,9 @@ static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
  		pdgid = (union ib_gid *)&ipv6_hdr(skb)->daddr;
  	}
 
+	if (rdma_is_multicast_addr((struct in6_addr *)pdgid))
+		return 0;
+
  	gid_attr = rdma_find_gid_by_port(&rxe->ib_dev, pdgid,
  					 IB_GID_TYPE_ROCE_UDP_ENCAP,
  					 1, skb->dev);
@@ -322,8 +335,8 @@ void rxe_rcv(struct sk_buff *skb)
  	if (unlikely(skb->len < pkt->offset + RXE_BTH_BYTES))
  		goto drop;
 
-	if (rxe_match_dgid(rxe, skb) < 0) {
-		pr_warn_ratelimited("failed matching dgid\n");
+	if (rxe_chk_dgid(rxe, skb) < 0) {
+		pr_warn_ratelimited("failed checking dgid\n");
  		goto drop;
  	}
 
maintainer
#3 lore raw
Quoted 13 lines · Zhu Yanjun
Quoted 12 lines · Bob Pearson

- Fix a bug in rxe_rcv that causes all multicast packets to be dropped. Currently rxe_match_dgid is called for each packet to verify that the destination IP address matches one of the entries in the port source GID table. This is incorrect for IP multicast addresses since they do not appear in the GID table.
- Add code to detect multicast addresses.
- Change function name to rxe_chk_dgid which is clearer.

  drivers/infiniband/sw/rxe/rxe_recv.c | 19 ++++++++++++++++---
  1 file changed, 16 insertions(+), 3 deletions(-)
Quoted 15 lines

drivers/infiniband/sw/rxe/rxe_recv.c static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) · line 287

Quoted 13 lines
drivers/infiniband/sw/rxe/rxe_recv.c +7 -1
index a3eed4da1540..b6fee61b2aee 100644
+++ b/drivers/infiniband/sw/rxe/rxe_recv.c
@@ -280,7 +280,17 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb)
  	kfree_skb(skb);
  }
-static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
+/**
+ * rxe_chk_dgid - validate destination IP address
+ * @rxe: rxe device that received packet
+ * @skb: the received packet buffer
+ *
+ * Accept any loopback packets

About loopback packets, will rdma_find_gid_by_port return correct value?

I don't think you can use 127.0.0.0 with the RDMA devices, at least not on the wire. The CM has special code to swap it out with a real device address

Jason

#4 lore raw
Quoted 14 lines · Jason Gunthorpe
Quoted 13 lines · Zhu Yanjun
Quoted 12 lines · Bob Pearson

- Fix a bug in rxe_rcv that causes all multicast packets to be dropped. Currently rxe_match_dgid is called for each packet to verify that the destination IP address matches one of the entries in the port source GID table. This is incorrect for IP multicast addresses since they do not appear in the GID table.
- Add code to detect multicast addresses.
- Change function name to rxe_chk_dgid which is clearer.

  drivers/infiniband/sw/rxe/rxe_recv.c | 19 ++++++++++++++++---
  1 file changed, 16 insertions(+), 3 deletions(-)
Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
drivers/infiniband/sw/rxe/rxe_recv.c +1 -1
index a3eed4da1540..b6fee61b2aee 100644
+++ b/drivers/infiniband/sw/rxe/rxe_recv.c
@@ -280,7 +280,17 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb)
  	kfree_skb(skb);
  }
-static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
Quoted 8 lines

drivers/infiniband/sw/rxe/rxe_recv.c static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) · line 288

Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_recv.c +6 -0
+/**
+ * rxe_chk_dgid - validate destination IP address
+ * @rxe: rxe device that received packet
+ * @skb: the received packet buffer
+ *
+ * Accept any loopback packets

About loopback packets, will rdma_find_gid_by_port return correct value?

I didn't touch that but the RXE_LOOPBACK test comes before the call to rdma_find_gid_by_port so it should never get called for loopback packets.

Quoted 4 lines

I don't think you can use 127.0.0.0 with the RDMA devices, at least not on the wire. The CM has special code to swap it out with a real device address

The following does work:

$ ib_send_bw -d rxe_0 (in window A)                $ ib_send_bw -d rxe_0 127.0.0.1 (in window B)

This uses the LOOPBACK path and just hands the skb from sender to receiver. It never touches the IP stack.

I have never been able to get this to work:

$ ib_send_bw -d rxe_1 (at 10.0.0.1 in window A)    $ ib_send_bw -d rxe_2 10.0.0.1 (at 10.0.0.2 in window B)

If it did work I could test the full path.

Quoted 3 lines

Jason

maintainer
#5 lore raw
Quoted 29 lines · Bob Pearson
Quoted 14 lines · Jason Gunthorpe
Quoted 13 lines · Zhu Yanjun
Quoted 12 lines · Bob Pearson

- Fix a bug in rxe_rcv that causes all multicast packets to be dropped. Currently rxe_match_dgid is called for each packet to verify that the destination IP address matches one of the entries in the port source GID table. This is incorrect for IP multicast addresses since they do not appear in the GID table.
- Add code to detect multicast addresses.
- Change function name to rxe_chk_dgid which is clearer.

   drivers/infiniband/sw/rxe/rxe_recv.c | 19 ++++++++++++++++---
   1 file changed, 16 insertions(+), 3 deletions(-)
Quoted 14 lines
Quoted 6 lines
Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_recv.c +1 -0
index a3eed4da1540..b6fee61b2aee 100644
+++ b/drivers/infiniband/sw/rxe/rxe_recv.c
@@ -280,7 +280,17 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb)
   	kfree_skb(skb);
   }
Quoted 8 lines

drivers/infiniband/sw/rxe/rxe_recv.c static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) · line 288

Quoted 7 lines
drivers/infiniband/sw/rxe/rxe_recv.c +6 -1
-static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
+/**
+ * rxe_chk_dgid - validate destination IP address
+ * @rxe: rxe device that received packet
+ * @skb: the received packet buffer
+ *
+ * Accept any loopback packets

About loopback packets, will rdma_find_gid_by_port return correct value?

Quoted 2 lines

I didn't touch that but the RXE_LOOPBACK test comes before the call to rdma_find_gid_by_port so it should never get called for loopback packets.

I confronted the loopback problem with rdma-core tests.

And I made a patch to fix it. If the following commit exists, this problem will not occur.

commit 5c99274be8864519328aa74bc550ba410095bc1c
Author: Zhu Yanjun <[email protected]>
Date:   Tue Jun 30 15:36:05 2020 +0300

    RDMA/rxe: Skip dgid check in loopback mode

    In the loopback tests, the following call trace occurs.

     Call Trace:
      __rxe_do_task+0x1a/0x30 [rdma_rxe]
      rxe_qp_destroy+0x61/0xa0 [rdma_rxe]
      rxe_destroy_qp+0x20/0x60 [rdma_rxe]
      ib_destroy_qp_user+0xcc/0x220 [ib_core]
      uverbs_free_qp+0x3c/0xc0 [ib_uverbs]
      destroy_hw_idr_uobject+0x24/0x70 [ib_uverbs]
      uverbs_destroy_uobject+0x43/0x1b0 [ib_uverbs]
      uobj_destroy+0x41/0x70 [ib_uverbs]
      __uobj_get_destroy+0x39/0x70 [ib_uverbs]
      ib_uverbs_destroy_qp+0x88/0xc0 [ib_uverbs]
      ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0xb9/0xf0 [ib_uverbs]
      ib_uverbs_cmd_verbs+0xb16/0xc30 [ib_uverbs]

    The root cause is that the actual RDMA connection is not created in the loopback tests and the rxe_match_dgid will fail randomly.

    To fix this call trace which appear in the loopback tests, skip check of the dgid.

Quoted 16 lines
Quoted 3 lines

I don't think you can use 127.0.0.0 with the RDMA devices, at least not on the wire. The CM has special code to swap it out with a real device address

The following does work:

$ ib_send_bw -d rxe_0 (in window A)                $ ib_send_bw -d rxe_0 127.0.0.1 (in window B)

This uses the LOOPBACK path and just hands the skb from sender to receiver. It never touches the IP stack.

I have never been able to get this to work:

$ ib_send_bw -d rxe_1 (at 10.0.0.1 in window A)    $ ib_send_bw -d rxe_2 10.0.0.1 (at 10.0.0.2 in window B)

If it did work I could test the full path.

Quoted 2 lines

Jason