Skip to contentSkip to conversation

Provider/rxe: Fix regression to UD traffic

PATCH In review Bob Pearson 2 participants 3 messages Oct 15, 2020 – Oct 16, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Provider/rxe: Fix regression to UD traffic

Update enum rdma_network_type copy to match kernel version. Without this change provider/rxe will send incorrect network types to the kernel in send WQEs.

This fix keeps rxe functional but should be replaced by a better implementation.

---
 providers/rxe/rxe.h | 1 +
 1 file changed, 1 insertion(+)
providers/rxe/rxe.h +1 -0
index 96f4ee9c..46e6ce72 100644
@@ -44,6 +44,7 @@
 
 enum rdma_network_type {
 	RDMA_NETWORK_IB,
+	RDMA_NETWORK_ROCE_V1,
 	RDMA_NETWORK_IPV4,
 	RDMA_NETWORK_IPV6
 };
Signature · 1 line
2.25.1
maintainer
#1 lore raw
Quoted 11 lines · Bob Pearson

Update enum rdma_network_type copy to match kernel version. Without this change provider/rxe will send incorrect network types to the kernel in send WQEs.

This fix keeps rxe functional but should be replaced by a better implementation.

---
 providers/rxe/rxe.h | 1 +
 1 file changed, 1 insertion(+)

Well, we can't just break user space so the kernel has to change in some way to accommodate this.

Obviously rxe should not have uAPI stuff that is not in include/uapi/rdma, so lets just fix that directly.

Please confirm I did this right. The PR for this merge window must be sent Friday.

From 8b20e1ceea413c98cb98930cb549be390226320f Mon Sep 17 00:00:00 2001
From: Jason Gunthorpe <[email protected]>
Date: Thu, 15 Oct 2020 20:42:18 -0300
Subject: [PATCH] RDMA/rxe: Move the definitions for rxe_av.network_type to
uAPI

RXE was wrongly using an internal kernel enum as part of its uAPI, split this out into a dedicated uAPI enum just for RXE. It only uses the IPv4 and IPv6 values.

This was exposed by changing the internal kernel enum definition which broke RXE.

Fixes 1c15b4f2a42f ("RDMA/core: Modify enum ib_gid_type and enum rdma_network_type")
---
 drivers/infiniband/sw/rxe/rxe_net.c | 8 ++++----
 include/uapi/rdma/rdma_user_rxe.h   | 6 ++++++
 2 files changed, 10 insertions(+), 4 deletions(-)
drivers/infiniband/sw/rxe/rxe_net.c +4 -4
index 31b93e7e1e2f41..575e1a4ec82121 100644
@@ -133,14 +133,14 @@ static struct dst_entry *rxe_find_route(struct net_device *ndev,
 		if (dst)
 			dst_release(dst);
 
-		if (av->network_type == RDMA_NETWORK_IPV4) {
+		if (av->network_type == RXE_NETWORK_TYPE_IPV4) {
 			struct in_addr *saddr;
 			struct in_addr *daddr;
 
 			saddr = &av->sgid_addr._sockaddr_in.sin_addr;
 			daddr = &av->dgid_addr._sockaddr_in.sin_addr;
 			dst = rxe_find_route4(ndev, saddr, daddr);
-		} else if (av->network_type == RDMA_NETWORK_IPV6) {
+		} else if (av->network_type == RXE_NETWORK_TYPE_IPV6) {
 			struct in6_addr *saddr6;
 			struct in6_addr *daddr6;
 
@@ -442,7 +442,7 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av,
 	if (IS_ERR(attr))
 		return NULL;
 
-	if (av->network_type == RDMA_NETWORK_IPV4)
+	if (av->network_type == RXE_NETWORK_TYPE_IPV6)
 		hdr_len = ETH_HLEN + sizeof(struct udphdr) +
 			sizeof(struct iphdr);
 	else
@@ -469,7 +469,7 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av,
 	skb->dev	= ndev;
 	rcu_read_unlock();
 
-	if (av->network_type == RDMA_NETWORK_IPV4)
+	if (av->network_type == RXE_NETWORK_TYPE_IPV4)
 		skb->protocol = htons(ETH_P_IP);
 	else
 		skb->protocol = htons(ETH_P_IPV6);
include/uapi/rdma/rdma_user_rxe.h +6 -0
index d8f2e0e46daba7..e591d8c1f3cf10 100644
@@ -39,6 +39,11 @@
 #include <linux/in.h>
 #include <linux/in6.h>
 
+enum {
+	RXE_NETWORK_TYPE_IPV4 = 1,
+	RXE_NETWORK_TYPE_IPV6 = 2,
+};
+
 union rxe_gid {
 	__u8	raw[16];
 	struct {
@@ -57,6 +62,7 @@ struct rxe_global_route {
 
 struct rxe_av {
 	__u8			port_num;
+	/* From RXE_NETWORK_TYPE_* */
 	__u8			network_type;
 	__u8			dmac[6];
 	struct rxe_global_route	grh;
Signature · 1 line
2.28.0
#2 lore raw
Quoted 13 lines · Jason Gunthorpe
Quoted 11 lines · Bob Pearson

Update enum rdma_network_type copy to match kernel version. Without this change provider/rxe will send incorrect network types to the kernel in send WQEs.

This fix keeps rxe functional but should be replaced by a better implementation.

---
 providers/rxe/rxe.h | 1 +
 1 file changed, 1 insertion(+)
Quoted 8 lines

Well, we can't just break user space so the kernel has to change in some way to accommodate this.

Obviously rxe should not have uAPI stuff that is not in include/uapi/rdma, so lets just fix that directly.

Please confirm I did this right. The PR for this merge window must be sent Friday.

That works too. I am figuring out how to convert ah_handle to ib_ah in rdma_rxe. I think I've mostly got it. uobj_get_obj_read(type, id, attr) is close to what I need. The problem is that all the code in this area assumes it is getting called in a user verb API call so there is an attr to pass around. What we need is an API like rdma_get_obj(ucontext, type, id) since we need a way to get to the user api data structures. Better would be a pointer from qp to ucontext or ufile that would be set for user QPs. If we did this rxe just use handles from the core instead of creating its own databases of objects.

Your patch will get us going for now. The above is a longer project that won't fit in your short term schedule.

Bob