Provider/rxe: Fix regression to UD traffic
Subsystems (2)
PATCH 1/1 Provider/rxe: Fix regression to UD traffic
Update enum rdma_network_type copy to match kernel version. Without this change provider/rxe will send incorrect network types to the kernel in send WQEs.
This fix keeps rxe functional but should be replaced by a better implementation.
--- providers/rxe/rxe.h | 1 + 1 file changed, 1 insertion(+)
providers/rxe/rxe.h +1 -0
@@ -44,6 +44,7 @@enum rdma_network_type {RDMA_NETWORK_IB , +RDMA_NETWORK_ROCE_V1 ,RDMA_NETWORK_IPV4 ,RDMA_NETWORK_IPV6 };
Signature · 1 line
2.25.1
Jason Gunthorpe
maintainerQuoted 11 lines · Bob Pearson
Update enum
rdma_network_typecopy to match kernel version. Without this change provider/rxe will send incorrect network types to the kernel in send WQEs.This fix keeps rxe functional but should be replaced by a better implementation.
Signed-off-by Bob Pearson <[email protected]>--- providers/rxe/rxe.h | 1 + 1 file changed, 1 insertion(+)
Well, we can't just break user space so the kernel has to change in some way to accommodate this.
Obviously rxe should not have uAPI stuff that is not in include/uapi/rdma, so lets just fix that directly.
Please confirm I did this right. The PR for this merge window must be sent Friday.
From 8b20e1ceea413c98cb98930cb549be390226320f Mon Sep 17 00:00:00 2001
From: Jason Gunthorpe <[email protected]>
Date: Thu, 15 Oct 2020 20:42:18 -0300
Subject: [PATCH] RDMA/rxe: Move the definitions for rxe_av.network_type to
uAPI
RXE was wrongly using an internal kernel enum as part of its uAPI, split this out into a dedicated uAPI enum just for RXE. It only uses the IPv4 and IPv6 values.
This was exposed by changing the internal kernel enum definition which broke RXE.
--- drivers/infiniband/sw/rxe/rxe_net.c | 8 ++++---- include/uapi/rdma/rdma_user_rxe.h | 6 ++++++ 2 files changed, 10 insertions(+), 4 deletions(-)
drivers/infiniband/sw/rxe/rxe_net.c +4 -4
@@ -133,14 +133,14 @@ static struct dst_entry *rxe_find_route(struct net_device *ndev,if (dst)dst_release (dst); -if (av->network_type ==RDMA_NETWORK_IPV4 ) { +if (av->network_type ==RXE_NETWORK_TYPE_IPV4 ) {struct in_addr *saddr;struct in_addr *daddr; saddr = &av->sgid_addr._sockaddr_in.sin_addr; daddr = &av->dgid_addr._sockaddr_in.sin_addr; dst =rxe_find_route4 (ndev, saddr, daddr); - }else if (av->network_type ==RDMA_NETWORK_IPV6 ) { + }else if (av->network_type ==RXE_NETWORK_TYPE_IPV6 ) {struct in6_addr *saddr6;struct in6_addr *daddr6;
@@ -442,7 +442,7 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av,if (IS_ERR (attr))return NULL ; -if (av->network_type ==RDMA_NETWORK_IPV4 ) +if (av->network_type ==RXE_NETWORK_TYPE_IPV6 ) hdr_len =ETH_HLEN +sizeof (struct udphdr ) +sizeof (struct iphdr );else
@@ -469,7 +469,7 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av, skb->dev = ndev;rcu_read_unlock (); -if (av->network_type ==RDMA_NETWORK_IPV4 ) +if (av->network_type ==RXE_NETWORK_TYPE_IPV4 ) skb->protocol =htons (ETH_P_IP );else skb->protocol =htons (ETH_P_IPV6 );
include/uapi/rdma/rdma_user_rxe.h +6 -0
@@ -39,6 +39,11 @@#include <linux/in.h> #include <linux/in6.h> +enum { +RXE_NETWORK_TYPE_IPV4 =1 , +RXE_NETWORK_TYPE_IPV6 =2 , +}; +union rxe_gid {__u8 raw[16 ];struct {
@@ -57,6 +62,7 @@ struct rxe_global_route {struct rxe_av {__u8 port_num; +/* From RXE_NETWORK_TYPE_* */ __u8 network_type;__u8 dmac[6 ];struct rxe_global_route grh;
Signature · 1 line
2.28.0
Quoted 13 lines · Jason Gunthorpe
Quoted 11 lines · Bob Pearson
Update enum
rdma_network_typecopy to match kernel version. Without this change provider/rxe will send incorrect network types to the kernel in send WQEs.This fix keeps rxe functional but should be replaced by a better implementation.
Signed-off-by Bob Pearson <[email protected]>--- providers/rxe/rxe.h | 1 + 1 file changed, 1 insertion(+)
Quoted 8 lines
Well, we can't just break user space so the kernel has to change in some way to accommodate this.
Obviously rxe should not have uAPI stuff that is not in
include/uapi/rdma, so lets just fix that directly.Please confirm I did this right. The PR for this merge window must be sent Friday.
That works too. I am figuring out how to convert ah_handle to ib_ah in rdma_rxe. I think I've mostly got it. uobj_get_obj_read(type, id, attr) is close to what I need. The problem is that all the code in this area assumes it is getting called in a user verb API call so there is an attr to pass around. What we need is an API like rdma_get_obj(ucontext, type, id) since we need a way to get to the user api data structures. Better would be a pointer from qp to ucontext or ufile that would be set for user QPs. If we did this rxe just use handles from the core instead of creating its own databases of objects.
Your patch will get us going for now. The above is a longer project that won't fit in your short term schedule.
Bob