RDMA/rxe: Cleanup rxe_init_packet()
Subsystems (2)
PATCH v2 1/1 RDMA/rxe: Cleanup rxe_init_packet()
Currently rxe_init_packet() recomputes ndev from the address vector but struct rxe_dev already holds a reference to ndev.
Cleanup rxe_init_packet to use the value of ndev in rxe and drop an unneeded parameter paylen since it is already carried in the packet info struct.
---
v2
Per Haris Iqbal remove redundant 'ack->paylen = paylen'
drivers/infiniband/sw/rxe/rxe_loc.h | 2 +- drivers/infiniband/sw/rxe/rxe_net.c | 49 +++++++++++----------------- drivers/infiniband/sw/rxe/rxe_req.c | 2 +- drivers/infiniband/sw/rxe/rxe_resp.c | 4 +-- 4 files changed, 23 insertions(+), 34 deletions(-)
drivers/infiniband/sw/rxe/rxe_loc.h +1 -1
@@ -93,7 +93,7 @@ void rxe_mw_cleanup(struct rxe_pool_elem *elem);/* rxe_net.c */ struct sk_buff *rxe_init_packet (struct rxe_dev *rxe,struct rxe_av *av, -int paylen,struct rxe_pkt_info *pkt); +struct rxe_pkt_info *pkt);int rxe_prepare (struct rxe_av *av,struct rxe_pkt_info *pkt,struct sk_buff *skb); int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt,
drivers/infiniband/sw/rxe/rxe_net.c +19 -30
74 lines not on this page · raw message
drivers/infiniband/sw/rxe/rxe_req.c +1 -1
@@ -397,7 +397,7 @@ static struct sk_buff *init_req_packet(struct rxe_qp *qp, pkt->paylen = paylen;/* init skb */ - skb =rxe_init_packet (rxe, av, paylen, pkt); + skb =rxe_init_packet (rxe, av, pkt);if (unlikely (!skb))return NULL ;
drivers/infiniband/sw/rxe/rxe_resp.c +2 -2
@@ -670,15 +670,15 @@ static struct sk_buff *prepare_ack_packet(struct rxe_qp *qp,*/ pad = (-payload) &0x3 ; paylen = rxe_opcode[opcode].length + payload + pad +RXE_ICRC_SIZE ; + ack->paylen = paylen; - skb =rxe_init_packet (rxe, &qp->pri_av, paylen, ack); + skb =rxe_init_packet (rxe, &qp->pri_av, ack);if (!skb)return NULL ; ack->qp = qp; ack->opcode = opcode; ack->mask = rxe_opcode[opcode].mask; - ack->paylen = paylen; ack->psn = psn; bth_init(ack, opcode, 0, 0, pad, IB_DEFAULT_PKEY_FULL,
Signature · 1 line
2.34.1
Jason Gunthorpe
maintainerdrivers/infiniband/sw/rxe/rxe_net.c int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt, · line 457
Quoted 10 lines · Bob Pearson
drivers/infiniband/sw/rxe/rxe_net.c +1
-6unsigned int hdr_len;struct sk_buff *skb =NULL ; -struct net_device *ndev; -const struct ib_gid_attr *attr; +struct net_device *ndev = rxe->ndev;const int port_num =1 ; - - attr =rdma_get_gid_attr (&rxe->ib_dev, port_num, av->grh.sgid_index); -if (IS_ERR (attr)) -return NULL ;
An ib_device can have many netdevs associated with the gid indexes, eg from VLANs or LAG. The core code creates these things
I think it is nonsense for rxe to work like this, and perhaps it doesn't work at all, but until rxe blocks creation of these other gid indexes I'm not sure it makes sense to delete this code..
Jason
Quoted 15 lines · Jason Gunthorpe
drivers/infiniband/sw/rxe/rxe_net.c int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt, · line 457
Quoted 10 lines · Bob Pearson
drivers/infiniband/sw/rxe/rxe_net.c +1
-6unsigned int hdr_len;struct sk_buff *skb =NULL ; -struct net_device *ndev; -const struct ib_gid_attr *attr; +struct net_device *ndev = rxe->ndev;const int port_num =1 ; - - attr =rdma_get_gid_attr (&rxe->ib_dev, port_num, av->grh.sgid_index); -if (IS_ERR (attr)) -return NULL ;An
ib_devicecan have many netdevs associated with the gid indexes, eg from VLANs or LAG. The core code creates these things
Quoted 5 lines
I think it is nonsense for rxe to work like this, and perhaps it doesn't work at all, but until rxe blocks creation of these other gid indexes I'm not sure it makes sense to delete this code..
Jason
Somehow I had the vague impression that rxe didn't support vlans but I just looked at the following commit
commit fd49ddaf7e266b5892d659eb99d9f77841e5b4c0
Author: Mohammad Heib <[email protected]>
Date: Tue Aug 11 18:04:15 2020 +0300
RDMA/rxe: prevent rxe creation on top of vlan interface
Creating rxe device on top of vlan interface will create a non-functional
device that has an empty gids table and can't be used for rdma cm
communication.
This is caused by the logic in
enum_all_gids_of_dev_cb()/is_eth_port_of_netdev(), which only considers
networks connected to "upper devices" of the configured network device,
resulting in an empty set of gids for a vlan interface, and attempts to
connect via this rdma device fail in cm_init_av_for_response because no
gids can be resolved.
Apparently, this behavior was implemented to fit the HW-RoCE devices that
create RoCE device per port, therefore RXE must behave the same like
HW-RoCE devices and create rxe device per real device only.
In order to communicate via a vlan interface, the user must use the gid
index of the vlan address instead of creating rxe over vlan.
which jibes with what you are saying. The immediate impact of this is that rxe->ndev should probably not be used unless you know you want the physical device.
Bob