Skip to contentSkip to conversation

RDMA/rxe: Receive wqe flush error completions on qp error

DISCUSSION Bob Pearson 3 participants 4 messages Oct 12, 2022 InfiniBand

lore
Share & tools
lore
#0 lore raw

Currently the rxe driver flushes the send queue on a transition of the qp to the error state and completes these wqes with flush errors but does not do this for the receive queue. The IBA spec requires that this happens. Is there a reason why this wasn't done or does it just not come up in practice?

Bob

#1 lore raw
Quoted 5 lines · Bob Pearson

Currently the rxe driver flushes the send queue on a transition of the qp to the error state and completes these wqes with flush errors but does not do this for the receive queue. The IBA spec requires that this happens. Is there a reason why this wasn't done or does it just not come up in practice?

It does of course happen in practise using HW HCAs. The ULP or user-space process need to free the resources used by the recv WRs. And it does so by receiving the work-completions (which in this case is FLUSHED_IN_ERROR).

Thxs, Håkon

#2 lore raw
Quoted 11 lines · Haakon Bugge
Quoted 5 lines · Bob Pearson

Currently the rxe driver flushes the send queue on a transition of the qp to the error state and completes these wqes with flush errors but does not do this for the receive queue. The IBA spec requires that this happens. Is there a reason why this wasn't done or does it just not come up in practice?

It does of course happen in practise using HW HCAs. The ULP or user-space process need to free the resources used by the recv WRs. And it does so by receiving the work-completions (which in this case is FLUSHED_IN_ERROR).

Thxs, Håkon

That is what I expected. I will try to fix this.

Bob

#3 lore raw
Quoted 7 lines · Bob Pearson
Quoted 6 lines · Haakon Bugge
Quoted 5 lines · Bob Pearson

Currently the rxe driver flushes the send queue on a transition of the qp to the error state and completes these wqes with flush errors but does not do this for the receive queue. The IBA spec requires that this happens. Is there a reason why this wasn't done or does it just not come up in practice?

Quoted 7 lines
Quoted 5 lines

It does of course happen in practise using HW HCAs. The ULP or user-space process need to free the resources used by the recv WRs. And it does so by receiving the work-completions (which in this case is FLUSHED_IN_ERROR).

Thxs, Håkon

That is what I expected. I will try to fix this.

That's definitely a bug to not flush the RQ. I'm a bit surprised the ULPs haven't been impacted by this.

Tom.