RXE fixes for 4.8

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/[email protected]>

Hi Doug,

Please find the following four patches from Yonatan which fix numerous issues in RXE code.

Thanks

Changelog v0->v1:
* Patch 1:
* removed second call to register_netdevice_notifier()
* fix indentation issues
* separated rxe_net_init() into two functions

Yonatan Cohen (4):
IB/rxe: Fix kernel panic in udp_setup_tunnel IB/rxe: Fix duplicate atomic request handling IB/rxe: Fix race condition between requester and completer IB/rxe: Fix kmem_cache leak

 drivers/infiniband/sw/rxe/rxe.c      | 25 ++++++++++++++--
 drivers/infiniband/sw/rxe/rxe_comp.c | 13 ++++++++
 drivers/infiniband/sw/rxe/rxe_net.c  | 55 ++++++++++++++++------------------
 drivers/infiniband/sw/rxe/rxe_net.h  |  5 +++-
 drivers/infiniband/sw/rxe/rxe_req.c  | 57 ++++++++++++++++++++++++++++--------
 drivers/infiniband/sw/rxe/rxe_resp.c | 11 +++----
 6 files changed, 114 insertions(+), 52 deletions(-)

--
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

Disable creation of a UDP socket for ipv6 when CONFIG_IPV6 is not enabeld. Since udp_sock_create6() returns 0 when CONFIG_IPV6 is not set

[   46.888632] IP: [<c220705a>] setup_udp_tunnel_sock+0x6/0x4f
[   46.891355] *pdpt = 0000000000000000 *pde = f000ff53f000ff53
[   46.893918] Oops: 0002 [#1] PREEMPT
[   46.896014] CPU: 0 PID: 1 Comm: swapper Not tainted 4.7.0-rc4-00001-g8700e3e #1
[   46.900280] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014
[   46.904905] task: cf06c040 ti: cf05e000 task.ti: cf05e000
[   46.907854] EIP: 0060:[<c220705a>] EFLAGS: 00210246 CPU: 0
[   46.911137] EIP is at setup_udp_tunnel_sock+0x6/0x4f
[   46.914070] EAX: 00000044 EBX: 00000001 ECX: cf05fef0 EDX: ca8142e0
[   46.917236] ESI: c2c4505b EDI: cf05fef0 EBP: cf05fed0 ESP: cf05fed0
[   46.919836]  DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068
[   46.922046] CR0: 80050033 CR2: 000001fc CR3: 02cec000 CR4: 000006b0
[   46.924550] Stack:
[   46.926014]  cf05ff10 c1fd4657 ca8142e0 0000000a 00000000 00000000 0000b712 00000008
[   46.931274]  00000000 6bb5bd01 c1fd48de 00000000 00000000 cf05ff1c 00000000 00000000
[   46.936122]  cf05ff1c c1fd4bdf 00000000 cf05ff28 c2c4507b ffffffff cf05ff88 c2bf1c74
[   46.942350] Call Trace:
[   46.944403]  [<c1fd4657>] rxe_setup_udp_tunnel+0x8f/0x99
[   46.947689]  [<c1fd48de>] ? net_to_rxe+0x4e/0x4e
[   46.950567]  [<c1fd4bdf>] rxe_net_init+0xe/0xa4
[   46.953147]  [<c2c4507b>] rxe_module_init+0x20/0x4c
[   46.955448]  [<c2bf1c74>] do_one_initcall+0x89/0x113
[   46.957797]  [<c2bf15eb>] ? set_debug_rodata+0xf/0xf
[   46.959966]  [<c2bf1dbc>] ? kernel_init_freeable+0xbe/0x15b
[   46.962262]  [<c2bf1ddc>] kernel_init_freeable+0xde/0x15b
[   46.964418]  [<c232eb54>] kernel_init+0x8/0xd0
[   46.966618]  [<c2333122>] ret_from_kernel_thread+0xe/0x24
[   46.969592]  [<c232eb4c>] ? rest_init+0x6f/0x6f

---
 drivers/infiniband/sw/rxe/rxe.c     | 25 +++++++++++++++--
 drivers/infiniband/sw/rxe/rxe_net.c | 55 +++++++++++++++++--------------------
 drivers/infiniband/sw/rxe/rxe_net.h |  5 +++-
 3 files changed, 51 insertions(+), 34 deletions(-)

index 55f0e8f..ddd5927 100644
@@ -362,15 +362,34 @@ static int __init rxe_module_init(void)
 		return err;
 	}
 
-	err = rxe_net_init();
+	err = rxe_net_ipv4_init();
 	if (err) {
-		pr_err("rxe: unable to init\n");
+		pr_err("rxe: unable to init ipv4 tunnel\n");
 		rxe_cache_exit();
-		return err;
+		goto exit;
+	}
+
+	err = rxe_net_ipv6_init();
+	if (err) {
+		pr_err("rxe: unable to init ipv6 tunnel\n");
+		rxe_cache_exit();
+		goto exit;
 	}
+
+	err = register_netdevice_notifier(&rxe_net_notifier);
+	if (err) {
+		pr_err("rxe: Failed to rigister netdev notifier\n");
+		goto exit;
+	}
+
 	pr_info("rxe: loaded\n");
 
 	return 0;
+
+exit:
+	rxe_release_udp_tunnel(recv_sockets.sk4);
+	rxe_release_udp_tunnel(recv_sockets.sk6);
+	return err;
 }
 
 static void __exit rxe_module_exit(void)

index 7b06f76..0daf7f0 100644
@@ -44,10 +44,13 @@ struct rxe_recv_sockets {
 };
 
 extern struct rxe_recv_sockets recv_sockets;
+extern struct notifier_block rxe_net_notifier;
+void rxe_release_udp_tunnel(struct socket *sk);
 
 struct rxe_dev *rxe_net_add(struct net_device *ndev);
 
-int rxe_net_init(void);
+int rxe_net_ipv4_init(void);
+int rxe_net_ipv6_init(void);
 void rxe_net_exit(void);
 
 #endif /* RXE_NET_H */

--
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

what an unnice way to settle a review discussion. I replied to all your arguments explaining
why the change log has to go, and the next step is to ignore this discussion and do it again
wrong w.o replying? so... again:

$ ./scripts/get_maintainer.pl drivers/infiniband/sw/rxe
Moni Shoua <monis-VPRAkNaXOzVWk0Htik3J/[email protected]> (supporter:SOFT-ROCE DRIVER (rxe))
Doug Ledford <[email protected]> (supporter:INFINIBAND SUBSYSTEM)
Sean Hefty <[email protected]> (supporter:INFINIBAND SUBSYSTEM)
Hal Rosenstock <[email protected]> (supporter:INFINIBAND SUBSYSTEM)
[email protected] (open list:SOFT-ROCE DRIVER (rxe))
[email protected] (open list)

Moni, can you please put your maintainer say here.

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

U29ycnkgT3IgYnV0IEkgbWlnaHQgaGF2ZSBtaXNzZWQgeW91ciByZXZpZXcgY29tbWVudCByZWdh cmRpbmcgdGhlIGNvbW1pdCBtZXNzYWdlDQpJIGNhbid0IGZpbmQgaXQgbm93IHNvIHBsZWFzZSBz ZW5kIGl0IGFnYWluDQoNClRoYW5rcw0KDQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpG cm9tOiBPciBHZXJsaXR6IFttYWlsdG86Z2VybGl0ei5vckBnbWFpbC5jb21dIA0KU2VudDogV2Vk bmVzZGF5LCBTZXB0ZW1iZXIgMDcsIDIwMTYgMjoyMSBQTQ0KVG86IExlb24gUm9tYW5vdnNreSA8 bGVvbnJvQG1lbGxhbm94LmNvbT47IE1vbmkgU2hvdWEgPG1vbmlzQG1lbGxhbm94LmNvbT4NCkNj OiBEb3VnIExlZGZvcmQgPGRsZWRmb3JkQHJlZGhhdC5jb20+OyBsaW51eC1yZG1hQHZnZXIua2Vy bmVsLm9yZzsgWW9uYXRhbiBDb2hlbiA8eW9uYXRhbmNAbWVsbGFub3guY29tPg0KU3ViamVjdDog UmU6IFtQQVRDSCByZG1hLXJjIFYxIDEvNF0gSUIvcnhlOiBGaXgga2VybmVsIHBhbmljIGluIHVk cF9zZXR1cF90dW5uZWwNCg0KT24gV2VkLCBTZXAgNywgMjAxNiBhdCAyOjA0IFBNLCBMZW9uIFJv bWFub3Zza3kgPGxlb25Aa2VybmVsLm9yZz4gd3JvdGU6DQo+IEZyb206IFlvbmF0YW4gQ29oZW4g PHlvbmF0YW5jQG1lbGxhbm94LmNvbT4NCj4NCj4gRGlzYWJsZSBjcmVhdGlvbiBvZiBhIFVEUCBz b2NrZXQgZm9yIGlwdjYgd2hlbg0KPiBDT05GSUdfSVBWNiBpcyBub3QgZW5hYmVsZC4gU2luY2Ug dWRwX3NvY2tfY3JlYXRlNigpIHJldHVybnMgMCB3aGVuIA0KPiBDT05GSUdfSVBWNiBpcyBub3Qg c2V0DQo+DQo+IFsgICA0Ni44ODg2MzJdIElQOiBbPGMyMjA3MDVhPl0gc2V0dXBfdWRwX3R1bm5l bF9zb2NrKzB4Ni8weDRmDQo+IFsgICA0Ni44OTEzNTVdICpwZHB0ID0gMDAwMDAwMDAwMDAwMDAw MCAqcGRlID0gZjAwMGZmNTNmMDAwZmY1Mw0KPiBbICAgNDYuODkzOTE4XSBPb3BzOiAwMDAyIFsj MV0gUFJFRU1QVA0KPiBbICAgNDYuODk2MDE0XSBDUFU6IDAgUElEOiAxIENvbW06IHN3YXBwZXIg Tm90IHRhaW50ZWQgNC43LjAtcmM0LTAwMDAxLWc4NzAwZTNlICMxDQo+IFsgICA0Ni45MDAyODBd IEhhcmR3YXJlIG5hbWU6IFFFTVUgU3RhbmRhcmQgUEMgKGk0NDBGWCArIFBJSVgsIDE5OTYpLCBC SU9TIERlYmlhbi0xLjguMi0xIDA0LzAxLzIwMTQNCj4gWyAgIDQ2LjkwNDkwNV0gdGFzazogY2Yw NmMwNDAgdGk6IGNmMDVlMDAwIHRhc2sudGk6IGNmMDVlMDAwDQo+IFsgICA0Ni45MDc4NTRdIEVJ UDogMDA2MDpbPGMyMjA3MDVhPl0gRUZMQUdTOiAwMDIxMDI0NiBDUFU6IDANCj4gWyAgIDQ2Ljkx MTEzN10gRUlQIGlzIGF0IHNldHVwX3VkcF90dW5uZWxfc29jaysweDYvMHg0Zg0KPiBbICAgNDYu OTE0MDcwXSBFQVg6IDAwMDAwMDQ0IEVCWDogMDAwMDAwMDEgRUNYOiBjZjA1ZmVmMCBFRFg6IGNh ODE0MmUwDQo+IFsgICA0Ni45MTcyMzZdIEVTSTogYzJjNDUwNWIgRURJOiBjZjA1ZmVmMCBFQlA6 IGNmMDVmZWQwIEVTUDogY2YwNWZlZDANCj4gWyAgIDQ2LjkxOTgzNl0gIERTOiAwMDdiIEVTOiAw MDdiIEZTOiAwMDAwIEdTOiAwMGUwIFNTOiAwMDY4DQo+IFsgICA0Ni45MjIwNDZdIENSMDogODAw NTAwMzMgQ1IyOiAwMDAwMDFmYyBDUjM6IDAyY2VjMDAwIENSNDogMDAwMDA2YjANCj4gWyAgIDQ2 LjkyNDU1MF0gU3RhY2s6DQo+IFsgICA0Ni45MjYwMTRdICBjZjA1ZmYxMCBjMWZkNDY1NyBjYTgx NDJlMCAwMDAwMDAwYSAwMDAwMDAwMCAwMDAwMDAwMCAwMDAwYjcxMiAwMDAwMDAwOA0KPiBbICAg NDYuOTMxMjc0XSAgMDAwMDAwMDAgNmJiNWJkMDEgYzFmZDQ4ZGUgMDAwMDAwMDAgMDAwMDAwMDAg Y2YwNWZmMWMgMDAwMDAwMDAgMDAwMDAwMDANCj4gWyAgIDQ2LjkzNjEyMl0gIGNmMDVmZjFjIGMx ZmQ0YmRmIDAwMDAwMDAwIGNmMDVmZjI4IGMyYzQ1MDdiIGZmZmZmZmZmIGNmMDVmZjg4IGMyYmYx Yzc0DQo+IFsgICA0Ni45NDIzNTBdIENhbGwgVHJhY2U6DQo+IFsgICA0Ni45NDQ0MDNdICBbPGMx ZmQ0NjU3Pl0gcnhlX3NldHVwX3VkcF90dW5uZWwrMHg4Zi8weDk5DQo+IFsgICA0Ni45NDc2ODld ICBbPGMxZmQ0OGRlPl0gPyBuZXRfdG9fcnhlKzB4NGUvMHg0ZQ0KPiBbICAgNDYuOTUwNTY3XSAg WzxjMWZkNGJkZj5dIHJ4ZV9uZXRfaW5pdCsweGUvMHhhNA0KPiBbICAgNDYuOTUzMTQ3XSAgWzxj MmM0NTA3Yj5dIHJ4ZV9tb2R1bGVfaW5pdCsweDIwLzB4NGMNCj4gWyAgIDQ2Ljk1NTQ0OF0gIFs8 YzJiZjFjNzQ+XSBkb19vbmVfaW5pdGNhbGwrMHg4OS8weDExMw0KPiBbICAgNDYuOTU3Nzk3XSAg WzxjMmJmMTVlYj5dID8gc2V0X2RlYnVnX3JvZGF0YSsweGYvMHhmDQo+IFsgICA0Ni45NTk5NjZd ICBbPGMyYmYxZGJjPl0gPyBrZXJuZWxfaW5pdF9mcmVlYWJsZSsweGJlLzB4MTViDQo+IFsgICA0 Ni45NjIyNjJdICBbPGMyYmYxZGRjPl0ga2VybmVsX2luaXRfZnJlZWFibGUrMHhkZS8weDE1Yg0K PiBbICAgNDYuOTY0NDE4XSAgWzxjMjMyZWI1ND5dIGtlcm5lbF9pbml0KzB4OC8weGQwDQo+IFsg ICA0Ni45NjY2MThdICBbPGMyMzMzMTIyPl0gcmV0X2Zyb21fa2VybmVsX3RocmVhZCsweGUvMHgy NA0KPiBbICAgNDYuOTY5NTkyXSAgWzxjMjMyZWI0Yz5dID8gcmVzdF9pbml0KzB4NmYvMHg2Zg0K DQp3aGF0IGFuIHVubmljZSB3YXkgdG8gc2V0dGxlIGEgcmV2aWV3IGRpc2N1c3Npb24uIEkgcmVw bGllZCB0byBhbGwgeW91ciBhcmd1bWVudHMgZXhwbGFpbmluZyB3aHkgdGhlIGNoYW5nZSBsb2cg aGFzIHRvIGdvLCBhbmQgdGhlIG5leHQgc3RlcCBpcyB0byBpZ25vcmUgdGhpcyBkaXNjdXNzaW9u IGFuZCBkbyBpdCBhZ2FpbiB3cm9uZyB3Lm8gcmVwbHlpbmc/IHNvLi4uIGFnYWluOg0KDQokIC4v c2NyaXB0cy9nZXRfbWFpbnRhaW5lci5wbCBkcml2ZXJzL2luZmluaWJhbmQvc3cvcnhlIE1vbmkg U2hvdWEgPG1vbmlzQG1lbGxhbm94LmNvbT4gKHN1cHBvcnRlcjpTT0ZULVJPQ0UgRFJJVkVSIChy eGUpKSBEb3VnIExlZGZvcmQgPGRsZWRmb3JkQHJlZGhhdC5jb20+IChzdXBwb3J0ZXI6SU5GSU5J QkFORCBTVUJTWVNURU0pIFNlYW4gSGVmdHkgPHNlYW4uaGVmdHlAaW50ZWwuY29tPiAoc3VwcG9y dGVyOklORklOSUJBTkQgU1VCU1lTVEVNKSBIYWwgUm9zZW5zdG9jayA8aGFsLnJvc2Vuc3RvY2tA Z21haWwuY29tPiAoc3VwcG9ydGVyOklORklOSUJBTkQgU1VCU1lTVEVNKSBsaW51eC1yZG1hQHZn ZXIua2VybmVsLm9yZyAob3BlbiBsaXN0OlNPRlQtUk9DRSBEUklWRVIgKHJ4ZSkpIGxpbnV4LWtl cm5lbEB2Z2VyLmtlcm5lbC5vcmcgKG9wZW4gbGlzdCkNCg0KTW9uaSwgY2FuIHlvdSBwbGVhc2Ug cHV0IHlvdXIgbWFpbnRhaW5lciBzYXkgaGVyZS4NCg0KDQoNCk9yLg0K
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

http://marc.info/?t=147307741300005&r=1&w=2
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Or,

You didn't like the format of commit message, while I liked - that's all, it is the end of discussion.

We are not in debate club here and while commit messages are important, the code and bug fixes are much more.

If you think that kernel oops doesn't belong to commit message, please fix SubmittingPatches [1] guide and don't forget to announce it and enforce it in various subsystems.

Do you think that there is something in the code which needs to be addressed before Doug's take this patchset and releases RXE with less unaddressed issues?

You are sitting next to him and you can freely ask him if he reviewed it. The answer will be YES. Moni reviewed first version and I reviewed the second one.

[1] http://lxr.free-electrons.com/source/Documentation/SubmittingPatches#L691

I made **concrete** **arguments** replying to yours why to my opinion the oops stack listing doesn't help anyone and hence doesn't belong to the kernel logs and need not be there forever. This is not liking thing, this is technical talking thing.

so strip this stupid stack trace which helps no one and set the patch free.

yes, see my comments to the rest of the patches of the series, some small fixes are needed.

Your assumption is (1) wrong and (2) irrelevant to this discussion

There's a reviewer discussion on a patch for a driver he (and not you) maintain,
he has to put his maintainer say.

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Your arguments were wrong though. I myself have gone flipping through git commit messages looking for a fix for a specific issue. Having at least the signature call trace in the commit message is very useful to someone in that position. *I* want the trace in the commit message. I may trim it down to just the relevant signature backtrace, but I would still like it there. So let's please drop this and move on.

and hence doesn't belong to

Doug Ledford <[email protected]>
    GPG Key ID: 0E572FDD

I see that panic messages are common in change logs and personally I think it adds to info about the necessity of the patch. Therefore I tend to agree with Leon.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

When handling ack for atomic opcodes like "fetch&add"
or "cmp&swp", the method send_atomic_ack() saves the ack before sending it, in case it gets lost and never reach the requester. In which case the method duplicate_request() will need to find it using the duplicated request.psn.
But send_atomic_ack() used a wrong psn value and thus
the above ack was never found.
This fix uses the ack.psn to locate the ack in case
its needed.
This fix also copies the ack packet to the skb's control buffer since duplicate_request() will need it when calling rxe_xmit_packet()

---
 drivers/infiniband/sw/rxe/rxe_resp.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

index ebb03b4..3e0f0f2 100644
@@ -972,11 +972,13 @@ static int send_atomic_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt,
 	free_rd_atomic_resource(qp, res);
 	rxe_advance_resp_resource(qp);
 
+	memcpy(SKB_TO_PKT(skb), &ack_pkt, sizeof(skb->cb));
+
 	res->type = RXE_ATOMIC_MASK;
 	res->atomic.skb = skb;
-	res->first_psn = qp->resp.psn;
-	res->last_psn = qp->resp.psn;
-	res->cur_psn = qp->resp.psn;
+	res->first_psn = ack_pkt.psn;
+	res->last_psn  = ack_pkt.psn;
+	res->cur_psn   = ack_pkt.psn;
 
 	rc = rxe_xmit_packet(rxe, qp, &ack_pkt, skb_copy);
 	if (rc) {

@@ -1116,8 +1118,7 @@ static enum resp_states duplicate_request(struct rxe_qp *qp,
 				rc = RESPST_CLEANUP;
 				goto out;
 			}
-			bth_set_psn(SKB_TO_PKT(skb_copy),
-				    qp->resp.psn - 1);
+
 			/* Resend the result. */
 			rc = rxe_xmit_packet(to_rdev(qp->ibqp.device), qp,
 					     pkt, skb_copy);

--
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Are you using tiny screen? any special reason to limit the change-log lines to be ~55 chars long and not 70 or 80?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

rxe_requester() is sending a pkt with rxe_xmit_packet() and then calls rxe_update() to update the wqe and qp's psn values. But sometimes the response is received before the requester had time to update the wqe in which case the completer acts on errornous wqe values.
This fix updates the wqe and qp before actually sending the request and rolls back when xmit fails.

---
 drivers/infiniband/sw/rxe/rxe_req.c | 57 ++++++++++++++++++++++++++++---------
 1 file changed, 44 insertions(+), 13 deletions(-)

index 33b2d9d..13a848a 100644
@@ -511,24 +511,21 @@ static int fill_packet(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
 }
 
 static void update_wqe_state(struct rxe_qp *qp,
-			     struct rxe_send_wqe *wqe,
-			     struct rxe_pkt_info *pkt,
-			     enum wqe_state *prev_state)
+		struct rxe_send_wqe *wqe,
+		struct rxe_pkt_info *pkt)
 {
-	enum wqe_state prev_state_ = wqe->state;
-
 	if (pkt->mask & RXE_END_MASK) {
 		if (qp_type(qp) == IB_QPT_RC)
 			wqe->state = wqe_state_pending;
 	} else {
 		wqe->state = wqe_state_processing;
 	}
-
-	*prev_state = prev_state_;
 }
 
-static void update_state(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
-			 struct rxe_pkt_info *pkt, int payload)
+static void update_wqe_psn(struct rxe_qp *qp,
+			   struct rxe_send_wqe *wqe,
+			   struct rxe_pkt_info *pkt,
+			   int payload)
 {
 	/* number of packets left to send including current one */
 	int num_pkt = (wqe->dma.resid + payload + qp->mtu - 1) / qp->mtu;

@@ -546,9 +543,34 @@ static void update_state(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
 		qp->req.psn = (wqe->first_psn + num_pkt) & BTH_PSN_MASK;
 	else
 		qp->req.psn = (qp->req.psn + 1) & BTH_PSN_MASK;
+}
 
-	qp->req.opcode = pkt->opcode;
+static void save_state(struct rxe_send_wqe *wqe,
+		       struct rxe_qp *qp,
+		       struct rxe_send_wqe *rollback_wqe,
+		       struct rxe_qp *rollback_qp)
+{
+	rollback_wqe->state     = wqe->state;
+	rollback_wqe->first_psn = wqe->first_psn;
+	rollback_wqe->last_psn  = wqe->last_psn;
+	rollback_qp->req.psn    = qp->req.psn;
+}
 
+static void rollback_state(struct rxe_send_wqe *wqe,
+			   struct rxe_qp *qp,
+			   struct rxe_send_wqe *rollback_wqe,
+			   struct rxe_qp *rollback_qp)
+{
+	wqe->state     = rollback_wqe->state;
+	wqe->first_psn = rollback_wqe->first_psn;
+	wqe->last_psn  = rollback_wqe->last_psn;
+	qp->req.psn    = rollback_qp->req.psn;
+}
+
+static void update_state(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
+			 struct rxe_pkt_info *pkt, int payload)
+{
+	qp->req.opcode = pkt->opcode;
 
 	if (pkt->mask & RXE_END_MASK)
 		qp->req.wqe_index = next_index(qp->sq.queue, qp->req.wqe_index);

@@ -571,7 +593,8 @@ int rxe_requester(void *arg)
 	int mtu;
 	int opcode;
 	int ret;
-	enum wqe_state prev_state;
+	struct rxe_qp rollback_qp;
+	struct rxe_send_wqe rollback_wqe;
 
 next_wqe:
 	if (unlikely(!qp->valid || qp->req.state == QP_STATE_ERROR))

@@ -688,13 +711,21 @@ next_wqe:
 		goto err;
 	}
 
-	update_wqe_state(qp, wqe, &pkt, &prev_state);
+	/*
+	 * To prevent a race on wqe access between requester and completer,
+	 * wqe members state and psn need to be set before calling
+	 * rxe_xmit_packet().
+	 * Otherwise, completer might initiate an unjustified retry flow.
+	 */
+	save_state(wqe, qp, &rollback_wqe, &rollback_qp);
+	update_wqe_state(qp, wqe, &pkt);
+	update_wqe_psn(qp, wqe, &pkt, payload);
 	ret = rxe_xmit_packet(to_rdev(qp->ibqp.device), qp, &pkt, skb);
 	if (ret) {
 		qp->need_req_skb = 1;
 		kfree_skb(skb);
 
-		wqe->state = prev_state;
+		rollback_state(wqe, qp, &rollback_wqe, &rollback_qp);
 
 		if (ret == -EAGAIN) {
 			rxe_run_task(&qp->req.task, 1);

--
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

pkt --> packet will make this much more readable
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

Decrement qp reference when handling error path
in completer to prevent kmem_cache leak.

---
 drivers/infiniband/sw/rxe/rxe_comp.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

index 36f67de..1c59ef2 100644
@@ -689,7 +689,14 @@ int rxe_completer(void *arg)
 					qp->req.need_retry = 1;
 					rxe_run_task(&qp->req.task, 1);
 				}
+
+				if (pkt) {
+					rxe_drop_ref(pkt->qp);
+					kfree_skb(skb);
+				}
+
 				goto exit;
+
 			} else {
 				wqe->status = IB_WC_RETRY_EXC_ERR;
 				state = COMPST_ERROR;

@@ -716,6 +723,12 @@ int rxe_completer(void *arg)
 		case COMPST_ERROR:
 			do_complete(qp, wqe);
 			rxe_qp_error(qp);
+
+			if (pkt) {
+				rxe_drop_ref(pkt->qp);
+				kfree_skb(skb);
+			}
+
 			goto exit;
 		}
 	}

--
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

"error path in completer" is pretty much cryptic, please user higher language (completing what)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Thanks, series applied.

Doug Ledford <[email protected]>
    GPG Key ID: 0E572FDD