Skip to contentSkip to conversation

9p/virtio: add a read barrier in p9_virtio_zc_request

PATCH In review Dominique Martinet 2 participants 3 messages Dec 10, 2022 – Dec 13, 2022 9P file system

lore
Share & tools
lore

Dominique Martinet

author maintainer
#0 lore raw

PATCH 1/1 9p/virtio: add a read barrier in p9_virtio_zc_request

The request receiving thread writes into request then marks the request valid in p9_client_cb by setting status after a write barrier.

p9_virtio_zc_request must like p9_client_rpc issue a read barrier after getting notified of the new request status before reading other request files.

(This has been noticed while fixing the usage of READ/WRITE_ONCE macros for request status)

---
 net/9p/trans_virtio.c | 6 ++++++
 1 file changed, 6 insertions(+)
net/9p/trans_virtio.c +6 -0
index 3c27ffb781e3..98425c63b3c3 100644
@@ -533,6 +533,12 @@ p9_virtio_zc_request(struct p9_client *client, struct p9_req_t *req,
 	p9_debug(P9_DEBUG_TRANS, "virtio request kicked\n");
 	err = wait_event_killable(req->wq,
 			          READ_ONCE(req->status) >= REQ_STATUS_RCVD);
+
+	/* Make sure our req is coherent with regard to updates in other
+	 * threads - echoes to wmb() in the callback like p9_client_rpc
+	 */
+	smp_rmb();
+
 	// RERROR needs reply (== error string) in static data
 	if (READ_ONCE(req->status) == REQ_STATUS_RCVD &&
 	    unlikely(req->rc.sdata[4] == P9_RERROR))
Signature · 1 line
2.38.1
reviewer Reviewed-by on v1 1/1
#1 lore raw
Quoted 22 lines · CET Dominique Martinet

The request receiving thread writes into request then marks the request valid in p9_client_cb by setting status after a write barrier.

p9_virtio_zc_request must like p9_client_rpc issue a read barrier after getting notified of the new request status before reading other request files.

(This has been noticed while fixing the usage of READ/WRITE_ONCE macros for request status)

---
 net/9p/trans_virtio.c | 6 ++++++
 1 file changed, 6 insertions(+)
net/9p/trans_virtio.c +0 -0
index 3c27ffb781e3..98425c63b3c3 100644

net/9p/trans_virtio.c p9_virtio_zc_request(struct p9_client *client, struct p9_req_t *req, · line 541

Quoted 9 lines
net/9p/trans_virtio.c +6 -0
@@ -533,6 +533,12 @@ p9_virtio_zc_request(struct p9_client *client, struct p9_req_t *req,
 	p9_debug(P9_DEBUG_TRANS, "virtio request kicked\n");
 	err = wait_event_killable(req->wq,
 			          READ_ONCE(req->status) >= REQ_STATUS_RCVD);
+
+	/* Make sure our req is coherent with regard to updates in other
+	 * threads - echoes to wmb() in the callback like p9_client_rpc
+	 */
+	smp_rmb();
+

Oh, I had p9_client_zc_rpc() for this in mind, but I see why you chose this place in p9_virtio_zc_request() instead. LGTM

I also made some tests to check whether this barrier would hurt performance, but I measured no difference. So this should be good to go:

Quoted 4 lines
net/9p/trans_virtio.c +0 -0
 	// RERROR needs reply (== error string) in static data
 	if (READ_ONCE(req->status) == REQ_STATUS_RCVD &&
 	    unlikely(req->rc.sdata[4] == P9_RERROR))
author maintainer
#2 lore raw

Christian Schoenebeck wrote on Mon, Dec 12, 2022 at 02:35:39PM +0100:

Quoted 9 lines
Quoted 9 lines
net/9p/trans_virtio.c +1 -0
index 3c27ffb781e3..98425c63b3c3 100644
@@ -533,6 +533,12 @@ p9_virtio_zc_request(struct p9_client *client, struct p9_req_t *req,
 	p9_debug(P9_DEBUG_TRANS, "virtio request kicked\n");
 	err = wait_event_killable(req->wq,
 			          READ_ONCE(req->status) >= REQ_STATUS_RCVD);
+
Quoted 8 lines

net/9p/trans_virtio.c p9_virtio_zc_request(struct p9_client *client, struct p9_req_t *req, · line 541

Quoted 5 lines
net/9p/trans_virtio.c +5 -0
+	/* Make sure our req is coherent with regard to updates in other
+	 * threads - echoes to wmb() in the callback like p9_client_rpc
+	 */
+	smp_rmb();
+

Oh, I had p9_client_zc_rpc() for this in mind, but I see why you chose this place in p9_virtio_zc_request() instead. LGTM

Yes, we access req data here so as much as it'd make more sense to keep it symetrical in p9_client_zc_rpc (like p9_client_rpc) I think we need it here.

Quoted 2 lines

I also made some tests to check whether this barrier would hurt performance, but I measured no difference. So this should be good to go:

Thanks!
The assembly generated with the barrier is actually slightly shorter for x86_64, but it's hard to tell the actual performance impact....

Quoted 1 line

Cheers, I've queued this patch as well: let's make that this merge windows' batch unless a problem comes up.

Signature · 1 line
Dominique