9p/virtio: add a read barrier in p9_virtio_zc_request
Dominique Martinet
maintainerPATCH 1/1 9p/virtio: add a read barrier in p9_virtio_zc_request
The request receiving thread writes into request then marks the request valid in p9_client_cb by setting status after a write barrier.
p9_virtio_zc_request must like p9_client_rpc issue a read barrier after getting notified of the new request status before reading other request files.
(This has been noticed while fixing the usage of READ/WRITE_ONCE macros for request status)
--- net/9p/trans_virtio.c | 6 ++++++ 1 file changed, 6 insertions(+)
net/9p/trans_virtio.c +6 -0
@@ -533,6 +533,12 @@ p9_virtio_zc_request(struct p9_client *client, struct p9_req_t *req,p9_debug (P9_DEBUG_TRANS ,"virtio request kicked\n" ); err =wait_event_killable (req->wq,READ_ONCE (req->status) >=REQ_STATUS_RCVD ); + +/* Make sure our req is coherent with regard to updates in other +* threads - echoes to wmb() in the callback like p9_client_rpc +*/ +smp_rmb (); +// RERROR needs reply (== error string) in static data if (READ_ONCE (req->status) ==REQ_STATUS_RCVD &&unlikely (req->rc.sdata[4 ] ==P9_RERROR ))
Signature · 1 line
2.38.1
Quoted 22 lines · CET Dominique Martinet
The request receiving thread writes into request then marks the request valid in
p9_client_cbby setting status after a write barrier.
p9_virtio_zc_requestmust likep9_client_rpcissue a read barrier after getting notified of the new request status before reading other request files.(This has been noticed while fixing the usage of READ/WRITE_ONCE macros for request status)
Reported-by Christian Schoenebeck <[email protected]>Signed-off-by Dominique Martinet <[email protected]>--- net/9p/trans_virtio.c | 6 ++++++ 1 file changed, 6 insertions(+)net/9p/trans_virtio.c +0
-0
net/9p/trans_virtio.c p9_virtio_zc_request(struct p9_client *client, struct p9_req_t *req, · line 541
Quoted 9 lines
net/9p/trans_virtio.c +6
-0@@ -533,6 +533,12 @@ p9_virtio_zc_request(struct p9_client *client, struct p9_req_t *req,p9_debug (P9_DEBUG_TRANS ,"virtio request kicked\n" ); err =wait_event_killable (req->wq,READ_ONCE (req->status) >=REQ_STATUS_RCVD ); + +/* Make sure our req is coherent with regard to updates in other +* threads - echoes to wmb() in the callback like p9_client_rpc +*/ +smp_rmb (); +
Oh, I had p9_client_zc_rpc() for this in mind, but I see why you chose this place in p9_virtio_zc_request() instead. LGTM
I also made some tests to check whether this barrier would hurt performance, but I measured no difference. So this should be good to go:
Quoted 4 lines
net/9p/trans_virtio.c +0
-0// RERROR needs reply (== error string) in static data if (READ_ONCE (req->status) ==REQ_STATUS_RCVD &&unlikely (req->rc.sdata[4 ] ==P9_RERROR ))
Dominique Martinet
maintainerChristian Schoenebeck wrote on Mon, Dec 12, 2022 at 02:35:39PM +0100:
Quoted 9 lines
Quoted 9 lines
net/9p/trans_virtio.c +1
-0@@ -533,6 +533,12 @@ p9_virtio_zc_request(struct p9_client *client, struct p9_req_t *req,p9_debug (P9_DEBUG_TRANS ,"virtio request kicked\n" ); err =wait_event_killable (req->wq,READ_ONCE (req->status) >=REQ_STATUS_RCVD ); +
Quoted 8 lines
net/9p/trans_virtio.c p9_virtio_zc_request(struct p9_client *client, struct p9_req_t *req, · line 541
Quoted 5 lines
net/9p/trans_virtio.c +5
-0+/* Make sure our req is coherent with regard to updates in other +* threads - echoes to wmb() in the callback like p9_client_rpc +*/ +smp_rmb (); +Oh, I had
p9_client_zc_rpc()for this in mind, but I see why you chose this place inp9_virtio_zc_request()instead. LGTM
Yes, we access req data here so as much as it'd make more sense to keep it symetrical in p9_client_zc_rpc (like p9_client_rpc) I think we need it here.
Quoted 2 lines
I also made some tests to check whether this barrier would hurt performance, but I measured no difference. So this should be good to go:
Thanks!
The assembly generated with the barrier is actually slightly shorter for x86_64, but it's hard to tell the actual performance impact....
Quoted 1 line
Reviewed-by Christian Schoenebeck <[email protected]>
Cheers, I've queued this patch as well: let's make that this merge windows' batch unless a problem comes up.
Signature · 1 line
Dominique