virtiofs: Fix various races and cleanups round 1
Subsystems (4)
Vivek Goyal
maintainercover virtiofs: Fix various races and cleanups round 1
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4
I have tested these patches with mount/umount and device removal using qemu monitor. For example.
virsh qemu-monitor-command --hmp vm9-f28 device_del myvirtiofs
Now a mounted device can be removed and file system will return errors -ENOTCONN and one can unmount it.
Miklos, if you are fine with the patches, I am fine if you fold these all into existing patch. I kept them separate so that review is easier.
Any feedback or comments are welcome.
Thanks
Vivek
Vivek Goyal (18):
virtiofs: Remove request from processing list before calling end
virtiofs: Check whether hiprio queue is connected at submission time
virtiofs: Pass fsvq instead of vq as parameter to
virtio_fs_enqueue_req
virtiofs: Check connected state for VQ_REQUEST queue as well Maintain count of in flight requests for VQ_REQUEST queue
virtiofs: ->remove should not clean virtiofs fuse devices
virtiofs: Stop virtiofs queues when device is being removed
virtiofs: Drain all pending requests during ->remove time
virtiofs: Add an helper to start all the queues
virtiofs: Do not use device managed mem for virtio_fs and virtio_fs_vq
virtiofs: stop and drain queues after sending DESTROY
virtiofs: Use virtio_fs_free_devs() in error path
virtiofs: Do not access virtqueue in request submission path
virtiofs: Add a fuse_iqueue operation to put() reference
virtiofs: Make virtio_fs object refcounted
virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path
virtiofs: Remove TODO to quiesce/end_requests
virtiofs: Remove TODO item from virtio_fs_free_devs()
fs/fuse/fuse_i.h | 5 + fs/fuse/inode.c | 6 + fs/fuse/virtio_fs.c | 259 ++++++++++++++++++++++++++++++++------------ 3 files changed, 198 insertions(+), 72 deletions(-)
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 1/18 virtiofs: Remove request from processing list before calling end
In error path we are calling fuse_request_end() but we need to clear FR_SENT bit as well as remove request from processing queue. Otherwise fuse_request_end() triggers a warning as well as other issues show up.
--- fs/fuse/virtio_fs.c | 4 ++++ 1 file changed, 4 insertions(+)
fs/fuse/virtio_fs.c +4 -0
@@ -826,6 +826,10 @@ __releases(fiq->waitq.lock) } req->out.h.error = ret;pr_err ("virtio-fs: virtio_fs_enqueue_req() failed %d\n" , ret); +spin_lock (&fpq->lock); +clear_bit (FR_SENT , &req->flags); +list_del_init (&req->list); +spin_unlock (&fpq->lock);fuse_request_end (fc, req);return ; }
Signature · 1 line
2.20.1
Re: [PATCH 01/18] virtiofs: Remove request from processing list before calling end
Quoted 8 lines · Vivek Goyal
In error path we are calling
fuse_request_end()but we need to clearFR_SENTbit as well as remove request from processing queue. Otherwisefuse_request_end()triggers a warning as well as other issues show up.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 4 ++++ 1 file changed, 4 insertions(+)
Re: [PATCH 01/18] virtiofs: Remove request from processing list before calling end
In reply to Vivek GoyalQuoted 8 lines · Vivek Goyal
In error path we are calling
fuse_request_end()but we need to clearFR_SENTbit as well as remove request from processing queue. Otherwisefuse_request_end()triggers a warning as well as other issues show up.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 4 ++++ 1 file changed, 4 insertions(+)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 2/18 virtiofs: Check whether hiprio queue is connected at submission time
In reply to Vivek GoyalFor hiprio queue (forget requests), we are keeping a state in queue whether queue is connected or not. If queue is not connected, do not try to submit request and return error instead.
As of now, we are checking for this state only in path where worker tries to submit forget after first attempt failed. Check this state even in the path when request is being submitted first time.
--- fs/fuse/virtio_fs.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
fs/fuse/virtio_fs.c +8 -1
@@ -577,9 +577,16 @@ __releases(fiq->waitq.lock)sg_init_one (&sg, forget,sizeof (*forget));/* Enqueue the request */ +spin_lock (&fsvq->lock); + +if (!fsvq->connected) { +kfree (forget); +spin_unlock (&fsvq->lock); + goto out; + } + vq = fsvq->vq;dev_dbg (&vq->vdev->dev,"%s\n" , __func__); -spin_lock (&fsvq->lock); ret =virtqueue_add_sgs (vq, sgs,1 ,0 , forget,GFP_ATOMIC );if (ret <0 ) {
Signature · 1 line
2.20.1
Re: [PATCH 02/18] virtiofs: Check whether hiprio queue is connected at submission time
Quoted 4 lines · Vivek Goyal
For hiprio queue (forget requests), we are keeping a state in queue whether queue is connected or not. If queue is not connected, do not try to submit request and return error instead.
Quoted 8 lines
As of now, we are checking for this state only in path where worker tries to submit forget after first attempt failed. Check this state even in the path when request is being submitted first time.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
Re: [PATCH 02/18] virtiofs: Check whether hiprio queue is connected at submission time
In reply to Vivek GoyalQuoted 4 lines · Vivek Goyal
For hiprio queue (forget requests), we are keeping a state in queue whether queue is connected or not. If queue is not connected, do not try to submit request and return error instead.
Quoted 8 lines
As of now, we are checking for this state only in path where worker tries to submit forget after first attempt failed. Check this state even in the path when request is being submitted first time.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 2/18 virtiofs: Check whether hiprio queue is connected at submission time
In reply to Vivek GoyalFor hiprio queue (forget requests), we are keeping a state in queue whether queue is connected or not. If queue is not connected, do not try to submit request and return error instead.
As of now, we are checking for this state only in path where worker tries to submit forget after first attempt failed. Check this state even in the path when request is being submitted first time.
--- fs/fuse/virtio_fs.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
fs/fuse/virtio_fs.c +8 -1
@@ -577,9 +577,16 @@ __releases(fiq->waitq.lock)sg_init_one (&sg, forget,sizeof (*forget));/* Enqueue the request */ +spin_lock (&fsvq->lock); + +if (!fsvq->connected) { +kfree (forget); +spin_unlock (&fsvq->lock); + goto out; + } + vq = fsvq->vq;dev_dbg (&vq->vdev->dev,"%s\n" , __func__); -spin_lock (&fsvq->lock); ret =virtqueue_add_sgs (vq, sgs,1 ,0 , forget,GFP_ATOMIC );if (ret <0 ) {
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 3/18 virtiofs: Pass fsvq instead of vq as parameter to virtio_fs_enqueue_req
In reply to Vivek GoyalPass fsvq instead of vq as parameter to virtio_fs_enqueue_req(). We will retrieve vq from fsvq under spin lock.
Later in the patch series we will retrieve vq only if fsvq is still connected other vq might have been cleaned up by device ->remove code and we will return error.
--- fs/fuse/virtio_fs.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
fs/fuse/virtio_fs.c +5 -4
@@ -698,14 +698,15 @@ static unsigned int sg_init_fuse_args(struct scatterlist *sg, }/* Add a request to a virtqueue and kick the device */ -static int virtio_fs_enqueue_req (struct virtqueue *vq,struct fuse_req *req) +static int virtio_fs_enqueue_req (struct virtio_fs_vq *fsvq, +struct fuse_req *req) {/* requests need at least 4 elements */ struct scatterlist *stack_sgs[6 ];struct scatterlist stack_sg[ARRAY_SIZE (stack_sgs)];struct scatterlist **sgs = stack_sgs;struct scatterlist *sg = stack_sg; -struct virtio_fs_vq *fsvq; +struct virtqueue *vq;unsigned int argbuf_used =0 ;unsigned int out_sgs =0 ;unsigned int in_sgs =0 ;
@@ -752,9 +753,9 @@ static int virtio_fs_enqueue_req(struct virtqueue *vq, struct fuse_req *req)for (i =0 ; i < total_sgs; i++) sgs[i] = &sg[i]; - fsvq =vq_to_fsvq (vq);spin_lock (&fsvq->lock); + vq = fsvq->vq; ret =virtqueue_add_sgs (vq, sgs, out_sgs, in_sgs, req,GFP_ATOMIC );if (ret <0 ) {/* TODO handle full virtqueue */
@@ -824,7 +825,7 @@ __releases(fiq->waitq.lock)/* TODO check for FR_INTERRUPTED? */ retry: - ret =virtio_fs_enqueue_req (fs->vqs[queue_id].vq, req); + ret =virtio_fs_enqueue_req (&fs->vqs[queue_id], req);if (ret <0 ) {if (ret == -ENOMEM || ret == -ENOSPC ) {/* Virtqueue full. Retry submission */
Signature · 1 line
2.20.1
Re: [PATCH 03/18] virtiofs: Pass fsvq instead of vq as parameter to virtio_fs_enqueue_req
Quoted 11 lines · Vivek Goyal
Pass fsvq instead of vq as parameter to
virtio_fs_enqueue_req(). We will retrieve vq from fsvq under spin lock.Later in the patch series we will retrieve vq only if fsvq is still connected other vq might have been cleaned up by device ->remove code and we will return error.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
Re: [PATCH 03/18] virtiofs: Pass fsvq instead of vq as parameter to virtio_fs_enqueue_req
In reply to Vivek GoyalQuoted 11 lines · Vivek Goyal
Pass fsvq instead of vq as parameter to
virtio_fs_enqueue_req(). We will retrieve vq from fsvq under spin lock.Later in the patch series we will retrieve vq only if fsvq is still connected other vq might have been cleaned up by device ->remove code and we will return error.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 4/18 virtiofs: Check connected state for VQ_REQUEST queue as well
In reply to Vivek GoyalRight now we are checking ->connected state only for VQ_HIPRIO. Now we want to make use of this method for all queues. So check it for VQ_REQUEST as well.
This will be helpful if device has been removed and virtqueue is gone. In that case ->connected will be false and request can't be submitted anymore and user space will see error -ENOTCONN.
--- fs/fuse/virtio_fs.c | 6 ++++++ 1 file changed, 6 insertions(+)
fs/fuse/virtio_fs.c +6 -0
@@ -755,6 +755,12 @@ static int virtio_fs_enqueue_req(struct virtio_fs_vq *fsvq,spin_lock (&fsvq->lock); +if (!fsvq->connected) { +spin_unlock (&fsvq->lock); + ret = -ENOTCONN ; + goto out; + } + vq = fsvq->vq; ret =virtqueue_add_sgs (vq, sgs, out_sgs, in_sgs, req,GFP_ATOMIC );if (ret <0 ) {
Signature · 1 line
2.20.1
Re: [PATCH 04/18] virtiofs: Check connected state for VQ_REQUEST queue as well
Quoted 4 lines · Vivek Goyal
Right now we are checking ->connected state only for
VQ_HIPRIO. Now we want to make use of this method for all queues. So check it forVQ_REQUESTas well.
Quoted 8 lines
This will be helpful if device has been removed and virtqueue is gone. In that case ->connected will be false and request can't be submitted anymore and user space will see error
-ENOTCONN.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 6 ++++++ 1 file changed, 6 insertions(+)
Re: [PATCH 04/18] virtiofs: Check connected state for VQ_REQUEST queue as well
In reply to Vivek GoyalQuoted 4 lines · Vivek Goyal
Right now we are checking ->connected state only for
VQ_HIPRIO. Now we want to make use of this method for all queues. So check it forVQ_REQUESTas well.
Quoted 8 lines
This will be helpful if device has been removed and virtqueue is gone. In that case ->connected will be false and request can't be submitted anymore and user space will see error
-ENOTCONN.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 6 ++++++ 1 file changed, 6 insertions(+)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 4/18 virtiofs: Check connected state for VQ_REQUEST queue as well
In reply to Vivek GoyalRight now we are checking ->connected state only for VQ_HIPRIO. Now we want to make use of this method for all queues. So check it for VQ_REQUEST as well.
This will be helpful if device has been removed and virtqueue is gone. In that case ->connected will be false and request can't be submitted anymore and user space will see error -ENOTCONN.
--- fs/fuse/virtio_fs.c | 6 ++++++ 1 file changed, 6 insertions(+)
fs/fuse/virtio_fs.c +6 -0
@@ -755,6 +755,12 @@ static int virtio_fs_enqueue_req(struct virtio_fs_vq *fsvq,spin_lock (&fsvq->lock); +if (!fsvq->connected) { +spin_unlock (&fsvq->lock); + ret = -ENOTCONN ; + goto out; + } + vq = fsvq->vq; ret =virtqueue_add_sgs (vq, sgs, out_sgs, in_sgs, req,GFP_ATOMIC );if (ret <0 ) {
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 5/18 Maintain count of in flight requests for VQ_REQUEST queue
In reply to Vivek GoyalAs of now we maintain this count only for VQ_HIPRIO. Maintain it for VQ_REQUEST as well so that later it can be used to drain VQ_REQUEST queue.
--- fs/fuse/virtio_fs.c | 4 ++++ 1 file changed, 4 insertions(+)
fs/fuse/virtio_fs.c +4 -0
@@ -360,6 +360,9 @@ static void virtio_fs_requests_done_work(struct work_struct *work)spin_unlock (&fpq->lock);fuse_request_end (fc, req); +spin_lock (&fsvq->lock); + fsvq->in_flight--; +spin_unlock (&fsvq->lock); } }
@@ -769,6 +772,7 @@ static int virtio_fs_enqueue_req(struct virtio_fs_vq *fsvq, goto out; } + fsvq->in_flight++; notify =virtqueue_kick_prepare (vq);spin_unlock (&fsvq->lock);
Signature · 1 line
2.20.1
Re: [PATCH 05/18] Maintain count of in flight requests for VQ_REQUEST queue
Quoted 8 lines · Vivek Goyal
As of now we maintain this count only for
VQ_HIPRIO. Maintain it forVQ_REQUESTas well so that later it can be used to drainVQ_REQUESTqueue.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 4 ++++ 1 file changed, 4 insertions(+)
Re: [PATCH 05/18] Maintain count of in flight requests for VQ_REQUEST queue
In reply to Vivek GoyalQuoted 8 lines · Vivek Goyal
As of now we maintain this count only for
VQ_HIPRIO. Maintain it forVQ_REQUESTas well so that later it can be used to drainVQ_REQUESTqueue.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 4 ++++ 1 file changed, 4 insertions(+)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 5/18 Maintain count of in flight requests for VQ_REQUEST queue
In reply to Vivek GoyalAs of now we maintain this count only for VQ_HIPRIO. Maintain it for VQ_REQUEST as well so that later it can be used to drain VQ_REQUEST queue.
--- fs/fuse/virtio_fs.c | 4 ++++ 1 file changed, 4 insertions(+)
fs/fuse/virtio_fs.c +4 -0
@@ -360,6 +360,9 @@ static void virtio_fs_requests_done_work(struct work_struct *work)spin_unlock (&fpq->lock);fuse_request_end (fc, req); +spin_lock (&fsvq->lock); + fsvq->in_flight--; +spin_unlock (&fsvq->lock); } }
@@ -769,6 +772,7 @@ static int virtio_fs_enqueue_req(struct virtio_fs_vq *fsvq, goto out; } + fsvq->in_flight++; notify =virtqueue_kick_prepare (vq);spin_unlock (&fsvq->lock);
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 6/18 virtiofs: ->remove should not clean virtiofs fuse devices
In reply to Vivek GoyalWe maintain a fuse device per virt queue. This fuse devices are allocated and installed during mount time and should be cleaned up when super block is going away. Device removal should not clean it. Device removal should stop queues and virtuques can go away.
--- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-)
fs/fuse/virtio_fs.c +0 -2
@@ -497,8 +497,6 @@ static void virtio_fs_remove(struct virtio_device *vdev) {struct virtio_fs *fs = vdev->priv; -virtio_fs_free_devs (fs); - vdev->config->reset (vdev);virtio_fs_cleanup_vqs (vdev, fs);
Signature · 1 line
2.20.1
Re: [PATCH 06/18] virtiofs: ->remove should not clean virtiofs fuse devices
Quoted 9 lines · Vivek Goyal
We maintain a fuse device per virt queue. This fuse devices are allocated and installed during mount time and should be cleaned up when super block is going away. Device removal should not clean it. Device removal should stop queues and virtuques can go away.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-)
Re: [PATCH 06/18] virtiofs: ->remove should not clean virtiofs fuse devices
In reply to Vivek GoyalQuoted 9 lines · Vivek Goyal
We maintain a fuse device per virt queue. This fuse devices are allocated and installed during mount time and should be cleaned up when super block is going away. Device removal should not clean it. Device removal should stop queues and virtuques can go away.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 6/18 virtiofs: ->remove should not clean virtiofs fuse devices
In reply to Vivek GoyalWe maintain a fuse device per virt queue. This fuse devices are allocated and installed during mount time and should be cleaned up when super block is going away. Device removal should not clean it. Device removal should stop queues and virtuques can go away.
--- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-)
fs/fuse/virtio_fs.c +0 -2
@@ -497,8 +497,6 @@ static void virtio_fs_remove(struct virtio_device *vdev) {struct virtio_fs *fs = vdev->priv; -virtio_fs_free_devs (fs); - vdev->config->reset (vdev);virtio_fs_cleanup_vqs (vdev, fs);
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 7/18 virtiofs: Stop virtiofs queues when device is being removed
In reply to Vivek GoyalStop all the virt queues when device is going away. This will ensure that no new requests are submitted to virtqueue and and request will end with error -ENOTCONN.
--- fs/fuse/virtio_fs.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
fs/fuse/virtio_fs.c +14 -0
@@ -493,10 +493,24 @@ static int virtio_fs_probe(struct virtio_device *vdev)return ret; } +static void virtio_fs_stop_all_queues (struct virtio_fs *fs) +{ +struct virtio_fs_vq *fsvq; +int i; + +for (i =0 ; i < fs->nvqs; i++) { + fsvq = &fs->vqs[i]; +spin_lock (&fsvq->lock); + fsvq->connected = false; +spin_unlock (&fsvq->lock); + } +} +static void virtio_fs_remove (struct virtio_device *vdev) {struct virtio_fs *fs = vdev->priv; +virtio_fs_stop_all_queues (fs); vdev->config->reset (vdev);virtio_fs_cleanup_vqs (vdev, fs);
Signature · 1 line
2.20.1
Re: [PATCH 07/18] virtiofs: Stop virtiofs queues when device is being removed
Quoted 8 lines · Vivek Goyal
Stop all the virt queues when device is going away. This will ensure that no new requests are submitted to virtqueue and and request will end with error
-ENOTCONN.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
Re: [PATCH 07/18] virtiofs: Stop virtiofs queues when device is being removed
In reply to Vivek GoyalQuoted 8 lines · Vivek Goyal
Stop all the virt queues when device is going away. This will ensure that no new requests are submitted to virtqueue and and request will end with error
-ENOTCONN.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 7/18 virtiofs: Stop virtiofs queues when device is being removed
In reply to Vivek GoyalStop all the virt queues when device is going away. This will ensure that no new requests are submitted to virtqueue and and request will end with error -ENOTCONN.
--- fs/fuse/virtio_fs.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
fs/fuse/virtio_fs.c +14 -0
@@ -493,10 +493,24 @@ static int virtio_fs_probe(struct virtio_device *vdev)return ret; } +static void virtio_fs_stop_all_queues (struct virtio_fs *fs) +{ +struct virtio_fs_vq *fsvq; +int i; + +for (i =0 ; i < fs->nvqs; i++) { + fsvq = &fs->vqs[i]; +spin_lock (&fsvq->lock); + fsvq->connected = false; +spin_unlock (&fsvq->lock); + } +} +static void virtio_fs_remove (struct virtio_device *vdev) {struct virtio_fs *fs = vdev->priv; +virtio_fs_stop_all_queues (fs); vdev->config->reset (vdev);virtio_fs_cleanup_vqs (vdev, fs);
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 8/18 virtiofs: Drain all pending requests during ->remove time
In reply to Vivek GoyalWhen device is going away, drain all pending requests.
--- fs/fuse/virtio_fs.c | 83 ++++++++++++++++++++++++++++----------------- 1 file changed, 51 insertions(+), 32 deletions(-)
fs/fuse/virtio_fs.c +51 -32
@@ -63,6 +63,55 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq)return &vq_to_fsvq (vq)->fud->pq; } +static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) +{ +WARN_ON (fsvq->in_flight <0 ); + +/* Wait for in flight requests to finish.*/ +while (1 ) { +spin_lock (&fsvq->lock); +if (!fsvq->in_flight) { +spin_unlock (&fsvq->lock); +break ; + } +spin_unlock (&fsvq->lock); +usleep_range (1000 ,2000 ); + } + +flush_work (&fsvq->done_work); +flush_delayed_work (&fsvq->dispatch_work); +} + +static inline void drain_hiprio_queued_reqs (struct virtio_fs_vq *fsvq) +{ +struct virtio_fs_forget *forget; + +spin_lock (&fsvq->lock); +while (1 ) { + forget =list_first_entry_or_null (&fsvq->queued_reqs, + struct virtio_fs_forget, list); +if (!forget) +break ; +list_del (&forget->list); +kfree (forget); + } +spin_unlock (&fsvq->lock); +} + +static void virtio_fs_drain_all_queues (struct virtio_fs *fs) +{ +struct virtio_fs_vq *fsvq; +int i; + +for (i =0 ; i < fs->nvqs; i++) { + fsvq = &fs->vqs[i]; +if (i ==VQ_HIPRIO ) +drain_hiprio_queued_reqs (fsvq); + +virtio_fs_drain_queue (fsvq); + } +} +/* Add a new instance to the list or return -EEXIST if tag name exists*/ static int virtio_fs_add_instance (struct virtio_fs *fs) {
@@ -511,6 +560,7 @@ static void virtio_fs_remove(struct virtio_device *vdev)struct virtio_fs *fs = vdev->priv;virtio_fs_stop_all_queues (fs); +virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);virtio_fs_cleanup_vqs (vdev, fs);
@@ -865,37 +915,6 @@ __releases(fiq->waitq.lock) } } -static void virtio_fs_flush_hiprio_queue (struct virtio_fs_vq *fsvq) -{ -struct virtio_fs_forget *forget; - -WARN_ON (fsvq->in_flight <0 ); - -/* Go through pending forget requests and free them */ -spin_lock (&fsvq->lock); -while (1 ) { - forget =list_first_entry_or_null (&fsvq->queued_reqs, - struct virtio_fs_forget, list); -if (!forget) -break ; -list_del (&forget->list); -kfree (forget); - } - -spin_unlock (&fsvq->lock); - -/* Wait for in flight requests to finish.*/ -while (1 ) { -spin_lock (&fsvq->lock); -if (!fsvq->in_flight) { -spin_unlock (&fsvq->lock); -break ; - } -spin_unlock (&fsvq->lock); -usleep_range (1000 ,2000 ); - } -} -const static struct fuse_iqueue_ops virtio_fs_fiq_ops = { .wake_forget_and_unlock = virtio_fs_wake_forget_and_unlock, .wake_interrupt_and_unlock = virtio_fs_wake_interrupt_and_unlock,
@@ -988,7 +1007,7 @@ static void virtio_kill_sb(struct super_block *sb)spin_lock (&fsvq->lock); fsvq->connected = false;spin_unlock (&fsvq->lock); -virtio_fs_flush_hiprio_queue (fsvq); +virtio_fs_drain_all_queues (vfs);fuse_kill_sb_anon (sb);virtio_fs_free_devs (vfs);
Signature · 1 line
2.20.1
Stefan Hajnoczi
maintainerRe: [PATCH 08/18] virtiofs: Drain all pending requests during ->remove time
Quoted 4 lines · Vivek Goyal
fs/fuse/virtio_fs.c +4
-0+static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) +{ +WARN_ON (fsvq->in_flight <0 ); +
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 79
Quoted 10 lines
fs/fuse/virtio_fs.c +10
-0+/* Wait for in flight requests to finish.*/ +while (1 ) { +spin_lock (&fsvq->lock); +if (!fsvq->in_flight) { +spin_unlock (&fsvq->lock); +break ; + } +spin_unlock (&fsvq->lock); +usleep_range (1000 ,2000 ); + }
I think all contexts that call this allow sleeping so we could avoid usleep here.
Vivek Goyal
maintainerRe: [PATCH 08/18] virtiofs: Drain all pending requests during ->remove time
Quoted 10 lines · Stefan Hajnoczi
Quoted 9 lines · Vivek Goyal
fs/fuse/virtio_fs.c +9
-0+static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) +{ +WARN_ON (fsvq->in_flight <0 ); + +/* Wait for in flight requests to finish.*/ +while (1 ) { +spin_lock (&fsvq->lock); +if (!fsvq->in_flight) { +spin_unlock (&fsvq->lock);
Quoted 8 lines
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 79
Quoted 5 lines
fs/fuse/virtio_fs.c +5
-0+break ; + } +spin_unlock (&fsvq->lock); +usleep_range (1000 ,2000 ); + }I think all contexts that call this allow sleeping so we could avoid usleep here.
usleep_range() is supposed to be used from non-atomic context.
https://github.com/torvalds/linux/blob/master/Documentation/timers/timers-howto.rst
What construct you are thinking of?
Vivek
Michael S. Tsirkin
maintainerRe: [PATCH 08/18] virtiofs: Drain all pending requests during ->remove time
Quoted 20 lines · Vivek Goyal
Quoted 10 lines · Stefan Hajnoczi
Quoted 9 lines · Vivek Goyal
fs/fuse/virtio_fs.c +9
-0+static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) +{ +WARN_ON (fsvq->in_flight <0 ); + +/* Wait for in flight requests to finish.*/ +while (1 ) { +spin_lock (&fsvq->lock); +if (!fsvq->in_flight) { +spin_unlock (&fsvq->lock);Quoted 8 lines
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 79
Quoted 5 lines
fs/fuse/virtio_fs.c +5
-0+break ; + } +spin_unlock (&fsvq->lock); +usleep_range (1000 ,2000 ); + }I think all contexts that call this allow sleeping so we could avoid usleep here.
Quoted 7 lines
usleep_range()is supposed to be used from non-atomic context.https://github.com/torvalds/linux/blob/master/Documentation/timers/timers-howto.rst
What construct you are thinking of?
Vivek
completion + signal on vq callback?
Signature · 1 line
MST
Stefan Hajnoczi
maintainerRe: [PATCH 08/18] virtiofs: Drain all pending requests during ->remove time
Quoted 23 lines · Michael S. Tsirkin
Quoted 22 lines · Vivek Goyal
Quoted 10 lines · Stefan Hajnoczi
Quoted 9 lines · Vivek Goyal
fs/fuse/virtio_fs.c +9
-0+static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) +{ +WARN_ON (fsvq->in_flight <0 ); + +/* Wait for in flight requests to finish.*/ +while (1 ) { +spin_lock (&fsvq->lock); +if (!fsvq->in_flight) { +spin_unlock (&fsvq->lock);Quoted 8 lines
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 79
Quoted 5 lines
fs/fuse/virtio_fs.c +5
-0+break ; + } +spin_unlock (&fsvq->lock); +usleep_range (1000 ,2000 ); + }I think all contexts that call this allow sleeping so we could avoid usleep here.
usleep_range()is supposed to be used from non-atomic context.
Quoted 7 lines
Quoted 5 lines
https://github.com/torvalds/linux/blob/master/Documentation/timers/timers-howto.rst
What construct you are thinking of?
Vivek
completion + signal on vq callback?
Yes. Time-based sleep() is sub-optimal because we could wake up exactly when in_flight is decremented from the vq callback. This avoids unnecessary sleep wakeups and the extra time spent sleeping after in_flight has been decremented.
Stefan
Stefan Hajnoczi
maintainerRe: [PATCH 08/18] virtiofs: Drain all pending requests during ->remove time
In reply to Michael S. TsirkinQuoted 23 lines · Michael S. Tsirkin
Quoted 22 lines · Vivek Goyal
Quoted 10 lines · Stefan Hajnoczi
Quoted 9 lines · Vivek Goyal
fs/fuse/virtio_fs.c +9
-0+static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) +{ +WARN_ON (fsvq->in_flight <0 ); + +/* Wait for in flight requests to finish.*/ +while (1 ) { +spin_lock (&fsvq->lock); +if (!fsvq->in_flight) { +spin_unlock (&fsvq->lock);Quoted 8 lines
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 79
Quoted 5 lines
fs/fuse/virtio_fs.c +5
-0+break ; + } +spin_unlock (&fsvq->lock); +usleep_range (1000 ,2000 ); + }I think all contexts that call this allow sleeping so we could avoid usleep here.
usleep_range()is supposed to be used from non-atomic context.
Quoted 7 lines
Quoted 5 lines
https://github.com/torvalds/linux/blob/master/Documentation/timers/timers-howto.rst
What construct you are thinking of?
Vivek
completion + signal on vq callback?
Yes. Time-based sleep() is sub-optimal because we could wake up exactly when in_flight is decremented from the vq callback. This avoids unnecessary sleep wakeups and the extra time spent sleeping after in_flight has been decremented.
Stefan
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerRe: [PATCH 08/18] virtiofs: Drain all pending requests during ->remove time
In reply to Vivek GoyalQuoted 20 lines · Vivek Goyal
Quoted 10 lines · Stefan Hajnoczi
Quoted 9 lines · Vivek Goyal
fs/fuse/virtio_fs.c +9
-0+static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) +{ +WARN_ON (fsvq->in_flight <0 ); + +/* Wait for in flight requests to finish.*/ +while (1 ) { +spin_lock (&fsvq->lock); +if (!fsvq->in_flight) { +spin_unlock (&fsvq->lock);Quoted 8 lines
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 79
Quoted 5 lines
fs/fuse/virtio_fs.c +5
-0+break ; + } +spin_unlock (&fsvq->lock); +usleep_range (1000 ,2000 ); + }I think all contexts that call this allow sleeping so we could avoid usleep here.
Quoted 7 lines
usleep_range()is supposed to be used from non-atomic context.https://github.com/torvalds/linux/blob/master/Documentation/timers/timers-howto.rst
What construct you are thinking of?
Vivek
completion + signal on vq callback?
Signature · 1 line
MST
Vivek Goyal
maintainerRe: [PATCH 08/18] virtiofs: Drain all pending requests during ->remove time
In reply to Stefan HajnocziQuoted 10 lines · Stefan Hajnoczi
Quoted 9 lines · Vivek Goyal
fs/fuse/virtio_fs.c +9
-0+static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) +{ +WARN_ON (fsvq->in_flight <0 ); + +/* Wait for in flight requests to finish.*/ +while (1 ) { +spin_lock (&fsvq->lock); +if (!fsvq->in_flight) { +spin_unlock (&fsvq->lock);
Quoted 8 lines
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 79
Quoted 5 lines
fs/fuse/virtio_fs.c +5
-0+break ; + } +spin_unlock (&fsvq->lock); +usleep_range (1000 ,2000 ); + }I think all contexts that call this allow sleeping so we could avoid usleep here.
usleep_range() is supposed to be used from non-atomic context.
https://github.com/torvalds/linux/blob/master/Documentation/timers/timers-howto.rst
What construct you are thinking of?
Vivek
Stefan Hajnoczi
maintainerRe: [PATCH 08/18] virtiofs: Drain all pending requests during ->remove time
In reply to Vivek GoyalQuoted 4 lines · Vivek Goyal
fs/fuse/virtio_fs.c +4
-0+static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) +{ +WARN_ON (fsvq->in_flight <0 ); +
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 79
Quoted 10 lines
fs/fuse/virtio_fs.c +10
-0+/* Wait for in flight requests to finish.*/ +while (1 ) { +spin_lock (&fsvq->lock); +if (!fsvq->in_flight) { +spin_unlock (&fsvq->lock); +break ; + } +spin_unlock (&fsvq->lock); +usleep_range (1000 ,2000 ); + }
I think all contexts that call this allow sleeping so we could avoid usleep here.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Virtio-fs] [PATCH 08/18] virtiofs: Drain all pending requests during ->remove time
In reply to Vivek GoyalQuoted 12 lines · Vivek Goyal
When device is going away, drain all pending requests.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 83 ++++++++++++++++++++++++++++----------------- 1 file changed, 51 insertions(+), 32 deletions(-)fs/fuse/virtio_fs.c +0
-0
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 70
Quoted 8 lines
fs/fuse/virtio_fs.c +5
-0@@ -63,6 +63,55 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq)return &vq_to_fsvq (vq)->fud->pq; } +static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) +{ +WARN_ON (fsvq->in_flight <0 ); + +/* Wait for in flight requests to finish.*/
blank space missed after *finish.*.
Quoted 5 lines
fs/fuse/virtio_fs.c +5
-0+while (1 ) { +spin_lock (&fsvq->lock); +if (!fsvq->in_flight) { +spin_unlock (&fsvq->lock); +break ;
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 85
Quoted 10 lines
fs/fuse/virtio_fs.c +10
-0+ } +spin_unlock (&fsvq->lock); +usleep_range (1000 ,2000 ); + } + +flush_work (&fsvq->done_work); +flush_delayed_work (&fsvq->dispatch_work); +} + +static inline void drain_hiprio_queued_reqs (struct virtio_fs_vq *fsvq)
Should we add *virtio_fs* prefix for this function? And I wonder if there are only forget reqs to drain? Maybe we should call it *virtio_fs_drain_queued_forget_reqs* or someone containing *forget_reqs*.
Thanks,
Jun
Quoted 88 lines
fs/fuse/virtio_fs.c +31
-32+{ +struct virtio_fs_forget *forget; + +spin_lock (&fsvq->lock); +while (1 ) { + forget =list_first_entry_or_null (&fsvq->queued_reqs, + struct virtio_fs_forget, list); +if (!forget) +break ; +list_del (&forget->list); +kfree (forget); + } +spin_unlock (&fsvq->lock); +} + +static void virtio_fs_drain_all_queues (struct virtio_fs *fs) +{ +struct virtio_fs_vq *fsvq; +int i; + +for (i =0 ; i < fs->nvqs; i++) { + fsvq = &fs->vqs[i]; +if (i ==VQ_HIPRIO ) +drain_hiprio_queued_reqs (fsvq); + +virtio_fs_drain_queue (fsvq); + } +} +/* Add a new instance to the list or return -EEXIST if tag name exists*/ static int virtio_fs_add_instance (struct virtio_fs *fs) {@@ -511,6 +560,7 @@ static void virtio_fs_remove(struct virtio_device *vdev)struct virtio_fs *fs = vdev->priv;virtio_fs_stop_all_queues (fs); +virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);virtio_fs_cleanup_vqs (vdev, fs);@@ -865,37 +915,6 @@ __releases(fiq->waitq.lock) } } -static void virtio_fs_flush_hiprio_queue (struct virtio_fs_vq *fsvq) -{ -struct virtio_fs_forget *forget; - -WARN_ON (fsvq->in_flight <0 ); - -/* Go through pending forget requests and free them */ -spin_lock (&fsvq->lock); -while (1 ) { - forget =list_first_entry_or_null (&fsvq->queued_reqs, - struct virtio_fs_forget, list); -if (!forget) -break ; -list_del (&forget->list); -kfree (forget); - } - -spin_unlock (&fsvq->lock); - -/* Wait for in flight requests to finish.*/ -while (1 ) { -spin_lock (&fsvq->lock); -if (!fsvq->in_flight) { -spin_unlock (&fsvq->lock); -break ; - } -spin_unlock (&fsvq->lock); -usleep_range (1000 ,2000 ); - } -} -const static struct fuse_iqueue_ops virtio_fs_fiq_ops = { .wake_forget_and_unlock = virtio_fs_wake_forget_and_unlock, .wake_interrupt_and_unlock = virtio_fs_wake_interrupt_and_unlock,@@ -988,7 +1007,7 @@ static void virtio_kill_sb(struct super_block *sb)spin_lock (&fsvq->lock); fsvq->connected = false;spin_unlock (&fsvq->lock); -virtio_fs_flush_hiprio_queue (fsvq); +virtio_fs_drain_all_queues (vfs);fuse_kill_sb_anon (sb);virtio_fs_free_devs (vfs);
Vivek Goyal
maintainerPATCH 9/18 virtiofs: Add an helper to start all the queues
In reply to Vivek GoyalThis just marks are the queues are connected and ready to accept the request.
--- fs/fuse/virtio_fs.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
fs/fuse/virtio_fs.c +16 -3
@@ -112,6 +112,19 @@ static void virtio_fs_drain_all_queues(struct virtio_fs *fs) } } +static void virtio_fs_start_all_queues (struct virtio_fs *fs) +{ +struct virtio_fs_vq *fsvq; +int i; + +for (i =0 ; i < fs->nvqs; i++) { + fsvq = &fs->vqs[i]; +spin_lock (&fsvq->lock); + fsvq->connected = true; +spin_unlock (&fsvq->lock); + } +} +/* Add a new instance to the list or return -EEXIST if tag name exists*/ static int virtio_fs_add_instance (struct virtio_fs *fs) {
@@ -483,10 +496,10 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev,if (ret <0 ) goto out; -for (i =0 ; i < fs->nvqs; i++) { +for (i =0 ; i < fs->nvqs; i++) fs->vqs[i].vq = vqs[i]; - fs->vqs[i].connected = true; - } + +virtio_fs_start_all_queues (fs); out:kfree (names);kfree (callbacks);
Signature · 1 line
2.20.1
Re: [PATCH 09/18] virtiofs: Add an helper to start all the queues
Quoted 7 lines · Vivek Goyal
This just marks are the queues are connected and ready to accept the request.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
Re: [PATCH 09/18] virtiofs: Add an helper to start all the queues
In reply to Vivek GoyalQuoted 7 lines · Vivek Goyal
This just marks are the queues are connected and ready to accept the request.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 9/18 virtiofs: Add an helper to start all the queues
In reply to Vivek GoyalThis just marks are the queues are connected and ready to accept the request.
--- fs/fuse/virtio_fs.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
fs/fuse/virtio_fs.c +16 -3
@@ -112,6 +112,19 @@ static void virtio_fs_drain_all_queues(struct virtio_fs *fs) } } +static void virtio_fs_start_all_queues (struct virtio_fs *fs) +{ +struct virtio_fs_vq *fsvq; +int i; + +for (i =0 ; i < fs->nvqs; i++) { + fsvq = &fs->vqs[i]; +spin_lock (&fsvq->lock); + fsvq->connected = true; +spin_unlock (&fsvq->lock); + } +} +/* Add a new instance to the list or return -EEXIST if tag name exists*/ static int virtio_fs_add_instance (struct virtio_fs *fs) {
@@ -483,10 +496,10 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev,if (ret <0 ) goto out; -for (i =0 ; i < fs->nvqs; i++) { +for (i =0 ; i < fs->nvqs; i++) fs->vqs[i].vq = vqs[i]; - fs->vqs[i].connected = true; - } + +virtio_fs_start_all_queues (fs); out:kfree (names);kfree (callbacks);
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 10/18 virtiofs: Do not use device managed mem for virtio_fs and virtio_fs_vq
In reply to Vivek GoyalThese data structures should go away when virtio_fs object is going away. When deivce is going away, we need to just make sure virtqueues can go away and after that none of the code accesses vq and all the requests get error.
So allocate memory for virtio_fs and virtio_fs_vq normally and free it at right time.
This patch still frees up memory during device remove time. A later patch will make virtio_fs object reference counted and this memory will be freed when last reference to object is dropped.
--- fs/fuse/virtio_fs.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
fs/fuse/virtio_fs.c +8 -5
@@ -446,7 +446,7 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev,vq_callback_t **callbacks;const char **names;unsigned int i; -int ret; +int ret =0 ; virtio_cread(vdev, struct virtio_fs_config, num_queues, &fs->num_queues);
@@ -454,9 +454,7 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev,return -EINVAL ; fs->nvqs =1 + fs->num_queues; - - fs->vqs =devm_kcalloc (&vdev->dev, fs->nvqs, -sizeof (fs->vqs[VQ_HIPRIO ]),GFP_KERNEL ); + fs->vqs =kcalloc (fs->nvqs,sizeof (fs->vqs[VQ_HIPRIO ]),GFP_KERNEL );if (!fs->vqs)return -ENOMEM ;
@@ -504,6 +502,8 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev,kfree (names);kfree (callbacks);kfree (vqs); +if (ret) +kfree (fs->vqs);return ret; }
@@ -519,7 +519,7 @@ static int virtio_fs_probe(struct virtio_device *vdev)struct virtio_fs *fs;int ret; - fs =devm_kzalloc (&vdev->dev,sizeof (*fs),GFP_KERNEL ); + fs =kzalloc (sizeof (*fs),GFP_KERNEL );if (!fs)return -ENOMEM ; vdev->priv = fs;
@@ -552,6 +552,7 @@ static int virtio_fs_probe(struct virtio_device *vdev) out: vdev->priv =NULL ; +kfree (fs);return ret; }
@@ -582,6 +583,8 @@ static void virtio_fs_remove(struct virtio_device *vdev)mutex_unlock (&virtio_fs_mutex); vdev->priv =NULL ; +kfree (fs->vqs); +kfree (fs); }#ifdef CONFIG_PM_SLEEP
Signature · 1 line
2.20.1
Re: [PATCH 10/18] virtiofs: Do not use device managed mem for virtio_fs and virtio_fs_vq
Quoted 8 lines · Vivek Goyal
These data structures should go away when
virtio_fsobject is going away. When deivce is going away, we need to just make sure virtqueues can go away and after that none of the code accesses vq and all the requests get error.So allocate memory for
virtio_fsandvirtio_fs_vqnormally and free it at right time.
Quoted 8 lines
This patch still frees up memory during device remove time. A later patch will make
virtio_fsobject reference counted and this memory will be freed when last reference to object is dropped.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
Re: [PATCH 10/18] virtiofs: Do not use device managed mem for virtio_fs and virtio_fs_vq
In reply to Vivek GoyalQuoted 8 lines · Vivek Goyal
These data structures should go away when
virtio_fsobject is going away. When deivce is going away, we need to just make sure virtqueues can go away and after that none of the code accesses vq and all the requests get error.So allocate memory for
virtio_fsandvirtio_fs_vqnormally and free it at right time.
Quoted 8 lines
This patch still frees up memory during device remove time. A later patch will make
virtio_fsobject reference counted and this memory will be freed when last reference to object is dropped.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 10/18 virtiofs: Do not use device managed mem for virtio_fs and virtio_fs_vq
In reply to Vivek GoyalThese data structures should go away when virtio_fs object is going away. When deivce is going away, we need to just make sure virtqueues can go away and after that none of the code accesses vq and all the requests get error.
So allocate memory for virtio_fs and virtio_fs_vq normally and free it at right time.
This patch still frees up memory during device remove time. A later patch will make virtio_fs object reference counted and this memory will be freed when last reference to object is dropped.
--- fs/fuse/virtio_fs.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
fs/fuse/virtio_fs.c +8 -5
@@ -446,7 +446,7 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev,vq_callback_t **callbacks;const char **names;unsigned int i; -int ret; +int ret =0 ; virtio_cread(vdev, struct virtio_fs_config, num_queues, &fs->num_queues);
@@ -454,9 +454,7 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev,return -EINVAL ; fs->nvqs =1 + fs->num_queues; - - fs->vqs =devm_kcalloc (&vdev->dev, fs->nvqs, -sizeof (fs->vqs[VQ_HIPRIO ]),GFP_KERNEL ); + fs->vqs =kcalloc (fs->nvqs,sizeof (fs->vqs[VQ_HIPRIO ]),GFP_KERNEL );if (!fs->vqs)return -ENOMEM ;
@@ -504,6 +502,8 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev,kfree (names);kfree (callbacks);kfree (vqs); +if (ret) +kfree (fs->vqs);return ret; }
@@ -519,7 +519,7 @@ static int virtio_fs_probe(struct virtio_device *vdev)struct virtio_fs *fs;int ret; - fs =devm_kzalloc (&vdev->dev,sizeof (*fs),GFP_KERNEL ); + fs =kzalloc (sizeof (*fs),GFP_KERNEL );if (!fs)return -ENOMEM ; vdev->priv = fs;
@@ -552,6 +552,7 @@ static int virtio_fs_probe(struct virtio_device *vdev) out: vdev->priv =NULL ; +kfree (fs);return ret; }
@@ -582,6 +583,8 @@ static void virtio_fs_remove(struct virtio_device *vdev)mutex_unlock (&virtio_fs_mutex); vdev->priv =NULL ; +kfree (fs->vqs); +kfree (fs); }#ifdef CONFIG_PM_SLEEP
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 11/18 virtiofs: stop and drain queues after sending DESTROY
In reply to Vivek GoyalDuring virtio_kill_sb() we first stop forget queue and drain it and then call fuse_kill_sb_anon(). This will result in sending DESTROY request to fuse server. Once finished, stop all the queues and drain one more time just to be sure and then free up the devices.
Given drain queues will call flush_work() on various workers, remove this logic from virtio_free_devs().
--- fs/fuse/virtio_fs.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
fs/fuse/virtio_fs.c +8 -3
@@ -180,9 +180,6 @@ static void virtio_fs_free_devs(struct virtio_fs *fs)if (!fsvq->fud)continue ; -flush_work (&fsvq->done_work); -flush_delayed_work (&fsvq->dispatch_work); -/* TODO need to quiesce/end_requests/decrement dev_count */ fuse_dev_free (fsvq->fud); fsvq->fud =NULL ;
@@ -994,6 +991,8 @@ static int virtio_fs_fill_super(struct super_block *sb)atomic_inc (&fc->dev_count); } +/* Previous unmount will stop all queues. Start these again */ +virtio_fs_start_all_queues (fs);fuse_send_init (fc, init_req);return 0 ;
@@ -1026,6 +1025,12 @@ static void virtio_kill_sb(struct super_block *sb)virtio_fs_drain_all_queues (vfs);fuse_kill_sb_anon (sb); + +/* fuse_kill_sb_anon() must have sent destroy. Stop all queues +* and drain one more time and free fuse devices. +*/ +virtio_fs_stop_all_queues (vfs); +virtio_fs_drain_all_queues (vfs);virtio_fs_free_devs (vfs); }
Signature · 1 line
2.20.1
Re: [PATCH 11/18] virtiofs: stop and drain queues after sending DESTROY
Quoted 5 lines · Vivek Goyal
During
virtio_kill_sb()we first stop forget queue and drain it and then callfuse_kill_sb_anon(). This will result in sending DESTROY request to fuse server. Once finished, stop all the queues and drain one more time just to be sure and then free up the devices.
Quoted 7 lines
Given drain queues will call
flush_work()on various workers, remove this logic fromvirtio_free_devs().Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
Vivek Goyal
maintainerPATCH 11/18 virtiofs: stop and drain queues after sending DESTROY
In reply to Vivek GoyalDuring virtio_kill_sb() we first stop forget queue and drain it and then call fuse_kill_sb_anon(). This will result in sending DESTROY request to fuse server. Once finished, stop all the queues and drain one more time just to be sure and then free up the devices.
Given drain queues will call flush_work() on various workers, remove this logic from virtio_free_devs().
--- fs/fuse/virtio_fs.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
fs/fuse/virtio_fs.c +8 -3
@@ -180,9 +180,6 @@ static void virtio_fs_free_devs(struct virtio_fs *fs)if (!fsvq->fud)continue ; -flush_work (&fsvq->done_work); -flush_delayed_work (&fsvq->dispatch_work); -/* TODO need to quiesce/end_requests/decrement dev_count */ fuse_dev_free (fsvq->fud); fsvq->fud =NULL ;
@@ -994,6 +991,8 @@ static int virtio_fs_fill_super(struct super_block *sb)atomic_inc (&fc->dev_count); } +/* Previous unmount will stop all queues. Start these again */ +virtio_fs_start_all_queues (fs);fuse_send_init (fc, init_req);return 0 ;
@@ -1026,6 +1025,12 @@ static void virtio_kill_sb(struct super_block *sb)virtio_fs_drain_all_queues (vfs);fuse_kill_sb_anon (sb); + +/* fuse_kill_sb_anon() must have sent destroy. Stop all queues +* and drain one more time and free fuse devices. +*/ +virtio_fs_stop_all_queues (vfs); +virtio_fs_drain_all_queues (vfs);virtio_fs_free_devs (vfs); }
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 12/18 virtiofs: Use virtio_fs_free_devs() in error path
In reply to Vivek GoyalWe already have an helper to cleanup fuse devices. Use that instead of duplicating the code.
--- fs/fuse/virtio_fs.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
fs/fuse/virtio_fs.c +1 -2
@@ -999,8 +999,7 @@ static int virtio_fs_fill_super(struct super_block *sb) err_free_init_req:fuse_request_free (init_req); err_free_fuse_devs: -for (i =0 ; i < fs->nvqs; i++) -fuse_dev_free (fs->vqs[i].fud); +virtio_fs_free_devs (fs); err:return err; }
Signature · 1 line
2.20.1
Re: [PATCH 12/18] virtiofs: Use virtio_fs_free_devs() in error path
Quoted 7 lines · Vivek Goyal
We already have an helper to cleanup fuse devices. Use that instead of duplicating the code.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
Re: [PATCH 12/18] virtiofs: Use virtio_fs_free_devs() in error path
In reply to Vivek GoyalQuoted 7 lines · Vivek Goyal
We already have an helper to cleanup fuse devices. Use that instead of duplicating the code.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 12/18 virtiofs: Use virtio_fs_free_devs() in error path
In reply to Vivek GoyalWe already have an helper to cleanup fuse devices. Use that instead of duplicating the code.
--- fs/fuse/virtio_fs.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
fs/fuse/virtio_fs.c +1 -2
@@ -999,8 +999,7 @@ static int virtio_fs_fill_super(struct super_block *sb) err_free_init_req:fuse_request_free (init_req); err_free_fuse_devs: -for (i =0 ; i < fs->nvqs; i++) -fuse_dev_free (fs->vqs[i].fud); +virtio_fs_free_devs (fs); err:return err; }
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 13/18 virtiofs: Do not access virtqueue in request submission path
In reply to Vivek GoyalIn request submission path it is possible that virtqueue is already gone due to driver->remove(). So do not access it in dev_dbg(). Use pr_debug() instead.
If virtuqueue is gone, this will result in NULL pointer deference.
--- fs/fuse/virtio_fs.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
fs/fuse/virtio_fs.c +4 -4
@@ -888,10 +888,10 @@ __releases(fiq->waitq.lock) fs = fiq->priv; fc = fs->vqs[queue_id].fud->fc; -dev_dbg (&fs->vqs[queue_id].vq->vdev->dev, -"%s: opcode %u unique %#llx nodeid %#llx in.len %u out.len %u\n" , - __func__, req->in.h.opcode, req->in.h.unique, req->in.h.nodeid, - req->in.h.len,fuse_len_args (req->out.numargs, req->out.args)); +pr_debug ("%s: opcode %u unique %#llx nodeid %#llx in.len %u out.len %u" +"\n" , __func__, req->in.h.opcode, req->in.h.unique, + req->in.h.nodeid, req->in.h.len, +fuse_len_args (req->out.numargs, req->out.args)); fpq = &fs->vqs[queue_id].fud->pq;spin_lock (&fpq->lock);
Signature · 1 line
2.20.1
Re: [PATCH 13/18] virtiofs: Do not access virtqueue in request submission path
Quoted 10 lines · Vivek Goyal
In request submission path it is possible that virtqueue is already gone due to
driver->remove(). So do not access it indev_dbg(). Usepr_debug()instead.If virtuqueue is gone, this will result in NULL pointer deference.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
Re: [PATCH 13/18] virtiofs: Do not access virtqueue in request submission path
In reply to Vivek GoyalQuoted 10 lines · Vivek Goyal
In request submission path it is possible that virtqueue is already gone due to
driver->remove(). So do not access it indev_dbg(). Usepr_debug()instead.If virtuqueue is gone, this will result in NULL pointer deference.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 13/18 virtiofs: Do not access virtqueue in request submission path
In reply to Vivek GoyalIn request submission path it is possible that virtqueue is already gone due to driver->remove(). So do not access it in dev_dbg(). Use pr_debug() instead.
If virtuqueue is gone, this will result in NULL pointer deference.
--- fs/fuse/virtio_fs.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
fs/fuse/virtio_fs.c +4 -4
@@ -888,10 +888,10 @@ __releases(fiq->waitq.lock) fs = fiq->priv; fc = fs->vqs[queue_id].fud->fc; -dev_dbg (&fs->vqs[queue_id].vq->vdev->dev, -"%s: opcode %u unique %#llx nodeid %#llx in.len %u out.len %u\n" , - __func__, req->in.h.opcode, req->in.h.unique, req->in.h.nodeid, - req->in.h.len,fuse_len_args (req->out.numargs, req->out.args)); +pr_debug ("%s: opcode %u unique %#llx nodeid %#llx in.len %u out.len %u" +"\n" , __func__, req->in.h.opcode, req->in.h.unique, + req->in.h.nodeid, req->in.h.len, +fuse_len_args (req->out.numargs, req->out.args)); fpq = &fs->vqs[queue_id].fud->pq;spin_lock (&fpq->lock);
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 14/18 virtiofs: Add a fuse_iqueue operation to put() reference
In reply to Vivek GoyalSoon I will make virtio_fs object reference counted, where reference will be taken by device as well as by fuse_conn (fuse_conn->fuse_iqueue->fiq_priv).
When fuse_connection is going away, it should put its reference on virtio_fs object.
So add a fuse_iqueue method which can be used to call into virtio_fs to put the reference on the object (fiq_priv).
--- fs/fuse/fuse_i.h | 5 +++++ fs/fuse/inode.c | 6 ++++++ 2 files changed, 11 insertions(+)
fs/fuse/fuse_i.h +5 -0
@@ -479,6 +479,11 @@ struct fuse_iqueue_ops {*/ void (*wake_pending_and_unlock)(struct fuse_iqueue *fiq) __releases(fiq->waitq.lock); + +/** +* Put a reference on fiq_priv. +*/ +void (*put)(struct fuse_iqueue *fiq); };/** /dev/fuse input queue operations */
fs/fuse/inode.c +6 -0
@@ -631,8 +631,14 @@ EXPORT_SYMBOL_GPL(fuse_conn_init);void fuse_conn_put (struct fuse_conn *fc) {if (refcount_dec_and_test (&fc->count)) { +struct fuse_iqueue *fiq = &fc->iq; +if (fc->destroy_req)fuse_request_free (fc->destroy_req); +if (fiq->priv && fiq->ops->put) { + fiq->ops->put (fiq); + fiq->priv =NULL ; + }put_pid_ns (fc->pid_ns);put_user_ns (fc->user_ns); fc->release (fc);
Signature · 1 line
2.20.1
Stefan Hajnoczi
maintainerRe: [PATCH 14/18] virtiofs: Add a fuse_iqueue operation to put() reference
fs/fuse/fuse_i.h struct fuse_iqueue_ops { · line 484
Quoted 11 lines · Vivek Goyal
fs/fuse/fuse_i.h +3
-0@@ -479,6 +479,11 @@ struct fuse_iqueue_ops {*/ void (*wake_pending_and_unlock)(struct fuse_iqueue *fiq) __releases(fiq->waitq.lock); + +/** +* Put a reference on fiq_priv.
I'm a bit confused about fiq->priv's role in this. The callback takes struct fuse_iqueue *fiq as the argument, not void *priv, so it could theoretically do more than just release priv.
I think one of the following would be clearer:
/**
* Drop a reference to fiq->priv.
*/
void (*put_priv)(void *priv);
Or:
/**
* Clean up when fuse_iqueue is destroyed.
*/
void (*release)(struct fuse_iqueue *fiq);
In the second case fuse_conn_put() shouldn't check fiq->priv.
Vivek Goyal
maintainerRe: [PATCH 14/18] virtiofs: Add a fuse_iqueue operation to put() reference
Quoted 24 lines · Stefan Hajnoczi
fs/fuse/fuse_i.h struct fuse_iqueue_ops { · line 484
Quoted 11 lines · Vivek Goyal
fs/fuse/fuse_i.h +3
-0@@ -479,6 +479,11 @@ struct fuse_iqueue_ops {*/ void (*wake_pending_and_unlock)(struct fuse_iqueue *fiq) __releases(fiq->waitq.lock); + +/** +* Put a reference on fiq_priv. I'm a bit confused about fiq->priv's role in this. The callback takes struct
fuse_iqueue*fiq as the argument, not void *priv, so it could theoretically do more than just release priv.I think one of the following would be clearer:
/**
* Drop a reference tofiq->priv.
*/
void (*put_priv)(void *priv);
Quoted 8 lines
Or:
/**
* Clean up whenfuse_iqueueis destroyed.
*/
void (*release)(structfuse_iqueue*fiq);In the second case
fuse_conn_put()shouldn't checkfiq->priv.
Hi Stefan,
I like using ->release() method sounds better. Here is the updated patch. This also changes the next patch, so will post that as well.
Thanks
Vivek
Subject: virtiofs: Add a fuse_iqueue operation to put() reference
Soon I will make virtio_fs object reference counted, where reference will be taken by device as well as by fuse_conn (fuse_conn->fuse_iqueue->fiq_priv).
When fuse_connection is going away, it should put its reference on virtio_fs object.
So add a fuse_iqueue method which can be used to call into virtio_fs to put the reference on the object (fiq_priv).
--- fs/fuse/fuse_i.h | 5 +++++ fs/fuse/inode.c | 4 ++++ 2 files changed, 9 insertions(+)
fs/fuse/fuse_i.h +5 -0
@@ -479,6 +479,11 @@ struct fuse_iqueue_ops {*/ void (*wake_pending_and_unlock)(struct fuse_iqueue *fiq) __releases(fiq->waitq.lock); + +/** +* Cleanup up when fuse_iqueue is destroyed +*/ +void (*release)(struct fuse_iqueue *fiq); };/** /dev/fuse input queue operations */
fs/fuse/inode.c +4 -0
@@ -631,8 +631,12 @@ EXPORT_SYMBOL_GPL(fuse_conn_init);void fuse_conn_put (struct fuse_conn *fc) {if (refcount_dec_and_test (&fc->count)) { +struct fuse_iqueue *fiq = &fc->iq; +if (fc->destroy_req)fuse_request_free (fc->destroy_req); +if (fiq->ops->release) + fiq->ops->release (fiq);put_pid_ns (fc->pid_ns);put_user_ns (fc->user_ns); fc->release (fc);
Vivek Goyal
maintainerRe: [PATCH 14/18] virtiofs: Add a fuse_iqueue operation to put() reference
In reply to Stefan HajnocziQuoted 24 lines · Stefan Hajnoczi
fs/fuse/fuse_i.h struct fuse_iqueue_ops { · line 484
Quoted 11 lines · Vivek Goyal
fs/fuse/fuse_i.h +3
-0@@ -479,6 +479,11 @@ struct fuse_iqueue_ops {*/ void (*wake_pending_and_unlock)(struct fuse_iqueue *fiq) __releases(fiq->waitq.lock); + +/** +* Put a reference on fiq_priv. I'm a bit confused about fiq->priv's role in this. The callback takes struct
fuse_iqueue*fiq as the argument, not void *priv, so it could theoretically do more than just release priv.I think one of the following would be clearer:
/**
* Drop a reference tofiq->priv.
*/
void (*put_priv)(void *priv);
Quoted 8 lines
Or:
/**
* Clean up whenfuse_iqueueis destroyed.
*/
void (*release)(structfuse_iqueue*fiq);In the second case
fuse_conn_put()shouldn't checkfiq->priv.
Hi Stefan,
I like using ->release() method sounds better. Here is the updated patch. This also changes the next patch, so will post that as well.
Thanks
Vivek
Subject: virtiofs: Add a fuse_iqueue operation to put() reference
Soon I will make virtio_fs object reference counted, where reference will be taken by device as well as by fuse_conn (fuse_conn->fuse_iqueue->fiq_priv).
When fuse_connection is going away, it should put its reference on virtio_fs object.
So add a fuse_iqueue method which can be used to call into virtio_fs to put the reference on the object (fiq_priv).
--- fs/fuse/fuse_i.h | 5 +++++ fs/fuse/inode.c | 4 ++++ 2 files changed, 9 insertions(+)
fs/fuse/fuse_i.h +5 -0
@@ -479,6 +479,11 @@ struct fuse_iqueue_ops {*/ void (*wake_pending_and_unlock)(struct fuse_iqueue *fiq) __releases(fiq->waitq.lock); + +/** +* Cleanup up when fuse_iqueue is destroyed +*/ +void (*release)(struct fuse_iqueue *fiq); };/** /dev/fuse input queue operations */
fs/fuse/inode.c +4 -0
@@ -631,8 +631,12 @@ EXPORT_SYMBOL_GPL(fuse_conn_init);void fuse_conn_put (struct fuse_conn *fc) {if (refcount_dec_and_test (&fc->count)) { +struct fuse_iqueue *fiq = &fc->iq; +if (fc->destroy_req)fuse_request_free (fc->destroy_req); +if (fiq->ops->release) + fiq->ops->release (fiq);put_pid_ns (fc->pid_ns);put_user_ns (fc->user_ns); fc->release (fc);
Stefan Hajnoczi
maintainerRe: [PATCH 14/18] virtiofs: Add a fuse_iqueue operation to put() reference
In reply to Vivek Goyalfs/fuse/fuse_i.h struct fuse_iqueue_ops { · line 484
Quoted 11 lines · Vivek Goyal
fs/fuse/fuse_i.h +3
-0@@ -479,6 +479,11 @@ struct fuse_iqueue_ops {*/ void (*wake_pending_and_unlock)(struct fuse_iqueue *fiq) __releases(fiq->waitq.lock); + +/** +* Put a reference on fiq_priv.
I'm a bit confused about fiq->priv's role in this. The callback takes struct fuse_iqueue *fiq as the argument, not void *priv, so it could theoretically do more than just release priv.
I think one of the following would be clearer:
/**
* Drop a reference to fiq->priv.
*/
void (*put_priv)(void *priv);
Or:
/**
* Clean up when fuse_iqueue is destroyed.
*/
void (*release)(struct fuse_iqueue *fiq);
In the second case fuse_conn_put() shouldn't check fiq->priv.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 14/18 virtiofs: Add a fuse_iqueue operation to put() reference
In reply to Vivek GoyalSoon I will make virtio_fs object reference counted, where reference will be taken by device as well as by fuse_conn (fuse_conn->fuse_iqueue->fiq_priv).
When fuse_connection is going away, it should put its reference on virtio_fs object.
So add a fuse_iqueue method which can be used to call into virtio_fs to put the reference on the object (fiq_priv).
--- fs/fuse/fuse_i.h | 5 +++++ fs/fuse/inode.c | 6 ++++++ 2 files changed, 11 insertions(+)
fs/fuse/fuse_i.h +5 -0
@@ -479,6 +479,11 @@ struct fuse_iqueue_ops {*/ void (*wake_pending_and_unlock)(struct fuse_iqueue *fiq) __releases(fiq->waitq.lock); + +/** +* Put a reference on fiq_priv. +*/ +void (*put)(struct fuse_iqueue *fiq); };/** /dev/fuse input queue operations */
fs/fuse/inode.c +6 -0
@@ -631,8 +631,14 @@ EXPORT_SYMBOL_GPL(fuse_conn_init);void fuse_conn_put (struct fuse_conn *fc) {if (refcount_dec_and_test (&fc->count)) { +struct fuse_iqueue *fiq = &fc->iq; +if (fc->destroy_req)fuse_request_free (fc->destroy_req); +if (fiq->priv && fiq->ops->put) { + fiq->ops->put (fiq); + fiq->priv =NULL ; + }put_pid_ns (fc->pid_ns);put_user_ns (fc->user_ns); fc->release (fc);
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 15/18 virtiofs: Make virtio_fs object refcounted
In reply to Vivek GoyalThis object is used both by fuse_connection as well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.
Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but virtio_fs object will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.
Removing a device also stops all virt queues and any new reuqest gets error -ENOTCONN. All existing in flight requests are drained before ->remove returns.
--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 9 deletions(-)
fs/fuse/virtio_fs.c +43 -9
@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;
@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq)return &vq_to_fsvq (vq)->fud->pq; } +static void release_virtiofs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtiofs_put (struct virtio_fs *fs) +{ +mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtiofs_obj); +mutex_unlock (&virtio_fs_mutex); +} + +static void virtio_fs_put (struct fuse_iqueue *fiq) +{ +struct virtio_fs *vfs = fiq->priv; +virtiofs_put (vfs); +} +static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) {WARN_ON (fsvq->in_flight <0 );
@@ -156,8 +178,10 @@ static struct virtio_fs *virtio_fs_find_instance(const char *tag)mutex_lock (&virtio_fs_mutex);list_for_each_entry (fs, &virtio_fs_instances, list) { -if (strcmp (fs->tag, tag) ==0 ) +if (strcmp (fs->tag, tag) ==0 ) { +kref_get (&fs->refcount); goto found; + } } fs =NULL ;/* not found */
@@ -519,6 +543,7 @@ static int virtio_fs_probe(struct virtio_device *vdev) fs =kzalloc (sizeof (*fs),GFP_KERNEL );if (!fs)return -ENOMEM ; +kref_init (&fs->refcount); vdev->priv = fs; ret =virtio_fs_read_tag (vdev, fs);
@@ -570,18 +595,18 @@ static void virtio_fs_remove(struct virtio_device *vdev) {struct virtio_fs *fs = vdev->priv; +mutex_lock (&virtio_fs_mutex); +list_del_init (&fs->list); +mutex_unlock (&virtio_fs_mutex); +virtio_fs_stop_all_queues (fs);virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);virtio_fs_cleanup_vqs (vdev, fs); -mutex_lock (&virtio_fs_mutex); -list_del (&fs->list); -mutex_unlock (&virtio_fs_mutex); - vdev->priv =NULL ; -kfree (fs->vqs); -kfree (fs); +/* Put device reference on virtio_fs object */ +virtiofs_put (fs); }#ifdef CONFIG_PM_SLEEP
@@ -932,6 +957,7 @@ const static struct fuse_iqueue_ops virtio_fs_fiq_ops = { .wake_forget_and_unlock = virtio_fs_wake_forget_and_unlock, .wake_interrupt_and_unlock = virtio_fs_wake_interrupt_and_unlock, .wake_pending_and_unlock = virtio_fs_wake_pending_and_unlock, + .put = virtio_fs_put, };static int virtio_fs_fill_super (struct super_block *sb)
@@ -1026,7 +1052,9 @@ static void virtio_kill_sb(struct super_block *sb)fuse_kill_sb_anon (sb);/* fuse_kill_sb_anon() must have sent destroy. Stop all queues -* and drain one more time and free fuse devices. +* and drain one more time and free fuse devices. Freeing fuse +* devices will drop their reference on fuse_conn and that in +* turn will drop its reference on virtio_fs object. */ virtio_fs_stop_all_queues (vfs);virtio_fs_drain_all_queues (vfs);
@@ -1060,6 +1088,10 @@ static int virtio_fs_get_tree(struct fs_context *fsc)struct fuse_conn *fc;int err; +/* This gets a reference on virtio_fs object. This ptr gets installed +* in fc->iq->priv. Once fuse_conn is going away, it calls ->put() +* to drop the reference to this object. +*/ fs =virtio_fs_find_instance (fsc->source);if (!fs) {pr_info ("virtio-fs: tag <%s> not found\n" , fsc->source);
@@ -1067,8 +1099,10 @@ static int virtio_fs_get_tree(struct fs_context *fsc) } fc =kzalloc (sizeof (struct fuse_conn ),GFP_KERNEL ); -if (!fc) +if (!fc) { +virtiofs_put (fs);return -ENOMEM ; + }fuse_conn_init (fc,get_user_ns (current_user_ns ()), &virtio_fs_fiq_ops, fs);
Signature · 1 line
2.20.1
Stefan Hajnoczi
maintainerRe: [PATCH 15/18] virtiofs: Make virtio_fs object refcounted
Quoted 32 lines · Vivek Goyal
This object is used both by
fuse_connectionas well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but
virtio_fsobject will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.Removing a device also stops all virt queues and any new reuqest gets error
-ENOTCONN. All existing in flight requests are drained before ->remove returns.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 9 deletions(-)fs/fuse/virtio_fs.c +1
-0@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq)return &vq_to_fsvq (vq)->fud->pq; }
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 75
Quoted 10 lines
fs/fuse/virtio_fs.c +9
-0+static void release_virtiofs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtiofs_put (struct virtio_fs *fs)
Why do the two function names above contain "virtiofs" instead of "virtio_fs"? I'm not sure if this is intentional and is supposed to mean something, but it's confusing.
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 82
Quoted 7 lines
fs/fuse/virtio_fs.c +7
-0+{ +mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtiofs_obj); +mutex_unlock (&virtio_fs_mutex); +} + +static void virtio_fs_put (struct fuse_iqueue *fiq)
Minor issue: this function name is confusingly similar to virtiofs_put(). Please rename to virtio_fs_fiq_put().
Vivek Goyal
maintainerRe: [PATCH 15/18] virtiofs: Make virtio_fs object refcounted
Quoted 48 lines · Stefan Hajnoczi
Quoted 32 lines · Vivek Goyal
This object is used both by
fuse_connectionas well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but
virtio_fsobject will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.Removing a device also stops all virt queues and any new reuqest gets error
-ENOTCONN. All existing in flight requests are drained before ->remove returns.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 9 deletions(-)fs/fuse/virtio_fs.c +1
-0@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq)return &vq_to_fsvq (vq)->fud->pq; }fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 75
Quoted 10 lines
fs/fuse/virtio_fs.c +9
-0+static void release_virtiofs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtiofs_put (struct virtio_fs *fs)Why do the two function names above contain "virtiofs" instead of "
virtio_fs"? I'm not sure if this is intentional and is supposed to mean something, but it's confusing.
Quoted 10 lines
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 82
Quoted 7 lines
fs/fuse/virtio_fs.c +7
-0+{ +mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtiofs_obj); +mutex_unlock (&virtio_fs_mutex); +} + +static void virtio_fs_put (struct fuse_iqueue *fiq)Minor issue: this function name is confusingly similar to
virtiofs_put(). Please rename tovirtio_fs_fiq_put().
Fixed with ->release semantics. Replaced "virtiofs" with "virtio_fs".
Subject: virtiofs: Make virtio_fs object refcounted
This object is used both by fuse_connection as well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.
Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but virtio_fs object will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.
Removing a device also stops all virt queues and any new reuqest gets error -ENOTCONN. All existing in flight requests are drained before ->remove returns.
--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 43 insertions(+), 9 deletions(-)
fs/fuse/virtio_fs.c +43 -9
@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;
@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_return &vq_to_fsvq (vq)->fud->pq; } +static void release_virtio_fs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtio_fs_put (struct virtio_fs *fs) +{ +mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtio_fs_obj); +mutex_unlock (&virtio_fs_mutex); +} + +static void virtio_fs_fiq_release (struct fuse_iqueue *fiq) +{ +struct virtio_fs *vfs = fiq->priv; +virtio_fs_put (vfs); +} +static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) {WARN_ON (fsvq->in_flight <0 );
@@ -156,8 +178,10 @@ static struct virtio_fs *virtio_fs_find_mutex_lock (&virtio_fs_mutex);list_for_each_entry (fs, &virtio_fs_instances, list) { -if (strcmp (fs->tag, tag) ==0 ) +if (strcmp (fs->tag, tag) ==0 ) { +kref_get (&fs->refcount); goto found; + } } fs =NULL ;/* not found */
@@ -519,6 +543,7 @@ static int virtio_fs_probe(struct virtio fs =kzalloc (sizeof (*fs),GFP_KERNEL );if (!fs)return -ENOMEM ; +kref_init (&fs->refcount); vdev->priv = fs; ret =virtio_fs_read_tag (vdev, fs);
@@ -570,18 +595,18 @@ static void virtio_fs_remove(struct virt {struct virtio_fs *fs = vdev->priv; +mutex_lock (&virtio_fs_mutex); +list_del_init (&fs->list); +mutex_unlock (&virtio_fs_mutex); +virtio_fs_stop_all_queues (fs);virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);virtio_fs_cleanup_vqs (vdev, fs); -mutex_lock (&virtio_fs_mutex); -list_del (&fs->list); -mutex_unlock (&virtio_fs_mutex); - vdev->priv =NULL ; -kfree (fs->vqs); -kfree (fs); +/* Put device reference on virtio_fs object */ +virtio_fs_put (fs); }#ifdef CONFIG_PM_SLEEP
@@ -932,6 +957,7 @@ const static struct fuse_iqueue_ops virt .wake_forget_and_unlock = virtio_fs_wake_forget_and_unlock, .wake_interrupt_and_unlock = virtio_fs_wake_interrupt_and_unlock, .wake_pending_and_unlock = virtio_fs_wake_pending_and_unlock, + .release = virtio_fs_fiq_release, };static int virtio_fs_fill_super (struct super_block *sb)
@@ -1026,7 +1052,9 @@ static void virtio_kill_sb(struct super_fuse_kill_sb_anon (sb);/* fuse_kill_sb_anon() must have sent destroy. Stop all queues -* and drain one more time and free fuse devices. +* and drain one more time and free fuse devices. Freeing fuse +* devices will drop their reference on fuse_conn and that in +* turn will drop its reference on virtio_fs object. */ virtio_fs_stop_all_queues (vfs);virtio_fs_drain_all_queues (vfs);
@@ -1060,6 +1088,10 @@ static int virtio_fs_get_tree(struct fs_struct fuse_conn *fc;int err; +/* This gets a reference on virtio_fs object. This ptr gets installed +* in fc->iq->priv. Once fuse_conn is going away, it calls ->put() +* to drop the reference to this object. +*/ fs =virtio_fs_find_instance (fsc->source);if (!fs) {pr_info ("virtio-fs: tag <%s> not found\n" , fsc->source);
@@ -1067,8 +1099,10 @@ static int virtio_fs_get_tree(struct fs_ } fc =kzalloc (sizeof (struct fuse_conn ),GFP_KERNEL ); -if (!fc) +if (!fc) { +virtio_fs_put (fs);return -ENOMEM ; + }fuse_conn_init (fc,get_user_ns (current_user_ns ()), &virtio_fs_fiq_ops, fs);
Re: [PATCH 15/18] virtiofs: Make virtio_fs object refcounted
Quoted 199 lines · Vivek Goyal
Quoted 48 lines · Stefan Hajnoczi
Quoted 32 lines · Vivek Goyal
This object is used both by
fuse_connectionas well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but
virtio_fsobject will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.Removing a device also stops all virt queues and any new reuqest gets error
-ENOTCONN. All existing in flight requests are drained before ->remove returns.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 9 deletions(-)fs/fuse/virtio_fs.c +1
-0@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq)return &vq_to_fsvq (vq)->fud->pq; }fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 75
Quoted 10 lines
fs/fuse/virtio_fs.c +9
-0+static void release_virtiofs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtiofs_put (struct virtio_fs *fs)Why do the two function names above contain "virtiofs" instead of "
virtio_fs"? I'm not sure if this is intentional and is supposed to mean something, but it's confusing.Quoted 10 lines
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_
Quoted 7 lines
fs/fuse/virtio_fs.c +7
-0+{ +mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtiofs_obj); +mutex_unlock (&virtio_fs_mutex); +} + +static void virtio_fs_put (struct fuse_iqueue *fiq)Minor issue: this function name is confusingly similar to
virtiofs_put(). Please rename tovirtio_fs_fiq_put().Fixed with ->release semantics. Replaced "virtiofs" with "
virtio_fs".Subject: virtiofs: Make
virtio_fsobject refcountedThis object is used both by
fuse_connectionas well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but
virtio_fsobject will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.Removing a device also stops all virt queues and any new reuqest gets error
-ENOTCONN. All existing in flight requests are drained before ->remove returns.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 43 insertions(+), 9 deletions(-)Index:
rhvgoyal-linux-fuse/fs/fuse/virtio_fs.c
===================================================================fs/fuse/virtio_fs.c +40
-8@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_return &vq_to_fsvq (vq)->fud->pq; } +static void release_virtio_fs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtio_fs_put (struct virtio_fs *fs) +{ +mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtio_fs_obj); +mutex_unlock (&virtio_fs_mutex); +} + +static void virtio_fs_fiq_release (struct fuse_iqueue *fiq) +{ +struct virtio_fs *vfs = fiq->priv; +virtio_fs_put (vfs); +} +static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) {WARN_ON (fsvq->in_flight <0 );@@ -156,8 +178,10 @@ static struct virtio_fs *virtio_fs_find_mutex_lock (&virtio_fs_mutex);list_for_each_entry (fs, &virtio_fs_instances, list) { -if (strcmp (fs->tag, tag) ==0 ) +if (strcmp (fs->tag, tag) ==0 ) { +kref_get (&fs->refcount); goto found; + } } fs =NULL ;/* not found */ @@ -519,6 +543,7 @@ static int virtio_fs_probe(struct virtio fs =kzalloc (sizeof (*fs),GFP_KERNEL );if (!fs)return -ENOMEM ; +kref_init (&fs->refcount); vdev->priv = fs; ret =virtio_fs_read_tag (vdev, fs);@@ -570,18 +595,18 @@ static void virtio_fs_remove(struct virt {struct virtio_fs *fs = vdev->priv; +mutex_lock (&virtio_fs_mutex); +list_del_init (&fs->list); +mutex_unlock (&virtio_fs_mutex); +virtio_fs_stop_all_queues (fs);virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);virtio_fs_cleanup_vqs (vdev, fs); -mutex_lock (&virtio_fs_mutex); -list_del (&fs->list); -mutex_unlock (&virtio_fs_mutex); - vdev->priv =NULL ; -kfree (fs->vqs); -kfree (fs); +/* Put device reference on virtio_fs object */ +virtio_fs_put (fs); }#ifdef CONFIG_PM_SLEEP @@ -932,6 +957,7 @@ const static struct fuse_iqueue_ops virt .wake_forget_and_unlock = virtio_fs_wake_forget_and_unlock, .wake_interrupt_and_unlock = virtio_fs_wake_interrupt_and_unlock, .wake_pending_and_unlock = virtio_fs_wake_pending_and_unlock, + .release = virtio_fs_fiq_release, };static int virtio_fs_fill_super (struct super_block *sb)@@ -1026,7 +1052,9 @@ static void virtio_kill_sb(struct super_fuse_kill_sb_anon (sb);/* fuse_kill_sb_anon() must have sent destroy. Stop all queues -* and drain one more time and free fuse devices. +* and drain one more time and free fuse devices. Freeing fuse +* devices will drop their reference on fuse_conn and that in +* turn will drop its reference on virtio_fs object. */ virtio_fs_stop_all_queues (vfs);virtio_fs_drain_all_queues (vfs);@@ -1060,6 +1088,10 @@ static int virtio_fs_get_tree(struct fs_struct fuse_conn *fc;int err; +/* This gets a reference on virtio_fs object. This ptr gets installed +* in fc->iq->priv. Once fuse_conn is going away, it calls ->put() +* to drop the reference to this object. +*/ fs =virtio_fs_find_instance (fsc->source);if (!fs) {pr_info ("virtio-fs: tag <%s> not found\n" , fsc->source);@@ -1067,8 +1099,10 @@ static int virtio_fs_get_tree(struct fs_ }
fs/fuse/virtio_fs.c static int virtio_fs_get_tree(struct fs_ · line 1108
Quoted 10 lines
fs/fuse/virtio_fs.c +3
-1fc =kzalloc (sizeof (struct fuse_conn ),GFP_KERNEL ); -if (!fc) +if (!fc) { +virtio_fs_put (fs);return -ENOMEM ; + }fuse_conn_init (fc,get_user_ns (current_user_ns ()), &virtio_fs_fiq_ops, fs);
Vivek Goyal
maintainerRe: [PATCH 15/18] virtiofs: Make virtio_fs object refcounted
In reply to Stefan HajnocziQuoted 48 lines · Stefan Hajnoczi
Quoted 32 lines · Vivek Goyal
This object is used both by
fuse_connectionas well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but
virtio_fsobject will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.Removing a device also stops all virt queues and any new reuqest gets error
-ENOTCONN. All existing in flight requests are drained before ->remove returns.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 9 deletions(-)fs/fuse/virtio_fs.c +1
-0@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq)return &vq_to_fsvq (vq)->fud->pq; }fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 75
Quoted 10 lines
fs/fuse/virtio_fs.c +9
-0+static void release_virtiofs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtiofs_put (struct virtio_fs *fs)Why do the two function names above contain "virtiofs" instead of "
virtio_fs"? I'm not sure if this is intentional and is supposed to mean something, but it's confusing.
Quoted 10 lines
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 82
Quoted 7 lines
fs/fuse/virtio_fs.c +7
-0+{ +mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtiofs_obj); +mutex_unlock (&virtio_fs_mutex); +} + +static void virtio_fs_put (struct fuse_iqueue *fiq)Minor issue: this function name is confusingly similar to
virtiofs_put(). Please rename tovirtio_fs_fiq_put().
Fixed with ->release semantics. Replaced "virtiofs" with "virtio_fs".
Subject: virtiofs: Make virtio_fs object refcounted
This object is used both by fuse_connection as well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.
Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but virtio_fs object will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.
Removing a device also stops all virt queues and any new reuqest gets error -ENOTCONN. All existing in flight requests are drained before ->remove returns.
--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 43 insertions(+), 9 deletions(-)
fs/fuse/virtio_fs.c +43 -9
@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;
@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_return &vq_to_fsvq (vq)->fud->pq; } +static void release_virtio_fs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtio_fs_put (struct virtio_fs *fs) +{ +mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtio_fs_obj); +mutex_unlock (&virtio_fs_mutex); +} + +static void virtio_fs_fiq_release (struct fuse_iqueue *fiq) +{ +struct virtio_fs *vfs = fiq->priv; +virtio_fs_put (vfs); +} +static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) {WARN_ON (fsvq->in_flight <0 );
@@ -156,8 +178,10 @@ static struct virtio_fs *virtio_fs_find_mutex_lock (&virtio_fs_mutex);list_for_each_entry (fs, &virtio_fs_instances, list) { -if (strcmp (fs->tag, tag) ==0 ) +if (strcmp (fs->tag, tag) ==0 ) { +kref_get (&fs->refcount); goto found; + } } fs =NULL ;/* not found */
@@ -519,6 +543,7 @@ static int virtio_fs_probe(struct virtio fs =kzalloc (sizeof (*fs),GFP_KERNEL );if (!fs)return -ENOMEM ; +kref_init (&fs->refcount); vdev->priv = fs; ret =virtio_fs_read_tag (vdev, fs);
@@ -570,18 +595,18 @@ static void virtio_fs_remove(struct virt {struct virtio_fs *fs = vdev->priv; +mutex_lock (&virtio_fs_mutex); +list_del_init (&fs->list); +mutex_unlock (&virtio_fs_mutex); +virtio_fs_stop_all_queues (fs);virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);virtio_fs_cleanup_vqs (vdev, fs); -mutex_lock (&virtio_fs_mutex); -list_del (&fs->list); -mutex_unlock (&virtio_fs_mutex); - vdev->priv =NULL ; -kfree (fs->vqs); -kfree (fs); +/* Put device reference on virtio_fs object */ +virtio_fs_put (fs); }#ifdef CONFIG_PM_SLEEP
@@ -932,6 +957,7 @@ const static struct fuse_iqueue_ops virt .wake_forget_and_unlock = virtio_fs_wake_forget_and_unlock, .wake_interrupt_and_unlock = virtio_fs_wake_interrupt_and_unlock, .wake_pending_and_unlock = virtio_fs_wake_pending_and_unlock, + .release = virtio_fs_fiq_release, };static int virtio_fs_fill_super (struct super_block *sb)
@@ -1026,7 +1052,9 @@ static void virtio_kill_sb(struct super_fuse_kill_sb_anon (sb);/* fuse_kill_sb_anon() must have sent destroy. Stop all queues -* and drain one more time and free fuse devices. +* and drain one more time and free fuse devices. Freeing fuse +* devices will drop their reference on fuse_conn and that in +* turn will drop its reference on virtio_fs object. */ virtio_fs_stop_all_queues (vfs);virtio_fs_drain_all_queues (vfs);
@@ -1060,6 +1088,10 @@ static int virtio_fs_get_tree(struct fs_struct fuse_conn *fc;int err; +/* This gets a reference on virtio_fs object. This ptr gets installed +* in fc->iq->priv. Once fuse_conn is going away, it calls ->put() +* to drop the reference to this object. +*/ fs =virtio_fs_find_instance (fsc->source);if (!fs) {pr_info ("virtio-fs: tag <%s> not found\n" , fsc->source);
@@ -1067,8 +1099,10 @@ static int virtio_fs_get_tree(struct fs_ } fc =kzalloc (sizeof (struct fuse_conn ),GFP_KERNEL ); -if (!fc) +if (!fc) { +virtio_fs_put (fs);return -ENOMEM ; + }fuse_conn_init (fc,get_user_ns (current_user_ns ()), &virtio_fs_fiq_ops, fs);
Stefan Hajnoczi
maintainerRe: [PATCH 15/18] virtiofs: Make virtio_fs object refcounted
In reply to Vivek GoyalQuoted 32 lines · Vivek Goyal
This object is used both by
fuse_connectionas well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but
virtio_fsobject will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.Removing a device also stops all virt queues and any new reuqest gets error
-ENOTCONN. All existing in flight requests are drained before ->remove returns.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 9 deletions(-)fs/fuse/virtio_fs.c +1
-0@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq)return &vq_to_fsvq (vq)->fud->pq; }
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 75
Quoted 10 lines
fs/fuse/virtio_fs.c +9
-0+static void release_virtiofs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtiofs_put (struct virtio_fs *fs)
Why do the two function names above contain "virtiofs" instead of "virtio_fs"? I'm not sure if this is intentional and is supposed to mean something, but it's confusing.
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 82
Quoted 7 lines
fs/fuse/virtio_fs.c +7
-0+{ +mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtiofs_obj); +mutex_unlock (&virtio_fs_mutex); +} + +static void virtio_fs_put (struct fuse_iqueue *fiq)
Minor issue: this function name is confusingly similar to virtiofs_put(). Please rename to virtio_fs_fiq_put().
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Virtio-fs] [PATCH 15/18] virtiofs: Make virtio_fs object refcounted
In reply to Vivek GoyalQuoted 43 lines · Vivek Goyal
This object is used both by
fuse_connectionas well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but
virtio_fsobject will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.Removing a device also stops all virt queues and any new reuqest gets error
-ENOTCONN. All existing in flight requests are drained before ->remove returns.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 9 deletions(-)fs/fuse/virtio_fs.c +11
-0@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq)return &vq_to_fsvq (vq)->fud->pq; } +static void release_virtiofs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtio_fs_put (struct virtio_fs *fs) +{
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 86
Quoted 10 lines
fs/fuse/virtio_fs.c +10
-0+mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtiofs_obj); +mutex_unlock (&virtio_fs_mutex); +} + +static void virtio_fs_put (struct fuse_iqueue *fiq) +{ +struct virtio_fs *vfs = fiq->priv; +virtiofs_put (vfs); +}
It's a little confusing that virtiofs_put() looks like virtiofs_put(), and could we use __virtio_fs_put to replace virtio_fs_put?
Thanks,
Jun
Vivek Goyal
maintainerRe: [Virtio-fs] [PATCH 15/18] virtiofs: Make virtio_fs object refcounted
Quoted 51 lines · piaojun
Quoted 48 lines · Vivek Goyal
This object is used both by
fuse_connectionas well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but
virtio_fsobject will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.Removing a device also stops all virt queues and any new reuqest gets error
-ENOTCONN. All existing in flight requests are drained before ->remove returns.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 9 deletions(-)fs/fuse/virtio_fs.c +16
-0@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq)return &vq_to_fsvq (vq)->fud->pq; } +static void release_virtiofs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtio_fs_put (struct virtio_fs *fs) +{ +mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtiofs_obj); +mutex_unlock (&virtio_fs_mutex); +} +
Quoted 8 lines
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 86
Quoted 5 lines
fs/fuse/virtio_fs.c +5
-0+static void virtio_fs_put (struct fuse_iqueue *fiq) +{ +struct virtio_fs *vfs = fiq->priv; +virtiofs_put (vfs); +}It's a little confusing that
virtiofs_put()looks likevirtiofs_put(), and could we use__virtio_fs_putto replacevirtio_fs_put?
Fixed this in follow up patch I posted.
https://www.redhat.com/archives/virtio-fs/2019-September/msg00091.html
Vivek
Vivek Goyal
maintainerRe: [Virtio-fs] [PATCH 15/18] virtiofs: Make virtio_fs object refcounted
In reply to piaojunQuoted 51 lines · piaojun
Quoted 48 lines · Vivek Goyal
This object is used both by
fuse_connectionas well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but
virtio_fsobject will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.Removing a device also stops all virt queues and any new reuqest gets error
-ENOTCONN. All existing in flight requests are drained before ->remove returns.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 9 deletions(-)fs/fuse/virtio_fs.c +16
-0@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq)return &vq_to_fsvq (vq)->fud->pq; } +static void release_virtiofs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtio_fs_put (struct virtio_fs *fs) +{ +mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtiofs_obj); +mutex_unlock (&virtio_fs_mutex); +} +
Quoted 8 lines
fs/fuse/virtio_fs.c static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) · line 86
Quoted 5 lines
fs/fuse/virtio_fs.c +5
-0+static void virtio_fs_put (struct fuse_iqueue *fiq) +{ +struct virtio_fs *vfs = fiq->priv; +virtiofs_put (vfs); +}It's a little confusing that
virtiofs_put()looks likevirtiofs_put(), and could we use__virtio_fs_putto replacevirtio_fs_put?
Fixed this in follow up patch I posted.
https://www.redhat.com/archives/virtio-fs/2019-September/msg00091.html
Vivek
Vivek Goyal
maintainerPATCH 15/18 virtiofs: Make virtio_fs object refcounted
In reply to Vivek GoyalThis object is used both by fuse_connection as well virt device. So make this object reference counted and that makes it easy to define life cycle of the object.
Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but virtio_fs object will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference.
Removing a device also stops all virt queues and any new reuqest gets error -ENOTCONN. All existing in flight requests are drained before ->remove returns.
--- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 9 deletions(-)
fs/fuse/virtio_fs.c +43 -9
@@ -37,6 +37,7 @@ struct virtio_fs_vq {/* A virtio-fs device instance */ struct virtio_fs { +struct kref refcount;struct list_head list;/* on virtio_fs_instances */ char *tag;struct virtio_fs_vq *vqs;
@@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq)return &vq_to_fsvq (vq)->fud->pq; } +static void release_virtiofs_obj (struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + +kfree (vfs->vqs); +kfree (vfs); +} + +static void virtiofs_put (struct virtio_fs *fs) +{ +mutex_lock (&virtio_fs_mutex); +kref_put (&fs->refcount, release_virtiofs_obj); +mutex_unlock (&virtio_fs_mutex); +} + +static void virtio_fs_put (struct fuse_iqueue *fiq) +{ +struct virtio_fs *vfs = fiq->priv; +virtiofs_put (vfs); +} +static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq) {WARN_ON (fsvq->in_flight <0 );
@@ -156,8 +178,10 @@ static struct virtio_fs *virtio_fs_find_instance(const char *tag)mutex_lock (&virtio_fs_mutex);list_for_each_entry (fs, &virtio_fs_instances, list) { -if (strcmp (fs->tag, tag) ==0 ) +if (strcmp (fs->tag, tag) ==0 ) { +kref_get (&fs->refcount); goto found; + } } fs =NULL ;/* not found */
@@ -519,6 +543,7 @@ static int virtio_fs_probe(struct virtio_device *vdev) fs =kzalloc (sizeof (*fs),GFP_KERNEL );if (!fs)return -ENOMEM ; +kref_init (&fs->refcount); vdev->priv = fs; ret =virtio_fs_read_tag (vdev, fs);
@@ -570,18 +595,18 @@ static void virtio_fs_remove(struct virtio_device *vdev) {struct virtio_fs *fs = vdev->priv; +mutex_lock (&virtio_fs_mutex); +list_del_init (&fs->list); +mutex_unlock (&virtio_fs_mutex); +virtio_fs_stop_all_queues (fs);virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);virtio_fs_cleanup_vqs (vdev, fs); -mutex_lock (&virtio_fs_mutex); -list_del (&fs->list); -mutex_unlock (&virtio_fs_mutex); - vdev->priv =NULL ; -kfree (fs->vqs); -kfree (fs); +/* Put device reference on virtio_fs object */ +virtiofs_put (fs); }#ifdef CONFIG_PM_SLEEP
@@ -932,6 +957,7 @@ const static struct fuse_iqueue_ops virtio_fs_fiq_ops = { .wake_forget_and_unlock = virtio_fs_wake_forget_and_unlock, .wake_interrupt_and_unlock = virtio_fs_wake_interrupt_and_unlock, .wake_pending_and_unlock = virtio_fs_wake_pending_and_unlock, + .put = virtio_fs_put, };static int virtio_fs_fill_super (struct super_block *sb)
@@ -1026,7 +1052,9 @@ static void virtio_kill_sb(struct super_block *sb)fuse_kill_sb_anon (sb);/* fuse_kill_sb_anon() must have sent destroy. Stop all queues -* and drain one more time and free fuse devices. +* and drain one more time and free fuse devices. Freeing fuse +* devices will drop their reference on fuse_conn and that in +* turn will drop its reference on virtio_fs object. */ virtio_fs_stop_all_queues (vfs);virtio_fs_drain_all_queues (vfs);
@@ -1060,6 +1088,10 @@ static int virtio_fs_get_tree(struct fs_context *fsc)struct fuse_conn *fc;int err; +/* This gets a reference on virtio_fs object. This ptr gets installed +* in fc->iq->priv. Once fuse_conn is going away, it calls ->put() +* to drop the reference to this object. +*/ fs =virtio_fs_find_instance (fsc->source);if (!fs) {pr_info ("virtio-fs: tag <%s> not found\n" , fsc->source);
@@ -1067,8 +1099,10 @@ static int virtio_fs_get_tree(struct fs_context *fsc) } fc =kzalloc (sizeof (struct fuse_conn ),GFP_KERNEL ); -if (!fc) +if (!fc) { +virtiofs_put (fs);return -ENOMEM ; + }fuse_conn_init (fc,get_user_ns (current_user_ns ()), &virtio_fs_fiq_ops, fs);
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 16/18 virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path
In reply to Vivek GoyalIt is possible that a mount is in progress and device is being removed at the same time. Use virtio_fs_mutex to avoid races.
This also takes care of bunch of races and removes some TODO items.
--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)
fs/fuse/virtio_fs.c +22 -10
@@ -13,7 +13,9 @@#include <linux/highmem.h> #include "fuse_i.h" -/* List of virtio-fs device instances and a lock for the list */ +/* List of virtio-fs device instances and a lock for the list. Also provides +* mutual exclusion in device removal and mounting path +*/ static DEFINE_MUTEX (virtio_fs_mutex);static LIST_HEAD (virtio_fs_instances);
@@ -72,17 +74,19 @@ static void release_virtiofs_obj(struct kref *ref)kfree (vfs); } +/* Make sure virtiofs_mutex is held */ static void virtiofs_put (struct virtio_fs *fs) { -mutex_lock (&virtio_fs_mutex);kref_put (&fs->refcount, release_virtiofs_obj); -mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_put (struct fuse_iqueue *fiq) {struct virtio_fs *vfs = fiq->priv; + +mutex_lock (&virtio_fs_mutex);virtiofs_put (vfs); +mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq)
@@ -596,9 +600,8 @@ static void virtio_fs_remove(struct virtio_device *vdev)struct virtio_fs *fs = vdev->priv;mutex_lock (&virtio_fs_mutex); +/* This device is going away. No one should get new reference */ list_del_init (&fs->list); -mutex_unlock (&virtio_fs_mutex); -virtio_fs_stop_all_queues (fs);virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);
@@ -607,6 +610,7 @@ static void virtio_fs_remove(struct virtio_device *vdev) vdev->priv =NULL ;/* Put device reference on virtio_fs object */ virtiofs_put (fs); +mutex_unlock (&virtio_fs_mutex); }#ifdef CONFIG_PM_SLEEP
@@ -978,10 +982,15 @@ static int virtio_fs_fill_super(struct super_block *sb) .no_force_umount = true, }; -/* TODO lock */ -if (fs->vqs[VQ_REQUEST ].fud) { -pr_err ("virtio-fs: device already in use\n" ); - err = -EBUSY ; +mutex_lock (&virtio_fs_mutex); + +/* After holding mutex, make sure virtiofs device is still there. +* Though we are holding a refernce to it, drive ->remove might +* still have cleaned up virtual queues. In that case bail out. +*/ + err = -EINVAL ; +if (list_empty (&fs->list)) { +pr_info ("virtio-fs: tag <%s> not found\n" , fs->tag); goto err; }
@@ -1007,7 +1016,6 @@ static int virtio_fs_fill_super(struct super_block *sb) fc = fs->vqs[VQ_REQUEST ].fud->fc; -/* TODO take fuse_mutex around this loop? */ for (i =0 ; i < fs->nvqs; i++) {struct virtio_fs_vq *fsvq = &fs->vqs[i];
@@ -1020,6 +1028,7 @@ static int virtio_fs_fill_super(struct super_block *sb)/* Previous unmount will stop all queues. Start these again */ virtio_fs_start_all_queues (fs);fuse_send_init (fc, init_req); +mutex_unlock (&virtio_fs_mutex);return 0 ; err_free_init_req:
@@ -1027,6 +1036,7 @@ static int virtio_fs_fill_super(struct super_block *sb) err_free_fuse_devs:virtio_fs_free_devs (fs); err: +mutex_unlock (&virtio_fs_mutex);return err; }
@@ -1100,7 +1110,9 @@ static int virtio_fs_get_tree(struct fs_context *fsc) fc =kzalloc (sizeof (struct fuse_conn ),GFP_KERNEL );if (!fc) { +mutex_lock (&virtio_fs_mutex);virtiofs_put (fs); +mutex_unlock (&virtio_fs_mutex);return -ENOMEM ; }
Signature · 1 line
2.20.1
Re: [PATCH 16/18] virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path
Quoted 9 lines · Vivek Goyal
It is possible that a mount is in progress and device is being removed at the same time. Use
virtio_fs_mutexto avoid races.This also takes care of bunch of races and removes some TODO items.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)
Let's move to a per-device mutex in the future. That way a single device that fails to drain/complete requests will not hang all other virtio-fs instances. This is fine for now.
Vivek Goyal
maintainerRe: [PATCH 16/18] virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path
Quoted 6 lines · Stefan Hajnoczi
Quoted 5 lines · Vivek Goyal
It is possible that a mount is in progress and device is being removed at the same time. Use
virtio_fs_mutexto avoid races.This also takes care of bunch of races and removes some TODO items.
Quoted 8 lines
Quoted 4 lines
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)Let's move to a per-device mutex in the future. That way a single device that fails to drain/complete requests will not hang all other virtio-fs instances. This is fine for now.
Good point. For now I updated the patch so that it applies cleanly after previous two patches changed.
Subject: virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path
It is possible that a mount is in progress and device is being removed at the same time. Use virtio_fs_mutex to avoid races.
This also takes care of bunch of races and removes some TODO items.
--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)
fs/fuse/virtio_fs.c +22 -10
@@ -13,7 +13,9 @@#include <linux/highmem.h> #include "fuse_i.h" -/* List of virtio-fs device instances and a lock for the list */ +/* List of virtio-fs device instances and a lock for the list. Also provides +* mutual exclusion in device removal and mounting path +*/ static DEFINE_MUTEX (virtio_fs_mutex);static LIST_HEAD (virtio_fs_instances);
@@ -72,17 +74,19 @@ static void release_virtio_fs_obj(structkfree (vfs); } +/* Make sure virtiofs_mutex is held */ static void virtio_fs_put (struct virtio_fs *fs) { -mutex_lock (&virtio_fs_mutex);kref_put (&fs->refcount, release_virtio_fs_obj); -mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_fiq_release (struct fuse_iqueue *fiq) {struct virtio_fs *vfs = fiq->priv; + +mutex_lock (&virtio_fs_mutex);virtio_fs_put (vfs); +mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq)
@@ -596,9 +600,8 @@ static void virtio_fs_remove(struct virtstruct virtio_fs *fs = vdev->priv;mutex_lock (&virtio_fs_mutex); +/* This device is going away. No one should get new reference */ list_del_init (&fs->list); -mutex_unlock (&virtio_fs_mutex); -virtio_fs_stop_all_queues (fs);virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);
@@ -607,6 +610,7 @@ static void virtio_fs_remove(struct virt vdev->priv =NULL ;/* Put device reference on virtio_fs object */ virtio_fs_put (fs); +mutex_unlock (&virtio_fs_mutex); }#ifdef CONFIG_PM_SLEEP
@@ -978,10 +982,15 @@ static int virtio_fs_fill_super(struct s .no_force_umount = true, }; -/* TODO lock */ -if (fs->vqs[VQ_REQUEST ].fud) { -pr_err ("virtio-fs: device already in use\n" ); - err = -EBUSY ; +mutex_lock (&virtio_fs_mutex); + +/* After holding mutex, make sure virtiofs device is still there. +* Though we are holding a refernce to it, drive ->remove might +* still have cleaned up virtual queues. In that case bail out. +*/ + err = -EINVAL ; +if (list_empty (&fs->list)) { +pr_info ("virtio-fs: tag <%s> not found\n" , fs->tag); goto err; }
@@ -1007,7 +1016,6 @@ static int virtio_fs_fill_super(struct s fc = fs->vqs[VQ_REQUEST ].fud->fc; -/* TODO take fuse_mutex around this loop? */ for (i =0 ; i < fs->nvqs; i++) {struct virtio_fs_vq *fsvq = &fs->vqs[i];
@@ -1020,6 +1028,7 @@ static int virtio_fs_fill_super(struct s/* Previous unmount will stop all queues. Start these again */ virtio_fs_start_all_queues (fs);fuse_send_init (fc, init_req); +mutex_unlock (&virtio_fs_mutex);return 0 ; err_free_init_req:
@@ -1027,6 +1036,7 @@ err_free_init_req: err_free_fuse_devs:virtio_fs_free_devs (fs); err: +mutex_unlock (&virtio_fs_mutex);return err; }
@@ -1100,7 +1110,9 @@ static int virtio_fs_get_tree(struct fs_ fc =kzalloc (sizeof (struct fuse_conn ),GFP_KERNEL );if (!fc) { +mutex_lock (&virtio_fs_mutex);virtio_fs_put (fs); +mutex_unlock (&virtio_fs_mutex);return -ENOMEM ; }
Re: [PATCH 16/18] virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path
Quoted 133 lines · Vivek Goyal
Quoted 6 lines · Stefan Hajnoczi
Quoted 5 lines · Vivek Goyal
It is possible that a mount is in progress and device is being removed at the same time. Use
virtio_fs_mutexto avoid races.This also takes care of bunch of races and removes some TODO items.
Quoted 8 lines
Quoted 4 lines
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)Let's move to a per-device mutex in the future. That way a single device that fails to drain/complete requests will not hang all other virtio-fs instances. This is fine for now.
Good point. For now I updated the patch so that it applies cleanly after previous two patches changed.
Subject: virtiofs: Use
virtio_fs_mutexfor races w.r.t ->remove and mount pathIt is possible that a mount is in progress and device is being removed at the same time. Use
virtio_fs_mutexto avoid races.This also takes care of bunch of races and removes some TODO items.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)Index:
rhvgoyal-linux-fuse/fs/fuse/virtio_fs.c
===================================================================fs/fuse/virtio_fs.c +20
-10@@ -13,7 +13,9 @@#include <linux/highmem.h> #include "fuse_i.h" -/* List of virtio-fs device instances and a lock for the list */ +/* List of virtio-fs device instances and a lock for the list. Also provides +* mutual exclusion in device removal and mounting path +*/ static DEFINE_MUTEX (virtio_fs_mutex);static LIST_HEAD (virtio_fs_instances);@@ -72,17 +74,19 @@ static void release_virtio_fs_obj(structkfree (vfs); } +/* Make sure virtiofs_mutex is held */ static void virtio_fs_put (struct virtio_fs *fs) { -mutex_lock (&virtio_fs_mutex);kref_put (&fs->refcount, release_virtio_fs_obj); -mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_fiq_release (struct fuse_iqueue *fiq) {struct virtio_fs *vfs = fiq->priv; + +mutex_lock (&virtio_fs_mutex);virtio_fs_put (vfs); +mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq)@@ -596,9 +600,8 @@ static void virtio_fs_remove(struct virtstruct virtio_fs *fs = vdev->priv;mutex_lock (&virtio_fs_mutex); +/* This device is going away. No one should get new reference */ list_del_init (&fs->list); -mutex_unlock (&virtio_fs_mutex); -virtio_fs_stop_all_queues (fs);virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);@@ -607,6 +610,7 @@ static void virtio_fs_remove(struct virt vdev->priv =NULL ;/* Put device reference on virtio_fs object */ virtio_fs_put (fs); +mutex_unlock (&virtio_fs_mutex); }#ifdef CONFIG_PM_SLEEP @@ -978,10 +982,15 @@ static int virtio_fs_fill_super(struct s .no_force_umount = true, }; -/* TODO lock */ -if (fs->vqs[VQ_REQUEST ].fud) { -pr_err ("virtio-fs: device already in use\n" ); - err = -EBUSY ; +mutex_lock (&virtio_fs_mutex); + +/* After holding mutex, make sure virtiofs device is still there. +* Though we are holding a refernce to it, drive ->remove might +* still have cleaned up virtual queues. In that case bail out. +*/ + err = -EINVAL ; +if (list_empty (&fs->list)) { +pr_info ("virtio-fs: tag <%s> not found\n" , fs->tag); goto err; }@@ -1007,7 +1016,6 @@ static int virtio_fs_fill_super(struct s fc = fs->vqs[VQ_REQUEST ].fud->fc; -/* TODO take fuse_mutex around this loop? */ for (i =0 ; i < fs->nvqs; i++) {struct virtio_fs_vq *fsvq = &fs->vqs[i];@@ -1020,6 +1028,7 @@ static int virtio_fs_fill_super(struct s/* Previous unmount will stop all queues. Start these again */ virtio_fs_start_all_queues (fs);fuse_send_init (fc, init_req); +mutex_unlock (&virtio_fs_mutex);return 0 ; err_free_init_req:@@ -1027,6 +1036,7 @@ err_free_init_req: err_free_fuse_devs:virtio_fs_free_devs (fs); err: +mutex_unlock (&virtio_fs_mutex);return err; }
fs/fuse/virtio_fs.c static int virtio_fs_get_tree(struct fs_ · line 1118
Quoted 9 lines
fs/fuse/virtio_fs.c +2
-0@@ -1100,7 +1110,9 @@ static int virtio_fs_get_tree(struct fs_ fc =kzalloc (sizeof (struct fuse_conn ),GFP_KERNEL );if (!fc) { +mutex_lock (&virtio_fs_mutex);virtio_fs_put (fs); +mutex_unlock (&virtio_fs_mutex);return -ENOMEM ; }
Re: [PATCH 16/18] virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path
In reply to Vivek GoyalQuoted 133 lines · Vivek Goyal
Quoted 6 lines · Stefan Hajnoczi
Quoted 5 lines · Vivek Goyal
It is possible that a mount is in progress and device is being removed at the same time. Use
virtio_fs_mutexto avoid races.This also takes care of bunch of races and removes some TODO items.
Quoted 8 lines
Quoted 4 lines
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)Let's move to a per-device mutex in the future. That way a single device that fails to drain/complete requests will not hang all other virtio-fs instances. This is fine for now.
Good point. For now I updated the patch so that it applies cleanly after previous two patches changed.
Subject: virtiofs: Use
virtio_fs_mutexfor races w.r.t ->remove and mount pathIt is possible that a mount is in progress and device is being removed at the same time. Use
virtio_fs_mutexto avoid races.This also takes care of bunch of races and removes some TODO items.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)Index:
rhvgoyal-linux-fuse/fs/fuse/virtio_fs.c
===================================================================fs/fuse/virtio_fs.c +20
-10@@ -13,7 +13,9 @@#include <linux/highmem.h> #include "fuse_i.h" -/* List of virtio-fs device instances and a lock for the list */ +/* List of virtio-fs device instances and a lock for the list. Also provides +* mutual exclusion in device removal and mounting path +*/ static DEFINE_MUTEX (virtio_fs_mutex);static LIST_HEAD (virtio_fs_instances);@@ -72,17 +74,19 @@ static void release_virtio_fs_obj(structkfree (vfs); } +/* Make sure virtiofs_mutex is held */ static void virtio_fs_put (struct virtio_fs *fs) { -mutex_lock (&virtio_fs_mutex);kref_put (&fs->refcount, release_virtio_fs_obj); -mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_fiq_release (struct fuse_iqueue *fiq) {struct virtio_fs *vfs = fiq->priv; + +mutex_lock (&virtio_fs_mutex);virtio_fs_put (vfs); +mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq)@@ -596,9 +600,8 @@ static void virtio_fs_remove(struct virtstruct virtio_fs *fs = vdev->priv;mutex_lock (&virtio_fs_mutex); +/* This device is going away. No one should get new reference */ list_del_init (&fs->list); -mutex_unlock (&virtio_fs_mutex); -virtio_fs_stop_all_queues (fs);virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);@@ -607,6 +610,7 @@ static void virtio_fs_remove(struct virt vdev->priv =NULL ;/* Put device reference on virtio_fs object */ virtio_fs_put (fs); +mutex_unlock (&virtio_fs_mutex); }#ifdef CONFIG_PM_SLEEP @@ -978,10 +982,15 @@ static int virtio_fs_fill_super(struct s .no_force_umount = true, }; -/* TODO lock */ -if (fs->vqs[VQ_REQUEST ].fud) { -pr_err ("virtio-fs: device already in use\n" ); - err = -EBUSY ; +mutex_lock (&virtio_fs_mutex); + +/* After holding mutex, make sure virtiofs device is still there. +* Though we are holding a refernce to it, drive ->remove might +* still have cleaned up virtual queues. In that case bail out. +*/ + err = -EINVAL ; +if (list_empty (&fs->list)) { +pr_info ("virtio-fs: tag <%s> not found\n" , fs->tag); goto err; }@@ -1007,7 +1016,6 @@ static int virtio_fs_fill_super(struct s fc = fs->vqs[VQ_REQUEST ].fud->fc; -/* TODO take fuse_mutex around this loop? */ for (i =0 ; i < fs->nvqs; i++) {struct virtio_fs_vq *fsvq = &fs->vqs[i];@@ -1020,6 +1028,7 @@ static int virtio_fs_fill_super(struct s/* Previous unmount will stop all queues. Start these again */ virtio_fs_start_all_queues (fs);fuse_send_init (fc, init_req); +mutex_unlock (&virtio_fs_mutex);return 0 ; err_free_init_req:@@ -1027,6 +1036,7 @@ err_free_init_req: err_free_fuse_devs:virtio_fs_free_devs (fs); err: +mutex_unlock (&virtio_fs_mutex);return err; }
fs/fuse/virtio_fs.c static int virtio_fs_get_tree(struct fs_ · line 1118
Quoted 9 lines
fs/fuse/virtio_fs.c +2
-0@@ -1100,7 +1110,9 @@ static int virtio_fs_get_tree(struct fs_ fc =kzalloc (sizeof (struct fuse_conn ),GFP_KERNEL );if (!fc) { +mutex_lock (&virtio_fs_mutex);virtio_fs_put (fs); +mutex_unlock (&virtio_fs_mutex);return -ENOMEM ; }
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerRe: [PATCH 16/18] virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path
In reply to Stefan HajnocziQuoted 6 lines · Stefan Hajnoczi
Quoted 5 lines · Vivek Goyal
It is possible that a mount is in progress and device is being removed at the same time. Use
virtio_fs_mutexto avoid races.This also takes care of bunch of races and removes some TODO items.
Quoted 8 lines
Quoted 4 lines
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)Let's move to a per-device mutex in the future. That way a single device that fails to drain/complete requests will not hang all other virtio-fs instances. This is fine for now.
Good point. For now I updated the patch so that it applies cleanly after previous two patches changed.
Subject: virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path
It is possible that a mount is in progress and device is being removed at the same time. Use virtio_fs_mutex to avoid races.
This also takes care of bunch of races and removes some TODO items.
--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)
fs/fuse/virtio_fs.c +22 -10
@@ -13,7 +13,9 @@#include <linux/highmem.h> #include "fuse_i.h" -/* List of virtio-fs device instances and a lock for the list */ +/* List of virtio-fs device instances and a lock for the list. Also provides +* mutual exclusion in device removal and mounting path +*/ static DEFINE_MUTEX (virtio_fs_mutex);static LIST_HEAD (virtio_fs_instances);
@@ -72,17 +74,19 @@ static void release_virtio_fs_obj(structkfree (vfs); } +/* Make sure virtiofs_mutex is held */ static void virtio_fs_put (struct virtio_fs *fs) { -mutex_lock (&virtio_fs_mutex);kref_put (&fs->refcount, release_virtio_fs_obj); -mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_fiq_release (struct fuse_iqueue *fiq) {struct virtio_fs *vfs = fiq->priv; + +mutex_lock (&virtio_fs_mutex);virtio_fs_put (vfs); +mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq)
@@ -596,9 +600,8 @@ static void virtio_fs_remove(struct virtstruct virtio_fs *fs = vdev->priv;mutex_lock (&virtio_fs_mutex); +/* This device is going away. No one should get new reference */ list_del_init (&fs->list); -mutex_unlock (&virtio_fs_mutex); -virtio_fs_stop_all_queues (fs);virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);
@@ -607,6 +610,7 @@ static void virtio_fs_remove(struct virt vdev->priv =NULL ;/* Put device reference on virtio_fs object */ virtio_fs_put (fs); +mutex_unlock (&virtio_fs_mutex); }#ifdef CONFIG_PM_SLEEP
@@ -978,10 +982,15 @@ static int virtio_fs_fill_super(struct s .no_force_umount = true, }; -/* TODO lock */ -if (fs->vqs[VQ_REQUEST ].fud) { -pr_err ("virtio-fs: device already in use\n" ); - err = -EBUSY ; +mutex_lock (&virtio_fs_mutex); + +/* After holding mutex, make sure virtiofs device is still there. +* Though we are holding a refernce to it, drive ->remove might +* still have cleaned up virtual queues. In that case bail out. +*/ + err = -EINVAL ; +if (list_empty (&fs->list)) { +pr_info ("virtio-fs: tag <%s> not found\n" , fs->tag); goto err; }
@@ -1007,7 +1016,6 @@ static int virtio_fs_fill_super(struct s fc = fs->vqs[VQ_REQUEST ].fud->fc; -/* TODO take fuse_mutex around this loop? */ for (i =0 ; i < fs->nvqs; i++) {struct virtio_fs_vq *fsvq = &fs->vqs[i];
@@ -1020,6 +1028,7 @@ static int virtio_fs_fill_super(struct s/* Previous unmount will stop all queues. Start these again */ virtio_fs_start_all_queues (fs);fuse_send_init (fc, init_req); +mutex_unlock (&virtio_fs_mutex);return 0 ; err_free_init_req:
@@ -1027,6 +1036,7 @@ err_free_init_req: err_free_fuse_devs:virtio_fs_free_devs (fs); err: +mutex_unlock (&virtio_fs_mutex);return err; }
@@ -1100,7 +1110,9 @@ static int virtio_fs_get_tree(struct fs_ fc =kzalloc (sizeof (struct fuse_conn ),GFP_KERNEL );if (!fc) { +mutex_lock (&virtio_fs_mutex);virtio_fs_put (fs); +mutex_unlock (&virtio_fs_mutex);return -ENOMEM ; }
Re: [PATCH 16/18] virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path
In reply to Vivek GoyalQuoted 9 lines · Vivek Goyal
It is possible that a mount is in progress and device is being removed at the same time. Use
virtio_fs_mutexto avoid races.This also takes care of bunch of races and removes some TODO items.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)
Let's move to a per-device mutex in the future. That way a single device that fails to drain/complete requests will not hang all other virtio-fs instances. This is fine for now.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Virtio-fs] [PATCH 16/18] virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path
In reply to Vivek GoyalQuoted 26 lines · Vivek Goyal
It is possible that a mount is in progress and device is being removed at the same time. Use
virtio_fs_mutexto avoid races.This also takes care of bunch of races and removes some TODO items.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)fs/fuse/virtio_fs.c +3
-1@@ -13,7 +13,9 @@#include <linux/highmem.h> #include "fuse_i.h" -/* List of virtio-fs device instances and a lock for the list */ +/* List of virtio-fs device instances and a lock for the list. Also provides +* mutual exclusion in device removal and mounting path +*/ static DEFINE_MUTEX (virtio_fs_mutex);static LIST_HEAD (virtio_fs_instances);
fs/fuse/virtio_fs.c static void release_virtiofs_obj(struct kref *ref) · line 77
Quoted 4 lines
fs/fuse/virtio_fs.c +1
-0@@ -72,17 +74,19 @@ static void release_virtiofs_obj(struct kref *ref)kfree (vfs); } +/* Make sure virtiofs_mutex is held */
Typo? virtiofs_mutex->virtio_fs_mutex
Jun
Quoted 91 lines
fs/fuse/virtio_fs.c +18
-9static void virtiofs_put (struct virtio_fs *fs) { -mutex_lock (&virtio_fs_mutex);kref_put (&fs->refcount, release_virtiofs_obj); -mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_put (struct fuse_iqueue *fiq) {struct virtio_fs *vfs = fiq->priv; + +mutex_lock (&virtio_fs_mutex);virtiofs_put (vfs); +mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq)@@ -596,9 +600,8 @@ static void virtio_fs_remove(struct virtio_device *vdev)struct virtio_fs *fs = vdev->priv;mutex_lock (&virtio_fs_mutex); +/* This device is going away. No one should get new reference */ list_del_init (&fs->list); -mutex_unlock (&virtio_fs_mutex); -virtio_fs_stop_all_queues (fs);virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);@@ -607,6 +610,7 @@ static void virtio_fs_remove(struct virtio_device *vdev) vdev->priv =NULL ;/* Put device reference on virtio_fs object */ virtiofs_put (fs); +mutex_unlock (&virtio_fs_mutex); }#ifdef CONFIG_PM_SLEEP @@ -978,10 +982,15 @@ static int virtio_fs_fill_super(struct super_block *sb) .no_force_umount = true, }; -/* TODO lock */ -if (fs->vqs[VQ_REQUEST ].fud) { -pr_err ("virtio-fs: device already in use\n" ); - err = -EBUSY ; +mutex_lock (&virtio_fs_mutex); + +/* After holding mutex, make sure virtiofs device is still there. +* Though we are holding a refernce to it, drive ->remove might +* still have cleaned up virtual queues. In that case bail out. +*/ + err = -EINVAL ; +if (list_empty (&fs->list)) { +pr_info ("virtio-fs: tag <%s> not found\n" , fs->tag); goto err; }@@ -1007,7 +1016,6 @@ static int virtio_fs_fill_super(struct super_block *sb) fc = fs->vqs[VQ_REQUEST ].fud->fc; -/* TODO take fuse_mutex around this loop? */ for (i =0 ; i < fs->nvqs; i++) {struct virtio_fs_vq *fsvq = &fs->vqs[i];@@ -1020,6 +1028,7 @@ static int virtio_fs_fill_super(struct super_block *sb)/* Previous unmount will stop all queues. Start these again */ virtio_fs_start_all_queues (fs);fuse_send_init (fc, init_req); +mutex_unlock (&virtio_fs_mutex);return 0 ; err_free_init_req:@@ -1027,6 +1036,7 @@ static int virtio_fs_fill_super(struct super_block *sb) err_free_fuse_devs:virtio_fs_free_devs (fs); err: +mutex_unlock (&virtio_fs_mutex);return err; }@@ -1100,7 +1110,9 @@ static int virtio_fs_get_tree(struct fs_context *fsc) fc =kzalloc (sizeof (struct fuse_conn ),GFP_KERNEL );if (!fc) { +mutex_lock (&virtio_fs_mutex);virtiofs_put (fs); +mutex_unlock (&virtio_fs_mutex);return -ENOMEM ; }
Vivek Goyal
maintainerPATCH 16/18 virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path
In reply to Vivek GoyalIt is possible that a mount is in progress and device is being removed at the same time. Use virtio_fs_mutex to avoid races.
This also takes care of bunch of races and removes some TODO items.
--- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)
fs/fuse/virtio_fs.c +22 -10
@@ -13,7 +13,9 @@#include <linux/highmem.h> #include "fuse_i.h" -/* List of virtio-fs device instances and a lock for the list */ +/* List of virtio-fs device instances and a lock for the list. Also provides +* mutual exclusion in device removal and mounting path +*/ static DEFINE_MUTEX (virtio_fs_mutex);static LIST_HEAD (virtio_fs_instances);
@@ -72,17 +74,19 @@ static void release_virtiofs_obj(struct kref *ref)kfree (vfs); } +/* Make sure virtiofs_mutex is held */ static void virtiofs_put (struct virtio_fs *fs) { -mutex_lock (&virtio_fs_mutex);kref_put (&fs->refcount, release_virtiofs_obj); -mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_put (struct fuse_iqueue *fiq) {struct virtio_fs *vfs = fiq->priv; + +mutex_lock (&virtio_fs_mutex);virtiofs_put (vfs); +mutex_unlock (&virtio_fs_mutex); }static void virtio_fs_drain_queue (struct virtio_fs_vq *fsvq)
@@ -596,9 +600,8 @@ static void virtio_fs_remove(struct virtio_device *vdev)struct virtio_fs *fs = vdev->priv;mutex_lock (&virtio_fs_mutex); +/* This device is going away. No one should get new reference */ list_del_init (&fs->list); -mutex_unlock (&virtio_fs_mutex); -virtio_fs_stop_all_queues (fs);virtio_fs_drain_all_queues (fs); vdev->config->reset (vdev);
@@ -607,6 +610,7 @@ static void virtio_fs_remove(struct virtio_device *vdev) vdev->priv =NULL ;/* Put device reference on virtio_fs object */ virtiofs_put (fs); +mutex_unlock (&virtio_fs_mutex); }#ifdef CONFIG_PM_SLEEP
@@ -978,10 +982,15 @@ static int virtio_fs_fill_super(struct super_block *sb) .no_force_umount = true, }; -/* TODO lock */ -if (fs->vqs[VQ_REQUEST ].fud) { -pr_err ("virtio-fs: device already in use\n" ); - err = -EBUSY ; +mutex_lock (&virtio_fs_mutex); + +/* After holding mutex, make sure virtiofs device is still there. +* Though we are holding a refernce to it, drive ->remove might +* still have cleaned up virtual queues. In that case bail out. +*/ + err = -EINVAL ; +if (list_empty (&fs->list)) { +pr_info ("virtio-fs: tag <%s> not found\n" , fs->tag); goto err; }
@@ -1007,7 +1016,6 @@ static int virtio_fs_fill_super(struct super_block *sb) fc = fs->vqs[VQ_REQUEST ].fud->fc; -/* TODO take fuse_mutex around this loop? */ for (i =0 ; i < fs->nvqs; i++) {struct virtio_fs_vq *fsvq = &fs->vqs[i];
@@ -1020,6 +1028,7 @@ static int virtio_fs_fill_super(struct super_block *sb)/* Previous unmount will stop all queues. Start these again */ virtio_fs_start_all_queues (fs);fuse_send_init (fc, init_req); +mutex_unlock (&virtio_fs_mutex);return 0 ; err_free_init_req:
@@ -1027,6 +1036,7 @@ static int virtio_fs_fill_super(struct super_block *sb) err_free_fuse_devs:virtio_fs_free_devs (fs); err: +mutex_unlock (&virtio_fs_mutex);return err; }
@@ -1100,7 +1110,9 @@ static int virtio_fs_get_tree(struct fs_context *fsc) fc =kzalloc (sizeof (struct fuse_conn ),GFP_KERNEL );if (!fc) { +mutex_lock (&virtio_fs_mutex);virtiofs_put (fs); +mutex_unlock (&virtio_fs_mutex);return -ENOMEM ; }
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 17/18 virtiofs: Remove TODO to quiesce/end_requests
In reply to Vivek GoyalWe now stop queues and drain all the pending requests from all virtqueues. So this is not a TODO anymore.
Got rid of incrementing fc->dev_count as well. It did not seem meaningful for virtio_fs.
--- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-)
fs/fuse/virtio_fs.c +0 -2
@@ -208,7 +208,6 @@ static void virtio_fs_free_devs(struct virtio_fs *fs)if (!fsvq->fud)continue ; -/* TODO need to quiesce/end_requests/decrement dev_count */ fuse_dev_free (fsvq->fud); fsvq->fud =NULL ; }
@@ -1022,7 +1021,6 @@ static int virtio_fs_fill_super(struct super_block *sb)if (i ==VQ_REQUEST )continue ;/* already initialized */ fuse_dev_install (fsvq->fud, fc); -atomic_inc (&fc->dev_count); }/* Previous unmount will stop all queues. Start these again */
Signature · 1 line
2.20.1
Re: [PATCH 17/18] virtiofs: Remove TODO to quiesce/end_requests
Quoted 10 lines · Vivek Goyal
We now stop queues and drain all the pending requests from all virtqueues. So this is not a TODO anymore.
Got rid of incrementing
fc->dev_countas well. It did not seem meaningful forvirtio_fs.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-)
Re: [PATCH 17/18] virtiofs: Remove TODO to quiesce/end_requests
In reply to Vivek GoyalQuoted 10 lines · Vivek Goyal
We now stop queues and drain all the pending requests from all virtqueues. So this is not a TODO anymore.
Got rid of incrementing
fc->dev_countas well. It did not seem meaningful forvirtio_fs.Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 17/18 virtiofs: Remove TODO to quiesce/end_requests
In reply to Vivek GoyalWe now stop queues and drain all the pending requests from all virtqueues. So this is not a TODO anymore.
Got rid of incrementing fc->dev_count as well. It did not seem meaningful for virtio_fs.
--- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-)
fs/fuse/virtio_fs.c +0 -2
@@ -208,7 +208,6 @@ static void virtio_fs_free_devs(struct virtio_fs *fs)if (!fsvq->fud)continue ; -/* TODO need to quiesce/end_requests/decrement dev_count */ fuse_dev_free (fsvq->fud); fsvq->fud =NULL ; }
@@ -1022,7 +1021,6 @@ static int virtio_fs_fill_super(struct super_block *sb)if (i ==VQ_REQUEST )continue ;/* already initialized */ fuse_dev_install (fsvq->fud, fc); -atomic_inc (&fc->dev_count); }/* Previous unmount will stop all queues. Start these again */
Signature · 1 line
2.20.1
Vivek Goyal
maintainerPATCH 18/18 virtiofs: Remove TODO item from virtio_fs_free_devs()
In reply to Vivek Goyalvirtio_fs_free_devs() is now called from ->kill_sb(). By this time all device queues have been quiesced. I am assuming that while ->kill_sb() is in progress, another mount instance will wait for it to finish (sb->s_umount mutex provides mutual exclusion).
W.r.t ->remove path, we should be fine as we are not touching vdev or virtqueues. And we have reference on virtio_fs object, so we know rest of the data structures are valid.
So I can't see the need of any additional locking yet.
--- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-)
fs/fuse/virtio_fs.c +0 -2
@@ -200,8 +200,6 @@ static void virtio_fs_free_devs(struct virtio_fs *fs) {unsigned int i; -/* TODO lock */ -for (i =0 ; i < fs->nvqs; i++) {struct virtio_fs_vq *fsvq = &fs->vqs[i];
Signature · 1 line
2.20.1
Re: [PATCH 18/18] virtiofs: Remove TODO item from virtio_fs_free_devs()
Quoted 9 lines · Vivek Goyal
virtio_fs_free_devs()is now called from ->kill_sb(). By this time all device queues have been quiesced. I am assuming that while ->kill_sb() is in progress, another mount instance will wait for it to finish (sb->s_umountmutex provides mutual exclusion).W.r.t ->remove path, we should be fine as we are not touching vdev or virtqueues. And we have reference on
virtio_fsobject, so we know rest of the data structures are valid.
Quoted 6 lines
So I can't see the need of any additional locking yet.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-)
Re: [PATCH 18/18] virtiofs: Remove TODO item from virtio_fs_free_devs()
In reply to Vivek GoyalQuoted 9 lines · Vivek Goyal
virtio_fs_free_devs()is now called from ->kill_sb(). By this time all device queues have been quiesced. I am assuming that while ->kill_sb() is in progress, another mount instance will wait for it to finish (sb->s_umountmutex provides mutual exclusion).W.r.t ->remove path, we should be fine as we are not touching vdev or virtqueues. And we have reference on
virtio_fsobject, so we know rest of the data structures are valid.
Quoted 6 lines
So I can't see the need of any additional locking yet.
Signed-off-by Vivek Goyal <[email protected]>--- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-)
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Vivek Goyal
maintainerPATCH 18/18 virtiofs: Remove TODO item from virtio_fs_free_devs()
In reply to Vivek Goyalvirtio_fs_free_devs() is now called from ->kill_sb(). By this time all device queues have been quiesced. I am assuming that while ->kill_sb() is in progress, another mount instance will wait for it to finish (sb->s_umount mutex provides mutual exclusion).
W.r.t ->remove path, we should be fine as we are not touching vdev or virtqueues. And we have reference on virtio_fs object, so we know rest of the data structures are valid.
So I can't see the need of any additional locking yet.
--- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-)
fs/fuse/virtio_fs.c +0 -2
@@ -200,8 +200,6 @@ static void virtio_fs_free_devs(struct virtio_fs *fs) {unsigned int i; -/* TODO lock */ -for (i =0 ; i < fs->nvqs; i++) {struct virtio_fs_vq *fsvq = &fs->vqs[i];
Signature · 1 line
2.20.1
Miklos Szeredi
maintainerQuoted 9 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
Quoted 6 lines
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Thanks,
Miklos
Stefan Hajnoczi
maintainerQuoted 12 lines · Miklos Szeredi
Quoted 11 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
Quoted 8 lines
Quoted 4 lines
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Stefan
Miklos Szeredi
maintainerQuoted 19 lines · Stefan Hajnoczi
Quoted 17 lines · Miklos Szeredi
Quoted 15 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Quoted 6 lines
Quoted 3 lines
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Can you give an example use case?
Thanks,
Miklos
Vivek Goyal
maintainerQuoted 20 lines · Miklos Szeredi
Quoted 19 lines · Stefan Hajnoczi
Quoted 17 lines · Miklos Szeredi
Quoted 15 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Can you give an example use case?
David Gilbert mentioned this could be useful if daemon stops responding or dies. One could remove device. That will fail all future requests and allow unmounting filesystem.
Havind said that, current implementation will help in above situation only if there are no pending requests. If there are pending requests and daemon stops responding, then removal will hang too, as we wait for draining the queues.
So at some point of time, we also need some sort of timeout functionality where we end requests with error after a timeout.
I feel we should support removing device at some point of time. But its not necessarily a must have feature for first round.
Thanks
Vivek
Miklos Szeredi
maintainerQuoted 40 lines · Vivek Goyal
Quoted 20 lines · Miklos Szeredi
Quoted 19 lines · Stefan Hajnoczi
Quoted 17 lines · Miklos Szeredi
Quoted 15 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Can you give an example use case?
David Gilbert mentioned this could be useful if daemon stops responding or dies. One could remove device. That will fail all future requests and allow unmounting filesystem.
Havind said that, current implementation will help in above situation only if there are no pending requests. If there are pending requests and daemon stops responding, then removal will hang too, as we wait for draining the queues.
Quoted 5 lines
So at some point of time, we also need some sort of timeout functionality where we end requests with error after a timeout.
I feel we should support removing device at some point of time. But its not necessarily a must have feature for first round.
If there's no compelling reason to do it in the first round, than I'd
prefer to not do it. More complexity -> more bugs.
Thanks,
Miklos
Michael S. Tsirkin
maintainerQuoted 39 lines · Vivek Goyal
Quoted 20 lines · Miklos Szeredi
Quoted 19 lines · Stefan Hajnoczi
Quoted 17 lines · Miklos Szeredi
Quoted 15 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Can you give an example use case?
David Gilbert mentioned this could be useful if daemon stops responding or dies. One could remove device. That will fail all future requests and allow unmounting filesystem.
Havind said that, current implementation will help in above situation only if there are no pending requests. If there are pending requests and daemon stops responding, then removal will hang too, as we wait for draining the queues.
Quoted 8 lines
So at some point of time, we also need some sort of timeout functionality where we end requests with error after a timeout.
I feel we should support removing device at some point of time. But its not necessarily a must have feature for first round.
Thanks
Vivek
Without removal how do we stop guest poking at some files if we want to?
I guess we could invent a special event to block accesses, but unplug will just do it.
blk and scsi support removal out of box, if this is supposed to be a drop in replacement then I think yes, you want this support.
Signature · 1 line
MST
Miklos Szeredi
maintainerQuoted 50 lines · Michael S. Tsirkin
Quoted 39 lines · Vivek Goyal
Quoted 20 lines · Miklos Szeredi
Quoted 19 lines · Stefan Hajnoczi
Quoted 17 lines · Miklos Szeredi
Quoted 15 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Can you give an example use case?
David Gilbert mentioned this could be useful if daemon stops responding or dies. One could remove device. That will fail all future requests and allow unmounting filesystem.
Havind said that, current implementation will help in above situation only if there are no pending requests. If there are pending requests and daemon stops responding, then removal will hang too, as we wait for draining the queues.
Quoted 8 lines
So at some point of time, we also need some sort of timeout functionality where we end requests with error after a timeout.
I feel we should support removing device at some point of time. But its not necessarily a must have feature for first round.
Thanks
Vivek
Quoted 8 lines
Without removal how do we stop guest poking at some files if we want to?
I guess we could invent a special event to block accesses, but unplug will just do it.
blk and scsi support removal out of box, if this is supposed to be a drop in replacement then I think yes, you want this support.
This is not a drop in replacement for blk and scsi transports. More for virtio-9p. Does that have anything similar?
If we get a request for this feature, then yes, what you are saying
makes sense. But that hasn't happened yet, so I think this can wait.
Thanks,
Miklos
Michael S. Tsirkin
maintainerQuoted 2 lines · Miklos Szeredi
This is not a drop in replacement for blk and scsi transports. More for virtio-9p. Does that have anything similar?
9p seems to supports unplug, yes. It's not great in that it blocks until we close the channel, but it's there and it does not crash or leak memory.
Michael S. Tsirkin
maintainerQuoted 2 lines · Miklos Szeredi
This is not a drop in replacement for blk and scsi transports. More for virtio-9p. Does that have anything similar?
9p seems to supports unplug, yes. It's not great in that it blocks until we close the channel, but it's there and it does not crash or leak memory.
Vivek Goyal
maintainerQuoted 20 lines · Miklos Szeredi
Quoted 19 lines · Stefan Hajnoczi
Quoted 17 lines · Miklos Szeredi
Quoted 15 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Can you give an example use case?
David Gilbert mentioned this could be useful if daemon stops responding or dies. One could remove device. That will fail all future requests and allow unmounting filesystem.
Havind said that, current implementation will help in above situation only if there are no pending requests. If there are pending requests and daemon stops responding, then removal will hang too, as we wait for draining the queues.
So at some point of time, we also need some sort of timeout functionality where we end requests with error after a timeout.
I feel we should support removing device at some point of time. But its not necessarily a must have feature for first round.
Thanks
Vivek
Quoted 20 lines · Miklos Szeredi
Quoted 19 lines · Stefan Hajnoczi
Quoted 17 lines · Miklos Szeredi
Quoted 15 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Can you give an example use case?
I think VirtFS migration need hot plugging, or it may cause QEMU crash or some problems.
Thanks,
Jun
Stefan Hajnoczi
maintainerQuoted 27 lines · piaojun
Quoted 24 lines · Miklos Szeredi
Quoted 23 lines · Stefan Hajnoczi
Quoted 20 lines · Miklos Szeredi
Quoted 9 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
Quoted 6 lines
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Can you give an example use case?
I think VirtFS migration need hot plugging, or it may cause QEMU crash or some problems.
Live migration is currently unsupported. Hot unplugging the virtio-fs device would allow the guest to live migrate successfully, so it's a useful feature to work around the missing live migration support.
Is this what you mean?
Stefan
Michael S. Tsirkin
maintainerQuoted 36 lines · Stefan Hajnoczi
Quoted 27 lines · piaojun
Quoted 24 lines · Miklos Szeredi
Quoted 23 lines · Stefan Hajnoczi
Quoted 20 lines · Miklos Szeredi
Quoted 9 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
Quoted 6 lines
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Can you give an example use case?
I think VirtFS migration need hot plugging, or it may cause QEMU crash or some problems.
Quoted 7 lines
Live migration is currently unsupported. Hot unplugging the virtio-fs device would allow the guest to live migrate successfully, so it's a useful feature to work around the missing live migration support.
Is this what you mean?
Stefan
Exactly. That's what I also said. To add to that, hotplug can not be negotiated, so it's not a feature we can easily add down the road if old guests crash on unplug. Thus a driver that does not support unplug should not claim to support removal.
Signature · 1 line
MST
Michael S. Tsirkin
maintainerQuoted 36 lines · Stefan Hajnoczi
Quoted 27 lines · piaojun
Quoted 24 lines · Miklos Szeredi
Quoted 23 lines · Stefan Hajnoczi
Quoted 20 lines · Miklos Szeredi
Quoted 9 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
Quoted 6 lines
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Can you give an example use case?
I think VirtFS migration need hot plugging, or it may cause QEMU crash or some problems.
Quoted 7 lines
Live migration is currently unsupported. Hot unplugging the virtio-fs device would allow the guest to live migrate successfully, so it's a useful feature to work around the missing live migration support.
Is this what you mean?
Stefan
Exactly. That's what I also said. To add to that, hotplug can not be negotiated, so it's not a feature we can easily add down the road if old guests crash on unplug. Thus a driver that does not support unplug should not claim to support removal.
Signature · 1 line
MST
Quoted 33 lines · Stefan Hajnoczi
Quoted 32 lines · piaojun
Quoted 28 lines · Miklos Szeredi
Quoted 19 lines · Stefan Hajnoczi
Quoted 17 lines · Miklos Szeredi
Quoted 15 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Quoted 6 lines
Quoted 3 lines
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Can you give an example use case?
Quoted 8 lines
Quoted 2 lines
I think VirtFS migration need hot plugging, or it may cause QEMU crash or some problems.
Live migration is currently unsupported. Hot unplugging the virtio-fs device would allow the guest to live migrate successfully, so it's a useful feature to work around the missing live migration support.
Is this what you mean?
Agreed, migration support is necessary for user, and hot plugging/unplugging is also common for virtio device.
Jun
Stefan Hajnoczi
maintainerQuoted 27 lines · piaojun
Quoted 24 lines · Miklos Szeredi
Quoted 23 lines · Stefan Hajnoczi
Quoted 20 lines · Miklos Szeredi
Quoted 9 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
Quoted 6 lines
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Can you give an example use case?
I think VirtFS migration need hot plugging, or it may cause QEMU crash or some problems.
Live migration is currently unsupported. Hot unplugging the virtio-fs device would allow the guest to live migrate successfully, so it's a useful feature to work around the missing live migration support.
Is this what you mean?
Stefan
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Stefan Hajnoczi
maintainerQuoted 12 lines · Miklos Szeredi
Quoted 11 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
Quoted 8 lines
Quoted 4 lines
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
Is device removal mandatory? Can't this be made a non-removable device? Is there a good reason why removing the virtio-fs device makes sense?
Hot plugging and unplugging virtio PCI adapters is common. I'd very much like removal to work from the beginning.
Stefan
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerQuoted 20 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
virsh qemu-monitor-command --hmp vm9-f28
device_delmyvirtiofsNow a mounted device can be removed and file system will return errors
-ENOTCONNand one can unmount it.
Quoted 8 lines
Miklos, if you are fine with the patches, I am fine if you fold these all into existing patch. I kept them separate so that review is easier.
Any feedback or comments are welcome.
Thanks
Vivek
Another version of a patch with fixes all rolled in would also be nice.
Quoted 28 lines
Vivek Goyal (18):
virtiofs: Remove request from processing list before calling end
virtiofs: Check whether hiprio queue is connected at submission time
virtiofs: Pass fsvq instead of vq as parameter to
virtio_fs_enqueue_req
virtiofs: Check connected state forVQ_REQUESTqueue as well Maintain count of in flight requests forVQ_REQUESTqueue
virtiofs: ->remove should not clean virtiofs fuse devices
virtiofs: Stop virtiofs queues when device is being removed
virtiofs: Drain all pending requests during ->remove time
virtiofs: Add an helper to start all the queues
virtiofs: Do not use device managed mem forvirtio_fsandvirtio_fs_vq
virtiofs: stop and drain queues after sending DESTROY
virtiofs: Usevirtio_fs_free_devs()in error path
virtiofs: Do not access virtqueue in request submission path
virtiofs: Add afuse_iqueueoperation toput()reference
virtiofs: Makevirtio_fsobject refcounted
virtiofs: Usevirtio_fs_mutexfor races w.r.t ->remove and mount path
virtiofs: Remove TODO to quiesce/end_requests
virtiofs: Remove TODO item fromvirtio_fs_free_devs()fs/fuse/fuse_i.h | 5 + fs/fuse/inode.c | 6 + fs/fuse/virtio_fs.c | 259 ++++++++++++++++++++++++++++++++------------ 3 files changed, 198 insertions(+), 72 deletions(-)Signature · 1 line
2.20.1
Michael S. Tsirkin
maintainerQuoted 20 lines · Vivek Goyal
Hi,
Michael Tsirkin pointed out issues w.r.t various locking related TODO items and races w.r.t device removal.
In this first round of cleanups, I have taken care of most pressing issues.
These patches apply on top of following.
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git#virtiofs-v4I have tested these patches with mount/umount and device removal using qemu monitor. For example.
virsh qemu-monitor-command --hmp vm9-f28
device_delmyvirtiofsNow a mounted device can be removed and file system will return errors
-ENOTCONNand one can unmount it.
Quoted 8 lines
Miklos, if you are fine with the patches, I am fine if you fold these all into existing patch. I kept them separate so that review is easier.
Any feedback or comments are welcome.
Thanks
Vivek
Another version of a patch with fixes all rolled in would also be nice.
Quoted 28 lines
Vivek Goyal (18):
virtiofs: Remove request from processing list before calling end
virtiofs: Check whether hiprio queue is connected at submission time
virtiofs: Pass fsvq instead of vq as parameter to
virtio_fs_enqueue_req
virtiofs: Check connected state forVQ_REQUESTqueue as well Maintain count of in flight requests forVQ_REQUESTqueue
virtiofs: ->remove should not clean virtiofs fuse devices
virtiofs: Stop virtiofs queues when device is being removed
virtiofs: Drain all pending requests during ->remove time
virtiofs: Add an helper to start all the queues
virtiofs: Do not use device managed mem forvirtio_fsandvirtio_fs_vq
virtiofs: stop and drain queues after sending DESTROY
virtiofs: Usevirtio_fs_free_devs()in error path
virtiofs: Do not access virtqueue in request submission path
virtiofs: Add afuse_iqueueoperation toput()reference
virtiofs: Makevirtio_fsobject refcounted
virtiofs: Usevirtio_fs_mutexfor races w.r.t ->remove and mount path
virtiofs: Remove TODO to quiesce/end_requests
virtiofs: Remove TODO item fromvirtio_fs_free_devs()fs/fuse/fuse_i.h | 5 + fs/fuse/inode.c | 6 + fs/fuse/virtio_fs.c | 259 ++++++++++++++++++++++++++++++++------------ 3 files changed, 198 insertions(+), 72 deletions(-)Signature · 1 line
2.20.1