block/nfs: add support for nfs_umount
PATCH 1/1 block/nfs: add support for nfs_umount
libnfs recently added support for unmounting. Add support in Qemu too.
--- block/nfs.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
block/nfs.c +4 -1
@@ -1,7 +1,7 @@/* * QEMU Block driver for native access to files on NFS shares * -* Copyright (c) 2014-2017 Peter Lieven <[email protected]> +* Copyright (c) 2014-2019 Peter Lieven <[email protected]> * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to deal
@@ -394,6 +394,9 @@ static void nfs_client_close(NFSClient *client)nfs_close (client->context, client->fh); client->fh =NULL ; } +#ifdef LIBNFS_FEATURE_UMOUNT +nfs_umount (client->context); +#endif aio_set_fd_handler (client->aio_context,nfs_get_fd (client->context), false,NULL ,NULL ,NULL ,NULL );nfs_destroy_context (client->context);
Signature · 1 line
2.17.1
Kevin Wolf
AppliedAm 03.09.2019 um 15:44 hat Peter Lieven geschrieben:
Quoted 4 lines
libnfs recently added support for unmounting. Add support in Qemu too.
Signed-off-by Peter Lieven <[email protected]>
Looks trivial enough to review even for me. :-)
Thanks, applied to the block branch.
Kevin
Quoted 8 lines
Am 03.09.2019 um 16:56 schrieb Kevin Wolf <[email protected]>:
Am 03.09.2019 um 15:44 hat Peter Lieven geschrieben:
Quoted 3 lines
libnfs recently added support for unmounting. Add support in Qemu too.
Quoted 7 lines
Quoted 1 line
Signed-off-by Peter Lieven <[email protected]>Looks trivial enough to review even for me. :-)
Thanks, applied to the block branch.
Kevin
I am not sure what the reason is, but with this patch I sometimes run into nfs_process_read being called for a cdrom mounted from nfs after I ejected it (and the whole nfs client context is already destroyed).
It seems that the following fixes the issue. It might be that the nfs_umount call just reveals this bug. nfs_close is also doing sync I/O with the libnfs library. If we mount a nfs share we are doing everything with sync calls and then set up the aio stuff with nfs_set_events. I think that we need to stop the aio before we are executing the sync calls to nfs_close and nfs_umount. Also not sure if holding the mutex is necessary here.
block/nfs.c +4 -2
@@ -389,6 +389,10 @@ static void nfs_attach_aio_context(BlockDriverState *bs,static void nfs_client_close (NFSClient *client) {if (client->context) { +qemu_mutex_lock (&client->mutex); +aio_set_fd_handler (client->aio_context,nfs_get_fd (client->context), + false,NULL ,NULL ,NULL ,NULL ); +qemu_mutex_unlock (&client->mutex);if (client->fh) {nfs_close (client->context, client->fh); client->fh =NULL ;
@@ -396,8 +400,6 @@ static void nfs_client_close(NFSClient *client)#ifdef LIBNFS_FEATURE_UMOUNT nfs_umount (client->context);#endif -aio_set_fd_handler (client->aio_context,nfs_get_fd (client->context), - false,NULL ,NULL ,NULL ,NULL );nfs_destroy_context (client->context); client->context =NULL ; }
Peter
Am 03.09.2019 um 21:52 hat Peter Lieven geschrieben:
Quoted 12 lines
Quoted 12 lines
Am 03.09.2019 um 16:56 schrieb Kevin Wolf <[email protected]>:
Am 03.09.2019 um 15:44 hat Peter Lieven geschrieben:
Quoted 4 lines
libnfs recently added support for unmounting. Add support in Qemu too.
Signed-off-by Peter Lieven <[email protected]>Looks trivial enough to review even for me. :-)
Quoted 7 lines
Quoted 3 lines
Thanks, applied to the block branch.
Kevin
I am not sure what the reason is, but with this patch I sometimes run into
nfs_process_readbeing called for a cdrom mounted from nfs after I ejected it (and the whole nfs client context is already destroyed).
Does this mean that nfs_umount() gets some response, but we don't properly wait for it? Or is some older request still in flight?
Quoted 6 lines
It seems that the following fixes the issue. It might be that the
nfs_umountcall just reveals this bug.nfs_closeis also doing sync I/O with the libnfs library. If we mount a nfs share we are doing everything with sync calls and then set up the aio stuff withnfs_set_events. I think that we need to stop the aio before we are executing the sync calls tonfs_closeandnfs_umount.
Disabling the fd handlers is probably just papering over the problem. If there is something in flight, we should wait for its completion. In theory, block devices are drained before calling .bdrv_close, so either we have a bug with draining nfs nodes correctly, or .bdrv_close initiates something new itself that we should wait for.
Quoted 1 line
Also not sure if holding the mutex is necessary here.
Not for aio_set_fd_handler(), but I don't know the requirements of nfs_get_fd().
Kevin
Quoted 24 lines
block/nfs.c +4
-2@@ -389,6 +389,10 @@ static void nfs_attach_aio_context(BlockDriverState *bs,static void nfs_client_close (NFSClient *client) {if (client->context) { +qemu_mutex_lock (&client->mutex); +aio_set_fd_handler (client->aio_context,nfs_get_fd (client->context), + false,NULL ,NULL ,NULL ,NULL ); +qemu_mutex_unlock (&client->mutex);if (client->fh) {nfs_close (client->context, client->fh); client->fh =NULL ;@@ -396,8 +400,6 @@ static void nfs_client_close(NFSClient *client)#ifdef LIBNFS_FEATURE_UMOUNT nfs_umount (client->context);#endif -aio_set_fd_handler (client->aio_context,nfs_get_fd (client->context), - false,NULL ,NULL ,NULL ,NULL );nfs_destroy_context (client->context); client->context =NULL ; }
Am 04.09.19 um 11:34 schrieb Kevin Wolf:
Quoted 11 lines
Am 03.09.2019 um 21:52 hat Peter Lieven geschrieben:
Quoted 10 lines
Quoted 10 lines
Am 03.09.2019 um 16:56 schrieb Kevin Wolf <[email protected]>:
Am 03.09.2019 um 15:44 hat Peter Lieven geschrieben:
Quoted 4 lines
libnfs recently added support for unmounting. Add support in Qemu too.
Signed-off-by Peter Lieven <[email protected]>Looks trivial enough to review even for me. :-)
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Thanks, applied to the block branch.
Kevin
I am not sure what the reason is, but with this patch I sometimes run into
nfs_process_readbeing called for a cdrom mounted from nfs after I ejected it (and the whole nfs client context is already destroyed).Does this mean that
nfs_umount()gets some response, but we don't properly wait for it? Or is some older request still in flight?
nfs_umount itself is a sync call and should only terminate when
the call is done. But there is an independent I/O handler in that
function polling on the fd. (wait_for_nfs_reply in libnfs-sync.c).
This is why I thought the right solution is to stop the Qemu I/O handler
before calling nfs_close and nfs_umount. nfs_close also uses this
sync I/O handler, but for some reason it seems not to make trouble.
The other solution would be to use the async versions of close and umount,
but that would make the code in Qemu more complex.
Peter
Quoted 31 lines · Peter Lieven
Am 04.09.19 um 11:34 schrieb Kevin Wolf:
Quoted 11 lines
Am 03.09.2019 um 21:52 hat Peter Lieven geschrieben:
Quoted 10 lines
Quoted 10 lines
Am 03.09.2019 um 16:56 schrieb Kevin Wolf <[email protected]>:
Am 03.09.2019 um 15:44 hat Peter Lieven geschrieben:
Quoted 4 lines
libnfs recently added support for unmounting. Add support in Qemu too.
Signed-off-by Peter Lieven <[email protected]>Looks trivial enough to review even for me. :-)
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Thanks, applied to the block branch.
Kevin
I am not sure what the reason is, but with this patch I sometimes run into
nfs_process_readbeing called for a cdrom mounted from nfs after I ejected it (and the whole nfs client context is already destroyed).Does this mean that
nfs_umount()gets some response, but we don't properly wait for it? Or is some older request still in flight?
nfs_umountitself is a sync call and should only terminate whenthe call is done. But there is an independent I/O handler in that
function polling on the fd. (
wait_for_nfs_replyinlibnfs-sync.c).This is why I thought the right solution is to stop the Qemu I/O handler
Quoted 10 lines
before calling
nfs_closeandnfs_umount.nfs_closealso uses thissync I/O handler, but for some reason it seems not to make trouble.
The other solution would be to use the async versions of close and umount,
but that would make the code in Qemu more complex.
NFS umount is pretty messy so I think you should continue using the sync version.
In NFSv3 (there is no mount protocol in v4) the Mount call (fetch the root filehandle)
and the Umount calls (tell server we should no longer show up in showexports -a output)
are not part of the NFS protocol but a different service running on a separate port.
This does not map well to libnfs since it is centered around a "struct nfs_context".
To use nfs_umount() from QEMU I would suggest :
1, make sure all commands in flight have finished, because you will soon disconnect from the NFS server and will never receive any in-flight responses.
2, unregister the nfs->fh filedescriptor from your eventsystem. Because the fd is about to be closed so there is great chance it will be recycled for a completely different purpose if you open any other files from qemu.
3, call nfs_umount() Internally this will close the socket to the
NFS server, then go through thr process to open a new socket to the portmapper to discover the mount server, then close that socket and reconnect a new socket again to the mount server and perform the UMNT call.
ronnie sahlberg
Quoted 3 lines
Peter
Am 05.09.19 um 12:05 schrieb ronnie sahlberg:
Quoted 60 lines · Peter Lieven
Quoted 28 lines
Am 04.09.19 um 11:34 schrieb Kevin Wolf:
Quoted 10 lines
Am 03.09.2019 um 21:52 hat Peter Lieven geschrieben:
Quoted 9 lines
Quoted 9 lines
Am 03.09.2019 um 16:56 schrieb Kevin Wolf <[email protected]>:
Am 03.09.2019 um 15:44 hat Peter Lieven geschrieben:
Quoted 4 lines
libnfs recently added support for unmounting. Add support in Qemu too.
Signed-off-by Peter Lieven <[email protected]>Looks trivial enough to review even for me. :-)
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Thanks, applied to the block branch.
Kevin
I am not sure what the reason is, but with this patch I sometimes run into
nfs_process_readbeing called for a cdrom mounted from nfs after I ejected it (and the whole nfs client context is already destroyed).Does this mean that
nfs_umount()gets some response, but we don't properly wait for it? Or is some older request still in flight?
nfs_umountitself is a sync call and should only terminate whenthe call is done. But there is an independent I/O handler in that
function polling on the fd. (
wait_for_nfs_replyinlibnfs-sync.c).This is why I thought the right solution is to stop the Qemu I/O handler
Quoted 10 lines
before calling
nfs_closeandnfs_umount.nfs_closealso uses thissync I/O handler, but for some reason it seems not to make trouble.
The other solution would be to use the async versions of close and umount,
but that would make the code in Qemu more complex.
NFS umount is pretty messy so I think you should continue using the sync version.
In NFSv3 (there is no mount protocol in v4) the Mount call (fetch the root filehandle)
and the Umount calls (tell server we should no longer show up in showexports -a output)
are not part of the NFS protocol but a different service running on a separate port.This does not map well to libnfs since it is centered around a "struct
nfs_context".To use
nfs_umount()from QEMU I would suggest :
1, make sure all commands in flight have finished, because you will soon disconnect from the NFS server and will never receive any in-flight responses.
2, unregister thenfs->fhfiledescriptor from your eventsystem. Because the fd is about to be closed so there is great chance it will be recycled for a completely different purpose if you open any other files from qemu.
Quoted 5 lines
3, call
nfs_umount()Internally this will close the socket to the
NFS server, then go through thr process to open a new socket to the portmapper to discover the mount server, then close that socket and reconnect a new socket again to the mount server and perform the UMNT call.
What we currently do in Qemu is:
1) bdrv_drain
2) bdrv_close which in the end calls nfs_client_close from block/nfs.c.
There we call:
2a) nfs_close(client->fh)
2b) aio_set_fd_handler(NULL)
2c) nfs_destroy_context(client->context);
My first patch added a nfs_umount between 2a) and 2b) so that we have
2a) nfs_close(client->fh)
2b) nfs_umount(client->context)
2c) aio_set_fd_handler(NULL)
2d) nfs_destroy_context(client->context);
This leads to triggering to assertion for an uninitialized client->mutex which is called from an invocation
of nfs_process_read after nfs_destroy_context was called.
If I change the order as following I see no more assertions:
2a) aio_set_fd_handler(NULL)
2b) nfs_close(client->fh)
2c) nfs_umount(client->context)
2d) nfs_destroy_context(client->context);
I think we should have done this in the first place, because nfs_close (and nfs_umount) poll on the nfs_fd in parallel
if we use the sync calls.
Peter
Quoted 103 lines · Peter Lieven
Am 05.09.19 um 12:05 schrieb ronnie sahlberg:
Quoted 60 lines · Peter Lieven
Quoted 28 lines
Am 04.09.19 um 11:34 schrieb Kevin Wolf:
Quoted 10 lines
Am 03.09.2019 um 21:52 hat Peter Lieven geschrieben:
Quoted 9 lines
Quoted 9 lines
Am 03.09.2019 um 16:56 schrieb Kevin Wolf <[email protected]>:
Am 03.09.2019 um 15:44 hat Peter Lieven geschrieben:
Quoted 4 lines
libnfs recently added support for unmounting. Add support in Qemu too.
Signed-off-by Peter Lieven <[email protected]>Looks trivial enough to review even for me. :-)
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Thanks, applied to the block branch.
Kevin
I am not sure what the reason is, but with this patch I sometimes run into
nfs_process_readbeing called for a cdrom mounted from nfs after I ejected it (and the whole nfs client context is already destroyed).Does this mean that
nfs_umount()gets some response, but we don't properly wait for it? Or is some older request still in flight?
nfs_umountitself is a sync call and should only terminate whenthe call is done. But there is an independent I/O handler in that
function polling on the fd. (
wait_for_nfs_replyinlibnfs-sync.c).This is why I thought the right solution is to stop the Qemu I/O handler
Quoted 10 lines
before calling
nfs_closeandnfs_umount.nfs_closealso uses thissync I/O handler, but for some reason it seems not to make trouble.
The other solution would be to use the async versions of close and umount,
but that would make the code in Qemu more complex.
NFS umount is pretty messy so I think you should continue using the sync version.
In NFSv3 (there is no mount protocol in v4) the Mount call (fetch the root filehandle)
and the Umount calls (tell server we should no longer show up in showexports -a output)
are not part of the NFS protocol but a different service running on a separate port.This does not map well to libnfs since it is centered around a "struct
nfs_context".To use
nfs_umount()from QEMU I would suggest :
1, make sure all commands in flight have finished, because you will soon disconnect from the NFS server and will never receive any in-flight responses.
2, unregister thenfs->fhfiledescriptor from your eventsystem. Because the fd is about to be closed so there is great chance it will be recycled for a completely different purpose if you open any other files from qemu.Quoted 5 lines
3, call
nfs_umount()Internally this will close the socket to the
NFS server, then go through thr process to open a new socket to the portmapper to discover the mount server, then close that socket and reconnect a new socket again to the mount server and perform the UMNT call.What we currently do in Qemu is:
1)
bdrv_drain2)
bdrv_closewhich in the end callsnfs_client_closefromblock/nfs.c.There we call: 2a) nfs_close(client->fh) 2b) aio_set_fd_handler(NULL) 2c) nfs_destroy_context(client->context);My first patch added a
nfs_umountbetween 2a) and 2b) so that we have2a) nfs_close(client->fh) 2b) nfs_umount(client->context) 2c) aio_set_fd_handler(NULL) 2d) nfs_destroy_context(client->context);This leads to triggering to assertion for an uninitialized
client->mutexwhich is called from an invocationof
nfs_process_readafternfs_destroy_contextwas called.If I change the order as following I see no more assertions:
Quoted 7 lines
2a) aio_set_fd_handler(NULL) 2b) nfs_close(client->fh) 2c) nfs_umount(client->context) 2d) nfs_destroy_context(client->context);
That makes sense and looks correct to me.
Quoted 11 lines
I think we should have done this in the first place, because
nfs_close(andnfs_umount) poll on thenfs_fdin parallelif we use the sync calls.
Peter
Am 05.09.19 um 12:28 schrieb ronnie sahlberg:
Quoted 101 lines · Peter Lieven
Quoted 100 lines
Am 05.09.19 um 12:05 schrieb ronnie sahlberg:
Quoted 59 lines · Peter Lieven
Quoted 27 lines
Am 04.09.19 um 11:34 schrieb Kevin Wolf:
Quoted 10 lines
Am 03.09.2019 um 21:52 hat Peter Lieven geschrieben:
Quoted 9 lines
Quoted 9 lines
Am 03.09.2019 um 16:56 schrieb Kevin Wolf <[email protected]>:
Am 03.09.2019 um 15:44 hat Peter Lieven geschrieben:
Quoted 4 lines
libnfs recently added support for unmounting. Add support in Qemu too.
Signed-off-by Peter Lieven <[email protected]>Looks trivial enough to review even for me. :-)
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Thanks, applied to the block branch.
Kevin
I am not sure what the reason is, but with this patch I sometimes run into
nfs_process_readbeing called for a cdrom mounted from nfs after I ejected it (and the whole nfs client context is already destroyed).Does this mean that
nfs_umount()gets some response, but we don't properly wait for it? Or is some older request still in flight?
nfs_umountitself is a sync call and should only terminate whenthe call is done. But there is an independent I/O handler in that
function polling on the fd. (
wait_for_nfs_replyinlibnfs-sync.c).This is why I thought the right solution is to stop the Qemu I/O handler
Quoted 10 lines
before calling
nfs_closeandnfs_umount.nfs_closealso uses thissync I/O handler, but for some reason it seems not to make trouble.
The other solution would be to use the async versions of close and umount,
but that would make the code in Qemu more complex.
NFS umount is pretty messy so I think you should continue using the sync version.
In NFSv3 (there is no mount protocol in v4) the Mount call (fetch the root filehandle)
and the Umount calls (tell server we should no longer show up in showexports -a output)
are not part of the NFS protocol but a different service running on a separate port.This does not map well to libnfs since it is centered around a "struct
nfs_context".To use
nfs_umount()from QEMU I would suggest :
1, make sure all commands in flight have finished, because you will soon disconnect from the NFS server and will never receive any in-flight responses.
2, unregister thenfs->fhfiledescriptor from your eventsystem. Because the fd is about to be closed so there is great chance it will be recycled for a completely different purpose if you open any other files from qemu.Quoted 5 lines
3, call
nfs_umount()Internally this will close the socket to the
NFS server, then go through thr process to open a new socket to the portmapper to discover the mount server, then close that socket and reconnect a new socket again to the mount server and perform the UMNT call.What we currently do in Qemu is:
1)
bdrv_drain2)
bdrv_closewhich in the end callsnfs_client_closefromblock/nfs.c.There we call: 2a) nfs_close(client->fh) 2b) aio_set_fd_handler(NULL) 2c) nfs_destroy_context(client->context);My first patch added a
nfs_umountbetween 2a) and 2b) so that we have2a) nfs_close(client->fh) 2b) nfs_umount(client->context) 2c) aio_set_fd_handler(NULL) 2d) nfs_destroy_context(client->context);This leads to triggering to assertion for an uninitialized
client->mutexwhich is called from an invocationof
nfs_process_readafternfs_destroy_contextwas called.If I change the order as following I see no more assertions:
Quoted 8 lines
Quoted 7 lines
2a) aio_set_fd_handler(NULL) 2b) nfs_close(client->fh) 2c) nfs_umount(client->context) 2d) nfs_destroy_context(client->context);That makes sense and looks correct to me.
I am unsure if we should acquire the client->mutex before we fiddle with
the aio_handler because in theory there could be something happen to the
nfs_fd in the background.
Best,
Peter