hv_utils: Add the support of hibernation
Subsystems (2)
Dexuan Cui
maintainerv3 cover hv_utils: Add the support of hibernation
Hi,
This is an updated version of the v2 patchset
(https://lkml.org/lkml/2020/1/13/42).
I addressed all the comments from Michael, and documented the changes after the "---" line in every patch's changelog.
Please review.
Thanks!
Dexuan Cui (4):
Tools: hv: Reopen the devices if read() or write() returns errors
hv_utils: Support host-initiated restart request
hv_utils: Support host-initiated hibernation request
hv_utils: Add the support of hibernation
drivers/hv/hv_fcopy.c | 54 +++++++++++++- drivers/hv/hv_kvp.c | 43 ++++++++++- drivers/hv/hv_snapshot.c | 55 +++++++++++++- drivers/hv/hv_util.c | 144 ++++++++++++++++++++++++++++++++++++- drivers/hv/hyperv_vmbus.h | 6 ++ include/linux/hyperv.h | 2 + tools/hv/hv_fcopy_daemon.c | 33 +++++++-- tools/hv/hv_kvp_daemon.c | 36 ++++++---- tools/hv/hv_vss_daemon.c | 49 ++++++++++--- 9 files changed, 385 insertions(+), 37 deletions(-)
Signature · 1 line
2.19.1
Dexuan Cui
maintainerPATCH v3 1/4 Tools: hv: Reopen the devices if read() or write() returns errors
The state machine in the hv_utils driver can run out of order in some corner cases, e.g. if the kvp daemon doesn't call write() fast enough due to some reason, kvp_timeout_func() can run first and move the state to HVUTIL_READY; next, when kvp_on_msg() is called it returns -EINVAL since kvp_transaction.state is smaller than HVUTIL_USERSPACE_REQ; later, the daemon's write() gets an error -EINVAL, and the daemon will exit().
We can reproduce the issue by sending a SIGSTOP signal to the daemon, wait for 1 minute, and send a SIGCONT signal to the daemon: the daemon will exit() quickly.
We can fix the issue by forcing a reset of the device (which means the daemon can close() and open() the device again) and doing extra necessary clean-up.
---
Changes in v2:
This is actually a new patch that makes the daemons more robust.
Changes in v3 (I addressed Michael's comments):
Don't reset target_fd, since that's unnecessary. Reset target_fname by: target_fname[0] = '\0'; Added the missing "fs_frozen = true;" in vss_operate(). After reopen_vss_fd: if vss_operate(VSS_OP_THAW) can not clear
fs_frozen due to an error, we just exit().
Added comments.
tools/hv/hv_fcopy_daemon.c | 33 +++++++++++++++++++++---- tools/hv/hv_kvp_daemon.c | 36 ++++++++++++++++------------ tools/hv/hv_vss_daemon.c | 49 +++++++++++++++++++++++++++++--------- 3 files changed, 88 insertions(+), 30 deletions(-)
tools/hv/hv_fcopy_daemon.c +29 -4
@@ -80,6 +80,8 @@ static int hv_start_fcopy(struct hv_start_fcopy *smsg) error =0 ; done: +if (error) + target_fname[0 ] ='\0' ;return error; }
@@ -108,15 +110,29 @@ static int hv_copy_data(struct hv_do_fcopy *cpmsg)return ret; } +/* +* Reset target_fname to "" in the two below functions for hibernation: if +* the fcopy operation is aborted by hibernation, the daemon should remove the +* partially-copied file; to achieve this, the hv_utils driver always fakes a +* CANCEL_FCOPY message upon suspend, and later when the VM resumes back, +* the daemon calls hv_copy_cancel() to remove the file; if a file is copied +* successfully before suspend, hv_copy_finished() must reset target_fname to +* avoid that the file can be incorrectly removed upon resume, since the faked +* CANCEL_FCOPY message is spurious in this case. +*/ static int hv_copy_finished (void ) {close (target_fd); + target_fname[0 ] ='\0' ;return 0 ; }static int hv_copy_cancel (void ) {close (target_fd); -unlink (target_fname); +if (strlen (target_fname) >0 ) { +unlink (target_fname); + target_fname[0 ] ='\0' ; + }return 0 ; }
@@ -141,7 +157,7 @@ int main(int argc, char *argv[])struct hv_do_fcopy copy;__u32 kernel_modver; } buffer = { }; -int in_handshake =1 ; +int in_handshake;static struct option long_options[] = { {"help" , no_argument,0 ,'h' },
@@ -170,6 +186,10 @@ int main(int argc, char *argv[])openlog ("HV_FCOPY" ,0 ,LOG_USER );syslog (LOG_INFO ,"starting; pid is:%d" ,getpid ()); +reopen_fcopy_fd: +/* Remove any possible partially-copied file on error */ +hv_copy_cancel (); + in_handshake =1 ; fcopy_fd =open ("/dev/vmbus/hv_fcopy" ,O_RDWR );if (fcopy_fd <0 ) {
@@ -196,7 +216,7 @@ int main(int argc, char *argv[]) len =pread (fcopy_fd, &buffer,sizeof (buffer),0 );if (len <0 ) {syslog (LOG_ERR ,"pread failed: %s" ,strerror (errno)); -exit (EXIT_FAILURE ); + goto reopen_fcopy_fd; }if (in_handshake) {
@@ -231,9 +251,14 @@ int main(int argc, char *argv[]) } +/* +* pwrite() may return an error due to the faked CANCEL_FCOPY +* message upon hibernation. Ignore the error by resetting the +* dev file, i.e. closing and re-opening it. +*/ if (pwrite (fcopy_fd, &error,sizeof (int ),0 ) !=sizeof (int )) {syslog (LOG_ERR ,"pwrite failed: %s" ,strerror (errno)); -exit (EXIT_FAILURE ); + goto reopen_fcopy_fd; } } }
tools/hv/hv_kvp_daemon.c +21 -15
@@ -76,7 +76,7 @@ enum { DNS }; -static int in_hand_shake =1 ; +static int in_hand_shake;static char *os_name ="" ;static char *os_major ="" ;
@@ -1360,7 +1360,7 @@ void print_usage(char *argv[])int main (int argc,char *argv[]) { -int kvp_fd, len; +int kvp_fd =-1 , len;int error;struct pollfd pfd;char *p;
@@ -1400,14 +1400,6 @@ int main(int argc, char *argv[])openlog ("KVP" ,0 ,LOG_USER );syslog (LOG_INFO ,"KVP starting; pid is:%d" ,getpid ()); - kvp_fd =open ("/dev/vmbus/hv_kvp" ,O_RDWR |O_CLOEXEC ); - -if (kvp_fd <0 ) { -syslog (LOG_ERR ,"open /dev/vmbus/hv_kvp failed; error: %d %s" , - errno,strerror (errno)); -exit (EXIT_FAILURE ); - } -/* * Retrieve OS release information. */
@@ -1423,6 +1415,18 @@ int main(int argc, char *argv[])exit (EXIT_FAILURE ); } +reopen_kvp_fd: +if (kvp_fd !=-1 ) +close (kvp_fd); + in_hand_shake =1 ; + kvp_fd =open ("/dev/vmbus/hv_kvp" ,O_RDWR |O_CLOEXEC ); + +if (kvp_fd <0 ) { +syslog (LOG_ERR ,"open /dev/vmbus/hv_kvp failed; error: %d %s" , + errno,strerror (errno)); +exit (EXIT_FAILURE ); + } +/* * Register ourselves with the kernel. */
@@ -1456,9 +1460,7 @@ int main(int argc, char *argv[])if (len !=sizeof (struct hv_kvp_msg )) {syslog (LOG_ERR ,"read failed; error:%d %s" , errno,strerror (errno)); - -close (kvp_fd); -return EXIT_FAILURE ; + goto reopen_kvp_fd; }/*
@@ -1617,13 +1619,17 @@ int main(int argc, char *argv[])break ; } -/* Send the value back to the kernel. */ +/* +* Send the value back to the kernel. Note: the write() may +* return an error due to hibernation; we can ignore the error +* by resetting the dev file, i.e. closing and re-opening it. +*/ kvp_done: len =write (kvp_fd, hv_msg,sizeof (struct hv_kvp_msg ));if (len !=sizeof (struct hv_kvp_msg )) {syslog (LOG_ERR ,"write failed; error: %d %s" , errno,strerror (errno)); -exit (EXIT_FAILURE ); + goto reopen_kvp_fd; } }
tools/hv/hv_vss_daemon.c +38 -11
109 lines not on this page · raw message
Signature · 1 line
2.19.1
RE: [PATCH v3 1/4] Tools: hv: Reopen the devices if read() or write() returns errors
From: Dexuan Cui <[email protected]> Sent: Saturday, January 25, 2020 11:54 AM
Quoted 89 lines
The state machine in the
hv_utilsdriver can run out of order in some corner cases, e.g. if the kvp daemon doesn't callwrite()fast enough due to some reason,kvp_timeout_func()can run first and move the state toHVUTIL_READY; next, whenkvp_on_msg()is called it returns-EINVALsince kvp_transaction.state is smaller thanHVUTIL_USERSPACE_REQ; later, the daemon'swrite()gets an error-EINVAL, and the daemon willexit().We can reproduce the issue by sending a SIGSTOP signal to the daemon, wait for 1 minute, and send a SIGCONT signal to the daemon: the daemon will
exit()quickly.We can fix the issue by forcing a reset of the device (which means the daemon can
close()andopen()the device again) and doing extra necessary clean-up.Signed-off-by Dexuan Cui <[email protected]>---
Changes in v2:
This is actually a new patch that makes the daemons more robust.Changes in v3 (I addressed Michael's comments):
Don't resettarget_fd, since that's unnecessary. Resettarget_fnameby: target_fname[0] = '\0'; Added the missing "fs_frozen= true;" invss_operate(). Afterreopen_vss_fd: ifvss_operate(VSS_OP_THAW)can not clear
fs_frozendue to an error, we justexit().
Added comments.tools/hv/hv_fcopy_daemon.c | 33 +++++++++++++++++++++---- tools/hv/hv_kvp_daemon.c | 36 ++++++++++++++++------------ tools/hv/hv_vss_daemon.c | 49 +++++++++++++++++++++++++++++--------- 3 files changed, 88 insertions(+), 30 deletions(-)tools/hv/hv_fcopy_daemon.c +18
-2@@ -80,6 +80,8 @@ static int hv_start_fcopy(struct hv_start_fcopy *smsg) error =0 ; done: +if (error) + target_fname[0 ] ='\0' ;return error; }@@ -108,15 +110,29 @@ static int hv_copy_data(struct hv_do_fcopy *cpmsg)return ret; } +/* +* Reset target_fname to "" in the two below functions for hibernation: if +* the fcopy operation is aborted by hibernation, the daemon should remove the +* partially-copied file; to achieve this, the hv_utils driver always fakes a +* CANCEL_FCOPY message upon suspend, and later when the VM resumes back, +* the daemon calls hv_copy_cancel() to remove the file; if a file is copied +* successfully before suspend, hv_copy_finished() must reset target_fname to +* avoid that the file can be incorrectly removed upon resume, since the faked +* CANCEL_FCOPY message is spurious in this case. +*/ static int hv_copy_finished (void ) {close (target_fd); + target_fname[0 ] ='\0' ;return 0 ; }static int hv_copy_cancel (void ) {close (target_fd); -unlink (target_fname); +if (strlen (target_fname) >0 ) { +unlink (target_fname); + target_fname[0 ] ='\0' ; + }return 0 ; }@@ -141,7 +157,7 @@ int main(int argc, char *argv[])struct hv_do_fcopy copy;__u32 kernel_modver; } buffer = { }; -int in_handshake =1 ; +int in_handshake;static struct option long_options[] = { {"help" , no_argument,0 ,'h' },
tools/hv/hv_fcopy_daemon.c int main(int argc, char *argv[]) · line 191
Quoted 6 lines
tools/hv/hv_fcopy_daemon.c +3
-0@@ -170,6 +186,10 @@ int main(int argc, char *argv[])openlog ("HV_FCOPY" ,0 ,LOG_USER );syslog (LOG_INFO ,"starting; pid is:%d" ,getpid ()); +reopen_fcopy_fd: +/* Remove any possible partially-copied file on error */ +hv_copy_cancel ();
Since you have removed the calls to close(fcopy_fd) after a pread() or pwrite() failure that were in v2 of the patch, I was expecting to see
if (fcopy_fd != -1) close(fcopy_fd)
here, like you've done with the kvp and vss code. And remember to initialize fcopy_fd to -1. :-)
Michael
Quoted 29 lines
tools/hv/hv_fcopy_daemon.c +8
-2+ in_handshake =1 ; fcopy_fd =open ("/dev/vmbus/hv_fcopy" ,O_RDWR );if (fcopy_fd <0 ) {@@ -196,7 +216,7 @@ int main(int argc, char *argv[]) len =pread (fcopy_fd, &buffer,sizeof (buffer),0 );if (len <0 ) {syslog (LOG_ERR ,"pread failed: %s" ,strerror (errno)); -exit (EXIT_FAILURE ); + goto reopen_fcopy_fd; }if (in_handshake) {@@ -231,9 +251,14 @@ int main(int argc, char *argv[]) } +/* +* pwrite() may return an error due to the faked CANCEL_FCOPY +* message upon hibernation. Ignore the error by resetting the +* dev file, i.e. closing and re-opening it. +*/ if (pwrite (fcopy_fd, &error,sizeof (int ),0 ) !=sizeof (int )) {syslog (LOG_ERR ,"pwrite failed: %s" ,strerror (errno)); -exit (EXIT_FAILURE ); + goto reopen_fcopy_fd; } } }
Dexuan Cui
maintainerRE: [PATCH v3 1/4] Tools: hv: Reopen the devices if read() or write() returns errors
Quoted 10 lines
From: Michael Kelley <[email protected]>
Sent: Saturday, January 25, 2020 8:49 PMtools/hv/hv_fcopy_daemon.c int main(int argc, char *argv[]) · line 191
Quoted 3 lines
tools/hv/hv_fcopy_daemon.c +3
-0+reopen_fcopy_fd: +/* Remove any possible partially-copied file on error */ +hv_copy_cancel ();Since you have removed the calls to
close(fcopy_fd)after apread()orpwrite()failure that were in v2 of the patch, I was expecting to see
Quoted 2 lines
if (fcopy_fd != -1) close(fcopy_fd)
I missed this... Thanks for catching it!
Quoted 4 lines
here, like you've done with the kvp and vss code. And remember to initialize
fcopy_fdto -1. :-)Michael
Will do it in v4.
Thanks,
-- Dexuan
Dexuan Cui
maintainerPATCH v3 2/4 hv_utils: Support host-initiated restart request
In reply to Dexuan CuiThe hv_utils driver currently supports a "shutdown" operation initiated from the Hyper-V host. Newer versions of Hyper-V also support a "restart" operation. So add support for the updated protocol version that has "restart" support, and perform a clean reboot when such a message is received from Hyper-V.
To test the restart functionality, run this PowerShell command on the Hyper-V host:
Restart-VM <vmname> -Type Reboot
---
Changes in v2:
It's the same as v1.
Changes in v3 (I addressed Michael's comments):
Used a better version of changelog from Michael. Added a comment about the meaning of shutdown_msg->flags. Call schedule_work() at the end of the function for consistency.
drivers/hv/hv_util.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-)
drivers/hv/hv_util.c +31 -1
@@ -24,6 +24,8 @@#define SD_MAJOR 3#define SD_MINOR 0 +#define SD_MINOR_1 1 +#define SD_VERSION_3_1 (SD_MAJOR << 16 | SD_MINOR_1)#define SD_VERSION (SD_MAJOR << 16 | SD_MINOR)#define SD_MAJOR_1 1
@@ -50,8 +52,9 @@ static int sd_srv_version;static int ts_srv_version;static int hb_srv_version; -#define SD_VER_COUNT 2 +#define SD_VER_COUNT 3static const int sd_versions[] = { +SD_VERSION_3_1 ,SD_VERSION ,SD_VERSION_1 };
@@ -118,17 +121,28 @@ static void perform_shutdown(struct work_struct *dummy)orderly_poweroff (true); } +static void perform_restart (struct work_struct *dummy) +{ +orderly_reboot (); +} +/* * Perform the shutdown operation in a thread context. */ static DECLARE_WORK(shutdown_work, perform_shutdown); +/* +* Perform the restart operation in a thread context. +*/ +static DECLARE_WORK(restart_work, perform_restart); +static void shutdown_onchannelcallback (void *context) {struct vmbus_channel *channel = context;u32 recvlen;u64 requestid;bool execute_shutdown = false; +bool execute_reboot = false;u8 *shut_txf_buf = util_shutdown.recv_buffer;struct shutdown_msg_data *shutdown_msg;
@@ -157,6 +171,12 @@ static void shutdown_onchannelcallback(void *context)sizeof (struct vmbuspipe_hdr ) + sizeof(struct icmsg_hdr)]; +/* +* shutdown_msg->flags can be 0 (shut down), 2(reboot), +* or 4(hibernate). It may bitwise-OR 1, which means +* performing the request by force. Linux always tries +* to perform the request by force. +*/ switch (shutdown_msg->flags) {case 0 :case 1 :
@@ -166,6 +186,14 @@ static void shutdown_onchannelcallback(void *context)pr_info ("Shutdown request received -" " graceful shutdown initiated\n" );break ; +case 2 : +case 3 : + icmsghdrp->status =HV_S_OK ; + execute_reboot = true; + +pr_info ("Restart request received -" +" graceful restart initiated\n" ); +break ;default : icmsghdrp->status =HV_E_FAIL ; execute_shutdown = false;
@@ -186,6 +214,8 @@ static void shutdown_onchannelcallback(void *context)if (execute_shutdown == true)schedule_work (&shutdown_work); +if (execute_reboot == true) +schedule_work (&restart_work); }/*
Signature · 1 line
2.19.1
RE: [PATCH v3 2/4] hv_utils: Support host-initiated restart request
From: Dexuan Cui <[email protected]> Sent: Saturday, January 25, 2020 11:54 AM
Quoted 108 lines
The
hv_utilsdriver currently supports a "shutdown" operation initiated from the Hyper-V host. Newer versions of Hyper-V also support a "restart" operation. So add support for the updated protocol version that has "restart" support, and perform a clean reboot when such a message is received from Hyper-V.To test the restart functionality, run this PowerShell command on the Hyper-V host:
Restart-VM <vmname> -Type Reboot
Signed-off-by Dexuan Cui <[email protected]>---
Changes in v2:
It's the same as v1.Changes in v3 (I addressed Michael's comments):
Used a better version of changelog from Michael. Added a comment about the meaning ofshutdown_msg->flags. Callschedule_work()at the end of the function for consistency.drivers/hv/hv_util.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-)drivers/hv/hv_util.c +29
-1@@ -24,6 +24,8 @@#define SD_MAJOR 3#define SD_MINOR 0 +#define SD_MINOR_1 1 +#define SD_VERSION_3_1 (SD_MAJOR << 16 | SD_MINOR_1)#define SD_VERSION (SD_MAJOR << 16 | SD_MINOR)#define SD_MAJOR_1 1@@ -50,8 +52,9 @@ static int sd_srv_version;static int ts_srv_version;static int hb_srv_version; -#define SD_VER_COUNT 2 +#define SD_VER_COUNT 3static const int sd_versions[] = { +SD_VERSION_3_1 ,SD_VERSION ,SD_VERSION_1 };@@ -118,17 +121,28 @@ static void perform_shutdown(struct work_struct *dummy)orderly_poweroff (true); } +static void perform_restart (struct work_struct *dummy) +{ +orderly_reboot (); +} +/* * Perform the shutdown operation in a thread context. */ static DECLARE_WORK(shutdown_work, perform_shutdown); +/* +* Perform the restart operation in a thread context. +*/ +static DECLARE_WORK(restart_work, perform_restart); +static void shutdown_onchannelcallback (void *context) {struct vmbus_channel *channel = context;u32 recvlen;u64 requestid;bool execute_shutdown = false; +bool execute_reboot = false;u8 *shut_txf_buf = util_shutdown.recv_buffer;struct shutdown_msg_data *shutdown_msg;@@ -157,6 +171,12 @@ static void shutdown_onchannelcallback(void *context)sizeof (struct vmbuspipe_hdr ) + sizeof(struct icmsg_hdr)]; +/* +* shutdown_msg->flags can be 0 (shut down), 2(reboot), +* or 4(hibernate). It may bitwise-OR 1, which means +* performing the request by force. Linux always tries +* to perform the request by force. +*/ switch (shutdown_msg->flags) {case 0 :case 1 :@@ -166,6 +186,14 @@ static void shutdown_onchannelcallback(void *context)pr_info ("Shutdown request received -" " graceful shutdown initiated\n" );break ; +case 2 : +case 3 : + icmsghdrp->status =HV_S_OK ; + execute_reboot = true; + +pr_info ("Restart request received -" +" graceful restart initiated\n" ); +break ;default : icmsghdrp->status =HV_E_FAIL ; execute_shutdown = false;
drivers/hv/hv_util.c static void shutdown_onchannelcallback(void *context) · line 218
Quoted 5 lines
drivers/hv/hv_util.c +2
-0@@ -186,6 +214,8 @@ static void shutdown_onchannelcallback(void *context)if (execute_shutdown == true)schedule_work (&shutdown_work); +if (execute_reboot == true) +schedule_work (&restart_work);
This works, and responds to my comment. But FWIW, a more compact approach would be to drop the boolean execute_shutdown, execute_restart, etc. local variables and have just this local variable:
struct work_struct *work_to_do = NULL;
In the "case" branches do:
work_to_do = &shutdown_work;
or
work_to_do = &restart_work;
Then at the bottom of the function, just do:
if (work_to_do) schedule_work(work_to_do);
Patch 3 of this series would then be a little simpler as well.
Quoted 5 lines
drivers/hv/hv_util.c +0
-1}/* -- 2.19.1
Dexuan Cui
maintainerRE: [PATCH v3 2/4] hv_utils: Support host-initiated restart request
Quoted 21 lines
From: Michael Kelley <[email protected]>
Sent: Saturday, January 25, 2020 8:57 PMReview static void shutdown_onchannelcallback(void
Quoted 1 line
(quoted diff) +0
-0@@ -186,6 +214,8 @@ static void shutdown_onchannelcallback(void*context)
drivers/hv/hv_util.c static void shutdown_onchannelcallback(void *context) · line 218
Quoted 5 lines
drivers/hv/hv_util.c +2
-0if (execute_shutdown == true)schedule_work (&shutdown_work); +if (execute_reboot == true) +schedule_work (&restart_work);This works, and responds to my comment. But FWIW, a more compact approach would be to drop the boolean
execute_shutdown,execute_restart, etc. local variables and have just this local variable:struct work_struct *work_to_do = NULL;In the "case" branches do:
work_to_do = &shutdown_work;or
Quoted 8 lines
work_to_do = &restart_work;Then at the bottom of the function, just do:
if (work_to_do) schedule_work(work_to_do);Patch 3 of this series would then be a little simpler as well.
Good idea! I'll do this in v4.
Thanks,
-- Dexuan
Dexuan Cui
maintainerPATCH v3 3/4 hv_utils: Support host-initiated hibernation request
In reply to Dexuan CuiUpdate the Shutdown IC version to 3.2, which is required for the host to send the hibernation request.
The user is expected to create the below udev rule file, which is applied upon the host-initiated hibernation request:
root@localhost:~# cat /usr/lib/udev/rules.d/40-vm-hibernation.rules
SUBSYSTEM=="vmbus", ACTION=="change", DRIVER=="hv_utils", ENV{EVENT}=="hibernate", RUN+="/usr/bin/systemctl hibernate"---
Changes in v2:
Send the host-initiated hibernation request to the user space via udev. (v1 used call_usermodehelper() and "/sbin/hyperv-hibernate".)
Changes in v3 (I addressed Michael's comoments):
Fixed the order issue in sd_versions[].
Moved schedule_work() to a later place for consistency.
drivers/hv/hv_util.c | 52 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 51 insertions(+), 1 deletion(-)
drivers/hv/hv_util.c +51 -1
@@ -25,7 +25,9 @@#define SD_MAJOR 3#define SD_MINOR 0#define SD_MINOR_1 1 +#define SD_MINOR_2 2#define SD_VERSION_3_1 (SD_MAJOR << 16 | SD_MINOR_1) +#define SD_VERSION_3_2 (SD_MAJOR << 16 | SD_MINOR_2)#define SD_VERSION (SD_MAJOR << 16 | SD_MINOR)#define SD_MAJOR_1 1
@@ -52,8 +54,9 @@ static int sd_srv_version;static int ts_srv_version;static int hb_srv_version; -#define SD_VER_COUNT 3 +#define SD_VER_COUNT 4static const int sd_versions[] = { +SD_VERSION_3_2 ,SD_VERSION_3_1 ,SD_VERSION ,SD_VERSION_1
@@ -78,9 +81,45 @@ static const int fw_versions[] = {UTIL_WS2K8_FW_VERSION }; +/* +* Send the "hibernate" udev event in a thread context. +*/ +struct hibernate_work_context { +struct work_struct work; +struct hv_device *dev; +}; + +static struct hibernate_work_context hibernate_context; +static bool hibernation_supported; + +static void send_hibernate_uevent (struct work_struct *work) +{ +char *uevent_env[2 ] = {"EVENT=hibernate" ,NULL }; +struct hibernate_work_context *ctx; + + ctx = container_of(work, struct hibernate_work_context, work); + +kobject_uevent_env (&ctx->dev->device.kobj,KOBJ_CHANGE , uevent_env); + +pr_info ("Sent hibernation uevent\n" ); +} + +static int hv_shutdown_init (struct hv_util_service *srv) +{ +struct vmbus_channel *channel = srv->channel; + +INIT_WORK (&hibernate_context.work, send_hibernate_uevent); + hibernate_context.dev = channel->device_obj; + + hibernation_supported =hv_is_hibernation_supported (); + +return 0 ; +} +static void shutdown_onchannelcallback (void *context);static struct hv_util_service util_shutdown = { .util_cb = shutdown_onchannelcallback, + .util_init = hv_shutdown_init, };static int hv_timesync_init (struct hv_util_service *srv);
@@ -143,6 +182,7 @@ static void shutdown_onchannelcallback(void *context)u64 requestid;bool execute_shutdown = false;bool execute_reboot = false; +bool execute_hibernate = false;u8 *shut_txf_buf = util_shutdown.recv_buffer;struct shutdown_msg_data *shutdown_msg;
@@ -194,6 +234,14 @@ static void shutdown_onchannelcallback(void *context)pr_info ("Restart request received -" " graceful restart initiated\n" );break ; +case 4 : +case 5 : +pr_info ("Hibernation request received\n" ); + + icmsghdrp->status = hibernation_supported ? +HV_S_OK :HV_E_FAIL ; + execute_hibernate = hibernation_supported; +break ;default : icmsghdrp->status =HV_E_FAIL ; execute_shutdown = false;
@@ -216,6 +264,8 @@ static void shutdown_onchannelcallback(void *context)schedule_work (&shutdown_work);if (execute_reboot == true)schedule_work (&restart_work); +if (execute_hibernate == true) +schedule_work (&hibernate_context.work); }/*
Signature · 1 line
2.19.1
Dexuan Cui
maintainerPATCH v3 4/4 hv_utils: Add the support of hibernation
In reply to Dexuan CuiAdd util_pre_suspend() and util_pre_resume() for some hv_utils devices (e.g. kvp/vss/fcopy), because they need special handling before util_suspend() calls vmbus_close().
For kvp, all the possible pending work items should be cancelled.
For vss and fcopy, some extra clean-up needs to be done, i.e. fake a THAW message for hv_vss_daemon and fake a CANCEL_FCOPY message for hv_fcopy_daemon, otherwise when the VM resums back, the daemons can end up in an inconsistent state (i.e. the file systems are frozen but will never be thawed; the file transmitted via fcopy may not be complete). Note: there is an extra patch for the daemons:
"Tools: hv: Reopen the devices if read() or write() returns errors", because the hv_utils driver can not guarantee the whole transaction finishes completely once util_suspend() starts to run (at this time, all the userspace processes are frozen).
util_probe() disables channel->callback_event to avoid the race with the channel callback.
---
Changes in v2:
Handles fcopy/vss specially to avoid possible inconsistent states.
Changes in v3 (I addressed Michael's comments):
Removed unneeded blank lines.
Simplified the error handling logic by allocating memory earlier. Added a comment before util_suspend(): when we're in the function,
all the userspace processes have been frozen.
drivers/hv/hv_fcopy.c | 54 +++++++++++++++++++++++++++++++++- drivers/hv/hv_kvp.c | 43 +++++++++++++++++++++++++-- drivers/hv/hv_snapshot.c | 55 ++++++++++++++++++++++++++++++++-- drivers/hv/hv_util.c | 62 ++++++++++++++++++++++++++++++++++++++- drivers/hv/hyperv_vmbus.h | 6 ++++ include/linux/hyperv.h | 2 ++ 6 files changed, 216 insertions(+), 6 deletions(-)
drivers/hv/hv_fcopy.c +53 -1
@@ -346,9 +346,61 @@ int hv_fcopy_init(struct hv_util_service *srv)return 0 ; } +static void hv_fcopy_cancel_work (void ) +{ +cancel_delayed_work_sync (&fcopy_timeout_work); +cancel_work_sync (&fcopy_send_work); +} + +int hv_fcopy_pre_suspend (void ) +{ +struct vmbus_channel *channel = fcopy_transaction.recv_channel; +struct hv_fcopy_hdr *fcopy_msg; + +/* +* Fake a CANCEL_FCOPY message for the user space daemon in case the +* daemon is in the middle of copying some file. It doesn't matter if +* there is already a message pending to be delivered to the user +* space since we force fcopy_transaction.state to be HVUTIL_READY, so +* the user space daemon's write() will fail with EINVAL (see +* fcopy_on_msg()), and the daemon will reset the device by closing +* and re-opening it. +*/ + fcopy_msg =kzalloc (sizeof (*fcopy_msg),GFP_KERNEL ); +if (!fcopy_msg) +return -ENOMEM ; + +tasklet_disable (&channel->callback_event); + + fcopy_msg->operation =CANCEL_FCOPY ; + +hv_fcopy_cancel_work (); + +/* We don't care about the return value. */ +hvutil_transport_send (hvt, fcopy_msg,sizeof (*fcopy_msg),NULL ); + +kfree (fcopy_msg); + + fcopy_transaction.state =HVUTIL_READY ; + +/* tasklet_enable() will be called in hv_fcopy_pre_resume(). */ +return 0 ; +} + +int hv_fcopy_pre_resume (void ) +{ +struct vmbus_channel *channel = fcopy_transaction.recv_channel; + +tasklet_enable (&channel->callback_event); + +return 0 ; +} +void hv_fcopy_deinit (void ) { fcopy_transaction.state =HVUTIL_DEVICE_DYING ; -cancel_delayed_work_sync (&fcopy_timeout_work); + +hv_fcopy_cancel_work (); +hvutil_transport_destroy (hvt); }
drivers/hv/hv_kvp.c +41 -2
52 lines not on this page · raw message
drivers/hv/hv_snapshot.c +53 -2
63 lines not on this page · raw message
drivers/hv/hv_util.c +61 -1
114 lines not on this page · raw message
drivers/hv/hyperv_vmbus.h +6 -0
@@ -352,14 +352,20 @@ void vmbus_on_msg_dpc(unsigned long data);int hv_kvp_init (struct hv_util_service *srv);void hv_kvp_deinit (void ); +int hv_kvp_pre_suspend (void ); +int hv_kvp_pre_resume (void );void hv_kvp_onchannelcallback (void *context);int hv_vss_init (struct hv_util_service *srv);void hv_vss_deinit (void ); +int hv_vss_pre_suspend (void ); +int hv_vss_pre_resume (void );void hv_vss_onchannelcallback (void *context);int hv_fcopy_init (struct hv_util_service *srv);void hv_fcopy_deinit (void ); +int hv_fcopy_pre_suspend (void ); +int hv_fcopy_pre_resume (void );void hv_fcopy_onchannelcallback (void *context);void vmbus_initiate_unload (bool crash);
include/linux/hyperv.h +2 -0
@@ -1435,6 +1435,8 @@ struct hv_util_service {void (*util_cb)(void *);int (*util_init)(struct hv_util_service *);void (*util_deinit)(void ); +int (*util_pre_suspend)(void ); +int (*util_pre_resume)(void ); };struct vmbuspipe_hdr {
Signature · 1 line
2.19.1
From: Dexuan Cui <[email protected]> Sent: Saturday, January 25, 2020 11:54 AM
Quoted 33 lines
Add
util_pre_suspend()andutil_pre_resume()for somehv_utilsdevices (e.g. kvp/vss/fcopy), because they need special handling beforeutil_suspend()callsvmbus_close().For kvp, all the possible pending work items should be cancelled.
For vss and fcopy, some extra clean-up needs to be done, i.e. fake a THAW message for
hv_vss_daemonand fake aCANCEL_FCOPYmessage forhv_fcopy_daemon, otherwise when the VM resums back, the daemons can end up in an inconsistent state (i.e. the file systems are frozen but will never be thawed; the file transmitted via fcopy may not be complete). Note: there is an extra patch for the daemons:
"Tools: hv: Reopen the devices ifread()orwrite()returns errors", because thehv_utilsdriver can not guarantee the whole transaction finishes completely onceutil_suspend()starts to run (at this time, all the userspace processes are frozen).
util_probe()disableschannel->callback_eventto avoid the race with the channel callback.Signed-off-by Dexuan Cui <[email protected]>---
Changes in v2:
Handles fcopy/vss specially to avoid possible inconsistent states.Changes in v3 (I addressed Michael's comments):
Removed unneeded blank lines.
Simplified the error handling logic by allocating memory earlier. Added a comment beforeutil_suspend(): when we're in the function,
all the userspace processes have been frozen.
Quoted 8 lines
drivers/hv/hv_fcopy.c | 54 +++++++++++++++++++++++++++++++++- drivers/hv/hv_kvp.c | 43 +++++++++++++++++++++++++-- drivers/hv/hv_snapshot.c | 55 ++++++++++++++++++++++++++++++++-- drivers/hv/hv_util.c | 62 ++++++++++++++++++++++++++++++++++++++- drivers/hv/hyperv_vmbus.h | 6 ++++ include/linux/hyperv.h | 2 ++ 6 files changed, 216 insertions(+), 6 deletions(-)