qemu-ga: add hook to quiesce the guest on fsfreeze-freeze/thaw
v8 cover qemu-ga: add hook to quiesce the guest on fsfreeze-freeze/thaw
Hi,
This is version 8 of the qemu-ga fsfreeze hook patchset.
*Changes from v7: ( http://patchwork.ozlabs.org/patch/204426/ )
1/2: Not changed.
2/2: Moved .gitignore exception for fsfreeze-hook.d right below "*.d".
Make fsfreeze success even if fsfreeze-hook.d doesn't exist. Use printf(1) instead of echo to output logs.
Use "$MYSQL" $MYSQL_OPTS instead of $MYSQL_CMD for safety. Improve signal handling in flush-mysql.sh.
Fix misspelling.
This patchset depends on Luiz Capitulino's patchset to improve error handling, which is available at:
https://lists.gnu.org/archive/html/qemu-devel/2012-11/msg03016.html
---
Tomoki Sekiyama (2):
qemu-ga: execute hook to quiesce the guest on fsfreeze-freeze/thaw
qemu-ga: sample fsfreeze hooks
.gitignore | 1 Makefile | 2 - qemu-ga.c | 42 ++++++++++++ qga/commands-posix.c | 69 ++++++++++++++++++++ qga/guest-agent-core.h | 1 scripts/qemu-guest-agent/fsfreeze-hook | 33 ++++++++++ .../fsfreeze-hook.d/mysql-flush.sh.sample | 56 ++++++++++++++++ 7 files changed, 202 insertions(+), 2 deletions(-) create mode 100755 scripts/qemu-guest-agent/fsfreeze-hook create mode 100755 scripts/qemu-guest-agent/fsfreeze-hook.d/mysql-flush.sh.sample
Thanks,
Signature · 3 lines
Tomoki Sekiyama <[email protected]> Linux Technology Center Hitachi, Ltd., Yokohama Research Laboratory
Re: [Qemu-devel] [PATCH v8 2/2] qemu-ga: sample fsfreeze hooks
Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
- fsfreeze-hook : execute scripts in fsfreeze-hook.d/
- fsfreeze-hook.d/mysql-flush.sh.sample : quiesce MySQL before snapshot
--- .gitignore | 1 Makefile | 2 - scripts/qemu-guest-agent/fsfreeze-hook | 33 ++++++++++++ .../fsfreeze-hook.d/mysql-flush.sh.sample | 56 ++++++++++++++++++++ 4 files changed, 91 insertions(+), 1 deletion(-) create mode 100755 scripts/qemu-guest-agent/fsfreeze-hook create mode 100755 scripts/qemu-guest-agent/fsfreeze-hook.d/mysql-flush.sh.sample
.gitignore +1 -0
@@ -68,6 +68,7 @@ fsdev/virtfs-proxy-helper.pod *.tp *.vr *.d +!scripts/qemu-guest-agent/fsfreeze-hook.d *.o *.lo *.la
Makefile +1 -1
@@ -248,7 +248,7 @@ clean: # avoid old build problems by removing potentially incorrect old files rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h rm -f qemu-options.def - find . -name '*.[od]' -exec rm -f {} + + find . -name '*.[od]' -type f -exec rm -f {} + rm -f *.a *.lo $(TOOLS) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~ rm -Rf .libs rm -f qemu-img-cmds.h
scripts/qemu-guest-agent/fsfreeze-hook new file +33 -0
@@ -0,0 +1,33 @@ +#!/bin/sh + +# This script is executed when a guest agent receives fsfreeze-freeze and +# fsfreeze-thaw command, if it is specified in --fsfreeze-hook (-F) +# option of qemu-ga or placed in default path (/etc/qemu/fsfreeze-hook). +# When the agent receives fsfreeze-freeze request, this script is issued with +# "freeze" argument before the filesystem is frozen. And for fsfreeze-thaw +# request, it is issued with "thaw" argument after filesystem is thawed. + +LOGFILE=/var/log/qga-fsfreeze-hook.log +FSFREEZE_D=$(dirname -- "$0")/fsfreeze-hook.d + +# Check whether file $1 is a backup or rpm-generated file and should be ignored +is_ignored_file() { + case "$1" in + *~ | *.bak | *.orig | *.rpmnew | *.rpmorig | *.rpmsave | *.sample) + return 0 ;; + esac + return 1 +} + +# Iterate executables in directory "fsfreeze-hook.d" with the specified args +[ ! -d "$FSFREEZE_D" ] && exit 0 +for file in "$FSFREEZE_D"/* ; do + is_ignored_file "$file" && continue + [ -x "$file" ] || continue + printf "$(date): execute $file $@\n" >>$LOGFILE + "$file" "$@" >>$LOGFILE 2>&1 + STATUS=$? + printf "$(date): $file finished with status=$STATUS\n" >>$LOGFILE +done + +exit 0
scripts/qemu-guest-agent/fsfreeze-hook.d/mysql-flush.sh.sample new file +56 -0
56 lines not on this page · raw message
Luiz Capitulino
Reviewed-byRe: [Qemu-devel] [PATCH v8 2/2] qemu-ga: sample fsfreeze hooks
Quoted 5 lines · On Wed, 12 Dec 2012 12:55:57 +0900 Tomoki Sekiyama…
Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
- fsfreeze-hook : execute scripts in fsfreeze-hook.d/
- fsfreeze-hook.d/mysql-flush.sh.sample : quiesce MySQL before snapshotSigned-off-by Tomoki Sekiyama <[email protected]>
Looks good to me:
Quoted 136 lines
--- .gitignore | 1 Makefile | 2 - scripts/qemu-guest-agent/fsfreeze-hook | 33 ++++++++++++ .../fsfreeze-hook.d/mysql-flush.sh.sample | 56 ++++++++++++++++++++ 4 files changed, 91 insertions(+), 1 deletion(-) create mode 100755 scripts/qemu-guest-agent/fsfreeze-hook create mode 100755 scripts/qemu-guest-agent/fsfreeze-hook.d/mysql-flush.sh.sample.gitignore +1
-0@@ -68,6 +68,7 @@ fsdev/virtfs-proxy-helper.pod *.tp *.vr *.d +!scripts/qemu-guest-agent/fsfreeze-hook.d *.o *.lo *.laMakefile +1
-1@@ -248,7 +248,7 @@ clean: # avoid old build problems by removing potentially incorrect old files rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h rm -f qemu-options.def - find . -name '*.[od]' -exec rm -f {} + + find . -name '*.[od]' -type f -exec rm -f {} + rm -f *.a *.lo $(TOOLS) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~ rm -Rf .libs rm -f qemu-img-cmds.hscripts/qemu-guest-agent/fsfreeze-hook new file +33
-0@@ -0,0 +1,33 @@ +#!/bin/sh + +# This script is executed when a guest agent receives fsfreeze-freeze and +# fsfreeze-thaw command, if it is specified in --fsfreeze-hook (-F) +# option of qemu-ga or placed in default path (/etc/qemu/fsfreeze-hook). +# When the agent receives fsfreeze-freeze request, this script is issued with +# "freeze" argument before the filesystem is frozen. And for fsfreeze-thaw +# request, it is issued with "thaw" argument after filesystem is thawed. + +LOGFILE=/var/log/qga-fsfreeze-hook.log +FSFREEZE_D=$(dirname -- "$0")/fsfreeze-hook.d + +# Check whether file $1 is a backup or rpm-generated file and should be ignored +is_ignored_file() { + case "$1" in + *~ | *.bak | *.orig | *.rpmnew | *.rpmorig | *.rpmsave | *.sample) + return 0 ;; + esac + return 1 +} + +# Iterate executables in directory "fsfreeze-hook.d" with the specified args +[ ! -d "$FSFREEZE_D" ] && exit 0 +for file in "$FSFREEZE_D"/* ; do + is_ignored_file "$file" && continue + [ -x "$file" ] || continue + printf "$(date): execute $file $@\n" >>$LOGFILE + "$file" "$@" >>$LOGFILE 2>&1 + STATUS=$? + printf "$(date): $file finished with status=$STATUS\n" >>$LOGFILE +done + +exit 0scripts/qemu-guest-agent/fsfreeze-hook.d/mysql-flush.sh.sample new file +56
-0@@ -0,0 +1,56 @@ +#!/bin/sh + +# Flush MySQL tables to the disk before the filesystem is frozen. +# At the same time, this keeps a read lock in order to avoid write accesses +# from the other clients until the filesystem is thawed. + +MYSQL="/usr/bin/mysql" +MYSQL_OPTS="-uroot" #"-prootpassword" +FIFO=/var/run/mysql-flush.fifo + +# Check mysql is installed and the server running +[ -x "$MYSQL" ] && "$MYSQL" $MYSQL_OPTS < /dev/null || exit 0 + +flush_and_wait() { + printf "FLUSH TABLES WITH READ LOCK \\G\n" + trap 'printf "$(date): $0 is killed\n">&2' HUP INT QUIT ALRM TERM + read < $FIFO + printf "UNLOCK TABLES \\G\n" + rm -f $FIFO +} + +case "$1" in + freeze) + mkfifo $FIFO || exit 1 + flush_and_wait | "$MYSQL" $MYSQL_OPTS & + # wait until every block is flushed + while [ "$(echo 'SHOW STATUS LIKE "Key_blocks_not_flushed"' |\ + "$MYSQL" $MYSQL_OPTS | tail -1 | cut -f 2)" -gt 0 ]; do + sleep 1 + done + # for InnoDB, wait until every log is flushed + INNODB_STATUS=$(mktemp /tmp/mysql-flush.XXXXXX) + [ $? -ne 0 ] && exit 2 + trap "rm -f $INNODB_STATUS; exit 1" HUP INT QUIT ALRM TERM + while :; do + printf "SHOW ENGINE INNODB STATUS \\G" |\ + "$MYSQL" $MYSQL_OPTS > $INNODB_STATUS + LOG_CURRENT=$(grep 'Log sequence number' $INNODB_STATUS |\ + tr -s ' ' | cut -d' ' -f4) + LOG_FLUSHED=$(grep 'Log flushed up to' $INNODB_STATUS |\ + tr -s ' ' | cut -d' ' -f5) + [ "$LOG_CURRENT" = "$LOG_FLUSHED" ] && break + sleep 1 + done + rm -f $INNODB_STATUS + ;; + + thaw) + [ ! -p $FIFO ] && exit 1 + echo > $FIFO + ;; + + *) + exit 1 + ;; +esac
Re: [Qemu-devel] [PATCH v8 1/2] qemu-ga: execute hook to quiesce the guest on fsfreeze-freeze/thaw
In reply to Tomoki SekiyamaTo use the online disk snapshot for online-backup, application-level consistency of the snapshot image is required. However, currently the guest agent can provide only filesystem-level consistency, and the snapshot may contain dirty data, for example, incomplete transactions. This patch provides the opportunity to quiesce applications before snapshot is taken.
If --fsfreeze-hook option is specified, the hook is executed with "freeze" argument before the filesystem is frozen by fsfreeze-freeze command. As for fsfreeze-thaw command, the hook is executed with "thaw" argument after the filesystem is thawed.
This patch depends on patchset to improve error reporting by Luiz Capitulino:
http://lists.gnu.org/archive/html/qemu-devel/2012-11/msg03016.html
--- qemu-ga.c | 42 +++++++++++++++++++++++++++++ qga/commands-posix.c | 69 ++++++++++++++++++++++++++++++++++++++++++++++++ qga/guest-agent-core.h | 1 + 3 files changed, 111 insertions(+), 1 deletion(-)
qemu-ga.c +41 -1
@@ -34,6 +34,12 @@#include "qga/service-win32.h" #include <windows.h> #endif +#ifdef __linux__ +#include <linux/fs.h> +#ifdef FIFREEZE +#define CONFIG_FSFREEZE +#endif +#endif #ifndef _WIN32#define QGA_VIRTIO_PATH_DEFAULT "/dev/virtio-ports/org.qemu.guest_agent.0"
@@ -42,6 +48,9 @@#endif #define QGA_STATEDIR_DEFAULT CONFIG_QEMU_LOCALSTATEDIR "/run"#define QGA_PIDFILE_DEFAULT QGA_STATEDIR_DEFAULT "/qemu-ga.pid" +#ifdef CONFIG_FSFREEZE +#define QGA_FSFREEZE_HOOK_DEFAULT CONFIG_QEMU_CONFDIR "/fsfreeze-hook" +#endif #define QGA_SENTINEL_BYTE 0xFFstruct GAState {
@@ -64,6 +73,9 @@ struct GAState {const char *log_filepath;const char *pid_filepath; } deferred_options; +#ifdef CONFIG_FSFREEZE +const char *fsfreeze_hook; +#endif };struct GAState *ga_state;
@@ -153,6 +165,10 @@ static void usage(const char *cmd) " %s)\n" " -l, --logfile set logfile path, logs to stderr by default\n" " -f, --pidfile specify pidfile (default is %s)\n" +#ifdef CONFIG_FSFREEZE +" -F, --fsfreeze-hook\n" +" enable fsfreeze hook (default is %s)\n" +#endif" -t, --statedir specify dir to store state information (absolute paths\n" " only, default is %s)\n" " -v, --verbose log extra debugging information\n"
@@ -167,6 +183,9 @@ static void usage(const char *cmd)"\n" "Report bugs to <[email protected]>\n" , cmd,QEMU_VERSION ,QGA_VIRTIO_PATH_DEFAULT ,QGA_PIDFILE_DEFAULT , +#ifdef CONFIG_FSFREEZE +QGA_FSFREEZE_HOOK_DEFAULT , +#endif QGA_STATEDIR_DEFAULT); }
@@ -401,6 +420,13 @@ void ga_unset_frozen(GAState *s) } } +#ifdef CONFIG_FSFREEZE +const char *ga_fsfreeze_hook (GAState *s) +{ +return s->fsfreeze_hook; +} +#endif +static void become_daemon (const char *pidfile) {#ifndef _WIN32
@@ -678,10 +704,13 @@ VOID WINAPI service_main(DWORD argc, TCHAR *argv[])int main (int argc,char **argv) { -const char *sopt ="hVvdm:p:l:f:b:s:t:" ; +const char *sopt ="hVvdm:p:l:f:F::b:s:t:" ;const char *method =NULL , *path =NULL ;const char *log_filepath =NULL ;const char *pid_filepath =QGA_PIDFILE_DEFAULT ; +#ifdef CONFIG_FSFREEZE +const char *fsfreeze_hook =NULL ; +#endif const char *state_dir =QGA_STATEDIR_DEFAULT ;#ifdef _WIN32const char *service =NULL ;
@@ -691,6 +720,9 @@ int main(int argc, char **argv) { "version", 0, NULL, 'V' }, { "logfile", 1, NULL, 'l' }, { "pidfile", 1, NULL, 'f' }, +#ifdef CONFIG_FSFREEZE + { "fsfreeze-hook", 2, NULL, 'F' }, +#endif { "verbose", 0, NULL, 'v' }, { "method", 1, NULL, 'm' }, { "path", 1, NULL, 'p' },
@@ -723,6 +755,11 @@ int main(int argc, char **argv)case 'f' : pid_filepath = optarg;break ; +#ifdef CONFIG_FSFREEZE +case 'F' : + fsfreeze_hook = optarg ? optarg :QGA_FSFREEZE_HOOK_DEFAULT ; +break ; +#endif case 't' : state_dir = optarg;break ;
@@ -786,6 +823,9 @@ int main(int argc, char **argv) s =g_malloc0 (sizeof (GAState)); s->log_level = log_level; s->log_file = stderr; +#ifdef CONFIG_FSFREEZE + s->fsfreeze_hook = fsfreeze_hook; +#endif g_log_set_default_handler (ga_log, s);g_log_set_fatal_mask (NULL ,G_LOG_LEVEL_ERROR );ga_enable_logging (s);
qga/commands-posix.c +69 -0
87 lines not on this page · raw message
qga/guest-agent-core.h +1 -0
@@ -34,6 +34,7 @@ void ga_set_response_delimited(GAState *s);bool ga_is_frozen (GAState *s);void ga_set_frozen (GAState *s);void ga_unset_frozen (GAState *s); +const char *ga_fsfreeze_hook (GAState *s);#ifndef _WIN32void reopen_fd_to_null (int fd);
mdroth
AppliedQuoted 3 lines · Tomoki Sekiyama
Hi,
This is version 8 of the qemu-ga fsfreeze hook patchset.
Thanks, applied to qga branch with some minor clarifications for the help output.
Quoted 35 lines
*Changes from v7: ( http://patchwork.ozlabs.org/patch/204426/ )
1/2: Not changed.
2/2: Moved .gitignore exception for fsfreeze-hook.d right below "*.d".
Make fsfreeze success even if fsfreeze-hook.d doesn't exist. Use printf(1) instead of echo to output logs.
Use "$MYSQL" $MYSQL_OPTS instead of $MYSQL_CMD for safety. Improve signal handling inflush-mysql.sh.
Fix misspelling.This patchset depends on Luiz Capitulino's patchset to improve error handling, which is available at:
https://lists.gnu.org/archive/html/qemu-devel/2012-11/msg03016.html
---
Tomoki Sekiyama (2):
qemu-ga: execute hook to quiesce the guest on fsfreeze-freeze/thaw
qemu-ga: sample fsfreeze hooks.gitignore | 1 Makefile | 2 - qemu-ga.c | 42 ++++++++++++ qga/commands-posix.c | 69 ++++++++++++++++++++ qga/guest-agent-core.h | 1 scripts/qemu-guest-agent/fsfreeze-hook | 33 ++++++++++ .../fsfreeze-hook.d/mysql-flush.sh.sample | 56 ++++++++++++++++ 7 files changed, 202 insertions(+), 2 deletions(-) create mode 100755 scripts/qemu-guest-agent/fsfreeze-hook create mode 100755 scripts/qemu-guest-agent/fsfreeze-hook.d/mysql-flush.sh.sampleThanks,
Signature · 3 lines
Tomoki Sekiyama <[email protected]> Linux Technology Center Hitachi, Ltd., Yokohama Research Laboratory