migration: fix the memory overwriting risk in add_to_iovec
PATCH 1/1 migration: fix the memory overwriting risk in add_to_iovec
From: Feng Lin <[email protected]>
When testing migration, a Segmentation fault qemu core is generated.
0 error_free (err=0x1) 1 0x00007f8b862df647 in qemu_fclose (f=f@entry=0x55e06c247640) 2 0x00007f8b8516d59a in migrate_fd_cleanup (s=s@entry=0x55e06c0e1ef0) 3 0x00007f8b8516d66c in migrate_fd_cleanup_bh (opaque=0x55e06c0e1ef0) 4 0x00007f8b8626a47f in aio_bh_poll (ctx=ctx@entry=0x55e06b5a16d0) 5 0x00007f8b8626e71f in aio_dispatch (ctx=0x55e06b5a16d0) 6 0x00007f8b8626a33d in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) 7 0x00007f8b866bdba4 in g_main_context_dispatch () 8 0x00007f8b8626cde9 in glib_pollfds_poll () 9 0x00007f8b8626ce62 in os_host_main_loop_wait (timeout=<optimized out>)
10 0x00007f8b8626cffd in main_loop_wait (nonblocking=nonblocking@entry=0) 11 0x00007f8b862ef01f in main_loop ()
Using gdb print the struct QEMUFile f = {
...,
iovcnt = 65, last_error = 21984,
last_error_obj = 0x1, shutdown = true
}
Well iovcnt is overflow, because the max size of MAX_IOV_SIZE is 64.
struct QEMUFile {
...;
struct iovec iov[MAX_IOV_SIZE];
unsigned int iovcnt;
int last_error;
Error *last_error_obj;
bool shutdown;
};iovcnt and last_error is overwrited by add_to_iovec().
Right now, add_to_iovec() increase iovcnt before check the limit. And it seems that add_to_iovec() assumes that iovcnt will set to zero in qemu_fflush(). But qemu_fflush() will directly return when f->shutdown is true.
The situation may occur when libvirtd restart during migration, after f->shutdown is set, before calling qemu_file_set_error() in qemu_file_shutdown().
So the safiest way is checking the iovcnt before increasing it.
--- migration/qemu-file.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
migration/qemu-file.c +4 -2
@@ -419,8 +419,10 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size,if (may_free) {set_bit (f->iovcnt, f->may_free); } - f->iov[f->iovcnt].iov_base = (uint8_t *)buf; - f->iov[f->iovcnt++].iov_len = size; +if (f->iovcnt <MAX_IOV_SIZE ) { + f->iov[f->iovcnt].iov_base = (uint8_t *)buf; + f->iov[f->iovcnt++].iov_len = size; + } }if (f->iovcnt >=MAX_IOV_SIZE ) {
Signature · 1 line
2.23.0
PATCH v2 1/1 migration: fix the memory overwriting risk in add_to_iovec
From: Feng Lin <[email protected]>
When testing migration, a Segmentation fault qemu core is generated.
0 error_free (err=0x1) 1 0x00007f8b862df647 in qemu_fclose (f=f@entry=0x55e06c247640) 2 0x00007f8b8516d59a in migrate_fd_cleanup (s=s@entry=0x55e06c0e1ef0) 3 0x00007f8b8516d66c in migrate_fd_cleanup_bh (opaque=0x55e06c0e1ef0) 4 0x00007f8b8626a47f in aio_bh_poll (ctx=ctx@entry=0x55e06b5a16d0) 5 0x00007f8b8626e71f in aio_dispatch (ctx=0x55e06b5a16d0) 6 0x00007f8b8626a33d in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) 7 0x00007f8b866bdba4 in g_main_context_dispatch () 8 0x00007f8b8626cde9 in glib_pollfds_poll () 9 0x00007f8b8626ce62 in os_host_main_loop_wait (timeout=<optimized out>)
10 0x00007f8b8626cffd in main_loop_wait (nonblocking=nonblocking@entry=0) 11 0x00007f8b862ef01f in main_loop ()
Using gdb print the struct QEMUFile f = {
...,
iovcnt = 65, last_error = 21984,
last_error_obj = 0x1, shutdown = true
}
Well iovcnt is overflow, because the max size of MAX_IOV_SIZE is 64.
struct QEMUFile {
...;
struct iovec iov[MAX_IOV_SIZE];
unsigned int iovcnt;
int last_error;
Error *last_error_obj;
bool shutdown;
};iovcnt and last_error is overwrited by add_to_iovec().
Right now, add_to_iovec() increase iovcnt before check the limit. And it seems that add_to_iovec() assumes that iovcnt will set to zero in qemu_fflush(). But qemu_fflush() will directly return when f->shutdown is true.
The situation may occur when libvirtd restart during migration, after f->shutdown is set, before calling qemu_file_set_error() in qemu_file_shutdown().
So the safiest way is checking the iovcnt before increasing it.
--- migration/qemu-file.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
migration/qemu-file.c +8 -5
@@ -416,6 +416,9 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, { f->iov[f->iovcnt -1 ].iov_len += size; } else { +if (f->iovcnt >=MAX_IOV_SIZE ) { + goto fflush; + }if (may_free) {set_bit (f->iovcnt, f->may_free); }
@@ -423,12 +426,12 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, f->iov[f->iovcnt++].iov_len = size; } -if (f->iovcnt >=MAX_IOV_SIZE ) { -qemu_fflush (f); -return 1 ; +if (f->iovcnt <MAX_IOV_SIZE ) { +return 0 ; } - -return 0 ; +fflush: +qemu_fflush (); +return 1 ; }static void add_buf_to_iovec (QEMUFile *f,size_t len)
Signature · 1 line
2.23.0
PATCH v3 1/1 migration: fix the memory overwriting risk in add_to_iovec
In reply to Lin FengFrom: Feng Lin <[email protected]>
When testing migration, a Segmentation fault qemu core is generated.
0 error_free (err=0x1) 1 0x00007f8b862df647 in qemu_fclose (f=f@entry=0x55e06c247640) 2 0x00007f8b8516d59a in migrate_fd_cleanup (s=s@entry=0x55e06c0e1ef0) 3 0x00007f8b8516d66c in migrate_fd_cleanup_bh (opaque=0x55e06c0e1ef0) 4 0x00007f8b8626a47f in aio_bh_poll (ctx=ctx@entry=0x55e06b5a16d0) 5 0x00007f8b8626e71f in aio_dispatch (ctx=0x55e06b5a16d0) 6 0x00007f8b8626a33d in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) 7 0x00007f8b866bdba4 in g_main_context_dispatch () 8 0x00007f8b8626cde9 in glib_pollfds_poll () 9 0x00007f8b8626ce62 in os_host_main_loop_wait (timeout=<optimized out>)
10 0x00007f8b8626cffd in main_loop_wait (nonblocking=nonblocking@entry=0) 11 0x00007f8b862ef01f in main_loop ()
Using gdb print the struct QEMUFile f = {
...,
iovcnt = 65, last_error = 21984,
last_error_obj = 0x1, shutdown = true
}
Well iovcnt is overflow, because the max size of MAX_IOV_SIZE is 64.
struct QEMUFile {
...;
struct iovec iov[MAX_IOV_SIZE];
unsigned int iovcnt;
int last_error;
Error *last_error_obj;
bool shutdown;
};iovcnt and last_error is overwrited by add_to_iovec().
Right now, add_to_iovec() increase iovcnt before check the limit. And it seems that add_to_iovec() assumes that iovcnt will set to zero in qemu_fflush(). But qemu_fflush() will directly return when f->shutdown is true.
The situation may occur when libvirtd restart during migration, after f->shutdown is set, before calling qemu_file_set_error() in qemu_file_shutdown().
So the safiest way is checking the iovcnt before increasing it.
--- migration/qemu-file.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
migration/qemu-file.c +8 -5
@@ -416,6 +416,9 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, { f->iov[f->iovcnt -1 ].iov_len += size; } else { +if (f->iovcnt >=MAX_IOV_SIZE ) { + goto fflush; + }if (may_free) {set_bit (f->iovcnt, f->may_free); }
@@ -423,12 +426,12 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, f->iov[f->iovcnt++].iov_len = size; } -if (f->iovcnt >=MAX_IOV_SIZE ) { -qemu_fflush (f); -return 1 ; +if (f->iovcnt <MAX_IOV_SIZE ) { +return 0 ; } - -return 0 ; +fflush: +qemu_fflush (f); +return 1 ; }static void add_buf_to_iovec (QEMUFile *f,size_t len)
Signature · 1 line
2.23.0
Quoted 51 lines · * Lin Feng ([email protected]) wrote:
From: Feng Lin <[email protected]>
When testing migration, a Segmentation fault qemu core is generated.
0 error_free (err=0x1) 1 0x00007f8b862df647 in qemu_fclose (f=f@entry=0x55e06c247640) 2 0x00007f8b8516d59a in migrate_fd_cleanup (s=s@entry=0x55e06c0e1ef0) 3 0x00007f8b8516d66c in migrate_fd_cleanup_bh (opaque=0x55e06c0e1ef0) 4 0x00007f8b8626a47f in aio_bh_poll (ctx=ctx@entry=0x55e06b5a16d0) 5 0x00007f8b8626e71f in aio_dispatch (ctx=0x55e06b5a16d0) 6 0x00007f8b8626a33d in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) 7 0x00007f8b866bdba4 in g_main_context_dispatch () 8 0x00007f8b8626cde9 in glib_pollfds_poll () 9 0x00007f8b8626ce62 in os_host_main_loop_wait (timeout=<optimized out>)10
0x00007f8b8626cffdinmain_loop_wait(nonblocking=nonblocking@entry=0) 110x00007f8b862ef01finmain_loop()
Using gdb print the struct QEMUFile f = {
...,
iovcnt = 65,last_error= 21984,
last_error_obj=0x1, shutdown = true
}
Well iovcnt is overflow, because the max size ofMAX_IOV_SIZEis 64.struct QEMUFile { ...; struct iovec iov[MAX_IOV_SIZE]; unsigned int iovcnt; int last_error; Error *last_error_obj; bool shutdown; };iovcnt and
last_erroris overwrited byadd_to_iovec().
Right now,add_to_iovec()increase iovcnt before check the limit. And it seems thatadd_to_iovec()assumes that iovcnt will set to zero inqemu_fflush(). Butqemu_fflush()will directly return whenf->shutdownis true.The situation may occur when libvirtd restart during migration, after
f->shutdownis set, before callingqemu_file_set_error()inqemu_file_shutdown().So the safiest way is checking the iovcnt before increasing it.
Signed-off-by Feng Lin <[email protected]>--- migration/qemu-file.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)migration/qemu-file.c +0
-0
migration/qemu-file.c static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, · line 421
Quoted 6 lines
migration/qemu-file.c +3
-0@@ -416,6 +416,9 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, { f->iov[f->iovcnt -1 ].iov_len += size; } else { +if (f->iovcnt >=MAX_IOV_SIZE ) { + goto fflush; + }
Why call qemu_fflush in this case?
If I understand what you're saying, then we only get to here if a previous qemu_fflush has failed, so this should fail as well?
How about, something like:
if (f->iovcnt >= MAX_IOV_SIZE) {
/* Should only happen if a previous fflush failed */
assert(f->shutdown || !qemu_file_is_writeable(f));
return 1;
}?
Dave
Quoted 24 lines
migration/qemu-file.c +5
-5if (may_free) {set_bit (f->iovcnt, f->may_free); }@@ -423,12 +426,12 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, f->iov[f->iovcnt++].iov_len = size; } -if (f->iovcnt >=MAX_IOV_SIZE ) { -qemu_fflush (f); -return 1 ; +if (f->iovcnt <MAX_IOV_SIZE ) { +return 0 ; } - -return 0 ; +fflush: +qemu_fflush (f); +return 1 ; }static void add_buf_to_iovec (QEMUFile *f,size_t len)Signature · 1 line
2.23.0
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Quoted 48 lines · * Dr. David Alan Gilbert(mailto:[email protected]) wrote:
Quoted 10 lines · * Lin Feng ([email protected]) wrote:
From: Feng Lin <[email protected]>
When testing migration, a Segmentation fault qemu core is generated.
0 error_free (err=0x1) 1 0x00007f8b862df647 in qemu_fclose (f=f@entry=0x55e06c247640) 2 0x00007f8b8516d59a in migrate_fd_cleanup (s=s@entry=0x55e06c0e1ef0) 3 0x00007f8b8516d66c in migrate_fd_cleanup_bh (opaque=0x55e06c0e1ef0) 4 0x00007f8b8626a47f in aio_bh_poll (ctx=ctx@entry=0x55e06b5a16d0) 5 0x00007f8b8626e71f in aio_dispatch (ctx=0x55e06b5a16d0) 6 0x00007f8b8626a33d in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>,user_data=<optimized out>)
Quoted 36 lines
7 0x00007f8b866bdba4 in g_main_context_dispatch () 8 0x00007f8b8626cde9 in glib_pollfds_poll () 9 0x00007f8b8626ce62 in os_host_main_loop_wait (timeout=<optimized out>)10
0x00007f8b8626cffdinmain_loop_wait(nonblocking=nonblocking@entry=0) 110x00007f8b862ef01finmain_loop()
Using gdb print the struct QEMUFile f = {
...,
iovcnt = 65,last_error= 21984,
last_error_obj=0x1, shutdown = true
}
Well iovcnt is overflow, because the max size ofMAX_IOV_SIZEis 64.struct QEMUFile { ...; struct iovec iov[MAX_IOV_SIZE]; unsigned int iovcnt; int last_error; Error *last_error_obj; bool shutdown; };iovcnt and
last_erroris overwrited byadd_to_iovec().
Right now,add_to_iovec()increase iovcnt before check the limit. And it seems thatadd_to_iovec()assumes that iovcnt will set to zero inqemu_fflush(). Butqemu_fflush()will directly return whenf->shutdownis true.The situation may occur when libvirtd restart during migration, after
f->shutdownis set, before callingqemu_file_set_error()inqemu_file_shutdown().So the safiest way is checking the iovcnt before increasing it.
Signed-off-by Feng Lin <[email protected]>--- migration/qemu-file.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
Quoted 15 lines
migration/qemu-file.c static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, · line 421
Quoted 11 lines
migration/qemu-file.c +3
-0@@ -416,6 +416,9 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, { f->iov[f->iovcnt -1 ].iov_len += size; } else { +if (f->iovcnt >=MAX_IOV_SIZE ) { + goto fflush; + }Why call
qemu_fflushin this case?
If I understand what you're saying, then we only get to here if a previousqemu_fflushhas failed, so this should fail as well?
Yes, that's what I mean.
Quoted 9 lines
How about, something like:
if (f->iovcnt >= MAX_IOV_SIZE) { /* Should only happen if a previous fflush failed */ assert(f->shutdown || !qemu_file_is_writeable(f)); return 1; }?
At first, I'm just thinking that overwriting requires qemu_fflush to reset iovcnt and do not consider the possibility of packet loss caused by other exceptions. It makes more sense to make an assertion here. Thank you for your suggestions.
Quoted 29 lines
Dave
Quoted 22 lines
migration/qemu-file.c +5
-6if (may_free) {set_bit (f->iovcnt, f->may_free); }@@ -423,12 +426,12 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, f->iov[f->iovcnt++].iov_len = size; } -if (f->iovcnt >=MAX_IOV_SIZE ) { -qemu_fflush (f); -return 1 ; +if (f->iovcnt <MAX_IOV_SIZE ) { +return 0 ; } - -return 0 ; +fflush: +qemu_fflush (f); +return 1 ; }static void add_buf_to_iovec (QEMUFile *f,size_t len) --Quoted 2 lines
2.23.0
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK
PATCH v4 1/1 migration: fix the memory overwriting risk in add_to_iovec
In reply to Lin FengFrom: Feng Lin <[email protected]>
When testing migration, a Segmentation fault qemu core is generated.
0 error_free (err=0x1) 1 0x00007f8b862df647 in qemu_fclose (f=f@entry=0x55e06c247640) 2 0x00007f8b8516d59a in migrate_fd_cleanup (s=s@entry=0x55e06c0e1ef0) 3 0x00007f8b8516d66c in migrate_fd_cleanup_bh (opaque=0x55e06c0e1ef0) 4 0x00007f8b8626a47f in aio_bh_poll (ctx=ctx@entry=0x55e06b5a16d0) 5 0x00007f8b8626e71f in aio_dispatch (ctx=0x55e06b5a16d0) 6 0x00007f8b8626a33d in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) 7 0x00007f8b866bdba4 in g_main_context_dispatch () 8 0x00007f8b8626cde9 in glib_pollfds_poll () 9 0x00007f8b8626ce62 in os_host_main_loop_wait (timeout=<optimized out>)
10 0x00007f8b8626cffd in main_loop_wait (nonblocking=nonblocking@entry=0) 11 0x00007f8b862ef01f in main_loop ()
Using gdb print the struct QEMUFile f = {
...,
iovcnt = 65, last_error = 21984,
last_error_obj = 0x1, shutdown = true
}
Well iovcnt is overflow, because the max size of MAX_IOV_SIZE is 64.
struct QEMUFile {
...;
struct iovec iov[MAX_IOV_SIZE];
unsigned int iovcnt;
int last_error;
Error *last_error_obj;
bool shutdown;
};iovcnt and last_error is overwrited by add_to_iovec().
Right now, add_to_iovec() increase iovcnt before check the limit. And it seems that add_to_iovec() assumes that iovcnt will set to zero in qemu_fflush(). But qemu_fflush() will directly return when f->shutdown is true.
The situation may occur when libvirtd restart during migration, after f->shutdown is set, before calling qemu_file_set_error() in qemu_file_shutdown().
So the safiest way is checking the iovcnt before increasing it.
--- migration/qemu-file.c | 5 +++++ 1 file changed, 5 insertions(+)
migration/qemu-file.c +5 -0
@@ -416,6 +416,11 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, { f->iov[f->iovcnt -1 ].iov_len += size; } else { +if (f->iovcnt >=MAX_IOV_SIZE ) { +/* Should only happen if a previous fflush failed */ +assert (f->shutdown || !qemu_file_is_writeable (f)); +return 1 ; + }if (may_free) {set_bit (f->iovcnt, f->may_free); }
Signature · 1 line
2.23.0
Quoted 35 lines · * Lin Feng ([email protected]) wrote:
From: Feng Lin <[email protected]>
When testing migration, a Segmentation fault qemu core is generated.
0 error_free (err=0x1) 1 0x00007f8b862df647 in qemu_fclose (f=f@entry=0x55e06c247640) 2 0x00007f8b8516d59a in migrate_fd_cleanup (s=s@entry=0x55e06c0e1ef0) 3 0x00007f8b8516d66c in migrate_fd_cleanup_bh (opaque=0x55e06c0e1ef0) 4 0x00007f8b8626a47f in aio_bh_poll (ctx=ctx@entry=0x55e06b5a16d0) 5 0x00007f8b8626e71f in aio_dispatch (ctx=0x55e06b5a16d0) 6 0x00007f8b8626a33d in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) 7 0x00007f8b866bdba4 in g_main_context_dispatch () 8 0x00007f8b8626cde9 in glib_pollfds_poll () 9 0x00007f8b8626ce62 in os_host_main_loop_wait (timeout=<optimized out>)10
0x00007f8b8626cffdinmain_loop_wait(nonblocking=nonblocking@entry=0) 110x00007f8b862ef01finmain_loop()
Using gdb print the struct QEMUFile f = {
...,
iovcnt = 65,last_error= 21984,
last_error_obj=0x1, shutdown = true
}
Well iovcnt is overflow, because the max size ofMAX_IOV_SIZEis 64.struct QEMUFile { ...; struct iovec iov[MAX_IOV_SIZE]; unsigned int iovcnt; int last_error; Error *last_error_obj; bool shutdown; };iovcnt and
last_erroris overwrited byadd_to_iovec().
Right now,add_to_iovec()increase iovcnt before check the limit. And it seems thatadd_to_iovec()assumes that iovcnt will set to zero inqemu_fflush(). Butqemu_fflush()will directly return whenf->shutdownis true.
Quoted 7 lines
The situation may occur when libvirtd restart during migration, after
f->shutdownis set, before callingqemu_file_set_error()inqemu_file_shutdown().So the safiest way is checking the iovcnt before increasing it.
Signed-off-by Feng Lin <[email protected]>
Quoted 23 lines
--- migration/qemu-file.c | 5 +++++ 1 file changed, 5 insertions(+)migration/qemu-file.c +5
-0@@ -416,6 +416,11 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, { f->iov[f->iovcnt -1 ].iov_len += size; } else { +if (f->iovcnt >=MAX_IOV_SIZE ) { +/* Should only happen if a previous fflush failed */ +assert (f->shutdown || !qemu_file_is_writeable (f)); +return 1 ; + }if (may_free) {set_bit (f->iovcnt, f->may_free); }Signature · 1 line
2.23.0
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Quoted 35 lines · * Lin Feng ([email protected]) wrote:
From: Feng Lin <[email protected]>
When testing migration, a Segmentation fault qemu core is generated.
0 error_free (err=0x1) 1 0x00007f8b862df647 in qemu_fclose (f=f@entry=0x55e06c247640) 2 0x00007f8b8516d59a in migrate_fd_cleanup (s=s@entry=0x55e06c0e1ef0) 3 0x00007f8b8516d66c in migrate_fd_cleanup_bh (opaque=0x55e06c0e1ef0) 4 0x00007f8b8626a47f in aio_bh_poll (ctx=ctx@entry=0x55e06b5a16d0) 5 0x00007f8b8626e71f in aio_dispatch (ctx=0x55e06b5a16d0) 6 0x00007f8b8626a33d in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) 7 0x00007f8b866bdba4 in g_main_context_dispatch () 8 0x00007f8b8626cde9 in glib_pollfds_poll () 9 0x00007f8b8626ce62 in os_host_main_loop_wait (timeout=<optimized out>)10
0x00007f8b8626cffdinmain_loop_wait(nonblocking=nonblocking@entry=0) 110x00007f8b862ef01finmain_loop()
Using gdb print the struct QEMUFile f = {
...,
iovcnt = 65,last_error= 21984,
last_error_obj=0x1, shutdown = true
}
Well iovcnt is overflow, because the max size ofMAX_IOV_SIZEis 64.struct QEMUFile { ...; struct iovec iov[MAX_IOV_SIZE]; unsigned int iovcnt; int last_error; Error *last_error_obj; bool shutdown; };iovcnt and
last_erroris overwrited byadd_to_iovec().
Right now,add_to_iovec()increase iovcnt before check the limit. And it seems thatadd_to_iovec()assumes that iovcnt will set to zero inqemu_fflush(). Butqemu_fflush()will directly return whenf->shutdownis true.
Quoted 7 lines
The situation may occur when libvirtd restart during migration, after
f->shutdownis set, before callingqemu_file_set_error()inqemu_file_shutdown().So the safiest way is checking the iovcnt before increasing it.
Signed-off-by Feng Lin <[email protected]>
Queued
Quoted 24 lines
--- migration/qemu-file.c | 5 +++++ 1 file changed, 5 insertions(+)migration/qemu-file.c +5
-0@@ -416,6 +416,11 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, { f->iov[f->iovcnt -1 ].iov_len += size; } else { +if (f->iovcnt >=MAX_IOV_SIZE ) { +/* Should only happen if a previous fflush failed */ +assert (f->shutdown || !qemu_file_is_writeable (f)); +return 1 ; + }if (may_free) {set_bit (f->iovcnt, f->may_free); }Signature · 1 line
2.23.0
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Quoted 40 lines · * Dr. David Alan Gilbert ([email protected]) wrote:
Quoted 39 lines · * Lin Feng ([email protected]) wrote:
From: Feng Lin <[email protected]>
When testing migration, a Segmentation fault qemu core is generated.
0 error_free (err=0x1) 1 0x00007f8b862df647 in qemu_fclose (f=f@entry=0x55e06c247640) 2 0x00007f8b8516d59a in migrate_fd_cleanup (s=s@entry=0x55e06c0e1ef0) 3 0x00007f8b8516d66c in migrate_fd_cleanup_bh (opaque=0x55e06c0e1ef0) 4 0x00007f8b8626a47f in aio_bh_poll (ctx=ctx@entry=0x55e06b5a16d0) 5 0x00007f8b8626e71f in aio_dispatch (ctx=0x55e06b5a16d0) 6 0x00007f8b8626a33d in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) 7 0x00007f8b866bdba4 in g_main_context_dispatch () 8 0x00007f8b8626cde9 in glib_pollfds_poll () 9 0x00007f8b8626ce62 in os_host_main_loop_wait (timeout=<optimized out>)10
0x00007f8b8626cffdinmain_loop_wait(nonblocking=nonblocking@entry=0) 110x00007f8b862ef01finmain_loop()
Using gdb print the struct QEMUFile f = {
...,
iovcnt = 65,last_error= 21984,
last_error_obj=0x1, shutdown = true
}
Well iovcnt is overflow, because the max size ofMAX_IOV_SIZEis 64.struct QEMUFile { ...; struct iovec iov[MAX_IOV_SIZE]; unsigned int iovcnt; int last_error; Error *last_error_obj; bool shutdown; };iovcnt and
last_erroris overwrited byadd_to_iovec().
Right now,add_to_iovec()increase iovcnt before check the limit. And it seems thatadd_to_iovec()assumes that iovcnt will set to zero inqemu_fflush(). Butqemu_fflush()will directly return whenf->shutdownis true.The situation may occur when libvirtd restart during migration, after
f->shutdownis set, before callingqemu_file_set_error()inqemu_file_shutdown().
Quoted 5 lines
Quoted 3 lines
So the safiest way is checking the iovcnt before increasing it.
Signed-off-by Feng Lin <[email protected]>Queued
Hmm this didn't actually build because that function is actually misnamed 'qemu_file_is_writable' (no e!); I've fixed that, but can you just reconfirm that you've tested this fixes your original problem?
Dave
Quoted 26 lines
Quoted 24 lines
--- migration/qemu-file.c | 5 +++++ 1 file changed, 5 insertions(+)migration/qemu-file.c +5
-0@@ -416,6 +416,11 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, { f->iov[f->iovcnt -1 ].iov_len += size; } else { +if (f->iovcnt >=MAX_IOV_SIZE ) { +/* Should only happen if a previous fflush failed */ +assert (f->shutdown || !qemu_file_is_writeable (f)); +return 1 ; + }if (may_free) {set_bit (f->iovcnt, f->may_free); }Signature · 1 line
2.23.0Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Quoted 48 lines · * Dr. David Alan Gilbert ([email protected]) wrote:
Subject: Re: [v4] migration: fix the memory overwriting risk in
add_to_iovecQuoted 45 lines · * Dr. David Alan Gilbert ([email protected]) wrote:
Quoted 44 lines · * Lin Feng ([email protected]) wrote:
From: Feng Lin <[email protected]>
When testing migration, a Segmentation fault qemu core is generated.
0 error_free (err=0x1) 1 0x00007f8b862df647 in qemu_fclose (f=f@entry=0x55e06c247640) 2 0x00007f8b8516d59a in migrate_fd_cleanup (s=s@entry=0x55e06c0e1ef0) 3 0x00007f8b8516d66c in migrate_fd_cleanup_bh (opaque=0x55e06c0e1ef0) 4 0x00007f8b8626a47f in aio_bh_poll (ctx=ctx@entry=0x55e06b5a16d0) 5 0x00007f8b8626e71f in aio_dispatch (ctx=0x55e06b5a16d0) 6 0x00007f8b8626a33d in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) 7 0x00007f8b866bdba4 in g_main_context_dispatch () 8 0x00007f8b8626cde9 in glib_pollfds_poll () 9 0x00007f8b8626ce62 in os_host_main_loop_wait (timeout=<optimizedout>)
100x00007f8b8626cffdinmain_loop_wait(nonblocking=nonblocking@entry=0)
110x00007f8b862ef01finmain_loop() Using gdb print the struct QEMUFile f = {
...,
iovcnt = 65,last_error= 21984,
last_error_obj=0x1, shutdown = true } Well iovcnt is overflow,
because the max size ofMAX_IOV_SIZEis 64.struct QEMUFile { ...; struct iovec iov[MAX_IOV_SIZE]; unsigned int iovcnt; int last_error; Error *last_error_obj; bool shutdown; };iovcnt and
last_erroris overwrited byadd_to_iovec(). Right now,add_to_iovec()increase iovcnt before check the limit. And it seems thatadd_to_iovec()assumes that iovcnt will set to zero inqemu_fflush(). Butqemu_fflush()will directly return whenf->shutdownis true.The situation may occur when libvirtd restart during migration, after
f->shutdownis set, before callingqemu_file_set_error()inqemu_file_shutdown().
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
So the safiest way is checking the iovcnt before increasing it.
Signed-off-by Feng Lin <[email protected]>Queued
Hmm this didn't actually build because that function is actually misnamed '
qemu_file_is_writable' (no e!); I've fixed that, but can you just reconfirm that you've tested this fixes your original problem?
Sorry for that rookie mistake. I have tested it again with gdb-fault injection. It can fix my original problem. Thanks for helping me complete my first qemu patch submission. Really helped a lot.
Quoted 31 lines
Dave
Quoted 18 lines
Quoted 18 lines
--- migration/qemu-file.c | 5 +++++ 1 file changed, 5 insertions(+)migration/qemu-file.c index renamed from migration/qemu-file.c +0
-0d6e03dbc0e..6879615197 100644
migration/qemu-file.c +5
-0@@ -416,6 +416,11 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, { f->iov[f->iovcnt -1 ].iov_len += size; } else { +if (f->iovcnt >=MAX_IOV_SIZE ) { +/* Should only happen if a previous fflush failed */ +assert (f->shutdown || !qemu_file_is_writeable (f)); +return 1 ; + }if (may_free) {Quoted 8 lines
Quoted 6 lines
migration/qemu-file.c +0
-1set_bit (f->iovcnt, f->may_free); } --2.23.0
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK--
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Quoted 52 lines · * linfeng (M) ([email protected]) wrote:
Quoted 50 lines · * Dr. David Alan Gilbert ([email protected]) wrote:
Subject: Re: [v4] migration: fix the memory overwriting risk in
add_to_iovecQuoted 47 lines · * Dr. David Alan Gilbert ([email protected]) wrote:
Quoted 46 lines · * Lin Feng ([email protected]) wrote:
From: Feng Lin <[email protected]>
When testing migration, a Segmentation fault qemu core is generated.
0 error_free (err=0x1) 1 0x00007f8b862df647 in qemu_fclose (f=f@entry=0x55e06c247640) 2 0x00007f8b8516d59a in migrate_fd_cleanup (s=s@entry=0x55e06c0e1ef0) 3 0x00007f8b8516d66c in migrate_fd_cleanup_bh (opaque=0x55e06c0e1ef0) 4 0x00007f8b8626a47f in aio_bh_poll (ctx=ctx@entry=0x55e06b5a16d0) 5 0x00007f8b8626e71f in aio_dispatch (ctx=0x55e06b5a16d0) 6 0x00007f8b8626a33d in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) 7 0x00007f8b866bdba4 in g_main_context_dispatch () 8 0x00007f8b8626cde9 in glib_pollfds_poll () 9 0x00007f8b8626ce62 in os_host_main_loop_wait (timeout=<optimizedout>)
100x00007f8b8626cffdinmain_loop_wait(nonblocking=nonblocking@entry=0)
110x00007f8b862ef01finmain_loop() Using gdb print the struct QEMUFile f = {
...,
iovcnt = 65,last_error= 21984,
last_error_obj=0x1, shutdown = true } Well iovcnt is overflow,
because the max size ofMAX_IOV_SIZEis 64.struct QEMUFile { ...; struct iovec iov[MAX_IOV_SIZE]; unsigned int iovcnt; int last_error; Error *last_error_obj; bool shutdown; };iovcnt and
last_erroris overwrited byadd_to_iovec(). Right now,add_to_iovec()increase iovcnt before check the limit. And it seems thatadd_to_iovec()assumes that iovcnt will set to zero inqemu_fflush(). Butqemu_fflush()will directly return whenf->shutdownis true.The situation may occur when libvirtd restart during migration, after
f->shutdownis set, before callingqemu_file_set_error()inqemu_file_shutdown().So the safiest way is checking the iovcnt before increasing it.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Feng Lin <[email protected]>Queued
Hmm this didn't actually build because that function is actually misnamed '
qemu_file_is_writable' (no e!); I've fixed that, but can you just reconfirm that you've tested this fixes your original problem?Sorry for that rookie mistake. I have tested it again with gdb-fault injection. It can fix my original problem. Thanks for helping me complete my first qemu patch submission. Really helped a lot.
Thanks for retesting.
Dave
Quoted 32 lines
Quoted 31 lines
Dave
Quoted 18 lines
Quoted 18 lines
--- migration/qemu-file.c | 5 +++++ 1 file changed, 5 insertions(+)migration/qemu-file.c index renamed from migration/qemu-file.c +0
-0d6e03dbc0e..6879615197 100644
migration/qemu-file.c +5
-0@@ -416,6 +416,11 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf, size_t size, { f->iov[f->iovcnt -1 ].iov_len += size; } else { +if (f->iovcnt >=MAX_IOV_SIZE ) { +/* Should only happen if a previous fflush failed */ +assert (f->shutdown || !qemu_file_is_writeable (f)); +return 1 ; + }if (may_free) {Quoted 8 lines
Quoted 6 lines
migration/qemu-file.c +0
-1set_bit (f->iovcnt, f->may_free); } --2.23.0
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK--
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK