xen/blkback: unmap all persistent grants when frontend gets disconnected
Subsystems (2)
PATCH 1/1 xen/blkback: unmap all persistent grants when frontend gets disconnected
blkback does not unmap persistent grants when frontend goes to Closed state (e.g. when blkfront module is being removed). This leads to the following in guest's dmesg:
[ 343.243825] xen:grant_table: WARNING: g.e. 0x445 still in use! [ 343.243825] xen:grant_table: WARNING: g.e. 0x42a still in use!
...
When load module -> use device -> unload module sequence is performed multiple times it is possible to hit BUG() condition in blkfront module:
[ 343.243825] kernel BUG at drivers/block/xen-blkfront.c:954! [ 343.243825] invalid opcode: 0000 [#1] SMP [ 343.243825] Modules linked in: xen_blkfront(-) ata_generic pata_acpi [last unloaded: xen_blkfront] ... [ 343.243825] Call Trace: [ 343.243825] [<ffffffff814111ef>] ? unregister_xenbus_watch+0x16f/0x1e0 [ 343.243825] [<ffffffffa0016fbf>] blkfront_remove+0x3f/0x140 [xen_blkfront] ... [ 343.243825] RIP [<ffffffffa0016aae>] blkif_free+0x34e/0x360 [xen_blkfront] [ 343.243825] RSP <ffff88001eb8fdc0>
We don't need to keep these grants if we're disconnecting as frontend might already forgot about them. Solve the issue by moving xen_blkbk_free_caches() call from xen_blkif_free() to xen_blkif_disconnect().
Now we can see the following:
[ 928.590893] xen:grant_table: WARNING: g.e. 0x587 still in use! [ 928.591861] xen:grant_table: WARNING: g.e. 0x372 still in use! ... [ 929.592146] xen:grant_table: freeing g.e. 0x587 [ 929.597174] xen:grant_table: freeing g.e. 0x372
...
Backend does not keep persistent grants any more, reconnect works fine.
--- drivers/block/xen-blkback/xenbus.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/block/xen-blkback/xenbus.c +3 -3
@@ -270,6 +270,9 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) blkif->blk_rings.common.sring =NULL ; } +/* Remove all persistent grants and the cache of ballooned pages. */ +xen_blkbk_free_caches (blkif); +return 0 ; }
@@ -281,9 +284,6 @@ static void xen_blkif_free(struct xen_blkif *blkif)xen_blkif_disconnect (blkif);xen_vbd_free (&blkif->vbd); -/* Remove all persistent grants and the cache of ballooned pages. */ -xen_blkbk_free_caches (blkif); -/* Make sure everything is drained before shutting down */ BUG_ON (blkif->persistent_gnt_c !=0 );BUG_ON (atomic_read (&blkif->persistent_gnt_in_use) !=0 );
Signature · 1 line
1.9.3
Quoted 27 lines · Vitaly Kuznetsov
blkback does not unmap persistent grants when frontend goes to Closed state (e.g. when blkfront module is being removed). This leads to the following in guest's dmesg:
[ 343.243825] xen:grant_table: WARNING: g.e. 0x445 still in use! [ 343.243825] xen:grant_table: WARNING: g.e. 0x42a still in use!...
When load module -> use device -> unload module sequence is performed multiple times it is possible to hit
BUG()condition in blkfront module:[ 343.243825] kernel BUG at drivers/block/xen-blkfront.c:954! [ 343.243825] invalid opcode: 0000 [#1] SMP [ 343.243825] Modules linked in: xen_blkfront(-) ata_generic pata_acpi [last unloaded: xen_blkfront] ... [ 343.243825] Call Trace: [ 343.243825] [<ffffffff814111ef>] ? unregister_xenbus_watch+0x16f/0x1e0 [ 343.243825] [<ffffffffa0016fbf>] blkfront_remove+0x3f/0x140 [xen_blkfront] ... [ 343.243825] RIP [<ffffffffa0016aae>] blkif_free+0x34e/0x360 [xen_blkfront] [ 343.243825] RSP <ffff88001eb8fdc0>We don't need to keep these grants if we're disconnecting as frontend might already forgot about them. Solve the issue by moving
xen_blkbk_free_caches()call fromxen_blkif_free()toxen_blkif_disconnect().Now we can see the following:
Quoted 8 lines
[ 928.590893] xen:grant_table: WARNING: g.e. 0x587 still in use! [ 928.591861] xen:grant_table: WARNING: g.e. 0x372 still in use! ... [ 929.592146] xen:grant_table: freeing g.e. 0x587 [ 929.597174] xen:grant_table: freeing g.e. 0x372...
Backend does not keep persistent grants any more, reconnect works fine.
David,
I usually combine the 'block' related patches in one branch and send them to Jens for the merge window.
Want to do it the same way or do you want to alter the workflow?
Quoted 33 lines
Signed-off-by Vitaly Kuznetsov <[email protected]>--- drivers/block/xen-blkback/xenbus.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)drivers/block/xen-blkback/xenbus.c +3
-3@@ -270,6 +270,9 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) blkif->blk_rings.common.sring =NULL ; } +/* Remove all persistent grants and the cache of ballooned pages. */ +xen_blkbk_free_caches (blkif); +return 0 ; }@@ -281,9 +284,6 @@ static void xen_blkif_free(struct xen_blkif *blkif)xen_blkif_disconnect (blkif);xen_vbd_free (&blkif->vbd); -/* Remove all persistent grants and the cache of ballooned pages. */ -xen_blkbk_free_caches (blkif); -/* Make sure everything is drained before shutting down */ BUG_ON (blkif->persistent_gnt_c !=0 );BUG_ON (atomic_read (&blkif->persistent_gnt_in_use) !=0 );Signature · 1 line
1.9.3
Quoted 27 lines · Vitaly Kuznetsov
blkback does not unmap persistent grants when frontend goes to Closed state (e.g. when blkfront module is being removed). This leads to the following in guest's dmesg:
[ 343.243825] xen:grant_table: WARNING: g.e. 0x445 still in use! [ 343.243825] xen:grant_table: WARNING: g.e. 0x42a still in use!...
When load module -> use device -> unload module sequence is performed multiple times it is possible to hit
BUG()condition in blkfront module:[ 343.243825] kernel BUG at drivers/block/xen-blkfront.c:954! [ 343.243825] invalid opcode: 0000 [#1] SMP [ 343.243825] Modules linked in: xen_blkfront(-) ata_generic pata_acpi [last unloaded: xen_blkfront] ... [ 343.243825] Call Trace: [ 343.243825] [<ffffffff814111ef>] ? unregister_xenbus_watch+0x16f/0x1e0 [ 343.243825] [<ffffffffa0016fbf>] blkfront_remove+0x3f/0x140 [xen_blkfront] ... [ 343.243825] RIP [<ffffffffa0016aae>] blkif_free+0x34e/0x360 [xen_blkfront] [ 343.243825] RSP <ffff88001eb8fdc0>We don't need to keep these grants if we're disconnecting as frontend might already forgot about them. Solve the issue by moving
xen_blkbk_free_caches()call fromxen_blkif_free()toxen_blkif_disconnect().Now we can see the following:
Quoted 8 lines
[ 928.590893] xen:grant_table: WARNING: g.e. 0x587 still in use! [ 928.591861] xen:grant_table: WARNING: g.e. 0x372 still in use! ... [ 929.592146] xen:grant_table: freeing g.e. 0x587 [ 929.597174] xen:grant_table: freeing g.e. 0x372...
Backend does not keep persistent grants any more, reconnect works fine.
David,
I usually combine the 'block' related patches in one branch and send them to Jens for the merge window.
Want to do it the same way or do you want to alter the workflow?
Quoted 33 lines
Signed-off-by Vitaly Kuznetsov <[email protected]>--- drivers/block/xen-blkback/xenbus.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)drivers/block/xen-blkback/xenbus.c +3
-3@@ -270,6 +270,9 @@ static int xen_blkif_disconnect(struct xen_blkif *blkif) blkif->blk_rings.common.sring =NULL ; } +/* Remove all persistent grants and the cache of ballooned pages. */ +xen_blkbk_free_caches (blkif); +return 0 ; }@@ -281,9 +284,6 @@ static void xen_blkif_free(struct xen_blkif *blkif)xen_blkif_disconnect (blkif);xen_vbd_free (&blkif->vbd); -/* Remove all persistent grants and the cache of ballooned pages. */ -xen_blkbk_free_caches (blkif); -/* Make sure everything is drained before shutting down */ BUG_ON (blkif->persistent_gnt_c !=0 );BUG_ON (atomic_read (&blkif->persistent_gnt_in_use) !=0 );Signature · 1 line
1.9.3