fix hangup when multiple processes are trying to write to the same file
PATCH 1/1 fix hangup when multiple processes are trying to write to the same file
This fixes a problem I encountered while running bonnie++. When you have one thread that opens a file and starts to write to it, and then another thread that tries to open and write to the same file, the second thread will loop forever trying to grab the inode lock for that inode. Basically we come in through generic_buffered_file_write, which calls gfs2_prepare_write, which then attempts to grab the glock. Because we don't own the lock, gfs2_prepare_write gets GLR_TRYFAILED, which returns AOP_TRUNCATED_PAGE to generic_buffered_file_write. At this point generic_buffered_file_write loops around again and immediately retries the prepare_write. This means that the second process never gets off of the processor in order to allow the process that holds the lock to finish its work and let go of the lock. This patch makes gfs2_glock_nq schedule() if it gets back a GLR_TRYFAILED, which resolves this problem. Please let me know if this is in the wrong place, ie if it should go in gfs2_prepare_write instead of gfs2_glock_nq, or if its completely wrong :).
fs/gfs2/glock.c renamed from fs/gfs2/glock.c.josef +2 -1
@@ -1185,7 +1185,8 @@if (error ==GLR_CANCELED ) {msleep (100 ); goto restart; - } + }else if (error ==GLR_TRYFAILED ) +schedule (); }return error;
PATCH 1/1 fix hangup when multiple processes are trying to write to the same file
Quoted 6 lines · Josef Whiter
This fixes a problem I encountered while running bonnie++. When you have one thread that opens a file and starts to write to it, and then another thread that tries to open and write to the same file, the second thread will loop forever trying to grab the inode lock for that inode. Basically we come in through
generic_buffered_file_write, which callsgfs2_prepare_write, which then attempts to grab the glock. Because we don't own the lock,gfs2_prepare_writegets
Quoted 9 lines
GLR_TRYFAILED, which returnsAOP_TRUNCATED_PAGEtogeneric_buffered_file_write. At this pointgeneric_buffered_file_writeloops around again and immediately retries theprepare_write. This means that the second process never gets off of the processor in order to allow the process that holds the lock to finish its work and let go of the lock. This patch makesgfs2_glock_nqschedule()if it gets back aGLR_TRYFAILED, which resolves this problem. Please let me know if this is in the wrong place, ie if it should go ingfs2_prepare_writeinstead ofgfs2_glock_nq, or if its completely wrong :).
Ok at the suggestion of Steven I have made it so that we just yield in the places where this would be a problem, so in gfs2_prepare_write and gfs2_readpage.
fs/gfs2/ops_address.c renamed from fs/gfs2/ops_address.c.josef +4 -1
@@ -266,8 +266,10 @@ skip_lock: out:return error; out_unlock: -if (error ==GLR_TRYFAILED ) +if (error ==GLR_TRYFAILED ) { error =AOP_TRUNCATED_PAGE ; +yield (); + }unlock_page (page);if (do_unlock)gfs2_holder_uninit (&gh);
@@ -364,6 +366,7 @@ static int gfs2_prepare_write(struct filif (error ==GLR_TRYFAILED ) {unlock_page (page); error =AOP_TRUNCATED_PAGE ; +yield (); } goto out_uninit; }
PATCH 1/1 fix hangup when multiple processes are trying to write to the same file
Hi,
Quoted 17 lines · Josef Whiter
Quoted 6 lines · Josef Whiter
This fixes a problem I encountered while running bonnie++. When you have one thread that opens a file and starts to write to it, and then another thread that tries to open and write to the same file, the second thread will loop forever trying to grab the inode lock for that inode. Basically we come in through
generic_buffered_file_write, which callsgfs2_prepare_write, which then attempts to grab the glock. Because we don't own the lock,gfs2_prepare_writegetsQuoted 9 lines
GLR_TRYFAILED, which returnsAOP_TRUNCATED_PAGEtogeneric_buffered_file_write. At this pointgeneric_buffered_file_writeloops around again and immediately retries theprepare_write. This means that the second process never gets off of the processor in order to allow the process that holds the lock to finish its work and let go of the lock. This patch makesgfs2_glock_nqschedule()if it gets back aGLR_TRYFAILED, which resolves this problem. Please let me know if this is in the wrong place, ie if it should go ingfs2_prepare_writeinstead ofgfs2_glock_nq, or if its completely wrong :).
Quoted 6 lines
Ok at the suggestion of Steven I have made it so that we just yield in the places where this would be a problem, so in
gfs2_prepare_writeandgfs2_readpage.Signed-off-by Josef Whiter <[email protected]>
Thats looks good, but one small comment:
fs/gfs2/ops_address.c skip_lock: · line 273
Quoted 12 lines
fs/gfs2/ops_address.c renamed from fs/gfs2/ops_address.c.josef +3
-1@@ -266,8 +266,10 @@ skip_lock: out:return error; out_unlock: -if (error ==GLR_TRYFAILED ) +if (error ==GLR_TRYFAILED ) { error =AOP_TRUNCATED_PAGE ; +yield (); + }unlock_page (page);
Can you move the above unlock_page to directly after the out_unlock: ? That way we won't schedule while holding the page lock. The other part of the patch does that in the right order already and looks good,
Steve.
Quoted 11 lines
fs/gfs2/ops_address.c renamed from fs/gfs2/ops_address.c.josef +1
-0if (do_unlock)gfs2_holder_uninit (&gh);@@ -364,6 +366,7 @@ static int gfs2_prepare_write(struct filif (error ==GLR_TRYFAILED ) {unlock_page (page); error =AOP_TRUNCATED_PAGE ; +yield (); } goto out_uninit; }
PATCH 1/1 fix hangup when multiple processes are trying to write to the same file
Quoted 44 lines · Steven Whitehouse
Hi,
Quoted 17 lines · Josef Whiter
Quoted 6 lines · Josef Whiter
This fixes a problem I encountered while running bonnie++. When you have one thread that opens a file and starts to write to it, and then another thread that tries to open and write to the same file, the second thread will loop forever trying to grab the inode lock for that inode. Basically we come in through
generic_buffered_file_write, which callsgfs2_prepare_write, which then attempts to grab the glock. Because we don't own the lock,gfs2_prepare_writegetsQuoted 9 lines
GLR_TRYFAILED, which returnsAOP_TRUNCATED_PAGEtogeneric_buffered_file_write. At this pointgeneric_buffered_file_writeloops around again and immediately retries theprepare_write. This means that the second process never gets off of the processor in order to allow the process that holds the lock to finish its work and let go of the lock. This patch makesgfs2_glock_nqschedule()if it gets back aGLR_TRYFAILED, which resolves this problem. Please let me know if this is in the wrong place, ie if it should go ingfs2_prepare_writeinstead ofgfs2_glock_nq, or if its completely wrong :).Quoted 6 lines
Ok at the suggestion of Steven I have made it so that we just yield in the places where this would be a problem, so in
gfs2_prepare_writeandgfs2_readpage.Signed-off-by Josef Whiter <[email protected]>Thats looks good, but one small comment:
fs/gfs2/ops_address.c skip_lock: · line 273
Quoted 12 lines
fs/gfs2/ops_address.c renamed from fs/gfs2/ops_address.c.josef +3
-1@@ -266,8 +266,10 @@ skip_lock: out:return error; out_unlock: -if (error ==GLR_TRYFAILED ) +if (error ==GLR_TRYFAILED ) { error =AOP_TRUNCATED_PAGE ; +yield (); + }unlock_page (page);Can you move the above
unlock_pageto directly after theout_unlock: ? That way we won't schedule while holding the page lock. The other part of the patch does that in the right order already and looks good,
Quoted 2 lines
Steve.
Doh sorry about that. Here's the correct version
fs/gfs2/ops_address.c renamed from fs/gfs2/ops_address.c.josef +5 -2
@@ -266,9 +266,11 @@ skip_lock: out:return error; out_unlock: -if (error ==GLR_TRYFAILED ) - error =AOP_TRUNCATED_PAGE ;unlock_page (page); +if (error ==GLR_TRYFAILED ) { + error =AOP_TRUNCATED_PAGE ; +yield (); + }if (do_unlock)gfs2_holder_uninit (&gh); goto out;
@@ -364,6 +366,7 @@ static int gfs2_prepare_write(struct filif (error ==GLR_TRYFAILED ) {unlock_page (page); error =AOP_TRUNCATED_PAGE ; +yield (); } goto out_uninit; }
PATCH 1/1 fix hangup when multiple processes are trying to write to the same file
Hi,
Now in the -nmw git tree. Thanks,
Steve.
Quoted 75 lines · Josef Whiter
Quoted 44 lines · Steven Whitehouse
Hi,
Quoted 17 lines · Josef Whiter
Quoted 6 lines · Josef Whiter
This fixes a problem I encountered while running bonnie++. When you have one thread that opens a file and starts to write to it, and then another thread that tries to open and write to the same file, the second thread will loop forever trying to grab the inode lock for that inode. Basically we come in through
generic_buffered_file_write, which callsgfs2_prepare_write, which then attempts to grab the glock. Because we don't own the lock,gfs2_prepare_writegetsQuoted 9 lines
GLR_TRYFAILED, which returnsAOP_TRUNCATED_PAGEtogeneric_buffered_file_write. At this pointgeneric_buffered_file_writeloops around again and immediately retries theprepare_write. This means that the second process never gets off of the processor in order to allow the process that holds the lock to finish its work and let go of the lock. This patch makesgfs2_glock_nqschedule()if it gets back aGLR_TRYFAILED, which resolves this problem. Please let me know if this is in the wrong place, ie if it should go ingfs2_prepare_writeinstead ofgfs2_glock_nq, or if its completely wrong :).Quoted 6 lines
Ok at the suggestion of Steven I have made it so that we just yield in the places where this would be a problem, so in
gfs2_prepare_writeandgfs2_readpage.Signed-off-by Josef Whiter <[email protected]>Thats looks good, but one small comment:
fs/gfs2/ops_address.c skip_lock: · line 273
Quoted 12 lines
fs/gfs2/ops_address.c renamed from fs/gfs2/ops_address.c.josef +3
-1@@ -266,8 +266,10 @@ skip_lock: out:return error; out_unlock: -if (error ==GLR_TRYFAILED ) +if (error ==GLR_TRYFAILED ) { error =AOP_TRUNCATED_PAGE ; +yield (); + }unlock_page (page);Can you move the above
unlock_pageto directly after theout_unlock: ? That way we won't schedule while holding the page lock. The other part of the patch does that in the right order already and looks good,Quoted 2 lines
Steve.
Doh sorry about that. Here's the correct version
fs/gfs2/ops_address.c renamed from fs/gfs2/ops_address.c.josef +5
-2@@ -266,9 +266,11 @@ skip_lock: out:return error; out_unlock: -if (error ==GLR_TRYFAILED ) - error =AOP_TRUNCATED_PAGE ;unlock_page (page); +if (error ==GLR_TRYFAILED ) { + error =AOP_TRUNCATED_PAGE ; +yield (); + }if (do_unlock)gfs2_holder_uninit (&gh); goto out;@@ -364,6 +366,7 @@ static int gfs2_prepare_write(struct filif (error ==GLR_TRYFAILED ) {unlock_page (page); error =AOP_TRUNCATED_PAGE ; +yield (); } goto out_uninit; }