dcache: return -ESTALE not -EBUSY on distributed fs race
Subsystems (3)
PATCH 1/1 dcache: return -ESTALE not -EBUSY on distributed fs race
From: "J. Bruce Fields" <[email protected]>
On a distributed filesystem it's possible for lookup to discover that a directory it just found is already cached elsewhere in the directory heirarchy. The dcache won't let us keep the directory in both places, so we have to move the dentry to the new location from the place we previously had it cached.
If the parent has changed, then this requires all the same locks as we'd need to do a cross-directory rename. But we're already in lookup holding one parent's i_mutex, so it's too late to acquire those locks in the right order.
The (unreliable) solution in __d_unalias is to trylock() the required locks and return -EBUSY if it fails.
I see no particular reason for returning -EBUSY, and -ESTALE is already the result of some other lookup races on NFS. I think -ESTALE is the more helpful error return. It also allows us to take advantage of the logic Jeff Layton added in c6a9428401c0 "vfs: fix renameat to retry on ESTALE errors" and ancestors, which hopefully resolves some of these errors before they're returned to userspace.
I can reproduce these cases using NFS with:
ssh root@$client '
mount -olookupcache=pos '$server':'$export' /mnt/ mkdir /mnt/TO
mkdir /mnt/DIR
touch /mnt/DIR/test.txt
while true; do
strace -e open cat /mnt/DIR/test.txt 2>&1 | grep EBUSY
done
'
ssh root@$server '
while true; do
mv $export/DIR $export/TO/DIR
mv $export/TO/DIR $export/DIR
done
'
It also helps to add some other concurrent use of the directory on the client (e.g., "ls /mnt/TO"). And you can replace the server-side mv's by client-side mv's that are repeatedly killed. (If the client is interrupted while waiting for the RENAME response then it's left with a dentry that has to go under one parent or the other, but it doesn't yet know which.)
--- fs/dcache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/dcache.c +1 -1
@@ -2612,7 +2612,7 @@ static struct dentry *__d_unalias(struct inode *inode, struct dentry *dentry, struct dentry *alias) { struct mutex *m1 = NULL, *m2 = NULL; -struct dentry *ret =ERR_PTR (-EBUSY ); +struct dentry *ret =ERR_PTR (-ESTALE );/* If alias and dentry share a parent, then no extra locks required */ if (alias->d_parent == dentry->d_parent)
Signature · 6 lines
1.9.3 -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Al, what's up with this patch?
--b.
Quoted 70 lines · J. Bruce Fields
From: "J. Bruce Fields" <[email protected]>
On a distributed filesystem it's possible for lookup to discover that a directory it just found is already cached elsewhere in the directory heirarchy. The dcache won't let us keep the directory in both places, so we have to move the dentry to the new location from the place we previously had it cached.
If the parent has changed, then this requires all the same locks as we'd need to do a cross-directory rename. But we're already in lookup holding one parent's
i_mutex, so it's too late to acquire those locks in the right order.The (unreliable) solution in
__d_unaliasis totrylock()the required locks and return-EBUSYif it fails.I see no particular reason for returning
-EBUSY, and-ESTALEis already the result of some other lookup races on NFS. I think-ESTALEis the more helpful error return. It also allows us to take advantage of the logic Jeff Layton added inc6a9428401c0"vfs: fix renameat to retry on ESTALE errors" and ancestors, which hopefully resolves some of these errors before they're returned to userspace.I can reproduce these cases using NFS with:
ssh root@$client '
mount -olookupcache=pos '$server':'$export'/mnt/mkdir/mnt/TO
mkdir/mnt/DIR
touch/mnt/DIR/test.txt
while true; do
strace -e open cat/mnt/DIR/test.txt2>&1 | grep EBUSY
done
'
ssh root@$server '
while true; do
mv $export/DIR $export/TO/DIR
mv $export/TO/DIR $export/DIR
done
'It also helps to add some other concurrent use of the directory on the client (e.g., "ls
/mnt/TO"). And you can replace the server-side mv's by client-side mv's that are repeatedly killed. (If the client is interrupted while waiting for the RENAME response then it's left with a dentry that has to go under one parent or the other, but it doesn't yet know which.)Acked-by Jeff Layton <[email protected]>Signed-off-by J. Bruce Fields <[email protected]>--- fs/dcache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/dcache.c +1
-1@@ -2612,7 +2612,7 @@ static struct dentry *__d_unalias(struct inode *inode, struct dentry *dentry, struct dentry *alias) { struct mutex *m1 = NULL, *m2 = NULL; -struct dentry *ret =ERR_PTR (-EBUSY ); +struct dentry *ret =ERR_PTR (-ESTALE );/* If alias and dentry share a parent, then no extra locks required */ if (alias->d_parent == dentry->d_parent)Signature · 1 line
1.9.3
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH 1/1 dcache: return -ESTALE not -EBUSY on distributed fs race
From: "J. Bruce Fields" <[email protected]>
On a distributed filesystem it's possible for lookup to discover that a directory it just found is already cached elsewhere in the directory heirarchy. The dcache won't let us keep the directory in both places, so we have to move the dentry to the new location from the place we previously had it cached.
If the parent has changed, then this requires all the same locks as we'd need to do a cross-directory rename. But we're already in lookup holding one parent's i_mutex, so it's too late to acquire those locks in the right order.
The (unreliable) solution in __d_unalias is to trylock() the required locks and return -EBUSY if it fails.
I see no particular reason for returning -EBUSY, and -ESTALE is already the result of some other lookup races on NFS. I think -ESTALE is the more helpful error return. It also allows us to take advantage of the logic Jeff Layton added in c6a9428401c0 "vfs: fix renameat to retry on ESTALE errors" and ancestors, which hopefully resolves some of these errors before they're returned to userspace.
I can reproduce these cases using NFS with:
ssh root@$client '
mount -olookupcache=pos '$server':'$export' /mnt/ mkdir /mnt/TO
mkdir /mnt/DIR
touch /mnt/DIR/test.txt
while true; do
strace -e open cat /mnt/DIR/test.txt 2>&1 | grep EBUSY
done
'
ssh root@$server '
while true; do
mv $export/DIR $export/TO/DIR
mv $export/TO/DIR $export/DIR
done
'
It also helps to add some other concurrent use of the directory on the client (e.g., "ls /mnt/TO"). And you can replace the server-side mv's by client-side mv's that are repeatedly killed. (If the client is interrupted while waiting for the RENAME response then it's left with a dentry that has to go under one parent or the other, but it doesn't yet know which.)
--- fs/dcache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Resending with minor conflict resolution needed due to b5ae6b15bd73 "merge d_materialise_unique() into d_splice_alias()"
fs/dcache.c +1 -1
@@ -2690,7 +2690,7 @@ static int __d_unalias(struct inode *inode, struct dentry *dentry, struct dentry *alias) { struct mutex *m1 = NULL, *m2 = NULL; -int ret = -EBUSY ; +int ret = -ESTALE ;/* If alias and dentry share a parent, then no extra locks required */ if (alias->d_parent == dentry->d_parent)
Signature · 5 lines
1.9.3 -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html