Problem: LTP linkat01 test fails on nfs directory (NFS v3)
Hello.
Tested kernels: 2.6.18, 2.6.22, 2.6.23-rc2
Steps to reproduce: Suppose that we have mounted some directory from nfs v3 server with default options. Also we have the two directories in this mountpoint and each directory has hard linked file. Try to open those files and write to one and read from another. Data will not be equal. (Testcase:
attached hardlink_test.c)
Analysis: Although these hard linked files have same nfs_fattr::fileid but have different nfs_fh fhandle. In this case nfs_find_actor() function (fs/nfs/inode.c) returns false during opening each file:
nfs_find_actor()
{
...
if (nfs_compare_fh(NFS_FH(inode), fh)) return 0;
...
}
Therefore for each of hard links new struct inode is allocated. It leads to cache aliasing.
Please explain why nfs_find_actor() function compares file handles?
Signature · 2 lines
Thanks, Vitaliy Gusev
#include <stdio.h>
#include <assert.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <err.h>
#define SLEEP 15
int main(int argc, char *argv[])
{
int fd1,fd2;
char *f1 = argv[1];
char *f2 = argv[2];
char buf[1000]= {0,};
char wrbuf[200];
int ret;
assert(argc == 3);
printf("write to: %s\nread from: %s\n\n",
f1, f2);
fd1 = open(f1, O_RDWR);
fd2 = open(f2, O_RDONLY);
if (fd1 < 0 || fd2 < 0)
err(1, "error open");
sprintf(wrbuf, "test message-%d", getpid());
printf("Write: [%s]\n", wrbuf);
ret = write(fd1, wrbuf, strlen(wrbuf));
if (ret < 0)
err(1, "error write");
sleep(1);
ret = read(fd2, buf, sizeof(buf) - 1);
if (ret < 0)
err(1, "error read");
printf("Read: [%s]\n", buf);
return 0;
}Hello.
Tested kernels: 2.6.18, 2.6.22, 2.6.23-rc2
Steps to reproduce: Suppose that we have mounted some directory from nfs v3 server with default options. Also we have the two directories in this mountpoint and each directory has hard linked file. Try to open those files and write to one and read from another. Data will not be equal. (Testcase:
attached hardlink_test.c)
Analysis: Although these hard linked files have same nfs_fattr::fileid but have different nfs_fh fhandle. In this case nfs_find_actor() function (fs/nfs/inode.c) returns false during opening each file:
nfs_find_actor()
{
...
if (nfs_compare_fh(NFS_FH(inode), fh)) return 0;
...
}
Therefore for each of hard links new struct inode is allocated. It leads to cache aliasing.
Please explain why nfs_find_actor() function compares file handles?
Signature · 2 lines
Thanks, Vitaliy Gusev
#include <stdio.h>
#include <assert.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <err.h>
#define SLEEP 15
int main(int argc, char *argv[])
{
int fd1,fd2;
char *f1 = argv[1];
char *f2 = argv[2];
char buf[1000]= {0,};
char wrbuf[200];
int ret;
assert(argc == 3);
printf("write to: %s\nread from: %s\n\n",
f1, f2);
fd1 = open(f1, O_RDWR);
fd2 = open(f2, O_RDONLY);
if (fd1 < 0 || fd2 < 0)
err(1, "error open");
sprintf(wrbuf, "test message-%d", getpid());
printf("Write: [%s]\n", wrbuf);
ret = write(fd1, wrbuf, strlen(wrbuf));
if (ret < 0)
err(1, "error write");
sleep(1);
ret = read(fd2, buf, sizeof(buf) - 1);
if (ret < 0)
err(1, "error read");
printf("Read: [%s]\n", buf);
return 0;
}Quoted 9 lines · Vitaliy Gusev
Hello.
Tested kernels: 2.6.18, 2.6.22, 2.6.23-rc2
Steps to reproduce: Suppose that we have mounted some directory from nfs v3 server with default options. Also we have the two directories in this mountpoint and each directory has hard linked file. Try to open those files and write to one and read from another. Data will not be equal. (Testcase:
attachedhardlink_test.c)
Please retry after re-exporting the filesystem using the highly recommended "no_subtree_check" option. The default subtree_check option is broken: it changes the filehandle of the file upon a cross-directory rename() of that file.
Quoted 9 lines
Analysis: Although these hard linked files have same
nfs_fattr::fileidbut have differentnfs_fhfhandle. In this casenfs_find_actor()function (fs/nfs/inode.c) returns false during opening each file:
nfs_find_actor()
{
...if (nfs_compare_fh(NFS_FH(inode), fh)) return 0;
Quoted 7 lines
...
}Therefore for each of hard links new struct inode is allocated. It leads to cache aliasing.
Please explain why
nfs_find_actor()function compares file handles?
'cos this is the only way to know that two files are the same. fileid is not always supported by servers: it is an optional NFSv4 attribute, and on NFSv3, most non-posix filesystems will fake it using something like iunique().
Comparing filehandles allows you to be certain that two files are the same if the filehandles are equal. If they are not equal, then that does not guarantee that the files are different, but then how else are you going to determine it?
Trond
Quoted 11 lines · On the Friday 21 September 2007 16:37 Trond Myklebust, wrote:
Quoted 9 lines · Vitaliy Gusev
Hello.
Tested kernels: 2.6.18, 2.6.22, 2.6.23-rc2
Steps to reproduce: Suppose that we have mounted some directory from nfs v3 server with default options. Also we have the two directories in this mountpoint and each directory has hard linked file. Try to open those files and write to one and read from another. Data will not be equal. (Testcase: attached
hardlink_test.c)
Quoted 4 lines
Please retry after re-exporting the filesystem using the highly recommended "
no_subtree_check" option. The defaultsubtree_checkoption is broken: it changes the filehandle of the file upon a cross-directoryrename()of that file.
Ok, problem goes out. Default option depends on nfs-utils release.
Quoted 7 lines
Quoted 2 lines
Please explain why
nfs_find_actor()function compares file handles?'cos this is the only way to know that two files are the same. fileid is not always supported by servers: it is an optional NFSv4 attribute, and
From RFC3010 ( NFSv4 ) section 2.4.1 :
"For example, if paths /a/b/c and /a/d/c refer to the same file, the
server SHOULD return the same filehandle for both path names traversals".
For NFSv4 problem should not be shown.
Quoted 8 lines
on NFSv3, most non-posix filesystems will fake it using something like
iunique().Comparing filehandles allows you to be certain that two files are the same if the filehandles are equal. If they are not equal, then that does not guarantee that the files are different, but then how else are you going to determine it?
Signature · 2 lines
Thanks, Vitaliy Gusev