Setting project quotas on special files
Hi everyone
It started with the following error:
dev # cp -al dvd dvd2
cp: cannot create hard link ‘dvd2’ to ‘dvd’: Invalid cross-device link
"dvd" is in this case a symlink (but I also did the test with special devices).
What I did was: copy that symlink from somewhere else, trying to turn on project quotas on a parent directoy of that "dev" directory which gives me:
xfs_quota: skipping special file /var/foo/dev/dvd
When I examine the inode using xfs_db I get for my "dvd" symlink:
core.projid_lo = 0
core.projid_hi = 0
When I create a new symlink "foo" (with the same uid+gid as "dvd") and examine it's inode using xfs_db:
core.projid_lo = 2398
core.projid_hi = 61
What I therefore seem to encounter is the problem mentioned in xfs_quota for the project subcommand:
An attempt to create a hard link to a file in the tree will only succeed if the project identi‐
fier matches the project identifier for the tree.
I could now use xfs_io to fix all special files once (since newly created ones will carry the project id directly), but why does "xfs_quota -x -c 'project -s test1' /var" skip special files if it is clearly possible and required to set the project id on them to have hardlinks working again within the project dir?
Thanks in advance,
best regards,
Tiziano
Signature · 11 lines
stepping stone GmbH Neufeldstrasse 9 CH-3012 Bern Telefon: +41 31 332 53 63 www.stepping-stone.ch [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 9 lines · Tiziano Müller
Hi everyone
It started with the following error:
dev # cp -al dvd dvd2
cp: cannot create hard link ‘dvd2’ to ‘dvd’: Invalid cross-device link"dvd" is in this case a symlink (but I also did the test with special devices).
This definitely would fail if you trying to create hard links across volumes.
Quoted 6 lines
What I did was: copy that symlink from somewhere else, trying to turn on project quotas on a parent directoy of that "dev" directory which gives me:
xfs_quota: skipping special file/var/foo/dev/dvd
symlink files will be skipped and xfs_quota consider it as special file just like fifo, sock, etc...
If xfs_quota do check with follow symlinks, then the project id is potentially applied to files on another filesystem.
Quoted 15 lines
When I examine the inode using
xfs_dbI get for my "dvd" symlink:core.projid_lo = 0
core.projid_hi = 0When I create a new symlink "foo" (with the same uid+gid as "dvd") and examine it's inode using
xfs_db:core.projid_lo = 2398
core.projid_hi = 61What I therefore seem to encounter is the problem mentioned in
xfs_quotafor the project subcommand:
Quoted 8 lines
An attempt to create a hard link to a file in the tree will only succeed if the project identi‐
fier matches the project identifier for the tree.I could now use
xfs_ioto fix all special files once (since newly created ones will carry the project id directly), but why does "xfs_quota-x -c 'project -s test1' /var" skip special files if it is clearly possible and required to set the project id on them to have hardlinks working again within the project dir?
Sorry if I misunderstood your question, but the existing hard link files would
notbe skipped IMO. For the project subcommand, it means we could not create hardlinks cross different project trees.
Thanks,
-Jeff
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Hi Jeff
Sorry for top-posting, but it may be better to illustrate this with an isolated example:
I have a volume named "backup" mounted with option "pquota", then I do the following inside that mountpoint:
localhost backup # mkdir test
localhost backup # ln -s /invalid/location test/symlinkA
localhost backup # echo 42:/var/backup/test >> /etc/projects
localhost backup # echo test:42 >> /etc/projid
localhost backup # xfs_quota -x -c 'project -s test' /var/backup
Setting up project test (path /var/backup/test)...xfs_quota: skipping special file /var/backup/test/symlinkA
Processed 1 (/etc/projects and cmdline) paths for project test with recursion depth infinite (-1). localhost backup # cp -al test/symlinkA test/symlinkB
cp: cannot create hard link "test/symlinkB" to "test/symlinkA": Invalid cross-device link localhost backup #
Creating a symlink after setting up project quotas in the same directory yields a different behaviour:
localhost backup # ln -s /invalid/location test/symlinkC localhost backup # cp -al test/symlinkC test/symlinkD localhost backup #
The following shows that a symlink alone (and not its target) can have a project id assigned and that the project id must be assigned to be able to create a hardlink of a symlink (no matter where it points to).
For each of the symlinks test/symlink{A,B} run stat to get the inode then use xfs_db on that inode to get the attributes:
localhost backup # xfs_db -r -c 'inode 4362' -c 'p' /dev/vdb1 | grep projid core.projid_lo = 0
core.projid_hi = 0
localhost backup # xfs_db -r -c 'inode 4363' -c 'p' /dev/vdb1 | grep projid core.projid_lo = 42
core.projid_hi = 0
Unfortunately xfs_io tries to follow the symlinks so it can not be used to set the project manually.
What works though is renaming the file:
localhost backup # mv test/symlinkA test/symlinkAtmp localhost backup # mv test/symlinkAtmp test/symlinkA localhost backup # stat test/symlinkA
File: ‘test/symlinkA’ -> ‘/invalid/location’
Size: 17 Blocks: 0 IO Block: 4096 symbolic link
Device: fe11h/65041d Inode: 4364 Links: 1
Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-11-10 15:49:44.870068178 +0100
Modify: 2013-11-10 15:49:44.870068178 +0100
Change: 2013-11-10 16:00:54.957571504 +0100
Birth: -
localhost backup # xfs_db -r -c 'inode 4364' -c 'p' /dev/vdb1 | grep projid
core.projid_lo = 42
core.projid_hi = 0
localhost backup # cp -al test/symlinkA test/symlinkB localhost backup #
Best regards,
Tiziano
Am Sonntag, den 10.11.2013, 02:46 -0800 schrieb Jeff Liu:
Quoted 59 lines · Tiziano Müller
Quoted 9 lines
Hi everyone
It started with the following error:
dev # cp -al dvd dvd2
cp: cannot create hard link ‘dvd2’ to ‘dvd’: Invalid cross-device link"dvd" is in this case a symlink (but I also did the test with special devices).
This definitely would fail if you trying to create hard links across volumes.
Quoted 6 lines
What I did was: copy that symlink from somewhere else, trying to turn on project quotas on a parent directoy of that "dev" directory which gives me:
xfs_quota: skipping special file/var/foo/dev/dvdsymlink files will be skipped and
xfs_quotaconsider it as special file just like fifo, sock, etc...If
xfs_quotado check with follow symlinks, then the project id is potentially applied to files on another filesystem.Quoted 15 lines
When I examine the inode using
xfs_dbI get for my "dvd" symlink:core.projid_lo = 0
core.projid_hi = 0When I create a new symlink "foo" (with the same uid+gid as "dvd") and examine it's inode using
xfs_db:core.projid_lo = 2398
core.projid_hi = 61What I therefore seem to encounter is the problem mentioned in
xfs_quotafor the project subcommand:Quoted 8 lines
An attempt to create a hard link to a file in the tree will only succeed if the project identi‐
fier matches the project identifier for the tree.I could now use
xfs_ioto fix all special files once (since newly created ones will carry the project id directly), but why does "xfs_quota-x -c 'project -s test1' /var" skip special files if it is clearly possible and required to set the project id on them to have hardlinks working again within the project dir?Sorry if I misunderstood your question, but the existing hard link files would
notbe skipped IMO. For the project subcommand, it means we could not create hardlinks cross different project trees.Thanks,
-Jeff_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Signature · 11 lines
stepping stone GmbH Neufeldstrasse 9 CH-3012 Bern Telefon: +41 31 332 53 63 www.stepping-stone.ch [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Hi Tiziano,
Quoted 9 lines · Tiziano Müller
Hi Jeff
Sorry for top-posting, but it may be better to illustrate this with an isolated example:
I have a volume named "backup" mounted with option "pquota", then I do the following inside that mountpoint:
Quoted 7 lines
localhost backup # mkdir test
localhost backup # ln -s /invalid/location test/symlinkA localhost backup # echo 42:/var/backup/test >>/etc/projectslocalhost backup # echo test:42 >>/etc/projidlocalhost backup #xfs_quota-x -c 'project -s test'/var/backupSetting up project test (path/var/backup/test)...xfs_quota: skipping special file/var/backup/test/symlinkA
So the symlink file is skipped.
Quoted 3 lines
Processed 1 (
/etc/projectsand cmdline) paths for project test with recursion depth infinite (-1). localhost backup # cp -al test/symlinkA test/symlinkB
cp: cannot create hard link "test/symlinkB" to "test/symlinkA": Invalid cross-device link
This is expected as you attempting to create a hardlink file upon a symlink which is located at a different volume and the symlink was not inherited from the project id(42 in this case) at this stage(i.e, it's projid is 0 as a default projid, which is different to 42).
This restriction can be got from xfs_vn_link():
/*
* If we are using project inheritance, we only allow hard link
* creation in our tree when the project IDs are the same; else
* the tree quota mechanism could be circumvented.
*/
if (unlikely((tdp->i_d.di_flags & XFS_DIFLAG_PROJINHERIT) &&
(xfs_get_projid(tdp) != xfs_get_projid(sip)))) { error = XFS_ERROR(EXDEV);
goto error_return;
}
Quoted 6 lines
localhost backup #
Creating a symlink after setting up project quotas in the same directory yields a different behaviour:
localhost backup # ln -s /invalid/location test/symlinkC
So a symlink file is created and it is inherited the projid in this case, because symlink file quota space is accounted according to it's path length. IOWs, it would be reflect on xfs_quota -xc 'report -[b|h]' if you create it with a longer path name exceeds the inline inode space.
Quoted 1 line
localhost backup # cp -al test/symlinkC test/symlinkD
Therefore create a hardlink from this symlink file with an inherited projid is desired to succeed.
Quoted 9 lines
localhost backup #
The following shows that a symlink alone (and not its target) can have a project id assigned and that the project id must be assigned to be able to create a hardlink of a symlink (no matter where it points to).
For each of the symlinks test/symlink{A,B} run
statto get the inode then usexfs_dbon that inode to get the attributes:
Quoted 6 lines
localhost backup #
xfs_db-r -c 'inode 4362' -c 'p'/dev/vdb1| grep projid core.projid_lo = 0
core.projid_hi = 0
localhost backup #xfs_db-r -c 'inode 4363' -c 'p'/dev/vdb1| grep projid core.projid_lo = 42
core.projid_hi = 0
Just as above mentioned...
Quoted 2 lines
Unfortunately
xfs_iotries to follow the symlinks so it can not be used to set the project manually.
Well, why you want to set project quota via xfs_io manually? :-P.
Quoted 5 lines
What works though is renaming the file:
localhost backup # mv test/symlinkA test/symlinkAtmp localhost backup # mv test/symlinkAtmp test/symlinkA
rename(2) will create a new dentry as well as an inode, hence it would inherit the projid.
Quoted 7 lines
localhost backup # stat test/symlinkA
File: ‘test/symlinkA’ -> ‘/invalid/location’
Size: 17 Blocks: 0 IO Block: 4096 symbolic link
Device: fe11h/65041d Inode: 4364 Links: 1
Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-11-10 15:49:44.870068178 +0100
Modify: 2013-11-10 15:49:44.870068178 +0100
Quoted 8 lines
Change: 2013-11-10 16:00:54.957571504 +0100
Birth: -
localhost backup #xfs_db-r -c 'inode 4364' -c 'p'/dev/vdb1| grep projid
core.projid_lo = 42
core.projid_hi = 0
localhost backup # cp -al test/symlinkA test/symlinkB localhost backup #
In terms of project quota anyway, hardlink file is only accounted once for the owner of the inode. It's welcome to report back if you observed different behaviour.
Thanks,
-Jeff
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Hi Jeff
Am Montag, den 11.11.2013, 14:43 +0800 schrieb Jeff Liu:
Quoted 19 lines
Hi Tiziano,
Quoted 9 lines · Tiziano Müller
Hi Jeff
Sorry for top-posting, but it may be better to illustrate this with an isolated example:
I have a volume named "backup" mounted with option "pquota", then I do the following inside that mountpoint:
Quoted 7 lines
localhost backup # mkdir test
localhost backup # ln -s /invalid/location test/symlinkA localhost backup # echo 42:/var/backup/test >>/etc/projectslocalhost backup # echo test:42 >>/etc/projidlocalhost backup #xfs_quota-x -c 'project -s test'/var/backupSetting up project test (path/var/backup/test)...xfs_quota: skipping special file/var/backup/test/symlinkA
Quoted 7 lines
So the symlink file is skipped.
Quoted 3 lines
Processed 1 (
/etc/projectsand cmdline) paths for project test with recursion depth infinite (-1). localhost backup # cp -al test/symlinkA test/symlinkB
cp: cannot create hard link "test/symlinkB" to "test/symlinkA": Invalid cross-device linkThis is expected as you attempting to create a hardlink file upon a symlink which is located at a different volume
No, 'cp -al' does not follow the symlink (which is invalid in this example btw) but tries to create a hardlink for that symlink.
Quoted 3 lines
and the symlink was not inherited
from the project id(42 in this case) at thisstage(i.e, it's projid is 0 as a default projid, which is different to 42).
Yes.
Quoted 44 lines
This restriction can be got from
xfs_vn_link():/*
* If we are using project inheritance, we only allow hard link
* creation in our tree when the project IDs are the same; else
* the tree quota mechanism could be circumvented.
*/
if (unlikely((tdp->i_d.di_flags &XFS_DIFLAG_PROJINHERIT) &&(xfs_get_projid(tdp) != xfs_get_projid(sip)))) {(quoted diff) +0
-0error = XFS_ERROR(EXDEV); goto error_return; }Quoted 7 lines
localhost backup #
Creating a symlink after setting up project quotas in the same directory yields a different behaviour:
localhost backup # ln -s /invalid/location test/symlinkC
So a symlink file is created and it is inherited the projid in this case, because symlink file quota space is accounted according to it's path length. IOWs, it would be reflect on
xfs_quota-xc 'report -[b|h]' if you create it with a longer path name exceeds the inline inode space.Quoted 1 line
localhost backup # cp -al test/symlinkC test/symlinkD
Therefore create a hardlink from this symlink file with an inherited projid is desired to succeed.
Quoted 9 lines
localhost backup #
The following shows that a symlink alone (and not its target) can have a project id assigned and that the project id must be assigned to be able to create a hardlink of a symlink (no matter where it points to).
For each of the symlinks test/symlink{A,B} run
statto get the inode then usexfs_dbon that inode to get the attributes:Quoted 6 lines
localhost backup #
xfs_db-r -c 'inode 4362' -c 'p'/dev/vdb1| grep projid core.projid_lo = 0
core.projid_hi = 0
localhost backup #xfs_db-r -c 'inode 4363' -c 'p'/dev/vdb1| grep projid core.projid_lo = 42
core.projid_hi = 0
Quoted 5 lines
Just as above mentioned...
Quoted 2 lines
Unfortunately
xfs_iotries to follow the symlinks so it can not be used to set the project manually.Well, why you want to set project quota via
xfs_iomanually? :-P.
To set the project id for special files which did not get the project id set during "xfs_quota -x -c 'project -s test'", which was leaving my tree in a somewhat inconsistent state wrt the project id.
As Dave wrote me on IRC, the following works (I only tested it, credit goes to him):
1. find the inode of all special files where the project id was not set
2. determine the project id to be set by recording their parent directorie's project id. You will end up with a tuple (for 16 bit project ids): (inode, projid_lo), resp. a triple (for 32 bit project ids): (inode, projid_lo, projid_hi)
3. umount the volume
4. use 'xfs_db -x -c "inode <inode>" -c "write core.projid_lo <projid_lo>" -c "write core.projid_hi <projid_hi>" <dev>' to set the project id on those inodes manually
5. mount the volume but without "pquota" and umount again (this will trigger a quotacheck on the next mount with pquota)
6. mount the volume with the "pquota" option again
As for the "why can xfs_quota not set the project id on special files":
it does it via ioctl(2), which needs a file handle, which is retrieved via open(2), which "follows" the symlinks/device files.
Quoted 24 lines
Quoted 6 lines
What works though is renaming the file:
localhost backup # mv test/symlinkA test/symlinkAtmp localhost backup # mv test/symlinkAtmp test/symlinkA
rename(2) will create a new dentry as well as an inode, hence it would inherit the projid.
Quoted 7 lines
localhost backup # stat test/symlinkA
File: ‘test/symlinkA’ -> ‘/invalid/location’
Size: 17 Blocks: 0 IO Block: 4096 symbolic link
Device: fe11h/65041d Inode: 4364 Links: 1
Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-11-10 15:49:44.870068178 +0100
Modify: 2013-11-10 15:49:44.870068178 +0100Quoted 8 lines
Change: 2013-11-10 16:00:54.957571504 +0100
Birth: -
localhost backup #xfs_db-r -c 'inode 4364' -c 'p'/dev/vdb1| grep projid
core.projid_lo = 42
core.projid_hi = 0
localhost backup # cp -al test/symlinkA test/symlinkB localhost backup #
Quoted 6 lines
In terms of project quota anyway, hardlink file is only accounted once for the owner of the inode. It's welcome to report back if you observed different behaviour.
Thanks,
-Jeff
best regards,
Tiziano
Signature · 11 lines
stepping stone GmbH Neufeldstrasse 9 CH-3012 Bern Telefon: +41 31 332 53 63 www.stepping-stone.ch [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 85 lines · Tiziano Müller
Hi Jeff
Am Montag, den 11.11.2013, 14:43 +0800 schrieb Jeff Liu:
Quoted 19 lines
Hi Tiziano,
Quoted 9 lines · Tiziano Müller
Hi Jeff
Sorry for top-posting, but it may be better to illustrate this with an isolated example:
I have a volume named "backup" mounted with option "pquota", then I do the following inside that mountpoint:
Quoted 7 lines
localhost backup # mkdir test
localhost backup # ln -s /invalid/location test/symlinkA localhost backup # echo 42:/var/backup/test >>/etc/projectslocalhost backup # echo test:42 >>/etc/projidlocalhost backup #xfs_quota-x -c 'project -s test'/var/backupSetting up project test (path/var/backup/test)...xfs_quota: skipping special file/var/backup/test/symlinkAQuoted 7 lines
So the symlink file is skipped.
Quoted 3 lines
Processed 1 (
/etc/projectsand cmdline) paths for project test with recursion depth infinite (-1). localhost backup # cp -al test/symlinkA test/symlinkB
cp: cannot create hard link "test/symlinkB" to "test/symlinkA": Invalid cross-device linkThis is expected as you attempting to create a hardlink file upon a symlink which is located at a different volume
No, 'cp -al' does not follow the symlink (which is invalid in this example btw) but tries to create a hardlink for that symlink.
Quoted 3 lines
and the symlink was not inherited
from the project id(42 in this case) at thisstage(i.e, it's projid is 0 as a default projid, which is different to 42).Yes.
Quoted 46 lines
This restriction can be got from
xfs_vn_link():/*
* If we are using project inheritance, we only allow hard link
* creation in our tree when the project IDs are the same; else
* the tree quota mechanism could be circumvented.
*/
if (unlikely((tdp->i_d.di_flags &XFS_DIFLAG_PROJINHERIT) &&(xfs_get_projid(tdp) != xfs_get_projid(sip)))) {(quoted diff) +0
-0error = XFS_ERROR(EXDEV); goto error_return; }Quoted 7 lines
localhost backup #
Creating a symlink after setting up project quotas in the same directory yields a different behaviour:
localhost backup # ln -s /invalid/location test/symlinkC
So a symlink file is created and it is inherited the projid in this case, because symlink file quota space is accounted according to it's path length. IOWs, it would be reflect on
xfs_quota-xc 'report -[b|h]' if you create it with a longer path name exceeds the inline inode space.Quoted 1 line
localhost backup # cp -al test/symlinkC test/symlinkD
Therefore create a hardlink from this symlink file with an inherited projid is desired to succeed.
Quoted 9 lines
localhost backup #
The following shows that a symlink alone (and not its target) can have a project id assigned and that the project id must be assigned to be able to create a hardlink of a symlink (no matter where it points to).
For each of the symlinks test/symlink{A,B} run
statto get the inode then usexfs_dbon that inode to get the attributes:Quoted 6 lines
localhost backup #
xfs_db-r -c 'inode 4362' -c 'p'/dev/vdb1| grep projid core.projid_lo = 0
core.projid_hi = 0
localhost backup #xfs_db-r -c 'inode 4363' -c 'p'/dev/vdb1| grep projid core.projid_lo = 42
core.projid_hi = 0Just as above mentioned...
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
Unfortunately
xfs_iotries to follow the symlinks so it can not be used to set the project manually.Well, why you want to set project quota via
xfs_iomanually? :-P.To set the project id for special files which did not get the project id set during "
xfs_quota-x -c 'project -s test'", which was leaving my tree in a somewhat inconsistent state wrt the project id.
Oh, now I can understand that you want to set projid to special files.
Quoted 17 lines
As Dave wrote me on IRC, the following works (I only tested it, credit goes to him):
1. find the inode of all special files where the project id was not set
2. determine the project id to be set by recording their parent directorie's project id. You will end up with a tuple (for 16 bit project ids): (inode,projid_lo), resp. a triple (for 32 bit project ids): (inode,projid_lo,projid_hi)
3. umount the volume
4. use 'xfs_db-x -c "inode <inode>" -c "write core.projid_lo <projid_lo>" -c "write core.projid_hi <projid_hi>" <dev>' to set the project id on those inodes manually
5. mount the volume but without "pquota" and umount again (this will trigger a quotacheck on the next mount with pquota)
6. mount the volume with the "pquota" option again
Quoted 3 lines
As for the "why can
xfs_quotanot set the project id on special files":
it does it via ioctl(2), which needs a file handle, which is retrieved via open(2), which "follows" the symlinks/device files.
Thank for letting me know this as well, I have been blocked for logging into IRC these days due to network issue.
Thanks,
-Jeff
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs