Skip to contentSkip to conversation

Setting project quotas on special files

DISCUSSION Tiziano Müller 2 participants 6 messages Nov 10, 2013 – Nov 11, 2013

lore
Share & tools
lore
#0 lore raw

Hi everyone

It started with the following error:

dev # cp -al dvd dvd2
cp: cannot create hard link ‘dvd2’ to ‘dvd’: Invalid cross-device link

"dvd" is in this case a symlink (but I also did the test with special devices).

What I did was: copy that symlink from somewhere else, trying to turn on project quotas on a parent directoy of that "dev" directory which gives me:

xfs_quota: skipping special file /var/foo/dev/dvd

When I examine the inode using xfs_db I get for my "dvd" symlink:

core.projid_lo = 0
core.projid_hi = 0

When I create a new symlink "foo" (with the same uid+gid as "dvd") and examine it's inode using xfs_db:

core.projid_lo = 2398
core.projid_hi = 61

What I therefore seem to encounter is the problem mentioned in xfs_quota for the project subcommand:

An attempt to create a hard link to a file in the tree will only succeed if the project identi‐
fier matches the project identifier for the tree.

I could now use xfs_io to fix all special files once (since newly created ones will carry the project id directly), but why does "xfs_quota -x -c 'project -s test1' /var" skip special files if it is clearly possible and required to set the project id on them to have hardlinks working again within the project dir?

Thanks in advance,
best regards,
Tiziano

Signature · 11 lines
stepping stone GmbH
Neufeldstrasse 9
CH-3012 Bern
Telefon: +41 31 332 53 63
www.stepping-stone.ch
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#1 lore raw
Quoted 9 lines · Tiziano Müller

Hi everyone

It started with the following error:

dev # cp -al dvd dvd2
cp: cannot create hard link ‘dvd2’ to ‘dvd’: Invalid cross-device link

"dvd" is in this case a symlink (but I also did the test with special devices).

This definitely would fail if you trying to create hard links across volumes.

Quoted 6 lines

What I did was: copy that symlink from somewhere else, trying to turn on project quotas on a parent directoy of that "dev" directory which gives me:

xfs_quota: skipping special file /var/foo/dev/dvd

symlink files will be skipped and xfs_quota consider it as special file just like fifo, sock, etc...

If xfs_quota do check with follow symlinks, then the project id is potentially applied to files on another filesystem.

Quoted 15 lines

When I examine the inode using xfs_db I get for my "dvd" symlink:

core.projid_lo = 0
core.projid_hi = 0

When I create a new symlink "foo" (with the same uid+gid as "dvd") and examine it's inode using xfs_db:

core.projid_lo = 2398
core.projid_hi = 61

What I therefore seem to encounter is the problem mentioned in xfs_quota for the project subcommand:

Quoted 8 lines

An attempt to create a hard link to a file in the tree will only succeed if the project identi‐
fier matches the project identifier for the tree.

I could now use xfs_io to fix all special files once (since newly created ones will carry the project id directly), but why does "xfs_quota -x -c 'project -s test1' /var" skip special files if it is clearly possible and required to set the project id on them to have hardlinks working again within the project dir?

Sorry if I misunderstood your question, but the existing hard link files would
notbe skipped IMO. For the project subcommand, it means we could not create hardlinks cross different project trees.

Thanks,
-Jeff

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw

Hi Jeff

Sorry for top-posting, but it may be better to illustrate this with an isolated example:

I have a volume named "backup" mounted with option "pquota", then I do the following inside that mountpoint:

localhost backup # mkdir test
localhost backup # ln -s /invalid/location test/symlinkA
localhost backup # echo 42:/var/backup/test >> /etc/projects
localhost backup # echo test:42 >> /etc/projid
localhost backup # xfs_quota -x -c 'project -s test' /var/backup
Setting up project test (path /var/backup/test)...
xfs_quota: skipping special file /var/backup/test/symlinkA
Processed 1 (/etc/projects and cmdline) paths for project test with recursion depth infinite (-1). localhost backup # cp -al test/symlinkA test/symlinkB
cp: cannot create hard link "test/symlinkB" to "test/symlinkA": Invalid cross-device link localhost backup #

Creating a symlink after setting up project quotas in the same directory yields a different behaviour:

localhost backup # ln -s /invalid/location test/symlinkC localhost backup # cp -al test/symlinkC test/symlinkD localhost backup #

The following shows that a symlink alone (and not its target) can have a project id assigned and that the project id must be assigned to be able to create a hardlink of a symlink (no matter where it points to).

For each of the symlinks test/symlink{A,B} run stat to get the inode then use xfs_db on that inode to get the attributes:

localhost backup # xfs_db -r -c 'inode 4362' -c 'p' /dev/vdb1 | grep projid core.projid_lo = 0
core.projid_hi = 0
localhost backup # xfs_db -r -c 'inode 4363' -c 'p' /dev/vdb1 | grep projid core.projid_lo = 42
core.projid_hi = 0

Unfortunately xfs_io tries to follow the symlinks so it can not be used to set the project manually.

What works though is renaming the file:

localhost backup # mv test/symlinkA test/symlinkAtmp localhost backup # mv test/symlinkAtmp test/symlinkA localhost backup # stat test/symlinkA
File: ‘test/symlinkA’ -> ‘/invalid/location’
Size: 17 Blocks: 0 IO Block: 4096 symbolic link
Device: fe11h/65041d Inode: 4364 Links: 1
Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-11-10 15:49:44.870068178 +0100
Modify: 2013-11-10 15:49:44.870068178 +0100
Change: 2013-11-10 16:00:54.957571504 +0100
Birth: -
localhost backup # xfs_db -r -c 'inode 4364' -c 'p' /dev/vdb1 | grep projid
core.projid_lo = 42
core.projid_hi = 0
localhost backup # cp -al test/symlinkA test/symlinkB localhost backup #

Best regards,
Tiziano

Am Sonntag, den 10.11.2013, 02:46 -0800 schrieb Jeff Liu:

Quoted 59 lines · Tiziano Müller
Quoted 9 lines

Hi everyone

It started with the following error:

dev # cp -al dvd dvd2
cp: cannot create hard link ‘dvd2’ to ‘dvd’: Invalid cross-device link

"dvd" is in this case a symlink (but I also did the test with special devices).

This definitely would fail if you trying to create hard links across volumes.

Quoted 6 lines

What I did was: copy that symlink from somewhere else, trying to turn on project quotas on a parent directoy of that "dev" directory which gives me:

xfs_quota: skipping special file /var/foo/dev/dvd

symlink files will be skipped and xfs_quota consider it as special file just like fifo, sock, etc...

If xfs_quota do check with follow symlinks, then the project id is potentially applied to files on another filesystem.

Quoted 15 lines

When I examine the inode using xfs_db I get for my "dvd" symlink:

core.projid_lo = 0
core.projid_hi = 0

When I create a new symlink "foo" (with the same uid+gid as "dvd") and examine it's inode using xfs_db:

core.projid_lo = 2398
core.projid_hi = 61

What I therefore seem to encounter is the problem mentioned in xfs_quota for the project subcommand:

Quoted 8 lines

An attempt to create a hard link to a file in the tree will only succeed if the project identi‐
fier matches the project identifier for the tree.

I could now use xfs_io to fix all special files once (since newly created ones will carry the project id directly), but why does "xfs_quota -x -c 'project -s test1' /var" skip special files if it is clearly possible and required to set the project id on them to have hardlinks working again within the project dir?

Sorry if I misunderstood your question, but the existing hard link files would
notbe skipped IMO. For the project subcommand, it means we could not create hardlinks cross different project trees.

Thanks,
-Jeff

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Signature · 11 lines
stepping stone GmbH
Neufeldstrasse 9
CH-3012 Bern
Telefon: +41 31 332 53 63
www.stepping-stone.ch
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#3 lore raw

Hi Tiziano,

Quoted 9 lines · Tiziano Müller

Hi Jeff

Sorry for top-posting, but it may be better to illustrate this with an isolated example:

I have a volume named "backup" mounted with option "pquota", then I do the following inside that mountpoint:

Quoted 7 lines

localhost backup # mkdir test
localhost backup # ln -s /invalid/location test/symlinkA localhost backup # echo 42:/var/backup/test >> /etc/projects localhost backup # echo test:42 >> /etc/projid localhost backup # xfs_quota -x -c 'project -s test' /var/backup Setting up project test (path /var/backup/test)...
xfs_quota: skipping special file /var/backup/test/symlinkA

So the symlink file is skipped.

Quoted 3 lines

Processed 1 (/etc/projects and cmdline) paths for project test with recursion depth infinite (-1). localhost backup # cp -al test/symlinkA test/symlinkB
cp: cannot create hard link "test/symlinkB" to "test/symlinkA": Invalid cross-device link

This is expected as you attempting to create a hardlink file upon a symlink which is located at a different volume and the symlink was not inherited from the project id(42 in this case) at this stage(i.e, it's projid is 0 as a default projid, which is different to 42).

This restriction can be got from xfs_vn_link():

/*
* If we are using project inheritance, we only allow hard link
* creation in our tree when the project IDs are the same; else
* the tree quota mechanism could be circumvented.
*/
if (unlikely((tdp->i_d.di_flags & XFS_DIFLAG_PROJINHERIT) &&

     (xfs_get_projid(tdp) != xfs_get_projid(sip)))) {

error = XFS_ERROR(EXDEV);
goto error_return;
}

Quoted 6 lines

localhost backup #

Creating a symlink after setting up project quotas in the same directory yields a different behaviour:

localhost backup # ln -s /invalid/location test/symlinkC

So a symlink file is created and it is inherited the projid in this case, because symlink file quota space is accounted according to it's path length. IOWs, it would be reflect on xfs_quota -xc 'report -[b|h]' if you create it with a longer path name exceeds the inline inode space.

Quoted 1 line

localhost backup # cp -al test/symlinkC test/symlinkD

Therefore create a hardlink from this symlink file with an inherited projid is desired to succeed.

Quoted 9 lines

localhost backup #

The following shows that a symlink alone (and not its target) can have a project id assigned and that the project id must be assigned to be able to create a hardlink of a symlink (no matter where it points to).

For each of the symlinks test/symlink{A,B} run stat to get the inode then use xfs_db on that inode to get the attributes:

Quoted 6 lines

localhost backup # xfs_db -r -c 'inode 4362' -c 'p' /dev/vdb1 | grep projid core.projid_lo = 0
core.projid_hi = 0
localhost backup # xfs_db -r -c 'inode 4363' -c 'p' /dev/vdb1 | grep projid core.projid_lo = 42
core.projid_hi = 0

Just as above mentioned...

Quoted 2 lines

Unfortunately xfs_io tries to follow the symlinks so it can not be used to set the project manually.

Well, why you want to set project quota via xfs_io manually? :-P.

Quoted 5 lines

What works though is renaming the file:

localhost backup # mv test/symlinkA test/symlinkAtmp localhost backup # mv test/symlinkAtmp test/symlinkA

rename(2) will create a new dentry as well as an inode, hence it would inherit the projid.

Quoted 7 lines

localhost backup # stat test/symlinkA
File: ‘test/symlinkA’ -> ‘/invalid/location’
Size: 17 Blocks: 0 IO Block: 4096 symbolic link
Device: fe11h/65041d Inode: 4364 Links: 1
Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-11-10 15:49:44.870068178 +0100
Modify: 2013-11-10 15:49:44.870068178 +0100

Quoted 8 lines

Change: 2013-11-10 16:00:54.957571504 +0100
Birth: -
localhost backup # xfs_db -r -c 'inode 4364' -c 'p' /dev/vdb1 | grep projid
core.projid_lo = 42
core.projid_hi = 0
localhost backup # cp -al test/symlinkA test/symlinkB localhost backup #

In terms of project quota anyway, hardlink file is only accounted once for the owner of the inode. It's welcome to report back if you observed different behaviour.

Thanks,
-Jeff

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#4 lore raw

Hi Jeff

Am Montag, den 11.11.2013, 14:43 +0800 schrieb Jeff Liu:

Quoted 19 lines

Hi Tiziano,

Quoted 9 lines · Tiziano Müller

Hi Jeff

Sorry for top-posting, but it may be better to illustrate this with an isolated example:

I have a volume named "backup" mounted with option "pquota", then I do the following inside that mountpoint:

Quoted 7 lines

localhost backup # mkdir test
localhost backup # ln -s /invalid/location test/symlinkA localhost backup # echo 42:/var/backup/test >> /etc/projects localhost backup # echo test:42 >> /etc/projid localhost backup # xfs_quota -x -c 'project -s test' /var/backup Setting up project test (path /var/backup/test)...
xfs_quota: skipping special file /var/backup/test/symlinkA

Quoted 7 lines

So the symlink file is skipped.

Quoted 3 lines

Processed 1 (/etc/projects and cmdline) paths for project test with recursion depth infinite (-1). localhost backup # cp -al test/symlinkA test/symlinkB
cp: cannot create hard link "test/symlinkB" to "test/symlinkA": Invalid cross-device link

This is expected as you attempting to create a hardlink file upon a symlink which is located at a different volume

No, 'cp -al' does not follow the symlink (which is invalid in this example btw) but tries to create a hardlink for that symlink.

Quoted 3 lines

and the symlink was not inherited
from the project id(42 in this case) at this stage(i.e, it's projid is 0 as a default projid, which is different to 42).

Yes.

Quoted 44 lines

This restriction can be got from xfs_vn_link():

/*
* If we are using project inheritance, we only allow hard link
* creation in our tree when the project IDs are the same; else
* the tree quota mechanism could be circumvented.
*/
if (unlikely((tdp->i_d.di_flags & XFS_DIFLAG_PROJINHERIT) &&

     (xfs_get_projid(tdp) != xfs_get_projid(sip)))) {
(quoted diff) +0 -0
 	error = XFS_ERROR(EXDEV);
 	goto error_return;
 }
Quoted 7 lines

localhost backup #

Creating a symlink after setting up project quotas in the same directory yields a different behaviour:

localhost backup # ln -s /invalid/location test/symlinkC

So a symlink file is created and it is inherited the projid in this case, because symlink file quota space is accounted according to it's path length. IOWs, it would be reflect on xfs_quota -xc 'report -[b|h]' if you create it with a longer path name exceeds the inline inode space.

Quoted 1 line

localhost backup # cp -al test/symlinkC test/symlinkD

Therefore create a hardlink from this symlink file with an inherited projid is desired to succeed.

Quoted 9 lines

localhost backup #

The following shows that a symlink alone (and not its target) can have a project id assigned and that the project id must be assigned to be able to create a hardlink of a symlink (no matter where it points to).

For each of the symlinks test/symlink{A,B} run stat to get the inode then use xfs_db on that inode to get the attributes:

Quoted 6 lines

localhost backup # xfs_db -r -c 'inode 4362' -c 'p' /dev/vdb1 | grep projid core.projid_lo = 0
core.projid_hi = 0
localhost backup # xfs_db -r -c 'inode 4363' -c 'p' /dev/vdb1 | grep projid core.projid_lo = 42
core.projid_hi = 0

Quoted 5 lines

Just as above mentioned...

Quoted 2 lines

Unfortunately xfs_io tries to follow the symlinks so it can not be used to set the project manually.

Well, why you want to set project quota via xfs_io manually? :-P.

To set the project id for special files which did not get the project id set during "xfs_quota -x -c 'project -s test'", which was leaving my tree in a somewhat inconsistent state wrt the project id.

As Dave wrote me on IRC, the following works (I only tested it, credit goes to him):

1. find the inode of all special files where the project id was not set
2. determine the project id to be set by recording their parent directorie's project id. You will end up with a tuple (for 16 bit project ids): (inode, projid_lo), resp. a triple (for 32 bit project ids): (inode, projid_lo, projid_hi)
3. umount the volume
4. use 'xfs_db -x -c "inode <inode>" -c "write core.projid_lo <projid_lo>" -c "write core.projid_hi <projid_hi>" <dev>' to set the project id on those inodes manually
5. mount the volume but without "pquota" and umount again (this will trigger a quotacheck on the next mount with pquota)
6. mount the volume with the "pquota" option again

As for the "why can xfs_quota not set the project id on special files":
it does it via ioctl(2), which needs a file handle, which is retrieved via open(2), which "follows" the symlinks/device files.

Quoted 24 lines
Quoted 6 lines

What works though is renaming the file:

localhost backup # mv test/symlinkA test/symlinkAtmp localhost backup # mv test/symlinkAtmp test/symlinkA

rename(2) will create a new dentry as well as an inode, hence it would inherit the projid.

Quoted 7 lines

localhost backup # stat test/symlinkA
File: ‘test/symlinkA’ -> ‘/invalid/location’
Size: 17 Blocks: 0 IO Block: 4096 symbolic link
Device: fe11h/65041d Inode: 4364 Links: 1
Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-11-10 15:49:44.870068178 +0100
Modify: 2013-11-10 15:49:44.870068178 +0100

Quoted 8 lines

Change: 2013-11-10 16:00:54.957571504 +0100
Birth: -
localhost backup # xfs_db -r -c 'inode 4364' -c 'p' /dev/vdb1 | grep projid
core.projid_lo = 42
core.projid_hi = 0
localhost backup # cp -al test/symlinkA test/symlinkB localhost backup #

Quoted 6 lines

In terms of project quota anyway, hardlink file is only accounted once for the owner of the inode. It's welcome to report back if you observed different behaviour.

Thanks,
-Jeff

best regards,
Tiziano

Signature · 11 lines
stepping stone GmbH
Neufeldstrasse 9
CH-3012 Bern
Telefon: +41 31 332 53 63
www.stepping-stone.ch
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#5 lore raw
Quoted 85 lines · Tiziano Müller

Hi Jeff

Am Montag, den 11.11.2013, 14:43 +0800 schrieb Jeff Liu:

Quoted 19 lines

Hi Tiziano,

Quoted 9 lines · Tiziano Müller

Hi Jeff

Sorry for top-posting, but it may be better to illustrate this with an isolated example:

I have a volume named "backup" mounted with option "pquota", then I do the following inside that mountpoint:

Quoted 7 lines

localhost backup # mkdir test
localhost backup # ln -s /invalid/location test/symlinkA localhost backup # echo 42:/var/backup/test >> /etc/projects localhost backup # echo test:42 >> /etc/projid localhost backup # xfs_quota -x -c 'project -s test' /var/backup Setting up project test (path /var/backup/test)...
xfs_quota: skipping special file /var/backup/test/symlinkA

Quoted 7 lines

So the symlink file is skipped.

Quoted 3 lines

Processed 1 (/etc/projects and cmdline) paths for project test with recursion depth infinite (-1). localhost backup # cp -al test/symlinkA test/symlinkB
cp: cannot create hard link "test/symlinkB" to "test/symlinkA": Invalid cross-device link

This is expected as you attempting to create a hardlink file upon a symlink which is located at a different volume

No, 'cp -al' does not follow the symlink (which is invalid in this example btw) but tries to create a hardlink for that symlink.

Quoted 3 lines

and the symlink was not inherited
from the project id(42 in this case) at this stage(i.e, it's projid is 0 as a default projid, which is different to 42).

Yes.

Quoted 46 lines

This restriction can be got from xfs_vn_link():

/*
* If we are using project inheritance, we only allow hard link
* creation in our tree when the project IDs are the same; else
* the tree quota mechanism could be circumvented.
*/
if (unlikely((tdp->i_d.di_flags & XFS_DIFLAG_PROJINHERIT) &&

     (xfs_get_projid(tdp) != xfs_get_projid(sip)))) {
(quoted diff) +0 -0
 	error = XFS_ERROR(EXDEV);
 	goto error_return;
 }
Quoted 7 lines

localhost backup #

Creating a symlink after setting up project quotas in the same directory yields a different behaviour:

localhost backup # ln -s /invalid/location test/symlinkC

So a symlink file is created and it is inherited the projid in this case, because symlink file quota space is accounted according to it's path length. IOWs, it would be reflect on xfs_quota -xc 'report -[b|h]' if you create it with a longer path name exceeds the inline inode space.

Quoted 1 line

localhost backup # cp -al test/symlinkC test/symlinkD

Therefore create a hardlink from this symlink file with an inherited projid is desired to succeed.

Quoted 9 lines

localhost backup #

The following shows that a symlink alone (and not its target) can have a project id assigned and that the project id must be assigned to be able to create a hardlink of a symlink (no matter where it points to).

For each of the symlinks test/symlink{A,B} run stat to get the inode then use xfs_db on that inode to get the attributes:

Quoted 6 lines

localhost backup # xfs_db -r -c 'inode 4362' -c 'p' /dev/vdb1 | grep projid core.projid_lo = 0
core.projid_hi = 0
localhost backup # xfs_db -r -c 'inode 4363' -c 'p' /dev/vdb1 | grep projid core.projid_lo = 42
core.projid_hi = 0

Just as above mentioned...

Quoted 7 lines
Quoted 3 lines
Quoted 2 lines

Unfortunately xfs_io tries to follow the symlinks so it can not be used to set the project manually.

Well, why you want to set project quota via xfs_io manually? :-P.

To set the project id for special files which did not get the project id set during "xfs_quota -x -c 'project -s test'", which was leaving my tree in a somewhat inconsistent state wrt the project id.

Oh, now I can understand that you want to set projid to special files.

Quoted 17 lines

As Dave wrote me on IRC, the following works (I only tested it, credit goes to him):

1. find the inode of all special files where the project id was not set
2. determine the project id to be set by recording their parent directorie's project id. You will end up with a tuple (for 16 bit project ids): (inode, projid_lo), resp. a triple (for 32 bit project ids): (inode, projid_lo, projid_hi)
3. umount the volume
4. use 'xfs_db -x -c "inode <inode>" -c "write core.projid_lo <projid_lo>" -c "write core.projid_hi <projid_hi>" <dev>' to set the project id on those inodes manually
5. mount the volume but without "pquota" and umount again (this will trigger a quotacheck on the next mount with pquota)
6. mount the volume with the "pquota" option again

Quoted 3 lines

As for the "why can xfs_quota not set the project id on special files":
it does it via ioctl(2), which needs a file handle, which is retrieved via open(2), which "follows" the symlinks/device files.

Thank for letting me know this as well, I have been blocked for logging into IRC these days due to network issue.

Thanks,
-Jeff

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs