Skip to contentSkip to conversation

RFC: permit link(2) to work across --bind mounts ?

PATCH Mark Lord 7 participants 19 messages Dec 18, 2007 – Dec 30, 2007 VFS

lore
Share & tools
lore

Mark Lord

author
#0 lore raw

Why does link(2) not support hard-linking across bind mount points of the same underlying filesystem ?

Is it as simple as something like this patch below (minus the printk)? Not likely, but then I'm not a filesystem guru.

???

fs/namei.c +0 -0
@@ -2398,8 +2398,11 @@
	if (error)
		goto out;
	error = -EXDEV;

- if (old_nd.mnt != nd.mnt)
- goto out_release;
+ if (old_nd.mnt != nd.mnt) {
+ if (old_nd.mnt->mnt_sb != nd.mnt->mnt_sb)
+ goto out_release;
+ printk("sys_linkat: old_nd.mnt != nd.mnt, but sb is the same. Continuing..\n");
+ }

	new_dentry = lookup_create(&nd, 0);
	error = PTR_ERR(new_dentry);
	if (IS_ERR(new_dentry))
author
#1 lore raw
Quoted 24 lines · Mark Lord

Why does link(2) not support hard-linking across bind mount points of the same underlying filesystem ?

Is it as simple as something like this patch below (minus the printk)? Not likely, but then I'm not a filesystem guru.

???

fs/namei.c 2007-12-18 17:37:04.000000000 -0500 renamed from fs/namei.c 2007-12-15 12:33:13.000000000 -0500 +5 -2
@@ -2398,8 +2398,11 @@
    if (error)
        goto out;
    error = -EXDEV;
-    if (old_nd.mnt != nd.mnt)
-        goto out_release;
+    if (old_nd.mnt != nd.mnt) {
+        if (old_nd.mnt->mnt_sb != nd.mnt->mnt_sb)
+            goto out_release;
+        printk("sys_linkat: old_nd.mnt != nd.mnt, but sb is the same. 
 Continuing..\n");
+    }
    new_dentry = lookup_create(&nd, 0);
    error = PTR_ERR(new_dentry);
Quoted 1 line
    if (IS_ERR(new_dentry))

..

The patch seems to work for me after some light testing on ext3 here. But I have no idea about other filesystems, or if there's some kind of race condition or something. Or maybe we just never bothered ?

Cheers

#2 lore raw
In reply to Mark Lord
Quoted 2 lines · Mark Lord

Why does link(2) not support hard-linking across bind mount points of the same underlying filesystem ?

Because it gives you a security boundary around a subtree.

#3 lore raw
Quoted 5 lines · Al Viro
Quoted 2 lines · Mark Lord

Why does link(2) not support hard-linking across bind mount points of the same underlying filesystem ?

Because it gives you a security boundary around a subtree.

PS: that had been discussed quite a few times, but to avoid searches:
consider e.g. mount --bind /tmp /tmp; now you've got a situation when users can't create links to elsewhere no root fs, even though they have /tmp writable to them. Similar technics works for other isolation needs - basically, you can confine rename/link to given subtree. IOW, it's a deliberate feature. Note that you can bind a bunch of trees into chroot and get predictable restrictions regardless of how the stuff might get rearranged a year later in the main tree, etc.

author
#4 lore raw
Quoted 6 lines · Al Viro
Quoted 4 lines · Al Viro
Quoted 2 lines · Mark Lord

Why does link(2) not support hard-linking across bind mount points of the same underlying filesystem ?

Because it gives you a security boundary around a subtree.

Quoted 8 lines

PS: that had been discussed quite a few times, but to avoid searches:
consider e.g. mount --bind /tmp /tmp; now you've got a situation when users can't create links to elsewhere no root fs, even though they have /tmp writable to them. Similar technics works for other isolation needs - basically, you can confine rename/link to given subtree. IOW, it's a deliberate feature. Note that you can bind a bunch of trees into chroot and get predictable restrictions regardless of how the stuff might get rearranged a year later in the main tree, etc.

..

Thanks, Al. That makes sense for a multi-user system, so I'm happy.

But.. pity there's no mount flag override for smaller systems, where bind mounts might be more useful with link(2) actually working.

The patch is simple enough when needed, though.

Cheers

fs/namei.c +0 -0
@@ -2398,7 +2398,7 @@
	if (error)
		goto out;
	error = -EXDEV;
-	if (old_nd.mnt != nd.mnt)
+	if (old_nd.mnt->mnt_sb != nd.mnt->mnt_sb)
		goto out_release;
	new_dentry = lookup_create(&nd, 0);
	error = PTR_ERR(new_dentry);
#5 lore raw
Quoted 2 lines · Mark Lord

But.. pity there's no mount flag override for smaller systems, where bind mounts might be more useful with link(2) actually working.

I don't see it. You always can make hard link on the underlying filesystem. If you need to make it on the bound mount, that is, if you can't locate the underlying filesystem to make the hard link, you can use a symbolic link.

author
#6 lore raw
Quoted 5 lines · David Newall
Quoted 2 lines · Mark Lord

But.. pity there's no mount flag override for smaller systems, where bind mounts might be more useful with link(2) actually working.

I don't see it. You always can make hard link on the underlying filesystem. If you need to make it on the bound mount, that is, if you can't locate the underlying filesystem to make the hard link, you can use a symbolic link.

..

Where people run into trouble with this, is when they simply go to move a huge file (DVD image) from one directory to another, where the target happens to be on a different bind point of the same underlying filesystem.

This can happen in a variety of common ways, including accessing over CIFS or Samba.

And the result is a very very long delay while we copy 4-8GB of data instead of simply moving the link.

Cheers

#7 lore raw
Quoted 5 lines · Mark Lord
Quoted 4 lines · David Newall
Quoted 2 lines · Mark Lord

But.. pity there's no mount flag override for smaller systems, where bind mounts might be more useful with link(2) actually working.

Quoted 7 lines
Quoted 1 line

I don't see it. You always can make hard link on the underlying filesystem. If you need to make it on the bound mount, that is, if you can't locate the underlying filesystem to make the hard link, you can use a symbolic link.

..

Where people run into trouble with this, is when they simply go to move a huge file (DVD image) from one directory to another, where the target happens to be on a different bind point of the same underlying filesystem.

Does that prevent you from seeing the underlying filesystem?

#18 lore raw
In reply to Mark Lord
Quoted 2 lines · Mark Lord

Why does link(2) not support hard-linking across bind mount points of the same underlying filesystem ?

do we need link(2) at all? bind mounts are supposed to be (hard/soft) link minus the headaches.

Signature · 1 line
Democracy is about two wolves and a sheep deciding what to eat for dinner.

begin:vcard
fn:Rogelio M. Serrano Jr
n:M. Serrano Jr;Rogelio
org:SMSG Communications Philippines;Technical Department adr:;;;;;;Republic of the Philippines email;internet:[email protected] title:Programmer
tel;work:+6327534145
tel;home:+6329527026
tel;cell:+639209202267
x-mozilla-html:FALSE
version:2.1
end:vcard

#8 lore raw
In reply to David Newall
Quoted 7 lines · David Newall
Quoted 2 lines · Mark Lord

But.. pity there's no mount flag override for smaller systems, where bind mounts might be more useful with link(2) actually working.

I don't see it. You always can make hard link on the underlying filesystem. If you need to make it on the bound mount, that is, if you can't locate the underlying filesystem to make the hard link, you can use a symbolic link.

i run into it on a system where /home is a bind mount of /var/home ... i did this because:

- i prefer /home to be nosuid,nodev (multi-user system)
- i prefer /home to not be on same fs as /
- the system has only one raid1 array, and i can't stand having two writable filesystems competing on the same set of spindles (i like to imagine that one fs competing for the spindles can potentially result in better seek patterns)
- i didn't want to do /var -> /home/var or vice versa ... because i don't like seeing "/var/home/dean" when i'm in my home dir and such.
- i didn't want to try to balance disk space between /var and /home
- i didn't want to use a volume mgr just to handle disk space balance...

so i gave a bind mount a try.

i was surprised to see that mv(1) between /var and /home causes the file to be copied due to the link(1) failing...

it does seem like something which should be configurable per mount point... maybe that can be done with the patches i've seen going around supporting per-bind mount read-only/etc options?

-dean

p.s. in retrospect i probably could have arranged it more like this:

  mount /dev/md1 $tmpmntpoint
  mount --bind $tmpmntpoint/var /var
  mount --bind $tmpmntpoint/home /home
  umount $tmpmntpoint

except i can't easily specify that in fstab... and neither of the bind mounts would show up in df(1). seems like it wouldn't be hard to support this type of subtree mount though. mount(8) could support a single subtree mount using this technique but the second subtree mount attempt would fail because you can't temporarily remount the device because the mount point is gone.

#9 lore raw
Quoted 7 lines · dean gaudet

p.s. in retrospect i probably could have arranged it more like this:

mount /dev/md1 $tmpmntpoint
mount --bind $tmpmntpoint/var /var
mount --bind $tmpmntpoint/home /home
umount $tmpmntpoint

Quoted 6 lines

except i can't easily specify that in fstab... and neither of the bind mounts would show up in df(1). seems like it wouldn't be hard to support this type of subtree mount though. mount(8) could support a single subtree mount using this technique but the second subtree mount attempt would fail because you can't temporarily remount the device because the mount point is gone.

Why is it gone?

mount /dev/md1 /tmpmnt
mount --bind /tmpmnt/var /var
mount --bind /tmpmnt/home /home

Is perfectly fine, and /tmpmnt is still alive and mounted. Additionally, you can

umount /tmpmnt

now, which leaves only /var and /home.

#10 lore raw
Quoted 22 lines · Jan Engelhardt
Quoted 7 lines · dean gaudet

p.s. in retrospect i probably could have arranged it more like this:

mount /dev/md1 $tmpmntpoint
mount --bind $tmpmntpoint/var /var
mount --bind $tmpmntpoint/home /home
umount $tmpmntpoint

Quoted 6 lines

except i can't easily specify that in fstab... and neither of the bind mounts would show up in df(1). seems like it wouldn't be hard to support this type of subtree mount though. mount(8) could support a single subtree mount using this technique but the second subtree mount attempt would fail because you can't temporarily remount the device because the mount point is gone.

Why is it gone?

mount /dev/md1 /tmpmnt
mount --bind /tmpmnt/var /var
mount --bind /tmpmnt/home /home

Quoted 6 lines

Is perfectly fine, and /tmpmnt is still alive and mounted. Additionally, you can

umount /tmpmnt

now, which leaves only /var and /home.

i was trying to come up with a userland-only change in mount(8) which would behave like so:

# mount --subtree var /dev/md1 /var
  internally mount does:
  - mount /dev/md1 /tmpmnt
  - mount --bind /tmpmnt/var /var
  - umount /tmpmnt
# mount --subtree home /dev/md1 /home
  internally mount does:
  - mount /dev/md1 /tmpmnt
  - mount --bind /tmpmnt/home /home
  - umount /tmpmnt

but that second mount would fail because /dev/md1 is already mounted (but the mount point is gone)...

it certainly works if i issue the commands individually as i described -- but a change within mount(8) would have the benefit of working with /etc/fstab too.

-dean

#11 lore raw
Quoted 10 lines · dean gaudet

i was trying to come up with a userland-only change in mount(8) which would behave like so:

# mount --subtree var /dev/md1 /var
 internally mount does:
 - mount /dev/md1 /tmpmnt
 - mount --bind /tmpmnt/var /var
 - umount /tmpmnt
Quoted 8 lines
# mount --subtree home /dev/md1 /home
 internally mount does:
 - mount /dev/md1 /tmpmnt
 - mount --bind /tmpmnt/home /home
 - umount /tmpmnt

but that second mount would fail because /dev/md1 is already mounted (but the mount point is gone)...

I do not think it would fail. Like this:

# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/home     208486056 158605472  49880584  77% /home
# mount /dev/mapper/home /mnt
# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/home     208486056 158605472  49880584  77% /home
/dev/mapper/home     208486056 158605472  49880584  77% /mnt
#12 lore raw
In reply to dean gaudet
Quoted 7 lines · dean gaudet
Quoted 5 lines · David Newall
Quoted 3 lines · Mark Lord

But.. pity there's no mount flag override for smaller systems, where bind mounts might be more useful with link(2) actually working.

Quoted 9 lines
Quoted 4 lines

I don't see it. You always can make hard link on the underlying filesystem. If you need to make it on the bound mount, that is, if you can't locate the underlying filesystem to make the hard link, you can use a symbolic link.

i run into it on a system where /home is a bind mount of /var/home ... i did this because:

- i prefer /home to be nosuid,nodev (multi-user system)

Whatever security /home has, /var/home is the one that restricts because users can still access their files that way.

Quoted 9 lines

- i prefer /home to not be on same fs as /
- the system has only one raid1 array, and i can't stand having two writable filesystems competing on the same set of spindles (i like to imagine that one fs competing for the spindles can potentially result in better seek patterns)
...
- i didn't want to try to balance disk space between /var and /home
- i didn't want to use a volume mgr just to handle disk space balance...

Pffuff. That's what volume managers are for! You do have (at least) two independent spindles in your RAID1 array, which give you less need to worry about head-stack contention. You probably want different mount restrictions on /home than /var, so you really must use separate filesystems. LVM is your friend.

But with regards to bind mounts and hard links: If you want to be able to hard-link /home/me/log to /var/tmp/my-log, then I see nothing to prevent hard-linking /var/home/me/log to /var/tmp/my-log.

I think it's possible to be too precious about preserving the illusion of one file-system structure when the reality is something different. Don't lose site of reality.

#13 lore raw
Quoted 15 lines · David Newall
Quoted 14 lines · dean gaudet
Quoted 11 lines · David Newall
Quoted 3 lines · Mark Lord

But.. pity there's no mount flag override for smaller systems, where bind mounts might be more useful with link(2) actually working.

I don't see it. You always can make hard link on the underlying filesystem.
If you need to make it on the bound mount, that is, if you can't locate the
underlying filesystem to make the hard link, you can use a symbolic link.

Quoted 8 lines
Quoted 5 lines

i run into it on a system where /home is a bind mount of /var/home ... i did this because:

- i prefer /home to be nosuid,nodev (multi-user system)

Whatever security /home has, /var/home is the one that restricts because users can still access their files that way.

yep. and /var is nosuid,nodev as well.

Quoted 10 lines
Quoted 9 lines

- i prefer /home to not be on same fs as /
- the system has only one raid1 array, and i can't stand having two writable filesystems competing on the same set of spindles (i like to imagine that one fs competing for the spindles can potentially result in better seek patterns)
...
- i didn't want to try to balance disk space between /var and /home
- i didn't want to use a volume mgr just to handle disk space balance...

Quoted 3 lines

Pffuff. That's what volume managers are for! You do have (at least) two independent spindles in your RAID1 array, which give you less need to worry about head-stack contention.

this system is write intensive and writes go to all spindles, so you're assertion is wrong. a quick look at iostat shows the system has averaged 50/50 reads/writes over 34 days. that means 50% of the IO is going to both spindles.

Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
sda 1.96 2.24 33.65 33.16 755.50 465.45 36.55 0.56 8.43 5.98 39.96

Quoted 2 lines

You probably want different mount restrictions
on /home than /var, so you really must use separate filesystems.

not sure why you think i want different restrictions... i'm running fine with nosuid,nodev for /var.

the main worry i have is some user maliciously hardlinks everything under /var/log somewhere else and slowly fills up the file system with old rotated logs. the users otherwise have quotas so they can't fill things up on their own. i could probably set up XFS quota trees (aka "projects") but haven't gone to this effort yet.

Quoted 1 line

LVM is your friend.

i disagree. but this is getting into personal taste -- i find volume managers to be an unnecessary layer of complexity. given i need quotas for the users anyhow i don't see why i should both manage my disk space via quotas and via an extra block layer.

Quoted 4 lines

But with regards to bind mounts and hard links: If you want to be able to hard-link /home/me/log to /var/tmp/my-log, then I see nothing to prevent hard-linking /var/home/me/log to /var/tmp/my-log.

you probably missed the point where i said that i was surprised i couldn't hardlink across the bind mount and actually wanted it to work.

-dean

#14 lore raw
Quoted 7 lines · dean gaudet
Quoted 4 lines

Pffuff. That's what volume managers are for! You do have (at least) two independent spindles in your RAID1 array, which give you less need to worry about head-stack contention.

this system is write intensive and writes go to all spindles, so you're assertion is wrong.

I don't know what you think I was asserting, but you were wrong. Of course I/O is distributed across both spindles. You would expect no less. THAT is what I was telling you.

Quoted 6 lines

the main worry i have is some user maliciously hardlinks everything under /var/log somewhere else and slowly fills up the file system with old rotated logs. the users otherwise have quotas so they can't fill things up on their own. i could probably set up XFS quota trees (aka "projects") but haven't gone to this effort yet.

See, this is where you show that you don't understand the system. I'll explain it, just once. /var/home contains home directories. /var/log and /var/home are on the same filesystem. So /var/log/* can be linked to /var/home/malicious, and that's just one of your basic misunderstandings.

Quoted 5 lines
Quoted 2 lines

LVM is your friend.

i disagree. but this is getting into personal taste -- i find volume managers to be an unnecessary layer of complexity.

Right... But wanting to change the semantics of link(2), so that you can do something that you already can do, anyway, this is simple, is it?

Quoted 3 lines

you probably missed the point where i said that i was surprised i couldn't hardlink across the bind mount and actually wanted it to work.

No. Look, you obviously haven't read what I've told you. I mean, it's very obvious you haven't. I'm wasting my time on you and I'm now out of generosity. Good luck to you. I think you need it.

And no, you can't change link(2). You don't need to.

#15 lore raw
Quoted 7 lines · David Newall
Quoted 6 lines · dean gaudet
Quoted 5 lines

Pffuff. That's what volume managers are for! You do have (at least) two independent spindles in your RAID1 array, which give you less need to worry
about head-stack contention.

Quoted 6 lines
Quoted 2 lines

this system is write intensive and writes go to all spindles, so you're assertion is wrong.

I don't know what you think I was asserting, but you were wrong. Of course I/O is distributed across both spindles. You would expect no less. THAT is what I was telling you.

are you on crack?

it's a raid1. writes go to all spindles. they have to. by definition. reads can be spread around, but writes are mirrored.

Quoted 8 lines
Quoted 7 lines

the main worry i have is some user maliciously hardlinks everything under /var/log somewhere else and slowly fills up the file system with old rotated logs. the users otherwise have quotas so they can't fill things up on their own. i could probably set up XFS quota trees (aka "projects") but haven't gone to this effort yet.

Quoted 4 lines

See, this is where you show that you don't understand the system. I'll explain it, just once. /var/home contains home directories. /var/log and /var/home are on the same filesystem. So /var/log/* can be linked to /var/home/malicious, and that's just one of your basic misunderstandings.

yes you are on crack.

i told you i understand this exactly. it's right there in the message sent.

Quoted 3 lines

No. Look, you obviously haven't read what I've told you. I mean, it's very obvious you haven't. I'm wasting my time on you and I'm now out of generosity. Good luck to you. I think you need it.

you're the idiot not actually reading my messages.

-dean

#16 lore raw
Quoted 9 lines · dean gaudet
Quoted 4 lines

See, this is where you show that you don't understand the system. I'll explain it, just once. /var/home contains home directories. /var/log and /var/home are on the same filesystem. So /var/log/* can be linked to /var/home/malicious, and that's just one of your basic misunderstandings.

yes you are on crack.

i told you i understand this exactly. it's right there in the message sent.

So... You understand that if /var/home and /var/log are on one file system, you can hard-link, and you set your system up knowing that, and then you're *surprised* that:

Quoted 3 lines

the main worry i have is some user maliciously hardlinks everything under /var/log somewhere else and slowly fills up the file system with old rotated logs.

"Doctor, it hurts when I do this.." "Well, don't do that then".

I think the first time I saw the recommendation "Put /home on its own filesystem and don't give users directly writable directories on /var (except via set-uid helpers) so they can't play hardlink games" back in 1983 or so. I know that when SunOS 3.1 came out, that was already well-understood basic sysadmining. Sometimes, there's actual good reasons behind 20-year-old voodoo.. ;)

You sure you don't want to redesign your filesystem layout so you don't have to worry about your malicious users hardlinking stuff? Might be a lot easier than trying to get the kernel to do what you want in this case....

#17 lore raw
Quoted 7 lines · On Sat, 29 Dec 2007, [email protected] wrote:
Quoted 3 lines · dean gaudet

the main worry i have is some user maliciously hardlinks everything under /var/log somewhere else and slowly fills up the file system with old rotated logs.

"Doctor, it hurts when I do this.." "Well, don't do that then".

actually it doesn't hurt. i have other mechanisms which would pick this up fairly quickly.

-dean