Optional switching off cow in overlayfs

DISCUSSION Jan Olszak 3 participants 15 messages Jun 18, 2015 – Jun 24, 2015 Overlay filesystem

Share & tools

Jan Olszak

author

2015-06-18 13:25 UTC #0 lore raw

Hi there!
I was wondering about a small improvement to overlayfs - optional, per file copy of write.

1. By default overlayfs would work as usual.
2. If a file in 'lower' filesystem has an xattr set (e.g. trusted.overlay.cow=n) the file would not be copied.

What do you think?

Cheers,
Jan

David Howells

2015-06-18 13:39 UTC #1 lore raw

Quoted 8 lines · Jan Olszak

I was wondering about a small improvement to overlayfs - optional, per file copy of write.
1. By default overlayfs would work as usual.
2. If a file in 'lower' filesystem has an xattr set (e.g. trusted.overlay.cow=n) the file would not be copied.
What do you think?

Ummm. What's the use case? How do you handle the file being part of multiple overlays?

David

Jan Olszak

author

2015-06-18 14:43 UTC #2 lore raw

Quoted 9 lines · David Howells

Quoted 7 lines · Jan Olszak
I was wondering about a small improvement to overlayfs - optional, per file copy of write.
1. By default overlayfs would work as usual.
2. If a file in 'lower' filesystem has an xattr set (e.g. trusted.overlay.cow=n) the file would not be copied.

Quoted 6 lines

Quoted 1 line
What do you think?
Ummm. What's the use case? How do you handle the file being part of multiple overlays?
David

I wan't to achieve 'files' granularity in overlayfs instead of 'directory' granularity. It would be useful in Linux containers configuration. Sometimes you don't have the power to change the images that are run inside containers and in one folder you have both:
- files that have to be copied
- files that don't really need to be copied

trusted.overlay.cow==n indicates switching off copying the file to the upper layer. So with multiple overlays you would always have only one file from the lower-most dir.

Jan

Miklos Szeredi

maintainer

2015-06-22 11:32 UTC #3 lore raw

Quoted 33 lines · Jan Olszak

Quoted 11 lines · David Howells
Quoted 8 lines · Jan Olszak
I was wondering about a small improvement to overlayfs - optional, per file
copy of write.
1. By default overlayfs would work as usual.
2. If a file in 'lower' filesystem has an xattr set (e.g. trusted.overlay.cow=n) the file would not be copied.
Quoted 8 lines
Quoted 1 line
What do you think?
Ummm. What's the use case? How do you handle the file being part of multiple
overlays?
David
I wan't to achieve 'files' granularity in overlayfs instead of 'directory' granularity.
It would be useful in Linux containers configuration. Sometimes you don't have the
power to change the images that are run inside containers and in one folder you have both:
- files that have to be copied
- files that don't really need to be copied

Quoted 4 lines

trusted.overlay.cow==n indicates switching off copying the file to the upper layer.
So with multiple overlays you would always have only one file from the lower-most dir.

What type filesystem is the lowest layer? Can you set immutable on
it (chattr +i)? That would be the logical way to say "nobody can modify this file, not even root".

Thanks,
Miklos

Jan Olszak

author

2015-06-22 13:45 UTC #4 lore raw

Quoted 34 lines · Miklos Szeredi

Quoted 29 lines · Jan Olszak
Quoted 10 lines · David Howells
Quoted 8 lines · Jan Olszak
I was wondering about a small improvement to overlayfs - optional, per file
copy of write.
1. By default overlayfs would work as usual.
2. If a file in 'lower' filesystem has an xattr set (e.g. trusted.overlay.cow=n) the file would not be copied.
Quoted 7 lines
Quoted 1 line
What do you think?
Ummm. What's the use case? How do you handle the file being part of multiple
overlays?
David
I wan't to achieve 'files' granularity in overlayfs instead of 'directory' granularity.
It would be useful in Linux containers configuration. Sometimes you don't have the
power to change the images that are run inside containers and in one folder you have both:
- files that have to be copied
- files that don't really need to be copied
Quoted 4 lines
trusted.overlay.cow==n indicates switching off copying the file to the upper layer.
So with multiple overlays you would always have only one file from the lower-most dir.

Quoted 9 lines

What type filesystem is the lowest layer? Can you set immutable on
it (chattr +i)? That would be the logical way to say "nobody can modify this file, not even root".
Thanks,
Miklos
--
To unsubscribe from this list: send the line "unsubscribe linux-unionfs" in

The lower fs is most likely ext4.
Unfortunately making a file immutable won't solve the problem. There's nothing wrong in modifying the file and it should stay possible.

I just want to switch off copy on write for some files. Can you see another way?

If you are OK with the optional xattr, then I'll prepare the patch.

Thanks,
Jan

Miklos Szeredi

maintainer

2015-06-22 13:53 UTC #5 lore raw

Quoted 6 lines · Jan Olszak

The lower fs is most likely ext4.
Unfortunately making a file immutable won't solve the problem. There's nothing wrong in modifying the file and it should stay possible.
I just want to switch off copy on write for some files. Can you see another way?

Ah, you want to allow modification of file on the lower layer?

You can bind mount individual files from the lower layer to the overlay. That fixes the "allow modification" part. But what should happen on rename or unlink?

Thanks,
Miklos

Jan Olszak

author

2015-06-22 14:26 UTC #6 lore raw

Quoted 9 lines · Miklos Szeredi

Quoted 6 lines · Jan Olszak
The lower fs is most likely ext4.
Unfortunately making a file immutable won't solve the problem. There's nothing wrong in modifying the file and it should stay possible.
I just want to switch off copy on write for some files. Can you see another way?
Ah, you want to allow modification of file on the lower layer?

Exactly

Quoted 2 lines

You can bind mount individual files from the lower layer to the overlay. That fixes the "allow modification" part.

Well bind mounting every file that should have COW is unmaintainable - if new files appear admin has to mount each one. Plus all the unnecessary mounts. It's much easier to do it the opposite way - by default have COW and only for some files switch it off.

Quoted 1 line

But what should happen on rename or unlink?

Both would operate on the original inode.

Setting "trusted.overlay.cow" is a privileged operation so it's admin's decision to allow this.

Each layer could potentially make the a file COW again by setting/removing "trusted.overlay.cow" attr.

Thanks,
Jan

Miklos Szeredi

maintainer

2015-06-22 14:37 UTC #7 lore raw

Quoted 5 lines · Jan Olszak

Quoted 2 lines
You can bind mount individual files from the lower layer to the overlay. That fixes the "allow modification" part.
Well bind mounting every file that should have COW is unmaintainable - if new files appear admin has to mount each one.

No. I was proposing bind mounts for non-COW ones.

mount -t overlay -olowerdir=/lower,upperdir=/upper,workdir=/work overlay /ovl mount --bind /lower/this-file-is-non-cow /ovl/this-file-is-non-cow
...

Quoted 3 lines

Quoted 1 line
But what should happen on rename or unlink?
Both would operate on the original inode.

And that would make this hugely complex.

Thanks,
Miklos

Jan Olszak

author

2015-06-22 15:32 UTC #8 lore raw

Quoted 10 lines · Miklos Szeredi

Quoted 4 lines · Jan Olszak
Quoted 2 lines
You can bind mount individual files from the lower layer to the overlay. That fixes the "allow modification" part.
Well bind mounting every file that should have COW is unmaintainable - if new files appear admin has to mount each one.
No. I was proposing bind mounts for non-COW ones.
mount -t overlay -olowerdir=/lower,upperdir=/upper,workdir=/work overlay /ovl mount --bind /lower/this-file-is-non-cow /ovl/this-file-is-non-cow
...

Yes, you are right. This should work, but introduces an overhead of many mounts for each Linux container.

The proposed alternative is: Use setfattr only once for each file and then forget about it.

Quoted 3 lines

Quoted 2 lines
Quoted 1 line
But what should happen on rename or unlink?
Both would operate on the original inode.
And that would make this hugely complex.

Are you sure? I haven't looked into rename and unlink routines but this patch covers opening the file and is pretty simple (though it probably breaks fchmod/fchown...).

I can try to prepare a real solution if you like the idea.

From 20e590c0e37f087955b25edfbc2ab4c9c034b5b1 Mon Sep 17 00:00:00 2001
From: Jan Olszak <[email protected]>
Date: Mon, 22 Jun 2015 17:07:31 +0200
Subject: [PATCH] Optional copy on write in overlayfs

---
 fs/overlayfs/inode.c     | 17 +++++++++++++++++
 fs/overlayfs/overlayfs.h |  1 +
 fs/overlayfs/super.c     |  5 -----
 3 files changed, 18 insertions(+), 5 deletions(-)

fs/overlayfs/inode.c +17 -0

index 04f1248..4f9f399 100644
@@ -321,6 +321,20 @@ out:
     return err;
 }
 
+static bool ovl_dentry_need_copy_up(struct dentry *realdentry)
+{
+    int res;
+    char val;
+    struct inode *inode = realdentry->d_inode;
+
+    res = inode->i_op->getxattr(realdentry, OVL_XATTR_COW, &val, 1);
+    if (res == 1 && val == 'n'){
+        return false;
+    }
+
+    return true;
+}
+
 static bool ovl_open_need_copy_up(int flags, enum ovl_path_type type,
                   struct dentry *realdentry)
 {

@@ -330,6 +344,9 @@ static bool ovl_open_need_copy_up(int flags, enum ovl_path_type type,
     if (special_file(realdentry->d_inode->i_mode))
         return false;
 
+    if (!ovl_dentry_need_copy_up(realdentry))
+        return false;
+
     if (!(OPEN_FMODE(flags) & FMODE_WRITE) && !(flags & O_TRUNC))
         return false;

fs/overlayfs/overlayfs.h +1 -0

index 17ac5af..b364888 100644
@@ -26,6 +26,7 @@ enum ovl_path_type {
 #define OVL_XATTR_PRE_NAME "trusted.overlay."
 #define OVL_XATTR_PRE_LEN  16
 #define OVL_XATTR_OPAQUE   OVL_XATTR_PRE_NAME"opaque"
+#define OVL_XATTR_COW      OVL_XATTR_PRE_NAME"cow"
 
 static inline int ovl_do_rmdir(struct inode *dir, struct dentry *dentry)
 {

fs/overlayfs/super.c +0 -5

index bf8537c..916a4b5 100644
@@ -944,11 +944,6 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
             pr_err("overlayfs: failed to clone lowerpath\n");
             goto out_put_lower_mnt;
         }
-        /*
-         * Make lower_mnt R/O.  That way fchmod/fchown on lower file
-         * will fail instead of modifying lower fs.
-         */
-        mnt->mnt_flags |= MNT_READONLY;
 
         ufs->lower_mnt[ufs->numlower] = mnt;
         ufs->numlower++;

Signature · 8 lines

1.9.1





Thanks,
Jan

David Howells

2015-06-23 09:27 UTC #9 lore raw

Quoted 4 lines · Jan Olszak

Quoted 3 lines
Quoted 2 lines
Quoted 1 line
But what should happen on rename or unlink?
Both would operate on the original inode.
And that would make this hugely complex.
Are you sure?

Unlink would be the same as now - you just whiteout over the lower layer.

Rename, though, would be complex - unless you go for the trivial option and just error out.

The problem is that the only connection between the upper layer and lower layer is coincidence governed by filename. If you move the upper layer object, the lower layer object is no longer coincident and is effectively lost. You cannot move the lower layer object because the lower layer may be shared between multiple overlays.

At the moment, rename deals

You could, I suppose, store a "fall through" inode in the upper layer that has the NFS fh of the lower layer object attached in some fashion (eg. xattr), but it's messy.

Hard linking would be even more complex - you'd have to hard link the "fall through" inodes on the upper layer.

David

Jan Olszak

author

2015-06-23 10:36 UTC #10 lore raw

Quoted 19 lines · David Howells

Quoted 4 lines · Jan Olszak
Quoted 3 lines
Quoted 2 lines
Quoted 1 line
But what should happen on rename or unlink?
Both would operate on the original inode.
And that would make this hugely complex.
Are you sure?
Unlink would be the same as now - you just whiteout over the lower layer.
Rename, though, would be complex - unless you go for the trivial option and just error out.
The problem is that the only connection between the upper layer and lower layer is coincidence governed by filename. If you move the upper layer object, the lower layer object is no longer coincident and is effectively lost. You cannot move the lower layer object because the lower layer may be shared between multiple overlays.
At the moment, rename deals

Quoted 8 lines

You could, I suppose, store a "fall through" inode in the upper layer that has the NFS fh of the lower layer object attached in some fashion (eg. xattr), but it's messy.
Hard linking would be even more complex - you'd have to hard link the "fall through" inodes on the upper layer.
David

"Fall through" inodes seem a messy solution, agreed.

Why not just say: "rename causes copy" and leave everything as it is now.
Renaming a file shared by many mounts seems to be a corner case and I guess overlayfs handles this well right now.

Thanks,
Jan

David Howells

2015-06-23 13:23 UTC #11 lore raw

Quoted 1 line · Jan Olszak

Why not just say: "rename causes copy" and leave everything as it is now.

That's another option - but your whole proposal was that we specifically don't copy up these files IIRC.

David

Jan Olszak

author

2015-06-23 16:05 UTC #12 lore raw

Quoted 5 lines · David Howells

Quoted 1 line · Jan Olszak
Why not just say: "rename causes copy" and leave everything as it is now.
That's another option - but your whole proposal was that we specifically don't copy up these files IIRC.

Yes, so let's modify the proposal:

Overlayfs checks xattr "trusted.overlay.cow" to switch COW off whenever it's possible.

1. This would be an optimization/convenience option, not a sercurity option.
2. By default overlayfs uses COW like it does now
3. trusted.overlay.cow=="n" means:
1. Unlink, open, setattr etc. use the original inode from lowerdir
2. Unmodifying operations also use inode from lowerdir
3. Other operations (rename) may copy the file if it's required by the implementation of overlayfs

Thanks,
Jan

Jan Olszak

author

2015-06-23 16:08 UTC #13 lore raw

In reply to David Howells

Quoted 6 lines · David Howells

Quoted 1 line · Jan Olszak
Why not just say: "rename causes copy" and leave everything as it is now.
That's another option - but your whole proposal was that we specifically don't copy up these files IIRC.

Yes, so let's modify the proposal:

Overlayfs checks xattr "trusted.overlay.cow" to switch COW off whenever it's possible.

1. This would be an optimization/convenience option, not a security option
2. By default overlayfs uses COW like it does now
3. trusted.overlay.cow=="n" means:
- Unlink, open, setattr etc. use the original inode from lowerdir
- Unmodifying operations also use inode from lowerdir
- Other operations (rename) may copy the file if it's required by the implementation of overlayfs

Thanks,
Jan

Jan Olszak

author

2015-06-24 16:03 UTC #14 lore raw

Quoted 13 lines · Jan Olszak

Quoted 6 lines · David Howells
Quoted 1 line · Jan Olszak
Why not just say: "rename causes copy" and leave everything as it is now.
That's another option - but your whole proposal was that we specifically don't copy up these files IIRC.
Yes, so let's modify the proposal:
Overlayfs checks xattr "trusted.overlay.cow" to switch COW off whenever it's possible.

Quoted 8 lines

1. This would be an optimization/convenience option, not a security option
2. By default overlayfs uses COW like it does now
3. trusted.overlay.cow=="n" means:
- Unlink, open, setattr etc. use the original inode from lowerdir
- Unmodifying operations also use inode from lowerdir
- Other operations (rename) may copy the file if it's required by the implementation of overlayfs

So does it sound upstreamable?

Jan