Skip to contentSkip to conversation

ovl: support stacked SEEK_HOLE/SEEK_DATA

PATCH Superseded Amir Goldstein 3 participants 7 messages Feb 26, 2019 – Feb 27, 2019 Overlay filesystem

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Amir Goldstein

author maintainer
#0 lore raw

PATCH 1/1 ovl: support stacked SEEK_HOLE/SEEK_DATA

Overlay file f_pos is the master copy that is preserved through copy up, but only real fs knows how to SEEK_HOLE/ SEEK_DATA. So we copy f_pos from overlay file, perform the seek on real file and copy f_pos back to overlay file.

Fixes d1d04ef8572b ("ovl: stack file ops")

---

Miklos,

I verified no regressions with xfstests quick tests. The improved generic/seek tests that I posted to fstests are failing on master and passing with this fix.

Thanks,
Amir.

 fs/overlayfs/file.c | 27 +++++++++++++++++++++++----
 1 file changed, 23 insertions(+), 4 deletions(-)
fs/overlayfs/file.c +23 -4
index 84dd957efa24..0d472940ce9e 100644
@@ -145,11 +145,30 @@ static int ovl_release(struct inode *inode, struct file *file)
 
 static loff_t ovl_llseek(struct file *file, loff_t offset, int whence)
 {
-	struct inode *realinode = ovl_inode_real(file_inode(file));
+	struct fd real;
+	const struct cred *old_cred;
+	ssize_t ret;
 
-	return generic_file_llseek_size(file, offset, whence,
-					realinode->i_sb->s_maxbytes,
-					i_size_read(realinode));
+	ret = ovl_real_fdget(file, &real);
+	if (ret)
+		return ret;
+
+	/*
+	 * Overlay file f_pos is the master copy that is preserved
+	 * through copy up, but only real fs knows how to SEEK_HOLE/
+	 * SEEK_DATA.
+	 */
+	real.file->f_pos = file->f_pos;
+
+	old_cred = ovl_override_creds(file_inode(file)->i_sb);
+	ret = vfs_llseek(real.file, offset, whence);
+	revert_creds(old_cred);
+
+	file->f_pos = real.file->f_pos;
+
+	fdput(real);
+
+	return ret;
 }
 
 static void ovl_file_accessed(struct file *file)
Signature · 1 line
2.17.1
maintainer
#1 lore raw
Quoted 39 lines · Amir Goldstein

Overlay file f_pos is the master copy that is preserved through copy up, but only real fs knows how to SEEK_HOLE/ SEEK_DATA. So we copy f_pos from overlay file, perform the seek on real file and copy f_pos back to overlay file.

Fixes d1d04ef8572b ("ovl: stack file ops")

---

Miklos,

I verified no regressions with xfstests quick tests. The improved generic/seek tests that I posted to fstests are failing on master and passing with this fix.

Thanks,
Amir.

 fs/overlayfs/file.c | 27 +++++++++++++++++++++++----
 1 file changed, 23 insertions(+), 4 deletions(-)
fs/overlayfs/file.c +3 -4
index 84dd957efa24..0d472940ce9e 100644
@@ -145,11 +145,30 @@ static int ovl_release(struct inode *inode, struct file *file)
 
 static loff_t ovl_llseek(struct file *file, loff_t offset, int whence)
 {
-       struct inode *realinode = ovl_inode_real(file_inode(file));
+       struct fd real;
+       const struct cred *old_cred;
+       ssize_t ret;
 
-       return generic_file_llseek_size(file, offset, whence,
-                                       realinode->i_sb->s_maxbytes,
-                                       i_size_read(realinode));

fs/overlayfs/file.c static int ovl_release(struct inode *inode, struct file *file) · line 161

Quoted 10 lines
fs/overlayfs/file.c +10 -0
+       ret = ovl_real_fdget(file, &real);
+       if (ret)
+               return ret;
+
+       /*
+        * Overlay file f_pos is the master copy that is preserved
+        * through copy up, but only real fs knows how to SEEK_HOLE/
+        * SEEK_DATA.
+        */
+       real.file->f_pos = file->f_pos;

Parallel invocations of ovl_llseek would mean trouble. I suggest
doing generic_file_llseek_size for SET/CUR/END and doing the recursive one under ovl inode lock for HOLE/DATA.

Thanks,
Miklos

Quoted 16 lines
fs/overlayfs/file.c +10 -1
+
+       old_cred = ovl_override_creds(file_inode(file)->i_sb);
+       ret = vfs_llseek(real.file, offset, whence);
+       revert_creds(old_cred);
+
+       file->f_pos = real.file->f_pos;
+
+       fdput(real);
+
+       return ret;
 }
 
 static void ovl_file_accessed(struct file *file)
--

2.17.1

author maintainer
#2 lore raw
Quoted 47 lines · Miklos Szeredi
Quoted 45 lines · Amir Goldstein

Overlay file f_pos is the master copy that is preserved through copy up, but only real fs knows how to SEEK_HOLE/ SEEK_DATA. So we copy f_pos from overlay file, perform the seek on real file and copy f_pos back to overlay file.

Fixes d1d04ef8572b ("ovl: stack file ops")

---

Miklos,

I verified no regressions with xfstests quick tests. The improved generic/seek tests that I posted to fstests are failing on master and passing with this fix.

Thanks,
Amir.

 fs/overlayfs/file.c | 27 +++++++++++++++++++++++----
 1 file changed, 23 insertions(+), 4 deletions(-)
fs/overlayfs/file.c +9 -4
index 84dd957efa24..0d472940ce9e 100644
@@ -145,11 +145,30 @@ static int ovl_release(struct inode *inode, struct file *file)
 
 static loff_t ovl_llseek(struct file *file, loff_t offset, int whence)
 {
-       struct inode *realinode = ovl_inode_real(file_inode(file));
+       struct fd real;
+       const struct cred *old_cred;
+       ssize_t ret;
 
-       return generic_file_llseek_size(file, offset, whence,
-                                       realinode->i_sb->s_maxbytes,
-                                       i_size_read(realinode));
+       ret = ovl_real_fdget(file, &real);
+       if (ret)
+               return ret;
+
+       /*
+        * Overlay file f_pos is the master copy that is preserved
Quoted 9 lines

fs/overlayfs/file.c static int ovl_release(struct inode *inode, struct file *file) · line 161

Quoted 4 lines
fs/overlayfs/file.c +4 -0
+        * through copy up, but only real fs knows how to SEEK_HOLE/
+        * SEEK_DATA.
+        */
+       real.file->f_pos = file->f_pos;

Parallel invocations of ovl_llseek would mean trouble. I suggest
doing generic_file_llseek_size for SET/CUR/END and doing the recursive one under ovl inode lock for HOLE/DATA.

OK. What about Eddie's question:
If realinode is nested overlay inode, then realinode->i_sb->s_maxbytes will not reflect the "real" inode's maxbytes.
Not to mention other crazy things that filesystems may do for SET/CUR/END. Take ext4_llseek() for example, for the less common case (ext3 format inode) the value of inode->i_sb->s_maxbytes is not used as maxbytes, but a smaller value.

Would it be worse or better than exclusive lock if we allocate a new realfile instance for every call to ovl_llseek? (as we do anyway for the transient state of ro fd of file that was copied up).

Also, I see that we need to use vfs_setpos() to copy f_pos back from real file in order to reset file->f_version.

Thanks,
Amir.

Quoted 20 lines

Thanks,
Miklos

Quoted 16 lines
fs/overlayfs/file.c +10 -1
+
+       old_cred = ovl_override_creds(file_inode(file)->i_sb);
+       ret = vfs_llseek(real.file, offset, whence);
+       revert_creds(old_cred);
+
+       file->f_pos = real.file->f_pos;
+
+       fdput(real);
+
+       return ret;
 }
 
 static void ovl_file_accessed(struct file *file)
--

2.17.1

maintainer
#3 lore raw
Quoted 67 lines · Amir Goldstein
Quoted 47 lines · Miklos Szeredi
Quoted 45 lines · Amir Goldstein

Overlay file f_pos is the master copy that is preserved through copy up, but only real fs knows how to SEEK_HOLE/ SEEK_DATA. So we copy f_pos from overlay file, perform the seek on real file and copy f_pos back to overlay file.

Fixes d1d04ef8572b ("ovl: stack file ops")

---

Miklos,

I verified no regressions with xfstests quick tests. The improved generic/seek tests that I posted to fstests are failing on master and passing with this fix.

Thanks,
Amir.

 fs/overlayfs/file.c | 27 +++++++++++++++++++++++----
 1 file changed, 23 insertions(+), 4 deletions(-)
fs/overlayfs/file.c +9 -4
index 84dd957efa24..0d472940ce9e 100644
@@ -145,11 +145,30 @@ static int ovl_release(struct inode *inode, struct file *file)
 
 static loff_t ovl_llseek(struct file *file, loff_t offset, int whence)
 {
-       struct inode *realinode = ovl_inode_real(file_inode(file));
+       struct fd real;
+       const struct cred *old_cred;
+       ssize_t ret;
 
-       return generic_file_llseek_size(file, offset, whence,
-                                       realinode->i_sb->s_maxbytes,
-                                       i_size_read(realinode));
+       ret = ovl_real_fdget(file, &real);
+       if (ret)
+               return ret;
+
+       /*
+        * Overlay file f_pos is the master copy that is preserved
Quoted 9 lines

fs/overlayfs/file.c static int ovl_release(struct inode *inode, struct file *file) · line 161

Quoted 4 lines
fs/overlayfs/file.c +4 -0
+        * through copy up, but only real fs knows how to SEEK_HOLE/
+        * SEEK_DATA.
+        */
+       real.file->f_pos = file->f_pos;

Parallel invocations of ovl_llseek would mean trouble. I suggest
doing generic_file_llseek_size for SET/CUR/END and doing the recursive one under ovl inode lock for HOLE/DATA.

OK. What about Eddie's question:
If realinode is nested overlay inode, then realinode->i_sb->s_maxbytes will not reflect the "real" inode's maxbytes.
Not to mention other crazy things that filesystems may do for SET/CUR/END. Take ext4_llseek() for example, for the less common case (ext3 format inode) the value of inode->i_sb->s_maxbytes is not used as maxbytes, but a smaller value.

Quoted 3 lines

Would it be worse or better than exclusive lock if we allocate a new realfile instance for every call to ovl_llseek? (as we do anyway for the transient state of ro fd of file that was copied up).

Hell knows. Simplest is definitely the exclusive inode lock. Then we
can look at optimizing that... I don't think lseek would be particularly performance sensitive: those would be using the pread/pwrite interfaces anyway.

Thanks,
Miklos

author maintainer
#4 lore raw
Quoted 69 lines · Miklos Szeredi
Quoted 67 lines · Amir Goldstein
Quoted 47 lines · Miklos Szeredi
Quoted 45 lines · Amir Goldstein

Overlay file f_pos is the master copy that is preserved through copy up, but only real fs knows how to SEEK_HOLE/ SEEK_DATA. So we copy f_pos from overlay file, perform the seek on real file and copy f_pos back to overlay file.

Fixes d1d04ef8572b ("ovl: stack file ops")

---

Miklos,

I verified no regressions with xfstests quick tests. The improved generic/seek tests that I posted to fstests are failing on master and passing with this fix.

Thanks,
Amir.

 fs/overlayfs/file.c | 27 +++++++++++++++++++++++----
 1 file changed, 23 insertions(+), 4 deletions(-)
fs/overlayfs/file.c +9 -4
index 84dd957efa24..0d472940ce9e 100644
@@ -145,11 +145,30 @@ static int ovl_release(struct inode *inode, struct file *file)
 
 static loff_t ovl_llseek(struct file *file, loff_t offset, int whence)
 {
-       struct inode *realinode = ovl_inode_real(file_inode(file));
+       struct fd real;
+       const struct cred *old_cred;
+       ssize_t ret;
 
-       return generic_file_llseek_size(file, offset, whence,
-                                       realinode->i_sb->s_maxbytes,
-                                       i_size_read(realinode));
+       ret = ovl_real_fdget(file, &real);
+       if (ret)
+               return ret;
+
+       /*
+        * Overlay file f_pos is the master copy that is preserved
Quoted 9 lines

fs/overlayfs/file.c static int ovl_release(struct inode *inode, struct file *file) · line 161

Quoted 4 lines
fs/overlayfs/file.c +4 -0
+        * through copy up, but only real fs knows how to SEEK_HOLE/
+        * SEEK_DATA.
+        */
+       real.file->f_pos = file->f_pos;

Parallel invocations of ovl_llseek would mean trouble. I suggest
doing generic_file_llseek_size for SET/CUR/END and doing the recursive one under ovl inode lock for HOLE/DATA.

OK. What about Eddie's question:
If realinode is nested overlay inode, then realinode->i_sb->s_maxbytes will not reflect the "real" inode's maxbytes.
Not to mention other crazy things that filesystems may do for SET/CUR/END. Take ext4_llseek() for example, for the less common case (ext3 format inode) the value of inode->i_sb->s_maxbytes is not used as maxbytes, but a smaller value.

Quoted 9 lines
Quoted 3 lines

Would it be worse or better than exclusive lock if we allocate a new realfile instance for every call to ovl_llseek? (as we do anyway for the transient state of ro fd of file that was copied up).

Hell knows. Simplest is definitely the exclusive inode lock. Then we
can look at optimizing that... I don't think lseek would be particularly performance sensitive: those would be using the pread/pwrite interfaces anyway.

Just to be clear, so exclusive inode lock and recursive for any type of seek? To be on the safe side?

Thanks,
Amir.

maintainer
#5 lore raw
Quoted 75 lines · Amir Goldstein
Quoted 73 lines · Miklos Szeredi
Quoted 70 lines · Amir Goldstein
Quoted 47 lines · Miklos Szeredi
Quoted 45 lines · Amir Goldstein

Overlay file f_pos is the master copy that is preserved through copy up, but only real fs knows how to SEEK_HOLE/ SEEK_DATA. So we copy f_pos from overlay file, perform the seek on real file and copy f_pos back to overlay file.

Fixes d1d04ef8572b ("ovl: stack file ops")

---

Miklos,

I verified no regressions with xfstests quick tests. The improved generic/seek tests that I posted to fstests are failing on master and passing with this fix.

Thanks,
Amir.

 fs/overlayfs/file.c | 27 +++++++++++++++++++++++----
 1 file changed, 23 insertions(+), 4 deletions(-)
fs/overlayfs/file.c +9 -4
index 84dd957efa24..0d472940ce9e 100644
@@ -145,11 +145,30 @@ static int ovl_release(struct inode *inode, struct file *file)
 
 static loff_t ovl_llseek(struct file *file, loff_t offset, int whence)
 {
-       struct inode *realinode = ovl_inode_real(file_inode(file));
+       struct fd real;
+       const struct cred *old_cred;
+       ssize_t ret;
 
-       return generic_file_llseek_size(file, offset, whence,
-                                       realinode->i_sb->s_maxbytes,
-                                       i_size_read(realinode));
+       ret = ovl_real_fdget(file, &real);
+       if (ret)
+               return ret;
+
+       /*
+        * Overlay file f_pos is the master copy that is preserved
Quoted 9 lines

fs/overlayfs/file.c static int ovl_release(struct inode *inode, struct file *file) · line 161

Quoted 4 lines
fs/overlayfs/file.c +4 -0
+        * through copy up, but only real fs knows how to SEEK_HOLE/
+        * SEEK_DATA.
+        */
+       real.file->f_pos = file->f_pos;

Parallel invocations of ovl_llseek would mean trouble. I suggest
doing generic_file_llseek_size for SET/CUR/END and doing the recursive one under ovl inode lock for HOLE/DATA.

OK. What about Eddie's question:
If realinode is nested overlay inode, then realinode->i_sb->s_maxbytes will not reflect the "real" inode's maxbytes.
Not to mention other crazy things that filesystems may do for SET/CUR/END. Take ext4_llseek() for example, for the less common case (ext3 format inode) the value of inode->i_sb->s_maxbytes is not used as maxbytes, but a smaller value.

Would it be worse or better than exclusive lock if we allocate a new realfile instance for every call to ovl_llseek? (as we do anyway for the transient state of ro fd of file that was copied up).

Quoted 8 lines
Quoted 5 lines

Hell knows. Simplest is definitely the exclusive inode lock. Then we
can look at optimizing that... I don't think lseek would be particularly performance sensitive: those would be using the pread/pwrite interfaces anyway.

Just to be clear, so exclusive inode lock and recursive for any type of seek? To be on the safe side?

Yes. As a trivial optimization SEEK_CUR/0 (return current pos) could be done without messing with underlying fs. Probably same for SEEK_SET/0.

Looking ahead, inode lock is definitely excessive, since we are not doing anything to the inode here, so ideally this would be a per-file mutex, but we've nowhere to put such a thing at the moment.

Thanks,
Miklos

#6 lore raw
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

OK. What about Eddie's question:
If realinode is nested overlay inode, then realinode->i_sb->s_maxbytes will not reflect the "real" inode's maxbytes.
Not to mention other crazy things that filesystems may do for SET/CUR/END. Take ext4_llseek() for example, for the less common case (ext3 format inode) the value of inode->i_sb->s_maxbytes is not used as maxbytes, but a smaller value.

Hi Amir,
Thanks your detail information and the patch.
I ran into this issue by using a proprietary stacked file system as lower and the file system does not initial it's s_maxbytes, therefore caused file larger than 2GB fails lseek. A quick workaround is to set the fs s_maxbytes to MAX_LFS_FILESIZE, but like you said, for certain cases this might be a problem.
I have verified this patch with same configuration and the lower inode f_op->llseek
does get invoked, looks like enough to resolve my particular case.

thanks,
Eddie