Skip to contentSkip to conversation

reduce bmv_count in xfs_vn_fiemap

PATCH Superseded Eric Sandeen 3 participants 10 messages Jul 15, 2009 – Jul 27, 2009 XFS filesystem

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 reduce bmv_count in xfs_vn_fiemap

commit 6321e3ed2acf3ee9643cdd403e1c88605d7944ba caused the full bmv_count's worth of getbmapx structures to get allocated; telling it to do MAXEXTNUM was a bit insane, resulting in ENOMEM.

Chop it down to something reasonable, the caller can loop over this if the file has > 64 extents.

(this is a regression, FWIW)

---

iff --git a/fs/xfs/linux-2.6/xfs_iops.c b/fs/xfs/linux-2.6/xfs_iops.c index 58973bb..370e9a7 100644

fs/xfs/linux-2.6/xfs_iops.c +6 -2
@@ -680,8 +680,12 @@ xfs_vn_fiemap(
 	else
 		bm.bmv_length = BTOBB(length);
 
-	/* our formatter will tell xfs_getbmap when to stop. */
-	bm.bmv_count = MAXEXTNUM;
+	/*
+	 * xfs_getbmap allocates based on that; pick a count that's
+	 * not too outragous.
+	 * Our formatter will tell xfs_getbmap when to stop.
+	 */
+	bm.bmv_count = 64;
 	bm.bmv_iflags = BMV_IF_PREALLOC;
 	if (fieinfo->fi_flags & FIEMAP_FLAG_XATTR)
 		bm.bmv_iflags |= BMV_IF_ATTRFORK;

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#1 lore raw
Quoted 7 lines · Eric Sandeen

commit 6321e3ed2acf3ee9643cdd403e1c88605d7944ba caused the full bmv_count's worth of getbmapx structures to get allocated; telling it to do MAXEXTNUM was a bit insane, resulting in ENOMEM.

Chop it down to something reasonable, the caller can loop over this if the file has > 64 extents.

It does seem to me that this will result in an unusal case for the caller, in that it will get fewer extents than fit in the provided buffer, yet should loop. Do current callers know that they can hit this case, detect it, and loop accordingly? Or is this just pushing the problem/regression to userspace?

Quoted 24 lines

(this is a regression, FWIW)

---

iff --git a/fs/xfs/linux-2.6/xfs_iops.c b/fs/xfs/linux-2.6/xfs_iops.c index 58973bb..370e9a7 100644

fs/xfs/linux-2.6/xfs_iops.c +6 -2
@@ -680,8 +680,12 @@ xfs_vn_fiemap(
 	else
 		bm.bmv_length = BTOBB(length);
 
-	/* our formatter will tell xfs_getbmap when to stop. */
-	bm.bmv_count = MAXEXTNUM;
+	/*
+	 * xfs_getbmap allocates based on that; pick a count that's
+	 * not too outragous.
+	 * Our formatter will tell xfs_getbmap when to stop.
+	 */
+	bm.bmv_count = 64;
 	bm.bmv_iflags = BMV_IF_PREALLOC;
 	if (fieinfo->fi_flags & FIEMAP_FLAG_XATTR)
 		bm.bmv_iflags |= BMV_IF_ATTRFORK;
Signature · 9 lines
Olaf Weber                 SGI               Phone:  +31(0)30-6696752
                           Veldzigt 2b       Fax:    +31(0)30-6696799
Technical Lead             3454 PW de Meern  Vnet:   955-7151
Storage Software           The Netherlands   Email:  [email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#2 lore raw
Quoted 7 lines · Olaf Weber
Quoted 4 lines · Eric Sandeen

commit 6321e3ed2acf3ee9643cdd403e1c88605d7944ba caused the full bmv_count's worth of getbmapx structures to get allocated; telling it to do MAXEXTNUM was a bit insane, resulting in ENOMEM.

Quoted 8 lines
Quoted 2 lines

Chop it down to something reasonable, the caller can loop over this if the file has > 64 extents.

It does seem to me that this will result in an unusal case for the caller, in that it will get fewer extents than fit in the provided buffer, yet should loop. Do current callers know that they can hit this case, detect it, and loop accordingly? Or is this just pushing the problem/regression to userspace?

Well, userspace just keeps calling until it gets FIEMAP_LAST in the flags. But yeah, I forgot that we were given the nr of user extents, I'll send a better V2.

-Eric

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#3 lore raw

PATCH v2 1/1 reduce bmv_count in xfs_vn_fiemap

In reply to Olaf Weber

commit 6321e3ed2acf3ee9643cdd403e1c88605d7944ba caused the full bmv_count's worth of getbmapx structures to get allocated; telling it to do MAXEXTNUM was a bit insane, resulting in ENOMEM every time.

Chop it down to something reasonable, the number of slots in the caller's input buffer. If this is too large the caller may get ENOMEM but the reason should not be a mystery, and they can try again with something smaller.

---

V2: set it to the fiemap info's fi_extents_max

fs/xfs/linux-2.6/xfs_iops.c +1 -2
index 58973bb..954d701 100644
@@ -680,8 +680,7 @@ xfs_vn_fiemap(
 	else
 		bm.bmv_length = BTOBB(length);
 
-	/* our formatter will tell xfs_getbmap when to stop. */
-	bm.bmv_count = MAXEXTNUM;
+	bm.bmv_count = fieinfo->fi_extents_max;
 	bm.bmv_iflags = BMV_IF_PREALLOC;
 	if (fieinfo->fi_flags & FIEMAP_FLAG_XATTR)
 		bm.bmv_iflags |= BMV_IF_ATTRFORK;

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Reviewed-by on v2 1/1
#4 lore raw
Quoted 9 lines · Eric Sandeen

commit 6321e3ed2acf3ee9643cdd403e1c88605d7944ba caused the full bmv_count's worth of getbmapx structures to get allocated; telling it to do MAXEXTNUM was a bit insane, resulting in ENOMEM every time.

Chop it down to something reasonable, the number of slots in the caller's input buffer. If this is too large the caller may get ENOMEM but the reason should not be a mystery, and they can try again with something smaller.

This version looks a lot better. Thanks.

Quoted 1 line
Quoted 23 lines

---

V2: set it to the fiemap info's fi_extents_max

fs/xfs/linux-2.6/xfs_iops.c +1 -2
index 58973bb..954d701 100644
@@ -680,8 +680,7 @@ xfs_vn_fiemap(
 	else
 		bm.bmv_length = BTOBB(length);
 
-	/* our formatter will tell xfs_getbmap when to stop. */
-	bm.bmv_count = MAXEXTNUM;
+	bm.bmv_count = fieinfo->fi_extents_max;
 	bm.bmv_iflags = BMV_IF_PREALLOC;
 	if (fieinfo->fi_flags & FIEMAP_FLAG_XATTR)
 		bm.bmv_iflags |= BMV_IF_ATTRFORK;

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Signature · 9 lines
Olaf Weber                 SGI               Phone:  +31(0)30-6696752
                           Veldzigt 2b       Fax:    +31(0)30-6696799
Technical Lead             3454 PW de Meern  Vnet:   955-7151
Storage Software           The Netherlands   Email:  [email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#5 lore raw

PATCH v3 1/1 reduce bmv_count in xfs_vn_fiemap

In reply to Eric Sandeen

commit 6321e3ed2acf3ee9643cdd403e1c88605d7944ba caused the full bmv_count's worth of getbmapx structures to get allocated; telling it to do MAXEXTNUM was a bit insane, resulting in ENOMEM every time.

Chop it down to something reasonable, the number of slots in the caller's input buffer. If this is too large the caller may get ENOMEM but the reason should not be a mystery, and they can try again with something smaller.

We add 1 to the value because in the normal getbmap world, bmv_count includes the header and xfs_getbmap does:

        nex = bmv->bmv_count - 1;
        if (nex <= 0)
                return XFS_ERROR(EINVAL);

---

V2: set it to the fiemap info's fi_extents_max
V3: bump up by one to accomodate expected bmv header in core

fs/xfs/linux-2.6/xfs_iops.c +1 -2
index 58973bb..954d701 100644
@@ -680,8 +680,7 @@ xfs_vn_fiemap(
 	else
 		bm.bmv_length = BTOBB(length);
 
-	/* our formatter will tell xfs_getbmap when to stop. */
-	bm.bmv_count = MAXEXTNUM;
+	bm.bmv_count = fieinfo->fi_extents_max + 1;
 	bm.bmv_iflags = BMV_IF_PREALLOC;
 	if (fieinfo->fi_flags & FIEMAP_FLAG_XATTR)
 		bm.bmv_iflags |= BMV_IF_ATTRFORK;

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Reviewed-by on v3 1/1
#6 lore raw
Quoted 10 lines · Eric Sandeen

commit 6321e3ed2acf3ee9643cdd403e1c88605d7944ba caused the full bmv_count's worth of getbmapx structures to get allocated; telling it to do MAXEXTNUM was a bit insane, resulting in ENOMEM every time.

Chop it down to something reasonable, the number of slots in the caller's input buffer. If this is too large the caller may get ENOMEM but the reason should not be a mystery, and they can try again with something smaller.

Quoted 8 lines

We add 1 to the value because in the normal getbmap world, bmv_count includes the header and xfs_getbmap does:

        nex = bmv->bmv_count - 1;
        if (nex <= 0)
                return XFS_ERROR(EINVAL);
Quoted 19 lines

---

V2: set it to the fiemap info's fi_extents_max
V3: bump up by one to accomodate expected bmv header in core

fs/xfs/linux-2.6/xfs_iops.c +1 -2
index 58973bb..954d701 100644
@@ -680,8 +680,7 @@ xfs_vn_fiemap(
 	else
 		bm.bmv_length = BTOBB(length);
 
-	/* our formatter will tell xfs_getbmap when to stop. */
-	bm.bmv_count = MAXEXTNUM;
+	bm.bmv_count = fieinfo->fi_extents_max + 1;
 	bm.bmv_iflags = BMV_IF_PREALLOC;
 	if (fieinfo->fi_flags & FIEMAP_FLAG_XATTR)
 		bm.bmv_iflags |= BMV_IF_ATTRFORK;
Signature · 9 lines
Olaf Weber                 SGI               Phone:  +31(0)30-6696752
                           Veldzigt 2b       Fax:    +31(0)30-6696799
Technical Lead             3454 PW de Meern  Vnet:   955-7151
Storage Software           The Netherlands   Email:  [email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Reviewed-by on v3 1/1
#7 lore raw
In reply to Eric Sandeen
Quoted 1 line · Eric Sandeen

V3: bump up by one to accomodate expected bmv header in core

That is probably woth a comment next to setting it in the code.

Otherwise looks good to me, and thanks for the fiemap tests so that we get test coverage for this.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#8 lore raw

PATCH v4 1/1 reduce bmv_count in xfs_vn_fiemap

In reply to Eric Sandeen

commit 6321e3ed2acf3ee9643cdd403e1c88605d7944ba caused the full bmv_count's worth of getbmapx structures to get allocated; telling it to do MAXEXTNUM was a bit insane, resulting in ENOMEM every time.

Chop it down to something reasonable, the number of slots in the caller's input buffer. If this is too large the caller may get ENOMEM but the reason should not be a mystery, and they can try again with something smaller.

We add 1 to the value because in the normal getbmap world, bmv_count includes the header and xfs_getbmap does:

        nex = bmv->bmv_count - 1;
        if (nex <= 0)
                return XFS_ERROR(EINVAL);

---

V2: set it to the fiemap info's fi_extents_max
V3: bump up by one to accomodate expected bmv header in core
V4: comments

fs/xfs/linux-2.6/xfs_iops.c +2 -2
index 58973bb..8070b34 100644
@@ -680,8 +680,8 @@ xfs_vn_fiemap(
 	else
 		bm.bmv_length = BTOBB(length);
 
-	/* our formatter will tell xfs_getbmap when to stop. */
-	bm.bmv_count = MAXEXTNUM;
+	/* We add one because in getbmap world count includes the header */
+	bm.bmv_count = fieinfo->fi_extents_max + 1;
 	bm.bmv_iflags = BMV_IF_PREALLOC;
 	if (fieinfo->fi_flags & FIEMAP_FLAG_XATTR)
 		bm.bmv_iflags |= BMV_IF_ATTRFORK;

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Reviewed-by on v4 1/1
#9 lore raw
Quoted 10 lines · Eric Sandeen

commit 6321e3ed2acf3ee9643cdd403e1c88605d7944ba caused the full bmv_count's worth of getbmapx structures to get allocated; telling it to do MAXEXTNUM was a bit insane, resulting in ENOMEM every time.

Chop it down to something reasonable, the number of slots in the caller's input buffer. If this is too large the caller may get ENOMEM but the reason should not be a mystery, and they can try again with something smaller.

Quoted 8 lines

We add 1 to the value because in the normal getbmap world, bmv_count includes the header and xfs_getbmap does:

        nex = bmv->bmv_count - 1;
        if (nex <= 0)
                return XFS_ERROR(EINVAL);

Thanks,

Olaf

Signature · 9 lines
Olaf Weber                 SGI               Phone:  +31(0)30-6696752
                           Veldzigt 2b       Fax:    +31(0)30-6696799
Technical Lead             3454 PW de Meern  Vnet:   955-7151
Storage Software           The Netherlands   Email:  [email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs