Discrepancy in 'df' output between kernel 3.0 and 3.2 for ext4?

PATCH New Zachary Mark 3 participants 10 messages Jun 13, 2012 – Jun 19, 2012 ext4 file system

Share & tools

Zachary Mark

author

2012-06-13 18:09 UTC #0 lore raw

Ext4 developers,

I recently upgraded my kernel from 3.0.24 to 3.2.18, and discovered that df is now reporting different statistics for my ext4 file systems (sda1 is ext3 and unaffected). Notice the difference between the 1K-blocks column and Used column between kernel versions (Available remains constant, as it is merely Used subtracted from the total size):

3.0.24:
[root@box ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 5080796 951932 3866608 20% /
tmpfs 12368192 32 12368160 1% /dev/shm

/dev/sdb1            2907178636    205876 2906972760   1% /media/9c2fb7c6-31db-4e29-b1b4-c199261301f9
/dev/sdc1            2907178636    205876 2906972760   1% /media/a564e4e1-9b49-40f0-afbd-e0ec4789da70
/dev/sdd1            2907178636    205876 2906972760   1% /media/f628cda4-ed56-4b8d-be1e-133917569b47
/dev/sde1            2907178636    359512 2906819124   1% /media/648da0a1-3b1b-42f7-a032-d592bb1435e2
/dev/sdf1            2907178636    205876 2906972760   1% /media/ce380174-701c-43e8-8986-256d23c3cbd2
/dev/sdg1            2907178636    205876 2906972760   1% /media/a61227f7-ec93-43ba-b657-6c95b81e51e7
/dev/sdh1            2907178636    205876 2906972760   1% /media/62b23b60-b70d-4097-ba35-0ffa57de8a44
/dev/sdi1            2907178636    205876 2906972760   1% /media/479bcb00-f4b0-41e5-9a06-56f02259912c
/dev/sdj1            2907178636    359512 2906819124   1% /media/0ccf9659-15d1-4e2b-81c7-b3d1fbb61076
/dev/sdk1            2907178636    205876 2906972760   1% /media/096c90c5-b28f-409a-9f5b-8ae09569ecb0
/dev/sdl1            2907178636    205876 2906972760   1% /media/671e195b-2e10-4d2d-83cf-96fd07daf3da
/dev/sdm1            2907178636    205876 2906972760   1% /media/0ec3c3ed-374f-4d52-9ccf-f8a42fb246c5

3.2.18:
[root@box ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 5080796 953576 3864964 20% /
tmpfs 12368792 32 12368760 1% /dev/shm

/dev/sdb1            2928918024  21945264 2906972760   1% /media/9c2fb7c6-31db-4e29-b1b4-c199261301f9
/dev/sdc1            2928918024  21945264 2906972760   1% /media/a564e4e1-9b49-40f0-afbd-e0ec4789da70
/dev/sdd1            2928918024  21945264 2906972760   1% /media/f628cda4-ed56-4b8d-be1e-133917569b47
/dev/sde1            2928918024  22098900 2906819124   1% /media/648da0a1-3b1b-42f7-a032-d592bb1435e2
/dev/sdf1            2928918024  21945264 2906972760   1% /media/ce380174-701c-43e8-8986-256d23c3cbd2
/dev/sdg1            2928918024  21945264 2906972760   1% /media/a61227f7-ec93-43ba-b657-6c95b81e51e7
/dev/sdh1            2928918024  21945264 2906972760   1% /media/62b23b60-b70d-4097-ba35-0ffa57de8a44
/dev/sdi1            2928918024  21945264 2906972760   1% /media/479bcb00-f4b0-41e5-9a06-56f02259912c
/dev/sdj1            2928918024  22098900 2906819124   1% /media/0ccf9659-15d1-4e2b-81c7-b3d1fbb61076
/dev/sdk1            2928918024  21945264 2906972760   1% /media/096c90c5-b28f-409a-9f5b-8ae09569ecb0
/dev/sdl1            2928918024  21945264 2906972760   1% /media/671e195b-2e10-4d2d-83cf-96fd07daf3da
/dev/sdm1            2928918024  21945264 2906972760   1% /media/0ec3c3ed-374f-4d52-9ccf-f8a42fb246c5

I bisected the change to the following commit:

f975d6bcc7a698a10cc755115e27d3612dcfe322 is the first bad commit commit f975d6bcc7a698a10cc755115e27d3612dcfe322

Author: Theodore Ts'o <[email protected]>
Date:   Fri Sep 9 19:00:51 2011 -0400

ext4: teach ext4_statfs() to deal with clusters if bigalloc is enabled

Signed-off-by "Theodore Ts'o" <[email protected]>

Reverting this commit on top of 3.2.18 returns the output to pre-3.2 levels.

Here's my /proc/mounts:

[root@box ~]# cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,relatime,errors=continue,user_xattr,acl,barrier=1,data=ordered 0 0 /dev /dev tmpfs rw,relatime,mode=755 0 0
/proc /proc proc rw,relatime 0 0
/sys /sys sysfs rw,relatime 0 0
devpts /dev/pts devpts rw,relatime,gid=5,mode=620 0 0
tmpfs /dev/shm tmpfs rw,relatime 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0
/dev/sdb1 /media/9c2fb7c6-31db-4e29-b1b4-c199261301f9 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdc1 /media/a564e4e1-9b49-40f0-afbd-e0ec4789da70 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdd1 /media/f628cda4-ed56-4b8d-be1e-133917569b47 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sde1 /media/648da0a1-3b1b-42f7-a032-d592bb1435e2 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdf1 /media/ce380174-701c-43e8-8986-256d23c3cbd2 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdg1 /media/a61227f7-ec93-43ba-b657-6c95b81e51e7 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdh1 /media/62b23b60-b70d-4097-ba35-0ffa57de8a44 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdi1 /media/479bcb00-f4b0-41e5-9a06-56f02259912c ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdj1 /media/0ccf9659-15d1-4e2b-81c7-b3d1fbb61076 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdk1 /media/096c90c5-b28f-409a-9f5b-8ae09569ecb0 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdl1 /media/671e195b-2e10-4d2d-83cf-96fd07daf3da ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdm1 /media/0ec3c3ed-374f-4d52-9ccf-f8a42fb246c5 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0

Is this discrepancy between the df outputs on the two kernel versions expected given my mount options? I decided to come to the list because I don't have the technical depth with regard to ext4 to be able to analyze the ext4_statfs changes that went into making bigalloc work, and I haven't found any reports of similar symptoms via web searches or the Documentation/filesystems/ext4.txt. This is not a production machine, so I am free to make any alterations or patches needed (if such a thing is required). Let me know if there is anything additional I need to provide.

Also, I'm not subscribed to the list, so please cc-me directly on any replies.

Thank you in advance,

Zachary Mark

Eric Sandeen

2012-06-13 18:27 UTC #1 lore raw

Quoted 78 lines · Zachary Mark

Ext4 developers,
I recently upgraded my kernel from 3.0.24 to 3.2.18, and discovered that df is now reporting different statistics for my ext4 file systems (sda1 is ext3 and unaffected). Notice the difference between the 1K-blocks column and Used column between kernel versions (Available remains constant, as it is merely Used subtracted from the total size):
3.0.24:
[root@box ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 5080796 951932 3866608 20% /
tmpfs 12368192 32 12368160 1% /dev/shm
/dev/sdb1            2907178636    205876 2906972760   1% /media/9c2fb7c6-31db-4e29-b1b4-c199261301f9
/dev/sdc1            2907178636    205876 2906972760   1% /media/a564e4e1-9b49-40f0-afbd-e0ec4789da70
/dev/sdd1            2907178636    205876 2906972760   1% /media/f628cda4-ed56-4b8d-be1e-133917569b47
/dev/sde1            2907178636    359512 2906819124   1% /media/648da0a1-3b1b-42f7-a032-d592bb1435e2
/dev/sdf1            2907178636    205876 2906972760   1% /media/ce380174-701c-43e8-8986-256d23c3cbd2
/dev/sdg1            2907178636    205876 2906972760   1% /media/a61227f7-ec93-43ba-b657-6c95b81e51e7
/dev/sdh1            2907178636    205876 2906972760   1% /media/62b23b60-b70d-4097-ba35-0ffa57de8a44
/dev/sdi1            2907178636    205876 2906972760   1% /media/479bcb00-f4b0-41e5-9a06-56f02259912c
/dev/sdj1            2907178636    359512 2906819124   1% /media/0ccf9659-15d1-4e2b-81c7-b3d1fbb61076
/dev/sdk1            2907178636    205876 2906972760   1% /media/096c90c5-b28f-409a-9f5b-8ae09569ecb0
/dev/sdl1            2907178636    205876 2906972760   1% /media/671e195b-2e10-4d2d-83cf-96fd07daf3da
/dev/sdm1            2907178636    205876 2906972760   1% /media/0ec3c3ed-374f-4d52-9ccf-f8a42fb246c5
3.2.18:
[root@box ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 5080796 953576 3864964 20% /
tmpfs 12368792 32 12368760 1% /dev/shm
/dev/sdb1            2928918024  21945264 2906972760   1% /media/9c2fb7c6-31db-4e29-b1b4-c199261301f9
/dev/sdc1            2928918024  21945264 2906972760   1% /media/a564e4e1-9b49-40f0-afbd-e0ec4789da70
/dev/sdd1            2928918024  21945264 2906972760   1% /media/f628cda4-ed56-4b8d-be1e-133917569b47
/dev/sde1            2928918024  22098900 2906819124   1% /media/648da0a1-3b1b-42f7-a032-d592bb1435e2
/dev/sdf1            2928918024  21945264 2906972760   1% /media/ce380174-701c-43e8-8986-256d23c3cbd2
/dev/sdg1            2928918024  21945264 2906972760   1% /media/a61227f7-ec93-43ba-b657-6c95b81e51e7
/dev/sdh1            2928918024  21945264 2906972760   1% /media/62b23b60-b70d-4097-ba35-0ffa57de8a44
/dev/sdi1            2928918024  21945264 2906972760   1% /media/479bcb00-f4b0-41e5-9a06-56f02259912c
/dev/sdj1            2928918024  22098900 2906819124   1% /media/0ccf9659-15d1-4e2b-81c7-b3d1fbb61076
/dev/sdk1            2928918024  21945264 2906972760   1% /media/096c90c5-b28f-409a-9f5b-8ae09569ecb0
/dev/sdl1            2928918024  21945264 2906972760   1% /media/671e195b-2e10-4d2d-83cf-96fd07daf3da
/dev/sdm1            2928918024  21945264 2906972760   1% /media/0ec3c3ed-374f-4d52-9ccf-f8a42fb246c5
I bisected the change to the following commit:
f975d6bcc7a698a10cc755115e27d3612dcfe322 is the first bad commit commit f975d6bcc7a698a10cc755115e27d3612dcfe322
Author: Theodore Ts'o <[email protected]>
Date:   Fri Sep 9 19:00:51 2011 -0400
ext4: teach ext4_statfs() to deal with clusters if bigalloc is enabled
Signed-off-by "Theodore Ts'o" <[email protected]>
Reverting this commit on top of 3.2.18 returns the output to pre-3.2 levels.
Here's my /proc/mounts:
[root@box ~]# cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,relatime,errors=continue,user_xattr,acl,barrier=1,data=ordered 0 0 /dev /dev tmpfs rw,relatime,mode=755 0 0
/proc /proc proc rw,relatime 0 0
/sys /sys sysfs rw,relatime 0 0
devpts /dev/pts devpts rw,relatime,gid=5,mode=620 0 0
tmpfs /dev/shm tmpfs rw,relatime 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0
/dev/sdb1 /media/9c2fb7c6-31db-4e29-b1b4-c199261301f9 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdc1 /media/a564e4e1-9b49-40f0-afbd-e0ec4789da70 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdd1 /media/f628cda4-ed56-4b8d-be1e-133917569b47 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sde1 /media/648da0a1-3b1b-42f7-a032-d592bb1435e2 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdf1 /media/ce380174-701c-43e8-8986-256d23c3cbd2 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdg1 /media/a61227f7-ec93-43ba-b657-6c95b81e51e7 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdh1 /media/62b23b60-b70d-4097-ba35-0ffa57de8a44 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdi1 /media/479bcb00-f4b0-41e5-9a06-56f02259912c ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdj1 /media/0ccf9659-15d1-4e2b-81c7-b3d1fbb61076 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdk1 /media/096c90c5-b28f-409a-9f5b-8ae09569ecb0 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdl1 /media/671e195b-2e10-4d2d-83cf-96fd07daf3da ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdm1 /media/0ec3c3ed-374f-4d52-9ccf-f8a42fb246c5 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0

Quoted 6 lines

Is this discrepancy between the df outputs on the two kernel versions expected given my mount options? I decided to come to the list because I don't have the technical depth with regard to ext4 to be able to analyze the ext4_statfs changes that went into making bigalloc work, and I haven't found any reports of similar symptoms via web searches or the Documentation/filesystems/ext4.txt. This is

https://bugzilla.redhat.com/show_bug.cgi?id=830412 just came in and is probably the same root cause.

Thanks for the bisect! I'll let Ted worry about it for now, at least until I have more time.

-Eric

Quoted 15 lines

not a production machine, so I am free to make any alterations or patches needed (if such a thing is required). Let me know if there is anything additional I need to provide.
Also, I'm not subscribed to the list, so please cc-me directly on any replies.
Thank you in advance,
Zachary Mark
Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Zachary Mark

author

2012-06-13 22:21 UTC #2 lore raw

Quoted 86 lines · Eric Sandeen

Quoted 78 lines · Zachary Mark
Ext4 developers,
I recently upgraded my kernel from 3.0.24 to 3.2.18, and discovered that df is now reporting different statistics for my ext4 file systems (sda1 is ext3 and unaffected). Notice the difference between the 1K-blocks column and Used column between kernel versions (Available remains constant, as it is merely Used subtracted from the total size):
3.0.24:
[root@box ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 5080796 951932 3866608 20% /
tmpfs 12368192 32 12368160 1% /dev/shm
/dev/sdb1            2907178636    205876 2906972760   1% /media/9c2fb7c6-31db-4e29-b1b4-c199261301f9
/dev/sdc1            2907178636    205876 2906972760   1% /media/a564e4e1-9b49-40f0-afbd-e0ec4789da70
/dev/sdd1            2907178636    205876 2906972760   1% /media/f628cda4-ed56-4b8d-be1e-133917569b47
/dev/sde1            2907178636    359512 2906819124   1% /media/648da0a1-3b1b-42f7-a032-d592bb1435e2
/dev/sdf1            2907178636    205876 2906972760   1% /media/ce380174-701c-43e8-8986-256d23c3cbd2
/dev/sdg1            2907178636    205876 2906972760   1% /media/a61227f7-ec93-43ba-b657-6c95b81e51e7
/dev/sdh1            2907178636    205876 2906972760   1% /media/62b23b60-b70d-4097-ba35-0ffa57de8a44
/dev/sdi1            2907178636    205876 2906972760   1% /media/479bcb00-f4b0-41e5-9a06-56f02259912c
/dev/sdj1            2907178636    359512 2906819124   1% /media/0ccf9659-15d1-4e2b-81c7-b3d1fbb61076
/dev/sdk1            2907178636    205876 2906972760   1% /media/096c90c5-b28f-409a-9f5b-8ae09569ecb0
/dev/sdl1            2907178636    205876 2906972760   1% /media/671e195b-2e10-4d2d-83cf-96fd07daf3da
/dev/sdm1            2907178636    205876 2906972760   1% /media/0ec3c3ed-374f-4d52-9ccf-f8a42fb246c5
3.2.18:
[root@box ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 5080796 953576 3864964 20% /
tmpfs 12368792 32 12368760 1% /dev/shm
/dev/sdb1            2928918024  21945264 2906972760   1% /media/9c2fb7c6-31db-4e29-b1b4-c199261301f9
/dev/sdc1            2928918024  21945264 2906972760   1% /media/a564e4e1-9b49-40f0-afbd-e0ec4789da70
/dev/sdd1            2928918024  21945264 2906972760   1% /media/f628cda4-ed56-4b8d-be1e-133917569b47
/dev/sde1            2928918024  22098900 2906819124   1% /media/648da0a1-3b1b-42f7-a032-d592bb1435e2
/dev/sdf1            2928918024  21945264 2906972760   1% /media/ce380174-701c-43e8-8986-256d23c3cbd2
/dev/sdg1            2928918024  21945264 2906972760   1% /media/a61227f7-ec93-43ba-b657-6c95b81e51e7
/dev/sdh1            2928918024  21945264 2906972760   1% /media/62b23b60-b70d-4097-ba35-0ffa57de8a44
/dev/sdi1            2928918024  21945264 2906972760   1% /media/479bcb00-f4b0-41e5-9a06-56f02259912c
/dev/sdj1            2928918024  22098900 2906819124   1% /media/0ccf9659-15d1-4e2b-81c7-b3d1fbb61076
/dev/sdk1            2928918024  21945264 2906972760   1% /media/096c90c5-b28f-409a-9f5b-8ae09569ecb0
/dev/sdl1            2928918024  21945264 2906972760   1% /media/671e195b-2e10-4d2d-83cf-96fd07daf3da
/dev/sdm1            2928918024  21945264 2906972760   1% /media/0ec3c3ed-374f-4d52-9ccf-f8a42fb246c5
I bisected the change to the following commit:
f975d6bcc7a698a10cc755115e27d3612dcfe322 is the first bad commit commit f975d6bcc7a698a10cc755115e27d3612dcfe322
Author: Theodore Ts'o<[email protected]>
Date:   Fri Sep 9 19:00:51 2011 -0400
ext4: teach ext4_statfs() to deal with clusters if bigalloc is enabled
Signed-off-by "Theodore Ts'o"<[email protected]>
Reverting this commit on top of 3.2.18 returns the output to pre-3.2 levels.
Here's my /proc/mounts:
[root@box ~]# cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,relatime,errors=continue,user_xattr,acl,barrier=1,data=ordered 0 0 /dev /dev tmpfs rw,relatime,mode=755 0 0
/proc /proc proc rw,relatime 0 0
/sys /sys sysfs rw,relatime 0 0
devpts /dev/pts devpts rw,relatime,gid=5,mode=620 0 0
tmpfs /dev/shm tmpfs rw,relatime 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0
/dev/sdb1 /media/9c2fb7c6-31db-4e29-b1b4-c199261301f9 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdc1 /media/a564e4e1-9b49-40f0-afbd-e0ec4789da70 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdd1 /media/f628cda4-ed56-4b8d-be1e-133917569b47 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sde1 /media/648da0a1-3b1b-42f7-a032-d592bb1435e2 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdf1 /media/ce380174-701c-43e8-8986-256d23c3cbd2 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdg1 /media/a61227f7-ec93-43ba-b657-6c95b81e51e7 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdh1 /media/62b23b60-b70d-4097-ba35-0ffa57de8a44 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdi1 /media/479bcb00-f4b0-41e5-9a06-56f02259912c ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdj1 /media/0ccf9659-15d1-4e2b-81c7-b3d1fbb61076 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdk1 /media/096c90c5-b28f-409a-9f5b-8ae09569ecb0 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdl1 /media/671e195b-2e10-4d2d-83cf-96fd07daf3da ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0 /dev/sdm1 /media/0ec3c3ed-374f-4d52-9ccf-f8a42fb246c5 ext4 rw,noatime,user_xattr,barrier=1,data=ordered 0 0
Quoted 6 lines
Is this discrepancy between the df outputs on the two kernel versions expected given my mount options? I decided to come to the list because I don't have the technical depth with regard to ext4 to be able to analyze the ext4_statfs changes that went into making bigalloc work, and I haven't found any reports of similar symptoms via web searches or the Documentation/filesystems/ext4.txt. This is

Quoted 5 lines

https://bugzilla.redhat.com/show_bug.cgi?id=830412 just came in and is probably the same root cause.
Thanks for the bisect! I'll let Ted worry about it for now, at least until I have more time.
-Eric

Thanks for the link. I'll be monitoring the bug report, but definitely keep me updated if possible.

-- Zach

Quoted 17 lines

Quoted 16 lines
not a production machine, so I am free to make any alterations or patches needed (if such a thing is required). Let me know if there is anything additional I need to provide.
Also, I'm not subscribed to the list, so please cc-me directly on any replies.
Thank you in advance,
Zachary Mark
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Eric Sandeen

2012-06-14 19:00 UTC #3 lore raw

In reply to Eric Sandeen

Quoted 9 lines · Eric Sandeen

Quoted 8 lines · Zachary Mark
Ext4 developers,
I recently upgraded my kernel from 3.0.24 to 3.2.18, and discovered that df is now reporting different statistics for my ext4 file systems (sda1 is ext3 and unaffected). Notice the difference between the 1K-blocks column and Used column between kernel versions (Available remains constant, as it is merely Used subtracted from the total size):

...

Quoted 10 lines

Quoted 6 lines
Is this discrepancy between the df outputs on the two kernel versions expected given my mount options? I decided to come to the list because I don't have the technical depth with regard to ext4 to be able to analyze the ext4_statfs changes that went into making bigalloc work, and I haven't found any reports of similar symptoms via web searches or the Documentation/filesystems/ext4.txt. This is
https://bugzilla.redhat.com/show_bug.cgi?id=830412 just came in and is probably the same root cause.
Thanks for the bisect! I'll let Ted worry about it for now, at least until I have more time.

Ted, one thing that seems very weird to me. When using BSD-style df, which is supposed to ignore basic metadata overhead, shouldn't a freshly mkfs'd filesystem always show free blocks == total blocks? It doesn't do that either before or after your changes, which seems odd to me. Am I misunderstanding what "bsddf" is supposed to do?

-eric

Theodore Ts'o

maintainer

2012-06-18 21:45 UTC #4 lore raw

In reply to Zachary Mark

Zachary,

Hopefully this should fix the problem which you noted.

Theodore Ts'o (2):
ext4: pass a char * to ext4_count_free() instead of a buffer_head ptr
ext4: fix overhead calculation used by ext4_statfs()

 fs/ext4/balloc.c |   3 +-
 fs/ext4/bitmap.c |  12 +---
 fs/ext4/ext4.h   |   6 +-
 fs/ext4/ialloc.c |   3 +-
 fs/ext4/resize.c |   7 ++-
 fs/ext4/super.c  | 174 +++++++++++++++++++++++++++++++++++++++----------------
 6 files changed, 140 insertions(+), 65 deletions(-)

Signature · 1 line

1.7.11

Theodore Ts'o

maintainer

2012-06-18 21:45 UTC #5 lore raw

PATCH 1/2 ext4: pass a char * to ext4_count_free() instead of a buffer_head ptr

Make it possible for ext4_count_free to operate on buffers and not just data in buffer_heads.

Signed-off-by "Theodore Ts'o" <[email protected]>

---
 fs/ext4/balloc.c | 3 ++-
 fs/ext4/bitmap.c | 8 +++-----
 fs/ext4/ext4.h   | 2 +-
 fs/ext4/ialloc.c | 3 ++-
 4 files changed, 8 insertions(+), 8 deletions(-)

fs/ext4/balloc.c +2 -1

index cee7812..d23b31c 100644
@@ -609,7 +609,8 @@ ext4_fsblk_t ext4_count_free_clusters(struct super_block *sb)
 		if (bitmap_bh == NULL)
 			continue;
 
-		x = ext4_count_free(bitmap_bh, sb->s_blocksize);
+		x = ext4_count_free(bitmap_bh->b_data,
+				    EXT4_BLOCKS_PER_GROUP(sb) / 8);
 		printk(KERN_DEBUG "group %u: stored = %d, counted = %u\n",
 			i, ext4_free_group_clusters(sb, gdp), x);
 		bitmap_count += x;

fs/ext4/bitmap.c +3 -5

index b319721..7e86a6d 100644
@@ -15,15 +15,13 @@
 
 static const int nibblemap[] = {4, 3, 3, 2, 3, 2, 2, 1, 3, 2, 2, 1, 2, 1, 1, 0};
 
-unsigned int ext4_count_free(struct buffer_head *map, unsigned int numchars)
+unsigned int ext4_count_free(char *bitmap, unsigned int numchars)
 {
 	unsigned int i, sum = 0;
 
-	if (!map)
-		return 0;
 	for (i = 0; i < numchars; i++)
-		sum += nibblemap[map->b_data[i] & 0xf] +
-			nibblemap[(map->b_data[i] >> 4) & 0xf];
+		sum += nibblemap[bitmap[i] & 0xf] +
+			nibblemap[(bitmap[i] >> 4) & 0xf];
 	return sum;
 }

fs/ext4/ext4.h +1 -1

index cfc4e01..293fa1c 100644
@@ -1852,7 +1852,7 @@ struct mmpd_data {
 # define NORET_AND	noreturn,
 
 /* bitmap.c */
-extern unsigned int ext4_count_free(struct buffer_head *, unsigned);
+extern unsigned int ext4_count_free(char *bitmap, unsigned numchars);
 void ext4_inode_bitmap_csum_set(struct super_block *sb, ext4_group_t group,
 				struct ext4_group_desc *gdp,
 				struct buffer_head *bh, int sz);

fs/ext4/ialloc.c +2 -1

index d48e8b1..6866bc2 100644
@@ -1054,7 +1054,8 @@ unsigned long ext4_count_free_inodes(struct super_block *sb)
 		if (!bitmap_bh)
 			continue;
 
-		x = ext4_count_free(bitmap_bh, EXT4_INODES_PER_GROUP(sb) / 8);
+		x = ext4_count_free(bitmap_bh->b_data,
+				    EXT4_INODES_PER_GROUP(sb) / 8);
 		printk(KERN_DEBUG "group %lu: stored = %d, counted = %lu\n",
 			(unsigned long) i, ext4_free_inodes_count(sb, gdp), x);
 		bitmap_count += x;

Signature · 1 line

1.7.11

Theodore Ts'o

maintainer

2012-06-18 21:45 UTC #6 lore raw

PATCH 2/2 ext4: fix overhead calculation used by ext4_statfs()

In reply to Theodore Ts'o

Commit f975d6bcc7a introduced bug which caused ext4_statfs() to miscalculate the number of file system overhead blocks. This causes the f_blocks field in the statfs structure to be larger than it should be. This would in turn cause the "df" output to show the number of data blocks in the file system and the number of data blocks used to be larger than they should be.

Signed-off-by "Theodore Ts'o" <[email protected]>

---
 fs/ext4/bitmap.c |   4 --
 fs/ext4/ext4.h   |   4 +-
 fs/ext4/resize.c |   7 ++-
 fs/ext4/super.c  | 174 +++++++++++++++++++++++++++++++++++++++----------------
 4 files changed, 132 insertions(+), 57 deletions(-)

fs/ext4/bitmap.c +0 -4

index 7e86a6d..a94b9c6 100644
@@ -11,8 +11,6 @@
 #include <linux/jbd2.h>
 #include "ext4.h"
 
-#ifdef EXT4FS_DEBUG
-
 static const int nibblemap[] = {4, 3, 3, 2, 3, 2, 2, 1, 3, 2, 2, 1, 2, 1, 1, 0};
 
 unsigned int ext4_count_free(char *bitmap, unsigned int numchars)

@@ -25,8 +23,6 @@ unsigned int ext4_count_free(char *bitmap, unsigned int numchars)
 	return sum;
 }
 
-#endif  /*  EXT4FS_DEBUG  */
-
 int ext4_inode_bitmap_csum_verify(struct super_block *sb, ext4_group_t group,
 				  struct ext4_group_desc *gdp,
 				  struct buffer_head *bh, int sz)

fs/ext4/ext4.h +2 -2

index 293fa1c..01434f2 100644
@@ -1161,8 +1161,7 @@ struct ext4_sb_info {
 	unsigned long s_desc_per_block;	/* Number of group descriptors per block */
 	ext4_group_t s_groups_count;	/* Number of groups in the fs */
 	ext4_group_t s_blockfile_groups;/* Groups acceptable for non-extent files */
-	unsigned long s_overhead_last;  /* Last calculated overhead */
-	unsigned long s_blocks_last;    /* Last seen block count */
+	unsigned long s_overhead;  /* # of fs overhead clusters */
 	unsigned int s_cluster_ratio;	/* Number of blocks per cluster */
 	unsigned int s_cluster_bits;	/* log2 of s_cluster_ratio */
 	loff_t s_bitmap_maxbytes;	/* max bytes for bitmap files */

@@ -2037,6 +2036,7 @@ extern int ext4_group_extend(struct super_block *sb,
 extern int ext4_resize_fs(struct super_block *sb, ext4_fsblk_t n_blocks_count);
 
 /* super.c */
+extern int ext4_calculate_overhead(struct super_block *sb);
 extern int ext4_superblock_csum_verify(struct super_block *sb,
 				       struct ext4_super_block *es);
 extern void ext4_superblock_csum_set(struct super_block *sb,

fs/ext4/resize.c +6 -1

index 7ea6cbb..17d38de 100644
@@ -1197,7 +1197,7 @@ static void ext4_update_super(struct super_block *sb,
 	struct ext4_new_group_data *group_data = flex_gd->groups;
 	struct ext4_sb_info *sbi = EXT4_SB(sb);
 	struct ext4_super_block *es = sbi->s_es;
-	int i;
+	int i, ret;
 
 	BUG_ON(flex_gd->count == 0 || group_data == NULL);
 	/*

@@ -1272,6 +1272,11 @@ static void ext4_update_super(struct super_block *sb,
 			   &sbi->s_flex_groups[flex_group].free_inodes);
 	}
 
+	/*
+	 * Update the fs overhead information
+	 */
+	ext4_calculate_overhead(sb);
+
 	if (test_opt(sb, DEBUG))
 		printk(KERN_DEBUG "EXT4-fs: added group %u:"
 		       "%llu blocks(%llu free %llu reserved)\n", flex_gd->count,

fs/ext4/super.c +124 ~~-50~~

203 lines not on this page · raw message

Signature · 1 line

1.7.11

Zachary Mark

author

2012-06-19 19:55 UTC #7 lore raw

In reply to Theodore Ts'o

Quoted 9 lines · Theodore Ts'o

Zachary,
Hopefully this should fix the problem which you noted.
Theodore Ts'o (2): ext4: pass a char * to ext4_count_free() instead of a buffer_head ptr ext4: fix overhead calculation used by ext4_statfs()

Quoted 6 lines

fs/ext4/balloc.c | 3 +- fs/ext4/bitmap.c | 12 +--- fs/ext4/ext4.h
|   6 +- fs/ext4/ialloc.c |   3 +- fs/ext4/resize.c |   7 ++-
fs/ext4/super.c  | 174
+++++++++++++++++++++++++++++++++++++++---------------- 6 files changed, 140 insertions(+), 65 deletions(-)

Ted, thanks for the patches! I've tested your patches against 3.5~rc3.
I had to return the machine on which I first spotted the problem, but
here are results from a box with identical hardware:

df from 3.0:
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 5080796 891164 3927376 19% /
tmpfs 12368192 32 12368160 1% /dev/shm
/dev/sdh1 2907178636 205816 2906972820 1%
/media/012a0d3e-8210-4eb1-94a9-a2a1fdeb62f3
/dev/sdi1 2907178636 1056768 2906121868 1%
/media/20a46e68-8203-459e-8364-0626510c2ff9

df from 3.2.20 (identical to 3.5~rc3 without your patches):
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 5080796 928376 3890164 20% /
tmpfs 12368792 32 12368760 1% /dev/shm
/dev/sdh1 2928733612 21760792 2906972820 1%
/media/012a0d3e-8210-4eb1-94a9-a2a1fdeb62f3
/dev/sdi1 2928733612 22611744 2906121868 1%
/media/20a46e68-8203-459e-8364-0626510c2ff9

df from 3.5~rc3 with your patches applied (as they didn't apply to 3.2):
[root@lab-s2210-0331 20a46e68-8203-459e-8364-0626510c2ff9]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 5080796 909656 3908884 19% /
tmpfs 12368708 32 12368676 1% /dev/shm
/dev/sdh1 2907178636 205816 2906972820 1%
/media/012a0d3e-8210-4eb1-94a9-a2a1fdeb62f3
/dev/sdi1 2907178636 1060936 2906117700 1%
/media/20a46e68-8203-459e-8364-0626510c2ff9

(duplicate parts of output removed)

sdh1 is mostly empty. sdi1 has about 6700 128k files written to it plus everything on sdh1. There seems to be slightly more overhead accounted for after your patches. Not sure if this is to be expected or not. In any case, they seem to be a step in the right direction if not an outright solution. Let me know if you need anything additional from me.

-- Zachary Mark

Ted Ts'o

maintainer

2012-06-19 20:13 UTC #8 lore raw

Quoted 16 lines · Zachary Mark

Ted, thanks for the patches! I've tested your patches against 3.5~rc3. I had to return the machine on which I first spotted the problem, but here are results from a box with identical hardware:
df from 3.0:
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sdh1            2907178636    205816 2906972820   1%
/dev/sdi1            2907178636   1056768 2906121868   1%
df from 3.2.20 (identical to 3.5~rc3 without your patches):
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sdh1            2928733612  21760792 2906972820   1%
/dev/sdi1            2928733612  22611744 2906121868   1%

Quoted 7 lines

df from 3.5~rc3 with your patches applied (as they didn't apply to 3.2):
/dev/sdh1            2907178636    205816 2906972820   1%
/dev/sdi1            2907178636   1060936 2906117700   1%
sdh1 is mostly empty. sdi1 has about 6700 128k files written to it plus everything on sdh1. There seems to be slightly more overhead accounted for after your patches. Not sure if this is to be expected or not.

Hmm... it looks like df output /dev/sdh1 is identical between 3.0 and 3.5~rc3 with my patches. I'm not sure why there is a difference for /dev/sdi1. However, I note that the "Available" figure is the same between 3.0, 3.2.20 and 3.5~rc3 for /dev/sdh1, but there is a difference in the Available column between 3.2.20 and 3.5~rc3 for /dev/sdi3. Could it be that some files got written to /dev/sdi between your test run?

It would be good if we could get this sorted out. I was pretty careful to account for all of the fs overhead blocks between when I did my patch with an empty file system.

If this can't be accounted by more files being written to /dev/sdi1, could you send me the (compressed, since they will be large) output of dumpe2fs for /dev/sdh1 and /dev/sdi1 with the "df" output from your three test kernel so I can investigate further?

Thanks,

- Ted

Zachary Mark

author

2012-06-19 21:01 UTC #9 lore raw

Quoted 37 lines · Ted Ts'o

Quoted 16 lines · Zachary Mark
Ted, thanks for the patches! I've tested your patches against 3.5~rc3. I had to return the machine on which I first spotted the problem, but here are results from a box with identical hardware:
df from 3.0:
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sdh1            2907178636    205816 2906972820   1%
/dev/sdi1            2907178636   1056768 2906121868   1%
df from 3.2.20 (identical to 3.5~rc3 without your patches):
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sdh1            2928733612  21760792 2906972820   1%
/dev/sdi1            2928733612  22611744 2906121868   1%
Quoted 7 lines
df from 3.5~rc3 with your patches applied (as they didn't apply to 3.2):
/dev/sdh1            2907178636    205816 2906972820   1%
/dev/sdi1            2907178636   1060936 2906117700   1%
sdh1 is mostly empty. sdi1 has about 6700 128k files written to it plus everything on sdh1. There seems to be slightly more overhead accounted for after your patches. Not sure if this is to be expected or not.
Hmm... it looks like df output /dev/sdh1 is identical between 3.0 and 3.5~rc3 with my patches. I'm not sure why there is a difference for /dev/sdi1. However, I note that the "Available" figure is the same between 3.0, 3.2.20 and 3.5~rc3 for /dev/sdh1, but there is a difference in the Available column between 3.2.20 and 3.5~rc3 for /dev/sdi3. Could it be that some files got written to /dev/sdi between your test run?
It would be good if we could get this sorted out. I was pretty careful to account for all of the fs overhead blocks between when I did my patch with an empty file system.

Quoted 8 lines

If this can't be accounted by more files being written to /dev/sdi1, could you send me the (compressed, since they will be large) output of dumpe2fs for /dev/sdh1 and /dev/sdi1 with the "df" output from your three test kernel so I can investigate further?
Thanks,
- Ted

Actually, you're right, I must have screwed up my original test somehow.
I just repeated the test. The 3.0 and 3.5~rc3-patched numbers are
identical to each other, and the 3.2.20/3.5~rc3-unpatched are also identical to each other.

-- Zach