Possible performance problem introduced between kernel 2.6.15 and 2.6.17 on xfs code
Hi all,
i noticed that changing from kernel 2.6.15 to 2.6.17, gnomevfs-copy operations suffered an heavy performance hit (20 times slower).
I checked quickly (i am not a kernel hacker) and i think it could be related to libgnomevfs calling fadvice POSIX_FADV_DONTNEED (in gnome_vfs_forget_cache, disabling this behaviour gets back normal performance). I am not able to dig deeper than this, anyway it could be a starting point.
The relevant gnome bugzilla reference is the following: http://bugzilla.gnome.org/show_bug.cgi?id=363400
Francesco Degrassi
Quoted 9 lines · fdegrassi
Hi all,
i noticed that changing from kernel 2.6.15 to 2.6.17, gnomevfs-copy operations suffered an heavy performance hit (20 times slower).
I checked quickly (i am not a kernel hacker) and i think it could be related to libgnomevfs calling fadvicePOSIX_FADV_DONTNEED(ingnome_vfs_forget_cache, disabling this behaviour gets back normal performance). I am not able to dig deeper than this, anyway it could be a starting point.
The relevant gnome bugzilla reference is the following: http://bugzilla.gnome.org/show_bug.cgi?id=363400Francesco Degrassi
Is a copy command from the shell affected, or just gnome point-n-click?
-Eric
The specific problem is related to gnome point-n-click only.
Anyway, I could be wrong, but i don't think it is a bug in gnome because:
1. it affects specifically local XFS filesystems only
2. the problem is not present on kernel 2.6.15 (I tested 2.6.17 and 2.6.15, both on ubuntu 6.10)
3. the problem does not manifest if i disable the call to gnome_vfs_forget_cache, that, for what i understand, simply calls fadvice POSIX_FADV_DONTNEED.
More info can be found on gnome bugzilla entry.
Francesco Degrassi
Quoted 14 lines · Eric Sandeen
Quoted 9 lines · fdegrassi
Hi all,
i noticed that changing from kernel 2.6.15 to 2.6.17, gnomevfs-copy operations suffered an heavy performance hit (20 times slower).
I checked quickly (i am not a kernel hacker) and i think it could be related to libgnomevfs calling fadvicePOSIX_FADV_DONTNEED(ingnome_vfs_forget_cache, disabling this behaviour gets back normal performance). I am not able to dig deeper than this, anyway it could be a starting point.
The relevant gnome bugzilla reference is the following: http://bugzilla.gnome.org/show_bug.cgi?id=363400Francesco Degrassi
Is a copy command from the shell affected, or just gnome point-n-click?
-Eric
Quoted 9 lines · fdegrassi
The specific problem is related to gnome point-n-click only.
Anyway, I could be wrong, but i don't think it is a bug in gnome because:
1. it affects specifically local XFS filesystems only
2. the problem is not present on kernel 2.6.15 (I tested 2.6.17 and 2.6.15, both on ubuntu 6.10)
3. the problem does not manifest if i disable the call tognome_vfs_forget_cache, that, for what i understand, simply calls fadvicePOSIX_FADV_DONTNEED.More info can be found on gnome bugzilla entry.
Francesco Degrassi
Thanks. I didn't mean to suggest that it was necessarily a gnome bug, just wanted to narrow down the problem. Do you have any idea if other filesystems on 2.6.17 also show this slowdown?
-Eric
I tried ext2, ext3, jfs, reiser3, vfat and the problem does not show up. Anyway i discovered that if i create a regular file in an XFS partition, create an XFS filesystem on it as if it was a regular block device, and mount it with the loop device, the problem does NOT manifest. So it seems to affect XFS filesystems on regular block devices only ? I'm confused.
____________EXAMPLE________________
root@francesco:/tmp$ mount | grep tmp /dev/sda7 on /tmp type xfs (rw) root@francesco:/tmp$ dd if=/dev/zero of=bdev bs=1024 count=200000 root@francesco:/tmp$ mkfs.xfs -f bdev root@francesco:/tmp$ mkdir mpoint root@francesco:/tmp$ mount -o loop bdev mpoint/ root@francesco:/tmp$ time gnomevfs-copy \ /home/fdegrassi/Desktop/downloads/systemrescuecd-x86-1.2.18.iso mpoint/
real 0m3.998s
user 0m0.080s
sys 0m1.292s
root@francesco:/tmp$ time gnomevfs-copy \ /home/fdegrassi/Desktop/downloads/systemrescuecd-x86-1.2.18.iso /tmp
real 2m55.448s
user 0m0.100s
sys 0m1.984s
________________________END_____________________
Thanks for your time, Eric.
Francesco Degrassi
Quoted 19 lines · Eric Sandeen
Quoted 9 lines · fdegrassi
The specific problem is related to gnome point-n-click only. Anyway, I could be wrong, but i don't think it is a bug in gnome because:
1. it affects specifically local XFS filesystems only
2. the problem is not present on kernel 2.6.15 (I tested 2.6.17 and 2.6.15, both on ubuntu 6.10)
3. the problem does not manifest if i disable the call tognome_vfs_forget_cache, that, for what i understand, simply calls fadvicePOSIX_FADV_DONTNEED.Quoted 3 lines
More info can be found on gnome bugzilla entry.
Francesco DegrassiThanks. I didn't mean to suggest that it was necessarily a gnome bug, just wanted to narrow down the problem. Do you have any idea if other filesystems on 2.6.17 also show this slowdown?
-Eric
Quoted 6 lines · Francesco Degrassi
I tried ext2, ext3, jfs, reiser3, vfat and the problem does not show up. Anyway i discovered that if i create a regular file in an XFS partition, create an XFS filesystem on it as if it was a regular block device, and mount it with the loop device, the problem does NOT manifest. So it seems to affect XFS filesystems on regular block devices only ? I'm confused.
This is probably not related, but I remember that barriers were added sometime between 2.6.15 and 2.6.17 and probably barriers are not enabled on a loop filesystem. Maybe the gnome vfs thing does some sync which force flushes or such. I'm no expert, just guessing.
Just to invalidate this (probably wrong) guess, could you try to mount /tmp with "-o nobarrier" and re-test?
Iustin, who wishes md raid1 would support barriers.
Hi Iustin,
i tried and it and indeed the performance problem does not manifest if i mount the xfs filesystem with the -o nobarrier option. Seems like we are a little bit closer to the real problem now.
The code in gnomevfs-copy calls fadvice POSIX_FADV_DONTNEED (in gnome_vfs_forget_cache) and disabling this behaviour gets back normal performance, so it has something to do with that.
Thanks for your help
Francesco
Quoted 17 lines · Iustin Pop
Quoted 6 lines · Francesco Degrassi
I tried ext2, ext3, jfs, reiser3, vfat and the problem does not show up. Anyway i discovered that if i create a regular file in an XFS partition, create an XFS filesystem on it as if it was a regular block device, and mount it with the loop device, the problem does NOT manifest. So it seems to affect XFS filesystems on regular block devices only ? I'm confused.
This is probably not related, but I remember that barriers were added sometime between 2.6.15 and 2.6.17 and probably barriers are not enabled on a loop filesystem. Maybe the gnome vfs thing does some sync which force flushes or such. I'm no expert, just guessing.
Just to invalidate this (probably wrong) guess, could you try to mount /tmp with "-o nobarrier" and re-test?
Iustin, who wishes md raid1 would support barriers.