newer /bin/cp have worse btrfs fiemap performance.
Hi,
newer /bin/cp have worse btrfs fiemap performance.
btrfs version: misc-next 750de989d367(the lastest of 2022/11/05) /bin/cp versions(run them on the same server)
8.22 copy from centos/7.9
8.30 rocklinux/9.0
9.1 local build, https://kojipkgs.fedoraproject.org/packages/coreutils/9.1/8.eln121/test case:
/bin/cp /mnt/test/file1 /dev/null
/mnt/test/file1 is created by https://lore.kernel.org/linux-btrfs/[email protected]/T/#T
and /mnt/test/file1 is 256M.
file is not cached: 'echo 3 >/proc/sys/vm/drop_caches' file is cached: run '/bin/cp /mnt/test/file1 /dev/null' again.
performance result(/bin/cp /mnt/test/file1 /dev/null):/bin/cp 9.1
file is not cached: 94.85(1:34.85)
file is cached: 1982.43(33:02.43)
/bin/cp 8.30
file is not cached: 1.48(0:01.48)
file is cached:14.07(0:14.07)
/bin/cp 8.22
file is not cached: 0.53(0:00.53)
file is cached: 0.10(0:00.10)as a compare, we test it on xfs too.
1) /bin/cp 8.22/8.30/9.1 have almost same performance.
2) the case(the file is cached) is faster than the case(the file is not cached).
strace show that the logical of /bin/cp 8.30 and 9.1 are different. /bin/cp 8.30
lseek(3, 198737920, SEEK_SET) = 198737920
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
lseek(3, 198746112, SEEK_SET) = 198746112
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096/bin/cp 9.1
lseek(3, 880640, SEEK_DATA) = 884736
lseek(3, 884736, SEEK_HOLE) = 888832
lseek(3, 884736, SEEK_SET) = 884736
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
lseek(3, 888832, SEEK_DATA) = 892928
lseek(3, 892928, SEEK_HOLE) = 897024
lseek(3, 892928, SEEK_SET) = 892928
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096Do we need some job in btrfs to support /bin/cp 9.1 well, or /bin/cp 9.1 is just wrong?
Best Regards
Wang Yugui ([email protected])
2022/11/06
Hi,
a fix.
Quoted 7 lines
Hi,
newer
/bin/cphave worse btrfs fiemap performance.btrfs version: misc-next 750de989d367(the lastest of 2022/11/05)
/bin/cpversions(run them on the same server)8.22 copy from centos/7.9
this
Quoted 1 line
8.30 rocklinux/9.0
should be
8.30 rocklinux/8
Best Regards
Wang Yugui ([email protected])
2022/11/06
Quoted 58 lines
9.1 local build, https://kojipkgs.fedoraproject.org/packages/coreutils/9.1/8.eln121/test case:
/bin/cp/mnt/test/file1/dev/null
/mnt/test/file1is created by https://lore.kernel.org/linux-btrfs/[email protected]/T/#T
and/mnt/test/file1is 256M.file is not cached: 'echo 3 >/proc/sys/vm/drop_caches' file is cached: run '
/bin/cp/mnt/test/file1/dev/null' again.performance result(/bin/cp
/mnt/test/file1/dev/null):/bin/cp9.1file is not cached: 94.85(1:34.85) file is cached: 1982.43(33:02.43) /bin/cp 8.30 file is not cached: 1.48(0:01.48) file is cached:14.07(0:14.07) /bin/cp 8.22 file is not cached: 0.53(0:00.53) file is cached: 0.10(0:00.10)as a compare, we test it on xfs too.
1)/bin/cp8.22/8.30/9.1 have almost same performance.
2) the case(the file is cached) is faster than the case(the file is not cached).strace show that the logical of
/bin/cp8.30 and 9.1 are different./bin/cp8.30lseek(3, 198737920, SEEK_SET) = 198737920 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 lseek(3, 198746112, SEEK_SET) = 198746112 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
/bin/cp9.1lseek(3, 880640, SEEK_DATA) = 884736 lseek(3, 884736, SEEK_HOLE) = 888832 lseek(3, 884736, SEEK_SET) = 884736 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 lseek(3, 888832, SEEK_DATA) = 892928 lseek(3, 892928, SEEK_HOLE) = 897024 lseek(3, 892928, SEEK_SET) = 892928 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096Do we need some job in btrfs to support
/bin/cp9.1 well, or/bin/cp9.1 is just wrong?Best Regards
Wang Yugui ([email protected])
2022/11/06
Quoted 4 lines · Wang Yugui
Hi,
newer
/bin/cphave worse btrfs fiemap performance.
New versions of coreutils (9.0 and 9.1) have worse cp performance, but it has nothing to do with fiemap.
This was pointed out in another thread ([1]) sometime ago, that starting with coreutils 9.0, cp no longer uses fiemap. That happened in coreutils' commit:
https://github.com/coreutils/coreutils/commit/26eccf6c98696c50f4416ba2967edc8676870716
You can also confirm that by checking that strace no longer shows any lines with:
ioctl(3, FS_IOC_FIEMAP, (...)
Quoted 26 lines
btrfs version: misc-next 750de989d367(the lastest of 2022/11/05)
/bin/cpversions(run them on the same server)8.22 copy from centos/7.9 8.30 rocklinux/9.0 9.1 local build, https://kojipkgs.fedoraproject.org/packages/coreutils/9.1/8.eln121/test case:
/bin/cp/mnt/test/file1/dev/null
/mnt/test/file1is created by https://lore.kernel.org/linux-btrfs/[email protected]/T/#T
and/mnt/test/file1is 256M.file is not cached: 'echo 3 >/proc/sys/vm/drop_caches' file is cached: run '
/bin/cp/mnt/test/file1/dev/null' again.performance result(/bin/cp
/mnt/test/file1/dev/null):/bin/cp9.1file is not cached: 94.85(1:34.85) file is cached: 1982.43(33:02.43) /bin/cp 8.30 file is not cached: 1.48(0:01.48) file is cached:14.07(0:14.07) /bin/cp 8.22 file is not cached: 0.53(0:00.53) file is cached: 0.10(0:00.10)
Quoted 6 lines
as a compare, we test it on xfs too.
1)/bin/cp8.22/8.30/9.1 have almost same performance.
2) the case(the file is cached) is faster than the case(the file is not cached).strace show that the logical of
/bin/cp8.30 and 9.1 are different.
Yes, as pointed out before.
Before coreutils 9.0, cp used fiemap to determine where holes and extents are in a file.
However starting with 9.0 it uses SEEK_DATA/SEEK_HOLE from lseek to do that, completely
dropping fiemap usage.
Quoted 24 lines
/bin/cp8.30lseek(3, 198737920, SEEK_SET) = 198737920 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 lseek(3, 198746112, SEEK_SET) = 198746112 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
/bin/cp9.1lseek(3, 880640, SEEK_DATA) = 884736 lseek(3, 884736, SEEK_HOLE) = 888832 lseek(3, 884736, SEEK_SET) = 884736 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 lseek(3, 888832, SEEK_DATA) = 892928 lseek(3, 892928, SEEK_HOLE) = 897024 lseek(3, 892928, SEEK_SET) = 892928 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
Quoted 2 lines
Do we need some job in btrfs to support
/bin/cp9.1 well, or/bin/cp9.1 is just wrong?
There's nothing wrong with cp using lseek's SEEK_DATA/HOLE features. It's just that the data/hole seek calls are where cp is spending most time, so the way to go is
to improve the efficiency of those calls in btrfs. I'll do some work to make it better.
Keep in mind that the case being tested is pretty much the worst possible scenario. It's an extremely
sparse file, where each extent has a hole before it and another hole after it. More often than not,
there are significantly more extents than holes in a typical file.
Thanks.
Quoted 6 lines
Best Regards
Wang Yugui ([email protected])
2022/11/06