Btrfs progs release 6.2
David Sterba
maintainerHi,
btrfs-progs version 6.2 have been released.
Changelog:
* receive: fix a corruption when decompressing zstd extents
* subvol sync: print total number and deletion progress
* accelerated hash algorithm implementations in fallback mode on x86_64
* fi mkswapfile: new option --uuid
* new global option --log=level to set the verbosity level directly
* other:
* experimental: update checksum conversion (not usable yet)
* build actually requires -std=gnu11
* refactor help option formatting, auto wrap long lines
Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
Git: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git
Quoted 16 lines · David Sterba
Hi,
btrfs-progs version 6.2 have been released.
Changelog:
* receive: fix a corruption when decompressing zstd extents
* subvol sync: print total number and deletion progress
* accelerated hash algorithm implementations in fallback mode onx86_64
* fi mkswapfile: new option --uuid
* new global option --log=level to set the verbosity level directly
* other:
* experimental: update checksum conversion (not usable yet)
* build actually requires -std=gnu11
* refactor help option formatting, auto wrap long lines
Quoted 2 lines
Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
Git:git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git
This release does not compile on Fedora Rawhide.
With the current btrfs-progs spec file in Fedora bumped to 6.2, I get this error:
Quoted 11 lines
[LD] btrfstune
gcc -o btrfstunetune/main.otune/seeding.otune/change-uuid.otune/change-metadata-uuid.otune/convert-bgt.otune/change-csum.okernel-lib/list_sort.okernel-lib/raid56.okernel-lib/rbtree.okernel-lib/tables.okernel-shared/backref.okernel-shared/ctree.okernel-shared/delayed-ref.okernel-shared/dir-item.okernel-shared/disk-io.okernel-shared/extent-tree.okernel-shared/extent_io.okernel-shared/file-item.okernel-shared/file.okernel-shared/free-space-cache.okernel-shared/free-space-tree.okernel-shared/inode-item.okernel-shared/inode.okernel-shared/print-tree.okernel-shared/root-tree.okernel-shared/transaction.okernel-shared/ulist.okernel-shared/uuid-tree.okernel-shared/volumes.okernel-shared/zoned.ocommon/cpu-utils.ocommon/device-scan.ocommon/device-utils.ocommon/extent-cache.ocommon/filesystem-utils.ocommon/format-output.ocommon/fsfeatures.ocommon/help.ocommon/messages.ocommon/open-utils.ocommon/parse-utils.ocommon/path-utils.ocommon/rbtree-utils.ocommon/send-stream.ocommon/send-utils.ocommon/string-table.ocommon/string-utils.ocommon/task-utils.ocommon/units.ocommon/utils.ocheck/qgroup-verify.ocheck/repair.ocmds/receive-dump.ocrypto/crc32c.ocrypto/hash.ocrypto/xxhash.olibbtrfsutil/stubs.olibbtrfsutil/subvolume.olibbtrfsutil.a -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes -rdynamic -L. -luuid -lblkid -ludev -L. -pthread -lgcrypt/usr/bin/ld:/tmp/ccPRdR5s.ltrans5.ltrans.o: in function `hash_init_accel':
/builddir/build/BUILD/btrfs-progs-v6.2/crypto/hash.c:26: undefined reference toblake2_init_accel' /usr/bin/ld: /builddir/build/BUILD/btrfs-progs-v6.2/crypto/hash.c:27: undefined reference tosha256_init_accel'/usr/bin/ld:/tmp/ccPRdR5s.ltrans5.ltrans.o: in function `hash_init_blake2':
/builddir/build/BUILD/btrfs-progs-v6.2/crypto/hash.c:37: undefined reference toblake2_init_accel' /usr/bin/ld: /tmp/ccPRdR5s.ltrans5.ltrans.o: in functionhash_init_sha256':
/builddir/build/BUILD/btrfs-progs-v6.2/crypto/hash.c:42: undefined reference to `sha256_init_accel'
collect2: error: ld returned 1 exit status
make: *** [Makefile:673: btrfstune] Error 1
Signature · 1 line
真実はいつも一つ!/ Always, there's only one truth!
Quoted 17 lines · Neal Gompa
Quoted 16 lines · David Sterba
Hi,
btrfs-progs version 6.2 have been released.
Changelog:
* receive: fix a corruption when decompressing zstd extents
* subvol sync: print total number and deletion progress
* accelerated hash algorithm implementations in fallback mode onx86_64
* fi mkswapfile: new option --uuid
* new global option --log=level to set the verbosity level directly
* other:
* experimental: update checksum conversion (not usable yet)
* build actually requires -std=gnu11
* refactor help option formatting, auto wrap long lines
Quoted 19 lines
Quoted 2 lines
Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
Git:git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.gitThis release does not compile on Fedora Rawhide.
With the current btrfs-progs spec file in Fedora bumped to 6.2, I get this error:
Quoted 11 lines
[LD] btrfstune
gcc -o btrfstunetune/main.otune/seeding.otune/change-uuid.otune/change-metadata-uuid.otune/convert-bgt.otune/change-csum.okernel-lib/list_sort.okernel-lib/raid56.okernel-lib/rbtree.okernel-lib/tables.okernel-shared/backref.okernel-shared/ctree.okernel-shared/delayed-ref.okernel-shared/dir-item.okernel-shared/disk-io.okernel-shared/extent-tree.okernel-shared/extent_io.okernel-shared/file-item.okernel-shared/file.okernel-shared/free-space-cache.okernel-shared/free-space-tree.okernel-shared/inode-item.okernel-shared/inode.okernel-shared/print-tree.okernel-shared/root-tree.okernel-shared/transaction.okernel-shared/ulist.okernel-shared/uuid-tree.okernel-shared/volumes.okernel-shared/zoned.ocommon/cpu-utils.ocommon/device-scan.ocommon/device-utils.ocommon/extent-cache.ocommon/filesystem-utils.ocommon/format-output.ocommon/fsfeatures.ocommon/help.ocommon/messages.ocommon/open-utils.ocommon/parse-utils.ocommon/path-utils.ocommon/rbtree-utils.ocommon/send-stream.ocommon/send-utils.ocommon/string-table.ocommon/string-utils.ocommon/task-utils.ocommon/units.ocommon/utils.ocheck/qgroup-verify.ocheck/repair.ocmds/receive-dump.ocrypto/crc32c.ocrypto/hash.ocrypto/xxhash.olibbtrfsutil/stubs.olibbtrfsutil/subvolume.olibbtrfsutil.a -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes -rdynamic -L. -luuid -lblkid -ludev -L. -pthread -lgcrypt/usr/bin/ld:/tmp/ccPRdR5s.ltrans5.ltrans.o: in function `hash_init_accel':
/builddir/build/BUILD/btrfs-progs-v6.2/crypto/hash.c:26: undefined reference toblake2_init_accel' /usr/bin/ld: /builddir/build/BUILD/btrfs-progs-v6.2/crypto/hash.c:27: undefined reference tosha256_init_accel'/usr/bin/ld:/tmp/ccPRdR5s.ltrans5.ltrans.o: in function `hash_init_blake2':
/builddir/build/BUILD/btrfs-progs-v6.2/crypto/hash.c:37: undefined reference toblake2_init_accel' /usr/bin/ld: /tmp/ccPRdR5s.ltrans5.ltrans.o: in functionhash_init_sha256':
/builddir/build/BUILD/btrfs-progs-v6.2/crypto/hash.c:42: undefined reference to `sha256_init_accel'
collect2: error: ld returned 1 exit status
make: *** [Makefile:673: btrfstune] Error 1
I've been chasing this error on various build target and compiler combinations. What's the gcc version and architecture?
Quoted 39 lines · David Sterba
Quoted 17 lines · Neal Gompa
Quoted 16 lines · David Sterba
Hi,
btrfs-progs version 6.2 have been released.
Changelog:
* receive: fix a corruption when decompressing zstd extents
* subvol sync: print total number and deletion progress
* accelerated hash algorithm implementations in fallback mode onx86_64
* fi mkswapfile: new option --uuid
* new global option --log=level to set the verbosity level directly
* other:
* experimental: update checksum conversion (not usable yet)
* build actually requires -std=gnu11
* refactor help option formatting, auto wrap long linesQuoted 19 lines
Quoted 2 lines
Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
Git:git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.gitThis release does not compile on Fedora Rawhide.
With the current btrfs-progs spec file in Fedora bumped to 6.2, I get this error:
Quoted 11 lines
[LD] btrfstune
gcc -o btrfstunetune/main.otune/seeding.otune/change-uuid.otune/change-metadata-uuid.otune/convert-bgt.otune/change-csum.okernel-lib/list_sort.okernel-lib/raid56.okernel-lib/rbtree.okernel-lib/tables.okernel-shared/backref.okernel-shared/ctree.okernel-shared/delayed-ref.okernel-shared/dir-item.okernel-shared/disk-io.okernel-shared/extent-tree.okernel-shared/extent_io.okernel-shared/file-item.okernel-shared/file.okernel-shared/free-space-cache.okernel-shared/free-space-tree.okernel-shared/inode-item.okernel-shared/inode.okernel-shared/print-tree.okernel-shared/root-tree.okernel-shared/transaction.okernel-shared/ulist.okernel-shared/uuid-tree.okernel-shared/volumes.okernel-shared/zoned.ocommon/cpu-utils.ocommon/device-scan.ocommon/device-utils.ocommon/extent-cache.ocommon/filesystem-utils.ocommon/format-output.ocommon/fsfeatures.ocommon/help.ocommon/messages.ocommon/open-utils.ocommon/parse-utils.ocommon/path-utils.ocommon/rbtree-utils.ocommon/send-stream.ocommon/send-utils.ocommon/string-table.ocommon/string-utils.ocommon/task-utils.ocommon/units.ocommon/utils.ocheck/qgroup-verify.ocheck/repair.ocmds/receive-dump.ocrypto/crc32c.ocrypto/hash.ocrypto/xxhash.olibbtrfsutil/stubs.olibbtrfsutil/subvolume.olibbtrfsutil.a -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes -rdynamic -L. -luuid -lblkid -ludev -L. -pthread -lgcrypt/usr/bin/ld:/tmp/ccPRdR5s.ltrans5.ltrans.o: in function `hash_init_accel':
/builddir/build/BUILD/btrfs-progs-v6.2/crypto/hash.c:26: undefined reference toblake2_init_accel' /usr/bin/ld: /builddir/build/BUILD/btrfs-progs-v6.2/crypto/hash.c:27: undefined reference tosha256_init_accel'/usr/bin/ld:/tmp/ccPRdR5s.ltrans5.ltrans.o: in function `hash_init_blake2':
/builddir/build/BUILD/btrfs-progs-v6.2/crypto/hash.c:37: undefined reference toblake2_init_accel' /usr/bin/ld: /tmp/ccPRdR5s.ltrans5.ltrans.o: in functionhash_init_sha256':
/builddir/build/BUILD/btrfs-progs-v6.2/crypto/hash.c:42: undefined reference to `sha256_init_accel'
collect2: error: ld returned 1 exit status
make: *** [Makefile:673: btrfstune] Error 1
Quoted 2 lines
I've been chasing this error on various build target and compiler combinations. What's the gcc version and architecture?
GCC 13.0.1 on x86_64. I didn't bother trying other architectures yet.
--
真実はいつも一つ!/ Always, there's only one truth!
Quoted 5 lines · David Sterba
Hi,
btrfs-progs version 6.2 have been released.
Quoted 10 lines
Changelog:
* receive: fix a corruption when decompressing zstd extents
* subvol sync: print total number and deletion progress
* accelerated hash algorithm implementations in fallback mode onx86_64
* fi mkswapfile: new option --uuid
* new global option --log=level to set the verbosity level directly
* other:
* experimental: update checksum conversion (not usable yet)
* build actually requires -std=gnu11
* refactor help option formatting, auto wrap long lines
Just tested new dist tar ball and looks like something is wrong because linking fails on:
[LD] btrfs/usr/bin/gcc -o btrfs btrfs.o kernel-lib/list_sort.o kernel-lib/raid56.o kernel-lib/rbtree.o kernel-lib/tables.o kernel-shared/backref.o kernel-shared/ctree.o kernel-shared/delayed-ref.o kernel-shared/dir-item.o kernel-shared/disk-io.o kernel-shared/extent-tree.o kernel-shared/extent_io.o kernel-shared/file-item.o kernel-shared/file.o kernel-shared/free-space-cache.o kernel-shared/free-space-tree.o kernel-shared/inode-item.o kernel-shared/inode.o kernel-shared/print-tree.o kernel-shared/root-tree.o kernel-shared/transaction.o kernel-shared/ulist.o kernel-shared/uuid-tree.o kernel-shared/volumes.o kernel-shared/zoned.o common/cpu-utils.o common/device-scan.o common/device-utils.o common/extent-cache.o common/filesystem-utils.o common/format-output.o common/fsfeatures.o common/help.o common/messages.o common/open-utils.o common/parse-utils.o common/path-utils.o common/rbtree-utils.o common/send-stream.o common/send-utils.o common/string-table.o common/string-utils.o common/task-utils.o common/units.o common/utils.o check/qgroup-verify.o check/repair.o cmds/receive-dump.o crypto/crc32c.o crypto/hash.o crypto/xxhash.o libbtrfsutil/stubs.o libbtrfsutil/subvolume.o cmds/subvolume.o cmds/subvolume-list.o cmds/filesystem.o cmds/device.o cmds/scrub.o cmds/inspect.o cmds/balance.o cmds/send.o cmds/receive.o cmds/quota.o cmds/qgroup.o cmds/replace.o check/main.o cmds/restore.o cmds/rescue.o cmds/rescue-chunk-recover.o cmds/rescue-super-recover.o cmds/property.o cmds/filesystem-usage.o cmds/inspect-dump-tree.o cmds/inspect-dump-super.o cmds/inspect-tree-stats.o cmds/filesystem-du.o cmds/reflink.o mkfs/common.o check/mode-common.o check/mode-lowmem.o check/clear-cache.o libbtrfsutil.a -Wl,--as-needed -Wl,--gc-sections -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -flto=auto -flto-partition=none -fuse-linker-plugin -Wl,--build-id=sha1 -rdynamic
-L. -luuid -lblkid -ludev -L. -pthread -lkcapi -lz -llzo2 -lzstd/usr/bin/ld: /tmp/ccTTIULh.lto.o: in function `hash_init_accel':/home/tkloczko/rpmbuild/BUILD/btrfs-progs-v6.2/crypto/hash.c:26:
undefined reference to `blake2_init_accel'/usr/bin/ld: /home/tkloczko/rpmbuild/BUILD/btrfs-progs-v6.2/crypto/hash.c:27:
undefined reference to `sha256_init_accel'/usr/bin/ld: /tmp/ccTTIULh.lto.o: in function `hash_init_blake2':/home/tkloczko/rpmbuild/BUILD/btrfs-progs-v6.2/crypto/hash.c:37:
undefined reference to `blake2_init_accel'/usr/bin/ld: /tmp/ccTTIULh.lto.o: in function `hash_init_sha256':/home/tkloczko/rpmbuild/BUILD/btrfs-progs-v6.2/crypto/hash.c:42:
undefined reference to `sha256_init_accel'
collect2: error: ld returned 1 exit status
make: *** [Makefile:629: btrfs] Error 1
kloczek
--
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
Quoted 59 lines · Tomasz Kłoczko
Quoted 5 lines · David Sterba
Hi,
btrfs-progs version 6.2 have been released.
Quoted 10 lines
Changelog:
* receive: fix a corruption when decompressing zstd extents
* subvol sync: print total number and deletion progress
* accelerated hash algorithm implementations in fallback mode onx86_64
* fi mkswapfile: new option --uuid
* new global option --log=level to set the verbosity level directly
* other:
* experimental: update checksum conversion (not usable yet)
* build actually requires -std=gnu11
* refactor help option formatting, auto wrap long linesJust tested new dist tar ball and looks like something is wrong because linking fails on:
[LD] btrfs
/usr/bin/gcc-o btrfs btrfs.okernel-lib/list_sort.okernel-lib/raid56.okernel-lib/rbtree.okernel-lib/tables.okernel-shared/backref.okernel-shared/ctree.okernel-shared/delayed-ref.okernel-shared/dir-item.okernel-shared/disk-io.okernel-shared/extent-tree.okernel-shared/extent_io.okernel-shared/file-item.okernel-shared/file.okernel-shared/free-space-cache.okernel-shared/free-space-tree.okernel-shared/inode-item.okernel-shared/inode.okernel-shared/print-tree.okernel-shared/root-tree.okernel-shared/transaction.okernel-shared/ulist.okernel-shared/uuid-tree.okernel-shared/volumes.okernel-shared/zoned.ocommon/cpu-utils.ocommon/device-scan.ocommon/device-utils.ocommon/extent-cache.ocommon/filesystem-utils.ocommon/format-output.ocommon/fsfeatures.ocommon/help.ocommon/messages.ocommon/open-utils.ocommon/parse-utils.ocommon/path-utils.ocommon/rbtree-utils.ocommon/send-stream.ocommon/send-utils.ocommon/string-table.ocommon/string-utils.ocommon/task-utils.ocommon/units.ocommon/utils.ocheck/qgroup-verify.ocheck/repair.ocmds/receive-dump.ocrypto/crc32c.ocrypto/hash.ocrypto/xxhash.olibbtrfsutil/stubs.olibbtrfsutil/subvolume.ocmds/subvolume.ocmds/subvolume-list.ocmds/filesystem.ocmds/device.ocmds/scrub.ocmds/inspect.ocmds/balance.ocmds/send.ocmds/receive.ocmds/quota.ocmds/qgroup.ocmds/replace.ocheck/main.ocmds/restore.ocmds/rescue.ocmds/rescue-chunk-recover.ocmds/rescue-super-recover.ocmds/property.ocmds/filesystem-usage.ocmds/inspect-dump-tree.ocmds/inspect-dump-super.ocmds/inspect-tree-stats.ocmds/filesystem-du.ocmds/reflink.omkfs/common.ocheck/mode-common.ocheck/mode-lowmem.ocheck/clear-cache.olibbtrfsutil.a -Wl,--as-needed(quoted diff) +0
-4-Wl,--gc-sections -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -flto=auto -flto-partition=none -fuse-linker-plugin -Wl,--build-id=sha1 -rdynamic -L. -luuid -lblkid -ludev -L. -pthread -lkcapi -lz -llzo2 -lzstd
/usr/bin/ld:/tmp/ccTTIULh.lto.o: in function `hash_init_accel':/home/tkloczko/rpmbuild/BUILD/btrfs-progs-v6.2/crypto/hash.c:26:
undefined reference to `blake2_init_accel'/usr/bin/ld:/home/tkloczko/rpmbuild/BUILD/btrfs-progs-v6.2/crypto/hash.c:27:
undefined reference to `sha256_init_accel'
Quoted 8 lines
/usr/bin/ld:/tmp/ccTTIULh.lto.o: in function `hash_init_blake2':/home/tkloczko/rpmbuild/BUILD/btrfs-progs-v6.2/crypto/hash.c:37:
undefined reference to `blake2_init_accel'/usr/bin/ld:/tmp/ccTTIULh.lto.o: in function `hash_init_sha256':/home/tkloczko/rpmbuild/BUILD/btrfs-progs-v6.2/crypto/hash.c:42:
undefined reference to `sha256_init_accel'
collect2: error: ld returned 1 exit status
make: *** [Makefile:629: btrfs] Error 1
Compared to my passing build (Archlinux, thus most of things should be pretty close to rawhide), it looks like Fedora is missing the blake2 and sha256 object files:
[LD] btrfs
gcc -o btrfs btrfs.o kernel-lib/list_sort.o kernel-lib/raid56.o kernel-lib/rbtree.o kernel-lib/tables.o kernel-shared/backref.o kernel-shared/ctree.o kernel-shared/delayed-ref.o kernel-shared/dir-item.o kernel-shared/disk-io.o kernel-shared/extent-tree.o kernel-shared/extent_io.o kernel-shared/file-item.o kernel-shared/file.o kernel-shared/free-space-cache.o kernel-shared/free-space-tree.o kernel-shared/inode-item.o kernel-shared/inode.o kernel-shared/print-tree.o kernel-shared/root-tree.o kernel-shared/transaction.o kernel-shared/ulist.o kernel-shared/uuid-tree.o kernel-shared/volumes.o kernel-shared/zoned.o common/cpu-utils.o common/device-scan.o common/device-utils.o common/extent-cache.o common/filesystem-utils.o common/format-output.o common/fsfeatures.o common/help.o common/messages.o common/open-utils.o common/parse-utils.o common/path-utils.o common/rbtree-utils.o common/send-stream.o common/send-utils.o common/string-table.o common/string-utils.o common/task-utils.o common/units.o common/utils.o check/qgroup-verify.o check/repair.o cmds/receive-dump.o crypto/crc32c.o crypto/hash.o crypto/xxhash.o crypto/sha224-256.o crypto/blake2b-ref.o crypto/blake2b-sse2.o crypto/blake2b-sse41.o crypto/blake2b-avx2.o crypto/sha256-x86.o libbtrfsutil/stubs.o libbtrfsutil/subvolume.o cmds/subvolume.o cmds/subvolume-list.o cmds/filesystem.o cmds/device.o cmds/scrub.o cmds/inspect.o cmds/balance.o cmds/send.o cmds/receive.o cmds/quota.o cmds/qgroup.o cmds/replace.o check/main.o cmds/restore.o cmds/rescue.o cmds/rescue-chunk-recover.o cmds/rescue-super-recover.o cmds/property.o cmds/filesystem-usage.o cmds/inspect-dump-tree.o cmds/inspect-dump-super.o cmds/inspect-tree-stats.o cmds/filesystem-du.o cmds/reflink.o mkfs/common.o check/mode-common.o check/mode-lowmem.o check/clear-cache.o libbtrfsutil.a -rdynamic -L. -luuid -lblkid -ludev -L. -pthread -lz -llzo2 -lzstd
According to the Makefile, it looks like Fedora build is not using the built-in crypto code.
If using libsodium, I got the same error, as libsodium goes a different name for its blake2b_init (crypto_generichash_blake2b_init).
We need more work to support external crypto libraries.
For now, please use builtin crypto to pass the build.
Thanks,
Qu
Quoted 4 lines
kloczek
--
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
Quoted 8 lines · Qu Wenruo
Quoted 1 line · Tomasz Kłoczko
On Tue, 28 Feb 2023 at 20:07, David Sterba <[email protected]> wrote:
cmds/property.ocmds/filesystem-usage.ocmds/inspect-dump-tree.ocmds/inspect-dump-super.ocmds/inspect-tree-stats.ocmds/filesystem-du.ocmds/reflink.omkfs/common.ocheck/mode-common.ocheck/mode-lowmem.ocheck/clear-cache.olibbtrfsutil.a -rdynamic -L. -luuid -lblkid
-ludev -L. -pthread -lz -llzo2 -lzstd
Quoted 5 lines
According to the
Makefile, it looks like Fedora build is not using the built-in crypto code.If using libsodium, I got the same error, as libsodium goes a different name for its
blake2b_init(crypto_generichash_blake2b_init).
Oh right, thanks, I can reproduce it now.
Quoted 9 lines · David Sterba
Quoted 8 lines · Qu Wenruo
Quoted 1 line · Tomasz Kłoczko
On Tue, 28 Feb 2023 at 20:07, David Sterba <[email protected]> wrote:
cmds/property.ocmds/filesystem-usage.ocmds/inspect-dump-tree.ocmds/inspect-dump-super.ocmds/inspect-tree-stats.ocmds/filesystem-du.ocmds/reflink.omkfs/common.ocheck/mode-common.ocheck/mode-lowmem.ocheck/clear-cache.olibbtrfsutil.a -rdynamic -L. -luuid -lblkid
-ludev -L. -pthread -lz -llzo2 -lzstd
Quoted 7 lines
Quoted 5 lines
According to the
Makefile, it looks like Fedora build is not using the built-in crypto code.If using libsodium, I got the same error, as libsodium goes a different name for its
blake2b_init(crypto_generichash_blake2b_init).Oh right, thanks, I can reproduce it now.
And bisection points to the following two patches:
bbf703bfd3f68958d33d139eb22057ab397e6c68 btrfs-progs: crypto: call sha256 implementations by pointer d1c366ee42bd3d2abb4fd855ac4a496b720d8bb6 btrfs-progs: crypto: call blake2 implementations by pointer
Thanks,
Qu
Quoted 18 lines · Qu Wenruo
Quoted 15 lines · David Sterba
Quoted 13 lines · Qu Wenruo
Quoted 1 line · Tomasz Kłoczko
On Tue, 28 Feb 2023 at 20:07, David Sterba <[email protected]> wrote:
cmds/property.ocmds/filesystem-usage.ocmds/inspect-dump-tree.ocmds/inspect-dump-super.ocmds/inspect-tree-stats.ocmds/filesystem-du.ocmds/reflink.omkfs/common.ocheck/mode-common.ocheck/mode-lowmem.ocheck/clear-cache.olibbtrfsutil.a -rdynamic -L. -luuid -lblkid
-ludev -L. -pthread -lz -llzo2 -lzstdAccording to the
Makefile, it looks like Fedora build is not using the built-in crypto code.If using libsodium, I got the same error, as libsodium goes a different name for its
blake2b_init(crypto_generichash_blake2b_init).
Quoted 8 lines
Quoted 1 line
Oh right, thanks, I can reproduce it now.
And bisection points to the following two patches:
bbf703bfd3f68958d33d139eb22057ab397e6c68btrfs-progs: crypto: call sha256 implementations by pointerd1c366ee42bd3d2abb4fd855ac4a496b720d8bb6btrfs-progs: crypto: call blake2 implementations by pointer
Yeah I know, I did some last minute changes to the commits because 32bit builds on intel and arm failed due to the flag and feature detection. The fix is not straightforward but now I have something that works.
Quoted 23 lines · David Sterba
Quoted 22 lines · Qu Wenruo
Quoted 9 lines · David Sterba
Quoted 8 lines · Qu Wenruo
Quoted 1 line · Tomasz Kłoczko
On Tue, 28 Feb 2023 at 20:07, David Sterba <[email protected]> wrote:
cmds/property.ocmds/filesystem-usage.ocmds/inspect-dump-tree.ocmds/inspect-dump-super.ocmds/inspect-tree-stats.ocmds/filesystem-du.ocmds/reflink.omkfs/common.ocheck/mode-common.ocheck/mode-lowmem.ocheck/clear-cache.olibbtrfsutil.a -rdynamic -L. -luuid -lblkid
-ludev -L. -pthread -lz -llzo2 -lzstdQuoted 7 lines
Quoted 5 lines
According to the
Makefile, it looks like Fedora build is not using the built-in crypto code.If using libsodium, I got the same error, as libsodium goes a different name for its
blake2b_init(crypto_generichash_blake2b_init).Oh right, thanks, I can reproduce it now.
And bisection points to the following two patches:
Quoted 8 lines
Quoted 4 lines
bbf703bfd3f68958d33d139eb22057ab397e6c68btrfs-progs: crypto: call sha256 implementations by pointerd1c366ee42bd3d2abb4fd855ac4a496b720d8bb6btrfs-progs: crypto: call blake2 implementations by pointerYeah I know, I did some last minute changes to the commits because 32bit builds on intel and arm failed due to the flag and feature detection. The fix is not straightforward but now I have something that works.
Just one question, why we support external crypto libraries in the first place?
IIRC the built-in ones are already utilizing various optimization, and I'd argue that performance should not be the critical aspect for btrfs anyway, as only btrfs check and btrfs-restore are really involved for performance but they are all single-thread workload, far from performance critical.
For best compatibility, the external crypto libs can even be a problem.
So clues are appreciated for why we're supporting external crypto libraries for hash functions.
Thanks,
Qu
Quoted 2 lines · Qu Wenruo
Just one question, why we support external crypto libraries in the first place?
For static build of btrfs-progs we need a fallback implementation because distros don't provide static versions of the crypto libraries.
External libraries can get a certification so if distro has to meet the criteria then it must use only the certified libraries. This was requested e.g. for SLES back then for the authenticated hashes. More libraries give users the freedom to choose the one they use.
Quoted 1 line
IIRC the built-in ones are already utilizing various optimization,
Only crc32c had an optimized version and not the fastest one, so in 6.2 there's at least one acelerated if possible.
Quoted 5 lines
and
I'd argue that performance should not be the critical aspect for btrfs anyway, as only btrfs check and btrfs-restore are really involved for performance but they are all single-thread workload, far from performance critical.
The checksums are still verified at read time and on a fast device the CPU is the bottleneck, not the IO, so this is basically to keep up with hardware speeds.
That it's single threaded is actually a problem and there's one issue asking to make 'check --init-csum-tree' parallel. The proper checksum switch does only checksumming so this should be accelerated and/or parallelized. If this can shorten time to do eg. check or uuid switch or other things then it IMO makes sense to implement.
Quoted 1 line
For best compatibility, the external crypto libs can even be a problem.
Yes, so there's the fallback.
Quoted 15 lines · David Sterba
Hi,
btrfs-progs version 6.2 have been released.
Changelog:
* receive: fix a corruption when decompressing zstd extents
* subvol sync: print total number and deletion progress
* accelerated hash algorithm implementations in fallback mode onx86_64
* fi mkswapfile: new option --uuid
* new global option --log=level to set the verbosity level directly
* other:
* experimental: update checksum conversion (not usable yet)
* build actually requires -std=gnu11
* refactor help option formatting, auto wrap long lines
Quoted 2 lines
Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
Git:git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git
Fixed version is in devel, I did some more checks on the CI images and the OBS build on different arches also pass so it should be fine. Bugfix release will be in a day.