Skip to contentSkip to conversation

More checksumming algorithms for btrfs

DISCUSSION David Sterba 1 participant 1 message Oct 21, 2019 Btrfs file system

lore
Share & tools
lore

David Sterba

author maintainer
#0 lore raw

The work to add more checksums is nearly finished, we're now in a good state to let interested users do some testing and benchmarking.

New hashes: xxhash64, sha256, blake2b-256

Quick start:

git://github.com/kdave/btrfs-devel preview/checksums-5 (build with CRYPTO_BLAKE2B=m)

git://github.com/kdave/btrfs-progs devel

  $ mkfs.btrfs --csum blake2 /dev/sda
  $ mount /dev/sda /mnt

Warning: use only for testing!

The increased size of checksums is allocating more memory when the data are being written so this can produce some warnings regarding size of the allocation. This will be addressed later.

Selection results
~~~~~~~~~~~~~~~~~

fast hash: xxhash
- 64bit digest
- optimized for 64bit platforms, leveraging CPU pipelining

cryptographically strong hash 1: sha256
- 256bit digest
- FIPS certification

cryptographically strong hash 2: blake2
- blake2b with 256bit digest
- '2b' as it targets 64bit platforms

Microbenchmark
~~~~~~~~~~~~~~

  $ cd btrfs-progs.git
  $ make hash-speedtest
  $ ./hash-speedtest [iterations]

Block size: 4096
Iterations: 100000

    NULL-NOP: cycles:     53638797, c/i      536
 NULL-MEMCPY: cycles:     59547932, c/i      595
      CRC32C: cycles:    179251924, c/i     1792
      XXHASH: cycles:    137327470, c/i     1373
      SHA256: cycles:  10719756126, c/i   107197
     BLAKE2b: cycles:   2264316924, c/i    22643

Compatibility
~~~~~~~~~~~~~

There's no new incompat bit, the checksum algorithm is detected at mount time and unknown type will fail to mount.

The crypto modules implementing the digests must be either built-in or loadable, lack of thereof will fail mount. The actual digest implementation is up to the crypto API to choose. Check /sys/fs/UUID/features/checksum .

Target release
~~~~~~~~~~~~~~

The plan is to queue the patches for 5.5, the blake2b patches seem to be on a good track so both shall be in the same release.