fs: Debug config option to disable filesystem checksum verification for fuzzing
Subsystems (8)
cover fs: Debug config option to disable filesystem checksum verification for fuzzing
From: Hrutvik Kanabar <[email protected]>
Fuzzing is a proven technique to discover exploitable bugs in the Linux kernel. But fuzzing filesystems is tricky: highly structured disk images use redundant checksums to verify data integrity. Therefore, randomly-mutated images are quickly rejected as corrupt, testing only error-handling code effectively.
The Janus [1] and Hydra [2] projects probe filesystem code deeply by correcting checksums after mutation. But their ad-hoc checksum-correcting code supports only a few filesystems, and it is difficult to support new ones - requiring significant duplication of filesystem logic which must also be kept in sync with upstream changes. Corrected checksums cannot be guaranteed to be valid, and reusing this code across different fuzzing frameworks is non-trivial.
Instead, this RFC suggests a config option:DISABLE_FS_CSUM_VERIFICATION. When it is enabled, all filesystems should bypass redundant checksum verification, proceeding as if checksums are valid. Setting of checksums should be unaffected. Mutated images will no longer be rejected due to invalid checksums, allowing testing of deeper code paths. Though some filesystems implement their own flags to disable some checksums, this option should instead disable all checksums for all filesystems uniformly. Critically, any bugs found remain reproducible on production systems: redundant checksums in mutated images can be fixed up to satisfy verification.
The patches below suggest a potential implementation for a few filesystems, though we may have missed some checksums. The option requires DEBUG_KERNEL and is not intended for production systems.
The first user of the option would be syzbot. We ran preliminary local syzkaller tests to compare behaviour with and without these patches. With the patches, we found a 19% increase in coverage, as well as many new crash types and increases in the total number of crashes:
Filesystem | % new crash types | % increase in crashes —----------|-------------------|----------------------
ext4 | 60% | 1400% btrfs | 25% | 185% f2fs | 63% | 16%
[1] Fuzzing file systems via two-dimensional input space exploration, Xu et al., 2019, IEEE Symposium on Security and Privacy,
doi: 10.1109/SP.2019.00035
[2] Finding semantic bugs in file systems with an extensible fuzzing framework, Kim et al., 2019, ACM Symposium on Operating Systems Principles, doi: 10.1145/3341301.3359662
Hrutvik Kanabar (7):
fs: create DISABLE_FS_CSUM_VERIFICATION config option fs/ext4: support DISABLE_FS_CSUM_VERIFICATION config option fs/btrfs: support DISABLE_FS_CSUM_VERIFICATION config option fs/exfat: support DISABLE_FS_CSUM_VERIFICATION config option fs/xfs: support DISABLE_FS_CSUM_VERIFICATION config option fs/ntfs: support DISABLE_FS_CSUM_VERIFICATION config option fs/f2fs: support DISABLE_FS_CSUM_VERIFICATION config option
fs/Kconfig.debug | 20 ++++++++++++++++++++ fs/btrfs/check-integrity.c | 3 ++- fs/btrfs/disk-io.c | 6 ++++-- fs/btrfs/free-space-cache.c | 3 ++- fs/btrfs/inode.c | 3 ++- fs/btrfs/scrub.c | 9 ++++++--- fs/exfat/nls.c | 3 ++- fs/exfat/super.c | 3 +++ fs/ext4/bitmap.c | 6 ++++-- fs/ext4/extents.c | 3 ++- fs/ext4/inode.c | 3 ++- fs/ext4/ioctl.c | 3 ++- fs/ext4/mmp.c | 3 ++- fs/ext4/namei.c | 6 ++++-- fs/ext4/orphan.c | 3 ++- fs/ext4/super.c | 6 ++++-- fs/ext4/xattr.c | 3 ++- fs/f2fs/checkpoint.c | 3 ++- fs/f2fs/compress.c | 3 ++- fs/f2fs/f2fs.h | 2 ++ fs/f2fs/inode.c | 3 +++ fs/ntfs/super.c | 3 ++- fs/xfs/libxfs/xfs_cksum.h | 5 ++++- lib/Kconfig.debug | 6 ++++++ 24 files changed, 86 insertions(+), 25 deletions(-) create mode 100644 fs/Kconfig.debug
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
RFC PATCH 1/7 fs: create `DISABLE_FS_CSUM_VERIFICATION` config option
From: Hrutvik Kanabar <[email protected]>
When implemented and enabled, this should circumvent all redundant checksum verification in filesystem code. However, setting of checksums is not affected.
The aim is to aid fuzzing efforts which randomly mutate disk images and so invalidate checksums. Checksum verification often rejects these mutated disk images, hindering fuzzer coverage of deep code paths. By disabling checksum verification, all mutated images are considered valid and so exploration of interesting code paths can continue.
This option requires the DEBUG_KERNEL option, and is not intended to be used on production systems.
--- fs/Kconfig.debug | 20 ++++++++++++++++++++ lib/Kconfig.debug | 6 ++++++ 2 files changed, 26 insertions(+) create mode 100644 fs/Kconfig.debug
fs/Kconfig.debug new file +20 -0
@@ -0,0 +1,20 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config DISABLE_FS_CSUM_VERIFICATION + bool "Disable redundant checksum verification for filesystems" + depends on DEBUG_KERNEL + help + Disable filesystem checksum verification for checksums which can be + trivially recomputed from the on-disk data (i.e. no encryption). + Note that this does not affect setting of checksums. + + This option is useful for filesystem testing. For example, fuzzing + with randomly mutated disk images can uncover bugs exploitable by + specially-crafted disks. Redundant checksums are orthogonal to these + exploits, as they can be recomputed for crafted disks. However, for + testing it is more reliable to disable checksums within the kernel + than to maintain image generators which faithfully reimplement + per-filesystem checksum recomputation. + + Say N if you are unsure. Disable this for production systems! +
lib/Kconfig.debug +6 -0
@@ -979,6 +979,12 @@ source "lib/Kconfig.kmsan" endmenu # "Memory Debugging" +menu "Filesystem Debugging" + +source "fs/Kconfig.debug" + +endmenu # "Filesystem Debugging" + config DEBUG_SHIRQ bool "Debug shared IRQ handlers" depends on DEBUG_KERNEL
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
RFC PATCH 2/7 fs/ext4: support `DISABLE_FS_CSUM_VERIFICATION` config option
In reply to Hrutvik KanabarFrom: Hrutvik Kanabar <[email protected]>
When DISABLE_FS_CSUM_VERIFICATION is enabled, bypass checks in key checksum verification functions.
--- fs/ext4/bitmap.c | 6 ++++-- fs/ext4/extents.c | 3 ++- fs/ext4/inode.c | 3 ++- fs/ext4/ioctl.c | 3 ++- fs/ext4/mmp.c | 3 ++- fs/ext4/namei.c | 6 ++++-- fs/ext4/orphan.c | 3 ++- fs/ext4/super.c | 6 ++++-- fs/ext4/xattr.c | 3 ++- 9 files changed, 24 insertions(+), 12 deletions(-)
fs/ext4/bitmap.c +4 -2
@@ -24,7 +24,8 @@ int ext4_inode_bitmap_csum_verify(struct super_block *sb, ext4_group_t group,__u32 provided, calculated;struct ext4_sb_info *sbi =EXT4_SB (sb); -if (!ext4_has_metadata_csum (sb)) +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) || + !ext4_has_metadata_csum (sb))return 1 ; provided =le16_to_cpu (gdp->bg_inode_bitmap_csum_lo);
@@ -63,7 +64,8 @@ int ext4_block_bitmap_csum_verify(struct super_block *sb, ext4_group_t group,struct ext4_sb_info *sbi =EXT4_SB (sb);int sz =EXT4_CLUSTERS_PER_GROUP (sb) /8 ; -if (!ext4_has_metadata_csum (sb)) +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) || + !ext4_has_metadata_csum (sb))return 1 ; provided =le16_to_cpu (gdp->bg_block_bitmap_csum_lo);
fs/ext4/extents.c +2 -1
@@ -63,7 +63,8 @@ static int ext4_extent_block_csum_verify(struct inode *inode, {struct ext4_extent_tail *et; -if (!ext4_has_metadata_csum (inode->i_sb)) +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) || + !ext4_has_metadata_csum (inode->i_sb))return 1 ; et =find_ext4_extent_tail (eh);
fs/ext4/inode.c +2 -1
@@ -86,7 +86,8 @@ static int ext4_inode_csum_verify(struct inode *inode, struct ext4_inode *raw, {__u32 provided, calculated; -if (EXT4_SB (inode->i_sb)->s_es->s_creator_os != +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) || +EXT4_SB (inode->i_sb)->s_es->s_creator_os !=cpu_to_le32 (EXT4_OS_LINUX ) || !ext4_has_metadata_csum (inode->i_sb))return 1 ;
fs/ext4/ioctl.c +2 -1
@@ -142,7 +142,8 @@ static int ext4_update_backup_sb(struct super_block *sb, es = (struct ext4_super_block *) (bh->b_data + offset);lock_buffer (bh); -if (ext4_has_metadata_csum (sb) && +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +ext4_has_metadata_csum (sb) && es->s_checksum !=ext4_superblock_csum (sb, es)) {ext4_msg (sb,KERN_ERR ,"Invalid checksum for backup " "superblock %llu\n" , sb_block);
fs/ext4/mmp.c +2 -1
@@ -21,7 +21,8 @@ static __le32 ext4_mmp_csum(struct super_block *sb, struct mmp_struct *mmp)static int ext4_mmp_csum_verify (struct super_block *sb,struct mmp_struct *mmp) { -if (!ext4_has_metadata_csum (sb)) +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) || + !ext4_has_metadata_csum (sb))return 1 ;return mmp->mmp_checksum ==ext4_mmp_csum (sb, mmp);
fs/ext4/namei.c +4 -2
@@ -396,7 +396,8 @@ int ext4_dirblock_csum_verify(struct inode *inode, struct buffer_head *bh) {struct ext4_dir_entry_tail *t; -if (!ext4_has_metadata_csum (inode->i_sb)) +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) || + !ext4_has_metadata_csum (inode->i_sb))return 1 ; t =get_dirent_tail (inode, bh);
@@ -491,7 +492,8 @@ static int ext4_dx_csum_verify(struct inode *inode,struct dx_tail *t;int count_offset, limit, count; -if (!ext4_has_metadata_csum (inode->i_sb)) +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) || + !ext4_has_metadata_csum (inode->i_sb))return 1 ; c =get_dx_countlimit (inode, dirent, &count_offset);
fs/ext4/orphan.c +2 -1
@@ -537,7 +537,8 @@ static int ext4_orphan_file_block_csum_verify(struct super_block *sb,struct ext4_orphan_block_tail *ot;__le64 dsk_block_nr =cpu_to_le64 (bh->b_blocknr); -if (!ext4_has_metadata_csum (sb)) +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) || + !ext4_has_metadata_csum (sb))return 1 ; ot =ext4_orphan_block_tail (sb, bh);
fs/ext4/super.c +4 -2
@@ -287,7 +287,8 @@ __le32 ext4_superblock_csum(struct super_block *sb,static int ext4_superblock_csum_verify (struct super_block *sb,struct ext4_super_block *es) { -if (!ext4_has_metadata_csum (sb)) +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) || + !ext4_has_metadata_csum (sb))return 1 ;return es->s_checksum ==ext4_superblock_csum (sb, es);
@@ -3198,7 +3199,8 @@ static __le16 ext4_group_desc_csum(struct super_block *sb, __u32 block_group,int ext4_group_desc_csum_verify (struct super_block *sb,__u32 block_group,struct ext4_group_desc *gdp) { -if (ext4_has_group_desc_csum (sb) && +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +ext4_has_group_desc_csum (sb) && (gdp->bg_checksum !=ext4_group_desc_csum (sb, block_group, gdp)))return 0 ;
fs/ext4/xattr.c +2 -1
@@ -154,7 +154,8 @@ static int ext4_xattr_block_csum_verify(struct inode *inode,struct ext4_xattr_header *hdr =BHDR (bh);int ret =1 ; -if (ext4_has_metadata_csum (inode->i_sb)) { +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +ext4_has_metadata_csum (inode->i_sb)) {lock_buffer (bh); ret = (hdr->h_checksum ==ext4_xattr_block_csum (inode, bh->b_blocknr, hdr));
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
RFC PATCH 3/7 fs/btrfs: support `DISABLE_FS_CSUM_VERIFICATION` config option
In reply to Hrutvik KanabarFrom: Hrutvik Kanabar <[email protected]>
When DISABLE_FS_CSUM_VERIFICATION is enabled, bypass checksum verification.
--- fs/btrfs/check-integrity.c | 3 ++- fs/btrfs/disk-io.c | 6 ++++-- fs/btrfs/free-space-cache.c | 3 ++- fs/btrfs/inode.c | 3 ++- fs/btrfs/scrub.c | 9 ++++++--- 5 files changed, 16 insertions(+), 8 deletions(-)
fs/btrfs/check-integrity.c +2 -1
@@ -1671,7 +1671,8 @@ static noinline_for_stack int btrfsic_test_for_metadata(crypto_shash_update (shash, data, sublen); }crypto_shash_final (shash, csum); -if (memcmp (csum, h->csum, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (csum, h->csum, fs_info->csum_size))return 1 ;return 0 ;/* is metadata */
fs/btrfs/disk-io.c +4 -2
@@ -184,7 +184,8 @@ static int btrfs_check_super_csum(struct btrfs_fs_info *fs_info,crypto_shash_digest (shash, raw_disk_sb +BTRFS_CSUM_SIZE ,BTRFS_SUPER_INFO_SIZE -BTRFS_CSUM_SIZE , result); -if (memcmp (disk_sb->csum, result, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (disk_sb->csum, result, fs_info->csum_size))return 1 ;return 0 ;
@@ -494,7 +495,8 @@ static int validate_extent_buffer(struct extent_buffer *eb) header_csum =page_address (eb->pages[0 ]) +get_eb_offset_in_page (eb, offsetof(struct btrfs_header , csum)); -if (memcmp (result, header_csum, csum_size) !=0 ) { +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (result, header_csum, csum_size) !=0 ) { btrfs_warn_rl(fs_info, "checksum verify failed on logical %llu mirror %u wanted " CSUM_FMT " found " CSUM_FMT " level %d", eb->start, eb->read_mirror,
fs/btrfs/free-space-cache.c +2 -1
@@ -574,7 +574,8 @@ static int io_ctl_check_crc(struct btrfs_io_ctl *io_ctl, int index)io_ctl_map_page (io_ctl,0 ); crc =btrfs_crc32c (crc, io_ctl->orig + offset,PAGE_SIZE - offset);btrfs_crc32c_final (crc, (u8 *)&crc); -if (val != crc) { +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && + val != crc) {btrfs_err_rl (io_ctl->fs_info,"csum mismatch on free space cache" );io_ctl_unmap_page (io_ctl);
fs/btrfs/inode.c +2 -1
@@ -3434,7 +3434,8 @@ int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page,crypto_shash_digest (shash, kaddr, fs_info->sectorsize, csum);kunmap_local (kaddr); -if (memcmp (csum, csum_expected, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (csum, csum_expected, fs_info->csum_size))return -EIO ;return 0 ; }
fs/btrfs/scrub.c +6 -3
@@ -1997,7 +1997,8 @@ static int scrub_checksum_data(struct scrub_block *sblock)crypto_shash_digest (shash, kaddr, fs_info->sectorsize, csum); -if (memcmp (csum, sector->csum, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (csum, sector->csum, fs_info->csum_size)) sblock->checksum_error =1 ;return sblock->checksum_error; }
@@ -2062,7 +2063,8 @@ static int scrub_checksum_tree_block(struct scrub_block *sblock) }crypto_shash_final (shash, calculated_csum); -if (memcmp (calculated_csum, on_disk_csum, sctx->fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (calculated_csum, on_disk_csum, sctx->fs_info->csum_size)) sblock->checksum_error =1 ;return sblock->header_error || sblock->checksum_error;
@@ -2099,7 +2101,8 @@ static int scrub_checksum_super(struct scrub_block *sblock)crypto_shash_digest (shash, kaddr +BTRFS_CSUM_SIZE ,BTRFS_SUPER_INFO_SIZE -BTRFS_CSUM_SIZE , calculated_csum); -if (memcmp (calculated_csum, s->csum, sctx->fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (calculated_csum, s->csum, sctx->fs_info->csum_size)) ++fail_cor;return fail_cor + fail_gen;
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
Re: [PATCH RFC 3/7] fs/btrfs: support `DISABLE_FS_CSUM_VERIFICATION` config option
Quoted 6 lines · Hrutvik Kanabar
From: Hrutvik Kanabar <[email protected]>
When
DISABLE_FS_CSUM_VERIFICATIONis enabled, bypass checksum verification.Signed-off-by Hrutvik Kanabar <[email protected]>
I always want more fuzz for btrfs, so overall this is pretty good.
But there are some comments related to free space cache part.
Despite the details, I'm wondering would it be possible for your fuzzing tool to do a better job at user space? Other than relying on loosen checks from kernel?
For example, implement a (mostly) read-only tool to do the following workload:
- Open the fs
Including understand the checksum algo, how to re-generate the csum.
- Read out the used space bitmap
In btrfs case, it's going to read the extent tree, process the backrefs items.
- Choose the victim sectors and corrupt them
Obviously, vitims should be choosen from above used space bitmap.
- Re-calculate the checksum for above corrupted sectors For btrfs, if it's a corrupted metadata, re-calculate the checksum.
By this, we can avoid such change to kernel, and still get a much better coverage.
If you need some help on such user space tool, I'm pretty happy to provide help.
Quoted 37 lines
--- fs/btrfs/check-integrity.c | 3 ++- fs/btrfs/disk-io.c | 6 ++++-- fs/btrfs/free-space-cache.c | 3 ++- fs/btrfs/inode.c | 3 ++- fs/btrfs/scrub.c | 9 ++++++--- 5 files changed, 16 insertions(+), 8 deletions(-)fs/btrfs/check-integrity.c +2
-1@@ -1671,7 +1671,8 @@ static noinline_for_stack int btrfsic_test_for_metadata(crypto_shash_update (shash, data, sublen); }crypto_shash_final (shash, csum); -if (memcmp (csum, h->csum, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (csum, h->csum, fs_info->csum_size))return 1 ;return 0 ;/* is metadata */ fs/btrfs/disk-io.c +2
-1@@ -184,7 +184,8 @@ static int btrfs_check_super_csum(struct btrfs_fs_info *fs_info,crypto_shash_digest (shash, raw_disk_sb +BTRFS_CSUM_SIZE ,BTRFS_SUPER_INFO_SIZE -BTRFS_CSUM_SIZE , result); -if (memcmp (disk_sb->csum, result, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (disk_sb->csum, result, fs_info->csum_size))return 1 ;return 0 ;
fs/btrfs/disk-io.c static int validate_extent_buffer(struct extent_buffer *eb) · line 499
Quoted 6 lines
fs/btrfs/disk-io.c +2
-1@@ -494,7 +495,8 @@ static int validate_extent_buffer(struct extent_buffer *eb) header_csum =page_address (eb->pages[0 ]) +get_eb_offset_in_page (eb, offsetof(struct btrfs_header , csum)); -if (memcmp (result, header_csum, csum_size) !=0 ) { +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (result, header_csum, csum_size) !=0 ) {
I believe this is the main thing fuzzing would take advantage of.
It would be much better if this is the only override...
Quoted 8 lines
fs/btrfs/disk-io.c +0
-0btrfs_warn_rl(fs_info, "checksum verify failed on logical %llu mirror %u wanted " CSUM_FMT " found " CSUM_FMT " level %d", eb->start, eb->read_mirror,fs/btrfs/free-space-cache.c +0
-0
fs/btrfs/free-space-cache.c static int io_ctl_check_crc(struct btrfs_io_ctl *io_ctl, int index) · line 578
Quoted 6 lines
fs/btrfs/free-space-cache.c +2
-1@@ -574,7 +574,8 @@ static int io_ctl_check_crc(struct btrfs_io_ctl *io_ctl, int index)io_ctl_map_page (io_ctl,0 ); crc =btrfs_crc32c (crc, io_ctl->orig + offset,PAGE_SIZE - offset);btrfs_crc32c_final (crc, (u8 *)&crc); -if (val != crc) { +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && + val != crc) {
I'm already seeing this to cause problems, especially for btrfs.
Btrfs has a very strong dependency on free space tracing, as all of our metadata (and data by default) relies on COW to keep the fs consistent.
I tried a lot of different methods in the past to make sure we won't write into previously used space, but it's causing a lot of performance impact.
Unlike tree-checker, we can not easily got a centerlized space to handle all the free space cross-check thing (thus it's only verified by things like btrfs-check).
Furthermore, even if you skip this override, with latest default free-space-tree feature, free space info is stored in regular btrfs metadata (tree blocks), with regular metadata checksum protection.
Thus I'm pretty sure we will have tons of reports on this, and unfortunately we can only go whac-a-mole way for it.
Quoted 8 lines
fs/btrfs/free-space-cache.c +0
-0btrfs_err_rl (io_ctl->fs_info,"csum mismatch on free space cache" );io_ctl_unmap_page (io_ctl);fs/btrfs/inode.c +0
-0
fs/btrfs/inode.c int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page, · line 3438
Quoted 6 lines
fs/btrfs/inode.c +2
-1@@ -3434,7 +3434,8 @@ int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page,crypto_shash_digest (shash, kaddr, fs_info->sectorsize, csum);kunmap_local (kaddr); -if (memcmp (csum, csum_expected, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (csum, csum_expected, fs_info->csum_size))
This skips data csum check, I don't know how valueable it is, but this should be harmless mostly.
If we got reports related to this, it would be a nice surprise.
Quoted 8 lines
fs/btrfs/inode.c +0
-0return -EIO ;return 0 ; }fs/btrfs/scrub.c +0
-0
fs/btrfs/scrub.c static int scrub_checksum_data(struct scrub_block *sblock) · line 2001
Quoted 6 lines
fs/btrfs/scrub.c +2
-1@@ -1997,7 +1997,8 @@ static int scrub_checksum_data(struct scrub_block *sblock)crypto_shash_digest (shash, kaddr, fs_info->sectorsize, csum); -if (memcmp (csum, sector->csum, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (csum, sector->csum, fs_info->csum_size))
Same as data csum verification overide.
Not necessary/useful but good to have.
fs/btrfs/scrub.c static int scrub_checksum_tree_block(struct scrub_block *sblock) · line 2067
Quoted 10 lines
fs/btrfs/scrub.c +2
-1sblock->checksum_error =1 ;return sblock->checksum_error; }@@ -2062,7 +2063,8 @@ static int scrub_checksum_tree_block(struct scrub_block *sblock) }crypto_shash_final (shash, calculated_csum); -if (memcmp (calculated_csum, on_disk_csum, sctx->fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (calculated_csum, on_disk_csum, sctx->fs_info->csum_size))
This is much less valueable, since it's only affecting scrub, and scrub itself is already very little checking the content of metadata.
Thanks,
Qu
Quoted 13 lines
fs/btrfs/scrub.c +2
-1sblock->checksum_error =1 ;return sblock->header_error || sblock->checksum_error;@@ -2099,7 +2101,8 @@ static int scrub_checksum_super(struct scrub_block *sblock)crypto_shash_digest (shash, kaddr +BTRFS_CSUM_SIZE ,BTRFS_SUPER_INFO_SIZE -BTRFS_CSUM_SIZE , calculated_csum); -if (memcmp (calculated_csum, s->csum, sctx->fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (calculated_csum, s->csum, sctx->fs_info->csum_size)) ++fail_cor;return fail_cor + fail_gen;
Re: [PATCH RFC 3/7] fs/btrfs: support `DISABLE_FS_CSUM_VERIFICATION` config option
Quoted 115 lines · Qu Wenruo' via kasan-dev
Quoted 6 lines · Hrutvik Kanabar
From: Hrutvik Kanabar <[email protected]>
When
DISABLE_FS_CSUM_VERIFICATIONis enabled, bypass checksum verification.Signed-off-by Hrutvik Kanabar <[email protected]>I always want more fuzz for btrfs, so overall this is pretty good.
But there are some comments related to free space cache part.
Despite the details, I'm wondering would it be possible for your fuzzing tool to do a better job at user space? Other than relying on loosen checks from kernel?
For example, implement a (mostly) read-only tool to do the following workload:
- Open the fs
Including understand the checksum algo, how to re-generate the csum.- Read out the used space bitmap
In btrfs case, it's going to read the extent tree, process the backrefs items.- Choose the victim sectors and corrupt them
Obviously, vitims should be choosen from above used space bitmap.- Re-calculate the checksum for above corrupted sectors
For btrfs, if it's a corrupted metadata, re-calculate the checksum.By this, we can avoid such change to kernel, and still get a much better coverage.
If you need some help on such user space tool, I'm pretty happy to provide help.
Quoted 37 lines
--- fs/btrfs/check-integrity.c | 3 ++- fs/btrfs/disk-io.c | 6 ++++-- fs/btrfs/free-space-cache.c | 3 ++- fs/btrfs/inode.c | 3 ++- fs/btrfs/scrub.c | 9 ++++++--- 5 files changed, 16 insertions(+), 8 deletions(-)fs/btrfs/check-integrity.c +2
-1@@ -1671,7 +1671,8 @@ static noinline_for_stack int btrfsic_test_for_metadata(crypto_shash_update (shash, data, sublen); }crypto_shash_final (shash, csum); -if (memcmp (csum, h->csum, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (csum, h->csum, fs_info->csum_size))return 1 ;return 0 ;/* is metadata */ fs/btrfs/disk-io.c +2
-1@@ -184,7 +184,8 @@ static int btrfs_check_super_csum(struct btrfs_fs_info *fs_info,crypto_shash_digest (shash, raw_disk_sb +BTRFS_CSUM_SIZE ,BTRFS_SUPER_INFO_SIZE -BTRFS_CSUM_SIZE , result); -if (memcmp (disk_sb->csum, result, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (disk_sb->csum, result, fs_info->csum_size))return 1 ;return 0 ;fs/btrfs/disk-io.c static int validate_extent_buffer(struct extent_buffer *eb) · line 499
Quoted 6 lines
fs/btrfs/disk-io.c +2
-1@@ -494,7 +495,8 @@ static int validate_extent_buffer(struct extent_buffer *eb) header_csum =page_address (eb->pages[0 ]) +get_eb_offset_in_page (eb, offsetof(struct btrfs_header , csum)); -if (memcmp (result, header_csum, csum_size) !=0 ) { +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (result, header_csum, csum_size) !=0 ) {I believe this is the main thing fuzzing would take advantage of.
It would be much better if this is the only override...
Quoted 8 lines
fs/btrfs/disk-io.c +0
-0btrfs_warn_rl(fs_info, "checksum verify failed on logical %llu mirror %u wanted " CSUM_FMT " found " CSUM_FMT " level %d", eb->start, eb->read_mirror,fs/btrfs/free-space-cache.c +0
-0fs/btrfs/free-space-cache.c static int io_ctl_check_crc(struct btrfs_io_ctl *io_ctl, int index) · line 578
Quoted 6 lines
fs/btrfs/free-space-cache.c +2
-1@@ -574,7 +574,8 @@ static int io_ctl_check_crc(struct btrfs_io_ctl *io_ctl, int index)io_ctl_map_page (io_ctl,0 ); crc =btrfs_crc32c (crc, io_ctl->orig + offset,PAGE_SIZE - offset);btrfs_crc32c_final (crc, (u8 *)&crc); -if (val != crc) { +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && + val != crc) {I'm already seeing this to cause problems, especially for btrfs.
Btrfs has a very strong dependency on free space tracing, as all of our metadata (and data by default) relies on COW to keep the fs consistent.
I tried a lot of different methods in the past to make sure we won't write into previously used space, but it's causing a lot of performance impact.
Unlike tree-checker, we can not easily got a centerlized space to handle all the free space cross-check thing (thus it's only verified by things like btrfs-check).
Quoted 6 lines
Furthermore, even if you skip this override, with latest default free-space-tree feature, free space info is stored in regular btrfs metadata (tree blocks), with regular metadata checksum protection.
Thus I'm pretty sure we will have tons of reports on this, and unfortunately we can only go whac-a-mole way for it.
Hi Qu,
I don't fully understand what you mean. Could you please elaborate?
Do you mean that btrfs uses this checksum check to detect blocks that were written to w/o updating the checksum?
Quoted 44 lines
Quoted 8 lines
fs/btrfs/free-space-cache.c +0
-0btrfs_err_rl (io_ctl->fs_info,"csum mismatch on free space cache" );io_ctl_unmap_page (io_ctl);fs/btrfs/inode.c +0
-0fs/btrfs/inode.c int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page, · line 3438
Quoted 6 lines
fs/btrfs/inode.c +2
-1@@ -3434,7 +3434,8 @@ int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page,crypto_shash_digest (shash, kaddr, fs_info->sectorsize, csum);kunmap_local (kaddr); -if (memcmp (csum, csum_expected, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (csum, csum_expected, fs_info->csum_size))This skips data csum check, I don't know how valueable it is, but this should be harmless mostly.
If we got reports related to this, it would be a nice surprise.
Quoted 8 lines
fs/btrfs/inode.c +0
-0return -EIO ;return 0 ; }fs/btrfs/scrub.c +0
-0fs/btrfs/scrub.c static int scrub_checksum_data(struct scrub_block *sblock) · line 2001
Quoted 6 lines
fs/btrfs/scrub.c +2
-1@@ -1997,7 +1997,8 @@ static int scrub_checksum_data(struct scrub_block *sblock)crypto_shash_digest (shash, kaddr, fs_info->sectorsize, csum); -if (memcmp (csum, sector->csum, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (csum, sector->csum, fs_info->csum_size))Same as data csum verification overide.
Not necessary/useful but good to have.Quoted 6 lines
fs/btrfs/scrub.c +0
-0sblock->checksum_error =1 ;return sblock->checksum_error; }@@ -2062,7 +2063,8 @@ static int scrub_checksum_tree_block(struct scrub_block *sblock) }
Quoted 7 lines
fs/btrfs/scrub.c static int scrub_checksum_tree_block(struct scrub_block *sblock) · line 2067
Quoted 4 lines
fs/btrfs/scrub.c +2
-1crypto_shash_final (shash, calculated_csum); -if (memcmp (calculated_csum, on_disk_csum, sctx->fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (calculated_csum, on_disk_csum, sctx->fs_info->csum_size))This is much less valueable, since it's only affecting scrub, and scrub itself is already very little checking the content of metadata.
Could you please elaborate here as well?
This is less valuable from what perspective?
The data loaded from disk can have any combination of (correct/incorrect metadata) x (correct/incorrect checksum). Correctness of metadata and checksum are effectively orthogonal, right?
Quoted 16 lines
Thanks,
QuQuoted 13 lines
fs/btrfs/scrub.c +2
-1sblock->checksum_error =1 ;return sblock->header_error || sblock->checksum_error;@@ -2099,7 +2101,8 @@ static int scrub_checksum_super(struct scrub_block *sblock)crypto_shash_digest (shash, kaddr +BTRFS_CSUM_SIZE ,BTRFS_SUPER_INFO_SIZE -BTRFS_CSUM_SIZE , calculated_csum); -if (memcmp (calculated_csum, s->csum, sctx->fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (calculated_csum, s->csum, sctx->fs_info->csum_size)) ++fail_cor;return fail_cor + fail_gen;
Re: [PATCH RFC 3/7] fs/btrfs: support `DISABLE_FS_CSUM_VERIFICATION` config option
Quoted 124 lines · Dmitry Vyukov
Quoted 115 lines · Qu Wenruo' via kasan-dev
Quoted 6 lines · Hrutvik Kanabar
From: Hrutvik Kanabar <[email protected]>
When
DISABLE_FS_CSUM_VERIFICATIONis enabled, bypass checksum verification.Signed-off-by Hrutvik Kanabar <[email protected]>I always want more fuzz for btrfs, so overall this is pretty good.
But there are some comments related to free space cache part.
Despite the details, I'm wondering would it be possible for your fuzzing tool to do a better job at user space? Other than relying on loosen checks from kernel?
For example, implement a (mostly) read-only tool to do the following workload:
- Open the fs
Including understand the checksum algo, how to re-generate the csum.- Read out the used space bitmap
In btrfs case, it's going to read the extent tree, process the backrefs items.- Choose the victim sectors and corrupt them
Obviously, vitims should be choosen from above used space bitmap.- Re-calculate the checksum for above corrupted sectors
For btrfs, if it's a corrupted metadata, re-calculate the checksum.By this, we can avoid such change to kernel, and still get a much better coverage.
If you need some help on such user space tool, I'm pretty happy to provide help.
Quoted 37 lines
--- fs/btrfs/check-integrity.c | 3 ++- fs/btrfs/disk-io.c | 6 ++++-- fs/btrfs/free-space-cache.c | 3 ++- fs/btrfs/inode.c | 3 ++- fs/btrfs/scrub.c | 9 ++++++--- 5 files changed, 16 insertions(+), 8 deletions(-)fs/btrfs/check-integrity.c +2
-1@@ -1671,7 +1671,8 @@ static noinline_for_stack int btrfsic_test_for_metadata(crypto_shash_update (shash, data, sublen); }crypto_shash_final (shash, csum); -if (memcmp (csum, h->csum, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (csum, h->csum, fs_info->csum_size))return 1 ;return 0 ;/* is metadata */ fs/btrfs/disk-io.c +2
-1@@ -184,7 +184,8 @@ static int btrfs_check_super_csum(struct btrfs_fs_info *fs_info,crypto_shash_digest (shash, raw_disk_sb +BTRFS_CSUM_SIZE ,BTRFS_SUPER_INFO_SIZE -BTRFS_CSUM_SIZE , result); -if (memcmp (disk_sb->csum, result, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (disk_sb->csum, result, fs_info->csum_size))return 1 ;return 0 ;fs/btrfs/disk-io.c static int validate_extent_buffer(struct extent_buffer *eb) · line 499
Quoted 6 lines
fs/btrfs/disk-io.c +2
-1@@ -494,7 +495,8 @@ static int validate_extent_buffer(struct extent_buffer *eb) header_csum =page_address (eb->pages[0 ]) +get_eb_offset_in_page (eb, offsetof(struct btrfs_header , csum)); -if (memcmp (result, header_csum, csum_size) !=0 ) { +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (result, header_csum, csum_size) !=0 ) {I believe this is the main thing fuzzing would take advantage of.
It would be much better if this is the only override...
Quoted 8 lines
fs/btrfs/disk-io.c +0
-0btrfs_warn_rl(fs_info, "checksum verify failed on logical %llu mirror %u wanted " CSUM_FMT " found " CSUM_FMT " level %d", eb->start, eb->read_mirror,fs/btrfs/free-space-cache.c +0
-0fs/btrfs/free-space-cache.c static int io_ctl_check_crc(struct btrfs_io_ctl *io_ctl, int index) · line 578
Quoted 6 lines
fs/btrfs/free-space-cache.c +2
-1@@ -574,7 +574,8 @@ static int io_ctl_check_crc(struct btrfs_io_ctl *io_ctl, int index)io_ctl_map_page (io_ctl,0 ); crc =btrfs_crc32c (crc, io_ctl->orig + offset,PAGE_SIZE - offset);btrfs_crc32c_final (crc, (u8 *)&crc); -if (val != crc) { +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && + val != crc) {I'm already seeing this to cause problems, especially for btrfs.
Btrfs has a very strong dependency on free space tracing, as all of our metadata (and data by default) relies on COW to keep the fs consistent.
I tried a lot of different methods in the past to make sure we won't write into previously used space, but it's causing a lot of performance impact.
Unlike tree-checker, we can not easily got a centerlized space to handle all the free space cross-check thing (thus it's only verified by things like btrfs-check).
Quoted 6 lines
Furthermore, even if you skip this override, with latest default free-space-tree feature, free space info is stored in regular btrfs metadata (tree blocks), with regular metadata checksum protection.
Thus I'm pretty sure we will have tons of reports on this, and unfortunately we can only go whac-a-mole way for it.
Quoted 6 lines
Hi Qu,
I don't fully understand what you mean. Could you please elaborate?
Do you mean that btrfs uses this checksum check to detect blocks that were written to w/o updating the checksum?
I mean, btrfs uses this particular checksum for its (free) space cache, and currently btrfs just trust the space cache completely to do COW.
This means, if we ignore the checksum for free space cache, we can easily screw up the COW, e.g. allocate a range for the new metadata to be written into.
But the truth is, that range is still being utilized by some other metadata. Thus would completely break COW.
This is indeed a problem for btrfs, but it is not that easiy to fix, since this involves cross-check 3 different data (free space cache for free space, extent tree for used space, and the metadata itself).
Thus my concern is, disabling free space cache csum can easily lead to various crashes, all related to broken COW, and we don't have a good enough way to validate the result.
Quoted 56 lines
Quoted 48 lines
Quoted 12 lines
fs/btrfs/free-space-cache.c +0
-0btrfs_err_rl (io_ctl->fs_info,"csum mismatch on free space cache" );io_ctl_unmap_page (io_ctl);fs/btrfs/inode.c +0
-0fs/btrfs/inode.c int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page, · line 3438
Quoted 6 lines
fs/btrfs/inode.c +2
-1@@ -3434,7 +3434,8 @@ int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page,crypto_shash_digest (shash, kaddr, fs_info->sectorsize, csum);kunmap_local (kaddr); -if (memcmp (csum, csum_expected, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (csum, csum_expected, fs_info->csum_size))This skips data csum check, I don't know how valueable it is, but this should be harmless mostly.
If we got reports related to this, it would be a nice surprise.
Quoted 8 lines
fs/btrfs/inode.c +0
-0return -EIO ;return 0 ; }fs/btrfs/scrub.c +0
-0fs/btrfs/scrub.c static int scrub_checksum_data(struct scrub_block *sblock) · line 2001
Quoted 6 lines
fs/btrfs/scrub.c +2
-1@@ -1997,7 +1997,8 @@ static int scrub_checksum_data(struct scrub_block *sblock)crypto_shash_digest (shash, kaddr, fs_info->sectorsize, csum); -if (memcmp (csum, sector->csum, fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (csum, sector->csum, fs_info->csum_size))Same as data csum verification overide.
Not necessary/useful but good to have.Quoted 6 lines
fs/btrfs/scrub.c +0
-0sblock->checksum_error =1 ;return sblock->checksum_error; }@@ -2062,7 +2063,8 @@ static int scrub_checksum_tree_block(struct scrub_block *sblock) }Quoted 7 lines
fs/btrfs/scrub.c static int scrub_checksum_tree_block(struct scrub_block *sblock) · line 2067
Quoted 4 lines
fs/btrfs/scrub.c +2
-1crypto_shash_final (shash, calculated_csum); -if (memcmp (calculated_csum, on_disk_csum, sctx->fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (calculated_csum, on_disk_csum, sctx->fs_info->csum_size))This is much less valueable, since it's only affecting scrub, and scrub itself is already very little checking the content of metadata.
Quoted 1 line
Could you please elaborate here as well?
These checksum verification is only done in the scrub path (just as the file name indicates).
Quoted 1 line
This is less valuable from what perspective?
It's just much harder to trigger, regular filesystem operations won't go into scrub path.
Unless there is also a full ioctl fuzzing tests, after corrupting the image.
Quoted 3 lines
The data loaded from disk can have any combination of (correct/incorrect metadata) x (correct/incorrect checksum). Correctness of metadata and checksum are effectively orthogonal,
Oh, I almost forgot another problem with the compile time csum verification skip.
If we skip csum check completely, just like the patch, it may cause less path coverage (this is very btrfs specific)
The problem is, btrfs has some repair path (scrub, and read-time), which requires to have a checksum mismatch (and a good copy with good checksum).
Thus if we ignore csum completely, the repair path will never be covered (as we treat them all as csum match).
Thanks,
Qu
Quoted 14 lines
right?
Quoted 10 lines
Thanks,
QuQuoted 7 lines
fs/btrfs/scrub.c +0
-0sblock->checksum_error =1 ;return sblock->header_error || sblock->checksum_error;@@ -2099,7 +2101,8 @@ static int scrub_checksum_super(struct scrub_block *sblock)crypto_shash_digest (shash, kaddr +BTRFS_CSUM_SIZE ,BTRFS_SUPER_INFO_SIZE -BTRFS_CSUM_SIZE , calculated_csum);
Quoted 6 lines
Quoted 6 lines
fs/btrfs/scrub.c static int scrub_checksum_super(struct scrub_block *sblock) · line 2108
Quoted 6 lines
fs/btrfs/scrub.c +2
-1-if (memcmp (calculated_csum, s->csum, sctx->fs_info->csum_size)) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +memcmp (calculated_csum, s->csum, sctx->fs_info->csum_size)) ++fail_cor;return fail_cor + fail_gen;
_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
RFC PATCH 4/7 fs/exfat: support `DISABLE_FS_CSUM_VERIFICATION` config option
In reply to Hrutvik KanabarFrom: Hrutvik Kanabar <[email protected]>
When DISABLE_FS_CSUM_VERIFICATION is enabled, bypass checksum verification.
--- fs/exfat/nls.c | 3 ++- fs/exfat/super.c | 3 +++ 2 files changed, 5 insertions(+), 1 deletion(-)
fs/exfat/nls.c +2 -1
@@ -696,7 +696,8 @@ static int exfat_load_upcase_table(struct super_block *sb,brelse (bh); } -if (index >=0xFFFF && utbl_checksum == chksum) +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) || + (index >=0xFFFF && utbl_checksum == chksum))return 0 ; exfat_err(sb, "failed to load upcase table (idx : 0x%08x, chksum : 0x%08x, utbl_chksum : 0x%08x)",
fs/exfat/super.c +3 -0
@@ -564,6 +564,9 @@ static int exfat_verify_boot_region(struct super_block *sb)if (!bh)return -EIO ; +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION )) +return 0 ; +for (i =0 ; i < sb->s_blocksize; i +=sizeof (u32)) { p_chksum = (__le32 *)&bh->b_data[i];if (le32_to_cpu (*p_chksum) != chksum) {
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
RFC PATCH 5/7 fs/xfs: support `DISABLE_FS_CSUM_VERIFICATION` config option
In reply to Hrutvik KanabarFrom: Hrutvik Kanabar <[email protected]>
When DISABLE_FS_CSUM_VERIFICATION is enabled, return truthy value for xfs_verify_cksum, which is the key function implementing checksum verification for XFS.
--- fs/xfs/libxfs/xfs_cksum.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_cksum.h +4 -1
@@ -76,7 +76,10 @@ xfs_verify_cksum(char *buffer, size_t length, unsigned long cksum_offset) {uint32_t crc =xfs_start_cksum_safe (buffer, length, cksum_offset); -return *(__le32 *)(buffer + cksum_offset) ==xfs_end_cksum (crc); +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION )) +return 1 ; +else +return *(__le32 *)(buffer + cksum_offset) ==xfs_end_cksum (crc); }#endif /* _XFS_CKSUM_H */
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
Darrick J. Wong
NAKRe: [PATCH RFC 5/7] fs/xfs: support `DISABLE_FS_CSUM_VERIFICATION` config option
Quoted 7 lines · Hrutvik Kanabar
From: Hrutvik Kanabar <[email protected]>
When
DISABLE_FS_CSUM_VERIFICATIONis enabled, return truthy value forxfs_verify_cksum, which is the key function implementing checksum verification for XFS.Signed-off-by Hrutvik Kanabar <[email protected]>
NAK, we're not going to break XFS for the sake of automated fuzz tools.
You'll have to adapt your fuzzing tools to rewrite the block header checksums, like the existing xfs fuzz testing framework does. See the xfs_db 'fuzz -d' command and the relevant fstests.
--D
Quoted 23 lines
--- fs/xfs/libxfs/xfs_cksum.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)fs/xfs/libxfs/xfs_cksum.h +4
-1@@ -76,7 +76,10 @@ xfs_verify_cksum(char *buffer, size_t length, unsigned long cksum_offset) {uint32_t crc =xfs_start_cksum_safe (buffer, length, cksum_offset); -return *(__le32 *)(buffer + cksum_offset) ==xfs_end_cksum (crc); +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION )) +return 1 ; +else +return *(__le32 *)(buffer + cksum_offset) ==xfs_end_cksum (crc); }#endif /* _XFS_CKSUM_H */ Signature · 1 line
2.38.0.413.g74048e4d9e-goog
Re: [PATCH RFC 5/7] fs/xfs: support `DISABLE_FS_CSUM_VERIFICATION` config option
Quoted 11 lines · Darrick J. Wong
Quoted 7 lines · Hrutvik Kanabar
From: Hrutvik Kanabar <[email protected]>
When
DISABLE_FS_CSUM_VERIFICATIONis enabled, return truthy value forxfs_verify_cksum, which is the key function implementing checksum verification for XFS.Signed-off-by Hrutvik Kanabar <[email protected]>NAK, we're not going to break XFS for the sake of automated fuzz tools.
Hi Darrick,
What do you mean by "break"? If this config is not enabled the behavior is not affected as far as I see.
Quoted 25 lines
You'll have to adapt your fuzzing tools to rewrite the block header checksums, like the existing xfs fuzz testing framework does. See the
xfs_db'fuzz -d' command and the relevant fstests.--D
Quoted 19 lines
--- fs/xfs/libxfs/xfs_cksum.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)fs/xfs/libxfs/xfs_cksum.h +4
-1@@ -76,7 +76,10 @@ xfs_verify_cksum(char *buffer, size_t length, unsigned long cksum_offset) {uint32_t crc =xfs_start_cksum_safe (buffer, length, cksum_offset); -return *(__le32 *)(buffer + cksum_offset) ==xfs_end_cksum (crc); +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION )) +return 1 ; +else +return *(__le32 *)(buffer + cksum_offset) ==xfs_end_cksum (crc); }
Quoted 9 lines
Quoted 4 lines
fs/xfs/libxfs/xfs_cksum.h +0
-1#endif /* _XFS_CKSUM_H */ --2.38.0.413.g74048e4d9e-goog
--
You received this message because you are subscribed to the Google Groups "kasan-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/Y0mD0LcNvu%2BQTlQ9%40magnolia.
_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
RFC PATCH 6/7 fs/ntfs: support `DISABLE_FS_CSUM_VERIFICATION` config option
In reply to Hrutvik KanabarFrom: Hrutvik Kanabar <[email protected]>
When DISABLE_FS_CSUM_VERIFICATION is enabled, bypass checksum verification in is_boot_sector_ntfs.
--- fs/ntfs/super.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/ntfs/super.c +2 -1
@@ -582,7 +582,8 @@ static bool is_boot_sector_ntfs(const struct super_block *sb,for (i =0 , u = (le32 *)b; u < (le32 *)(&b->checksum); ++u) i +=le32_to_cpup (u); -if (le32_to_cpu (b->checksum) != i) +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && +le32_to_cpu (b->checksum) != i)ntfs_warning (sb,"Invalid boot sector checksum." ); }/* Check OEMidentifier is "NTFS " */
Signature · 8 lines
2.38.0.413.g74048e4d9e-goog _______________________________________________ Linux-f2fs-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
RFC PATCH 7/7 fs/f2fs: support `DISABLE_FS_CSUM_VERIFICATION` config option
In reply to Hrutvik KanabarFrom: Hrutvik Kanabar <[email protected]>
When DISABLE_FS_CSUM_VERIFICATION is enabled, bypass checksum verification.
--- fs/f2fs/checkpoint.c | 3 ++- fs/f2fs/compress.c | 3 ++- fs/f2fs/f2fs.h | 2 ++ fs/f2fs/inode.c | 3 +++ 4 files changed, 9 insertions(+), 2 deletions(-)
fs/f2fs/checkpoint.c +2 -1
@@ -864,7 +864,8 @@ static int get_checkpoint_version(struct f2fs_sb_info *sbi, block_t cp_addr, } crc =f2fs_checkpoint_chksum (sbi, *cp_block); -if (crc !=cur_cp_crc (*cp_block)) { +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && + crc !=cur_cp_crc (*cp_block)) {f2fs_put_page (*cp_page,1 );f2fs_warn (sbi,"invalid crc value" );return -EINVAL ;
fs/f2fs/compress.c +2 -1
@@ -772,7 +772,8 @@ void f2fs_decompress_cluster(struct decompress_io_ctx *dic, bool in_task)u32 provided =le32_to_cpu (dic->cbuf->chksum);u32 calculated =f2fs_crc32 (sbi, dic->cbuf->cdata, dic->clen); -if (provided != calculated) { +if (!IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION ) && + provided != calculated) {if (!is_inode_flag_set (dic->inode,FI_COMPRESS_CORRUPT )) {set_inode_flag (dic->inode,FI_COMPRESS_CORRUPT ); printk_ratelimited(
fs/f2fs/f2fs.h +2 -0
@@ -1976,6 +1976,8 @@ static inline u32 f2fs_crc32(struct f2fs_sb_info *sbi, const void *address,static inline bool f2fs_crc_valid (struct f2fs_sb_info *sbi,__u32 blk_crc,void *buf,size_t buf_size) { +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION )) +return true;return f2fs_crc32 (sbi, buf, buf_size) == blk_crc; }
fs/f2fs/inode.c +3 -0
@@ -181,6 +181,9 @@ bool f2fs_inode_chksum_verify(struct f2fs_sb_info *sbi, struct page *page)#endif return true; +if (IS_ENABLED (CONFIG_DISABLE_FS_CSUM_VERIFICATION )) +return true; + ri = &F2FS_NODE (page)->i; provided =le32_to_cpu (ri->i_inode_checksum); calculated =f2fs_inode_chksum (sbi, page);
Signature · 8 lines
2.38.0.413.g74048e4d9e-goog _______________________________________________ Linux-f2fs-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Quoted 27 lines · Hrutvik Kanabar
From: Hrutvik Kanabar <[email protected]>
Fuzzing is a proven technique to discover exploitable bugs in the Linux kernel. But fuzzing filesystems is tricky: highly structured disk images use redundant checksums to verify data integrity. Therefore, randomly-mutated images are quickly rejected as corrupt, testing only error-handling code effectively.
The Janus [1] and Hydra [2] projects probe filesystem code deeply by correcting checksums after mutation. But their ad-hoc checksum-correcting code supports only a few filesystems, and it is difficult to support new ones - requiring significant duplication of filesystem logic which must also be kept in sync with upstream changes. Corrected checksums cannot be guaranteed to be valid, and reusing this code across different fuzzing frameworks is non-trivial.
Instead, this RFC suggests a config option:
DISABLE_FS_CSUM_VERIFICATION. When it is enabled, all filesystems should bypass redundant checksum verification, proceeding as if checksums are valid. Setting of checksums should be unaffected. Mutated images will no longer be rejected due to invalid checksums, allowing testing of deeper code paths. Though some filesystems implement their own flags to disable some checksums, this option should instead disable all checksums for all filesystems uniformly. Critically, any bugs found remain reproducible on production systems: redundant checksums in mutated images can be fixed up to satisfy verification.
Quoted 8 lines
The patches below suggest a potential implementation for a few filesystems, though we may have missed some checksums. The option requires
DEBUG_KERNELand is not intended for production systems.The first user of the option would be syzbot. We ran preliminary local syzkaller tests to compare behaviour with and without these patches. With the patches, we found a 19% increase in coverage, as well as many new crash types and increases in the total number of crashes:
I think the build-time option inflexible, but I see the point when you're testing several filesystems that it's one place to set up the environment. Alternatively I suggest to add sysfs knob available in debuging builds to enable/disable checksum verification per filesystem.
As this may not fit to other filesystems I don't suggest to do that for all but I am willing to do that for btrfs, with eventual extension to the config option you propose. The increased fuzzing coverage would be good to have.
Quoted 38 lines · David Sterba
Quoted 27 lines · Hrutvik Kanabar
From: Hrutvik Kanabar <[email protected]>
Fuzzing is a proven technique to discover exploitable bugs in the Linux kernel. But fuzzing filesystems is tricky: highly structured disk images use redundant checksums to verify data integrity. Therefore, randomly-mutated images are quickly rejected as corrupt, testing only error-handling code effectively.
The Janus [1] and Hydra [2] projects probe filesystem code deeply by correcting checksums after mutation. But their ad-hoc checksum-correcting code supports only a few filesystems, and it is difficult to support new ones - requiring significant duplication of filesystem logic which must also be kept in sync with upstream changes. Corrected checksums cannot be guaranteed to be valid, and reusing this code across different fuzzing frameworks is non-trivial.
Instead, this RFC suggests a config option:
DISABLE_FS_CSUM_VERIFICATION. When it is enabled, all filesystems should bypass redundant checksum verification, proceeding as if checksums are valid. Setting of checksums should be unaffected. Mutated images will no longer be rejected due to invalid checksums, allowing testing of deeper code paths. Though some filesystems implement their own flags to disable some checksums, this option should instead disable all checksums for all filesystems uniformly. Critically, any bugs found remain reproducible on production systems: redundant checksums in mutated images can be fixed up to satisfy verification.Quoted 8 lines
The patches below suggest a potential implementation for a few filesystems, though we may have missed some checksums. The option requires
DEBUG_KERNELand is not intended for production systems.The first user of the option would be syzbot. We ran preliminary local syzkaller tests to compare behaviour with and without these patches. With the patches, we found a 19% increase in coverage, as well as many new crash types and increases in the total number of crashes:
Quoted 4 lines
I think the build-time option inflexible, but I see the point when you're testing several filesystems that it's one place to set up the environment. Alternatively I suggest to add sysfs knob available in debuging builds to enable/disable checksum verification per filesystem.
Hi David,
What usage scenarios do you have in mind for runtime changing of this option? I see this option intended only for very narrow use cases which require a specially built kernel in a number of other ways (lots of which are not tunable at runtime, e.g. debugging configs).
Quoted 4 lines
As this may not fit to other filesystems I don't suggest to do that for all but I am willing to do that for btrfs, with eventual extension to the config option you propose. The increased fuzzing coverage would be good to have.
Quoted 8 lines · Dmitry Vyukov
Quoted 6 lines · David Sterba
Quoted 1 line · Hrutvik Kanabar
From: Hrutvik Kanabar <[email protected]>
I think the build-time option inflexible, but I see the point when you're testing several filesystems that it's one place to set up the environment. Alternatively I suggest to add sysfs knob available in debuging builds to enable/disable checksum verification per filesystem.
Quoted 4 lines
What usage scenarios do you have in mind for runtime changing of this option? I see this option intended only for very narrow use cases which require a specially built kernel in a number of other ways (lots of which are not tunable at runtime, e.g. debugging configs).
For my own development and testing usecase I'd like to build the kernel from the same config all the time, then start a VM and run random tests that do not skip the checksum verification. Then as the last also run fuzzing with checksums skipped. The debugging (lockdep, various sanity checks, ...) config options are enabled. We both have a narrow usecase, what I'm suggesting is a common way to enable them.
_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel