btrfs-progs: check update

PATCH In review Qu Wenruo 3 participants 3 messages Nov 26, 2018 – Dec 18, 2018 Btrfs file system

lore

Share & tools

lore

Qu Wenruo

author

2018-11-26 06:04 UTC #0 lore raw

cover btrfs-progs: check update

Hi David,

Please merge this pull request:
https://github.com/kdave/btrfs-progs/pull/155

This patch can be fetch from the following branch:
https://github.com/adam900710/btrfs-progs/tree/check-next

The base commit is:
commit 5d64c40240135cc22f4ba2b902bfe20418a599ea (david/devel)

Author: David Sterba <[email protected]>
Date:   Tue Nov 20 11:13:08 2018 +0100

btrfs-progs: docs: fix rendering of exponents in manual pages

Reported on IRC that the inode number limit appears to be 264, while the actual value is 2^64. Fix that for the manual page backend by redefining the format.

Signed-off-by David Sterba <[email protected]>

This small patchset contains 2 check related functionality:
1) Ability to repair dir item with mismatch hash
Both lowmem and original check, along with test case update.

2) Check qgroup limit exceed
The patch to deprecate BTRFS_QGROUP_LIMIT_RSV_RFER|EXCL is dropped for this pull request, as it's still uncertain if it's OK or not, and it's very easy to rebase that patch.

The patchset passes all selftest with one exception:
misc-tests/021-image-multi-devices

That test case failure is caused by poorly recovered multi-device btrfs-image.
The problem exists from the very beginning, just recent enhanced kernel code will refuse to mount and I'll address it later in another patchset.

There is some other check related patches, naming "btrfs-progs: fixes of file extent in original and lowmem check" from Fujitsu, which is still being updated. I'll push them in next update.

Thanks,
Qu

Qu Wenruo (5):
btrfs-progs: lowmem check: Add ability to repair dir item with
mismatch hash
btrfs-progs: original check: Use mismatch_dir_hash_record to record
bad dir items
btrfs-progs: original check: Add ability to repair dir item with
invalid hash
btrfs-progs: fsck-tests: Make 026-bad-dir-item-name test case to
verify if btrfs-check can also repair it
btrfs-progs: qgroup-verify: Check if qgroup limit is exceeded

 check/main.c                                  | 121 +++++++++++++++++-
 check/mode-common.c                           |  51 ++++++++
 check/mode-common.h                           |   5 +-
 check/mode-lowmem.c                           |  46 ++++++-
 check/mode-lowmem.h                           |   1 +
 check/mode-original.h                         |  14 ++
 ctree.h                                       |   3 +
 dir-item.c                                    |   6 +-
 qgroup-verify.c                               |  82 ++++++++++++
 .../026-bad-dir-item-name/description.txt     |  41 ++++++
 .../fsck-tests/026-bad-dir-item-name/test.sh  |  13 --
 11 files changed, 354 insertions(+), 29 deletions(-)
 create mode 100644 tests/fsck-tests/026-bad-dir-item-name/description.txt
 delete mode 100755 tests/fsck-tests/026-bad-dir-item-name/test.sh

Signature · 1 line

2.19.2

David Sterba

2018-12-17 19:59 UTC #1 lore raw

Quoted 44 lines · Qu Wenruo

Hi David,
Please merge this pull request:
https://github.com/kdave/btrfs-progs/pull/155
This patch can be fetch from the following branch:
https://github.com/adam900710/btrfs-progs/tree/check-next
The base commit is:
commit 5d64c40240135cc22f4ba2b902bfe20418a599ea (david/devel)
Author: David Sterba <[email protected]>
Date:   Tue Nov 20 11:13:08 2018 +0100
btrfs-progs: docs: fix rendering of exponents in manual pages
Reported on IRC that the inode number limit appears to be 264, while the actual value is 2^64. Fix that for the manual page backend by redefining the format.
Signed-off-by David Sterba <[email protected]>
This small patchset contains 2 check related functionality:
1) Ability to repair dir item with mismatch hash
Both lowmem and original check, along with test case update.
2) Check qgroup limit exceed
The patch to deprecate BTRFS_QGROUP_LIMIT_RSV_RFER|EXCL is dropped for this pull request, as it's still uncertain if it's OK or not, and it's very easy to rebase that patch.
The patchset passes all selftest with one exception:
misc-tests/021-image-multi-devices
That test case failure is caused by poorly recovered multi-device btrfs-image.
The problem exists from the very beginning, just recent enhanced kernel code will refuse to mount and I'll address it later in another patchset.
There is some other check related patches, naming "btrfs-progs: fixes of file extent in original and lowmem check" from Fujitsu, which is still being updated. I'll push them in next update.

Quoted 8 lines

Thanks,
Qu
Qu Wenruo (5):
btrfs-progs: lowmem check: Add ability to repair dir item with
mismatch hash
btrfs-progs: original check: Use mismatch_dir_hash_record to record
bad dir items

This patch changes the output style from "report when it happens" to "print a summary". I find the latter a bit more unfriendly as it can take long time before something gets printed at all and it looks like there's no problem with the dir items until that point.
Also if memory allocation fails at the time the entry is added to list, the information is lost.

We can do both but would need to set the verbosity, the default can be the summary. For the verbose level 'more-is-better'. The output of check needs to be improved, that's known, so I'll take the patch as-is, the overall output changes would need a full code audit anyway.

Quoted 4 lines

btrfs-progs: original check: Add ability to repair dir item with
invalid hash
btrfs-progs: fsck-tests: Make 026-bad-dir-item-name test case to
verify if btrfs-check can also repair it

The patches look good.

Quoted 1 line

btrfs-progs: qgroup-verify: Check if qgroup limit is exceeded

This patch has very short changelog compared to what it does, can you please update and resend? Thanks.

Qu Wenruo

2018-12-18 01:13 UTC #2 lore raw

Quoted 54 lines · David Sterba

Quoted 44 lines · Qu Wenruo
Hi David,
Please merge this pull request:
https://github.com/kdave/btrfs-progs/pull/155
This patch can be fetch from the following branch:
https://github.com/adam900710/btrfs-progs/tree/check-next
The base commit is:
commit 5d64c40240135cc22f4ba2b902bfe20418a599ea (david/devel)
Author: David Sterba <[email protected]>
Date:   Tue Nov 20 11:13:08 2018 +0100
btrfs-progs: docs: fix rendering of exponents in manual pages
Reported on IRC that the inode number limit appears to be 264, while the actual value is 2^64. Fix that for the manual page backend by redefining the format.
Signed-off-by David Sterba <[email protected]>
This small patchset contains 2 check related functionality:
1) Ability to repair dir item with mismatch hash
Both lowmem and original check, along with test case update.
2) Check qgroup limit exceed
The patch to deprecate BTRFS_QGROUP_LIMIT_RSV_RFER|EXCL is dropped for this pull request, as it's still uncertain if it's OK or not, and it's very easy to rebase that patch.
The patchset passes all selftest with one exception:
misc-tests/021-image-multi-devices
That test case failure is caused by poorly recovered multi-device btrfs-image.
The problem exists from the very beginning, just recent enhanced kernel code will refuse to mount and I'll address it later in another patchset.
There is some other check related patches, naming "btrfs-progs: fixes of file extent in original and lowmem check" from Fujitsu, which is still being updated. I'll push them in next update.
Quoted 8 lines
Thanks,
Qu
Qu Wenruo (5):
btrfs-progs: lowmem check: Add ability to repair dir item with
mismatch hash
btrfs-progs: original check: Use mismatch_dir_hash_record to record
bad dir items

Quoted 6 lines

This patch changes the output style from "report when it happens" to "print a summary". I find the latter a bit more unfriendly as it can take long time before something gets printed at all and it looks like there's no problem with the dir items until that point.
Also if memory allocation fails at the time the entry is added to list, the information is lost.

I though the summary mode is the default behavior for original mode. I'm happier to change it to report-when-found behavior.

Quoted 12 lines

We can do both but would need to set the verbosity, the default can be the summary. For the verbose level 'more-is-better'. The output of check needs to be improved, that's known, so I'll take the patch as-is, the overall output changes would need a full code audit anyway.
Quoted 4 lines
btrfs-progs: original check: Add ability to repair dir item with
invalid hash
btrfs-progs: fsck-tests: Make 026-bad-dir-item-name test case to
verify if btrfs-check can also repair it

Quoted 7 lines

The patches look good.
Quoted 1 line
btrfs-progs: qgroup-verify: Check if qgroup limit is exceeded
This patch has very short changelog compared to what it does, can you please update and resend? Thanks.

No problem.

Thanks,
Qu