Correction to check_filetype()
PATCH 1/1 Correction to check_filetype()
Hi,
If the mode of a directory gets corrupted, check_filetype() makes wrong decisions for all its sub-directories. For example, using debugfs we can corrupt the mode of a directory to 0140755 (i.e. a socket). e2fsck will set the filetype of all its subdirectories as 6 (filetype for socket). All the subdirectories would be moved to lost+found, and in second run of e2fsck their filetype would be set back to 2.
By the time we come to check_filetype(), we have already verified the "." and ".." entries, so we special case these dirents in check_filetype().
Please consider for review.
e2fsck/pass2.c +3 -1
@@ -495,7 +495,9 @@ static _INLINE_ int check_filetype(e2fscreturn 1 ; } -if (ext2fs_test_inode_bitmap (ctx->inode_dir_map, dirent->inode)) { +if (ext2fs_test_inode_bitmap (ctx->inode_dir_map, dirent->inode) || + ((dirent->name_len &&0xFF ) <=2 && dirent->name[0 ] =='.' && + (dirent->name[1 ] =='.' || dirent->name[1 ] =='\0' ))) { should_be =EXT2_FT_DIR ; }else if (ext2fs_test_inode_bitmap (ctx->inode_reg_map, dirent->inode)) {
Hi,
The other issue this brings up is maybe pass1 should be checking whether it is the inode mode that is corrupted (by trying to verify block[0] has "." and ".."in it) instead of truncating off those blocks. This would actually be a redundant check as pass2 would also make the same check.
We can have some more checks like if i_blocks!=0 then the file may not be special file, etc. Does e2fsck need to have more such checks to avoid making decisions just based on the mode?
Thanks,
Kalpak.
Quoted 33 lines · Kalpak Shah
Hi,
If the mode of a directory gets corrupted,
check_filetype()makes wrong decisions for all its sub-directories. For example, using debugfs we can corrupt the mode of a directory to 0140755 (i.e. a socket). e2fsck will set the filetype of all its subdirectories as 6 (filetype for socket). All the subdirectories would be moved to lost+found, and in second run of e2fsck their filetype would be set back to 2.By the time we come to
check_filetype(), we have already verified the "." and ".." entries, so we special case these dirents incheck_filetype().Please consider for review.
Signed-off-by Andreas Dilger <[email protected]>Signed-off-by Kalpak Shah <[email protected]>Index:
e2fsprogs-1.39/e2fsck/pass2.c
===================================================================e2fsck/pass2.c +3
-2@@ -495,7 +495,9 @@ static _INLINE_ int check_filetype(e2fscreturn 1 ; } -if (ext2fs_test_inode_bitmap (ctx->inode_dir_map, dirent->inode)) { +if (ext2fs_test_inode_bitmap (ctx->inode_dir_map, dirent->inode) || + ((dirent->name_len &&0xFF ) <=2 && dirent->name[0 ] =='.' && + (dirent->name[1 ] =='.' || dirent->name[1 ] =='\0' ))) { should_be =EXT2_FT_DIR ; }else if (ext2fs_test_inode_bitmap (ctx->inode_reg_map, dirent->inode)) { -To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 17 lines · Kalpak Shah
Hi,
If the mode of a directory gets corrupted,
check_filetype()makes wrong decisions for all its sub-directories. For example, using debugfs we can corrupt the mode of a directory to 0140755 (i.e. a socket). e2fsck will set the filetype of all its subdirectories as 6 (filetype for socket). All the subdirectories would be moved to lost+found, and in second run of e2fsck their filetype would be set back to 2.By the time we come to
check_filetype(), we have already verified the "." and ".." entries, so we special case these dirents incheck_filetype().Please consider for review.
Signed-off-by Andreas Dilger <[email protected]>Signed-off-by Kalpak Shah <[email protected]>Index:
e2fsprogs-1.39/e2fsck/pass2.c
===================================================================
e2fsck/pass2.c static _INLINE_ int check_filetype(e2fsc · line 500
Quoted 7 lines
e2fsck/pass2.c +2
-1@@ -495,7 +495,9 @@ static _INLINE_ int check_filetype(e2fscreturn 1 ; } -if (ext2fs_test_inode_bitmap (ctx->inode_dir_map, dirent->inode)) { + if (ext2fs_test_inode_bitmap(ctx->inode_dir_map, dirent->inode) || + ((dirent->name_len && 0xFF) <= 2 && dirent->name[0] == '.' &&
What is the "&& 0xFF" part for? At the very least, it should probably be "& 0xFF". It doesn't seem like this mask should be needed at all though, I think.
Thanx...
ps
Quoted 11 lines
e2fsck/pass2.c +1
-0+ (dirent->name[1] == '.' || dirent->name[1] == '\0'))) { should_be =EXT2_FT_DIR ; } else if (ext2fs_test_inode_bitmap(ctx->inode_reg_map,dirent->inode)) {-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi Peter,
Thanks for pointing out the typo. In the kernel, struct ext3_dir_entry_2 has a 8-bit name_len field followed by 8-bit field for filetype. Whereas in e2fsck, struct ext2_dir_entry has a 16-bit name_len field, the upper 8 bits of which store the filetype. Hence e2fsck masks the upper 8 bits while using name_len.
Here is the patch with the change.
e2fsck/pass2.c +3 -1
@@ -495,7 +495,9 @@ static _INLINE_ int check_filetype(e2fscreturn 1 ; } -if (ext2fs_test_inode_bitmap (ctx->inode_dir_map, dirent->inode)) { +if (ext2fs_test_inode_bitmap (ctx->inode_dir_map, dirent->inode) || + ((dirent->name_len &0xFF ) <=2 && dirent->name[0 ] =='.' && + (dirent->name[1 ] =='.' || dirent->name[1 ] =='\0' ))) { should_be =EXT2_FT_DIR ; }else if (ext2fs_test_inode_bitmap (ctx->inode_reg_map, dirent->inode)) {
Thanks,
Kalpak Shah. <kalpak-KYPl3Ael/[email protected]>
Quoted 45 lines · Peter Staubach
Quoted 17 lines · Kalpak Shah
Hi,
If the mode of a directory gets corrupted,
check_filetype()makes wrong decisions for all its sub-directories. For example, using debugfs we can corrupt the mode of a directory to 0140755 (i.e. a socket). e2fsck will set the filetype of all its subdirectories as 6 (filetype for socket). All the subdirectories would be moved to lost+found, and in second run of e2fsck their filetype would be set back to 2.By the time we come to
check_filetype(), we have already verified the "." and ".." entries, so we special case these dirents incheck_filetype().Please consider for review.
Signed-off-by Andreas Dilger <adilger-KYPl3Ael/[email protected]>Signed-off-by Kalpak Shah <kalpak-KYPl3Ael/[email protected]>Index:
e2fsprogs-1.39/e2fsck/pass2.c
===================================================================e2fsck/pass2.c static _INLINE_ int check_filetype(e2fsc · line 500
Quoted 7 lines
e2fsck/pass2.c +2
-1@@ -495,7 +495,9 @@ static _INLINE_ int check_filetype(e2fscreturn 1 ; } -if (ext2fs_test_inode_bitmap (ctx->inode_dir_map, dirent->inode)) { + if (ext2fs_test_inode_bitmap(ctx->inode_dir_map, dirent->inode) || + ((dirent->name_len && 0xFF) <= 2 && dirent->name[0] == '.' &&What is the "&&
0xFF" part for? At the very least, it should probably be "&0xFF". It doesn't seem like this mask should be needed at all though, I think.Thanx... psQuoted 11 lines
e2fsck/pass2.c +1
-0+ (dirent->name[1] == '.' || dirent->name[1] == '\0'))) { should_be =EXT2_FT_DIR ; } else if (ext2fs_test_inode_bitmap(ctx->inode_reg_map,dirent->inode)) {-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 8 lines · Kalpak Shah
If the mode of a directory gets corrupted,
check_filetype()makes wrong decisions for all its sub-directories. For example, using debugfs we can corrupt the mode of a directory to 0140755 (i.e. a socket). e2fsck will set the filetype of all its subdirectories as 6 (filetype for socket). All the subdirectories would be moved to lost+found, and in second run of e2fsck their filetype would be set back to 2.
Um, I'm not seeing this. Using stock e2fsprogs, given the following test image, I'm not seeing the behavior you describe.
It does require a second e2fsck to fix things, which is bad; we need to make sure the i_size is fixed in pass 1.
- Ted
e2fsck 1.40-WIP (14-Nov-2006)
Pass 1: Checking inodes, blocks, and sizes
Inode 12, i_blocks is 2, should be 0. Fix? yes
Pass 2: Checking directory structure
Entry 'dir' in / (2) has an incorrect filetype (was 2, should be 6). Fix? yes
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Inode 2 ref count is 4, should be 3. Fix? yes
Inode 12 ref count is 2, should be 1. Fix? yes
Unattached inode 13
Connect to /lost+found? yes
Inode 13 ref count is 2, should be 1. Fix? yes
Unattached inode 14
Connect to /lost+found? yes
Inode 14 ref count is 2, should be 1. Fix? yes
Unattached inode 15
Connect to /lost+found? yes
Inode 15 ref count is 2, should be 1. Fix? yes
Pass 5: Checking group summary information
Block bitmap differences: -21
Fix? yes
Free blocks count wrong for group #0 (75, counted=76). Fix? yes
Free blocks count wrong (75, counted=76).
Fix? yes
Directories count wrong for group #0 (3, counted=2). Fix? yes
foo.img: ***** FILE SYSTEM WAS MODIFIED *****
foo.img: 15/16 files (0.0% non-contiguous), 24/100 blocks <tytso@candygram> {/usr/projects/e2fsprogs/e2fsprogs/build/e2fsck} 730% e2fsck -fy foo.img
e2fsck 1.40-WIP (14-Nov-2006)
Pass 1: Checking inodes, blocks, and sizes
Special (device/socket/fifo) inode 12 has non-zero size. Fix? yes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
foo.img: ***** FILE SYSTEM WAS MODIFIED ***** foo.img: 15/16 files (0.0% non-contiguous), 24/100 blocks
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 11 lines · Theodore Tso
Quoted 7 lines · Kalpak Shah
If the mode of a directory gets corrupted,
check_filetype()makes wrong decisions for all its sub-directories. For example, using debugfs we can corrupt the mode of a directory to 0140755 (i.e. a socket). e2fsck will set the filetype of all its subdirectories as 6 (filetype for socket). All the subdirectories would be moved to lost+found, and in second run of e2fsck their filetype would be set back to 2.Um, I'm not seeing this. Using stock e2fsprogs, given the following test image, I'm not seeing the behavior you describe.
The log of the e2fsck (on a test filesystem with deliberately introduced corruption) is available at:
https://bugzilla.lustre.org/show_bug.cgi?id=11645
I've also added a testcase (created by making a parent directory with several subdirs, then using debugfs to change the mode of the parent directory).
https://bugzilla.lustre.org/attachment.cgi?id=9958
(also attached here).
It doesn't exhibit the filetype breakage in the upstream e2fsck because the test is run with the patch applied, but it does still show the "size is wrong on second e2fsck" problem you observed. The test case is created as if that problem was also fixed already (i.e. second e2fsck is clean).
Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.
tests/f_check_filetype/expect.1 new file +65 -0
@@ -0,0 +1,65 @@ +Pass 1: Checking inodes, blocks, and sizes +Inode 12, i_blocks is 2, should be 0. Fix? yes + +Pass 2: Checking directory structure +Entry 'dir1' in / (2) has an incorrect filetype (was 2, should be 3). +Fix? yes + +Pass 3: Checking directory connectivity +Unconnected directory inode 13 (???) +Connect to /lost+found? yes + +Unconnected directory inode 14 (???) +Connect to /lost+found? yes + +Unconnected directory inode 15 (???) +Connect to /lost+found? yes + +Unconnected directory inode 16 (???) +Connect to /lost+found? yes + +Unconnected directory inode 17 (???) +Connect to /lost+found? yes + +Unconnected directory inode 18 (???) +Connect to /lost+found? yes + +Unconnected directory inode 19 (???) +Connect to /lost+found? yes + +Pass 4: Checking reference counts +Inode 2 ref count is 4, should be 3. Fix? yes + +Inode 12 ref count is 2, should be 1. Fix? yes + +Inode 13 ref count is 3, should be 2. Fix? yes + +Inode 14 ref count is 3, should be 2. Fix? yes + +Inode 15 ref count is 3, should be 2. Fix? yes + +Inode 16 ref count is 3, should be 2. Fix? yes + +Inode 17 ref count is 3, should be 2. Fix? yes + +Inode 18 ref count is 3, should be 2. Fix? yes + +Inode 19 ref count is 3, should be 2. Fix? yes + +Pass 5: Checking group summary information +Block bitmap differences: -1112 +Fix? yes + +Free blocks count wrong for group #0 (7072, counted=7073). +Fix? yes + +Free blocks count wrong (7072, counted=7073). +Fix? yes + +Directories count wrong for group #0 (10, counted=9). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 19/256 files (5.3% non-contiguous), 1119/8192 blocks +Exit status is 1
tests/f_check_filetype/expect.2 new file +8 -0
@@ -0,0 +1,8 @@ +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information + +test_filesys: 19/256 files (5.3% non-contiguous), 1119/8192 blocks +Exit status is 0
tests/f_check_filetype/name new file +1 -0
@@ -0,0 +1 @@ +parent directory with corrupt mode
tests/f_check_filetype/image.gz new file +6 -0
@@ -0,0 +1,6 @@ +�RFimage ���Uu���;��A(v'HHأ�" +ȀJ/*���bww��.kww�vww'&v��.3� ��/�|>���{����9���uf,�:�$-\(���q�����h��r��Mti3g�.%�f�ʏJ&�76�����fK{��������{��36]��[����]�ï�c~�����^��'�u�`��*��w��w�4���Jk��ş�WN+S��'m�6N����_���iP<��j���fT�������_��& ��8��ӽ��&N�寲 ����|l�]&�g��W�SV(���ꊗ�SV^(��^'�j�����ϊ�����_RX��������h�98y���MY�W�L�S��Q�8m���~���Aip�����ZQ��n�WZhVc�Ω��uҺ�kZ/uK� R��#���|����_q�wouU_�?��o����g�{������j�{��t����b�y����j1~��0~�Z�?�t@-Ɵo:�?��7������O���b�~e| fL����셒�?.���șٲ<ga�ґی�~�Ͷ�a�� +�'��K���ʖ����~2�'�U���(� ����������������z0S�9͒��i����(�H R��(5NMR�B�w� +����i��,͛�K���i��`Z(-�����խ�t<a�3��ɋ%ջ�i���K瞱�:��2d����~E���_w��p������J����է�V��oR}~���Nŧ}z�6zĎ�G�i9b����tl�rۡ#F�<j�[�oվ�*��6kϵ��ڽ���]������cF��2rȘ-zn3b��-��oպͤ��o6b��k^i�Z}{w�?b�Б;�r�Q;��r̰�C2RFi��T�T�����N���ܫׯs���^���z���^�)a���[�٪c�|�-����÷��m�n������Q���5~qt��v��N�zu���AEe�����a;5f�1��mڷk�ӭ�c���ο���Ա�X?s�Ͷ���f����m��X�s������S�v�5���]�����n�_��O���[��g�ڿc�ֿq���Ps[���s굇�l�U�jס��Z�nkb�_�?L�g-Z��_|�>���I� uDI՛q'���0�/��.ߴA�b%�f�ʏJ&�76����xS����Jm��O,��� � MsI��O���>�U�����i�7���A�?G��V�ީO�(m��/n��~���Ai�T���n^`}��"��X5�'7�-�f5�뜊_X'�����R��~� uO=R��`��Uo�^�q'7y���mR��h�����if�����s�U�\�����m������������w�ʧz���]�� �w��_y�����֬�mo�{ǯ~����^�ͫڿ7����o������||�4��+��hĈ9&���k���Mk����?<��-��_|�P����=��oa�� ��U���>P�����V��>\��U�OT�xKCҦih�1< O����iD�2m�F��Ө�M�6m�F�1i��Cڱ��&�vN��]�ni��GڳPu�@�e�>iߴ_�?�L���!��tX:<��LG���1��t\�{�G:>wp'�����tj:-�3�K�J��3ҙ�tv:'���K�҅�tq�$]�.K��� W�����tm�.]�nH7������>8ݖ��nOw�;�]��tO�7ݗ�O��C���Hz4=�/�FKO������lz.=�^H/������jz-���Ho������bһ��~����M�0}�>N��⇌���!�<}��L_���7���]�>��&V��-����e�<͔fN��z�~�5͖fOs��aj��&�i�3͕�N�fi�4_j��O��Bi�HZ4-�OK�%�Ri�LZ6-��O+��eZ1�J�S��6��ڥ��C�VN��U�ji��FZ3U���کSꜺ�uҺ�kZ/uK� R��#�L�^�2�N}�Fi�I����i@��?���W�����q����= ������W=�s����N��9�{ouU_����;y�{�����b�y����j1~��0~�Z�?�t@-Ɵ�WƟ�^��io����O>��x|�������������/T��`�r�S��4��ܶ�啚LYnW�ۀ㪎lokÌ�������Pg52������?���ͧ���j1���a����Tc��Pg51���jj�C�U��������8�gs �e���PRZ(lZf[@]3$��44 K��fi�E��L[��i�4*m��Mۥ�iL�>�vLc�Ni�K�5�vO{�=�^i�O�7��O��A��tH:4�OG�#�Q��tL:6������O'��I��tJ:5����ƥ�����tV:;���M�����tQ�8]�.M������tU�:]��Mץ�� ��tS�9ݒnM�������tW�;ݓ�M�������Pz8=�M�������Tz:=��Mϥ�����Rz9��^M�������Vz;���M���i|�0}�>N��O�giB�<}��L_���7���]�>��&�BwJRi*K�i�4s�%�K�Ӭi�4{�#5H S��85IMӜi�4w�'5K��R�4Z -�J�EҢi��xZ"-��JK�eҲi��|Z!�H-ӊ�Uj�ڤ�i��.�ORǴrZ%��VK��5Қ�"���N�R��%���M]�z�[Z?m����g�0�J��w�6J�MR��/�O��4( N+�� ��ii�04 K��fi�E��L[��i�4*m��Mۥ�iL�>�vLc�Ni�K�5�vO{�=�^i�O�7��O��A��tH:4�OG�#�Q��tL:6������O'��I��tJ:5����ƥ�����tV:;���M�����tQ�8]�.M������tU�:]��Mץ�� ��tS�9ݒnM�������tW�;ݓ�M�������Pz8=�M�������Tz:=��Mϥ�����Rz9��^M�������Vz;���M���i|�0}�>N��O�giB�<}��L_���7���]�>��&�BY�P�JSY*O3���,�^��fM�����Aj��ƩIj��Ls���<�Y�7͗����i��PZ8-�M����iɴTZ:-��M˥�� +�Ej�VL�R��&�M+�v�}�:���*iմZZ=���Li��v�:�.i��n��K���i��=�H=ӆ�W�L�S��Q�8m���~���Aip�[Rf� LkhY�0, O����iD�2m�F��Ө�M�6m�F�1i��C�1�M;���.i״[�=��L{���>iߴ_�?�L���!��tX:<��LG���1��t\�{�G:>��NL'���)��tZ�g���NOg�3�Y��tN:7���O��E��tI�4]�.OW�+�U��tM�6]��O7��M��tK�5ݖ��nOw�;�]��tO�7ݗ�O��C���Hz4=�OO�'�S���Lz6=��O/��K���Jz5��^Oo�7�[���Nz7���O�����Q�8}�>M�� ���E�2}��Nߤo�w���C�� +�BI*Me�<͔fN��z�~�5͖fOs��aj��&�i�3͕�N�fi�4_j��O��Bi�HZ4-�OK�%�Ri�LZ6-��O+��eZ1�J�S��6��ڥ��C�VN��U�ji��FZ3U���کSꜺ�uҺ�kZ/uK� R��#�L�^�2�N}�Fi�r�4 `�4��JJ[�t�rii���c�ry��[5��ܺ��6��,�m<ey�&S��5��I��mm���Qc��ʪ�?�uVC��F�?�Y�������Pg55� ����� @����T � ���V �
_______________________________________________
Lustre-discuss mailing list Lustre-discuss-KYPl3Ael/[email protected]
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
Quoted 9 lines · Andreas Dilger
Quoted 8 lines · Theodore Tso
Quoted 7 lines · Kalpak Shah
If the mode of a directory gets corrupted,
check_filetype()makes wrong decisions for all its sub-directories. For example, using debugfs we can corrupt the mode of a directory to 0140755 (i.e. a socket). e2fsck will set the filetype of all its subdirectories as 6 (filetype for socket). All the subdirectories would be moved to lost+found, and in second run of e2fsck their filetype would be set back to 2.
Ah, OK, I misunderstand the original bug report. It wasn't that the filetype of all of the subdirectories are set incorrectly; it was the filetype of the .. entry in all of the subdirectories. And my test case created files, not directories in the corrupted directory.
So the correct fix for this problem isn't to add a special case in pass2's check_filetype() --- since the directory entry being pointed to really isn't a directory --- but rather in pass3's fix_dot_dot_proc(), which gets executed when we reparent the subdirectories to lost+found:
e2fsck/pass3.c +1 -0
@@ -645,6 +645,7 @@ static int fix_dotdot_proc(struct ext2_dfix_problem (fp->ctx,PR_3_ADJUST_INODE , &pctx); } dirent->inode = fp->parent; + dirent->name_len = (dirent->name_len &0xFF ) |EXT2_FT_DIR <<8 ; fp->done++;return DIRENT_ABORT |DIRENT_CHANGED ;
The question though is whether we should be doing anything more to notice that the real problem isn't the file type information is incorrect, but rather that i_mode in the parent directory is incorrect. Put another way, we can detect that perhaps i_mode is what is inconsistent, given the following clues:
* The inode contains data blocks (although we have to be careful since a block or char device will have a non-zero i_block[0])
* The inode's i_block[0] is valid, isn't being used by any other inode, and the contents of the data block pointed to be i_block[0] appears to be a directory (i.e., has a . and .. entry where '.' points to itself).
* One or more directory entries point to the inode a filetype of EXT2_FT_DIR.
On the other hand, this is difficult to do right, because the first clue has a significant chance of false positives in the case of block and character devices; the second clue can only be completely verified after pass 1 is completely finished; and the third clue can only be verified while executing pass 2.
Furthermore, how often and how likely that we would suffer a single-bit error in i_mode, where the rest of the inode (especially i_blocks and i_block) won't be trashed, and the worst case if we get this wrong is that the files end up in lost+found so we don't lose any data anyway? (This would be less true in the 64-bit "inode in directory" design, which is one of the reasons why while I like it a lot, it is definitely a lot more fragile and will require some very careful handling.)
And of course, if we're going to handle the case where a directory was wrongly turned into a socket/device/fifo, we should also handle the case where a regular file is wrongly turned into a socket/device/fifo, although this gets more difficult since we can't do a test based on '.' and '..' in the case of a regular file has its i_mode field corrupted.
What we can do in the directory case is to perform the '.' and '..' check in pass1, although it would require a fair amount of special case code, and hope that it isn't a false positive or that it happens to be pointing at another (valid) directory --- although if '.' points to itself that would seem unlikely.
dir->name_len = (dir->name_len & 0xFF) | EXT2_FT_DIR << 8;
And we can't really check the case where a directory gets turned into a regular file without destroying e2fsck's performance, since that would require reading the first block of every single file, and this also significantly increases the chance of false positives. So it's a lot of complexity for what seems to have always been an artificial test case.
Let me see what I can do, but the controlling consideration is going to be "first, do no harm", which means (a) it must not harm performance, and (b) it must never cause e2fsck to stop and ask a question on a valid filesystem.
- Ted
Quoted 6 lines · Theodore Tso
And we can't really check the case where a directory gets turned into a regular file without destroying e2fsck's performance, since that would require reading the first block of every single file, and this also significantly increases the chance of false positives. So it's a lot of complexity for what seems to have always been an artificial test case.
OK, this is what I have done so far. Not checked in yet, but it looks right to me. Note that this makes your testcase have a very booring result. :-)
I'm going to let this one soak for a bit to make sure we don't pick up any fase positives or negatives in the hueristics.
- Ted
e2fsck/message.c +20 -0
@@ -35,6 +35,7 @@* %Id <inode> -> i_dir_acl * %Iu <inode> -> i_uid * %Ig <inode> -> i_gid +* %It <inode type> * %j <ino2> inode number * %m <com_err error message> * %N <num>
@@ -310,6 +311,25 @@ static _INLINE_ void expand_inode_expresprintf ("%d" , (inode->i_gid | (inode->osd2.linux2.l_i_gid_high <<16 )));break ; +case 't' : +if (LINUX_S_ISREG (inode->i_mode)) +printf (_ ("regular file" )); +else if (LINUX_S_ISDIR (inode->i_mode)) +printf (_ ("directory" )); +else if (LINUX_S_ISCHR (inode->i_mode)) +printf (_ ("character device" )); +else if (LINUX_S_ISBLK (inode->i_mode)) +printf (_ ("block device" )); +else if (LINUX_S_ISFIFO (inode->i_mode)) +printf (_ ("named pipe" )); +else if (LINUX_S_ISLNK (inode->i_mode)) +printf (_ ("symbolic link" )); +else if (LINUX_S_ISSOCK (inode->i_mode)) +printf (_ ("socket" )); +else +printf (_ ("unknown file type with mode 0%o" ), + inode->i_mode); +break ;default : no_inode:printf ("%%I%c" , ch);
e2fsck/pass1.c +71 -4
94 lines not on this page · raw message
e2fsck/pass3.c +1 -0
@@ -645,6 +645,7 @@ static int fix_dotdot_proc(struct ext2_dfix_problem (fp->ctx,PR_3_ADJUST_INODE , &pctx); } dirent->inode = fp->parent; + dirent->name_len = (dirent->name_len &0xFF ) |EXT2_FT_DIR <<8 ; fp->done++;return DIRENT_ABORT |DIRENT_CHANGED ;
e2fsck/problem.c +5 -0
@@ -778,6 +778,11 @@ static struct e2fsck_problem problem_tab {PR_1_ATTR_HASH ,N_ ("@a in @i %i has a hash (%N) which is @n (must be 0)\n" ), PROMPT_CLEAR, PR_PREEN_OK }, + +/* inode appears to be a directory */ + { PR_1_TREAT_AS_DIRECTORY, +N_ ("@i %i is a %It but it looks like it is really a directory\n" ), + PROMPT_FIX, PR_PREEN_OK },/* Pass 1b errors */
e2fsck/problem.h +3 -0
@@ -452,6 +452,9 @@ struct problem_context {/* wrong EA hash value */ #define PR_1_ATTR_HASH 0x010054 +/* inode appears to be a directory */ +#define PR_1_TREAT_AS_DIRECTORY 0x010055 +/* * Pass 1b errors */
Quoted 7 lines · Theodore Tso
Quoted 6 lines · Theodore Tso
And we can't really check the case where a directory gets turned into a regular file without destroying e2fsck's performance, since that would require reading the first block of every single file, and this also significantly increases the chance of false positives. So it's a lot of complexity for what seems to have always been an artificial test case.
Yes, you can't check whether each regular file is a directory or not. The only case which is easy to check is whether the mode of a directory is corrupted since directories will always have a "." and ".." entry.
Note that we will still lose all data of a regular file if its mode is changed to lets say, a socket. But there is hardly anything we can do about this since there is no special check that can be associated with a regular file.
Since dir_index feature is going to be set by default in ext3/4, maybe we can check for EXT2_INDEX_FL in check_is_really_dir() and if this flag is set then check if such a file is a directory or not. So we would not be reading the first block of each file and yet would be able to correct most directory mode corruptions.
Thanks,
Kalpak.
A message that is not in the archive (sent off-list or not yet received): [email protected]
e2fsck/pass1.c int e2fsck_pass1_check_device_inode(ext2 · line 137
Quoted 11 lines · Theodore Tso
I'm going to let this one soak for a bit to make sure we don't pick up any fase positives or negatives in the hueristics.
e2fsck/pass1.c +3
-2@@ -133,11 +133,10 @@ int e2fsck_pass1_check_device_inode(ext2 +* If the index flag is set, then this is a bogus +* device/fifo/socket */ -if ((ext2fs_inode_data_blocks (fs, inode) !=0 ) || - (inode->i_flags &EXT2_INDEX_FL )) +if (inode->i_flags &EXT2_INDEX_FL )return 0 ;
There were ancient versions of the kernel that left EXT2_INDEX_FL set on all files, instead of just directories... I'm not sure if those were in actual released kernels, or just in patches.
e2fsck/pass1.c static void check_inode_extra_space(e2fs · line 415
Quoted 5 lines
e2fsck/pass1.c +5
-0+static void check_is_really_dir (e2fsck_t ctx,struct problem_context *pctx, +char *buf) +{ +if (ext2fs_read_dir_block (ctx->fs, inode->i_block[0 ], buf)) +return ;
Do we call check_blocks() on this inode shortly thereafter? If we do then the overhead of reading the first block is offset by not reading it again later. Otherwise, this could slow things down.
Quoted 13 lines
e2fsck/pass1.c +13
-0+ dirent = (struct ext2_dir_entry *) buf; +if (((dirent->name_len &0xFF ) !=1 ) || + (dirent->name[0 ] !='.' ) || + (dirent->inode != pctx->ino) || + (dirent->rec_len <12 ) || + (dirent->rec_len %4 ) || + (dirent->rec_len >= ctx->fs->blocksize -12 )) +return ; + + dirent = (struct ext2_dir_entry *) (buf + dirent->rec_len); + if (((dirent->name_len & 0xFF) != 2) || + (dirent->name[0] != '.') || + (dirent->name[1] != '.') ||
e2fsck/pass1.c static void check_inode_extra_space(e2fs · line 439
Quoted 10 lines
e2fsck/pass1.c +9
-0+ (dirent->rec_len <12 ) || + (dirent->rec_len % 4)) + return; + +if (fix_problem (ctx,PR_1_TREAT_AS_DIRECTORY , pctx)) { + inode->i_mode = (inode->i_mode &07777 ) |LINUX_S_IFDIR ; +e2fsck_write_inode_full (ctx, pctx->ino, inode, +EXT2_INODE_SIZE (ctx->fs->super), +"check_is_really_dir" ); }
The one worry I have here (though I don't think it is necessarily IN the code you propose) is that someone could create a regular file which looks like a directory and somehow get it linked into the filesystem tree, giving them escalated access (e.g. device files owned by them, suid executables, links to otherwise unreadable files, etc).
It would seem that this is only a danger if the mode on the file is corrupted, which shouldn't really be doable by a regular user, but it is definitely something to consider.
I take it that this code fixes the test image I previously posted?
Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.
Quoted 4 lines · Andreas Dilger
Quoted 3 lines · Theodore Tso
I'm going to let this one soak for a bit to make sure we don't pick up any fase positives or negatives in the hueristics.
Quoted 12 lines
e2fsck/pass1.c int e2fsck_pass1_check_device_inode(ext2 · line 137
Quoted 8 lines
e2fsck/pass1.c +3
-2@@ -133,11 +133,10 @@ int e2fsck_pass1_check_device_inode(ext2 +* If the index flag is set, then this is a bogus +* device/fifo/socket */ -if ((ext2fs_inode_data_blocks (fs, inode) !=0 ) || - (inode->i_flags &EXT2_INDEX_FL )) +if (inode->i_flags &EXT2_INDEX_FL )return 0 ;There were ancient versions of the kernel that left
EXT2_INDEX_FLset on all files, instead of just directories... I'm not sure if those were in actual released kernels, or just in patches.
Well, we've been running with this in e2fsprogs for quite some time (August 2002, e2fsprogs 1.28), and no one has complained, so I think we're safe...
Quoted 9 lines
e2fsck/pass1.c static void check_inode_extra_space(e2fs · line 415
Quoted 5 lines
e2fsck/pass1.c +5
-0+static void check_is_really_dir (e2fsck_t ctx,struct problem_context *pctx, +char *buf) +{ +if (ext2fs_read_dir_block (ctx->fs, inode->i_block[0 ], buf)) +return ;Do we call
check_blocks()on this inode shortly thereafter? If we do then the overhead of reading the first block is offset by not reading it again later. Otherwise, this could slow things down.
This is why we only do this on special devices; check_is_really_dir() doesn't do anything on directory or regular files.
Quoted 5 lines
The one worry I have here (though I don't think it is necessarily IN the code you propose) is that someone could create a regular file which looks like a directory and somehow get it linked into the filesystem tree, giving them escalated access (e.g. device files owned by them, suid executables, links to otherwise unreadable files, etc).
I thought of that, but I didn't worry about it because I'm not doing this check on regular files. Come to think of it I should add a check so we don't do this for long symlinks, for similar reasons. It would only matter if the user can force filesystem check, which makes it a long shot, but we should avoid that issue as well.
Quoted 1 line
I take it that this code fixes the test image I previously posted?
Yup. Take a look at what I checked in into Mercurial. I added two separate test cases. One of them directly tests the filetype issue, and the other tests a mutated directory into a char device case.
- Ted