Online filesystem check framework

LSF/MM TOPIC Goldwyn Rodrigues 3 participants 4 messages Jan 21, 2016 – Jan 25, 2016 VFS

Share & tools

Goldwyn Rodrigues

author

2016-01-21 17:15 UTC #0 lore raw

Topic: Generic Online filesystem Check framework

Motivation:
+ Better uptime - Filesystems turn read-only at the first error
encountered and it may block critical applications which have not encountered the error.
+ Unmountable Filesystems - Some filesystems such as clustered
filesystem may not be unmountable because they are used by too many computers to be taken offline.
+ Autofix - may sound dangerous as fixing is without user
intervention, but an option may help admins which are looking for a good uptime.
+ Logic inbuilt - most logic of access to filesystem is already in the
filesystem driver. The fix/check would make use of existing functionality.

Framework would be around providing a generic interface framework which would use inode numbers as the basic unit to check or fix. Other metadata may need special parameters. Userspace scripts will issue check/fixes to the system, which may/may not be driven by

My other interests include VFS, access control and security, and future plans with the VFS and storage subsystems. I am also interested in multi-device arrays such as MD and DM.

Signature · 1 line

Goldwyn

Jan Kara

reviewer

2016-01-22 09:36 UTC #1 lore raw

Hello,

Quoted 13 lines · Goldwyn Rodrigues

Topic: Generic Online filesystem Check framework
Motivation:
+ Better uptime - Filesystems turn read-only at the first error encountered
and it may block critical applications which have not encountered the error.
+ Unmountable Filesystems - Some filesystems such as clustered filesystem
may not be unmountable because they are used by too many computers to be taken offline.
+ Autofix - may sound dangerous as fixing is without user intervention, but
an option may help admins which are looking for a good uptime.
+ Logic inbuilt - most logic of access to filesystem is already in the
filesystem driver. The fix/check would make use of existing functionality.

Quoted 4 lines

Framework would be around providing a generic interface framework which would use inode numbers as the basic unit to check or fix. Other metadata may need special parameters. Userspace scripts will issue check/fixes to the system, which may/may not be driven by

Well, this is a very difficult topic so I think we need some concrete proposal and ideally some RFC code about what you think needs to be done to make online fsck possible. Without that it will be just a useless handwaving since I don't think anybody is able to come up with a decent proposal during that half an hour session...
Honza

Signature · 2 lines

Jan Kara <[email protected]>
SUSE Labs, CR

Dave Chinner

2016-01-22 12:04 UTC #2 lore raw

Quoted 21 lines · Jan Kara

Hello,
Quoted 13 lines · Goldwyn Rodrigues
Topic: Generic Online filesystem Check framework
Motivation:
+ Better uptime - Filesystems turn read-only at the first error encountered
and it may block critical applications which have not encountered the error.
+ Unmountable Filesystems - Some filesystems such as clustered filesystem
may not be unmountable because they are used by too many computers to be taken offline.
+ Autofix - may sound dangerous as fixing is without user intervention, but
an option may help admins which are looking for a good uptime.
+ Logic inbuilt - most logic of access to filesystem is already in the
filesystem driver. The fix/check would make use of existing functionality.
Quoted 4 lines
Framework would be around providing a generic interface framework which would use inode numbers as the basic unit to check or fix. Other metadata may need special parameters. Userspace scripts will issue check/fixes to the system, which may/may not be driven by

Quoted 5 lines

Well, this is a very difficult topic so I think we need some concrete proposal and ideally some RFC code about what you think needs to be done to make online fsck possible. Without that it will be just a useless handwaving since I don't think anybody is able to come up with a decent proposal during that half an hour session...

I don't see it being generic, either, because metadata structurei, dependencies, redundancy and algorithms are highly filesystem specific. It just doesn't seem feasible to me - if it was then we'd only need one userspace fsck/repair program....

FWIW, for XFS we already have a plan towards implementing online repair[*] which we've been working towards for the past 5 years, but it's another 5 years worth of work before we'll get to the point of being able to dynamically repair corruptions as they are detected...

[*] I wrote this document back in 2008:

http://xfs.org/index.php/Reliable_Detection_and_Repair_of_Metadata_Corruption

We've got metadata CRCs now, and reverse mapping is just about ready to be merged. Then we need parent pointers (i.e. directory structure reverse mappings), transaction rollback on error, etc, before we can even start to think about online repair algorithms.

Cheers,

Dave.