blkid: excessive random reads probing for ZFS on NTFS filesystem
Hi,
I noticed blkid causing a noticeable amount of disk hits while identifying an NTFS (Win7) partition, and did an strace to find out why this was. It seems that the probe_zfs() function does a seek-and-read-41-bytes pattern 64 times in the 130 to 520 KB region of the partition. This of course is slow on a traditional hard disk, and seems excessive. I don't know the internals of NTFS or of ZFS. Can someone with a little more knowledge in this area provide some pointers as to why this might be happening?
-- John Lindgren
Karel Zak
maintainerQuoted 5 lines · John Lindgren
I noticed blkid causing a noticeable amount of disk hits while identifying an NTFS (Win7) partition, and did an strace to find out why this was. It seems that the
probe_zfs()function does a seek-and-read-41-bytes pattern 64 times in the 130 to 520 KB region of the partition.
Is it really so big problem on traditional HDDs?
Quoted 4 lines
This of course is slow on a traditional hard disk, and
seems excessive. I don't know the internals of NTFS or of ZFS. Can someone with a little more knowledge in this area provide some pointers as to why this might be happening?
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Quoted 7 lines · Karel Zak
Quoted 5 lines · John Lindgren
I noticed blkid causing a noticeable amount of disk hits while identifying an NTFS (Win7) partition, and did an strace to find out why this was. It seems that the
probe_zfs()function does a seek-and-read-41-bytes pattern 64 times in the 130 to 520 KB region of the partition.Is it really so big problem on traditional HDDs?
Many traditional HD's are rated with a seek time of ~10 ms (though I work with older hardware mostly; newer ones may be better). So by a naive calculation that would make probe_zfs take 0.64 seconds. Now, in the case of probe_zfs, the seeks have good locality, so in practice it will not take that long -- provided blkid is the only program accessing the disk at that time. If the system has some sort of read-ahead process running during boot (as I do), then locality is lost for both blkid and the read-ahead process, and you get thrashing.
Also take into consideration that while 64 seeks may not seem excessive in itself, that is for only one filesystem, and blkid checks something on the order of 40 different filesystems. (Fortunately none of the others take anywhere near the number of seeks that zfs_probe does.)
Andreas: As far as implementation, I think the ZFS code could be improved by parsing the name-value pair list *before* searching for a valid uberblock, rather than after. I don't have any ZFS filesystems around to test with, or I would try to do this myself.
-- John
Karel Zak
maintainerQuoted 17 lines · John Lindgren
Quoted 7 lines · Karel Zak
Quoted 5 lines · John Lindgren
I noticed blkid causing a noticeable amount of disk hits while identifying an NTFS (Win7) partition, and did an strace to find out why this was. It seems that the
probe_zfs()function does a seek-and-read-41-bytes pattern 64 times in the 130 to 520 KB region of the partition.Is it really so big problem on traditional HDDs?
Many traditional HD's are rated with a seek time of ~10 ms (though I work with older hardware mostly; newer ones may be better). So by a naive calculation that would make
probe_zfstake 0.64 seconds. Now, in the case ofprobe_zfs, the seeks have good locality, so in practice it will not take that long -- provided blkid is the only program accessing the disk at that time. If the system has some sort of read-ahead process running during boot (as I do), then locality is lost for both blkid and the read-ahead process, and you get thrashing.
Quoted 4 lines
Also take into consideration that while 64 seeks may not seem excessive in itself, that is for only one filesystem, and blkid checks something on the order of 40 different filesystems. (Fortunately none of the others take anywhere near the number of seeks that
zfs_probedoes.)
Unfortunately, the old less expensive version of the ZFS detection code was insufficient. See Andreas' patches:
commit e54a76ca076625b1883ddf0595162eb8de81d5d1
Author: Andreas Dilger <[email protected]>
Date: Wed Feb 17 10:21:27 2010 +0100
commit a1fbeb3df35d1441c4ef64ea7e04c2b1fda38ba2
Author: Andreas Dilger <[email protected]>
Date: Thu Mar 11 15:16:46 2010 +0100
Quoted 3 lines
Andreas: As far as implementation, I think the ZFS code could be improved by parsing the name-value pair list *before* searching for a valid uberblock, rather than after.
Doesn't depend the position of the list on the position of the valid uberblock?
Quoted 2 lines
I don't have any ZFS filesystems
around to test with, or I would try to do this myself.
There is a test image, see tests/ts/blkid/images-fs/zfs.img.bz2
Maybe you can try to move zfs to the end of the probed filesystems in libblkid/src/superblocks/superblocks.c. It would be better to probe for exotic and "expensive" filesytems later.
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Quoted 10 lines · Karel Zak
Unfortunately, the old less expensive version of the ZFS detection code was insufficient. See Andreas' patches:
commit
e54a76ca076625b1883ddf0595162eb8de81d5d1
Author: Andreas Dilger <[email protected]>
Date: Wed Feb 17 10:21:27 2010 +0100commit
a1fbeb3df35d1441c4ef64ea7e04c2b1fda38ba2
Author: Andreas Dilger <[email protected]>
Date: Thu Mar 11 15:16:46 2010 +0100
Understood. The older implementation performed a less thorough search for a valid uberblock, which I would expect to sometimes fail.
Quoted 5 lines
Quoted 3 lines
Andreas: As far as implementation, I think the ZFS code could be improved by parsing the name-value pair list *before* searching for a valid uberblock, rather than after.
Doesn't depend the position of the list on the position of the valid uberblock?
No, it's at a fixed offset of 16k within the vdev label (before the uberblock array, which is at 128k).
Quoted 3 lines
Quoted 2 lines
I don't have any ZFS filesystems
around to test with, or I would try to do this myself.There is a test image, see tests/ts/blkid/images-fs/zfs.img.bz2
Okay, thanks.
Quoted 3 lines
Maybe you can try to move zfs to the end of the probed filesystems in
libblkid/src/superblocks/superblocks.c. It would be better to probe for exotic and "expensive" filesytems later.
I did try that; however, blkid checks *all* filesystems, even after it finds a match, just in case two probes turn up positive. Even passing "-n nozfs" doesn't actually prevent it from checking for ZFS.
Maybe it would be better to change this behavior and leave the ZFS code as is?
-- John
Karel Zak
maintainerQuoted 3 lines · John Lindgren
I did try that; however, blkid checks *all* filesystems, even after it finds a match, just in case two probes turn up positive. Even passing "-n nozfs" doesn't actually prevent it from checking for ZFS.
The name of the FS in libblkid is zfs_member :-)
blkid -p -o udev -n nozfs_member
Yeah, crazy name, see our TODO file:
- add something like "blkid --list-known" to list all supported filesystems/raids
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Quoted 6 lines · Karel Zak
Quoted 3 lines · John Lindgren
I did try that; however, blkid checks *all* filesystems, even after it finds a match, just in case two probes turn up positive. Even passing "-n nozfs" doesn't actually prevent it from checking for ZFS.
The name of the FS in libblkid is
zfs_member:-)
Quoted 8 lines
blkid -p -o udev -n
nozfs_memberYeah, crazy name, see our TODO file:
- add something like "blkid --list-known" to list all supported filesystems/raids
Karel
My bad. "-n nozfs_member" does work, so I will add it to my udev rules locally.
Perhaps a "-q/--quick" option could be added to blkid to allow an early exit once the first positive match is found? Then more difficult checks such as ZFS could be moved to the end of the list, as you suggest, with a performance benefit.
-- John
Karel Zak
maintainerQuoted 5 lines · John Lindgren
Quoted 4 lines · Karel Zak
Quoted 3 lines · John Lindgren
I did try that; however, blkid checks *all* filesystems, even after it finds a match, just in case two probes turn up positive. Even passing "-n nozfs" doesn't actually prevent it from checking for ZFS.
Quoted 8 lines
Quoted 8 lines
The name of the FS in libblkid is
zfs_member:-)blkid -p -o udev -n
nozfs_memberYeah, crazy name, see our TODO file:
- add something like "blkid --list-known" to list all supported filesystems/raids
Implemented,
blkid -k
lists all known filesystems/RAIDs now.
Quoted 2 lines
Perhaps a "-q/--quick" option could be added to blkid to allow an early exit once the first positive match is found?
That's dangerous, and many times rejected by udev upstream. The mainstream distributions rely on the automatic FS detection on many places. We are not able to control mkfs and partitioning tools, so we are not sure that the device has been properly wiped and there is really only one valid superblock. We already have bad experience with FAT, linux swap, luks, ... some people already lost data.
Quoted 3 lines
Then more difficult checks
such as ZFS could be moved to the end of the list, as you suggest, with a performance benefit.
What about to set to blkid (during boot) greater I/O priority than to read-ahead?
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Quoted 1 line · Karel Zak
CC Andreas who is the code (co)author.
His sun.com address is not active any more (as you have probably noticed).
-- John