android-xfstests: fix finding FS_TYPE when userdata is on dm device
PATCH RESEND 1/1 android-xfstests: fix finding FS_TYPE when userdata is on dm device
From: Eric Biggers <[email protected]>
If the userdata filesystem is on a device-mapper device, then shrinking the partition underneath it breaks 'blkid' probing, which android-setup-partitions now uses find the filesystem type. Use blkid's '-p' and '-S' options to do a low-level probe with explicitly specified size, which still works in this case.
--- kvm-xfstests/test-appliance/android-setup-partitions | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
kvm-xfstests/test-appliance/android-setup-partitions +5 -2
@@ -259,7 +259,8 @@ get_fs_size() # userdata partition. That isn't necessary, since resizing the underlying # partition to a size smaller than the dm device just causes I/O requests to the # truncated region to fail, and normally there should be no I/O occurring beyond -# the end of the filesystem. +# the end of the filesystem. Exception: this makes 'blkid' stop reporting +# information about the device, unless blkid's -p and -S options are used. shrink_userdata_partition() { local fs_size part_size
@@ -380,7 +381,9 @@ else fi # Type of the userdata filesystem, e.g. ext4 or f2fs -USERDATA_FS_TYPE=$(blkid -s TYPE -o value "$USERDATA_FS_DEV") +USERDATA_FS_TYPE=$(blkid -s TYPE -o value \ + -p -S $(get_partition_size "$USERDATA_RAW_DEV") \ + "$USERDATA_FS_DEV") if ! all_partitions_present ; then # Free up as much space as we can, then create the partitions.
Signature · 1 line
2.19.0.rc2.392.g5ba43deb5a-goog
Theodore Y. Ts'o
AppliedQuoted 9 lines · Eric Biggers
From: Eric Biggers <[email protected]>
If the userdata filesystem is on a device-mapper device, then shrinking the partition underneath it breaks 'blkid' probing, which android-setup-partitions now uses find the filesystem type. Use blkid's '-p' and '-S' options to do a low-level probe with explicitly specified size, which still works in this case.
Signed-off-by Eric Biggers <[email protected]>
Thanks, applied.
- Ted
Quoted 9 lines · Theodore Y. Ts'o
Quoted 8 lines · Eric Biggers
From: Eric Biggers <[email protected]>
If the userdata filesystem is on a device-mapper device, then shrinking the partition underneath it breaks 'blkid' probing, which android-setup-partitions now uses find the filesystem type. Use blkid's '-p' and '-S' options to do a low-level probe with explicitly specified size, which still works in this case.
Quoted 5 lines
Quoted 1 line
Signed-off-by Eric Biggers <[email protected]>Thanks, applied.
- Ted
Hi Ted, are you planning to push this out?
- Eric
Quoted 3 lines · Eric Biggers
Hi Ted, are you planning to push this out?
Oops, sorry. Pushed out now.
- Ted
Quoted 8 lines · Theodore Y. Ts'o
Quoted 3 lines · Eric Biggers
Hi Ted, are you planning to push this out?
Oops, sorry. Pushed out now.
Ted,
FYI, this is pushed to github, but not kernel.org tree. Intentional?
FYI2, I started trying to build a kvm-xfstests image with Debian buster. First thing I ran into with setup-buildchroot is that golang-1.8-go package is not available.
I suppose golang is for syzbot? Their wiki recommends to use golang-go for debian based releases, which does work for buster.
Now I hit an error in ./gen-image:
W: Couldn't download package openssl (ver 1.1.0f-3+deb9u1 arch amd64) at
http://mirrors.kernel.org/debian/pool/main/o/openssl/openssl_1.1.0f-3+deb9u1_amd64.deb
I see that this pkg doesn't exist in the mirror, but this pkg does exists (deb9u2):
http://mirrors.edge.kernel.org/debian/pool/main/o/openssl/openssl_1.1.0f-3+deb9u2_amd64.deb
Do you know what that is about?
Thanks,
Amir.
Quoted 3 lines · Amir Goldstein
FYI, this is pushed to github, but not kernel.org tree. Intentional?
Nope, just another oversight; fixed now.
Quoted 5 lines
FYI2, I started trying to build a kvm-xfstests image with Debian buster. First thing I ran into with setup-buildchroot is that golang-1.8-go package is not available.
I suppose golang is for syzbot? Their wiki recommends to use golang-go for debian based releases, which does work for buster.
At the time syzbot required golang 1.8. I just checked, and according to their docs[1] syzbot now requires golang 1.9. I haven't been bothering quite as much with syzkaller lately because there are now C reproducers, and the syzkaller reproducers are not stable --- that is, if you don't use *precisely* the same version of syzkaller, down to the commit id, the syzkaller repro might not work at all. I've tried talking to Dmitry about providing at least *backwards* compatibility (e.g., that a newer syzkaller can use a repro produced by an older syzbot), but he's not willing to provide even that level of compatibility.
[1] https://github.com/google/syzkaller/blob/master/docs/linux/setup.md
So building syzkaller is currently optional, and the only time I would recommend it is that there apparently some cases where there is a syzkaller reproducer, but no C reproducer, which is very said given the complete lack of backwards compatibility.
I suppose we could include git, the syzkaller git tree and the go compiler into the test VM, and build the appropriate version syzkaller for each repro --- but it would really bloat the kvm-xfstests, and that information isn't even encoded into the syzkaller repro file. :-(
So it's all rather a mess at this point. Maybe this is something we could do just for the gce-xfstests test appliance. Patches appreciated. :-)
On your other note, I haven't bothered using setup-buildchroot since my laptop environment is buster. Making setup-buildchroot install the newest possible version of go available for buster or stretch sounds like the best approach for now.
Quoted 8 lines
Now I hit an error in ./gen-image:
W: Couldn't download package openssl (ver 1.1.0f-3+deb9u1 arch amd64) at
http://mirrors.kernel.org/debian/pool/main/o/openssl/openssl_1.1.0f-3+deb9u1_amd64.deb
I see that this pkg doesn't exist in the mirror, but this pkg does exists (deb9u2):
http://mirrors.edge.kernel.org/debian/pool/main/o/openssl/openssl_1.1.0f-3+deb9u2_amd64.debDo you know what that is about?
That's generally due to the mirror not being fully synced up so the Debian distro metadata files weren't synced up with the debian package files. If you try again in 30 minutes or so, it should resolve itself.
- Ted
Quoted 32 lines · Theodore Y. Ts'o
Quoted 3 lines · Amir Goldstein
FYI, this is pushed to github, but not kernel.org tree. Intentional?
Nope, just another oversight; fixed now.
Quoted 5 lines
FYI2, I started trying to build a kvm-xfstests image with Debian buster. First thing I ran into with setup-buildchroot is that golang-1.8-go package is not available.
I suppose golang is for syzbot? Their wiki recommends to use golang-go for debian based releases, which does work for buster.At the time syzbot required golang 1.8. I just checked, and according to their docs[1] syzbot now requires golang 1.9. I haven't been bothering quite as much with syzkaller lately because there are now C reproducers, and the syzkaller reproducers are not stable --- that is, if you don't use *precisely* the same version of syzkaller, down to the commit id, the syzkaller repro might not work at all. I've tried talking to Dmitry about providing at least *backwards* compatibility (e.g., that a newer syzkaller can use a repro produced by an older syzbot), but he's not willing to provide even that level of compatibility.
[1] https://github.com/google/syzkaller/blob/master/docs/linux/setup.md
So building syzkaller is currently optional, and the only time I would recommend it is that there apparently some cases where there is a syzkaller reproducer, but no C reproducer, which is very said given the complete lack of backwards compatibility.
Quoted 5 lines
I suppose we could include git, the syzkaller git tree and the go compiler into the test VM, and build the appropriate version syzkaller for each repro --- but it would really bloat the kvm-xfstests, and that information isn't even encoded into the syzkaller repro file. :-(
That's a shame.
Quoted 9 lines
So it's all rather a mess at this point. Maybe this is something we could do just for the gce-xfstests test appliance. Patches appreciated. :-)
On your other note, I haven't bothered using setup-buildchroot since my laptop environment is buster. Making setup-buildchroot install the newest possible version of go available for buster or stretch sounds like the best approach for now.
That's what I did. will post one liner once everything is working.
Quoted 8 lines
Quoted 8 lines
Now I hit an error in ./gen-image:
W: Couldn't download package openssl (ver 1.1.0f-3+deb9u1 arch amd64) at
http://mirrors.kernel.org/debian/pool/main/o/openssl/openssl_1.1.0f-3+deb9u1_amd64.deb
I see that this pkg doesn't exist in the mirror, but this pkg does exists (deb9u2):
http://mirrors.edge.kernel.org/debian/pool/main/o/openssl/openssl_1.1.0f-3+deb9u2_amd64.deb
Quoted 7 lines
Quoted 1 line
Do you know what that is about?
That's generally due to the mirror not being fully synced up so the Debian distro metadata files weren't synced up with the debian package files. If you try again in 30 minutes or so, it should resolve itself.
Note that the 2 packages I referred to above are different. One is +deb9u1 (missing) the other is +deb9u2 (available) I see that some packages downloaded have the +deb9u1 suffix some have the +deb9u2 suffix and some no suffix at all.
I know absolutely zero about that distro black magic, so if you have a lead for me, please let me know.
Thanks,
Amir.