Skip to contentSkip to conversation

crypt issue with 2.6.30/gcc 4.4.0?

DISCUSSION Udo van den Heuvel 3 participants 7 messages Jun 14, 2009

lore
Share & tools
lore
#0 lore raw

Hello,

I am having trouble decrypting my dm-crypted disks with 2.6.30. 2.6.29.4 works fine with the same encrypted disks.
I've been upgrading to Fedora 11 in the past few days so now GCC 4.4.0 is the compiler, that may (also) be a factor.

On 2.6.92.4 I see the encryption password prompt being buried by a lot of USB-messages from connected gear. I can 'blindly' type my password and asterisks appear for each letter. At the end of the password I press Enter; a complaint about the partition table for md1 is displayed and booting continues.
With 2.6.30 I see the Password prompt being repeated, after all the USB messages, as soon as I press the first letter of the password. Each time the password is entered it is not accepted and a new Password: prompt is printed up to a total of three tries.

For all of these boots into 2.6.29.4 or 2.6.30 the userland etc is the same (Fedora 11), it is just that 2.6.29.4 was built while running Fedora 10 and 2.6.30 is now built with the tools in 2.6.30. I did boot into 2.6.30 (built while running Fedora 10) successfully once, right after the upgrade to Fedora 11. Later I made small changes to the kernel config and rebuilt the kernel and this made the encryption password fail. A make clean and rebuild does not fix the issue

How can I find out what's the cause for the password issue?

Kind regards,
Udo

#1 lore raw
Quoted 25 lines · Udo van den Heuvel

Hello,

I am having trouble decrypting my dm-crypted disks with 2.6.30. 2.6.29.4 works fine with the same encrypted disks.
I've been upgrading to Fedora 11 in the past few days so now GCC 4.4.0 is the compiler, that may (also) be a factor.

On 2.6.92.4 I see the encryption password prompt being buried by a lot of USB-messages from connected gear. I can 'blindly' type my password and asterisks appear for each letter. At the end of the password I press Enter; a complaint about the partition table for md1 is displayed and booting continues.
With 2.6.30 I see the Password prompt being repeated, after all the USB messages, as soon as I press the first letter of the password. Each time the password is entered it is not accepted and a new Password: prompt is printed up to a total of three tries.

For all of these boots into 2.6.29.4 or 2.6.30 the userland etc is the same (Fedora 11), it is just that 2.6.29.4 was built while running Fedora 10 and 2.6.30 is now built with the tools in 2.6.30. I did boot into 2.6.30 (built while running Fedora 10) successfully once, right after the upgrade to Fedora 11. Later I made small changes to the kernel config and rebuilt the kernel and this made the encryption password fail. A make clean and rebuild does not fix the issue

Quoted 1 line

How can I find out what's the cause for the password issue?

Instead of changing two variables at once (kernel version AND gcc version) you keep one constant and vary the other.

IOW, rebuild 2.6.29.4 with gcc-4.4 and test. Then rebuild 2.6.30 with gcc-4.3 or older and test.

#2 lore raw
Quoted 8 lines · Mikael Pettersson
Quoted 1 line · Udo van den Heuvel

How can I find out what's the cause for the password issue?

Instead of changing two variables at once (kernel version AND gcc version) you keep one constant and vary the other.

IOW, rebuild 2.6.29.4 with gcc-4.4 and test. Then rebuild 2.6.30 with gcc-4.3 or older and test.

I can do the 2.6.29.4 build with gcc-4.4 but the gcc-4.3 case is harder to do: I need to convince Fedora to install a 2nd compiler-version (and rest of toolchain).
I'll post back with the 2.6.29.4 results using gcc-4.4.

Udo

#3 lore raw
Quoted 7 lines · Udo van den Heuvel
Quoted 6 lines · Mikael Pettersson
Quoted 1 line · Udo van den Heuvel

How can I find out what's the cause for the password issue?

Instead of changing two variables at once (kernel version AND gcc version) you keep one constant and vary the other.

Quoted 6 lines
Quoted 2 lines

IOW, rebuild 2.6.29.4 with gcc-4.4 and test. Then rebuild 2.6.30 with gcc-4.3 or older and test.

I can do the 2.6.29.4 build with gcc-4.4 but the gcc-4.3 case is harder to do: I need to convince Fedora to install a 2nd compiler-version (and rest of toolchain).

Not true. Just as you can choose from multiple self-built kernels you can build and choose dynamically from multiple self-built GCCs.

E.g., for this particular case, download gcc-4.3.3.tar.bz2, tar jxf it, mkdir objdir, cd objdir, /path/to/gcc-4.3.3/configure --prefix=/tmp/gcc43 --enable-languages=c, make, make install, cd /path/to/linux-2.6.30, PATH=/tmp/gcc43/bin:$PATH make, and so on.

#4 lore raw
In reply to Mikael Pettersson
Quoted 1 line · Mikael Pettersson

IOW, rebuild 2.6.29.4 with gcc-4.4 and test.

This testcase gives the same results as 2.6.30 with gcc-4.4, i.e.:
password is not accepted.
So it is not the kernel I guess. Perhaps gcc or something that goes in the ramdisk?

Udo

#5 lore raw
In reply to Udo van den Heuvel
Quoted 5 lines · Udo van den Heuvel

I am having trouble decrypting my dm-crypted disks with 2.6.30. 2.6.29.4 works fine with the same encrypted disks.
I've been upgrading to Fedora 11 in the past few days so now GCC 4.4.0 is the compiler, that may (also) be a factor.

Quoted 9 lines

On 2.6.92.4 I see the encryption password prompt being buried by a lot of USB-messages from connected gear. I can 'blindly' type my password and asterisks appear for each letter. At the end of the password I press Enter; a complaint about the partition table for md1 is displayed and booting continues.
With 2.6.30 I see the Password prompt being repeated, after all the USB messages, as soon as I press the first letter of the password. Each time the password is entered it is not accepted and a new Password: prompt is printed up to a total of three tries.

Hi,
password prompt in Fedora is handled completely in userspace in initrd by plymouth, then it send password to cryptsetup.

You did provide neither kernel log messages nor description of changes to your kernel config... Try run it in text mode with full kernel log, also try enable some debug mode in plymouth and check exact error message.

Anyway, the problem is probably in some misconfiguration in you initrd (like missing kernel module - some new module maybe needed now for your config, bad order of module loading, device name changes of underlying device, maybe some timing problem initialization of USB keyboard etc.) and not in kernel itself. (Are you using mkinitrd from F11? Also see fedora bug 470740)

Milan
--
[email protected]

#6 lore raw
Quoted 7 lines · Milan Broz
Quoted 4 lines

With 2.6.30 I see the Password prompt being repeated, after all the USB messages, as soon as I press the first letter of the password. Each time the password is entered it is not accepted and a new Password: prompt is printed up to a total of three tries.

password prompt in Fedora is handled completely in userspace in initrd by plymouth, then it send password to cryptsetup.

Yes. I saw that in mkinitrd source...

Quoted 1 line

You did provide neither kernel log messages

The kernel is just booting up, messages scroll by fast, no strange stuff appears as far as I can see.

Quoted 2 lines

nor description of changes to
your kernel config...

I got the 2.6.29.4 .config, copied it into 2.6.30 tree and added ext4 as fs and nfs4 server support. Nothing much in the area of where the problem is?

Quoted 1 line

Try run it in text mode with full kernel log,

How to retrieve the log?

Quoted 1 line

also try enable some debug mode in plymouth and check exact error message.

Will have look into possibilties there.

Quoted 1 line

(Are you using mkinitrd from F11? Also see fedora bug 470740)

Indeed. This looks very much like it, although I use a normal PS/2 keyboard and I'll have to count the asterisks.
Thanks for this tip!!

Udo