The policy on initramfs decompression failure
As part of the multi-compression-formats patch, the issue has come up as to what is the preferred policy is on initramfs decompression failure, due to either corruption or due to the use of a compression format which the kernel does not support.
I had personally assumed the proper policy would be to panic, since it is unlikely to mean the system can be booted. However, Ingo brought up the case where the initramfs is auxilliary to being able to boot the full system, for example the initramfs supplied is primarily a data carrier, and either the builtin initramfs or the kernel itself is sufficient to boot.
By this argument, we should change initramfs decoding failure to a KERN_CRIT message, and in the (presumably most common) case that it does not suffice to boot the system, we will get a panic in short order as the system is unable to find init.
This argument seems to mostly hold water, but it does implement a policy change over the current code. Furthermore, it does make me concerned that a *partial* decoding failure (such as can be caused by a corrupt image, or, say, a gzipped image concatenated to a bzip2 image, with the kernel only supporting bzip2) could cause a booted-but-dysfunctional system, which is in many configurations a worse failure mode than a panic.
Hence I would like to solicit opinions about what the policy should be.
-hpa
Quoted 24 lines · H. Peter Anvin
As part of the multi-compression-formats patch, the issue has come up as to what is the preferred policy is on initramfs decompression failure, due to either corruption or due to the use of a compression format which the kernel does not support.
I had personally assumed the proper policy would be to panic, since it is unlikely to mean the system can be booted. However, Ingo brought up the case where the initramfs is auxilliary to being able to boot the full system, for example the initramfs supplied is primarily a data carrier, and either the builtin initramfs or the kernel itself is sufficient to boot.
By this argument, we should change initramfs decoding failure to a
KERN_CRITmessage, and in the (presumably most common) case that it does not suffice to boot the system, we will get a panic in short order as the system is unable to find init.This argument seems to mostly hold water, but it does implement a policy change over the current code. Furthermore, it does make me concerned that a *partial* decoding failure (such as can be caused by a corrupt image, or, say, a gzipped image concatenated to a bzip2 image, with the kernel only supporting bzip2) could cause a booted-but-dysfunctional system, which is in many configurations a worse failure mode than a panic.
Quoted 3 lines
Hence I would like to solicit opinions about what the policy should be.
-hpa
There is also the additional issue that continuing to boot might hide the original error message. Indeed, the kernel might panic eventually (as you said, for example due to missing init), but in the meantime the original "junk in compressed archive" might have scrolled off the screen. And after a panic, shift+pageup does not work to inspect past messages.
Regards,
Alain
Quoted 2 lines · Alain Knaff
There is also the additional issue that continuing to boot might hide the original error message. [...]
I have hit this pointless panic during testing, that's the motivation for this whole question. The initrd was unimportant in that bootup - but that is generally true of bzImage bootups.
And your argument makes little sense: if there is something wrong then one looks at the logs _anyway_. Are you suggesting that all warnings that signal some potential badness should result in a panic? That is nonsensical.
What you seem to be arguing for is to introduce a kernel option that says "panic on warnings" - so that folks cannot miss warnings. _That_ would be a fair argument.
Panics are rarely good, unless the user asks for it, period. We've been flipping over BUG_ON()s to WARN_ON() everywhere where it matters in practice.
Ingo
Quoted 2 lines · Ingo Molnar
And your argument makes little sense: if there is something wrong then one looks at the logs _anyway_.
Unfortunately, not everybody has the knowledge or equipment ready to set up a serial console... And logs in the classical sense (in a logfile...) don't exist yet at that early stage of boot, because it happens _before_ the kernel is able to write to the filesystem...
Quoted 3 lines
Are you suggesting that all warnings that
signal some potential badness should result in a panic? That is nonsensical.
There must be some misunderstanding somewhere. I didn't make any such suggestion. I agree with you, such a suggestion would be nonsensical.
Quoted 3 lines
What you seem to be arguing for is to introduce a kernel option that says "panic on warnings" - so that folks cannot miss warnings. _That_ would be a fair argument.
That would be an interesting idea, but might lead to the opposite problem (kernel stopping _before_ the real problem happens).
Maybe what we could do is "fix" panic() such that it doesn't disable Shift-Pgup. But I admit that such a change may not be trivial to implement, as there may be cases where the interrupt system is fubar, and all interrupt handlers (including keyboard) would need to be disabled.
Quoted 5 lines
Panics are rarely good, unless the user asks for it, period. We've been flipping over BUG_ON()s to
WARN_ON()everywhere where it matters in practice.Ingo
That is a valid philosophical discussion. But shouldn't we move it to a thread of its own?
Regards,
Alain
Quoted 6 lines · Alain Knaff
Quoted 2 lines · Ingo Molnar
And your argument makes little sense: if there is something wrong then one looks at the logs _anyway_.
Unfortunately, not everybody has the knowledge or equipment ready to set up a serial console... [...]
By your argument the ton of warnings we emit in various situations are wrong too and all should be panic()s. That argument is bogus.
Not looking at the logs makes boot problem analysis harder of course. Nor is your argument actually true: you can use printk_delay or any other method. Or you can use the VGA console and use shift-pageup ...
Ingo
Quoted 10 lines · Ingo Molnar
Quoted 5 lines · Alain Knaff
Quoted 2 lines · Ingo Molnar
And your argument makes little sense: if there is something wrong then one looks at the logs _anyway_.
Unfortunately, not everybody has the knowledge or equipment ready to set up a serial console... [...]
By your argument the ton of warnings we emit in various situations are wrong too and all should be panic()s.
That is not my argument. I never said something like that.
I don't know, but I have to wonder about the strength of _your_ position if the only way to defend it is to put words into other people's mouth.
Quoted 1 line
That argument is bogus.
Indeed that argument is bogus. However, I'm not sure where it is coming from...
Quoted 3 lines
Not looking at the logs makes boot problem analysis harder of course. Nor is your argument actually true: you can use
printk_delayor any other method.
This is interesting information. Where can I find documentation about this? Neither google, nor a find-grep in the kernel sources turned up anything useful...
Quoted 1 line
Or you can use the VGA console and use shift-pageup ...
No you can't. Try it. Or is this only a kvm artifact?
Quoted 2 lines
Ingo
Alain
Quoted 6 lines · Alain Knaff
Quoted 5 lines · Ingo Molnar
Quoted 3 lines · Alain Knaff
Quoted 2 lines · Ingo Molnar
And your argument makes little sense: if there is something wrong then one looks at the logs _anyway_.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Unfortunately, not everybody has the knowledge or equipment ready to set up a serial console... [...]
By your argument the ton of warnings we emit in various situations are wrong too and all should be panic()s.
That is not my argument. I never said something like that.
I did not say that it is your argument, i said it is _by_ your argument:
i.e. it is a logical extension of your argument.
Exactly how is such a warning different from other warnings that the kernel already emits? For which people supposedly have to set up a serial console? (which they dont have to)
Answer: it is not different, and it is exactly as hard or easy to find as the other ones. I.e. why should this warning get a special treatment? I already told the kernel that i dont want a gzip ramfs image decompressor by turning off the (otherwise default-enabled) option. panic()ing on that decision, overriding my decision and escallating it into a non-working system is silly and a bug.
Ingo
Quoted 4 lines · Ingo Molnar
By your argument the ton of warnings we emit in various situations are wrong too and all should be panic()s. That argument is bogus.
Thought about this whole thing some more, and it seems to me as follows:
what we really want, and need, is a "panic-level=X" option, where X will naturally vary for differnet users. I suspect there are many users today who would prefer a panic (and reboot) on a KERN_CRIT message, even at runtime. For finer control, we need a message subsystem tag, but that is something that would be highly desirable anyway.
As such, the initramfs decompression failure should be a KERN_CRIT or KERN_ALERT message, and not a panic per se.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 11 lines · H. Peter Anvin
As part of the multi-compression-formats patch, the issue has come up as to what is the preferred policy is on initramfs decompression failure, due to either corruption or due to the use of a compression format which the kernel does not support.
I had personally assumed the proper policy would be to panic, since it is unlikely to mean the system can be booted. However, Ingo brought up the case where the initramfs is auxilliary to being able to boot the full system, for example the initramfs supplied is primarily a data carrier, and either the builtin initramfs or the kernel itself is sufficient to boot.
I would suggest a default policy of "panic", and a way of overriding the policy, probably with a boot command-line option. The reality is that most of the time, the failure case of a failed or partially failed initramfs is not going to be well tested, as you have pointed out, the downsidesof a booted-but-dysfunctional system is often going to be worse than a hard failure. The partially decoded case is going to be even worse, so I can see a three-way policy:
failed-initramfs-decode=panic Panic on failed initramfs
failed-initramfs-decode=partial If the initramfs fails part-way in,
decode what you can and let the boot system see what files could be fully decypted
failed-initramfs-decode=allow If the initramfs decryption fails
part-of-the-way in, continue the boot, but do not provide the partial initramfs --- i.e., this is the all-or-nothing option
If this is too complicated, I'd be happy with the "panic on failed initramfs". After all, the user can always simply delete the initrd specifier from their grub boot configuration, and simply retry the boot....
- Ted
Quoted 10 lines · Theodore Tso
failed-initramfs-decode=panic Panic on failed initramfs
failed-initramfs-decode=partial If the initramfs fails part-way in,
decode what you can and let the boot system see what files could be fully decypted
failed-initramfs-decode=allow If the initramfs decryption fails
part-of-the-way in, continue the boot, but do not provide the partial initramfs --- i.e., this is the all-or-nothing option
Interesting approach... but wouldn't it make more sense to have that be global? Or else, eventually every single panic will have such a tri-state switch, with associated option parsing and overhead, leading to bloat.
Quoted 4 lines
If this is too complicated, I'd be happy with the "panic on failed initramfs". After all, the user can always simply delete the initrd specifier from their grub boot configuration, and simply retry the boot....
Exactly!
Regards,
Alain