BMC Image Signing Proposal
Hi all,
During the hackaton meetup, we touched on BMC image verification and signing and concluded that the community would like to see two verification steps, one prior to writing the image to flash (via digital signature verification), and another one by checking FIT in U-Boot prior to booting from the new image.
The proposal would be to implement the digital signature verification first.
At a high level, during the build:
* A SHA-256 hash would be calculated over tbd files that make up the firmware image.
* The hash would be signed by a private key that's part of the repository (community key). Companies could overwrite it with their own private key when building production images.
* The encrypted hash (digital signature) and corresponding public key would be added to the firmware image.
* Yocto may already provide a way to sign images and generate keys.
On the BMC:
* The hash would be calculated on the image files that were uploaded to the BMC.
* The signature would be decrypted using an existing public key on the BMC (this validates the new public key delivered with the image is also valid).
* Accept image if both values are the same.
Any thoughts or comments?
Hi, Anoo!
The thoughts are as follows:
1. BMC usually runs in a secured environment where probability of tampering with flash IC contents by means other than BMC's firmware itself is negligible.
2. U-Boot already performs image checksum validation before booting a FIT image
3. User input really needs validation, at least to make the system fool-proof
Having said that, I suggest that the only thing that really needs doing is signing (and checking) of the overall firmware image file that is supplied by the user (admin) during the firmware upgrade procedure. Applying asymmetric cryptography to a digest hash looks to me like a good idea as it indeed allows for verifying the supplier of the firmware image.
Regards,
Alexander Amelkin
YADRO
Quoted 19 lines · anoo
Hi all,
During the hackaton meetup, we touched on BMC image verification and signing and concluded that the community would like to see two verification steps, one prior to writing the image to flash (via digital signature verification), and another one by checking FIT in U-Boot prior to booting from the new image.
The proposal would be to implement the digital signature verification first.
At a high level, during the build:
* A SHA-256 hash would be calculated over tbd files that make up the firmware image.
* The hash would be signed by a private key that's part of the repository (community key). Companies could overwrite it with their own private key when building production images.
* The encrypted hash (digital signature) and corresponding public key would be added to the firmware image.
* Yocto may already provide a way to sign images and generate keys.On the BMC:
* The hash would be calculated on the image files that were uploaded to the BMC.
* The signature would be decrypted using an existing public key on the BMC (this validates the new public key delivered with the image is also valid).
* Accept image if both values are the same.Any thoughts or comments?
Hi Alexander,
Quoted 7 lines · Alexander Amelkin
Hi, Anoo!
The thoughts are as follows:
1. BMC usually runs in a secured environment where probability of tampering with flash IC contents by means other than BMC's firmware itself is negligible.
This does bring up the issue of developing a threat model to develop against; we should probably do that. However, without one, I feel like we should design *for* defense in depth and allow people to remove protections as they see fit for their environment, rather than make potentially compromising assumptions about what that environment is at the outset.
For instance, whilst BMCs might typically be isolated from customer traffic on a separate LAN, there's still the in-band interface which can be used to poke at the BMC. Vulnerabilities in the IPMI stack could lead to BMC compromise and undesirable flash writes*. Therefore BMC flash integrity remains a valid concern despite network isolation.
* This ties into Michael Brown's talks of isolating daemons to their own user/group and enforcing SELinux policy against them.
Quoted 3 lines
2. U-Boot already performs image checksum validation before booting a FIT image
Typically the rootfs is not part of the FIT, so it will not be checked.
Some systems supported by OpenBMC directly mount the rootfs rather
than booting through an initrd, which makes rootfs authentication somewhat tricky. Regardless, with signed images we should expand the FIT hash check to be a full signature check.
Quoted 10 lines
3. User input really needs validation, at least to make the system fool-proof
Having said that, I suggest that the only thing that really needs doing is signing (and checking) of the overall firmware image file that is supplied by the user (admin) during the firmware upgrade procedure. Applying asymmetric cryptography to a digest hash looks to me like a good idea as it indeed allows for verifying the supplier of the firmware image.
Agreed - do Yadro do this already? If so, what did you do to integrate image signing into the build process? If not, then I'd be keen to keep the discussion alive on how we can achieve it.
Cheers,
Andrew
Hi,
Quoted 28 lines · Andrew Jeffery
Hi Alexander,
Quoted 7 lines · Alexander Amelkin
Hi, Anoo!
The thoughts are as follows:
1. BMC usually runs in a secured environment where probability of tampering with flash IC contents by means other than BMC's firmware itself is negligible.
This does bring up the issue of developing a threat model to develop against; we should probably do that. However, without one, I feel like we should design *for* defense in depth and allow people to remove protections as they see fit for their environment, rather than make potentially compromising assumptions about what that environment is at the outset.
For instance, whilst BMCs might typically be isolated from customer traffic on a separate LAN, there's still the in-band interface which can be used to poke at the BMC. Vulnerabilities in the IPMI stack could lead to BMC compromise and undesirable flash writes*. Therefore BMC flash integrity remains a valid concern despite network isolation.
* This ties into Michael Brown's talks of isolating daemons to their own user/group and enforcing SELinux policy against them.
Quoted 1 line
Quoted 8 lines
Quoted 2 lines
2. U-Boot already performs image checksum validation before booting a FIT image
Typically the rootfs is not part of the FIT, so it will not be checked.
Some systems supported by OpenBMC directly mount the rootfs rather
than booting through an initrd, which makes rootfs authentication somewhat tricky. Regardless, with signed images we should expand the FIT hash check to be a full signature check.
If you use an initrd in the FIT, it can be checked in the same way as the kernel. FIT supports a full signature check.
If you use a separate rootfs, verity can be used if it is read-only. Otherwise I am not sure.
Quoted 12 lines
Quoted 11 lines
3. User input really needs validation, at least to make the system fool-proof
Having said that, I suggest that the only thing that really needs doing is signing (and checking) of the overall firmware image file that is supplied by the user (admin) during the firmware upgrade procedure. Applying asymmetric cryptography to a digest hash looks to me like a good idea as it indeed allows for verifying the supplier of the firmware image.
Quoted 7 lines
Agreed - do Yadro do this already? If so, what did you do to integrate image signing into the build process? If not, then I'd be keen to keep the discussion alive on how we can achieve it.
Cheers,
Andrew
Regards,
Simon
Quoted 29 lines · Simon Glass
Hi,
Quoted 26 lines · Andrew Jeffery
Hi Alexander,
Quoted 7 lines · Alexander Amelkin
Hi, Anoo!
The thoughts are as follows:
1. BMC usually runs in a secured environment where probability of tampering with flash IC contents by means other than BMC's firmware itself is negligible.
This does bring up the issue of developing a threat model to develop against; we should probably do that. However, without one, I feel like we should design *for* defense in depth and allow people to remove protections as they see fit for their environment, rather than make potentially compromising assumptions about what that environment is at the outset.
For instance, whilst BMCs might typically be isolated from customer traffic on a separate LAN, there's still the in-band interface which can be used to poke at the BMC. Vulnerabilities in the IPMI stack could lead to BMC compromise and undesirable flash writes*. Therefore BMC flash integrity remains a valid concern despite network isolation.
* This ties into Michael Brown's talks of isolating daemons to their own user/group and enforcing SELinux policy against them.
We would prefer that no environment is used for any action of consequence. It is completely untrusted, and only used for things like hints of preferences. For example, we would store the last preferred image (A or B), but ultimately, the one chosen would have to be authenticated before booting anyway; it is just the preferred image would be checked first and only fall back to the other one if verification failed.
Along the same lines, if the user stopped the BMC from automatically booting, we are now in a situation where the Linux runtime's root of trust could be compromised (specifically by forcing a boot of something that does not verify or by passing custom kernel command line arguments). I would prefer if there was a way that if stopped in u-boot, you would have to reset the BMC to actually boot to the kernel.
Quoted 10 lines
Quoted 9 lines
Quoted 3 lines
2. U-Boot already performs image checksum validation before booting a FIT image
Typically the rootfs is not part of the FIT, so it will not be checked.
Some systems supported by OpenBMC directly mount the rootfs rather
than booting through an initrd, which makes rootfs authentication somewhat tricky. Regardless, with signed images we should expand the FIT hash check to be a full signature check.
Quoted 5 lines
If you use an initrd in the FIT, it can be checked in the same way as the kernel. FIT supports a full signature check.
If you use a separate rootfs, verity can be used if it is read-only. Otherwise I am not sure.
Historically, we have just booted with an initramfs (squashfs) that is the root filesystem. This means that it is possible to update in place without worrying about corrupting the current runtime. Normally our updates are A->B->A, but we do have one mode that is always in place.
Our current plan was to have our FIT include U-Boot, Linux, rootfs (as squashfs), and DTB, with a signature on U-Boot individually, and then another signature on all the parts. We would use an SPL to verify the signature of U-Boot and then boot it, and then U-Boot would verify the signature of the full FIT before booting Linux.
--Vernon
Quoted 21 lines
Quoted 11 lines
Quoted 10 lines
3. User input really needs validation, at least to make the system fool-proof
Having said that, I suggest that the only thing that really needs doing is signing (and checking) of the overall firmware image file that is supplied by the user (admin) during the firmware upgrade procedure. Applying asymmetric cryptography to a digest hash looks to me like a good idea as it indeed allows for verifying the supplier of the firmware image.
Quoted 7 lines
Agreed - do Yadro do this already? If so, what did you do to integrate image signing into the build process? If not, then I'd be keen to keep the discussion alive on how we can achieve it.
Cheers,
Andrew
Regards,
Simon
Quoted 16 lines · Andrew Jeffery
Quoted 7 lines · Alexander Amelkin
Hi, Anoo!
The thoughts are as follows:
1. BMC usually runs in a secured environment where probability of tampering with flash IC contents by means other than BMC's firmware itself is negligible.
This does bring up the issue of developing a threat model to develop against; we should probably do that. However, without one, I feel like we should design *for* defense in depth and allow people to remove protections as they see fit for their environment, rather than make potentially compromising assumptions about what that environment is at the outset.
Quoted 8 lines
For instance, whilst BMCs might typically be isolated from customer traffic on a separate LAN, there's still the in-band interface which can be used to poke at the BMC. Vulnerabilities in the IPMI stack could lead to BMC compromise and undesirable flash writes*. Therefore BMC flash integrity remains a valid concern despite network isolation.
* This ties into Michael Brown's talks of isolating daemons to their own user/group and enforcing SELinux policy against them.
using isolation technologies that are used by containers may also add extra depth, and be a good interim solution as SELinux policy is developed and tuned.
Quoted 8 lines
Quoted 2 lines
2. U-Boot already performs image checksum validation before booting a FIT image
Typically the rootfs is not part of the FIT, so it will not be checked.
Some systems supported by OpenBMC directly mount the rootfs rather
than booting through an initrd, which makes rootfs authentication somewhat tricky. Regardless, with signed images we should expand the FIT hash check to be a full signature check.
dm-verity would solve that (for a ro rootfs). The read/write parts will have to be carefully controlled of course, noexec is an obvious thing. Oh, and not having configuration as executable code and being *paranoid* about parsing config.
Signature · 2 lines
Stewart Smith OPAL Architect, IBM.
Quoted 6 lines · Stewart Smith
Quoted 5 lines · Andrew Jeffery
Quoted 2 lines · Alexander Amelkin
2. U-Boot already performs image checksum validation before booting a FIT image
Quoted 7 lines
Quoted 5 lines
Typically the rootfs is not part of the FIT, so it will not be checked.
Some systems supported by OpenBMC directly mount the rootfs rather
than booting through an initrd, which makes rootfs authentication somewhat tricky. Regardless, with signed images we should expand the FIT hash check to be a full signature check.dm-verity would solve that (for a ro rootfs).
dm-verity is a worthwhile technology, but being based on device mapper and therefore block devices, we can't use it for MTD devices, which is all of the upstream OpenBMC machines at this moment.
I would suggest using some kind of pre-mount verification of the raw MTD device against a stored checksum would be the way to go. This would imply the use of an initrd (as we would need somewhere to store the tools that do the verification). The initrd itself would be verified by u-boot checking the FIT.
Future contributors to OpenBMC that have eMMC hardware do have the option of using dm-verity.
Cheers,
Joel
Hi Joel,
Quoted 15 lines · Joel Stanley
Quoted 12 lines · Stewart Smith
Quoted 10 lines · Andrew Jeffery
Quoted 2 lines · Alexander Amelkin
2. U-Boot already performs image checksum validation before booting a FIT image
Typically the rootfs is not part of the FIT, so it will not be checked.
Some systems supported by OpenBMC directly mount the rootfs rather
than booting through an initrd, which makes rootfs authentication somewhat tricky. Regardless, with signed images we should expand the FIT hash check to be a full signature check.
Quoted 5 lines
Quoted 1 line
dm-verity would solve that (for a ro rootfs).
dm-verity is a worthwhile technology, but being based on device mapper and therefore block devices, we can't use it for MTD devices, which is all of the upstream OpenBMC machines at this moment.
You could use ubi to provide a block device, I haven't tried it though.
Quoted 9 lines
I would suggest using some kind of pre-mount verification of the raw MTD device against a stored checksum would be the way to go. This would imply the use of an initrd (as we would need somewhere to store the tools that do the verification). The initrd itself would be verified by u-boot checking the FIT.
Future contributors to OpenBMC that have eMMC hardware do have the option of using dm-verity.
Are you saying that dm-verity does not work with eMMC, or something else?
Regards,
SImon
Quoted 20 lines · Joel Stanley
Quoted 6 lines · Stewart Smith
Quoted 5 lines · Andrew Jeffery
Quoted 2 lines · Alexander Amelkin
2. U-Boot already performs image checksum validation before booting a FIT image
Quoted 7 lines
Quoted 5 lines
Typically the rootfs is not part of the FIT, so it will not be checked.
Some systems supported by OpenBMC directly mount the rootfs rather
than booting through an initrd, which makes rootfs authentication somewhat tricky. Regardless, with signed images we should expand the FIT hash check to be a full signature check.dm-verity would solve that (for a ro rootfs).
dm-verity is a worthwhile technology, but being based on device mapper and therefore block devices, we can't use it for MTD devices, which is all of the upstream OpenBMC machines at this moment.
Quoted 5 lines
I would suggest using some kind of pre-mount verification of the raw MTD device against a stored checksum would be the way to go. This would imply the use of an initrd (as we would need somewhere to store the tools that do the verification). The initrd itself would be verified by u-boot checking the FIT.
mtdblock could end up being okay for the dm-verity case? There's no writes there at least... although I haven't spent much/any time thinking of any implications to that - you're the bigger expert there than I.
Signature · 2 lines
Stewart Smith OPAL Architect, IBM.
Thanks everybody for your thoughts so far. I'll try to comment on the remaining open questions.
Summarizing, there are 2 different verifications that are desired, one prior to writing the image to flash (using signature check), and one prior to booting the image (FIT verification).
1. Signature check
Quoted 1 line
Can you take a look at swupdate and reply with your thoughts?
Yes, thanks Andrew for looking and suggesting this.
Quoted 2 lines
To overcome that the public key should be stored in OTP. We use this method on our Nuvoton Poleg BMC.
One concern would be if the key is deemed compromised. Avi, does the Nuvoton system have a mechanism for invalidating a key stored in OTP?
Quoted 1 line
Quoted 1 line
The "firmware update" public key in flash (and firmware update code itself) would be authenticated by BMC secure boot.
Per Eugene's comment, maybe we're ok with the key in flash?
Quoted 1 line
Whatever mechanism is used to generate the signature should be able to be overridden by an environment variable or something.
Yes, good point Vernon.
2. FIT verification
Quoted 2 lines
Regardless, with signed images we should expand the FIT hash check to be a full signature check.
Yes, I think there was some agreement on this statement during the hackaton.
Quoted 1 line
Also - has anybody mentioned using the FIT as a firmware update install image, in addition to using to boot the kernel?(i.e. FIT within a FIT). Think the native DFU command in u-boot supports this method. Just a thought
Yes, ideally we'd implement FIT verification of the rootfs in addition to the kernel to verify an image before booting.
Here are some charts with the image signing flow for comment:
https://drive.google.com/file/d/1IxfMYRttN8RbhRY7PwBmXsqCBvtv_yLJ/view?usp=sharing
Quoted 11 lines · Adriana Kobylak
Here are some charts with the image signing flow for comment:
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdri
ve.google.com%2Ffile%2Fd%2F1IxfMYRttN8RbhRY7PwBmXsqCBvtv_yLJ %2Fview%3Fusp%3Dsharing&data=04%7C01%7Cyupalani%40microsoft.c om%7C9a2d9f45a2144c59bc5908d56f3235c7%7Cee3303d7fb734b0c8589b cd847f1c277%7C1%7C0%7C636537183804895636%7CUnknown%7CTWFp bGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ %3D%3D%7C- 1&sdata=C0RBtfmjE3VHH1A8A8MwCBVFCUQyvF3WEdWH7bgjRC0%3D& reserved=0
Thanks for putting together a chart.
We should consider both of these requirements for image signing:
1. Update verification
2. Boot Verification
Appending signature to image meets verification during firmware update. To do verification on every boot, we need something like FIT.
https://chromium.googlesource.com/chromiumos/third_party/u-boot-next/+/chromeos-v2013.06/doc/uImage.FIT
As far as actual signing is concerned, we don't have access to private key for security reasons. We should support two models:
Model 1:
Source code has private key and signing is part of build process ("bitbake obmc-phosphor-image")
Model 2:
Source code does not have private key, Signing is done externally and some post-processing is done to add hash to image. (maybe a new task, "bitbake obmc-phosphor-image -c add_hash")
Quoted 4 lines · Yugi Mani
We should consider both of these requirements for image signing:
1. Update verification
2. Boot Verification
Yes, the boot verification via FIT is being tracked via issue https://github.com/openbmc/openbmc/issues/2829, it's planned to be implemented after the update verification.
Quoted 4 lines
Appending signature to image meets verification during firmware update. To do verification on every boot, we need something like FIT.
https://chromium.googlesource.com/chromiumos/third_party/u-boot-next/+/chromeos-v2013.06/doc/uImage.FIT
Thanks, I've added the link to the issue 2829 for reference.
Quoted 10 lines
As far as actual signing is concerned, we don't have access to private key for security reasons. We should support two models:
Model 1:
Source code has private key and signing is part of build process ("bitbake obmc-phosphor-image")Model 2:
Source code does not have private key, Signing is done externally and some post-processing is done to add hash to image. (maybe a new task, "bitbake obmc-phosphor-image -cadd_hash")
Yeah, these are good points, we were thinking also through a build environment variable this could be specified. Eddie is implementing this piece so he could expand on this.
For the location of the public keys on the BMC, seems /etc/ would be an appropriate place as the keys are static and unshareable (reference: http://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s07.html).
Any thoughts on storing the public keys on /etc/ssh/ for example?
Quoted 3 lines · Adriana Kobylak
For the location of the public keys on the BMC, seems
/etc/would be an appropriate place as the keys are static and unshareable (reference:
http://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s07.html).
What do you mean unsharable?
Quoted 1 line
Any thoughts on storing the public keys on
/etc/ssh/for example?
This is where ssh keys go :)
Storing them in /etc/ makes sense though.
Cheers,
Joel
Quoted 4 lines
What do you mean unsharable?
The public keys only apply to the BMC where the firmware image is being installed on and not are not intended to be shared with other BMCs / hosts. "Shareable files can be stored on one host and used on several others" (http://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch02.html).
On another note, I'll be adding the gerrit links to the main epic (https://github.com/openbmc/openbmc/issues/356) as changes start getting submitted for anyone that wants to leave comments or follow along. The first ones are:
https://gerrit.openbmc-project.xyz/#/c/8949/ - Add image signing framework and open keys
https://gerrit.openbmc-project.xyz/#/c/9025/ - BMC Digital Signature documentation
Quoted 7 lines · Yugi Mani
We should consider both of these requirements for image signing:
1. Update verification
2. Boot VerificationAppending signature to image meets verification during firmware update. To do verification on every boot, we need something like FIT.
https://chromium.googlesource.com/chromiumos/third_party/u-boot-next/+/chromeos-v2013.06/doc/uImage.FIT
Quoted 8 lines
As far as actual signing is concerned, we don't have access to private key for security reasons. We should support two models:
Model 1:
Source code has private key and signing is part of build process ("bitbake obmc-phosphor-image")Model 2:
Source code does not have private key, Signing is done externally and some post-processing is done to add hash to image. (maybe a new task, "bitbake obmc-phosphor-image -cadd_hash")
For reference, for OpenPOWER host firmware, we support three models:
Local mode (a.k.a. development mode) — Build the container and sign using locally available private keys. Signatures are generated using simple openssl operations. Because the private keys are exposed on the local system (the build machine), this mode should be used only for development signing, or when the user is confident that the build machine is secure against unauthorized access.
Independent mode — Generate the signing requests locally and export the requests for signing externally. External signing is by user's method of
choice: any method capable of generating a ECDSA p521 signature (the built-in support uses openssl). Resulting signatures are re-imported to the container build process, to create the completed container. No private or privileged information is exposed at the build machine.
Production mode — Build the container locally and interface with the remote signframework to retrieve signatures and (public) keys as needed. Signing is done remotely on a secure signing server using a hardware security module (HSM). Private keys are stored securely in the HSM and never exposed. Completed signatures are returned by the signframework and integrated into the container.
(gratuitously copy&pasted from Nick's great docs up at
https://github.com/open-power/sb-signing-utils/wiki )
Signature · 2 lines
Stewart Smith OPAL Architect, IBM.
Quoted 40 lines · Stewart Smith
Quoted 7 lines · Yugi Mani
We should consider both of these requirements for image signing:
1. Update verification
2. Boot VerificationAppending signature to image meets verification during firmware update. To do verification on every boot, we need something like FIT.
https://chromium.googlesource.com/chromiumos/third_party/u-boot-next/+/chromeos-v2013.06/doc/uImage.FITQuoted 8 lines
As far as actual signing is concerned, we don't have access to private key for security reasons. We should support two models:
Model 1:
Source code has private key and signing is part of build process ("bitbake obmc-phosphor-image")Model 2:
Source code does not have private key, Signing is done externally and some post-processing is done to add hash to image. (maybe a new task, "bitbake obmc-phosphor-image -cadd_hash")For reference, for OpenPOWER host firmware, we support three models:
Local mode (a.k.a. development mode) — Build the container and sign using locally available private keys. Signatures are generated using simple openssl operations. Because the private keys are exposed on the local system (the build machine), this mode should be used only for development signing, or when the user is confident that the build machine is secure against unauthorized access.
Independent mode — Generate the signing requests locally and export the requests for signing externally. External signing is by user's method of
choice: any method capable of generating a ECDSA p521 signature (the built-in support uses openssl). Resulting signatures are re-imported to the container build process, to create the completed container. No private or privileged information is exposed at the build machine.Production mode — Build the container locally and interface with the remote signframework to retrieve signatures and (public) keys as needed. Signing is done remotely on a secure signing server using a hardware security module (HSM). Private keys are stored securely in the HSM and never exposed. Completed signatures are returned by the signframework and integrated into the container.
Quoted 2 lines
(gratuitously copy&pasted from Nick's great docs up at
https://github.com/open-power/sb-signing-utils/wiki )
Historically, we have supported two signing modes, similar to one and three:
Local mode (a.k.a. development mode) - similar to what you describe where the keys are insecure and should not be used to generate production images.
Re-Sign mode (for Production images) - takes a binary image, parses and deconstructs it, verifies the signature, and puts it back together with a new signature and public key certificate. This makes it so we only ever sign 'blessed' images with the production key, where this blessing comes from the validation team. The production build server initially signs the release image with a non-production key and that image gets tested. Then it gets re-signed with the production key only if it passes validation.
--Vernon
Quoted 4 lines · Adriana Kobylak
Here are some charts with the image signing flow for comment:
https://drive.google.com/file/d/1IxfMYRttN8RbhRY7PwBmXsqCBvtv_yLJ/view?usp=sharing
Nancy, does anyone from your team want to have input on this?
Quoted 3 lines · Adriana Kobylak
Here are some charts with the image signing flow for comment:
https://drive.google.com/file/d/1IxfMYRttN8RbhRY7PwBmXsqCBvtv_yLJ/view?usp=sharing
Why are there changes to host pnor signing?
We already have secure boot and signatures on the host side, I'm not keen on adding in another set of signatures into yet-another already non-standard and undocumented file format.
Signature · 2 lines
Stewart Smith OPAL Architect, IBM.
Quoted 8 lines
Why are there changes to host pnor signing?
We already have secure boot and signatures on the host side, I'm not keen on adding in another set of signatures into yet-another already non-standard
and undocumented file format.
Secure boot validates individual pnor packages, so it could be possible to build an image with an older individual package (like an opal n-1) and still be able to boot the host. Signing the whole image would catch this scenario. This image signing feature in general is an extension of the current implementation of the software dbus interfaces, which provides options for a user/company to use.
I'd like to bring this topic again.
As I know image signing feature is completed for both BMC and PNOR:
https://github.com/openbmc/phosphor-bmc-code-mgmt
https://github.com/openbmc/openpower-pnor-code-mgmt
However, the above repos are only for systems with UBI-FS feature. Most of machines are still using the "legacy" obmc flash layout, and thus they do not have image singing feature.
So I would like to ask for ideas about how to support image signing feature for machines with "legacy" flash layout?
1. Should we use UBI-FS for machines that requires image signing feature?
2. Or should we implement image signing feature on "legacy" flash layout as well?
Thanks!
Good point. We at MSFT are using legacy (non-UBI) layout. We have a manifest for boot verification and we append the hash to image for update verification. I can share details about the design/implementation, if you have any specific questions.
Quoted 37 lines
-----Original Message-----
From: openbmc <openbmc- [email protected]> On Behalf Of Lei YU
Sent: Monday, May 14, 2018 7:06 PM
To: Adriana Kobylak <[email protected]>
Cc: Stewart Smith <[email protected]>; OpenBMC Maillist <[email protected]>
Subject: Re: BMC Image Signing ProposalI'd like to bring this topic again.
As I know image signing feature is completed for both BMC and PNOR:
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit
hub.com%2Fopenbmc%2Fphosphor-bmc-code- mgmt&data=02%7C01%7Cyupalani%40microsoft.com%7C12f250ae199b4 32be89a08d5ba089773%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C 0%7C636619468424007343&sdata=GHBBgm9g9kBpfNAzYxgveFoGWXXl1 QOr8fHwZNJNvNA%3D&reserved=0
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit
hub.com%2Fopenbmc%2Fopenpower-pnor-code- mgmt&data=02%7C01%7Cyupalani%40microsoft.com%7C12f250ae199b4 32be89a08d5ba089773%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C 0%7C636619468424007343&sdata=CR2o%2FgVnHPfMr7w8fwOYeRsbOdF B3VMiB%2FFUCnE2aHE%3D&reserved=0However, the above repos are only for systems with UBI-FS feature. Most of machines are still using the "legacy" obmc flash layout, and thus they do not have image singing feature.
So I would like to ask for ideas about how to support image signing feature for machines with "legacy" flash layout?
1. Should we use UBI-FS for machines that requires image signing feature?
2. Or should we implement image signing feature on "legacy" flash layout as well?Thanks!
Quoted 3 lines · Yugi Mani
Good point. We at MSFT are using legacy (non-UBI) layout. We have a manifest for boot verification and we append the hash to image for update verification. I can share details about the design/implementation, if you have any
specific questions.
I'd be interested in what you came up with as we have the same two issues with OpenPOWER: update verification (not security so much as stopping people doing something silly like flashing the wrong firmware image), as well as partial downgrade protection (i.e. the host has to verify the integrity of the whole image, not just individual components)
Signature · 2 lines
Stewart Smith OPAL Architect, IBM.
Quoted 2 lines · Yugi Mani
Good point. We at MSFT are using legacy (non-UBI) layout. We have a manifest for boot verification and we append the hash to image for update verification. I can share details about the design/implementation, if you have any specific questions.
At Intel, we are using a legacy layout as well, either ping-ponging between partitions or using a active/temp/recovery partition scheme depending on the secure boot mechanism for that platform.
--Vernon
Quoted 37 lines
Quoted 37 lines
-----Original Message-----
From: openbmc <openbmc- [email protected]> On Behalf Of Lei YU
Sent: Monday, May 14, 2018 7:06 PM
To: Adriana Kobylak <[email protected]>
Cc: Stewart Smith <[email protected]>; OpenBMC Maillist <[email protected]>
Subject: Re: BMC Image Signing ProposalI'd like to bring this topic again.
As I know image signing feature is completed for both BMC and PNOR:
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit
hub.com%2Fopenbmc%2Fphosphor-bmc-code- mgmt&data=02%7C01%7Cyupalani%40microsoft.com%7C12f250ae199b4 32be89a08d5ba089773%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C 0%7C636619468424007343&sdata=GHBBgm9g9kBpfNAzYxgveFoGWXXl1 QOr8fHwZNJNvNA%3D&reserved=0
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit
hub.com%2Fopenbmc%2Fopenpower-pnor-code- mgmt&data=02%7C01%7Cyupalani%40microsoft.com%7C12f250ae199b4 32be89a08d5ba089773%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C 0%7C636619468424007343&sdata=CR2o%2FgVnHPfMr7w8fwOYeRsbOdF B3VMiB%2FFUCnE2aHE%3D&reserved=0However, the above repos are only for systems with UBI-FS feature. Most of machines are still using the "legacy" obmc flash layout, and thus they do not have image singing feature.
So I would like to ask for ideas about how to support image signing feature for machines with "legacy" flash layout?
1. Should we use UBI-FS for machines that requires image signing feature?
2. Or should we implement image signing feature on "legacy" flash layout as well?Thanks!
Quoted 9 lines · Vernon Mauery
Quoted 6 lines · Yugi Mani
Good point. We at MSFT are using legacy (non-UBI) layout. We have a manifest for boot verification and we append the hash to image for update verification.
I can share details about the design/implementation, if you have any specific questions.
Quoted 4 lines
At Intel, we are using a legacy layout as well, either ping-ponging between partitions or using a active/temp/recovery partition scheme depending on the secure boot mechanism for that platform.
Thanks for info!
So I think it is better for OpenBMC project to have a common (or example) image signing tools/code, not for a specific machine or product, but for the general machines in this project using legacy flash layout. Let's discuss and get a design proposal?
Quoted 3 lines
--Vernon
Quoted 5 lines · Lei YU
So I think it is better for OpenBMC project to have a common (or example) image signing
tools/code, not for a specific machine or product, but for the general machines in this project using legacy flash layout. Let's discuss and get a design proposal?
Yeah, ideally we'd converge the legacy and ubi code update methods into one, to take advantage of the Software D-Bus interfaces and have the different features like signature verification, filesystem mirroring, etc be able to be picked up as separate packages. One starting point for the design proposal would be to determine how to separate in the build and the repo the different code update methods. Lei, want to take an initial stab at it? :)
Quoted 9 lines · Adriana Kobylak
Quoted 5 lines · Lei YU
So I think it is better for OpenBMC project to have a common (or example) image signing
tools/code, not for a specific machine or product, but for the general machines in this project using legacy flash layout. Let's discuss and get a design proposal?Yeah, ideally we'd converge the legacy and ubi code update methods into one, to take advantage of the Software D-Bus interfaces and have the different features like signature verification, filesystem mirroring, etc be able to be picked up as separate packages. One starting point for the design proposal would be to determine how to separate in the build and the repo the different code update methods. Lei, want to take an initial stab at it? :)
As far as the update methods go, just using a descriptive payload goes a long way. Historically, using a manifest type thing that told the update mechanism which bytes to write to where was pretty simple and very effective. The update mechanism would check the authenticity of the payload before trusting the manifest, but then the manifest and payload would all get written to the flash. This was for specific flash offsets and did not fully comprehend the notion of A/B or other redundant scenarios, so all the offsets were relative.
But doing something in the manifest so simple as this would work for a variety of scenarios:
MANIFEST:
purpose=xyz.openbmc_project.Software.Version.VersionPurpose.BMC version=v2.2-32-gc6712d3-dirty
KeyType=OpenBMC
HashType=RSA-SHA256
# Expected is a list of files alongside the manifest Expected=part1,part2,part3,partN
update_pre=<target to activate prior to fwupdate>
# obmc-flash-bmc-by-name knows where to place this part by name # (e.g. u-boot always goes at 0x00000000) [email protected]
# obmc-flash-bmc-at-offset places a part at a fixed offset # optionally relative to an optionally specified MTD partition # (default relative to /dev/mtd0) [email protected] 0x130000 # (default relative to /dev/mtd0) [email protected] 0x130000
# not a ubi person, so I can't comment on all the options :) [email protected]
partN_update=<service for updating partN>
update_post=<target to activate post fwupdate>
The idea is that we can have a series of service files that can handle the majority of operations generically and any special cases can supply their own. But since it is just a service, the dependencies are all taken care of.
Just some thoughts on how this might be done. This is actually something that I am working on right now, trying to get our redundant image booting working internally, so I am glad to hear that others are working toward similar goals.
--Vernon
Quoted 53 lines · Vernon Mauery
Quoted 10 lines · Adriana Kobylak
Quoted 6 lines · Lei YU
So I think it is better for OpenBMC project to have a common (or example) image signing
tools/code, not for a specific machine or product, but for the
general machines in this project using legacy flash layout. Let's discuss and get a design proposal?Quoted 6 lines
Yeah, ideally we'd converge the legacy and ubi code update methods into one, to take advantage of the Software D-Bus interfaces and have the different features like signature verification, filesystem mirroring, etc be able to be picked up as separate packages. One starting point for the design proposal would be to determine how to separate in the build and the repo the different code update methods. Lei, want to take an initial stab at it? :)
As far as the update methods go, just using a descriptive payload goes a long way. Historically, using a manifest type thing that told the update mechanism which bytes to write to where was pretty simple and very effective. The update mechanism would check the authenticity of the payload before trusting the manifest, but then the manifest and payload would all get written to the flash. This was for specific flash offsets and did not fully comprehend the notion of A/B or other redundant scenarios, so all the offsets were relative.
But doing something in the manifest so simple as this would work for a variety of scenarios:
MANIFEST:
purpose=xyz.openbmc_project.Software.Version.VersionPurpose.BMC version=v2.2-32-gc6712d3-dirty
KeyType=OpenBMC
HashType=RSA-SHA256# Expected is a list of files alongside the manifest Expected=part1,part2,part3,partN
update_pre=<target to activate prior to fwupdate>
# obmc-flash-bmc-by-name knows where to place this part by name # (e.g. u-boot always goes at 0x00000000) [email protected]# obmc-flash-bmc-at-offset places a part at a fixed offset # optionally relative to an optionally specified MTD partition # (default relative to /dev/mtd0) [email protected] 0x130000 # (default relative to /dev/mtd0) [email protected] 0x130000
Quoted 5 lines
# not a ubi person, so I can't comment on all the options :) [email protected]partN_update=<service for updating partN>
update_post=<target to activate post fwupdate>
This is interesting, and it looks like a general method for code updating to support both non-ubi and ubi layout. (Though it would require code changes in phosphor-software-manager).
I have several questions though.
Background:
* For non-ubi, there are two ways to do code update:
1. Copy image-bmc (or image-rofs, etc) to /run/initramfs, and reboot. A updater script in initramfs will run to program the image to flash. During the code update, BMC is NOT operational;
2. Invoking "prepareForUpdate" method to ask BMC reboot into a ramfs, then invoking the updater script to program the image to flash. During the code update, BMC is working.
3. There is not WebUI support (yet)
* For ubi:
1. The update happens when BMC is working (due to the fact it assumes the flash contains enough space for two images).
2. The WebUI will upload the image and "activate" it when BMC is running.
So here is my question: to support non-ubifs code update in a general way, should it do "prepareForUpdate", making BMC to reboot into ramfs, or should it do the update during reboot?
* If we prefer the "prepareForUpdate" way, then the WebUI should add this function to make BMC enter "update" mode;
* If we prefer to update during reboot, then the BMC will not be working for a few minutes during the code update, WebUI will not be working as well.
Quoted 14 lines
The idea is that we can have a series of service files that can handle the majority of operations generically and any special cases can supply their own. But since it is just a service, the dependencies are all taken care of.
Just some thoughts on how this might be done. This is actually something that I am working on right now, trying to get our redundant image booting working internally, so I am glad to hear that others are working toward similar goals.
--Vernon
Quoted 58 lines · Lei YU
Quoted 56 lines · Vernon Mauery
Quoted 10 lines · Adriana Kobylak
Quoted 6 lines · Lei YU
So I think it is better for OpenBMC project to have a common (or example) image signing
tools/code, not for a specific machine or product, but for the
general machines in this project using legacy flash layout. Let's discuss and get a design proposal?Quoted 6 lines
Yeah, ideally we'd converge the legacy and ubi code update methods into one, to take advantage of the Software D-Bus interfaces and have the different features like signature verification, filesystem mirroring, etc be able to be picked up as separate packages. One starting point for the design proposal would be to determine how to separate in the build and the repo the different code update methods. Lei, want to take an initial stab at it? :)
As far as the update methods go, just using a descriptive payload goes a long way. Historically, using a manifest type thing that told the update mechanism which bytes to write to where was pretty simple and very effective. The update mechanism would check the authenticity of the payload before trusting the manifest, but then the manifest and payload would all get written to the flash. This was for specific flash offsets and did not fully comprehend the notion of A/B or other redundant scenarios, so all the offsets were relative.
But doing something in the manifest so simple as this would work for a variety of scenarios:
MANIFEST:
purpose=xyz.openbmc_project.Software.Version.VersionPurpose.BMC version=v2.2-32-gc6712d3-dirty
KeyType=OpenBMC
HashType=RSA-SHA256# Expected is a list of files alongside the manifest Expected=part1,part2,part3,partN
update_pre=<target to activate prior to fwupdate>
# obmc-flash-bmc-by-name knows where to place this part by name # (e.g. u-boot always goes at 0x00000000) [email protected]# obmc-flash-bmc-at-offset places a part at a fixed offset # optionally relative to an optionally specified MTD partition # (default relative to /dev/mtd0) [email protected] 0x130000 # (default relative to /dev/mtd0) [email protected] 0x130000# not a ubi person, so I can't comment on all the options :) [email protected]
Quoted 6 lines
Quoted 2 lines
partN_update=<service for updating partN>
update_post=<target to activate post fwupdate>This is interesting, and it looks like a general method for code updating to support both non-ubi and ubi layout. (Though it would require code changes in phosphor-software-manager).
That was the idea :)
I think that if we really want to converge on some common code here, having the phosphor-software-manager handle several different flash formats is important. This is the sort of thing that could easily be selectable at build time, one method should have little impact on the other.
Quoted 16 lines
I have several questions though.
Background:
* For non-ubi, there are two ways to do code update:
1. Copy image-bmc (or image-rofs, etc) to/run/initramfs, and reboot. A updater script in initramfs will run to program the image to flash. During the code update, BMC is NOT operational;
2. Invoking "prepareForUpdate" method to ask BMC reboot into a ramfs, then invoking the updater script to program the image to flash. During the code update, BMC is working.
3. There is not WebUI support (yet)
* For ubi:
1. The update happens when BMC is working (due to the fact it assumes the flash contains enough space for two images).
2. The WebUI will upload the image and "activate" it when BMC is running.
Quoted 6 lines
So here is my question: to support non-ubifs code update in a general way, should it do "prepareForUpdate", making BMC to reboot into ramfs, or should it do the update during reboot?
* If we prefer the "prepareForUpdate" way, then the WebUI should add this function to make BMC enter "update" mode;
Intel has a requirement to do firmware updates while the BMC is active, so we will need to enable this feature in the WebUI. This is not a hard problem, given that a redfish mechanism is already in the works.
--Vernon
Quoted 17 lines
* If we prefer to update during reboot, then the BMC will not be working for a few minutes during the code update, WebUI will not be working as well.
Quoted 14 lines
The idea is that we can have a series of service files that can handle the majority of operations generically and any special cases can supply their own. But since it is just a service, the dependencies are all taken care of.
Just some thoughts on how this might be done. This is actually something that I am working on right now, trying to get our redundant image booting working internally, so I am glad to hear that others are working toward similar goals.
--Vernon
Quoted 59 lines · Vernon Mauery
Quoted 58 lines · Lei YU
Quoted 56 lines · Vernon Mauery
Quoted 10 lines · Adriana Kobylak
Quoted 6 lines · Lei YU
So I think it is better for OpenBMC project to have a common (or example) image signing
tools/code, not for a specific machine or product, but for the
general machines in this project using legacy flash layout. Let's discuss and get a design proposal?Quoted 6 lines
Yeah, ideally we'd converge the legacy and ubi code update methods into one, to take advantage of the Software D-Bus interfaces and have the different features like signature verification, filesystem mirroring, etc be able to be picked up as separate packages. One starting point for the design proposal would be to determine how to separate in the build and the repo the different code update methods. Lei, want to take an initial stab at it? :)
As far as the update methods go, just using a descriptive payload goes a long way. Historically, using a manifest type thing that told the update mechanism which bytes to write to where was pretty simple and very effective. The update mechanism would check the authenticity of the payload before trusting the manifest, but then the manifest and payload would all get written to the flash. This was for specific flash offsets and did not fully comprehend the notion of A/B or other redundant scenarios, so all the offsets were relative.
But doing something in the manifest so simple as this would work for a variety of scenarios:
MANIFEST:
purpose=xyz.openbmc_project.Software.Version.VersionPurpose.BMC version=v2.2-32-gc6712d3-dirty
KeyType=OpenBMC
HashType=RSA-SHA256# Expected is a list of files alongside the manifest Expected=part1,part2,part3,partN
update_pre=<target to activate prior to fwupdate>
# obmc-flash-bmc-by-name knows where to place this part by name # (e.g. u-boot always goes at 0x00000000) [email protected]# obmc-flash-bmc-at-offset places a part at a fixed offset # optionally relative to an optionally specified MTD partition # (default relative to /dev/mtd0) [email protected] 0x130000 # (default relative to /dev/mtd0) [email protected] 0x130000# not a ubi person, so I can't comment on all the options :) [email protected]
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
partN_update=<service for updating partN>
update_post=<target to activate post fwupdate>This is interesting, and it looks like a general method for code updating to support both non-ubi and ubi layout. (Though it would require code changes in phosphor-software-manager).
One other thought I had was that we could make the manifest a JSON file which makes for very simple parsing (since the parser is already written). Then we could go with something like this:
{
'purpose': 'xyz.openbmc_project.Software.Version.VersionPurpose.BMC',
'version': 'v2.2-32-gc6712d3-dirty',
'KeyType': 'OpenBMC',
'HashType': 'RSA-SHA256',
'ImageParts': [
'part1': {
'update': {
'service': '[email protected]','
'args': [ 0x130000 ]
}
},
'part2': {
'update': {
'service': '[email protected]','
'args': [ '/dev/mtd/image-a', 0x80000 ]
}
},
'part3': {
'update': {
'service': '[email protected]',
}
}
]
}This makes it very clear what all the bits mean. We would need to document the format of the JSON very clearly and define behavior for malformed structures, but I like this better than just a key/value flat text file.
--Vernon
Quoted 3 lines
That was the idea :)
I think that if we really want to converge on some common code here, having the phosphor-software-manager handle several different flash formats is important. This is the sort of thing that could easily be selectable at build time, one method should have little impact on the other.
Quoted 5 lines · Vernon Mauery
One other thought I had was that we could make the manifest a JSON file which makes for very simple parsing (since the parser is already written). Then we could go with something like this:
That's a good option, at least for the write to flash piece. We could even extend the manifest to include the names of the service files to delete/clean up the flash. Most of the rest of the code manages the D-Bus objects so that'd be common with all flash layouts.
Another option, or combination with a json manifest, would be to have different repos or different subdirectories for specific implementations.
Lei, thinking we could convert Romulus to ubi, and use the PNOR chip to store the alternate BMC version. I think that'd be more straight fwd and the advantage would be that the interfaces are tested and verified. And on the side we can continue this discussion on how to make the code more modular to support other layouts and we can start making the changes but at least we can get Romulus using signature validation in the mean time.
Quoted 7 lines · Adriana Kobylak
Quoted 5 lines · Vernon Mauery
One other thought I had was that we could make the manifest a JSON file which makes for very simple parsing (since the parser is already written). Then we could go with something like this:
Quoted 7 lines
That's a good option, at least for the write to flash piece. We could even extend the manifest to include the names of the service files to delete/clean up the flash. Most of the rest of the code manages the D-Bus objects so that'd be common with all flash layouts.
Another option, or combination with a json manifest, would be to have different repos or different subdirectories for specific implementations.
I would prefer not to need to split the implementation, but that may not always be possible.
One thing along this line of thinking is that Intel's BMCs don't have the notion of field mode, which is baked into the software manager. So it might be helpful to have some way to deal with this.
One thought was that external service files executed from the manifest could deal with this (keep the OEM service files in external repositories).
--Vernon
Quoted 9 lines
Lei, thinking we could convert Romulus to ubi, and use the PNOR chip to store the alternate BMC version. I think that'd be more straight fwd and the advantage would be that the interfaces are tested and verified. And on the side we can continue this discussion on how to make the code more modular to support other layouts and we can start making the changes but at least we can get Romulus using signature validation in the mean time.
Quoted 17 lines · Adriana Kobylak
Quoted 6 lines · Vernon Mauery
One other thought I had was that we could make the manifest a JSON file which makes for very simple parsing (since the parser is already written). Then we could go with something like this:
That's a good option, at least for the write to flash piece. We could even extend the manifest to include the names of the service files to delete/clean up the flash. Most of the rest of the code manages the D-Bus objects so that'd be common with all flash layouts.
Another option, or combination with a json manifest, would be to have different repos or different subdirectories for specific implementations.
Quoted 7 lines
Lei, thinking we could convert Romulus to ubi, and use the PNOR chip to store the alternate BMC version. I think that'd be more straight fwd and the advantage would be that the interfaces are tested and verified. And on the side we can continue this discussion on how to make the code more modular to support other layouts and we can start making the changes but at least we can get Romulus using signature validation in the mean time.
By default Romulus has a single 32MiB for BMC and 64MiB for PNOR, there is not enough room for Romulus to enable ubi with alternative BMC.
So I am afraid that it can not enable ubifs.
But I do have enabled image signature verification on fixed layout and tested on Romulus.
The related changes are:
https://gerrit.openbmc-project.xyz/#/c/10765/
https://gerrit.openbmc-project.xyz/#/c/10801/
https://gerrit.openbmc-project.xyz/#/c/10768/
https://gerrit.openbmc-project.xyz/#/c/10800/
I have tested on Romulus that:
* Code update successful with valid image, both REST API and in WebUI;
* Code update failure with invalid image (signature verify failure), if FieldMode is enabled;
I you think that is OK, we can merge the changes, as it adds the new features (code update and signature verify in phosphor-software-manager) without breaking existing services.
It could be a start point for phosphor-software-manager to handle both ubifs and non-ubifs.
Then we can continue to work on enhancing/refactoring to make it more generic as Vernon suggested.
Quoted 10 lines · Lei YU
I'd like to bring this topic again.
As I know image signing feature is completed for both BMC and PNOR:
https://github.com/openbmc/phosphor-bmc-code-mgmt
https://github.com/openbmc/openpower-pnor-code-mgmtHowever, the above repos are only for systems with UBI-FS feature. Most of machines are still using the "legacy" obmc flash layout, and thus they do not have image singing feature.
Quoted 5 lines
So I would like to ask for ideas about how to support image signing feature for machines with "legacy" flash layout?
1. Should we use UBI-FS for machines that requires image signing feature?
2. Or should we implement image signing feature on "legacy" flash layout as well?
Either option is valid. I think we should get it gonig for non-UBI-FS systems though, and we should do that through the host
Signature · 2 lines
Stewart Smith OPAL Architect, IBM.
Quoted 3 lines · Alexander Amelkin
1. BMC usually runs in a secured environment where probability of tampering with flash IC contents by means other than BMC's firmware itself is negligible.
Consider the host to be actively hostile.
In any metal-as-a-service environment, a privilege escalation is to get code onto the BMC, and thus survive to the next tenant. While the bmc to host interfaces *should* be nice and secure, they are software, and thus are *going* to have security issues. A solid secure-boot story on the BMC would add defense in depth.
Signature · 2 lines
Stewart Smith OPAL Architect, IBM.
Quoted 11 lines · anoo
Hi all,
During the hackaton meetup, we touched on BMC image verification and signing and concluded that the community would like to see two verification steps, one prior to writing the image to flash (via digital signature verification), and another one by checking FIT in U-Boot prior to booting from the new image.
The proposal would be to implement the digital signature verification first.
Quoted 9 lines
At a high level, during the build:
* A SHA-256 hash would be calculated over tbd files that make up the firmware image.
* The hash would be signed by a private key that's part of the repository (community key). Companies could overwrite it with their own private key when building production images.
* The encrypted hash (digital signature) and corresponding public key would be added to the firmware image.
* Yocto may already provide a way to sign images and generate keys.
I'm starting to look into signing of artefacts as part of the build process. Some quick searching suggests Yocto doesn't have support for it out of the box, but there is the swupdate project[1] and the corresponding meta-swupdate[2] Yocto layer that sound interesting. However, it may require a rework of how we manage image updates if we were to make use of it. I'll keep poking around, but if anyone has any other pointers please let me know.
[1] https://github.com/sbabic/swupdate
[2] https://github.com/sbabic/meta-swupdate
Quoted 10 lines
On the BMC:
* The hash would be calculated on the image files that were uploaded to the BMC.
* The signature would be decrypted using an existing public key on the BMC (this validates the new public key delivered with the image is also valid).
* Accept image if both values are the same.Any thoughts or comments?
Can you take a look at swupdate and reply with your thoughts?
Cheers,
Andrew
Quoted 12 lines · Andrew Jeffery
Quoted 11 lines · anoo
Hi all,
During the hackaton meetup, we touched on BMC image verification and signing and concluded that the community would like to see two verification steps, one prior to writing the image to flash (via digital signature verification), and another one by checking FIT in U-Boot prior to booting from the new image.
The proposal would be to implement the digital signature verification first.
Quoted 6 lines
Quoted 6 lines
At a high level, during the build:
* A SHA-256 hash would be calculated over tbd files that make up the firmware image.
* The hash would be signed by a private key that's part of the repository (community key). Companies could overwrite it with their own private key when building production images.
Can we not store any key in the tree at all, please. If we do, somebody will inevitably use that key for production images and be totally insecure. Generate a key by default and print a big warning that it is being signed by a local (insecure) key and that they should set up keys according to some documentation.
Provide a script that can be used to (somewhat) securely store openssl keys (with a passphrase) and then request the passphrase for each build. This could also be used to securely store keys on an HSM that openssl has ENGINE support for.
Quoted 12 lines
Quoted 3 lines
* The encrypted hash (digital signature) and corresponding public key would be added to the firmware image.
* Yocto may already provide a way to sign images and generate keys.I'm starting to look into signing of artefacts as part of the build process. Some quick searching suggests Yocto doesn't have support for it out of the box, but there is the swupdate project[1] and the corresponding meta-swupdate[2] Yocto layer that sound interesting. However, it may require a rework of how we manage image updates if we were to make use of it. I'll keep poking around, but if anyone has any other pointers please let me know.
Quoted 2 lines
[1] https://github.com/sbabic/swupdate
[2] https://github.com/sbabic/meta-swupdate
Whatever mechanism is used to generate the signature should be able to be overridden by an environment variable or something. We already do something like this for our pre-openbmc generations where a common developer key is stored outside the tree, then the image gets re-signed with the new key to turn it into a production-signed image. This way, we can use different mechanisms for generating the development and production signatures (openssl and a remote signing service).
--Vernon
Quoted 16 lines
Quoted 10 lines
On the BMC:
* The hash would be calculated on the image files that were uploaded to the BMC.
* The signature would be decrypted using an existing public key on the BMC (this validates the new public key delivered with the image is also valid).
* Accept image if both values are the same.Any thoughts or comments?
Can you take a look at swupdate and reply with your thoughts?
Cheers,
Andrew
Hi Anoo and all,
Regarding "The encrypted hash (digital signature) and corresponding public key would be added to the firmware image":
If both the decrypted signature and the public key are in flash, if flash was replaced (by hacking or physicaly) a malicious public key can be used with a signature that was decrypted by the paired malicious private key of the stored public key.
To overcome that the public key should be stored in OTP. We use this method on our Nuvoton Poleg BMC.
Does this sound reasonable?
Regards,
Avi
Quoted 29 lines · anoo
Hi all,
During the hackaton meetup, we touched on BMC image verification and signing and concluded that the community would like to see two verification steps, one prior to writing the image to flash (via digital signature verification), and another one by checking FIT in U-Boot prior to booting from the new image.
The proposal would be to implement the digital signature verification first.
At a high level, during the build:
* A SHA-256 hash would be calculated over tbd files that make up the firmware image.
* The hash would be signed by a private key that's part of the repository (community key). Companies could overwrite it with their own private key when building production images.
* The encrypted hash (digital signature) and corresponding public key would be added to the firmware image.
* Yocto may already provide a way to sign images and generate keys.On the BMC:
* The hash would be calculated on the image files that were uploaded to the BMC.
* The signature would be decrypted using an existing public key on the BMC (this validates the new public key delivered with the image is also valid).
* Accept image if both values are the same.Any thoughts or comments?
Dell - Internal Use - Confidential
The "firmware update" public key in flash (and firmware update code itself) would be authenticated by BMC secure boot.
Also - has anybody mentioned using the FIT as a firmware update install image, in addition to using to boot the kernel? (i.e. FIT within a FIT). Think the native DFU command in u-boot supports this method. Just a thought
-----Original Message-----
From: openbmc [mailto:[email protected]] On Behalf Of Avi Fishman
Sent: Monday, January 29, 2018 4:45 AM
To: anoo <[email protected]>
Cc: [email protected]; OpenBMC Maillist <[email protected]>; [email protected]; [email protected]
Subject: Re: BMC Image Signing Proposal
Hi Anoo and all,
Regarding "The encrypted hash (digital signature) and corresponding public key would be added to the firmware image":
If both the decrypted signature and the public key are in flash, if flash was replaced (by hacking or physicaly) a malicious public key can be used with a signature that was decrypted by the paired malicious private key of the stored public key.
To overcome that the public key should be stored in OTP. We use this method on our Nuvoton Poleg BMC.
Does this sound reasonable?
Regards,
Avi
Quoted 29 lines · anoo
Hi all,
During the hackaton meetup, we touched on BMC image verification and signing and concluded that the community would like to see two verification steps, one prior to writing the image to flash (via digital signature verification), and another one by checking FIT in U-Boot prior to booting from the new image.
The proposal would be to implement the digital signature verification first.
At a high level, during the build:
* A SHA-256 hash would be calculated over tbd files that make up the firmware image.
* The hash would be signed by a private key that's part of the repository (community key). Companies could overwrite it with their own private key when building production images.
* The encrypted hash (digital signature) and corresponding public key would be added to the firmware image.
* Yocto may already provide a way to sign images and generate keys.On the BMC:
* The hash would be calculated on the image files that were uploaded to the BMC.
* The signature would be decrypted using an existing public key on the BMC (this validates the new public key delivered with the image is also valid).
* Accept image if both values are the same.Any thoughts or comments?