Packaging and deploying multiple firmware image types in one
There was some discussion in the "PSU firmware update" proposal[1] about bundling firmware images together such as the BMC and PSU firmware. The Software interface defines these bundles as a "System" version[2].
At IBM, we're starting to create a "System" bundle of BMC + host firmware by adding a recipe that would add the host's firmware files to the BMC's squashfs[3].
One of the pending items is determining how to showcase the version of these System images in Redfish. We can assume that the version of a System image corresponds to the BMC's "FirmwareVersion", but it's unknown what else the bundle contains (host fw, PSU fw, PCIe fw,..), in order to populate the other version fields such as "BiosVersion". There's a D-Bus "Extended Version" property[4] that could be leveraged to add the additional image information.
Feel free to share if you have any thoughts on this item or any aspect of these "System" images.
---
[1] https://lists.ozlabs.org/pipermail/openbmc/2019-June/016573.html
[2] https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/xyz/openbmc_project/Software/Version.interface.yaml
[3] https://github.com/openbmc/meta-openpower/blob/master/recipes-phosphor/flash/host-fw_git.bb
[4] https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/xyz/openbmc_project/Software/ExtendedVersion.interface.yaml
Quoted 16 lines · Adriana Kobylak
There was some discussion in the "PSU firmware update" proposal[1] about bundling firmware images together such as the BMC and PSU firmware. The Software interface defines these bundles as a "System" version[2].
At IBM, we're starting to create a "System" bundle of BMC + host firmware by adding a recipe that would add the host's firmware files to the BMC's squashfs[3].
One of the pending items is determining how to showcase the version of these System images in Redfish. We can assume that the version of a System image corresponds to the BMC's "FirmwareVersion", but it's unknown what else the bundle contains (host fw, PSU fw, PCIe fw,..), in order to populate the other version fields such as "BiosVersion". There's a D-Bus "Extended Version" property[4] that could be leveraged to add the additional image information.
Quoted 3 lines
Feel free to share if you have any thoughts on this item or any aspect of these "System" images.
We use the system bundle of BMC + Host firmware on our VESNIN hardware. There are some things which cause discomfort a little bit:
- The uploaded system bundle isn't shown in the WebUI.
- The system bundle has only one version field which is common for BMC and Host firmwares.
- After rebooting BMC, which is required to complete update the BMC firmware the system bundle turns to two separated instances in D-Bus which has its own real versions.
Thus, I thought about putting the separate manifests for each part of the bundle.
Quoted 6 lines
---
[1] https://lists.ozlabs.org/pipermail/openbmc/2019-June/016573.html
[2] https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/xyz/openbmc_project/Software/Version.interface.yaml
[3] https://github.com/openbmc/meta-openpower/blob/master/recipes-phosphor/flash/host-fw_git.bb
[4] https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/xyz/openbmc_project/Software/ExtendedVersion.interface.yaml
Quoted 8 lines · Alexander A. Filippov
On Mon, Nov 11, 2019 at 01:28:11PM -0600, Adriana Kobylak wrote:
At IBM, we're starting to create a "System" bundle of BMC + host firmware by adding a recipe that would add the host's firmware files to the BMC's squashfs[3].
Quoted 6 lines
Feel free to share if you have any thoughts on this item or any aspect of these "System" images.
We use the system bundle of BMC + Host firmware on our VESNIN hardware. There are some things which cause discomfort a little bit:
- The uploaded system bundle isn't shown in the WebUI.
IBM is planning on supporting a bundled "System" firmware form the WebUI. The design is here:
https://ibm.invisionapp.com/share/4XNZ0JAMJ7B#/screens/319215228
Quoted 2 lines
Quoted 11 lines · Gunnar Mills
Quoted 5 lines · Alexander A. Filippov
On Mon, Nov 11, 2019 at 01:28:11PM -0600, Adriana Kobylak wrote:
We use the system bundle of BMC + Host firmware on our VESNIN hardware. There are some things which cause discomfort a little bit:
- The uploaded system bundle isn't shown in the WebUI.IBM is planning on supporting a bundled "System" firmware form the WebUI. The design is here:
https://ibm.invisionapp.com/share/4XNZ0JAMJ7B#/screens/319215228
Maybe in the mean time we can have the WebUI display the "System" versions alongside the "BMC" ones. I'll check if it's as simple as "if purpose is BMC or System" do what it currently does.
Quoted 2 lines
Quoted 2 lines
Quoted 3 lines · Alexander A. Filippov
On Mon, Nov 11, 2019 at 01:28:11PM -0600, Adriana Kobylak wrote:
We use the system bundle of BMC + Host firmware on our VESNIN hardware.
How are you currently building the bundled image, do you include the Host firmware in the BMC rootfs, or do you have separate image files (bmc fw file, host fw file) in a single tarball?
Quoted 5 lines
There are some things which cause discomfort a little bit:
- The uploaded system bundle isn't shown in the WebUI.
- The system bundle has only one version field which is common for BMC and Host
firmwares.
Do you think the ExtendedVersion d-bus property could help in this case? For example adding to the manifest "extended_version=host-v1.2."
Quoted 5 lines
- After rebooting BMC, which is required to complete update the BMC firmware
the system bundle turns to two separated instances in D-Bus which has
its own
real versions.
Yeah, the purpose is not currently preserved across reboots. I have a change here for that: https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-bmc-code-mgmt/+/27045
Quoted 3 lines
Thus, I thought about putting the separate manifests for each part of the bundle.
If you go the route of adding a second manifest, would you have them in a separate tarball (bmc image + manifest) and (host image + manifest) then put those tarballs inside a tarball? since the manifest file name would be the same.
Quoted 7 lines
Quoted 7 lines
---
[1] https://lists.ozlabs.org/pipermail/openbmc/2019-June/016573.html
[2] https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/xyz/openbmc_project/Software/Version.interface.yaml
[3] https://github.com/openbmc/meta-openpower/blob/master/recipes-phosphor/flash/host-fw_git.bb
[4] https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/xyz/openbmc_project/Software/ExtendedVersion.interface.yaml
Quoted 8 lines · Adriana Kobylak
Quoted 3 lines · Alexander A. Filippov
On Mon, Nov 11, 2019 at 01:28:11PM -0600, Adriana Kobylak wrote:
We use the system bundle of BMC + Host firmware on our VESNIN hardware.
How are you currently building the bundled image, do you include the Host firmware in the BMC rootfs, or do you have separate image files (bmc fw file, host fw file) in a single tarball?
We put separate image files in a single tarball.
Quoted 9 lines
Quoted 6 lines
There are some things which cause discomfort a little bit:
- The uploaded system bundle isn't shown in the WebUI.
- The system bundle has only one version field which is common for BMC and Host
firmwares.Do you think the ExtendedVersion d-bus property could help in this case? For example adding to the manifest "extended_version=host-v1.2."
I think, it is right only if the system bundle will be processed by its own service, which creates corresponding objects for each part of the bundle with their own hash, version's data and purpose.
E.g.: as you've proposed below with separate tarballs in one common tarball.
Quoted 11 lines
Quoted 6 lines
- After rebooting BMC, which is required to complete update the BMC firmware
the system bundle turns to two separated instances in D-Bus which has
its own
real versions.Yeah, the purpose is not currently preserved across reboots. I have a change here for that:
https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-bmc-code-mgmt/+/27045
We use static filesystem layout on our VESNIN hardware.
So, the version of active BMC is read from /etc/os-release and always has the BMC purpose.
The same way with the host firmware: it is read directly from the flash, which has a special partition with version's data and always has the Host purpose.
Quoted 10 lines
Quoted 4 lines
Thus, I thought about putting the separate manifests for each part of the
bundle.If you go the route of adding a second manifest, would you have them in a separate tarball (bmc image + manifest) and (host image + manifest) then put those tarballs inside a tarball? since the manifest file name would be the same.
Yeah, it's possible.
But when I researched that, I had found that the phosphor-image-updater and the openpower-update-manager both already have support for the system bundle. I decided, we shouldn't create any other implementation, but follow the upstream's way. So, it has led to the state I described earlier.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
---
[1] https://lists.ozlabs.org/pipermail/openbmc/2019-June/016573.html
[2] https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/xyz/openbmc_project/Software/Version.interface.yaml
[3] https://github.com/openbmc/meta-openpower/blob/master/recipes-phosphor/flash/host-fw_git.bb
[4] https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/xyz/openbmc_project/Software/ExtendedVersion.interface.yaml
Quoted 5 lines · Adriana Kobylak
Yeah, the purpose is not currently preserved across reboots. I have a change here for that:
https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-bmc-code-mgmt/+/27045
Quoted 4 lines · Adriana Kobylak (Code Review)
...
but let's continue on the mailing list about your thoughts on how you think the tarball of tarballs should be handled.
Ok, here are my thoughts:
The phosphor-version-software-manager might put all internal tarballs in the /tmp/images folder during processing the top level tarball. That will lead to creation of corresponding D-Bus objects. Each of them will have their own purpose, version, object path and so on.
The root D-Bus object and their folder might be removed after that to reduce a used file system space.
Quoted 13 lines · Alexander A. Filippov
Quoted 5 lines · Adriana Kobylak
Yeah, the purpose is not currently preserved across reboots. I have a change here for that:
https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-bmc-code-mgmt/+/27045Quoted 4 lines · Adriana Kobylak (Code Review)
...
but let's continue on the mailing list about your thoughts on how you think the tarball of tarballs should be handled.
Quoted 5 lines
Ok, here are my thoughts:
The phosphor-version-software-manager might put all internal tarballs in the/tmp/imagesfolder during processing the top level tarball. That will lead to creation of corresponding D-Bus objects. Each of them will have their own purpose, version, object path and so on.
Yeah agree. We may still need some way to let the phosphor-version-software-manager know that it needs to untar the internal tarballs, maybe a very simple MANIFEST with a new field, then each individual tarball would have their own MANIFEST that creates the D-Bus versions like you mentioned.
Quoted 3 lines
The root D-Bus object and their folder might be removed after that to reduce a used file system space.
Quoted 6 lines
Quoted 5 lines
Ok, here are my thoughts:
The phosphor-version-software-manager might put all internal tarballs in the/tmp/imagesfolder during processing the top level tarball. That will lead to creation of corresponding D-Bus objects. Each of them will have their own purpose, version, object path and so on.
Quoted 7 lines
Yeah agree. We may still need some way to let the phosphor-version-software-manager
know that it needs to untar the internal tarballs, maybe a very simple MANIFEST with a new field, then each individual tarball would have their own MANIFEST that
creates the D-Bus versions like you mentioned.
For the bmcweb change to support the System purpose (https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/27144), Andrew had the comment about if we could have the software manager determine the .BMC, .Host, .PSU details out of a System image and populate the individual version interfaces, which further confirms that the current Version D-Bus interface is insufficient to detail the individual components of a "System" firmware image, and that workarounds like adding individual Manifest files that Alexander proposed are needed.
Therefore, wanted to see what people thought about expanding the software D-Bus interface to add a 2nd version id to the path of a System version, which would contain the version information for each sub-element, but would not contain an Activation object:
--
Current, leave as is for individual firmware images:
/xyz/openbmc_project/software/<bmc-version-id>/
- Activation: Ready
- Purpose: .BMC
- Version: 2.8.0
/xyz/openbmc_project/software/<host-version-id>/
- Activation: Ready
- Purpose: .Host
- Version: 1.2.3
--
New expanded option for bundled images:
/xyz/openbmc_project/software/<system-version-id>/
- Activation: Ready
- Purpose: .System
- Version: CompanyZ-v1.0 /xyz/openbmc_project/software/<sysrem-version-id>/<bmc-version-id>/
- Purpose: .BMC
- Version: 2.8.0
/xyz/openbmc_project/software/<sysrem-version-id>/<host-version-id>/
- Purpose: .Host
- Version: 1.2.3
Quoted 31 lines · Adriana Kobylak
For the bmcweb change to support the System purpose
(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/27144), Andrew had the comment about if we could have the software manager determine the .BMC, .Host, .PSU details out of a System image and populate the individual version interfaces, which further confirms that the current Version D-Bus interface is insufficient to detail the individual components of a "System" firmware image, and that workarounds like adding individual Manifest files that Alexander proposed are needed.Therefore, wanted to see what people thought about expanding the software D-Bus interface to add a 2nd version id to the path of a System version, which would contain the version information for each sub-element, but would not contain an Activation object:
--
Current, leave as is for individual firmware images:/xyz/openbmc_project/software/<bmc-version-id>/
- Activation: Ready
- Purpose: .BMC
- Version: 2.8.0/xyz/openbmc_project/software/<host-version-id>/
- Activation: Ready
- Purpose: .Host
- Version: 1.2.3--
New expanded option for bundled images:
Quoted 10 lines
/xyz/openbmc_project/software/<system-version-id>/
- Activation: Ready
- Purpose: .System
- Version: CompanyZ-v1.0 /xyz/openbmc_project/software/<sysrem-version-id>/<bmc-version-id>/
- Purpose: .BMC
- Version: 2.8.0
/xyz/openbmc_project/software/<sysrem-version-id>/<host-version-id>/
- Purpose: .Host
- Version: 1.2.3
Makes sense to me. The phosphor-webui code could look for a system object and if found, only allow the user to update/switch images based on system level. They could still easily show the BMC, host, PSU levels though. Or maybe this will just be a config option for the GUI? Either way I think the GUI would want to show both the system and individual levels of firmware.
Redfish code in bmcweb can continue to work as-is for bmc, host, psu firmware inventory. I think we should see if we can get the DMTF to add a FirmwareVersion property to the ComputerSystem object where we could store this new system version.
Quoted 12 lines · Andrew Geissler
Quoted 10 lines · Adriana Kobylak
New expanded option for bundled images:
/xyz/openbmc_project/software/<system-version-id>/
- Activation: Ready
- Purpose: .System
- Version: CompanyZ-v1.0 /xyz/openbmc_project/software/<sysrem-version-id>/<bmc-version-id>/
- Purpose: .BMC
- Version: 2.8.0
Quoted 3 lines
Quoted 3 lines
/xyz/openbmc_project/software/<sysrem-version-id>/<host-version-id>/
- Purpose: .Host
- Version: 1.2.3
Ended up adding an array instead for the proposal, to avoid confusion with multiple version ids:
https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-dbus-interfaces/+/27395
"Purpose":
"xyz.openbmc_project.Software.Version.VersionPurpose.System",
"Version": "2.8.0-dev-882-gf0cbf42",
"VersionAggregate": [
[
"xyz.openbmc_project.Software.Version.VersionPurpose.BMC",
"2.8.0-dev",
""
],
[
"xyz.openbmc_project.Software.Version.VersionPurpose.Host",
"open-power-witherspoon-v2.2","buildroot-2018.11.1-7-g5d7cc8c,skiboot-v6.2,hostboot-17ba81e-pb264a73,occ-12c8088"
]
]Quoted 7 lines
Makes sense to me. The phosphor-webui code could look for a system object and if found, only allow the user to update/switch images based on system level. They could still easily show the BMC, host, PSU levels though. Or maybe this will just be a config option for the GUI? Either way I think the GUI would want to show both the system and individual levels of firmware.
Yeah, it'd make sense to also show the individual levels to provide the user with more information about the image.
Quoted 3 lines
Redfish code in bmcweb can continue to work as-is for bmc, host, psu firmware inventory.
Yeah, I'd just need to make a change in bmcweb to get the version string from that new Aggregate interface to populate the BMC FirmwareVersion field.
Quoted 3 lines
I think we should see if we can get the DMTF to
add a FirmwareVersion property to the ComputerSystem object where we could store this new system version.
Agree.