Skip to contentSkip to conversation

Deploying Yocto build images

DISCUSSION Stefano Babic 4 participants 7 messages Nov 21, 2013

lore
Share & tools
lore
#0 lore raw

Hi everybody,

in the last ELCE, David point out in his presentation that we should improve how to deploy Yocto images on the target.

I did some work this year to provide a reliable way for some customers of us to install Yocto's images in field, and I have published last week the sources. Here the link of the announcement:

	http://comments.gmane.org/gmane.comp.embedded.eldk/2397

Mainly, it is a tool that can be stored in the rootfs or will be put in a separate initrd image and whose goal is only to update the system. I have tried to describe pros and cons of several different solutions (updating via bootloader ? single copy against dual copy ?) - you can find details in the doc directory in the swupdate repository.

Maybe someone of you can find this helpful, and I will be happy if I could get some feedback.

Best regards,
Stefano Babic

Signature · 5 lines
=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: [email protected]
=====================================================================
#1 lore raw

Stefano,

This is a really great tool. I'm always developing a new software update tool for each new project, since there are different requirements. As I can see you are dealing with different scenarios, and this is really amazing.

I'm already cloning meta-swupdate to test it.
Looking at recipes I realized that swupdate is initialized in an sysVinit environment.
Did you plan to use systemd too?
Maybe I can contribute to this.

Thanks for sharing.

Regards,

--
*dS
Diego Sueiro

/*long live rock 'n roll*/

2013/11/21 Stefano Babic <[email protected]>

Quoted 34 lines

Hi everybody,

in the last ELCE, David point out in his presentation that we should improve how to deploy Yocto images on the target.

I did some work this year to provide a reliable way for some customers of us to install Yocto's images in field, and I have published last week the sources. Here the link of the announcement:

        http://comments.gmane.org/gmane.comp.embedded.eldk/2397

Mainly, it is a tool that can be stored in the rootfs or will be put in a separate initrd image and whose goal is only to update the system. I have tried to describe pros and cons of several different solutions (updating via bootloader ? single copy against dual copy ?) - you can find details in the doc directory in the swupdate repository.

Maybe someone of you can find this helpful, and I will be happy if I could get some feedback.

Best regards,
Stefano Babic

--
=====================================================================
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: [email protected]
=====================================================================
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

#2 lore raw

Hi Diego,

Quoted 4 lines · Diego Sueiro

Stefano,

This is a really great tool. I'm always developing a new software update tool for each new project, since there are different requirements.

That is right. I did the same in the past, until I recognize that I waste a lot of effort.

Quoted 3 lines

As I can see you are dealing with different scenarios, and this is really amazing.

Sure. I hope that the tool remains general enough to allow to include scenarios that I was not able to foresee.

Quoted 5 lines

I'm already cloning meta-swupdate to test it.
Looking at recipes I realized that swupdate is initialized in an sysVinit environment.
Did you plan to use systemd too?
Maybe I can contribute to this.

Let's say that how the tool is started is open and can be easy customized. Updating to systemd is surely nice.

One use of case I have is to not use neither sysVinit not systemd at all. The tool is put into a initrd and it is the only task running, that is I modified rcS to start only the tool and to speed up the booting process. But as I said, this is only a use case.

Best regards,
Stefano

Signature · 5 lines
=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: [email protected]
=====================================================================
#5 lore raw
In reply to Stefano Babic

Stefano,

Looks interesting. I have a question that I didn't see covered in the documentation (yet). I can see the benefit to having a single image firmware upgrade, but how does swupdate handle configuration differences? I can see cases in which the majority of the software will be the same but there will be configuration file differences from device to device. I like the idea of a single image upgrade, but not at the cost of wiping the device-specific configuration.

Thanks,
Bryan E.

Quoted 40 lines

-----Original Message-----
From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Stefano Babic
Sent: Thursday, November 21, 2013 7:11 AM
To: [email protected]
Subject: [yocto] Deploying Yocto build images

Hi everybody,

in the last ELCE, David point out in his presentation that we should improve how to deploy Yocto images on the target.

I did some work this year to provide a reliable way for some customers of us to install Yocto's images in field, and I have published last week the sources. Here the link of the announcement:

	http://comments.gmane.org/gmane.comp.embedded.eldk/2397

Mainly, it is a tool that can be stored in the rootfs or will be put in a separate initrd image and whose goal is only to update the system. I have tried to describe pros and cons of several different solutions (updating via bootloader ? single copy against dual copy ?) - you can find details in the doc directory in the swupdate repository.

Maybe someone of you can find this helpful, and I will be happy if I could get some feedback.

Best regards,
Stefano Babic

--
=====================================================================
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: [email protected]
=====================================================================
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

#6 lore raw

Hi Bryan,

Quoted 4 lines · Bryan Evenson

Stefano,

Looks interesting. I have a question that I didn't see covered in the documentation (yet).

I know, this is my fault. Writing documentation is a task that is always postponed ;-)

Quoted 6 lines

I can see the benefit to having a single image firmware upgrade, but how does swupdate handle configuration differences? I can see cases in which the majority of the software will be the same but there will be configuration file differences from device to device. I like the idea of a single image upgrade, but not at the cost of wiping the device-specific configuration.

I explain how it was done in a real project. The scope was to have a single image, containing update for multiple devices. Let's say this is an update of version Y for a whole family of products, but each product has its own parts (mostly kernel differs, but not only).

In that case, an external parser was used. The sw-description is written as XML, with a syntax like this:

<?xml version="1.0" encoding="UTF-8"?> <software version="1.0">
<name>Update Image</name> <version>1.0.0</version>
<description>Firmware for XXXXX Project</description>

<images>
<image device="firstdevice" version="0.9">

     <stream name="dev1-uImage" type="ubivol" volume="kernel" />
     <stream name="dev1.dtb" type="ubivol" volume="dtb" />
     <stream name="dev1-rootfs.ubifs" type="ubivol" volume="rootfs"/>
     <stream name="dev1-uboot-env" type="uboot" />
     <stream name="raw_vfat" type="raw" dest="/dev/mmcblk0p4" />
     <stream name="sdcard.lua" type="lua" />

</image>

<image device="seconddevice" version="0.9">

     <stream name="dev2.dtb" rev="0.9" type="dtb" />
     <stream name="dev2-uImage" rev="0.9" type="kernel" />

......

That is: there is a single description, that describes the components for all devices. Each device must recognized where it is running - this was done with a configuration file or reading directly from hardware (GPIOs, SPI EPROM).

When the image is downloaded, the description is parsed and only the significant parts are extracted - the rest is discarded.
Of course, the image can be significantly bigger - it contains the software for many devices. In my real project, this was not an issue - the scope was really to have a single image for a product family.

Note: this is only an example how the tool was used. But this shows also that the syntax of the sw-description is open. I could also add the LUA script/XML parser to the repository, it is at the moment not stored - I have thought at it as a custom specific requirement.

Best regards,
Stefano

Signature · 5 lines
=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: [email protected]
=====================================================================
#3 lore raw
In reply to Diego Sueiro

Thanks for posting this. It's very timely for a project I am in the initial stages of designing. I have looked over the docs and some of the code, and am interested in using this.

I would make a couple suggestions:

1) As you mention, documentation is important. Unless this is very well documented, it's difficult for other people to take up and use effectively, or to advocate for its use in a group

2) Progress indications: it's going to be fairly important to build in some way to notify other process about the state of the update. A flexible framework for sharing progress would be very much appreciated. There are a couple scenarios I have, but the main one is a network notification where one device is being updated by another. The updated device should be able to signal update progress to the updater device in a flexible fashion. I haven't looked at the code hard enough to see how difficult this would be to add.
--
Michael

From: [email protected] [mailto:[email protected]] On Behalf Of Diego Sueiro
Sent: Thursday, November 21, 2013 6:35 AM
To: Stefano Babic
Cc: [email protected]
Subject: Re: [yocto] Deploying Yocto build images

Stefano,

This is a really great tool. I'm always developing a new software update tool for each new project, since there are different requirements. As I can see you are dealing with different scenarios, and this is really amazing.

I'm already cloning meta-swupdate to test it.
Looking at recipes I realized that swupdate is initialized in an sysVinit environment. Did you plan to use systemd too?
Maybe I can contribute to this.

Thanks for sharing.

Regards,

--
*dS
Diego Sueiro

/*long live rock 'n roll*/

2013/11/21 Stefano Babic <[email protected]<mailto:[email protected]>> Hi everybody,

in the last ELCE, David point out in his presentation that we should improve how to deploy Yocto images on the target.

I did some work this year to provide a reliable way for some customers of us to install Yocto's images in field, and I have published last week the sources. Here the link of the announcement:

        http://comments.gmane.org/gmane.comp.embedded.eldk/2397

Mainly, it is a tool that can be stored in the rootfs or will be put in a separate initrd image and whose goal is only to update the system. I have tried to describe pros and cons of several different solutions (updating via bootloader ? single copy against dual copy ?) - you can find details in the doc directory in the swupdate repository.

Maybe someone of you can find this helpful, and I will be happy if I could get some feedback.

Best regards,
Stefano Babic

--
=====================================================================
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53<tel:%2B49-8142-66989-53> Fax: +49-8142-66989-80<tel:%2B49-8142-66989-80> Email: [email protected]<mailto:[email protected]>
=====================================================================
_______________________________________________
yocto mailing list
[email protected]<mailto:[email protected]>
https://lists.yoctoproject.org/listinfo/yocto

#4 lore raw

Hi Michael,

Quoted 8 lines · On 21/11/2013 19:20, [email protected] wrote:

Thanks for posting this. It’s very timely for a project I am in the initial stages of designing. I have looked over the docs and some of the code, and am interested in using this.

I would make a couple suggestions:

Thanks !

Quoted 3 lines

1) As you mention, documentation is important. Unless this is very
well documented, it’s difficult for other people to take up and use effectively, or to advocate for its use in a group

I know, and I am understand there are a lot of things that I have not explained. Documentation is first priority.

Quoted 4 lines

2) Progress indications: it’s going to be fairly important to build
in some way to notify other process about the state of the update. A
flexible framework for sharing progress would be very much appreciated.

There is an interface using Unix Domain Socket to communicate with the installer. There is only a couple of implemented messages (start installer - get status). It is quite rudimentary, but it can be extended.

This interface is also used by the AJAX interface (see in ./www) to report the current status to the operator's browser when a downloaded is started from network. Another use case is to output the status on a LCD:
a separate process can ask the installer and render the output for the operator.

Quoted 5 lines

There are a couple scenarios I have, but the main one is a network notification where one device is being updated by another. The updated device should be able to signal update progress to the updater device in a flexible fashion. I haven’t looked at the code hard enough to see how difficult this would be to add.

I think it is should be not very difficult. As I said, there is already an interface that can be easy extended.

Regards,
Stefano

Signature · 5 lines
=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: [email protected]
=====================================================================