Skip to contentSkip to conversation

Yocto and Google protobuffer

DISCUSSION Pietro 6 participants 21 messages Aug 31, 2016 – Sep 8, 2016

lore
Share & tools
lore

Pietro

author
#0 lore raw

Hi all,

I am new to the Yocto building system and I could be talking nonsense. I used to work with buildroot time ago and I remember there is an area where compiled software/packages/tools previously built are "staged" and used when building other packages.

Is there something like that available with Yocto ? Specifically I would like to add a package which uses the Google Protocol Buffer, I do not have administrator rights on the machine and I can't install the packages I need system wise.

Is it possible to add recipes and use them at building time without including them in the image being generated ?

A good example for that would be the protoc, the protocol buffer description file compiler.

Thanks in advance,
Pietro.

#1 lore raw
Quoted 12 lines · Pietro

Hi all,

I am new to the Yocto building system and I could be talking nonsense. I used to work with buildroot time ago and I remember there is an area where compiled software/packages/tools previously built are "staged" and used when building other packages.

Is there something like that available with Yocto ? Specifically I would like to add a package which uses the Google Protocol Buffer, I do not have administrator rights on the machine and I can't install the packages I need system wise.

Quoted 6 lines

Is it possible to add recipes and use them at building time without including them in the image being generated ?

A good example for that would be the protoc, the protocol buffer description file compiler.

There is a recipe for protobuf in meta-oe (2.6.1). All you need to do, is include meta-oe in your layers (bblayers.conf) and have protobuf-native listed in DEPENDS inside your package recipe.

The protoc compiler will be available in the PATH when package is being built, so autotools checks like AC_CHECK_PROG/AC_PATH_PROG should be able to find it.

Cheers,

Signature · 2 lines
Maciej Borzecki
RnDity
author
#2 lore raw

Maciej Borzęcki

Quoted 24 lines · <[email protected]> writes:
Quoted 12 lines · Pietro

Hi all,

I am new to the Yocto building system and I could be talking nonsense. I used to work with buildroot time ago and I remember there is an area where compiled software/packages/tools previously built are "staged" and used when building other packages.

Is there something like that available with Yocto ? Specifically I would like to add a package which uses the Google Protocol Buffer, I do not have administrator rights on the machine and I can't install the packages I need system wise.

Quoted 6 lines

Is it possible to add recipes and use them at building time without including them in the image being generated ?

A good example for that would be the protoc, the protocol buffer description file compiler.

There is a recipe for protobuf in meta-oe (2.6.1). All you need to do, is include meta-oe in your layers (bblayers.conf) and have protobuf-native listed in DEPENDS inside your package recipe.

Quoted 8 lines

The protoc compiler will be available in the PATH when package is being built, so autotools checks like AC_CHECK_PROG/AC_PATH_PROG should be able to find it.

Cheers,

Signature · 2 lines
Maciej Borzecki
RnDity

Thanks a lot.

I have added the DEPENDS line and it indeed downloads and build the some protobuf related stuff, where did you get the dependency name from ? I can't see the protobuf-native as a recipe in the meta-oe website :

https://layers.openembedded.org/layerindex/branch/master/layer/meta-oe/

I have just realised that GRPC (http://www.grpc.io/) is needed for my project, it is a library which uses the protobuffers, is there a recipe/package which provides them around or do I need to create my own recipe ?

I would like to check the root filesystem being generated as part of the build process, where is it ?

#3 lore raw
Quoted 35 lines · Pietro

Maciej Borzęcki

Quoted 24 lines · <[email protected]> writes:
Quoted 12 lines · Pietro

Hi all,

I am new to the Yocto building system and I could be talking nonsense. I used to work with buildroot time ago and I remember there is an area where compiled software/packages/tools previously built are "staged" and used when building other packages.

Is there something like that available with Yocto ? Specifically I would like to add a package which uses the Google Protocol Buffer, I do not have administrator rights on the machine and I can't install the packages I need system wise.

Quoted 6 lines

Is it possible to add recipes and use them at building time without including them in the image being generated ?

A good example for that would be the protoc, the protocol buffer description file compiler.

There is a recipe for protobuf in meta-oe (2.6.1). All you need to do, is include meta-oe in your layers (bblayers.conf) and have protobuf-native listed in DEPENDS inside your package recipe.

Quoted 8 lines

The protoc compiler will be available in the PATH when package is being built, so autotools checks like AC_CHECK_PROG/AC_PATH_PROG should be able to find it.

Cheers,
--
Maciej Borzecki
RnDity

Quoted 7 lines

Thanks a lot.

I have added the DEPENDS line and it indeed downloads and build the some protobuf related stuff, where did you get the dependency name from ? I can't see the protobuf-native as a recipe in the meta-oe website :

https://layers.openembedded.org/layerindex/branch/master/layer/meta-oe/

It's just a mechanism that allows for building the native (i.e. for the host) packages, and these are autmatically named ${PN}-native. There should a BBCLASSEXTEND = "..native.." stanza inside the protobuf recipe that enables this feature.

Quoted 5 lines

I have just realised that GRPC (http://www.grpc.io/) is needed for my project, it is a library which uses the protobuffers, is there a recipe/package which provides them around or do I need to create my own recipe ?

You can try searching for a recipe at http://layers.openembedded.org
If there's none, try to write your own. This might be a good changes to get yourself acquainted with devtool tool that will help you in the process.

Quoted 4 lines

I would like to check the root filesystem being generated as part of the build process, where is it ?

${TMPDIR}/<machine>_<distro?>/<imagename>/../rootfs

for instance, for my current build:

tmp/work/vexpress_qemu-poky-linux-gnueabi/core-image-minimal/1.0-r0/rootfs

Signature · 2 lines
Maciej Borzecki
RnDity
author
#4 lore raw

Maciej Borzęcki

Quoted 65 lines · <[email protected]> writes:
Quoted 35 lines · Pietro

Maciej Borzęcki

Quoted 24 lines · <[email protected]> writes:
Quoted 12 lines · Pietro

Hi all,

I am new to the Yocto building system and I could be talking nonsense. I used to work with buildroot time ago and I remember there is an area where compiled software/packages/tools previously built are "staged" and used when building other packages.

Is there something like that available with Yocto ? Specifically I would like to add a package which uses the Google Protocol Buffer, I do not have administrator rights on the machine and I can't install the packages I need system wise.

Quoted 6 lines

Is it possible to add recipes and use them at building time without including them in the image being generated ?

A good example for that would be the protoc, the protocol buffer description file compiler.

There is a recipe for protobuf in meta-oe (2.6.1). All you need to do, is include meta-oe in your layers (bblayers.conf) and have protobuf-native listed in DEPENDS inside your package recipe.

Quoted 8 lines

The protoc compiler will be available in the PATH when package is being built, so autotools checks like AC_CHECK_PROG/AC_PATH_PROG should be able to find it.

Cheers,
--
Maciej Borzecki
RnDity

Quoted 7 lines

Thanks a lot.

I have added the DEPENDS line and it indeed downloads and build the some protobuf related stuff, where did you get the dependency name from ? I can't see the protobuf-native as a recipe in the meta-oe website :

https://layers.openembedded.org/layerindex/branch/master/layer/meta-oe/

It's just a mechanism that allows for building the native (i.e. for the host) packages, and these are autmatically named ${PN}-native. There should a BBCLASSEXTEND = "..native.." stanza inside the protobuf recipe that enables this feature.

Quoted 5 lines

I have just realised that GRPC (http://www.grpc.io/) is needed for my project, it is a library which uses the protobuffers, is there a recipe/package which provides them around or do I need to create my own recipe ?

You can try searching for a recipe at http://layers.openembedded.org
If there's none, try to write your own. This might be a good changes to get yourself acquainted with devtool tool that will help you in the process.

Quoted 4 lines

I would like to check the root filesystem being generated as part of the build process, where is it ?

Quoted 9 lines

${TMPDIR}/<machine>_<distro?>/<imagename>/../rootfs

for instance, for my current build:

tmp/work/vexpress_qemu-poky-linux-gnueabi/core-image-minimal/1.0-r0/rootfs

Signature · 2 lines
Maciej Borzecki
RnDity

Thanks, much appreciated.

Do you know where the software which is not included in the image - such us protoc - is it stored ?

It turns out the recipe for the protobuffer uses version 2.6 while I would need version 3.0.0 to be used, is there a way to partially "override" a recipe ?

http://cgit.openembedded.org/cgit.cgi/meta-openembedded/tree/meta-oe/recipes-devtools/protobuf/protobuf_2.6.1.bb?h=master

Thanks a lot.
P.

#5 lore raw

Hi Pietro,

You can override the recipe by adding a recipe for version 3.0.0+ in your own layer and making sure your layer has a higher priority in bblayers.conf. See for example what we do in our project:

https://github.com/webOS-ports/webos-ports-setup/blob/testing/conf/bblayers.conf

openembedded-core provides ofono 1.1.7 for example with https://github.com/openembedded/openembedded-core/tree/krogoth/meta/recipes-connectivity/ofono

However we want to use ANOTHER version of ofono (1.1.7 based, but from a different repo/project).

So we have our own .bbappend at https://github.com/webOS-ports/meta-webos-ports/blob/krogoth/meta-luneos/recipes-connectivity/ofono/ofono_git.bbappend where we specify the different repo etc to use.

This doesn't apply 1:1 in your case, but you could simply add a protobuf_3.0.0.bb in your own layer and it should build that instead. Just make sure you have your layer at a higher position compared to meta-openembedded in your bblayers.conf

Herrie

Quoted 91 lines · Pietro

Maciej Borzęcki

Quoted 67 lines · <[email protected]> writes:
Quoted 35 lines · Pietro

Maciej Borzęcki

Quoted 24 lines · <[email protected]> writes:
Quoted 12 lines · Pietro

Hi all,

I am new to the Yocto building system and I could be talking nonsense. I used to work with buildroot time ago and I remember there is an area where compiled software/packages/tools previously built are "staged" and used when building other packages.

Is there something like that available with Yocto ? Specifically I would like to add a package which uses the Google Protocol Buffer, I do not have administrator rights on the machine and I can't install the packages I need system wise.

Quoted 6 lines

Is it possible to add recipes and use them at building time without including them in the image being generated ?

A good example for that would be the protoc, the protocol buffer description file compiler.

There is a recipe for protobuf in meta-oe (2.6.1). All you need to do, is include meta-oe in your layers (bblayers.conf) and have protobuf-native listed in DEPENDS inside your package recipe.

Quoted 8 lines

The protoc compiler will be available in the PATH when package is being built, so autotools checks like AC_CHECK_PROG/AC_PATH_PROG should be able to find it.

Cheers,
--
Maciej Borzecki
RnDity

Quoted 7 lines

Thanks a lot.

I have added the DEPENDS line and it indeed downloads and build the some protobuf related stuff, where did you get the dependency name from ? I can't see the protobuf-native as a recipe in the meta-oe website :

https://layers.openembedded.org/layerindex/branch/master/layer/meta-oe/

It's just a mechanism that allows for building the native (i.e. for the host) packages, and these are autmatically named ${PN}-native. There should a BBCLASSEXTEND = "..native.." stanza inside the protobuf recipe that enables this feature.

Quoted 5 lines

I have just realised that GRPC (http://www.grpc.io/) is needed for my project, it is a library which uses the protobuffers, is there a recipe/package which provides them around or do I need to create my own recipe ?

You can try searching for a recipe at http://layers.openembedded.org
If there's none, try to write your own. This might be a good changes to get yourself acquainted with devtool tool that will help you in the process.

Quoted 4 lines

I would like to check the root filesystem being generated as part of the build process, where is it ?

${TMPDIR}/<machine>_<distro?>/<imagename>/../rootfs

Quoted 7 lines

for instance, for my current build:

tmp/work/vexpress_qemu-poky-linux-gnueabi/core-image-minimal/1.0-r0/rootfs

--
Maciej Borzecki
RnDity

Thanks, much appreciated.

Do you know where the software which is not included in the image - such us protoc - is it stored ?

It turns out the recipe for the protobuffer uses version 2.6 while I would need version 3.0.0 to be used, is there a way to partially "override" a recipe ?

http://cgit.openembedded.org/cgit.cgi/meta-openembedded/tree/meta-oe/recipes-devtools/protobuf/protobuf_2.6.1.bb?h=master

Thanks a lot.
P.

#6 lore raw
Quoted 21 lines · Herman van Hazendonk

Hi Pietro,

You can override the recipe by adding a recipe for version 3.0.0+ in your own layer and making sure your layer has a higher priority in bblayers.conf. See for example what we do in our project:

https://github.com/webOS-ports/webos-ports-setup/blob/testin
g/conf/bblayers.conf

openembedded-core provides ofono 1.1.7 for example with
https://github.com/openembedded/openembedded-core/tree/
krogoth/meta/recipes-connectivity/ofono

However we want to use ANOTHER version of ofono (1.1.7 based, but from a different repo/project).

So we have our own .bbappend at https://github.com/webOS-ports
/meta-webos-ports/blob/krogoth/meta-luneos/recipes-connectiv ity/ofono/ofono_git.bbappend where we specify the different repo etc to use.

Quoted 5 lines

This doesn't apply 1:1 in your case, but you could simply add a protobuf_3.0.0.bb in your own layer and it should build that instead. Just make sure you have your layer at a higher position compared to meta-openembedded in your bblayers.conf

In the normal case (a simple upgrade to the newest version) the best choice would be to send a upgrade patch to openembedded-devel list: that way you never have to maintain it in your own layer.

Jussi

#7 lore raw

2016-09-01 12:34 GMT+02:00 Jussi Kukkonen <[email protected]>:

Quoted 27 lines · Herman van Hazendonk
Quoted 16 lines

Hi Pietro,

You can override the recipe by adding a recipe for version 3.0.0+ in your own layer and making sure your layer has a higher priority in bblayers.conf. See for example what we do in our project:

https://github.com/webOS-ports/webos-ports-setup/blob/testing/conf/bblayers.conf

openembedded-core provides ofono 1.1.7 for example with
https://github.com/openembedded/openembedded-core/tree/krogoth/meta/recipes-connectivity/ofono

However we want to use ANOTHER version of ofono (1.1.7 based, but from a different repo/project).

Quoted 8 lines

So we have our own .bbappend at
https://github.com/webOS-ports/meta-webos-ports/blob/krogoth/meta-luneos/recipes-connectivity/ofono/ofono_git.bbappend
where we specify the different repo etc to use.

This doesn't apply 1:1 in your case, but you could simply add a protobuf_3.0.0.bb in your own layer and it should build that instead. Just make sure you have your layer at a higher position compared to meta-openembedded in your bblayers.conf

Quoted 4 lines

In the normal case (a simple upgrade to the newest version) the best choice would be to send a upgrade patch to openembedded-devel list: that way you never have to maintain it in your own layer.

Hi,

protobuf 2.x and 3.x are incompatible, there is also a protobuf3 recipe in meta-maker.

Signature · 2 lines
Regards
Samuel
#8 lore raw
Quoted 36 lines · Samuel Stirtzel

2016-09-01 12:34 GMT+02:00 Jussi Kukkonen <[email protected]>:

Quoted 27 lines · Herman van Hazendonk
Quoted 16 lines

Hi Pietro,

You can override the recipe by adding a recipe for version 3.0.0+ in your own layer and making sure your layer has a higher priority in bblayers.conf. See for example what we do in our project:

https://github.com/webOS-ports/webos-ports-setup/blob/testing/conf/bblayers.conf

openembedded-core provides ofono 1.1.7 for example with
https://github.com/openembedded/openembedded-core/tree/krogoth/meta/recipes-connectivity/ofono

However we want to use ANOTHER version of ofono (1.1.7 based, but from a different repo/project).

Quoted 8 lines

So we have our own .bbappend at
https://github.com/webOS-ports/meta-webos-ports/blob/krogoth/meta-luneos/recipes-connectivity/ofono/ofono_git.bbappend
where we specify the different repo etc to use.

This doesn't apply 1:1 in your case, but you could simply add a protobuf_3.0.0.bb in your own layer and it should build that instead. Just make sure you have your layer at a higher position compared to meta-openembedded in your bblayers.conf

Quoted 4 lines

In the normal case (a simple upgrade to the newest version) the best choice would be to send a upgrade patch to openembedded-devel list: that way you never have to maintain it in your own layer.

Quoted 5 lines

Hi,

protobuf 2.x and 3.x are incompatible, there is also a protobuf3 recipe in meta-maker.

why is it in meta-maker and not in meta-oe ?

Quoted 9 lines

--
Regards
Samuel
--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

#9 lore raw

2016-09-06 20:11 GMT+02:00 Khem Raj <[email protected]>:

Quoted 11 lines
Quoted 8 lines · Samuel Stirtzel

Hi,

protobuf 2.x and 3.x are incompatible, there is also a protobuf3 recipe in meta-maker.

why is it in meta-maker and not in meta-oe ?

No idea,
it would be easier to ask the author (Koen).

Signature · 2 lines
Regards
Samuel
author
#10 lore raw
In reply to Jussi Kukkonen

Jussi Kukkonen <[email protected]>

Quoted 24 lines · writes:

On 1 September 2016 at 13:21, Herman van Hazendonk <[email protected]> wrote:

Hi Pietro,

You can override the recipe by adding a recipe for version 3.0.0+ in your own layer and making sure your layer has a higher priority in bblayers.conf. See for example what we do in our project:

https://github.com/webOS-ports/webos-ports-setup/blob/testing/conf/bblayers.conf

openembedded-core provides ofono 1.1.7 for example with
https://github.com/openembedded/openembedded-core/tree/krogoth/meta/recipes-
connectivity/ofono

However we want to use ANOTHER version of ofono (1.1.7 based, but from a different repo/project).

So we have our own .bbappend at
https://github.com/webOS-ports/meta-webos-ports/blob/krogoth/meta-luneos/recipes-connectiv
ity/ofono/ofono_git.bbappend where we specify the different repo
etc to use.

Quoted 4 lines

This doesn't apply 1:1 in your case, but you could simply add a protobuf_3.0.0.bb in your own layer and it should build that instead. Just make sure you have your layer at a higher position compared to meta-openembedded in your bblayers.conf

Thanks a lot.
I have written my own repice and added it into my own layer, it does not compile though :

|
| autoreconf: configure.ac: tracing
|
| autoreconf: configure.ac: subdirectory gmock not present
| autoreconf: configure.ac: not using Libtool
| autoreconf: running:
| /export/arm/pietro/PD15.1/build/tmp-glibc/sysroots/x86_64-linux/usr/bin/autoconf
| --include=/export/arm/pietro/PD15.1/build/tmp-glibc/work/cortexa8t2hf-vfp-neon-phytec-linux-gnueabi/proto
buf/3.0.0-r0/git/m4/ --force
|
| configure.ac:93: error: possibly undefined macro: AC_PROG_LIBTOOL
|
|       If this token and others are legitimate, please use
|       m4_pattern_allow.
|       See the Autoconf documentation.
|
| autoreconf:
| /export/arm/pietro/PD15.1/build/tmp-glibc/sysroots/x86_64-linux/usr/bin/autoconf
| failed with exit status: 1
|
| + bbfatal autoreconf execution failed.

I understand this is a completely different matter now, but has anybody else seen this before ? I have tried to compile the same revision on my local machine "natively" and it's built fine.

That library should be a dependency for another package/recipe I am working on, is it allowed to specify a version inside the DEPENDS recipe's clause ? I have tried to google the problem but I haven't found a working example as yet.

Cheers,
P.

author
#11 lore raw
Quoted 62 lines · Pietro

Jussi Kukkonen <[email protected]>

Quoted 24 lines · writes:

On 1 September 2016 at 13:21, Herman van Hazendonk <[email protected]> wrote:

Hi Pietro,

You can override the recipe by adding a recipe for version 3.0.0+ in your own layer and making sure your layer has a higher priority in bblayers.conf. See for example what we do in our project:

https://github.com/webOS-ports/webos-ports-setup/blob/testing/conf/bblayers.conf

openembedded-core provides ofono 1.1.7 for example with
https://github.com/openembedded/openembedded-core/tree/krogoth/meta/recipes-
connectivity/ofono

However we want to use ANOTHER version of ofono (1.1.7 based, but from a different repo/project).

So we have our own .bbappend at
https://github.com/webOS-ports/meta-webos-ports/blob/krogoth/meta-luneos/recipes-connectiv
ity/ofono/ofono_git.bbappend where we specify the different repo
etc to use.

Quoted 4 lines

This doesn't apply 1:1 in your case, but you could simply add a protobuf_3.0.0.bb in your own layer and it should build that instead. Just make sure you have your layer at a higher position compared to meta-openembedded in your bblayers.conf

Thanks a lot.
I have written my own repice and added it into my own layer, it does not compile though :

|
| autoreconf: configure.ac: tracing
|
| autoreconf: configure.ac: subdirectory gmock not present
| autoreconf: configure.ac: not using Libtool
| autoreconf: running:
| /export/arm/pietro/PD15.1/build/tmp-glibc/sysroots/x86_64-linux/usr/bin/autoconf
| --include=/export/arm/pietro/PD15.1/build/tmp-glibc/work/cortexa8t2hf-vfp-neon-phytec-linux-gnueabi/proto
buf/3.0.0-r0/git/m4/ --force
|
| configure.ac:93: error: possibly undefined macro: AC_PROG_LIBTOOL
|
|       If this token and others are legitimate, please use
|       m4_pattern_allow.
|       See the Autoconf documentation.
|
| autoreconf:
| /export/arm/pietro/PD15.1/build/tmp-glibc/sysroots/x86_64-linux/usr/bin/autoconf
| failed with exit status: 1
|
| + bbfatal autoreconf execution failed.

I understand this is a completely different matter now, but has anybody else seen this before ? I have tried to compile the same revision on my local machine "natively" and it's built fine.

Quoted 7 lines

That library should be a dependency for another package/recipe I am working on, is it allowed to specify a version inside the DEPENDS recipe's clause ? I have tried to google the problem but I haven't found a working example as yet.

Cheers,
P.

Forget about it, I was pointing to a broken commit it.
My recipe name is protobuf_3.0.0.bb, how do I make it a dependency of another package ?

I have tried many solution but none of them is working :
DEPENDS = "protobuf > 3.0.0" ... "protobuf_3.0.0" ... etc etc

Any thoughts ?

P.

#12 lore raw

Hi Pietro,

You shouldn't need to specify a version. DEPENDS = "protobuf" or DEPENDS = "protobuf-native" should do :)

Herrie

Quoted 86 lines · Pietro
Quoted 67 lines · Pietro

Jussi Kukkonen <[email protected]>

Quoted 24 lines · writes:

On 1 September 2016 at 13:21, Herman van Hazendonk <[email protected]> wrote:

Hi Pietro,

You can override the recipe by adding a recipe for version 3.0.0+ in your own layer and making sure your layer has a higher
priority
in bblayers.conf. See for example what we do in our project:

https://github.com/webOS-ports/webos-ports-setup/blob/testing/conf/bblayers.conf

openembedded-core provides ofono 1.1.7 for example with

https://github.com/openembedded/openembedded-core/tree/krogoth/meta/recipes-
connectivity/ofono

However we want to use ANOTHER version of ofono (1.1.7 based, but from a different repo/project).

So we have our own .bbappend at

Quoted 8 lines

https://github.com/webOS-ports/meta-webos-ports/blob/krogoth/meta-luneos/recipes-connectiv
ity/ofono/ofono_git.bbappend where we specify the different repo
etc to use.

This doesn't apply 1:1 in your case, but you could simply add a protobuf_3.0.0.bb in your own layer and it should build that instead. Just make sure you have your layer at a higher position compared to meta-openembedded in your bblayers.conf

Thanks a lot.
I have written my own repice and added it into my own layer, it does not compile though :

|
| autoreconf: configure.ac: tracing
|
| autoreconf: configure.ac: subdirectory gmock not present
| autoreconf: configure.ac: not using Libtool
| autoreconf: running:
| /export/arm/pietro/PD15.1/build/tmp-glibc/sysroots/x86_64-linux/usr/bin/autoconf
| 
--include=/export/arm/pietro/PD15.1/build/tmp-glibc/work/cortexa8t2hf-vfp-neon-phytec-linux-gnueabi/proto
buf/3.0.0-r0/git/m4/ --force
|
| configure.ac:93: error: possibly undefined macro: AC_PROG_LIBTOOL
|
|       If this token and others are legitimate, please use
|       m4_pattern_allow.
|       See the Autoconf documentation.
|
| autoreconf:
| /export/arm/pietro/PD15.1/build/tmp-glibc/sysroots/x86_64-linux/usr/bin/autoconf
| failed with exit status: 1
|
| + bbfatal autoreconf execution failed.

I understand this is a completely different matter now, but has anybody else seen this before ? I have tried to compile the same revision on my local machine "natively" and it's built fine.

Quoted 7 lines

That library should be a dependency for another package/recipe I am working on, is it allowed to specify a version inside the DEPENDS recipe's clause ? I have tried to google the problem but I haven't found a working example as yet.

Cheers,
P.

Forget about it, I was pointing to a broken commit it.
My recipe name is protobuf_3.0.0.bb, how do I make it a dependency of another package ?

I have tried many solution but none of them is working :
DEPENDS = "protobuf > 3.0.0" ... "protobuf_3.0.0" ... etc etc

Any thoughts ?

P.

author
#13 lore raw

Herman van Hazendonk <[email protected]>

Quoted 9 lines · writes:

Hi Pietro,

You shouldn't need to specify a version. DEPENDS = "protobuf" or DEPENDS = "protobuf-native" should do :)

Herrie

Indeed my recipe gets built first and I can see do_protobuf_3.0.0[..](), nonetheless when my package, which depends on libprotobuf compiles it fails since it does not find the right version of the library.

It still finds the older version

You don't have protoc 3.0.0 installed in your path.

| Please install Google protocol buffers 3.0.0 and its compiler.
| You can find it here:
|
|    https://github.com/google/protobuf/releases/tag/v3.0.0
|
| Here is what I get when trying to evaluate your version of protoc:
|
| protoc --version
| libprotoc 2.4.1
| make: [system-check] Error 1 (ignored)
|

The version should be 3.0.0. What am I doing wrong ?

Quoted 87 lines · Pietro
Quoted 86 lines
Quoted 66 lines · Pietro

Jussi Kukkonen <[email protected]>

Quoted 25 lines · writes:

On 1 September 2016 at 13:21, Herman van Hazendonk <[email protected]> wrote:

Hi Pietro,

You can override the recipe by adding a recipe for version 3.0.0+ in your own layer and making sure your layer has a higher
priority
in bblayers.conf. See for example what we do in our project:

https://github.com/webOS-ports/webos-ports-setup/blob/testing/conf/bblayers.conf

openembedded-core provides ofono 1.1.7 for example with
https://github.com/openembedded/openembedded-core/tree/krogoth/meta/recipes-
connectivity/ofono

However we want to use ANOTHER version of ofono (1.1.7 based, but from a different repo/project).

So we have our own .bbappend at
https://github.com/webOS-ports/meta-webos-ports/blob/krogoth/meta-luneos/recipes-connectiv
ity/ofono/ofono_git.bbappend where we specify the different repo
etc to use.

Quoted 4 lines

This doesn't apply 1:1 in your case, but you could simply add a protobuf_3.0.0.bb in your own layer and it should build that instead. Just make sure you have your layer at a higher position compared to meta-openembedded in your bblayers.conf

Thanks a lot.
I have written my own repice and added it into my own layer, it does not compile though :

|
| autoreconf: configure.ac: tracing
|
| autoreconf: configure.ac: subdirectory gmock not present
| autoreconf: configure.ac: not using Libtool
| autoreconf: running:
|

/export/arm/pietro/PD15.1/build/tmp-glibc/sysroots/x86_64-linux/usr/bin/autoconf
| --include=/export/arm/pietro/PD15.1/build/tmp-glibc/work/cortexa8t2hf-vfp-neon-phytec-linux-gnueabi/proto buf/3.0.0-r0/git/m4/ --force

|
| configure.ac:93: error: possibly undefined macro: AC_PROG_LIBTOOL
|
|       If this token and others are legitimate, please use
|       m4_pattern_allow.
|       See the Autoconf documentation.
|
| autoreconf:
|
/export/arm/pietro/PD15.1/build/tmp-glibc/sysroots/x86_64-linux/usr/bin/autoconf
| failed with exit status: 1
|
| + bbfatal autoreconf execution failed.

I understand this is a completely different matter now, but has anybody else seen this before ? I have tried to compile the same revision on my local machine "natively" and it's built fine.

Quoted 8 lines

That library should be a dependency for another package/recipe I am working on, is it allowed to specify a version inside the DEPENDS recipe's clause ? I have tried to google the problem but I haven't found
a working example as yet.

Cheers,
P.

Forget about it, I was pointing to a broken commit it.
My recipe name is protobuf_3.0.0.bb, how do I make it a dependency of another package ?

I have tried many solution but none of them is working :
DEPENDS = "protobuf > 3.0.0" ... "protobuf_3.0.0" ... etc etc

Any thoughts ?

P.

#14 lore raw

If it takes the wrong version, it could be your layers aren't in the right order. That's the first thing to check.

You might want to try to run bitbake with

-f protobuf -c cleanall or -f protobuf-native -c cleanall

This will remove all locally ;)

Then rebuild it... I haven't played much with the -native bits, so not sure how those work.

Quoted 131 lines · Pietro

Herman van Hazendonk <[email protected]>

Quoted 9 lines · writes:

Hi Pietro,

You shouldn't need to specify a version. DEPENDS = "protobuf" or DEPENDS = "protobuf-native" should do :)

Herrie

Indeed my recipe gets built first and I can see do_protobuf_3.0.0[..](), nonetheless when my package, which depends on libprotobuf compiles it fails since it does not find the right version of the library.

It still finds the older version

You don't have protoc 3.0.0 installed in your path.

| Please install Google protocol buffers 3.0.0 and its compiler.
| You can find it here:
|
|    https://github.com/google/protobuf/releases/tag/v3.0.0
|
| Here is what I get when trying to evaluate your version of protoc:
|
| protoc --version
| libprotoc 2.4.1
| make: [system-check] Error 1 (ignored)
|

The version should be 3.0.0. What am I doing wrong ?

Quoted 93 lines · Pietro
Quoted 92 lines
Quoted 72 lines · Pietro

Jussi Kukkonen <[email protected]>

Quoted 30 lines · writes:

On 1 September 2016 at 13:21, Herman van Hazendonk <[email protected]> wrote:

Hi Pietro,

You can override the recipe by adding a recipe for version
3.0.0+
in your own layer and making sure your layer has a higher
priority
in bblayers.conf. See for example what we do in our project:

https://github.com/webOS-ports/webos-ports-setup/blob/testing/conf/bblayers.conf

openembedded-core provides ofono 1.1.7 for example with

https://github.com/openembedded/openembedded-core/tree/krogoth/meta/recipes-
connectivity/ofono

However we want to use ANOTHER version of ofono (1.1.7 based,
but
from a different repo/project).

So we have our own .bbappend at

https://github.com/webOS-ports/meta-webos-ports/blob/krogoth/meta-luneos/recipes-connectiv
ity/ofono/ofono_git.bbappend where we specify the different repo
etc to use.

Quoted 5 lines

This doesn't apply 1:1 in your case, but you could simply add a protobuf_3.0.0.bb in your own layer and it should build that instead. Just make sure you have your layer at a higher
position
compared to meta-openembedded in your bblayers.conf

Thanks a lot.
I have written my own repice and added it into my own layer, it does not compile though :

|
| autoreconf: configure.ac: tracing
|
| autoreconf: configure.ac: subdirectory gmock not present
| autoreconf: configure.ac: not using Libtool
| autoreconf: running:
|

/export/arm/pietro/PD15.1/build/tmp-glibc/sysroots/x86_64-linux/usr/bin/autoconf
| --include=/export/arm/pietro/PD15.1/build/tmp-glibc/work/cortexa8t2hf-vfp-neon-phytec-linux-gnueabi/proto buf/3.0.0-r0/git/m4/ --force

|
| configure.ac:93: error: possibly undefined macro: AC_PROG_LIBTOOL
|
|       If this token and others are legitimate, please use
|       m4_pattern_allow.
|       See the Autoconf documentation.
|
| autoreconf:
|
/export/arm/pietro/PD15.1/build/tmp-glibc/sysroots/x86_64-linux/usr/bin/autoconf
| failed with exit status: 1
|
| + bbfatal autoreconf execution failed.

I understand this is a completely different matter now, but has anybody else seen this before ? I have tried to compile the same revision on my local machine "natively" and it's built fine.

Quoted 8 lines

That library should be a dependency for another package/recipe I am working on, is it allowed to specify a version inside the DEPENDS recipe's clause ? I have tried to google the problem but I haven't found
a working example as yet.

Cheers,
P.

Forget about it, I was pointing to a broken commit it.
My recipe name is protobuf_3.0.0.bb, how do I make it a dependency of another package ?

I have tried many solution but none of them is working :
DEPENDS = "protobuf > 3.0.0" ... "protobuf_3.0.0" ... etc etc

Any thoughts ?

P.

author
#15 lore raw

Herman van Hazendonk <[email protected]>

Quoted 12 lines · writes:

If it takes the wrong version, it could be your layers aren't in the right order. That's the first thing to check.

You might want to try to run bitbake with

-f protobuf -c cleanall or -f protobuf-native -c cleanall

This will remove all locally ;)

Then rebuild it... I haven't played much with the -native bits, so not sure how those work.

I needed to specify DEPENDS = "protobuf-native" to get it working, I would be really interested to understand exactly what that does, but I must admit to myself I can't understand everything in a single shot.

So never mind.

I still have another question :-)

I am about to create another recipe for the gprc library for C++, to build it on my local machine has been pretty simple:

$ git clone -b $(curl -L http://grpc.io/release)
https://github.com/grpc/grpc
$ cd grpc
$ git submodule update --init
$ make
$ [sudo] make install

The recipe I have created so far would clone/checkout the source code from a GIT repo or something similar and then the build process could start straight away.

In my case I have an additional step:

git submodule update --init

Is there a function/hook I can override in the recipe ?

Thanks,
P.

#16 lore raw
Quoted 10 lines · Pietro

Herman van Hazendonk <[email protected]>

Quoted 7 lines · writes:

If it takes the wrong version, it could be your layers aren't in the right order. That's the first thing to check.

You might want to try to run bitbake with

-f protobuf -c cleanall or -f protobuf-native -c cleanall

Quoted 8 lines
Quoted 5 lines

This will remove all locally ;)

Then rebuild it... I haven't played much with the -native bits, so not sure how those work.

I needed to specify DEPENDS = "protobuf-native" to get it working, I would be really interested to understand exactly what that does, but I must admit to myself I can't understand everything in a single shot.

Quoting my first email:

Quoted 3 lines
Quoted 3 lines

All you need to do,
is include meta-oe in your layers (bblayers.conf) and have protobuf-native listed in DEPENDS inside your package recipe.

DEPENDS lists build time dependencies, like libraries, tools etc. protobuf-native, by convention, means that the package was built for your build host. This enables you to run protoc during the build to generate proper language bindings.

Quoted 15 lines

So never mind.

I still have another question :-)

I am about to create another recipe for the gprc library for C++, to build it on my local machine has been pretty simple:

$ git clone -b $(curl -L http://grpc.io/release)
https://github.com/grpc/grpc
$ cd grpc
$ git submodule update --init
$ make
$ [sudo] make install
Quoted 8 lines

The recipe I have created so far would clone/checkout the source code from a GIT repo or something similar and then the build process could start straight away.

In my case I have an additional step:

git submodule update --init

There's a submodule fetcher, when setting SRC_URI use gitsm:// instead of git:// see
https://www.yoctoproject.org/docs/2.1/bitbake-user-manual/bitbake-user-manual.html#gitsm-fetcher
for details.

Cheers,

Signature · 2 lines
Maciej Borzecki
RnDity
author
#17 lore raw

Maciej Borzęcki

Quoted 21 lines · <[email protected]> writes:
Quoted 10 lines · Pietro

Herman van Hazendonk <[email protected]>

Quoted 7 lines · writes:

If it takes the wrong version, it could be your layers aren't in the right order. That's the first thing to check.

You might want to try to run bitbake with

-f protobuf -c cleanall or -f protobuf-native -c cleanall

Quoted 8 lines
Quoted 5 lines

This will remove all locally ;)

Then rebuild it... I haven't played much with the -native bits, so not sure how those work.

I needed to specify DEPENDS = "protobuf-native" to get it working, I would be really interested to understand exactly what that does, but I must admit to myself I can't understand everything in a single shot.

Quoting my first email:

Quoted 9 lines
Quoted 3 lines
Quoted 3 lines

All you need to do,
is include meta-oe in your layers (bblayers.conf) and have protobuf-native listed in DEPENDS inside your package recipe.

DEPENDS lists build time dependencies, like libraries, tools etc. protobuf-native, by convention, means that the package was built for your build host. This enables you to run protoc during the build to generate proper language bindings.

Where are all these "native" binaries stored ? Is it "sysroot" ?

How can I declare run-time dependencies ?

Such library should be cross-compiled in order to successfully link my package's binaries against it, on the other hand when building the "protoc" the target should be the host machine as such tool will be run on the host machine when required.

I wonder if is the "native" keyword in the DEPENDS statement to draw the line, so in a recipe A:

i) DEPENDS = "protobuf"
tells that the recipe depends on the protobuf at run-time - namely during the linking and when the binary resulting from A is executed. So protobuf is cross-compiled.

ii) DEPENDS = "protobuf-native"
tells that the recipe depends on the prottobuf at build-time and a result of that protobuf is NOT cross compiled.

Quoted 24 lines
Quoted 15 lines

So never mind.

I still have another question :-)

I am about to create another recipe for the gprc library for C++, to build it on my local machine has been pretty simple:

$ git clone -b $(curl -L http://grpc.io/release)
https://github.com/grpc/grpc
$ cd grpc
$ git submodule update --init
$ make
$ [sudo] make install
Quoted 8 lines

The recipe I have created so far would clone/checkout the source code from a GIT repo or something similar and then the build process could start straight away.

In my case I have an additional step:

git submodule update --init

Quoted 9 lines

There's a submodule fetcher, when setting SRC_URI use gitsm:// instead of git:// see
https://www.yoctoproject.org/docs/2.1/bitbake-user-manual/bitbake-user-manual.html#gitsm-fetcher
for details.

Cheers,

Signature · 2 lines
Maciej Borzecki
RnDity

Thanks, I'll give to it a go.

#18 lore raw
Quoted 28 lines · Pietro

Maciej Borzęcki

Quoted 25 lines · <[email protected]> writes:
Quoted 10 lines · Pietro

Herman van Hazendonk <[email protected]>

Quoted 7 lines · writes:

If it takes the wrong version, it could be your layers aren't in the right order. That's the first thing to check.

You might want to try to run bitbake with

-f protobuf -c cleanall or -f protobuf-native -c cleanall

Quoted 8 lines
Quoted 5 lines

This will remove all locally ;)

Then rebuild it... I haven't played much with the -native bits, so not sure how those work.

I needed to specify DEPENDS = "protobuf-native" to get it working, I would be really interested to understand exactly what that does, but I must admit to myself I can't understand everything in a single shot.

Quoting my first email:

Quoted 3 lines
Quoted 3 lines

All you need to do,
is include meta-oe in your layers (bblayers.conf) and have protobuf-native listed in DEPENDS inside your package recipe.

Quoted 8 lines
Quoted 5 lines

DEPENDS lists build time dependencies, like libraries, tools etc. protobuf-native, by convention, means that the package was built for your build host. This enables you to run protoc during the build to generate proper language bindings.

Where are all these "native" binaries stored ? Is it "sysroot" ?

How can I declare run-time dependencies ?

RDEPENDS, see https://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#var-RDEPENDS

Libraries should be taken care of automatically (at least for ELF binaries), so you will only need to list other services or packages that are needed during runtime.

Quoted 6 lines

Such library should be cross-compiled in order to successfully link my package's binaries against it, on the other hand when building the "protoc" the target should be the host machine as such tool will be run on the host machine when required.

Quoted 7 lines

I wonder if is the "native" keyword in the DEPENDS statement to draw the line, so in a recipe A:

i) DEPENDS = "protobuf"
tells that the recipe depends on the protobuf at run-time - namely during the linking and when the binary resulting from A is executed. So protobuf is cross-compiled.

So the protobuf recipe (meta-oe one) splits the package into 2 parts, protobuf (libraries that are used by generated bindings) and protobuf-compiler (one that generates bindings).
The build system will figure out that your package RDEPENDS on protobuf (not protobuf-compiler).

The other package, protobuf-compiler, is not really needed in your target filesystem, unless you want to generate bindings on the target (as in on your ARM board or similar). It is generally a good practice to spllt your packages so that the user does not have to pay the cost for things they don't need.

Quoted 4 lines

ii) DEPENDS = "protobuf-native"
tells that the recipe depends on the prottobuf at build-time and a result of that protobuf is NOT cross compiled.

Yes. You need protoc to generate bindings at build time, hence you need a version of protobuf that can be run by your native host (hence -native).

Signature · 2 lines
Maciej Borzecki
RnDity
author
#19 lore raw
In reply to Maciej Borzęcki

Maciej Borzęcki

Quoted 54 lines · <[email protected]> writes:
Quoted 10 lines · Pietro

Herman van Hazendonk <[email protected]>

Quoted 7 lines · writes:

If it takes the wrong version, it could be your layers aren't in the right order. That's the first thing to check.

You might want to try to run bitbake with

-f protobuf -c cleanall or -f protobuf-native -c cleanall

Quoted 8 lines
Quoted 5 lines

This will remove all locally ;)

Then rebuild it... I haven't played much with the -native bits, so not sure how those work.

I needed to specify DEPENDS = "protobuf-native" to get it working, I would be really interested to understand exactly what that does, but I must admit to myself I can't understand everything in a single shot.

Quoting my first email:

Quoted 3 lines
Quoted 3 lines

All you need to do,
is include meta-oe in your layers (bblayers.conf) and have protobuf-native listed in DEPENDS inside your package recipe.

DEPENDS lists build time dependencies, like libraries, tools etc. protobuf-native, by convention, means that the package was built for your build host. This enables you to run protoc during the build to generate proper language bindings.

Quoted 15 lines

So never mind.

I still have another question :-)

I am about to create another recipe for the gprc library for C++, to build it on my local machine has been pretty simple:

$ git clone -b $(curl -L http://grpc.io/release)
https://github.com/grpc/grpc
$ cd grpc
$ git submodule update --init
$ make
$ [sudo] make install
Quoted 8 lines

The recipe I have created so far would clone/checkout the source code from a GIT repo or something similar and then the build process could start straight away.

In my case I have an additional step:

git submodule update --init

Quoted 9 lines

There's a submodule fetcher, when setting SRC_URI use gitsm:// instead of git:// see
https://www.yoctoproject.org/docs/2.1/bitbake-user-manual/bitbake-user-manual.html#gitsm-fetcher
for details.

Cheers,

Signature · 2 lines
Maciej Borzecki
RnDity

Thanks a lot.

It looks like the project I am trying to clone does not follow the convention the gitsm module expects.

ERROR: Function failed: Fetcher failure: Fetch command failed with exit code 1, output:
cp: cannot stat '/.../build/downloads/git2/github.com.grpc.grpc/modules': No such file or directory

Is there a way to get around it ?

I have also noticed that the tag parameter does not work properly - it might be me making a mistake though.

SRC_URI = "git://github.com/grpc/grpc;protocol=http;rev=3df9bdf88013e4c9cb5b5f092ac7cd1aad11fa96

Works fine, but:

ERROR: Function failed: Fetcher failure for URL:
'git://github.com/grpc/grpc;protocol=http;tag=v1.0.0'. The command git -c core.fsyncobjectfiles=0 ls-remote http://github.com/grpc/grpc
refs/heads/v1.0.0 refs/t
ags/v1.0.0^{} gave empty output unexpectedly

And indeed the command

git -c core.fsyncobjectfiles=0 ls-remote http://github.com/grpc/grpc
refs/heads/v1.0.0 refs/tags/v1.0.0^{}

Does not result in any output while the command :

git -c core.fsyncobjectfiles=0 ls-remote http://github.com/grpc/grpc
refs/heads/v1.0.0 refs/tags/v1.0.0

Results in :

2a69139aa7f609e439c24a46754252a5f9d37500 refs/tags/v1.0.0

What's the use of '^{}' ?

#20 lore raw
In reply to Pietro
Quoted 81 lines · Pietro

Maciej Borzęcki

Quoted 5 lines · <[email protected]> writes:
Quoted 4 lines · Pietro

Maciej Borzęcki

Quoted 1 line · <[email protected]> writes:

On Wed, Aug 31, 2016 at 6:44 PM, Pietro <[email protected]>

Quoted 3 lines · wrote:
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Hi all,

I am new to the Yocto building system and I could be talking

nonsense. I

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

used to work with buildroot time ago and I remember there is an area where compiled software/packages/tools previously built are "staged"

and

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

used when building other packages.

Is there something like that available with Yocto ? Specifically I

would

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

like to add a package which uses the Google Protocol Buffer, I do not

have

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

administrator rights on the machine and I can't install the packages I need system wise.

Is it possible to add recipes and use them at building time without including them in the image being generated ?

A good example for that would be the protoc, the protocol buffer

description

Quoted 44 lines
Quoted 15 lines
Quoted 7 lines
Quoted 2 lines

file compiler.

There is a recipe for protobuf in meta-oe (2.6.1). All you need to do, is include meta-oe in your layers (bblayers.conf) and have protobuf-native listed in DEPENDS inside your package recipe.

Quoted 8 lines

The protoc compiler will be available in the PATH when package is being built, so autotools checks like AC_CHECK_PROG/AC_PATH_PROG should be able to find it.

Cheers,
--
Maciej Borzecki
RnDity

Quoted 7 lines

Thanks a lot.

I have added the DEPENDS line and it indeed downloads and build the some protobuf related stuff, where did you get the dependency name from ? I can't see the protobuf-native as a recipe in the meta-oe website :

https://layers.openembedded.org/layerindex/branch/master/layer/meta-oe/

It's just a mechanism that allows for building the native (i.e. for the host) packages, and these are autmatically named ${PN}-native. There should a BBCLASSEXTEND = "..native.." stanza inside the protobuf recipe that enables this feature.

Quoted 5 lines

I have just realised that GRPC (http://www.grpc.io/) is needed for my project, it is a library which uses the protobuffers, is there a recipe/package which provides them around or do I need to create my own recipe ?

You can try searching for a recipe at http://layers.openembedded.org
If there's none, try to write your own. This might be a good changes to get yourself acquainted with devtool tool that will help you in the process.

Quoted 4 lines

I would like to check the root filesystem being generated as part of the build process, where is it ?

Quoted 5 lines

${TMPDIR}/<machine>_<distro?>/<imagename>/../rootfs

for instance, for my current build:

tmp/work/vexpress_qemu-poky-linux-gnueabi/core-image-

minimal/1.0-r0/rootfs

Quoted 1 line
Quoted 9 lines
Quoted 3 lines

--
Maciej Borzecki
RnDity

Thanks, much appreciated.

Do you know where the software which is not included in the image - such us protoc - is it stored ?

The sysroots are in ${TMPDIR}/sysroots/: The native sysroot (probably x86_64-linux) will be one of those.

The corresponding protobuf work directory will be in ${TMPDIR}/work/<target_system>/protobuf

See the Yocto dev manual and reference manual for more details about these.
http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html

HTH,
Jussi