Skip to contentSkip to conversation

mesa: avoid unnecessary rerunning of tasks

PATCH Richard Purdie 3 participants 8 messages Sep 5, 2013 – Sep 13, 2013 2 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): 2b4e12bfea634eaecac61b471a9d607fe127d1cd.1378361078.git.nitin.a.kamble@intel.com

#2 lore raw
Quoted 88 lines · On Thu, 2013-09-05 at 06:15 +0000, [email protected] wrote:

From: Nitin A Kamble <[email protected]>

When this recipe is rebuilt for a different machine with same arch, then the previous change to EXTRA_OECONF was causing rerun of all the tasks starting from do_configure.

This commit avoids this behavior, needing only rerun of the do_populate_sysroot task. This is achieved by letting the mesa recipe build the GL components provided by the emgd-driver-bin recipe, but avoid these conflicting files getting populated in the machine specific sysroot.

Fixes bug:
[YOCTO #5120]

---
 common/recipes-graphics/mesa/mesa_9.1.6.bbappend | 69 +++++++++++++++++-------
 1 file changed, 49 insertions(+), 20 deletions(-)
common/recipes-graphics/mesa/mesa_9.1.6.bbappend +40 -20
index b92831d..8024f0f 100644
@@ -1,24 +1,53 @@
-
 # The emgd binary driver also provides egl, gles1, gles2 library & headers.
-# To avoid conflict disable egl, gles1, gles2 from meta-dri if the BSP image
-# is bundling the emgd driver.
+# To avoid conflict do not populate sysroot with  egl, gles1, gles2 files
+# from mesa recipe if the BSP image is bundling the emgd driver.
+
+EXCLUDE_FROM_STAGING="${libdir}/libGLESv2.so.2 \
+                             ${libdir}/libEGL.so.1 \
+                             ${libdir}/libGLESv2.so \
+                             ${libdir}/libEGL.so \
+                             ${libdir}/pkgconfig/egl.pc \
+                             ${libdir}/pkgconfig/glesv2.pc \
+                             ${includedir}/KHR/khrplatform.h \
+                             ${includedir}/GLES/glext.h \
+                             ${includedir}/GLES/glplatform.h \
+                             ${includedir}/GLES/gl.h \
+                             ${includedir}/EGL/eglext.h \
+                             ${includedir}/EGL/eglplatform.h \
+                             ${includedir}/EGL/egl.h \
+                             ${includedir}/GLES2/gl2.h \
+                             ${includedir}/GLES2/gl2platform.h \
+                             ${includedir}/GLES2/gl2ext.h \
+                             "
+
+# override the function from the staging.bbclass
+# An exclude list is conveyed to the tar
+sysroot_stage_dir () {
+        src="$1"
+        dest="$2"
+        # if the src doesn't exist don't do anything
+        if [ ! -d "$src" ]; then
+                 return
+        fi
 
-python __anonymous () {
-    import re
-    xserver = d.getVar('XSERVER', True)
-    if xserver and 'emgd-driver-bin' in xserver.split(' '):
-        extra_oeconf = d.getVar('EXTRA_OECONF', True).split()
-        take_out = ["--enable-egl", "--enable-gles1", "--enable-gles2"]
-        put_in = ["--disable-egl", "--disable-gles1", "--disable-gles2"]
-        pattern = re.compile("--with-egl-platforms")
-        new_extra_oeconf = [ ]
-        for i in extra_oeconf:
-            if ( i not in take_out ) and ( not pattern.match(i)):
-                new_extra_oeconf.append(i)
-        for i in put_in:
-            new_extra_oeconf.append(i)
+        # We only want to stage the contents of $src if it's non-empty so first rmdir $src
+        # then if it still exists (rmdir on non-empty dir fails) we can copy its contents
+        rmdir "$src" 2> /dev/null || true
+        # However we always want to stage a $src itself, even if it's empty
+        mkdir -p "$dest"
+
+        if [ -d "$src" ]; then
+                tar -X ${T}/exclude_from_staging -cf - -C "$src" -ps . | tar -xf - -C "$dest"
+        fi
+}
 
-        d.setVar('EXTRA_OECONF', ' '.join(new_extra_oeconf))
-        depends = d.getVar('DEPENDS', True)
-        d.setVar('DEPENDS', depends + " emgd-driver-bin")

common/recipes-graphics/mesa/mesa_9.1.6.bbappend line 53

Quoted 10 lines
common/recipes-graphics/mesa/mesa_9.1.6.bbappend +9 -0
+# here the exclude list for tar is prepared
+sysroot_stage_all_prepend() {
+        truncate --size 0 ${T}/exclude_from_staging
+        if [[ "${XSERVER}" =~ "emgd-driver-bin" ]]; then
+                for i in ${EXCLUDE_FROM_STAGING}
+                do
+                        echo ${D}${i} >> ${T}/exclude_from_staging
+                done
+        fi
 }

This is certainly better and is heading in the right direction. The two issues I can see are:

a) do_populate_sysroot still needs to run again
b) do_packagedata installs pkgdata (shlibs) which says mesa provides some of these libs yet is no shouldn't for the given MACHINE.

I did think this shouldn't be too difficult to fix however life is never quite straightforward.

For a), rather than change things before we create the sstate package, I was thinking we could change than at installation of the sstate package. We have SSTATEPOSTINSTFUNCS however these run after we have placed the files in the sysroot so its too late. SSTATEPREINSTFUNCS sounds better but this actually runs before unpacking the sstate files which is too early for us.

There is only one user of SSTATEPREINSTFUNCS (useradd.bbclass) so I'm going to propose we rename that one to a more logical SSTATEPREUNPACKFUNCS and add PREINSTFUNCS at a more sensible place. We can then add a SSTATEPREINSTFUNCS to the mesa recipe which for do_populate_sysroot and do_populate_sysroot_setscene deletes the files in question. That should resolve a).

b) is more tricky since pkgdata is arch specific, not machine specific. I had thought it was machine specific but checking now, its not.

To be quite honest, I don't like what pkgdata does at the moment. We have cleaned things up a lot compared to where they used to be but the search process the system uses to find information is truly horrid.

It would be rather tempting to decide to install pkgdata in a machine specific directory much like we do with populate_sysroot already. There would be minimal performance overhead since the files are small and we could then bin all the "search all, then arch, then machine" type lookup code we have right now. If we did make it machine specific, we could then use the same trick we use in a) but for do_packagedata and instead of removing files, it would run some carefully crafted sed operations.

Right now, we might not need to fix this 100% properly. If emgd installs itself as machine specific, it should be first in the providers search paths so the second lot of pkgdata should get ignored. emgd is of course not machine specific at the moment, nor should it be apart from this issue :/.

This also raises some other questions. The existing code in meta-intel adds EMGD into the DEPENDS. It does this so the normal PROVIDES for virtual/libgl and friends don't have to change. It does mean the sstate checksums for anything depending on mesa change however. If we don't have the dependency, things may try and build before emgd is built, libGL would be missing and things would break.

So somehow we need to add in the dependency let leave the sstate checksums unchanged. This is possible in the sstate checksum validation code but its probably easiest to change that in the core, not meta-intel :/.

I'd also appreciate someone (Ross maybe?) confirming that if we build application A against mesa, then change over to a machine that uses emgd and swizzle the libs around in the sysroot, do we need to recompile the app? This approach is assuming we do not need to do that. If we do, there is a different approach we need to take.

So in summary, this problem is far from simple, we do have most of the pieces we need to fix it with some tweaking but some of the changes are invasive and might need to be deferred to 1.6. As it stands, the change will cause issues due to the changed DEPENDS. We do need to think carefully about this and solve it once and for all in a way which works optimally but the existing workaround might be best until we can pull all the better pieces together.

I hope this was a relatively clear explanation of the problem, let me know if I need to clarify anything. I've cc'd OE-Core since I think more people probably need to see/think about this.

Cheers,

Richard

#3 lore raw
Quoted 71 lines · Richard Purdie

This is certainly better and is heading in the right direction. The two issues I can see are:

a) do_populate_sysroot still needs to run again
b) do_packagedata installs pkgdata (shlibs) which says mesa provides some of these libs yet is no shouldn't for the given MACHINE.

I did think this shouldn't be too difficult to fix however life is never quite straightforward.

For a), rather than change things before we create the sstate package, I was thinking we could change than at installation of the sstate package. We have SSTATEPOSTINSTFUNCS however these run after we have placed the files in the sysroot so its too late. SSTATEPREINSTFUNCS sounds better but this actually runs before unpacking the sstate files which is too early for us.

There is only one user of SSTATEPREINSTFUNCS (useradd.bbclass) so I'm going to propose we rename that one to a more logical SSTATEPREUNPACKFUNCS and add PREINSTFUNCS at a more sensible place. We can then add a SSTATEPREINSTFUNCS to the mesa recipe which for do_populate_sysroot and do_populate_sysroot_setscene deletes the files in question. That should resolve a).

b) is more tricky since pkgdata is arch specific, not machine specific. I had thought it was machine specific but checking now, its not.

To be quite honest, I don't like what pkgdata does at the moment. We have cleaned things up a lot compared to where they used to be but the search process the system uses to find information is truly horrid.

It would be rather tempting to decide to install pkgdata in a machine specific directory much like we do with populate_sysroot already. There would be minimal performance overhead since the files are small and we could then bin all the "search all, then arch, then machine" type lookup code we have right now. If we did make it machine specific, we could then use the same trick we use in a) but for do_packagedata and instead of removing files, it would run some carefully crafted sed operations.

Right now, we might not need to fix this 100% properly. If emgd installs itself as machine specific, it should be first in the providers search paths so the second lot of pkgdata should get ignored. emgd is of course not machine specific at the moment, nor should it be apart from this issue :/.

This also raises some other questions. The existing code in meta-intel adds EMGD into the DEPENDS. It does this so the normal PROVIDES for virtual/libgl and friends don't have to change. It does mean the sstate checksums for anything depending on mesa change however. If we don't have the dependency, things may try and build before emgd is built, libGL would be missing and things would break.

So somehow we need to add in the dependency let leave the sstate checksums unchanged. This is possible in the sstate checksum validation code but its probably easiest to change that in the core, not meta-intel :/.

I'd also appreciate someone (Ross maybe?) confirming that if we build application A against mesa, then change over to a machine that uses emgd and swizzle the libs around in the sysroot, do we need to recompile the app? This approach is assuming we do not need to do that. If we do, there is a different approach we need to take.

So in summary, this problem is far from simple, we do have most of the pieces we need to fix it with some tweaking but some of the changes are invasive and might need to be deferred to 1.6. As it stands, the change will cause issues due to the changed DEPENDS. We do need to think carefully about this and solve it once and for all in a way which works optimally but the existing workaround might be best until we can pull all the better pieces together.

Quoted 3 lines

I hope this was a relatively clear explanation of the problem, let me know if I need to clarify anything. I've cc'd OE-Core since I think more people probably need to see/think about this.

It actually get worse (or maybe slightly better) when you think about the shlibs code.

Imagine application A links against libGL. It therefore gets an RDEPENDS on the package that provides libGL. With emgd in the mix, the package providing libGL changes and as things stand today, I suspect it gets rebuild.

So lets assume that we add leave the mesa pkgdata/shlibs code in place and actively stop emgd putting conflicting shlibs provides in place (easy to do).

We could then make the emgd-binary packages RPROVIDE/RCONFLICT the right mesa packages. We'd then need to make sure the package manager behaviour is correct in that:

a) If an image is mesa based, installing the app pulls in libGL as normal, doesn't touch emgd.
b) In an EMGD image, installing the app pulls in libgl, however XSERVER also pulls in emgd-binary and this then replaces the libGL package.

In theory this should all be possible and would resolve the problem in
b) in my original email without the need to make pkgdata machine specific (we should do that for other reasons but we're getting off track).

So perhaps there is a way out of this but it isn't simple...

Cheers,

Richard

#4 lore raw
In reply to Richard Purdie

On Thu, Sep 5, 2013 at 5:33 AM, Richard Purdie
<[email protected]> wrote:
....

Quoted 12 lines

This also raises some other questions. The existing code in meta-intel adds EMGD into the DEPENDS. It does this so the normal PROVIDES for virtual/libgl and friends don't have to change. It does mean the sstate checksums for anything depending on mesa change however. If we don't have the dependency, things may try and build before emgd is built, libGL would be missing and things would break.

So somehow we need to add in the dependency let leave the sstate checksums unchanged. This is possible in the sstate checksum validation code but its probably easiest to change that in the core, not meta-intel :/.

Quoted 5 lines

I'd also appreciate someone (Ross maybe?) confirming that if we build application A against mesa, then change over to a machine that uses emgd and swizzle the libs around in the sysroot, do we need to recompile the app? This approach is assuming we do not need to do that. If we do, there is a different approach we need to take.

In theory both cases are possible; the API must be the same but we cannot guarantee ABI will match.

Quoted 11 lines

So in summary, this problem is far from simple, we do have most of the pieces we need to fix it with some tweaking but some of the changes are invasive and might need to be deferred to 1.6. As it stands, the change will cause issues due to the changed DEPENDS. We do need to think carefully about this and solve it once and for all in a way which works optimally but the existing workaround might be best until we can pull all the better pieces together.

I hope this was a relatively clear explanation of the problem, let me know if I need to clarify anything. I've cc'd OE-Core since I think more people probably need to see/think about this.

We have same issues in meta-fsl-arm; there we made mesa and gpu packages machine specific as we /mangle/ them.

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
#5 lore raw
In reply to Richard Purdie
Quoted 5 lines · Richard Purdie

I'd also appreciate someone (Ross maybe?) confirming that if we build application A against mesa, then change over to a machine that uses emgd and swizzle the libs around in the sysroot, do we need to recompile the app? This approach is assuming we do not need to do that. If we do, there is a different approach we need to take.

So what's happening is that we're revising the original OpenGL policy that I posted in October last year:

http://lists.openembedded.org/pipermail/openembedded-core/2012-October/070122.html

tl;dr: I proposed that 1) we rename all GL packages so they identify their provider (ie libgl-mesa, we do that), 2) no dependencies on specific GL implementations, 3) only Mesa stages. The theory was that Mesa can be considered a canonical implementation of the various GL platforms and that we can *build* against Mesa but *install* the right hardware-specific GL implementation into the images.

Nice in theory and it should work, but sadly that isn't the case - the thread has more details. Short version is that some applications *do* rely on specific GL headers, so you can't just swap them around in reality.

Maybe it is time to have a mesa-gl recipe alongside mesa that *just* builds the GL libraries. EMGD can depend on it for the driver modules it installs, and presumably other vendors with binary drivers can install it for the software rendering/GLX support (Otavio etc, please step in here!).

Ross

#6 lore raw
Quoted 25 lines · Ross
Quoted 5 lines · Richard Purdie

I'd also appreciate someone (Ross maybe?) confirming that if we build application A against mesa, then change over to a machine that uses emgd and swizzle the libs around in the sysroot, do we need to recompile the app? This approach is assuming we do not need to do that. If we do, there is a different approach we need to take.

So what's happening is that we're revising the original OpenGL policy that I posted in October last year:

http://lists.openembedded.org/pipermail/openembedded-core/2012-October/070122.html

tl;dr: I proposed that 1) we rename all GL packages so they identify their provider (ie libgl-mesa, we do that), 2) no dependencies on specific GL implementations, 3) only Mesa stages. The theory was that Mesa can be considered a canonical implementation of the various GL platforms and that we can *build* against Mesa but *install* the right hardware-specific GL implementation into the images.

Nice in theory and it should work, but sadly that isn't the case - the thread has more details. Short version is that some applications *do* rely on specific GL headers, so you can't just swap them around in reality.

Quoted 5 lines

Maybe it is time to have a mesa-gl recipe alongside mesa that *just* builds the GL libraries. EMGD can depend on it for the driver modules it installs, and presumably other vendors with binary drivers can install it for the software rendering/GLX support (Otavio etc, please step in here!).

Yes; we have both situations at Freescale ARM BSP.

For the AMD GPU binaries, which are used in MX5 SoCs we *just* use mesa GL and do software rendering. For Vivante GPU binaries, used for MX6, we use mesa GL *headers* but Vivante provides a libGL binary.

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
#7 lore raw
Quoted 10 lines · Otavio Salvador
Quoted 5 lines

Maybe it is time to have a mesa-gl recipe alongside mesa that *just* builds the GL libraries. EMGD can depend on it for the driver modules it installs, and presumably other vendors with binary drivers can install it for the software rendering/GLX support (Otavio etc, please step in here!).

Yes; we have both situations at Freescale ARM BSP.

For the AMD GPU binaries, which are used in MX5 SoCs we *just* use mesa GL and do software rendering.

So mesa-gl will work fine there.

Quoted 2 lines

For Vivante GPU binaries, used for
MX6, we use mesa GL *headers* but Vivante provides a libGL binary.

My, that's horrific. I'd forgotten about this, for good reason clearly.

Ross

#8 lore raw
Quoted 12 lines · Ross
Quoted 10 lines · Otavio Salvador
Quoted 5 lines

Maybe it is time to have a mesa-gl recipe alongside mesa that *just* builds the GL libraries. EMGD can depend on it for the driver modules it installs, and presumably other vendors with binary drivers can install it for the software rendering/GLX support (Otavio etc, please step in here!).

Yes; we have both situations at Freescale ARM BSP.

For the AMD GPU binaries, which are used in MX5 SoCs we *just* use mesa GL and do software rendering.

Quoted 6 lines

So mesa-gl will work fine there.

Quoted 2 lines

For Vivante GPU binaries, used for
MX6, we use mesa GL *headers* but Vivante provides a libGL binary.

My, that's horrific. I'd forgotten about this, for good reason clearly.

So I now have actual code which is better than any thought experiment :)

First, I have this applied:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t1&id=3e2c058ffab5b4e92523dac3078c1bc02d0eb8b8

This creates a machine specific pkgdata/shlibs directory instead of the current multiple directories and iterations we use. Its not complete yet (buildhistory is broken for example), its another proof of concept. What this does is ensure that the main TUNE_PKGARCH directory doesn't get corrupted with emgd shlibs below, accidentally or otherwise.

We then have two PoC patches one for OE-Core, one for meta-intel. These put the gl bits into "i586-emgd" instead of "i586" so they're isolated.

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t1&id=258e068bbb309877b6e36eeec1107806fc710cde
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/metaintel&id=917286eebc5e4eeb13fcc1b0db6adb50ccd847a0

I hacked emenlow so it shared qemux86's TUNE_PKGARCH for testing purposes. You can then build both qemux86 and emenlow in the same build directory without them trampling clutter.

cogl would also need to inherit the gl class so its rough around the edges but basically works.

Thoughts?

Cheers,

Richard

#9 lore raw
Quoted 38 lines · Richard Purdie
Quoted 12 lines · Ross
Quoted 10 lines · Otavio Salvador
Quoted 5 lines

Maybe it is time to have a mesa-gl recipe alongside mesa that *just* builds the GL libraries. EMGD can depend on it for the driver modules it installs, and presumably other vendors with binary drivers can install it for the software rendering/GLX support (Otavio etc, please step in here!).

Yes; we have both situations at Freescale ARM BSP.

For the AMD GPU binaries, which are used in MX5 SoCs we *just* use mesa GL and do software rendering.

Quoted 6 lines

So mesa-gl will work fine there.

Quoted 2 lines

For Vivante GPU binaries, used for
MX6, we use mesa GL *headers* but Vivante provides a libGL binary.

My, that's horrific. I'd forgotten about this, for good reason clearly.

So I now have actual code which is better than any thought experiment :)

First, I have this applied:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t1&id=3e2c058ffab5b4e92523dac3078c1bc02d0eb8b8

This creates a machine specific pkgdata/shlibs directory instead of the current multiple directories and iterations we use. Its not complete yet (buildhistory is broken for example), its another proof of concept. What this does is ensure that the main TUNE_PKGARCH directory doesn't get corrupted with emgd shlibs below, accidentally or otherwise.

We then have two PoC patches one for OE-Core, one for meta-intel. These put the gl bits into "i586-emgd" instead of "i586" so they're isolated.

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t1&id=258e068bbb309877b6e36eeec1107806fc710cde
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/metaintel&id=917286eebc5e4eeb13fcc1b0db6adb50ccd847a0

Quoted 8 lines

I hacked emenlow so it shared qemux86's TUNE_PKGARCH for testing purposes. You can then build both qemux86 and emenlow in the same build directory without them trampling clutter.

cogl would also need to inherit the gl class so its rough around the edges but basically works.

Thoughts?

Sorry by delayed reply.

I did look at the changes and I did enjoy it a lot. I had something like this in O.S. Systems fork of OpenEmbedded-Classic to share multiple builds against compatible geode machines... so let's stop with nostalgy and focus in 2013 ;-)

It does looks great. I like the idea however I'd prefer if you could name it like "subarch" or something like that. For example we have some uses for this in FSL layer:

* xserver-xorg
* gpu-viv
* amd-gpu
* gst-fsl-plugin

All these are subarch compatible but currently are marked as machine specific (which slow down builds).

Regards,

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750