Skip to contentSkip to conversation

dmalloc: fix parallel builds

PATCH New Alex Suykov 2 participants 6 messages Jan 3, 2015 – Jan 7, 2015

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 dmalloc: fix parallel builds

---
This patch should fix
http://autobuild.buildroot.net/results/2cc40ae3fc8b7a287c43528b3e4ffdbcd5033c09/

 package/dmalloc/dmalloc-deps.patch | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
 create mode 100644 package/dmalloc/dmalloc-deps.patch
package/dmalloc/dmalloc-deps.patch new file +14 -0
new file mode 100644
index 0000000..c717e1a
@@ -0,0 +1,14 @@
+Missing dmallocc.o <- dmalloc.h dependency may break parallel builds.
+dmalloc.h is generated, and may not be around by the time gcc starts
+parsing dmallocc.cc.
+
+--- a/Makefile.in
++++ b/Makefile.in
+@@ -399,6 +399,7 @@
+ compat.o: compat.c dmalloc.h conf.h settings.h compat.h dmalloc_loc.h
+ dmalloc.o: dmalloc.c dmalloc_argv.h conf.h settings.h dmalloc.h \
+  compat.h debug_tok.h dmalloc_loc.h env.h error_val.h version.h
++dmallocc.o: dmallocc.cc dmalloc.h return.h conf.h settings.h
+ dmalloc_argv.o: dmalloc_argv.c conf.h settings.h dmalloc_argv.h \
+  dmalloc_argv_loc.h compat.h
+ dmalloc_rand.o: dmalloc_rand.c conf.h settings.h dmalloc_rand.h
Signature · 1 line
2.0.3
#1 lore raw

PATCH 1/1 dmalloc: fix parallel builds

Dear Alex Suykov,

Quoted 8 lines · Alex Suykov

---
This patch should fix
http://autobuild.buildroot.net/results/2cc40ae3fc8b7a287c43528b3e4ffdbcd5033c09/

 package/dmalloc/dmalloc-deps.patch | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
 create mode 100644 package/dmalloc/dmalloc-deps.patch

While it indeed looks like a parallel build issue, I haven't been able to reproduce it: I've started a loop building dmalloc over and over again on my build server (on which the issue was reproduced by the autobuilders), and never managed to see it after an entire night looping on dmalloc builds. So I'm a bit confused on this one.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#2 lore raw

PATCH 1/1 dmalloc: fix parallel builds

Quoted 10 lines · Thomas Petazzoni
Quoted 8 lines · Alex Suykov

---
This patch should fix
http://autobuild.buildroot.net/results/2cc40ae3fc8b7a287c43528b3e4ffdbcd5033c09/

 package/dmalloc/dmalloc-deps.patch | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
 create mode 100644 package/dmalloc/dmalloc-deps.patch
Quoted 5 lines

While it indeed looks like a parallel build issue, I haven't been able to reproduce it: I've started a loop building dmalloc over and over again on my build server (on which the issue was reproduced by the autobuilders), and never managed to see it after an entire night looping on dmalloc builds. So I'm a bit confused on this one.

The following sequence seems to trigger it more or less reliably for me:

    ./configure --disable-static --enable-shared --enable-cxx --enable-threads
    sed -i -e "s at cat \$(srcdir)/dmalloc.h.1.*@&; sleep 1@" Makefile
    make clean
    make -j4

What happens is a race condition between "cat ...; mv ... dmalloc.h" and "g++ -c dmallocc.cc". As long as cat-mv pair is faster, it works well. But if there's a delay, it breaks.

This should be rare, I can only reproduce it with an artificial delay, and only with --disable-static. But on the other hand, it did happen on autobuild somehow.

#3 lore raw

PATCH 1/1 dmalloc: fix parallel builds

Dear Alex Suykov,

Quoted 7 lines · Alex Suykov

The following sequence seems to trigger it more or less reliably for me:

    ./configure --disable-static --enable-shared --enable-cxx --enable-threads
    sed -i -e "s at cat \$(srcdir)/dmalloc.h.1.*@&; sleep 1@" Makefile
    make clean
    make -j4
Quoted 7 lines

What happens is a race condition between "cat ...; mv ... dmalloc.h" and "g++ -c dmallocc.cc". As long as cat-mv pair is faster, it works well. But if there's a delay, it breaks.

This should be rare, I can only reproduce it with an artificial delay, and only with --disable-static. But on the other hand, it did happen on autobuild somehow.

Ok, thanks a lot for the explanation! I've committed your patch, with a few minor tweaks:

[Thomas:
- Rename patch to the new naming convention.
- Add SoB line from Alex inside the patch itself.
- Adjust the commit log to contain the reference to the autobuilder failure.]

If you're using dmalloc, I just noticed that the version we have in Buildroot is 5.4.3, while upstream has a 5.5.2 release. If you want to send a patch to bump the dmalloc version, that would be great.

Also, don't hesitate to send your patch to the upstream dmalloc developers.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#4 lore raw

PATCH 1/1 dmalloc: fix parallel builds

Quoted 3 lines · Thomas Petazzoni

If you're using dmalloc, I just noticed that the version we have in Buildroot is 5.4.3, while upstream has a 5.5.2 release. If you want to send a patch to bump the dmalloc version, that would be great.

No, I don't use it myself. I was just looking through autobuild logs for unrelated reasons and kind of decided to fix what was clearly fixable.

Anyway, I'll send a version bump patch now, it looks like the changes are minor, and both Gentoo and OpenWRT have 5.5.2 already.

I wonder if it makes sense to contact whoever sent that config to autobuild and ask for some feedback. Since dmalloc was enabled there, it was probably used for something.

#5 lore raw

PATCH 1/1 dmalloc: fix parallel builds

Dear Alex Suykov,

Quoted 5 lines · Alex Suykov

No, I don't use it myself. I was just looking through autobuild logs for unrelated reasons and kind of decided to fix what was clearly fixable.

Anyway, I'll send a version bump patch now, it looks like the changes are minor, and both Gentoo and OpenWRT have 5.5.2 already.

Good, thanks!

Quoted 3 lines

I wonder if it makes sense to contact whoever sent that config to autobuild and ask for some feedback. Since dmalloc was enabled there, it was probably used for something.

The configs tested in the autobuilders are randomly generated:

* A random architecture/toolchain configuration is selected amongst
http://autobuild.buildroot.org/toolchains/configs/.

* A random selection of packages is generated using "randpackageconfig".

So the configuration tested in the autobuilders are not submitted by people, they are randomly generated by a script. That's how we test so many combinations of packages/platforms.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com