Fix webkit-gtk 'Memory Exhaust' issue
cover Fix webkit-gtk 'Memory Exhaust' issue
The following changes since commit 31e6eee860b5f9f4ac9ef0889bcff5648de6e3f9:
poky-tiny.conf: blacklist core-image-weston option (2013-07-18 21:26:58 +0100)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib kangkai/webkit-gtk
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kangkai/webkit-gtk
Kai Kang (1):
webkit-gtk: fix 'Memory exhausted' error
meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb | 2 ++ 1 file changed, 2 insertions(+)
Signature · 1 line
1.8.1.2
PATCH 1/1 webkit-gtk: fix 'Memory exhausted' error
Build webkit-gtk occupies lot of memory and fails on some host:
| ...tmp/sysroots/i686-linux/usr/libexec/x86_64-wrs-linux/gcc/x86_64-wrs-linux/4.8.1/ld:
failed to set dynamic section sizes: Memory exhausted
| collect2: error: ld returned 1 exit status
| make[1]: *** [libwebkitgtk-1.0.la] Error 1
Add ld option to fix it.
Refer to:
http://comments.gmane.org/gmane.os.opendarwin.webkit.devel/20078
--- meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb +2 -0
@@ -60,6 +60,8 @@ CPPFLAGS_append_powerpc = " -I${STAGING_INCDIR}/pango-1.0 \ EXTRA_AUTORECONF = " -I Source/autotools " +# Fix compile error: "ld: failed to set dynamic section sizes: Memory exhausted" +LDFLAGS += " -Wl,--no-keep-memory" #| ./Source/JavaScriptCore/heap/HandleTypes.h: In static member function 'static T* JSC::HandleTypes<T>::getFromSlot(JSC::HandleSlot) [with T = JSC::Structure, JSC::HandleTypes<T>::ExternalType = JSC::Structure*, JSC::HandleSlot = JSC::JSValue*]': #| ./Source/JavaScriptCore/heap/Handle.h:141:79: instantiated from 'JSC::Handle<T>::ExternalType JSC::Handle<T>::get() const [with T = JSC::Structure, JSC::Handle<T>::ExternalType = JSC::Structure*]'
Signature · 1 line
1.8.1.2
Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
Hi,
Instead of penalising everybody, would it be possible to make this change dependent on the build machine (ram + swap)? Or to only add -Wl,--no-keep-memory if the build failed without it?
Not sure if it would make any difference, but do you see this only with full debug enabled (-ggdb)? If yes, can the debug flags be changed, or that linker flag only be added, if full debug is enabled?
Cheers,
Andre'
Quoted 29 lines · Kai Kang
Build webkit-gtk occupies lot of memory and fails on some host:
| ...tmp/sysroots/i686-linux/usr/libexec/x86_64-wrs-linux/gcc/x86_64-wrs-linux/4.8.1/ld:
failed to set dynamic section sizes: Memory exhausted
| collect2: error: ld returned 1 exit status
| make[1]: *** [libwebkitgtk-1.0.la] Error 1Add ld option to fix it.
Refer to:
http://comments.gmane.org/gmane.os.opendarwin.webkit.devel/20078Signed-off-by Kai Kang <[email protected]>--- meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb | 2 ++ 1 file changed, 2 insertions(+)meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb +2
-0@@ -60,6 +60,8 @@ CPPFLAGS_append_powerpc = " -I${STAGING_INCDIR}/pango-1.0 \ EXTRA_AUTORECONF = " -I Source/autotools " +# Fix compile error: "ld: failed to set dynamic section sizes: Memory exhausted" +LDFLAGS += " -Wl,--no-keep-memory" #| ./Source/JavaScriptCore/heap/HandleTypes.h: In static member function 'static T* JSC::HandleTypes<T>::getFromSlot(JSC::HandleSlot) [with T = JSC::Structure, JSC::HandleTypes<T>::ExternalType = JSC::Structure*, JSC::HandleSlot = JSC::JSValue*]': #| ./Source/JavaScriptCore/heap/Handle.h:141:79: instantiated from 'JSC::Handle<T>::ExternalType JSC::Handle<T>::get() const [with T = JSC::Structure, JSC::Handle<T>::ExternalType = JSC::Structure*]'
Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
Agreed, I think this sort of thing belongs in site.conf or some such place, not in the recipe.
p.
Quoted 50 lines · André Draszik
Hi,
Instead of penalising everybody, would it be possible to make this change dependent on the build machine (ram + swap)? Or to only add -Wl,--no-keep-memory if the build failed without it?
Not sure if it would make any difference, but do you see this only with full debug enabled (-ggdb)? If yes, can the debug flags be changed, or that linker flag only be added, if full debug is enabled?
Cheers,
Andre'Quoted 21 lines · Kai Kang
Build webkit-gtk occupies lot of memory and fails on some host:
| ...tmp/sysroots/i686-linux/usr/libexec/x86_64-wrs-linux/gcc/x86_64-wrs-linux/4.8.1/ld:
failed to set dynamic section sizes: Memory exhausted
| collect2: error: ld returned 1 exit status
| make[1]: *** [libwebkitgtk-1.0.la] Error 1Add ld option to fix it.
Refer to:
http://comments.gmane.org/gmane.os.opendarwin.webkit.devel/20078Signed-off-by Kai Kang <[email protected]>--- meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb | 2 ++ 1 file changed, 2 insertions(+)meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb +0
-0meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb CPPFLAGS_append_powerpc = " -I${STAGING_INCDIR}/pango-1.0 \ · line 67
Quoted 8 lines
meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb +2
-0@@ -60,6 +60,8 @@ CPPFLAGS_append_powerpc = " -I${STAGING_INCDIR}/pango-1.0 \ EXTRA_AUTORECONF = " -I Source/autotools " +# Fix compile error: "ld: failed to set dynamic section sizes: Memory exhausted" +LDFLAGS += " -Wl,--no-keep-memory" #| ./Source/JavaScriptCore/heap/HandleTypes.h: In static member function 'static T* JSC::HandleTypes<T>::getFromSlot(JSC::HandleSlot) [with T = JSC::Structure, JSC::HandleTypes<T>::ExternalType = JSC::Structure*, JSC::HandleSlot = JSC::JSValue*]': #| ./Source/JavaScriptCore/heap/Handle.h:141:79: instantiated from 'JSC::Handle<T>::ExternalType JSC::Handle<T>::get() const [with T = JSC::Structure, JSC::Handle<T>::ExternalType = JSC::Structure*]'_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
In reply to André DraszikQuoted 5 lines · André Draszik
Hi,
Instead of penalising everybody, would it be possible to make this change dependent on the build machine (ram + swap)? Or to only add -Wl,--no-keep-memory if the build failed without it?
Yes, it doesn't fail on every build machine. But it seems that build fails randomly on different build machines.
Quoted 4 lines
Not sure if it would make any difference, but do you see this only with full debug enabled (-ggdb)? If yes, can the debug flags be changed, or that linker flag only be added, if full debug is enabled?
webkit-gtk set configure option "--enable-debug=no" to turn off debug. And no such "-ggdb" flags.
Thanks,
Kai
Quoted 42 lines
Cheers,
Andre'Quoted 21 lines · Kai Kang
Build webkit-gtk occupies lot of memory and fails on some host:
| ...tmp/sysroots/i686-linux/usr/libexec/x86_64-wrs-linux/gcc/x86_64-wrs-linux/4.8.1/ld:
failed to set dynamic section sizes: Memory exhausted
| collect2: error: ld returned 1 exit status
| make[1]: *** [libwebkitgtk-1.0.la] Error 1Add ld option to fix it.
Refer to:
http://comments.gmane.org/gmane.os.opendarwin.webkit.devel/20078Signed-off-by Kai Kang <[email protected]>--- meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb | 2 ++ 1 file changed, 2 insertions(+)meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb +0
-0meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb CPPFLAGS_append_powerpc = " -I${STAGING_INCDIR}/pango-1.0 \ · line 67
Quoted 8 lines
meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb +2
-0@@ -60,6 +60,8 @@ CPPFLAGS_append_powerpc = " -I${STAGING_INCDIR}/pango-1.0 \ EXTRA_AUTORECONF = " -I Source/autotools " +# Fix compile error: "ld: failed to set dynamic section sizes: Memory exhausted" +LDFLAGS += " -Wl,--no-keep-memory" #| ./Source/JavaScriptCore/heap/HandleTypes.h: In static member function 'static T* JSC::HandleTypes<T>::getFromSlot(JSC::HandleSlot) [with T = JSC::Structure, JSC::HandleTypes<T>::ExternalType = JSC::Structure*, JSC::HandleSlot = JSC::JSValue*]': #| ./Source/JavaScriptCore/heap/Handle.h:141:79: instantiated from 'JSC::Handle<T>::ExternalType JSC::Handle<T>::get() const [with T = JSC::Structure, JSC::Handle<T>::ExternalType = JSC::Structure*]'_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 2 lines
Regards, Neil | Kai Kang
Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
Quoted 6 lines · Kang Kai
Quoted 3 lines
Instead of penalising everybody, would it be possible to make this change dependent on the build machine (ram + swap)? Or to only add -Wl,--no-keep-memory if the build failed without it?
Yes, it doesn't fail on every build machine. But it seems that build fails randomly on different build machines.
I'm in agreement with the others - this is how individual build machine admins make a webkit-gtk build work on their machine if it doesn't have enough memory available. We should probably document how to do this in local.conf in the recipe for reference, but not make it a default.
Ross
Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
Quoted 5 lines · Ross
Quoted 4 lines · Kang Kai
Quoted 3 lines
Instead of penalising everybody, would it be possible to make this change dependent on the build machine (ram + swap)? Or to only add -Wl,--no-keep-memory if the build failed without it?
Quoted 9 lines
Quoted 2 lines
Yes, it doesn't fail on every build machine. But it seems that build fails randomly on different build machines.
I'm in agreement with the others - this is how individual build machine admins make a webkit-gtk build work on their machine if it doesn't have enough memory available. We should probably document how to do this in local.conf in the recipe for reference, but not make it a default.
Perhaps create a bbclass for this that can be globally inherited in local.conf? I ran into a similar thing with gnuradio and had to drop PARALLEL_MAKE to get the build to complete with my amount of RAM+swap. Something like "lean-build.bbclass" that handles known issues for systems with limited memory.
Would variables like LDFLAGS_webkit-gtk and PARALLEL_MAKE_gnuradio be applied to the appropriate recipes if they were set in local.conf or a globally inherited bbclass, or am I misunderstanding how bitbake parses variables?
--
Paul Barker
Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
Quoted 4 lines · Paul Barker
Would variables like LDFLAGS_webkit-gtk and
PARALLEL_MAKE_gnuradiobe applied to the appropriate recipes if they were set in local.conf or a globally inherited bbclass, or am I misunderstanding how bitbake parses variables?
LDFLAGS_pn-webkit-gtk and PARALLEL_MAKE_pn-gnuradio, yes. Any of that stuff can go in site.conf and/or local.conf, and this is where this sort of build-environment-specific customisation belongs.
p.
Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
Quoted 9 lines · Phil Blundell
Quoted 4 lines · Paul Barker
Would variables like LDFLAGS_webkit-gtk and
PARALLEL_MAKE_gnuradiobe applied to the appropriate recipes if they were set in local.conf or a globally inherited bbclass, or am I misunderstanding how bitbake parses variables?LDFLAGS_pn-webkit-gtk and PARALLEL_MAKE_pn-gnuradio, yes. Any of that stuff can go in site.conf and/or local.conf, and this is where this sort of build-environment-specific customisation belongs.
I disagree.
The default behaviour should be that the system builds as long as the hosts has some minimum RAM. If this means that we pay a < ~5% build time penalty, we should not flinch. I'm not sure how much extra time I'm willing to accept but for a given package even a factor of two could be acceptable as long as the overall build did not take twice as long.
Kai, when you have a chance to test this, can you tell us what sort of build time differences we are talking about for the full package as well as just for the link phase if that's easy to extract? You could use /usr/bin/time to get the cpu and memory usage.
Are we really expecting all system builders to track all these limitations and put them in place on low-end machines? I realize that we need, and most of us have, a beefy box for builds but a few people are still trying to build using 32 bit, 4 GB systems and many people likely expect that 8 GB of RAM is plenty, in fact the system where this problem occurred had 8 GB and was running with:
BB_NUMBER_THREADS ?= "2"
PARALLEL_MAKE ?= "-j 4"
so there shouldn't have been much memory pressure.
Do we have minimum build system specs?
Regardless of what default behaviour we pick, switching a build to be tuned for a high/low end machine should be easy to do at set-up time as Paul suggested. Embedding such limitations in individual recipes and activating them by setting a single variable in local.conf is appealing.
// Randy
Quoted 9 lines
p.
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 2 lines
# Randy MacLeod. SMTS, Linux, Wind River Direct: 613.963.1350
Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
In reply to André DraszikQuoted 5 lines · André Draszik
Hi,
Instead of penalising everybody, would it be possible to make this change dependent on the build machine (ram + swap)? Or to only add -Wl,--no-keep-memory if the build failed without it?
penalty is not as much when you do parallel builds. However it will be good to measure how much extra built time is needed when this options is active. if its regressing the build time considerably then we need to make a choice.
Quoted 45 lines
Not sure if it would make any difference, but do you see this only with full debug enabled (-ggdb)? If yes, can the debug flags be changed, or that linker flag only be added, if full debug is enabled?
Cheers,
Andre'Quoted 27 lines · Kai Kang
Build webkit-gtk occupies lot of memory and fails on some host:
| ...tmp/sysroots/i686-linux/usr/libexec/x86_64-wrs-linux/gcc/x86_64-wrs-linux/4.8.1/ld:
failed to set dynamic section sizes: Memory exhausted
| collect2: error: ld returned 1 exit status
| make[1]: *** [libwebkitgtk-1.0.la] Error 1Add ld option to fix it.
Refer to:
http://comments.gmane.org/gmane.os.opendarwin.webkit.devel/20078Signed-off-by Kai Kang <[email protected]>---
meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb | 2 ++1 file changed, 2 insertions(+)meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb +0
-0@@ -60,6 +60,8 @@ CPPFLAGS_append_powerpc = " -I${STAGING_INCDIR}/pango-1.0 \
EXTRA_AUTORECONF= " -I Source/autotools "meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb +2
-0+# Fix compile error: "ld: failed to set dynamic section sizes: Memory exhausted" +LDFLAGS += " -Wl,--no-keep-memory"Quoted 2 lines
#|
./Source/JavaScriptCore/heap/HandleTypes.h: In static member function 'static T* JSC::HandleTypes<T>::getFromSlot(JSC::HandleSlot) [with T =JSC::Structure, JSC::HandleTypes<T>::ExternalType = JSC::Structure*,JSC::HandleSlot= JSC::JSValue*]':
#|./Source/JavaScriptCore/heap/Handle.h:141:79: instantiated from 'JSC::Handle<T>::ExternalType JSC::Handle<T>::get() const [with T =JSC::Structure, JSC::Handle<T>::ExternalType = JSC::Structure*]'_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core