qemu needs > 2 GB of RAM to build?
Hi all,
I'm testing building core-image-minimal in a resource-constrained environment (a VirtualBox VM with 2300 MB of RAM allocated to it). With PARALLEL_MAKE set to -j4 (the VM does have two CPUs allocated), I'm finding that the build of qemu-native fails because the OOM killer steps in and kills gcc. This is happening during the linking phase of building qemu.
If this is true, I assume it would pretty much mean that no one can build our images without more than 2 GB of RAM. Is this true? Is this something for us to be concerned about?
Scott
Signature · 3 lines
Scott Garman Embedded Linux Engineer - Yocto Project Intel Open Source Technology Center
Quoted 8 lines · Scott Garman
Hi all,
I'm testing building core-image-minimal in a resource-constrained environment (a VirtualBox VM with 2300 MB of RAM allocated to it). With
PARALLEL_MAKEset to -j4 (the VM does have two CPUs allocated), I'm finding that the build of qemu-native fails because the OOM killer steps in and kills gcc. This is happening during the linking phase of building qemu.
what distro are you running on guest ? it could be something wrong with the distro gcc or system
Quoted 3 lines
If this is true, I assume it would pretty much mean that no one can build our images without more than 2 GB of RAM. Is this true? Is this something for us to be concerned about?
I have not seen such an error on a system which has 2G RAM all builds qemu-native fine here
Quoted 13 lines
Scott
--
Scott Garman
Embedded Linux Engineer - Yocto Project
Intel Open Source Technology Center_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Quoted 8 lines · Khem Raj
Quoted 7 lines · Scott Garman
Hi all,
I'm testing building core-image-minimal in a resource-constrained environment (a VirtualBox VM with 2300 MB of RAM allocated to it). With
PARALLEL_MAKEset to -j4 (the VM does have two CPUs allocated), I'm finding that the build of qemu-native fails because the OOM killer steps in and kills gcc. This is happening during the linking phase of building qemu.
What's BB_NUMBER_THREADS set to? I had to reduce to 1 to build qemu-native on a laptop with 2GB RAM. iirc it had a habit of linking something equally large such as eglibc, qt or the kernel at the same time.
Quoted 2 lines
what distro are you running on guest ? it could be something wrong with the distro gcc or system
I've seen this on F14 (Gnome 2.x) and F15 (Gnome 3) with a laptop with only 2GB RAM.
Quoted 7 lines
Quoted 4 lines
If this is true, I assume it would pretty much mean that no one can build our images without more than 2 GB of RAM. Is this true? Is this something for us to be concerned about?
I have not seen such an error on a system which has 2G RAM all builds qemu-native fine here
I think BB_NUMBER_THREADS is the key here. And whether you're running much in the way of a desktop environment.
Joshua
Signature · 3 lines
Joshua Lock
Yocto Project "Johannes factotum"
Intel Open Source Technology CentreQuoted 12 lines · Joshua Lock
Quoted 8 lines · Khem Raj
Quoted 7 lines · Scott Garman
Hi all,
I'm testing building core-image-minimal in a resource-constrained environment (a VirtualBox VM with 2300 MB of RAM allocated to it). With
PARALLEL_MAKEset to -j4 (the VM does have two CPUs allocated), I'm finding that the build of qemu-native fails because the OOM killer steps in and kills gcc. This is happening during the linking phase of building qemu.
Quoted 3 lines
What's
BB_NUMBER_THREADSset to? I had to reduce to 1 to build qemu-native on a laptop with 2GB RAM. iirc it had a habit of linking something equally large such as eglibc, qt or the kernel at the same time.
I had it set to 2, but I'm able to reproduce the problem when rebuilding *only* qemu-native.
Quoted 5 lines
Quoted 2 lines
what distro are you running on guest ? it could be something wrong with the distro gcc or system
I've seen this on F14 (Gnome 2.x) and F15 (Gnome 3) with a laptop with only 2GB RAM.
This is a Fedora 14 host running GNOME 2.x
Quoted 2 lines
I think
BB_NUMBER_THREADSis the key here. And whether you're running much in the way of a desktop environment.
I would have thought PARALLEL_MAKE would have been more of the issue - I think I saw two instances of ld running via top when the crash occurred.
In any case, the purpose behind this exercise for me is to test the feasibility of using a VM environment for performing builds. The idea is to get Windows users started with some tutorial screencasts they can follow along with using the VM until they feel compelled enough to set up a native Linux development system.
It sounds like a system requirement of using this VM will have to be that the host system have 6 GB or more of RAM (since the VM guest won't be able to use all of that - generally around 50-60%). This pretty much rules out its use on all but the most recent laptops.
Scott
Signature · 3 lines
Scott Garman Embedded Linux Engineer - Yocto Project Intel Open Source Technology Center
Quoted 29 lines · Scott Garman
Quoted 14 lines · Joshua Lock
Quoted 10 lines · Khem Raj
Quoted 8 lines · Scott Garman
Hi all,
I'm testing building core-image-minimal in a resource-constrained environment (a VirtualBox VM with 2300 MB of RAM allocated to it). With
PARALLEL_MAKEset to -j4 (the VM does have two CPUs allocated), I'm finding
that the build of qemu-native fails because the OOM killer steps in and kills gcc. This is happening during the linking phase of building qemu.Quoted 4 lines
What's
BB_NUMBER_THREADSset to? I had to reduce to 1 to build qemu-native on a laptop with 2GB RAM. iirc it had a habit of linking something equally large such as eglibc, qt or the kernel at the same time.I had it set to 2, but I'm able to reproduce the problem when rebuilding *only* qemu-native.
Quoted 5 lines
Quoted 2 lines
what distro are you running on guest ? it could be something wrong with the distro gcc or system
I've seen this on F14 (Gnome 2.x) and F15 (Gnome 3) with a laptop with only 2GB RAM.
Quoted 7 lines
This is a Fedora 14 host running GNOME 2.x
Quoted 2 lines
I think
BB_NUMBER_THREADSis the key here. And whether you're running much in the way of a desktop environment.I would have thought
PARALLEL_MAKEwould have been more of the issue - I think I saw two instances of ld running via top when the crash occurred.
Both doing linking for qemu-native? I thought I remembered one giant link phase but it's been a while.
Quoted 11 lines
In any case, the purpose behind this exercise for me is to test the feasibility of using a VM environment for performing builds. The idea is to get Windows users started with some tutorial screencasts they can follow along with using the VM until they feel compelled enough to set up a native Linux development system.
It sounds like a system requirement of using this VM will have to be that the host system have 6 GB or more of RAM (since the VM guest won't be able to use all of that - generally around 50-60%). This pretty much rules out its use on all but the most recent laptops.
Can you not include some canny sstate packages in the VM image? i.e qemu-native?
Or work with qemu upstream to re-work the link phase?
Or both. One short-term and one long-term :-)
Cheers,
Joshua
Signature · 3 lines
Joshua Lock
Yocto Project "Johannes factotum"
Intel Open Source Technology CentreQuoted 40 lines · Scott Garman
Quoted 12 lines · Joshua Lock
Quoted 8 lines · Khem Raj
Quoted 7 lines · Scott Garman
Hi all,
I'm testing building core-image-minimal in a resource-constrained environment (a VirtualBox VM with 2300 MB of RAM allocated to it). With
PARALLEL_MAKEset to -j4 (the VM does have two CPUs allocated), I'm finding that the build of qemu-native fails because the OOM killer steps in and kills gcc. This is happening during the linking phase of building qemu.Quoted 3 lines
What's
BB_NUMBER_THREADSset to? I had to reduce to 1 to build qemu-native on a laptop with 2GB RAM. iirc it had a habit of linking something equally large such as eglibc, qt or the kernel at the same time.I had it set to 2, but I'm able to reproduce the problem when rebuilding *only* qemu-native.
Quoted 5 lines
Quoted 2 lines
what distro are you running on guest ? it could be something wrong with the distro gcc or system
I've seen this on F14 (Gnome 2.x) and F15 (Gnome 3) with a laptop with only 2GB RAM.
This is a Fedora 14 host running GNOME 2.x
Quoted 2 lines
I think
BB_NUMBER_THREADSis the key here. And whether you're running much in the way of a desktop environment.I would have thought
PARALLEL_MAKEwould have been more of the issue - I think I saw two instances of ld running via top when the crash occurred.In any case, the purpose behind this exercise for me is to test the feasibility of using a VM environment for performing builds. The idea is to get Windows users started with some tutorial screencasts they can follow along with using the VM until they feel compelled enough to set up a native Linux development system.
Quoted 4 lines
It sounds like a system requirement of using this VM will have to be that the host system have 6 GB or more of RAM (since the VM guest won't be able to use all of that - generally around 50-60%). This pretty much rules out its use on all but the most recent laptops.
Why not enable more swap? It's slow but works for short peaks in mem usage. I have to extend our swap from 2G to 4G on VM with 2G RAM for some builds.. also needed during linking unstripped libs :/.
Regards,
Quoted 12 lines
Scott
Signature · 8 lines
Scott Garman Embedded Linux Engineer - Yocto Project Intel Open Source Technology Center _______________________________________________ Openembedded-core mailing list [email protected] http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
Quoted 3 lines · Martin Jansa
Why not enable more swap? It's slow but works for short peaks in mem usage. I have to extend our swap from 2G to 4G on VM with 2G RAM for some builds.. also needed during linking unstripped libs :/.
Thank you, Marin - this is an excellent idea. :)
Scott
Signature · 3 lines
Scott Garman Embedded Linux Engineer - Yocto Project Intel Open Source Technology Center
I have encountered the same problem on archlinux in VM. after googled it, maybe it is GCC's defect
Quoted 24 lines · Scott Garman
Hi all,
I'm testing building core-image-minimal in a resource-constrained environment (a VirtualBox VM with 2300 MB of RAM allocated to it). With
PARALLEL_MAKEset to -j4 (the VM does have two CPUs allocated), I'm finding that the build of qemu-native fails because the OOM killer steps in and kills gcc. This is happening during the linking phase of building qemu.If this is true, I assume it would pretty much mean that no one can build our images without more than 2 GB of RAM. Is this true? Is this something for us to be concerned about?
Scott
--
Scott Garman
Embedded Linux Engineer - Yocto Project
Intel Open Source Technology Center_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Signature · 3 lines
Yi Qingliang [email protected] https://niqingliang2003.wordpress.com
Quoted 8 lines · Scott Garman
Hi all,
I'm testing building core-image-minimal in a resource-constrained environment (a VirtualBox VM with 2300 MB of RAM allocated to it). With
PARALLEL_MAKEset to -j4 (the VM does have two CPUs allocated), I'm finding that the build of qemu-native fails because the OOM killer steps in and kills gcc. This is happening during the linking phase of building qemu.
Quoted 4 lines
If this is true, I assume it would pretty much mean that no one can build our images without more than 2 GB of RAM. Is this true? Is this something for us to be concerned about?
I think this could be because of gcc bug
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48190
that bug has now been fixed and backported to gcc-4.5 and gcc 4.6 branches as well
however this fix has to go into the gcc in distro that you are running on build host
we will get this fix into oe-core gcc as well but that may not help with qemu-native
-Khem
Do you mean it need the native-gcc on host to fix this bug?
Quoted 29 lines · Khem Raj
Quoted 8 lines · Scott Garman
Hi all,
I'm testing building core-image-minimal in a resource-constrained environment (a VirtualBox VM with 2300 MB of RAM allocated to it). With
PARALLEL_MAKEset to -j4 (the VM does have two CPUs allocated), I'm finding that the build of qemu-native fails because the OOM killer steps in and kills gcc. This is happening during the linking phase of building qemu.Quoted 4 lines
If this is true, I assume it would pretty much mean that no one can build our images without more than 2 GB of RAM. Is this true? Is this something for us to be concerned about?
I think this could be because of gcc bug
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48190
that bug has now been fixed and backported to gcc-4.5 and gcc 4.6 branches as well
however this fix has to go into the gcc in distro that you are running on build host
we will get this fix into oe-core gcc as well but that may not help with qemu-native-Khem
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Signature · 3 lines
Yi Qingliang [email protected] https://niqingliang2003.wordpress.com