Skip to contentSkip to conversation

webkitgtk: limit the number of compilation threads

PATCH In review Thomas Perrot 4 participants 7 messages May 20, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 webkitgtk: limit the number of compilation threads

Otherwise the compilation may lead to out-of-memories.

---
 meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +2 -1
index 65757c36a733..e7cfe49e2b87 100644
@@ -80,6 +80,8 @@ setup_python_link() {
 	fi
 }
 
+PARALLEL_MAKE = ""
+
 EXTRA_OECMAKE = " \
 		-DPORT=GTK \
 		${@bb.utils.contains('GI_DATA_ENABLED', 'True', '-DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \
@@ -164,4 +166,3 @@ src_package_preprocess () {
             ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h
 
 }
-
Signature · 1 line
2.35.3
#1 lore raw
Quoted 22 lines · Thomas Perrot via lists.openembedded.org

Otherwise the compilation may lead to out-of-memories.

---
  meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +2 -0
index 65757c36a733..e7cfe49e2b87 100644
@@ -80,6 +80,8 @@ setup_python_link() {
  	fi
  }
 
+PARALLEL_MAKE = ""
+
  EXTRA_OECMAKE = " \
  		-DPORT=GTK \
  		${@bb.utils.contains('GI_DATA_ENABLED', 'True', '-DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \

meta/recipes-sato/webkit/webkitgtk_2.36.1.bb src_package_preprocess () { · line 167

Quoted 4 lines
meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +0 -1
@@ -164,4 +166,3 @@ src_package_preprocess () {
              ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h
 
  }
-

Why punish build machines with lots of RAM ? Maybe it could be set depending on available RAM/number of cores ?

Jacob

#2 lore raw

The correct way to address this is to restrict the amount of webkit make jobs in site.conf, specific to the build host. Or add RAM.

Ale

Quoted 46 lines · Jacob Kroon

Review

Quoted 8 lines · Thomas Perrot via lists.openembedded.org

Otherwise the compilation may lead to out-of-memories.

---
  meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb

meta/recipes-sato/webkit/webkitgtk_2.36.1.bb setup_python_link() { · line 87

Quoted 12 lines

index 65757c36a733..e7cfe49e2b87 100644

meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +2 -0
@@ -80,6 +80,8 @@ setup_python_link() {
      fi
  }
 
+PARALLEL_MAKE = ""
+
  EXTRA_OECMAKE = " \
              -DPORT=GTK \
              ${@bb.utils.contains('GI_DATA_ENABLED', 'True',

'-DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \

meta/recipes-sato/webkit/webkitgtk_2.36.1.bb src_package_preprocess () { · line 167

Quoted 6 lines
meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +0 -1
@@ -164,4 +166,3 @@ src_package_preprocess () {
              ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h
 
  }
-

Why punish build machines with lots of RAM ? Maybe it could be set depending on available RAM/number of cores ?

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#165928):
https://lists.openembedded.org/g/openembedded-core/message/165928
Mute This Topic: https://lists.openembedded.org/mt/91229473/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#3 lore raw

Hi Alexander,

Quoted 3 lines · Alexander Kanavin

The correct way to address this is to restrict the amount of webkit make jobs in site.conf, specific to the build host. Or add RAM.

I heard you

Kind regards,
Thomas Perrot

Quoted 53 lines

Ale

Quoted 33 lines · Jacob Kroon

Review

Quoted 8 lines · Thomas Perrot via lists.openembedded.org

Otherwise the compilation may lead to out-of-memories.

---
   meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++-

   1 file changed, 2 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb

Quoted 9 lines

index 65757c36a733..e7cfe49e2b87 100644

meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +0 -0
@@ -80,6 +80,8 @@ setup_python_link() {

       fi
   }

meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +2 -0
+PARALLEL_MAKE = ""
+
Quoted 3 lines

   EXTRA_OECMAKE = " \
               -DPORT=GTK \
               ${@bb.utils.contains('GI_DATA_ENABLED', 'True', '-

DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \

Quoted 6 lines
meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +0 -0
@@ -164,4 +166,3 @@ src_package_preprocess () {

               ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h
   }
-

Why punish build machines with lots of RAM ? Maybe it could be set depending on available RAM/number of cores ?

Quoted 4 lines

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#165929):
https://lists.openembedded.org/g/openembedded-core/message/165929
Mute This Topic: https://lists.openembedded.org/mt/91229473/5443093
Group Owner: [email protected] Unsubscribe:
https://lists.openembedded.org/g/openembedded-core/unsub [
[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 3 lines
Thomas Perrot, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
#4 lore raw
Quoted 8 lines · Thomas Perrot via lists.openembedded.org

Hi Alexander,

Quoted 3 lines · Alexander Kanavin

The correct way to address this is to restrict the amount of webkit make jobs in site.conf, specific to the build host. Or add RAM.

I heard you

I do agree with Alex and Jacob here. Whilst this may not work on your system, the same could be said for other recipes too. If you're going to run builds on resource constrained systems, you will need to configure the parallelism options accordingly and setting limits on webkit isn't unusual. We do set load based limits on paralellism on the autobuilder for example.

Cheers,

Richard

#5 lore raw
In reply to Jacob Kroon

Hello Jacob,

Quoted 24 lines · Jacob Kroon
Quoted 23 lines · Thomas Perrot via lists.openembedded.org

Otherwise the compilation may lead to out-of-memories.

---
  meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++-

  1 file changed, 2 insertions(+), 1 deletion(-)
meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +0 -0
index 65757c36a733..e7cfe49e2b87 100644
@@ -80,6 +80,8 @@ setup_python_link() {

        fi
  }

meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +2 -0
+PARALLEL_MAKE = ""
+

  EXTRA_OECMAKE = " \
                -DPORT=GTK \
                ${@bb.utils.contains('GI_DATA_ENABLED', 'True', '- DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \

meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +0 -0
@@ -164,4 +166,3 @@ src_package_preprocess () {
Quoted 9 lines
Quoted 5 lines

              ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h
  }
-

Why punish build machines with lots of RAM ? Maybe it could be set depending on available RAM/number of cores ?

Why not.

I tested on a build server with more than ~12GB ram available and 16 cores, then I tried to reduce the number of threads and when the value is greater than 1 the build fails.

Kind regards,
Thomas Perrot

Quoted 13 lines

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#165928):
https://lists.openembedded.org/g/openembedded-core/message/165928
Mute This Topic: https://lists.openembedded.org/mt/91229473/5443093
Group Owner: [email protected] Unsubscribe:
https://lists.openembedded.org/g/openembedded-core/unsub [
[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 3 lines
Thomas Perrot, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
#6 lore raw
Quoted 37 lines · Thomas Perrot

Hello Jacob,

Quoted 26 lines · Jacob Kroon
Quoted 25 lines · Thomas Perrot via lists.openembedded.org

Otherwise the compilation may lead to out-of-memories.

---
   meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++-
   1 file changed, 2 insertions(+), 1 deletion(-)
meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +2 -0
index 65757c36a733..e7cfe49e2b87 100644
@@ -80,6 +80,8 @@ setup_python_link() {
         fi
   }
 
+PARALLEL_MAKE = ""
+
   EXTRA_OECMAKE = " \
                 -DPORT=GTK \
                 ${@bb.utils.contains('GI_DATA_ENABLED', 'True', '-

DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \

meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +0 -0
@@ -164,4 +166,3 @@ src_package_preprocess () {
               ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h
 
Quoted 7 lines

meta/recipes-sato/webkit/webkitgtk_2.36.1.bb src_package_preprocess () { · line 167

Quoted 3 lines
meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +0 -1
   }
-

Why punish build machines with lots of RAM ? Maybe it could be set depending on available RAM/number of cores ?

Quoted 2 lines

Why not.

Because it slows down the build for them, which is not good.

Quoted 4 lines

I tested on a build server with more than ~12GB ram available and 16 cores, then I tried to reduce the number of threads and when the value is greater than 1 the build fails.

12GB RAM does not sound like much for a 16 core system.

Jacob