Skip to contentSkip to conversation

webkitgtk: Use linker options to reduce memory overhead during linking

PATCH In review Khem Raj 3 participants 5 messages Mar 8, 2021 – Mar 9, 2021

lore
Share & tools
lore

Khem Raj

author
#0 lore raw

PATCH 1/1 webkitgtk: Use linker options to reduce memory overhead during linking

Helps builds on machines with less RAM ( 8G )

---
 .../webkitgtk/reduce-memory-overheads.patch   | 22 +++++++++++++++++++
 meta/recipes-sato/webkit/webkitgtk_2.30.5.bb  |  1 +
 2 files changed, 23 insertions(+)
 create mode 100644 meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch
meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch new file +22 -0
new file mode 100644
index 0000000000..0493150a91
@@ -0,0 +1,22 @@
+From: Alberto Garcia <[email protected]>
+Subject: Reduce memory usage when not using the Gold linker
+Bug-Debian: https://bugs.debian.org/949621
+Forwarded: no
+
+Upstream-Status: Pending
+Signed-off-by: Khem Raj <[email protected]>
+--- a/Source/cmake/OptionsCommon.cmake
++++ b/Source/cmake/OptionsCommon.cmake
+@@ -95,6 +95,12 @@ option(GCC_OFFLINEASM_SOURCE_MAP
+   "Produce debug line information for offlineasm-generated code"
+   ${GCC_OFFLINEASM_SOURCE_MAP_DEFAULT})
+ 
++# Pass --reduce-memory-overheads to the bfd linker in order to save memory
++if (NOT USE_LD_GOLD)
++    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--reduce-memory-overheads")
++    set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--reduce-memory-overheads")
++endif ()
++
+ # Enable the usage of OpenMP.
+ #  - At this moment, OpenMP is only used as an alternative implementation
+ #    to native threads for the parallelization of the SVG filters.
meta/recipes-sato/webkit/webkitgtk_2.30.5.bb +1 -0
index 2928b8e55b..a1dd76ce81 100644
@@ -18,6 +18,7 @@ SRC_URI = "https://www.webkitgtk.org/releases/${BPN}-${PV}.tar.xz \
            file://0001-Fix-build-with-musl.patch \
            file://include_array.patch \
            file://include_xutil.patch \
+           file://reduce-memory-overheads.patch \
            "
 SRC_URI[sha256sum] = "7d0dab08e3c5ae07bec80b2822ef42e952765d5724cac86eb23999bfed5a7f1f"
 
Signature · 1 line
2.30.1
#1 lore raw

Can you please explain the context for this? Why would we want to support building webkit on such constrained machines? 8g of ram means they can’t run more than 4-6 gcc processes, so webkit builds will be unbearably slow anyway.

Alex

Quoted 65 lines · Khem Raj

Helps builds on machines with less RAM ( 8G )

---
 .../webkitgtk/reduce-memory-overheads.patch   | 22 +++++++++++++++++++
 meta/recipes-sato/webkit/webkitgtk_2.30.5.bb  |  1 +
 2 files changed, 23 insertions(+)
 create mode 100644

meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch

diff --git a/meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch b/meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch new file mode 100644
index 0000000000..0493150a91

meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch new file +14 -0
@@ -0,0 +1,22 @@
+From: Alberto Garcia <[email protected]>
+Subject: Reduce memory usage when not using the Gold linker
+Bug-Debian: https://bugs.debian.org/949621
+Forwarded: no
+
+Upstream-Status: Pending
+Signed-off-by: Khem Raj <[email protected]>
+--- a/Source/cmake/OptionsCommon.cmake
++++ b/Source/cmake/OptionsCommon.cmake
+@@ -95,6 +95,12 @@ option(GCC_OFFLINEASM_SOURCE_MAP
+   "Produce debug line information for offlineasm-generated code"
+   ${GCC_OFFLINEASM_SOURCE_MAP_DEFAULT})
+
++# Pass --reduce-memory-overheads to the bfd linker in order to save

memory
++if (NOT USE_LD_GOLD)
++ set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}

meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch new file +6 -2
-Wl,--reduce-memory-overheads")
++    set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}
-Wl,--reduce-memory-overheads")
++endif ()
++
+ # Enable the usage of OpenMP.
+ #  - At this moment, OpenMP is only used as an alternative implementation
+ #    to native threads for the parallelization of the SVG filters.
meta/recipes-sato/webkit/webkitgtk_2.30.5.bb +1 -0
index 2928b8e55b..a1dd76ce81 100644
@@ -18,6 +18,7 @@ SRC_URI = "
 https://www.webkitgtk.org/releases/${BPN}-${PV}.tar.xz \
            file://0001-Fix-build-with-musl.patch \
            file://include_array.patch \
            file://include_xutil.patch \
+           file://reduce-memory-overheads.patch \
            "
 SRC_URI[sha256sum] =

"7d0dab08e3c5ae07bec80b2822ef42e952765d5724cac86eb23999bfed5a7f1f"

--
2.30.1

author
#2 lore raw
Quoted 3 lines · Alexander Kanavin

Can you please explain the context for this? Why would we want to support building webkit on such constrained machines? 8g of ram means they can’t run more than 4-6 gcc processes, so webkit builds will be unbearably slow anyway.

this helps to avoid unwanted build breaks since gcc crashes when memory pressure is too high,
when doing parallel builds this can happen very often since there are other tasks running in parallel, so these situations can arise often we have seen this happen on machines with 16GB RAM therefore saving some of the memory during linking helps reduce these breakages. they can still happen if say machine was doing more than 1 builds in parallel but at least tools have done what they could.

Quoted 59 lines

Alex

Quoted 56 lines · Khem Raj

Helps builds on machines with less RAM ( 8G )

---
 .../webkitgtk/reduce-memory-overheads.patch   | 22 +++++++++++++++++++
 meta/recipes-sato/webkit/webkitgtk_2.30.5.bb  |  1 +
 2 files changed, 23 insertions(+)
 create mode 100644 meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch
meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch new file +22 -0
new file mode 100644
index 0000000000..0493150a91
@@ -0,0 +1,22 @@
+From: Alberto Garcia <[email protected]>
+Subject: Reduce memory usage when not using the Gold linker
+Bug-Debian: https://bugs.debian.org/949621
+Forwarded: no
+
+Upstream-Status: Pending
+Signed-off-by: Khem Raj <[email protected]>
+--- a/Source/cmake/OptionsCommon.cmake
++++ b/Source/cmake/OptionsCommon.cmake
+@@ -95,6 +95,12 @@ option(GCC_OFFLINEASM_SOURCE_MAP
+   "Produce debug line information for offlineasm-generated code"
+   ${GCC_OFFLINEASM_SOURCE_MAP_DEFAULT})
+
++# Pass --reduce-memory-overheads to the bfd linker in order to save memory
++if (NOT USE_LD_GOLD)
++    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--reduce-memory-overheads")
++    set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--reduce-memory-overheads")
++endif ()
++
+ # Enable the usage of OpenMP.
+ #  - At this moment, OpenMP is only used as an alternative implementation
+ #    to native threads for the parallelization of the SVG filters.
meta/recipes-sato/webkit/webkitgtk_2.30.5.bb +1 -1
index 2928b8e55b..a1dd76ce81 100644
@@ -18,6 +18,7 @@ SRC_URI = "https://www.webkitgtk.org/releases/${BPN}-${PV}.tar.xz \
            file://0001-Fix-build-with-musl.patch \
            file://include_array.patch \
            file://include_xutil.patch \
+           file://reduce-memory-overheads.patch \
            "
 SRC_URI[sha256sum] = "7d0dab08e3c5ae07bec80b2822ef42e952765d5724cac86eb23999bfed5a7f1f"
 
--

2.30.1

#3 lore raw

Thanks Khem, this makes sense.

Alex

Quoted 88 lines · Khem Raj
Quoted 2 lines · Alexander Kanavin

Can you please explain the context for this? Why would we want to

support building webkit on such constrained machines? 8g of ram means they can’t run more than 4-6 gcc processes, so webkit builds will be unbearably slow anyway.

Quoted 1 line

this helps to avoid unwanted build breaks since gcc crashes when memory pressure is too high,
when doing parallel builds this can happen very often since there are other tasks running in parallel, so these situations can arise often we have seen this happen on machines with 16GB RAM therefore saving some of the memory during linking helps reduce these breakages. they can still happen if say machine was doing more than 1 builds in parallel but at least tools have done what they could.

Quoted 12 lines

Alex

Quoted 9 lines · Khem Raj

Helps builds on machines with less RAM ( 8G )

---
 .../webkitgtk/reduce-memory-overheads.patch   | 22 +++++++++++++++++++
 meta/recipes-sato/webkit/webkitgtk_2.30.5.bb  |  1 +
 2 files changed, 23 insertions(+)
 create mode 100644

meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch

Quoted 2 lines
Quoted 2 lines

diff --git

a/meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch b/meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch

Quoted 11 lines
Quoted 11 lines

new file mode 100644
index 0000000000..0493150a91

meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch new file +6 -0
@@ -0,0 +1,22 @@
+From: Alberto Garcia <[email protected]>
+Subject: Reduce memory usage when not using the Gold linker
+Bug-Debian: https://bugs.debian.org/949621
+Forwarded: no
+
+Upstream-Status: Pending
Quoted 8 lines

meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch

Quoted 8 lines
meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch new file +8 -0
+Signed-off-by: Khem Raj <[email protected]>
+--- a/Source/cmake/OptionsCommon.cmake
++++ b/Source/cmake/OptionsCommon.cmake
+@@ -95,6 +95,12 @@ option(GCC_OFFLINEASM_SOURCE_MAP
+   "Produce debug line information for offlineasm-generated code"
+   ${GCC_OFFLINEASM_SOURCE_MAP_DEFAULT})
+
++# Pass --reduce-memory-overheads to the bfd linker in order to save

memory

Quoted 2 lines

meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch line 16

Quoted 2 lines
meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch new file +2 -0
++if (NOT USE_LD_GOLD)
++    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}

-Wl,--reduce-memory-overheads")

Quoted 1 line

meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch line 17

Quoted 1 line
meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch new file +1 -0
++    set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}

-Wl,--reduce-memory-overheads")

Quoted 4 lines

meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch

Quoted 4 lines
meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch new file +4 -0
++endif ()
++
+ # Enable the usage of OpenMP.
+ #  - At this moment, OpenMP is only used as an alternative

implementation

Quoted 2 lines

Review

Quoted 2 lines
meta/recipes-sato/webkit/webkitgtk/reduce-memory-overheads.patch new file +1 -0
+ #    to native threads for the parallelization of the SVG filters.
(quoted diff) +0 -0

b/meta/recipes-sato/webkit/webkitgtk_2.30.5.bb

Quoted 4 lines

meta/recipes-sato/webkit/webkitgtk_2.30.5.bb SRC_URI = "

Quoted 4 lines

index 2928b8e55b..a1dd76ce81 100644

meta/recipes-sato/webkit/webkitgtk_2.30.5.bb +0 -0
@@ -18,6 +18,7 @@ SRC_URI = "

https://www.webkitgtk.org/releases/${BPN}-${PV}.tar.xz \

Quoted 6 lines

meta/recipes-sato/webkit/webkitgtk_2.30.5.bb SRC_URI = "https://www.webkitgtk.org/releases/${BPN}-${PV}.tar.xz \

Quoted 6 lines
meta/recipes-sato/webkit/webkitgtk_2.30.5.bb +1 -0
            file://0001-Fix-build-with-musl.patch \
            file://include_array.patch \
            file://include_xutil.patch \
+           file://reduce-memory-overheads.patch \
            "
 SRC_URI[sha256sum] =

"7d0dab08e3c5ae07bec80b2822ef42e952765d5724cac86eb23999bfed5a7f1f"

Quoted 7 lines
Quoted 7 lines
meta/recipes-sato/webkit/webkitgtk_2.30.5.bb +0 -1
--

2.30.1

#4 lore raw
Quoted 5 lines · Alexander Kanavin

Thanks Khem, this makes sense.

Alex

webkitgtk is one of the top swappers for my (very old) machines - would be cool if this has an end

Andreas