Update WebKitGTK+ to 2.20.0

Hi everybody,

This patch series adds a new dependency needed by WebKitGTK+: woff. In turn, that depends on brotli, which the patch series adds as well.

I am adding myself as maintainer for those two new packages (brotli and woff2) in the DEVELOPERS file and I will be keeping an eye on their releases.

This series starts with the same three patches as the previous submission, and adds on top three additional commits to make WebKitGTK+ available on AArch64 (it's a platform we support upstream), and adds a couple of patches which are already merged in the upstream repository, but are not in the release and are worth applying.

Hopefully Patchwork will pick the series in full :-)

Regards,

Adrian Perez de Castro (4):
brotli: new package
woff2: new package
webkitgtk: update to version 2.20.0
webkitgtk: Add upstream patch for better memory monitoring

 DEVELOPERS                                         |  2 +
 package/Config.in                                  |  2 +
 package/brotli/Config.in                           | 12 +++
 package/brotli/brotli.hash                         |  5 ++
 package/brotli/brotli.mk                           | 17 ++++
 ...memory-monitor-thresholds-should-be-bette.patch | 93 ++++++++++++++++++++++
 ...ld-failure-when-ENABLE_VIDEO-ENABLE_WEB_A.patch | 32 ++++++++
 package/webkitgtk/Config.in                        |  3 +
 package/webkitgtk/webkitgtk.hash                   |  8 +-
 package/webkitgtk/webkitgtk.mk                     |  4 +-
 package/woff2/Config.in                            |  9 +++
 package/woff2/woff2.hash                           |  5 ++
 package/woff2/woff2.mk                             | 17 ++++
 13 files changed, 203 insertions(+), 6 deletions(-)
 create mode 100644 package/brotli/Config.in
 create mode 100644 package/brotli/brotli.hash
 create mode 100644 package/brotli/brotli.mk
 create mode 100644 package/webkitgtk/0001-WebProcess-memory-monitor-thresholds-should-be-bette.patch
 create mode 100644 package/webkitgtk/0002-WPE-GTK-Build-failure-when-ENABLE_VIDEO-ENABLE_WEB_A.patch
 create mode 100644 package/woff2/Config.in
 create mode 100644 package/woff2/woff2.hash
 create mode 100644 package/woff2/woff2.mk

2.16.2

Add the brotli package to Buildroot. This is needed by woff2, which in turn is needed by webkitgtk from version 2.20.0 onwards. WebKitGTK+ used to bundle a copy of the library, but it stopped doing so now that the upstream has started making releases.

---
 DEVELOPERS                 |  1 +
 package/Config.in          |  1 +
 package/brotli/Config.in   | 12 ++++++++++++
 package/brotli/brotli.hash |  5 +++++
 package/brotli/brotli.mk   | 17 +++++++++++++++++
 5 files changed, 36 insertions(+)
 create mode 100644 package/brotli/Config.in
 create mode 100644 package/brotli/brotli.hash
 create mode 100644 package/brotli/brotli.mk

index 1faefc45fc..2d5f2108a8 100644
@@ -62,6 +62,7 @@ F:	package/setools/
 F:	package/sngrep/
 
 N:	Adrian Perez de Castro <[email protected]>
+F:	package/brotli/
 F:	package/libepoxy/
 F:	package/webkitgtk/
 

index 58dfd6306d..26ded5ebb2 100644
@@ -63,6 +63,7 @@ menu "Audio and video applications"
 endmenu
 
 menu "Compressors and decompressors"
+	source "package/brotli/Config.in"
 	source "package/bzip2/Config.in"
 	source "package/gzip/Config.in"
 	source "package/lz4/Config.in"

new file mode 100644
index 0000000000..e40f5b6b9f
@@ -0,0 +1,12 @@
+config BR2_PACKAGE_BROTLI
+	bool "brotli"
+	help
+	  Generic-purpose lossless compression library. The algorithm
+	  compresses data using a combination of a modern variant of
+	  the LZ77 algorithm, Huffman coding and 2nd order context
+	  modeling, with a compression ratio comparable to the best
+	  currently available general-purpose compression methods. It
+	  is similar in speed with deflate but offers more dense
+	  compression.
+
+	  https://github.com/google/brotli

new file mode 100644
index 0000000000..7bdefcb7dd
@@ -0,0 +1,5 @@
+# Locally generated:
+sha512  93adcf437d730ac403e444285ac8aefbb2c8a6b5e1b064e8ee33684c067287a8159e0ee73d2217c167881e87da73fa494792d963a15508fd42b2ac4a5b52823c  v1.0.3.tar.gz
+
+# Hash for license files:
+sha512  16d7a62757f9854cd0ff9750eb0b374a1360deded0ba1646fde430f79a8f7c36a85f4884faaf1ea90db02537cd8d8c314cfcbcf492be25af24e1e77aa5919ebb  LICENSE

new file mode 100644
index 0000000000..cf1e811543
@@ -0,0 +1,17 @@
+################################################################################
+#
+# brotli
+#
+################################################################################
+
+BROTLI_VERSION = 1.0.3
+BROTLI_SOURCE = v$(BROTLI_VERSION).tar.gz
+BROTLI_SITE = https://github.com/google/brotli/archive
+BROTLI_LICENSE = MIT
+BROTLI_LICENSE_FILES = LICENSE
+BROTLI_INSTALL_STAGING = YES
+BROTLI_CONF_OPTS = \
+	-DBROTLI_DISABLE_TESTS=ON \
+	-DBROTLI_BUNDLED_MODE=OFF
+
+$(eval $(cmake-package))

2.16.2

Hello,

Brotli is detected by libcurl, and it causes a build failure, at least in some situations:

http://autobuild.buildroot.net/results/64b/64bc0dfe284206390ae0680b94c0876863a3c0f3/build-end.log

Could you have a look ?

Thanks!

Thomas

Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

Sure, from the looks of it the curl config scripts are not adding -lbrotlicommon when doing static builds, hence the missing symbols. I'll see what I can come up with, likely this will end up with a patch sent to curl upstream.

Now it's a bit late for me, so I'll continue with it tomorrow.

Thanks for the heads up!

--
Adri?n ?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180402/d4399987/attachment.asc>

Hi,

I already committed Baruch' fix:

https://git.buildroot.net/buildroot/commit/?id=e7d658e0291b78553bf4eca704f0d0202bcf031f

Bye, Peter Korsgaard

Oh, that's lovely ? thanks!

(Odd that the curl ?configure? script only uses ?pkg-config? when the flag ?--with-brotli? is passed explicitly... ?\_(?)_/?)

--
Adri?n ?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180403/963fcc3f/attachment.asc>

Add the woff2 package to Builroot. This is needed by webkitgtk from version 2.20.0 onwards. WebKitGTK+ used to bundle a copy of the library, but it stopped doing so now that the upstream is has been making releases.

---
 DEVELOPERS               |  1 +
 package/Config.in        |  1 +
 package/woff2/Config.in  |  9 +++++++++
 package/woff2/woff2.hash |  5 +++++
 package/woff2/woff2.mk   | 17 +++++++++++++++++
 5 files changed, 33 insertions(+)
 create mode 100644 package/woff2/Config.in
 create mode 100644 package/woff2/woff2.hash
 create mode 100644 package/woff2/woff2.mk

index 2d5f2108a8..031af3cdd0 100644
@@ -65,6 +65,7 @@ N:	Adrian Perez de Castro <[email protected]>
 F:	package/brotli/
 F:	package/libepoxy/
 F:	package/webkitgtk/
+F:	package/woff2/
 
 N:	Adrien Gallou?t <[email protected]>
 F:	package/glorytun/

index 26ded5ebb2..ae5a09bd5b 100644
@@ -1186,6 +1186,7 @@ menu "Graphics"
 	source "package/waylandpp/Config.in"
 	source "package/webkitgtk/Config.in"
 	source "package/webp/Config.in"
+	source "package/woff2/Config.in"
 	source "package/zbar/Config.in"
 	source "package/zxing-cpp/Config.in"
 endmenu

new file mode 100644
index 0000000000..48672570dd
@@ -0,0 +1,9 @@
+config BR2_PACKAGE_WOFF2
+	bool "woff2"
+	depends on BR2_INSTALL_LIBSTDCPP
+	select BR2_PACKAGE_BROTLI
+	help
+	  Reference implementation for the WOFF2 font file format,
+	  typically used for Web fonts
+
+	  https://github.com/google/woff2

new file mode 100644
index 0000000000..2e9b62bde1
@@ -0,0 +1,5 @@
+# Locally generated:
+sha512  c788bba1530aec463e755e901f9342f4b599e3a07f54645fef1dc388ab5d5c30625535e5dd38e9e792e04a640574baa50eeefb6b7338ab403755f4a4e0c3044d  v1.0.2.tar.gz
+
+# Hash for license files:
+sha512  16d7a62757f9854cd0ff9750eb0b374a1360deded0ba1646fde430f79a8f7c36a85f4884faaf1ea90db02537cd8d8c314cfcbcf492be25af24e1e77aa5919ebb  LICENSE

new file mode 100644
index 0000000000..23b88a5e9a
@@ -0,0 +1,17 @@
+################################################################################
+#
+# woff2
+#
+################################################################################
+
+WOFF2_VERSION = 1.0.2
+WOFF2_SOURCE = v$(WOFF2_VERSION).tar.gz
+WOFF2_SITE = https://github.com/google/woff2/archive
+WOFF2_LICENSE = MIT
+WOFF2_LICENSE_FILES = LICENSE
+WOFF2_INSTALL_STAGING = YES
+WOFF2_DEPENDENCIES = brotli
+WOFF2_CONF_OPTS = \
+	-DNOISY_LOGGING=OFF
+
+$(eval $(cmake-package))

2.16.2

Hello Adrian,

This package causes a build failure on m68k:

http://autobuild.buildroot.net/results/78d/78d25015ac464c985e5a550ce9a80d85080dc6f2/build-end.log

http://autobuild.buildroot.net/?reason=woff2-1.0.2

It's a compiler issue. It would be nice to investigate whether it has been fixed in newer versions of gcc, if a workaround or fix is possible.

Thanks!

Thomas

Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

Release notes:

    https://webkitgtk.org/2018/03/12/webkitgtk2.20.0-released.html

No corresponding WebKit Security Advisory (WSA) has been published for now.

This also adds a dependency on the woff2 package. In previous versions WebKitGTK+ bundled a known-to-work snapshot of its Git repository due to upstream not providing releases. The situation has changed and releases are now available, and the bundled copy of woff2 has been removed.

Last but not least, this patch selects BR2_PACKAGE_WEBP_DEMUX, which is now required.

---
Changes v1 -> v2:
- Marked as supported on AArch64, which is officially supported upstream.
- Add upstream patch for fixing build failure with multimedia/XSLT support disabled.

---
 ...ld-failure-when-ENABLE_VIDEO-ENABLE_WEB_A.patch | 32 ++++++++++++++++++++++
 package/webkitgtk/Config.in                        |  3 ++
 package/webkitgtk/webkitgtk.hash                   |  8 +++---
 package/webkitgtk/webkitgtk.mk                     |  4 +--
 4 files changed, 41 insertions(+), 6 deletions(-)
 create mode 100644 package/webkitgtk/0002-WPE-GTK-Build-failure-when-ENABLE_VIDEO-ENABLE_WEB_A.patch

new file mode 100644
index 0000000000..2c69fa89ae
@@ -0,0 +1,32 @@
+From ecfba9336863c2dad22c9ad1dfd1f6ea1801ccdb Mon Sep 17 00:00:00 2001
+From: "aperez at igalia.com"
+ <[email protected]@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
+Date: Thu, 22 Mar 2018 13:48:34 +0000
+Subject: [PATCH xserver 2/2] [WPE][GTK] Build failure when ENABLE_VIDEO,
+ ENABLE_WEB_AUDIO and ENABLE_XSLT are disabled
+ https://bugs.webkit.org/show_bug.cgi?id=183896
+
+Reviewed by Yusuke Suzuki.
+
+No new tests needed.
+
+* bindings/js/JSWebAnimationCustom.cpp: Add missing #include of Document.h
+
+Signed-off-by: Adrian Perez de Castro <[email protected]>
+Upstream-Status: Committed
+
+diff --git a/Source/WebCore/bindings/js/JSWebAnimationCustom.cpp b/Source/WebCore/bindings/js/JSWebAnimationCustom.cpp
+index 8b56f86c151..b8716be0f1c 100644
+--- a/Source/WebCore/bindings/js/JSWebAnimationCustom.cpp
++++ b/Source/WebCore/bindings/js/JSWebAnimationCustom.cpp
+@@ -26,6 +26,7 @@
+ #include "config.h"
+ #include "JSWebAnimation.h"
+ 
++#include "Document.h"
+ #include "JSAnimationEffect.h"
+ #include "JSAnimationTimeline.h"
+ #include "JSDOMConstructor.h"
+-- 
+2.16.2
+

index 31794ed2a4..99755d8d69 100644
@@ -2,6 +2,7 @@ config BR2_PACKAGE_WEBKITGTK_ARCH_SUPPORTS
 	bool
 	# ARM needs BLX, so v5t+, BE completely untested so disabled
 	default y if BR2_arm && !BR2_ARM_CPU_ARMV4
+	default y if BR2_aarch64
 	default y if BR2_i386 || BR2_x86_64
 	# Disabled on MIPS big endian due to sigbus
 	default y if BR2_mipsel || BR2_mips64el

@@ -41,6 +42,8 @@ config BR2_PACKAGE_WEBKITGTK
 	select BR2_PACKAGE_LIBXSLT
 	select BR2_PACKAGE_SQLITE
 	select BR2_PACKAGE_WEBP
+	select BR2_PACKAGE_WEBP_DEMUX
+	select BR2_PACKAGE_WOFF2
 	select BR2_PACKAGE_XLIB_LIBXCOMPOSITE if BR2_PACKAGE_LIBGTK3_X11
 	select BR2_PACKAGE_XLIB_LIBXDAMAGE if BR2_PACKAGE_LIBGTK3_X11
 	select BR2_PACKAGE_XLIB_LIBXRENDER if BR2_PACKAGE_LIBGTK3_X11

index dd2b28d372..28468a5297 100644
@@ -1,7 +1,7 @@
-# From https://webkitgtk.org/releases/webkitgtk-2.18.6.tar.xz.sums
-md5 c1a548595135ee75ad3bf2e18ac83112 webkitgtk-2.18.6.tar.xz
-sha1 fb0daa85142cfe8822de518dfaa7bd5c3cdd6c23 webkitgtk-2.18.6.tar.xz
-sha256 93912cc2f40f12e452be1ca4babdbdaac0ec4f828d441257a6b06c2963bbac3c webkitgtk-2.18.6.tar.xz
+# From https://webkitgtk.org/releases/webkitgtk-2.20.0.tar.xz.sums
+md5 51cc47345c3ec53802718cd6fc9b746f webkitgtk-2.20.0.tar.xz
+sha1 f1e021318f06085fff5455667459bf26716d610f webkitgtk-2.20.0.tar.xz
+sha256 57f640f720bd9a8a7207f3321cf803a15c2f207b4e7b75ff1be17bc1eeb00a3c webkitgtk-2.20.20.tar.xz
 
 # Hashes for license files:
 sha256 0b5d3a7cc325942567373b0ecd757d07c132e0ebd7c97bfc63f7e1a76094edb4 Source/WebCore/LICENSE-APPLE

index eb39664b1e..e27f970104 100644
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-WEBKITGTK_VERSION = 2.18.6
+WEBKITGTK_VERSION = 2.20.0
 WEBKITGTK_SITE = http://www.webkitgtk.org/releases
 WEBKITGTK_SOURCE = webkitgtk-$(WEBKITGTK_VERSION).tar.xz
 WEBKITGTK_INSTALL_STAGING = YES

@@ -14,7 +14,7 @@ WEBKITGTK_LICENSE_FILES = \
 	Source/WebCore/LICENSE-LGPL-2.1
 WEBKITGTK_DEPENDENCIES = host-ruby host-flex host-bison host-gperf \
 	enchant harfbuzz icu jpeg libgcrypt libgtk3 libsecret libsoup \
-	libtasn1 libxml2 libxslt sqlite webp
+	libtasn1 libxml2 libxslt sqlite webp woff2
 WEBKITGTK_CONF_OPTS = \
 	-DENABLE_API_TESTS=OFF \
 	-DENABLE_GEOLOCATION=OFF \

2.16.2

Your introductory mail talked about 3 additional patches needed for aarch64 support, but I don't see them here?

Bye, Peter Korsgaard

Hello Adrian,

Could you give some feedback on this question, so that we can move forward with applying the webkitgtk bump ?

Thanks!

Thomas

Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

Hi!

The change for AArch64 is just listing BR2_aarch64 in WEBKITGTK_ARCH_SUPPORTS, the addition is in the first patch.

Anyway, I was waiting a bit until the 2.20.1 release, which just happened:

https://webkitgtk.org/2018/04/10/webkitgtk2.20.1-released.html

Updating to 2.20.1 directly is IMHO better because it includes the fix for building with multimedia disabled, and a fix to improve memory consumption on devices with smaller amounts of memory ? which is particularly welcome for embedded :-)

I'm making a test build with 2.20.1 as I wrote this mail, if all goes well the new patch set will supersede this one.

Cheers,

--
Adri?n ?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180410/40a17eb8/attachment.asc>

Hello,

Then the following paragraph that you wrote:

This series starts with the same three patches as the previous submission, and adds on top three additional commits to make WebKitGTK+ available on AArch64 (it's a platform we support upstream), and adds a couple of patches which are already merged in the upstream repository, but are not in the release and are worth applying.

is a bit unclear. You're saying "adds on top three additional commits to make WebKitGTK+ available on AArch6". What those three additional commits ?

For the record, your patch series was:

brotli: new package
woff2: new package
webkitgtk: update to version 2.20.0
webkitgtk: Add upstream patch for better memory monitoring

OK, thanks. Some clarification about the above would be nice. Perhaps it was just a mistake in your cover letter ?

Best regards,

Thomas

Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

Hi Thomas,

Ah, right: first I thought of making a separate patch for the small change of marking as available on AArch64. That, plus one commit for the upstream build fix, plus one commit for the upstream memory monitor patch. The first I ended up stashing with ?webkitgtk: update to version 2.20.0?, and the other two are not going to be needed for 2.20.1 (they were merged upstream).

Sorry if I wasn't being very clear here O:-)

If you think enabling the package for AArch64 deserves its own separate commit, let me know and I'll split it back. FWIW, I've been trying AArch64 builds for the RaspberryPi 3, and they work fine; we also use them at work for a couple of clients as well without issues.

Cheers,

--
Adri?n ?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180410/506c727b/attachment.asc>

Hello,

OK, thanks for the clarification.

I think it's OK to enable on AArch64 at the same time as you do the bump. It could be done in separate commits (each change stands on its own), but I'm fine with those changes being in the same commit.

Best regards,

Thomas

Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

Hi again,

Also, I have found out from one of my test builds that in both 2.20.x releases we have an ugly regression of the JSC JIT support on 32-bit MIPS and ARM when using the softfp ABI.

A patch for this is at https://bugs.webkit.org/show_bug.cgi?id=183786 but we (the WebKitGTK upstream) haven't decided yet whether it will be in the next 2.20.x release. Some of us think it is worth a try, so I am in the middle of backporting the patch (and a few other patches needed by it) on top of 2.20.1, to include them in Buildroot, which is a good way of trying getting to try them on top of 2.20.1 and decide if they will go in a release as well.

If you think it would be better to first disable JIT for 32-bit ARM and MIPS when doing the webkitgtk package version bump, and later on re-enabling it with the backpoted changes, please let me know.

Most likely I will keep it along with the version bump, then.

Thanks for all the feedback!

--
Adri?n ?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180410/83293599/attachment.asc>

Hello,

Since I don't know what is the extent of the breakage, and I have a limited understanding of WebkitGtk anyway, I'll let you decide what is best.

By "softfp ABI", are you talking soft-float (i.e the hardware has no FPU at all) or about -mfloat-abi=softfp on ARM, which means that the FPU may be used, but that integer registers are used to pass floating point arguments ?

In either case, it would mean that ARM EABIhf is not broken, and therefore JIT should be not be disabled for all ARM configurations, but only the ones using softfp.

Best regards,

Thomas

Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

Hello,

On certain situations, the code generated by the JavaScriptCore JIT compiler may contain calls to C++ functions which are part of the runtime. The JIT compiler has to generate those calls using the same ABI conventions as the C/C++ compiler. Any code generating calls into the runtime will either crash or behave in unpredictable ways.

The patch is non-trivial, but fortunately it's just a backport of an existing fix so there are decent chances that I may have it ready and tested by next week.

I mean using ?-mfloat-abi=softfp? (FPU may be used, integer regs used).

<trivia>
The JavaScriptCore JIT does not run on soft-float hardware at all, and can emit NEON code, so the build recipe includes a check on BR2_ARM_CPU_HAS_NEON.

There is a mode that does not use NEON, but instead it won't use Thumb2 either, just traditional ARM 32-bit instructions... but the traditional-ARM configuration has been bitrotting for quite some time. Anyway, nowadays most ARM processors with enough power to run WebKit decently all have NEON support, so probably not a big deal.
</trivia>

That is correct: ARM EABIhf (including AArch64) works well. If I don't manage to have the patch working on top of 2.20.1 in the next few days I will submit patch set to update to 2.20.1 with JIT support disabled for ARM when using softfp.

As for MIPS, currently the webkitgtk recipe never enables the JIT support. With patch backported it will be possible to also enable it for 32-bit MIPS (and possibly for 64-bit as well, but I don't dare to do that because I lack hardware to run tests.)

Cheers,

--
Adri?n ?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180410/25e56add/attachment.asc>

This imports a patch which has been committed upstream, but is not part of the 2.20.0 release which modifies WebKitGTK+'s memory monitor to take into account the available memory when calculating the thresholds at which the Web engine will try to return memory back to the system. The net result is that on constrained systems WebKitGTK+ will try to free memory it does not need more often.

---
 ...memory-monitor-thresholds-should-be-bette.patch | 93 ++++++++++++++++++++++
 1 file changed, 93 insertions(+)
 create mode 100644 package/webkitgtk/0001-WebProcess-memory-monitor-thresholds-should-be-bette.patch

new file mode 100644
index 0000000000..9848ec8526
@@ -0,0 +1,93 @@
+From 25fe9c19e4fb3519e34df7e16b3d761812b921aa Mon Sep 17 00:00:00 2001
+From: Adrian Perez de Castro <[email protected]>
+Date: Wed, 21 Mar 2018 01:05:53 +0200
+Subject: [PATCH xserver 1/2] WebProcess memory monitor thresholds should be
+ better tuned for embedded systems.
+ https://bugs.webkit.org/show_bug.cgi?id=183773
+
+Patch by Carlos Alberto Lopez Perez <[email protected]> on 2018-03-20
+Reviewed by NOBODY (OOPS!).
+
+Take into account the total system RAM for the thresholds calculation.
+
+For systems with more than 3GB the conservative and strict thresholds remain as they are,
+but for systems with less RAM the thresholds are dynamically configured as follows:
+
+- Conservative threshold (release non critical memory) if WebProcess using more than 33% of the total RAM.
+- Strict threshold (release all possible memory) if WebProcess using more than 50% of the total RAM.
+
+The Kill threshold is also modified. Now it is capped at 90% of the total RAM.
+
+* wtf/MemoryPressureHandler.cpp:
+(WTF::thresholdForMemoryKillWithProcessState):
+(WTF::thresholdForPolicy):
+(WTF::MemoryPressureHandler::shrinkOrDie):
+
+Signed-off-by: Adrian Perez de Castro <[email protected]>
+Upstream-Status: Committed
+
+diff --git a/Source/WTF/wtf/MemoryPressureHandler.cpp b/Source/WTF/wtf/MemoryPressureHandler.cpp
+index 0e1d1a60fe0..d31180883eb 100644
+--- a/Source/WTF/wtf/MemoryPressureHandler.cpp
++++ b/Source/WTF/wtf/MemoryPressureHandler.cpp
+@@ -28,6 +28,7 @@
+ 
+ #include <wtf/MemoryFootprint.h>
+ #include <wtf/NeverDestroyed.h>
++#include <wtf/RAMSize.h>
+ 
+ #define LOG_CHANNEL_PREFIX Log
+ 
+@@ -84,20 +85,17 @@ static const char* toString(MemoryUsagePolicy policy)
+ 
+ static size_t thresholdForMemoryKillWithProcessState(WebsamProcessState processState, unsigned tabCount)
+ {
++    size_t baseThreshold = 2 * GB;
+ #if CPU(X86_64) || CPU(ARM64)
+-    size_t baseThreshold;
+     if (processState == WebsamProcessState::Active)
+         baseThreshold = 4 * GB;
+-    else
+-        baseThreshold = 2 * GB;
+-    if (tabCount <= 1)
+-        return baseThreshold;
+-    return baseThreshold + (std::min(tabCount - 1, 4u) * 1 * GB);
++    if (tabCount > 1)
++        baseThreshold += std::min(tabCount - 1, 4u) * 1 * GB;
+ #else
+-    UNUSED_PARAM(processState);
+-    UNUSED_PARAM(tabCount);
+-    return 3 * GB;
++    if ((tabCount > 1) || (processState == WebsamProcessState::Active))
++        baseThreshold = 3 * GB;
+ #endif
++    return std::min(baseThreshold, static_cast<size_t>(ramSize() * 0.9));
+ }
+ 
+ void MemoryPressureHandler::setPageCount(unsigned pageCount)
+@@ -114,11 +112,12 @@ size_t MemoryPressureHandler::thresholdForMemoryKill()
+ 
+ static size_t thresholdForPolicy(MemoryUsagePolicy policy)
+ {
++    const size_t baseThresholdForPolicy = std::min(3 * GB, ramSize());
+     switch (policy) {
+     case MemoryUsagePolicy::Conservative:
+-        return 1 * GB;
++        return baseThresholdForPolicy / 3;
+     case MemoryUsagePolicy::Strict:
+-        return 1.5 * GB;
++        return baseThresholdForPolicy / 2;
+     case MemoryUsagePolicy::Unrestricted:
+     default:
+         ASSERT_NOT_REACHED();
+@@ -150,6 +149,7 @@ void MemoryPressureHandler::shrinkOrDie()
+         return;
+     }
+ 
++    WTFLogAlways("Unable to shrink memory footprint of process (%lu MB) below the kill thresold (%lu MB). Killed\n", footprint.value() / MB, thresholdForMemoryKill() / MB);
+     RELEASE_ASSERT(m_memoryKillCallback);
+     m_memoryKillCallback();
+ }
+-- 
+2.16.2
+

2.16.2

Hello Adrian,

In the discussion around those two patches, you said you would resend a new version that takes into account my comments, and also uses the latest 2.20.1 release.

Therefore, I'll mark the current patches as "Changes Requested" in patchwork. Don't forget to send a new iteration that bumps to 2.20.1. Thanks!

Best regards,

Thomas

Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com