Qt WebEngine support

Hi all,

The purpose of this patch series is to continue the integration of Qt5 WebEngine in buildroot; started long time ago by Akihiko Odaki [1] and Julien Corjon [2].

The first patch adds the rpi-userland OpenGL libraries symlinks. WebEngine eglfs backend loads dynamically libEGL.so.1 and libGLESv2.so.2 [3].

	$ ls -l output/target/usr/lib/libEGL* output/target/usr/lib/libGLESv2*
	-rwxr-xr-x 1 buildroot buildroot 130032 Aug 10 21:17 output/target/usr/lib/libEGL.so
	lrwxrwxrwx 1 buildroot buildroot      9 Aug  9 12:30 output/target/usr/lib/libEGL.so.1 -> libEGL.so
	-rwxr-xr-x 1 buildroot buildroot  71088 Aug 10 21:16 output/target/usr/lib/libGLESv2.so
	lrwxrwxrwx 1 buildroot buildroot     12 Aug  9 12:30 output/target/usr/lib/libGLESv2.so.2 -> libGLESv2.so

A PR has been made [4].

The second patch provides the Qt WebEngine package. The additionnal patch fixes a build issue when samples are compiled without the support of printing [5]. This patch is already merged [6] and concerns only 5.9.1.

It fixes the error below.

	.obj/browsermainwindow.o: In function `BrowserMainWindow::printRequested(QWebEnginePage*)': browsermainwindow.cpp:(.text+0x2cc0): undefined reference to `QWebEnginePage::print(QPrinter*, QWebEngineCallback<bool> const&)'

The last patch is a special defconfig that provides a Qt WebEngine setup on raspberrypi3.

With this patchset, one can run the Qt quicknanobrowser sample on a rpi3 with the following options:
- BR2_TOOLCHAIN_BUILDROOT_LIBC="glibc" and BR2_TOOLCHAIN_BUILDROOT_CXX=y (Qt 5 needs a toolchain w/ wchar, NPTL, C++, dynamic library)
- BR2_PACKAGE_LIBERATION (Qt 5.8 requires a least one font)
- BR2_PACKAGE_RPI_USERLAND (to enable OpenGL backend)
- BR2_PACKAGE_QT5BASE_EXAMPLES (to install quicknanobrowser sample)
- BR2_PACKAGE_QT5QUICKCONTROLS (needed by quicknanobrowser)
- BR2_PACKAGE_QT5WEBENGINE (because it is what we want :))

To browse for HTTPS websites, please consider adding the following options as well:
- BR2_PACKAGE_CA_CERT (for certificates)
- BR2_PACKAGE_NTPD (to sync date)

To run quicknanobrowser:

	# cd /usr/lib/qt/examples/webengine/quicknanobrowser/
	# ./quicknanobrowser

I have tested it with a rpi 3; and I will be glad if someone else has another EGL platform to test it.

Enjoy ;)

Note: For now, quicknanobrowser will just display the qt website without any input backend.

Note 2: Since bump to 5.9.1, the chromium's copy of opus fails with neon [7]. BR2_PACKAGE_QT5WEBENGINE_SYSTEM_FFMPEG option allows one to use buildroot copy (which compiles fine).

	In file included from ../../3rdparty/chromium/third_party/opus/src/silk/arm/NSQ_neon.c:31:0:
	/home/gportay/src/buildroot/output-rpi3-qt5.9/host/lib/gcc/arm-buildroot-linux-gnueabihf/6.4.0/include/arm_neon.h:8997:1: error: inlining failed in call to always_inline ?vld1q_s32?: target specific option mismatch
	 vld1q_s32 (const int32_t * __a)
	 ^~~~~~~~~
	../../3rdparty/chromium/third_party/opus/src/silk/arm/NSQ_neon.c:40:15: note: called from here
	     int32x4_t coef0 = vld1q_s32(coef32);
		       ^~~~~
	../../3rdparty/chromium/third_party/opus/src/silk/arm/NSQ_neon.c: At top level:
	cc1: warning: unrecognized command line option ?-Wno-#pragma-messages?

Note 3: Since bump to 5.9.1, chromium requires udev.

Note 4: Tested against:
- GCC 6.x and binutils 2.27 on rpi 3.
- GCC 7.x and binutils 2.28 on rpi 3.

Note 5: Version 5.6.2 causes a build issue while building chromium. I disabled
the build against this version and am waiting for the release 5.6.3.

[1] https://patchwork.ozlabs.org/patch/640633/
[2] http://lists.busybox.net/pipermail/buildroot/2015-July/132010.html
[3] https://github.com/qt/qtwebengine/blob/5.9/src/core/surface_factory_qt.cpp#L76-L86
[4] https://github.com/raspberrypi/userland/pull/379
[5] https://bugreports.qt.io/browse/QTBUG-61510
[6] https://codereview.qt-project.org/#/c/198041/
[7] https://patchwork.ozlabs.org/patch/791332/

Note 6: The chromium.inc has been generated thanks
( echo 'CHROMIUM_LICENSE_FILES = \' &&
cd output/build/qt5webengine-5.9.1/ && \
find "src/3rdparty/chromium/" -type f -iname "*LICENSE*" -o -iname "*COPYING*" -o -iname "*GPL*" | \

  sed -e '/\.asm$/d' \
      -e '/\.h$/d' \
      -e '/\.c$/d' \
      -e '/\.cc$/d' \
      -e '/\.cpp$/d' \
      -e '/\.pyc\?$/d' \
      -e '/\.pl$/d' \
      -e '/\.sha1$/d' \
      -e '/\.patch$/d' \
      -e '/licensecheck/d' \
      -e 's,^,\t,' \
      -e 's,$, \\,' | \

sort && \
echo '' ) >package/qt5/qt5webengine/chromium.inc

Changes since RFC:
- rebase to master (use SPDX licenses, remove license approved, update qt5_defconfig...)
- remove the patch "prefer arm tune"
- disable build with Qt 5.6 LTS (not working, waiting for 5.6.3 in August)
- re-order patches

Changes since v1:
- manually create symlinks to preserve SONAME
- add my signed-off-by to the qt webengine patch
- make qtwe package depends on supported architectures
- fix Kconfig comment about Qt 5.6
- sort alphabetically chromium licenses

Changes since v2:
- fix build issue when I sorted alphabetically the chromium licenses
- replace legacy option BR2_TARGET_ROOTFS_EXT2_BLOCK to new option BR2_TARGET_ROOTFS_EXT2_SIZE

Changes since v3:
- add my signed-off-by and the PR link to the rpi-userland patch
- move rpi patch index from 0001 to 0007

Changes since v4:
- bump to Qt WebEngine to 5.9.1
- added the mainlined patch that fixes the build issue with samples when print is not supported
- udev is selected (required by chromium)
- add config entry to use system ffmpeg and set it to yes by default
- the patch concerning the path to CA certificates has been removed since it is no more relevant in 5.9.x
- increase BR2_TARGET_ROOTFS_EXT2_SIZE to 360MB to support

   $ du -sk target/
   312480	target/

Changes since v5:
- really add the mainlined patch that fixes the build issue with samples when print is not supported
- add BR2_PACKAGE_QT5WEBENGINE_ARCH_SUPPORTS to list all supported architectures
- add BR2_PACKAGE_QT5WEBENGINE missing dependencies (tc, mmu, wchar...)
- stop using && for depends on relationships
- update chromium licence list in chromium.inc
- remove option BR2_PACKAGE_QT5WEBENGINE_SYSTEM_FFMPEG;
and compile ffmpeg, opus, libvpx and webp packages
- reword help and add url to documentation of package qt5webengine
- remove hash of 5.6.2 version
- add QT5WEBENGINE_ namespace to QMAKEFLAGS
- add missing TARGET_MAKE_ENV to BUILD_CMD
- create python symlink form $(HOST_DIR)/bin (was $(HOST_DIR)/usr/bin)
- add myself to DEVELOPERS

Changes since v6:
- rpi-userland commit message:
fix output of ls -l snippet
update url to link to qtwe-5.9.1
add upstream-status
- qtwe:
add missing host-bison, host-flex and host-pkgconf dependencies select BR2_HOSTARCH_NEEDS_IA32_COMPILER for 32-bit targets

Ga?l PORTAY (3):
rpi-userland: fix opengl library symlinks
qt5webengine: new package
configs: new raspberrypi3 qt5 config

 DEVELOPERS                                         |   1 +
 configs/raspberrypi3_qt5_defconfig                 |  48 ++++
 package/qt5/Config.in                              |   1 +
 .../0001-Always-compile-QWebEnginePage-print.patch | 244 ++++++++++++++++
 package/qt5/qt5webengine/Config.in                 |  75 +++++
 package/qt5/qt5webengine/chromium.inc              | 310 +++++++++++++++++++++
 package/qt5/qt5webengine/qt5webengine.hash         |   2 +
 package/qt5/qt5webengine/qt5webengine.mk           |  80 ++++++
 .../0007-Add-OpenGL-library-versioning.patch       |  41 +++
 9 files changed, 802 insertions(+)
 create mode 100644 configs/raspberrypi3_qt5_defconfig
 create mode 100644 package/qt5/qt5webengine/5.9.1/0001-Always-compile-QWebEnginePage-print.patch
 create mode 100644 package/qt5/qt5webengine/Config.in
 create mode 100644 package/qt5/qt5webengine/chromium.inc
 create mode 100644 package/qt5/qt5webengine/qt5webengine.hash
 create mode 100644 package/qt5/qt5webengine/qt5webengine.mk
 create mode 100644 package/rpi-userland/0007-Add-OpenGL-library-versioning.patch

2.13.2

The Qt WebEngine module loads libEGL.so.1 and libGLESv2.so.2 libraries. Paths are hard-coded [1].

This patch sets up cmake to create those library links.

	$ ls -l output/target/usr/lib/libEGL* output/target/usr/lib/libGLESv2*
	-rwxr-xr-x 1 buildroot buildroot 130032 Aug 10 21:17 output/target/usr/lib/libEGL.so
	lrwxrwxrwx 1 buildroot buildroot      9 Aug  9 12:30 output/target/usr/lib/libEGL.so.1 -> libEGL.so
	-rwxr-xr-x 1 buildroot buildroot  71088 Aug 10 21:16 output/target/usr/lib/libGLESv2.so
	lrwxrwxrwx 1 buildroot buildroot     12 Aug  9 12:30 output/target/usr/lib/libGLESv2.so.2 -> libGLESv2.so

A pull request is pending [2].

[1] https://github.com/qt/qtwebengine/blob/5.9.1/src/core/surface_factory_qt.cpp#L76-L86
[2] https://github.com/raspberrypi/userland/pull/379

---
 .../0007-Add-OpenGL-library-versioning.patch       | 41 ++++++++++++++++++++++
 1 file changed, 41 insertions(+)
 create mode 100644 package/rpi-userland/0007-Add-OpenGL-library-versioning.patch

new file mode 100644
index 000000000..9a22a66d2
@@ -0,0 +1,41 @@
+From a6abc80168512993f390cfdb40531c50829af6da Mon Sep 17 00:00:00 2001
+From: =?utf-8?q?Ga=C3=ABl=20PORTAY?= <[email protected]>
+Date: Fri, 3 Mar 2017 17:28:41 -0500
+Subject: [PATCH] Add OpenGL library versioning
+MIME-Version: 1.0
+Content-Type: text/plain; charset=utf-8
+Content-Transfer-Encoding: 8bit
+
+Some applications like QtWebEngine dynamically load libraries using
+dlopen. The library name is hardcoded, and it includes the library
+version [1].
+
+This patch creates symlinks for EGL and GLES libraries.
+
+	libEGL.so.1 -> libEGL.so
+	libGLESv2.so.2 -> libGLESv2.so
+
+[1] https://github.com/qt/qtwebengine/blob/v5.9.1/src/core/surface_factory_qt.cpp#L76-L86
+
+Upstream-Status: https://github.com/raspberrypi/userland/pull/379
+Signed-off-by: Ga?l PORTAY <[email protected]>
+---
+ host_applications/android/apps/vidtex/CMakeLists.txt | 5 +++++
+ 1 file changed, 5 insertions(+)
+
+diff --git a/host_applications/android/apps/vidtex/CMakeLists.txt b/host_applications/android/apps/vidtex/CMakeLists.txt
+index 1f705ef..d68ae5f 100644
+--- a/host_applications/android/apps/vidtex/CMakeLists.txt
++++ b/host_applications/android/apps/vidtex/CMakeLists.txt
+@@ -10,3 +10,8 @@ set (VIDTEX_SOURCES
+    vidtex.c)
+ add_executable(vidtex ${VIDTEX_SOURCES})
+ target_link_libraries(vidtex GLESv2 EGL m bcm_host mmal_core mmal_components mmal_util mmal_vc_client vcos)
++
++set(DESTDIR \$ENV{DESTDIR})
++include(GNUInstallDirs)
++install(CODE "execute_process(COMMAND ${CMAKE_COMMAND} -E create_symlink libEGL.so ${DESTDIR}/${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}/libEGL.so.1)")
++install(CODE "execute_process(COMMAND ${CMAKE_COMMAND} -E create_symlink libGLESv2.so ${DESTDIR}/${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}/libGLESv2.so.2)")
+-- 
+2.13.2
+

2.13.2

Ga?l, All,

On 2017-08-13 09:56 -0400, Ga?l PORTAY spake thusly:

But now they no longer do that:

    https://github.com/qt/qtwebengine/commit/d4c621f6a6b87f2a86069fa393b9f7c4f9e7b9ad

Upstream was not too happy with your MR... :-(

So, do you think you could look at backporting the Qt patch instead?

Regards,
Yann E. MORIN.

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Hello Yann,

Good catch!

For sure, I will restart a build with this commit and without my patch.

Regards,
Gael

This patch is based on works [1] and [2].

The chromium.inc has been generated using the following command.

	( echo 'CHROMIUM_LICENSE_FILES = \' &&
	  cd output/build/qt5webengine-5.9.1/ && \
	  find "src/3rdparty/chromium/" -type f -iname "*LICENSE*" -o -iname "*COPYING*" -o -iname "*GPL*" | \
	  sed -e '/\.asm$/d' \
	      -e '/\.h$/d' \
	      -e '/\.c$/d' \
	      -e '/\.cc$/d' \
	      -e '/\.cpp$/d' \
	      -e '/\.pyc\?$/d' \
	      -e '/\.pl$/d' \
	      -e '/\.sha1$/d' \
	      -e '/\.patch$/d' \
	      -e '/licensecheck/d' \
	      -e 's,^,\t,' \
	      -e 's,$, \\,' | \
	  sort && \
	  echo '' ) >package/qt5/qt5webengine/chromium.inc

[1]: http://lists.busybox.net/pipermail/buildroot/2015-July/132010.html
[2]: https://patchwork.ozlabs.org/patch/640633/

---
 DEVELOPERS                                         |   1 +
 package/qt5/Config.in                              |   1 +
 .../0001-Always-compile-QWebEnginePage-print.patch | 244 ++++++++++++++++
 package/qt5/qt5webengine/Config.in                 |  75 +++++
 package/qt5/qt5webengine/chromium.inc              | 310 +++++++++++++++++++++
 package/qt5/qt5webengine/qt5webengine.hash         |   2 +
 package/qt5/qt5webengine/qt5webengine.mk           |  80 ++++++
 7 files changed, 713 insertions(+)
 create mode 100644 package/qt5/qt5webengine/5.9.1/0001-Always-compile-QWebEnginePage-print.patch
 create mode 100644 package/qt5/qt5webengine/Config.in
 create mode 100644 package/qt5/qt5webengine/chromium.inc
 create mode 100644 package/qt5/qt5webengine/qt5webengine.hash
 create mode 100644 package/qt5/qt5webengine/qt5webengine.mk

index f3944e274..a6f84e77e 100644
@@ -630,6 +630,7 @@ F:	package/zxing-cpp/
 
 N:	Ga?l Portay <[email protected]>
 F:	package/qt5/qt5virtualkeyboard/
+F:	package/qt5/qt5webengine/
 
 N:	Gary Bisson <[email protected]>
 F:	board/boundarydevices/

index abaa58499..f2a1f378c 100644
@@ -79,6 +79,7 @@ source "package/qt5/qt5tools/Config.in"
 source "package/qt5/qt5virtualkeyboard/Config.in"
 source "package/qt5/qt5webchannel/Config.in"
 source "package/qt5/qt5webkit/Config.in"
+source "package/qt5/qt5webengine/Config.in"
 source "package/qt5/qt5websockets/Config.in"
 source "package/qt5/qt5x11extras/Config.in"
 source "package/qt5/qt5xmlpatterns/Config.in"

new file mode 100644
index 000000000..d45500c44
@@ -0,0 +1,2 @@
+# Hash from: http://download.qt.io/official_releases/qt/5.9/5.9.1/submodules/qtwebengine-opensource-src-5.9.1.tar.xz.mirrorlist
+sha256 f6a37eeb9188474a16d29ede498fce959396ab80329a0a83eaeb925251686401  qtwebengine-opensource-src-5.9.1.tar.xz

2.13.2

This config sets Qt WebEngine for raspberrypi 3.

---
 configs/raspberrypi3_qt5_defconfig | 48 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
 create mode 100644 configs/raspberrypi3_qt5_defconfig

new file mode 100644
index 000000000..7cc850d09
@@ -0,0 +1,48 @@
+BR2_arm=y
+BR2_cortex_a53=y
+BR2_ARM_FPU_NEON_VFPV4=y
+
+BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
+BR2_TOOLCHAIN_BUILDROOT_CXX=y
+
+BR2_SYSTEM_DHCP="eth0"
+
+# Linux headers same as kernel, a 4.9 series
+BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_9=y
+
+BR2_LINUX_KERNEL=y
+BR2_LINUX_KERNEL_CUSTOM_GIT=y
+BR2_LINUX_KERNEL_CUSTOM_REPO_URL="https://github.com/raspberrypi/linux.git"
+BR2_LINUX_KERNEL_CUSTOM_REPO_VERSION="5e4ee836560d4c0371e109bf469e1ad808ae7a44"
+BR2_LINUX_KERNEL_DEFCONFIG="bcm2709"
+
+# Build the DTB from the kernel sources
+BR2_LINUX_KERNEL_DTS_SUPPORT=y
+BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2710-rpi-3-b bcm2710-rpi-cm3"
+
+# Build Qt5 WebEngine
+BR2_PACKAGE_LIBERATION=y
+BR2_PACKAGE_QT5=y
+BR2_PACKAGE_QT5BASE_EXAMPLES=y
+BR2_PACKAGE_QT5QUICKCONTROLS=y
+BR2_PACKAGE_QT5WEBENGINE=y
+BR2_PACKAGE_QT5WEBENGINE_PROPRIETARY_CODECS=y
+BR2_PACKAGE_RPI_FIRMWARE=y
+BR2_PACKAGE_RPI_USERLAND=y
+BR2_PACKAGE_CA_CERTIFICATES=y
+BR2_PACKAGE_NTP=y
+
+# Required tools to create the SD image
+BR2_PACKAGE_HOST_DOSFSTOOLS=y
+BR2_PACKAGE_HOST_GENIMAGE=y
+BR2_PACKAGE_HOST_MTOOLS=y
+
+# Filesystem / image
+BR2_TARGET_ROOTFS_EXT2=y
+BR2_TARGET_ROOTFS_EXT2_4=y
+BR2_TARGET_ROOTFS_EXT2_SIZE="360000"
+# BR2_TARGET_ROOTFS_TAR is not set
+BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV=y
+BR2_ROOTFS_POST_BUILD_SCRIPT="board/raspberrypi3/post-build.sh"
+BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi3/post-image.sh"
+BR2_ROOTFS_POST_SCRIPT_ARGS="--add-pi3-miniuart-bt-overlay"

2.13.2

Hello,

These details would be good to have in the commit log of the qtwebengine package.

Those notes too.

This note too.

BR2_HOSTARCH_NEEDS_IA32_COMPILER has nothing to do with 32-bit targets. It has to do with the fact that qtwebengine builds programs for the *host* (and not the target) using -m32. Therefore, on x86-64 machines, it requires multilib support, in order to build (and run) 32 bits programs.

Or, do you have some evidence/explanation that qtwebengine builds its host programs 32 bits when the target is 32 bits, and builds its host programs 64 bits when the target is 64 bits ?

Best regards,

Thomas

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Hello Thomas,

Indeed, I will reword the commit message.

Sorry if the changelog was not clear enough.

Does the comment I added in the Config.in is more clear?

# For 32-bits targets, v8 (a chromium 3rd-party) adds -m32 flag to build # internal host-tools.
# https://github.com/v8/v8/blob/5.6.326.55/gypfiles/toolchain.gypi#L1036-L1037

I had a quick look to where v8 adds the -m32 flag and I found this [1].

My understanding is that v8 adds this flag when target is 32-bits and when host compiler is biarch:

      ['(OS=="linux" or OS=="freebsd" or OS=="openbsd" or OS=="solaris" \
         or OS=="netbsd" or OS=="mac" or OS=="android" or OS=="qnx") and \
        (v8_target_arch=="arm" or v8_target_arch=="ia32" or \
         v8_target_arch=="x87" or v8_target_arch=="mips" or \
         v8_target_arch=="mipsel" or v8_target_arch=="ppc" or \
         v8_target_arch=="s390")', {
        'target_conditions': [
          ['_toolset=="host"', {
            'conditions': [
              ['host_cxx_is_biarch==1', {
                'conditions': [
                  ['host_arch=="s390" or host_arch=="s390x"', {
                    'cflags': [ '-m31' ],
                    'ldflags': [ '-m31' ]
                  },{
                   'cflags': [ '-m32' ],
                   'ldflags': [ '-m32' ]
                  }],
                ],
              }],
            ],
            'xcode_settings': {
              'ARCHS': [ 'i386' ],
            },
           ...
      }],

biarch is set when host is x86[-64], PPC[64] or s390[x] (see [2]).

    ['host_arch=="ia32" or host_arch=="x64" or \
      host_arch=="ppc" or host_arch=="ppc64" or \
      host_arch=="s390" or host_arch=="s390x" or \
      clang==1', {
      'variables': {
        'host_cxx_is_biarch%': 1,
       },
     }, {
      'variables': {
        'host_cxx_is_biarch%': 0,
      },
    }],

So v8 seems to add -m32 flag to build its host-tools if host is intel/ppc/s390 (32/64-bits) and the target is 32-bits.

Also, it seems to apply the -m64 flag for 64-bits targets [3].

      ['(OS=="linux" or OS=="android") and \
        (v8_target_arch=="x64" or v8_target_arch=="arm64" or \
         v8_target_arch=="ppc64" or v8_target_arch=="s390x")', {
        'target_conditions': [
          ['_toolset=="host"', {
            'conditions': [
              ['host_cxx_is_biarch==1', {
                'cflags': [ '-m64' ],
                'ldflags': [ '-m64' ]
              }],
             ],
           }],
           ...,
         ],
      }],

[1] https://github.com/v8/v8/blob/5.6.326.55/gypfiles/toolchain.gypi#L1021-L1045
[2] https://github.com/v8/v8/blob/5.6.326.55/gypfiles/toolchain.gypi#L138-L150
[3] https://github.com/v8/v8/blob/5.6.326.55/gypfiles/toolchain.gypi#L1066-L1087

Best regards,
Gael