nodejs: drop 'gyp' PACKAGECONFIG
PATCH 1/3 nodejs: drop 'gyp' PACKAGECONFIG
From: André Draszik <[email protected]>
During the python3 / nodejs update, the dependencies weren't updated, so
using system-gyp ends up trying to use the python2 version of system-
gyp, which will of course fail.
Fixing this to depend on the python3 version of gyp still doesn't
doesn't make things work, though:
ERROR: nodejs-native-12.14.1-r0 do_configure: Execution of '.../nodejs-native/12.14.1-r0/temp/run.do_configure.26054' failed with exit code 1:
gyp: Error importing pymod_do_mainmodule (ForEachFormat): No module named 'ForEachFormat' while loading dependencies of .../nodejs-native/12.14.1-r0/node-v12.14.1/node.gyp while trying to load .../nodejs-native/12.14.1-r0/node-v12.14.1/node.gyp Error running GYP
The reason is the following patch nodejs has applied to its version of gyp as of NodeJS v12 (commit fff922afee6e ("deps,build: compute torque_outputs in v8.gyp") upstream):
tools/gyp/pylib/gyp/input.py 2020-03-02 12:16:09.956707788 +0000 renamed from pylib/gyp/input.py 2020-03-02 12:36:30.788248197 +0000 +2 -0
@@ -890,6 +881,7 @@ def ExpandVariables(input, phase, variab oldwd = os.getcwd() # Python doesn't like os.open('.'): no fchdir. if build_file_dir: # build_file_dir may be None (see above). os.chdir(build_file_dir) + sys.path.append(os.getcwd()) try: parsed_contents = shlex.split(contents)
@@ -900,6 +892,7 @@ def ExpandVariables(input, phase, variab "module (%s): %s" % (parsed_contents[0], e)) replacement = str(py_module.DoMain(parsed_contents[1:])).rstrip() finally: + sys.path.pop() os.chdir(oldwd) assert replacement != None elif command_string:
Since I'm not sure how to deal with that when using system-gyp, and because the original intention for using system-gyp was to make the previous nodejs version compatible with python3 by ultimately switching to the python3 version of system-gyp which isn't necessary anymore, and given nobody else seems to be using this PACKAGECONFIG, just drop it.
--- meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb | 3 --- 1 file changed, 3 deletions(-)
meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb +0 -3
@@ -53,7 +53,6 @@ ARCHFLAGS ?= "" PACKAGECONFIG ??= "ares icu libuv zlib" PACKAGECONFIG[ares] = "--shared-cares,,c-ares" -PACKAGECONFIG[gyp] = ",,gyp-py2-native" PACKAGECONFIG[icu] = "--with-intl=system-icu,--without-intl,icu" PACKAGECONFIG[libuv] = "--shared-libuv,,libuv" PACKAGECONFIG[nghttp2] = "--shared-nghttp2,,nghttp2"
@@ -82,8 +81,6 @@ python do_unpack() { shutil.rmtree(d.getVar('S') + '/deps/openssl', True) if 'ares' in d.getVar('PACKAGECONFIG'): shutil.rmtree(d.getVar('S') + '/deps/cares', True) - if 'gyp' in d.getVar('PACKAGECONFIG'): - shutil.rmtree(d.getVar('S') + '/tools/gyp', True) if 'libuv' in d.getVar('PACKAGECONFIG'): shutil.rmtree(d.getVar('S') + '/deps/uv', True) if 'nghttp2' in d.getVar('PACKAGECONFIG'):
Signature · 1 line
2.23.0.rc1
PATCH 2/3 brotli: allow building a -native version
From: André Draszik <[email protected]>
nodejs (-native) can be built against a system-installed version of the brotli libraries. Allow doing that by provide a -native version of brotli.
--- meta-oe/recipes-extended/brotli/brotli_1.0.7.bb | 2 ++ 1 file changed, 2 insertions(+)
meta-oe/recipes-extended/brotli/brotli_1.0.7.bb +2 -0
@@ -18,3 +18,5 @@ do_install_append () { mv -v "${lib}" "$(echo ${lib} | sed s/-static//)" done } + +BBCLASSEXTEND = "native"
Signature · 1 line
2.23.0.rc1
PATCH 3/3 nodejs: allow use of system brotli (and make default)
In reply to André DraszikFrom: André Draszik <[email protected]>
Use system brotli via PACKAGECONFIG by default. So far, nodejs had been built using its embedded copy of brotli, which we generally try to avoid, for the known reasons (independent updates, cve & license checks, etc).
The nodejs patches to enable this have been submitted. brotli is in meta-oe, so enabling this by default should not be a problem.
--- ...-passing-multiple-libs-to-pkg_config.patch | 41 ++++++++++++ ...allow-use-of-system-installed-brotli.patch | 66 +++++++++++++++++++ .../recipes-devtools/nodejs/nodejs_12.14.1.bb | 7 +- 3 files changed, 113 insertions(+), 1 deletion(-) create mode 100644 meta-oe/recipes-devtools/nodejs/nodejs/0001-build-allow-passing-multiple-libs-to-pkg_config.patch create mode 100644 meta-oe/recipes-devtools/nodejs/nodejs/0002-build-allow-use-of-system-installed-brotli.patch
meta-oe/recipes-devtools/nodejs/nodejs/0001-build-allow-passing-multiple-libs-to-pkg_config.patch new file +41 -0
@@ -0,0 +1,41 @@ +From fdaa0e3bef93c5c72a7258b5f1e30718e7d81f9b Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Andr=C3=A9=20Draszik?= <[email protected]> +Date: Mon, 2 Mar 2020 12:17:09 +0000 +Subject: [PATCH 1/2] build: allow passing multiple libs to pkg_config +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Sometimes it's necessary to pass multiple library names to pkg-config, +e.g. the brotli shared libraries can be pulled in with + pkg-config libbrotlienc libbrotlidec + +Update the code to handle both, strings (as used so far), and lists +of strings. + +Signed-off-by: André Draszik <[email protected]> +--- +Upstream-Status: Submitted [https://github.com/nodejs/node/pull/32046] + configure.py | 6 +++++- + 1 file changed, 5 insertions(+), 1 deletion(-) + +diff --git a/configure.py b/configure.py +index beb08df088..e3f78f2fed 100755 +--- a/configure.py ++++ b/configure.py +@@ -680,7 +680,11 @@ def pkg_config(pkg): + retval = () + for flag in ['--libs-only-l', '--cflags-only-I', + '--libs-only-L', '--modversion']: +- args += [flag, pkg] ++ args += [flag] ++ if isinstance(pkg, list): ++ args += pkg ++ else: ++ args += [pkg] + try: + proc = subprocess.Popen(shlex.split(pkg_config) + args, + stdout=subprocess.PIPE) +-- +2.25.0 +
meta-oe/recipes-devtools/nodejs/nodejs/0002-build-allow-use-of-system-installed-brotli.patch new file +66 -0
66 lines not on this page · raw message
meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb +6 -1
@@ -20,6 +20,8 @@ SRC_URI = "http://nodejs.org/dist/v${PV}/node-v${PV}.tar.xz \ file://0003-Install-both-binaries-and-use-libdir.patch \ file://0004-v8-don-t-override-ARM-CFLAGS.patch \ file://big-endian.patch \ + file://0001-build-allow-passing-multiple-libs-to-pkg_config.patch \ + file://0002-build-allow-use-of-system-installed-brotli.patch \ " SRC_URI_append_class-target = " \ file://0002-Using-native-binaries.patch \
@@ -51,8 +53,9 @@ ARCHFLAGS_arm = "${@bb.utils.contains('TUNE_FEATURES', 'callconvention-hard', '- GYP_DEFINES_append_mipsel = " mips_arch_variant='r1' " ARCHFLAGS ?= "" -PACKAGECONFIG ??= "ares icu libuv zlib" +PACKAGECONFIG ??= "ares brotli icu libuv zlib" PACKAGECONFIG[ares] = "--shared-cares,,c-ares" +PACKAGECONFIG[brotli] = "--shared-brotli,,brotli" PACKAGECONFIG[icu] = "--with-intl=system-icu,--without-intl,icu" PACKAGECONFIG[libuv] = "--shared-libuv,,libuv" PACKAGECONFIG[nghttp2] = "--shared-nghttp2,,nghttp2"
@@ -81,6 +84,8 @@ python do_unpack() { shutil.rmtree(d.getVar('S') + '/deps/openssl', True) if 'ares' in d.getVar('PACKAGECONFIG'): shutil.rmtree(d.getVar('S') + '/deps/cares', True) + if 'brotli' in d.getVar('PACKAGECONFIG'): + shutil.rmtree(d.getVar('S') + '/deps/brotli', True) if 'libuv' in d.getVar('PACKAGECONFIG'): shutil.rmtree(d.getVar('S') + '/deps/uv', True) if 'nghttp2' in d.getVar('PACKAGECONFIG'):
Signature · 1 line
2.23.0.rc1
can you rebase it on top of master-next and resend
Quoted 75 lines · André Draszik
From: André Draszik <[email protected]>
During the python3 / nodejs update, the dependencies weren't updated, so
using system-gyp ends up trying to use the python2 version of system-
gyp, which will of course fail.
Fixing this to depend on the python3 version of gyp still doesn't
doesn't make things work, though:
ERROR: nodejs-native-12.14.1-r0do_configure: Execution of '.../nodejs-native/12.14.1-r0/temp/run.do_configure.26054' failed with exit code 1:
gyp: Error importingpymod_do_mainmodule(ForEachFormat): No module named 'ForEachFormat' while loading dependencies of .../nodejs-native/12.14.1-r0/node-v12.14.1/node.gyp while trying to load .../nodejs-native/12.14.1-r0/node-v12.14.1/node.gyp Error running GYPThe reason is the following patch nodejs has applied to its version of gyp as of NodeJS v12 (commit
fff922afee6e("deps,build: computetorque_outputsin v8.gyp") upstream):tools/gyp/pylib/gyp/input.py 2020-03-02 12:16:09.956707788 +0000 renamed from pylib/gyp/input.py 2020-03-02 12:36:30.788248197 +0000 +2
-0@@ -890,6 +881,7 @@ def ExpandVariables(input, phase, variab oldwd = os.getcwd() # Python doesn't like os.open('.'): no fchdir. if build_file_dir: # build_file_dir may be None (see above). os.chdir(build_file_dir) + sys.path.append(os.getcwd()) try: parsed_contents = shlex.split(contents)@@ -900,6 +892,7 @@ def ExpandVariables(input, phase, variab "module (%s): %s" % (parsed_contents[0], e)) replacement = str(py_module.DoMain(parsed_contents[1:])).rstrip() finally: + sys.path.pop() os.chdir(oldwd) assert replacement != None elif command_string:Since I'm not sure how to deal with that when using system-gyp, and because the original intention for using system-gyp was to make the previous nodejs version compatible with python3 by ultimately switching to the python3 version of system-gyp which isn't necessary anymore, and given nobody else seems to be using this PACKAGECONFIG, just drop it.
Signed-off-by André Draszik <[email protected]>--- meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb | 3 --- 1 file changed, 3 deletions(-)meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb +0
-4@@ -53,7 +53,6 @@ ARCHFLAGS ?= "" PACKAGECONFIG ??= "ares icu libuv zlib" PACKAGECONFIG[ares] = "--shared-cares,,c-ares" -PACKAGECONFIG[gyp] = ",,gyp-py2-native" PACKAGECONFIG[icu] = "--with-intl=system-icu,--without-intl,icu" PACKAGECONFIG[libuv] = "--shared-libuv,,libuv" PACKAGECONFIG[nghttp2] = "--shared-nghttp2,,nghttp2"@@ -82,8 +81,6 @@ python do_unpack() { shutil.rmtree(d.getVar('S') + '/deps/openssl', True) if 'ares' in d.getVar('PACKAGECONFIG'): shutil.rmtree(d.getVar('S') + '/deps/cares', True) - if 'gyp' in d.getVar('PACKAGECONFIG'): - shutil.rmtree(d.getVar('S') + '/tools/gyp', True) if 'libuv' in d.getVar('PACKAGECONFIG'): shutil.rmtree(d.getVar('S') + '/deps/uv', True) if 'nghttp2' in d.getVar('PACKAGECONFIG'): --2.23.0.rc1
--
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel
Re: [meta-oe][PATCH 3/3] nodejs: allow use of system brotli (and make default)
In reply to André Draszikcan you rebase it on top of master-next and resend
Quoted 181 lines · André Draszik
From: André Draszik <[email protected]>
Use system brotli via PACKAGECONFIG by default. So far, nodejs had been built using its embedded copy of brotli, which we generally try to avoid, for the known reasons (independent updates, cve & license checks, etc).
The nodejs patches to enable this have been submitted. brotli is in meta-oe, so enabling this by default should not be a problem.
Signed-off-by André Draszik <[email protected]>--- ...-passing-multiple-libs-to-pkg_config.patch | 41 ++++++++++++ ...allow-use-of-system-installed-brotli.patch | 66 +++++++++++++++++++ .../recipes-devtools/nodejs/nodejs_12.14.1.bb | 7 +- 3 files changed, 113 insertions(+), 1 deletion(-) create mode 100644 meta-oe/recipes-devtools/nodejs/nodejs/0001-build-allow-passing-multiple-libs-to-pkg_config.patch create mode 100644 meta-oe/recipes-devtools/nodejs/nodejs/0002-build-allow-use-of-system-installed-brotli.patchmeta-oe/recipes-devtools/nodejs/nodejs/0001-build-allow-passing-multiple-libs-to-pkg_config.patch new file +41
-0@@ -0,0 +1,41 @@ +From fdaa0e3bef93c5c72a7258b5f1e30718e7d81f9b Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Andr=C3=A9=20Draszik?= <[email protected]> +Date: Mon, 2 Mar 2020 12:17:09 +0000 +Subject: [PATCH 1/2] build: allow passing multiple libs to pkg_config +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Sometimes it's necessary to pass multiple library names to pkg-config, +e.g. the brotli shared libraries can be pulled in with + pkg-config libbrotlienc libbrotlidec + +Update the code to handle both, strings (as used so far), and lists +of strings. + +Signed-off-by: André Draszik <[email protected]> +--- +Upstream-Status: Submitted [https://github.com/nodejs/node/pull/32046] + configure.py | 6 +++++- + 1 file changed, 5 insertions(+), 1 deletion(-) + +diff --git a/configure.py b/configure.py +index beb08df088..e3f78f2fed 100755 +--- a/configure.py ++++ b/configure.py +@@ -680,7 +680,11 @@ def pkg_config(pkg): + retval = () + for flag in ['--libs-only-l', '--cflags-only-I', + '--libs-only-L', '--modversion']: +- args += [flag, pkg] ++ args += [flag] ++ if isinstance(pkg, list): ++ args += pkg ++ else: ++ args += [pkg] + try: + proc = subprocess.Popen(shlex.split(pkg_config) + args, + stdout=subprocess.PIPE) +-- +2.25.0 +meta-oe/recipes-devtools/nodejs/nodejs/0002-build-allow-use-of-system-installed-brotli.patch new file +66
-0@@ -0,0 +1,66 @@ +From f0f927feee8cb1fb173835d5c3f6beb6bf7d5e54 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Andr=C3=A9=20Draszik?= <[email protected]> +Date: Mon, 2 Mar 2020 12:17:35 +0000 +Subject: [PATCH 2/2] build: allow use of system-installed brotli +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +brotli is available as a shared library since 2016, so it makes sense +to allow its use as a system-installed version. + +Some of the infrastructure was in place already (node.gyp and +node.gypi), but some bits in the configure script here were missing. + +Add them, keeping the default as before, to use the bundled version. + +Refs: https://github.com/google/brotli/pull/421 +Signed-off-by: André Draszik <[email protected]> +--- +Upstream-Status: Submitted [https://github.com/nodejs/node/pull/32046] + configure.py | 22 ++++++++++++++++++++++ + 1 file changed, 22 insertions(+) + +diff --git a/configure.py b/configure.py +index e3f78f2fed..0190e31b41 100755 +--- a/configure.py ++++ b/configure.py +@@ -301,6 +301,27 @@ shared_optgroup.add_option('--shared-zlib-libpath', + dest='shared_zlib_libpath', + help='a directory to search for the shared zlib DLL') + ++shared_optgroup.add_option('--shared-brotli', ++ action='store_true', ++ dest='shared_brotli', ++ help='link to a shared brotli DLL instead of static linking') ++ ++shared_optgroup.add_option('--shared-brotli-includes', ++ action='store', ++ dest='shared_brotli_includes', ++ help='directory containing brotli header files') ++ ++shared_optgroup.add_option('--shared-brotli-libname', ++ action='store', ++ dest='shared_brotli_libname', ++ default='brotlidec,brotlienc', ++ help='alternative lib name to link to [default: %default]') ++ ++shared_optgroup.add_option('--shared-brotli-libpath', ++ action='store', ++ dest='shared_brotli_libpath', ++ help='a directory to search for the shared brotli DLL') ++ + shared_optgroup.add_option('--shared-cares', + action='store_true', + dest='shared_cares', +@@ -1692,6 +1713,7 @@ configure_napi(output) + configure_library('zlib', output) + configure_library('http_parser', output) + configure_library('libuv', output) ++configure_library('brotli', output, pkgname=['libbrotlidec', 'libbrotlienc']) + configure_library('cares', output, pkgname='libcares') + configure_library('nghttp2', output, pkgname='libnghttp2') + configure_v8(output) +-- +2.25.0 +meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb +6
-2@@ -20,6 +20,8 @@ SRC_URI = "http://nodejs.org/dist/v${PV}/node-v${PV}.tar.xz \ file://0003-Install-both-binaries-and-use-libdir.patch \ file://0004-v8-don-t-override-ARM-CFLAGS.patch \ file://big-endian.patch \ + file://0001-build-allow-passing-multiple-libs-to-pkg_config.patch \ + file://0002-build-allow-use-of-system-installed-brotli.patch \ " SRC_URI_append_class-target = " \ file://0002-Using-native-binaries.patch \@@ -51,8 +53,9 @@ ARCHFLAGS_arm = "${@bb.utils.contains('TUNE_FEATURES', 'callconvention-hard', '- GYP_DEFINES_append_mipsel = " mips_arch_variant='r1' " ARCHFLAGS ?= "" -PACKAGECONFIG ??= "ares icu libuv zlib" +PACKAGECONFIG ??= "ares brotli icu libuv zlib" PACKAGECONFIG[ares] = "--shared-cares,,c-ares" +PACKAGECONFIG[brotli] = "--shared-brotli,,brotli" PACKAGECONFIG[icu] = "--with-intl=system-icu,--without-intl,icu" PACKAGECONFIG[libuv] = "--shared-libuv,,libuv" PACKAGECONFIG[nghttp2] = "--shared-nghttp2,,nghttp2"@@ -81,6 +84,8 @@ python do_unpack() { shutil.rmtree(d.getVar('S') + '/deps/openssl', True) if 'ares' in d.getVar('PACKAGECONFIG'): shutil.rmtree(d.getVar('S') + '/deps/cares', True) + if 'brotli' in d.getVar('PACKAGECONFIG'): + shutil.rmtree(d.getVar('S') + '/deps/brotli', True) if 'libuv' in d.getVar('PACKAGECONFIG'): shutil.rmtree(d.getVar('S') + '/deps/uv', True) if 'nghttp2' in d.getVar('PACKAGECONFIG'): --2.23.0.rc1
--
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-devel