Skip to contentSkip to conversation

nodejs: drop 'gyp' PACKAGECONFIG

PATCH Superseded André Draszik 2 participants 4 messages Mar 3, 2020

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 nodejs: drop 'gyp' PACKAGECONFIG

From: André Draszik <[email protected]>

During the python3 / nodejs update, the dependencies weren't updated, so
using system-gyp ends up trying to use the python2 version of system-
gyp, which will of course fail.
Fixing this to depend on the python3 version of gyp still doesn't
doesn't make things work, though:
ERROR: nodejs-native-12.14.1-r0 do_configure: Execution of '.../nodejs-native/12.14.1-r0/temp/run.do_configure.26054' failed with exit code 1:
gyp: Error importing pymod_do_mainmodule (ForEachFormat): No module named 'ForEachFormat' while loading dependencies of .../nodejs-native/12.14.1-r0/node-v12.14.1/node.gyp while trying to load .../nodejs-native/12.14.1-r0/node-v12.14.1/node.gyp Error running GYP

The reason is the following patch nodejs has applied to its version of gyp as of NodeJS v12 (commit fff922afee6e ("deps,build: compute torque_outputs in v8.gyp") upstream):

tools/gyp/pylib/gyp/input.py 2020-03-02 12:16:09.956707788 +0000 renamed from pylib/gyp/input.py 2020-03-02 12:36:30.788248197 +0000 +2 -0
@@ -890,6 +881,7 @@ def ExpandVariables(input, phase, variab
           oldwd = os.getcwd()  # Python doesn't like os.open('.'): no fchdir.
           if build_file_dir:  # build_file_dir may be None (see above).
             os.chdir(build_file_dir)
+          sys.path.append(os.getcwd())
           try:
 
             parsed_contents = shlex.split(contents)
@@ -900,6 +892,7 @@ def ExpandVariables(input, phase, variab
                              "module (%s): %s" % (parsed_contents[0], e))
             replacement = str(py_module.DoMain(parsed_contents[1:])).rstrip()
           finally:
+            sys.path.pop()
             os.chdir(oldwd)
           assert replacement != None
         elif command_string:

Since I'm not sure how to deal with that when using system-gyp, and because the original intention for using system-gyp was to make the previous nodejs version compatible with python3 by ultimately switching to the python3 version of system-gyp which isn't necessary anymore, and given nobody else seems to be using this PACKAGECONFIG, just drop it.

---
 meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb | 3 ---
 1 file changed, 3 deletions(-)
meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb +0 -3
index 6eb52c209..49bb71e28 100644
@@ -53,7 +53,6 @@ ARCHFLAGS ?= ""
 
 PACKAGECONFIG ??= "ares icu libuv zlib"
 PACKAGECONFIG[ares] = "--shared-cares,,c-ares"
-PACKAGECONFIG[gyp] = ",,gyp-py2-native"
 PACKAGECONFIG[icu] = "--with-intl=system-icu,--without-intl,icu"
 PACKAGECONFIG[libuv] = "--shared-libuv,,libuv"
 PACKAGECONFIG[nghttp2] = "--shared-nghttp2,,nghttp2"
@@ -82,8 +81,6 @@ python do_unpack() {
     shutil.rmtree(d.getVar('S') + '/deps/openssl', True)
     if 'ares' in d.getVar('PACKAGECONFIG'):
         shutil.rmtree(d.getVar('S') + '/deps/cares', True)
-    if 'gyp' in d.getVar('PACKAGECONFIG'):
-        shutil.rmtree(d.getVar('S') + '/tools/gyp', True)
     if 'libuv' in d.getVar('PACKAGECONFIG'):
         shutil.rmtree(d.getVar('S') + '/deps/uv', True)
     if 'nghttp2' in d.getVar('PACKAGECONFIG'):
Signature · 1 line
2.23.0.rc1
#1 lore raw

PATCH v2 2/2 nodejs: allow use of system brotli (and make default)

From: André Draszik <[email protected]>

Use system brotli via PACKAGECONFIG by default. So far, nodejs had been built using its embedded copy of brotli, which we generally try to avoid, for the known reasons (independent updates, cve & license checks, etc).

The nodejs patches to enable this have been submitted. brotli is in meta-oe, so enabling this by default should not be a problem.

---
 ...-passing-multiple-libs-to-pkg_config.patch | 41 ++++++++++++
 ...allow-use-of-system-installed-brotli.patch | 66 +++++++++++++++++++
 .../recipes-devtools/nodejs/nodejs_12.14.1.bb |  7 +-
 3 files changed, 113 insertions(+), 1 deletion(-)
 create mode 100644 meta-oe/recipes-devtools/nodejs/nodejs/0001-build-allow-passing-multiple-libs-to-pkg_config.patch
 create mode 100644 meta-oe/recipes-devtools/nodejs/nodejs/0002-build-allow-use-of-system-installed-brotli.patch
meta-oe/recipes-devtools/nodejs/nodejs/0001-build-allow-passing-multiple-libs-to-pkg_config.patch new file +41 -0
new file mode 100644
index 000000000..13edf229b
@@ -0,0 +1,41 @@
+From fdaa0e3bef93c5c72a7258b5f1e30718e7d81f9b Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Andr=C3=A9=20Draszik?= <[email protected]>
+Date: Mon, 2 Mar 2020 12:17:09 +0000
+Subject: [PATCH 1/2] build: allow passing multiple libs to pkg_config
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+Sometimes it's necessary to pass multiple library names to pkg-config,
+e.g. the brotli shared libraries can be pulled in with
+    pkg-config libbrotlienc libbrotlidec
+
+Update the code to handle both, strings (as used so far), and lists
+of strings.
+
+Signed-off-by: André Draszik <[email protected]>
+---
+Upstream-Status: Submitted [https://github.com/nodejs/node/pull/32046]
+ configure.py | 6 +++++-
+ 1 file changed, 5 insertions(+), 1 deletion(-)
+
+diff --git a/configure.py b/configure.py
+index beb08df088..e3f78f2fed 100755
+--- a/configure.py
++++ b/configure.py
+@@ -680,7 +680,11 @@ def pkg_config(pkg):
+   retval = ()
+   for flag in ['--libs-only-l', '--cflags-only-I',
+                '--libs-only-L', '--modversion']:
+-    args += [flag, pkg]
++    args += [flag]
++    if isinstance(pkg, list):
++      args += pkg
++    else:
++      args += [pkg]
+     try:
+       proc = subprocess.Popen(shlex.split(pkg_config) + args,
+                               stdout=subprocess.PIPE)
+-- 
+2.25.0
+
meta-oe/recipes-devtools/nodejs/nodejs/0002-build-allow-use-of-system-installed-brotli.patch new file +66 -0

66 lines not on this page · raw message

meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb +6 -1
index 49bb71e28..1ea438c5b 100644
@@ -20,6 +20,8 @@ SRC_URI = "http://nodejs.org/dist/v${PV}/node-v${PV}.tar.xz \
            file://0003-Install-both-binaries-and-use-libdir.patch \
            file://0004-v8-don-t-override-ARM-CFLAGS.patch \
            file://big-endian.patch \
+           file://0001-build-allow-passing-multiple-libs-to-pkg_config.patch \
+           file://0002-build-allow-use-of-system-installed-brotli.patch \
            "
 SRC_URI_append_class-target = " \
            file://0002-Using-native-binaries.patch \
@@ -51,8 +53,9 @@ ARCHFLAGS_arm = "${@bb.utils.contains('TUNE_FEATURES', 'callconvention-hard', '-
 GYP_DEFINES_append_mipsel = " mips_arch_variant='r1' "
 ARCHFLAGS ?= ""
 
-PACKAGECONFIG ??= "ares icu libuv zlib"
+PACKAGECONFIG ??= "ares brotli icu libuv zlib"
 PACKAGECONFIG[ares] = "--shared-cares,,c-ares"
+PACKAGECONFIG[brotli] = "--shared-brotli,,brotli"
 PACKAGECONFIG[icu] = "--with-intl=system-icu,--without-intl,icu"
 PACKAGECONFIG[libuv] = "--shared-libuv,,libuv"
 PACKAGECONFIG[nghttp2] = "--shared-nghttp2,,nghttp2"
@@ -81,6 +84,8 @@ python do_unpack() {
     shutil.rmtree(d.getVar('S') + '/deps/openssl', True)
     if 'ares' in d.getVar('PACKAGECONFIG'):
         shutil.rmtree(d.getVar('S') + '/deps/cares', True)
+    if 'brotli' in d.getVar('PACKAGECONFIG'):
+        shutil.rmtree(d.getVar('S') + '/deps/brotli', True)
     if 'libuv' in d.getVar('PACKAGECONFIG'):
         shutil.rmtree(d.getVar('S') + '/deps/uv', True)
     if 'nghttp2' in d.getVar('PACKAGECONFIG'):
Signature · 1 line
2.23.0.rc1
#2 lore raw
In reply to André Draszik
Quoted 26 lines · André Draszik

From: André Draszik <[email protected]>

During the python3 / nodejs update, the dependencies weren't updated, so
using system-gyp ends up trying to use the python2 version of system-
gyp, which will of course fail.
Fixing this to depend on the python3 version of gyp still doesn't
doesn't make things work, though:
ERROR: nodejs-native-12.14.1-r0 do_configure: Execution of '.../nodejs-native/12.14.1-r0/temp/run.do_configure.26054' failed with exit code 1:
gyp: Error importing pymod_do_mainmodule (ForEachFormat): No module named 'ForEachFormat' while loading dependencies of .../nodejs-native/12.14.1-r0/node-v12.14.1/node.gyp while trying to load .../nodejs-native/12.14.1-r0/node-v12.14.1/node.gyp Error running GYP

The reason is the following patch nodejs has applied to its version of gyp as of NodeJS v12 (commit fff922afee6e ("deps,build: compute torque_outputs in v8.gyp") upstream):

tools/gyp/pylib/gyp/input.py 2020-03-02 12:16:09.956707788 +0000 renamed from pylib/gyp/input.py 2020-03-02 12:36:30.788248197 +0000 +1 -0
@@ -890,6 +881,7 @@ def ExpandVariables(input, phase, variab
            oldwd = os.getcwd()  # Python doesn't like os.open('.'): no fchdir.
            if build_file_dir:  # build_file_dir may be None (see above).
              os.chdir(build_file_dir)
+          sys.path.append(os.getcwd())
            try:
 
              parsed_contents = shlex.split(contents)

tools/gyp/pylib/gyp/input.py 2020-03-02 12:16:09.956707788 +0000 def ExpandVariables(input, phase, variab · line 898

Quoted 7 lines
tools/gyp/pylib/gyp/input.py 2020-03-02 12:16:09.956707788 +0000 renamed from pylib/gyp/input.py 2020-03-02 12:36:30.788248197 +0000 +1 -0
@@ -900,6 +892,7 @@ def ExpandVariables(input, phase, variab
                               "module (%s): %s" % (parsed_contents[0], e))
              replacement = str(py_module.DoMain(parsed_contents[1:])).rstrip()
            finally:
+            sys.path.pop()
              os.chdir(oldwd)
            assert replacement != None
          elif command_string:

Please avoid adding diffs to commit message, this confuses git am perhaps better to express the change with few words.

Quoted 34 lines

Since I'm not sure how to deal with that when using system-gyp, and because the original intention for using system-gyp was to make the previous nodejs version compatible with python3 by ultimately switching to the python3 version of system-gyp which isn't necessary anymore, and given nobody else seems to be using this PACKAGECONFIG, just drop it.

---
  meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb | 3 ---
  1 file changed, 3 deletions(-)
meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb +0 -3
index 6eb52c209..49bb71e28 100644
@@ -53,7 +53,6 @@ ARCHFLAGS ?= ""
 
  PACKAGECONFIG ??= "ares icu libuv zlib"
  PACKAGECONFIG[ares] = "--shared-cares,,c-ares"
-PACKAGECONFIG[gyp] = ",,gyp-py2-native"
  PACKAGECONFIG[icu] = "--with-intl=system-icu,--without-intl,icu"
  PACKAGECONFIG[libuv] = "--shared-libuv,,libuv"
  PACKAGECONFIG[nghttp2] = "--shared-nghttp2,,nghttp2"
@@ -82,8 +81,6 @@ python do_unpack() {
      shutil.rmtree(d.getVar('S') + '/deps/openssl', True)
      if 'ares' in d.getVar('PACKAGECONFIG'):
          shutil.rmtree(d.getVar('S') + '/deps/cares', True)
-    if 'gyp' in d.getVar('PACKAGECONFIG'):
-        shutil.rmtree(d.getVar('S') + '/tools/gyp', True)
      if 'libuv' in d.getVar('PACKAGECONFIG'):
          shutil.rmtree(d.getVar('S') + '/deps/uv', True)
      if 'nghttp2' in d.getVar('PACKAGECONFIG'):
#3 lore raw
Quoted 29 lines · Khem Raj
Quoted 27 lines · André Draszik

From: André Draszik <[email protected]>

During the python3 / nodejs update, the dependencies weren't updated, so using system-gyp ends up trying to use the python2 version of system- gyp, which will of course fail.
Fixing this to depend on the python3 version of gyp still doesn't doesn't make things work, though:

     ERROR: nodejs-native-12.14.1-r0 do_configure: Execution of '.../nodejs-native/12.14.1-

r0/temp/run.do_configure.26054' failed with exit code 1:
gyp: Error importing pymod_do_mainmodule (ForEachFormat): No module named 'ForEachFormat' while loading
dependencies of .../nodejs-native/12.14.1-r0/node-v12.14.1/node.gyp while trying to load .../nodejs-native/12.14.1- r0/node-v12.14.1/node.gyp
Error running GYP

The reason is the following patch nodejs has applied to its version of gyp as of NodeJS v12 (commit fff922afee6e ("deps,build: compute torque_outputs in v8.gyp") upstream):

tools/gyp/pylib/gyp/input.py 2020-03-02 12:16:09.956707788 +0000 renamed from pylib/gyp/input.py 2020-03-02 12:36:30.788248197 +0000 +1 -0
@@ -890,6 +881,7 @@ def ExpandVariables(input, phase, variab
            oldwd = os.getcwd()  # Python doesn't like os.open('.'): no fchdir.
            if build_file_dir:  # build_file_dir may be None (see above).
              os.chdir(build_file_dir)
+          sys.path.append(os.getcwd())
            try:
 
Quoted 12 lines

tools/gyp/pylib/gyp/input.py 2020-03-02 12:16:09.956707788 +0000 def ExpandVariables(input, phase, variab · line 898

Quoted 9 lines
tools/gyp/pylib/gyp/input.py 2020-03-02 12:16:09.956707788 +0000 renamed from pylib/gyp/input.py 2020-03-02 12:36:30.788248197 +0000 +1 -0
              parsed_contents = shlex.split(contents)
@@ -900,6 +892,7 @@ def ExpandVariables(input, phase, variab
                               "module (%s): %s" % (parsed_contents[0], e))
              replacement = str(py_module.DoMain(parsed_contents[1:])).rstrip()
            finally:
+            sys.path.pop()
              os.chdir(oldwd)
            assert replacement != None
          elif command_string:

Please avoid adding diffs to commit message, this confuses git am perhaps better to express the change with few words.

Sorry Khem, I didn't think of that... Done.

A.

Quoted 34 lines
Quoted 34 lines

Since I'm not sure how to deal with that when using system-gyp, and because the original intention for using system-gyp was to make the previous nodejs version compatible with python3 by ultimately switching to the python3 version of system-gyp which isn't necessary anymore, and given nobody else seems to be using this PACKAGECONFIG, just drop it.

---
  meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb | 3 ---
  1 file changed, 3 deletions(-)
meta-oe/recipes-devtools/nodejs/nodejs_12.14.1.bb +0 -3
index 6eb52c209..49bb71e28 100644
@@ -53,7 +53,6 @@ ARCHFLAGS ?= ""
 
  PACKAGECONFIG ??= "ares icu libuv zlib"
  PACKAGECONFIG[ares] = "--shared-cares,,c-ares"
-PACKAGECONFIG[gyp] = ",,gyp-py2-native"
  PACKAGECONFIG[icu] = "--with-intl=system-icu,--without-intl,icu"
  PACKAGECONFIG[libuv] = "--shared-libuv,,libuv"
  PACKAGECONFIG[nghttp2] = "--shared-nghttp2,,nghttp2"
@@ -82,8 +81,6 @@ python do_unpack() {
      shutil.rmtree(d.getVar('S') + '/deps/openssl', True)
      if 'ares' in d.getVar('PACKAGECONFIG'):
          shutil.rmtree(d.getVar('S') + '/deps/cares', True)
-    if 'gyp' in d.getVar('PACKAGECONFIG'):
-        shutil.rmtree(d.getVar('S') + '/tools/gyp', True)
      if 'libuv' in d.getVar('PACKAGECONFIG'):
          shutil.rmtree(d.getVar('S') + '/deps/uv', True)
      if 'nghttp2' in d.getVar('PACKAGECONFIG'):