Skip to contentSkip to conversation

pkg-luarocks: remove LUAROCKS_RUN_ENV

PATCH New Francois Perrad 3 participants 16 messages Sep 18, 2019 – Mar 20, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/6 pkg-luarocks: remove LUAROCKS_RUN_ENV

---
 package/luarocks/luarocks.mk | 5 -----
 package/pkg-luarocks.mk      | 4 ++--
 2 files changed, 2 insertions(+), 7 deletions(-)
package/luarocks/luarocks.mk +0 -5
index cd547e280..9cf5aba8a 100644
@@ -48,11 +48,6 @@ endef
 
 $(eval $(host-generic-package))
 
-LUAROCKS_RUN_ENV = \
-	LUA_PATH="$(HOST_DIR)/share/lua/$(LUAINTERPRETER_ABIVER)/?.lua" \
-	TARGET_CC="$(TARGET_CC)" \
-	TARGET_CFLAGS="$(LUAROCKS_CFLAGS)" \
-	TARGET_LDFLAGS="$(TARGET_LDFLAGS)"
 LUAROCKS_RUN_CMD = $(LUA_RUN) $(HOST_DIR)/bin/luarocks
 
 define LUAROCKS_FINALIZE_TARGET
package/pkg-luarocks.mk +2 -2
index 78d6c325f..ede9b2c83 100644
@@ -53,7 +53,7 @@ ifndef $(2)_EXTRACT_CMDS
 define $(2)_EXTRACT_CMDS
 	mkdir -p $$($(2)_DIR)/luarocks-extract
 	cd $$($(2)_DIR)/luarocks-extract && \
-		$$(LUAROCKS_RUN_ENV) $$(LUAROCKS_RUN_CMD) unpack --force $$($(2)_DL_DIR)/$$($(2)_SOURCE)
+		$$(LUAROCKS_RUN_CMD) unpack --force $$($(2)_DL_DIR)/$$($(2)_SOURCE)
 	mv $$($(2)_DIR)/luarocks-extract/*/* $$($(2)_DIR)
 endef
 endif
@@ -63,7 +63,7 @@ endif
 #
 ifndef $(2)_INSTALL_TARGET_CMDS
 define $(2)_INSTALL_TARGET_CMDS
-	cd $$($(2)_SRCDIR) && $$(LUAROCKS_RUN_ENV) \
+	cd $$($(2)_SRCDIR) && \
 		$$(LUAROCKS_RUN_CMD) make --keep $$($(2)_ROCKSPEC) $$($(2)_BUILD_OPTS)
 endef
 endif
Signature · 1 line
2.20.1
#1 lore raw

PATCH 2/6 pkg-luarocks: refactor infra

all variables containing BR data (except external_deps_dirs) are now options of the command luarocks make

paths for external dependencies are now handled by the BR addon which generates the variable _BUILD_OPTS with the content excepted by luarocks make

and the remaining content of luarocks-br-config.lua is now inlined in luarocks.mk

here, the config file is usable for host-luarocks

---
 docs/manual/adding-packages-luarocks.txt | 12 ++++++------
 package/luarocks/buildroot.lua           | 21 ++++++++++++++++++++-
 package/luarocks/luarocks-br-config.lua  | 13 -------------
 package/luarocks/luarocks.mk             | 17 +++--------------
 package/pkg-luarocks.mk                  | 16 +++++++++++++++-
 5 files changed, 44 insertions(+), 35 deletions(-)
 delete mode 100644 package/luarocks/luarocks-br-config.lua
docs/manual/adding-packages-luarocks.txt +6 -6
index df97cdd50..f7104f373 100644
@@ -19,10 +19,10 @@ with an example :
 06:
 07: LUA_FOO_VERSION = 1.0.2-1
 08: LUA_FOO_NAME_UPSTREAM = foo
-09: LUA_FOO_DEPENDENCIES = bar
-10:
-11: LUA_FOO_BUILD_OPTS += BAR_INCDIR=$(STAGING_DIR)/usr/include
-12: LUA_FOO_BUILD_OPTS += BAR_LIBDIR=$(STAGING_DIR)/usr/lib
+09:
+10: LUA_FOO_DEPENDENCIES = bar
+11: LUA_FOO_BUILD_OPTS = BAR_DIR=$(STAGING_DIR)/usr
+12:
 13: LUA_FOO_LICENSE = luaFoo license
 14: LUA_FOO_LICENSE_FILES = $(LUA_FOO_SUBDIR)/COPYING
 15:
@@ -38,10 +38,10 @@ Buildroot, we give Lua-related packages a name that starts with "lua", so the
 Buildroot name is different from the upstream name. +LUA_FOO_NAME_UPSTREAM+
 makes the link between the two names.
 
-On line 9, we declare our dependencies against native libraries, so that they
+On line 10, we declare our dependencies against native libraries, so that they
 are built before the build process of our package starts.
 
-On lines 11-12, we tell Buildroot to pass custom options to LuaRocks when it is
+On line 11, we tell Buildroot to pass custom options to LuaRocks when it is
 building the package.
 
 On lines 13-14, we specify the licensing terms for the package.
package/luarocks/buildroot.lua +20 -1
index e829b6db0..e3ab3f7f1 100644
@@ -94,9 +94,18 @@ local function get_main_modules (rockspec)
    return t
 end
 
-local function get_external_dependencies (rockspec)
+local function get_raw_external_dependencies (rockspec)
    local t = {}
    for k in pairs(rockspec.external_dependencies or {}) do
+      t[#t+1] = k
+   end
+   table.sort(t)
+   return t
+end
+
+local function get_external_dependencies (rockspec)
+   local t = {}
+   for _, k in ipairs(get_raw_external_dependencies(rockspec)) do
       k = k:lower()
       if fs.is_dir('package/' .. k) then
          t[#t+1] = k
@@ -254,6 +263,16 @@ local function generate_mk (rockspec, lcname, licenses)
    end
    if #external_dependencies > 0 then
       f:write(ucname .. '_DEPENDENCIES = ' .. table.concat(external_dependencies, ' ') .. '\n')
+      local deps = get_raw_external_dependencies(rockspec)
+      if #deps == 1 then
+         f:write(ucname .. '_BUILD_OPTS = ' .. deps[1] .. '_DIR="$(STAGING_DIR)/usr"\n')
+      else
+         f:write(ucname .. '_BUILD_OPTS =')
+         for i = 1, #deps do
+            f:write(' \\\n\t' .. deps[i] .. '_DIR="$(STAGING_DIR)/usr"')
+         end
+         f:write('\n')
+      end
    end
    f:write('\n$(eval $(luarocks-package))\n')
    f:close()
package/luarocks/luarocks-br-config.lua deleted +0 -13
deleted file mode 100644
index 32d71e1e2..000000000
@@ -1,13 +0,0 @@
--- BR cross-compilation
-local function getenv (name) return os_getenv(name) or '' end
-variables.LUA_INCDIR = getenv('STAGING_DIR') .. [[/usr/include]]
-variables.LUA_LIBDIR = getenv('STAGING_DIR') .. [[/usr/lib]]
-variables.CC = getenv('TARGET_CC')
-variables.LD = getenv('TARGET_CC')
-variables.CFLAGS = getenv('TARGET_CFLAGS')
-variables.LIBFLAG = [[-shared ]] .. getenv('TARGET_LDFLAGS')
-external_deps_dirs = { getenv('STAGING_DIR') .. [[/usr]] }
-gcc_rpath = false
-rocks_trees = { getenv('TARGET_DIR') .. [[/usr]] }
-wrap_bin_scripts = false
-deps_mode = [[none]]
package/luarocks/luarocks.mk +3 -14
index 9cf5aba8a..27fb753d5 100644
@@ -13,10 +13,6 @@ HOST_LUAROCKS_DEPENDENCIES = host-luainterpreter
 
 LUAROCKS_CONFIG_DIR = $(HOST_DIR)/etc
 LUAROCKS_CONFIG_FILE = $(LUAROCKS_CONFIG_DIR)/luarocks/config-$(LUAINTERPRETER_ABIVER).lua
-LUAROCKS_CFLAGS = $(TARGET_CFLAGS) -fPIC
-ifeq ($(BR2_PACKAGE_LUA_5_3),y)
-LUAROCKS_CFLAGS += -DLUA_COMPAT_5_2
-endif
 
 define LUAROCKS_ADDON_EXTRACT
 	mkdir $(@D)/src/luarocks/cmd/external
@@ -33,23 +29,16 @@ define HOST_LUAROCKS_CONFIGURE_CMDS
 	cd $(@D) && ./configure $(HOST_LUAROCKS_CONF_OPTS)
 endef
 
-ifeq ($(BR2_PACKAGE_LUAJIT),y)
-define LUAROCKS_CONFIGURE_INTERPRETER_LUAJIT
-	echo "lua_interpreter = [[luajit]]" >> $(LUAROCKS_CONFIG_FILE)
-endef
-endif
-
 define HOST_LUAROCKS_INSTALL_CMDS
 	rm -f $(LUAROCKS_CONFIG_FILE)
 	$(MAKE1) -C $(@D) install
-	cat $(HOST_LUAROCKS_PKGDIR)/luarocks-br-config.lua >> $(LUAROCKS_CONFIG_FILE)
-	$(LUAROCKS_CONFIGURE_INTERPRETER_LUAJIT)
+	echo "-- BR tricks"                     >> $(LUAROCKS_CONFIG_FILE)
+	echo "gcc_rpath = false"                >> $(LUAROCKS_CONFIG_FILE)
+	echo "wrap_bin_scripts = false"         >> $(LUAROCKS_CONFIG_FILE)
 endef
 
 $(eval $(host-generic-package))
 
-LUAROCKS_RUN_CMD = $(LUA_RUN) $(HOST_DIR)/bin/luarocks
-
 define LUAROCKS_FINALIZE_TARGET
 	rm -rf $(TARGET_DIR)/usr/lib/luarocks
 endef
package/pkg-luarocks.mk +15 -1
index ede9b2c83..23906f4a8 100644
@@ -17,6 +17,12 @@
 #
 ################################################################################
 
+LUAROCKS_RUN_CMD = $(HOST_DIR)/bin/luarocks
+LUAROCKS_CFLAGS = $(TARGET_CFLAGS) -fPIC
+ifeq ($(BR2_PACKAGE_LUA_5_3),y)
+LUAROCKS_CFLAGS += -DLUA_COMPAT_5_2
+endif
+
 ################################################################################
 # inner-luarocks-package -- defines how the configuration, compilation and
 # installation of a LuaRocks package should be done, implements a few hooks to
@@ -64,7 +70,15 @@ endif
 ifndef $(2)_INSTALL_TARGET_CMDS
 define $(2)_INSTALL_TARGET_CMDS
 	cd $$($(2)_SRCDIR) && \
-		$$(LUAROCKS_RUN_CMD) make --keep $$($(2)_ROCKSPEC) $$($(2)_BUILD_OPTS)
+		$$(LUAROCKS_RUN_CMD) make --keep --deps-mode none \
+			--tree "$$(TARGET_DIR)/usr" \
+			LUA_INCDIR="$$(STAGING_DIR)/usr/include" \
+			LUA_LIBDIR="$$(STAGING_DIR)/usr/lib" \
+			CC=$$(TARGET_CC) \
+			LD=$$(TARGET_CC) \
+			CFLAGS="$$(LUAROCKS_CFLAGS)" \
+			LIBFLAG="-shared $$(TARGET_LDFLAGS)" \
+			$$($(2)_BUILD_OPTS) $$($(2)_ROCKSPEC)
 endef
 endif
 
Signature · 1 line
2.20.1
#2 lore raw

PATCH 3/6 pkg-luarocks: regen packages with external dependencies

In reply to Francois Perrad
---
 package/lsqlite3/lsqlite3.mk             | 1 +
 package/lua-curl/lua-curl.mk             | 1 +
 package/lua-flu/lua-flu.mk               | 3 +++
 package/lua-iconv/lua-iconv.mk           | 1 +
 package/luadbi-sqlite3/luadbi-sqlite3.mk | 1 +
 package/luaexpat/luaexpat.mk             | 1 +
 package/luaossl/luaossl.mk               | 3 +++
 package/luasec/luasec.mk                 | 1 +
 package/luasql-sqlite3/luasql-sqlite3.mk | 1 +
 package/wsapi-fcgi/wsapi-fcgi.mk         | 1 +
 10 files changed, 14 insertions(+)
package/lsqlite3/lsqlite3.mk +1 -0
index bbf6a1db1..51f9516fd 100644
@@ -8,5 +8,6 @@ LSQLITE3_VERSION = 0.9.5-1
 LSQLITE3_SUBDIR = lsqlite3_fsl09y
 LSQLITE3_LICENSE = MIT
 LSQLITE3_DEPENDENCIES = sqlite
+LSQLITE3_BUILD_OPTS = SQLITE_DIR="$(STAGING_DIR)/usr"
 
 $(eval $(luarocks-package))
package/lua-curl/lua-curl.mk +1 -0
index d887cb511..13b1d6d9f 100644
@@ -10,5 +10,6 @@ LUA_CURL_SUBDIR = Lua-cURLv3-$(LUA_CURL_VERSION_UPSTREAM)
 LUA_CURL_LICENSE = MIT
 LUA_CURL_LICENSE_FILES = $(LUA_CURL_SUBDIR)/LICENSE
 LUA_CURL_DEPENDENCIES = libcurl
+LUA_CURL_BUILD_OPTS = CURL_DIR="$(STAGING_DIR)/usr"
 
 $(eval $(luarocks-package))
package/lua-flu/lua-flu.mk +3 -0
index 4c7b76547..0c65e6da6 100644
@@ -10,5 +10,8 @@ LUA_FLU_SUBDIR = doub-flu-63f077a988cd
 LUA_FLU_LICENSE = MIT
 LUA_FLU_LICENSE_FILES = $(LUA_FLU_SUBDIR)/doc/LICENSE.txt
 LUA_FLU_DEPENDENCIES = attr libfuse
+LUA_FLU_BUILD_OPTS = \
+	ATTR_DIR="$(STAGING_DIR)/usr" \
+	FUSE_DIR="$(STAGING_DIR)/usr"
 
 $(eval $(luarocks-package))
package/lua-iconv/lua-iconv.mk +1 -0
index ea8e0dd8b..ea0cca29b 100644
@@ -8,5 +8,6 @@ LUA_ICONV_VERSION = 7-1
 LUA_ICONV_LICENSE = MIT
 LUA_ICONV_LICENSE_FILES = $(LUA_ICONV_SUBDIR)/COPYING
 LUA_ICONV_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv)
+LUA_ICONV_BUILD_OPTS = ICONV_DIR="$(STAGING_DIR)/usr"
 
 $(eval $(luarocks-package))
package/luadbi-sqlite3/luadbi-sqlite3.mk +1 -0
index 4ae2b5cf3..755803c43 100644
@@ -9,5 +9,6 @@ LUADBI_SQLITE3_SUBDIR = luadbi
 LUADBI_SQLITE3_LICENSE = MIT
 LUADBI_SQLITE3_LICENSE_FILES = $(LUADBI_SQLITE3_SUBDIR)/COPYING
 LUADBI_SQLITE3_DEPENDENCIES = sqlite
+LUADBI_SQLITE3_BUILD_OPTS = SQLITE_DIR="$(STAGING_DIR)/usr"
 
 $(eval $(luarocks-package))
package/luaexpat/luaexpat.mk +1 -0
index 8c86b24f2..74533385c 100644
@@ -8,5 +8,6 @@ LUAEXPAT_VERSION = 1.3.3-1
 LUAEXPAT_LICENSE = MIT
 LUAEXPAT_LICENSE_FILES = $(LUAEXPAT_SUBDIR)/LICENSE
 LUAEXPAT_DEPENDENCIES = expat
+LUAEXPAT_BUILD_OPTS = EXPAT_DIR="$(STAGING_DIR)/usr"
 
 $(eval $(luarocks-package))
package/luaossl/luaossl.mk +3 -0
index 128c320d1..8a636dc74 100644
@@ -10,5 +10,8 @@ LUAOSSL_SUBDIR = luaossl-rel-$(LUAOSSL_VERSION_UPSTREAM)
 LUAOSSL_LICENSE = MIT
 LUAOSSL_LICENSE_FILES = $(LUAOSSL_SUBDIR)/LICENSE
 LUAOSSL_DEPENDENCIES = openssl
+LUAOSSL_BUILD_OPTS = \
+	CRYPTO_DIR="$(STAGING_DIR)/usr" \
+	OPENSSL_DIR="$(STAGING_DIR)/usr"
 
 $(eval $(luarocks-package))
package/luasec/luasec.mk +1 -0
index 9ce358753..e338bb74a 100644
@@ -10,5 +10,6 @@ LUASEC_SUBDIR = luasec-luasec-$(LUASEC_VERSION_UPSTREAM)
 LUASEC_LICENSE = MIT
 LUASEC_LICENSE_FILES = $(LUASEC_SUBDIR)/LICENSE
 LUASEC_DEPENDENCIES = openssl
+LUASEC_BUILD_OPTS = OPENSSL_DIR="$(STAGING_DIR)/usr"
 
 $(eval $(luarocks-package))
package/luasql-sqlite3/luasql-sqlite3.mk +1 -0
index 69fc956c9..bcb8676af 100644
@@ -9,5 +9,6 @@ LUASQL_SQLITE3_SUBDIR = luasql
 LUASQL_SQLITE3_LICENSE = MIT
 LUASQL_SQLITE3_LICENSE_FILES = $(LUASQL_SQLITE3_SUBDIR)/doc/us/license.html
 LUASQL_SQLITE3_DEPENDENCIES = sqlite
+LUASQL_SQLITE3_BUILD_OPTS = SQLITE_DIR="$(STAGING_DIR)/usr"
 
 $(eval $(luarocks-package))
package/wsapi-fcgi/wsapi-fcgi.mk +1 -0
index 677bb774b..d1cb06f79 100644
@@ -11,5 +11,6 @@ WSAPI_FCGI_LICENSE_FILES = \
 	$(WSAPI_FCGI_SUBDIR)/doc/us/license.html \
 	$(WSAPI_FCGI_SUBDIR)/doc/us/license.md
 WSAPI_FCGI_DEPENDENCIES = libfcgi
+WSAPI_FCGI_BUILD_OPTS = FASTCGI_DIR="$(STAGING_DIR)/usr"
 
 $(eval $(luarocks-package))
Signature · 1 line
2.20.1
#3 lore raw

PATCH 3/6 pkg-luarocks: regen packages with external dependencies

Quoted 19 lines · On Wed, 18 Sep 2019 08:19:12 +0200 Francois Perrad <[email protected]> wrote:
---
 package/lsqlite3/lsqlite3.mk             | 1 +
 package/lua-curl/lua-curl.mk             | 1 +
 package/lua-flu/lua-flu.mk               | 3 +++
 package/lua-iconv/lua-iconv.mk           | 1 +
 package/luadbi-sqlite3/luadbi-sqlite3.mk | 1 +
 package/luaexpat/luaexpat.mk             | 1 +
 package/luaossl/luaossl.mk               | 3 +++
 package/luasec/luasec.mk                 | 1 +
 package/luasql-sqlite3/luasql-sqlite3.mk | 1 +
 package/wsapi-fcgi/wsapi-fcgi.mk         | 1 +
 10 files changed, 14 insertions(+)
package/lsqlite3/lsqlite3.mk +0 -0
index bbf6a1db1..51f9516fd 100644

package/lsqlite3/lsqlite3.mk LSQLITE3_VERSION = 0.9.5-1 · line 11

Quoted 4 lines
package/lsqlite3/lsqlite3.mk +1 -0
@@ -8,5 +8,6 @@ LSQLITE3_VERSION = 0.9.5-1
 LSQLITE3_SUBDIR = lsqlite3_fsl09y
 LSQLITE3_LICENSE = MIT
 LSQLITE3_DEPENDENCIES = sqlite
+LSQLITE3_BUILD_OPTS = SQLITE_DIR="$(STAGING_DIR)/usr"

I'm not sure to understand this change. It seems a bit redundant for all those packages to say "the native libraries you need are in $(STAGING_DIR)/usr", since native libraries are always there. Why isn't the common infrastructure taking care of this ?

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#4 lore raw

PATCH 3/6 pkg-luarocks: regen packages with external dependencies

Le mer. 18 sept. 2019 ? 09:35, Thomas Petazzoni <
[email protected]> a ?crit :

Quoted 17 lines
Quoted 14 lines · On Wed, 18 Sep 2019 08:19:12 +0200 Francois Perrad <[email protected]> wrote:
---
 package/lsqlite3/lsqlite3.mk             | 1 +
 package/lua-curl/lua-curl.mk             | 1 +
 package/lua-flu/lua-flu.mk               | 3 +++
 package/lua-iconv/lua-iconv.mk           | 1 +
 package/luadbi-sqlite3/luadbi-sqlite3.mk | 1 +
 package/luaexpat/luaexpat.mk             | 1 +
 package/luaossl/luaossl.mk               | 3 +++
 package/luasec/luasec.mk                 | 1 +
 package/luasql-sqlite3/luasql-sqlite3.mk | 1 +
 package/wsapi-fcgi/wsapi-fcgi.mk         | 1 +
 10 files changed, 14 insertions(+)
Quoted 16 lines

package/lsqlite3/lsqlite3.mk LSQLITE3_VERSION = 0.9.5-1 · line 11

Quoted 9 lines
package/lsqlite3/lsqlite3.mk +1 -0
index bbf6a1db1..51f9516fd 100644
@@ -8,5 +8,6 @@ LSQLITE3_VERSION = 0.9.5-1
 LSQLITE3_SUBDIR = lsqlite3_fsl09y
 LSQLITE3_LICENSE = MIT
 LSQLITE3_DEPENDENCIES = sqlite
+LSQLITE3_BUILD_OPTS = SQLITE_DIR="$(STAGING_DIR)/usr"

I'm not sure to understand this change. It seems a bit redundant for all those packages to say "the native libraries you need are in $(STAGING_DIR)/usr", since native libraries are always there. Why isn't the common infrastructure taking care of this ?

Yes, it looks redundant.
_BUILD_OPT contains a list of key/value,
the value is always the same ($(STAGING_DIR)/usr), but the key (SQLITE_DIR in this case) depends of the rock.

This is the LuaRocks way.

Fran?ois

Quoted 10 lines

Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190918/bb88f0f3/attachment.html>

#5 lore raw

PATCH 3/6 pkg-luarocks: regen packages with external dependencies

Quoted 6 lines · On Wed, 18 Sep 2019 20:19:14 +0200 Fran?ois Perrad <[email protected]>…

Yes, it looks redundant.
_BUILD_OPT contains a list of key/value,
the value is always the same ($(STAGING_DIR)/usr), but the key (SQLITE_DIR in this case) depends of the rock.

This is the LuaRocks way.

I understand, but before your change such xyz_DIR were not needed, and now they are needed. This looks like a "regression" in the sense that things are worse than they were.

Could you explain why the existing solution no longer works ?

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#6 lore raw

PATCH 3/6 pkg-luarocks: regen packages with external dependencies

Le mer. 18 sept. 2019 ? 21:23, Thomas Petazzoni <
[email protected]> a ?crit :

Quoted 8 lines
Quoted 5 lines · On Wed, 18 Sep 2019 20:19:14 +0200 Fran?ois Perrad <[email protected]>…

Yes, it looks redundant.
_BUILD_OPT contains a list of key/value,
the value is always the same ($(STAGING_DIR)/usr), but the key (SQLITE_DIR in this case) depends of the rock.

Quoted 9 lines
Quoted 1 line

This is the LuaRocks way.

I understand, but before your change such xyz_DIR were not needed, and now they are needed. This looks like a "regression" in the sense that things are worse than they were.

Could you explain why the existing solution no longer works ?

Previously, there is in the config file:

    external_deps_dirs = { getenv('STAGING_DIR') .. [[/usr]] }

and this path was globally used.

The config file is common for host-luarocks and luarocks, so this way is no longer doable.

external_deps_dirs has no equivalent as CLI option.
But for each external dependency BAR, a CLI option BAR_DIR is available and must be set to $(STAGING_DIR)/usr (This is the LuaRocks way).

There are some explanations in the commit log 2/6.

If you look at the final point of the serie, the LuaRocks infrastructure is greatly improved:
- host-luarocks is now supported
- luarocks.mk contains no longer the cross-compilation stuff
- the macro _INSTALL_TARGET_CMDS in pkg-luarocks, contains all the
cross-compilation stuff (better design compatible with per-package folders)
- luarocks is called without the full path of host-lua and without
LUAROCKS_RUN_ENV

Fran?ois

Quoted 8 lines

Thanks!

Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190919/e09262f5/attachment.html>

#7 lore raw

PATCH 3/6 pkg-luarocks: regen packages with external dependencies

Hello Fran?ois,

Quoted 6 lines · On Thu, 19 Sep 2019 18:38:14 +0200 Fran?ois Perrad <[email protected]>…

Previously, there is in the config file:

    external_deps_dirs = { getenv('STAGING_DIR') .. [[/usr]] }

and this path was globally used.

The config file is common for host-luarocks and luarocks, so this way is no longer doable.

We cannot pass a separate configuration file when building things for the host and building things for the target ?

Quoted 6 lines

external_deps_dirs has no equivalent as CLI option.
But for each external dependency BAR, a CLI option BAR_DIR is available and must be set to $(STAGING_DIR)/usr (This is the LuaRocks way).

There are some explanations in the commit log 2/6.

Quoted 8 lines

If you look at the final point of the serie, the LuaRocks infrastructure is greatly improved:
- host-luarocks is now supported
- luarocks.mk contains no longer the cross-compilation stuff
- the macro _INSTALL_TARGET_CMDS in pkg-luarocks, contains all the
cross-compilation stuff (better design compatible with per-package folders)
- luarocks is called without the full path of host-lua and without
LUAROCKS_RUN_ENV

Don't get me wrong, I definitely agree that the series brings a number of useful improvements. I just find it annoying that now all packages need to pass a BAR_DIR = $(STAGING_DIR)/usr for all their native dependencies, while it was not needed before. So that's why I'm asking a few questions to see if we could do better.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#8 lore raw

PATCH 3/6 pkg-luarocks: regen packages with external dependencies

In reply to François Perrad

Hello Fran?ois,

Quoted 6 lines · On Thu, 19 Sep 2019 18:38:14 +0200 Fran?ois Perrad <[email protected]>…

Previously, there is in the config file:

    external_deps_dirs = { getenv('STAGING_DIR') .. [[/usr]] }

and this path was globally used.

The config file is common for host-luarocks and luarocks, so this way is no longer doable.

According to
https://github.com/luarocks/luarocks/wiki/config-file-format, the path to the configuration file can be overridden using the environment variable LUAROCKS_CONFIG. This would allow to have one config file to use when building host LuaRocks packages, and one to use when building target LuaRocks packages. Thanks to that, we could keep the external_deps_dirs in the LuaRocks config file used for target packages, and avoid adding all those BAR_DIR, FOO_DIR variables in a number of packages.

Could you have a look at whether this is possible ?

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#9 lore raw

PATCH 4/6 pkg-luarocks: add support of host-luarocks-package

In reply to Francois Perrad
---
 docs/manual/adding-packages-luarocks.txt |  4 +--
 package/pkg-luarocks.mk                  | 31 +++++++++++++++++-------
 2 files changed, 23 insertions(+), 12 deletions(-)
docs/manual/adding-packages-luarocks.txt +1 -3
index f7104f373..5bab0c799 100644
@@ -73,9 +73,7 @@ infrastructures in Buildroot, respectively.
 
 The main macro of the LuaRocks package infrastructure is +luarocks-package+:
 like +generic-package+ it works by defining a number of variables providing
-metadata information about the package, and then calling +luarocks-package+. It
-is worth mentioning that building LuaRocks packages for the host is not
-supported, so the macro +host-luarocks-package+ is not implemented.
+metadata information about the package, and then calling +luarocks-package+.
 
 Just like the generic infrastructure, the LuaRocks infrastructure works
 by defining a number of variables before calling the +luarocks-package+
package/pkg-luarocks.mk +22 -9
index 23906f4a8..28218fde0 100644
@@ -19,8 +19,10 @@
 
 LUAROCKS_RUN_CMD = $(HOST_DIR)/bin/luarocks
 LUAROCKS_CFLAGS = $(TARGET_CFLAGS) -fPIC
+HOST_LUAROCKS_CFLAGS = $(HOST_CFLAGS) -fPIC
 ifeq ($(BR2_PACKAGE_LUA_5_3),y)
 LUAROCKS_CFLAGS += -DLUA_COMPAT_5_2
+HOST_LUAROCKS_CFLAGS += -DLUA_COMPAT_5_2
 endif
 
 ################################################################################
@@ -40,15 +42,16 @@ endif
 define inner-luarocks-package
 
 $(2)_BUILD_OPTS		?=
-$(2)_NAME_UPSTREAM	?= $(1)
-$(2)_SUBDIR		?= $$($(2)_NAME_UPSTREAM)-$$(shell echo "$$($(2)_VERSION)" | sed -e "s/-[0-9]$$$$//")
-$(2)_ROCKSPEC		?= $$(call LOWERCASE,$$($(2)_NAME_UPSTREAM))-$$($(2)_VERSION).rockspec
-$(2)_SOURCE		?= $$(call LOWERCASE,$$($(2)_NAME_UPSTREAM))-$$($(2)_VERSION).src.rock
-$(2)_SITE		?= $$(call qstrip,$$(BR2_LUAROCKS_MIRROR))
+$(3)_NAME_UPSTREAM	?= $(1)
+$(3)_SUBDIR		?= $$($(3)_NAME_UPSTREAM)-$$(shell echo "$$($(3)_VERSION)" | sed -e "s/-[0-9]$$$$//")
+$(3)_ROCKSPEC		?= $$(call LOWERCASE,$$($(3)_NAME_UPSTREAM))-$$($(3)_VERSION).rockspec
+$(3)_SOURCE		?= $$(call LOWERCASE,$$($(3)_NAME_UPSTREAM))-$$($(3)_VERSION).src.rock
+$(3)_SITE		?= $$(call qstrip,$$(BR2_LUAROCKS_MIRROR))
 
-# Since we do not support host-luarocks-package, we know this is
-# a target package, and can just add the required dependencies
-$(2)_DEPENDENCIES	+= luainterpreter
+ifeq ($(4),target)
+$(2)_DEPENDENCIES += luainterpreter
+endif
+# host-luarocks implies host-luainterpreter
 $(2)_EXTRACT_DEPENDENCIES += host-luarocks
 
 #
@@ -82,6 +85,16 @@ define $(2)_INSTALL_TARGET_CMDS
 endef
 endif
 
+ifndef $(2)_INSTALL_CMDS
+define $(2)_INSTALL_CMDS
+	cd $$($(2)_SRCDIR) && \
+		$$(LUAROCKS_RUN_CMD) make --keep --deps-mode none \
+			CFLAGS="$$(HOST_LUAROCKS_CFLAGS)" \
+			LIBFLAG="-shared $$(HOST_LDFLAGS)" \
+			$$($(2)_BUILD_OPTS) $$($(2)_ROCKSPEC)
+endef
+endif
+
 # Call the generic package infrastructure to generate the necessary
 # make targets
 $(call inner-generic-package,$(1),$(2),$(3),$(4))
@@ -99,4 +112,4 @@ endef
 ################################################################################
 
 luarocks-package = $(call inner-luarocks-package,$(pkgname),$(call UPPERCASE,$(pkgname)),$(call UPPERCASE,$(pkgname)),target)
-# host-luarocks-package not supported
+host-luarocks-package = $(call inner-luarocks-package,host-$(pkgname),$(call UPPERCASE,host-$(pkgname)),$(call UPPERCASE,$(pkgname)),host)
Signature · 1 line
2.20.1
#10 lore raw

PATCH 4/6 pkg-luarocks: add support of host-luarocks-package

Hello Fran?ois,

Quoted 11 lines · On Wed, 18 Sep 2019 08:19:13 +0200 Francois Perrad <[email protected]> wrote:

$(2)_BUILD_OPTS ?=
-$(2)_NAME_UPSTREAM ?= $(1)
-$(2)_SUBDIR ?= $$($(2)_NAME_UPSTREAM)-$$(shell echo "$$($(2)_VERSION)" | sed -e "s/-[0-9]$$$$//")
-$(2)_ROCKSPEC ?= $$(call LOWERCASE,$$($(2)_NAME_UPSTREAM))-$$($(2)_VERSION).rockspec
-$(2)_SOURCE ?= $$(call LOWERCASE,$$($(2)_NAME_UPSTREAM))-$$($(2)_VERSION).src.rock
-$(2)_SITE ?= $$(call qstrip,$$(BR2_LUAROCKS_MIRROR))
+$(3)_NAME_UPSTREAM ?= $(1)
+$(3)_SUBDIR ?= $$($(3)_NAME_UPSTREAM)-$$(shell echo "$$($(3)_VERSION)" | sed -e "s/-[0-9]$$$$//")
+$(3)_ROCKSPEC ?= $$(call LOWERCASE,$$($(3)_NAME_UPSTREAM))-$$($(3)_VERSION).rockspec
+$(3)_SOURCE ?= $$(call LOWERCASE,$$($(3)_NAME_UPSTREAM))-$$($(3)_VERSION).src.rock
+$(3)_SITE ?= $$(call qstrip,$$(BR2_LUAROCKS_MIRROR))

I think this is not good, because it means that if a package "lua-foo" has both a host and a target variant, then LUA_FOO_NAME_UPSTREAM will be defined twice by the package infrastructure.

Instead, we want to do something like this:

ifndef $(2)_NAME_UPSTREAM
ifdef $(3)_NAME_UPSTREAM

    $(2)_NAME_UPSTREAM = $($(3)_NAME_UPSTREAM)
  else
    $(2)_NAME_UPSTREAM ?= $(1)

endif
endif

And so on for the different variables.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#11 lore raw

PATCH 5/6 package/lua: allows to load native modules from host-lua

In reply to Francois Perrad

so, we need a liblua.so
and some modules require Lua 5.2 functions

---
 package/lua/lua.mk | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
package/lua/lua.mk +4 -1
index 8a5f9258a..2ff2fdf75 100644
@@ -55,6 +55,9 @@ LUA_POST_PATCH_HOOKS += LUA_32BITS_LUACONF
 endif
 
 HOST_LUA_CFLAGS = -Wall -fPIC -DLUA_USE_DLOPEN -DLUA_USE_POSIX
+ifeq ($(BR2_PACKAGE_LUA_5_3),y)
+HOST_LUA_CFLAGS += -DLUA_COMPAT_5_2
+endif
 HOST_LUA_MYLIBS = -ldl
 
 define LUA_BUILD_CMDS
@@ -74,7 +77,7 @@ define HOST_LUA_BUILD_CMDS
 	CFLAGS="$(HOST_LUA_CFLAGS)" \
 	MYLDFLAGS="$(HOST_LDFLAGS)" \
 	MYLIBS="$(HOST_LUA_MYLIBS)" \
-	BUILDMODE=static \
+	BUILDMODE=dynamic \
 	PKG_VERSION=$(LUA_VERSION) -C $(@D)/src all
 	sed -e "s/@VERSION@/$(LUA_VERSION)/;s/@ABI@/$(LUAINTERPRETER_ABIVER)/;s/@MYLIBS@/$(HOST_LUA_MYLIBS)/" \
 		package/lua/lua.pc.in > $(@D)/lua.pc
Signature · 1 line
2.20.1
#12 lore raw

PATCH 6/6 package/lua: allows host-lua to find installed modules

In reply to Francois Perrad
---
 package/lua/lua.mk | 6 ++++++
 1 file changed, 6 insertions(+)
package/lua/lua.mk +6 -0
index 2ff2fdf75..30dbf0d68 100644
@@ -54,6 +54,12 @@ endef
 LUA_POST_PATCH_HOOKS += LUA_32BITS_LUACONF
 endif
 
+define HOST_LUA_LUACONF
+	$(SED) 's|#define LUA_ROOT.*|#define LUA_ROOT "$(HOST_DIR)/usr/"|' $(@D)/src/luaconf.h
+endef
+
+HOST_LUA_POST_PATCH_HOOKS += HOST_LUA_LUACONF
+
 HOST_LUA_CFLAGS = -Wall -fPIC -DLUA_USE_DLOPEN -DLUA_USE_POSIX
 ifeq ($(BR2_PACKAGE_LUA_5_3),y)
 HOST_LUA_CFLAGS += -DLUA_COMPAT_5_2
Signature · 1 line
2.20.1
#14 lore raw

PATCH 1/6 pkg-luarocks: remove LUAROCKS_RUN_ENV

Le mer. 18 sept. 2019 ? 09:31, Thomas Petazzoni <
[email protected]> a ?crit :

Quoted 8 lines
Quoted 1 line · On Wed, 18 Sep 2019 08:19:10 +0200 Francois Perrad <[email protected]> wrote:

This requires more details in the commit log. Why can you do this ?

just a preliminary step,
could be squashed with the next commit 2/6

Quoted 12 lines

Thanks,

Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190918/26b1999b/attachment.html>

#15 lore raw

PATCH 1/6 pkg-luarocks: remove LUAROCKS_RUN_ENV

Hello Fran?ois,

I am getting back to this series, trying to get it finally merged.

Quoted 6 lines · On Wed, 18 Sep 2019 20:18:18 +0200 Fran?ois Perrad <[email protected]>…

Le mer. 18 sept. 2019 ? 09:31, Thomas Petazzoni <
thomas.petazzoni at bootlin.com> a ?crit :

Quoted 3 lines

On Wed, 18 Sep 2019 08:19:10 +0200
Francois Perrad <[email protected]> wrote:

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

This requires more details in the commit log. Why can you do this ?

just a preliminary step,
could be squashed with the next commit 2/6

So, preliminary steps are good to have separately if possible. But you still didn't explain how this patch works. How is it possible to simply delete LUAROCKS_RUN_ENV ? The variables passed in LUAROCKS_RUN_ENV are not needed ? If they are not needed, why do they exist in the first place ?

We really need a better explanation.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com