Skip to contentSkip to conversation

python-lxml: allow build as host package

PATCH Superseded Carlos Santos 3 participants 7 messages Jan 23, 2017 – Apr 1, 2017

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 python-lxml: allow build as host package

While currently there is no in-tree Buildroot package which depends on host-python-lxml, we (DATACOM) have some proprietary modules that use it in their test scripts.

We tested python-lxml as host package and confirmed that it builds and works correctly. Someone else might require it, so we are proposing its inclusion.

---
Additional background (DATACOM-specific): so far we circumvent the lack of host-python-lxml invoking python with an absolute path in the tests (/usr/bin/python) but this forces us to install python-lxml on all build hosts and developer workstations.

Change-Id If51b034e9579e938a4e7470a58689d7eb34a6a3f
---
 package/python-lxml/python-lxml.mk | 4 ++++
 1 file changed, 4 insertions(+)
package/python-lxml/python-lxml.mk +4 -0
index 62c2980..22287b9 100644
@@ -27,5 +27,9 @@ PYTHON_LXML_DEPENDENCIES = libxml2 libxslt zlib
 PYTHON_LXML_BUILD_OPTS = \
 	--with-xslt-config=$(STAGING_DIR)/usr/bin/xslt-config \
 	--with-xml2-config=$(STAGING_DIR)/usr/bin/xml2-config
+HOST_PYTHON_LXML_BUILD_OPTS = \
+	--with-xslt-config=$(HOST_DIR)/usr/bin/xslt-config \
+	--with-xml2-config=$(HOST_DIR)/usr/bin/xml2-config
 
 $(eval $(python-package))
+$(eval $(host-python-package))
Signature · 1 line
2.7.4
#1 lore raw

PATCH 1/1 python-lxml: allow build as host package

Hello,

Quoted 9 lines · Carlos Santos

While currently there is no in-tree Buildroot package which depends on host-python-lxml, we (DATACOM) have some proprietary modules that use it in their test scripts.

We tested python-lxml as host package and confirmed that it builds and works correctly. Someone else might require it, so we are proposing its inclusion.

I remain a little bit uncomfortable with host packages that have no in-tree user, as it means the code is dead and never tested by our autobuilder infrastructure.

That being said, lots of aspects of Buildroot are already not tested by our autobuilder infra, so maybe my argument is not so good after all.

Peter, what is your feeling on this ?

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#2 lore raw

PATCH v2 1/1 python-lxml: allow build as host package

In reply to Carlos Santos

While currently there is no in-tree Buildroot package which depends on host-python-lxml, we (DATACOM) have some proprietary modules that use it in their test scripts.

We tested python-lxml as host package and confirmed that it builds and works correctly. Someone else might require it, so we are proposing its inclusion.

---
Changes v1->v2:
- Add explicit host package dependencies, as required after commit 4bdb067e380e72c3a6cb9c6ccf3791a6cf8661e0 (infra: remove auto derivation of host dependencies).

Additional background (DATACOM-specific): so far we circumvent the lack of host-python-lxml invoking python with an absolute path in the tests (/usr/bin/python) but this forces us to install python-lxml on all build hosts and developer workstations.

Change-Id If51b034e9579e938a4e7470a58689d7eb34a6a3f
---
 package/python-lxml/python-lxml.mk | 5 +++++
 1 file changed, 5 insertions(+)
package/python-lxml/python-lxml.mk +5 -0
index 62c2980..977ec5f 100644
@@ -21,11 +21,16 @@ PYTHON_LXML_LICENSE_FILES = \
 PYTHON_LXML_SETUP_TYPE = setuptools
 
 PYTHON_LXML_DEPENDENCIES = libxml2 libxslt zlib
+HOST_PYTHON_LXML_DEPENDENCIES = host-libxml2 host-libxslt host-zlib
 
 # python-lxml needs these scripts in order to properly detect libxml2 and
 # libxslt compiler and linker flags
 PYTHON_LXML_BUILD_OPTS = \
 	--with-xslt-config=$(STAGING_DIR)/usr/bin/xslt-config \
 	--with-xml2-config=$(STAGING_DIR)/usr/bin/xml2-config
+HOST_PYTHON_LXML_BUILD_OPTS = \
+	--with-xslt-config=$(HOST_DIR)/usr/bin/xslt-config \
+	--with-xml2-config=$(HOST_DIR)/usr/bin/xml2-config
 
 $(eval $(python-package))
+$(eval $(host-python-package))
Signature · 1 line
2.7.4
#3 lore raw

PATCH v2 1/1 python-lxml: allow build as host package

Quoted 6 lines

From: "Carlos Santos" <[email protected]>
To: buildroot at buildroot.org
Cc: "Thomas Petazzoni" <[email protected]>
Sent: Wednesday, February 1, 2017 3:51:08 PM
Subject: [Buildroot] [PATCH v2] python-lxml: allow build as host package

Quoted 7 lines

While currently there is no in-tree Buildroot package which depends on host-python-lxml, we (DATACOM) have some proprietary modules that use it in their test scripts.

We tested python-lxml as host package and confirmed that it builds and works correctly. Someone else might require it, so we are proposing its inclusion.

Hello,

Any news about this proposal? I think it could fall into the new policy of "loosen up a little on accepting host tools"[1] if I add the corresponding Config.in.host, right? I know I'm pushing here, but host-python-lxml would help us (DATACOM) to have more reproducible builds.

1. http://elinux.org/Buildroot:DeveloperDaysFOSDEM2017#Host_packages_with_no_in-tree_user

Carlos Santos (Casantos) - DATACOM, P&D
?Something must be done. This is something. Therefore we must do it.? (from the BBC series ?Yes, Prime Minister?, ?Power to the people?)

#4 lore raw

PATCH v2 1/1 python-lxml: allow build as host package

Quoted 14 lines · Carlos Santos
Quoted 6 lines

From: "Carlos Santos" <[email protected]>
To: buildroot at buildroot.org
Cc: "Thomas Petazzoni" <[email protected]>
Sent: Wednesday, February 1, 2017 3:51:08 PM
Subject: [Buildroot] [PATCH v2] python-lxml: allow build as host package

Quoted 7 lines

While currently there is no in-tree Buildroot package which depends on host-python-lxml, we (DATACOM) have some proprietary modules that use it in their test scripts.

We tested python-lxml as host package and confirmed that it builds and works correctly. Someone else might require it, so we are proposing its inclusion.

Quoted 8 lines

Hello,

Any news about this proposal? I think it could fall into the new policy of "loosen up a little on accepting host tools"[1] if I add the corresponding Config.in.host, right? I know I'm pushing here, but host-python-lxml would help us (DATACOM) to have more reproducible builds.

1. http://elinux.org/Buildroot:DeveloperDaysFOSDEM2017#Host_packages_with_no_in-tree_user

We didn't reach a satisfactory conclusion for this case. We (or at least I :-)
certainly understand your need. But it is really weird to see a completely random python package appear in the host tools menu. I personally don't mind that much, but if all 265 host packages would appear in the menu it would get a little bit overloaded...

*Some* way to include it in the autobuilders is certainly needed. As written in
the report: "Maybe for these it's not even so important that they are tested in the autobuilder - almost nobody is going to use it anyway. However, in that case it's not very useful to have it anyway. Especially because it almost certainly will break at some point. "

So, no clear answer I'm afraid...

Regards,
Arnout

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
#5 lore raw

PATCH v2 1/1 python-lxml: allow build as host package

Quoted 37 lines

From: "Arnout Vandecappelle" <[email protected]>
To: "Carlos Santos" <[email protected]>, buildroot at buildroot.org
Cc: "Thomas Petazzoni" <[email protected]>
Sent: Thursday, February 16, 2017 8:08:17 PM
Subject: Re: [Buildroot] [PATCH v2] python-lxml: allow build as host package

Quoted 16 lines · Carlos Santos
Quoted 6 lines

From: "Carlos Santos" <[email protected]>
To: buildroot at buildroot.org
Cc: "Thomas Petazzoni" <[email protected]>
Sent: Wednesday, February 1, 2017 3:51:08 PM
Subject: [Buildroot] [PATCH v2] python-lxml: allow build as host package

Quoted 7 lines

While currently there is no in-tree Buildroot package which depends on host-python-lxml, we (DATACOM) have some proprietary modules that use it in their test scripts.

We tested python-lxml as host package and confirmed that it builds and works correctly. Someone else might require it, so we are proposing its inclusion.

Hello,

Quoted 7 lines

Any news about this proposal? I think it could fall into the new policy of "loosen up a little on accepting host tools"[1] if I add the corresponding Config.in.host, right? I know I'm pushing here, but host-python-lxml would help us (DATACOM) to have more reproducible builds.

1.
http://elinux.org/Buildroot:DeveloperDaysFOSDEM2017#Host_packages_with_no_in-tree_user

We didn't reach a satisfactory conclusion for this case. We (or at least I :-) certainly understand your need. But it is really weird to see a completely random python package appear in the host tools menu. I personally don't mind that much, but if all 265 host packages would appear in the menu it would get a little bit overloaded...

Quoted 7 lines

*Some* way to include it in the autobuilders is certainly needed. As written in the report: "Maybe for these it's not even so important that they are tested in the autobuilder - almost nobody is going to use it anyway. However, in that case it's not very useful to have it anyway. Especially because it almost certainly will break at some point. "

So, no clear answer I'm afraid...

Hello,

Could we have a verdict on this, please?

http://patchwork.ozlabs.org/patch/722644/

Signature · 4 lines
Carlos Santos (Casantos) - DATACOM, P&D
?The greatest triumph that modern PR can offer is the transcendent 
success of having your words and actions judged by your reputation, 
rather than the other way about.? ? Christopher Hitchens
#6 lore raw

PATCH v2 1/1 python-lxml: allow build as host package

In reply to Carlos Santos

Hello,

Quoted 8 lines · Carlos Santos

While currently there is no in-tree Buildroot package which depends on host-python-lxml, we (DATACOM) have some proprietary modules that use it in their test scripts.

We tested python-lxml as host package and confirmed that it builds and works correctly. Someone else might require it, so we are proposing its inclusion.

Quoted 7 lines

---
Changes v1->v2:
- Add explicit host package dependencies, as required after commit 4bdb067e380e72c3a6cb9c6ccf3791a6cf8661e0 (infra: remove auto derivation of host dependencies).

In the end, I've applied to master, after adding a Config.in.host file that makes the host package show up in menuconfig.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com