Skip to contentSkip to conversation

package/python-jinja2: fix runtime python modules dependencies

PATCH Superseded Peter Seiderer 3 participants 9 messages Mar 22, 2020 – Apr 8, 2020

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 package/python-jinja2: fix runtime python modules dependencies

- add python unicodedata runtime dependency, fixes:
File "usr/lib/python3.8/site-packages/jinja2/lexer.py", line 46, in <module>
ModuleNotFoundError: No module named 'unicodedata'

---
 package/python-jinja2/Config.in | 2 ++
 1 file changed, 2 insertions(+)
package/python-jinja2/Config.in +2 -0
index a1900e94d5..5b99df75b8 100644
@@ -1,6 +1,8 @@
 config BR2_PACKAGE_PYTHON_JINJA2
 	bool "python-jinja2"
 	select BR2_PACKAGE_PYTHON_MARKUPSAFE # runtime
+	select BR2_PACKAGE_PYTHON_UNICODEDATA if BR2_PACKAGE_PYTHON
+	select BR2_PACKAGE_PYTHON3_UNICODEDATA if BR2_PACKAGE_PYTHON3
 	help
 	  Jinja2 is a template engine written in pure Python. It
 	  provides a Django inspired non-XML syntax but supports
Signature · 1 line
2.25.1
#1 lore raw

PATCH 2/2 package/python-flask: fix runtime python modules dependencies

- add python setuptools runtime dependency, fixes:
ModuleNotFoundError: No module named 'pkg_resources'

- add python libxml2 runtime dependency, fixes:
ModuleNotFoundError: No module named 'xml'

- add python ssl runtime dependency, fixes:
AttributeError: 'NoneType' object has no attribute 'SSLContext'

---
 package/python-flask/Config.in | 4 ++++
 1 file changed, 4 insertions(+)
package/python-flask/Config.in +4 -0
index 6f77913ed1..72fcb61999 100644
@@ -2,8 +2,12 @@ config BR2_PACKAGE_PYTHON_FLASK
 	bool "python-flask"
 	select BR2_PACKAGE_PYTHON_CLICK # runtime
 	select BR2_PACKAGE_PYTHON_JINJA2 # runtime
+	select BR2_PACKAGE_PYTHON_LXML # runtime
 	select BR2_PACKAGE_PYTHON_WERKZEUG # runtime
 	select BR2_PACKAGE_PYTHON_ITSDANGEROUS # runtime
+	select BR2_PACKAGE_PYTHON_SETUPTOOLS # runtime
+	select BR2_PACKAGE_PYTHON_SSL if BR2_PACKAGE_PYTHON
+	select BR2_PACKAGE_PYTHON3_SSL if BR2_PACKAGE_PYTHON3
 	select BR2_PACKAGE_PYTHON_ZLIB if BR2_PACKAGE_PYTHON
 	select BR2_PACKAGE_PYTHON3_ZLIB if BR2_PACKAGE_PYTHON3
 	help
Signature · 1 line
2.25.1
#2 lore raw

PATCH 2/2 package/python-flask: fix runtime python modules dependencies

Quoted 8 lines · On Sun, 22 Mar 2020 22:10:02 +0100 Peter Seiderer <[email protected]> wrote:

- add python setuptools runtime dependency, fixes:
ModuleNotFoundError: No module named 'pkg_resources'

- add python libxml2 runtime dependency, fixes:
ModuleNotFoundError: No module named 'xml'

- add python ssl runtime dependency, fixes:
AttributeError: 'NoneType' object has no attribute 'SSLContext'

Are all these dependencies intentionally mandatory, or are they mandatory "by mistake" ?

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#3 lore raw

PATCH 2/2 package/python-flask: fix runtime python modules dependencies

Hello Thomas,

Quoted 6 lines · Thomas Petazzoni
Quoted 3 lines · On Sun, 22 Mar 2020 22:10:02 +0100 Peter Seiderer <[email protected]> wrote:

- add python setuptools runtime dependency, fixes:
ModuleNotFoundError: No module named 'pkg_resources'

Quoted 8 lines
Quoted 5 lines

- add python libxml2 runtime dependency, fixes:
ModuleNotFoundError: No module named 'xml'

- add python ssl runtime dependency, fixes:
AttributeError: 'NoneType' object has no attribute 'SSLContext'

Are all these dependencies intentionally mandatory, or are they mandatory "by mistake" ?

No python expert - from my side intentionally mandatory, started with a minimal defconfig only selected python3/python-flask and added packages until I got a error free invocation of flask for an hello world example...

Regards,
Peter

Quoted 2 lines

Thomas

#4 lore raw

PATCH 2/2 package/python-flask: fix runtime python modules dependencies

In reply to Peter Seiderer
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Peter" == Peter Seiderer <[email protected]> writes:

- add python setuptools runtime dependency, fixes:
ModuleNotFoundError: No module named 'pkg_resources'

Hmm, the only instance of pkg_resources I see is in src/flask/cli.py:

def _load_plugin_commands(self):
if self._loaded_plugin_commands:
return
try:
import pkg_resources
except ImportError:
self._loaded_plugin_commands = True
return

for ep in pkg_resources.iter_entry_points("flask.commands"):
self.add_command(ep.load(), ep.name)
self._loaded_plugin_commands = True

Which seems to nicely deal with missing pkg_resources.

Quoted 2 lines

- add python libxml2 runtime dependency, fixes:
ModuleNotFoundError: No module named 'xml'

The python-lxml package provides a 'lxml' module, NOT a 'xml' one. I don't see an 'import xml' line anywhere, where did you see that error?

Quoted 2 lines

- add python ssl runtime dependency, fixes:
AttributeError: 'NoneType' object has no attribute 'SSLContext'

This again seems to only be referenced in src/flask/cli.py if you pass a --key option. Presumably it works without SSL if you don't use --key / --cert on flask run?

Signature · 1 line
Bye, Peter Korsgaard
#5 lore raw

PATCH 2/2 package/python-flask: fix runtime python modules dependencies

Hello Peter,

Quoted 16 lines · Peter Korsgaard
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Peter" == Peter Seiderer <[email protected]> writes:

- add python setuptools runtime dependency, fixes:
ModuleNotFoundError: No module named 'pkg_resources'

Hmm, the only instance of pkg_resources I see is in src/flask/cli.py:

def _load_plugin_commands(self):
if self._loaded_plugin_commands:
return
try:
import pkg_resources
except ImportError:
self._loaded_plugin_commands = True
return

Quoted 5 lines

for ep in pkg_resources.iter_entry_points("flask.commands"):
self.add_command(ep.load(), ep.name)
self._loaded_plugin_commands = True

Which seems to nicely deal with missing pkg_resources.

Did re-run my flask test (reverted this patch) and try to run a simple hello world example (from [1]):

        $ cat hello.py
from flask import Flask

app = Flask(__name__)

@app.route('/')
def hello_world():
return 'Hello, World!'

$ env FLASK_APP=hello.py flask run --host=0.0.0.0
Traceback (most recent call last):
File "/usr/bin/flask", line 6, in <module>
from pkg_resources import load_entry_point
ModuleNotFoundError: No module named 'pkg_resources'

Quoted 9 lines
Quoted 4 lines

- add python libxml2 runtime dependency, fixes:
ModuleNotFoundError: No module named 'xml'

The python-lxml package provides a 'lxml' module, NOT a 'xml' one. I don't see an 'import xml' line anywhere, where did you see that error?

Quoted 7 lines
Quoted 2 lines

- add python ssl runtime dependency, fixes:
AttributeError: 'NoneType' object has no attribute 'SSLContext'

This again seems to only be referenced in src/flask/cli.py if you pass a --key option. Presumably it works without SSL if you don't use --key / --cert on flask run?

Now re-added 'select BR2_PACKAGE_PYTHON_SETUPTOOLS':

	$ diff -u .config.old .config
--- .config.old 2020-04-07 23:28:50.829044089 +0200
+++ .config     2020-04-08 19:49:46.487754298 +0200
@@ -1206,7 +1206,7 @@
 # BR2_PACKAGE_PYTHON3_SSL is not set
 BR2_PACKAGE_PYTHON3_UNICODEDATA=y
 # BR2_PACKAGE_PYTHON3_SQLITE is not set
-# BR2_PACKAGE_PYTHON3_PYEXPAT is not set
+BR2_PACKAGE_PYTHON3_PYEXPAT=y
 # BR2_PACKAGE_PYTHON3_XZ is not set
 BR2_PACKAGE_PYTHON3_ZLIB=y

@@ -1480,7 +1480,7 @@
# BR2_PACKAGE_PYTHON_SERIAL_ASYNCIO is not set
# BR2_PACKAGE_PYTHON_SERVICE_IDENTITY is not set
# BR2_PACKAGE_PYTHON_SETPROCTITLE is not set
-# BR2_PACKAGE_PYTHON_SETUPTOOLS is not set
+BR2_PACKAGE_PYTHON_SETUPTOOLS=y
# BR2_PACKAGE_PYTHON_SH is not set
# BR2_PACKAGE_PYTHON_SHUTILWHICH is not set
# BR2_PACKAGE_PYTHON_SIMPLEAUDIO is not set
@@ -1947,7 +1947,7 @@
#
# BR2_PACKAGE_BENEJSON is not set
# BR2_PACKAGE_CJSON is not set
-# BR2_PACKAGE_EXPAT is not set
+BR2_PACKAGE_EXPAT=y
# BR2_PACKAGE_EZXML is not set
# BR2_PACKAGE_JANSSON is not set
# BR2_PACKAGE_JOSE is not set

	$ $ env FLASK_APP=hello.py flask run --host=0.0.0.0
[   22.431766] random: flask: uninitialized urandom read (24 bytes read)
[   22.443232] random: crng init done
Traceback (most recent call last):
  File "/usr/bin/flask", line 11, in <module>
    load_entry_point('Flask==1.1.1', 'console_scripts', 'flask')()
  File "usr/lib/python3.8/site-packages/flask/cli.py", line 966, in main
  File "usr/lib/python3.8/site-packages/flask/cli.py", line 586, in main
  File "usr/lib/python3.8/site-packages/click/core.py", line 717, in main
  File "usr/lib/python3.8/site-packages/click/core.py", line 1135, in invoke
  File "usr/lib/python3.8/site-packages/click/core.py", line 641, in make_context
  File "usr/lib/python3.8/site-packages/click/core.py", line 940, in parse_args
  File "usr/lib/python3.8/site-packages/click/core.py", line 1476, in handle_parse_result
  File "usr/lib/python3.8/site-packages/click/core.py", line 96, in invoke_param_callback
  File "usr/lib/python3.8/site-packages/flask/cli.py", line 742, in _validate_key
AttributeError: 'NoneType' object has no attribute 'SSLContext'

Now re-added 'select BR2_PACKAGE_PYTHON3_SSL':

	$ diff -u .config.old .config
--- .config.old 2020-04-08 19:49:46.487754298 +0200
+++ .config     2020-04-08 19:56:34.155512376 +0200
@@ -1203,7 +1203,7 @@
 # BR2_PACKAGE_PYTHON3_DECIMAL is not set
 # BR2_PACKAGE_PYTHON3_OSSAUDIODEV is not set
 # BR2_PACKAGE_PYTHON3_READLINE is not set
-# BR2_PACKAGE_PYTHON3_SSL is not set
+BR2_PACKAGE_PYTHON3_SSL=y
 BR2_PACKAGE_PYTHON3_UNICODEDATA=y
 # BR2_PACKAGE_PYTHON3_SQLITE is not set
 BR2_PACKAGE_PYTHON3_PYEXPAT=y
@@ -1674,7 +1674,14 @@
 # BR2_PACKAGE_LIBUECC is not set
 # BR2_PACKAGE_MBEDTLS is not set
 # BR2_PACKAGE_NETTLE is not set
-# BR2_PACKAGE_OPENSSL is not set
+BR2_PACKAGE_OPENSSL=y
+BR2_PACKAGE_LIBOPENSSL=y
+BR2_PACKAGE_LIBOPENSSL_TARGET_ARCH="linux-armv4"
+# BR2_PACKAGE_LIBOPENSSL_BIN is not set
+# BR2_PACKAGE_LIBOPENSSL_ENGINES is not set
+# BR2_PACKAGE_LIBRESSL is not set
+BR2_PACKAGE_HAS_OPENSSL=y
+BR2_PACKAGE_PROVIDES_OPENSSL="libopenssl"
 BR2_PACKAGE_PROVIDES_HOST_OPENSSL="host-libopenssl"
 # BR2_PACKAGE_RHASH is not set
 # BR2_PACKAGE_TINYDTLS is not set
	$ env FLASK_APP=hello.py flask run --host=0.0.0.0
 * Serving Flask app "hello.py"
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
172.16.0.1 - - [01/Jan/1970 00:02:17] "GET / HTTP/1.1" 200 -

Voila! A running flask hello-world...

Your are right, the adding of the python-lxml was wrong (suspect the selectiob of expat/pyexpat via setuptools is enough and the thing I missed on the first run), but the other dependencies are needed..., will send an patch to revert the python-lxml dependency...

Regards,
Peter

[1] https://flask.palletsprojects.com/en/1.1.x/quickstart/#quickstart

#6 lore raw

PATCH 2/2 package/python-flask: fix runtime python modules dependencies

Quoted 39 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Peter" == Peter Seiderer <[email protected]> writes:

Hello Peter,

Quoted 16 lines · Peter Korsgaard
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Peter" == Peter Seiderer <[email protected]> writes:

- add python setuptools runtime dependency, fixes:
ModuleNotFoundError: No module named 'pkg_resources'

Hmm, the only instance of pkg_resources I see is in src/flask/cli.py:

def _load_plugin_commands(self):
if self._loaded_plugin_commands:
return
try:
import pkg_resources
except ImportError:
self._loaded_plugin_commands = True
return

Quoted 5 lines

for ep in pkg_resources.iter_entry_points("flask.commands"):
self.add_command(ep.load(), ep.name) self._loaded_plugin_commands = True

Which seems to nicely deal with missing pkg_resources.

Did re-run my flask test (reverted this patch) and try to run a simple hello world example (from [1]):

        $ cat hello.py
from flask import Flask

app = Flask(__name__)

@app.route('/')
def hello_world():
return 'Hello, World!'

Quoted 5 lines

$ env FLASK_APP=hello.py flask run --host=0.0.0.0
Traceback (most recent call last):
File "/usr/bin/flask", line 6, in <module>
from pkg_resources import load_entry_point
ModuleNotFoundError: No module named 'pkg_resources'

Ahh, indeed. The click stuff generates the flask script, so that is why it isn't in the source tree and I didn't see it with grep.

The other feedback should be correct though.

Signature · 1 line
Bye, Peter Korsgaard
#7 lore raw

PATCH 1/2 package/python-jinja2: fix runtime python modules dependencies

In reply to Peter Seiderer
Quoted 6 lines · On Sun, 22 Mar 2020 22:10:01 +0100 Peter Seiderer <[email protected]> wrote:

- add python unicodedata runtime dependency, fixes:
File "usr/lib/python3.8/site-packages/jinja2/lexer.py", line 46, in <module>
ModuleNotFoundError: No module named 'unicodedata'

---
 package/python-jinja2/Config.in | 2 ++
 1 file changed, 2 insertions(+)

These two patches lack your Signed-off-by. Also, what about adding runtime tests for those two packages, to detect such missing runtime dependencies ?

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#8 lore raw

PATCH 1/2 package/python-jinja2: fix runtime python modules dependencies

Hello Thomas,

Quoted 10 lines · Thomas Petazzoni
Quoted 6 lines · On Sun, 22 Mar 2020 22:10:01 +0100 Peter Seiderer <[email protected]> wrote:

- add python unicodedata runtime dependency, fixes:
File "usr/lib/python3.8/site-packages/jinja2/lexer.py", line 46, in <module>
ModuleNotFoundError: No module named 'unicodedata'

---
 package/python-jinja2/Config.in | 2 ++
 1 file changed, 2 insertions(+)
Quoted 3 lines

These two patches lack your Signed-off-by. Also, what about adding runtime tests for those two packages, to detect such missing runtime dependencies ?

Ups, will re-send with the Signed-off-by added...

Will take a look at the runtime tests...

Regards,
Peter

Quoted 4 lines

Thanks!

Thomas