Skip to contentSkip to conversation

support/testing: add python-mako runtime test

PATCH Applied Romain Naour 4 participants 13 messages Jun 9, 2023 – Sep 30, 2023

lore
Share & tools
lore

Romain Naour

author
#0 lore raw

PATCH 1/4 support/testing: add python-mako runtime test

This new runtime test allows to reproduce the issue reported by https://bugs.busybox.net/show_bug.cgi?id=15628
with a simple python script.

Traceback (most recent call last):
File "/bin/piglit", line 107, in <module>
import framework.programs.summary as summary
File "/usr/lib64/piglit/framework/programs/summary.py", line 30, in <module>
from framework import summary, status, core, backends, exceptions
File "/usr/lib64/piglit/framework/summary/__init__.py", line 31, in <module>
from .html_ import html, feat
File "/usr/lib64/piglit/framework/summary/html_.py", line 37, in <module>
from mako.lookup import TemplateLookup
File "/usr/lib/python3.11/site-packages/mako/lookup.py", line 13, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
ModuleNotFoundError: No module named 'markupsafe

Signed-off-by Romain Naour <[email protected]>
---
 support/testing/tests/package/sample_python_mako.py |  1 +
 support/testing/tests/package/test_python_mako.py   | 12 ++++++++++++
 2 files changed, 13 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_mako.py
 create mode 100644 support/testing/tests/package/test_python_mako.py
support/testing/tests/package/sample_python_mako.py new file +1 -0
new file mode 100644
index 0000000000..aedae7ebad
@@ -0,0 +1 @@
+from mako import runtime
support/testing/tests/package/test_python_mako.py new file +12 -0
new file mode 100644
index 0000000000..b6075bde6a
@@ -0,0 +1,12 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy3Mako(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_MAKO=y
+        """
+    sample_scripts = ["tests/package/sample_python_mako.py"]
+    timeout = 30
Signature · 6 lines
2.40.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
author
#1 lore raw

PATCH 2/4 package/python-mako: add optionnal runtime dependency on python-pygments

Signed-off-by Romain Naour <[email protected]>
---
 package/python-mako/Config.in | 11 +++++++++++
 1 file changed, 11 insertions(+)
package/python-mako/Config.in +11 -0
index e6a10c9c2e..2e1094189f 100644
@@ -7,3 +7,14 @@ config BR2_PACKAGE_PYTHON_MAKO
 	  for maximum performance.
 
 	  https://pypi.python.org/pypi/Mako
+
+if BR2_PACKAGE_PYTHON_MAKO
+
+config BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN
+	bool "mako extension pygmentplugin"
+	select BR2_PACKAGE_PYTHON_PYGMENTS # runtime
+	help
+	  Provide python-pygments support for
+	  mako.ext.pygmentplugin
+
+endif
Signature · 6 lines
2.40.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Applied
#2 lore raw

Re: [Buildroot] [PATCH 2/4] package/python-mako: add optionnal runtime dependency on python-pygments

Quoted 4 lines · On Fri, 9 Jun 2023 19:49:53 +0200 Romain Naour <[email protected]> wrote:
Signed-off-by Romain Naour <[email protected]>
---
 package/python-mako/Config.in | 11 +++++++++++
 1 file changed, 11 insertions(+)

Applied to master, thanks.

Thomas

Signature · 7 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
author
#3 lore raw

PATCH 3/4 package/python-mako: add optionnal runtime dependency on python-babel

In reply to Romain Naour
Signed-off-by Romain Naour <[email protected]>
---
 package/python-mako/Config.in | 7 +++++++
 1 file changed, 7 insertions(+)
package/python-mako/Config.in +7 -0
index 2e1094189f..7bba31943e 100644
@@ -17,4 +17,11 @@ config BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN
 	  Provide python-pygments support for
 	  mako.ext.pygmentplugin
 
+config BR2_PACKAGE_PYTHON_MAKO_EXT_BABELPLUGIN
+	bool "mako extension babelplugin"
+	select BR2_PACKAGE_PYTHON_BABEL # runtime
+	help
+	  Provide python-babel support for
+	  mako.ext.babelplugin
+
 endif
Signature · 6 lines
2.40.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Applied
#4 lore raw

Re: [Buildroot] [PATCH 3/4] package/python-mako: add optionnal runtime dependency on python-babel

Quoted 4 lines · On Fri, 9 Jun 2023 19:49:54 +0200 Romain Naour <[email protected]> wrote:
Signed-off-by Romain Naour <[email protected]>
---
 package/python-mako/Config.in | 7 +++++++
 1 file changed, 7 insertions(+)

Applied to master, thanks.

Thomas

Signature · 7 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
author
#5 lore raw

PATCH 4/4 support/testing: TestPythonPy3MakoExt: add a test for mako external plugin (pygmentsplugin, babel)

In reply to Romain Naour

Mako provide some external plugins that requires additionnal and optionnal runtime dependencies.

Signed-off-by Romain Naour <[email protected]>
---
 .../testing/tests/package/sample_python_mako_ext.py  |  3 +++
 support/testing/tests/package/test_python_mako.py    | 12 ++++++++++++
 2 files changed, 15 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_mako_ext.py
support/testing/tests/package/sample_python_mako_ext.py new file +3 -0
new file mode 100644
index 0000000000..5b06a28181
@@ -0,0 +1,3 @@
+from mako import runtime
+import mako.ext.pygmentplugin
+import mako.ext.babelplugin
support/testing/tests/package/test_python_mako.py +12 -0
index b6075bde6a..f1031d1d59 100644
@@ -10,3 +10,15 @@ class TestPythonPy3Mako(TestPythonPackageBase):
         """
     sample_scripts = ["tests/package/sample_python_mako.py"]
     timeout = 30
+
+class TestPythonPy3MakoExt(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_MAKO=y
+        BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN=y
+        BR2_PACKAGE_PYTHON_MAKO_EXT_BABELPLUGIN=y
+        """
+    sample_scripts = ["tests/package/sample_python_mako_ext.py"]
+    timeout = 30
Signature · 6 lines
2.40.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Applied
#6 lore raw

Re: [Buildroot] [PATCH 4/4] support/testing: TestPythonPy3MakoExt: add a test for mako external plugin (pygmentsplugin, babel)

Quoted 9 lines · On Fri, 9 Jun 2023 19:49:55 +0200 Romain Naour <[email protected]> wrote:

Mako provide some external plugins that requires additionnal and optionnal runtime dependencies.

Signed-off-by Romain Naour <[email protected]>
---
 .../testing/tests/package/sample_python_mako_ext.py  |  3 +++
 support/testing/tests/package/test_python_mako.py    | 12 ++++++++++++
 2 files changed, 15 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_mako_ext.py

Applied to master after adding entries in the DEVELOPERS file, and fixing flake8 warnings. Thanks!

Thomas

Signature · 7 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#7 lore raw
In reply to Romain Naour

Hello Romain,

Quoted 20 lines · On Fri, 9 Jun 2023 19:49:52 +0200 Romain Naour <[email protected]> wrote:

This new runtime test allows to reproduce the issue reported by https://bugs.busybox.net/show_bug.cgi?id=15628
with a simple python script.

Traceback (most recent call last):
File "/bin/piglit", line 107, in <module>
import framework.programs.summary as summary
File "/usr/lib64/piglit/framework/programs/summary.py", line 30, in <module>
from framework import summary, status, core, backends, exceptions
File "/usr/lib64/piglit/framework/summary/__init__.py", line 31, in <module>
from .html_ import html, feat
File "/usr/lib64/piglit/framework/summary/html_.py", line 37, in <module>
from mako.lookup import TemplateLookup
File "/usr/lib/python3.11/site-packages/mako/lookup.py", line 13, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
ModuleNotFoundError: No module named 'markupsafe

Quoted 1 line
Signed-off-by Romain Naour <[email protected]>

I'm unclear on a number of things:

- How does this interact with the patch proposed by Sebastian at
https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/,
which takes a different approach ?

- How does your patch series fixes bug #15628 ? The patch from Sebastian fixes it, but not your series as far as I can see.

- I don't see in your series what ensures that the test TestPythonPy3Mako properly works. It would work with Sebastian patch applied, but Sebastian's patch collides with your series because Sebastian patch makes BR2_PACKAGE_PYTHON_PYGMENTS a mandatory dependency of python-mako, while your patch series makes it an optional dependency, enabled when BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN is enabled.

I might not have grasped the full scale of the issue, but my feeling is that:

- python-mako needs markupsafe unconditionally, as demonstrated by your test TestPythonPy3Mako. So python-mako should unconditionally select BR2_PACKAGE_PYTHON_MARKUPSAFE

- python-mako only needs pygments/babel as optional dependencies so your patches 2/4 and 3/4 are fine

- in order to fix bug #15628, we need to ensure that the piglit package selects BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN, so that python-mako is installed with pygments support

Does that make sense ?

Best regards,

Thomas

Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#8 lore raw
Quoted 7 lines · On Wed, 12 Jul 2023 21:43:57 +0200 Thomas Petazzoni…

I might not have grasped the full scale of the issue, but my feeling is that:

- python-mako needs markupsafe unconditionally, as demonstrated by your test TestPythonPy3Mako. So python-mako should unconditionally select BR2_PACKAGE_PYTHON_MARKUPSAFE

Quoted 8 lines

- python-mako only needs pygments/babel as optional dependencies so your patches 2/4 and 3/4 are fine

- in order to fix bug #15628, we need to ensure that the piglit package selects BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN, so that python-mako is installed with pygments support

Does that make sense ?

I got it wrong, your basic test shows that both pygments and markupsafe are needed for a basic usage of python-mako:

$ ./support/testing/run-tests -o ../outputs/ -k tests.package.test_python_mako.TestPythonPy3Mako
21:32:04 TestPythonPy3Mako                        Starting
21:32:04 TestPythonPy3Mako                        Building
21:34:55 TestPythonPy3Mako                        Building done
21:35:01 TestPythonPy3Mako                        Cleaning up
F
======================================================================
FAIL: test_run (tests.package.test_python_mako.TestPythonPy3Mako)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 137, in test_run
    self.run_sample_scripts()
  File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 132, in run_sample_scripts
    self.assertRunOk(cmd, timeout=self.timeout)
  File "/home/thomas/buildroot/buildroot/support/testing/infra/basetest.py", line 95, in assertRunOk
    self.assertEqual(
AssertionError: 1 != 0 : 
Failed to run: python sample_python_mako.py
output was:
  Traceback (most recent call last):
    File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 287, in _install_highlighting
    File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 271, in _install_pygments
    File "/usr/lib/python3.11/site-packages/mako/ext/pygmentplugin.py", line 7, in <module>
  ModuleNotFoundError: No module named 'pygments'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/sample_python_mako.py", line 1, in <module>
from mako import runtime
File "/usr/lib/python3.11/site-packages/mako/runtime.py", line 15, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
ModuleNotFoundError: No module named 'markupsafe'

----------------------------------------------------------------------
Ran 1 test in 177.157s

FAILED (failures=1)

So to me it looks like the patch from Sebastian is correct, and therefore your PATCH 2/4 adding an option to enable pygments support does not make sense because pygments is needed unconditionally by python-mako.

Thoughts?

Thomas

Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#9 lore raw

Hello Thomas, All,

Le 12/07/2023 à 21:46, Thomas Petazzoni via buildroot a écrit :

Quoted 62 lines
Quoted 7 lines · On Wed, 12 Jul 2023 21:43:57 +0200 Thomas Petazzoni…

I might not have grasped the full scale of the issue, but my feeling is that:

- python-mako needs markupsafe unconditionally, as demonstrated by your test TestPythonPy3Mako. So python-mako should unconditionally select BR2_PACKAGE_PYTHON_MARKUPSAFE

Quoted 8 lines

- python-mako only needs pygments/babel as optional dependencies so your patches 2/4 and 3/4 are fine

- in order to fix bug #15628, we need to ensure that the piglit package selects BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN, so that python-mako is installed with pygments support

Does that make sense ?

I got it wrong, your basic test shows that both pygments and markupsafe are needed for a basic usage of python-mako:

$ ./support/testing/run-tests -o ../outputs/ -k tests.package.test_python_mako.TestPythonPy3Mako
21:32:04 TestPythonPy3Mako                        Starting
21:32:04 TestPythonPy3Mako                        Building
21:34:55 TestPythonPy3Mako                        Building done
21:35:01 TestPythonPy3Mako                        Cleaning up
F
======================================================================
FAIL: test_run (tests.package.test_python_mako.TestPythonPy3Mako)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 137, in test_run
    self.run_sample_scripts()
  File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 132, in run_sample_scripts
    self.assertRunOk(cmd, timeout=self.timeout)
  File "/home/thomas/buildroot/buildroot/support/testing/infra/basetest.py", line 95, in assertRunOk
    self.assertEqual(
AssertionError: 1 != 0 : 
Failed to run: python sample_python_mako.py
output was:
  Traceback (most recent call last):
    File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 287, in _install_highlighting
    File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 271, in _install_pygments
    File "/usr/lib/python3.11/site-packages/mako/ext/pygmentplugin.py", line 7, in <module>
  ModuleNotFoundError: No module named 'pygments'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/sample_python_mako.py", line 1, in <module>
from mako import runtime
File "/usr/lib/python3.11/site-packages/mako/runtime.py", line 15, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
ModuleNotFoundError: No module named 'markupsafe'

----------------------------------------------------------------------
Ran 1 test in 177.157s

Quoted 8 lines

FAILED (failures=1)

So to me it looks like the patch from Sebastian is correct, and therefore your PATCH 2/4 adding an option to enable pygments support does not make sense because pygments is needed unconditionally by python-mako.

Thoughts?

Actually the patch from Sebastian should be part of this series but without selecting pygments.

It's unclear for me if pygments and babel are really mandatory runtime dependencies of mako. Indeed, pygments and babel are used by "mako extension" and may not be used at all when using python-mako.

Do we really want to handle such "mako extension" ?

Otherwise, I'm fine to add them as runtime dependencies of mako.

Best regards,
Romain

Quoted 2 lines

Thomas

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#10 lore raw

Le 13/07/2023 à 09:36, Romain Naour a écrit :

Quoted 49 lines

Hello Thomas, All,

Le 12/07/2023 à 21:46, Thomas Petazzoni via buildroot a écrit :

Quoted 46 lines
Quoted 7 lines · On Wed, 12 Jul 2023 21:43:57 +0200 Thomas Petazzoni…

I might not have grasped the full scale of the issue, but my feeling is that:

- python-mako needs markupsafe unconditionally, as demonstrated by your test TestPythonPy3Mako. So python-mako should unconditionally select BR2_PACKAGE_PYTHON_MARKUPSAFE

Quoted 8 lines

- python-mako only needs pygments/babel as optional dependencies so your patches 2/4 and 3/4 are fine

- in order to fix bug #15628, we need to ensure that the piglit package selects BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN, so that python-mako is installed with pygments support

Does that make sense ?

I got it wrong, your basic test shows that both pygments and markupsafe are needed for a basic usage of python-mako:

$ ./support/testing/run-tests -o ../outputs/ -k tests.package.test_python_mako.TestPythonPy3Mako
21:32:04 TestPythonPy3Mako                        Starting
21:32:04 TestPythonPy3Mako                        Building
21:34:55 TestPythonPy3Mako                        Building done
21:35:01 TestPythonPy3Mako                        Cleaning up
F
======================================================================
FAIL: test_run (tests.package.test_python_mako.TestPythonPy3Mako)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 137, in test_run
    self.run_sample_scripts()
  File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 132, in run_sample_scripts
    self.assertRunOk(cmd, timeout=self.timeout)
  File "/home/thomas/buildroot/buildroot/support/testing/infra/basetest.py", line 95, in assertRunOk
    self.assertEqual(
AssertionError: 1 != 0 : 
Failed to run: python sample_python_mako.py
output was:
  Traceback (most recent call last):
    File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 287, in _install_highlighting
    File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 271, in _install_pygments
    File "/usr/lib/python3.11/site-packages/mako/ext/pygmentplugin.py", line 7, in <module>
  ModuleNotFoundError: No module named 'pygments'

IIRC, I didn't have this error while testing with TestPythonPy3Mako with Sebastian patch applied and ammanded to remove pygments dependency.

Best regards,
Romain

Quoted 32 lines
Quoted 16 lines

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/sample_python_mako.py", line 1, in <module>
from mako import runtime
File "/usr/lib/python3.11/site-packages/mako/runtime.py", line 15, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
ModuleNotFoundError: No module named 'markupsafe'

----------------------------------------------------------------------
Ran 1 test in 177.157s

Quoted 8 lines

FAILED (failures=1)

So to me it looks like the patch from Sebastian is correct, and therefore your PATCH 2/4 adding an option to enable pygments support does not make sense because pygments is needed unconditionally by python-mako.

Thoughts?

Actually the patch from Sebastian should be part of this series but without selecting pygments.

It's unclear for me if pygments and babel are really mandatory runtime dependencies of mako. Indeed, pygments and babel are used by "mako extension" and may not be used at all when using python-mako.

Quoted 10 lines

Do we really want to handle such "mako extension" ?

Otherwise, I'm fine to add them as runtime dependencies of mako.

Best regards,
Romain

Quoted 2 lines

Thomas

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#11 lore raw

Hello all,

Quoted 18 lines · Romain Naour
Quoted 16 lines · Romain Naour

Hello Thomas, All,

Quoted 13 lines · Thomas Petazzoni via buildroot
Quoted 10 lines · On Wed, 12 Jul 2023 21:43:57 +0200 Thomas Petazzoni…

I might not have grasped the full scale of the issue, but my feeling is that:

- python-mako needs markupsafe unconditionally, as demonstrated by your test TestPythonPy3Mako. So python-mako should unconditionally select BR2_PACKAGE_PYTHON_MARKUPSAFE

- python-mako only needs pygments/babel as optional dependencies so your patches 2/4 and 3/4 are fine

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

- in order to fix bug #15628, we need to ensure that the piglit package selects BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN, so that python-mako is installed with pygments support

Does that make sense ?

I got it wrong, your basic test shows that both pygments and markupsafe are needed for a basic usage of python-mako:

After checking again I'm not even sure that is necessarily the case. When you don't
select either you get the error that both are missing. However it seems that just
adding one of BR2_PACKAGE_PYTHON_MARKUPSAFE=y or BR2_PACKAGE_PYTHON_PYGMENTS=y in the TestPythonPy3Mako config
makes the test work. I don't know if we should add both or just one of them as a default dependency in this case. It does seem though that markupsafe is necessary
If the optional babel dependency is selected.

./support/testing/run-tests -o test-output -d test-dl/ -k tests.package.test_python_mako.TestPythonPy3MakoExt
11:16:50 TestPythonPy3MakoExt Starting
11:16:59 TestPythonPy3MakoExt Cleaning up
F
======================================================================
FAIL: test_run (tests.package.test_python_mako.TestPythonPy3MakoExt)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/sewey/projects/buildroot/support/testing/tests/package/test_python.py",
line 137, in test_run
self.run_sample_scripts()
File "/home/sewey/projects/buildroot/support/testing/tests/package/test_python.py",
line 132, in run_sample_scripts
self.assertRunOk(cmd, timeout=self.timeout)
File "/home/sewey/projects/buildroot/support/testing/infra/basetest.py",
line 95, in assertRunOk

    self.assertEqual(

AssertionError: 1 != 0 :
Failed to run: python sample_python_mako_ext.py
output was:
Traceback (most recent call last):
File "/root/sample_python_mako_ext.py", line 3, in <module>
import mako.ext.babelplugin
File "/usr/lib/python3.11/site-packages/mako/ext/babelplugin.py",
line 10, in <module>
File "/usr/lib/python3.11/site-packages/mako/ext/extract.py", line
11, in <module>
File "/usr/lib/python3.11/site-packages/mako/lexer.py", line 13, in <module> File "/usr/lib/python3.11/site-packages/mako/parsetree.py", line
13, in <module>

    File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14,

in <module>
ModuleNotFoundError: No module named 'markupsafe'

----------------------------------------------------------------------
Ran 1 test in 9.392s

FAILED (failures=1)

Best regards,
Sebastian

Quoted 76 lines
Quoted 17 lines
Quoted 17 lines
$ ./support/testing/run-tests -o ../outputs/ -k tests.package.test_python_mako.TestPythonPy3Mako
21:32:04 TestPythonPy3Mako                        Starting
21:32:04 TestPythonPy3Mako                        Building
21:34:55 TestPythonPy3Mako                        Building done
21:35:01 TestPythonPy3Mako                        Cleaning up
F
======================================================================
FAIL: test_run (tests.package.test_python_mako.TestPythonPy3Mako)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 137, in test_run
    self.run_sample_scripts()
  File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 132, in run_sample_scripts
    self.assertRunOk(cmd, timeout=self.timeout)
  File "/home/thomas/buildroot/buildroot/support/testing/infra/basetest.py", line 95, in assertRunOk
    self.assertEqual(
Quoted 8 lines
Quoted 8 lines

AssertionError: 1 != 0 :
Failed to run: python sample_python_mako.py
output was:
Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 287, in _install_highlighting File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 271, in _install_pygments File "/usr/lib/python3.11/site-packages/mako/ext/pygmentplugin.py", line 7, in <module>
ModuleNotFoundError: No module named 'pygments'

IIRC, I didn't have this error while testing with TestPythonPy3Mako with Sebastian patch applied and ammanded to remove pygments dependency.

Best regards,
Romain

Quoted 42 lines
Quoted 16 lines

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/sample_python_mako.py", line 1, in <module>
from mako import runtime
File "/usr/lib/python3.11/site-packages/mako/runtime.py", line 15, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
ModuleNotFoundError: No module named 'markupsafe'

----------------------------------------------------------------------
Ran 1 test in 177.157s

Quoted 8 lines

FAILED (failures=1)

So to me it looks like the patch from Sebastian is correct, and therefore your PATCH 2/4 adding an option to enable pygments support does not make sense because pygments is needed unconditionally by python-mako.

Thoughts?

Actually the patch from Sebastian should be part of this series but without selecting pygments.

It's unclear for me if pygments and babel are really mandatory runtime dependencies of mako. Indeed, pygments and babel are used by "mako extension" and may not be used at all when using python-mako.

Do we really want to handle such "mako extension" ?

Otherwise, I'm fine to add them as runtime dependencies of mako.

Best regards,
Romain

Quoted 2 lines

Thomas

Signature · 15 lines
Pour la planète, n'imprimez ce mail que si c'est nécessaire


10-12 Rue Nicolas Appert
44100 NANTES

Sebastian WEYER
Stagiaire production

[email protected]
http://www.smile.eu
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#12 lore raw
In reply to Romain Naour
Quoted 20 lines · On Fri, 9 Jun 2023 19:49:52 +0200 Romain Naour <[email protected]> wrote:

This new runtime test allows to reproduce the issue reported by https://bugs.busybox.net/show_bug.cgi?id=15628
with a simple python script.

Traceback (most recent call last):
File "/bin/piglit", line 107, in <module>
import framework.programs.summary as summary
File "/usr/lib64/piglit/framework/programs/summary.py", line 30, in <module>
from framework import summary, status, core, backends, exceptions
File "/usr/lib64/piglit/framework/summary/__init__.py", line 31, in <module>
from .html_ import html, feat
File "/usr/lib64/piglit/framework/summary/html_.py", line 37, in <module>
from mako.lookup import TemplateLookup
File "/usr/lib/python3.11/site-packages/mako/lookup.py", line 13, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module> File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
ModuleNotFoundError: No module named 'markupsafe

Quoted 7 lines
Signed-off-by Romain Naour <[email protected]>
---
 support/testing/tests/package/sample_python_mako.py |  1 +
 support/testing/tests/package/test_python_mako.py   | 12 ++++++++++++
 2 files changed, 13 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_mako.py
 create mode 100644 support/testing/tests/package/test_python_mako.py

I've applied after changing the commit log, which was confusing, because the problem does not reproduce with this test, as it has been fixed by the previous commit. Thanks!

Thomas

Signature · 7 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot