package/python-babel: add purge locale data support
RFC 1/1 package/python-babel: add purge locale data support
Add purge loacle data support (inspired by BR2_ENABLE_LOCALE_PURGE handling in Makefile) to reduce target space allocation by the python-babel package, e.g. from 32MB to 24K for target/usr/lib/python3.11/site-packages/babel/locale-data with BR2_ENABLE_LOCALE_WHITELIST="C en_US de_DE".
--- package/python-babel/python-babel.mk | 12 ++++++++++++ 1 file changed, 12 insertions(+)
package/python-babel/python-babel.mk +12 -0
@@ -12,5 +12,17 @@ PYTHON_BABEL_LICENSE = BSD-3-Clause PYTHON_BABEL_LICENSE_FILES = LICENSE HOST_PYTHON_BABEL_DEPENDENCIES = host-python-pytz +# purge locale data (if enabled), keep special en_US_POSIX data by default +ifeq ($(BR2_ENABLE_LOCALE_PURGE),y) +define PYTHON_BABEL_CLEANUP_LOCALE + for i in `ls $(TARGET_DIR)/usr/lib/python3.11/site-packages/babel/locale-data/*.dat`; \ + do \ + i_base=`basename "$$i" .dat`; \ + echo "$(BR2_ENABLE_LOCALE_WHITELIST) en_US_POSIX" | grep -qw "$$i_base" || rm "$$i"; \ + done +endef +endif +PYTHON_BABEL_POST_INSTALL_TARGET_HOOKS += PYTHON_BABEL_CLEANUP_LOCALE + $(eval $(python-package)) $(eval $(host-python-package))
Signature · 6 lines
2.40.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Peter, All,
On 2023-04-18 23:32 +0200, Peter Seiderer spake thusly:
Quoted 16 lines
Add purge loacle data support (inspired by
BR2_ENABLE_LOCALE_PURGEhandling inMakefile) to reduce target space allocation by the python-babel package, e.g. from 32MB to 24K for target/usr/lib/python3.11/site-packages/babel/locale-data with BR2_ENABLE_LOCALE_WHITELIST="Cen_USde_DE".Signed-off-by Peter Seiderer <[email protected]>--- package/python-babel/python-babel.mk | 12 ++++++++++++ 1 file changed, 12 insertions(+)package/python-babel/python-babel.mk +0
-0
package/python-babel/python-babel.mk PYTHON_BABEL_LICENSE = BSD-3-Clause · line 21
Quoted 10 lines
package/python-babel/python-babel.mk +7
-0@@ -12,5 +12,17 @@ PYTHON_BABEL_LICENSE = BSD-3-Clause PYTHON_BABEL_LICENSE_FILES = LICENSE HOST_PYTHON_BABEL_DEPENDENCIES = host-python-pytz +# purge locale data (if enabled), keep special en_US_POSIX data by default +ifeq ($(BR2_ENABLE_LOCALE_PURGE),y) +define PYTHON_BABEL_CLEANUP_LOCALE + for i in `ls $(TARGET_DIR)/usr/lib/python3.11/site-packages/babel/locale-data/*.dat`; \ + do \ + i_base=`basename "$$i" .dat`; \ + echo "$(BR2_ENABLE_LOCALE_WHITELIST) en_US_POSIX" | grep -qw "$$i_base" || rm "$$i"; \
Why do you force keeping the en_US_POSIX locale specifically?
If there is the need that at least one locale be present, then can we keep a more generic one, like 'C', or 'POSIX', or whatever is not specific to a country?
package/python-babel/python-babel.mk PYTHON_BABEL_LICENSE = BSD-3-Clause · line 25
Quoted 4 lines
package/python-babel/python-babel.mk +4
-0+ done +endef +endif +PYTHON_BABEL_POST_INSTALL_TARGET_HOOKS += PYTHON_BABEL_CLEANUP_LOCALE
Like for the toolchain locale purge, it should be a target-fnalize hook;
PYTHON_BABEL_TARGET_FINALIZE_HOOKS
(Note that the systemd locale purge is done as a ROOTFS_PRE_CMD_HOOKS, but that is wrong, IMNSHO, and it too should have been a target finalize hook.)
Regards,
Yann E. MORIN.
Quoted 9 lines
package/python-babel/python-babel.mk +0
-0$(eval $(python-package)) $(eval $(host-python-package))Signature · 6 lines
2.40.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Hello Yann,
Quoted 22 lines · Yann E. MORIN
Peter, All,
On 2023-04-18 23:32 +0200, Peter Seiderer spake thusly:
Quoted 19 lines
Add purge loacle data support (inspired by
BR2_ENABLE_LOCALE_PURGEhandling inMakefile) to reduce target space allocation by the python-babel package, e.g. from 32MB to 24K for target/usr/lib/python3.11/site-packages/babel/locale-data with BR2_ENABLE_LOCALE_WHITELIST="Cen_USde_DE".Signed-off-by Peter Seiderer <[email protected]>--- package/python-babel/python-babel.mk | 12 ++++++++++++ 1 file changed, 12 insertions(+)package/python-babel/python-babel.mk +0
-0@@ -12,5 +12,17 @@ PYTHON_BABEL_LICENSE = BSD-3-Clause PYTHON_BABEL_LICENSE_FILES = LICENSE HOST_PYTHON_BABEL_DEPENDENCIES = host-python-pytz
Quoted 9 lines
package/python-babel/python-babel.mk PYTHON_BABEL_LICENSE = BSD-3-Clause · line 21
Quoted 7 lines
package/python-babel/python-babel.mk +7
-0+# purge locale data (if enabled), keep special en_US_POSIX data by default +ifeq ($(BR2_ENABLE_LOCALE_PURGE),y) +define PYTHON_BABEL_CLEANUP_LOCALE + for i in `ls $(TARGET_DIR)/usr/lib/python3.11/site-packages/babel/locale-data/*.dat`; \ + do \ + i_base=`basename "$$i" .dat`; \ + echo "$(BR2_ENABLE_LOCALE_WHITELIST) en_US_POSIX" | grep -qw "$$i_base" || rm "$$i"; \Why do you force keeping the
en_US_POSIXlocale specifically?
Because it is the python-babel default, see [1]...
Quoted 4 lines
If there is the need that at least one locale be present, then can we keep a more generic one, like 'C', or 'POSIX', or whatever is not specific to a country?
See above...
Quoted 8 lines
package/python-babel/python-babel.mk PYTHON_BABEL_LICENSE = BSD-3-Clause · line 25
Quoted 5 lines
package/python-babel/python-babel.mk +4
-0+ done +endef +endif +PYTHON_BABEL_POST_INSTALL_TARGET_HOOKS += PYTHON_BABEL_CLEANUP_LOCALELike for the toolchain locale purge, it should be a target-fnalize hook;
PYTHON_BABEL_TARGET_FINALIZE_HOOKS
Will fix it...
Thanks for review!
Regards,
Peter
Quoted 18 lines
(Note that the systemd locale purge is done as a
ROOTFS_PRE_CMD_HOOKS, but that is wrong, IMNSHO, and it too should have been a target finalize hook.)Regards,
Yann E. MORIN.Quoted 9 lines
package/python-babel/python-babel.mk +0
-1$(eval $(python-package)) $(eval $(host-python-package)) --2.40.0
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Peter, All,
On 2023-05-15 22:49 +0200, Peter Seiderer spake thusly:
Quoted 7 lines · Yann E. MORIN
Quoted 6 lines
On 2023-04-18 23:32 +0200, Peter Seiderer spake thusly:
Quoted 5 lines
Add purge loacle data support (inspired by
BR2_ENABLE_LOCALE_PURGEhandling inMakefile) to reduce target space allocation by the python-babel package, e.g. from 32MB to 24K for target/usr/lib/python3.11/site-packages/babel/locale-data with BR2_ENABLE_LOCALE_WHITELIST="Cen_USde_DE".
[--SNIP--]
Quoted 3 lines
Quoted 2 lines
package/python-babel/python-babel.mk PYTHON_BABEL_LICENSE = BSD-3-Clause · line 21
Quoted 1 line
package/python-babel/python-babel.mk +1
-0+ echo "$(BR2_ENABLE_LOCALE_WHITELIST) en_US_POSIX" | grep -qw "$$i_base" || rm "$$i"; \Why do you force keeping the
en_US_POSIXlocale specifically?Because it is the python-babel default, see [1]...
OK, Thanks! Say so explicitly in the commit log when you respin.
Regards,
Yann E. MORIN.
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 5 lines · On Tue, 18 Apr 2023 23:32:41 +0200 Peter Seiderer <[email protected]> wrote:
Add purge loacle data support (inspired by
BR2_ENABLE_LOCALE_PURGEhandling inMakefile) to reduce target space allocation by the python-babel package, e.g. from 32MB to 24K for target/usr/lib/python3.11/site-packages/babel/locale-data with BR2_ENABLE_LOCALE_WHITELIST="Cen_USde_DE".
I updated the commit log to add the explanation about en_US_POSIX, as requested by Yann.
Quoted 11 lines
Signed-off-by Peter Seiderer <[email protected]>--- package/python-babel/python-babel.mk | 12 ++++++++++++ 1 file changed, 12 insertions(+)package/python-babel/python-babel.mk +0
-0
package/python-babel/python-babel.mk PYTHON_BABEL_LICENSE = BSD-3-Clause · line 18
Quoted 7 lines
package/python-babel/python-babel.mk +4
-0@@ -12,5 +12,17 @@ PYTHON_BABEL_LICENSE = BSD-3-Clause PYTHON_BABEL_LICENSE_FILES = LICENSE HOST_PYTHON_BABEL_DEPENDENCIES = host-python-pytz +# purge locale data (if enabled), keep special en_US_POSIX data by default +ifeq ($(BR2_ENABLE_LOCALE_PURGE),y) +define PYTHON_BABEL_CLEANUP_LOCALE + for i in `ls $(TARGET_DIR)/usr/lib/python3.11/site-packages/babel/locale-data/*.dat`; \
The hardcoded 3.11 was not good, it would break at the next major bump of Python, replace with $(PYTHON3_VERSION_MAJOR).
package/python-babel/python-babel.mk PYTHON_BABEL_LICENSE = BSD-3-Clause · line 25
Quoted 7 lines
package/python-babel/python-babel.mk +7
-0+ do \ + i_base=`basename "$$i" .dat`; \ + echo "$(BR2_ENABLE_LOCALE_WHITELIST) en_US_POSIX" | grep -qw "$$i_base" || rm "$$i"; \ + done +endef +endif +PYTHON_BABEL_POST_INSTALL_TARGET_HOOKS += PYTHON_BABEL_CLEANUP_LOCALE
Changed to PYTHON_LABEL_TARGET_FINALIZE_HOOKS, as suggested by Yann, and moved inside the ifeq ... endif block.
Applied with those changes.
Thanks a lot!
Thomas
Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin Embedded Linux and Kernel engineering and training https://bootlin.com _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Hello Thomas, *,
Quoted 45 lines · Thomas Petazzoni via buildroot
Quoted 5 lines · On Tue, 18 Apr 2023 23:32:41 +0200 Peter Seiderer <[email protected]> wrote:
Add purge loacle data support (inspired by
BR2_ENABLE_LOCALE_PURGEhandling inMakefile) to reduce target space allocation by the python-babel package, e.g. from 32MB to 24K for target/usr/lib/python3.11/site-packages/babel/locale-data with BR2_ENABLE_LOCALE_WHITELIST="Cen_USde_DE".I updated the commit log to add the explanation about
en_US_POSIX, as requested by Yann.Quoted 11 lines
Signed-off-by Peter Seiderer <[email protected]>--- package/python-babel/python-babel.mk | 12 ++++++++++++ 1 file changed, 12 insertions(+)package/python-babel/python-babel.mk +0
-0package/python-babel/python-babel.mk PYTHON_BABEL_LICENSE = BSD-3-Clause · line 18
Quoted 7 lines
package/python-babel/python-babel.mk +4
-0@@ -12,5 +12,17 @@ PYTHON_BABEL_LICENSE = BSD-3-Clause PYTHON_BABEL_LICENSE_FILES = LICENSE HOST_PYTHON_BABEL_DEPENDENCIES = host-python-pytz +# purge locale data (if enabled), keep special en_US_POSIX data by default +ifeq ($(BR2_ENABLE_LOCALE_PURGE),y) +define PYTHON_BABEL_CLEANUP_LOCALE + for i in `ls $(TARGET_DIR)/usr/lib/python3.11/site-packages/babel/locale-data/*.dat`; \The hardcoded 3.11 was not good, it would break at the next major bump of Python, replace with $(PYTHON3_VERSION_MAJOR).
package/python-babel/python-babel.mk PYTHON_BABEL_LICENSE = BSD-3-Clause · line 25
Quoted 7 lines
package/python-babel/python-babel.mk +7
-0+ do \ + i_base=`basename "$$i" .dat`; \ + echo "$(BR2_ENABLE_LOCALE_WHITELIST) en_US_POSIX" | grep -qw "$$i_base" || rm "$$i"; \ + done +endef +endif +PYTHON_BABEL_POST_INSTALL_TARGET_HOOKS += PYTHON_BABEL_CLEANUP_LOCALEChanged to
PYTHON_LABEL_TARGET_FINALIZE_HOOKS, as suggested by Yann, and moved inside the ifeq ... endif block.
Quoted 3 lines
Applied with those changes.
Thanks a lot!
Seems I missed to send an updated (non RFC) version, many thanks for fixing and applying....
Regards,
Peter
Quoted 2 lines
Thomas
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot