support/size-stats: actually handle .pyc files
v2 cover support/size-stats: actually handle .pyc files
From: "Yann E. MORIN" <[email protected]>
Hello All!
This small two-patch series properly associates .pyc files to the corresponding package, to account for the PEP 3147 layout.
Changes v1 -> v2:
- use python's cache_from_source() instead of open-coding the path (Vincent)
Regards,
Yann E. MORIN.
The following changes since commit 04d7d7bdeb907483e7db0005b403484f379cf84b
package/mosh: bump gcc version requirement due to protobuf (2026-09-13 21:59:28 +0200)
are available as patches in this mail series,
for you to apply patches up to 9e49dfb4368becd006de1be4977695caea90c237
support/scripts/size-stats: properly assign pre-compiled .pyc files (2026-09-14 14:29:42 +0200)
----------------------------------------------------------------
Yann E. MORIN (2):
support/scripts/size-stats: prepare for more .py associated files support/scripts/size-stats: properly assign pre-compiled .pyc files
support/scripts/size-stats | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-)
--
____________ .-----------------.--------------------: _ :------------------. | Yann E. MORIN | Real-Time Embedded | __/ ) | /"\ ASCII RIBBON | | | Software Designer | _/ - /' | \ / CAMPAIGN | | +33 638.411.245 '--------------------: (_ `--, | X AGAINST | | yann.morin (at) orange.com |_=" ,--' | / \ HTML MAIL |
'--------------------------------------:______/_____:------------------'
____________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 1/2 support/scripts/size-stats: prepare for more .py associated files
From: "Yann E. MORIN" <[email protected]>
--- support/scripts/size-stats | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-)
support/scripts/size-stats +18 -3
@@ -54,9 +54,8 @@ class Config: # pkg: package to which the file belongs # def add_file(filesdict, relpath, abspath, pkg): - if relpath.endswith(".py"): - # also check for compiled .pyc file - add_file(filesdict, relpath + "c", abspath + "c", pkg) + for relpath2, abspath2 in get_associated_files(relpath, abspath): + add_file(filesdict, relpath2, abspath2, pkg) if not os.path.exists(abspath): return if os.path.islink(abspath):
@@ -65,6 +64,22 @@ def add_file(filesdict, relpath, abspath, pkg): filesdict[relpath] = (pkg, sz) +# +# Returns an iterator over the files associated with the given one +# as a 2-tuple of (relpath, abspath). +# +# Supported files: +# *.py => yields the list of corresponding pre-compiled .pyc files +# +# relpath: relative path of the file +# fullpath: absolute path to the file +# +def get_associated_files(relpath, abspath): + # also check for compiled .pyc file + if relpath.endswith(".py"): + yield (relpath + "c", abspath + "c") + + # # This function returns a dict where each key is the path of a file in # the root filesystem, and the value is a tuple containing two
Signature · 18 lines
2.43.0 ____________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 2/2 support/scripts/size-stats: properly assign pre-compiled .pyc files
In reply to [email protected]From: "Yann E. MORIN" <[email protected]>
Since commit 3fed42456693 (package/python3: use the provided pyc compiler), we abide by the PEP 3147 guidelines on where the pyc files are located. In practice, this means that:
1. BR2_PACKAGE_PYTHON3_PY_ONLY: there is no precompiled file; this was still working so far;
2. BR2_PACKAGE_PYTHON3_PYC_ONLY: there is only the .pyc file in the same directory as the original .py file; tjhis was still working so far, as it uses the same legacy pre-PEP 3147 layout;
3. BR2_PACKAGE_PYTHON3_PY_PYC: for a .py file /foo/bar.py, the corresponding .pyc file will be /foo/__pycache__/bar.pythonX.Y.pyc (where X.Y is the current major.minor python version, e.g. 3.14); this is what is broken now.
Add support for this layout in the size-stats script.
When we associate files to a package, we only have the original name, and we must construct the associated filenames. We could construct the path to the .pyc file manually (like we do for the legacy layout), ut this is brittel, and will break if/when python changes the layout. Instead, use importlib's helper cache_from_source(), as suggested by Vincent.
---
Changes v1 -> v2:
- use cache_from_source() rather than compute the path manually (Veincent)
--- support/scripts/size-stats | 10 ++++++++++ 1 file changed, 10 insertions(+)
support/scripts/size-stats +10 -0
@@ -23,6 +23,7 @@ import argparse import csv import collections import math +import importlib try: import matplotlib
@@ -77,7 +78,16 @@ def add_file(filesdict, relpath, abspath, pkg): def get_associated_files(relpath, abspath): # also check for compiled .pyc file if relpath.endswith(".py"): + # Legacy .pyc next to .py, or .pyc-only setup yield (relpath + "c", abspath + "c") + # PEP 3147 layout + # importlib._bootstrap_external.cache_from_source() works on the + # path string, not on actual files being present/absent, and so + # it also works for relative paths... + yield ( + importlib._bootstrap_external.cache_from_source(relpath), + importlib._bootstrap_external.cache_from_source(abspath), + ) #
Signature · 18 lines
2.43.0 ____________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
All,
+Peter, +Thomas
On 2026-09-14 14:48 +0200, [email protected] spake thusly:
Quoted 2 lines
This small two-patch series properly associates .pyc files to the corresponding package, to account for the PEP 3147 layout.
Weird: this patch series did not end up in patchwork, although it is on lore:
https://lore.kernel.org/buildroot/[email protected]/
Should I resend it?
Regards,
Yann E. MORIN.
Signature · 23 lines
____________ .-----------------.--------------------: _ :------------------. | Yann E. MORIN | Real-Time Embedded | __/ ) | /"\ ASCII RIBBON | | | Software Designer | _/ - /' | \ / CAMPAIGN | | +33 638.411.245 '--------------------: (_ `--, | X AGAINST | | yann.morin (at) orange.com |_=" ,--' | / \ HTML MAIL | '--------------------------------------:______/_____:------------------' ____________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"yann" == yann morin <[email protected]> writes:
All,
+Peter, +ThomasOn 2026-09-14 14:48 +0200, [email protected] spake thusly:
Quoted 8 lines
Quoted 2 lines
This small two-patch series properly associates .pyc files to the corresponding package, to account for the PEP 3147 layout.
Weird: this patch series did not end up in patchwork, although it is on lore:
https://lore.kernel.org/buildroot/[email protected]/Should I resend it?
It it certainly easier to have it in patchwork, please do.
Signature · 5 lines
Bye, Peter Korsgaard _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot