Skip to content

Partially prepare for python3 and fix old Packages file reuse

Martin Jansa yocto 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest22 patches 7 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/22opkg-make-index: don't error out when some package disappearsopkg-make-index+24 -10 replies
2/22opkg-make-index: convert mtime to int before comparing itopkg-make-index+1 -10 replies
3/22opkg.py: Add knowledge about License field in ipk headersopkg.py+8 -00 replies
4/22arfile.py: handle six digit UIDsarfile.py+6 -10 replies
5/22Use python via the PATH, rather than hardcoding /usr/bin/pythonopkg-list-fields opkg-make-index opkg-show-deps +2 more+5 -50 replies
6/22opkg.py: use hashlib instead of old md5 moduleopkg.py+3 -30 replies
7/22opkg.py, arfile.py: report which file has wrong format and use the same testarfile.py opkg.py+11 -160 replies
8/22arfile: decode read lines as ascii string before getting fields from itarfile.py+1 -00 replies
9/222to3: use subprocess instead of commandsopkg-compare-indexes opkg-make-index opkg-show-deps +2 more+6 -90 replies
10/222to3: print fixesarfile.py makePackage opkg-compare-indexes +5 more+33 -330 replies
11/222to3: dictionary fixesarfile.py opkg-compare-indexes opkg-make-index +3 more+26 -260 replies
12/222to3: exception handling fixesarfile.py opkg-make-index opkg.py+5 -50 replies
13/22opkg-make-index: show OSError/IOErroropkg-make-index+10 -100 replies
14/22opkg.py: use string funtcions directly on string variableopkg-make-index opkg-show-deps opkg.py+8 -110 replies
15/22opkg.py: catch TypeError when reading control file, to show which one is failingopkg.py+10 -30 replies
16/22arfile: fix testarfile.py+4 -10 replies
17/22opkg-make-index: don't use stdout for Packages output and use __str__ instead of __repr__opkg-make-index opkg.py+11 -100 replies
18/22opkg.py: cast lines from controlfile as stringopkg.py+1 -10 replies
19/22opkg.py: computeFileMD5 only when we have fn, otherwise fails to read None fileopkg.py+11 -80 replies
20/22opkg.py: fix write_package when called from main testopkg.py+3 -30 replies
21/22opkg.py: use textwrap for description writingopkg.py+4 -10 replies
22/22opkg.py: improve test so it prints temporary control file with long description and then reads it backopkg.py+11 -10 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [opkg-utils][PATCH 00/22] Partially prepare for python3 and fix old Packages file reuse

First 5 patches are taken from oe-core.

Then there are some cleanups and fixes for issues I've found while looking for cause of very long package-index calls.

Some patches uses compatible changes from 2to3 (tested with python2.7), but python3 support is not 100% complete, but were closer.

And in the end I've found the cause.. printing long fields (e. g. description) with opkg.py was producing empty lines in Packages file (which are Packages entry separators) so only first entries till first empty line were processed from old Packages file and the rest was always extracted from control files in packaged .ipk.

And this also fixes Packages.filelist generation.

I don't use python very much, so please review.

And the speedup? Almost 60 times :)
before:

real    20m10.484s
user    18m32.335s
sys     0m11.885s

after

real    0m24.492s
user    0m23.748s
sys     0m0.565s

The following changes since commit 002d29bc605d7c2d02e4cf20a43c5277c15f5597:

[opkg-utils] fix install fail problem Thanks for khorben's patch :-) https://docs.openmoko.org/trac/attachment/ticket/2072/patch-opkg-utils_Makefile_install_path.diff (2008-11-03 03:59:59 +0000)

are available in the git repository at:
git://github.com/shr-project/opkg-utils jansa/pull
https://github.com/shr-project/opkg-utils/tree/jansa/pull

Christopher Larson (1):
Use python via the PATH, rather than hardcoding /usr/bin/python

Enrico Scholz (1):
opkg-make-index: convert mtime to int before comparing it

Khem Raj (1):
opkg.py: Add knowledge about License field in ipk headers

Martin Jansa (17):
opkg.py: use hashlib instead of old md5 module
opkg.py, arfile.py: report which file has wrong format and use the
same test
arfile: decode read lines as ascii string before getting fields from
it
2to3: use subprocess instead of commands
2to3: print fixes
2to3: dictionary fixes
2to3: exception handling fixes
opkg-make-index: show OSError/IOError
opkg.py: use string funtcions directly on string variable opkg.py: catch TypeError when reading control file, to show which one
is failing
arfile: fix test
opkg-make-index: don't use stdout for Packages output and use __str__
instead of __repr__
opkg.py: cast lines from controlfile as string
opkg.py: computeFileMD5 only when we have fn, otherwise fails to read
None file
opkg.py: fix write_package when called from main test opkg.py: use textwrap for description writing
opkg.py: improve test so it prints temporary control file with long
description and then reads it back

Richard Purdie (1):
opkg-make-index: don't error out when some package disappears

Scott Anderson (1):
arfile.py: handle six digit UIDs

 arfile.py            |   55 +++++++++++++---------
 makePackage          |    2 +-
 opkg-compare-indexes |   22 ++++----
 opkg-list-fields     |    4 +-
 opkg-make-index      |   74 +++++++++++++++++++----------
 opkg-show-deps       |   28 +++++------
 opkg-unbuild         |    4 +-
 opkg-update-index    |    5 +-
 opkg.py              |  125 +++++++++++++++++++++++++++++--------------------
 9 files changed, 186 insertions(+), 133 deletions(-)
Signature · 1 line
1.7.8.5