Skip to contentSkip to conversation

meta: Cleanup python2

PATCH In review Robert Yang 2 participants 14 messages Jun 28, 2019 – Jul 12, 2019

lore
Share & tools
lore
#0 lore raw

cover meta: Cleanup python2

The following changes since commit 9f674a88c781c7092d5b3460922a1579b9fe4bf9:

busybox: Fix typo in syslog initscript (2019-06-27 13:27:54 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib rbt/py3
http://cgit.openembedded.org//log/?h=rbt/py3

Robert Yang (6):
ksum.py: python -> python3
wic: python2 -> python3
ext-sdk-prepare.py: python2 -> python3
oeqa: Cleanup /usr/bin/env python
package_rpm.bbclass: python2 -> python3
waf: python2 -> python3

 meta/classes/package_rpm.bbclass        |  2 +-
 meta/files/ext-sdk-prepare.py           |  2 +-
 meta/lib/oeqa/selftest/cases/wic.py     |  1 -
 meta/lib/oeqa/utils/logparser.py        |  1 -
 meta/recipes-graphics/eglinfo/files/waf |  4 +--
 scripts/lib/wic/__init__.py             |  2 +-
 scripts/lib/wic/ksparser.py             |  2 +-
 scripts/lib/wic/pluginbase.py           |  2 +-
 scripts/tiny/ksum.py                    | 44 ++++++++++++++++-----------------
 9 files changed, 29 insertions(+), 31 deletions(-)
Signature · 1 line
2.7.4
#1 lore raw

PATCH 1/6 ksum.py: python -> python3

* Testinfo:
In kernel's builddir:

  $ /path/to/oe-core/scripts/tiny/ksum.py

Collecting object files [DONE]

Totals:

vmlinux:

    text        data            bss             total
    10933110    3824470         1605632         16363212

modules (2004):

    text        data            bss             total
    46144408    3047516         167580          49359504

vmlinux + modules:

    text        data            bss             total
    57077518    6871986         1773212         65722716
---
 scripts/tiny/ksum.py | 44 ++++++++++++++++++++++----------------------
 1 file changed, 22 insertions(+), 22 deletions(-)
scripts/tiny/ksum.py +22 -22
index f38ae05..8f0e4c0 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 #
 # Copyright (c) 2016, Intel Corporation.
 #
@@ -62,7 +62,7 @@ def is_ko_file(filename):
     return False
 
 def collect_object_files():
-    print "Collecting object files recursively from %s..." % os.getcwd()
+    print("Collecting object files recursively from %s..." % os.getcwd())
     for dirpath, dirs, files in os.walk(os.getcwd()):
         for filename in files:
             if is_ko_file(filename):
@@ -70,7 +70,7 @@ def collect_object_files():
             elif is_vmlinux_file(filename):
                 global vmlinux_file
                 vmlinux_file = os.path.join(dirpath, filename)
-    print "Collecting object files [DONE]"
+    print("Collecting object files [DONE]")
 
 def add_ko_file(filename):
         p = Popen("size -t " + filename, shell=True, stdout=PIPE, stderr=PIPE)
@@ -78,9 +78,9 @@ def add_ko_file(filename):
         if len(output) > 2:
             sizes = output[-1].split()[0:4]
             if verbose:
-                print "     %10d %10d %10d %10d\t" % \
-                    (int(sizes[0]), int(sizes[1]), int(sizes[2]), int(sizes[3])),
-                print "%s" % filename[len(os.getcwd()) + 1:]
+                print("     %10d %10d %10d %10d\t" % \
+                    (int(sizes[0]), int(sizes[1]), int(sizes[2]), int(sizes[3])), end=' ')
+                print("%s" % filename[len(os.getcwd()) + 1:])
             global n_ko_files, ko_text, ko_data, ko_bss, ko_total
             ko_text += int(sizes[0])
             ko_data += int(sizes[1])
@@ -94,9 +94,9 @@ def get_vmlinux_totals():
         if len(output) > 2:
             sizes = output[-1].split()[0:4]
             if verbose:
-                print "     %10d %10d %10d %10d\t" % \
-                    (int(sizes[0]), int(sizes[1]), int(sizes[2]), int(sizes[3])),
-                print "%s" % vmlinux_file[len(os.getcwd()) + 1:]
+                print("     %10d %10d %10d %10d\t" % \
+                    (int(sizes[0]), int(sizes[1]), int(sizes[2]), int(sizes[3])), end=' ')
+                print("%s" % vmlinux_file[len(os.getcwd()) + 1:])
             global vmlinux_text, vmlinux_data, vmlinux_bss, vmlinux_total
             vmlinux_text += int(sizes[0])
             vmlinux_data += int(sizes[1])
@@ -129,20 +129,20 @@ def main():
     sum_ko_files()
     get_vmlinux_totals()
 
-    print "\nTotals:"
-    print "\nvmlinux:"
-    print "    text\tdata\t\tbss\t\ttotal"
-    print "    %-10d\t%-10d\t%-10d\t%-10d" % \
-        (vmlinux_text, vmlinux_data, vmlinux_bss, vmlinux_total)
-    print "\nmodules (%d):" % n_ko_files
-    print "    text\tdata\t\tbss\t\ttotal"
-    print "    %-10d\t%-10d\t%-10d\t%-10d" % \
-        (ko_text, ko_data, ko_bss, ko_total)
-    print "\nvmlinux + modules:"
-    print "    text\tdata\t\tbss\t\ttotal"
-    print "    %-10d\t%-10d\t%-10d\t%-10d" % \
+    print("\nTotals:")
+    print("\nvmlinux:")
+    print("    text\tdata\t\tbss\t\ttotal")
+    print("    %-10d\t%-10d\t%-10d\t%-10d" % \
+        (vmlinux_text, vmlinux_data, vmlinux_bss, vmlinux_total))
+    print("\nmodules (%d):" % n_ko_files)
+    print("    text\tdata\t\tbss\t\ttotal")
+    print("    %-10d\t%-10d\t%-10d\t%-10d" % \
+        (ko_text, ko_data, ko_bss, ko_total))
+    print("\nvmlinux + modules:")
+    print("    text\tdata\t\tbss\t\ttotal")
+    print("    %-10d\t%-10d\t%-10d\t%-10d" % \
         (vmlinux_text + ko_text, vmlinux_data + ko_data, \
-         vmlinux_bss + ko_bss, vmlinux_total + ko_total)
+         vmlinux_bss + ko_bss, vmlinux_total + ko_total))
 
 if __name__ == "__main__":
     try:
Signature · 1 line
2.7.4
#2 lore raw

PATCH 2/6 wic: python2 -> python3

In reply to Robert Yang
---
 scripts/lib/wic/__init__.py   | 2 +-
 scripts/lib/wic/ksparser.py   | 2 +-
 scripts/lib/wic/pluginbase.py | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
scripts/lib/wic/__init__.py +1 -1
index ba2d614..8556793 100644
@@ -1,4 +1,4 @@
-#!/usr/bin/env python -tt
+#!/usr/bin/env python3
 #
 # Copyright (c) 2007 Red Hat, Inc.
 # Copyright (c) 2011 Intel, Inc.
scripts/lib/wic/ksparser.py +1 -1
index 6204821..3e67003 100644
@@ -1,4 +1,4 @@
-#!/usr/bin/env python -tt
+#!/usr/bin/env python3
 #
 # Copyright (c) 2016 Intel, Inc.
 #
scripts/lib/wic/pluginbase.py +1 -1
index bfb73ca..f74d643 100644
@@ -1,4 +1,4 @@
-#!/usr/bin/env python -tt
+#!/usr/bin/env python3
 #
 # Copyright (c) 2011 Intel, Inc.
 #
Signature · 1 line
2.7.4
#3 lore raw

PATCH 3/6 ext-sdk-prepare.py: python2 -> python3

In reply to Robert Yang

This script is python3 indeed.

---
 meta/files/ext-sdk-prepare.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/files/ext-sdk-prepare.py +1 -1
index 96c5212..163d5e9 100644
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # Prepare the build system within the extensible SDK
 
Signature · 1 line
2.7.4
#4 lore raw

PATCH 4/6 oeqa: Cleanup /usr/bin/env python

In reply to Robert Yang
---
 meta/lib/oeqa/selftest/cases/wic.py | 1 -
 meta/lib/oeqa/utils/logparser.py    | 1 -
 2 files changed, 2 deletions(-)
meta/lib/oeqa/selftest/cases/wic.py +0 -1
index d16eae5..f1c6f39 100644
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 #
 # Copyright (c) 2015, Intel Corporation.
 #
meta/lib/oeqa/utils/logparser.py +0 -1
index b31214b..7313df8 100644
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 #
 # SPDX-License-Identifier: MIT
 #
Signature · 1 line
2.7.4
#5 lore raw

PATCH 5/6 package_rpm.bbclass: python2 -> python3

In reply to Robert Yang
---
 meta/classes/package_rpm.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/package_rpm.bbclass +1 -1
index 1a64cb2..a605a57 100644
@@ -36,7 +36,7 @@ def write_rpm_perfiledata(srcname, d):
     pkgd = d.getVar('PKGD')
 
     def dump_filerdeps(varname, outfile, d):
-        outfile.write("#!/usr/bin/env python\n\n")
+        outfile.write("#!/usr/bin/env python3\n\n")
         outfile.write("# Dependency table\n")
         outfile.write('deps = {\n')
         for pkg in packages.split():
Signature · 1 line
2.7.4
#6 lore raw

PATCH 6/6 waf: python2 -> python3

In reply to Robert Yang
---
 meta/recipes-graphics/eglinfo/files/waf | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-graphics/eglinfo/files/waf +2 -2
index 04ddd9f..2c83ed4 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # encoding: latin-1
 # Thomas Nagy, 2005-2018
 #
@@ -113,7 +113,7 @@ def unpack_wafdir(dir, src):
 	os.remove(tmp)
 	os.chdir(cwd)
 
-	try: dir = unicode(dir, 'mbcs')
+	try: dir = str(dir, 'mbcs')
 	except: pass
 	try:
 		from ctypes import windll
Signature · 1 line
2.7.4
#7 lore raw

Re: [PATCH 6/6] waf: python2 -> python3

Did you just do the minimum required, or take a new copy of waf from upstream?

(https://gitlab.com/ita1024/waf/)

Ross

Quoted 33 lines · Robert Yang
---
 meta/recipes-graphics/eglinfo/files/waf | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-graphics/eglinfo/files/waf +2 -3
index 04ddd9f..2c83ed4 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # encoding: latin-1
 # Thomas Nagy, 2005-2018
 #
@@ -113,7 +113,7 @@ def unpack_wafdir(dir, src):
        os.remove(tmp)
        os.chdir(cwd)
 
-       try: dir = unicode(dir, 'mbcs')
+       try: dir = str(dir, 'mbcs')
        except: pass
        try:
                from ctypes import windll
--

2.7.4

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#8 lore raw

Re: [PATCH 6/6] waf: python2 -> python3

Quoted 3 lines · Ross

Did you just do the minimum required, or take a new copy of waf from upstream?

(https://gitlab.com/ita1024/waf/)

No, I didn't take it from upstream, I just modified it and ran:

$ bitbake eglinfo-fb eglinfo-x11 eglinfo-wayland

// RObert

Quoted 38 lines

Ross

Quoted 33 lines · Robert Yang
---
  meta/recipes-graphics/eglinfo/files/waf | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-graphics/eglinfo/files/waf +2 -3
index 04ddd9f..2c83ed4 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
  # encoding: latin-1
  # Thomas Nagy, 2005-2018
  #
@@ -113,7 +113,7 @@ def unpack_wafdir(dir, src):
         os.remove(tmp)
         os.chdir(cwd)
 
-       try: dir = unicode(dir, 'mbcs')
+       try: dir = str(dir, 'mbcs')
         except: pass
         try:
                 from ctypes import windll
--

2.7.4

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#9 lore raw

Re: [PATCH 6/6] waf: python2 -> python3

In reply to Burton, Ross

Hi Ross,

Quoted 3 lines · Ross

Did you just do the minimum required, or take a new copy of waf from upstream?

(https://gitlab.com/ita1024/waf/)

I checked the one from upstream, it is still python2. The waf is the only one which still requires python2 in oe-core. We can get rid of python2 after this patch, what's your opinion, please?

// Robert

Quoted 38 lines

Ross

Quoted 33 lines · Robert Yang
---
  meta/recipes-graphics/eglinfo/files/waf | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-graphics/eglinfo/files/waf +2 -3
index 04ddd9f..2c83ed4 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
  # encoding: latin-1
  # Thomas Nagy, 2005-2018
  #
@@ -113,7 +113,7 @@ def unpack_wafdir(dir, src):
         os.remove(tmp)
         os.chdir(cwd)
 
-       try: dir = unicode(dir, 'mbcs')
+       try: dir = str(dir, 'mbcs')
         except: pass
         try:
                 from ctypes import windll
--

2.7.4

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#10 lore raw

Re: [PATCH 6/6] waf: python2 -> python3

Quoted 8 lines · Robert Yang
Quoted 3 lines · Ross

Did you just do the minimum required, or take a new copy of waf from upstream?

(https://gitlab.com/ita1024/waf/)

I checked the one from upstream, it is still python2. The waf is the only one which still requires python2 in oe-core. We can get rid of python2 after this patch, what's your opinion, please?

If that's the case then YES YES YES. :)

Ross

#11 lore raw

Re: [PATCH 6/6] waf: python2 -> python3

Quoted 5 lines · Ross
Quoted 3 lines · Robert Yang
Quoted 2 lines · Ross

Did you just do the minimum required, or take a new copy of waf from upstream?

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

(https://gitlab.com/ita1024/waf/)

I checked the one from upstream, it is still python2. The waf is the only one which still requires python2 in oe-core. We can get rid of python2 after this patch, what's your opinion, please?

If that's the case then YES YES YES. :)

Follow-up: do you mean remove python2 from HOSTTOOLS, or the recipe from oe-core?

$ bitbake -g world
$ grep -- '-> "python"' recipe-depends.dot
"kernel-devsrc" -> "python"
"ltp" -> "python"
"meta-world-pkgdata" -> "python"
"packagegroup-core-tools-profile" -> "python"
"packagegroup-core-tools-testapps" -> "python"
"packagegroup-self-hosted" -> "python"
"perf" -> "python"
"python-nose" -> "python"
"python-numpy" -> "python"
"python-scons" -> "python"
"python-setuptools" -> "python"

Nothing depends on the python-* in core, we can ignore the packagegroups, so that leaves kernel-devsrc, ltp, and perf. Also u-boot uses nativepython.

HOSTTOOLS is more complicated and there's still plenty of pieces that use host python. I've an old poky-contrib:ross/nopy2 branch that fixes a few more users of host python2 that needs rebasing and retesting.

Ross

Ross

#12 lore raw

Re: [PATCH 6/6] waf: python2 -> python3

Quoted 31 lines · Ross
Quoted 5 lines · Ross
Quoted 3 lines · Robert Yang
Quoted 2 lines · Ross

Did you just do the minimum required, or take a new copy of waf from upstream?

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

(https://gitlab.com/ita1024/waf/)

I checked the one from upstream, it is still python2. The waf is the only one which still requires python2 in oe-core. We can get rid of python2 after this patch, what's your opinion, please?

If that's the case then YES YES YES. :)

Follow-up: do you mean remove python2 from HOSTTOOLS, or the recipe from oe-core?

$ bitbake -g world
$ grep -- '-> "python"' recipe-depends.dot
"kernel-devsrc" -> "python"
"ltp" -> "python"
"meta-world-pkgdata" -> "python"
"packagegroup-core-tools-profile" -> "python"
"packagegroup-core-tools-testapps" -> "python"
"packagegroup-self-hosted" -> "python"
"perf" -> "python"
"python-nose" -> "python"
"python-numpy" -> "python"
"python-scons" -> "python"
"python-setuptools" -> "python"
Quoted 8 lines

Nothing depends on the python-* in core, we can ignore the packagegroups, so that leaves kernel-devsrc, ltp, and perf. Also u-boot uses nativepython.

HOSTTOOLS is more complicated and there's still plenty of pieces that use host python. I've an old poky-contrib:ross/nopy2 branch that fixes a few more users of host python2 that needs rebasing and retesting.

I meant remove python2 from HOSTTOOLS, but it seems not easy to do atm, so maybe we still need keep it for many years. I have this idea is because host such as Redhat Linux 8 doesn't install python2 by default any more.

// Robert

Quoted 5 lines

Ross

Ross

#13 lore raw

Re: [PATCH 6/6] waf: python2 -> python3

Quoted 8 lines · Robert Yang
Quoted 4 lines

HOSTTOOLS is more complicated and there's still plenty of pieces that use host python. I've an old poky-contrib:ross/nopy2 branch that fixes a few more users of host python2 that needs rebasing and retesting.

I meant remove python2 from HOSTTOOLS, but it seems not easy to do atm, so maybe we still need keep it for many years. I have this idea is because host such as Redhat Linux 8 doesn't install python2 by default any more.

I left a world build going last night, some more fixes in nopy2. There's still about 10 recipes failing.

Ross