Skip to contentSkip to conversation

utils/genrandconfig: fix runtime issue with Python 3

PATCH New Thomas Petazzoni 3 participants 8 messages Dec 3, 2019 – Dec 6, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 utils/genrandconfig: fix runtime issue with Python 3

With Python 3.7, genrandconfig fails with:

'str' object has no attribute 'decode'

We are already working on str objects, and there is no need to decode them, so we drop the call to decode_byte_list() and its definition as it was only used there.

Cc Arnout Vandecappelle (Essensium/Mind) <[email protected]>

---
This is of course very strange, as the decode_byte_list() function was introduced specifically to handle Python 3.x. Does anyone has an explanation?

---
 utils/genrandconfig | 10 +---------
 1 file changed, 1 insertion(+), 9 deletions(-)
utils/genrandconfig +1 -9
index fb28f4d356..5b60bc21d6 100755
@@ -39,14 +39,6 @@ def urlopen_closing(uri):
     return contextlib.closing(_urllib.urlopen(uri))
 
 
-if sys.hexversion >= 0x3000000:
-    def decode_byte_list(bl):
-        return [b.decode() for b in bl]
-else:
-    def decode_byte_list(e):
-        return e
-
-
 class SystemInfo:
     DEFAULT_NEEDED_PROGS = ["make", "git", "gcc", "timeout"]
     DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar", "diffoscope"]
@@ -128,7 +120,7 @@ def get_toolchain_configs(toolchains_csv, buildrootdir):
     with open(toolchains_csv) as r:
         # filter empty lines and comments
         lines = [t for t in r.readlines() if len(t.strip()) > 0 and t[0] != '#']
-        toolchains = decode_byte_list(lines)
+        toolchains = lines
     configs = []
 
     (_, _, _, _, hostarch) = os.uname()
Signature · 1 line
2.23.0
#1 lore raw

PATCH 1/1 utils/genrandconfig: fix runtime issue with Python 3

Quoted 8 lines · Thomas Petazzoni

With Python 3.7, genrandconfig fails with:

'str' object has no attribute 'decode'

We are already working on str objects, and there is no need to decode them, so we drop the call to decode_byte_list() and its definition as it was only used there.

Quoted 6 lines
Cc Arnout Vandecappelle (Essensium/Mind) <[email protected]>

---
This is of course very strange, as the decode_byte_list() function was introduced specifically to handle Python 3.x. Does anyone has an explanation?

Just guessing here, but: originally this file was downloaded with
urlopen_closing() instead of being read from a local file. Since we only use genrandconfig in the autobuilders in python2 (cfr. the shebang), nobody noticed that this broke when I switched to reading from file instead of URL two years ago (84929a53a4).

Regards,
Arnout

Quoted 33 lines
---
 utils/genrandconfig | 10 +---------
 1 file changed, 1 insertion(+), 9 deletions(-)
utils/genrandconfig +1 -9
index fb28f4d356..5b60bc21d6 100755
@@ -39,14 +39,6 @@ def urlopen_closing(uri):
     return contextlib.closing(_urllib.urlopen(uri))
 
 
-if sys.hexversion >= 0x3000000:
-    def decode_byte_list(bl):
-        return [b.decode() for b in bl]
-else:
-    def decode_byte_list(e):
-        return e
-
-
 class SystemInfo:
     DEFAULT_NEEDED_PROGS = ["make", "git", "gcc", "timeout"]
     DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar", "diffoscope"]
@@ -128,7 +120,7 @@ def get_toolchain_configs(toolchains_csv, buildrootdir):
     with open(toolchains_csv) as r:
         # filter empty lines and comments
         lines = [t for t in r.readlines() if len(t.strip()) > 0 and t[0] != '#']
-        toolchains = decode_byte_list(lines)
+        toolchains = lines
     configs = []
 
     (_, _, _, _, hostarch) = os.uname()
#2 lore raw

PATCH 1/1 utils/genrandconfig: fix runtime issue with Python 3

Hello,

Quoted 2 lines · On Tue, 3 Dec 2019 20:36:17 +0100 Arnout Vandecappelle <[email protected]> wrote:

Just guessing here, but: originally this file was downloaded with
urlopen_closing() instead of being read from a local file.

Ah, that must be it!

Quoted 2 lines

Since we only use genrandconfig in the autobuilders in python2 (cfr. the shebang)

I'm not sure why you say "in python2" ? The shebang in genrandconfig is:

#!/usr/bin/env python

and I indeed noticed the issue with Python 3.x because I started using genrandconfig on my updated Fedora system, which now uses Python 3.x as "python".

Quoted 2 lines

, nobody noticed that this broke when I switched to reading from file instead of URL two years ago (84929a53a4).

Indeed. I'll update the commit log and resend.

Thanks for the hint!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#3 lore raw

PATCH 1/1 utils/genrandconfig: fix runtime issue with Python 3

Quoted 8 lines · Thomas Petazzoni

Hello,

Quoted 2 lines · On Tue, 3 Dec 2019 20:36:17 +0100 Arnout Vandecappelle <[email protected]> wrote:

Just guessing here, but: originally this file was downloaded with
urlopen_closing() instead of being read from a local file.

Quoted 8 lines

Ah, that must be it!

Quoted 2 lines

Since we only use genrandconfig in the autobuilders in python2 (cfr. the shebang)

I'm not sure why you say "in python2" ? The shebang in genrandconfig is:

#!/usr/bin/env python

Almost all distros resolve this to Python2. I think even now only Arch and
Fedora 31 resolve it to Python3.

Quoted 8 lines

and I indeed noticed the issue with Python 3.x because I started using genrandconfig on my updated Fedora system, which now uses Python 3.x as "python".

Quoted 2 lines

, nobody noticed that this broke when I switched to reading from file instead of URL two years ago (84929a53a4).

Indeed. I'll update the commit log and resend.

Didn't I mention that I already merged it?

Regards,
Arnout

#4 lore raw

PATCH 1/1 utils/genrandconfig: fix runtime issue with Python 3

Quoted 10 lines · On Wed, 4 Dec 2019 13:52:48 +0100 Arnout Vandecappelle <[email protected]> wrote:
Quoted 8 lines

and I indeed noticed the issue with Python 3.x because I started using genrandconfig on my updated Fedora system, which now uses Python 3.x as "python".

Quoted 2 lines

, nobody noticed that this broke when I switched to reading from file instead of URL two years ago (84929a53a4).

Indeed. I'll update the commit log and resend.

Didn't I mention that I already merged it?

You did, but I saw it after replying to this e-mail :)

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#5 lore raw

PATCH 1/1 utils/genrandconfig: fix runtime issue with Python 3

In reply to Thomas Petazzoni
Quoted 10 lines · Thomas Petazzoni

With Python 3.7, genrandconfig fails with:

'str' object has no attribute 'decode'

We are already working on str objects, and there is no need to decode them, so we drop the call to decode_byte_list() and its definition as it was only used there.

Cc Arnout Vandecappelle (Essensium/Mind) <[email protected]>

Applied to master, thanks.

Regards,
Arnout

Quoted 37 lines

---
This is of course very strange, as the decode_byte_list() function was introduced specifically to handle Python 3.x. Does anyone has an explanation?

---
 utils/genrandconfig | 10 +---------
 1 file changed, 1 insertion(+), 9 deletions(-)
utils/genrandconfig +1 -9
index fb28f4d356..5b60bc21d6 100755
@@ -39,14 +39,6 @@ def urlopen_closing(uri):
     return contextlib.closing(_urllib.urlopen(uri))
 
 
-if sys.hexversion >= 0x3000000:
-    def decode_byte_list(bl):
-        return [b.decode() for b in bl]
-else:
-    def decode_byte_list(e):
-        return e
-
-
 class SystemInfo:
     DEFAULT_NEEDED_PROGS = ["make", "git", "gcc", "timeout"]
     DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar", "diffoscope"]
@@ -128,7 +120,7 @@ def get_toolchain_configs(toolchains_csv, buildrootdir):
     with open(toolchains_csv) as r:
         # filter empty lines and comments
         lines = [t for t in r.readlines() if len(t.strip()) > 0 and t[0] != '#']
-        toolchains = decode_byte_list(lines)
+        toolchains = lines
     configs = []
 
     (_, _, _, _, hostarch) = os.uname()
#6 lore raw

PATCH 1/1 utils/genrandconfig: fix runtime issue with Python 3

In reply to Thomas Petazzoni
Quoted 11 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

With Python 3.7, genrandconfig fails with:
'str' object has no attribute 'decode'

We are already working on str objects, and there is no need to decode them, so we drop the call to decode_byte_list() and its definition as it was only used there.

Cc Arnout Vandecappelle (Essensium/Mind) <[email protected]>

Committed to 2019.02.x and 2019.08.x, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#7 lore raw

PATCH 1/1 utils/genrandconfig: fix runtime issue with Python 3

In reply to Thomas Petazzoni
Quoted 11 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

With Python 3.7, genrandconfig fails with:
'str' object has no attribute 'decode'

We are already working on str objects, and there is no need to decode them, so we drop the call to decode_byte_list() and its definition as it was only used there.

Cc Arnout Vandecappelle (Essensium/Mind) <[email protected]>

Committed to 2019.11.x, thanks.

Signature · 1 line
Bye, Peter Korsgaard