Skip to contentSkip to conversation

devpyshell fixes

PATCH New Ed Bartosh 1 participant 4 messages Jul 4, 2016

lore
Share & tools
lore

Ed Bartosh

author
#0 lore raw

v2 cover devpyshell fixes

Hi,

This patchset fixes several devpyshell bugs caused by moving to Python 3.

Please, note that it contains fix for double unbuffering, which was sent in a separate patchset targeting jethro, krogoth and master. It's included here just to keep it together with the rest of devpyshell fixes for master.

Changes in v2: removed bitbake patch from the series

The following changes since commit 5c11e365e19357f721c49d076971567e7b64b61b:

lib/oeqa: add Galculator to SDK and runtime tests (2016-07-01 16:22:48 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib ed/oe-core/devpyshell-python3
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/oe-core/devpyshell-python3

Ed Bartosh (3):
devshell.bbclass: fix double unbuffering
devpyshell: python3: flush stdout explicitly
oepydevshell-internal: python3: encode/decode pty content

 meta/classes/devshell.bbclass    |  5 ++---
 scripts/oepydevshell-internal.py | 12 +++++-------
 2 files changed, 7 insertions(+), 10 deletions(-)

--
Regards,
Ed

author
#1 lore raw

PATCH v2 1/3 devshell.bbclass: fix double unbuffering

stdout is already unbuffered in bitbake code. Attempt to do it again in devshell.bbclass causes this crash when running devpyshell:
File "scripts/oepydevshell-internal.py", line 29, in <module>

      pty = open(sys.argv[1], "w+b", 0)

IOError: [Errno 13] Permission denied: '/dev/pts/6'

Signed-off-by Ed Bartosh <[email protected]>
---
 meta/classes/devshell.bbclass | 1 -
 1 file changed, 1 deletion(-)
meta/classes/devshell.bbclass +0 -1
index 341d9c0..041ed15 100644
@@ -65,7 +65,6 @@ def devpyshell(d):
         os.dup2(m, sys.stdout.fileno())
         os.dup2(m, sys.stderr.fileno())
 
-        sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
         sys.stdin = os.fdopen(sys.stdin.fileno(), 'r', 0)
 
         bb.utils.nonblockingfd(sys.stdout)
Signature · 1 line
2.1.4
author
#2 lore raw

PATCH v2 2/3 devpyshell: python3: flush stdout explicitly

In reply to Ed Bartosh

Opening text stream in unbuffered mode raises the following exception In Python 3:
ValueError: can't have unbuffered text I/O

Fixed by leaving std* streams in text mode and flushing stdout explicitly.

Signed-off-by Ed Bartosh <[email protected]>
---
 meta/classes/devshell.bbclass    | 4 ++--
 scripts/oepydevshell-internal.py | 4 +---
 2 files changed, 3 insertions(+), 5 deletions(-)
meta/classes/devshell.bbclass +2 -2
index 041ed15..be71aff 100644
@@ -65,8 +65,6 @@ def devpyshell(d):
         os.dup2(m, sys.stdout.fileno())
         os.dup2(m, sys.stderr.fileno())
 
-        sys.stdin = os.fdopen(sys.stdin.fileno(), 'r', 0)
-
         bb.utils.nonblockingfd(sys.stdout)
         bb.utils.nonblockingfd(sys.stderr)
         bb.utils.nonblockingfd(sys.stdin)

@@ -92,6 +90,7 @@ def devpyshell(d):
             else:
                 prompt = ps1
             sys.stdout.write(prompt)
+            sys.stdout.flush()
 
         # Restore Ctrl+C since bitbake masks this
         def signal_handler(signal, frame):

@@ -113,6 +112,7 @@ def devpyshell(d):
                         continue
                 except EOFError as e:
                     sys.stdout.write("\n")
+                    sys.stdout.flush()
                 except (OSError, IOError) as e:
                     if e.errno == 11:
                         continue
scripts/oepydevshell-internal.py +1 -3
index 7761f66..31a75ac 100755
@@ -29,9 +29,6 @@ if len(sys.argv) != 3:
 pty = open(sys.argv[1], "w+b", 0)
 parent = int(sys.argv[2])
 
-# Don't buffer output by line endings
-sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
-sys.stdin = os.fdopen(sys.stdin.fileno(), 'r', 0)
 nonblockingfd(pty)
 nonblockingfd(sys.stdin)
 

@@ -64,6 +61,7 @@ try:
                     # Write a page at a time to avoid overflowing output 
                     # d.keys() is a good way to do that
                     sys.stdout.write(i[:4096])
+                    sys.stdout.flush()
                     i = i[4096:]
                 if sys.stdin in ready:
                     echonocbreak(sys.stdin.fileno())
Signature · 1 line
2.1.4
author
#3 lore raw

PATCH v2 3/3 oepydevshell-internal: python3: encode/decode pty content

In reply to Ed Bartosh

As /dev/pty opened in binary mode its content has to be decoded when reading from it and encoded when writing to it.

Signed-off-by Ed Bartosh <[email protected]>
---
 scripts/oepydevshell-internal.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
scripts/oepydevshell-internal.py +4 -4
index 31a75ac..a22bec3 100755
@@ -47,7 +47,7 @@ try:
     # Need cbreak/noecho whilst in select so we trigger on any keypress
     cbreaknoecho(sys.stdin.fileno())
     # Send our PID to the other end so they can kill us.
-    pty.write(str(os.getpid()) + "\n")
+    pty.write(str(os.getpid()).encode('utf-8') + b"\n")
     while True:
         try:
             writers = []

@@ -56,7 +56,7 @@ try:
             (ready, _, _) = select.select([pty, sys.stdin], writers , [], 0)
             try:
                 if pty in ready:
-                    i = i + pty.read()
+                    i = i + pty.read().decode('utf-8')
                 if i:
                     # Write a page at a time to avoid overflowing output 
                     # d.keys() is a good way to do that

@@ -65,9 +65,9 @@ try:
                     i = i[4096:]
                 if sys.stdin in ready:
                     echonocbreak(sys.stdin.fileno())
-                    o = input()
+                    o = input().encode('utf-8')
                     cbreaknoecho(sys.stdin.fileno())
-                    pty.write(o + "\n")
+                    pty.write(o + b"\n")
             except (IOError, OSError) as e:
                 if e.errno == 11:
                     continue
Signature · 1 line
2.1.4