Skip to contentSkip to conversation

oe-pkgdata-util: ignore SIGPIPE

PATCH In review Chen Qi 2 participants 4 messages Apr 20, 2021 – May 6, 2021

lore
Share & tools
lore

Chen Qi

author
#0 lore raw

PATCH 1/1 oe-pkgdata-util: ignore SIGPIPE

oe-pkgdata-util sometimes outputs a large amount of data. When used with pipe, it's likely to get the following error.

BrokenPipeError: [Errno 32] Broken pipe

The problem could be reproduced by running `oe-pkgdata-util list-pkg | less'. Type 'q' after running the above command, and we get the error.

---
 scripts/oe-pkgdata-util | 5 +++++
 1 file changed, 5 insertions(+)
scripts/oe-pkgdata-util +5 -0
index 75dd23efa3..4aeb28879d 100755
@@ -17,6 +17,7 @@ import re
 import argparse
 import logging
 from collections import defaultdict, OrderedDict
+from signal import signal, SIGPIPE, SIG_DFL
 
 scripts_path = os.path.dirname(os.path.realpath(__file__))
 lib_path = scripts_path + '/lib'
@@ -615,6 +616,10 @@ def main():
         logger.error('Unable to find pkgdata directory %s' % args.pkgdata_dir)
         sys.exit(1)
 
+    # It's possible that this program will output large contents, and when used with a pipe in command line,
+    # we will get a 'BrokenPipeError: [Errno 32] Broken pipe'. Ignore the SIGPIPE to avoid such error.
+    signal(SIGPIPE, SIG_DFL)
+
     ret = args.func(args)
 
     return ret
Signature · 1 line
2.17.1
#1 lore raw
Quoted 5 lines · Chen Qi

oe-pkgdata-util sometimes outputs a large amount of data. When used with pipe, it's likely to get the following error.

  BrokenPipeError: [Errno 32] Broken pipe

Quoted 7 lines

The problem could be reproduced by running `oe-pkgdata-util list-pkg | less'. Type 'q' after running the above command, and we get the error.

---
 scripts/oe-pkgdata-util | 5 +++++

 1 file changed, 5 insertions(+)

This is far from clearcut.

https://mail.python.org/pipermail/python-dev/2018-June/154191.html

Not quite sure what to do about this as there are pros/cons...

Cheers,

Richard

Quoted 34 lines
scripts/oe-pkgdata-util +0 -0
index 75dd23efa3..4aeb28879d 100755
@@ -17,6 +17,7 @@ import re

 import argparse
 import logging
 from collections import defaultdict, OrderedDict
+from signal import signal, SIGPIPE, SIG_DFL

 scripts_path = os.path.dirname(os.path.realpath(__file__)) lib_path = scripts_path + '/lib'

scripts/oe-pkgdata-util +0 -0
@@ -615,6 +616,10 @@ def main():
          logger.error('Unable to find pkgdata directory %s' % args.pkgdata_dir)
          sys.exit(1)

scripts/oe-pkgdata-util +4 -0
+    # It's possible that this program will output large contents, and when used with a pipe in command line,
+    # we will get a 'BrokenPipeError: [Errno 32] Broken pipe'. Ignore the SIGPIPE to avoid such error.
+    signal(SIGPIPE, SIG_DFL)
+

     ret = args.func(args)

     return ret

author
#2 lore raw
Quoted 15 lines · Richard Purdie
Quoted 5 lines · Chen Qi

oe-pkgdata-util sometimes outputs a large amount of data. When used with pipe, it's likely to get the following error.

BrokenPipeError: [Errno 32] Broken pipe

Quoted 7 lines

The problem could be reproduced by running `oe-pkgdata-util list-pkg | less'. Type 'q' after running the above command, and we get the error.

---
  scripts/oe-pkgdata-util | 5 +++++
  1 file changed, 5 insertions(+)

This is far from clearcut.

Quoted 7 lines

https://mail.python.org/pipermail/python-dev/2018-June/154191.html

Not quite sure what to do about this as there are pros/cons...

Cheers,

Richard

Thanks Richard. I've read the material above and grabbed more knowledge on this topic.

At first, I also wanted to use the 'try ... except' clause to handle the output of oe-pkgdata-utils. But there are multiple places for the output, so I figured it's ugly and not necessary.

People mentioned situations where setting SIG_DFL for SIGPIPE is harmful. That's true. But in respect to this oe-pkgutils-data program, when setting SIG_DFL for SIGPIPE, what would be the actual problem in practice?

So I think https://github.com/python/cpython/pull/6773 is meaningful, but it does not apply to all programs.

Anyway, if you think using the 'try...except' clause is better, I could do it and send out V2. But I'm still wondering what extra benefit it could bring us.

Best Regards,
Chen Qi

Quoted 36 lines
Quoted 34 lines
scripts/oe-pkgdata-util +1 -0
index 75dd23efa3..4aeb28879d 100755
@@ -17,6 +17,7 @@ import re
  import argparse
  import logging
  from collections import defaultdict, OrderedDict
+from signal import signal, SIGPIPE, SIG_DFL
 
 
 

scripts_path = os.path.dirname(os.path.realpath(__file__)) lib_path = scripts_path + '/lib'

scripts/oe-pkgdata-util +4 -0
@@ -615,6 +616,10 @@ def main():
          logger.error('Unable to find pkgdata directory %s' % args.pkgdata_dir)
          sys.exit(1)
 
 
 
 
+    # It's possible that this program will output large contents, and when used with a pipe in command line,
+    # we will get a 'BrokenPipeError: [Errno 32] Broken pipe'. Ignore the SIGPIPE to avoid such error.
+    signal(SIGPIPE, SIG_DFL)
+

ret = args.func(args)

return ret

author
#3 lore raw
In reply to Richard Purdie
Quoted 15 lines · Richard Purdie
Quoted 5 lines · Chen Qi

oe-pkgdata-util sometimes outputs a large amount of data. When used with pipe, it's likely to get the following error.

BrokenPipeError: [Errno 32] Broken pipe

Quoted 7 lines

The problem could be reproduced by running `oe-pkgdata-util list-pkg | less'. Type 'q' after running the above command, and we get the error.

---
  scripts/oe-pkgdata-util | 5 +++++
  1 file changed, 5 insertions(+)

This is far from clearcut.

Quoted 7 lines

https://mail.python.org/pipermail/python-dev/2018-June/154191.html

Not quite sure what to do about this as there are pros/cons...

Cheers,

Richard

Thanks Richard. I've read the material above and grabbed more knowledge on this topic.

At first, I also wanted to use the 'try ... except' clause to handle the output of oe-pkgdata-utils. But there are multiple places for the output, so I figured it's ugly and not necessary.

People mentioned situations where setting SIG_DFL for SIGPIPE is harmful. That's true. But in respect to this oe-pkgutils-data program, when setting SIG_DFL for SIGPIPE, what would be the actual problem in practice?

So I think https://github.com/python/cpython/pull/6773 is meaningful, but it does not apply to all programs.

Anyway, if you think using the 'try...except' clause is better, I could do it and send out V2. But I'm still wondering what extra benefit it could bring us.

Best Regards,
Chen Qi

Quoted 36 lines
Quoted 34 lines
scripts/oe-pkgdata-util +1 -0
index 75dd23efa3..4aeb28879d 100755
@@ -17,6 +17,7 @@ import re
  import argparse
  import logging
  from collections import defaultdict, OrderedDict
+from signal import signal, SIGPIPE, SIG_DFL
 
 
 

scripts_path = os.path.dirname(os.path.realpath(__file__)) lib_path = scripts_path + '/lib'

scripts/oe-pkgdata-util +4 -0
@@ -615,6 +616,10 @@ def main():
          logger.error('Unable to find pkgdata directory %s' % args.pkgdata_dir)
          sys.exit(1)
 
 
 
 
+    # It's possible that this program will output large contents, and when used with a pipe in command line,
+    # we will get a 'BrokenPipeError: [Errno 32] Broken pipe'. Ignore the SIGPIPE to avoid such error.
+    signal(SIGPIPE, SIG_DFL)
+

ret = args.func(args)

return ret