Skip to contentSkip to conversation

resulttool: Add reproducible log extraction

PATCH In review Joshua Watt 2 participants 3 messages Sep 5, 2019 – Sep 6, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 resulttool: Add reproducible log extraction

Adds an argument to the log subcommand to extract the raw logs from the reproducible selftest.

To prevent ambiguity, the "--raw" argument has been renamed "--raw-ptest", although the old "--raw" argument is kept around for compatibility.

[YOCTO #13324]

---
 scripts/lib/resulttool/log.py | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)
scripts/lib/resulttool/log.py +30 -3
index 25c3396717e..2352c767d91 100644
@@ -16,6 +16,16 @@ def show_ptest(result, ptest, logger):
     print("ptest '%s' not found" % ptest)
     return 1
 
+def show_reproducible(result, reproducible, logger):
+    try:
+        print(result['reproducible'][reproducible]['diffoscope.text'])
+        return 0
+
+    except KeyError:
+        print("reproducible '%s' not found" % reproducible)
+        return 1
+
+
 def log(args, logger):
     results = resultutils.load_resultsdata(args.source)
 
@@ -40,17 +50,28 @@ def log(args, logger):
                         with open(dest, 'w') as f:
                             f.write(ptest['log'])
 
-        if args.raw:
+        if args.raw_ptest:
             if 'ptestresult.rawlogs' in r:
                 print(r['ptestresult.rawlogs']['log'])
             else:
-                print('Raw logs not found')
+                print('Raw ptest logs not found')
+                return 1
+
+        if args.raw_reproducible:
+            if 'reproducible.rawlogs' in r:
+                print(r['reproducible.rawlogs']['log'])
+            else:
+                print('Raw reproducible logs not found')
                 return 1
 
         for ptest in args.ptest:
             if not show_ptest(r, ptest, logger):
                 return 1
 
+        for reproducible in args.reproducible:
+            if not show_reproducible(r, reproducible, logger):
+                return 1
+
 def register_commands(subparsers):
     """Register subcommands from this plugin"""
     parser = subparsers.add_parser('log', help='show logs',
@@ -63,9 +84,15 @@ def register_commands(subparsers):
             help='show logs for a ptest')
     parser.add_argument('--dump-ptest', metavar='DIR',
             help='Dump all ptest log files to the specified directory.')
+    parser.add_argument('--reproducible', action='append', default=[],
+            help='show logs for a reproducible test')
     parser.add_argument('--prepend-run', action='store_true',
             help='''Dump ptest results to a subdirectory named after the test run when using --dump-ptest.
                     Required if more than one test run is present in the result file''')
     parser.add_argument('--raw', action='store_true',
-            help='show raw logs')
+            help='show raw (ptest) logs. Deprecated. Alias for "--raw-ptest"', dest='raw_ptest')
+    parser.add_argument('--raw-ptest', action='store_true',
+            help='show raw ptest log')
+    parser.add_argument('--raw-reproducible', action='store_true',
+            help='show raw reproducible build logs')
 
Signature · 1 line
2.21.0
#1 lore raw
Quoted 39 lines · Joshua Watt

Adds an argument to the log subcommand to extract the raw logs from the
reproducible selftest.

To prevent ambiguity, the "--raw" argument has been renamed "--raw-ptest", although the old "--raw" argument is kept around for compatibility.

[YOCTO #13324]

---
 scripts/lib/resulttool/log.py | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)
scripts/lib/resulttool/log.py +10 -0
index 25c3396717e..2352c767d91 100644
@@ -16,6 +16,16 @@ def show_ptest(result, ptest, logger):
     print("ptest '%s' not found" % ptest)
     return 1
 
+def show_reproducible(result, reproducible, logger):
+    try:
+        print(result['reproducible'][reproducible]['diffoscope.text'
 ])
+        return 0
+
+    except KeyError:
+        print("reproducible '%s' not found" % reproducible)
+        return 1
+
+
 def log(args, logger):
     results = resultutils.load_resultsdata(args.source)

scripts/lib/resulttool/log.py def log(args, logger): · line 52

Quoted 3 lines
scripts/lib/resulttool/log.py +0 -0
@@ -40,17 +50,28 @@ def log(args, logger):
                         with open(dest, 'w') as f:
                             f.write(ptest['log'])
 

You might want to consider this in autobuilder context:

http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder-helper/tree/scripts/generate-testresult-index.py#n121

Since the indexer could extract reproducible logs as well as ptest ones.

Nathan: We probably need to think about the new testsuite pieces in this context too...

Cheers,

Richard

#2 lore raw
Quoted 45 lines · Richard Purdie
Quoted 39 lines · Joshua Watt

Adds an argument to the log subcommand to extract the raw logs from the
reproducible selftest.

To prevent ambiguity, the "--raw" argument has been renamed "--raw-ptest", although the old "--raw" argument is kept around for compatibility.

[YOCTO #13324]

---
  scripts/lib/resulttool/log.py | 33 ++++++++++++++++++++++++++++++---
  1 file changed, 30 insertions(+), 3 deletions(-)
scripts/lib/resulttool/log.py +10 -0
index 25c3396717e..2352c767d91 100644
@@ -16,6 +16,16 @@ def show_ptest(result, ptest, logger):
      print("ptest '%s' not found" % ptest)
      return 1
 
+def show_reproducible(result, reproducible, logger):
+    try:
+        print(result['reproducible'][reproducible]['diffoscope.text'
 ])
+        return 0
+
+    except KeyError:
+        print("reproducible '%s' not found" % reproducible)
+        return 1
+
+
  def log(args, logger):
      results = resultutils.load_resultsdata(args.source)

scripts/lib/resulttool/log.py def log(args, logger): · line 52

Quoted 3 lines
scripts/lib/resulttool/log.py +0 -0
@@ -40,17 +50,28 @@ def log(args, logger):
                          with open(dest, 'w') as f:
                              f.write(ptest['log'])
 

You might want to consider this in autobuilder context:

Quoted 4 lines

http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder-helper/tree/scripts/generate-testresult-index.py#n121

Since the indexer could extract reproducible logs as well as ptest ones.

Yes, I've though about that, but so far I've purposely not done anything because I'm not sure what to do. Originally, I tried making the QA build run diffoscope on the package files that failed to build reproducible, but this took way too long and generated way too much data for the logs. I think that alternative *might* be to include the binary data from the packages so that diffoscope can be run on them offline, but this still seems like too much data.

Anyway, until we decide on a reasonable thing to do, I've omitted the --dump-reproducible option from resulttool. This does have the unfortunate side effect that you can't easily debug non-reproducible builds when the AB finds them, short of re-running the test locally, so it is something we should resolve soon.

Quoted 8 lines

Nathan: We probably need to think about the new testsuite pieces in this context too...

Cheers,

Richard