oeqa/reproducibility: Work around subunit protocol limitations/bugs
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 25 lines · Richard Purdie via lists.openembedded.org
We've seen reproducibility failures with:
File "
/usr/lib/python3/dist-packages/subunit/chunked.py", line 108, in_read_lengthraiseValueError("chunk header invalid: %r" % count_str)
ValueError: chunk header invalid: b'\n'which leads to:
2026-04-09 01:18:20,886 - oe-selftest - INFO - ERROR: broken-runnerand UNKNOWN status for builds queued in that runner.
This appears to be when reproducibility builds fail with:
Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/bringup-fast/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/reproducible.py", line 407, intest_reproducible_buildsself.fail('\n'.join(fails))
~~~~~~~~~^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.14/unittest/case.py", line 750, in fail
raiseself.failureException(msg)which is possibly from bad placement of '\n' in the byte stream. It doesn't 100% reproduce since the placement is critical but from the subunit code, it does use <int>\n as a header in the protocol.
Quoted 6 lines
I've wondered if the presence of "\n" in the traceback code is part of the issue so remove "\n" from the traceback by writing the python code differently just in case it helps.
Ideally we'd report the issue and fix subunit but I was unable to work out the exact sequence to reproduce and any fix would take time to propogate anyway.
There are other better ideas about what is going on here to cause this and this patch won't help. The issue is thankfully rare so we can drop this for now until we come up with something better. We can open a bug to track the issue.
Cheers,
Richard
Quoted 30 lines · Richard Purdie via lists.openembedded.org
Quoted 29 lines · Richard Purdie via lists.openembedded.org
We've seen reproducibility failures with:
File "
/usr/lib/python3/dist-packages/subunit/chunked.py", line 108, in_read_lengthraiseValueError("chunk header invalid: %r" % count_str)
ValueError: chunk header invalid: b'\n'which leads to:
2026-04-09 01:18:20,886 - oe-selftest - INFO - ERROR: broken-runnerand UNKNOWN status for builds queued in that runner.
This appears to be when reproducibility builds fail with:
Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/bringup-fast/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/reproducible.py", line 407, intest_reproducible_buildsself.fail('\n'.join(fails))
~~~~~~~~~^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.14/unittest/case.py", line 750, in fail
raiseself.failureException(msg)which is possibly from bad placement of '\n' in the byte stream. It doesn't 100% reproduce since the placement is critical but from the subunit code, it does use <int>\n as a header in the protocol.
I've wondered if the presence of "\n" in the traceback code is part of the issue so remove "\n" from the traceback by writing the python code differently just in case it helps.
Quoted 7 lines
Quoted 2 lines
Ideally we'd report the issue and fix subunit but I was unable to work out the exact sequence to reproduce and any fix would take time to propogate anyway.
There are other better ideas about what is going on here to cause this and this patch won't help. The issue is thankfully rare so we can drop this for now until we come up with something better. We can open a bug to track the issue.
For anyone following along, I've filed an issue in bugzilla:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=16242
Best regards,
Signature · 1 line
Paul Barker