Skip to contentSkip to conversation

oeqa/reproducibility: Work around subunit protocol limitations/bugs

PATCH Richard Purdie 2 participants 2 messages Apr 9 – Apr 10 1 message not in the archive

lore
Share & tools
lore
#1 lore raw
In reply to a message not in the archive
Quoted 25 lines · Richard Purdie via lists.openembedded.org

We've seen reproducibility failures with:

  File "/usr/lib/python3/dist-packages/subunit/chunked.py", line 108, in _read_length raise ValueError("chunk header invalid: %r" % count_str)
ValueError: chunk header invalid: b'\n'

which leads to:

2026-04-09 01:18:20,886 - oe-selftest - INFO - ERROR: broken-runner

and UNKNOWN status for builds queued in that runner.

This appears to be when reproducibility builds fail with:

Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/bringup-fast/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/reproducible.py", line 407, in test_reproducible_builds self.fail('\n'.join(fails))
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.14/unittest/case.py", line 750, in fail
    raise self.failureException(msg)

which is possibly from bad placement of '\n' in the byte stream. It doesn't 100% reproduce since the placement is critical but from the subunit code, it does use <int>\n as a header in the protocol.

Quoted 6 lines

I've wondered if the presence of "\n" in the traceback code is part of the issue so remove "\n" from the traceback by writing the python code differently just in case it helps.

Ideally we'd report the issue and fix subunit but I was unable to work out the exact sequence to reproduce and any fix would take time to propogate anyway.

There are other better ideas about what is going on here to cause this and this patch won't help. The issue is thankfully rare so we can drop this for now until we come up with something better. We can open a bug to track the issue.

Cheers,

Richard

#2 lore raw
Quoted 30 lines · Richard Purdie via lists.openembedded.org
Quoted 29 lines · Richard Purdie via lists.openembedded.org

We've seen reproducibility failures with:

  File "/usr/lib/python3/dist-packages/subunit/chunked.py", line 108, in _read_length raise ValueError("chunk header invalid: %r" % count_str)
ValueError: chunk header invalid: b'\n'

which leads to:

2026-04-09 01:18:20,886 - oe-selftest - INFO - ERROR: broken-runner

and UNKNOWN status for builds queued in that runner.

This appears to be when reproducibility builds fail with:

Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/bringup-fast/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/reproducible.py", line 407, in test_reproducible_builds self.fail('\n'.join(fails))
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.14/unittest/case.py", line 750, in fail
    raise self.failureException(msg)

which is possibly from bad placement of '\n' in the byte stream. It doesn't 100% reproduce since the placement is critical but from the subunit code, it does use <int>\n as a header in the protocol.

I've wondered if the presence of "\n" in the traceback code is part of the issue so remove "\n" from the traceback by writing the python code differently just in case it helps.

Quoted 7 lines
Quoted 2 lines

Ideally we'd report the issue and fix subunit but I was unable to work out the exact sequence to reproduce and any fix would take time to propogate anyway.

There are other better ideas about what is going on here to cause this and this patch won't help. The issue is thankfully rare so we can drop this for now until we come up with something better. We can open a bug to track the issue.

For anyone following along, I've filed an issue in bugzilla:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=16242

Best regards,

Signature · 1 line
Paul Barker