tests/unit/test_ksparser_parse: parse the .wks file, fix a diskid crash
cover tests/unit/test_ksparser_parse: parse the .wks file, fix a diskid crash
This continues the standalone unit-test work, moving up from ksparser's argparse types to the KickStart parser that drives a whole .wks file. It keeps the shape the suite settled on: the source fix is its own standalone commit, and the green test module lands last, so the suite passes at every commit.
The fix: a bootloader line with --ptable msdos and a non-integer --diskid builds its error message with "... % self.ptable". self is the KickStart instance, which has no ptable attribute, so the interpolation raises AttributeError before the intended KickStartError. The user gets an opaque traceback instead of a described error. Interpolate the offending value instead (matching the gpt branch just below), and fix the "--ptbale" typo in the same message.
The test module adds end-to-end coverage for KickStart(): happy-path part/bootloader lines, degenerate files (empty, comment-only, CRLF, unknown directive), mutually-exclusive and invalid option combinations, multiple-bootloader rejection, diskid parsing for both ptables, and include handling. get_bitbake_var() is neutralised so the parser runs with no BitBake and no host tools. Backing the fix out turns the two msdos --diskid cases red.
Trevor Woerner (2):
ksparser: fix crash on an invalid msdos --diskid tests/unit/test_ksparser_parse: cover the .wks KickStart parser
src/wic/ksparser.py | 4 +- tests/unit/test_ksparser_parse.py | 217 ++++++++++++++++++++++++++++++ 2 files changed, 219 insertions(+), 2 deletions(-) create mode 100644 tests/unit/test_ksparser_parse.py
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
PATCH 1/2 ksparser: fix crash on an invalid msdos --diskid
When a bootloader line sets --ptable msdos with a --diskid that is not a 32-bit integer, the error path builds its message with "... % self.ptable". self is the KickStart instance, which has no ptable attribute, so the interpolation raises AttributeError before the intended KickStartError is raised. Instead of a described error naming the bad value, the user sees an opaque traceback.
Interpolate the offending value (parsed.diskid) into the message, matching the gpt branch just below, so a bad msdos --diskid raises a KickStartError that names it. Fix the "--ptbale" typo in the same message while here.
AI-Generated: codex/claude-opus 4.8 (xhigh)
--- src/wic/ksparser.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
src/wic/ksparser.py +2 -2
@@ -322,9 +322,9 @@ class KickStart(): try: self.bootloader.diskid = int(parsed.diskid, 0) except ValueError: - err = "with --ptbale msdos only 32bit integers " \ + err = "with --ptable msdos only 32bit integers " \ "are allowed for --diskid. %s could not " \ - "be parsed" % self.ptable + "be parsed" % parsed.diskid raise KickStartError(err) else: try:
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
PATCH 2/2 tests/unit/test_ksparser_parse: cover the .wks KickStart parser
In reply to Trevor WoernerAdd end-to-end coverage for KickStart(): happy-path part/bootloader lines, degenerate files (empty, comment-only, CRLF, unknown directive), mutually-exclusive and invalid option combinations, multiple-bootloader rejection, diskid parsing for both ptables, and include handling. The bitbake variable lookup is neutralised so the parser runs with no BitBake and no host tools.
The msdos --diskid cases assert that a non-integer value raises a KickStartError naming the offending value.
AI-Generated: codex/claude-opus 4.8 (xhigh)
--- tests/unit/test_ksparser_parse.py | 217 ++++++++++++++++++++++++++++++ 1 file changed, 217 insertions(+) create mode 100644 tests/unit/test_ksparser_parse.py
tests/unit/test_ksparser_parse.py new file +217 -0
@@ -0,0 +1,217 @@ +r""" +End-to-end tests for the KickStart .wks parser (wic/ksparser.py). + +KickStart(confpath) reads a .wks file line by line, runs each line +through an argparse-based grammar, validates filesystem/option +combinations, and builds Partition objects plus a bootloader config. +It is pure logic apart from get_bitbake_var() (used for ${VAR} +expansion and the bootloader APPEND), which these tests neutralise so +no bitbake or host tools are needed. + +Each test writes a small .wks file and asserts on the parsed result or +on the specific error raised. +""" + +import sys +from pathlib import Path + +import pytest + +_SRC = Path(__file__).resolve().parent.parent.parent / "src" +if str(_SRC) not in sys.path: + sys.path.insert(0, str(_SRC)) + +import wic.misc as misc +import wic.ksparser as ksparser +from wic.ksparser import KickStart, KickStartError + + [email protected](autouse=True) +def no_bitbake(monkeypatch): + """Neutralise bitbake variable lookups in both modules. + + get_bitbake_var is imported into ksparser's namespace and also used + via misc; patch both so ${VAR} expansion and APPEND resolve to None + instead of trying to run `bitbake -e`. + """ + monkeypatch.setattr(misc, "get_bitbake_var", lambda *a, **k: None) + monkeypatch.setattr(ksparser, "get_bitbake_var", lambda *a, **k: None) + + +def _wks(tmp_path, text): + path = tmp_path / "test.wks" + path.write_text(text) + return str(path) + + +def _parse(tmp_path, text): + return KickStart(_wks(tmp_path, text)) + + +# --------------------------------------------------------------------------- +# Happy path +# --------------------------------------------------------------------------- + +class TestParseHappyPath: + + def test_single_part(self, tmp_path): + ks = _parse(tmp_path, "part / --source rootfs --fstype ext4 --size 100M\n") + assert len(ks.partitions) == 1 + p = ks.partitions[0] + assert p.fstype == "ext4" + assert p.size == 100 * 1024 + assert p.mountpoint == "/" + + def test_defaults_applied_without_fixed_size(self, tmp_path): + ks = _parse(tmp_path, "part / --source rootfs --fstype ext4 --size 1M\n") + p = ks.partitions[0] + assert p.overhead_factor == KickStart.DEFAULT_OVERHEAD_FACTOR + assert p.extra_filesystem_space == KickStart.DEFAULT_EXTRA_FILESYSTEM_SPACE + + def test_partnum_increments(self, tmp_path): + ks = _parse( + tmp_path, + "part /boot --source bootimg --fstype vfat --size 50M\n" + "part / --source rootfs --fstype ext4 --size 100M\n", + ) + assert [p.lineno for p in ks.partitions] == [1, 2] + assert len(ks.partitions) == 2 + + def test_bootloader_parsed(self, tmp_path): + ks = _parse(tmp_path, "bootloader --ptable gpt\n") + assert ks.bootloader.ptable == "gpt" + + def test_comment_and_blank_lines_ignored(self, tmp_path): + ks = _parse( + tmp_path, + "# a comment\n" + "\n" + " \n" + "part / --source rootfs --fstype ext4 --size 1M\n", + ) + assert len(ks.partitions) == 1 + + +# --------------------------------------------------------------------------- +# Degenerate files +# --------------------------------------------------------------------------- + +class TestParseDegenerateFiles: + + def test_empty_file(self, tmp_path): + ks = _parse(tmp_path, "") + assert ks.partitions == [] + assert ks.bootloader is not None # defaults filled in + + def test_comment_only_file(self, tmp_path): + ks = _parse(tmp_path, "# only comments\n# nothing else\n") + assert ks.partitions == [] + + def test_crlf_line_endings(self, tmp_path): + ks = _parse(tmp_path, "part / --source rootfs --fstype ext4 --size 1M\r\n") + assert len(ks.partitions) == 1 + + def test_unknown_directive_rejected(self, tmp_path): + with pytest.raises(KickStartError): + _parse(tmp_path, "frobnicate everything\n") + + +# --------------------------------------------------------------------------- +# Mutually-exclusive and invalid option combinations +# --------------------------------------------------------------------------- + +class TestParseInvalidCombinations: + + def test_size_and_fixed_size_conflict(self, tmp_path): + with pytest.raises(KickStartError): + _parse(tmp_path, "part / --source rootfs --fstype ext4 " + "--size 100M --fixed-size 200M\n") + + def test_fixed_size_with_overhead_factor_rejected(self, tmp_path): + with pytest.raises(KickStartError): + _parse(tmp_path, "part / --source rootfs --fstype ext4 " + "--fixed-size 100M --overhead-factor 1.5\n") + + def test_squashfs_with_label_rejected(self, tmp_path): + with pytest.raises(KickStartError): + _parse(tmp_path, "part / --source rootfs --fstype squashfs --label boot\n") + + def test_squashfs_with_fsuuid_rejected(self, tmp_path): + with pytest.raises(KickStartError): + _parse(tmp_path, "part / --source rootfs --fstype squashfs --fsuuid 0x1234\n") + + def test_erofs_with_label_rejected(self, tmp_path): + with pytest.raises(KickStartError): + _parse(tmp_path, "part / --source rootfs --fstype erofs --label x --size 1M\n") + + def test_use_label_without_label_rejected(self, tmp_path): + with pytest.raises(KickStartError): + _parse(tmp_path, "part / --source rootfs --fstype ext4 --size 1M --use-label\n") + + def test_msdos_fsuuid_too_long_rejected(self, tmp_path): + with pytest.raises(KickStartError): + _parse(tmp_path, "part / --source rootfs --fstype msdos " + "--size 1M --fsuuid 0xDEADBEEF12\n") + + +# --------------------------------------------------------------------------- +# Multiple bootloaders +# --------------------------------------------------------------------------- + +class TestParseBootloader: + + def test_two_bootloaders_rejected(self, tmp_path): + with pytest.raises(KickStartError): + _parse(tmp_path, "bootloader --ptable gpt\nbootloader --ptable msdos\n") + + def test_msdos_diskid_integer(self, tmp_path): + ks = _parse(tmp_path, "bootloader --ptable msdos --diskid 0x12345678\n") + assert ks.bootloader.diskid == 0x12345678 + + def test_gpt_bad_diskid_raises_kickstart_error(self, tmp_path): + with pytest.raises(KickStartError): + _parse(tmp_path, "bootloader --ptable gpt --diskid not-a-uuid\n") + + def test_msdos_bad_diskid_raises_kickstart_error(self, tmp_path): + # A non-integer --diskid on an msdos ptable must raise a described + # KickStartError, not an opaque AttributeError from the error path + # interpolating a missing attribute. + with pytest.raises(KickStartError) as excinfo: + _parse(tmp_path, "bootloader --ptable msdos --diskid not-an-int\n") + # The message names the offending value so the user can see what + # was rejected. + assert "not-an-int" in str(excinfo.value) + + def test_msdos_bad_diskid_hex_string_rejected(self, tmp_path): + # int(x, 0) parses "0x..." but a stray non-hex digit still fails; + # the same described KickStartError path handles it. + with pytest.raises(KickStartError) as excinfo: + _parse(tmp_path, "bootloader --ptable msdos --diskid 0xZZ\n") + assert "0xZZ" in str(excinfo.value) + + def test_gpt_diskid_uuid_parsed(self, tmp_path): + ks = _parse( + tmp_path, + "bootloader --ptable gpt " + "--diskid 12345678-1234-5678-1234-567812345678\n", + ) + assert str(ks.bootloader.diskid) == \ + "12345678-1234-5678-1234-567812345678" + + +# --------------------------------------------------------------------------- +# Include handling +# --------------------------------------------------------------------------- + +class TestParseInclude: + + def test_include_missing_file_rejected(self, tmp_path): + with pytest.raises(KickStartError): + _parse(tmp_path, "include does-not-exist.wks\n") + + def test_include_pulls_in_partitions(self, tmp_path): + inc = tmp_path / "inc.wks" + inc.write_text("part / --source rootfs --fstype ext4 --size 5M\n") + ks = _parse(tmp_path, "include inc.wks\n") + assert len(ks.partitions) == 1 + assert ks.partitions[0].size == 5 * 1024
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
Quoted 14 lines · Trevor Woerner via lists.yoctoproject.org
This continues the standalone unit-test work, moving up from ksparser's argparse types to the KickStart parser that drives a whole .wks file. It keeps the shape the suite settled on: the source fix is its own standalone commit, and the green test module lands last, so the suite passes at every commit.
The fix: a bootloader line with --ptable msdos and a non-integer --diskid builds its error message with "... % self.ptable". self is the KickStart instance, which has no ptable attribute, so the interpolation raises AttributeError before the intended KickStartError. The user gets an opaque traceback instead of a described error. Interpolate the offending value instead (matching the gpt branch just below), and fix the "--ptbale" typo in the same message.
Quoted 7 lines
The test module adds end-to-end coverage for
KickStart(): happy-path part/bootloader lines, degenerate files (empty, comment-only, CRLF, unknown directive), mutually-exclusive and invalid option combinations, multiple-bootloader rejection, diskid parsing for both ptables, and include handling.get_bitbake_var()is neutralised so the parser runs with no BitBake and no host tools. Backing the fix out turns the two msdos --diskid cases red.
Both patches LGTM.
Best regards,
Signature · 1 line
Paul Barker
Quoted 24 lines · Paul Barker via lists.yoctoproject.org
Quoted 14 lines · Trevor Woerner via lists.yoctoproject.org
This continues the standalone unit-test work, moving up from ksparser's argparse types to the KickStart parser that drives a whole .wks file. It keeps the shape the suite settled on: the source fix is its own standalone commit, and the green test module lands last, so the suite passes at every commit.
The fix: a bootloader line with --ptable msdos and a non-integer --diskid builds its error message with "... % self.ptable". self is the KickStart instance, which has no ptable attribute, so the interpolation raises AttributeError before the intended KickStartError. The user gets an opaque traceback instead of a described error. Interpolate the offending value instead (matching the gpt branch just below), and fix the "--ptbale" typo in the same message.
Quoted 7 lines
The test module adds end-to-end coverage for
KickStart(): happy-path part/bootloader lines, degenerate files (empty, comment-only, CRLF, unknown directive), mutually-exclusive and invalid option combinations, multiple-bootloader rejection, diskid parsing for both ptables, and include handling.get_bitbake_var()is neutralised so the parser runs with no BitBake and no host tools. Backing the fix out turns the two msdos --diskid cases red.
Quoted 3 lines
Both patches LGTM.
Reviewed-by Paul Barker <[email protected]>
Thanks Paul!
Applied to wic, master branch.
Quoted 16 lines
Best regards,
Signature · 11 lines
Paul Barker -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#4515): https://lists.yoctoproject.org/g/yocto-patches/message/4515 Mute This Topic: https://lists.yoctoproject.org/mt/120320310/900817 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/yocto-patches/leave/13168745/900817/63955952/xyzzy [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-