Introduce new acpi/smbios qtests using biosbits
cover Introduce new acpi/smbios qtests using biosbits
Biosbits is a software written by Josh Triplett that can be downloaded by visiting https://biosbits.org/. The github codebase can be found here:
https://github.com/biosbits/bits/tree/master. It is a software that exercizes the bios components such as acpi and smbios tables directly through acpica bios interpreter (a freely available C based library written by Intel, downloadable from https://acpica.org/ and is included with biosbits) without an operating system getting involved in between.
There are several advantages to directly testing the bios in a real physical machine or VM as opposed to indirectly discovering bios issues through the operating system. For one thing, the OSes tend to hide bios problems from the end user. The other is that we have more control of what we wanted to test and how by directly using acpica interpreter on top of the bios on a running system. More details on the inspiration for developing biosbits and its real life uses can be found in (a) and (b).
This patchset contains QEMU qtests written in python that exercizes the QEMU bios components using biosbits and reports test failures.
Details of each of the files added by this patchset are provided in the README file which is part of Patch 11. Every effort to contact Josh, through various means including email, twitter, linkedIn etc has failed. Hence, the changes to build biosbits with the newer compiler, upgrade acpica and other changes are currently maintained in a forked project in my personal github. We may want to maintain bits in a separate fork in a stable repository that is accessible by QEMU developers.
The newly introduced qtest currently only run for x86_64 platform. They pass both when running make check on a baremetal box as well as from inside a vm.
Thanks to Igor M for pointing me to this work.
(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf
(b) https://www.youtube.com/watch?v=36QIepyUuhg
Ani Sinha (12):
qtest: meson.build changes required to integrate python based qtests acpi/tests/bits: add prebuilt bios bits zip archive acpi/tests/bits: add prebuilt bits generated grub modules and scripts acpi/tests/bits: initial commit of test scripts that are run by
biosbits
acpi/tests/bits: disable acpi PSS tests that are failing in biosbits acpi/tests/bits: add smilatency test suite from bits in order to
disable it
acpi/tests/bits: disable smilatency test since it does not pass
everytime
acpi/tests/bits: add biosbits config file for running bios tests acpi/tests/bits: add acpi and smbios python tests that uses biosbits acpi/tests/bits: add acpi bits qtest directory in meson for running
tests
acpi/tests/bits: add README file for bits qtests
MAINTAINERS: add myself as the maintainer for acpi biosbits qtests
MAINTAINERS | 5 + tests/qtest/acpi-bits/README | 168 ++ tests/qtest/acpi-bits/acpi-bits-test-venv.sh | 59 + tests/qtest/acpi-bits/acpi-bits-test.py | 327 +++ .../qtest/acpi-bits/bits-config/bits-cfg.txt | 18 + tests/qtest/acpi-bits/bits-config/meson.build | 11 + tests/qtest/acpi-bits/bits-tests/meson.build | 11 + tests/qtest/acpi-bits/bits-tests/smbios.py | 2430 +++++++++++++++++ .../qtest/acpi-bits/bits-tests/smilatency.py | 103 + tests/qtest/acpi-bits/bits-tests/testacpi.py | 283 ++ tests/qtest/acpi-bits/bits-tests/testcpuid.py | 83 + tests/qtest/acpi-bits/meson.build | 39 + .../acpi-bits/prebuilt/bits-2095-grub.tar.gz | Bin 0 -> 41416278 bytes tests/qtest/acpi-bits/prebuilt/bits-2095.zip | Bin 0 -> 31922898 bytes tests/qtest/acpi-bits/prebuilt/meson.build | 11 + tests/qtest/acpi-bits/requirements.txt | 1 + tests/qtest/meson.build | 7 +- 17 files changed, 3555 insertions(+), 1 deletion(-) create mode 100644 tests/qtest/acpi-bits/README create mode 100644 tests/qtest/acpi-bits/acpi-bits-test-venv.sh create mode 100644 tests/qtest/acpi-bits/acpi-bits-test.py create mode 100644 tests/qtest/acpi-bits/bits-config/bits-cfg.txt create mode 100644 tests/qtest/acpi-bits/bits-config/meson.build create mode 100644 tests/qtest/acpi-bits/bits-tests/meson.build create mode 100644 tests/qtest/acpi-bits/bits-tests/smbios.py create mode 100644 tests/qtest/acpi-bits/bits-tests/smilatency.py create mode 100644 tests/qtest/acpi-bits/bits-tests/testacpi.py create mode 100644 tests/qtest/acpi-bits/bits-tests/testcpuid.py create mode 100644 tests/qtest/acpi-bits/meson.build create mode 100644 tests/qtest/acpi-bits/prebuilt/bits-2095-grub.tar.gz create mode 100644 tests/qtest/acpi-bits/prebuilt/bits-2095.zip create mode 100644 tests/qtest/acpi-bits/prebuilt/meson.build create mode 100644 tests/qtest/acpi-bits/requirements.txt
Signature · 1 line
2.25.1
PATCH 1/12 qtest: meson.build changes required to integrate python based qtests
These are some basic changes required in meson.build file in order to incorporate python based qtests later on. No new qtests have been added in this change.
--- tests/qtest/meson.build | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
tests/qtest/meson.build +4 -1
@@ -310,6 +310,8 @@ qtests += {'dbus-display-test': [dbus_display1, gio]} endif qtest_executables = {} +other_deps = [] + foreach dir : target_dirs if not dir.endswith('-softmmu') continue
@@ -327,6 +329,7 @@ foreach dir : target_dirs endif qtest_env.set('G_TEST_DBUS_DAEMON', meson.project_source_root() / 'tests/dbus-vmstate-daemon.sh') qtest_env.set('QTEST_QEMU_BINARY', './qemu-system-' + target_base) + qtest_env.set('QTEST_SOURCE_ROOT', meson.project_source_root()) if have_tools and have_vhost_user_blk_server qtest_env.set('QTEST_QEMU_STORAGE_DAEMON_BINARY', './storage-daemon/qemu-storage-daemon') test_deps += [qsd]
@@ -351,7 +354,7 @@ foreach dir : target_dirs endif test('qtest-@0@/@1@'.format(target_base, test), qtest_executables[test], - depends: [test_deps, qtest_emulator, emulator_modules], + depends: [test_deps, qtest_emulator, emulator_modules, other_deps], env: qtest_env, args: ['--tap', '-k'], protocol: 'tap',
Signature · 1 line
2.25.1
PATCH 4/12 acpi/tests/bits: initial commit of test scripts that are run by biosbits
In reply to Ani SinhaThis is initial commit of cpuid, acpi and smbios python test scripts for biosbits to execute. No change has been made to them from the original code written by the biosbits author Josh Triplett. They are required to be installed into the bits iso file and then run from within the virtual machine booted off with biosbits iso.
The original location of these tests are here:
https://github.com/biosbits/bits/blob/master/python/testacpi.py
https://github.com/biosbits/bits/blob/master/python/smbios.py
https://github.com/biosbits/bits/blob/master/python/testcpuid.py
--- tests/qtest/acpi-bits/bits-tests/meson.build | 11 + tests/qtest/acpi-bits/bits-tests/smbios.py | 2430 +++++++++++++++++ tests/qtest/acpi-bits/bits-tests/testacpi.py | 283 ++ tests/qtest/acpi-bits/bits-tests/testcpuid.py | 83 + 4 files changed, 2807 insertions(+) create mode 100644 tests/qtest/acpi-bits/bits-tests/meson.build create mode 100644 tests/qtest/acpi-bits/bits-tests/smbios.py create mode 100644 tests/qtest/acpi-bits/bits-tests/testacpi.py create mode 100644 tests/qtest/acpi-bits/bits-tests/testcpuid.py
tests/qtest/acpi-bits/bits-tests/meson.build new file +11 -0
@@ -0,0 +1,11 @@ +test_files = ['smbios.py', 'testacpi.py', 'testcpuid.py'] + +copytestfiles = custom_target('copy test files', + input : test_files, + output : test_files, + command : ['cp', '@INPUT@', '@OUTDIR@'], + install : true, + install_dir : 'bits-tests', + build_by_default : true) + +other_deps += copytestfiles
tests/qtest/acpi-bits/bits-tests/smbios.py new file +2430 -0
2,430 lines not on this page · raw message
tests/qtest/acpi-bits/bits-tests/testacpi.py new file +283 -0
283 lines not on this page · raw message
tests/qtest/acpi-bits/bits-tests/testcpuid.py new file +83 -0
83 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [PATCH 04/12] acpi/tests/bits: initial commit of test scripts that are run by biosbits
Quoted 2,786 lines · Ani Sinha
This is initial commit of cpuid, acpi and smbios python test scripts for biosbits to execute. No change has been made to them from the original code written by the biosbits author Josh Triplett. They are required to be installed into the bits iso file and then run from within the virtual machine booted off with biosbits iso.
The original location of these tests are here:
https://github.com/biosbits/bits/blob/master/python/testacpi.py
https://github.com/biosbits/bits/blob/master/python/smbios.py
https://github.com/biosbits/bits/blob/master/python/testcpuid.pySigned-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/bits-tests/meson.build | 11 + tests/qtest/acpi-bits/bits-tests/smbios.py | 2430 +++++++++++++++++ tests/qtest/acpi-bits/bits-tests/testacpi.py | 283 ++ tests/qtest/acpi-bits/bits-tests/testcpuid.py | 83 + 4 files changed, 2807 insertions(+)create mode 100644 tests/qtest/acpi-bits/bits-tests/meson.build create mode 100644
tests/qtest/acpi-bits/bits-tests/smbios.pycreate mode 100644tests/qtest/acpi-bits/bits-tests/testacpi.pycreate mode 100644tests/qtest/acpi-bits/bits-tests/testcpuid.pytests/qtest/acpi-bits/bits-tests/meson.build new file +11
-0@@ -0,0 +1,11 @@ +test_files = ['smbios.py', 'testacpi.py', 'testcpuid.py'] + +copytestfiles = custom_target('copy test files', + input : test_files, + output : test_files, + command : ['cp', '@INPUT@', '@OUTDIR@'], + install : true, + install_dir : 'bits-tests', + build_by_default : true) + +other_deps += copytestfilestests/qtest/acpi-bits/bits-tests/smbios.py new file +2430
-0@@ -0,0 +1,2430 @@ +# Copyright (c) 2015, Intel Corporation +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# * Redistributions of source code must retain the above copyright notice, +# this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# * Neither the name of Intel Corporation nor the names of its contributors +# may be used to endorse or promote products derived from this software +# without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""SMBIOS/DMI module.""" + +import bits +import bitfields +import ctypes +import redirect +import struct +import uuid +import unpack +import ttypager +import sys + +class SMBIOS(unpack.Struct): + def __new__(cls): + if sys.platform == "BITS-EFI": + import efi + sm_ptr = efi.system_table.ConfigurationTableDict.get(efi.SMBIOS_TABLE_GUID) + else: + address = 0xF0000 + mem = bits.memory(0xF0000, 0x10000) + for offset in range(0, len(mem), 16): + signature = (ctypes.c_char * 4).from_address(address + offset).value + if signature == "_SM_": + entry_point_length = ctypes.c_ubyte.from_address(address + offset + 5).value + csum = sum(map(ord, mem[offset:offset + entry_point_length])) & 0xff + if csum == 0: + sm_ptr = address + offset + break + else: + return None + + if not sm_ptr: + return None + + sm = super(SMBIOS, cls).__new__(cls) + sm._header_memory = bits.memory(sm_ptr, 0x1f) + return sm + + def __init__(self): + super(SMBIOS, self).__init__() + u = unpack.Unpackable(self._header_memory) + self.add_field('header', Header(u)) + self._structure_memory = bits.memory(self.header.structure_table_address, self.header.structure_table_length) + u = unpack.Unpackable(self._structure_memory) + self.add_field('structures', unpack.unpack_all(u, _smbios_structures, self), unpack.format_each("\n\n{!r}")) + + def structure_type(self, num): + '''Dumps structure of given Type if present''' + try: + types_present = [self.structures[x].smbios_structure_type for x in range(len(self.structures))] + matrix = dict() + for index in range(len(types_present)): + if types_present.count(types_present[index]) == 1: + matrix[types_present[index]] = self.structures[index] + else: # if multiple structures of the same type, return a list of structures for the type number + if matrix.has_key(types_present[index]): + matrix[types_present[index]].append(self.structures[index]) + else: + matrix[types_present[index]] = [self.structures[index]] + return matrix[num] + except: + print "Failure: Type {} - not found".format(num) + +class Header(unpack.Struct): + def __new__(cls, u): + return super(Header, cls).__new__(cls) + + def __init__(self, u): + super(Header, self).__init__() + self.raw_data = u.unpack_rest() + u = unpack.Unpackable(self.raw_data) + self.add_field('anchor_string', u.unpack_one("4s")) + self.add_field('checksum', u.unpack_one("B")) + self.add_field('length', u.unpack_one("B")) + self.add_field('major_version', u.unpack_one("B")) + self.add_field('minor_version', u.unpack_one("B")) + self.add_field('max_structure_size', u.unpack_one("<H")) + self.add_field('entry_point_revision', u.unpack_one("B")) + self.add_field('formatted_area', u.unpack_one("5s")) + self.add_field('intermediate_anchor_string', u.unpack_one("5s")) + self.add_field('intermediate_checksum', u.unpack_one("B")) + self.add_field('structure_table_length', u.unpack_one("<H")) + self.add_field('structure_table_address', u.unpack_one("<I")) + self.add_field('number_structures', u.unpack_one("<H")) + self.add_field('bcd_revision', u.unpack_one("B")) + if not u.at_end(): + self.add_field('data', u.unpack_rest()) + +class SmbiosBaseStructure(unpack.Struct): + def __new__(cls, u, sm): + t = u.unpack_peek_one("B") + if cls.smbios_structure_type is not None and t != cls.smbios_structure_type: + return None + return super(SmbiosBaseStructure, cls).__new__(cls) + + def __init__(self, u, sm): + super(SmbiosBaseStructure, self).__init__() + self.start_offset = u.offset + length = u.unpack_peek_one("<xB") + self.raw_data = u.unpack_raw(length) + self.u = unpack.Unpackable(self.raw_data) + + self.strings_offset = u.offset + def unpack_string(): + return "".join(iter(lambda: u.unpack_one("c"), "\x00")) + strings = list(iter(unpack_string, "")) + if not strings: + u.skip(1) + + self.strings_length = u.offset - self.strings_offset + self.raw_strings = str(bits.memory(sm.header.structure_table_address + self.strings_offset, self.strings_length)) + + if len(strings): + self.strings = strings + + self.add_field('type', self.u.unpack_one("B")) + self.add_field('length', self.u.unpack_one("B")) + self.add_field('handle', self.u.unpack_one("<H")) + + def fini(self): + if not self.u.at_end(): + self.add_field('data', self.u.unpack_rest()) + del self.u + + def fmtstr(self, i): + """Format the specified index and the associated string""" + return "{} '{}'".format(i, self.getstr(i)) + + def getstr(self, i): + """Get the string associated with the given index""" + if i == 0: + return "(none)" + if not hasattr(self, "strings"): + return "(error: structure has no strings)" + if i > len(self.strings): + return "(error: string index out of range)" + return self.strings[i - 1] + +class BIOSInformation(SmbiosBaseStructure): + smbios_structure_type = 0 + + def __init__(self, u, sm): + super(BIOSInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('vendor', u.unpack_one("B"), self.fmtstr) + self.add_field('version', u.unpack_one("B"), self.fmtstr) + self.add_field('starting_address_segment', u.unpack_one("<H")) + self.add_field('release_date', u.unpack_one("B"), self.fmtstr) + self.add_field('rom_size', u.unpack_one("B")) + self.add_field('characteristics', u.unpack_one("<Q")) + minor_version_str = str(sm.header.minor_version) # 34 is .34, 4 is .4, 41 is .41; compare ASCIIbetically to compare initial digits rather than numeric value + if (sm.header.major_version, minor_version_str) >= (2,"4"): + characteristic_bytes = 2 + else: + characteristic_bytes = self.length - 0x12 + self.add_field('characteristics_extensions', [u.unpack_one("B") for b in range(characteristic_bytes)]) + if (sm.header.major_version, minor_version_str) >= (2,"4"): + self.add_field('major_release', u.unpack_one("B")) + self.add_field('minor_release', u.unpack_one("B")) + self.add_field('ec_major_release', u.unpack_one("B")) + self.add_field('ec_minor_release', u.unpack_one("B")) + except: + self.decode_failure = True + print "Error parsing BIOSInformation" + import traceback + traceback.print_exc() + self.fini() + +class SystemInformation(SmbiosBaseStructure): + smbios_structure_type = 1 + + def __init__(self, u, sm): + super(SystemInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr) + self.add_field('product_name', u.unpack_one("B"), self.fmtstr) + self.add_field('version', u.unpack_one("B"), self.fmtstr) + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0x8: + self.add_field('uuid', uuid.UUID(bytes_le=u.unpack_one("16s"))) + wakeup_types = { + 0: 'Reserved', + 1: 'Other', + 2: 'Unknown', + 3: 'APM Timer', + 4: 'Modem Ring', + 5: 'LAN Remote', + 6: 'Power Switch', + 7: 'PCI PME#', + 8: 'AC Power Restored' + } + self.add_field('wakeup_type', u.unpack_one("B"), unpack.format_table("{}", wakeup_types)) + if self.length > 0x19: + self.add_field('sku_number', u.unpack_one("B"), self.fmtstr) + self.add_field('family', u.unpack_one("B"), self.fmtstr) + except: + self.decode_failure = True + print "Error parsing SystemInformation" + import traceback + traceback.print_exc() + self.fini() + +_board_types = { + 1: 'Unknown', + 2: 'Other', + 3: 'Server Blade', + 4: 'Connectivity Switch', + 5: 'System Management Module', + 6: 'Processor Module', + 7: 'I/O Module', + 8: 'Memory Module', + 9: 'Daughter Board', + 0xA: 'Motherboard', + 0xB: 'Processor/Memory Module', + 0xC: 'Processor/IO Module', + 0xD: 'Interconnect Board' +} + +class BaseboardInformation(SmbiosBaseStructure): + smbios_structure_type = 2 + + def __init__(self, u, sm): + super(BaseboardInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr) + self.add_field('product', u.unpack_one("B"), self.fmtstr) + self.add_field('version', u.unpack_one("B"), self.fmtstr) + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + + if self.length > 0x8: + self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr) + + if self.length > 0x9: + self.add_field('feature_flags', u.unpack_one("B")) + self.add_field('hosting_board', bool(bitfields.getbits(self.feature_flags, 0)), "feature_flags[0]={}") + self.add_field('requires_daughter_card', bool(bitfields.getbits(self.feature_flags, 1)), "feature_flags[1]={}") + self.add_field('removable', bool(bitfields.getbits(self.feature_flags, 2)), "feature_flags[2]={}") + self.add_field('replaceable', bool(bitfields.getbits(self.feature_flags, 3)), "feature_flags[3]={}") + self.add_field('hot_swappable', bool(bitfields.getbits(self.feature_flags, 4)), "feature_flags[4]={}") + + if self.length > 0xA: + self.add_field('location', u.unpack_one("B"), self.fmtstr) + + if self.length > 0xB: + self.add_field('chassis_handle', u.unpack_one("<H")) + + if self.length > 0xD: + self.add_field('board_type', u.unpack_one("B"), unpack.format_table("{}", _board_types)) + + if self.length > 0xE: + self.add_field('handle_count', u.unpack_one("B")) + if self.handle_count > 0: + self.add_field('contained_object_handles', tuple(u.unpack_one("<H") for i in range(self.handle_count))) + except: + self.decode_failure = True + print "Error parsing BaseboardInformation" + import traceback + traceback.print_exc() + self.fini() + +class SystemEnclosure(SmbiosBaseStructure): + smbios_structure_type = 3 + + def __init__(self, u, sm): + super(SystemEnclosure, self).__init__(u, sm) + u = self.u + try: + self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr) + self.add_field('enumerated_type', u.unpack_one("B")) + self.add_field('chassis_lock_present', bool(bitfields.getbits(self.enumerated_type, 7)), "enumerated_type[7]={}") + board_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Desktop', + 0x04: 'Low Profile Desktop', + 0x05: 'Pizza Box', + 0x06: 'Mini Tower', + 0x07: 'Tower', + 0x08: 'Portable', + 0x09: 'Laptop', + 0x0A: 'Notebook', + 0x0B: 'Hand Held', + 0x0C: 'Docking Station', + 0x0D: 'All in One', + 0x0E: 'Sub Notebook', + 0x0F: 'Space-saving', + 0x10: 'Lunch Box', + 0x11: 'Main Server Chassis', + 0x12: 'Expansion Chassis', + 0x13: 'SubChassis', + 0x14: 'Bus Expansion Chassis', + 0x15: 'Peripheral Chassis', + 0x16: 'RAID Chassis', + 0x17: 'Rack Mount Chassis', + 0x18: 'Sealed-case PC', + 0x19: 'Multi-system chassis W', + 0x1A: 'Compact PCI', + 0x1B: 'Advanced TCA', + 0x1C: 'Blade', + 0x1D: 'Blade Enclosure', + } + self.add_field('system_enclosure_type', bitfields.getbits(self.enumerated_type, 6, 0), unpack.format_table("enumerated_type[6:0]={}", board_types)) + self.add_field('version', u.unpack_one("B"), self.fmtstr) + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr) + minor_version_str = str(sm.header.minor_version) # 34 is .34, 4 is .4, 41 is .41; compare ASCIIbetically to compare initial digits rather than numeric value + if self.length > 9: + chassis_states = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Safe', + 0x04: 'Warning', + 0x05: 'Critical', + 0x06: 'Non-recoverable', + } + self.add_field('bootup_state', u.unpack_one("B"), unpack.format_table("{}", chassis_states)) + self.add_field('power_supply_state', u.unpack_one("B"), unpack.format_table("{}", chassis_states)) + self.add_field('thermal_state', u.unpack_one("B"), unpack.format_table("{}", chassis_states)) + security_states = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'None', + 0x04: 'External interface locked out', + 0x05: 'External interface enabled', + } + self.add_field('security_status', u.unpack_one("B"), unpack.format_table("{}", security_states)) + if self.length > 0xd: + self.add_field('oem_defined', u.unpack_one("<I")) + if self.length > 0x11: + self.add_field('height', u.unpack_one("B")) + self.add_field('num_power_cords', u.unpack_one("B")) + self.add_field('contained_element_count', u.unpack_one("B")) + self.add_field('contained_element_length', u.unpack_one("B")) + if getattr(self, 'contained_element_count', 0): + self.add_field('contained_elements', tuple(SystemEnclosureContainedElement(u, self.contained_element_length) for i in range(self.contained_element_count))) + if self.length > (0x15 + (getattr(self, 'contained_element_count', 0) * getattr(self, 'contained_element_length', 0))): + self.add_field('sku_number', u.unpack_one("B"), self.fmtstr) + except: + self.decode_failure = True + print "Error parsing SystemEnclosure" + import traceback + traceback.print_exc() + self.fini() + +class SystemEnclosureContainedElement(unpack.Struct): + def __init__(self, u, length): + super(SystemEnclosureContainedElement, self).__init__() + self.start_offset = u.offset + self.raw_data = u.unpack_raw(length) + self.u = unpack.Unpackable(self.raw_data) + u = self.u + self.add_field('contained_element_type', u.unpack_one("B")) + type_selections = { + 0: 'SMBIOS baseboard type enumeration', + 1: 'SMBIOS structure type enumeration', + } + self.add_field('type_select', bitfields.getbits(self.contained_element_type, 7), unpack.format_table("contained_element_type[7]={}", type_selections)) + self.add_field('type', bitfields.getbits(self.contained_element_type, 6, 0)) + if self.type_select == 0: + self.add_field('smbios_board_type', self.type, unpack.format_table("{}", _board_types)) + else: + self.add_field('smbios_structure_type', self.type) + self.add_field('minimum', u.unpack_one("B")) + self.add_field('maximum', u.unpack_one("B")) + if not u.at_end(): + self.add_field('data', u.unpack_rest()) + del self.u + +class ProcessorInformation(SmbiosBaseStructure): + smbios_structure_type = 4 + + def __init__(self, u, sm): + super(ProcessorInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('socket_designation', u.unpack_one("B"), self.fmtstr) + processor_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Central Processor', + 0x04: 'Math Processor', + 0x05: 'DSP Processor', + 0x06: 'Video Processor', + } + self.add_field('processor_type', u.unpack_one("B"), unpack.format_table("{}", processor_types)) + self.add_field('processor_family', u.unpack_one("B")) + self.add_field('processor_manufacturer', u.unpack_one("B"), self.fmtstr) + self.add_field('processor_id', u.unpack_one("<Q")) + self.add_field('processor_version', u.unpack_one("B"), self.fmtstr) + self.add_field('voltage', u.unpack_one("B")) + self.add_field('external_clock', u.unpack_one("<H")) + self.add_field('max_speed', u.unpack_one("<H")) + self.add_field('current_speed', u.unpack_one("<H")) + self.add_field('status', u.unpack_one("B")) + processor_upgrades = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Daughter Board', + 0x04: 'ZIF Socket', + 0x05: 'Replaceable Piggy Back', + 0x06: 'None', + 0x07: 'LIF Socket', + 0x08: 'Slot 1', + 0x09: 'Slot 2', + 0x0A: '370-pin socket', + 0x0B: 'Slot A', + 0x0C: 'Slot M', + 0x0D: 'Socket 423', + 0x0E: 'Socket A (Socket 462)', + 0x0F: 'Socket 478', + 0x10: 'Socket 754', + 0x11: 'Socket 940', + 0x12: 'Socket 939', + 0x13: 'Socket mPGA604', + 0x14: 'Socket LGA771', + 0x15: 'Socket LGA775', + 0x16: 'Socket S1', + 0x17: 'Socket AM2', + 0x18: 'Socket F (1207)', + 0x19: 'Socket LGA1366', + 0x1A: 'Socket G34', + 0x1B: 'Socket AM3', + 0x1C: 'Socket C32', + 0x1D: 'Socket LGA1156', + 0x1E: 'Socket LGA1567', + 0x1F: 'Socket PGA988A', + 0x20: 'Socket BGA1288', + 0x21: 'Socket rPGA988B', + 0x22: 'Socket BGA1023', + 0x23: 'Socket BGA1224', + 0x24: 'Socket BGA1155', + 0x25: 'Socket LGA1356', + 0x26: 'Socket LGA2011', + 0x27: 'Socket FS1', + 0x28: 'Socket FS2', + 0x29: 'Socket FM1', + 0x2A: 'Socket FM2', + } + self.add_field('processor_upgrade', u.unpack_one("B"), unpack.format_table("{}", processor_upgrades)) + if self.length > 0x1A: + self.add_field('l1_cache_handle', u.unpack_one("<H")) + self.add_field('l2_cache_handle', u.unpack_one("<H")) + self.add_field('l3_cache_handle', u.unpack_one("<H")) + if self.length > 0x20: + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr) + self.add_field('part_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0x24: + self.add_field('core_count', u.unpack_one("B")) + self.add_field('core_enabled', u.unpack_one("B")) + self.add_field('thread_count', u.unpack_one("B")) + self.add_field('processor_characteristics', u.unpack_one("<H")) + if self.length > 0x28: + self.add_field('processor_family_2', u.unpack_one("<H")) + if self.length > 0x2A: + self.add_field('core_count2', u.unpack_one("<H")) + self.add_field('core_enabled2', u.unpack_one("<H")) + self.add_field('thread_count2', u.unpack_one("<H")) + except: + self.decode_failure = True + print "Error parsing Processor Information" + import traceback + traceback.print_exc() + self.fini() + +class MemoryControllerInformation(SmbiosBaseStructure): #obsolete starting with v2.1 + smbios_structure_type = 5 + + def __init__(self, u, sm): + super(MemoryControllerInformation, self).__init__(u, sm) + u = self.u + try: + _error_detecting_method = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'None', + 0x04: '8-bit Parity', + 0x05: '32-bit ECC', + 0x06: '64-bit ECC', + 0x07: '128-bit ECC', + 0x08: 'CRC' + } + self.add_field('error_detecting_method', u.unpack_one("B"), unpack.format_table("{}", _error_detecting_method)) + self.add_field('error_correcting_capability', u.unpack_one("B")) + _interleaves = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'One-Way Interleave', + 0x04: 'Two-Way Interleave', + 0x05: 'Four-Way Interleave', + 0x06: 'Eight-Way Interleave', + 0x07: 'Sixteen-Way Interleave' + } + self.add_field('supported_interleave', u.unpack_one("B"), unpack.format_table("{}", _interleaves)) + self.add_field('current_interleave', u.unpack_one("B"), unpack.format_table("{}", _interleaves)) + self.add_field('max_memory_module_size', u.unpack_one("B"), self.fmtstr) + self.add_field('supported_speeds', u.unpack_one("<H")) + self.add_field('supported_memory_types', u.unpack_one("<H")) + self.add_field('memory_module_voltage', u.unpack_one("B")) + self.add_field('req_voltage_b2', bitfields.getbits(self.memory_module_voltage, 2), "memory_module_voltage[2]={}") + self.add_field('req_voltage_b1', bitfields.getbits(self.memory_module_voltage, 1), "memory_module_voltage[1]={}") + self.add_field('req_voltage_b0', bitfields.getbits(self.memory_module_voltage, 0), "memory_module_voltage[0]={}") + self.add_field('num_associated_memory_slots', u.unpack_one("B")) + self.add_field('memory_module_configuration_handles', u.unpack_one("<(self.num_associated_memory_slots)H")) + self.add_field('enabled_error_correcting_capabilities', u.unpack_one("B")) + except: + self.decode_failure = True + print "Error parsing MemoryControllerInformation" + import traceback + traceback.print_exc() + self.fini() + +class MemoryModuleInformation(SmbiosBaseStructure): #obsolete starting with v2.1 + smbios_structure_type = 6 + + def __init__(self, u, sm): + super(MemoryModuleInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('socket_designation', u.unpack_one("B"), self.fmtstr) + self.add_field('bank_connections', u.unpack_one("B")) + self.add_field('current_speed', u.unpack_one("B")) + self.add_field('current_memory_type', u.unpack_one("<H")) + _mem_connection = { + 0: 'single', + 1: 'double-bank' + } + self.add_field('installed_mem', u.unpack_one("B")) + self.add_field('installed_size', bitfields.getbits(self.installed_mem, 6, 0), "installed_mem[6:0]={}") + self.add_field('installed_memory_module_connection', bitfields.getbits(self.installed_mem, 7), unpack.format_table("installed_mem[7]={}", _mem_connection)) + self.add_field('enabled_mem', u.unpack_one("B")) + self.add_field('enabled_size', bitfields.getbits(self.installed_mem, 6, 0), "enabled_mem[6:0]={}") + self.add_field('enabled_memory_module_connection', bitfields.getbits(self.installed_mem, 7), unpack.format_table("enabled_mem[7]={}", _mem_connection)) + self.add_field('error_status', u.unpack_one("B")) + self.add_field('error_status_info_obstained_from_event_log', bool(bitfields.getbits(self.error_status, 2)), unpack.format_table("error_status[2]={}", _mem_connection)) + self.add_field('correctable_errors_received', bool(bitfields.getbits(self.error_status, 1)), unpack.format_table("error_status[1]={}", _mem_connection)) + self.add_field('uncorrectable_errors_received', bool(bitfields.getbits(self.error_status, 0)), unpack.format_table("error_status[0]={}", _mem_connection)) + except: + self.decode_failure = True + print "Error parsing MemoryModuleInformation" + import traceback + traceback.print_exc() + self.fini() + +class CacheInformation(SmbiosBaseStructure): + smbios_structure_type = 7 + + def __init__(self, u, sm): + super(CacheInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('socket_designation', u.unpack_one("B"), self.fmtstr) + processor_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Central Processor', + 0x04: 'Math Processor', + 0x05: 'DSP Processor', + 0x06: 'Video Processor', + } + self.add_field('cache_configuration', u.unpack_one("<H")) + _operational_mode = { + 0b00: 'Write Through', + 0b01: 'Write Back', + 0b10: 'Varies with Memory Address', + 0b11: 'Unknown' + } + self.add_field('operational_mode', bitfields.getbits(self.cache_configuration, 9, 8), unpack.format_table("cache_configuration[9:8]={}", _operational_mode)) + self.add_field('enabled_at_boot_time', bool(bitfields.getbits(self.cache_configuration, 7)), "cache_configuration[7]={}") + _location = { + 0b00: 'Internal', + 0b01: 'External', + 0b10: 'Reserved', + 0b11: 'Unknown' + } + self.add_field('location_relative_to_cpu_module', bitfields.getbits(self.cache_configuration, 6, 5), unpack.format_table("cache_configuration[6:5]={}", _location)) + self.add_field('cache_socketed', bool(bitfields.getbits(self.cache_configuration, 3)), "cache_configuration[3]={}") + self.add_field('cache_level', bitfields.getbits(self.cache_configuration, 2, 0), "cache_configuration[2:0]={}") + self.add_field('max_cache_size', u.unpack_one("<H")) + _granularity = { + 0: '1K granularity', + 1: '64K granularity' + } + self.add_field('max_granularity', bitfields.getbits(self.cache_configuration, 15), unpack.format_table("max_cache_size[15]={}", _granularity)) + self.add_field('max_size_in_granularity', bitfields.getbits(self.cache_configuration, 14, 0), "max_cache_size[14, 0]={}") + self.add_field('installed_size', u.unpack_one("<H")) + if self.installed_size != 0: + self.add_field('installed_granularity', bitfields.getbits(self.cache_configuration, 15), unpack.format_table("installed_size[15]={}", _granularity)) + self.add_field('installed_size_in_granularity', bitfields.getbits(self.cache_configuration, 14, 0), "installed_size[14, 0]={}") + self.add_field('supported_sram_type', u.unpack_one("<H")) + self.add_field('current_sram_type', u.unpack_one("<H")) + if self.length > 0x0F: + self.add_field('cache_speed', u.unpack_one("B")) + if self.length > 0x10: + _error_correction = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'None', + 0x04: 'Parity', + 0x05: 'Single-bit ECC', + 0x06: 'Multi-bit ECC' + } + self.add_field('error_correction', u.unpack_one("B"), unpack.format_table("{}", _error_correction)) + if self.length > 0x10: + _system_cache_type = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Instruction', + 0x04: 'Data', + 0x05: 'Unified' + } + self.add_field('system_cache_type', u.unpack_one("B"), unpack.format_table("{}", _system_cache_type)) + if self.length > 0x12: + _associativity = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Direct Mapped', + 0x04: '2-way Set-Associative', + 0x05: '4-way Set-Associative', + 0x06: 'Fully Associative', + 0x07: '8-way Set-Associative', + 0x08: '16-way Set-Associative', + 0x09: '12-way Set-Associative', + 0x0A: '24-way Set-Associative', + 0x0B: '32-way Set-Associative', + 0x0C: '48-way Set-Associative', + 0x0D: '64-way Set-Associative', + 0x0E: '20-way Set-Associative' + } + self.add_field('associativity', u.unpack_one("B"), unpack.format_table("{}", _associativity)) + + except: + self.decode_failure = True + print "Error parsing CacheInformation" + import traceback + traceback.print_exc() + self.fini() + +class PortConnectorInfo(SmbiosBaseStructure): + smbios_structure_type = 8 + + def __init__(self, u, sm): + super(PortConnectorInfo, self).__init__(u, sm) + u = self.u + try: + self.add_field('internal_reference_designator', u.unpack_one("B"), self.fmtstr) + connector_types = { + 0x00: 'None', + 0x01: 'Centronics', + 0x02: 'Mini Centronics', + 0x03: 'Proprietary', + 0x04: 'DB-25 pin male', + 0x05: 'DB-25 pin female', + 0x06: 'DB-15 pin male', + 0x07: 'DB-15 pin female', + 0x08: 'DB-9 pin male', + 0x09: 'DB-9 pin female', + 0x0A: 'RJ-11', + 0x0B: 'RJ-45', + 0x0C: '50-pin MiniSCSI', + 0x0D: 'Mini-DIN', + 0x0E: 'Micro-DIN', + 0x0F: 'PS/2', + 0x10: 'Infrared', + 0x11: 'HP-HIL', + 0x12: 'Access Bus (USB)', + 0x13: 'SSA SCSI', + 0x14: 'Circular DIN-8 male', + 0x15: 'Circular DIN-8 female', + 0x16: 'On Board IDE', + 0x17: 'On Board Floppy', + 0x18: '9-pin Dual Inline (pin 10 cut)', + 0x19: '25-pin Dual Inline (pin 26 cut)', + 0x1A: '50-pin Dual Inline', + 0x1B: '68-pin Dual Inline', + 0x1C: 'On Board Sound Input from CD-ROM', + 0x1D: 'Mini-Centronics Type-14', + 0x1E: 'Mini-Centronics Type-26', + 0x1F: 'Mini-jack (headphones)', + 0x20: 'BNC', + 0x21: '1394', + 0x22: 'SAS/SATA Plug Receptacle', + 0xA0: 'PC-98', + 0xA1: 'PC-98Hireso', + 0xA2: 'PC-H98', + 0xA3: 'PC-98Note', + 0xA4: 'PC-98Full', + 0xFF: 'Other', + } + self.add_field('internal_connector_type', u.unpack_one("B"), unpack.format_table("{}", connector_types)) + self.add_field('external_reference_designator', u.unpack_one("B"), self.fmtstr) + self.add_field('external_connector_type', u.unpack_one("B"), unpack.format_table("{}", connector_types)) + port_types = { + 0x00: 'None', + 0x01: 'Parallel Port XT/AT Compatible', + 0x02: 'Parallel Port PS/2', + 0x03: 'Parallel Port ECP', + 0x04: 'Parallel Port EPP', + 0x05: 'Parallel Port ECP/EPP', + 0x06: 'Serial Port XT/AT Compatible', + 0x07: 'Serial Port 16450 Compatible', + 0x08: 'Serial Port 16550 Compatible', + 0x09: 'Serial Port 16550A Compatible', + 0x0A: 'SCSI Port', + 0x0B: 'MIDI Port', + 0x0C: 'Joy Stick Port', + 0x0D: 'Keyboard Port', + 0x0E: 'Mouse Port', + 0x0F: 'SSA SCSI', + 0x10: 'USB', + 0x11: 'FireWire (IEEE P1394)', + 0x12: 'PCMCIA Type I2', + 0x13: 'PCMCIA Type II', + 0x14: 'PCMCIA Type III', + 0x15: 'Cardbus', + 0x16: 'Access Bus Port', + 0x17: 'SCSI II', + 0x18: 'SCSI Wide', + 0x19: 'PC-98', + 0x1A: 'PC-98-Hireso', + 0x1B: 'PC-H98', + 0x1C: 'Video Port', + 0x1D: 'Audio Port', + 0x1E: 'Modem Port', + 0x1F: 'Network Port', + 0x20: 'SATA', + 0x21: 'SAS', + 0xA0: '8251 Compatible', + 0xA1: '8251 FIFO Compatible', + 0xFF: 'Other', + } + self.add_field('port_type', u.unpack_one("B"), unpack.format_table("{}", port_types)) + except: + self.decodeFailure = True + print "Error parsing PortConnectorInfo" + import traceback + traceback.print_exc() + self.fini() + +class SystemSlots(SmbiosBaseStructure): + smbios_structure_type = 9 + + def __init__(self, u, sm): + super(SystemSlots, self).__init__(u, sm) + u = self.u + try: + self.add_field('designation', u.unpack_one("B"), self.fmtstr) + _slot_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'ISA', + 0x04: 'MCA', + 0x05: 'EISA', + 0x06: 'PCI', + 0x07: 'PC Card (PCMCIA)', + 0x08: 'VL-VESA', + 0x09: 'Proprietary', + 0x0A: 'Processor Card Slot', + 0x0B: 'Proprietary Memory Card Slot', + 0x0C: 'I/O Riser Card Slot', + 0x0D: 'NuBus', + 0x0E: 'PCI 66MHz Capable', + 0x0F: 'AGP', + 0x10: 'AGP 2X', + 0x11: 'AGP 4X', + 0x12: 'PCI-X', + 0x13: 'AGP 8X', + 0xA0: 'PC-98/C20', + 0xA1: 'PC-98/C24', + 0xA2: 'PC-98/E', + 0xA3: 'PC-98/Local Bus', + 0xA4: 'PC-98/Card', + 0xA5: 'PCI Express', + 0xA6: 'PCI Express x1', + 0xA7: 'PCI Express x2', + 0xA8: 'PCI Express x4', + 0xA9: 'PCI Express x8', + 0xAA: 'PCI Express x16', + 0xAB: 'PCI Express Gen 2', + 0xAC: 'PCI Express Gen 2 x1', + 0xAD: 'PCI Express Gen 2 x2', + 0xAE: 'PCI Express Gen 2 x4', + 0xAF: 'PCI Express Gen 2 x8', + 0xB0: 'PCI Express Gen 2 x16', + 0xB1: 'PCI Express Gen 3', + 0xB2: 'PCI Express Gen 3 x1', + 0xB3: 'PCI Express Gen 3 x2', + 0xB4: 'PCI Express Gen 3 x4', + 0xB5: 'PCI Express Gen 3 x8', + 0xB6: 'PCI Express Gen 3 x16', + } + self.add_field('slot_type', u.unpack_one("B"), unpack.format_table("{}", _slot_types)) + _slot_data_bus_widths = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: '8 bit', + 0x04: '16 bit', + 0x05: '32 bit', + 0x06: '64 bit', + 0x07: '128 bit', + 0x08: '1x or x1', + 0x09: '2x or x2', + 0x0A: '4x or x4', + 0x0B: '8x or x8', + 0x0C: '12x or x12', + 0x0D: '16x or x16', + 0x0E: '32x or x32', + } + self.add_field('slot_data_bus_width', u.unpack_one('B'), unpack.format_table("{}", _slot_data_bus_widths)) + _current_usages = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Available', + 0x04: 'In use', + } + self.add_field('current_usage', u.unpack_one('B'), unpack.format_table("{}", _current_usages)) + _slot_lengths = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Short Length', + 0x04: 'Long Length', + } + self.add_field('slot_length', u.unpack_one('B'), unpack.format_table("{}", _slot_lengths)) + self.add_field('slot_id', u.unpack_one('<H')) + self.add_field('characteristics1', u.unpack_one('B')) + self.add_field('characteristics_unknown', bool(bitfields.getbits(self.characteristics1, 0)), "characteristics1[0]={}") + self.add_field('provides_5_0_volts', bool(bitfields.getbits(self.characteristics1, 1)), "characteristics1[1]={}") + self.add_field('provides_3_3_volts', bool(bitfields.getbits(self.characteristics1, 2)), "characteristics1[2]={}") + self.add_field('shared_slot', bool(bitfields.getbits(self.characteristics1, 3)), "characteristics1[3]={}") + self.add_field('supports_pc_card_16', bool(bitfields.getbits(self.characteristics1, 4)), "characteristics1[4]={}") + self.add_field('supports_cardbus', bool(bitfields.getbits(self.characteristics1, 5)), "characteristics1[5]={}") + self.add_field('supports_zoom_video', bool(bitfields.getbits(self.characteristics1, 6)), "characteristics1[6]={}") + self.add_field('supports_modem_ring_resume', bool(bitfields.getbits(self.characteristics1, 7)), "characteristics1[7]={}") + if self.length > 0x0C: + self.add_field('characteristics2', u.unpack_one('B')) + self.add_field('supports_PME', bool(bitfields.getbits(self.characteristics2, 0)), "characteristics2[0]={}") + self.add_field('supports_hot_plug', bool(bitfields.getbits(self.characteristics2, 1)), "characteristics2[1]={}") + self.add_field('supports_smbus', bool(bitfields.getbits(self.characteristics2, 2)), "characteristics2[2]={}") + if self.length > 0x0D: + self.add_field('segment_group_number', u.unpack_one('<H')) + self.add_field('bus_number', u.unpack_one('B')) + self.add_field('device_function_number', u.unpack_one('B')) + self.add_field('device_number', bitfields.getbits(self.device_function_number, 7, 3), "device_function_number[7:3]={}") + self.add_field('function_number', bitfields.getbits(self.device_function_number, 2, 0), "device_function_number[2:0]={}") + except: + self.decodeFailure = True + print "Error parsing SystemSlots" + import traceback + traceback.print_exc() + self.fini() + +class OnBoardDevicesInformation(SmbiosBaseStructure): + smbios_structure_type = 10 + + def __init__(self, u, sm): + super(OnBoardDevicesInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('device_type', u.unpack_one("B")) + self.add_field('device_enabled', bool(bitfields.getbits(self.device_type, 7)), "device_type[7]={}") + _device_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Video', + 0x04: 'SCSI Controller', + 0x05: 'Ethernet', + 0x06: 'Token Ring', + 0x07: 'Sound', + 0x08: 'PATA Controller', + 0x09: 'SATA Controller', + 0x0A: 'SAS Controller' + } + self.add_field('type_of_device', bitfields.getbits(self.device_type, 6, 0), unpack.format_table("device_type[6:0]={}", _device_types)) + self.add_field('description_string', u.unpack_one("B"), self.fmtstr) + except: + self.decodeFailure = True + print "Error parsing OnBoardDevicesInformation" + import traceback + traceback.print_exc() + self.fini() + +class OEMStrings(SmbiosBaseStructure): + smbios_structure_type = 11 + + def __init__(self, u, sm): + super(OEMStrings, self).__init__(u, sm) + u = self.u + try: + self.add_field('count', u.unpack_one("B")) + except: + self.decodeFailure = True + print "Error parsing OEMStrings" + import traceback + traceback.print_exc() + self.fini() + +class SystemConfigOptions(SmbiosBaseStructure): + smbios_structure_type = 12 + + def __init__(self, u, sm): + super(SystemConfigOptions, self).__init__(u, sm) + u = self.u + try: + self.add_field('count', u.unpack_one("B")) + except: + self.decodeFailure = True + print "Error parsing SystemConfigOptions" + import traceback + traceback.print_exc() + self.fini() + +class BIOSLanguageInformation(SmbiosBaseStructure): + smbios_structure_type = 13 + + def __init__(self, u, sm): + super(BIOSLanguageInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('installable_languages', u.unpack_one("B")) + if self.length > 0x05: + self.add_field('flags', u.unpack_one('B')) + self.add_field('abbreviated_format', bool(bitfields.getbits(self.flags, 0)), "flags[0]={}") + if self.length > 0x6: + u.skip(15) + self.add_field('current_language', u.unpack_one('B'), self.fmtstr) + except: + self.decodeFailure = True + print "Error parsing BIOSLanguageInformation" + import traceback + traceback.print_exc() + self.fini() + +class GroupAssociations(SmbiosBaseStructure): + smbios_structure_type = 14 + + def __init__(self, u, sm): + super(GroupAssociations, self).__init__(u, sm) + u = self.u + try: + self.add_field('group_name', u.unpack_one("B"), self.fmtstr) + self.add_field('item_type', u.unpack_one('B')) + self.add_field('item_handle', u.unpack_one('<H')) + except: + self.decodeFailure = True + print "Error parsing GroupAssociations" + import traceback + traceback.print_exc() + self.fini() + +class SystemEventLog(SmbiosBaseStructure): + smbios_structure_type = 15 + + def __init__(self, u, sm): + super(SystemEventLog, self).__init__(u, sm) + u = self.u + try: + self.add_field('log_area_length', u.unpack_one("<H")) + self.add_field('log_header_start_offset', u.unpack_one('<H')) + self.add_field('log_data_start_offset', u.unpack_one('<H')) + _access_method = { + 0x00: 'Indexed I/O: 1 8-bit index port, 1 8-bit data port', + 0x01: 'Indexed I/O: 2 8-bit index ports, 1 8-bit data port', + 0x02: 'Indexed I/O: 1 16-bit index port, 1 8-bit data port', + 0x03: 'Memory-mapped physical 32-bit address', + 0x04: 'Available through General-Purpose NonVolatile Data functions', + xrange(0x05, 0x07F): 'Available for future assignment', + xrange(0x80, 0xFF): 'BIOS Vendor/OEM-specific' + } + self.add_field('access_method', u.unpack_one('B'), unpack.format_table("{}", _access_method)) + self.add_field('log_status', u.unpack_one('B')) + self.add_field('log_area_full', bool(bitfields.getbits(self.log_status, 1)), "log_status[1]={}") + self.add_field('log_area_valid', bool(bitfields.getbits(self.log_status, 0)), "log_status[0]={}") + self.add_field('log_change_token', u.unpack_one('<I')) + self.add_field('access_method_address', u.unpack_one('<I')) + if self.length > 0x14: + _log_header_formats = { + 0: 'No header', + 1: 'Type 1 log header', + xrange(2, 0x7f): 'Available for future assignment', + xrange(0x80, 0xff): 'BIOS vendor or OEM-specific format' + } + self.add_field('log_header_format', u.unpack_one("B"), unpack.format_table("{}", _log_header_formats)) + if self.length > 0x15: + self.add_field('num_supported_log_type_descriptors', u.unpack_one('B')) + if self.length > 0x16: + self.add_field('length_log_type_descriptor', u.unpack_one('B')) + if self.length != (0x17 + (self.num_supported_log_type_descriptors * self.length_log_type_descriptor)): + print "Error: structure length ({}) != 0x17 + (num_supported_log_type_descriptors ({}) * length_log_type_descriptor({}))".format(self.length, self.num_supported_log_type_descriptors, self.length_log_type_descriptor) + print "structure length = {}".format(self.length) + print "num_supported_log_type_descriptors = {}".format(self.num_supported_log_type_descriptors) + print "length_log_type_descriptor = {}".format(self.length_log_type_descriptor) + self.decodeFailure = True + self.add_field('descriptors', tuple(EventLogDescriptor.unpack(u) for i in range(self.num_supported_log_type_descriptors)), unpack.format_each("\n{!r}")) + except: + self.decodeFailure = True + print "Error parsing SystemEventLog" + import traceback + traceback.print_exc() + self.fini() + +class EventLogDescriptor(unpack.Struct): + @staticmethod + def _unpack(u): + _event_log_type_descriptors = { + 0x00: 'Reserved', + 0x01: 'Single-bit ECC memory error', + 0x02: 'Multi-bit ECC memory error', + 0x03: 'Parity memory error', + 0x04: 'Bus time-out', + 0x05: 'I/O Channel Check', + 0x06: 'Software NMI', + 0x07: 'POST Memory Resize', + 0x08: 'POST Error', + 0x09: 'PCI Parity Error', + 0x0A: 'PCI System Error', + 0x0B: 'CPU Failure', + 0x0C: 'EISA FailSafe Timer time-out', + 0x0D: 'Correctable memory log disabled', + 0x0E: 'Logging disabled for a specific Event Type - too many errors of the same type received in a short amount of time', + 0x0F: 'Reserved', + 0x10: 'System Limit Exceeded', + 0x11: 'Asynchronous hardware timer expired and issued a system reset', + 0x12: 'System configuration information', + 0x13: 'Hard-disk information', + 0x14: 'System reconfigured', + 0x15: 'Uncorrectable CPU-complex error', + 0x16: 'Log Area Reset/Cleared', + 0x17: 'System boot', + xrange(0x18, 0x7F): 'Unused, available for assignment', + xrange(0x80, 0xFE): 'Availalbe for system- and OEM-specific assignments', + 0xFF: 'End of log' + } + yield 'log_type', u.unpack_one('B'), unpack.format_table("{}", _event_log_type_descriptors) + _event_log_format = { + 0x00: 'None', + 0x01: 'Handle', + 0x02: 'Multiple-Event', + 0x03: 'Multiple-Event Handle', + 0x04: 'POST Results Bitmap', + 0x05: 'System Management Type', + 0x06: 'Multiple-Event System Management Type', + xrange(0x80, 0xFF): 'OEM assigned' + } + yield 'variable_data_format_type', u.unpack_one('B'), unpack.format_table("{}", _event_log_format) + +class PhysicalMemoryArray(SmbiosBaseStructure): + smbios_structure_type = 16 + + def __init__(self, u, sm): + super(PhysicalMemoryArray, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + _location_field = { + 0x01: "Other", + 0x02: "Unknown", + 0x03: "System board or motherboard", + 0x04: "ISA add-on card", + 0x05: "EISA add-on card", + 0x06: "PCI add-on card", + 0x07: "MCA add-on card", + 0x08: "PCMCIA add-on card", + 0x09: "Proprietary add-on card", + 0x0A: "NuBus", + 0xA0: "PC-98/C20 add-on card", + 0xA1: "PC-98/C24 add-on card", + 0xA2: "PC-98/E add-on card", + 0xA3: "PC-98/Local bus add-on card" + } + self.add_field('location', u.unpack_one("B"), unpack.format_table("{}", _location_field)) + if self.length > 0x05: + _use = { + 0x01: "Other", + 0x02: "Unknown", + 0x03: "System memory", + 0x04: "Video memory", + 0x05: "Flash memory", + 0x06: "Non-volatile RAM", + 0x07: "Cache memory" + } + self.add_field('use', u.unpack_one('B'), unpack.format_table("{}", _use)) + if self.length > 0x06: + _error_correction = { + 0x01: "Other", + 0x02: "Unknown", + 0x03: "None", + 0x04: "Parity", + 0x05: "Single-bit ECC", + 0x06: "Multi-bit ECC", + 0x07: "CRC" + } + self.add_field('memory_error_correction', u.unpack_one('B'), unpack.format_table("{}", _error_correction)) + if self.length > 0x07: + self.add_field('maximum_capacity', u.unpack_one('<I')) + if self.length > 0x0B: + self.add_field('memory_error_information_handle', u.unpack_one('<H')) + if self.length > 0x0D: + self.add_field('num_memory_devices', u.unpack_one('<H')) + if self.length > 0x0F: + self.add_field('extended_maximum_capacity', u.unpack_one('<Q')) + except: + self.decodeFailure = True + print "Error parsing PhysicalMemoryArray" + import traceback + traceback.print_exc() + self.fini() + +class MemoryDevice(SmbiosBaseStructure): + smbios_structure_type = 17 + + def __init__(self, u, sm): + super(MemoryDevice, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('physical_memory_array_handle', u.unpack_one("<H")) + if self.length > 0x6: + self.add_field('memory_error_information_handle', u.unpack_one("<H")) + if self.length > 0x8: + self.add_field('total_width', u.unpack_one("<H")) + if self.length > 0xA: + self.add_field('data_width', u.unpack_one("<H")) + if self.length > 0xC: + self.add_field('size', u.unpack_one("<H")) + if self.length > 0xE: + _form_factors = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'SIMM', + 0x04: 'SIP', + 0x05: 'Chip', + 0x06: 'DIP', + 0x07: 'ZIP', + 0x08: 'Proprietary Card', + 0x09: 'DIMM', + 0x0A: 'TSOP', + 0x0B: 'Row of chips', + 0x0C: 'RIMM', + 0x0D: 'SODIMM', + 0x0E: 'SRIMM', + 0x0F: 'FB-DIMM' + } + self.add_field('form_factor', u.unpack_one("B"), unpack.format_table("{}", _form_factors)) + if self.length > 0xF: + self.add_field('device_set', u.unpack_one("B")) + if self.length > 0x10: + self.add_field('device_locator', u.unpack_one("B"), self.fmtstr) + if self.length > 0x11: + self.add_field('bank_locator', u.unpack_one("B"), self.fmtstr) + if self.length > 0x12: + _memory_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'DRAM', + 0x04: 'EDRAM', + 0x05: 'VRAM', + 0x06: 'SRAM', + 0x07: 'RAM', + 0x08: 'ROM', + 0x09: 'FLASH', + 0x0A: 'EEPROM', + 0x0B: 'FEPROM', + 0x0C: 'EPROM', + 0x0D: 'CDRAM', + 0x0E: '3DRAM', + 0x0F: 'SDRAM', + 0x10: 'SGRAM', + 0x11: 'RDRAM', + 0x12: 'DDR', + 0x13: 'DDR2', + 0x14: 'DDR2 FB-DIMM', + xrange(0x15, 0x17): 'Reserved', + 0x18: 'DDR3', + 0x19: 'FBD2' + } + self.add_field('memory_type', u.unpack_one("B"), unpack.format_table("{}", _memory_types)) + if self.length > 0x13: + self.add_field('type_detail', u.unpack_one('<H')) + if self.length > 0x15: + self.add_field('speed', u.unpack_one("<H")) + if self.length > 0x17: + self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr) + if self.length > 0x18: + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0x19: + self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr) + if self.length > 0x1A: + self.add_field('part_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0x1B: + self.add_field('attributes', u.unpack_one("B")) + self.add_field('rank', bitfields.getbits(self.attributes, 3, 0), "attributes[3:0]={}") + if self.length > 0x1C: + if self.size == 0x7FFF: + self.add_field('extended_size', u.unpack_one('<I')) + self.add_field('mem_size', bitfields.getbits(self.type_detail, 30, 0), "type_detail[30:0]={}") + else: + u.skip(4) + if self.length > 0x20: + self.add_field('configured_memory_clock_speed', u.unpack_one("<H")) + if self.length > 0x22: + self.add_field('minimum_voltage', u.unpack_one("<H")) + if self.length > 0x24: + self.add_field('maximum_voltage', u.unpack_one("<H")) + if self.length > 0x26: + self.add_field('configured_voltage', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing MemoryDevice" + import traceback + traceback.print_exc() + self.fini() + +class MemoryErrorInfo32Bit(SmbiosBaseStructure): + smbios_structure_type = 18 + + def __init__(self, u, sm): + super(MemoryErrorInfo32Bit, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + _error_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'OK', + 0x04: 'Bad read', + 0x05: 'Parity error', + 0x06: 'Single-bit error', + 0x07: 'Double-bit error', + 0x08: 'Multi-bit error', + 0x09: 'Nibble error', + 0x0A: 'Checksum error', + 0x0B: 'CRC error', + 0x0C: 'Corrected single-bit error', + 0x0D: 'Corrected error', + 0x0E: 'Uncorrectable error' + } + self.add_field('error_type', u.unpack_one("B"), unpack.format_table("{}", _error_types)) + if self.length > 0x5: + _error_granularity_field = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Device level', + 0x04: 'Memory partition level' + } + self.add_field('error_granularity', u.unpack_one("B"), unpack.format_table("{}", _error_granularity_field)) + if self.length > 0x6: + _error_operation_field = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Read', + 0x04: 'Write', + 0x05: 'Partial write' + } + self.add_field('error_operation', u.unpack_one("B"), unpack.format_table("{}", _error_operation_field)) + if self.length > 0x7: + self.add_field('vendor_syndrome', u.unpack_one("<I")) + if self.length > 0xB: + self.add_field('memory_array_error_address', u.unpack_one("<I")) + if self.length > 0xF: + self.add_field('device_error_address', u.unpack_one("<I")) + if self.length > 0x13: + self.add_field('error_resolution', u.unpack_one("<I")) + except: + self.decodeFailure = True + print "Error parsing MemoryErrorInfo32Bit" + import traceback + traceback.print_exc() + self.fini() + +class MemoryArrayMappedAddress(SmbiosBaseStructure): + smbios_structure_type = 19 + + def __init__(self, u, sm): + super(MemoryArrayMappedAddress, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('starting_address', u.unpack_one("<I")) + # if FFFF FFFF: address stored in Extended Starting Address + if self.length > 0x8: + self.add_field('ending_address', u.unpack_one("<I")) + if self.length > 0xC: + self.add_field('memory_array_handle', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('partition_width', u.unpack_one("B")) + if self.length > 0xF: + # valid if starting_address = FFFF FFFF + if self.starting_address == 0xFFFFFFFF: + self.add_field('extended_starting_address', u.unpack_one("<Q")) + if self.length > 0x17: + self.add_field('extended_ending_address', u.unpack_one("<Q")) + else: + u.skip(16) + + except: + self.decodeFailure = True + print "Error parsing MemoryArrayMappedAddress" + import traceback + traceback.print_exc() + self.fini() + +class MemoryDeviceMappedAddress(SmbiosBaseStructure): + smbios_structure_type = 20 + + def __init__(self, u, sm): + super(MemoryDeviceMappedAddress, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('starting_address', u.unpack_one("<I")) + # if FFFF FFFF: address stored in Extended Starting Address + if self.length > 0x8: + self.add_field('ending_address', u.unpack_one("<I")) + if self.length > 0xC: + self.add_field('memory_device_handle', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('memory_array_mapped_address_handle', u.unpack_one("<H")) + if self.length > 0x10: + self.add_field('partition_row_position', u.unpack_one("B")) + if self.length > 0x11: + self.add_field('interleave_position', u.unpack_one("B")) + if self.length > 0x12: + self.add_field('interleave_data_depth', u.unpack_one("B")) + if self.length > 0x13: + # valid if starting_address = FFFF FFFF + if self.starting_address == 0xFFFFFFFF: + self.add_field('extended_starting_address', u.unpack_one("<Q")) + if self.length > 0x1B: + self.add_field('extended_ending_address', u.unpack_one("<Q")) + else: + u.skip(16) + except: + self.decodeFailure = True + print "Error parsing MemoryDeviceMappedAddress" + import traceback + traceback.print_exc() + self.fini() + +class BuiltInPointingDevice(SmbiosBaseStructure): + smbios_structure_type = 21 + + def __init__(self, u, sm): + super(BuiltInPointingDevice, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + _pointing_device_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Mouse', + 0x04: 'Track Ball', + 0x05: 'Track Point', + 0x06: 'Glide Point', + 0x07: 'Touch Pad', + 0x08: 'Touch Screen', + 0x09: 'Optical Sensor' + } + self.add_field('pointing_device_type', u.unpack_one("B"), unpack.format_table("{}", _pointing_device_types)) + if self.length > 0x5: + _interfaces = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Serial', + 0x04: 'PS/2', + 0x05: 'Infared', + 0x06: 'HP-HIL', + 0x07: 'Bus mouse', + 0x08: 'ADB (Apple Desktop Bus)', + 0x09: 'Bus mouse DB-9', + 0x0A: 'Bus mouse micro-DIN', + 0x0B: 'USB' + } + self.add_field('interface', u.unpack_one("B"), unpack.format_table("{}", _interfaces)) + if self.length > 0x6: + self.add_field('num_buttons', u.unpack_one("B")) + except: + self.decodeFailure = True + print "Error parsing BuiltInPointingDevice" + import traceback + traceback.print_exc() + self.fini() + +class PortableBattery(SmbiosBaseStructure): + smbios_structure_type = 22 + + def __init__(self, u, sm): + super(PortableBattery, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('location', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr) + if self.length > 0x6: + self.add_field('manufacturer_date', u.unpack_one("B"), self.fmtstr) + if self.length > 0x7: + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0x8: + self.add_field('device_name', u.unpack_one("B"), self.fmtstr) + if self.length > 0x9: + _device_chemistry = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Lead Acid', + 0x04: 'Nickel Cadmium', + 0x05: 'Nickel metal hydride', + 0x06: 'Lithium-ion', + 0x07: 'Zinc air', + 0x08: 'Lithium Polymer' + } + self.add_field('device_chemistry', u.unpack_one("B"), unpack.format_table("{}", _device_chemistry)) + if self.length > 0xA: + self.add_field('design_capacity', u.unpack_one("<H")) + if self.length > 0xC: + self.add_field('design_voltage', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('sbds_version_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0xF: + self.add_field('max_error_battery_data', u.unpack_one("B"), self.fmtstr) + if self.length > 0x10: + if self.serial_number == 0: + self.add_field('sbds_serial_number', u.unpack_one("<H")) + else: + u.skip(2) + if self.length > 0x12: + if self.manufacturer_date == 0: + self.add_field('sbds_manufacture_date', u.unpack_one("<H")) + self.add_field('year_biased_by_1980', bitfields.getbits(self.sbds_manufacture_date, 15, 9), "sbds_manufacture_date[15:9]={}") + self.add_field('month', bitfields.getbits(self.sbds_manufacture_date, 8, 5), "sbds_manufacture_date[8:5]={}") + self.add_field('date', bitfields.getbits(self.sbds_manufacture_date, 4, 0), "sbds_manufacture_date[4:0]={}") + else: + u.skip(2) + if self.length > 0x14: + if self.device_chemistry == 0x02: + self.add_field('sbds_device_chemistry', u.unpack_one("B"), self.fmtstr) + else: + u.skip(1) + if self.length > 0x15: + self.add_field('design_capacity_multiplier', u.unpack_one("B")) + if self.length > 0x16: + self.add_field('oem_specific', u.unpack_one("<I")) + except: + self.decodeFailure = True + print "Error parsing PortableBattery" + import traceback + traceback.print_exc() + self.fini() + +class SystemReset(SmbiosBaseStructure): + smbios_structure_type = 23 + + def __init__(self, u, sm): + super(SystemReset, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('capabilities', u.unpack_one("B")) + self.add_field('contains_watchdog_timer', bool(bitfields.getbits(self.capabilities, 5)), "capabilities[5]={}") + _boot_option = { + 0b00: 'Reserved, do not use', + 0b01: 'Operating System', + 0b10: 'System utilities', + 0b11: 'Do not reboot' + } + self.add_field('boot_option_on_limit', bitfields.getbits(self.capabilities, 4, 3), unpack.format_table("capabilities[4:3]={}", _boot_option)) + self.add_field('boot_option_after_watchdog_reset', bitfields.getbits(self.capabilities, 2, 1), unpack.format_table("capabilities[2:1]={}", _boot_option)) + self.add_field('system_reset_enabled_by_user', bool(bitfields.getbits(self.capabilities, 0)), "capabilities[0]={}") + if self.length > 0x5: + self.add_field('reset_count', u.unpack_one("<H")) + if self.length > 0x5: + self.add_field('reset_limit', u.unpack_one("<H")) + if self.length > 0x9: + self.add_field('timer_interval', u.unpack_one("<H")) + if self.length > 0xB: + self.add_field('timeout', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing SystemReset" + import traceback + traceback.print_exc() + self.fini() + +class HardwareSecurity(SmbiosBaseStructure): + smbios_structure_type = 24 + + def __init__(self, u, sm): + super(HardwareSecurity, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('hardware_security_settings', u.unpack_one("B")) + _status = { + 0x00: 'Disabled', + 0x01: 'Enabled', + 0x02: 'Not Implemented', + 0x03: 'Unknown' + } + self.add_field('power_on_password_status', bitfields.getbits(self.hardware_security_settings, 7, 6), unpack.format_table("hardware_security_settings[7:6]={}", _status)) + self.add_field('keyboard_password_status', bitfields.getbits(self.hardware_security_settings, 5, 4), unpack.format_table("hardware_security_settings[5:4]={}", _status)) + self.add_field('admin_password_status', bitfields.getbits(self.hardware_security_settings, 3, 2), unpack.format_table("hardware_security_settings0[3:2]={}", _status)) + self.add_field('front_panel_reset_status', bitfields.getbits(self.hardware_security_settings, 1, 0), unpack.format_table("hardware_security_settings[1:0]={}", _status)) + except: + self.decodeFailure = True + print "Error parsing HardwareSecurity" + import traceback + traceback.print_exc() + self.fini() + +class SystemPowerControls(SmbiosBaseStructure): + smbios_structure_type = 25 + + def __init__(self, u, sm): + super(SystemPowerControls, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('next_scheduled_poweron_month', u.unpack_one("B")) + self.add_field('next_scheduled_poweron_day_of_month', u.unpack_one("B")) + self.add_field('next_scheduled_poweron_hour', u.unpack_one("B")) + self.add_field('next_scheduled_poweron_minute', u.unpack_one("B")) + self.add_field('next_scheduled_poweron_second', u.unpack_one("B")) + except: + self.decodeFailure = True + print "Error parsing SystemPowerControls" + import traceback + traceback.print_exc() + self.fini() + +class VoltageProbe(SmbiosBaseStructure): + smbios_structure_type = 26 + + def __init__(self, u, sm): + super(VoltageProbe, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('description', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('location_and_status', u.unpack_one("B")) + _status = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'OK', + 0b100: 'Non-critical', + 0b101: 'Critical', + 0b110: 'Non-recoverable' + } + _location = { + 0b00001: 'Other', + 0b00010: 'Unknown', + 0b00011: 'Processor', + 0b00100: 'Disk', + 0b00101: 'Peripheral Bay', + 0b00110: 'System Management Module', + 0b00111: 'Motherboard', + 0b01000: 'Memory Module', + 0b01001: 'Processor Module', + 0b01010: 'Power Unit', + 0b01011: 'Add-in Card' + } + self.add_field('status', bitfields.getbits(self.location_and_status, 7, 5), unpack.format_table("location_and_status[7:5]={}", _status)) + self.add_field('location', bitfields.getbits(self.location_and_status, 4, 0), unpack.format_table("location_and_status[4:0]={}", _location)) + if self.length > 0x6: + self.add_field('max_value', u.unpack_one("<H")) + if self.length > 0x8: + self.add_field('min_value', u.unpack_one("<H")) + if self.length > 0xA: + self.add_field('resolution', u.unpack_one("<H")) + if self.length > 0xC: + self.add_field('tolerance', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('accuracy', u.unpack_one("<H")) + if self.length > 0x10: + self.add_field('oem_defined', u.unpack_one("<I")) + if self.length > 0x14: + self.add_field('nominal_value', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing VoltageProbe" + import traceback + traceback.print_exc() + self.fini() + +class CoolingDevice(SmbiosBaseStructure): + smbios_structure_type = 27 + + def __init__(self, u, sm): + super(CoolingDevice, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('temperature_probe_handle', u.unpack_one("<H")) + if self.length > 0x6: + self.add_field('device_type_and_status', u.unpack_one("B")) + _status = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'OK', + 0b100: 'Non-critical', + 0b101: 'Critical', + 0b110: 'Non-recoverable' + } + _type = { + 0b00001: 'Other', + 0b00010: 'Unknown', + 0b00011: 'Fan', + 0b00100: 'Centrifugal Blower', + 0b00101: 'Chip Fan', + 0b00110: 'Cabinet Fan', + 0b00111: 'Power Supply Fan', + 0b01000: 'Heat Pipe', + 0b01001: 'Integrated Refrigeration', + 0b10000: 'Active Cooling', + 0b10001: 'Passive Cooling' + } + self.add_field('status', bitfields.getbits(self.device_type_and_status, 7, 5), unpack.format_table("device_type_and_status[7:5]={}", _status)) + self.add_field('device_type', bitfields.getbits(self.device_type_and_status, 4, 0), unpack.format_table("device_type_and_status[4:0]={}", _type)) + if self.length > 0x7: + self.add_field('cooling_unit_group', u.unpack_one("B")) + if self.length > 0x8: + self.add_field('OEM_defined', u.unpack_one("<I")) + if self.length > 0xC: + self.add_field('nominal_speed', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('description', u.unpack_one("B"), self.fmtstr) + except: + self.decodeFailure = True + print "Error parsing CoolingDevice" + import traceback + traceback.print_exc() + self.fini() + +class TemperatureProbe(SmbiosBaseStructure): + smbios_structure_type = 28 + + def __init__(self, u, sm): + super(TemperatureProbe, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('description', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('location_and_status', u.unpack_one("B")) + _status = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'OK', + 0b100: 'Non-critical', + 0b101: 'Critical', + 0b110: 'Non-recoverable' + } + _location = { + 0b00001: 'Other', + 0b00010: 'Unknown', + 0b00011: 'Processor', + 0b00100: 'Disk', + 0b00101: 'Peripheral Bay', + 0b00110: 'System Management Module', + 0b00111: 'Motherboard', + 0b01000: 'Memory Module', + 0b01001: 'Processor Module', + 0b01010: 'Power Unit', + 0b01011: 'Add-in Card', + 0b01100: 'Front Panel Board', + 0b01101: 'Back Panel Board', + 0b01110: 'Power System Board', + 0b01111: 'Drive Back Plane' + } + self.add_field('status', bitfields.getbits(self.location_and_status, 7, 5), unpack.format_table("location_and_status[7:5]={}", _status)) + self.add_field('location', bitfields.getbits(self.location_and_status, 4, 0), unpack.format_table("location_and_status[4:0]={}", _location)) + if self.length > 0x6: + self.add_field('maximum_value', u.unpack_one("<H")) + if self.length > 0x8: + self.add_field('minimum_value', u.unpack_one("<H")) + if self.length > 0xA: + self.add_field('resolution', u.unpack_one("<H")) + if self.length > 0xC: + self.add_field('tolerance', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('accuracy', u.unpack_one("<H")) + if self.length > 0x10: + self.add_field('OEM_defined', u.unpack_one("<I")) + if self.length > 0x14: + self.add_field('nominal_value', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing TemperatureProbe" + import traceback + traceback.print_exc() + self.fini() + +class ElectricalCurrentProbe(SmbiosBaseStructure): + smbios_structure_type = 29 + + def __init__(self, u, sm): + super(ElectricalCurrentProbe, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('description', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('location_and_status', u.unpack_one("B")) + _status = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'OK', + 0b100: 'Non-critical', + 0b101: 'Critical', + 0b110: 'Non-recoverable' + } + _location = { + 0b00001: 'Other', + 0b00010: 'Unknown', + 0b00011: 'Processor', + 0b00100: 'Disk', + 0b00101: 'Peripheral Bay', + 0b00110: 'System Management Module', + 0b00111: 'Motherboard', + 0b01000: 'Memory Module', + 0b01001: 'Processor Module', + 0b01010: 'Power Unit', + 0b01011: 'Add-in Card', + 0b01100: 'Front Panel Board', + 0b01101: 'Back Panel Board', + 0b01110: 'Power System Board', + 0b01111: 'Drive Back Plane' + } + self.add_field('status', bitfields.getbits(self.location_and_status, 7, 5), unpack.format_table("location_and_status[7:5]={}", _status)) + self.add_field('location', bitfields.getbits(self.location_and_status, 4, 0), unpack.format_table("location_and_status[4:0]={}", _location)) + if self.length > 0x6: + self.add_field('maximum_value', u.unpack_one("<H")) + if self.length > 0x8: + self.add_field('minimum_value', u.unpack_one("<H")) + if self.length > 0xA: + self.add_field('resolution', u.unpack_one("<H")) + if self.length > 0xC: + self.add_field('tolerance', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('accuracy', u.unpack_one("<H")) + if self.length > 0x10: + self.add_field('OEM_defined', u.unpack_one("<I")) + if self.length > 0x14: + self.add_field('nominal_value', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing ElectricalCurrentProbe" + import traceback + traceback.print_exc() + self.fini() + +class OutOfBandRemoteAccess(SmbiosBaseStructure): + smbios_structure_type = 30 + + def __init__(self, u, sm): + super(OutOfBandRemoteAccess, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('manufacturer_name', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('connections', u.unpack_one("B")) + self.add_field('outbound_connection_enabled', bool(bitfields.getbits(self.connections, 1)), "connections[1]={}") + self.add_field('inbound_connection_enabled', bool(bitfields.getbits(self.connections, 0)), "connections[0]={}") + except: + self.decodeFailure = True + print "Error parsing OutOfBandRemoteAccess" + import traceback + traceback.print_exc() + self.fini() + +class BootIntegrityServicesEntryPoint(SmbiosBaseStructure): + smbios_structure_type = 31 + +class SystemBootInformation(SmbiosBaseStructure): + smbios_structure_type = 32 + + def __init__(self, u, sm): + super(SystemBootInformation, self).__init__(u, sm) + u = self.u + try: + if self.length > 0xA: + u.skip(6) + _boot_status = { + 0: 'No errors detected', + 1: 'No bootable media', + 2: '"normal" operating system failed to load', + 3: 'Firmware-detected hardware failure, including "unknown" failure types', + 4: 'Operating system-detected hardware failure', + 5: 'User-requested boot, usually through a keystroke', + 6: 'System security violation', + 7: 'Previously-requested image', + 8: 'System watchdog timer expired, causing the system to reboot', + xrange(9,127): 'Reserved for future assignment', + xrange(128, 191): 'Vendor/OEM-specific implementations', + xrange(192, 255): 'Product-specific implementations' + } + self.add_field('boot_status', u.unpack_one("B"), unpack.format_table("{}", _boot_status)) + except: + self.decodeFailure = True + print "Error parsing SystemBootInformation" + import traceback + traceback.print_exc() + self.fini() + +class MemoryErrorInfo64Bit(SmbiosBaseStructure): + smbios_structure_type = 33 + + def __init__(self, u, sm): + super(MemoryErrorInfo64Bit, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + _error_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'OK', + 0x04: 'Bad read', + 0x05: 'Parity error', + 0x06: 'Single-bit error', + 0x07: 'Double-bit error', + 0x08: 'Multi-bit error', + 0x09: 'Nibble error', + 0x0A: 'Checksum error', + 0x0B: 'CRC error', + 0x0C: 'Corrected single-bit error', + 0x0D: 'Corrected error', + 0x0E: 'Uncorrectable error' + } + self.add_field('error_type', u.unpack_one("B"), unpack.format_table("{}", _error_types)) + if self.length > 0x5: + _error_granularity_field = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Device level', + 0x04: 'Memory partition level' + } + self.add_field('error_granularity', u.unpack_one("B"), unpack.format_table("{}", _error_granularity_field)) + if self.length > 0x6: + _error_operation_field = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Read', + 0x04: 'Write', + 0x05: 'Partial write' + } + self.add_field('error_operation', u.unpack_one("B"), unpack.format_table("{}", _error_operation_field)) + if self.length > 0x7: + self.add_field('vendor_syndrome', u.unpack_one("<I")) + if self.length > 0xB: + self.add_field('memory_array_error_address', u.unpack_one("<Q")) + if self.length > 0xF: + self.add_field('device_error_address', u.unpack_one("<Q")) + if self.length > 0x13: + self.add_field('error_resolution', u.unpack_one("<Q")) + except: + self.decodeFailure = True + print "Error parsing MemoryErrorInfo64Bit" + import traceback + traceback.print_exc() + self.fini() + +class ManagementDevice(SmbiosBaseStructure): + smbios_structure_type = 34 + + def __init__(self, u, sm): + super(ManagementDevice, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('description', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + _type = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'National Semiconductor LM75', + 0x04: 'National Semiconductor LM78', + 0x05: 'National Semiconductor LM79', + 0x06: 'National Semiconductor LM80', + 0x07: 'National Semiconductor LM81', + 0x08: 'Analog Devices ADM9240', + 0x09: 'Dallas Semiconductor DS1780', + 0x0A: 'Maxim 1617', + 0x0B: 'Genesys GL518SM', + 0x0C: 'Winbond W83781D', + 0x0D: 'Holtek HT82H791' + } + self.add_field('device_type', u.unpack_one("B"), unpack.format_table("{}", _type)) + if self.length > 0x6: + self.add_field('address', u.unpack_one("<I")) + if self.length > 0xA: + _address_type = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'I/O Port', + 0x04: 'Memory', + 0x05: 'SM Bus' + } + self.add_field('address_type', u.unpack_one("B"), unpack.format_table("{}", _address_type)) + except: + self.decodeFailure = True + print "Error parsing ManagementDevice" + import traceback + traceback.print_exc() + self.fini() + +class ManagementDeviceComponent(SmbiosBaseStructure): + smbios_structure_type = 35 + + def __init__(self, u, sm): + super(ManagementDeviceComponent, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('description', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('management_device_handle', u.unpack_one("<H")) + if self.length > 0x7: + self.add_field('component_handle', u.unpack_one("<H")) + if self.length > 0x9: + self.add_field('threshold_handle', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing ManagementDeviceComponent" + import traceback + traceback.print_exc() + self.fini() + +class ManagementDeviceThresholdData(SmbiosBaseStructure): + smbios_structure_type = 36 + + def __init__(self, u, sm): + super(ManagementDeviceThresholdData, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('lower_threshold_noncritical', u.unpack_one("<H")) + if self.length > 0x6: + self.add_field('upper_threshold_noncritical', u.unpack_one("<H")) + if self.length > 0x8: + self.add_field('lower_threshold_critical', u.unpack_one("<H")) + if self.length > 0xA: + self.add_field('upper_threshold_critical', u.unpack_one("<H")) + if self.length > 0xC: + self.add_field('lower_threshold_nonrecoverable', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('upper_threshold_nonrecoverable', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing ManagementDeviceThresholdData" + import traceback + traceback.print_exc() + self.fini() + +class MemoryChannel(SmbiosBaseStructure): + smbios_structure_type = 37 + + def __init__(self, u, sm): + super(MemoryChannel, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + _channel_type = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'RamBus', + 0x04: 'SyncLink' + } + self.add_field('channel_type', u.unpack_one("B"), unpack.format_table("{}", _channel_type)) + if self.length > 0x6: + self.add_field('max_channel_load', u.unpack_one("B")) + if self.length > 0x8: + self.add_field('memory_device_count', u.unpack_one("B")) + if self.length > 0xA: + self.add_field('memory_device_load', u.unpack_one("B")) + if self.length > 0xC: + self.add_field('memory_device_handle', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing MemoryChannel" + import traceback + traceback.print_exc() + self.fini() + +class IPMIDeviceInformation(SmbiosBaseStructure): + smbios_structure_type = 38 + + def __init__(self, u, sm): + super(IPMIDeviceInformation, self).__init__(u, sm) + u = self.u + try: + _interface_type = { + 0x00: 'Unknown', + 0x01: 'KCS: Keyboard Controller Style', + 0x02: 'SMIC: Server Management Interface Chip', + 0x03: 'BT: Block Transfer', + xrange(0x04, 0xFF): 'Reserved' + } + self.add_field('interface_type', u.unpack_one("B"), unpack.format_table("{}", _interface_type)) + self.add_field('ipmi_specification_revision', u.unpack_one("B")) + self.add_field('msd_revision', bitfields.getbits(self.ipmi_specification_revision, 7, 4), "ipmi_specification_revision[7:4]={}") + self.add_field('lsd_revision', bitfields.getbits(self.ipmi_specification_revision, 3, 0), "ipmi_specification_revision[3:0]={}") + + self.add_field('i2c_slave_address', u.unpack_one("B")) + self.add_field('nv_storage_device_address', u.unpack_one("B")) + self.add_field('base_address', u.unpack_one("<Q")) + # if lsb is 1, address is in IO space. otherwise, memory-mapped + self.add_field('base_address_modifier_interrupt_info', u.unpack_one("B")) + _reg_spacing = { + 0b00: 'Interface registers are on successive byte boundaries', + 0b01: 'Interface registers are on 32-bit boundaries', + 0b10: 'Interface registers are on 16-byte boundaries', + 0b11: 'Reserved' + } + self.add_field('register_spacing', bitfields.getbits(self.base_address_modifier_interrupt_info, 7, 6), unpack.format_table("base_address_modifier_interrupt_info[7:6]={}", _reg_spacing)) + self.add_field('ls_bit_for_addresses', bitfields.getbits(self.base_address_modifier_interrupt_info, 4), "base_address_modifier_interrupt_info[4]={}") + self.add_field('interrupt_info_specified', bool(bitfields.getbits(self.base_address_modifier_interrupt_info, 3)), "base_address_modifier_interrupt_info[3]={}") + _polarity = { + 0: 'active low', + 1: 'active high' + } + self.add_field('interrupt_polarity', bitfields.getbits(self.base_address_modifier_interrupt_info, 1), unpack.format_table("base_address_modifier_interrupt_info[1]={}", _polarity)) + _interrupt_trigger = { + 0: 'edge', + 1: 'level' + } + self.add_field('interrupt_trigger_mode', bitfields.getbits(self.base_address_modifier_interrupt_info, 0), unpack.format_table("base_address_modifier_interrupt_info[0]={}", _interrupt_trigger)) + self.add_field('interrupt_number', u.unpack_one("B")) + except: + self.decodeFailure = True + print "Error parsing IPMIDeviceInformation" + import traceback + traceback.print_exc() + self.fini() + +class SystemPowerSupply(SmbiosBaseStructure): + smbios_structure_type = 39 + + def __init__(self, u, sm): + super(SystemPowerSupply, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('power_unit_group', u.unpack_one("B")) + if self.length > 0x5: + self.add_field('location', u.unpack_one("B"), self.fmtstr) + if self.length > 0x6: + self.add_field('device_name', u.unpack_one("B"), self.fmtstr) + if self.length > 0x7: + self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr) + if self.length > 0x8: + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0x9: + self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr) + if self.length > 0xA: + self.add_field('model_part_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0xB: + self.add_field('revision_level', u.unpack_one("B"), self.fmtstr) + if self.length > 0xC: + self.add_field('max_power_capacity', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('power_supply_characteristics', u.unpack_one("<H")) + _dmtf_power_supply_type = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'Linear', + 0b100: 'Switching', + 0b101: 'Battery', + 0b110: 'UPS', + 0b111: 'Converter', + 0b1000: 'Regulator', + xrange(0b1001, 0b1111): 'Reserved' + } + self.add_field('dmtf_power_supply_type', bitfields.getbits(self.power_supply_characteristics, 13, 10), unpack.format_table("power_supply_characteristics[13:10]={}", _dmtf_power_supply_type)) + _status = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'OK', + 0b100: 'Non-critical', + 0b101: 'Critical; power supply has failed and has been taken off-line' + } + self.add_field('status', bitfields.getbits(self.power_supply_characteristics, 9, 7), unpack.format_table("power_supply_characteristics[9:7]={}", _status)) + _dmtf_input_voltage_range_switching = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'Manual', + 0b100: 'Auto-switch', + 0b101: 'Wide range', + 0b110: 'Not applicable', + xrange(0b0111, 0b1111): 'Reserved' + } + self.add_field('dmtf_input_voltage_range_switching', bitfields.getbits(self.power_supply_characteristics, 6, 3), unpack.format_table("power_supply_characteristics[6:3]={}", _dmtf_input_voltage_range_switching)) + self.add_field('power_supply_unplugged', bool(bitfields.getbits(self.power_supply_characteristics, 2)), "power_supply_characteristics[2]={}") + self.add_field('power_supply_present', bool(bitfields.getbits(self.power_supply_characteristics, 1)), "power_supply_characteristics[1]={}") + self.add_field('power_supply_hot_replaceable', bool(bitfields.getbits(self.power_supply_characteristics, 0)), "power_supply_characteristics[0]={}") + if self.length > 0x10: + self.add_field('input_voltage_probe_handle', u.unpack_one("<H")) + if self.length > 0x12: + self.add_field('cooling_device_handle', u.unpack_one("<H")) + if self.length > 0x14: + self.add_field('input_current_probe_handle', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing SystemPowerSupply" + import traceback + traceback.print_exc() + self.fini() + +class AdditionalInformation(SmbiosBaseStructure): + smbios_structure_type = 40 + + def __init__(self, u, sm): + super(AdditionalInformation, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('num_additional_information_entries', u.unpack_one("B")) + if self.length > 0x5: + self.add_field('additional_information_entry_length', u.unpack_one("B")) + self.add_field('referenced_handle', u.unpack_one("<H")) + self.add_field('referenced_offset', u.unpack_one("B")) + self.add_field('string', u.unpack_one("B"), self.fmtstr) + self.add_field('value', u.unpack_rest()) + except: + self.decodeFailure = True + print "Error parsing AdditionalInformation" + import traceback + traceback.print_exc() + self.fini() + +class OnboardDevicesExtendedInformation(SmbiosBaseStructure): + smbios_structure_type = 41 + + def __init__(self, u, sm): + super(OnboardDevicesExtendedInformation, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('reference_designation', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('device_type', u.unpack_one("B")) + self.add_field('device_enabled', bool(bitfields.getbits(self.device_type, 7)), "device_type[7]={}") + _device_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Video', + 0x04: 'SCSI Controller', + 0x05: 'Ethernet', + 0x06: 'Token Ring', + 0x07: 'Sound', + 0x08: 'PATA Controller', + 0x09: 'SATA Controller', + 0x0A: 'SAS Controller' + } + self.add_field('type_of_device', bitfields.getbits(self.device_type, 6, 0), unpack.format_table("device_type[6:0]={}", _device_types)) + if self.length > 0x6: + self.add_field('device_type_instance', u.unpack_one("B")) + if self.length > 0x7: + self.add_field('segment_group_number', u.unpack_one("<H")) + if self.length > 0x9: + self.add_field('bus_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0xA: + self.add_field('device_and_function_number', u.unpack_one("B")) + self.add_field('device_number', bitfields.getbits(self.device_type, 7, 3), "device_and_function_number[7:3]={}") + self.add_field('function_number', bitfields.getbits(self.device_type, 2, 0), "device_and_function_number[2:0]={}") + except: + self.decodeFailure = True + print "Error parsing OnboardDevicesExtendedInformation" + import traceback + traceback.print_exc() + self.fini() + +class ManagementControllerHostInterface(SmbiosBaseStructure): + smbios_structure_type = 42 + + def __init__(self, u, sm): + super(ManagementControllerHostInterface, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + _interface_types = { + 0x00: 'Reserved', + 0x01: 'Reserved', + 0x02: 'KCS: Keyboard Controller Style', + 0x03: '8250 UART Register Compatible', + 0x04: '16450 UART Register Compatible', + 0x05: '16550/16550A UART Register Compatible', + 0x06: '16650/16650A UART Register Compatible', + 0x07: '16750/16750A UART Register Compatible', + 0x08: '16850/16850A UART Register Compatible', + 0xF0: 'OEM' + } + self.add_field('interface_type', u.unpack_one("B"), unpack.format_table("{}", _interface_types)) + if self.length > 0x5: + self.add_field('mc_host_interface_data', u.unpack_rest(), self.fmtstr) + except: + self.decodeFailure = True + print "Error parsing ManagementControllerHostInterface" + import traceback + traceback.print_exc() + self.fini() + +class Inactive(SmbiosBaseStructure): + smbios_structure_type = 126 + + def __init__(self, u, sm): + super(Inactive, self).__init__(u, sm) + self.fini() + +class EndOfTable(SmbiosBaseStructure): + smbios_structure_type = 127 + + def __init__(self, u, sm): + super(EndOfTable, self).__init__(u, sm) + self.fini() + +class SmbiosStructureUnknown(SmbiosBaseStructure): + smbios_structure_type = None + + def __init__(self, u, sm): + super(SmbiosStructureUnknown, self).__init__(u, sm) + self.fini() + +_smbios_structures = [ + BIOSInformation, + SystemInformation, + BaseboardInformation, + SystemEnclosure, + ProcessorInformation, + MemoryControllerInformation, + MemoryModuleInformation, + CacheInformation, + PortConnectorInfo, + SystemSlots, + OnBoardDevicesInformation, + OEMStrings, + SystemConfigOptions, + BIOSLanguageInformation, + GroupAssociations, + SystemEventLog, + PhysicalMemoryArray, + MemoryDevice, + MemoryErrorInfo32Bit, + MemoryArrayMappedAddress, + MemoryDeviceMappedAddress, + BuiltInPointingDevice, + PortableBattery, + SystemReset, + HardwareSecurity, + SystemPowerControls, + VoltageProbe, + CoolingDevice, + TemperatureProbe, + ElectricalCurrentProbe, + OutOfBandRemoteAccess, + BootIntegrityServicesEntryPoint, + SystemBootInformation, + MemoryErrorInfo64Bit, + ManagementDevice, + ManagementDeviceComponent, + ManagementDeviceThresholdData, + MemoryChannel, + IPMIDeviceInformation, + SystemPowerSupply, + AdditionalInformation, + OnboardDevicesExtendedInformation, + ManagementControllerHostInterface, + Inactive, + EndOfTable, + SmbiosStructureUnknown, # Must always come last +] + +def log_smbios_info(): + with redirect.logonly(): + try: + sm = SMBIOS() + print + if sm is None: + print "No SMBIOS structures found" + return + output = {} + known_types = (0, 1) + for sm_struct in sm.structures: + if sm_struct.type in known_types: + output.setdefault(sm_struct.type, []).append(sm_struct) + if len(output) == len(known_types): + break + + print "SMBIOS information:" + for key in sorted(known_types): + for s in output.get(key, ["No structure of type {} found".format(key)]): + print ttypager._wrap("{}: {}".format(key, s)) + except: + print "Error parsing SMBIOS information:" + import traceback + traceback.print_exc() + +def dump_raw(): + try: + sm = SMBIOS() + if sm: + s = "SMBIOS -- Raw bytes and structure decode.\n\n" + + s += str(sm.header) + '\n' + s += bits.dumpmem(sm._header_memory) + '\n' + + s += "Raw bytes for the SMBIOS structures\n" + s += bits.dumpmem(sm._structure_memory) + '\n' + + for sm_struct in sm.structures: + s += str(sm_struct) + '\n' + s += bits.dumpmem(sm_struct.raw_data) + + s += "Strings:\n" + for n in range(1, len(getattr(sm_struct, "strings", [])) + 1): + s += str(sm_struct.fmtstr(n)) + '\n' + s += bits.dumpmem(sm_struct.raw_strings) + '\n' + else: + s = "No SMBIOS structures found" + ttypager.ttypager_wrap(s, indent=False) + except: + print "Error parsing SMBIOS information:" + import traceback + traceback.print_exc() + +def dump(): + try: + sm = SMBIOS() + if sm: + s = str(sm) + else: + s = "No SMBIOS structures found" + ttypager.ttypager_wrap(s, indent=False) + except: + print "Error parsing SMBIOS information:" + import traceback + traceback.print_exc() + +def annex_a_conformance(): + try: + sm = SMBIOS() + + # check: 1. The table anchor string "_SM_" is present in the address range 0xF0000 to 0xFFFFF on a 16-byte bound + + def table_entry_point_verification(): + ''' Verify table entry-point''' + if (sm.header.length < 0x1F): + print "Failure: Table entry-point - The entry-point Length must be at least 0x1F" + if sm.header.checksum != 0: + print "Failure: Table entry-point - The entry-point checksum must evaluate to 0" + if ((sm.header.major_version < 2) and (sm.header.minor_version < 4)): + print "Failure: Table entry-point - SMBIOS version must be at least 2.4" + if (sm.header.intermediate_anchor_string == '_DMI_'): + print "Failure: Table entry-point - The Intermediate Anchor String must be '_DMI_'" + if (sm.header.intermediate_checksum != 0): + print "Failure: Table entry-point - The Intermediate checksum must evaluate to 0" + + #check: 3. The structure-table is traversable and conforms to the entry-point specifications: + + def req_structures(): + '''Checks for required structures and corresponding data''' + types_present = [sm.structures[x].smbios_structure_type for x in range(len(sm.structures))] + required = [0, 1, 4, 7, 9, 16, 17, 19, 31, 32] + for s in required: + if s not in set(types_present): + print "Failure: Type {} required but not found".format(s) + + else: + if s == 0: + if types_present.count(s) > 1: + print "Failure: Type {} - One and only one structure of this type must be present.".format(s) + if sm.structure_type(s).length < 0x18: + print "Failure: Type {} - The structure Length field must be at least 0x18".format(s) + if sm.structure_type(s).version is None: + print "Failure: Type {} - BIOS Version string must be present and non-null.".format(s) + if sm.structure_type(s).release_date is None: + print "Failure: Type {} - BIOS Release Date string must be present, non-null, and include a 4-digit year".format(s) + if bitfields.getbits(sm.structure_type(s).characteristics, 3, 0) != 0 or bitfields.getbits(sm.structure_type(s).characteristics, 31, 4) == 0: + print "Failure: Type {} - BIOS Characteristics: bits 3:0 must all be 0, and at least one of bits 31:4 must be set to 1.".format(s) + elif s == 1: + if types_present.count(s) > 1: + print "Failure: Type {} - One and only one structure of this type must be present.".format(s) + if sm.structure_type(s).length < 0x1B: + print "Failure: Type {} - The structure Length field must be at least 0x1B".format(s) + if sm.structure_type(s).manufacturer == None: + print "Failure: Type {} - Manufacturer string must be present and non-null.".format(s) + if sm.structure_type(s).product_name == None: + print "Failure: Type {} - Product Name string must be present and non-null".format(s) + if sm.structure_type(s).uuid == '00000000 00000000' and sm.structure_type(s).uuid == 'FFFFFFFF FFFFFFFF': + print "Failure: Type {} - UUID field must be neither 00000000 00000000 nor FFFFFFFF FFFFFFFF.".format(s) + if sm.structure_type(s).wakeup_type == 00 and sm.structure_type(s).wakeup_type == 0x02: + print "Failure: Type {} - Wake-up Type field must be neither 00h (Reserved) nor 02h (Unknown).".format(s) + # continue for remaining required types + + # check remaining conformance guidelines + + table_entry_point_verification() + req_structures() + except: + print "Error checking ANNEX A conformance guidelines" + import traceback + traceback.print_exc()tests/qtest/acpi-bits/bits-tests/testacpi.py new file +283
-0@@ -0,0 +1,283 @@ +# Copyright (c) 2015, Intel Corporation +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# * Redistributions of source code must retain the above copyright notice, +# this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# * Neither the name of Intel Corporation nor the names of its contributors +# may be used to endorse or promote products derived from this software +# without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Tests for ACPI""" + +import acpi +import bits +import bits.mwait +import struct +import testutil +import testsuite +import time + +def register_tests(): + testsuite.add_test("ACPI _MAT (Multiple APIC Table Entry) under Processor objects", test_mat, submenu="ACPI Tests") + testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", test_pss, submenu="ACPI Tests") + testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, submenu="ACPI Tests") + testsuite.add_test("ACPI DSDT (Differentiated System Description Table)", test_dsdt, submenu="ACPI Tests") + testsuite.add_test("ACPI FACP (Fixed ACPI Description Table)", test_facp, submenu="ACPI Tests") + testsuite.add_test("ACPI HPET (High Precision Event Timer Table)", test_hpet, submenu="ACPI Tests") + testsuite.add_test("ACPI MADT (Multiple APIC Description Table)", test_apic, submenu="ACPI Tests") + testsuite.add_test("ACPI MPST (Memory Power State Table)", test_mpst, submenu="ACPI Tests") + testsuite.add_test("ACPI RSDP (Root System Description Pointer Structure)", test_rsdp, submenu="ACPI Tests") + testsuite.add_test("ACPI XSDT (Extended System Description Table)", test_xsdt, submenu="ACPI Tests") + +def test_mat(): + cpupaths = acpi.get_cpupaths() + apic = acpi.parse_apic() + procid_apicid = apic.procid_apicid + uid_x2apicid = apic.uid_x2apicid + for cpupath in cpupaths: + # Find the ProcId defined by the processor object + processor = acpi.evaluate(cpupath) + # Find the UID defined by the processor object's _UID method + uid = acpi.evaluate(cpupath + "._UID") + mat_buffer = acpi.evaluate(cpupath + "._MAT") + if mat_buffer is None: + continue + # Process each _MAT subtable + mat = acpi._MAT(mat_buffer) + for index, subtable in enumerate(mat): + if subtable.subtype == acpi.MADT_TYPE_LOCAL_APIC: + if subtable.flags.bits.enabled: + testsuite.test("{} Processor declaration ProcId = _MAT ProcId".format(cpupath), processor.ProcId == subtable.proc_id) + testsuite.print_detail("{} ProcId ({:#02x}) != _MAT ProcId ({:#02x})".format(cpupath, processor.ProcId, subtable.proc_id)) + testsuite.print_detail("Processor Declaration: {}".format(processor)) + testsuite.print_detail("_MAT entry[{}]: {}".format(index, subtable)) + if testsuite.test("{} with local APIC in _MAT has local APIC in MADT".format(cpupath), processor.ProcId in procid_apicid): + testsuite.test("{} ApicId derived using Processor declaration ProcId = _MAT ApicId".format(cpupath), procid_apicid[processor.ProcId] == subtable.apic_id) + testsuite.print_detail("{} ApicId derived from MADT ({:#02x}) != _MAT ApicId ({:#02x})".format(cpupath, procid_apicid[processor.ProcId], subtable.apic_id)) + testsuite.print_detail("Processor Declaration: {}".format(processor)) + testsuite.print_detail("_MAT entry[{}]: {}".format(index, subtable)) + if subtable.subtype == acpi.MADT_TYPE_LOCAL_X2APIC: + if subtable.flags.bits.enabled: + if testsuite.test("{} with x2Apic in _MAT has _UID".format(cpupath), uid is not None): + testsuite.test("{}._UID = _MAT UID".format(cpupath), uid == subtable.uid) + testsuite.print_detail("{}._UID ({:#x}) != _MAT UID ({:#x})".format(cpupath, uid, subtable.uid)) + testsuite.print_detail("_MAT entry[{}]: {}".format(index, subtable)) + if testsuite.test("{} with _MAT x2Apic has x2Apic in MADT".format(cpupath), subtable.uid in uid_x2apicid): + testsuite.test("{} x2ApicId derived from MADT using UID = _MAT x2ApicId".format(cpupath), uid_x2apicid[subtable.uid] == subtable.x2apicid) + testsuite.print_detail("{} x2ApicId derived from MADT ({:#02x}) != _MAT x2ApicId ({:#02x})".format(cpupath, uid_x2apicid[subtable.uid], subtable.x2apicid)) + testsuite.print_detail("_MAT entry[{}]: {}".format(index, subtable)) + +def test_pss(): + uniques = acpi.parse_cpu_method("_PSS") + # We special-case None here to avoid a double-failure for CPUs without a _PSS + testsuite.test("_PSS must be identical for all CPUs", len(uniques) <= 1 or (len(uniques) == 2 and None in uniques)) + for pss, cpupaths in uniques.iteritems(): + if not testsuite.test("_PSS must exist", pss is not None): + testsuite.print_detail(acpi.factor_commonprefix(cpupaths)) + testsuite.print_detail('No _PSS exists') + continue + + if not testsuite.test("_PSS must not be empty", pss.pstates): + testsuite.print_detail(acpi.factor_commonprefix(cpupaths)) + testsuite.print_detail('_PSS is empty') + continue + + testsuite.print_detail(acpi.factor_commonprefix(cpupaths)) + for index, pstate in enumerate(pss.pstates): + testsuite.print_detail("P[{}]: {}".format(index, pstate)) + + testsuite.test("_PSS must contain at most 16 Pstates", len(pss.pstates) <= 16) + testsuite.test("_PSS must have no duplicate Pstates", len(pss.pstates) == len(set(pss.pstates))) + + frequencies = [p.core_frequency for p in pss.pstates] + testsuite.test("_PSS must list Pstates in descending order of frequency", frequencies == sorted(frequencies, reverse=True)) + + testsuite.test("_PSS must have Pstates with no duplicate frequencies", len(frequencies) == len(set(frequencies))) + + dissipations = [p.power for p in pss.pstates] + testsuite.test("_PSS must list Pstates in descending order of power dissipation", dissipations == sorted(dissipations, reverse=True)) + +def test_pstates(): + """Execute and verify frequency for each Pstate in the _PSS""" + IA32_PERF_CTL = 0x199 + with bits.mwait.use_hint(), bits.preserve_msr(IA32_PERF_CTL): + cpupath_procid = acpi.find_procid() + cpupath_uid = acpi.find_uid() + apic = acpi.parse_apic() + procid_apicid = apic.procid_apicid + uid_x2apicid = apic.uid_x2apicid + def cpupath_apicid(cpupath): + if procid_apicid is not None: + procid = cpupath_procid.get(cpupath, None) + if procid is not None: + apicid = procid_apicid.get(procid, None) + if apicid is not None: + return apicid + if uid_x2apicid is not None: + uid = cpupath_uid.get(cpupath, None) + if uid is not None: + apicid = uid_x2apicid.get(uid, None) + if apicid is not None: + return apicid + return bits.cpus()[0] + + bclk = testutil.adjust_to_nearest(bits.bclk(), 100.0/12) * 1000000 + + uniques = acpi.parse_cpu_method("_PSS") + for pss, cpupaths in uniques.iteritems(): + if not testsuite.test("_PSS must exist", pss is not None): + testsuite.print_detail(acpi.factor_commonprefix(cpupaths)) + testsuite.print_detail('No _PSS exists') + continue + + for n, pstate in enumerate(pss.pstates): + for cpupath in cpupaths: + apicid = cpupath_apicid(cpupath) + if apicid is None: + print 'Failed to find apicid for cpupath {}'.format(cpupath) + continue + bits.wrmsr(apicid, IA32_PERF_CTL, pstate.control) + + # Detecting Turbo frequency requires at least 2 pstates + # since turbo frequency = max non-turbo frequency + 1 + turbo = False + if len(pss.pstates) >= 2: + turbo = (n == 0 and pstate.core_frequency == (pss.pstates[1].core_frequency + 1)) + if turbo: + # Needs to busywait, not sleep + start = time.time() + while (time.time() - start < 2): + pass + + for duration in (0.1, 1.0): + frequency_data = bits.cpu_frequency(duration) + # Abort the test if no cpu frequency is not available + if frequency_data is None: + continue + aperf = frequency_data[1] + aperf = testutil.adjust_to_nearest(aperf, bclk/2) + aperf = int(aperf / 1000000) + if turbo: + if aperf >= pstate.core_frequency: + break + else: + if aperf == pstate.core_frequency: + break + + if turbo: + testsuite.test("P{}: Turbo measured frequency {} >= expected {} MHz".format(n, aperf, pstate.core_frequency), aperf >= pstate.core_frequency) + else: + testsuite.test("P{}: measured frequency {} MHz == expected {} MHz".format(n, aperf, pstate.core_frequency), aperf == pstate.core_frequency) + +def test_psd_thread_scope(): + uniques = acpi.parse_cpu_method("_PSD") + if not testsuite.test("_PSD (P-State Dependency) must exist for each processor", None not in uniques): + testsuite.print_detail(acpi.factor_commonprefix(uniques[None])) + testsuite.print_detail('No _PSD exists') + return + unique_num_dependencies = {} + unique_num_entries = {} + unique_revision = {} + unique_domain = {} + unique_coordination_type = {} + unique_num_processors = {} + for value, cpupaths in uniques.iteritems(): + unique_num_dependencies.setdefault(len(value.dependencies), []).extend(cpupaths) + unique_num_entries.setdefault(value.dependencies[0].num_entries, []).extend(cpupaths) + unique_revision.setdefault(value.dependencies[0].revision, []).extend(cpupaths) + unique_domain.setdefault(value.dependencies[0].domain, []).extend(cpupaths) + unique_coordination_type.setdefault(value.dependencies[0].coordination_type, []).extend(cpupaths) + unique_num_processors.setdefault(value.dependencies[0].num_processors, []).extend(cpupaths) + def detail(d, fmt): + for value, cpupaths in sorted(d.iteritems(), key=(lambda (k,v): v)): + testsuite.print_detail(acpi.factor_commonprefix(cpupaths)) + testsuite.print_detail(fmt.format(value)) + + testsuite.test('Dependency count for each processor must be 1', unique_num_dependencies.keys() == [1]) + detail(unique_num_dependencies, 'Dependency count for each processor = {} (Expected 1)') + testsuite.test('_PSD.num_entries must be 5', unique_num_entries.keys() == [5]) + detail(unique_num_entries, 'num_entries = {} (Expected 5)') + testsuite.test('_PSD.revision must be 0', unique_revision.keys() == [0]) + detail(unique_revision, 'revision = {}') + testsuite.test('_PSD.coordination_type must be 0xFE (HW_ALL)', unique_coordination_type.keys() == [0xfe]) + detail(unique_coordination_type, 'coordination_type = {:#x} (Expected 0xFE HW_ALL)') + testsuite.test('_PSD.domain must be unique (thread-scoped) for each processor', len(unique_domain) == len(acpi.get_cpupaths())) + detail(unique_domain, 'domain = {:#x} (Expected a unique value for each processor)') + testsuite.test('_PSD.num_processors must be 1', unique_num_processors.keys() == [1]) + detail(unique_num_processors, 'num_processors = {} (Expected 1)') + +def test_table_checksum(data): + csum = sum(ord(c) for c in data) % 0x100 + testsuite.test('ACPI table cumulative checksum must equal 0', csum == 0) + testsuite.print_detail("Cumulative checksum = {} (Expected 0)".format(csum)) + +def test_apic(): + data = acpi.get_table("APIC") + if data is None: + return + test_table_checksum(data) + apic = acpi.parse_apic() + +def test_dsdt(): + data = acpi.get_table("DSDT") + if data is None: + return + test_table_checksum(data) + +def test_facp(): + data = acpi.get_table("FACP") + if data is None: + return + test_table_checksum(data) + facp = acpi.parse_facp() + +def test_hpet(): + data = acpi.get_table("HPET") + if data is None: + return + test_table_checksum(data) + hpet = acpi.parse_hpet() + +def test_mpst(): + data = acpi.get_table("MPST") + if data is None: + return + test_table_checksum(data) + mpst = acpi.MPST(data) + +def test_rsdp(): + data = acpi.get_table("RSD PTR ") + if data is None: + return + + # Checksum the first 20 bytes per ACPI 1.0 + csum = sum(ord(c) for c in data[:20]) % 0x100 + testsuite.test('ACPI 1.0 table first 20 bytes cummulative checksum must equal 0', csum == 0) + testsuite.print_detail("Cummulative checksum = {} (Expected 0)".format(csum)) + + test_table_checksum(data) + rsdp = acpi.parse_rsdp() + +def test_xsdt(): + data = acpi.get_table("XSDT") + if data is None: + return + test_table_checksum(data) + xsdt = acpi.parse_xsdt()tests/qtest/acpi-bits/bits-tests/testcpuid.py new file +15
-0@@ -0,0 +1,83 @@ +# Copyright (c) 2012, Intel Corporation +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# * Redistributions of source code must retain the above copyright notice, +# this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# * Neither the name of Intel Corporation nor the names of its contributors +# may be used to endorse or promote products derived from this software +# without specific prior written permission. +#
tests/qtest/acpi-bits/bits-tests/testcpuid.py line 25
Quoted 10 lines
tests/qtest/acpi-bits/bits-tests/testcpuid.py new file +10
-0+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Could you do me a favour and add a SPDX license identifier in front of the license boilerplate codes?
It's not required for the QEMU project, but at least I normally have troubles to find out what such longer license boilertexts mean ... is it a BSD license, is it a MIT license, something completely else...? A short SPDX line at the beginning helps to identify the license at a first glance.
Thomas
Re: [PATCH 04/12] acpi/tests/bits: initial commit of test scripts that are run by biosbits
On Tue, Jun 28, 2022 at 09:24:19AM +0200, Thomas Huth wrote:
... 3000 lines of irrelevant code ...
Quoted 35 lines
Quoted 21 lines
tests/qtest/acpi-bits/bits-tests/testcpuid.py new file +15
-0@@ -0,0 +1,83 @@ +# Copyright (c) 2012, Intel Corporation +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# * Redistributions of source code must retain the above copyright notice, +# this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# * Neither the name of Intel Corporation nor the names of its contributors +# may be used to endorse or promote products derived from this software +# without specific prior written permission. +#tests/qtest/acpi-bits/bits-tests/testcpuid.py line 25
Quoted 10 lines
tests/qtest/acpi-bits/bits-tests/testcpuid.py new file +10
-0+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.Could you do me a favour and add a SPDX license identifier in front of the license boilerplate codes?
Quoted 6 lines
It's not required for the QEMU project, but at least I normally have troubles to find out what such longer license boilertexts mean ... is it a BSD license, is it a MIT license, something completely else...? A short SPDX line at the beginning helps to identify the license at a first glance.
Thomas
Pls remember to cut out irrelevant text when you respond.
Thanks,
Signature · 1 line
MST
PATCH 5/12 acpi/tests/bits: disable acpi PSS tests that are failing in biosbits
In reply to Ani SinhaPSS tests in acpi test suite seems to be failing in biosbits. This is because the test is unable to find PSS support in QEMU bios. Let us disable them for now so that make check does not fail. We can fix the tests and re-enable them later.
Example failure:
---- ACPI _PSS (Pstate) table conformance tests ---- [assert] _PSS must exist FAIL
\_SB_.CPUS.C000
No _PSS exists
Summary: 1 passed, 1 failed
---- ACPI _PSS (Pstate) runtime tests ---- [assert] _PSS must exist FAIL
\_SB_.CPUS.C000
No _PSS exists
Summary: 0 passed, 1 failed
--- tests/qtest/acpi-bits/bits-tests/testacpi.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tests/qtest/acpi-bits/bits-tests/testacpi.py +2 -2
@@ -36,8 +36,8 @@ def register_tests(): testsuite.add_test("ACPI _MAT (Multiple APIC Table Entry) under Processor objects", test_mat, submenu="ACPI Tests") - testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", test_pss, submenu="ACPI Tests") - testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, submenu="ACPI Tests") +# testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", test_pss, submenu="ACPI Tests") +# testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, submenu="ACPI Tests") testsuite.add_test("ACPI DSDT (Differentiated System Description Table)", test_dsdt, submenu="ACPI Tests") testsuite.add_test("ACPI FACP (Fixed ACPI Description Table)", test_facp, submenu="ACPI Tests") testsuite.add_test("ACPI HPET (High Precision Event Timer Table)", test_hpet, submenu="ACPI Tests")
Signature · 1 line
2.25.1
PATCH 6/12 acpi/tests/bits: add smilatency test suite from bits in order to disable it
In reply to Ani Sinhasmilatency tests does not reliably pass every time it is run from QEMU. This change adds the test file unchanged from bits so that the next change can disable the test.
--- tests/qtest/acpi-bits/bits-tests/meson.build | 2 +- .../qtest/acpi-bits/bits-tests/smilatency.py | 102 ++++++++++++++++++ 2 files changed, 103 insertions(+), 1 deletion(-) create mode 100644 tests/qtest/acpi-bits/bits-tests/smilatency.py
tests/qtest/acpi-bits/bits-tests/meson.build +1 -1
@@ -1,4 +1,4 @@ -test_files = ['smbios.py', 'testacpi.py', 'testcpuid.py'] +test_files = ['smbios.py', 'smilatency.py', 'testacpi.py', 'testcpuid.py'] copytestfiles = custom_target('copy test files', input : test_files,
tests/qtest/acpi-bits/bits-tests/smilatency.py new file +102 -0
102 lines not on this page · raw message
Signature · 1 line
2.25.1
PATCH 7/12 acpi/tests/bits: disable smilatency test since it does not pass everytime
In reply to Ani Sinhasmilatency test is latency sensitive and does not pass deterministically when run in QEMU environment under biosbits. Disable the test suite for now.
Example failure:
==== SMI latency test ====
Warning: touching the keyboard can affect the results of this test. Starting test. Wait here, I will be back in 15 seconds. [assert] SMI latency < 150us to minimize risk of OS timeouts FAIL
1us < t <= 10us; average = 1372ns; count = 10912449
Times between first few observations: 176us 1646ns 1441ns 1450ns 1462ns
10us < t <= 100us; average = 16us; count = 1187
Times between first few observations: 15ms 3148us 5856us 49ms 33ms
100us < t <= 1ms; average = 259us; count = 8
Times between first few observations: 111ms 2227ms 1779ms 999ms 219ms
0 SMI detected using MSR_SMI_COUNT (MSR 0x34)
Summary of impact: observed maximum latency = 298us
Summary: 0 passed, 1 failed
--- tests/qtest/acpi-bits/bits-tests/smilatency.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tests/qtest/acpi-bits/bits-tests/smilatency.py +3 -2
@@ -33,8 +33,9 @@ import usb def register_tests(): - testsuite.add_test("SMI latency test", smi_latency); - testsuite.add_test("SMI latency test with USB disabled via BIOS handoff", test_with_usb_disabled, runall=False); + pass + # testsuite.add_test("SMI latency test", smi_latency); + # testsuite.add_test("SMI latency test with USB disabled via BIOS handoff", test_with_usb_disabled, runall=False); def smi_latency(): MSR_SMI_COUNT = 0x34
Signature · 1 line
2.25.1
PATCH 8/12 acpi/tests/bits: add biosbits config file for running bios tests
In reply to Ani SinhaThis change adds initial biosbits config file that instructs biosbits to run bios test suits in batch mode. Additionally acpi and smbios structures are also dumped.
--- tests/qtest/acpi-bits/bits-config/bits-cfg.txt | 18 ++++++++++++++++++ tests/qtest/acpi-bits/bits-config/meson.build | 11 +++++++++++ 2 files changed, 29 insertions(+) create mode 100644 tests/qtest/acpi-bits/bits-config/bits-cfg.txt create mode 100644 tests/qtest/acpi-bits/bits-config/meson.build
tests/qtest/acpi-bits/bits-config/bits-cfg.txt new file +18 -0
@@ -0,0 +1,18 @@ +# BITS configuration file +[bits] + +# To run BITS in batch mode, set batch to a list of one or more of the +# following keywords; BITS will then run all of the requested operations, then +# save the log file to disk. +# +# test: Run the full BITS testsuite. +# acpi: Dump all ACPI structures. +# smbios: Dump all SMBIOS structures. +# +# Leave batch set to an empty string to disable batch mode. +# batch = + +# Uncomment the following to run all available batch operations +# please take a look at boot/python/init.py in bits zip file +# to see how these options are parsed and used. +batch = test acpi smbios
tests/qtest/acpi-bits/bits-config/meson.build new file +11 -0
@@ -0,0 +1,11 @@ +config_files = ['bits-cfg.txt'] + +copycfgfiles = custom_target('copy cfg files', + input : config_files, + output : config_files, + command : ['cp', '@INPUT@', '@OUTPUT@'], + install : true, + install_dir : 'bits-config', + build_by_default : true) + +other_deps += copycfgfiles
Signature · 1 line
2.25.1
PATCH 9/12 acpi/tests/bits: add acpi and smbios python tests that uses biosbits
In reply to Ani SinhaThis change adds python based qtest framework that can be used to run qtests from within a virtual environment. A bash script creates the virtual environment and then runs the python based tests from within that environment. All dependent python packages are installed in the virtual environment using pip module. QEMU python test modules are also available in the environment for spawning the QEMU based VMs.
It also introduces QEMU acpi/smbios biosbits python test script which is run from within the python virtual environment.
--- tests/qtest/acpi-bits/acpi-bits-test-venv.sh | 59 ++++ tests/qtest/acpi-bits/acpi-bits-test.py | 327 +++++++++++++++++++ tests/qtest/acpi-bits/meson.build | 39 +++ tests/qtest/acpi-bits/requirements.txt | 1 + 4 files changed, 426 insertions(+) create mode 100644 tests/qtest/acpi-bits/acpi-bits-test-venv.sh create mode 100644 tests/qtest/acpi-bits/acpi-bits-test.py create mode 100644 tests/qtest/acpi-bits/meson.build create mode 100644 tests/qtest/acpi-bits/requirements.txt
tests/qtest/acpi-bits/acpi-bits-test-venv.sh new file +59 -0
@@ -0,0 +1,59 @@ +#!/usr/bin/env bash +# Generates a python virtual environment for the test to run. +# Then runs python test scripts from within that virtual environment. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# +# Author: Ani Sinha <[email protected]> + +set -e + +MYPATH=$(realpath ${BASH_SOURCE:-$0}) +MYDIR=$(dirname $MYPATH) + +if [ -z "$QTEST_SOURCE_ROOT" ]; then + echo -n "Please set QTEST_SOURCE_ROOT env pointing" + echo " to the root of the qemu source tree." + echo -n "This is required so that the test can find the " + echo "python modules that it needs for execution." + exit 1 +fi +SRCDIR=$QTEST_SOURCE_ROOT +TESTSCRIPTS=("acpi-bits-test.py") +PIPCMD="-m pip -q --disable-pip-version-check" +# we need to save the old value of PWD before we do a change-dir later +QTEST_PWD=$PWD + +TESTS_PYTHON=/usr/bin/python3 +TESTS_VENV_REQ=requirements.txt + +# sadly for pip -e and -t options do not work together. +# please see https://github.com/pypa/pip/issues/562 +cd $MYDIR + +$TESTS_PYTHON -m venv . +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ +[ -f $TESTS_VENV_REQ ] && \ + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ + +# venv is activated at this point. + +# run the test +for testscript in ${TESTSCRIPTS[@]} ; do + export QTEST_PWD; python3 $testscript +done + +cd $QTEST_PWD + +exit 0
tests/qtest/acpi-bits/acpi-bits-test.py new file +327 -0
327 lines not on this page · raw message
tests/qtest/acpi-bits/meson.build new file +39 -0
@@ -0,0 +1,39 @@ +xorriso = find_program('xorriso', required: true) +if not xorriso.found() + message('xorriso not found ... disabled bits acpi tests.') + subdir_done() +endif + +# biosbits qtests are currenly only supported on x86_64 platforms. +qtests_x86_64 += ['acpi-bits-test'] +qtests += {'acpi-bits-test' : files('acpi-bits-test.py'),} +slow_qtests += {'acpi-bits-test': 120, } + +subdir('prebuilt') +subdir('bits-tests') +subdir('bits-config') + +test_files = ['acpi-bits-test.py'] +requirements = 'requirements.txt' + +copytestfiles = custom_target('copy test files', + input : test_files, + output : test_files, + command : ['cp', '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) + +requirementsfiles = custom_target('copy py req files', + input : requirements, + output : requirements, + command : ['cp', '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) + +other_deps += [copytestfiles,requirementsfiles] + +qtest_executables += { + 'acpi-bits-test': configure_file(copy:true, + input:'acpi-bits-test-venv.sh', + output:'acpi-bits-test') +}
tests/qtest/acpi-bits/requirements.txt new file +1 -0
@@ -0,0 +1 @@ +tap.py
Signature · 1 line
2.25.1
Re: [PATCH 09/12] acpi/tests/bits: add acpi and smbios python tests that uses biosbits
Quoted 183 lines · Ani Sinha
This change adds python based qtest framework that can be used to run qtests from within a virtual environment. A bash script creates the virtual environment and then runs the python based tests from within that environment. All dependent python packages are installed in the virtual environment using pip module. QEMU python test modules are also available in the environment for spawning the QEMU based VMs.
It also introduces QEMU acpi/smbios biosbits python test script which is run from within the python virtual environment.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/acpi-bits-test-venv.sh | 59 ++++ tests/qtest/acpi-bits/acpi-bits-test.py | 327 +++++++++++++++++++ tests/qtest/acpi-bits/meson.build | 39 +++ tests/qtest/acpi-bits/requirements.txt | 1 + 4 files changed, 426 insertions(+)create mode 100644
tests/qtest/acpi-bits/acpi-bits-test-venv.shcreate mode 100644tests/qtest/acpi-bits/acpi-bits-test.pycreate mode 100644 tests/qtest/acpi-bits/meson.build create mode 100644tests/qtest/acpi-bits/requirements.txttests/qtest/acpi-bits/acpi-bits-test-venv.sh new file +59
-0@@ -0,0 +1,59 @@ +#!/usr/bin/env bash +# Generates a python virtual environment for the test to run. +# Then runs python test scripts from within that virtual environment. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# +# Author: Ani Sinha <[email protected]> + +set -e + +MYPATH=$(realpath ${BASH_SOURCE:-$0}) +MYDIR=$(dirname $MYPATH) + +if [ -z "$QTEST_SOURCE_ROOT" ]; then + echo -n "Please set QTEST_SOURCE_ROOT env pointing" + echo " to the root of the qemu source tree." + echo -n "This is required so that the test can find the " + echo "python modules that it needs for execution." + exit 1 +fi +SRCDIR=$QTEST_SOURCE_ROOT +TESTSCRIPTS=("acpi-bits-test.py") +PIPCMD="-m pip -q --disable-pip-version-check" +# we need to save the old value of PWD before we do a change-dir later +QTEST_PWD=$PWD + +TESTS_PYTHON=/usr/bin/python3 +TESTS_VENV_REQ=requirements.txt + +# sadly for pip -e and -t options do not work together. +# please see https://github.com/pypa/pip/issues/562 +cd $MYDIR + +$TESTS_PYTHON -m venv . +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ +[ -f $TESTS_VENV_REQ ] && \ + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ + +# venv is activated at this point. + +# run the test +for testscript in ${TESTSCRIPTS[@]} ; do + export QTEST_PWD; python3 $testscript +done + +cd $QTEST_PWD + +exit 0tests/qtest/acpi-bits/acpi-bits-test.py new file +90
-0@@ -0,0 +1,327 @@ +#!/usr/bin/env python3 +# group: rw quick +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, +# https://biosbits.org/ +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# +# Some parts are slightly taken from qtest.py and iotests.py +# +# Authors: +# Ani Sinha <[email protected]> + +# pylint: disable=invalid-name + +""" +QEMU bios tests using biosbits available at +https://biosbits.org/. +""" + +import logging +import os +import re +import shutil +import subprocess +import sys +import tarfile +import tempfile +import time +import unittest +import zipfile +from typing import ( + List, + Optional, + Sequence, +) +from tap import TAPTestRunner +from qemu.machine import QEMUMachine + +QTESTQEMUPROG = os.getenv('QTEST_QEMU_BINARY') +QTEST_PWD = os.getenv('QTEST_PWD') + +def get_arch(): + """finds the arch from the qemu binary name""" + match = re.search('.*qemu-system-(.*)', QTESTQEMUPROG) + if match: + return match.group(1) + return 'x86_64' + +ARCH = get_arch() + +class QEMUBitsMachine(QEMUMachine): + """ + A QEMU VM, with isa-debugcon enabled and bits iso passed + using -cdrom to QEMU commandline. + """ + def __init__(self, + binary: str, + args: Sequence[str] = (), + wrapper: Sequence[str] = (), + name: Optional[str] = None, + base_temp_dir: str = "/var/tmp", + debugcon_log: str = "debugcon-log.txt", + debugcon_addr: str = "0x403", + sock_dir: Optional[str] = None, + qmp_timer: Optional[float] = None): + # pylint: disable=too-many-arguments + + if name is None: + name = "qemu-bits-%d" % os.getpid() + if sock_dir is None: + sock_dir = base_temp_dir + super().__init__(binary, args, wrapper=wrapper, name=name, + base_temp_dir=base_temp_dir, + sock_dir=sock_dir, qmp_timer=qmp_timer) + self.debugcon_log = debugcon_log + self.debugcon_addr = debugcon_addr + self.base_temp_dir = base_temp_dir + + @property
tests/qtest/acpi-bits/acpi-bits-test.py line 100
Quoted 10 lines
tests/qtest/acpi-bits/acpi-bits-test.py new file +10
-0+ def _base_args(self) -> List[str]: + args = super()._base_args + args.extend([ + '-chardev', + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, + self.debugcon_log), + '-device', + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr, + ]) + return args
So is this patch series *really* related to qtests? qtests are using a special "accelerator" mode of QEMU where it gets launched with the "-qtest" parameter. I can't see that you're using "-qtest" here or anywhere else, so this rather looks like another framework to me to run python-based QEMU tests (similar to the avocado tests).
Thus if this is really not related to qtests, may I suggest to move this into another folder instead? Maybe tests/pytests/acpi-bits or something similar?
Thomas
Re: [PATCH 09/12] acpi/tests/bits: add acpi and smbios python tests that uses biosbits
Quoted 193 lines · Thomas Huth
Quoted 191 lines · Ani Sinha
This change adds python based qtest framework that can be used to run qtests from within a virtual environment. A bash script creates the virtual environment and then runs the python based tests from within that environment. All dependent python packages are installed in the virtual environment using pip module. QEMU python test modules are also available in the environment for spawning the QEMU based VMs.
It also introduces QEMU acpi/smbios biosbits python test script which is run from within the python virtual environment.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/acpi-bits-test-venv.sh | 59 ++++ tests/qtest/acpi-bits/acpi-bits-test.py | 327 +++++++++++++++++++ tests/qtest/acpi-bits/meson.build | 39 +++ tests/qtest/acpi-bits/requirements.txt | 1 + 4 files changed, 426 insertions(+)create mode 100644
tests/qtest/acpi-bits/acpi-bits-test-venv.shcreate mode 100644tests/qtest/acpi-bits/acpi-bits-test.pycreate mode 100644 tests/qtest/acpi-bits/meson.build create mode 100644tests/qtest/acpi-bits/requirements.txttests/qtest/acpi-bits/acpi-bits-test-venv.sh new file +59
-0@@ -0,0 +1,59 @@ +#!/usr/bin/env bash +# Generates a python virtual environment for the test to run. +# Then runs python test scripts from within that virtual environment. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# +# Author: Ani Sinha <[email protected]> + +set -e + +MYPATH=$(realpath ${BASH_SOURCE:-$0}) +MYDIR=$(dirname $MYPATH) + +if [ -z "$QTEST_SOURCE_ROOT" ]; then + echo -n "Please set QTEST_SOURCE_ROOT env pointing" + echo " to the root of the qemu source tree." + echo -n "This is required so that the test can find the " + echo "python modules that it needs for execution." + exit 1 +fi +SRCDIR=$QTEST_SOURCE_ROOT +TESTSCRIPTS=("acpi-bits-test.py") +PIPCMD="-m pip -q --disable-pip-version-check" +# we need to save the old value of PWD before we do a change-dir later +QTEST_PWD=$PWD + +TESTS_PYTHON=/usr/bin/python3 +TESTS_VENV_REQ=requirements.txt + +# sadly for pip -e and -t options do not work together. +# please see https://github.com/pypa/pip/issues/562 +cd $MYDIR + +$TESTS_PYTHON -m venv . +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ +[ -f $TESTS_VENV_REQ ] && \ + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ + +# venv is activated at this point. + +# run the test +for testscript in ${TESTSCRIPTS[@]} ; do + export QTEST_PWD; python3 $testscript +done + +cd $QTEST_PWD + +exit 0tests/qtest/acpi-bits/acpi-bits-test.py new file +98
-0@@ -0,0 +1,327 @@ +#!/usr/bin/env python3 +# group: rw quick +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, +# https://biosbits.org/ +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# +# Some parts are slightly taken from qtest.py and iotests.py +# +# Authors: +# Ani Sinha <[email protected]> + +# pylint: disable=invalid-name + +""" +QEMU bios tests using biosbits available at +https://biosbits.org/. +""" + +import logging +import os +import re +import shutil +import subprocess +import sys +import tarfile +import tempfile +import time +import unittest +import zipfile +from typing import ( + List, + Optional, + Sequence, +) +from tap import TAPTestRunner +from qemu.machine import QEMUMachine + +QTESTQEMUPROG = os.getenv('QTEST_QEMU_BINARY') +QTEST_PWD = os.getenv('QTEST_PWD') + +def get_arch(): + """finds the arch from the qemu binary name""" + match = re.search('.*qemu-system-(.*)', QTESTQEMUPROG) + if match: + return match.group(1) + return 'x86_64' + +ARCH = get_arch() + +class QEMUBitsMachine(QEMUMachine): + """ + A QEMU VM, with isa-debugcon enabled and bits iso passed + using -cdrom to QEMU commandline. + """ + def __init__(self, + binary: str, + args: Sequence[str] = (), + wrapper: Sequence[str] = (), + name: Optional[str] = None, + base_temp_dir: str = "/var/tmp", + debugcon_log: str = "debugcon-log.txt", + debugcon_addr: str = "0x403", + sock_dir: Optional[str] = None, + qmp_timer: Optional[float] = None): + # pylint: disable=too-many-arguments + + if name is None: + name = "qemu-bits-%d" % os.getpid() + if sock_dir is None: + sock_dir = base_temp_dir + super().__init__(binary, args, wrapper=wrapper, name=name, + base_temp_dir=base_temp_dir, + sock_dir=sock_dir, qmp_timer=qmp_timer) + self.debugcon_log = debugcon_log + self.debugcon_addr = debugcon_addr + self.base_temp_dir = base_temp_dir + + @property + def _base_args(self) -> List[str]: + args = super()._base_args + args.extend([ + '-chardev', + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, + self.debugcon_log), + '-device', + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr,
Quoted 8 lines
tests/qtest/acpi-bits/acpi-bits-test.py line 100
Quoted 2 lines
tests/qtest/acpi-bits/acpi-bits-test.py new file +2
-0+ ]) + return argsSo is this patch series *really* related to qtests? qtests are using a special "accelerator" mode of QEMU where it gets launched with the "-qtest" parameter. I can't see that you're using "-qtest" here or anywhere else, so this rather looks like another framework to me to run python-based QEMU tests (similar to the avocado tests).
yes you are right. This does not use or need the qtest accelerator because we are not inspecting the guest memory for anything.
Quoted 3 lines
Thus if this is really not related to qtests, may I suggest to move this into another folder instead? Maybe tests/pytests/acpi-bits or something similar?
The problem I faced with this test is that it does not quite fall into the qtest category. Nor does it fall into the integration test category. I asked Igor and he suggested I use the qtest framework. Should we invent a new class of tests then? How many such tests are we going to have in the future? Lets see what others think.
Quoted 3 lines
Thomas
Re: [PATCH 09/12] acpi/tests/bits: add acpi and smbios python tests that uses biosbits
Quoted 207 lines · Ani Sinha
Quoted 193 lines · Thomas Huth
Quoted 191 lines · Ani Sinha
This change adds python based qtest framework that can be used to run qtests from within a virtual environment. A bash script creates the virtual environment and then runs the python based tests from within that environment. All dependent python packages are installed in the virtual environment using pip module. QEMU python test modules are also available in the environment for spawning the QEMU based VMs.
It also introduces QEMU acpi/smbios biosbits python test script which is run from within the python virtual environment.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/acpi-bits-test-venv.sh | 59 ++++ tests/qtest/acpi-bits/acpi-bits-test.py | 327 +++++++++++++++++++ tests/qtest/acpi-bits/meson.build | 39 +++ tests/qtest/acpi-bits/requirements.txt | 1 + 4 files changed, 426 insertions(+)create mode 100644
tests/qtest/acpi-bits/acpi-bits-test-venv.shcreate mode 100644tests/qtest/acpi-bits/acpi-bits-test.pycreate mode 100644 tests/qtest/acpi-bits/meson.build create mode 100644tests/qtest/acpi-bits/requirements.txttests/qtest/acpi-bits/acpi-bits-test-venv.sh new file +59
-0@@ -0,0 +1,59 @@ +#!/usr/bin/env bash +# Generates a python virtual environment for the test to run. +# Then runs python test scripts from within that virtual environment. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# +# Author: Ani Sinha <[email protected]> + +set -e + +MYPATH=$(realpath ${BASH_SOURCE:-$0}) +MYDIR=$(dirname $MYPATH) + +if [ -z "$QTEST_SOURCE_ROOT" ]; then + echo -n "Please set QTEST_SOURCE_ROOT env pointing" + echo " to the root of the qemu source tree." + echo -n "This is required so that the test can find the " + echo "python modules that it needs for execution." + exit 1 +fi +SRCDIR=$QTEST_SOURCE_ROOT +TESTSCRIPTS=("acpi-bits-test.py") +PIPCMD="-m pip -q --disable-pip-version-check" +# we need to save the old value of PWD before we do a change-dir later +QTEST_PWD=$PWD + +TESTS_PYTHON=/usr/bin/python3 +TESTS_VENV_REQ=requirements.txt + +# sadly for pip -e and -t options do not work together. +# please see https://github.com/pypa/pip/issues/562 +cd $MYDIR + +$TESTS_PYTHON -m venv . +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ +[ -f $TESTS_VENV_REQ ] && \ + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ + +# venv is activated at this point. + +# run the test +for testscript in ${TESTSCRIPTS[@]} ; do + export QTEST_PWD; python3 $testscript +done + +cd $QTEST_PWD + +exit 0tests/qtest/acpi-bits/acpi-bits-test.py new file +98
-0@@ -0,0 +1,327 @@ +#!/usr/bin/env python3 +# group: rw quick +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, +# https://biosbits.org/ +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# +# Some parts are slightly taken from qtest.py and iotests.py +# +# Authors: +# Ani Sinha <[email protected]> + +# pylint: disable=invalid-name + +""" +QEMU bios tests using biosbits available at +https://biosbits.org/. +""" + +import logging +import os +import re +import shutil +import subprocess +import sys +import tarfile +import tempfile +import time +import unittest +import zipfile +from typing import ( + List, + Optional, + Sequence, +) +from tap import TAPTestRunner +from qemu.machine import QEMUMachine + +QTESTQEMUPROG = os.getenv('QTEST_QEMU_BINARY') +QTEST_PWD = os.getenv('QTEST_PWD') + +def get_arch(): + """finds the arch from the qemu binary name""" + match = re.search('.*qemu-system-(.*)', QTESTQEMUPROG) + if match: + return match.group(1) + return 'x86_64' + +ARCH = get_arch() + +class QEMUBitsMachine(QEMUMachine): + """ + A QEMU VM, with isa-debugcon enabled and bits iso passed + using -cdrom to QEMU commandline. + """ + def __init__(self, + binary: str, + args: Sequence[str] = (), + wrapper: Sequence[str] = (), + name: Optional[str] = None, + base_temp_dir: str = "/var/tmp", + debugcon_log: str = "debugcon-log.txt", + debugcon_addr: str = "0x403", + sock_dir: Optional[str] = None, + qmp_timer: Optional[float] = None): + # pylint: disable=too-many-arguments + + if name is None: + name = "qemu-bits-%d" % os.getpid() + if sock_dir is None: + sock_dir = base_temp_dir + super().__init__(binary, args, wrapper=wrapper, name=name, + base_temp_dir=base_temp_dir, + sock_dir=sock_dir, qmp_timer=qmp_timer) + self.debugcon_log = debugcon_log + self.debugcon_addr = debugcon_addr + self.base_temp_dir = base_temp_dir + + @property + def _base_args(self) -> List[str]: + args = super()._base_args + args.extend([ + '-chardev', + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, + self.debugcon_log), + '-device', + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr,Quoted 8 lines
tests/qtest/acpi-bits/acpi-bits-test.py line 100
Quoted 2 lines
tests/qtest/acpi-bits/acpi-bits-test.py new file +2
-0+ ]) + return argsSo is this patch series *really* related to qtests? qtests are using a special "accelerator" mode of QEMU where it gets launched with the "-qtest" parameter. I can't see that you're using "-qtest" here or anywhere else, so this rather looks like another framework to me to run python-based QEMU tests (similar to the avocado tests).
yes you are right. This does not use or need the qtest accelerator because we are not inspecting the guest memory for anything.
Quoted 1 line
Quoted 6 lines
Quoted 2 lines
Thus if this is really not related to qtests, may I suggest to move this into another folder instead? Maybe tests/pytests/acpi-bits or something similar?
The problem I faced with this test is that it does not quite fall into the qtest category. Nor does it fall into the integration test category. I asked Igor and he suggested I use the qtest framework.
My python-foo really lacks, so putting python stuff under tests/qtest where I'm the maintainer is a really bad idea ;-)
Quoted 2 lines
Should we invent a new class of tests then? How many such tests are we going to have in the future? Lets see what others think.
I think we want to have another folder under tests/ for python-based tests that do not use the avocado framework. John is working on better python bindings, and since we switched almost all other tests to the meson test runner now, the avocado-based tests (with their own test runner) rather feel like an oddball in the qemu test suite now, so I asume we will have more python-based tests in the future which will work without avocado.
The difficult question is - as always - to come up with a good name...
- tests/pytest/
- tests/integration/
- tests/pyqmygoose/
- ...
... I don't have a real good clue, as you can see ;-)
Thomas
Re: [PATCH 09/12] acpi/tests/bits: add acpi and smbios python tests that uses biosbits
In reply to Ani SinhaQuoted 203 lines · Ani Sinha
Quoted 193 lines · Thomas Huth
Quoted 191 lines · Ani Sinha
This change adds python based qtest framework that can be used to run qtests from within a virtual environment. A bash script creates the virtual environment and then runs the python based tests from within that environment. All dependent python packages are installed in the virtual environment using pip module. QEMU python test modules are also available in the environment for spawning the QEMU based VMs.
It also introduces QEMU acpi/smbios biosbits python test script which is run from within the python virtual environment.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/acpi-bits-test-venv.sh | 59 ++++ tests/qtest/acpi-bits/acpi-bits-test.py | 327 +++++++++++++++++++ tests/qtest/acpi-bits/meson.build | 39 +++ tests/qtest/acpi-bits/requirements.txt | 1 + 4 files changed, 426 insertions(+)create mode 100644
tests/qtest/acpi-bits/acpi-bits-test-venv.shcreate mode 100644tests/qtest/acpi-bits/acpi-bits-test.pycreate mode 100644 tests/qtest/acpi-bits/meson.build create mode 100644tests/qtest/acpi-bits/requirements.txttests/qtest/acpi-bits/acpi-bits-test-venv.sh new file +59
-0@@ -0,0 +1,59 @@ +#!/usr/bin/env bash +# Generates a python virtual environment for the test to run. +# Then runs python test scripts from within that virtual environment. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# +# Author: Ani Sinha <[email protected]> + +set -e + +MYPATH=$(realpath ${BASH_SOURCE:-$0}) +MYDIR=$(dirname $MYPATH) + +if [ -z "$QTEST_SOURCE_ROOT" ]; then + echo -n "Please set QTEST_SOURCE_ROOT env pointing" + echo " to the root of the qemu source tree." + echo -n "This is required so that the test can find the " + echo "python modules that it needs for execution." + exit 1 +fi +SRCDIR=$QTEST_SOURCE_ROOT +TESTSCRIPTS=("acpi-bits-test.py") +PIPCMD="-m pip -q --disable-pip-version-check" +# we need to save the old value of PWD before we do a change-dir later +QTEST_PWD=$PWD + +TESTS_PYTHON=/usr/bin/python3 +TESTS_VENV_REQ=requirements.txt + +# sadly for pip -e and -t options do not work together. +# please see https://github.com/pypa/pip/issues/562 +cd $MYDIR + +$TESTS_PYTHON -m venv . +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ +[ -f $TESTS_VENV_REQ ] && \ + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ + +# venv is activated at this point. + +# run the test +for testscript in ${TESTSCRIPTS[@]} ; do + export QTEST_PWD; python3 $testscript +done + +cd $QTEST_PWD + +exit 0tests/qtest/acpi-bits/acpi-bits-test.py new file +98
-0@@ -0,0 +1,327 @@ +#!/usr/bin/env python3 +# group: rw quick +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, +# https://biosbits.org/ +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# +# Some parts are slightly taken from qtest.py and iotests.py +# +# Authors: +# Ani Sinha <[email protected]> + +# pylint: disable=invalid-name + +""" +QEMU bios tests using biosbits available at +https://biosbits.org/. +""" + +import logging +import os +import re +import shutil +import subprocess +import sys +import tarfile +import tempfile +import time +import unittest +import zipfile +from typing import ( + List, + Optional, + Sequence, +) +from tap import TAPTestRunner +from qemu.machine import QEMUMachine + +QTESTQEMUPROG = os.getenv('QTEST_QEMU_BINARY') +QTEST_PWD = os.getenv('QTEST_PWD') + +def get_arch(): + """finds the arch from the qemu binary name""" + match = re.search('.*qemu-system-(.*)', QTESTQEMUPROG) + if match: + return match.group(1) + return 'x86_64' + +ARCH = get_arch() + +class QEMUBitsMachine(QEMUMachine): + """ + A QEMU VM, with isa-debugcon enabled and bits iso passed + using -cdrom to QEMU commandline. + """ + def __init__(self, + binary: str, + args: Sequence[str] = (), + wrapper: Sequence[str] = (), + name: Optional[str] = None, + base_temp_dir: str = "/var/tmp", + debugcon_log: str = "debugcon-log.txt", + debugcon_addr: str = "0x403", + sock_dir: Optional[str] = None, + qmp_timer: Optional[float] = None): + # pylint: disable=too-many-arguments + + if name is None: + name = "qemu-bits-%d" % os.getpid() + if sock_dir is None: + sock_dir = base_temp_dir + super().__init__(binary, args, wrapper=wrapper, name=name, + base_temp_dir=base_temp_dir, + sock_dir=sock_dir, qmp_timer=qmp_timer) + self.debugcon_log = debugcon_log + self.debugcon_addr = debugcon_addr + self.base_temp_dir = base_temp_dir + + @property + def _base_args(self) -> List[str]: + args = super()._base_args + args.extend([ + '-chardev', + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, + self.debugcon_log), + '-device', + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr,Quoted 8 lines
tests/qtest/acpi-bits/acpi-bits-test.py line 100
Quoted 2 lines
tests/qtest/acpi-bits/acpi-bits-test.py new file +2
-0+ ]) + return argsSo is this patch series *really* related to qtests? qtests are using a special "accelerator" mode of QEMU where it gets launched with the "-qtest" parameter. I can't see that you're using "-qtest" here or anywhere else, so this rather looks like another framework to me to run python-based QEMU tests (similar to the avocado tests).
Quoted 2 lines
yes you are right. This does not use or need the qtest accelerator because we are not inspecting the guest memory for anything.
Same is true for e.g. ./tests/qtest/bios-tables-test.c ...
Quoted 13 lines
Quoted 3 lines
Thus if this is really not related to qtests, may I suggest to move this into another folder instead? Maybe tests/pytests/acpi-bits or something similar?
The problem I faced with this test is that it does not quite fall into the qtest category. Nor does it fall into the integration test category. I asked Igor and he suggested I use the qtest framework. Should we invent a new class of tests then? How many such tests are we going to have in the future? Lets see what others think.
Quoted 3 lines
Thomas
Re: [PATCH 09/12] acpi/tests/bits: add acpi and smbios python tests that uses biosbits
Quoted 16 lines · Michael S. Tsirkin
Quoted 15 lines · Ani Sinha
Quoted 14 lines · Thomas Huth
Quoted 12 lines · Ani Sinha
This change adds python based qtest framework that can be used to run qtests from within a virtual environment. A bash script creates the virtual environment and then runs the python based tests from within that environment. All dependent python packages are installed in the virtual environment using pip module. QEMU python test modules are also available in the environment for spawning the QEMU based VMs.
It also introduces QEMU acpi/smbios biosbits python test script which is run from within the python virtual environment.
Signed-off-by Ani Sinha <[email protected]>---
[...]
Quoted 46 lines
Quoted 46 lines
Quoted 45 lines
Quoted 29 lines
tests/qtest/acpi-bits/acpi-bits-test.py new file +29
-0+class QEMUBitsMachine(QEMUMachine): + """ + A QEMU VM, with isa-debugcon enabled and bits iso passed + using -cdrom to QEMU commandline. + """ + def __init__(self, + binary: str, + args: Sequence[str] = (), + wrapper: Sequence[str] = (), + name: Optional[str] = None, + base_temp_dir: str = "/var/tmp", + debugcon_log: str = "debugcon-log.txt", + debugcon_addr: str = "0x403", + sock_dir: Optional[str] = None, + qmp_timer: Optional[float] = None): + # pylint: disable=too-many-arguments + + if name is None: + name = "qemu-bits-%d" % os.getpid() + if sock_dir is None: + sock_dir = base_temp_dir + super().__init__(binary, args, wrapper=wrapper, name=name, + base_temp_dir=base_temp_dir, + sock_dir=sock_dir, qmp_timer=qmp_timer) + self.debugcon_log = debugcon_log + self.debugcon_addr = debugcon_addr + self.base_temp_dir = base_temp_dir + + @propertytests/qtest/acpi-bits/acpi-bits-test.py line 100
Quoted 10 lines
tests/qtest/acpi-bits/acpi-bits-test.py new file +10
-0+ def _base_args(self) -> List[str]: + args = super()._base_args + args.extend([ + '-chardev', + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, + self.debugcon_log), + '-device', + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr, + ]) + return argsSo is this patch series *really* related to qtests? qtests are using a special "accelerator" mode of QEMU where it gets launched with the "-qtest" parameter. I can't see that you're using "-qtest" here or anywhere else, so this rather looks like another framework to me to run python-based QEMU tests (similar to the avocado tests).
Quoted 4 lines
Quoted 2 lines
yes you are right. This does not use or need the qtest accelerator because we are not inspecting the guest memory for anything.
Same is true for e.g.
./tests/qtest/bios-tables-test.c...
No, bios-tables-test still uses functions like qtest_memread() and qtest_readb(), so it still uses the qtest framework, even if it's actually running code via TCG.
Thomas
PATCH 10/12 acpi/tests/bits: add acpi bits qtest directory in meson for running tests
In reply to Ani Sinhaadded acpi-bits subdirectory in meson.build so that the tests in that subdirectory can be run as a part of make check.
--- tests/qtest/acpi-bits/acpi-bits-test.py | 4 ++-- tests/qtest/meson.build | 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-)
tests/qtest/acpi-bits/acpi-bits-test.py +2 -2
@@ -244,9 +244,9 @@ def parse_log(self): if os.getenv('V'): print('\nlogs from biosbits follows:') - print('==========================================\n\n') + print('==========================================\n') print(log) - print('\n==========================================\n') + print('==========================================\n') matchiter = re.finditer(r'(.*Summary: )(\d+ passed), (\d+ failed).*', log)
tests/qtest/meson.build +2 -0
@@ -312,6 +312,8 @@ endif qtest_executables = {} other_deps = [] +subdir('acpi-bits') + foreach dir : target_dirs if not dir.endswith('-softmmu') continue
Signature · 1 line
2.25.1
PATCH 11/12 acpi/tests/bits: add README file for bits qtests
In reply to Ani SinhaThe README file is added describing the directory structure and the purpose of every file it contains. It also describes how to add new tests, make changes to existing tests or bits config files or regenerate the bits software.
--- tests/qtest/acpi-bits/README | 168 +++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 tests/qtest/acpi-bits/README
tests/qtest/acpi-bits/README new file +168 -0
@@ -0,0 +1,168 @@ +============================================================================= +ACPI/SMBIOS QTESTS USING BIOSBITS +============================================================================= + +Biosbits is a software written by Josh Triplett that can be downloaded by +visiting https://biosbits.org/. The github codebase can be found here: +https://github.com/biosbits/bits/tree/master. It is a software that exercizes +the bios components such as acpi and smbios tables directly through acpica +bios interpreter (a freely available C based library written by Intel, +downloadable from https://acpica.org/ and is included with biosbits) without an +operating system getting involved in between. +There are several advantages to directly testing the bios in a real physical +machine or VM as opposed to indirectly discovering bios issues through the +operating system. For one thing, the OSes tend to hide bios problems from the +end user. The other is that we have more control of what we wanted to test +and how by directly using acpica interpreter on top of the bios on a running +system. More details on the inspiration for developing biosbits and its real +life uses can be found in (a) and (b). +This directory contains QEMU qtests written in python that exercizes the QEMU +bios components using biosbits and reports test failures. + +These tests use python virtual environment. In debian/ubuntu system, the tests +would require python3.8-venv and python3-pip packages to be installed. + +A brief description of the contents of this directory follows: + +├── acpi-bits-test.py +├── acpi-bits-test-venv.sh +├── bits-config +│ ├── bits-cfg.txt +│ └── meson.build +├── bits-tests +│ ├── meson.build +│ ├── smbios.py +│ ├── smilatency.py +│ ├── testacpi.py +│ └── testcpuid.py +├── meson.build +├── prebuilt +│ ├── bits-2095-grub.tar.gz +│ ├── bits-2095.zip +│ └── meson.build +├── README +└── requirements.txt + +acpi-bits: + - acpi-bits-test-venv.sh: This is a shell script that sets up the virtual + environment necessary for the actual python test script to run. Amongst + other things, it makes sure that QEMU python library is available within + that venv so that QEMU machines can be forked. QEMU python library can be + found within python/ directory in QEMU source. + After setting up the virtual environment, it runs the python test script + from within that environment. + If you want to enable verbose mode only for bits test and run make check, + one trick is to add V=1 before the call to execute the python script in + this file. + - acpi-bits-test.py: This is the main python test script that generates a + biosbits iso. It then spawns a QEMU VM with it, collects the logs and reports + test failures. This is the script one would be interested in if they wanted + to add or change some component of the log parsing, add a new commandline to + how QEMU is spawned etc. Test writers typically would not need to modify + this script unless they wanted to enhance or change the log parsing for + their tests. + - requirements.txt: This text file contains the dependent python libraries + needed for the tests to run. If a new dependent library is needed, it would + be added here as a new entry and then acpi-bits-test-venv.sh would install + it when setting up the python virtual environment using pip. + - README: This text file. + +acpi-bits/bits-config: + This location contains biosbits config files that determine how the software + runs the tests. + - bits-config.txt: this is the biosbits config file that determines what tests + or actions are performed by bits. The description of the config options are + provided in the file itself. + +acpi-bits/prebuilt: + This location contains prebuilt biosbits binaries that are used to generate + the bits iso file for testing. + - bits-2095.zip: The contents from this zip archive are the main contents of + the iso file that are used for testing. This binary zip archive also + contains the full source of the bits software including the full text of + the license agreement listed here: + https://github.com/biosbits/bits/blob/master/COPYING + The source tarball can be found in this location in the zip file: + boot/src/bits-2095.tar.gz + The additional changes beyond those that are present in the official + biosbits github repository can be found here: + https://github.com/ani-sinha/bits/tree/bits-qemu-logging + + Basically these changes can be categorized into the following: + (a) changes to make sure biosbits builds with the latest gcc compiler + (gcc 9.4). + (b) upgrade of acpica to the latest version (march 2022). + (c) send bits logs to the debug IO port at addresss 0x403 so that isa + debugcon can be used to collect the logs. + (d) send a gub halt command to shutdown the VM once all the tests have been + executed. + + This zip file is automatically generated by the bits build system. It can + be found in the bits build directory and it is suffixed by the bits build + number. + Normally, there is no need to make any alteration to this zip archive + unless one wanted to change the bits software itself (for example to add + a new config option or change actions of existing options or change the + debug IO port etc). When such changes are made and a new biosbits software + is needed to be generated, please refresh this zip archive as well as the + grub tarball at the same time. Tests will break if changes are made to bits + that are incompatible with existing behavior. So care must be taken to make + sure that the change is compatible with existing bits software as much as + possible. When a new config option is introduced for example, bits must + be upadated here first before introducing the new config option through + a later commit. + - prebuilt/bits-2095-grub.tar.gz: This tarbball contains bits generated grub + scripts and modules to the prebuilt directory. These prebuilt grub + artifacts are required in order to generate a bootable bits iso file that + can run tests. + In order to generate this tar archive, please put the following two + directories that can be found in the bits build directory in a single + tar archive file named as bits-<n>-grub.tar.gz where n is the version of + bits that generated the archive: + + grub-inst-x86_64-efi + grub-inst + + This version should be the same as the version number of bits that generated + bits-<n>.zip file. In other words, the two files must be in sync and should + come from the same build of biosbits of the same version. Things may still + work if they come from different versions but mixing version numbers are + not recommended. + There is normally no need to regenerate this archive unless some fixes or + changes have gone into the grub that are part of biosbits. + +acpi-bits/bits-tests: + This directory contains biosbits python based tests that are run from within + the biosbits environment in the spawned VM. New additions of test cases can + be made in the appropriate test file. For example, new acpi tests can go + into testacpi.py and one would call testsuite.add_test() to register the new + test so that it gets executed as a part of the ACPI tests. + It might be occassionally necessary to disable some subtests or add a new + test that belongs to a test suite not already present in this directory. To + do this, please extract the bits source from the zip file mentioned above. + Copy the test suite/script that needs modification (addition of new tests + or disabling them) from boot/python location of the extracted directory + into this directory. + + step (a): copy unmodified test script to this directory. + step (b): update meson.build and add this file to the list. + Commit (a) and (b) together in the same commit. + + step (c): perform modifications to the test. + Commit (c) in a separate commit. + + The test framework will then use your modified test script to run the test. + No further changes would be needed. Please check the logs to make sure that + appropriate changes have taken effect. + +meson.build files makes sure that the bits qtests are appropriately integrated +into the QEMU qtest framework and are run with "make check-qtest". +Currently, the bits test is configured to run only for x86_64 architectures. Bits +has been built only for this architecture. + + +Author: Ani Sinha <[email protected]> + +References: +(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf +(b) https://www.youtube.com/watch?v=36QIepyUuhg
Signature · 1 line
2.25.1
Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests
Quoted 29 lines · Ani Sinha
The README file is added describing the directory structure and the purpose of every file it contains. It also describes how to add new tests, make changes to existing tests or bits config files or regenerate the bits software.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/README | 168 +++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 tests/qtest/acpi-bits/READMEtests/qtest/acpi-bits/README new file +13
-0@@ -0,0 +1,168 @@ +============================================================================= +ACPI/SMBIOS QTESTS USING BIOSBITS +============================================================================= + +Biosbits is a software written by Josh Triplett that can be downloaded by +visiting https://biosbits.org/. The github codebase can be found here: +https://github.com/biosbits/bits/tree/master. It is a software that exercizes +the bios components such as acpi and smbios tables directly through acpica +bios interpreter (a freely available C based library written by Intel, +downloadable from https://acpica.org/ and is included with biosbits) without an +operating system getting involved in between. +There are several advantages to directly testing the bios in a real physical +machine or VM as opposed to indirectly discovering bios issues through the
tests/qtest/acpi-bits/README line 23
Quoted 10 lines
tests/qtest/acpi-bits/README new file +10
-0+operating system. For one thing, the OSes tend to hide bios problems from the +end user. The other is that we have more control of what we wanted to test +and how by directly using acpica interpreter on top of the bios on a running +system. More details on the inspiration for developing biosbits and its real +life uses can be found in (a) and (b). +This directory contains QEMU qtests written in python that exercizes the QEMU +bios components using biosbits and reports test failures. + +These tests use python virtual environment. In debian/ubuntu system, the tests +would require python3.8-venv and python3-pip packages to be installed.
Why do we mess with venv and pip? Certainly possible but what's wrong with using distro provided packages?
Quoted 80 lines
tests/qtest/acpi-bits/README new file +80
-0+ +A brief description of the contents of this directory follows: + +├── acpi-bits-test.py +├── acpi-bits-test-venv.sh +├── bits-config +│ ├── bits-cfg.txt +│ └── meson.build +├── bits-tests +│ ├── meson.build +│ ├── smbios.py +│ ├── smilatency.py +│ ├── testacpi.py +│ └── testcpuid.py +├── meson.build +├── prebuilt +│ ├── bits-2095-grub.tar.gz +│ ├── bits-2095.zip +│ └── meson.build +├── README +└── requirements.txt + +acpi-bits: + - acpi-bits-test-venv.sh: This is a shell script that sets up the virtual + environment necessary for the actual python test script to run. Amongst + other things, it makes sure that QEMU python library is available within + that venv so that QEMU machines can be forked. QEMU python library can be + found within python/ directory in QEMU source. + After setting up the virtual environment, it runs the python test script + from within that environment. + If you want to enable verbose mode only for bits test and run make check, + one trick is to add V=1 before the call to execute the python script in + this file. + - acpi-bits-test.py: This is the main python test script that generates a + biosbits iso. It then spawns a QEMU VM with it, collects the logs and reports + test failures. This is the script one would be interested in if they wanted + to add or change some component of the log parsing, add a new commandline to + how QEMU is spawned etc. Test writers typically would not need to modify + this script unless they wanted to enhance or change the log parsing for + their tests. + - requirements.txt: This text file contains the dependent python libraries + needed for the tests to run. If a new dependent library is needed, it would + be added here as a new entry and then acpi-bits-test-venv.sh would install + it when setting up the python virtual environment using pip. + - README: This text file. + +acpi-bits/bits-config: + This location contains biosbits config files that determine how the software + runs the tests. + - bits-config.txt: this is the biosbits config file that determines what tests + or actions are performed by bits. The description of the config options are + provided in the file itself. + +acpi-bits/prebuilt: + This location contains prebuilt biosbits binaries that are used to generate + the bits iso file for testing. + - bits-2095.zip: The contents from this zip archive are the main contents of + the iso file that are used for testing. This binary zip archive also + contains the full source of the bits software including the full text of + the license agreement listed here: + https://github.com/biosbits/bits/blob/master/COPYING + The source tarball can be found in this location in the zip file: + boot/src/bits-2095.tar.gz + The additional changes beyond those that are present in the official + biosbits github repository can be found here: + https://github.com/ani-sinha/bits/tree/bits-qemu-logging + + Basically these changes can be categorized into the following: + (a) changes to make sure biosbits builds with the latest gcc compiler + (gcc 9.4). + (b) upgrade of acpica to the latest version (march 2022). + (c) send bits logs to the debug IO port at addresss 0x403 so that isa + debugcon can be used to collect the logs. + (d) send a gub halt command to shutdown the VM once all the tests have been + executed. + + This zip file is automatically generated by the bits build system. It can + be found in the bits build directory and it is suffixed by the bits build + number. + Normally, there is no need to make any alteration to this zip archive
tests/qtest/acpi-bits/README line 113
Quoted 10 lines
tests/qtest/acpi-bits/README new file +10
-0+ unless one wanted to change the bits software itself (for example to add + a new config option or change actions of existing options or change the + debug IO port etc). When such changes are made and a new biosbits software + is needed to be generated, please refresh this zip archive as well as the + grub tarball at the same time. Tests will break if changes are made to bits + that are incompatible with existing behavior. So care must be taken to make + sure that the change is compatible with existing bits software as much as + possible. When a new config option is introduced for example, bits must + be upadated here first before introducing the new config option through + a later commit.
I don't think playing with source tarballs is a reasonable work environment. Let's use submodules just like e.g. firmware does?
Quoted 57 lines
tests/qtest/acpi-bits/README new file +55
-0+ - prebuilt/bits-2095-grub.tar.gz: This tarbball contains bits generated grub + scripts and modules to the prebuilt directory. These prebuilt grub + artifacts are required in order to generate a bootable bits iso file that + can run tests. + In order to generate this tar archive, please put the following two + directories that can be found in the bits build directory in a single + tar archive file named as bits-<n>-grub.tar.gz where n is the version of + bits that generated the archive: + + grub-inst-x86_64-efi + grub-inst + + This version should be the same as the version number of bits that generated + bits-<n>.zip file. In other words, the two files must be in sync and should + come from the same build of biosbits of the same version. Things may still + work if they come from different versions but mixing version numbers are + not recommended. + There is normally no need to regenerate this archive unless some fixes or + changes have gone into the grub that are part of biosbits. + +acpi-bits/bits-tests: + This directory contains biosbits python based tests that are run from within + the biosbits environment in the spawned VM. New additions of test cases can + be made in the appropriate test file. For example, new acpi tests can go + into testacpi.py and one would call testsuite.add_test() to register the new + test so that it gets executed as a part of the ACPI tests. + It might be occassionally necessary to disable some subtests or add a new + test that belongs to a test suite not already present in this directory. To + do this, please extract the bits source from the zip file mentioned above. + Copy the test suite/script that needs modification (addition of new tests + or disabling them) from boot/python location of the extracted directory + into this directory. + + step (a): copy unmodified test script to this directory. + step (b): update meson.build and add this file to the list. + Commit (a) and (b) together in the same commit. + + step (c): perform modifications to the test. + Commit (c) in a separate commit. + + The test framework will then use your modified test script to run the test. + No further changes would be needed. Please check the logs to make sure that + appropriate changes have taken effect. + +meson.build files makes sure that the bits qtests are appropriately integrated +into the QEMU qtest framework and are run with "make check-qtest". +Currently, the bits test is configured to run only for x86_64 architectures. Bits +has been built only for this architecture. + + +Author: Ani Sinha <[email protected]> + +References: +(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf +(b) https://www.youtube.com/watch?v=36QIepyUuhgSignature · 1 line
2.25.1
Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests
Quoted 36 lines · Michael S. Tsirkin
Quoted 34 lines · Ani Sinha
The README file is added describing the directory structure and the purpose of every file it contains. It also describes how to add new tests, make changes to existing tests or bits config files or regenerate the bits software.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/README | 168 +++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 tests/qtest/acpi-bits/READMEtests/qtest/acpi-bits/README new file +18
-0@@ -0,0 +1,168 @@ +============================================================================= +ACPI/SMBIOS QTESTS USING BIOSBITS +============================================================================= + +Biosbits is a software written by Josh Triplett that can be downloaded by +visiting https://biosbits.org/. The github codebase can be found here: +https://github.com/biosbits/bits/tree/master. It is a software that exercizes +the bios components such as acpi and smbios tables directly through acpica +bios interpreter (a freely available C based library written by Intel, +downloadable from https://acpica.org/ and is included with biosbits) without an +operating system getting involved in between. +There are several advantages to directly testing the bios in a real physical +machine or VM as opposed to indirectly discovering bios issues through the +operating system. For one thing, the OSes tend to hide bios problems from the +end user. The other is that we have more control of what we wanted to test +and how by directly using acpica interpreter on top of the bios on a running +system. More details on the inspiration for developing biosbits and its real +life uses can be found in (a) and (b).
Quoted 8 lines
tests/qtest/acpi-bits/README line 23
Quoted 5 lines
tests/qtest/acpi-bits/README new file +5
-0+This directory contains QEMU qtests written in python that exercizes the QEMU +bios components using biosbits and reports test failures. + +These tests use python virtual environment. In debian/ubuntu system, the tests +would require python3.8-venv and python3-pip packages to be installed.Why do we mess with venv and pip? Certainly possible but what's wrong with using distro provided packages?
There are two things:
(a) We are already using pip and venv for our avocado based integration tests. Look for TESTS_VENV_DIR in Makefile.include under tests.
(b) the venv is primarily needed because I wanted to take advantage of our rich python library that handles QEMU based machines. There are python qtest libraries as well. These are well tested and used with integration tests and I wanted to keep the test part of the code simple by simply reusing them. however, in order to use them, we need a venv environment within which these qemu python libraries are installed. Integration tests does the same thing.
A note about my language of choice - python. I gave a lot of thoughts on this. We do not do a lot of stuff here. All we do is:
(a) generate bits iso.
(b) spawn a QEMU vm with the iso which then runs a bunch of tests within the vm.
(c) collect and analyze logs.
We are not inspecting guest memory or manipulating devices or pci buses. We do not need the power of C here. We need something that is simple to write, easy to maintain and understand and can deal with things like manipulating text files and configs easily. Python seems a better fit for the role.
Quoted 85 lines
Quoted 85 lines
tests/qtest/acpi-bits/README new file +84
-0+ +A brief description of the contents of this directory follows: + +├── acpi-bits-test.py +├── acpi-bits-test-venv.sh +├── bits-config +│ ├── bits-cfg.txt +│ └── meson.build +├── bits-tests +│ ├── meson.build +│ ├── smbios.py +│ ├── smilatency.py +│ ├── testacpi.py +│ └── testcpuid.py +├── meson.build +├── prebuilt +│ ├── bits-2095-grub.tar.gz +│ ├── bits-2095.zip +│ └── meson.build +├── README +└── requirements.txt + +acpi-bits: + - acpi-bits-test-venv.sh: This is a shell script that sets up the virtual + environment necessary for the actual python test script to run. Amongst + other things, it makes sure that QEMU python library is available within + that venv so that QEMU machines can be forked. QEMU python library can be + found within python/ directory in QEMU source. + After setting up the virtual environment, it runs the python test script + from within that environment. + If you want to enable verbose mode only for bits test and run make check, + one trick is to add V=1 before the call to execute the python script in + this file. + - acpi-bits-test.py: This is the main python test script that generates a + biosbits iso. It then spawns a QEMU VM with it, collects the logs and reports + test failures. This is the script one would be interested in if they wanted + to add or change some component of the log parsing, add a new commandline to + how QEMU is spawned etc. Test writers typically would not need to modify + this script unless they wanted to enhance or change the log parsing for + their tests. + - requirements.txt: This text file contains the dependent python libraries + needed for the tests to run. If a new dependent library is needed, it would + be added here as a new entry and then acpi-bits-test-venv.sh would install + it when setting up the python virtual environment using pip. + - README: This text file. + +acpi-bits/bits-config: + This location contains biosbits config files that determine how the software + runs the tests. + - bits-config.txt: this is the biosbits config file that determines what tests + or actions are performed by bits. The description of the config options are + provided in the file itself. + +acpi-bits/prebuilt: + This location contains prebuilt biosbits binaries that are used to generate + the bits iso file for testing. + - bits-2095.zip: The contents from this zip archive are the main contents of + the iso file that are used for testing. This binary zip archive also + contains the full source of the bits software including the full text of + the license agreement listed here: + https://github.com/biosbits/bits/blob/master/COPYING + The source tarball can be found in this location in the zip file: + boot/src/bits-2095.tar.gz + The additional changes beyond those that are present in the official + biosbits github repository can be found here: + https://github.com/ani-sinha/bits/tree/bits-qemu-logging + + Basically these changes can be categorized into the following: + (a) changes to make sure biosbits builds with the latest gcc compiler + (gcc 9.4). + (b) upgrade of acpica to the latest version (march 2022). + (c) send bits logs to the debug IO port at addresss 0x403 so that isa + debugcon can be used to collect the logs. + (d) send a gub halt command to shutdown the VM once all the tests have been + executed. + + This zip file is automatically generated by the bits build system. It can + be found in the bits build directory and it is suffixed by the bits build + number. + Normally, there is no need to make any alteration to this zip archive + unless one wanted to change the bits software itself (for example to add + a new config option or change actions of existing options or change the + debug IO port etc). When such changes are made and a new biosbits software + is needed to be generated, please refresh this zip archive as well as the
Quoted 8 lines
tests/qtest/acpi-bits/README line 113
Quoted 6 lines
tests/qtest/acpi-bits/README new file +6
-0+ grub tarball at the same time. Tests will break if changes are made to bits + that are incompatible with existing behavior. So care must be taken to make + sure that the change is compatible with existing bits software as much as + possible. When a new config option is introduced for example, bits must + be upadated here first before introducing the new config option through + a later commit.I don't think playing with source tarballs is a reasonable work environment.
I agree. However, we do not do much with the source tarball. It is there as part of bits to satisfy the license requirement. If we need to manipulate any test script that is in the source file, we would copy it over and keep it in the bits-test directory and change it there.
Quoted 1 line
Let's use submodules just like e.g. firmware does?
Yes I also proposed this to Igor on IRC. We can/maybe need to figure out a place to store the bits source if we think my github is not the best place. However, we need the source only if/when we need to rebuild bits. I suspect it won't be too often if at all.
Quoted 61 lines
Quoted 60 lines
tests/qtest/acpi-bits/README new file +55
-1+ - prebuilt/bits-2095-grub.tar.gz: This tarbball contains bits generated grub + scripts and modules to the prebuilt directory. These prebuilt grub + artifacts are required in order to generate a bootable bits iso file that + can run tests. + In order to generate this tar archive, please put the following two + directories that can be found in the bits build directory in a single + tar archive file named as bits-<n>-grub.tar.gz where n is the version of + bits that generated the archive: + + grub-inst-x86_64-efi + grub-inst + + This version should be the same as the version number of bits that generated + bits-<n>.zip file. In other words, the two files must be in sync and should + come from the same build of biosbits of the same version. Things may still + work if they come from different versions but mixing version numbers are + not recommended. + There is normally no need to regenerate this archive unless some fixes or + changes have gone into the grub that are part of biosbits. + +acpi-bits/bits-tests: + This directory contains biosbits python based tests that are run from within + the biosbits environment in the spawned VM. New additions of test cases can + be made in the appropriate test file. For example, new acpi tests can go + into testacpi.py and one would call testsuite.add_test() to register the new + test so that it gets executed as a part of the ACPI tests. + It might be occassionally necessary to disable some subtests or add a new + test that belongs to a test suite not already present in this directory. To + do this, please extract the bits source from the zip file mentioned above. + Copy the test suite/script that needs modification (addition of new tests + or disabling them) from boot/python location of the extracted directory + into this directory. + + step (a): copy unmodified test script to this directory. + step (b): update meson.build and add this file to the list. + Commit (a) and (b) together in the same commit. + + step (c): perform modifications to the test. + Commit (c) in a separate commit. + + The test framework will then use your modified test script to run the test. + No further changes would be needed. Please check the logs to make sure that + appropriate changes have taken effect. + +meson.build files makes sure that the bits qtests are appropriately integrated +into the QEMU qtest framework and are run with "make check-qtest". +Currently, the bits test is configured to run only for x86_64 architectures. Bits +has been built only for this architecture. + + +Author: Ani Sinha <[email protected]> + +References: +(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf +(b) https://www.youtube.com/watch?v=36QIepyUuhg --2.25.1
Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests
Quoted 64 lines · Ani Sinha
Quoted 36 lines · Michael S. Tsirkin
Quoted 34 lines · Ani Sinha
The README file is added describing the directory structure and the purpose of every file it contains. It also describes how to add new tests, make changes to existing tests or bits config files or regenerate the bits software.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/README | 168 +++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 tests/qtest/acpi-bits/READMEtests/qtest/acpi-bits/README new file +18
-0@@ -0,0 +1,168 @@ +============================================================================= +ACPI/SMBIOS QTESTS USING BIOSBITS +============================================================================= + +Biosbits is a software written by Josh Triplett that can be downloaded by +visiting https://biosbits.org/. The github codebase can be found here: +https://github.com/biosbits/bits/tree/master. It is a software that exercizes +the bios components such as acpi and smbios tables directly through acpica +bios interpreter (a freely available C based library written by Intel, +downloadable from https://acpica.org/ and is included with biosbits) without an +operating system getting involved in between. +There are several advantages to directly testing the bios in a real physical +machine or VM as opposed to indirectly discovering bios issues through the +operating system. For one thing, the OSes tend to hide bios problems from the +end user. The other is that we have more control of what we wanted to test +and how by directly using acpica interpreter on top of the bios on a running +system. More details on the inspiration for developing biosbits and its real +life uses can be found in (a) and (b).Quoted 8 lines
tests/qtest/acpi-bits/README line 23
Quoted 5 lines
tests/qtest/acpi-bits/README new file +5
-0+This directory contains QEMU qtests written in python that exercizes the QEMU +bios components using biosbits and reports test failures. + +These tests use python virtual environment. In debian/ubuntu system, the tests +would require python3.8-venv and python3-pip packages to be installed.Why do we mess with venv and pip? Certainly possible but what's wrong with using distro provided packages?
There are two things:
(a) We are already using pip and venv for our avocado based integration tests. Look forTESTS_VENV_DIRin Makefile.include under tests.
(b) the venv is primarily needed because I wanted to take advantage of our rich python library that handles QEMU based machines. There are python qtest libraries as well. These are well tested and used with integration tests and I wanted to keep the test part of the code simple by simply reusing them. however, in order to use them, we need a venv environment within which these qemu python libraries are installed. Integration tests does the same thing.A note about my language of choice - python. I gave a lot of thoughts on this. We do not do a lot of stuff here. All we do is:
(a) generate bits iso.
(b) spawn a QEMU vm with the iso which then runs a bunch of tests within the vm.
(c) collect and analyze logs.
Quoted 5 lines
We are not inspecting guest memory or manipulating devices or pci buses. We do not need the power of C here. We need something that is simple to write, easy to maintain and understand and can deal with things like manipulating text files and configs easily. Python seems a better fit for the role.
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
Quoted 100 lines
Quoted 85 lines
Quoted 85 lines
tests/qtest/acpi-bits/README new file +84
-0+ +A brief description of the contents of this directory follows: + +├── acpi-bits-test.py +├── acpi-bits-test-venv.sh +├── bits-config +│ ├── bits-cfg.txt +│ └── meson.build +├── bits-tests +│ ├── meson.build +│ ├── smbios.py +│ ├── smilatency.py +│ ├── testacpi.py +│ └── testcpuid.py +├── meson.build +├── prebuilt +│ ├── bits-2095-grub.tar.gz +│ ├── bits-2095.zip +│ └── meson.build +├── README +└── requirements.txt + +acpi-bits: + - acpi-bits-test-venv.sh: This is a shell script that sets up the virtual + environment necessary for the actual python test script to run. Amongst + other things, it makes sure that QEMU python library is available within + that venv so that QEMU machines can be forked. QEMU python library can be + found within python/ directory in QEMU source. + After setting up the virtual environment, it runs the python test script + from within that environment. + If you want to enable verbose mode only for bits test and run make check, + one trick is to add V=1 before the call to execute the python script in + this file. + - acpi-bits-test.py: This is the main python test script that generates a + biosbits iso. It then spawns a QEMU VM with it, collects the logs and reports + test failures. This is the script one would be interested in if they wanted + to add or change some component of the log parsing, add a new commandline to + how QEMU is spawned etc. Test writers typically would not need to modify + this script unless they wanted to enhance or change the log parsing for + their tests. + - requirements.txt: This text file contains the dependent python libraries + needed for the tests to run. If a new dependent library is needed, it would + be added here as a new entry and then acpi-bits-test-venv.sh would install + it when setting up the python virtual environment using pip. + - README: This text file. + +acpi-bits/bits-config: + This location contains biosbits config files that determine how the software + runs the tests. + - bits-config.txt: this is the biosbits config file that determines what tests + or actions are performed by bits. The description of the config options are + provided in the file itself. + +acpi-bits/prebuilt: + This location contains prebuilt biosbits binaries that are used to generate + the bits iso file for testing. + - bits-2095.zip: The contents from this zip archive are the main contents of + the iso file that are used for testing. This binary zip archive also + contains the full source of the bits software including the full text of + the license agreement listed here: + https://github.com/biosbits/bits/blob/master/COPYING + The source tarball can be found in this location in the zip file: + boot/src/bits-2095.tar.gz + The additional changes beyond those that are present in the official + biosbits github repository can be found here: + https://github.com/ani-sinha/bits/tree/bits-qemu-logging + + Basically these changes can be categorized into the following: + (a) changes to make sure biosbits builds with the latest gcc compiler + (gcc 9.4). + (b) upgrade of acpica to the latest version (march 2022). + (c) send bits logs to the debug IO port at addresss 0x403 so that isa + debugcon can be used to collect the logs. + (d) send a gub halt command to shutdown the VM once all the tests have been + executed. + + This zip file is automatically generated by the bits build system. It can + be found in the bits build directory and it is suffixed by the bits build + number. + Normally, there is no need to make any alteration to this zip archive + unless one wanted to change the bits software itself (for example to add + a new config option or change actions of existing options or change the + debug IO port etc). When such changes are made and a new biosbits software + is needed to be generated, please refresh this zip archive as well as theQuoted 8 lines
tests/qtest/acpi-bits/README line 113
Quoted 6 lines
tests/qtest/acpi-bits/README new file +6
-0+ grub tarball at the same time. Tests will break if changes are made to bits + that are incompatible with existing behavior. So care must be taken to make + sure that the change is compatible with existing bits software as much as + possible. When a new config option is introduced for example, bits must + be upadated here first before introducing the new config option through + a later commit.I don't think playing with source tarballs is a reasonable work environment.
I agree. However, we do not do much with the source tarball. It is there as part of bits to satisfy the license requirement. If we need to manipulate any test script that is in the source file, we would copy it over and keep it in the bits-test directory and change it there.
Quoted 6 lines
Quoted 1 line
Let's use submodules just like e.g. firmware does?
Yes I also proposed this to Igor on IRC. We can/maybe need to figure out a place to store the bits source if we think my github is not the best place. However, we need the source only if/when we need to rebuild bits. I suspect it won't be too often if at all.
Why not on git.qemu.org ?
Quoted 61 lines
Quoted 61 lines
Quoted 60 lines
tests/qtest/acpi-bits/README new file +55
-1+ - prebuilt/bits-2095-grub.tar.gz: This tarbball contains bits generated grub + scripts and modules to the prebuilt directory. These prebuilt grub + artifacts are required in order to generate a bootable bits iso file that + can run tests. + In order to generate this tar archive, please put the following two + directories that can be found in the bits build directory in a single + tar archive file named as bits-<n>-grub.tar.gz where n is the version of + bits that generated the archive: + + grub-inst-x86_64-efi + grub-inst + + This version should be the same as the version number of bits that generated + bits-<n>.zip file. In other words, the two files must be in sync and should + come from the same build of biosbits of the same version. Things may still + work if they come from different versions but mixing version numbers are + not recommended. + There is normally no need to regenerate this archive unless some fixes or + changes have gone into the grub that are part of biosbits. + +acpi-bits/bits-tests: + This directory contains biosbits python based tests that are run from within + the biosbits environment in the spawned VM. New additions of test cases can + be made in the appropriate test file. For example, new acpi tests can go + into testacpi.py and one would call testsuite.add_test() to register the new + test so that it gets executed as a part of the ACPI tests. + It might be occassionally necessary to disable some subtests or add a new + test that belongs to a test suite not already present in this directory. To + do this, please extract the bits source from the zip file mentioned above. + Copy the test suite/script that needs modification (addition of new tests + or disabling them) from boot/python location of the extracted directory + into this directory. + + step (a): copy unmodified test script to this directory. + step (b): update meson.build and add this file to the list. + Commit (a) and (b) together in the same commit. + + step (c): perform modifications to the test. + Commit (c) in a separate commit. + + The test framework will then use your modified test script to run the test. + No further changes would be needed. Please check the logs to make sure that + appropriate changes have taken effect. + +meson.build files makes sure that the bits qtests are appropriately integrated +into the QEMU qtest framework and are run with "make check-qtest". +Currently, the bits test is configured to run only for x86_64 architectures. Bits +has been built only for this architecture. + + +Author: Ani Sinha <[email protected]> + +References: +(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf +(b) https://www.youtube.com/watch?v=36QIepyUuhg --2.25.1
Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests
Quoted 66 lines · Michael S. Tsirkin
Quoted 64 lines · Ani Sinha
Quoted 36 lines · Michael S. Tsirkin
Quoted 34 lines · Ani Sinha
The README file is added describing the directory structure and the purpose of every file it contains. It also describes how to add new tests, make changes to existing tests or bits config files or regenerate the bits software.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/README | 168 +++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 tests/qtest/acpi-bits/READMEtests/qtest/acpi-bits/README new file +18
-0@@ -0,0 +1,168 @@ +============================================================================= +ACPI/SMBIOS QTESTS USING BIOSBITS +============================================================================= + +Biosbits is a software written by Josh Triplett that can be downloaded by +visiting https://biosbits.org/. The github codebase can be found here: +https://github.com/biosbits/bits/tree/master. It is a software that exercizes +the bios components such as acpi and smbios tables directly through acpica +bios interpreter (a freely available C based library written by Intel, +downloadable from https://acpica.org/ and is included with biosbits) without an +operating system getting involved in between. +There are several advantages to directly testing the bios in a real physical +machine or VM as opposed to indirectly discovering bios issues through the +operating system. For one thing, the OSes tend to hide bios problems from the +end user. The other is that we have more control of what we wanted to test +and how by directly using acpica interpreter on top of the bios on a running +system. More details on the inspiration for developing biosbits and its real +life uses can be found in (a) and (b).Quoted 8 lines
tests/qtest/acpi-bits/README line 23
Quoted 5 lines
tests/qtest/acpi-bits/README new file +5
-0+This directory contains QEMU qtests written in python that exercizes the QEMU +bios components using biosbits and reports test failures. + +These tests use python virtual environment. In debian/ubuntu system, the tests +would require python3.8-venv and python3-pip packages to be installed.Why do we mess with venv and pip? Certainly possible but what's wrong with using distro provided packages?
There are two things:
(a) We are already using pip and venv for our avocado based integration tests. Look forTESTS_VENV_DIRin Makefile.include under tests.
(b) the venv is primarily needed because I wanted to take advantage of our rich python library that handles QEMU based machines. There are python qtest libraries as well. These are well tested and used with integration tests and I wanted to keep the test part of the code simple by simply reusing them. however, in order to use them, we need a venv environment within which these qemu python libraries are installed. Integration tests does the same thing.A note about my language of choice - python. I gave a lot of thoughts on this. We do not do a lot of stuff here. All we do is:
(a) generate bits iso.
(b) spawn a QEMU vm with the iso which then runs a bunch of tests within the vm.
(c) collect and analyze logs.
Quoted 8 lines
Quoted 5 lines
We are not inspecting guest memory or manipulating devices or pci buses. We do not need the power of C here. We need something that is simple to write, easy to maintain and understand and can deal with things like manipulating text files and configs easily. Python seems a better fit for the role.
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Quoted 101 lines
Quoted 101 lines
Quoted 86 lines
Quoted 86 lines
tests/qtest/acpi-bits/README new file +84
-0+ +A brief description of the contents of this directory follows: + +├── acpi-bits-test.py +├── acpi-bits-test-venv.sh +├── bits-config +│ ├── bits-cfg.txt +│ └── meson.build +├── bits-tests +│ ├── meson.build +│ ├── smbios.py +│ ├── smilatency.py +│ ├── testacpi.py +│ └── testcpuid.py +├── meson.build +├── prebuilt +│ ├── bits-2095-grub.tar.gz +│ ├── bits-2095.zip +│ └── meson.build +├── README +└── requirements.txt + +acpi-bits: + - acpi-bits-test-venv.sh: This is a shell script that sets up the virtual + environment necessary for the actual python test script to run. Amongst + other things, it makes sure that QEMU python library is available within + that venv so that QEMU machines can be forked. QEMU python library can be + found within python/ directory in QEMU source. + After setting up the virtual environment, it runs the python test script + from within that environment. + If you want to enable verbose mode only for bits test and run make check, + one trick is to add V=1 before the call to execute the python script in + this file. + - acpi-bits-test.py: This is the main python test script that generates a + biosbits iso. It then spawns a QEMU VM with it, collects the logs and reports + test failures. This is the script one would be interested in if they wanted + to add or change some component of the log parsing, add a new commandline to + how QEMU is spawned etc. Test writers typically would not need to modify + this script unless they wanted to enhance or change the log parsing for + their tests. + - requirements.txt: This text file contains the dependent python libraries + needed for the tests to run. If a new dependent library is needed, it would + be added here as a new entry and then acpi-bits-test-venv.sh would install + it when setting up the python virtual environment using pip. + - README: This text file. + +acpi-bits/bits-config: + This location contains biosbits config files that determine how the software + runs the tests. + - bits-config.txt: this is the biosbits config file that determines what tests + or actions are performed by bits. The description of the config options are + provided in the file itself. + +acpi-bits/prebuilt: + This location contains prebuilt biosbits binaries that are used to generate + the bits iso file for testing. + - bits-2095.zip: The contents from this zip archive are the main contents of + the iso file that are used for testing. This binary zip archive also + contains the full source of the bits software including the full text of + the license agreement listed here: + https://github.com/biosbits/bits/blob/master/COPYING + The source tarball can be found in this location in the zip file: + boot/src/bits-2095.tar.gz + The additional changes beyond those that are present in the official + biosbits github repository can be found here: + https://github.com/ani-sinha/bits/tree/bits-qemu-logging + + Basically these changes can be categorized into the following: + (a) changes to make sure biosbits builds with the latest gcc compiler + (gcc 9.4). + (b) upgrade of acpica to the latest version (march 2022). + (c) send bits logs to the debug IO port at addresss 0x403 so that isa + debugcon can be used to collect the logs. + (d) send a gub halt command to shutdown the VM once all the tests have been + executed. + + This zip file is automatically generated by the bits build system. It can + be found in the bits build directory and it is suffixed by the bits build + number. + Normally, there is no need to make any alteration to this zip archive + unless one wanted to change the bits software itself (for example to add + a new config option or change actions of existing options or change the + debug IO port etc). When such changes are made and a new biosbits software + is needed to be generated, please refresh this zip archive as well as theQuoted 8 lines
tests/qtest/acpi-bits/README line 113
Quoted 6 lines
tests/qtest/acpi-bits/README new file +6
-0+ grub tarball at the same time. Tests will break if changes are made to bits + that are incompatible with existing behavior. So care must be taken to make + sure that the change is compatible with existing bits software as much as + possible. When a new config option is introduced for example, bits must + be upadated here first before introducing the new config option through + a later commit.I don't think playing with source tarballs is a reasonable work environment.
I agree. However, we do not do much with the source tarball. It is there as part of bits to satisfy the license requirement. If we need to manipulate any test script that is in the source file, we would copy it over and keep it in the bits-test directory and change it there.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Let's use submodules just like e.g. firmware does?
Yes I also proposed this to Igor on IRC. We can/maybe need to figure out a place to store the bits source if we think my github is not the best place. However, we need the source only if/when we need to rebuild bits. I suspect it won't be too often if at all.
Why not on git.qemu.org ?
Sounds good to me. Do I get push access? :-)
Quoted 63 lines
Quoted 62 lines
Quoted 62 lines
Quoted 61 lines
tests/qtest/acpi-bits/README new file +55
-1+ - prebuilt/bits-2095-grub.tar.gz: This tarbball contains bits generated grub + scripts and modules to the prebuilt directory. These prebuilt grub + artifacts are required in order to generate a bootable bits iso file that + can run tests. + In order to generate this tar archive, please put the following two + directories that can be found in the bits build directory in a single + tar archive file named as bits-<n>-grub.tar.gz where n is the version of + bits that generated the archive: + + grub-inst-x86_64-efi + grub-inst + + This version should be the same as the version number of bits that generated + bits-<n>.zip file. In other words, the two files must be in sync and should + come from the same build of biosbits of the same version. Things may still + work if they come from different versions but mixing version numbers are + not recommended. + There is normally no need to regenerate this archive unless some fixes or + changes have gone into the grub that are part of biosbits. + +acpi-bits/bits-tests: + This directory contains biosbits python based tests that are run from within + the biosbits environment in the spawned VM. New additions of test cases can + be made in the appropriate test file. For example, new acpi tests can go + into testacpi.py and one would call testsuite.add_test() to register the new + test so that it gets executed as a part of the ACPI tests. + It might be occassionally necessary to disable some subtests or add a new + test that belongs to a test suite not already present in this directory. To + do this, please extract the bits source from the zip file mentioned above. + Copy the test suite/script that needs modification (addition of new tests + or disabling them) from boot/python location of the extracted directory + into this directory. + + step (a): copy unmodified test script to this directory. + step (b): update meson.build and add this file to the list. + Commit (a) and (b) together in the same commit. + + step (c): perform modifications to the test. + Commit (c) in a separate commit. + + The test framework will then use your modified test script to run the test. + No further changes would be needed. Please check the logs to make sure that + appropriate changes have taken effect. + +meson.build files makes sure that the bits qtests are appropriately integrated +into the QEMU qtest framework and are run with "make check-qtest". +Currently, the bits test is configured to run only for x86_64 architectures. Bits +has been built only for this architecture. + + +Author: Ani Sinha <[email protected]> + +References: +(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf +(b) https://www.youtube.com/watch?v=36QIepyUuhg --2.25.1
Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests
Quoted 76 lines · Ani Sinha
Quoted 66 lines · Michael S. Tsirkin
Quoted 64 lines · Ani Sinha
Quoted 36 lines · Michael S. Tsirkin
Quoted 34 lines · Ani Sinha
The README file is added describing the directory structure and the purpose of every file it contains. It also describes how to add new tests, make changes to existing tests or bits config files or regenerate the bits software.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/README | 168 +++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 tests/qtest/acpi-bits/READMEtests/qtest/acpi-bits/README new file +18
-0@@ -0,0 +1,168 @@ +============================================================================= +ACPI/SMBIOS QTESTS USING BIOSBITS +============================================================================= + +Biosbits is a software written by Josh Triplett that can be downloaded by +visiting https://biosbits.org/. The github codebase can be found here: +https://github.com/biosbits/bits/tree/master. It is a software that exercizes +the bios components such as acpi and smbios tables directly through acpica +bios interpreter (a freely available C based library written by Intel, +downloadable from https://acpica.org/ and is included with biosbits) without an +operating system getting involved in between. +There are several advantages to directly testing the bios in a real physical +machine or VM as opposed to indirectly discovering bios issues through the +operating system. For one thing, the OSes tend to hide bios problems from the +end user. The other is that we have more control of what we wanted to test +and how by directly using acpica interpreter on top of the bios on a running +system. More details on the inspiration for developing biosbits and its real +life uses can be found in (a) and (b).Quoted 8 lines
tests/qtest/acpi-bits/README line 23
Quoted 5 lines
tests/qtest/acpi-bits/README new file +5
-0+This directory contains QEMU qtests written in python that exercizes the QEMU +bios components using biosbits and reports test failures. + +These tests use python virtual environment. In debian/ubuntu system, the tests +would require python3.8-venv and python3-pip packages to be installed.Why do we mess with venv and pip? Certainly possible but what's wrong with using distro provided packages?
There are two things:
(a) We are already using pip and venv for our avocado based integration tests. Look forTESTS_VENV_DIRin Makefile.include under tests.
(b) the venv is primarily needed because I wanted to take advantage of our rich python library that handles QEMU based machines. There are python qtest libraries as well. These are well tested and used with integration tests and I wanted to keep the test part of the code simple by simply reusing them. however, in order to use them, we need a venv environment within which these qemu python libraries are installed. Integration tests does the same thing.A note about my language of choice - python. I gave a lot of thoughts on this. We do not do a lot of stuff here. All we do is:
(a) generate bits iso.
(b) spawn a QEMU vm with the iso which then runs a bunch of tests within the vm.
(c) collect and analyze logs.Quoted 8 lines
Quoted 5 lines
We are not inspecting guest memory or manipulating devices or pci buses. We do not need the power of C here. We need something that is simple to write, easy to maintain and understand and can deal with things like manipulating text files and configs easily. Python seems a better fit for the role.
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
Quoted 8 lines
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?
Quoted 175 lines
Quoted 101 lines
Quoted 101 lines
Quoted 86 lines
Quoted 86 lines
tests/qtest/acpi-bits/README new file +84
-0+ +A brief description of the contents of this directory follows: + +├── acpi-bits-test.py +├── acpi-bits-test-venv.sh +├── bits-config +│ ├── bits-cfg.txt +│ └── meson.build +├── bits-tests +│ ├── meson.build +│ ├── smbios.py +│ ├── smilatency.py +│ ├── testacpi.py +│ └── testcpuid.py +├── meson.build +├── prebuilt +│ ├── bits-2095-grub.tar.gz +│ ├── bits-2095.zip +│ └── meson.build +├── README +└── requirements.txt + +acpi-bits: + - acpi-bits-test-venv.sh: This is a shell script that sets up the virtual + environment necessary for the actual python test script to run. Amongst + other things, it makes sure that QEMU python library is available within + that venv so that QEMU machines can be forked. QEMU python library can be + found within python/ directory in QEMU source. + After setting up the virtual environment, it runs the python test script + from within that environment. + If you want to enable verbose mode only for bits test and run make check, + one trick is to add V=1 before the call to execute the python script in + this file. + - acpi-bits-test.py: This is the main python test script that generates a + biosbits iso. It then spawns a QEMU VM with it, collects the logs and reports + test failures. This is the script one would be interested in if they wanted + to add or change some component of the log parsing, add a new commandline to + how QEMU is spawned etc. Test writers typically would not need to modify + this script unless they wanted to enhance or change the log parsing for + their tests. + - requirements.txt: This text file contains the dependent python libraries + needed for the tests to run. If a new dependent library is needed, it would + be added here as a new entry and then acpi-bits-test-venv.sh would install + it when setting up the python virtual environment using pip. + - README: This text file. + +acpi-bits/bits-config: + This location contains biosbits config files that determine how the software + runs the tests. + - bits-config.txt: this is the biosbits config file that determines what tests + or actions are performed by bits. The description of the config options are + provided in the file itself. + +acpi-bits/prebuilt: + This location contains prebuilt biosbits binaries that are used to generate + the bits iso file for testing. + - bits-2095.zip: The contents from this zip archive are the main contents of + the iso file that are used for testing. This binary zip archive also + contains the full source of the bits software including the full text of + the license agreement listed here: + https://github.com/biosbits/bits/blob/master/COPYING + The source tarball can be found in this location in the zip file: + boot/src/bits-2095.tar.gz + The additional changes beyond those that are present in the official + biosbits github repository can be found here: + https://github.com/ani-sinha/bits/tree/bits-qemu-logging + + Basically these changes can be categorized into the following: + (a) changes to make sure biosbits builds with the latest gcc compiler + (gcc 9.4). + (b) upgrade of acpica to the latest version (march 2022). + (c) send bits logs to the debug IO port at addresss 0x403 so that isa + debugcon can be used to collect the logs. + (d) send a gub halt command to shutdown the VM once all the tests have been + executed. + + This zip file is automatically generated by the bits build system. It can + be found in the bits build directory and it is suffixed by the bits build + number. + Normally, there is no need to make any alteration to this zip archive + unless one wanted to change the bits software itself (for example to add + a new config option or change actions of existing options or change the + debug IO port etc). When such changes are made and a new biosbits software + is needed to be generated, please refresh this zip archive as well as theQuoted 8 lines
tests/qtest/acpi-bits/README line 113
Quoted 6 lines
tests/qtest/acpi-bits/README new file +6
-0+ grub tarball at the same time. Tests will break if changes are made to bits + that are incompatible with existing behavior. So care must be taken to make + sure that the change is compatible with existing bits software as much as + possible. When a new config option is introduced for example, bits must + be upadated here first before introducing the new config option through + a later commit.I don't think playing with source tarballs is a reasonable work environment.
I agree. However, we do not do much with the source tarball. It is there as part of bits to satisfy the license requirement. If we need to manipulate any test script that is in the source file, we would copy it over and keep it in the bits-test directory and change it there.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Let's use submodules just like e.g. firmware does?
Yes I also proposed this to Igor on IRC. We can/maybe need to figure out a place to store the bits source if we think my github is not the best place. However, we need the source only if/when we need to rebuild bits. I suspect it won't be too often if at all.
Why not on git.qemu.org ?
Sounds good to me. Do I get push access? :-)
Quoted 63 lines
Quoted 62 lines
Quoted 62 lines
Quoted 61 lines
tests/qtest/acpi-bits/README new file +55
-1+ - prebuilt/bits-2095-grub.tar.gz: This tarbball contains bits generated grub + scripts and modules to the prebuilt directory. These prebuilt grub + artifacts are required in order to generate a bootable bits iso file that + can run tests. + In order to generate this tar archive, please put the following two + directories that can be found in the bits build directory in a single + tar archive file named as bits-<n>-grub.tar.gz where n is the version of + bits that generated the archive: + + grub-inst-x86_64-efi + grub-inst + + This version should be the same as the version number of bits that generated + bits-<n>.zip file. In other words, the two files must be in sync and should + come from the same build of biosbits of the same version. Things may still + work if they come from different versions but mixing version numbers are + not recommended. + There is normally no need to regenerate this archive unless some fixes or + changes have gone into the grub that are part of biosbits. + +acpi-bits/bits-tests: + This directory contains biosbits python based tests that are run from within + the biosbits environment in the spawned VM. New additions of test cases can + be made in the appropriate test file. For example, new acpi tests can go + into testacpi.py and one would call testsuite.add_test() to register the new + test so that it gets executed as a part of the ACPI tests. + It might be occassionally necessary to disable some subtests or add a new + test that belongs to a test suite not already present in this directory. To + do this, please extract the bits source from the zip file mentioned above. + Copy the test suite/script that needs modification (addition of new tests + or disabling them) from boot/python location of the extracted directory + into this directory. + + step (a): copy unmodified test script to this directory. + step (b): update meson.build and add this file to the list. + Commit (a) and (b) together in the same commit. + + step (c): perform modifications to the test. + Commit (c) in a separate commit. + + The test framework will then use your modified test script to run the test. + No further changes would be needed. Please check the logs to make sure that + appropriate changes have taken effect. + +meson.build files makes sure that the bits qtests are appropriately integrated +into the QEMU qtest framework and are run with "make check-qtest". +Currently, the bits test is configured to run only for x86_64 architectures. Bits +has been built only for this architecture. + + +Author: Ani Sinha <[email protected]> + +References: +(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf +(b) https://www.youtube.com/watch?v=36QIepyUuhg --2.25.1
Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests
Quoted 83 lines · Michael S. Tsirkin
Quoted 81 lines · Ani Sinha
Quoted 66 lines · Michael S. Tsirkin
Quoted 64 lines · Ani Sinha
Quoted 36 lines · Michael S. Tsirkin
Quoted 34 lines · Ani Sinha
The README file is added describing the directory structure and the purpose of every file it contains. It also describes how to add new tests, make changes to existing tests or bits config files or regenerate the bits software.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/README | 168 +++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 tests/qtest/acpi-bits/READMEtests/qtest/acpi-bits/README new file +18
-0@@ -0,0 +1,168 @@ +============================================================================= +ACPI/SMBIOS QTESTS USING BIOSBITS +============================================================================= + +Biosbits is a software written by Josh Triplett that can be downloaded by +visiting https://biosbits.org/. The github codebase can be found here: +https://github.com/biosbits/bits/tree/master. It is a software that exercizes +the bios components such as acpi and smbios tables directly through acpica +bios interpreter (a freely available C based library written by Intel, +downloadable from https://acpica.org/ and is included with biosbits) without an +operating system getting involved in between. +There are several advantages to directly testing the bios in a real physical +machine or VM as opposed to indirectly discovering bios issues through the +operating system. For one thing, the OSes tend to hide bios problems from the +end user. The other is that we have more control of what we wanted to test +and how by directly using acpica interpreter on top of the bios on a running +system. More details on the inspiration for developing biosbits and its real +life uses can be found in (a) and (b).Quoted 8 lines
tests/qtest/acpi-bits/README line 23
Quoted 5 lines
tests/qtest/acpi-bits/README new file +5
-0+This directory contains QEMU qtests written in python that exercizes the QEMU +bios components using biosbits and reports test failures. + +These tests use python virtual environment. In debian/ubuntu system, the tests +would require python3.8-venv and python3-pip packages to be installed.Why do we mess with venv and pip? Certainly possible but what's wrong with using distro provided packages?
There are two things:
(a) We are already using pip and venv for our avocado based integration tests. Look forTESTS_VENV_DIRin Makefile.include under tests.
(b) the venv is primarily needed because I wanted to take advantage of our rich python library that handles QEMU based machines. There are python qtest libraries as well. These are well tested and used with integration tests and I wanted to keep the test part of the code simple by simply reusing them. however, in order to use them, we need a venv environment within which these qemu python libraries are installed. Integration tests does the same thing.A note about my language of choice - python. I gave a lot of thoughts on this. We do not do a lot of stuff here. All we do is:
(a) generate bits iso.
(b) spawn a QEMU vm with the iso which then runs a bunch of tests within the vm.
(c) collect and analyze logs.Quoted 8 lines
Quoted 5 lines
We are not inspecting guest memory or manipulating devices or pci buses. We do not need the power of C here. We need something that is simple to write, easy to maintain and understand and can deal with things like manipulating text files and configs easily. Python seems a better fit for the role.
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0
Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?
Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.
It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/
I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 177 lines
Quoted 176 lines
Quoted 102 lines
Quoted 102 lines
Quoted 87 lines
Quoted 87 lines
tests/qtest/acpi-bits/README new file +84
-0+ +A brief description of the contents of this directory follows: + +├── acpi-bits-test.py +├── acpi-bits-test-venv.sh +├── bits-config +│ ├── bits-cfg.txt +│ └── meson.build +├── bits-tests +│ ├── meson.build +│ ├── smbios.py +│ ├── smilatency.py +│ ├── testacpi.py +│ └── testcpuid.py +├── meson.build +├── prebuilt +│ ├── bits-2095-grub.tar.gz +│ ├── bits-2095.zip +│ └── meson.build +├── README +└── requirements.txt + +acpi-bits: + - acpi-bits-test-venv.sh: This is a shell script that sets up the virtual + environment necessary for the actual python test script to run. Amongst + other things, it makes sure that QEMU python library is available within + that venv so that QEMU machines can be forked. QEMU python library can be + found within python/ directory in QEMU source. + After setting up the virtual environment, it runs the python test script + from within that environment. + If you want to enable verbose mode only for bits test and run make check, + one trick is to add V=1 before the call to execute the python script in + this file. + - acpi-bits-test.py: This is the main python test script that generates a + biosbits iso. It then spawns a QEMU VM with it, collects the logs and reports + test failures. This is the script one would be interested in if they wanted + to add or change some component of the log parsing, add a new commandline to + how QEMU is spawned etc. Test writers typically would not need to modify + this script unless they wanted to enhance or change the log parsing for + their tests. + - requirements.txt: This text file contains the dependent python libraries + needed for the tests to run. If a new dependent library is needed, it would + be added here as a new entry and then acpi-bits-test-venv.sh would install + it when setting up the python virtual environment using pip. + - README: This text file. + +acpi-bits/bits-config: + This location contains biosbits config files that determine how the software + runs the tests. + - bits-config.txt: this is the biosbits config file that determines what tests + or actions are performed by bits. The description of the config options are + provided in the file itself. + +acpi-bits/prebuilt: + This location contains prebuilt biosbits binaries that are used to generate + the bits iso file for testing. + - bits-2095.zip: The contents from this zip archive are the main contents of + the iso file that are used for testing. This binary zip archive also + contains the full source of the bits software including the full text of + the license agreement listed here: + https://github.com/biosbits/bits/blob/master/COPYING + The source tarball can be found in this location in the zip file: + boot/src/bits-2095.tar.gz + The additional changes beyond those that are present in the official + biosbits github repository can be found here: + https://github.com/ani-sinha/bits/tree/bits-qemu-logging + + Basically these changes can be categorized into the following: + (a) changes to make sure biosbits builds with the latest gcc compiler + (gcc 9.4). + (b) upgrade of acpica to the latest version (march 2022). + (c) send bits logs to the debug IO port at addresss 0x403 so that isa + debugcon can be used to collect the logs. + (d) send a gub halt command to shutdown the VM once all the tests have been + executed. + + This zip file is automatically generated by the bits build system. It can + be found in the bits build directory and it is suffixed by the bits build + number. + Normally, there is no need to make any alteration to this zip archive + unless one wanted to change the bits software itself (for example to add + a new config option or change actions of existing options or change the + debug IO port etc). When such changes are made and a new biosbits software + is needed to be generated, please refresh this zip archive as well as theQuoted 8 lines
tests/qtest/acpi-bits/README line 113
Quoted 6 lines
tests/qtest/acpi-bits/README new file +6
-0+ grub tarball at the same time. Tests will break if changes are made to bits + that are incompatible with existing behavior. So care must be taken to make + sure that the change is compatible with existing bits software as much as + possible. When a new config option is introduced for example, bits must + be upadated here first before introducing the new config option through + a later commit.I don't think playing with source tarballs is a reasonable work environment.
I agree. However, we do not do much with the source tarball. It is there as part of bits to satisfy the license requirement. If we need to manipulate any test script that is in the source file, we would copy it over and keep it in the bits-test directory and change it there.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Let's use submodules just like e.g. firmware does?
Yes I also proposed this to Igor on IRC. We can/maybe need to figure out a place to store the bits source if we think my github is not the best place. However, we need the source only if/when we need to rebuild bits. I suspect it won't be too often if at all.
Why not on git.qemu.org ?
Sounds good to me. Do I get push access? :-)
Quoted 63 lines
Quoted 62 lines
Quoted 62 lines
Quoted 61 lines
tests/qtest/acpi-bits/README new file +55
-1+ - prebuilt/bits-2095-grub.tar.gz: This tarbball contains bits generated grub + scripts and modules to the prebuilt directory. These prebuilt grub + artifacts are required in order to generate a bootable bits iso file that + can run tests. + In order to generate this tar archive, please put the following two + directories that can be found in the bits build directory in a single + tar archive file named as bits-<n>-grub.tar.gz where n is the version of + bits that generated the archive: + + grub-inst-x86_64-efi + grub-inst + + This version should be the same as the version number of bits that generated + bits-<n>.zip file. In other words, the two files must be in sync and should + come from the same build of biosbits of the same version. Things may still + work if they come from different versions but mixing version numbers are + not recommended. + There is normally no need to regenerate this archive unless some fixes or + changes have gone into the grub that are part of biosbits. + +acpi-bits/bits-tests: + This directory contains biosbits python based tests that are run from within + the biosbits environment in the spawned VM. New additions of test cases can + be made in the appropriate test file. For example, new acpi tests can go + into testacpi.py and one would call testsuite.add_test() to register the new + test so that it gets executed as a part of the ACPI tests. + It might be occassionally necessary to disable some subtests or add a new + test that belongs to a test suite not already present in this directory. To + do this, please extract the bits source from the zip file mentioned above. + Copy the test suite/script that needs modification (addition of new tests + or disabling them) from boot/python location of the extracted directory + into this directory. + + step (a): copy unmodified test script to this directory. + step (b): update meson.build and add this file to the list. + Commit (a) and (b) together in the same commit. + + step (c): perform modifications to the test. + Commit (c) in a separate commit. + + The test framework will then use your modified test script to run the test. + No further changes would be needed. Please check the logs to make sure that + appropriate changes have taken effect. + +meson.build files makes sure that the bits qtests are appropriately integrated +into the QEMU qtest framework and are run with "make check-qtest". +Currently, the bits test is configured to run only for x86_64 architectures. Bits +has been built only for this architecture. + + +Author: Ani Sinha <[email protected]> + +References: +(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf +(b) https://www.youtube.com/watch?v=36QIepyUuhg --2.25.1
Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests
Quoted 96 lines · Ani Sinha
Quoted 83 lines · Michael S. Tsirkin
Quoted 81 lines · Ani Sinha
Quoted 66 lines · Michael S. Tsirkin
Quoted 64 lines · Ani Sinha
Quoted 36 lines · Michael S. Tsirkin
Quoted 34 lines · Ani Sinha
The README file is added describing the directory structure and the purpose of every file it contains. It also describes how to add new tests, make changes to existing tests or bits config files or regenerate the bits software.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/README | 168 +++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 tests/qtest/acpi-bits/READMEtests/qtest/acpi-bits/README new file +18
-0@@ -0,0 +1,168 @@ +============================================================================= +ACPI/SMBIOS QTESTS USING BIOSBITS +============================================================================= + +Biosbits is a software written by Josh Triplett that can be downloaded by +visiting https://biosbits.org/. The github codebase can be found here: +https://github.com/biosbits/bits/tree/master. It is a software that exercizes +the bios components such as acpi and smbios tables directly through acpica +bios interpreter (a freely available C based library written by Intel, +downloadable from https://acpica.org/ and is included with biosbits) without an +operating system getting involved in between. +There are several advantages to directly testing the bios in a real physical +machine or VM as opposed to indirectly discovering bios issues through the +operating system. For one thing, the OSes tend to hide bios problems from the +end user. The other is that we have more control of what we wanted to test +and how by directly using acpica interpreter on top of the bios on a running +system. More details on the inspiration for developing biosbits and its real +life uses can be found in (a) and (b).Quoted 8 lines
tests/qtest/acpi-bits/README line 23
Quoted 5 lines
tests/qtest/acpi-bits/README new file +5
-0+This directory contains QEMU qtests written in python that exercizes the QEMU +bios components using biosbits and reports test failures. + +These tests use python virtual environment. In debian/ubuntu system, the tests +would require python3.8-venv and python3-pip packages to be installed.Why do we mess with venv and pip? Certainly possible but what's wrong with using distro provided packages?
There are two things:
(a) We are already using pip and venv for our avocado based integration tests. Look forTESTS_VENV_DIRin Makefile.include under tests.
(b) the venv is primarily needed because I wanted to take advantage of our rich python library that handles QEMU based machines. There are python qtest libraries as well. These are well tested and used with integration tests and I wanted to keep the test part of the code simple by simply reusing them. however, in order to use them, we need a venv environment within which these qemu python libraries are installed. Integration tests does the same thing.A note about my language of choice - python. I gave a lot of thoughts on this. We do not do a lot of stuff here. All we do is:
(a) generate bits iso.
(b) spawn a QEMU vm with the iso which then runs a bunch of tests within the vm.
(c) collect and analyze logs.Quoted 8 lines
Quoted 5 lines
We are not inspecting guest memory or manipulating devices or pci buses. We do not need the power of C here. We need something that is simple to write, easy to maintain and understand and can deal with things like manipulating text files and configs easily. Python seems a better fit for the role.
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.
Quoted 6 lines
It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
It's pretty small for sure. Another submodule?
Quoted 177 lines
Quoted 177 lines
Quoted 176 lines
Quoted 102 lines
Quoted 102 lines
Quoted 87 lines
Quoted 87 lines
tests/qtest/acpi-bits/README new file +84
-0+ +A brief description of the contents of this directory follows: + +├── acpi-bits-test.py +├── acpi-bits-test-venv.sh +├── bits-config +│ ├── bits-cfg.txt +│ └── meson.build +├── bits-tests +│ ├── meson.build +│ ├── smbios.py +│ ├── smilatency.py +│ ├── testacpi.py +│ └── testcpuid.py +├── meson.build +├── prebuilt +│ ├── bits-2095-grub.tar.gz +│ ├── bits-2095.zip +│ └── meson.build +├── README +└── requirements.txt + +acpi-bits: + - acpi-bits-test-venv.sh: This is a shell script that sets up the virtual + environment necessary for the actual python test script to run. Amongst + other things, it makes sure that QEMU python library is available within + that venv so that QEMU machines can be forked. QEMU python library can be + found within python/ directory in QEMU source. + After setting up the virtual environment, it runs the python test script + from within that environment. + If you want to enable verbose mode only for bits test and run make check, + one trick is to add V=1 before the call to execute the python script in + this file. + - acpi-bits-test.py: This is the main python test script that generates a + biosbits iso. It then spawns a QEMU VM with it, collects the logs and reports + test failures. This is the script one would be interested in if they wanted + to add or change some component of the log parsing, add a new commandline to + how QEMU is spawned etc. Test writers typically would not need to modify + this script unless they wanted to enhance or change the log parsing for + their tests. + - requirements.txt: This text file contains the dependent python libraries + needed for the tests to run. If a new dependent library is needed, it would + be added here as a new entry and then acpi-bits-test-venv.sh would install + it when setting up the python virtual environment using pip. + - README: This text file. + +acpi-bits/bits-config: + This location contains biosbits config files that determine how the software + runs the tests. + - bits-config.txt: this is the biosbits config file that determines what tests + or actions are performed by bits. The description of the config options are + provided in the file itself. + +acpi-bits/prebuilt: + This location contains prebuilt biosbits binaries that are used to generate + the bits iso file for testing. + - bits-2095.zip: The contents from this zip archive are the main contents of + the iso file that are used for testing. This binary zip archive also + contains the full source of the bits software including the full text of + the license agreement listed here: + https://github.com/biosbits/bits/blob/master/COPYING + The source tarball can be found in this location in the zip file: + boot/src/bits-2095.tar.gz + The additional changes beyond those that are present in the official + biosbits github repository can be found here: + https://github.com/ani-sinha/bits/tree/bits-qemu-logging + + Basically these changes can be categorized into the following: + (a) changes to make sure biosbits builds with the latest gcc compiler + (gcc 9.4). + (b) upgrade of acpica to the latest version (march 2022). + (c) send bits logs to the debug IO port at addresss 0x403 so that isa + debugcon can be used to collect the logs. + (d) send a gub halt command to shutdown the VM once all the tests have been + executed. + + This zip file is automatically generated by the bits build system. It can + be found in the bits build directory and it is suffixed by the bits build + number. + Normally, there is no need to make any alteration to this zip archive + unless one wanted to change the bits software itself (for example to add + a new config option or change actions of existing options or change the + debug IO port etc). When such changes are made and a new biosbits software + is needed to be generated, please refresh this zip archive as well as theQuoted 8 lines
tests/qtest/acpi-bits/README line 113
Quoted 6 lines
tests/qtest/acpi-bits/README new file +6
-0+ grub tarball at the same time. Tests will break if changes are made to bits + that are incompatible with existing behavior. So care must be taken to make + sure that the change is compatible with existing bits software as much as + possible. When a new config option is introduced for example, bits must + be upadated here first before introducing the new config option through + a later commit.I don't think playing with source tarballs is a reasonable work environment.
I agree. However, we do not do much with the source tarball. It is there as part of bits to satisfy the license requirement. If we need to manipulate any test script that is in the source file, we would copy it over and keep it in the bits-test directory and change it there.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Let's use submodules just like e.g. firmware does?
Yes I also proposed this to Igor on IRC. We can/maybe need to figure out a place to store the bits source if we think my github is not the best place. However, we need the source only if/when we need to rebuild bits. I suspect it won't be too often if at all.
Why not on git.qemu.org ?
Sounds good to me. Do I get push access? :-)
Quoted 63 lines
Quoted 62 lines
Quoted 62 lines
Quoted 61 lines
tests/qtest/acpi-bits/README new file +55
-1+ - prebuilt/bits-2095-grub.tar.gz: This tarbball contains bits generated grub + scripts and modules to the prebuilt directory. These prebuilt grub + artifacts are required in order to generate a bootable bits iso file that + can run tests. + In order to generate this tar archive, please put the following two + directories that can be found in the bits build directory in a single + tar archive file named as bits-<n>-grub.tar.gz where n is the version of + bits that generated the archive: + + grub-inst-x86_64-efi + grub-inst + + This version should be the same as the version number of bits that generated + bits-<n>.zip file. In other words, the two files must be in sync and should + come from the same build of biosbits of the same version. Things may still + work if they come from different versions but mixing version numbers are + not recommended. + There is normally no need to regenerate this archive unless some fixes or + changes have gone into the grub that are part of biosbits. + +acpi-bits/bits-tests: + This directory contains biosbits python based tests that are run from within + the biosbits environment in the spawned VM. New additions of test cases can + be made in the appropriate test file. For example, new acpi tests can go + into testacpi.py and one would call testsuite.add_test() to register the new + test so that it gets executed as a part of the ACPI tests. + It might be occassionally necessary to disable some subtests or add a new + test that belongs to a test suite not already present in this directory. To + do this, please extract the bits source from the zip file mentioned above. + Copy the test suite/script that needs modification (addition of new tests + or disabling them) from boot/python location of the extracted directory + into this directory. + + step (a): copy unmodified test script to this directory. + step (b): update meson.build and add this file to the list. + Commit (a) and (b) together in the same commit. + + step (c): perform modifications to the test. + Commit (c) in a separate commit. + + The test framework will then use your modified test script to run the test. + No further changes would be needed. Please check the logs to make sure that + appropriate changes have taken effect. + +meson.build files makes sure that the bits qtests are appropriately integrated +into the QEMU qtest framework and are run with "make check-qtest". +Currently, the bits test is configured to run only for x86_64 architectures. Bits +has been built only for this architecture. + + +Author: Ani Sinha <[email protected]> + +References: +(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf +(b) https://www.youtube.com/watch?v=36QIepyUuhg --2.25.1
Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests
Quoted 98 lines · Michael S. Tsirkin
Quoted 96 lines · Ani Sinha
Quoted 83 lines · Michael S. Tsirkin
Quoted 81 lines · Ani Sinha
Quoted 66 lines · Michael S. Tsirkin
Quoted 64 lines · Ani Sinha
Quoted 36 lines · Michael S. Tsirkin
Quoted 34 lines · Ani Sinha
The README file is added describing the directory structure and the purpose of every file it contains. It also describes how to add new tests, make changes to existing tests or bits config files or regenerate the bits software.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/README | 168 +++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 tests/qtest/acpi-bits/READMEtests/qtest/acpi-bits/README new file +18
-0@@ -0,0 +1,168 @@ +============================================================================= +ACPI/SMBIOS QTESTS USING BIOSBITS +============================================================================= + +Biosbits is a software written by Josh Triplett that can be downloaded by +visiting https://biosbits.org/. The github codebase can be found here: +https://github.com/biosbits/bits/tree/master. It is a software that exercizes +the bios components such as acpi and smbios tables directly through acpica +bios interpreter (a freely available C based library written by Intel, +downloadable from https://acpica.org/ and is included with biosbits) without an +operating system getting involved in between. +There are several advantages to directly testing the bios in a real physical +machine or VM as opposed to indirectly discovering bios issues through the +operating system. For one thing, the OSes tend to hide bios problems from the +end user. The other is that we have more control of what we wanted to test +and how by directly using acpica interpreter on top of the bios on a running +system. More details on the inspiration for developing biosbits and its real +life uses can be found in (a) and (b).Quoted 8 lines
tests/qtest/acpi-bits/README line 23
Quoted 5 lines
tests/qtest/acpi-bits/README new file +5
-0+This directory contains QEMU qtests written in python that exercizes the QEMU +bios components using biosbits and reports test failures. + +These tests use python virtual environment. In debian/ubuntu system, the tests +would require python3.8-venv and python3-pip packages to be installed.Why do we mess with venv and pip? Certainly possible but what's wrong with using distro provided packages?
There are two things:
(a) We are already using pip and venv for our avocado based integration tests. Look forTESTS_VENV_DIRin Makefile.include under tests.
(b) the venv is primarily needed because I wanted to take advantage of our rich python library that handles QEMU based machines. There are python qtest libraries as well. These are well tested and used with integration tests and I wanted to keep the test part of the code simple by simply reusing them. however, in order to use them, we need a venv environment within which these qemu python libraries are installed. Integration tests does the same thing.A note about my language of choice - python. I gave a lot of thoughts on this. We do not do a lot of stuff here. All we do is:
(a) generate bits iso.
(b) spawn a QEMU vm with the iso which then runs a bunch of tests within the vm.
(c) collect and analyze logs.Quoted 8 lines
Quoted 5 lines
We are not inspecting guest memory or manipulating devices or pci buses. We do not need the power of C here. We need something that is simple to write, easy to maintain and understand and can deal with things like manipulating text files and configs easily. Python seems a better fit for the role.
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.
Quoted 8 lines
Quoted 6 lines
It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.
I am open to anything. Whatever feels right is fine to me.
Quoted 179 lines
Quoted 178 lines
Quoted 178 lines
Quoted 177 lines
Quoted 103 lines
Quoted 103 lines
Quoted 88 lines
Quoted 88 lines
tests/qtest/acpi-bits/README new file +84
-0+ +A brief description of the contents of this directory follows: + +├── acpi-bits-test.py +├── acpi-bits-test-venv.sh +├── bits-config +│ ├── bits-cfg.txt +│ └── meson.build +├── bits-tests +│ ├── meson.build +│ ├── smbios.py +│ ├── smilatency.py +│ ├── testacpi.py +│ └── testcpuid.py +├── meson.build +├── prebuilt +│ ├── bits-2095-grub.tar.gz +│ ├── bits-2095.zip +│ └── meson.build +├── README +└── requirements.txt + +acpi-bits: + - acpi-bits-test-venv.sh: This is a shell script that sets up the virtual + environment necessary for the actual python test script to run. Amongst + other things, it makes sure that QEMU python library is available within + that venv so that QEMU machines can be forked. QEMU python library can be + found within python/ directory in QEMU source. + After setting up the virtual environment, it runs the python test script + from within that environment. + If you want to enable verbose mode only for bits test and run make check, + one trick is to add V=1 before the call to execute the python script in + this file. + - acpi-bits-test.py: This is the main python test script that generates a + biosbits iso. It then spawns a QEMU VM with it, collects the logs and reports + test failures. This is the script one would be interested in if they wanted + to add or change some component of the log parsing, add a new commandline to + how QEMU is spawned etc. Test writers typically would not need to modify + this script unless they wanted to enhance or change the log parsing for + their tests. + - requirements.txt: This text file contains the dependent python libraries + needed for the tests to run. If a new dependent library is needed, it would + be added here as a new entry and then acpi-bits-test-venv.sh would install + it when setting up the python virtual environment using pip. + - README: This text file. + +acpi-bits/bits-config: + This location contains biosbits config files that determine how the software + runs the tests. + - bits-config.txt: this is the biosbits config file that determines what tests + or actions are performed by bits. The description of the config options are + provided in the file itself. + +acpi-bits/prebuilt: + This location contains prebuilt biosbits binaries that are used to generate + the bits iso file for testing. + - bits-2095.zip: The contents from this zip archive are the main contents of + the iso file that are used for testing. This binary zip archive also + contains the full source of the bits software including the full text of + the license agreement listed here: + https://github.com/biosbits/bits/blob/master/COPYING + The source tarball can be found in this location in the zip file: + boot/src/bits-2095.tar.gz + The additional changes beyond those that are present in the official + biosbits github repository can be found here: + https://github.com/ani-sinha/bits/tree/bits-qemu-logging + + Basically these changes can be categorized into the following: + (a) changes to make sure biosbits builds with the latest gcc compiler + (gcc 9.4). + (b) upgrade of acpica to the latest version (march 2022). + (c) send bits logs to the debug IO port at addresss 0x403 so that isa + debugcon can be used to collect the logs. + (d) send a gub halt command to shutdown the VM once all the tests have been + executed. + + This zip file is automatically generated by the bits build system. It can + be found in the bits build directory and it is suffixed by the bits build + number. + Normally, there is no need to make any alteration to this zip archive + unless one wanted to change the bits software itself (for example to add + a new config option or change actions of existing options or change the + debug IO port etc). When such changes are made and a new biosbits software + is needed to be generated, please refresh this zip archive as well as theQuoted 8 lines
tests/qtest/acpi-bits/README line 113
Quoted 6 lines
tests/qtest/acpi-bits/README new file +6
-0+ grub tarball at the same time. Tests will break if changes are made to bits + that are incompatible with existing behavior. So care must be taken to make + sure that the change is compatible with existing bits software as much as + possible. When a new config option is introduced for example, bits must + be upadated here first before introducing the new config option through + a later commit.I don't think playing with source tarballs is a reasonable work environment.
I agree. However, we do not do much with the source tarball. It is there as part of bits to satisfy the license requirement. If we need to manipulate any test script that is in the source file, we would copy it over and keep it in the bits-test directory and change it there.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Let's use submodules just like e.g. firmware does?
Yes I also proposed this to Igor on IRC. We can/maybe need to figure out a place to store the bits source if we think my github is not the best place. However, we need the source only if/when we need to rebuild bits. I suspect it won't be too often if at all.
Why not on git.qemu.org ?
Sounds good to me. Do I get push access? :-)
Quoted 63 lines
Quoted 62 lines
Quoted 62 lines
Quoted 61 lines
tests/qtest/acpi-bits/README new file +55
-1+ - prebuilt/bits-2095-grub.tar.gz: This tarbball contains bits generated grub + scripts and modules to the prebuilt directory. These prebuilt grub + artifacts are required in order to generate a bootable bits iso file that + can run tests. + In order to generate this tar archive, please put the following two + directories that can be found in the bits build directory in a single + tar archive file named as bits-<n>-grub.tar.gz where n is the version of + bits that generated the archive: + + grub-inst-x86_64-efi + grub-inst + + This version should be the same as the version number of bits that generated + bits-<n>.zip file. In other words, the two files must be in sync and should + come from the same build of biosbits of the same version. Things may still + work if they come from different versions but mixing version numbers are + not recommended. + There is normally no need to regenerate this archive unless some fixes or + changes have gone into the grub that are part of biosbits. + +acpi-bits/bits-tests: + This directory contains biosbits python based tests that are run from within + the biosbits environment in the spawned VM. New additions of test cases can + be made in the appropriate test file. For example, new acpi tests can go + into testacpi.py and one would call testsuite.add_test() to register the new + test so that it gets executed as a part of the ACPI tests. + It might be occassionally necessary to disable some subtests or add a new + test that belongs to a test suite not already present in this directory. To + do this, please extract the bits source from the zip file mentioned above. + Copy the test suite/script that needs modification (addition of new tests + or disabling them) from boot/python location of the extracted directory + into this directory. + + step (a): copy unmodified test script to this directory. + step (b): update meson.build and add this file to the list. + Commit (a) and (b) together in the same commit. + + step (c): perform modifications to the test. + Commit (c) in a separate commit. + + The test framework will then use your modified test script to run the test. + No further changes would be needed. Please check the logs to make sure that + appropriate changes have taken effect. + +meson.build files makes sure that the bits qtests are appropriately integrated +into the QEMU qtest framework and are run with "make check-qtest". +Currently, the bits test is configured to run only for x86_64 architectures. Bits +has been built only for this architecture. + + +Author: Ani Sinha <[email protected]> + +References: +(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf +(b) https://www.youtube.com/watch?v=36QIepyUuhg --2.25.1
venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 106 lines · Ani Sinha
Quoted 105 lines · Michael S. Tsirkin
Quoted 102 lines · Ani Sinha
Quoted 83 lines · Michael S. Tsirkin
Quoted 81 lines · Ani Sinha
Quoted 66 lines · Michael S. Tsirkin
Quoted 64 lines · Ani Sinha
Quoted 36 lines · Michael S. Tsirkin
Quoted 34 lines · Ani Sinha
The README file is added describing the directory structure and the purpose of every file it contains. It also describes how to add new tests, make changes to existing tests or bits config files or regenerate the bits software.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/README | 168 +++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+)create mode 100644 tests/qtest/acpi-bits/README
tests/qtest/acpi-bits/README new file +18
-0@@ -0,0 +1,168 @@ +============================================================================= +ACPI/SMBIOS QTESTS USING BIOSBITS +============================================================================= + +Biosbits is a software written by Josh Triplett that can be downloaded by +visiting https://biosbits.org/. The github codebase can be found here: +https://github.com/biosbits/bits/tree/master. It is a software that exercizes +the bios components such as acpi and smbios tables directly through acpica +bios interpreter (a freely available C based library written by Intel, +downloadable from https://acpica.org/ and is included with biosbits) without an +operating system getting involved in between. +There are several advantages to directly testing the bios in a real physical +machine or VM as opposed to indirectly discovering bios issues through the +operating system. For one thing, the OSes tend to hide bios problems from the +end user. The other is that we have more control of what we wanted to test +and how by directly using acpica interpreter on top of the bios on a running +system. More details on the inspiration for developing biosbits and its real +life uses can be found in (a) and (b).Quoted 8 lines
tests/qtest/acpi-bits/README line 23
Quoted 5 lines
tests/qtest/acpi-bits/README new file +5
-0+This directory contains QEMU qtests written in python that exercizes the QEMU +bios components using biosbits and reports test failures. + +These tests use python virtual environment. In debian/ubuntu system, the tests +would require python3.8-venv and python3-pip packages to be installed.Why do we mess with venv and pip? Certainly possible but what's wrong with using distro provided packages?
There are two things:
(a) We are already using pip and venv for our avocado based integration tests. Look forTESTS_VENV_DIRin Makefile.include under tests.
(b) the venv is primarily needed because I wanted to take advantage of our rich python library that handles QEMU based machines. There are python qtest libraries as well. These are well tested and used with integration tests and I wanted to keep the test part of the code simple by simply reusing them. however, in order to use them, we need a venv environment within which these qemu python libraries are installed. Integration tests does the same thing.A note about my language of choice - python. I gave a lot of thoughts on this. We do not do a lot of stuff here. All we do is:
(a) generate bits iso.
(b) spawn a QEMU vm with the iso which then runs a bunch of tests within the vm.
(c) collect and analyze logs.Quoted 8 lines
Quoted 5 lines
We are not inspecting guest memory or manipulating devices or pci buses. We do not need the power of C here. We need something that is simple to write, easy to maintain and understand and can deal with things like manipulating text files and configs easily. Python seems a better fit for the role.
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 34 lines · Thomas Huth
Quoted 34 lines
Quoted 20 lines
Quoted 20 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.Quoted 8 lines
Quoted 6 lines
It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.
Quoted 6 lines
Quoted 1 line
I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
Signature · 1 line
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 40 lines · Michael S. Tsirkin
Quoted 39 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Quoted 6 lines
Quoted 1 line
Thomas
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Thomas
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 43 lines · Thomas Huth
Quoted 42 lines · Michael S. Tsirkin
Quoted 40 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
Quoted 8 lines
Quoted 4 lines
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Definitely not submodules.
We need to get out of the mindset that submodules are needed for every new dependancy we add. Submodules are only appropriate if the external project is designed to be used as a copylib (eg the keycodemapdb tool), or if we need to bundle in order to prevent a regression for previously deployed QEMU installs where the dependancy is known not to exist on all our supported platforms.
This does not apply in this case, because the proposed use of tappy is merely for a test case. Meson just needs to check if tappy exists and if it does, then use it, otherwise skip the tests that need it. The user can arrange to install tappy, as they do with the majority of other deps.
If John's venv stuff is relevant, then we don't even need the meson checks, just delegate to the venv setup.
Regardless, no submodules are needed or desirable.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 68 lines · Daniel P. Berrangé
Quoted 43 lines · Thomas Huth
Quoted 42 lines · Michael S. Tsirkin
Quoted 40 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
Quoted 8 lines
Quoted 4 lines
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Definitely not submodules.
We need to get out of the mindset that submodules are needed for every new dependancy we add. Submodules are only appropriate if the external project is designed to be used as a copylib (eg the keycodemapdb tool), or if we need to bundle in order to prevent a regression for previously deployed QEMU installs where the dependancy is known not to exist on all our supported platforms.
This does not apply in this case, because the proposed use of tappy is merely for a test case. Meson just needs to check if tappy exists and if it does, then use it, otherwise skip the tests that need it. The user can arrange to install tappy, as they do with the majority of other deps.
Quoted 4 lines
If John's venv stuff is relevant, then we don't even need the meson checks, just delegate to the venv setup.
Regardless, no submodules are needed or desirable.
What about keeping biosbits stuff? Source or pre-built.
Quoted 8 lines
With regards,
Daniel
--|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 69 lines · Ani Sinha
Quoted 68 lines · Daniel P. Berrangé
Quoted 43 lines · Thomas Huth
Quoted 42 lines · Michael S. Tsirkin
Quoted 40 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
Quoted 8 lines
Quoted 4 lines
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Definitely not submodules.
We need to get out of the mindset that submodules are needed for every new dependancy we add. Submodules are only appropriate if the external project is designed to be used as a copylib (eg the keycodemapdb tool), or if we need to bundle in order to prevent a regression for previously deployed QEMU installs where the dependancy is known not to exist on all our supported platforms.
This does not apply in this case, because the proposed use of tappy is merely for a test case. Meson just needs to check if tappy exists and if it does, then use it, otherwise skip the tests that need it. The user can arrange to install tappy, as they do with the majority of other deps.
Quoted 6 lines
Quoted 4 lines
If John's venv stuff is relevant, then we don't even need the meson checks, just delegate to the venv setup.
Regardless, no submodules are needed or desirable.
What about keeping biosbits stuff? Source or pre-built.
Shipping them as pre-built binaries in QEMU is not a viable option IMHO, especially for grub as a GPL'd project we need to be extremely clear about the exact corresponding source and build process for any binary.
For this kind of thing I would generally expect the distro to provide packages that we consume. Looking at biosbits I see it is itself bundling a bunch more 3rd party projects, libffi, grub2, and including even an ancient version of python as a submodule.
So bundling a pre-built biosbits in QEMU appears to mean that we're in turn going to unexpectedly bundle a bunch of other 3rd party projects too, all with dubious license compliance. I don't think this looks like something we should have in qemu.git or qemu tarballs. It will also make it challenging for the distro to take biosbits at all, unless those 3rd party bundles can be eliminated in favour of using existing builds their have packaged for grub, python, libffi, etc.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 87 lines · Daniel P. Berrangé
Quoted 69 lines · Ani Sinha
Quoted 68 lines · Daniel P. Berrangé
Quoted 43 lines · Thomas Huth
Quoted 42 lines · Michael S. Tsirkin
Quoted 40 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
Quoted 8 lines
Quoted 4 lines
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Definitely not submodules.
We need to get out of the mindset that submodules are needed for every new dependancy we add. Submodules are only appropriate if the external project is designed to be used as a copylib (eg the keycodemapdb tool), or if we need to bundle in order to prevent a regression for previously deployed QEMU installs where the dependancy is known not to exist on all our supported platforms.
This does not apply in this case, because the proposed use of tappy is merely for a test case. Meson just needs to check if tappy exists and if it does, then use it, otherwise skip the tests that need it. The user can arrange to install tappy, as they do with the majority of other deps.
Quoted 6 lines
Quoted 4 lines
If John's venv stuff is relevant, then we don't even need the meson checks, just delegate to the venv setup.
Regardless, no submodules are needed or desirable.
What about keeping biosbits stuff? Source or pre-built.
Shipping them as pre-built binaries in QEMU is not a viable option IMHO, especially for grub as a GPL'd project we need to be extremely clear about the exact corresponding source and build process for any binary.
For this kind of thing I would generally expect the distro to provide packages that we consume. Looking at biosbits I see it is itself bundling a bunch more 3rd party projects, libffi, grub2, and including even an ancient version of python as a submodule.
Quoted 7 lines
So bundling a pre-built biosbits in QEMU appears to mean that we're in turn going to unexpectedly bundle a bunch of other 3rd party projects too, all with dubious license compliance. I don't think this looks like something we should have in qemu.git or qemu tarballs. It will also make it challenging for the distro to take biosbits at all, unless those 3rd party bundles can be eliminated in favour of using existing builds their have packaged for grub, python, libffi, etc.
So if this depends on some third party binary bits, I think this is pretty similar to the tests in the avocado directory ... there we download third party binaries, too... Wouldn't it make sense to adapt your tests to that framework?
Thomas
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 97 lines · Thomas Huth
Quoted 87 lines · Daniel P. Berrangé
Quoted 69 lines · Ani Sinha
Quoted 68 lines · Daniel P. Berrangé
Quoted 43 lines · Thomas Huth
Quoted 42 lines · Michael S. Tsirkin
Quoted 40 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
No problem with that. So that's venv. But do we need pip and pulling
packages from the net during testing?We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
Quoted 8 lines
Quoted 4 lines
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Definitely not submodules.
We need to get out of the mindset that submodules are needed for every new dependancy we add. Submodules are only appropriate if the external project is designed to be used as a copylib (eg the keycodemapdb tool), or if we need to bundle in order to prevent a regression for previously deployed QEMU installs where the dependancy is known not to exist on all our supported platforms.
This does not apply in this case, because the proposed use of tappy is merely for a test case. Meson just needs to check if tappy exists and if it does, then use it, otherwise skip the tests that need it. The user can arrange to install tappy, as they do with the majority of other deps.
Quoted 6 lines
Quoted 4 lines
If John's venv stuff is relevant, then we don't even need the meson checks, just delegate to the venv setup.
Regardless, no submodules are needed or desirable.
What about keeping biosbits stuff? Source or pre-built.
Shipping them as pre-built binaries in QEMU is not a viable option IMHO, especially for grub as a GPL'd project we need to be extremely clear about the exact corresponding source and build process for any binary.
For this kind of thing I would generally expect the distro to provide packages that we consume. Looking at biosbits I see it is itself bundling a bunch more 3rd party projects, libffi, grub2, and including even an ancient version of python as a submodule.
Quoted 7 lines
So bundling a pre-built biosbits in QEMU appears to mean that we're in turn going to unexpectedly bundle a bunch of other 3rd party projects too, all with dubious license compliance. I don't think this looks like something we should have in qemu.git or qemu tarballs. It will also make it challenging for the distro to take biosbits at all, unless those 3rd party bundles can be eliminated in favour of using existing builds their have packaged for grub, python, libffi, etc.
Quoted 4 lines
So if this depends on some third party binary bits, I think this is pretty similar to the tests in the avocado directory ... there we download third party binaries, too... Wouldn't it make sense to adapt your tests to that framework?
I do not want to bring in the whole avocado framework because it would unnecessarily make things complicated. I just need the qemu machine python library and that is enough. For downloading third party stuff, we can simply wget things from somewhere.
Quoted 3 lines
Thomas
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 115 lines · On Tue, Jun 28, 2022 at 14:05 Ani Sinha <[email protected]> wrote:
Quoted 4 lines · Thomas Huth
Quoted 2 lines · Daniel P. Berrangé
Quoted 1 line · Ani Sinha
On Tue, Jun 28, 2022 at 1:19 PM Daniel P. Berrangé <
Quoted 5 lines · [email protected]> wrote:
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines · Thomas Huth
Quoted 3 lines · Michael S. Tsirkin
Quoted 2 lines · Thomas Huth
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
No problem with that. So that's venv. But do we need pip
and pulling
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 14 lines
Quoted 10 lines
packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic
library
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
for parsing test output that maybe we can keep this somewhere
within
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
the python src tree? Not sure ...
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a
while and
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of
various QEMU
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 8 lines
Quoted 3 lines
bits, I think you should loop him into this discussion, too.
Thomas
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So
before you
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
add yet another submodule (which have a couple of other
disadvantages), I
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
think you could also directly use the venv here.
Definitely not submodules.
We need to get out of the mindset that submodules are needed for
every new
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
dependancy we add. Submodules are only appropriate if the external
project
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
is designed to be used as a copylib (eg the keycodemapdb tool), or
if we
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
need to bundle in order to prevent a regression for previously
deployed
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
QEMU installs where the dependancy is known not to exist on all our supported platforms.
This does not apply in this case, because the proposed use of tappy
is
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
merely for a test case. Meson just needs to check if tappy exists
and if
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
it does, then use it, otherwise skip the tests that need it. The
user can
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
arrange to install tappy, as they do with the majority of other deps.
If John's venv stuff is relevant, then we don't even need the meson
checks,
Quoted 24 lines
Quoted 16 lines
Quoted 5 lines
Quoted 3 lines
just delegate to the venv setup.
Regardless, no submodules are needed or desirable.
What about keeping biosbits stuff? Source or pre-built.
Shipping them as pre-built binaries in QEMU is not a viable option IMHO, especially for grub as a GPL'd project we need to be extremely clear about the exact corresponding source and build process for any binary.
For this kind of thing I would generally expect the distro to provide packages that we consume. Looking at biosbits I see it is itself bundling a bunch more 3rd party projects, libffi, grub2, and including even an ancient version of python as a submodule.
Quoted 7 lines
So bundling a pre-built biosbits in QEMU appears to mean that we're in turn going to unexpectedly bundle a bunch of other 3rd party projects too, all with dubious license compliance. I don't think this looks like something we should have in qemu.git or qemu tarballs. It will also make it challenging for the distro to take biosbits at all, unless those 3rd party bundles can be eliminated in favour of using existing builds their have packaged for grub, python, libffi, etc.
Quoted 1 line
So if this depends on some third party binary bits, I think this is
Quoted 8 lines
pretty
Quoted 3 lines
similar to the tests in the avocado directory ... there we download third party binaries, too... Wouldn't it make sense to adapt your tests to that framework?
I do not want to bring in the whole avocado framework because it would unnecessarily make things complicated. I just need the qemu machine python library and that is enough. For downloading third party stuff,
we can simply wget things from somewhere.
https://pypi.org/project/wget/
That get_asset() call is an overkill for downloading two archives.
Quoted 5 lines
Quoted 4 lines
Thomas
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 128 lines · Ani Sinha
On Tue, Jun 28, 2022 at 14:05 Ani Sinha <[email protected]> wrote:
On Tue, Jun 28, 2022 at 1:58 PM Thomas Huth <[email protected]> wrote:
>
> On 28/06/2022 10.23, Daniel P. Berrangé wrote:
> > On Tue, Jun 28, 2022 at 01:21:35PM +0530, Ani Sinha wrote:
> >> On Tue, Jun 28, 2022 at 1:19 PM Daniel P. Berrangé < [email protected]> wrote:
> >>>
> >>> On Tue, Jun 28, 2022 at 09:25:35AM +0200, Thomas Huth wrote:
> >>>> On 28/06/2022 09.10, Michael S. Tsirkin wrote:
> >>>>> On Tue, Jun 28, 2022 at 09:03:33AM +0200, Thomas Huth wrote:
> >>>>>>>>>>>> No problem with that. So that's venv. But do we need pip and pulling
> >>>>>>>>>>>> packages from the net during testing?
> >>>>>>>>>>>
> >>>>>>>>>>> We do that too. See requirements.txt in tests/
> >>>>>>>>>>> Following two are downloaded:
> >>>>>>>>>>> avocado-framework==88.1
> >>>>>>>>>>> pycdlib==1.11.0
> >>>>>>>>>>>
> >>>>>>>>>>> Also see this line in Makefie.include:
> >>>>>>>>>>>
> >>>>>>>>>>> $(call quiet-venv-pip,install -r $(TESTS_VENV_REQ)) > >>>>>>>>>>
> >>>>>>>>>> Right but that's avocado since it pulls lots of stuff from > >>>>>>>>>> the net anyway.
> >>>>>>>>>> Are the libraries in question not packaged on major distros? > >>>>>>>>>
> >>>>>>>>> Currently I only need this:
> >>>>>>>>> https://github.com/python-tap/tappy
> >>>>>>>>> which is the basic TAP processing library for python. > >>>>>>>>>
> >>>>>>>>> It seems its only installed through pip:
> >>>>>>>>> https://tappy.readthedocs.io/en/latest/
> >>>>>>>>>
> >>>>>>>>> I do not think this is packaged by default. It's such a basic library
> >>>>>>>>> for parsing test output that maybe we can keep this somewhere within
> >>>>>>>>> the python src tree? Not sure ...
> >>>>>>>>
> >>>>>>>> It's pretty small for sure. Another submodule?
> >>>>>>>
> >>>>>>> Unlike BITS, this one is likely going to be maintained for a while and
> >>>>>>> will receive new releases through
> >>>>>>> https://pypi.org/project/tap.py/
> >>>>>>> so forking is OK but someone has to keep this updated. > >>>>>>>
> >>>>>>> I am open to anything. Whatever feels right is fine to me. > >>>>>>
> >>>>>> John Snow is currently working on the "Pythonification" of various QEMU
> >>>>>> bits, I think you should loop him into this discussion, too. > >>>>>>
> >>>>>> Thomas
> >>>>>
> >>>>> submodule does not mean we fork necessarily. We could have > >>>>> all options: check for the module and use it if there, if not > >>>>> use one from system if not there install with pip ..
> >>>>> But yea, I'm not sure what's best either.
> >>>>
> >>>> submodules create a dependency on an internet connection, too. So before you
> >>>> add yet another submodule (which have a couple of other disadvantages), I
> >>>> think you could also directly use the venv here.
> >>>
> >>> Definitely not submodules.
> >>>
> >>> We need to get out of the mindset that submodules are needed for every new
> >>> dependancy we add. Submodules are only appropriate if the external project
> >>> is designed to be used as a copylib (eg the keycodemapdb tool), or if we
> >>> need to bundle in order to prevent a regression for previously deployed
> >>> QEMU installs where the dependancy is known not to exist on all our > >>> supported platforms.
> >>>
> >>> This does not apply in this case, because the proposed use of tappy is
> >>> merely for a test case. Meson just needs to check if tappy exists and if
> >>> it does, then use it, otherwise skip the tests that need it. The user can
> >>> arrange to install tappy, as they do with the majority of other deps. > >>>
> >>> If John's venv stuff is relevant, then we don't even need the meson checks,
> >>> just delegate to the venv setup.
> >>>
> >>> Regardless, no submodules are needed or desirable.
> >>
> >> What about keeping biosbits stuff? Source or pre-built.
> >
> > Shipping them as pre-built binaries in QEMU is not a viable option > > IMHO, especially for grub as a GPL'd project we need to be extremely > > clear about the exact corresponding source and build process for any > > binary.
> >
> > For this kind of thing I would generally expect the distro to provide > > packages that we consume. Looking at biosbits I see it is itself > > bundling a bunch more 3rd party projects, libffi, grub2, and including > > even an ancient version of python as a submodule.
> >
> > So bundling a pre-built biosbits in QEMU appears to mean that we're in > > turn going to unexpectedly bundle a bunch of other 3rd party projects > > too, all with dubious license compliance. I don't think this looks like > > something we should have in qemu.git or qemu tarballs. It will also > > make it challenging for the distro to take biosbits at all, unless > > those 3rd party bundles can be eliminated in favour of using existing > > builds their have packaged for grub, python, libffi, etc. >
> So if this depends on some third party binary bits, I think this is pretty
> similar to the tests in the avocado directory ... there we download third > party binaries, too... Wouldn't it make sense to adapt your tests to that > framework?I do not want to bring in the whole avocado framework because it would unnecessarily make things complicated. I just need the qemu machine python library and that is enough. For downloading third party stuff,
Quoted 7 lines
we can simply wget things from somewhere.
https://pypi.org/project/wget/
That
get_asset()call is an overkill for downloading two archives.
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
Quoted 6 lines
Quoted 5 lines
Thomas
Why we should avoid new submodules if possible
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]
Quoted 2 lines
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
As far as I know, you can also specify the version when using pip, can't you? So that's not really an advantage here.
On the contrary, submodules have a couple of disadvantages that I really dislike:
- submodules do not get updated automatically when doing a "git checkout", we have to update them via a script instead. This causes e.g. trouble if you rsync your source tree to a machine that has no access to the internet and you forgot to update the submodule before the sync
- the content of submodules is not added to the tarballs that get created on the git forges automatically. There were lots of requests from users in the past that tried to download a tarball from github and then wondered why they couldn't compile QEMU.
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule,
where people did not understand that they should report that
rather to the original project instead (i.e. you ship it - you
own it)
* People get the impression that QEMU is a huge monster
application if they count the number of code lines, run
their code scanner tools on the tarball contents, etc.
Remember "nemu", for example, where one of the main complaints
was that QEMU has too many lines of code?
- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
So in my opinion we should avoid new submodules if there is an alternative.
Thomas
Re: Why we should avoid new submodules if possible
Quoted 7 lines · Thomas Huth
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]Quoted 2 lines
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
As far as I know, you can also specify the version when using pip, can't you? So that's not really an advantage here.
But of course if you do you do not get updates ;) You do however rely on a 3rd party to faithfully provide you correct code based on the version, and host it forever.
Quoted 7 lines
On the contrary, submodules have a couple of disadvantages that I really dislike:
- submodules do not get updated automatically when doing a "git checkout", we have to update them via a script instead. This causes e.g. trouble if you rsync your source tree to a machine that has no access to the internet and you forgot to update the submodule before the sync
how is pip better?
Quoted 4 lines
- the content of submodules is not added to the tarballs that get created on the git forges automatically. There were lots of requests from users in the past that tried to download a tarball from github and then wondered why they couldn't compile QEMU.
how is pip better here?
Quoted 4 lines
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule,
Quoted 8 lines
where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?
I think we can skip the checkout in the tarball if we like. If people want to run the test they can checkout then.
Quoted 4 lines
- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
Come on, this is just a test. We *really* don't care if an ISO we use to test ACPI is using an exploitable version of grub.
Quoted 3 lines
So in my opinion we should avoid new submodules if there is an alternative.
Thomas
Interesting take generally, but I don't see how the logic applies in this case. Would appreciate hearing your answers to the above.
Signature · 1 line
MST
Re: Why we should avoid new submodules if possible
Quoted 14 lines · Michael S. Tsirkin
Quoted 4 lines · Thomas Huth
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule,Quoted 8 lines
where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?
Quoted 2 lines
I think we can skip the checkout in the tarball if we like. If people want to run the test they can checkout then.
For tarballs and submodules, we want to provide the code in the cases where we're providing binary blobs, and for where it's required to build QEMU proper.
Overall I think that the approach we use today for providing guest binaries (submodules with the code, pre-built blobs checked into git) is creaking at the seams and often awkward for downstream distros (who want to rebuild the binaries anyway).
Plus submodules in general in git work really badly and awkwardly, and I'd rather we didn't add them unless we really must.
We already have an approach for "tests that use binaries" -- the avocado test suites. Is that something we could use in this case ?
thanks
-- PMM
Re: Why we should avoid new submodules if possible
Quoted 30 lines · Peter Maydell
Quoted 14 lines · Michael S. Tsirkin
Quoted 4 lines · Thomas Huth
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule,Quoted 8 lines
where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?Quoted 2 lines
I think we can skip the checkout in the tarball if we like. If people want to run the test they can checkout then.
For tarballs and submodules, we want to provide the code in the cases where we're providing binary blobs, and for where it's required to build QEMU proper.
Overall I think that the approach we use today for providing guest binaries (submodules with the code, pre-built blobs checked into git) is creaking at the seams and often awkward for downstream distros (who want to rebuild the binaries anyway).
Plus submodules in general in git work really badly and awkwardly, and I'd rather we didn't add them unless we really must.
Quoted 6 lines
We already have an approach for "tests that use binaries" -- the avocado test suites. Is that something we could use in this case ?
thanks
-- PMM
From experience avocado is basically rolling its own reimplementation of git fetch. Result is weird failures when some corner case or another isn't handled perfectly. Kind of unavoidable since you can't have ISOs for a ton of guest OSes reasonably in a git repo. Well maybe with sparse checkouts you can. Haven't really looked at it.
These specific tests are weird, kind of special in that they are developer oriented. I *would* like anyone making changes to ACPI to run them, and for CI to run them on patches, but I do not believe e.g. distros necessarily need them.
git submodules are awkward basically because they are an automated wget. I don't think an explicit wget is much better ... but
looks like I'm alone in this. Oh well.
So it will be a weird dance of wget a tarball, unpack, generate ISO and run. God help you if you need to patch the test - it's wget all the way down.
Signature · 1 line
MST
Re: Why we should avoid new submodules if possible
Quoted 4 lines · Michael S. Tsirkin
git submodules are awkward basically because they are an automated wget. I don't think an explicit wget is much better ... but
looks like I'm alone in this. Oh well.
submodules are 90% of the hassles I have in upstreaming and updating the bsd-user stuff. They play terribly with branches, rebases, and any number of other things and are a pain to reset at times. They are easily the most brittle,
error-prone and difficult to use feature of git.
I hate them with a burning passion. You are not alone.
Warner
Re: Why we should avoid new submodules if possible
In reply to Michael S. TsirkinHello,
Quoted 7 lines · Michael S. Tsirkin
git submodules are awkward basically because they are an automated wget. I don't think an explicit wget is much better ... but
looks like I'm alone in this. Oh well.
So it will be a weird dance of wget a tarball, unpack, generate ISO and run. God help you if you need to patch the test - it's wget all the way down.
That's the problem - the submodules are not automated. They are half-automated, and the rules for when the automation works and for when the automation falls apart are not intellibible for the general Joe Developer.
You might spend a few days studying how they behave exactly, and then you will know. But unless you will use them every day you will forget again, because the rules do not lend themselves to some abstraction easily understood by humans.
Thanks
Michal
Re: Why we should avoid new submodules if possible
Quoted 17 lines · Michal Suchánek
Hello,
Quoted 7 lines · Michael S. Tsirkin
git submodules are awkward basically because they are an automated wget. I don't think an explicit wget is much better ... but
looks like I'm alone in this. Oh well.
So it will be a weird dance of wget a tarball, unpack, generate ISO and run. God help you if you need to patch the test - it's wget all the way down.That's the problem - the submodules are not automated. They are half-automated, and the rules for when the automation works and for when the automation falls apart are not intellibible for the general Joe Developer.
Quoted 8 lines
You might spend a few days studying how they behave exactly, and then you will know. But unless you will use them every day you will forget again, because the rules do not lend themselves to some abstraction easily understood by humans.
Thanks
Michal
But why would learning a different tool be easier?
Signature · 1 line
MST
Re: Why we should avoid new submodules if possible
Quoted 23 lines · Michael S. Tsirkin
Quoted 22 lines · Michal Suchánek
Hello,
Quoted 7 lines · Michael S. Tsirkin
git submodules are awkward basically because they are an automated wget. I don't think an explicit wget is much better ... but
looks like I'm alone in this. Oh well.
So it will be a weird dance of wget a tarball, unpack, generate ISO and run. God help you if you need to patch the test - it's wget all the way down.That's the problem - the submodules are not automated. They are half-automated, and the rules for when the automation works and for when the automation falls apart are not intellibible for the general Joe Developer.
You might spend a few days studying how they behave exactly, and then you will know. But unless you will use them every day you will forget again, because the rules do not lend themselves to some abstraction easily understood by humans.
Quoted 5 lines
Quoted 3 lines
Thanks
Michal
But why would learning a different tool be easier?
a) it's working lends itself to explaining in human-intelligible concepts
b) it's used elsewhere as well
Thanks
Michal
Re: Why we should avoid new submodules if possible
In reply to Michael S. TsirkinQuoted 16 lines · Michael S. Tsirkin
Quoted 7 lines · Thomas Huth
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]Quoted 2 lines
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
As far as I know, you can also specify the version when using pip, can't you? So that's not really an advantage here.
But of course if you do you do not get updates ;) You do however rely on a 3rd party to faithfully provide you correct code based on the version, and host it forever.
Quoted 3 lines
On the contrary, submodules have a couple of disadvantages that I really dislike:
Quoted 6 lines
Quoted 4 lines
- submodules do not get updated automatically when doing a "git checkout", we have to update them via a script instead. This causes e.g. trouble if you rsync your source tree to a machine that has no access to the internet and you forgot to update the submodule before the sync
how is pip better?
You don't end up with an inconsistent source tree in that case (which happens with submodules quite easily, at least for me it happened a couple of times already). Either the machine has an internet connection, so that pip can install the missing bits, or it does not and the test has to be skipped. But if I copy the wrong state of a submodule around, things get messed up quite easily in my experience. Ok, you could say that this is just my special setup with rsync, but already given the fact that "git checkout" creates an inconsistent state of your source tree until you run the script for updating the submodules the next time is an indication that submodules are rather a shaky thing (e.g. if you'd create a tarball for somebody else from your source tree right after doing a "git checkout").
Quoted 6 lines
Quoted 4 lines
- the content of submodules is not added to the tarballs that get created on the git forges automatically. There were lots of requests from users in the past that tried to download a tarball from github and then wondered why they couldn't compile QEMU.
how is pip better here?
You don't get incomplete/non-working tarballs in that case.
Quoted 13 lines
Quoted 4 lines
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule,Quoted 8 lines
where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?
Quoted 2 lines
I think we can skip the checkout in the tarball if we like. If people want to run the test they can checkout then.
Release tarballs don't include the ".git" folder infrastructur, so everybody who downloads a tarball will simply never be able to run the test.
Quoted 8 lines
Quoted 5 lines
- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
Come on, this is just a test. We *really* don't care if an ISO we use to test ACPI is using an exploitable version of grub.
Wait, I thought we were only talking about tappy here? The ISO binaries should certainly *not* be bundled in the QEMU tarballs (they are too big already anyway, we should rather think of moving the firmware binaries out of the tarball instead).
Thomas
Re: Why we should avoid new submodules if possible
Quoted 22 lines · Thomas Huth
Quoted 21 lines · Michael S. Tsirkin
Quoted 7 lines · Thomas Huth
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]Quoted 2 lines
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
As far as I know, you can also specify the version when using pip, can't you? So that's not really an advantage here.
But of course if you do you do not get updates ;) You do however rely on a 3rd party to faithfully provide you correct code based on the version, and host it forever.
Quoted 7 lines
On the contrary, submodules have a couple of disadvantages that I really dislike:
- submodules do not get updated automatically when doing a "git checkout", we have to update them via a script instead. This causes e.g. trouble if you rsync your source tree to a machine that has no access to the internet and you forgot to update the submodule before the sync
Quoted 7 lines
Quoted 1 line
how is pip better?
You don't end up with an inconsistent source tree in that case (which happens with submodules quite easily, at least for me it happened a couple of times already). Either the machine has an internet connection, so that pip can install the missing bits, or it does not and the test has to be skipped.
skipped tests are too easy to ignore ...
Quoted 7 lines
But if I copy the wrong state of a submodule around, things get messed up quite easily in my experience. Ok, you could say that this is just my special setup with rsync, but already given the fact that "git checkout" creates an inconsistent state of your source tree until you run the script for updating the submodules the next time is an indication that submodules are rather a shaky thing (e.g. if you'd create a tarball for somebody else from your source tree right after doing a "git checkout").
yea one has to remember to set submodule.recurse = true in .gitconfig
I agree it's annoying, I guess they don't change it for compat reasons.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
- the content of submodules is not added to the tarballs that get created on the git forges automatically. There were lots of requests from users in the past that tried to download a tarball from github and then wondered why they couldn't compile QEMU.
how is pip better here?
You don't get incomplete/non-working tarballs in that case.
So skip the test ;)
Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule, where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?
Quoted 5 lines
Quoted 2 lines
I think we can skip the checkout in the tarball if we like. If people want to run the test they can checkout then.
Release tarballs don't include the ".git" folder infrastructur, so everybody who downloads a tarball will simply never be able to run the test.
I actually think I'm fine with that for this specific case.
Quoted 9 lines
Quoted 8 lines
Quoted 5 lines
- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
Come on, this is just a test. We *really* don't care if an ISO we use to test ACPI is using an exploitable version of grub.
Quoted 6 lines
Wait, I thought we were only talking about tappy here? The ISO binaries should certainly *not* be bundled in the QEMU tarballs (they are too big already anyway, we should rather think of moving the firmware binaries out of the tarball instead).
Thomas
IIUC there are three things we are discussing
- biosbits source
- biosbits image
- tappy
Signature · 1 line
MST
Re: Why we should avoid new submodules if possible
Quoted 1 line · Michael S. Tsirkin
On Tue, Jun 28, 2022 at 12:50:06PM +0200, Thomas Huth wrote:
[...]
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Come on, this is just a test. We *really* don't care if an ISO we use to test ACPI is using an exploitable version of grub.
Wait, I thought we were only talking about tappy here? The ISO binaries should certainly *not* be bundled in the QEMU tarballs (they are too big already anyway, we should rather think of moving the firmware binaries out of the tarball instead).
Quoted 6 lines
Quoted 1 line
Thomas
IIUC there are three things we are discussing
- biosbits source
- biosbits image
- tappy
Oh well, I missed that part of the discussion so far since the corresponding patches did not make it to the mailing list ¯\_(ツ)_/¯
Anyway, that's just another indication that it might not be the right fit for inclusion into the QEMU source tree. So either download it similar to (or included in) the avocado tests, or maybe another solution would be to have a separate "qemu-ci" or "qemu-testing" repository for stuff like this ... ?
Thomas
Re: Why we should avoid new submodules if possible
Quoted 21 lines · Thomas Huth
Quoted 1 line · Michael S. Tsirkin
On Tue, Jun 28, 2022 at 12:50:06PM +0200, Thomas Huth wrote:
[...]
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Come on, this is just a test. We *really* don't care if an ISO we use to test ACPI is using an exploitable version of grub.
Wait, I thought we were only talking about tappy here? The ISO binaries should certainly *not* be bundled in the QEMU tarballs (they are too big already anyway, we should rather think of moving the firmware binaries out of the tarball instead).
Quoted 6 lines
Quoted 1 line
Thomas
IIUC there are three things we are discussing
- biosbits source
- biosbits image
- tappyOh well, I missed that part of the discussion so far since the corresponding patches did not make it to the mailing list ¯\_(ツ)_/¯
Quoted 7 lines
Anyway, that's just another indication that it might not be the right fit for inclusion into the QEMU source tree. So either download it similar to (or included in) the avocado tests, or maybe another solution would be to have a separate "qemu-ci" or "qemu-testing" repository for stuff like this ... ?
Thomas
I don't think anyone suggested sticking all those blobs in qemu.git: I proposed a submodule, others proposed the web (fetched with avocado or just wget). If it's a submodule we'll want to skip it in the tarball, and skip the test if not checked out. Whoever is trying to do development out of a tarball should just stop :).
Signature · 1 line
MST
Re: Why we should avoid new submodules if possible
In reply to Thomas HuthQuoted 19 lines · Thomas Huth
Quoted 1 line · Michael S. Tsirkin
On Tue, Jun 28, 2022 at 12:50:06PM +0200, Thomas Huth wrote:
[...]
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Come on, this is just a test. We *really* don't care if an ISO we use to test ACPI is using an exploitable version of grub.
Wait, I thought we were only talking about tappy here? The ISO binaries should certainly *not* be bundled in the QEMU tarballs (they are too big already anyway, we should rather think of moving the firmware binaries out of the tarball instead).
Quoted 6 lines
Quoted 1 line
Thomas
IIUC there are three things we are discussing
- biosbits source
- biosbits image
- tappy
Quoted 7 lines
Oh well, I missed that part of the discussion so far since the corresponding patches did not make it to the mailing list ¯\_(ツ)_/¯
Anyway, that's just another indication that it might not be the right fit for inclusion into the QEMU source tree. So either download it similar to (or included in) the avocado tests, or maybe another solution would be to have a separate "qemu-ci" or "qemu-testing" repository for stuff like this ... ?
Yes, I think we should have a separate repo for all testing related stuff like blobs, guest images etc. Curently, some of them are in personal github accounts which is not idea IMHo.
Bits stuff can reside in the same place.
Re: Why we should avoid new submodules if possible
Quoted 28 lines · Ani Sinha
Quoted 19 lines · Thomas Huth
Quoted 1 line · Michael S. Tsirkin
On Tue, Jun 28, 2022 at 12:50:06PM +0200, Thomas Huth wrote:
[...]
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Come on, this is just a test. We *really* don't care if an ISO we use to test ACPI is using an exploitable version of grub.
Wait, I thought we were only talking about tappy here? The ISO binaries should certainly *not* be bundled in the QEMU tarballs (they are too big already anyway, we should rather think of moving the firmware binaries out of the tarball instead).
Quoted 6 lines
Quoted 1 line
Thomas
IIUC there are three things we are discussing
- biosbits source
- biosbits image
- tappyQuoted 7 lines
Oh well, I missed that part of the discussion so far since the corresponding patches did not make it to the mailing list ¯\_(ツ)_/¯
Anyway, that's just another indication that it might not be the right fit for inclusion into the QEMU source tree. So either download it similar to (or included in) the avocado tests, or maybe another solution would be to have a separate "qemu-ci" or "qemu-testing" repository for stuff like this ... ?
Quoted 5 lines
Yes, I think we should have a separate repo for all testing related stuff like blobs, guest images etc. Curently, some of them are in personal github accounts which is not idea IMHo.
Bits stuff can reside in the same place.
FYI, the reason much of this is intentionally NOT under the /qemu-project gitlab namespace is that we did not want to be responsible for distributing arbitrary binary blobs/images. That in turn makes the QEMU project responsible for license compliance, which is non-trivial todo correctly for much of this stuff. As such it is highly desirable to delegate both the hosting the binaries and source to the third party who builds it.
I agree the use of personal github accounts is not nice, but it was the least worst solution identified.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: Why we should avoid new submodules if possible
Quoted 6 lines · Daniel P. Berrangé
FYI, the reason much of this is intentionally NOT under the /qemu-project gitlab namespace is that we did not want to be responsible for distributing arbitrary binary blobs/images. That in turn makes the QEMU project responsible for license compliance, which is non-trivial todo correctly for much of this stuff. As such it is highly desirable to delegate both the hosting the binaries and source to the third party who builds it.
This might be understadable for random guest OS images which include tons of stuff and are thus hard to audit. But not to biosbits which has its own license (more or less bsd) + gpl for grub:
https://github.com/biosbits/bits/blob/master/COPYING
Quoted 2 lines
I agree the use of personal github accounts is not nice, but it was the least worst solution identified.
Signature · 1 line
MST
Re: Why we should avoid new submodules if possible
Quoted 9 lines · Michael S. Tsirkin
Quoted 6 lines · Daniel P. Berrangé
FYI, the reason much of this is intentionally NOT under the /qemu-project gitlab namespace is that we did not want to be responsible for distributing arbitrary binary blobs/images. That in turn makes the QEMU project responsible for license compliance, which is non-trivial todo correctly for much of this stuff. As such it is highly desirable to delegate both the hosting the binaries and source to the third party who builds it.
Quoted 4 lines
This might be understadable for random guest OS images which include tons of stuff and are thus hard to audit. But not to biosbits which has its own license (more or less bsd) + gpl for grub:
https://github.com/biosbits/bits/blob/master/COPYING
These are all the dependencies:
https://github.com/biosbits/bits/tree/master/deps
We can go through the licenses for each and make a determination. The audit would be lost easier because there is a bounded number of dependencies for bits.
Quoted 8 lines
Quoted 3 lines
I agree the use of personal github accounts is not nice, but it was the least worst solution identified.
--
MST
Re: Why we should avoid new submodules if possible
Quoted 15 lines · Ani Sinha
Quoted 9 lines · Michael S. Tsirkin
Quoted 6 lines · Daniel P. Berrangé
FYI, the reason much of this is intentionally NOT under the /qemu-project gitlab namespace is that we did not want to be responsible for distributing arbitrary binary blobs/images. That in turn makes the QEMU project responsible for license compliance, which is non-trivial todo correctly for much of this stuff. As such it is highly desirable to delegate both the hosting the binaries and source to the third party who builds it.
Quoted 4 lines
This might be understadable for random guest OS images which include tons of stuff and are thus hard to audit. But not to biosbits which has its own license (more or less bsd) + gpl for grub:
https://github.com/biosbits/bits/blob/master/COPYING
Quoted 6 lines
These are all the dependencies:
https://github.com/biosbits/bits/tree/master/depsWe can go through the licenses for each and make a determination. The audit would be lost easier because there is a bounded number of dependencies for bits.
That's quite a bit of dependencies already ... I don't think that we should put the burden of keeping up with the licenses of those projects to the QEMU project. So just make sure that the binaries are available somewhere and then include your test into the avocado framework or download via curl/wget as suggested in other mails.
Thomas
Re: Why we should avoid new submodules if possible
Quoted 22 lines · Thomas Huth
Quoted 14 lines · Ani Sinha
Quoted 8 lines · Michael S. Tsirkin
Quoted 6 lines · Daniel P. Berrangé
FYI, the reason much of this is intentionally NOT under the /qemu-project gitlab namespace is that we did not want to be responsible for distributing arbitrary binary blobs/images. That in turn makes the QEMU project responsible for license compliance, which is non-trivial todo correctly for much of this stuff. As such it is highly desirable to delegate both the hosting the binaries and source to the third party who builds it.
Quoted 4 lines
This might be understadable for random guest OS images which include tons of stuff and are thus hard to audit. But not to biosbits which has its own license (more or less bsd) + gpl for grub:
https://github.com/biosbits/bits/blob/master/COPYINGQuoted 6 lines
These are all the dependencies:
https://github.com/biosbits/bits/tree/master/depsWe can go through the licenses for each and make a determination. The audit would be lost easier because there is a bounded number of dependencies for bits.
Quoted 5 lines
That's quite a bit of dependencies already ... I don't think that we should put the burden of keeping up with the licenses of those projects to the QEMU project. So just make sure that the binaries are available somewhere and then include your test into the avocado framework or download via curl/wget as suggested in other mails.
"available somewhere" until they get moved or deleted...
This happened a few times with archives/artifacts Avocado tests fetch.
We ended up committing one or a set of URLs pointing to a file, its usual name and its hash.
If you downloaded the file in time and don't flush the Avocado cache, you can keep testing.
If it is too late (the file disappeared) and you have time to dig for such a file online, and are lucky because you found it and its hash matches, then you can run some tests. As a maintainer I find this feature useful, and find it worse to lose the ability to test something that used to work.
My 2¢...
Phil.
Re: Why we should avoid new submodules if possible
In reply to Thomas HuthQuoted 34 lines · Thomas Huth
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]Quoted 2 lines
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
As far as I know, you can also specify the version when using pip, can't you? So that's not really an advantage here.
On the contrary, submodules have a couple of disadvantages that I really dislike:
- submodules do not get updated automatically when doing a "git checkout", we have to update them via a script instead. This causes e.g. trouble if you rsync your source tree to a machine that has no access to the internet and you forgot to update the submodule before the sync
- the content of submodules is not added to the tarballs that get created on the git forges automatically. There were lots of requests from users in the past that tried to download a tarball from github and then wondered why they couldn't compile QEMU.
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule, where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?
Quoted 8 lines
- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
So in my opinion we should avoid new submodules if there is an alternative.
Thomas
So looking at the latest proposals downloading files from CI, checksumming them etc etc. No auto checkout, not added automatically either, right?
This seems to be the only difference:
- we include the submodule content in our release tarballs
How about we just fix that? Thomas would that address your concern at least wrt tests?
Signature · 1 line
MST
Re: Why we should avoid new submodules if possible
Quoted 42 lines · Michael S. Tsirkin
Quoted 41 lines · Thomas Huth
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]Quoted 2 lines
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
As far as I know, you can also specify the version when using pip, can't you? So that's not really an advantage here.
On the contrary, submodules have a couple of disadvantages that I really dislike:
- submodules do not get updated automatically when doing a "git checkout", we have to update them via a script instead. This causes e.g. trouble if you rsync your source tree to a machine that has no access to the internet and you forgot to update the submodule before the sync
- the content of submodules is not added to the tarballs that get created on the git forges automatically. There were lots of requests from users in the past that tried to download a tarball from github and then wondered why they couldn't compile QEMU.
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule, where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
So in my opinion we should avoid new submodules if there is an alternative.
Quoted 8 lines
Quoted 1 line
Thomas
So looking at the latest proposals downloading files from CI, checksumming them etc etc. No auto checkout, not added automatically either, right?
This seems to be the only difference:
- we include the submodule content in our release tarballs
That's just one of the issues with submodules. Working with them in general is not a pleasant experiance. Thomas pointed out some of the issues, such as 'git checkout' ignoring submodules, requiring extra steps to sync them. The flipside on tarballs is that the auto-generated tarballs from gitlab do not include submodules, so the tests will be missing the files they wanted. There's also the perenial problem that developers frequently send patches that mistakenly include submodule changes, which is related to the way that 'git checkout' doesn't sync submodule state when switching branches. I'd really like to see us doing more to eliminate as much use of submodules as is possible over time.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: Why we should avoid new submodules if possible
Quoted 3 lines · Daniel P. Berrangé
There's also the perenial problem that developers frequently send patches that mistakenly include submodule changes, which is related to the way that 'git checkout' doesn't sync submodule state when switching branches.
Do you happen to know how exactly that happens?
Signature · 1 line
MST
Re: Why we should avoid new submodules if possible
Quoted 6 lines · Michael S. Tsirkin
Quoted 3 lines · Daniel P. Berrangé
There's also the perenial problem that developers frequently send patches that mistakenly include submodule changes, which is related to the way that 'git checkout' doesn't sync submodule state when switching branches.
Do you happen to know how exactly that happens?
For any given branch the submodule is synced to a given git commit hash. If the submodule checkout is not synced to the same commit hash it will show as dirty, and if you git add this pending change, it'll record that new submodule commit hash. Seeing dirty state is common when you switch between branches, either side of a git master change that updated a submodule.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: Why we should avoid new submodules if possible
Quoted 15 lines · Daniel P. Berrangé
Quoted 6 lines · Michael S. Tsirkin
Quoted 3 lines · Daniel P. Berrangé
There's also the perenial problem that developers frequently send patches that mistakenly include submodule changes, which is related to the way that 'git checkout' doesn't sync submodule state when switching branches.
Do you happen to know how exactly that happens?
For any given branch the submodule is synced to a given git commit hash. If the submodule checkout is not synced to the same commit hash it will show as dirty, and if you git add this pending change, it'll record that new submodule commit hash. Seeing dirty state is common when you switch between branches, either side of a git master change that updated a submodule.
Quoted 2 lines
With regards,
Daniel
I see. It is interesting.
So apparently what you want is ignore submodule changes, right? If yes this is close to what we want:
git submodule update --checkout testsub
git update-index --skip-worktree testsub
A script checking out the submodule can just run this command.
Quoted 4 lines
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: Why we should avoid new submodules if possible
Quoted 18 lines · Michael S. Tsirkin
Quoted 6 lines · Daniel P. Berrangé
Quoted 5 lines · Michael S. Tsirkin
Quoted 3 lines · Daniel P. Berrangé
There's also the perenial problem that developers frequently send patches that mistakenly include submodule changes, which is related to the way that 'git checkout' doesn't sync submodule state when switching branches.
Quoted 8 lines
Quoted 1 line
Do you happen to know how exactly that happens?
For any given branch the submodule is synced to a given git commit hash. If the submodule checkout is not synced to the same commit hash it will show as dirty, and if you git add this pending change, it'll record that new submodule commit hash. Seeing dirty state is common when you switch between branches, either side of a git master change that updated a submodule.
I see. It is interesting.
Quoted 8 lines
So apparently what you want is ignore submodule changes, right? If yes this is close to what we want:
git submodule update --checkout testsub
git update-index --skip-worktree testsubA script checking out the submodule can just run this command.
The problem happens not when you check out the submodule, but when you do basic not-submodule-related git operations like "git checkout my-working-branch". The fix would be if git itself automatically kept submodule state in sync with the main repo's branch. But it doesn't, so the UI is a massive beartrap.
-- PMM
Re: Why we should avoid new submodules if possible
Quoted 28 lines · Peter Maydell
Quoted 18 lines · Michael S. Tsirkin
Quoted 6 lines · Daniel P. Berrangé
Quoted 5 lines · Michael S. Tsirkin
Quoted 3 lines · Daniel P. Berrangé
There's also the perenial problem that developers frequently send patches that mistakenly include submodule changes, which is related to the way that 'git checkout' doesn't sync submodule state when switching branches.
Quoted 8 lines
Quoted 1 line
Do you happen to know how exactly that happens?
For any given branch the submodule is synced to a given git commit hash. If the submodule checkout is not synced to the same commit hash it will show as dirty, and if you git add this pending change, it'll record that new submodule commit hash. Seeing dirty state is common when you switch between branches, either side of a git master change that updated a submodule.
I see. It is interesting.
Quoted 8 lines
So apparently what you want is ignore submodule changes, right? If yes this is close to what we want:
git submodule update --checkout testsub
git update-index --skip-worktree testsubA script checking out the submodule can just run this command.
Quoted 7 lines
The problem happens not when you check out the submodule, but when you do basic not-submodule-related git operations like "git checkout my-working-branch". The fix would be if git itself automatically kept submodule state in sync with the main repo's branch. But it doesn't, so the UI is a massive beartrap.
-- PMM
If this is what you want, you can make it do that too:
$ cat .gitmodules
[submodule "testsub"]
path = testsub
recurse = true
url = /tmp/testsub/git checkout (and a bunch of other commands) will now recurse into this submodule.
Signature · 1 line
MST
Re: Why we should avoid new submodules if possible
In reply to Michael S. TsirkinQuoted 9 lines · Michael S. Tsirkin
Quoted 2 lines · Daniel P. Berrangé
There's also the perenial problem that developers frequently send patches that mistakenly include submodule changes, which is related to
the
Quoted 1 line
way that 'git checkout' doesn't sync submodule state when switching
branches.
Do you happen to know how exactly that happens?
Here's how it happens to me.
% git checkout master
% git checkout -b bsd-user-2022q4
# bring changes in, doesn't matter # Build a few times # time passes, master upstream evolves, submodules move % git checkout master % git pull # master fast forwards % git rebase -i master bsd-user-2022q4
at this point I have to do unnatural git submodule recursive things to make it stop
complaining about how all the submodules are now 'modified' because they have
moved on the master branch.
It is real, lasting, lingering pain and a large part of the reason that I refuse to allow FreeBSD
to use submodules[*]. It's just too much extra hassle for no gain at all (at least for me, I
see only the pain and no benefits to me or anybody else). I've sent at least three review
requests that have this pollution in it since it's hard to avoid if you press forward after the
rebase not to have something swept up into an innocuous change.
Warner
[*] They are also generally a poor match for FreeBSD since we rarely import things
verbatim, unchanged from upstreams that have well-oiled CI to give us assurance
that things are good...
Re: Why we should avoid new submodules if possible
In reply to Daniel P. BerrangéQuoted 49 lines · Daniel P. Berrangé
Quoted 48 lines · Michael S. Tsirkin
Quoted 34 lines · Thomas Huth
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]Quoted 2 lines
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
As far as I know, you can also specify the version when using pip, can't you? So that's not really an advantage here.
On the contrary, submodules have a couple of disadvantages that I really dislike:
- submodules do not get updated automatically when doing a "git checkout", we have to update them via a script instead. This causes e.g. trouble if you rsync your source tree to a machine that has no access to the internet and you forgot to update the submodule before the sync
- the content of submodules is not added to the tarballs that get created on the git forges automatically. There were lots of requests from users in the past that tried to download a tarball from github and then wondered why they couldn't compile QEMU.
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule, where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?Quoted 8 lines
- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
So in my opinion we should avoid new submodules if there is an alternative.
Thomas
So looking at the latest proposals downloading files from CI, checksumming them etc etc. No auto checkout, not added automatically either, right?
Quoted 5 lines
Quoted 2 lines
This seems to be the only difference:
- we include the submodule content in our release tarballsThat's just one of the issues with submodules. Working with them in general is not a pleasant experiance.
This is what I asked about at the maintainers summit. I'd like to map the issues and see if there's anything we can do to solve them. In particular we will likely keep using submodules where we historically did so it's time well spent.
I agree generally but the big question is what to replace these with. Below I assume the replacement is a script such as avocado or pytest with its own hashing, calling wget internally etc etc.
Quoted 2 lines
Thomas pointed out some of the issues, such
as 'git checkout' ignoring submodules, requiring extra steps to sync them.
Not different from a home grown SCM as part of test script, right?
Quoted 3 lines
The flipside on tarballs is that the auto-generated tarballs from gitlab do not include submodules, so the tests will be missing the files they wanted.
Not different from a home grown SCM as part of test script, right?
Quoted 2 lines
There's also the perenial problem that developers frequently send patches that mistakenly include submodule changes,
OK, so the thing to do would be to look for ways to exclude submodule changes from git commits.
Quoted 2 lines
which is related to the
way that 'git checkout' doesn't sync submodule state when switching branches.
Not different from a home grown SCM as part of test script, right?
Quoted 2 lines
I'd really like to see us doing more to eliminate as much use of submodules as is possible over time.
Or try to fix the problems, right?
Quoted 7 lines
With regards,
DanielSignature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: Why we should avoid new submodules if possible
Quoted 67 lines · Michael S. Tsirkin
Quoted 49 lines · Daniel P. Berrangé
Quoted 48 lines · Michael S. Tsirkin
Quoted 34 lines · Thomas Huth
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]Quoted 2 lines
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
As far as I know, you can also specify the version when using pip, can't you? So that's not really an advantage here.
On the contrary, submodules have a couple of disadvantages that I really dislike:
- submodules do not get updated automatically when doing a "git checkout", we have to update them via a script instead. This causes e.g. trouble if you rsync your source tree to a machine that has no access to the internet and you forgot to update the submodule before the sync
- the content of submodules is not added to the tarballs that get created on the git forges automatically. There were lots of requests from users in the past that tried to download a tarball from github and then wondered why they couldn't compile QEMU.
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule, where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?Quoted 8 lines
- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
So in my opinion we should avoid new submodules if there is an alternative.
Thomas
So looking at the latest proposals downloading files from CI, checksumming them etc etc. No auto checkout, not added automatically either, right?
Quoted 5 lines
Quoted 2 lines
This seems to be the only difference:
- we include the submodule content in our release tarballsThat's just one of the issues with submodules. Working with them in general is not a pleasant experiance.
This is what I asked about at the maintainers summit. I'd like to map the issues and see if there's anything we can do to solve them. In particular we will likely keep using submodules where we historically did so it's time well spent.
I agree generally but the big question is what to replace these with. Below I assume the replacement is a script such as avocado or pytest with its own hashing, calling wget internally etc etc.
Quoted 5 lines
Quoted 2 lines
Thomas pointed out some of the issues, such
as 'git checkout' ignoring submodules, requiring extra steps to sync them.Not different from a home grown SCM as part of test script, right?
We're not building a home grown SCM as part of a test script, so this answer is irrelevant.
Quoted 5 lines
Quoted 2 lines
There's also the perenial problem that developers frequently send patches that mistakenly include submodule changes,
OK, so the thing to do would be to look for ways to exclude submodule changes from git commits.
If someone wants to make git suck less with submodules great, but needs someone to actually do the work.
Quoted 4 lines
Quoted 2 lines
I'd really like to see us doing more to eliminate as much use of submodules as is possible over time.
Or try to fix the problems, right?
Again needs someone to actually make it happen.
Meanwhile QEMU already has an integrated test harness in the form of Avocado that does everything needed. If Avocado had just been used for this biosbits test in the first place, the test would likely have already been merged to QEMU, instead of us having this never ending debate on how to re-invent an alternative to what already avocado does.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: Why we should avoid new submodules if possible
Quoted 68 lines · Daniel P. Berrangé
Quoted 67 lines · Michael S. Tsirkin
Quoted 49 lines · Daniel P. Berrangé
Quoted 48 lines · Michael S. Tsirkin
Quoted 34 lines · Thomas Huth
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]Quoted 2 lines
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
As far as I know, you can also specify the version when using pip, can't you? So that's not really an advantage here.
On the contrary, submodules have a couple of disadvantages that I really dislike:
- submodules do not get updated automatically when doing a "git checkout", we have to update them via a script instead. This causes e.g. trouble if you rsync your source tree to a machine that has no access to the internet and you forgot to update the submodule before the sync
- the content of submodules is not added to the tarballs that get created on the git forges automatically. There were lots of requests from users in the past that tried to download a tarball from github and then wondered why they couldn't compile QEMU.
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule, where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?Quoted 8 lines
- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
So in my opinion we should avoid new submodules if there is an alternative.
Thomas
So looking at the latest proposals downloading files from CI, checksumming them etc etc. No auto checkout, not added automatically either, right?
Quoted 5 lines
Quoted 2 lines
This seems to be the only difference:
- we include the submodule content in our release tarballsThat's just one of the issues with submodules. Working with them in general is not a pleasant experiance.
This is what I asked about at the maintainers summit. I'd like to map the issues and see if there's anything we can do to solve them. In particular we will likely keep using submodules where we historically did so it's time well spent.
I agree generally but the big question is what to replace these with. Below I assume the replacement is a script such as avocado or pytest with its own hashing, calling wget internally etc etc.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Thomas pointed out some of the issues, such
as 'git checkout' ignoring submodules, requiring extra steps to sync them.Not different from a home grown SCM as part of test script, right?
We're not building a home grown SCM as part of a test script, so this answer is irrelevant.
If you are managing contents of files using code then you have a content managing system :)
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
There's also the perenial problem that developers frequently send patches that mistakenly include submodule changes,
OK, so the thing to do would be to look for ways to exclude submodule changes from git commits.
If someone wants to make git suck less with submodules great, but needs someone to actually do the work.
I have some experience with git, might be able to if I can figure out what the issues are. Or maybe there's already a flag to do what we want git is pretty configurable.
Quoted 14 lines
Quoted 4 lines
Quoted 2 lines
I'd really like to see us doing more to eliminate as much use of submodules as is possible over time.
Or try to fix the problems, right?
Again needs someone to actually make it happen.
Meanwhile QEMU already has an integrated test harness in the form of Avocado that does everything needed. If Avocado had just been used for this biosbits test in the first place, the test would likely have already been merged to QEMU, instead of us having this never ending debate on how to re-invent an alternative to what already avocado does.
Quoted 2 lines
With regards,
Daniel
I just think there's a ton of difference between linux guest images which we want to keep running indefinitely and a unit test firmware which we want to version with qemu.
Quoted 4 lines
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: Why we should avoid new submodules if possible
In reply to Daniel P. BerrangéQuoted 95 lines · Daniel P. Berrangé
Quoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Daniel P. Berrangé
Quoted 4 lines · Michael S. Tsirkin
Quoted 3 lines · Thomas Huth
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]Quoted 1 line
For biosbits if we are going this route then I feel a submodule
is much
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
better. It records which version exactly each qemu version
wants.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
As far as I know, you can also specify the version when using pip,
can't
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
you? So that's not really an advantage here.
On the contrary, submodules have a couple of disadvantages that I
really
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
dislike:
- submodules do not get updated automatically when doing a "git
checkout",
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
we have to update them via a script instead. This causes e.g.
trouble if you
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
rsync your source tree to a machine that has no access to the
internet and
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
you forgot to update the submodule before the sync
- the content of submodules is not added to the tarballs that get
created on
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
the git forges automatically. There were lots of requests from
users in the
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
past that tried to download a tarball from github and then
wondered why they
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
couldn't compile QEMU.
- we include the submodule content in our release tarballs, so
people get
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
the impression that hte submodule content is part of the QEMU
sources. This
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
has two disadvantages:
* We already got bug reports for the code in the submodule, where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
So in my opinion we should avoid new submodules if there is an
alternative.
Quoted 11 lines
Quoted 11 lines
Quoted 9 lines
Quoted 2 lines
Thomas
So looking at the latest proposals downloading files from CI, checksumming them etc etc. No auto checkout, not added automatically either, right?
This seems to be the only difference:
- we include the submodule content in our release tarballsThat's just one of the issues with submodules. Working with them in
general
Quoted 8 lines
Quoted 1 line
is not a pleasant experiance.
This is what I asked about at the maintainers summit. I'd like to map the issues and see if there's anything we can do to solve them. In particular we will likely keep using submodules where we historically did so it's time well spent.
Quoted 7 lines
I agree generally but the big question is what to replace these with. Below I assume the replacement is a script such as avocado or pytest with its own hashing, calling wget internally etc etc.
Quoted 2 lines
Thomas pointed out some of the issues, such
as 'git checkout' ignoring submodules, requiring extra steps to syncthem.
Quoted 3 lines
Not different from a home grown SCM as part of test script, right?
We're not building a home grown SCM as part of a test script, so this answer is irrelevant.
Quoted 3 lines
Quoted 2 lines
There's also the perenial problem that developers frequently send patches that mistakenly include submodule changes,
Quoted 7 lines
Quoted 1 line
OK, so the thing to do would be to look for ways to exclude submodule
changes
Quoted 1 line
from git commits.
If someone wants to make git suck less with submodules great, but needs someone to actually do the work.
A big part of the problem is knowing which of the following commands I have to
do to undo the uncommitted changes, the committed changes, the staged changes,
etc:
git submodule update --init --recursive git submodule update --init
git submodule foreach --recursive git reset --hard git submodule foreach --recursive git clean -xfd git submodule foreach --recursive git clean -xfg
(all of these are in my history, I honestly don't know the difference between the last two).
And each 'oops' takes time away from upstreaming bsd-user I don't really have that
much of. I've wasted hours on this over the past year between all the different ways
it can screw up.
To be fair, it is a relatively small fraction of the time, but as you can tell from the
animation in my email it inspires much passion.
Warner
Quoted 27 lines
Quoted 1 line
Quoted 1 line
I'd really like to see us doing more to eliminate as much use of
submodules
Quoted 3 lines
Quoted 1 line
as is possible over time.p
Or try to fix the problems, right?
Again needs someone to actually make it happen.
Meanwhile QEMU already has an integrated test harness in the form of Avocado that does everything needed. If Avocado had just been used for this biosbits test in the first place, the test would likely have already been merged to QEMU, instead of us having this never ending debate on how to re-invent an alternative to what already avocado does.
With regards,
Daniel
--
|: https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-
https://fstop138.berrange.com :|
|: https://entangle-photo.org -o-
https://www.instagram.com/dberrange :|
Re: Why we should avoid new submodules if possible
Quoted 125 lines · Warner Losh
On Wed, Sep 28, 2022 at 7:09 AM Daniel P. Berrangé <[email protected]> wrote:
On Wed, Sep 28, 2022 at 05:53:17AM -0400, Michael S. Tsirkin wrote:
> On Wed, Sep 28, 2022 at 10:37:14AM +0100, Daniel P. Berrangé wrote:
> > On Wed, Sep 28, 2022 at 05:26:42AM -0400, Michael S. Tsirkin wrote:
> > > On Tue, Jun 28, 2022 at 12:21:39PM +0200, Thomas Huth wrote:
> > > > On 28/06/2022 12.03, Michael S. Tsirkin wrote:
> > > > [...]
> > > > > For biosbits if we are going this route then I feel a submodule is much
> > > > > better. It records which version exactly each qemu version wants.
> > > >
> > > > As far as I know, you can also specify the version when using pip, can't
> > > > you? So that's not really an advantage here.
> > > >
> > > > On the contrary, submodules have a couple of disadvantages that I really
> > > > dislike:
> > > >
> > > > - submodules do not get updated automatically when doing a "git checkout",
> > > > we have to update them via a script instead. This causes e.g. trouble if you
> > > > rsync your source tree to a machine that has no access to the internet and
> > > > you forgot to update the submodule before the sync
> > > >
> > > > - the content of submodules is not added to the tarballs that get created on
> > > > the git forges automatically. There were lots of requests from users in the
> > > > past that tried to download a tarball from github and then wondered why they
> > > > couldn't compile QEMU.
> > > >
> > > > - we include the submodule content in our release tarballs, so people get
> > > > the impression that hte submodule content is part of the QEMU sources. This
> > > > has two disadvantages:
> > > > * We already got bug reports for the code in the submodule, > > > > where people did not understand that they should report that > > > > rather to the original project instead (i.e. you ship it - you > > > > own it)
> > > > * People get the impression that QEMU is a huge monster > > > > application if they count the number of code lines, run > > > > their code scanner tools on the tarball contents, etc. > > > > Remember "nemu", for example, where one of the main complaints > > > > was that QEMU has too many lines of code?
> > > >
> > > > - If programs includes code via submodules, this gets a higher > > > > burder for distro maintainers, since they have to patch each > > > > and every package when there is a bug, instead of being able to > > > > fix it in one central place.
> > > >
> > > > So in my opinion we should avoid new submodules if there is an alternative.
> > > >
> > > > Thomas
> > >
> > > So looking at the latest proposals downloading files from CI, > > > checksumming them etc etc. No auto checkout, not added automatically > > > either, right?
> > >
> > > This seems to be the only difference:
> > > - we include the submodule content in our release tarballs > >
> > That's just one of the issues with submodules. Working with them in general
> > is not a pleasant experiance.
>
> This is what I asked about at the maintainers summit.
> I'd like to map the issues and see if there's anything
> we can do to solve them. In particular we will likely
> keep using submodules where we historically did
> so it's time well spent.
>
> I agree generally but the big question is what to replace these with. > Below I assume the replacement is a script such as avocado or pytest > with its own hashing, calling wget internally etc etc.
>
>
> > Thomas pointed out some of the issues, such
> > as 'git checkout' ignoring submodules, requiring extra steps to sync them.
>
>
> Not different from a home grown SCM as part of test script, right?We're not building a home grown SCM as part of a test script, so this answer is irrelevant.
> > There's also the perenial problem that developers frequently send > > patches that mistakenly include submodule changes,
>
> OK, so the thing to do would be to look for ways to exclude submodule changes
> from git commits.If someone wants to make git suck less with submodules great, but needs someone to actually do the work.
A big part of the problem is knowing which of the following commands I have to do to undo the uncommitted changes, the committed changes, the staged changes, etc:
git submodule update --init --recursive git submodule update --init
git submodule foreach --recursive git reset --hard git submodule foreach --recursive git clean -xfd git submodule foreach --recursive git clean -xfg(all of these are in my history, I honestly don't know the difference between the last two).
And each 'oops' takes time away from upstreaming bsd-user I don't really have that
much of. I've wasted hours on this over the past year between all the different ways
it can screw up.
Quoted 6 lines
To be fair, it is a relatively small fraction of the time, but as you can tell from the
animation in my email it inspires much passion.Warner
OK this is understandable.
I think for what we do in qemu (never change upstream really)
git submodule update --checkout testsub git update-index --skip-worktree testsub
is more or less what most people want.
This assumes you have a script that basically does
git submodule deinit XYZ git submodule update --init XYZ git update-index --skip-worktree XYZ
each time, and do not need to work on the submodule proper.
Quoted 27 lines
Quoted 1 line
Quoted 1 line
I'd really like to see us doing more to eliminate as much use of
submodules
> > as is possible over time.p
>
> Or try to fix the problems, right?Again needs someone to actually make it happen.
Meanwhile QEMU already has an integrated test harness in the form of Avocado that does everything needed. If Avocado had just been used for this biosbits test in the first place, the test would likely have already been merged to QEMU, instead of us having this never ending debate on how to re-invent an alternative to what already avocado does.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange
:|
|: https://libvirt.org -o- https://fstop138.berrange.com
:|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange
:|
Re: Why we should avoid new submodules if possible
In reply to Michael S. TsirkinQuoted 48 lines · Michael S. Tsirkin
Quoted 34 lines · Thomas Huth
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]Quoted 2 lines
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
As far as I know, you can also specify the version when using pip, can't you? So that's not really an advantage here.
On the contrary, submodules have a couple of disadvantages that I really dislike:
- submodules do not get updated automatically when doing a "git checkout", we have to update them via a script instead. This causes e.g. trouble if you rsync your source tree to a machine that has no access to the internet and you forgot to update the submodule before the sync
- the content of submodules is not added to the tarballs that get created on the git forges automatically. There were lots of requests from users in the past that tried to download a tarball from github and then wondered why they couldn't compile QEMU.
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule, where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?Quoted 8 lines
- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
So in my opinion we should avoid new submodules if there is an alternative.
Thomas
So looking at the latest proposals downloading files from CI, checksumming them etc etc. No auto checkout, not added automatically either, right?
Quoted 5 lines
This seems to be the only difference:
- we include the submodule content in our release tarballsHow about we just fix that? Thomas would that address your concern at least wrt tests?
If I'm not forced to checkout that submodule, and if we don't add it to the release tarball, I guess there's not much left I can complain about ;-)
Thomas
Re: Why we should avoid new submodules if possible
Quoted 49 lines · Thomas Huth
Quoted 48 lines · Michael S. Tsirkin
Quoted 34 lines · Thomas Huth
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]Quoted 2 lines
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
As far as I know, you can also specify the version when using pip, can't you? So that's not really an advantage here.
On the contrary, submodules have a couple of disadvantages that I really dislike:
- submodules do not get updated automatically when doing a "git checkout", we have to update them via a script instead. This causes e.g. trouble if you rsync your source tree to a machine that has no access to the internet and you forgot to update the submodule before the sync
- the content of submodules is not added to the tarballs that get created on the git forges automatically. There were lots of requests from users in the past that tried to download a tarball from github and then wondered why they couldn't compile QEMU.
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule, where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?Quoted 8 lines
- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
So in my opinion we should avoid new submodules if there is an alternative.
Thomas
So looking at the latest proposals downloading files from CI, checksumming them etc etc. No auto checkout, not added automatically either, right?
Quoted 7 lines
Quoted 5 lines
This seems to be the only difference:
- we include the submodule content in our release tarballsHow about we just fix that? Thomas would that address your concern at least wrt tests?
If I'm not forced to checkout that submodule,
I think the make check script can do that?
Quoted 4 lines
and if we don't add it to the
release tarball, I guess there's not much left I can complain about ;-)Thomas
Re: Why we should avoid new submodules if possible
Quoted 53 lines · Michael S. Tsirkin
Quoted 52 lines · Thomas Huth
Quoted 51 lines · Michael S. Tsirkin
Quoted 34 lines · Thomas Huth
On 28/06/2022 12.03, Michael S. Tsirkin wrote:
[...]Quoted 2 lines
For biosbits if we are going this route then I feel a submodule is much better. It records which version exactly each qemu version wants.
As far as I know, you can also specify the version when using pip, can't you? So that's not really an advantage here.
On the contrary, submodules have a couple of disadvantages that I really dislike:
- submodules do not get updated automatically when doing a "git checkout", we have to update them via a script instead. This causes e.g. trouble if you rsync your source tree to a machine that has no access to the internet and you forgot to update the submodule before the sync
- the content of submodules is not added to the tarballs that get created on the git forges automatically. There were lots of requests from users in the past that tried to download a tarball from github and then wondered why they couldn't compile QEMU.
- we include the submodule content in our release tarballs, so people get the impression that hte submodule content is part of the QEMU sources. This has two disadvantages:
* We already got bug reports for the code in the submodule, where people did not understand that they should report that rather to the original project instead (i.e. you ship it - you own it)
* People get the impression that QEMU is a huge monster application if they count the number of code lines, run their code scanner tools on the tarball contents, etc. Remember "nemu", for example, where one of the main complaints was that QEMU has too many lines of code?Quoted 8 lines
- If programs includes code via submodules, this gets a higher burder for distro maintainers, since they have to patch each and every package when there is a bug, instead of being able to fix it in one central place.
So in my opinion we should avoid new submodules if there is an alternative.
Thomas
So looking at the latest proposals downloading files from CI, checksumming them etc etc. No auto checkout, not added automatically either, right?
This seems to be the only difference:
- we include the submodule content in our release tarballs
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
How about we just fix that? Thomas would that address your concern at least wrt tests?
If I'm not forced to checkout that submodule,
I think the make check script can do that?
Yes, currently there are only some submodules that are always checked out: dtc, meson, tests/fp/berkeley-softfloat-3, tests/fp/berkeley-testfloat-3 and ui/keycodemapdb. And for example the ones in the roms/ directory are not checked out by default.
Thomas
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Ani SinhaQuoted 103 lines · Ani Sinha
Quoted 97 lines · Thomas Huth
Quoted 87 lines · Daniel P. Berrangé
Quoted 69 lines · Ani Sinha
Quoted 68 lines · Daniel P. Berrangé
Quoted 43 lines · Thomas Huth
Quoted 42 lines · Michael S. Tsirkin
Quoted 40 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
No problem with that. So that's venv. But do we need pip and pulling
packages from the net during testing?We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
Quoted 8 lines
Quoted 4 lines
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Definitely not submodules.
We need to get out of the mindset that submodules are needed for every new dependancy we add. Submodules are only appropriate if the external project is designed to be used as a copylib (eg the keycodemapdb tool), or if we need to bundle in order to prevent a regression for previously deployed QEMU installs where the dependancy is known not to exist on all our supported platforms.
This does not apply in this case, because the proposed use of tappy is merely for a test case. Meson just needs to check if tappy exists and if it does, then use it, otherwise skip the tests that need it. The user can arrange to install tappy, as they do with the majority of other deps.
Quoted 6 lines
Quoted 4 lines
If John's venv stuff is relevant, then we don't even need the meson checks, just delegate to the venv setup.
Regardless, no submodules are needed or desirable.
What about keeping biosbits stuff? Source or pre-built.
Shipping them as pre-built binaries in QEMU is not a viable option IMHO, especially for grub as a GPL'd project we need to be extremely clear about the exact corresponding source and build process for any binary.
For this kind of thing I would generally expect the distro to provide packages that we consume. Looking at biosbits I see it is itself bundling a bunch more 3rd party projects, libffi, grub2, and including even an ancient version of python as a submodule.
Quoted 7 lines
So bundling a pre-built biosbits in QEMU appears to mean that we're in turn going to unexpectedly bundle a bunch of other 3rd party projects too, all with dubious license compliance. I don't think this looks like something we should have in qemu.git or qemu tarballs. It will also make it challenging for the distro to take biosbits at all, unless those 3rd party bundles can be eliminated in favour of using existing builds their have packaged for grub, python, libffi, etc.
Quoted 4 lines
So if this depends on some third party binary bits, I think this is pretty similar to the tests in the avocado directory ... there we download third party binaries, too... Wouldn't it make sense to adapt your tests to that framework?
Quoted 4 lines
I do not want to bring in the whole avocado framework because it would unnecessarily make things complicated. I just need the qemu machine python library and that is enough. For downloading third party stuff, we can simply wget things from somewhere.
Well avocado is already a part of QEMU's testing framework, so it is not bringing in a whole new framework. Arguably the introduction of python usage in qtest is what's introducing a new maint burden here.
If it is actually booting a real guest image (from biosbits) and interacting with it, then it does feel like the scope of this testing is more appropriate to QEMU's avocado framework than qtest, especially given the desire to use python for it all.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 7 lines · Daniel P. Berrangé
If it is actually booting a real guest image (from biosbits) and interacting with it, then it does feel like the scope of this testing is more appropriate to QEMU's avocado framework than qtest, especially given the desire to use python for it all.
With regards,
Daniel
I feel avocado is directed towards booting full fledged guest OS. It makes it much easier to figure out guest issues but it also prone to false positives and is harder to debug as a result. Booting a minimal image like this shouldn't require that.
Signature · 1 line
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 10 lines · Michael S. Tsirkin
Quoted 7 lines · Daniel P. Berrangé
If it is actually booting a real guest image (from biosbits) and interacting with it, then it does feel like the scope of this testing is more appropriate to QEMU's avocado framework than qtest, especially given the desire to use python for it all.
With regards,
DanielI feel avocado is directed towards booting full fledged guest OS.
That's essentially what this is doing - its a custom guest OS rather than a common distro IIUC
Quoted 3 lines
It makes it much easier to figure out guest issues but it also prone to false positives and is harder to debug as a result. Booting a minimal image like this shouldn't require that.
Well avocado is as reliable as the tests you write for it. The problems are largely around the images being used in avocado. If the biosbits testing system is reliable, then avocado will be too, and if they not reliable, then it will affect qtest too.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 7 lines · Daniel P. Berrangé
Quoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Daniel P. Berrangé
If it is actually booting a real guest image (from biosbits) and interacting with it, then it does feel like the scope of this testing is more appropriate to QEMU's avocado framework than qtest, especially given the desire to use python for it all.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
With regards,
DanielI feel avocado is directed towards booting full fledged guest OS.
That's essentially what this is doing - its a custom guest OS rather than a common distro IIUC
yes but then so is bios tables test - we generate the disk on the fly.
Quoted 6 lines
Quoted 3 lines
It makes it much easier to figure out guest issues but it also prone to false positives and is harder to debug as a result. Booting a minimal image like this shouldn't require that.
Well avocado is as reliable as the tests you write for it. The problems are largely around the images being used in avocado.
The idea of downloading large images as part of the test as opposed to part of setup is a large part of what makes it flaky - due to running into unpredictable latency and errors on both the internet and local IO side of things.
Quoted 3 lines
If the biosbits
testing system is reliable, then avocado will be too, and if they not reliable, then it will affect qtest too.
A larger system has more potential for bugs ;) I'm with Ani here. Using a system that is overkill will just make debugging more painful than it needs to be.
Quoted 6 lines
With regards,
DanielSignature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Daniel P. BerrangéQuoted 8 lines · Daniel P. Berrangé
Quoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Daniel P. Berrangé
If it is actually booting a real guest image (from biosbits) and interacting with it, then it does feel like the scope of this testing is more appropriate to QEMU's avocado framework than qtest, especially given the desire to use python for it all.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
With regards,
DanielI feel avocado is directed towards booting full fledged guest OS.
That's essentially what this is doing - its a custom guest OS rather
Daniel, there is no OS :-) Like there is no kernel image, no persistent real file system (there is a fake one called pyfs), no packages, nothing. Its very minimalistic image narrowly targeted for one thing alone - to test the bios.
Quoted 18 lines
than a common distro IIUC
Quoted 3 lines
It makes it much easier to figure out guest issues but it also prone to false positives and is harder to debug as a result. Booting a minimal image like this shouldn't require that.
Well avocado is as reliable as the tests you write for it. The problems are largely around the images being used in avocado. If the biosbits testing system is reliable, then avocado will be too, and if they not reliable, then it will affect qtest too.
With regards,
Daniel
--|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Michael S. TsirkinQuoted 7 lines · Michael S. Tsirkin
Quoted 5 lines · Daniel P. Berrangé
If it is actually booting a real guest image (from biosbits) and interacting with it, then it does feel like the scope of this testing is more appropriate to QEMU's avocado framework than qtest, especially given the desire to use python for it all.
Quoted 7 lines
Quoted 2 lines
With regards,
DanielI feel avocado is directed towards booting full fledged guest OS. It makes it much easier to figure out guest issues but it also prone to false positives and is harder to debug as a result. Booting a minimal image like this shouldn't require that.
Yes 100% agree with Michael on this. Biobits is *not* booting any OS image. It runs off grub, that is, directly from bootloader stage. The interraction with the VM is minimal.
Quoted 4 lines
--
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 11 lines · Ani Sinha
Quoted 10 lines · Michael S. Tsirkin
Quoted 7 lines · Daniel P. Berrangé
If it is actually booting a real guest image (from biosbits) and interacting with it, then it does feel like the scope of this testing is more appropriate to QEMU's avocado framework than qtest, especially given the desire to use python for it all.
With regards,
Daniel
Quoted 8 lines
Quoted 4 lines
I feel avocado is directed towards booting full fledged guest OS. It makes it much easier to figure out guest issues but it also prone to false positives and is harder to debug as a result. Booting a minimal image like this shouldn't require that.
Yes 100% agree with Michael on this. Biobits is *not* booting any OS image. It runs off grub, that is, directly from bootloader stage. The interraction with the VM is minimal.
Just because it doesn't run a whole Linux kernel, doesn't make it not a guest OS image. It is merely unsual in that it can do everything it needs from grub stage, because it is just poking low level BIOS stuff and doesn't need a full OS like Linux on top. This is still functional integration testing IMHO and relevant to avocado in QEMU.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 21 lines · Daniel P. Berrangé
Quoted 11 lines · Ani Sinha
Quoted 10 lines · Michael S. Tsirkin
Quoted 7 lines · Daniel P. Berrangé
If it is actually booting a real guest image (from biosbits) and interacting with it, then it does feel like the scope of this testing is more appropriate to QEMU's avocado framework than qtest, especially given the desire to use python for it all.
With regards,
DanielQuoted 8 lines
Quoted 4 lines
I feel avocado is directed towards booting full fledged guest OS. It makes it much easier to figure out guest issues but it also prone to false positives and is harder to debug as a result. Booting a minimal image like this shouldn't require that.
Yes 100% agree with Michael on this. Biobits is *not* booting any OS image. It runs off grub, that is, directly from bootloader stage. The interraction with the VM is minimal.
Quoted 8 lines
Just because it doesn't run a whole Linux kernel, doesn't make it not a guest OS image. It is merely unsual in that it can do everything it needs from grub stage, because it is just poking low level BIOS stuff and doesn't need a full OS like Linux on top. This is still functional integration testing IMHO and relevant to avocado in QEMU.
With regards,
Daniel
I wonder why it needs grub BTW. Let's just load all of disk into ram and be done with it?
Quoted 4 lines
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Daniel P. BerrangéQuoted 21 lines · Daniel P. Berrangé
Quoted 11 lines · Ani Sinha
Quoted 10 lines · Michael S. Tsirkin
Quoted 7 lines · Daniel P. Berrangé
If it is actually booting a real guest image (from biosbits) and interacting with it, then it does feel like the scope of this testing is more appropriate to QEMU's avocado framework than qtest, especially given the desire to use python for it all.
With regards,
DanielQuoted 8 lines
Quoted 4 lines
I feel avocado is directed towards booting full fledged guest OS. It makes it much easier to figure out guest issues but it also prone to false positives and is harder to debug as a result. Booting a minimal image like this shouldn't require that.
Yes 100% agree with Michael on this. Biobits is *not* booting any OS image. It runs off grub, that is, directly from bootloader stage. The interraction with the VM is minimal.
Quoted 8 lines
Just because it doesn't run a whole Linux kernel, doesn't make it not a guest OS image. It is merely unsual in that it can do everything it needs from grub stage, because it is just poking low level BIOS stuff and doesn't need a full OS like Linux on top. This is still functional integration testing IMHO and relevant to avocado in QEMU.
With regards,
Daniel
I think the main difference is not even in how it works, it's in what it does. Which is check that ACPI tables are sane. Who cares about that? Well developers do when they change the tables. Users really don't because for users we have the expected tables in tree and we check against these.
Signature · 1 line
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 5 lines · Michael S. Tsirkin
I think the main difference is not even in how it works, it's in what it does. Which is check that ACPI tables are sane. Who cares about that? Well developers do when they change the tables. Users really don't because for users we have the expected tables in tree and we check against these.
It wants to build and run a big guest binary blob -- that to me is the main difference. Users don't much care about any of our tests, whether they're under 'make check' or 'make check-avocado' or the iotests framework. The reason to pick one framework or another is mostly I think whether the properties of the test are such that one framework works better. Avocado is (for better or worse) the one we have for dealing with "actually run a guest machine with a big lump of guest code in it".
-- PMM
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 10 lines · Peter Maydell
Quoted 5 lines · Michael S. Tsirkin
I think the main difference is not even in how it works, it's in what it does. Which is check that ACPI tables are sane. Who cares about that? Well developers do when they change the tables. Users really don't because for users we have the expected tables in tree and we check against these.
It wants to build and run a big guest binary blob -- that to me is the main difference. Users don't much care about any of our tests,
perhaps but we do enforce patch submitters to make sure make check passes before submitting patches. make check-avocado is not run as part of make check, requires considerable disk space to download all guest images and hence generally not run by patch submitters. Making bits parts of avocado tests almost defeats the purpose of having this test at all.
Quoted 8 lines
whether they're under 'make check' or 'make check-avocado' or the iotests framework. The reason to pick one framework or another is mostly I think whether the properties of the test are such that one framework works better. Avocado is (for better or worse) the one we have for dealing with "actually run a guest machine with a big lump of guest code in it".
-- PMM
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 12 lines · Ani Sinha
Quoted 10 lines · Peter Maydell
Quoted 5 lines · Michael S. Tsirkin
I think the main difference is not even in how it works, it's in what it does. Which is check that ACPI tables are sane. Who cares about that? Well developers do when they change the tables. Users really don't because for users we have the expected tables in tree and we check against these.
It wants to build and run a big guest binary blob -- that to me is the main difference. Users don't much care about any of our tests,
Quoted 6 lines
perhaps but we do enforce patch submitters to make sure make check passes before submitting patches. make check-avocado is not run as part of make check, requires considerable disk space to download all guest images and hence generally not run by patch submitters. Making bits parts of avocado tests almost defeats the purpose of having this test at all.
You can define sane subsets of the avocado tests by using tags. For example, as I'm one of the s390x maintainers, I'm also only running the tests that are tagged with tags=arch:s390x there. You could simply introduce an "acpi" tag for your tests, too.
Thomas
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Ani SinhaQuoted 12 lines · Ani Sinha
Quoted 10 lines · Peter Maydell
Quoted 5 lines · Michael S. Tsirkin
I think the main difference is not even in how it works, it's in what it does. Which is check that ACPI tables are sane. Who cares about that? Well developers do when they change the tables. Users really don't because for users we have the expected tables in tree and we check against these.
It wants to build and run a big guest binary blob -- that to me is the main difference. Users don't much care about any of our tests,
Quoted 6 lines
perhaps but we do enforce patch submitters to make sure make check passes before submitting patches. make check-avocado is not run as part of make check, requires considerable disk space to download all guest images and hence generally not run by patch submitters. Making bits parts of avocado tests almost defeats the purpose of having this test at all.
This proposed biosbits test also involves a considerable download. The test is said to be irrelevant for anyone except those working on a fairly narrow set of QEMU firmware related bits. So by the same rationale we shouldn't impose that burden on everyone working on QEMU by having it in qtest. Making it entirely optional, only downloaded by avocado on demand, for the people who need to run the tests is best.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 14 lines · Daniel P. Berrangé
Quoted 12 lines · Ani Sinha
Quoted 10 lines · Peter Maydell
Quoted 5 lines · Michael S. Tsirkin
I think the main difference is not even in how it works, it's in what it does. Which is check that ACPI tables are sane. Who cares about that? Well developers do when they change the tables. Users really don't because for users we have the expected tables in tree and we check against these.
It wants to build and run a big guest binary blob -- that to me is the main difference. Users don't much care about any of our tests,
Quoted 8 lines
Quoted 6 lines
perhaps but we do enforce patch submitters to make sure make check passes before submitting patches. make check-avocado is not run as part of make check, requires considerable disk space to download all guest images and hence generally not run by patch submitters. Making bits parts of avocado tests almost defeats the purpose of having this test at all.
This proposed biosbits test also involves a considerable download.
I do not think 50 MB is "considerable" . Last time I tried to run avocado tests, my laptop ran out of disk space!
Quoted 2 lines
The test is said to be irrelevant for anyone except those working on a fairly narrow set of QEMU firmware related bits.
Well ok that is just a bad argument. You can say the same thing for most qtests. In fact, that is why most qtetes can run directly simply by passing QTEST_QEMU_BINARY in the environment. No need to go through make check. Same with the bits test. It can be run directly.
So by the same
Quoted 2 lines
rationale we shouldn't impose that burden on everyone working on QEMU by having it in qtest.
So why burden everyone by having bios-tables-test when it only affects acpi/smbios developers?
Making it entirely optional, only
Quoted 10 lines
downloaded by avocado on demand, for the people who need to run the tests is best.
With regards,
Daniel
--|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 5 lines · Ani Sinha
Quoted 1 line · Daniel P. Berrangé
This proposed biosbits test also involves a considerable download.
I do not think 50 MB is "considerable" . Last time I tried to run avocado tests, my laptop ran out of disk space!
I think 50MB is pretty big. It might be smaller than some other avocado tests, but it's not exactly the "no binary involved" that most qtests are.
Quoted 7 lines
Quoted 2 lines
The test is said to be irrelevant for anyone except those working on a fairly narrow set of QEMU firmware related bits.
Well ok that is just a bad argument. You can say the same thing for most qtests. In fact, that is why most qtetes can run directly simply by passing
QTEST_QEMU_BINARYin the environment. No need to go through make check. Same with the bits test. It can be run directly.
'make check' is generally the small, fast, no-binary-blobs tests. Very few 'make check' tests even run code in the guest.
Quoted 6 lines
So by the same
Quoted 2 lines
rationale we shouldn't impose that burden on everyone working on QEMU by having it in qtest.
So why burden everyone by having bios-tables-test when it only affects acpi/smbios developers?
Because it's small and fast and doesn't have a binary blob to download.
There are definitely some awkwardnesses with 'check-avocado', but we should work on fixing those, not use them as a reason to refuse to put tests into the avocado tests if that's where they fit best.
-- PMM
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 11 lines · On Tue, Jun 28, 2022 at 19:15 Peter Maydell <[email protected]> wrote:
Quoted 1 line · Ani Sinha
On Tue, Jun 28, 2022 at 6:25 PM Daniel P. Berrangé <[email protected]>
Quoted 4 lines · wrote:
Quoted 1 line
This proposed biosbits test also involves a considerable download.
I do not think 50 MB is "considerable" . Last time I tried to run avocado tests, my laptop ran out of disk space!
I think 50MB is pretty big. It might be smaller than some other avocado tests, but it's not exactly the "no binary involved" that most qtests are.
Well bios-tables-test uses the binary blobs of the acpi tables. Only difference is that in this case, we could maintain them within the qemu tree. In this case the blob in slightly larger and comes from a third party. That is the difference.
Quoted 10 lines
Quoted 8 lines
Quoted 3 lines
The test is said to be irrelevant for anyone except those working on a fairly narrow set of QEMU firmware related bits.
Well ok that is just a bad argument. You can say the same thing for most qtests. In fact, that is why most qtetes can run directly simply by passing
QTEST_QEMU_BINARYin the environment. No need to go through make check. Same with the bits test. It can be run directly.'make check' is generally the small, fast, no-binary-blobs tests.
See above.
Quoted 1 line
Very few 'make check' tests even run code in the guest.
So bits test is similar here. It runs code in the guest vm.
Quoted 8 lines
Quoted 7 lines
So by the same
Quoted 2 lines
rationale we shouldn't impose that burden on everyone working on QEMU by having it in qtest.
So why burden everyone by having bios-tables-test when it only affects acpi/smbios developers?
Quoted 6 lines
Because it's small and fast and doesn't have a binary blob to download.
There are definitely some awkwardnesses with 'check-avocado', but we should work on fixing those, not use them as a reason to refuse to put tests into the avocado tests if that's where they fit best.
I think this test fits best in the qtrst not with the integration test framework. Very few path developers will ever run it and wrote new tests for it if we have it there. I would be terribly discouraged if that’s where this test landed.
Quoted 3 lines
-- PMM
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 12 lines · Ani Sinha
Quoted 8 lines · On Tue, Jun 28, 2022 at 19:15 Peter Maydell <[email protected]> wrote:
Quoted 5 lines · Ani Sinha
Quoted 1 line · Daniel P. Berrangé
This proposed biosbits test also involves a considerable download.
I do not think 50 MB is "considerable" . Last time I tried to run avocado tests, my laptop ran out of disk space!
Quoted 6 lines
Quoted 3 lines
I think 50MB is pretty big. It might be smaller than some other avocado tests, but it's not exactly the "no binary involved" that most qtests are.
Well bios-tables-test uses the binary blobs of the acpi tables. Only difference is that in this case, we could maintain them within the qemu tree. In this case the blob in slightly larger and comes from a third party. That is the difference.
Yes. That is exactly the difference that means it should go in the avocado tests.
-- PMM
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Ani SinhaQuoted 13 lines · Ani Sinha
On Tue, Jun 28, 2022 at 19:15 Peter Maydell <[email protected] <mailto:[email protected]>> wrote:
On Tue, 28 Jun 2022 at 14:23, Ani Sinha <[email protected] <mailto:[email protected]>> wrote:
> On Tue, Jun 28, 2022 at 6:25 PM Daniel P. Berrangé
<[email protected] <mailto:[email protected]>> wrote:
> > This proposed biosbits test also involves a considerable download. >
> I do not think 50 MB is "considerable" . Last time I tried to run > avocado tests, my laptop ran out of disk space!
Quoted 6 lines
I think 50MB is pretty big. It might be smaller than some other avocado tests, but it's not exactly the "no binary involved" that most qtests are.
Well bios-tables-test uses the binary blobs of the acpi tables. Only difference is that in this case, we could maintain them within the qemu tree. In this case the blob in slightly larger and comes from a third party. That is the difference.
"slightly larger" ... it apparently contains already a complete grub and python 2.7 interpreter ... I'd consider that as *much* larger than the ~ 2k loc bios-tables-test ;-)
Quoted 3 lines
Very few 'make check' tests even run code in the guest.
So bits test is similar here. It runs code in the guest vm.
The qtests that run some TCG code only use some few lines of code. The tests that run third party firmware images are the avocado tests.
Quoted 6 lines
There are definitely some awkwardnesses with 'check-avocado', but we should work on fixing those, not use them as a reason to refuse to put tests into the avocado tests if that's where they fit best.
I think this test fits best in the qtrst not with the integration test framework.
I disagree. Third party binary blob tests are certainly nothing for the qtest directory.
Quoted 1 line
Very few path developers will ever run it and wrote new tests for it if we have it there. I would be terribly discouraged if that’s where this test landed.
I see your point - I'm also hardly ever running whole "make check-avocado" since it's too slow/fat/broken. But using this as a reason to stick your test into the qtest framework also does not work/scale: Being one of the s390x maintainers - What if I now also want my s390x related tests to be executed by everybody? The kernel + initrd images from tests/avocado/machine_s390_ccw_virtio.py are also not that big, ca. 50 MiB each. Should I now move that out of the avocade directory, too, and force everybody to consume it via a submodule? Then who's next? QEMU has 21 target architectures, if every maintainers adds a 50 MiB test to the tree, we're at more than a gigabyte of data already that you'd have to download before you were able to run "make check". This simply does not scale.
So the avocado framework is currently still the best place that we have for such tests. You just have to get used to consume it via "tags" instead of running the whole "make check-avocado" suite. For example, being a s390x maintainer, I'm only running:
make check-venv
./tests/venv/bin/avocado run -t arch:s390x tests/avocado/
and that finishes within some few minutes and only downloads some few hundreds of megabytes. You could do something similar with acpi. You could even wrap it in a "make check-avocado-acpi" target if you don't like to execute avocado directly.
I even wouldn't mind if you put your python stuff in a new directory like tests/pytests/ for example, as long as it downloads your binaries separately - as I wrote in another mail, the avocado framework rather looks like an oddball in our test framework nowadays since it uses a separate test runner and not the meson test harness, so having a new approach for python-based tests is maybe even a good idea. I just really want to avoid that this goes into tests/qtest (since it really does not belong there), and please don't add more external stuff via git submodules, that's really the wrong approach for this.
Thomas
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 9 lines · Thomas Huth
I even wouldn't mind if you put your python stuff in a new directory like tests/pytests/ for example, as long as it downloads your binaries separately
- as I wrote in another mail, the avocado framework rather looks like an oddball in our test framework nowadays since it uses a separate test runner and not the meson test harness, so having a new approach for python-based tests is maybe even a good idea. I just really want to avoid that this goes into tests/qtest (since it really does not belong there), and please don't add more external stuff via git submodules, that's really the wrong approach for this.
I get it, people hate submodules with passion. I think trying another approach for testing that is neither avocado nor qtest is not too bad. As long as this is not user visible, we can allow ourselves space to experiment.
OK so, how about this:
- put it in a new directory: tests/roms?
- create repo for a fork of biosbits under git.qemu.org
- roll our own analog to git submodules: a script that clones the repo
- new target make check-roms, if the clone exists locally - run the test, if not - skip it
- as for using pre-generates ISOs as an optimization, I'm not sure how important that is, if yes -
we can add another repo and another make target along the same lines
Signature · 1 line
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 17 lines · Michael S. Tsirkin
Quoted 9 lines · Thomas Huth
I even wouldn't mind if you put your python stuff in a new directory like tests/pytests/ for example, as long as it downloads your binaries separately
- as I wrote in another mail, the avocado framework rather looks like an oddball in our test framework nowadays since it uses a separate test runner and not the meson test harness, so having a new approach for python-based tests is maybe even a good idea. I just really want to avoid that this goes into tests/qtest (since it really does not belong there), and please don't add more external stuff via git submodules, that's really the wrong approach for this.I get it, people hate submodules with passion. I think trying another approach for testing that is neither avocado nor qtest is not too bad. As long as this is not user visible, we can allow ourselves space to experiment.
Quoted 5 lines
OK so, how about this:
- put it in a new directory: tests/roms?
- create repo for a fork of biosbits under git.qemu.org
- roll our own analog to git submodules: a script that clones the repo
No need to clone the whole repo. We can simply download the binaries that the girlab CI job would generate from the bits sources in that repo.
We need to clone if we are always building bits from source for every test. That is not necessary IMHO since much of the bits package would remain as is without modification.
Quoted 1 line
- new target make check-roms,
I think make pytest or some such is better and more generic if other such tests in other areas follow suit.
if the clone exists locally -
Quoted 1 line
run the test, if not - skip it
if download of the bits binaries fail, skip it.
Quoted 10 lines
- as for using pre-generates ISOs as an optimization, I'm not sure how important that is, if yes -
we can add another repo and another make target along the same lines--
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 24 lines · Ani Sinha
Quoted 17 lines · Michael S. Tsirkin
Quoted 9 lines · Thomas Huth
I even wouldn't mind if you put your python stuff in a new directory like tests/pytests/ for example, as long as it downloads your binaries separately
- as I wrote in another mail, the avocado framework rather looks like an oddball in our test framework nowadays since it uses a separate test runner and not the meson test harness, so having a new approach for python-based tests is maybe even a good idea. I just really want to avoid that this goes into tests/qtest (since it really does not belong there), and please don't add more external stuff via git submodules, that's really the wrong approach for this.I get it, people hate submodules with passion. I think trying another approach for testing that is neither avocado nor qtest is not too bad. As long as this is not user visible, we can allow ourselves space to experiment.
Quoted 5 lines
OK so, how about this:
- put it in a new directory: tests/roms?
- create repo for a fork of biosbits under git.qemu.org
- roll our own analog to git submodules: a script that clones the repo
Quoted 6 lines
No need to clone the whole repo. We can simply download the binaries that the girlab CI job would generate from the bits sources in that repo.
We need to clone if we are always building bits from source for every test. That is not necessary IMHO since much of the bits package would remain as is without modification.
IMHO CI job idea isn't great since isn't versioned at all, is it? Also as long as test passes, sure. But if it fails one will need the sources to investigate.
Let's start with building things from source. Add an option of prebuilt binaries as an optimization once things stabilize.
Quoted 4 lines
Quoted 1 line
- new target make check-roms,
I think make pytest or some such is better and more generic if other such tests in other areas follow suit.
The name is not critical in my mind, but I think we need to decide what exactly differentiates it from other tests.
Quoted 5 lines
if the clone exists locally -
Quoted 1 line
run the test, if not - skip it
if download of the bits binaries fail, skip it.
You seem to be recreating either git or avocado or both here.
Personally I want something that works offline.
Quoted 10 lines
Quoted 10 lines
- as for using pre-generates ISOs as an optimization, I'm not sure how important that is, if yes -
we can add another repo and another make target along the same lines--
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 26 lines · Michael S. Tsirkin
Quoted 24 lines · Ani Sinha
Quoted 22 lines · Michael S. Tsirkin
Quoted 9 lines · Thomas Huth
I even wouldn't mind if you put your python stuff in a new directory like tests/pytests/ for example, as long as it downloads your binaries separately
- as I wrote in another mail, the avocado framework rather looks like an oddball in our test framework nowadays since it uses a separate test runner and not the meson test harness, so having a new approach for python-based tests is maybe even a good idea. I just really want to avoid that this goes into tests/qtest (since it really does not belong there), and please don't add more external stuff via git submodules, that's really the wrong approach for this.I get it, people hate submodules with passion. I think trying another approach for testing that is neither avocado nor qtest is not too bad. As long as this is not user visible, we can allow ourselves space to experiment.
OK so, how about this:
- put it in a new directory: tests/roms?
- create repo for a fork of biosbits under git.qemu.org
- roll our own analog to git submodules: a script that clones the repo
Quoted 8 lines
Quoted 6 lines
No need to clone the whole repo. We can simply download the binaries that the girlab CI job would generate from the bits sources in that repo.
We need to clone if we are always building bits from source for every test. That is not necessary IMHO since much of the bits package would remain as is without modification.IMHO CI job idea isn't great since isn't versioned at all, is it?
bits is versioned yes, in a crude way. every time you make a commit in the top level repo, the version would increment by one.
Quoted 2 lines
Also as long as test passes, sure. But if it fails one will need the sources to investigate.
sources might also be needed to write the tests.
Quoted 2 lines
Let's start with building things from source.
hmm. bitys uses old autotools, not ninja and takes about 10/15 mins to build depending on parallelity and build host.
Add an option
Quoted 36 lines
of prebuilt binaries as an optimization once things stabilize.
Quoted 4 lines
Quoted 1 line
- new target make check-roms,
I think make pytest or some such is better and more generic if other such tests in other areas follow suit.
The name is not critical in my mind, but I think we need to decide what exactly differentiates it from other tests.
Quoted 5 lines
if the clone exists locally -
Quoted 1 line
run the test, if not - skip it
if download of the bits binaries fail, skip it.
You seem to be recreating either git or avocado or both here.
Personally I want something that works offline.
Quoted 10 lines
Quoted 10 lines
- as for using pre-generates ISOs as an optimization, I'm not sure how important that is, if yes -
we can add another repo and another make target along the same lines--
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 34 lines · Ani Sinha
Quoted 33 lines · Michael S. Tsirkin
Quoted 30 lines · Ani Sinha
Quoted 17 lines · Michael S. Tsirkin
Quoted 9 lines · Thomas Huth
I even wouldn't mind if you put your python stuff in a new directory like tests/pytests/ for example, as long as it downloads your binaries separately
- as I wrote in another mail, the avocado framework rather looks like an oddball in our test framework nowadays since it uses a separate test runner and not the meson test harness, so having a new approach for python-based tests is maybe even a good idea. I just really want to avoid that this goes into tests/qtest (since it really does not belong there), and please don't add more external stuff via git submodules, that's really the wrong approach for this.I get it, people hate submodules with passion. I think trying another approach for testing that is neither avocado nor qtest is not too bad. As long as this is not user visible, we can allow ourselves space to experiment.
Quoted 5 lines
OK so, how about this:
- put it in a new directory: tests/roms?
- create repo for a fork of biosbits under git.qemu.org
- roll our own analog to git submodules: a script that clones the repoNo need to clone the whole repo. We can simply download the binaries that the girlab CI job would generate from the bits sources in that repo.
We need to clone if we are always building bits from source for every test. That is not necessary IMHO since much of the bits package would remain as is without modification.
Quoted 4 lines
Quoted 1 line
IMHO CI job idea isn't great since isn't versioned at all, is it?
bits is versioned yes, in a crude way. every time you make a commit in the top level repo, the version would increment by one.
Is it easy to find out which source was this generated from? And is there a promise to keep these around indefinitely?
Quoted 10 lines
Quoted 2 lines
Also as long as test passes, sure. But if it fails one will need the sources to investigate.
sources might also be needed to write the tests.
Quoted 2 lines
Let's start with building things from source.
hmm. bitys uses old autotools, not ninja and takes about 10/15 mins to build depending on parallelity and build host.
Right. But whoever wants to use these just needs to do it once.
Quoted 37 lines
Add an option
Quoted 36 lines
of prebuilt binaries as an optimization once things stabilize.
Quoted 4 lines
Quoted 1 line
- new target make check-roms,
I think make pytest or some such is better and more generic if other such tests in other areas follow suit.
The name is not critical in my mind, but I think we need to decide what exactly differentiates it from other tests.
Quoted 5 lines
if the clone exists locally -
Quoted 1 line
run the test, if not - skip it
if download of the bits binaries fail, skip it.
You seem to be recreating either git or avocado or both here.
Personally I want something that works offline.
Quoted 10 lines
Quoted 10 lines
- as for using pre-generates ISOs as an optimization, I'm not sure how important that is, if yes -
we can add another repo and another make target along the same lines--
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 36 lines · Michael S. Tsirkin
Quoted 34 lines · Ani Sinha
Quoted 33 lines · Michael S. Tsirkin
Quoted 30 lines · Ani Sinha
Quoted 17 lines · Michael S. Tsirkin
Quoted 9 lines · Thomas Huth
I even wouldn't mind if you put your python stuff in a new directory like tests/pytests/ for example, as long as it downloads your binaries separately
- as I wrote in another mail, the avocado framework rather looks like an oddball in our test framework nowadays since it uses a separate test runner and not the meson test harness, so having a new approach for python-based tests is maybe even a good idea. I just really want to avoid that this goes into tests/qtest (since it really does not belong there), and please don't add more external stuff via git submodules, that's really the wrong approach for this.I get it, people hate submodules with passion. I think trying another approach for testing that is neither avocado nor qtest is not too bad. As long as this is not user visible, we can allow ourselves space to experiment.
Quoted 5 lines
OK so, how about this:
- put it in a new directory: tests/roms?
- create repo for a fork of biosbits under git.qemu.org
- roll our own analog to git submodules: a script that clones the repoNo need to clone the whole repo. We can simply download the binaries that the girlab CI job would generate from the bits sources in that repo.
We need to clone if we are always building bits from source for every test. That is not necessary IMHO since much of the bits package would remain as is without modification.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
IMHO CI job idea isn't great since isn't versioned at all, is it?
bits is versioned yes, in a crude way. every time you make a commit in the top level repo, the version would increment by one.
Is it easy to find out which source was this generated from?
yes because this is how the version number is generated:
expr 2000 + git rev-list HEAD 2>/dev/null | wc -l
That means we simply count the number of commits in the top level directory. If we overwrote history by force pushing, then well yes all bets are off.
Quoted 1 line
And is there a promise to keep these around indefinitely?
We do not need to maintain multiple versions of bits or increment the version number every time we wrote a test. Typically new commits/changes will be made if we wanted to fix a bug or upgrade acpica or whatever. Test scripts are python based and can be pushed into bits while generating a new iso for testing purposes.
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Also as long as test passes, sure. But if it fails one will need the sources to investigate.
sources might also be needed to write the tests.
Quoted 1 line
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Let's start with building things from source.
hmm. bitys uses old autotools, not ninja and takes about 10/15 mins to build depending on parallelity and build host.
Right. But whoever wants to use these just needs to do it once.
but I thought you were suggesting we built bits every time the test is run?
Quoted 40 lines
Quoted 39 lines
Add an option
Quoted 36 lines
of prebuilt binaries as an optimization once things stabilize.
Quoted 4 lines
Quoted 1 line
- new target make check-roms,
I think make pytest or some such is better and more generic if other such tests in other areas follow suit.
The name is not critical in my mind, but I think we need to decide what exactly differentiates it from other tests.
Quoted 5 lines
if the clone exists locally -
Quoted 1 line
run the test, if not - skip it
if download of the bits binaries fail, skip it.
You seem to be recreating either git or avocado or both here.
Personally I want something that works offline.
Quoted 10 lines
Quoted 10 lines
- as for using pre-generates ISOs as an optimization, I'm not sure how important that is, if yes -
we can add another repo and another make target along the same lines--
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 1 line · Ani Sinha
but I thought you were suggesting we built bits every time the test is run?
In my opinion 3 scenarios are worth supporting:
- people not touching ACPI, including users - simply don't run the tests, comparing tables with expected output should be enough
- people making changes indirectly affecting ACPI -
use tests to validate that tables are still well formed, using a pre built binary should be enough IMO
- people working on ACPI - use tests to analyse the tables, building from source might be necessary for debugging, sources change very rarely
- people developing the tests
building from source is required
So I would suggest basically two github repos, one with binaries one with sources. We'll keep relevant hashes to use in the script.
All in all not that different from submodules but I guess people have submodules and that is that.
And I personally would probably not tie it to CI whoever owns the repository can worry about the builds, and I think keeping things distributed is important.
So
- people not touching ACPI - make check should see directory not found and skip the test
- people making changes indirectly affecting ACPI -
check out binaries and use
- people working on ACPI -
see that source directory is present, go there
and run make. should not rebuild each time right?
- people developing the tests
building from source is required
Signature · 1 line
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 25 lines · Michael S. Tsirkin
Quoted 1 line · Ani Sinha
but I thought you were suggesting we built bits every time the test is run?
In my opinion 3 scenarios are worth supporting:
- people not touching ACPI, including users - simply don't run the tests, comparing tables with expected output should be enough
- people making changes indirectly affecting ACPI -
use tests to validate that tables are still well formed, using a pre built binary should be enough IMO
- people working on ACPI - use tests to analyse the tables, building from source might be necessary for debugging, sources change very rarely
- people developing the tests
building from source is requiredSo I would suggest basically two github repos, one with binaries one with sources. We'll keep relevant hashes to use in the script.
All in all not that different from submodules but I guess people have submodules and that is that.And I personally would probably not tie it to CI whoever owns the repository can worry about the builds, and I think keeping things distributed is important.
Quoted 10 lines
So
- people not touching ACPI - make check should see directory not found and skip the test
- people making changes indirectly affecting ACPI -
check out binaries and use
- people working on ACPI -
see that source directory is present, go there
and run make. should not rebuild each time right?
- people developing the tests
building from source is required
Ok I have now committed a Dockerfile that has all the build time dependencies and builds bits and generates the tarballs/zip file that my test requires:
https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile
We just need to fork the repo and generate automated builds with this Dockerfile and put the binaries somewhere. This should also help developers requiring to rebuild bits when necessary.
Oh and btw, I also made bits compliant with the latest gcc 11 compiler that comes with Ubuntu 22.04 LTS :-)
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 43 lines · Ani Sinha
Quoted 25 lines · Michael S. Tsirkin
Quoted 1 line · Ani Sinha
but I thought you were suggesting we built bits every time the test is run?
In my opinion 3 scenarios are worth supporting:
- people not touching ACPI, including users - simply don't run the tests, comparing tables with expected output should be enough
- people making changes indirectly affecting ACPI -
use tests to validate that tables are still well formed, using a pre built binary should be enough IMO
- people working on ACPI - use tests to analyse the tables, building from source might be necessary for debugging, sources change very rarely
- people developing the tests
building from source is requiredSo I would suggest basically two github repos, one with binaries one with sources. We'll keep relevant hashes to use in the script.
All in all not that different from submodules but I guess people have submodules and that is that.And I personally would probably not tie it to CI whoever owns the repository can worry about the builds, and I think keeping things distributed is important.
Quoted 10 lines
So
- people not touching ACPI - make check should see directory not found and skip the test
- people making changes indirectly affecting ACPI -
check out binaries and use
- people working on ACPI -
see that source directory is present, go there
and run make. should not rebuild each time right?
- people developing the tests
building from source is requiredOk I have now committed a Dockerfile that has all the build time dependencies and builds bits and generates the tarballs/zip file that my test requires:
https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile
Quoted 5 lines
We just need to fork the repo and generate automated builds with this Dockerfile and put the binaries somewhere. This should also help developers requiring to rebuild bits when necessary.
Oh and btw, I also made bits compliant with the latest gcc 11 compiler that comes with Ubuntu 22.04 LTS :-)
Pushed a build script here:
https://github.com/ani-sinha/bits/commit/90b99ef05d55ead4b33b2fb19ad07dfb9682ec92
and the bios bits binaries are in this branch:
https://github.com/ani-sinha/bits/tree/bits-builds
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Michael S. TsirkinQuoted 52 lines · Michael S. Tsirkin
Quoted 24 lines · Ani Sinha
Quoted 17 lines · Michael S. Tsirkin
Quoted 9 lines · Thomas Huth
I even wouldn't mind if you put your python stuff in a new directory like tests/pytests/ for example, as long as it downloads your binaries separately
- as I wrote in another mail, the avocado framework rather looks like an oddball in our test framework nowadays since it uses a separate test runner and not the meson test harness, so having a new approach for python-based tests is maybe even a good idea. I just really want to avoid that this goes into tests/qtest (since it really does not belong there), and please don't add more external stuff via git submodules, that's really the wrong approach for this.I get it, people hate submodules with passion. I think trying another approach for testing that is neither avocado nor qtest is not too bad. As long as this is not user visible, we can allow ourselves space to experiment.
Quoted 5 lines
OK so, how about this:
- put it in a new directory: tests/roms?
- create repo for a fork of biosbits under git.qemu.org
- roll our own analog to git submodules: a script that clones the repoQuoted 6 lines
No need to clone the whole repo. We can simply download the binaries that the girlab CI job would generate from the bits sources in that repo.
We need to clone if we are always building bits from source for every test. That is not necessary IMHO since much of the bits package would remain as is without modification.IMHO CI job idea isn't great since isn't versioned at all, is it? Also as long as test passes, sure. But if it fails one will need the sources to investigate.
Let's start with building things from source. Add an option of prebuilt binaries as an optimization once things stabilize.
Quoted 4 lines
Quoted 1 line
- new target make check-roms,
I think make pytest or some such is better and more generic if other such tests in other areas follow suit.
The name is not critical in my mind, but I think we need to decide what exactly differentiates it from other tests.
Quoted 1 line
Quoted 8 lines
Quoted 4 lines
if the clone exists locally -
Quoted 1 line
run the test, if not - skip it
if download of the bits binaries fail, skip it.
You seem to be recreating either git or avocado or both here.
Personally I want something that works offline.
Well we need to make a call as to how to make things work offline, if that's what we wanted. I have made changes in v2 to download the binaries from my github repo. It works great. If download fails, it skips the test. If those files are available before, it will not attempt to download them.
https://pastebin.com/nrcdN8WS and
https://pastebin.com/m7Dx3PZk
The Dockerfile and build script I have in the repo will generate these artifacts. So if you wanted to run the test offline, either build those from source and place them where the test expects to find them or build them from source using Dockerfile/build script and place them there before running the test.
Quoted 14 lines
Quoted 13 lines
Quoted 13 lines
- as for using pre-generates ISOs as an optimization, I'm not sure how important that is, if yes -
we can add another repo and another make target along the same lines--
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Ani SinhaQuoted 2 lines · Ani Sinha
So why burden everyone by having bios-tables-test when it only affects acpi/smbios developers?
No it doesn't. Any change in qemu can break ACPI - it's enough to register an MR overlapping the ACPI registers. So we have a basic test making sure ACPI is in expected state and have everyone run it.
Signature · 1 line
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Daniel P. BerrangéQuoted 28 lines · Daniel P. Berrangé
Quoted 12 lines · Ani Sinha
Quoted 10 lines · Peter Maydell
Quoted 5 lines · Michael S. Tsirkin
I think the main difference is not even in how it works, it's in what it does. Which is check that ACPI tables are sane. Who cares about that? Well developers do when they change the tables. Users really don't because for users we have the expected tables in tree and we check against these.
It wants to build and run a big guest binary blob -- that to me is the main difference. Users don't much care about any of our tests,
Quoted 6 lines
perhaps but we do enforce patch submitters to make sure make check passes before submitting patches. make check-avocado is not run as part of make check, requires considerable disk space to download all guest images and hence generally not run by patch submitters. Making bits parts of avocado tests almost defeats the purpose of having this test at all.
This proposed biosbits test also involves a considerable download. The test is said to be irrelevant for anyone except those working on a fairly narrow set of QEMU firmware related bits. So by the same rationale we shouldn't impose that burden on everyone working on QEMU by having it in qtest. Making it entirely optional, only downloaded by avocado on demand, for the people who need to run the tests is best.
Quoted 2 lines
With regards,
Daniel
Optional is fine. I would like to make life easy for developers working on ACPI though, the field is unforgiving enough as it is.
How about we skip the test if the submodule is not checked out?
Quoted 4 lines
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Thomas HuthQuoted 96 lines · Thomas Huth
Quoted 87 lines · Daniel P. Berrangé
Quoted 69 lines · Ani Sinha
Quoted 68 lines · Daniel P. Berrangé
Quoted 43 lines · Thomas Huth
Quoted 42 lines · Michael S. Tsirkin
Quoted 40 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
No problem with that. So that's venv. But do we need pip and pulling
packages from the net during testing?We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
Quoted 8 lines
Quoted 4 lines
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Definitely not submodules.
We need to get out of the mindset that submodules are needed for every new dependancy we add. Submodules are only appropriate if the external project is designed to be used as a copylib (eg the keycodemapdb tool), or if we need to bundle in order to prevent a regression for previously deployed QEMU installs where the dependancy is known not to exist on all our supported platforms.
This does not apply in this case, because the proposed use of tappy is merely for a test case. Meson just needs to check if tappy exists and if it does, then use it, otherwise skip the tests that need it. The user can arrange to install tappy, as they do with the majority of other deps.
Quoted 6 lines
Quoted 4 lines
If John's venv stuff is relevant, then we don't even need the meson checks, just delegate to the venv setup.
Regardless, no submodules are needed or desirable.
What about keeping biosbits stuff? Source or pre-built.
Shipping them as pre-built binaries in QEMU is not a viable option IMHO, especially for grub as a GPL'd project we need to be extremely clear about the exact corresponding source and build process for any binary.
For this kind of thing I would generally expect the distro to provide packages that we consume. Looking at biosbits I see it is itself bundling a bunch more 3rd party projects, libffi, grub2, and including even an ancient version of python as a submodule.
Quoted 7 lines
So bundling a pre-built biosbits in QEMU appears to mean that we're in turn going to unexpectedly bundle a bunch of other 3rd party projects too, all with dubious license compliance. I don't think this looks like something we should have in qemu.git or qemu tarballs. It will also make it challenging for the distro to take biosbits at all, unless those 3rd party bundles can be eliminated in favour of using existing builds their have packaged for grub, python, libffi, etc.
Quoted 4 lines
So if this depends on some third party binary bits, I think this is pretty similar to the tests in the avocado directory ... there we download third party binaries, too... Wouldn't it make sense to adapt your tests to that framework?
Now that you mention it, avocado does feel like a more appropriate fit. IIUC the biosbits project appears to be effectively providing a custom guest OS ISO image. IOW this testing is quite biased towards being integration testing which is the target of avocado, while qtest is much more to the unit testing end of the spectrum. This would avoid all the discussion and patches around introducing python to qtest
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 9 lines · Daniel P. Berrangé
Now that you mention it, avocado does feel like a more appropriate fit. IIUC the biosbits project appears to be effectively providing a custom guest OS ISO image. IOW this testing is quite biased towards being integration testing which is the target of avocado, while qtest is much more to the unit testing end of the spectrum. This would avoid all the discussion and patches around introducing python to qtest
With regards,
Daniel
Yes it's an ISO image but not a full OS. So IMHO not really a integration test, more like a unit test for ACPI - current tests we have just compare ACPI to expected file which is okay but very limited.
Signature · 1 line
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 8 lines · Michael S. Tsirkin
Quoted 7 lines · Daniel P. Berrangé
Now that you mention it, avocado does feel like a more appropriate fit. IIUC the biosbits project appears to be effectively providing a custom guest OS ISO image. IOW this testing is quite biased towards being integration testing which is the target of avocado, while qtest is much more to the unit testing end of the spectrum. This would avoid all the discussion and patches around introducing python to qtest
Quoted 4 lines
Quoted 2 lines
With regards,
DanielYes it's an ISO image but not a full OS.
The avocado framework is not meant for full OS testing only. We have a couple of tests there that just run a small firmware image. For example:
- tests/avocado/machine_m68k_nextcube.py : Just runs the original firmware of the NexT Cube machine
- tests/avocado/ppc_405.py : Just runs an U-Boot firmware image
Some of the tests even do not download anything at all and just run a QEMU instance via python, e.g.:
- tests/avocado/virtio_check_params.py
- tests/avocado/x86_cpu_model_versions.py
So I fail to see why biosbits should be so much different here.
Of course we could argue whether the avocado framework is really such a great fit for the QEMU test suite (as I said in another mail, it rather feels like an oddball nowadays), but that's a different argument.
Thomas
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Daniel P. BerrangéQuoted 103 lines · Daniel P. Berrangé
Quoted 96 lines · Thomas Huth
Quoted 87 lines · Daniel P. Berrangé
Quoted 69 lines · Ani Sinha
Quoted 68 lines · Daniel P. Berrangé
Quoted 43 lines · Thomas Huth
Quoted 42 lines · Michael S. Tsirkin
Quoted 40 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
No problem with that. So that's venv. But do we need pip and pulling
packages from the net during testing?We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
Quoted 8 lines
Quoted 4 lines
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Definitely not submodules.
We need to get out of the mindset that submodules are needed for every new dependancy we add. Submodules are only appropriate if the external project is designed to be used as a copylib (eg the keycodemapdb tool), or if we need to bundle in order to prevent a regression for previously deployed QEMU installs where the dependancy is known not to exist on all our supported platforms.
This does not apply in this case, because the proposed use of tappy is merely for a test case. Meson just needs to check if tappy exists and if it does, then use it, otherwise skip the tests that need it. The user can arrange to install tappy, as they do with the majority of other deps.
Quoted 6 lines
Quoted 4 lines
If John's venv stuff is relevant, then we don't even need the meson checks, just delegate to the venv setup.
Regardless, no submodules are needed or desirable.
What about keeping biosbits stuff? Source or pre-built.
Shipping them as pre-built binaries in QEMU is not a viable option IMHO, especially for grub as a GPL'd project we need to be extremely clear about the exact corresponding source and build process for any binary.
For this kind of thing I would generally expect the distro to provide packages that we consume. Looking at biosbits I see it is itself bundling a bunch more 3rd party projects, libffi, grub2, and including even an ancient version of python as a submodule.
Quoted 7 lines
So bundling a pre-built biosbits in QEMU appears to mean that we're in turn going to unexpectedly bundle a bunch of other 3rd party projects too, all with dubious license compliance. I don't think this looks like something we should have in qemu.git or qemu tarballs. It will also make it challenging for the distro to take biosbits at all, unless those 3rd party bundles can be eliminated in favour of using existing builds their have packaged for grub, python, libffi, etc.
Quoted 4 lines
So if this depends on some third party binary bits, I think this is pretty similar to the tests in the avocado directory ... there we download third party binaries, too... Wouldn't it make sense to adapt your tests to that framework?
Quoted 5 lines
Now that you mention it, avocado does feel like a more appropriate fit. IIUC the biosbits project appears to be effectively providing a custom guest OS ISO image. IOW this testing is quite biased towards being integration testing which is the target of avocado, while qtest is much more to the unit testing end of the spectrum.
This is more like unit testing than integration testing, now that you mention it. It tests only the bios, very narrowly and does not involve any OS at all.
This would avoid all the
Quoted 9 lines
discussion and patches around introducing python to qtest
With regards,
Daniel
--|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 111 lines · Ani Sinha
Quoted 103 lines · Daniel P. Berrangé
Quoted 96 lines · Thomas Huth
Quoted 87 lines · Daniel P. Berrangé
Quoted 69 lines · Ani Sinha
Quoted 68 lines · Daniel P. Berrangé
Quoted 43 lines · Thomas Huth
Quoted 42 lines · Michael S. Tsirkin
Quoted 40 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 15 lines
No problem with that. So that's venv. But do we need pip and pulling
packages from the net during testing?We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from
the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
Quoted 8 lines
Quoted 4 lines
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Definitely not submodules.
We need to get out of the mindset that submodules are needed for every new dependancy we add. Submodules are only appropriate if the external project is designed to be used as a copylib (eg the keycodemapdb tool), or if we need to bundle in order to prevent a regression for previously deployed QEMU installs where the dependancy is known not to exist on all our supported platforms.
This does not apply in this case, because the proposed use of tappy is merely for a test case. Meson just needs to check if tappy exists and if it does, then use it, otherwise skip the tests that need it. The user can arrange to install tappy, as they do with the majority of other deps.
Quoted 6 lines
Quoted 4 lines
If John's venv stuff is relevant, then we don't even need the meson checks, just delegate to the venv setup.
Regardless, no submodules are needed or desirable.
What about keeping biosbits stuff? Source or pre-built.
Shipping them as pre-built binaries in QEMU is not a viable option IMHO, especially for grub as a GPL'd project we need to be extremely clear about the exact corresponding source and build process for any binary.
For this kind of thing I would generally expect the distro to provide packages that we consume. Looking at biosbits I see it is itself bundling a bunch more 3rd party projects, libffi, grub2, and including even an ancient version of python as a submodule.
Quoted 7 lines
So bundling a pre-built biosbits in QEMU appears to mean that we're in turn going to unexpectedly bundle a bunch of other 3rd party projects too, all with dubious license compliance. I don't think this looks like something we should have in qemu.git or qemu tarballs. It will also make it challenging for the distro to take biosbits at all, unless those 3rd party bundles can be eliminated in favour of using existing builds their have packaged for grub, python, libffi, etc.
Quoted 4 lines
So if this depends on some third party binary bits, I think this is pretty similar to the tests in the avocado directory ... there we download third party binaries, too... Wouldn't it make sense to adapt your tests to that framework?
Quoted 5 lines
Now that you mention it, avocado does feel like a more appropriate fit. IIUC the biosbits project appears to be effectively providing a custom guest OS ISO image. IOW this testing is quite biased towards being integration testing which is the target of avocado, while qtest is much more to the unit testing end of the spectrum.
Quoted 3 lines
This is more like unit testing than integration testing, now that you mention it. It tests only the bios, very narrowly and does not involve any OS at all.
Another thing to consider is that integration testing is further down the line? Not for once when submitting patches on acpi have I run them. However, every time I have run make check to make sure bios-tables-test passes and I am not breaking anything. It's much more useful to have this kind of thing part of make check before patch submitters can quickly check for failures either in bios-tables-test or in bits. Also its lot easier to add new acpi/smbios tests as a part of this when bios-tables-test and this one are closer together.
Quoted 11 lines
This would avoid all the
Quoted 9 lines
discussion and patches around introducing python to qtest
With regards,
Daniel
--|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Daniel P. BerrangéQuoted 3 lines · Daniel P. Berrangé
So bundling a pre-built biosbits in QEMU appears to mean that we're in turn going to unexpectedly bundle a bunch of other 3rd party projects too, all with dubious license compliance.
Well looks like classical mere aggregation to me ... license issues need to be figured out if we are to distribute things but I think this is basically what distros do anyway.
And I doubt we want to support arbitrary versions of grub etc, they are very distro specific tools.
I don't see why we can't have the resulting ISOs in some submodule - nothing requires us to distribute it in qemu tarballs.
Signature · 1 line
MST
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 5 lines · Michael S. Tsirkin
Quoted 3 lines · Daniel P. Berrangé
So bundling a pre-built biosbits in QEMU appears to mean that we're in turn going to unexpectedly bundle a bunch of other 3rd party projects too, all with dubious license compliance.
Quoted 8 lines
Well looks like classical mere aggregation to me ... license issues need to be figured out if we are to distribute things but I think this is basically what distros do anyway.
And I doubt we want to support arbitrary versions of grub etc, they are very distro specific tools.
I don't see why we can't have the resulting ISOs in some submodule - nothing requires us to distribute it in qemu tarballs.
If we don't distribute it in the tarballs then these tests won't be usable for testing release builds of QEMU which feels wrong to me.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Daniel P. BerrangéQuoted 62 lines · Daniel P. Berrangé
Quoted 43 lines · Thomas Huth
Quoted 42 lines · Michael S. Tsirkin
Quoted 40 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
Quoted 8 lines
Quoted 4 lines
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Definitely not submodules.
We need to get out of the mindset that submodules are needed for every new dependancy we add. Submodules are only appropriate if the external project is designed to be used as a copylib (eg the keycodemapdb tool), or if we need to bundle in order to prevent a regression for previously deployed QEMU installs where the dependancy is known not to exist on all our supported platforms.
Quoted 4 lines
This does not apply in this case, because the proposed use of tappy is merely for a test case. Meson just needs to check if tappy exists and if it does, then use it, otherwise skip the tests that need it. The user can arrange to install tappy, as they do with the majority of other deps.
I think that's reasonable, except I think we should make configure do the checks as that is when users see the output of what is available.
By the way, it might make sense
to have a script setting up developer dependencies for qemu for major platforms.
Quoted 2 lines
If John's venv stuff is relevant, then we don't even need the meson checks, just delegate to the venv setup.
IIUC venv does not install things itself. pip does.
Quoted 8 lines
Regardless, no submodules are needed or desirable.
With regards,
DanielSignature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Thomas HuthQuoted 44 lines · Thomas Huth
Quoted 42 lines · Michael S. Tsirkin
Quoted 40 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
Quoted 8 lines
Quoted 4 lines
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Not sure what you mean by directly using the venv.
Quoted 3 lines
Thomas
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
Quoted 50 lines · Ani Sinha
Quoted 49 lines · Thomas Huth
Quoted 46 lines · Michael S. Tsirkin
Quoted 34 lines · Thomas Huth
Quoted 34 lines
Quoted 20 lines
Quoted 20 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.Quoted 8 lines
Quoted 6 lines
It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.Quoted 6 lines
Quoted 1 line
I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Thomas
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
Quoted 5 lines
Quoted 3 lines
submodules create a dependency on an internet connection, too. So before you add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.
Not sure what you mean by directly using the venv.
I meant installing the dependency via pip in a venv, as you are currently doing it via the acpi-bits-test-venv.sh script.
Thomas
Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)
In reply to Thomas HuthQuoted 41 lines · Thomas Huth
Quoted 40 lines · Michael S. Tsirkin
Quoted 39 lines · Thomas Huth
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
Quoted 8 lines
Quoted 1 line
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
John Snow is currently working on the "Pythonification" of various QEMU bits, I think you should loop him into this discussion, too.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Thomas
submodule does not mean we fork necessarily. We could have all options: check for the module and use it if there, if not use one from system if not there install with pip .. But yea, I'm not sure what's best either.
submodules create a dependency on an internet connection, too.
Yes but this dependency is explicit.
Quoted 5 lines
So before you
add yet another submodule (which have a couple of other disadvantages), I think you could also directly use the venv here.Thomas
Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests
In reply to Ani Sinha+danPb who pointed me to the TAP stuff. Maybe he has some good ideas on this.
Quoted 295 lines · Ani Sinha
Quoted 98 lines · Michael S. Tsirkin
Quoted 96 lines · Ani Sinha
Quoted 83 lines · Michael S. Tsirkin
Quoted 81 lines · Ani Sinha
Quoted 66 lines · Michael S. Tsirkin
Quoted 64 lines · Ani Sinha
Quoted 36 lines · Michael S. Tsirkin
Quoted 34 lines · Ani Sinha
The README file is added describing the directory structure and the purpose of every file it contains. It also describes how to add new tests, make changes to existing tests or bits config files or regenerate the bits software.
Signed-off-by Ani Sinha <[email protected]>--- tests/qtest/acpi-bits/README | 168 +++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 tests/qtest/acpi-bits/READMEtests/qtest/acpi-bits/README new file +18
-0@@ -0,0 +1,168 @@ +============================================================================= +ACPI/SMBIOS QTESTS USING BIOSBITS +============================================================================= + +Biosbits is a software written by Josh Triplett that can be downloaded by +visiting https://biosbits.org/. The github codebase can be found here: +https://github.com/biosbits/bits/tree/master. It is a software that exercizes +the bios components such as acpi and smbios tables directly through acpica +bios interpreter (a freely available C based library written by Intel, +downloadable from https://acpica.org/ and is included with biosbits) without an +operating system getting involved in between. +There are several advantages to directly testing the bios in a real physical +machine or VM as opposed to indirectly discovering bios issues through the +operating system. For one thing, the OSes tend to hide bios problems from the +end user. The other is that we have more control of what we wanted to test +and how by directly using acpica interpreter on top of the bios on a running +system. More details on the inspiration for developing biosbits and its real +life uses can be found in (a) and (b).Quoted 8 lines
tests/qtest/acpi-bits/README line 23
Quoted 5 lines
tests/qtest/acpi-bits/README new file +5
-0+This directory contains QEMU qtests written in python that exercizes the QEMU +bios components using biosbits and reports test failures. + +These tests use python virtual environment. In debian/ubuntu system, the tests +would require python3.8-venv and python3-pip packages to be installed.Why do we mess with venv and pip? Certainly possible but what's wrong with using distro provided packages?
There are two things:
(a) We are already using pip and venv for our avocado based integration tests. Look forTESTS_VENV_DIRin Makefile.include under tests.
(b) the venv is primarily needed because I wanted to take advantage of our rich python library that handles QEMU based machines. There are python qtest libraries as well. These are well tested and used with integration tests and I wanted to keep the test part of the code simple by simply reusing them. however, in order to use them, we need a venv environment within which these qemu python libraries are installed. Integration tests does the same thing.A note about my language of choice - python. I gave a lot of thoughts on this. We do not do a lot of stuff here. All we do is:
(a) generate bits iso.
(b) spawn a QEMU vm with the iso which then runs a bunch of tests within the vm.
(c) collect and analyze logs.Quoted 8 lines
Quoted 5 lines
We are not inspecting guest memory or manipulating devices or pci buses. We do not need the power of C here. We need something that is simple to write, easy to maintain and understand and can deal with things like manipulating text files and configs easily. Python seems a better fit for the role.
No problem with that. So that's venv. But do we need pip and pulling packages from the net during testing?
We do that too. See requirements.txt in tests/
Following two are downloaded:
avocado-framework==88.1
pycdlib==1.11.0Quoted 7 lines
Quoted 3 lines
Also see this line in Makefie.include:
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
Right but that's avocado since it pulls lots of stuff from the net anyway.
Are the libraries in question not packaged on major distros?Currently I only need this:
https://github.com/python-tap/tappy
which is the basic TAP processing library for python.Quoted 8 lines
Quoted 6 lines
It seems its only installed through pip:
https://tappy.readthedocs.io/en/latest/I do not think this is packaged by default. It's such a basic library for parsing test output that maybe we can keep this somewhere within the python src tree? Not sure ...
It's pretty small for sure. Another submodule?
Unlike BITS, this one is likely going to be maintained for a while and will receive new releases through
https://pypi.org/project/tap.py/
so forking is OK but someone has to keep this updated.I am open to anything. Whatever feels right is fine to me.
Quoted 179 lines
Quoted 178 lines
Quoted 178 lines
Quoted 177 lines
Quoted 103 lines
Quoted 103 lines
Quoted 88 lines
Quoted 88 lines
tests/qtest/acpi-bits/README new file +84
-0+ +A brief description of the contents of this directory follows: + +├── acpi-bits-test.py +├── acpi-bits-test-venv.sh +├── bits-config +│ ├── bits-cfg.txt +│ └── meson.build +├── bits-tests +│ ├── meson.build +│ ├── smbios.py +│ ├── smilatency.py +│ ├── testacpi.py +│ └── testcpuid.py +├── meson.build +├── prebuilt +│ ├── bits-2095-grub.tar.gz +│ ├── bits-2095.zip +│ └── meson.build +├── README +└── requirements.txt + +acpi-bits: + - acpi-bits-test-venv.sh: This is a shell script that sets up the virtual + environment necessary for the actual python test script to run. Amongst + other things, it makes sure that QEMU python library is available within + that venv so that QEMU machines can be forked. QEMU python library can be + found within python/ directory in QEMU source. + After setting up the virtual environment, it runs the python test script + from within that environment. + If you want to enable verbose mode only for bits test and run make check, + one trick is to add V=1 before the call to execute the python script in + this file. + - acpi-bits-test.py: This is the main python test script that generates a + biosbits iso. It then spawns a QEMU VM with it, collects the logs and reports + test failures. This is the script one would be interested in if they wanted + to add or change some component of the log parsing, add a new commandline to + how QEMU is spawned etc. Test writers typically would not need to modify + this script unless they wanted to enhance or change the log parsing for + their tests. + - requirements.txt: This text file contains the dependent python libraries + needed for the tests to run. If a new dependent library is needed, it would + be added here as a new entry and then acpi-bits-test-venv.sh would install + it when setting up the python virtual environment using pip. + - README: This text file. + +acpi-bits/bits-config: + This location contains biosbits config files that determine how the software + runs the tests. + - bits-config.txt: this is the biosbits config file that determines what tests + or actions are performed by bits. The description of the config options are + provided in the file itself. + +acpi-bits/prebuilt: + This location contains prebuilt biosbits binaries that are used to generate + the bits iso file for testing. + - bits-2095.zip: The contents from this zip archive are the main contents of + the iso file that are used for testing. This binary zip archive also + contains the full source of the bits software including the full text of + the license agreement listed here: + https://github.com/biosbits/bits/blob/master/COPYING + The source tarball can be found in this location in the zip file: + boot/src/bits-2095.tar.gz + The additional changes beyond those that are present in the official + biosbits github repository can be found here: + https://github.com/ani-sinha/bits/tree/bits-qemu-logging + + Basically these changes can be categorized into the following: + (a) changes to make sure biosbits builds with the latest gcc compiler + (gcc 9.4). + (b) upgrade of acpica to the latest version (march 2022). + (c) send bits logs to the debug IO port at addresss 0x403 so that isa + debugcon can be used to collect the logs. + (d) send a gub halt command to shutdown the VM once all the tests have been + executed. + + This zip file is automatically generated by the bits build system. It can + be found in the bits build directory and it is suffixed by the bits build + number. + Normally, there is no need to make any alteration to this zip archive + unless one wanted to change the bits software itself (for example to add + a new config option or change actions of existing options or change the + debug IO port etc). When such changes are made and a new biosbits software + is needed to be generated, please refresh this zip archive as well as theQuoted 8 lines
tests/qtest/acpi-bits/README line 113
Quoted 6 lines
tests/qtest/acpi-bits/README new file +6
-0+ grub tarball at the same time. Tests will break if changes are made to bits + that are incompatible with existing behavior. So care must be taken to make + sure that the change is compatible with existing bits software as much as + possible. When a new config option is introduced for example, bits must + be upadated here first before introducing the new config option through + a later commit.I don't think playing with source tarballs is a reasonable work environment.
I agree. However, we do not do much with the source tarball. It is there as part of bits to satisfy the license requirement. If we need to manipulate any test script that is in the source file, we would copy it over and keep it in the bits-test directory and change it there.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Let's use submodules just like e.g. firmware does?
Yes I also proposed this to Igor on IRC. We can/maybe need to figure out a place to store the bits source if we think my github is not the best place. However, we need the source only if/when we need to rebuild bits. I suspect it won't be too often if at all.
Why not on git.qemu.org ?
Sounds good to me. Do I get push access? :-)
Quoted 63 lines
Quoted 62 lines
Quoted 62 lines
Quoted 61 lines
tests/qtest/acpi-bits/README new file +55
-1+ - prebuilt/bits-2095-grub.tar.gz: This tarbball contains bits generated grub + scripts and modules to the prebuilt directory. These prebuilt grub + artifacts are required in order to generate a bootable bits iso file that + can run tests. + In order to generate this tar archive, please put the following two + directories that can be found in the bits build directory in a single + tar archive file named as bits-<n>-grub.tar.gz where n is the version of + bits that generated the archive: + + grub-inst-x86_64-efi + grub-inst + + This version should be the same as the version number of bits that generated + bits-<n>.zip file. In other words, the two files must be in sync and should + come from the same build of biosbits of the same version. Things may still + work if they come from different versions but mixing version numbers are + not recommended. + There is normally no need to regenerate this archive unless some fixes or + changes have gone into the grub that are part of biosbits. + +acpi-bits/bits-tests: + This directory contains biosbits python based tests that are run from within + the biosbits environment in the spawned VM. New additions of test cases can + be made in the appropriate test file. For example, new acpi tests can go + into testacpi.py and one would call testsuite.add_test() to register the new + test so that it gets executed as a part of the ACPI tests. + It might be occassionally necessary to disable some subtests or add a new + test that belongs to a test suite not already present in this directory. To + do this, please extract the bits source from the zip file mentioned above. + Copy the test suite/script that needs modification (addition of new tests + or disabling them) from boot/python location of the extracted directory + into this directory. + + step (a): copy unmodified test script to this directory. + step (b): update meson.build and add this file to the list. + Commit (a) and (b) together in the same commit. + + step (c): perform modifications to the test. + Commit (c) in a separate commit. + + The test framework will then use your modified test script to run the test. + No further changes would be needed. Please check the logs to make sure that + appropriate changes have taken effect. + +meson.build files makes sure that the bits qtests are appropriately integrated +into the QEMU qtest framework and are run with "make check-qtest". +Currently, the bits test is configured to run only for x86_64 architectures. Bits +has been built only for this architecture. + + +Author: Ani Sinha <[email protected]> + +References: +(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf +(b) https://www.youtube.com/watch?v=36QIepyUuhg --2.25.1
PATCH 12/12 MAINTAINERS: add myself as the maintainer for acpi biosbits qtests
In reply to Ani SinhaI wrote the biosbits test framework, the qtest for testing QEMU with biosbits and all the related changes. Making myself as the maintainer for biosbits related files and test scripts.
--- MAINTAINERS | 5 +++++ 1 file changed, 5 insertions(+)
MAINTAINERS +5 -0
@@ -1860,6 +1860,11 @@ S: Supported F: hw/acpi/viot.c F: hw/acpi/viot.h +ACPI/QTEST/BIOSBITS +M: Ani Sinha <[email protected]> +S: Supported +F: tests/qtest/acpi-bits/* + ACPI/HEST/GHES R: Dongjiu Geng <[email protected]> L: [email protected]
Signature · 1 line
2.25.1
Quoted 30 lines · Ani Sinha
Biosbits is a software written by Josh Triplett that can be downloaded by visiting https://biosbits.org/. The github codebase can be found here:
https://github.com/biosbits/bits/tree/master. It is a software that exercizes the bios components such as acpi and smbios tables directly through acpica bios interpreter (a freely available C based library written by Intel, downloadable from https://acpica.org/ and is included with biosbits) without an operating system getting involved in between.
There are several advantages to directly testing the bios in a real physical machine or VM as opposed to indirectly discovering bios issues through the operating system. For one thing, the OSes tend to hide bios problems from the end user. The other is that we have more control of what we wanted to test and how by directly using acpica interpreter on top of the bios on a running system. More details on the inspiration for developing biosbits and its real life uses can be found in (a) and (b).
This patchset contains QEMU qtests written in python that exercizes the QEMU bios components using biosbits and reports test failures.Details of each of the files added by this patchset are provided in the README file which is part of Patch 11. Every effort to contact Josh, through various means including email, twitter, linkedIn etc has failed. Hence, the changes to build biosbits with the newer compiler, upgrade acpica and other changes are currently maintained in a forked project in my personal github. We may want to maintain bits in a separate fork in a stable repository that is accessible by QEMU developers.
The newly introduced qtest currently only run for
x86_64platform. They pass both when running make check on a baremetal box as well as from inside a vm.Thanks to Igor M for pointing me to this work.
Quoted 7 lines
(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf
(b) https://www.youtube.com/watch?v=36QIepyUuhgAni Sinha (12):
qtest: meson.build changes required to integrate python based qtests acpi/tests/bits: add prebuilt bios bits zip archive acpi/tests/bits: add prebuilt bits generated grub modules and scripts
These two files didn't arrive on the mailing list, presumaby because pre-built binaries made the patches way too large.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 35 lines · Daniel P. Berrangé
Quoted 33 lines · Ani Sinha
Biosbits is a software written by Josh Triplett that can be downloaded by visiting https://biosbits.org/. The github codebase can be found here:
https://github.com/biosbits/bits/tree/master. It is a software that exercizes the bios components such as acpi and smbios tables directly through acpica bios interpreter (a freely available C based library written by Intel, downloadable from https://acpica.org/ and is included with biosbits) without an operating system getting involved in between.
There are several advantages to directly testing the bios in a real physical machine or VM as opposed to indirectly discovering bios issues through the operating system. For one thing, the OSes tend to hide bios problems from the end user. The other is that we have more control of what we wanted to test and how by directly using acpica interpreter on top of the bios on a running system. More details on the inspiration for developing biosbits and its real life uses can be found in (a) and (b).
This patchset contains QEMU qtests written in python that exercizes the QEMU bios components using biosbits and reports test failures.Details of each of the files added by this patchset are provided in the README file which is part of Patch 11. Every effort to contact Josh, through various means including email, twitter, linkedIn etc has failed. Hence, the changes to build biosbits with the newer compiler, upgrade acpica and other changes are currently maintained in a forked project in my personal github. We may want to maintain bits in a separate fork in a stable repository that is accessible by QEMU developers.
The newly introduced qtest currently only run for
x86_64platform. They pass both when running make check on a baremetal box as well as from inside a vm.Thanks to Igor M for pointing me to this work.
(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf
(b) https://www.youtube.com/watch?v=36QIepyUuhg
Quoted 7 lines
Quoted 4 lines
Ani Sinha (12):
qtest: meson.build changes required to integrate python based qtests acpi/tests/bits: add prebuilt bios bits zip archive acpi/tests/bits: add prebuilt bits generated grub modules and scriptsThese two files didn't arrive on the mailing list, presumaby because pre-built binaries made the patches way too large.
Yes they are over 25 MB and my gmail account does not support attachments more than that size.
Quoted 9 lines
With regards,
Daniel
--|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 41 lines · Ani Sinha
Quoted 40 lines · Daniel P. Berrangé
Quoted 37 lines · Ani Sinha
Biosbits is a software written by Josh Triplett that can be downloaded by visiting https://biosbits.org/. The github codebase can be found here:
https://github.com/biosbits/bits/tree/master. It is a software that exercizes the bios components such as acpi and smbios tables directly through acpica bios interpreter (a freely available C based library written by Intel, downloadable from https://acpica.org/ and is included with biosbits) without an operating system getting involved in between.
There are several advantages to directly testing the bios in a real physical machine or VM as opposed to indirectly discovering bios issues through the operating system. For one thing, the OSes tend to hide bios problems from the end user. The other is that we have more control of what we wanted to test and how by directly using acpica interpreter on top of the bios on a running system. More details on the inspiration for developing biosbits and its real life uses can be found in (a) and (b).
This patchset contains QEMU qtests written in python that exercizes the QEMU bios components using biosbits and reports test failures.Details of each of the files added by this patchset are provided in the README file which is part of Patch 11. Every effort to contact Josh, through various means including email, twitter, linkedIn etc has failed. Hence, the changes to build biosbits with the newer compiler, upgrade acpica and other changes are currently maintained in a forked project in my personal github. We may want to maintain bits in a separate fork in a stable repository that is accessible by QEMU developers.
The newly introduced qtest currently only run for
x86_64platform. They pass both when running make check on a baremetal box as well as from inside a vm.Thanks to Igor M for pointing me to this work.
(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf
(b) https://www.youtube.com/watch?v=36QIepyUuhgAni Sinha (12):
qtest: meson.build changes required to integrate python based qtests acpi/tests/bits: add prebuilt bios bits zip archive acpi/tests/bits: add prebuilt bits generated grub modules and scripts
Quoted 5 lines
Quoted 2 lines
These two files didn't arrive on the mailing list, presumaby because pre-built binaries made the patches way too large.
Yes they are over 25 MB and my gmail account does not support attachments more than that size.
Ok, with that kind of size, it is definitely not something we want to be committing to git either, nor consuming via a submodule since the latter would bloat the release tarballs too.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 2 lines · Daniel P. Berrangé
Ok, with that kind of size, it is definitely not something we want to be committing to git either,
Not to qemu git I think.
Quoted 2 lines
nor consuming via a submodule since the
latter would bloat the release tarballs too.
Hmm - why? We don't have to put the submodule in the tarball if we don't want to. People consuming tarballs probably do not need these tests anyway - just a basic smoketest is all that is needed.
Signature · 1 line
MST
Quoted 4 lines · Michael S. Tsirkin
Quoted 2 lines · Daniel P. Berrangé
Ok, with that kind of size, it is definitely not something we want to be committing to git either,
Quoted 8 lines
Not to qemu git I think.
Quoted 2 lines
nor consuming via a submodule since the
latter would bloat the release tarballs too.Hmm - why? We don't have to put the submodule in the tarball if we don't want to. People consuming tarballs probably do not need these tests anyway - just a basic smoketest is all that is needed.
That feels very dubious. Upstream doesnt test every scenario that users build & run in. Especially with Fedora rawhide we've often found problems which upstream QEMU missed, particularly where new GCC releases have bugs causing subtle mis-compilation of code.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 14 lines · Daniel P. Berrangé
Quoted 4 lines · Michael S. Tsirkin
Quoted 2 lines · Daniel P. Berrangé
Ok, with that kind of size, it is definitely not something we want to be committing to git either,
Quoted 8 lines
Not to qemu git I think.
Quoted 2 lines
nor consuming via a submodule since the
latter would bloat the release tarballs too.Hmm - why? We don't have to put the submodule in the tarball if we don't want to. People consuming tarballs probably do not need these tests anyway - just a basic smoketest is all that is needed.
Quoted 7 lines
That feels very dubious. Upstream doesnt test every scenario that users build & run in. Especially with Fedora rawhide we've often found problems which upstream QEMU missed, particularly where new GCC releases have bugs causing subtle mis-compilation of code.
With regards,
Daniel
IMHO these tests are not really useful for that.
What they do is verify that our ACPI tables are sane - in addition to the manual review with disassembler we do currently.
We already have tests that verify that qemu generates expected ACPI tables and that is enough for what you describe.
A miscompiled qemu will generate acpi tables that differ from expected ones and the simple bit for bit test will fail.
No need to run acpipica within guest.
Quoted 4 lines
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|