Python: Fix 'make check-dev' and modernize to 3.9+
Subsystems (8)
v2 cover Python: Fix 'make check-dev' and modernize to 3.9+
This series does a couple things that I'll probably end up splitting out into smaller series if history is any guide, but either way, here it goes:
A) Convert qemu.git/python/ to a PEP517/pyproject.toml package. Ultimately this means deleting setup.py and fully migrating to newer python infrastructure. I think this should be safe to do by now, but admittedly I am not *confident* as it relies on setuptools versions in the wild, not python versions. My motivation for trying it is to fix "make check-dev", which has been broken for the last two Fedora releases under newer setuptools which have started removing support for the pre-PEP517 packaging formats, which will only continue to get worse from here on out.
B) Sync changes from the qemu.qmp package back over to qemu.git. I know I need to decouple this badly, but in order to do so, I need to make sure they're synchronized to be assured that the switch to the standalone version won't break anything, so this is a necessary step. It's happening here because of the 3.6+ compat crud we are still carrying in qemu.git that has since been removed from the standalone library.
C) Move us to 3.9+ style type hints. They are deprecated in 3.9, and *could* be removed at any time. I figured now was a good time as any to get rid of them before they become a problem randomly some day in the future.
D) Update the mypy configuration to check under multiple Python versions more effectively and thoroughly.
Whew.
v2:
- Perform the 3.9+ syntax conversion using automated tooling instead
- Correct illegal escape sequences (pyupgrade whines otherwise)
- Use the correct shebang for all python scripts in tree
- Remove asterisk imports from scripts/codeconverter
- rebased on origin/master
John Snow (12):
python: convert packages to PEP517/pyproject.toml
python: update pylint ignores
python: sync changes from external qemu.qmp package
python: update shebangs to standard, using /usr/bin/env
python: fix illegal escape sequences
python: upgrade to python3.9+ syntax
fixup
python: further 3.9+ syntax upgrades
python: update mkvenv to type-check under different python versions
python: remove version restriction for mypy scripts/codeconverter: remove unused code scripts/codeconverter: remove * imports
docs/conf.py | 13 +- docs/sphinx/compat.py | 12 +- docs/sphinx/dbusdoc.py | 28 +- docs/sphinx/dbusdomain.py | 39 +-- docs/sphinx/dbusparser.py | 4 +- docs/sphinx/depfile.py | 11 +- docs/sphinx/fakedbusdoc.py | 5 +- docs/sphinx/hxtool.py | 7 +- docs/sphinx/kerneldoc.py | 8 +- docs/sphinx/qapi_domain.py | 81 +++--- docs/sphinx/qapidoc.py | 29 +- docs/sphinx/qapidoc_legacy.py | 1 - docs/sphinx/qmp_lexer.py | 7 +- python/README.rst | 33 ++- .gitlab-ci.d/check-dco.py | 9 +- .gitlab-ci.d/check-patch.py | 7 +- .gitlab-ci.d/check-units.py | 8 +- python/Makefile | 18 +- python/pyproject.toml | 10 + python/qemu/machine/console_socket.py | 6 +- python/qemu/machine/machine.py | 34 +-- python/qemu/machine/qtest.py | 13 +- python/qemu/qmp/error.py | 7 +- python/qemu/qmp/events.py | 72 +++-- python/qemu/qmp/legacy.py | 31 +- python/qemu/qmp/message.py | 38 +-- python/qemu/qmp/models.py | 17 +- python/qemu/qmp/protocol.py | 179 +++++++----- python/qemu/qmp/qmp_client.py | 147 +++++++--- python/qemu/qmp/qmp_shell.py | 182 ++++++++---- python/qemu/qmp/qmp_tui.py | 55 ++-- python/qemu/qmp/util.py | 116 +------- python/qemu/utils/accel.py | 6 +- python/qemu/utils/qemu_ga_client.py | 13 +- python/qemu/utils/qom_common.py | 15 +- python/qemu/utils/qom_fuse.py | 12 +- python/scripts/mkvenv.py | 40 ++- python/setup.cfg | 6 +- python/setup.py | 40 --- python/tests/minreqs.txt | 2 +- python/tests/protocol.py | 11 +- roms/edk2-build.py | 19 +- scripts/analyse-9p-simpletrace.py | 2 + scripts/analyse-locks-simpletrace.py | 5 +- scripts/analyze-migration.py | 48 ++-- scripts/block-coroutine-wrapper.py | 4 +- scripts/check_sparse.py | 9 +- scripts/ci/gitlab-pipeline-status | 4 +- .../codeconverter/codeconverter/patching.py | 76 ++--- .../codeconverter/codeconverter/qom_macros.py | 98 +++++-- .../codeconverter/qom_type_info.py | 86 ++++-- .../codeconverter/codeconverter/regexps.py | 5 +- .../codeconverter/test_patching.py | 6 +- .../codeconverter/test_regexps.py | 33 ++- scripts/codeconverter/codeconverter/utils.py | 10 +- scripts/codeconverter/converter.py | 16 +- scripts/compare-machine-types.py | 49 ++-- scripts/coverage/compare_gcov_json.py | 7 +- scripts/cpu-x86-uarch-abi.py | 6 +- scripts/decodetree.py | 12 +- scripts/device-crash-test | 18 +- scripts/dump-guest-memory.py | 5 +- scripts/feature_to_c.py | 7 +- scripts/kvm/kvm_flightrecorder | 7 +- scripts/kvm/vmxcap | 6 +- scripts/meson-buildoptions.py | 3 +- scripts/minikconf.py | 13 +- scripts/modinfo-collect.py | 7 +- scripts/modinfo-generate.py | 4 +- scripts/modules/module_block.py | 5 +- scripts/mtest2make.py | 8 +- scripts/oss-fuzz/minimize_qtest_trace.py | 18 +- scripts/oss-fuzz/output_reproducer.py | 18 +- .../oss-fuzz/reorder_fuzzer_qtest_trace.py | 6 +- scripts/performance/dissect.py | 2 +- scripts/performance/topN_callgrind.py | 2 +- scripts/performance/topN_perf.py | 2 +- scripts/probe-gdb-support.py | 4 +- scripts/python_qmp_updater.py | 1 + scripts/qapi-gen.py | 1 + scripts/qapi/commands.py | 13 +- scripts/qapi/common.py | 61 ++-- scripts/qapi/error.py | 1 - scripts/qapi/events.py | 8 +- scripts/qapi/expr.py | 21 +- scripts/qapi/features.py | 2 +- scripts/qapi/gen.py | 14 +- scripts/qapi/introspect.py | 42 ++- scripts/qapi/parser.py | 44 ++- scripts/qapi/schema.py | 271 +++++++++--------- scripts/qapi/source.py | 10 +- scripts/qapi/types.py | 20 +- scripts/qapi/visit.py | 14 +- scripts/qcow2-to-stdout.py | 8 +- scripts/qemu-gdb.py | 13 +- scripts/qemu-plugin-symbols.py | 4 +- scripts/qemu-stamp.py | 1 + scripts/qemu-trace-stap | 1 - scripts/qemugdb/aio.py | 1 + scripts/qemugdb/coroutine.py | 1 + scripts/qemugdb/mtree.py | 1 + scripts/qemugdb/tcg.py | 2 +- scripts/qemugdb/timers.py | 2 +- scripts/qmp/qemu-ga-client | 1 + scripts/qmp/qmp | 1 + scripts/qmp/qmp-shell | 1 + scripts/qmp/qmp-shell-wrap | 1 + scripts/qmp/qom-fuse | 1 + scripts/qmp/qom-get | 1 + scripts/qmp/qom-list | 1 + scripts/qmp/qom-set | 1 + scripts/qmp/qom-tree | 1 + scripts/qom-cast-macro-clean-cocci-gen.py | 3 +- scripts/render_block_graph.py | 28 +- scripts/replay-dump.py | 10 +- scripts/rust/rustc_args.py | 10 +- scripts/shaderinclude.py | 4 +- scripts/signrom.py | 3 +- scripts/simplebench/bench-backup.py | 9 +- scripts/simplebench/bench-example.py | 4 +- scripts/simplebench/bench_block_job.py | 9 +- scripts/simplebench/bench_prealloc.py | 10 +- scripts/simplebench/bench_write_req.py | 5 +- scripts/simplebench/img_bench_templater.py | 10 +- scripts/simplebench/results_to_text.py | 4 +- scripts/simplebench/simplebench.py | 2 +- scripts/simplebench/table_templater.py | 4 +- scripts/simpletrace.py | 16 +- scripts/symlink-install-tree.py | 3 +- scripts/tracetool.py | 5 +- scripts/tracetool/__init__.py | 11 +- scripts/tracetool/backend/__init__.py | 2 - scripts/tracetool/backend/dtrace.py | 2 - scripts/tracetool/backend/ftrace.py | 2 - scripts/tracetool/backend/log.py | 2 - scripts/tracetool/backend/simple.py | 2 - scripts/tracetool/backend/syslog.py | 2 - scripts/tracetool/backend/ust.py | 2 - scripts/tracetool/format/__init__.py | 2 - scripts/tracetool/format/c.py | 2 - scripts/tracetool/format/d.py | 5 +- scripts/tracetool/format/h.py | 2 - scripts/tracetool/format/log_stap.py | 3 +- scripts/tracetool/format/simpletrace_stap.py | 3 +- scripts/tracetool/format/stap.py | 2 - scripts/tracetool/format/ust_events_c.py | 2 - scripts/tracetool/format/ust_events_h.py | 2 - scripts/u2f-setup-gen.py | 15 +- scripts/undefsym.py | 3 +- scripts/userfaultfd-wrlat.py | 10 +- scripts/vmstate-static-checker.py | 1 + scripts/xml-preprocess-test.py | 3 +- scripts/xml-preprocess.py | 4 +- target/hexagon/gen_analyze_funcs.py | 3 - target/hexagon/gen_decodetree.py | 9 +- target/hexagon/gen_helper_funcs.py | 3 - target/hexagon/gen_helper_protos.py | 4 +- target/hexagon/gen_idef_parser_funcs.py | 4 - target/hexagon/gen_op_attribs.py | 6 +- target/hexagon/gen_opcodes_def.py | 6 +- target/hexagon/gen_printinsn.py | 5 +- target/hexagon/gen_tcg_func_table.py | 6 +- target/hexagon/gen_tcg_funcs.py | 3 - target/hexagon/gen_trans_funcs.py | 8 +- target/hexagon/hex_common.py | 15 +- tests/docker/docker.py | 37 ++- tests/functional/aspeed.py | 4 +- tests/functional/qemu_test/__init__.py | 30 +- tests/functional/qemu_test/archive.py | 2 +- tests/functional/qemu_test/asset.py | 10 +- tests/functional/qemu_test/decorators.py | 1 + tests/functional/qemu_test/linuxkernel.py | 5 +- tests/functional/qemu_test/ports.py | 3 +- tests/functional/qemu_test/testcase.py | 2 +- tests/functional/qemu_test/tuxruntest.py | 12 +- tests/functional/qemu_test/uncompress.py | 4 +- tests/functional/replay_kernel.py | 7 +- tests/functional/reverse_debugging.py | 5 +- .../functional/test_aarch64_aspeed_ast2700.py | 9 +- .../test_aarch64_aspeed_ast2700fc.py | 9 +- tests/functional/test_aarch64_imx8mp_evk.py | 4 +- tests/functional/test_aarch64_raspi3.py | 2 +- tests/functional/test_aarch64_raspi4.py | 7 +- tests/functional/test_aarch64_replay.py | 4 +- .../functional/test_aarch64_reverse_debug.py | 2 +- tests/functional/test_aarch64_rme_sbsaref.py | 8 +- tests/functional/test_aarch64_rme_virt.py | 11 +- tests/functional/test_aarch64_sbsaref.py | 9 +- .../functional/test_aarch64_sbsaref_alpine.py | 8 +- .../test_aarch64_sbsaref_freebsd.py | 8 +- tests/functional/test_aarch64_smmu.py | 8 +- tests/functional/test_aarch64_tcg_plugins.py | 4 +- tests/functional/test_aarch64_tuxrun.py | 1 + tests/functional/test_aarch64_virt.py | 11 +- tests/functional/test_aarch64_virt_gpu.py | 7 +- tests/functional/test_aarch64_xlnx_versal.py | 3 +- tests/functional/test_acpi_bits.py | 21 +- tests/functional/test_alpha_clipper.py | 2 +- tests/functional/test_arm_aspeed_ast1030.py | 7 +- tests/functional/test_arm_aspeed_ast2500.py | 2 +- tests/functional/test_arm_aspeed_ast2600.py | 11 +- tests/functional/test_arm_aspeed_bletchley.py | 2 +- tests/functional/test_arm_aspeed_palmetto.py | 2 +- tests/functional/test_arm_aspeed_rainier.py | 3 +- tests/functional/test_arm_aspeed_romulus.py | 2 +- .../functional/test_arm_aspeed_witherspoon.py | 2 +- tests/functional/test_arm_bflt.py | 8 +- tests/functional/test_arm_bpim2u.py | 10 +- tests/functional/test_arm_canona1100.py | 3 +- tests/functional/test_arm_collie.py | 2 +- tests/functional/test_arm_cubieboard.py | 10 +- tests/functional/test_arm_emcraft_sf2.py | 7 +- tests/functional/test_arm_integratorcp.py | 12 +- tests/functional/test_arm_microbit.py | 8 +- tests/functional/test_arm_orangepi.py | 11 +- tests/functional/test_arm_quanta_gsj.py | 9 +- tests/functional/test_arm_raspi2.py | 7 +- tests/functional/test_arm_smdkc210.py | 2 +- tests/functional/test_arm_stellaris.py | 8 +- tests/functional/test_arm_sx1.py | 2 +- tests/functional/test_arm_tuxrun.py | 1 + tests/functional/test_arm_vexpress.py | 2 +- tests/functional/test_arm_virt.py | 3 +- tests/functional/test_avr_mega2560.py | 2 +- tests/functional/test_avr_uno.py | 2 +- tests/functional/test_cpu_queries.py | 1 + tests/functional/test_empty_cpu_model.py | 1 + tests/functional/test_hppa_seabios.py | 4 +- tests/functional/test_i386_tuxrun.py | 1 + tests/functional/test_intel_iommu.py | 6 +- tests/functional/test_linux_initrd.py | 2 +- tests/functional/test_loongarch64_virt.py | 10 +- tests/functional/test_m68k_mcf5208evb.py | 2 +- tests/functional/test_m68k_nextcube.py | 8 +- tests/functional/test_m68k_q800.py | 3 +- tests/functional/test_m68k_tuxrun.py | 1 + tests/functional/test_mem_addr_space.py | 4 +- tests/functional/test_memlock.py | 9 +- .../functional/test_microblaze_s3adsp1800.py | 9 +- tests/functional/test_mips64_malta.py | 2 +- tests/functional/test_mips64_tuxrun.py | 1 + tests/functional/test_mips64el_fuloong2e.py | 11 +- tests/functional/test_mips64el_loongson3v.py | 8 +- tests/functional/test_mips64el_malta.py | 17 +- tests/functional/test_mips64el_tuxrun.py | 1 + tests/functional/test_mips_malta.py | 8 +- tests/functional/test_mips_tuxrun.py | 1 + tests/functional/test_mipsel_malta.py | 11 +- tests/functional/test_mipsel_tuxrun.py | 1 + tests/functional/test_multiprocess.py | 10 +- tests/functional/test_netdev_ethtool.py | 5 +- tests/functional/test_or1k_sim.py | 2 +- tests/functional/test_pc_cpu_hotplug_props.py | 1 + tests/functional/test_ppc64_e500.py | 7 +- tests/functional/test_ppc64_hv.py | 16 +- tests/functional/test_ppc64_mac99.py | 8 +- tests/functional/test_ppc64_powernv.py | 4 +- tests/functional/test_ppc64_pseries.py | 4 +- tests/functional/test_ppc64_reverse_debug.py | 2 +- tests/functional/test_ppc64_tuxrun.py | 3 +- tests/functional/test_ppc_40p.py | 10 +- tests/functional/test_ppc_74xx.py | 4 +- tests/functional/test_ppc_amiga.py | 3 +- tests/functional/test_ppc_bamboo.py | 9 +- tests/functional/test_ppc_mac.py | 2 +- tests/functional/test_ppc_mpc8544ds.py | 3 +- tests/functional/test_ppc_sam460ex.py | 7 +- tests/functional/test_ppc_tuxrun.py | 1 + tests/functional/test_ppc_virtex_ml507.py | 3 +- tests/functional/test_riscv32_tuxrun.py | 1 + tests/functional/test_riscv64_tuxrun.py | 1 + tests/functional/test_riscv_opensbi.py | 4 +- tests/functional/test_rx_gdbsim.py | 10 +- tests/functional/test_s390x_ccw_virtio.py | 9 +- tests/functional/test_s390x_topology.py | 11 +- tests/functional/test_s390x_tuxrun.py | 1 + tests/functional/test_sh4_r2d.py | 2 +- tests/functional/test_sh4_tuxrun.py | 1 + tests/functional/test_sh4eb_r2d.py | 7 +- tests/functional/test_sparc64_sun4u.py | 3 +- tests/functional/test_sparc64_tuxrun.py | 1 + tests/functional/test_sparc_sun4m.py | 2 +- tests/functional/test_virtio_balloon.py | 10 +- tests/functional/test_virtio_gpu.py | 14 +- tests/functional/test_virtio_version.py | 1 + tests/functional/test_x86_64_hotplug_blk.py | 6 +- tests/functional/test_x86_64_hotplug_cpu.py | 6 +- tests/functional/test_x86_64_kvm_xen.py | 8 +- tests/functional/test_x86_64_replay.py | 4 +- tests/functional/test_x86_64_reverse_debug.py | 2 +- tests/functional/test_x86_64_tuxrun.py | 1 + .../functional/test_x86_cpu_model_versions.py | 13 +- tests/functional/test_xtensa_lx60.py | 2 +- tests/guest-debug/run-test.py | 9 +- tests/guest-debug/test_gdbstub.py | 15 +- tests/image-fuzzer/qcow2/fuzz.py | 3 +- tests/image-fuzzer/qcow2/layout.py | 32 ++- tests/image-fuzzer/runner.py | 19 +- tests/lcitool/refresh | 6 +- tests/migration-stress/guestperf-batch.py | 1 + tests/migration-stress/guestperf-plot.py | 1 + tests/migration-stress/guestperf.py | 1 + .../migration-stress/guestperf/comparison.py | 3 +- tests/migration-stress/guestperf/engine.py | 8 +- tests/migration-stress/guestperf/hardware.py | 2 +- tests/migration-stress/guestperf/plot.py | 4 +- tests/migration-stress/guestperf/progress.py | 4 +- tests/migration-stress/guestperf/report.py | 9 +- tests/migration-stress/guestperf/scenario.py | 2 +- tests/migration-stress/guestperf/shell.py | 18 +- tests/migration-stress/guestperf/timings.py | 4 +- tests/qapi-schema/test-qapi.py | 2 +- tests/qemu-iotests/030 | 6 +- tests/qemu-iotests/040 | 6 +- tests/qemu-iotests/041 | 7 +- tests/qemu-iotests/044 | 11 +- tests/qemu-iotests/045 | 10 +- tests/qemu-iotests/055 | 4 +- tests/qemu-iotests/056 | 6 +- tests/qemu-iotests/057 | 5 +- tests/qemu-iotests/065 | 10 +- tests/qemu-iotests/093 | 9 +- tests/qemu-iotests/096 | 4 +- tests/qemu-iotests/118 | 4 +- tests/qemu-iotests/124 | 8 +- tests/qemu-iotests/129 | 2 + tests/qemu-iotests/132 | 3 +- tests/qemu-iotests/136 | 4 +- tests/qemu-iotests/139 | 3 +- tests/qemu-iotests/141 | 1 + tests/qemu-iotests/147 | 13 +- tests/qemu-iotests/148 | 2 + tests/qemu-iotests/149 | 7 +- tests/qemu-iotests/151 | 6 +- tests/qemu-iotests/152 | 2 + tests/qemu-iotests/155 | 2 + tests/qemu-iotests/163 | 11 +- tests/qemu-iotests/165 | 3 +- tests/qemu-iotests/194 | 7 +- tests/qemu-iotests/196 | 2 + tests/qemu-iotests/202 | 1 + tests/qemu-iotests/203 | 1 + tests/qemu-iotests/205 | 11 +- tests/qemu-iotests/206 | 1 + tests/qemu-iotests/207 | 6 +- tests/qemu-iotests/208 | 1 + tests/qemu-iotests/209 | 11 +- tests/qemu-iotests/210 | 1 + tests/qemu-iotests/211 | 1 + tests/qemu-iotests/212 | 1 + tests/qemu-iotests/213 | 1 + tests/qemu-iotests/216 | 1 + tests/qemu-iotests/218 | 1 + tests/qemu-iotests/219 | 1 + tests/qemu-iotests/224 | 13 +- tests/qemu-iotests/228 | 11 +- tests/qemu-iotests/234 | 4 +- tests/qemu-iotests/235 | 14 +- tests/qemu-iotests/236 | 1 + tests/qemu-iotests/237 | 2 + tests/qemu-iotests/238 | 3 +- tests/qemu-iotests/240 | 2 +- tests/qemu-iotests/242 | 23 +- tests/qemu-iotests/245 | 1 + tests/qemu-iotests/246 | 1 + tests/qemu-iotests/248 | 12 +- tests/qemu-iotests/254 | 3 +- tests/qemu-iotests/255 | 1 + tests/qemu-iotests/256 | 4 +- tests/qemu-iotests/257 | 28 +- tests/qemu-iotests/258 | 10 +- tests/qemu-iotests/260 | 8 +- tests/qemu-iotests/262 | 4 +- tests/qemu-iotests/264 | 7 +- tests/qemu-iotests/274 | 1 + tests/qemu-iotests/277 | 10 +- tests/qemu-iotests/280 | 2 +- tests/qemu-iotests/281 | 4 +- tests/qemu-iotests/283 | 1 + tests/qemu-iotests/295 | 6 +- tests/qemu-iotests/296 | 7 +- tests/qemu-iotests/297 | 8 +- tests/qemu-iotests/298 | 2 + tests/qemu-iotests/299 | 1 + tests/qemu-iotests/300 | 10 +- tests/qemu-iotests/302 | 4 +- tests/qemu-iotests/303 | 13 +- tests/qemu-iotests/304 | 3 +- tests/qemu-iotests/307 | 2 +- tests/qemu-iotests/310 | 1 + tests/qemu-iotests/check | 7 +- tests/qemu-iotests/fat16.py | 12 +- tests/qemu-iotests/findtests.py | 19 +- tests/qemu-iotests/iotests.py | 70 +++-- tests/qemu-iotests/linters.py | 9 +- tests/qemu-iotests/nbd-fault-injector.py | 13 +- tests/qemu-iotests/qcow2.py | 5 +- tests/qemu-iotests/qcow2_format.py | 14 +- tests/qemu-iotests/qed.py | 10 +- tests/qemu-iotests/testenv.py | 25 +- tests/qemu-iotests/testrunner.py | 32 +-- tests/qemu-iotests/tests/block-status-cache | 1 + .../tests/export-incoming-iothread | 1 + .../qemu-iotests/tests/graph-changes-while-io | 11 +- tests/qemu-iotests/tests/image-fleecing | 1 + tests/qemu-iotests/tests/inactive-node-nbd | 8 +- .../tests/iothreads-commit-active | 2 + tests/qemu-iotests/tests/iothreads-create | 2 + tests/qemu-iotests/tests/iothreads-nbd-export | 3 + tests/qemu-iotests/tests/iothreads-stream | 2 + tests/qemu-iotests/tests/luks-detached-header | 5 +- .../tests/migrate-bitmaps-postcopy-test | 10 +- tests/qemu-iotests/tests/migrate-bitmaps-test | 2 +- .../qemu-iotests/tests/migrate-during-backup | 1 + .../tests/mirror-change-copy-mode | 3 +- .../tests/mirror-ready-cancel-error | 1 + tests/qemu-iotests/tests/nbd-multiconn | 2 +- .../qemu-iotests/tests/nbd-reconnect-on-open | 11 +- .../qemu-iotests/tests/parallels-read-bitmap | 8 +- tests/qemu-iotests/tests/qsd-migrate | 2 +- .../tests/remove-bitmap-from-backing | 8 +- tests/qemu-iotests/tests/reopen-file | 3 +- .../qemu-iotests/tests/stream-error-on-reset | 8 +- .../tests/stream-unaligned-prefetch | 13 +- .../qemu-iotests/tests/stream-under-throttle | 4 +- tests/qemu-iotests/tests/vvfat | 7 +- tests/tcg/aarch64/gdbstub/test-mte.py | 2 +- tests/tcg/aarch64/gdbstub/test-sve-ioctl.py | 4 +- tests/tcg/aarch64/gdbstub/test-sve.py | 2 +- tests/tcg/i386/test-avx.py | 11 +- tests/tcg/i386/test-mmx.py | 9 +- tests/tcg/multiarch/gdbstub/catch-syscalls.py | 6 +- .../gdbstub/follow-fork-mode-child.py | 4 +- .../gdbstub/follow-fork-mode-parent.py | 2 +- tests/tcg/multiarch/gdbstub/interrupt.py | 1 - tests/tcg/multiarch/gdbstub/late-attach.py | 4 +- tests/tcg/multiarch/gdbstub/memory.py | 2 - tests/tcg/multiarch/gdbstub/prot-none.py | 5 +- tests/tcg/multiarch/gdbstub/registers.py | 3 +- tests/tcg/multiarch/gdbstub/sha1.py | 1 - .../multiarch/gdbstub/test-proc-mappings.py | 1 - .../multiarch/gdbstub/test-qxfer-auxv-read.py | 1 - .../gdbstub/test-qxfer-siginfo-read.py | 2 +- .../gdbstub/test-thread-breakpoint.py | 1 - .../system/validate-memory-counts.py | 7 +- tests/tcg/s390x/gdbstub/test-signals-s390x.py | 2 - tests/tcg/s390x/gdbstub/test-svc.py | 1 - tests/vm/aarch64vm.py | 16 +- tests/vm/basevm.py | 37 +-- tests/vm/centos.aarch64 | 11 +- tests/vm/freebsd | 7 +- tests/vm/haiku.x86_64 | 7 +- tests/vm/netbsd | 5 +- tests/vm/openbsd | 5 +- tests/vm/ubuntu.aarch64 | 6 +- tests/vm/ubuntuvm.py | 6 +- 456 files changed, 2624 insertions(+), 2058 deletions(-) create mode 100644 python/pyproject.toml delete mode 100755 python/setup.py
Signature · 1 line
2.48.1
PATCH v2 1/12 python: convert packages to PEP517/pyproject.toml
Newer versions of setuptools increasingly expect that packages are defined using the pyproject.toml/PEP517 packaging layout format. With 3.9 as our minimum, I believe it's finally appropriate to make the shift away from the legacy packaging format.
Update documentation and dependencies that change as a result of the different build/packaging/installation pathways.
This change has the effect of fixing "make check-dev", which has been broken on newer versions of Fedora for a while, now.
--- python/README.rst | 33 ++++++++++++++++----------------- python/Makefile | 18 +++++++++--------- python/pyproject.toml | 10 ++++++++++ python/setup.py | 40 ---------------------------------------- python/tests/minreqs.txt | 2 +- 5 files changed, 36 insertions(+), 67 deletions(-) create mode 100644 python/pyproject.toml delete mode 100755 python/setup.py
python/README.rst +16 -17
@@ -5,24 +5,23 @@ This directory houses Python tooling used by the QEMU project to build, configure, and test QEMU. It is organized by namespace (``qemu``), and then by package (e.g. ``qemu/machine``, ``qemu/qmp``, etc). -``setup.py`` is used by ``pip`` to install this tooling to the current -environment. ``setup.cfg`` provides the packaging configuration used by -``setup.py``. You will generally invoke it by doing one of the following: +``pyproject.toml`` and ``setup.cfg`` are used by ``pip`` to install this +tooling to the current environment. ``setup.cfg`` provides the packaging +configuration, while ``pyproject.toml`` describes the package build +system requirements. -1. ``pip3 install .`` will install these packages to your current - environment. If you are inside a virtual environment, they will - install there. If you are not, it will attempt to install to the - global environment, which is **not recommended**. +You will generally install these packages by invoking ``pip3 install +.``; which will install these packages to your current environment. If +you are inside a virtual environment, they will install there. If you +are not, modern versions of pip will attempt instead to install to your +local user environment. Older versions of pip will attempt to install to +the global environment, which is **not recommended**. -2. ``pip3 install --user .`` will install these packages to your user's - local python packages. If you are inside of a virtual environment, - this will fail; you want the first invocation above. - -If you append the ``--editable`` or ``-e`` argument to either invocation -above, pip will install in "editable" mode. This installs the package as -a forwarder ("qemu.egg-link") that points to the source tree. In so -doing, the installed package always reflects the latest version in your -source tree. +If you append the ``--editable`` or ``-e`` argument to the above +invocation, pip will install in "editable" mode. This installs the +package as a "forwarder" that points to the source tree, so that the +installed package always reflects the latest version in your source +tree. Installing ".[devel]" instead of "." will additionally pull in required packages for testing this package. They are not runtime requirements,
@@ -81,4 +80,4 @@ Files in this directory - ``VERSION`` contains the PEP-440 compliant version used to describe this package; it is referenced by ``setup.cfg``. - ``setup.cfg`` houses setuptools package configuration. -- ``setup.py`` is the setuptools installer used by pip; See above. +- ``pyproject.toml`` lists build system requirements for the Python packages.
python/Makefile +9 -9
@@ -15,8 +15,8 @@ help: @echo "make check-tox:" @echo " Run tests against multiple python versions." @echo " These tests use the newest dependencies." - @echo " Requires: Python 3.9 - 3.11, and tox." - @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.11'" + @echo " Requires: Python 3.9 - 3.13, and tox." + @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.13'" @echo " The variable QEMU_TOX_EXTRA_ARGS can be use to pass extra" @echo " arguments to tox". @echo ""
@@ -63,12 +63,12 @@ $(QEMU_MINVENV_DIR) $(QEMU_MINVENV_DIR)/bin/activate: setup.cfg tests/minreqs.tx @( \ echo "ACTIVATE $(QEMU_MINVENV_DIR)"; \ . $(QEMU_MINVENV_DIR)/bin/activate; \ - echo "INSTALL wheel $(QEMU_MINVENV_DIR)"; \ - $(PIP_INSTALL) wheel 1>/dev/null; \ + echo "INSTALL wheel $(QEMU_MINVENV_DIR)"; \ + $(PIP_INSTALL) wheel 1>/dev/null; \ echo "INSTALL -r tests/minreqs.txt $(QEMU_MINVENV_DIR)";\ $(PIP_INSTALL) -r tests/minreqs.txt 1>/dev/null; \ echo "INSTALL -e qemu $(QEMU_MINVENV_DIR)"; \ - $(PIP_INSTALL) -e . 1>/dev/null; \ + $(PIP_INSTALL) -e . --config-settings=editable_mode=compat 1>/dev/null; \ ) @touch $(QEMU_MINVENV_DIR)
@@ -103,7 +103,7 @@ check-dev: dev-venv .PHONY: develop develop: - $(PIP_INSTALL) -e .[devel] + $(PIP_INSTALL) -e .[devel] --config-settings=editable_mode=compat .PHONY: check check:
@@ -122,12 +122,12 @@ check-coverage: .PHONY: clean clean: - python3 setup.py clean --all - rm -f pyproject.toml + rm -rf build/ .PHONY: distclean distclean: clean - rm -rf qemu.egg-info/ .eggs/ dist/ + rm -rf qemu.egg-info/ rm -rf $(QEMU_VENV_DIR) $(QEMU_MINVENV_DIR) .tox/ + rm -rf .mypy_cache/ rm -f .coverage .coverage.* rm -rf htmlcov/
python/pyproject.toml new file +10 -0
@@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0-or-later + +[build-system] +requires = [ + "setuptools>=39.2", + "wheel", +] +build-backend = "setuptools.build_meta" + +[tool.setuptools_scm]
python/setup.py deleted +0 -40
@@ -1,40 +0,0 @@ -#!/usr/bin/env python3 -""" -QEMU tooling installer script -Copyright (c) 2020-2021 John Snow for Red Hat, Inc. -""" - -import setuptools -from setuptools.command import bdist_egg -import sys -import pkg_resources - - -class bdist_egg_guard(bdist_egg.bdist_egg): - """ - Protect against bdist_egg from being executed - - This prevents calling 'setup.py install' directly, as the 'install' - CLI option will invoke the deprecated bdist_egg hook. "pip install" - calls the more modern bdist_wheel hook, which is what we want. - """ - def run(self): - sys.exit( - 'Installation directly via setup.py is not supported.\n' - 'Please use `pip install .` instead.' - ) - - -def main(): - """ - QEMU tooling installer - """ - - # https://medium.com/@daveshawley/safely-using-setup-cfg-for-metadata-1babbe54c108 - pkg_resources.require('setuptools>=39.2') - - setuptools.setup(cmdclass={'bdist_egg': bdist_egg_guard}) - - -if __name__ == '__main__': - main()
python/tests/minreqs.txt +1 -1
@@ -15,6 +15,7 @@ # also pin setuptools to version 70 or below. Otherwise, the # installation of the QEMU package itself fails, failing to find # setuptools. +# Avocado also appears to require an older setuptools. setuptools<=70 # Dependencies for qapidoc/qapi_domain et al
@@ -55,7 +56,6 @@ astroid==2.15.4 dill==0.2 lazy-object-proxy==1.4.0 platformdirs==2.2.0 -toml==0.10.0 tomlkit==0.10.1 wrapt==1.14.0
Signature · 1 line
2.48.1
Re: [PATCH v2 01/12] python: convert packages to PEP517/pyproject.toml
Quoted 19 lines · John Snow
Newer versions of setuptools increasingly expect that packages are defined using the pyproject.toml/PEP517 packaging layout format. With 3.9 as our minimum, I believe it's finally appropriate to make the shift away from the legacy packaging format.
Update documentation and dependencies that change as a result of the different build/packaging/installation pathways.
This change has the effect of fixing "make check-dev", which has been broken on newer versions of Fedora for a while, now.
Signed-off-by John Snow <[email protected]>--- python/README.rst | 33 ++++++++++++++++----------------- python/Makefile | 18 +++++++++--------- python/pyproject.toml | 10 ++++++++++ python/setup.py | 40 ---------------------------------------- python/tests/minreqs.txt | 2 +- 5 files changed, 36 insertions(+), 67 deletions(-)
Quoted 2 lines
create mode 100644
python/pyproject.toml
delete mode 100755python/setup.py
Re: [PATCH v2 01/12] python: convert packages to PEP517/pyproject.toml
Quoted 23 lines · Thomas Huth
Quoted 19 lines · John Snow
Newer versions of setuptools increasingly expect that packages are defined using the pyproject.toml/PEP517 packaging layout format. With 3.9 as our minimum, I believe it's finally appropriate to make the shift away from the legacy packaging format.
Update documentation and dependencies that change as a result of the different build/packaging/installation pathways.
This change has the effect of fixing "make check-dev", which has been broken on newer versions of Fedora for a while, now.
Signed-off-by John Snow <[email protected]>--- python/README.rst | 33 ++++++++++++++++----------------- python/Makefile | 18 +++++++++--------- python/pyproject.toml | 10 ++++++++++ python/setup.py | 40 ---------------------------------------- python/tests/minreqs.txt | 2 +- 5 files changed, 36 insertions(+), 67 deletions(-)Quoted 2 lines
create mode 100644
python/pyproject.toml
delete mode 100755python/setup.py
Quoted 1 line
Reviewed-by Thomas Huth <[email protected]>
Thanks. If there are no objections, I might stage at least the first few reviewed patches here so I can get this stitched up sooner than later. The more disruptive parts I might wait a touch longer on.
PATCH v2 2/12 python: update pylint ignores
In reply to John SnowThe next patch will synchronize the qemu.qmp library with the external, standalone version. That synchronization will require an extra ignore for pylint, so do that now.
--- python/setup.cfg | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
python/setup.cfg +3 -2
@@ -140,11 +140,12 @@ ignore_missing_imports = True # --disable=W". disable=consider-using-f-string, consider-using-with, + no-member, # mypy also handles this better. too-many-arguments, - too-many-positional-arguments, too-many-function-args, # mypy handles this with less false positives. too-many-instance-attributes, - no-member, # mypy also handles this better. + too-many-positional-arguments, + unknown-option-value, [pylint.basic] # Good variable names which should always be accepted, separated by a comma.
Signature · 1 line
2.48.1
Re: [PATCH v2 02/12] python: update pylint ignores
Quoted 8 lines · John Snow
The next patch will synchronize the qemu.qmp library with the external, standalone version. That synchronization will require an extra ignore for pylint, so do that now.
Signed-off-by John Snow <[email protected]>--- python/setup.cfg | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
Re: [PATCH v2 02/12] python: update pylint ignores
In reply to John SnowQuoted 5 lines · John Snow
The next patch will synchronize the qemu.qmp library with the external, standalone version. That synchronization will require an extra ignore for pylint, so do that now.
Signed-off-by John Snow <[email protected]>
PATCH v2 3/12 python: sync changes from external qemu.qmp package
In reply to John SnowSynchronize changes from the standalone python-qemu-qmp repository for the primary purpose of removing some pre-3.9 compatibility crud that we no longer need or want.
My plan for this release cycle is to finally eliminate this duplication so this type of patch will no longer be necessary -- but for now, it is.
NB: A few changes are not synchronized; mostly license and documentation strings. Everything functional is fully copied verbatim. The license strings are not sync'd only because they point to different LICENSE filenames for the different repositories.
--- python/qemu/qmp/error.py | 7 +- python/qemu/qmp/events.py | 50 ++++++++-- python/qemu/qmp/legacy.py | 16 +++- python/qemu/qmp/message.py | 22 +++-- python/qemu/qmp/models.py | 6 +- python/qemu/qmp/protocol.py | 169 +++++++++++++++++++++------------- python/qemu/qmp/qmp_client.py | 133 +++++++++++++++++++------- python/qemu/qmp/qmp_shell.py | 167 +++++++++++++++++++++++---------- python/qemu/qmp/qmp_tui.py | 34 +++++-- python/qemu/qmp/util.py | 116 ++--------------------- python/tests/protocol.py | 10 +- 11 files changed, 443 insertions(+), 287 deletions(-)
python/qemu/qmp/error.py +5 -2
@@ -44,7 +44,10 @@ class ProtocolError(QMPError): :param error_message: Human-readable string describing the error. """ - def __init__(self, error_message: str): - super().__init__(error_message) + def __init__(self, error_message: str, *args: object): + super().__init__(error_message, *args) #: Human-readable error message, without any prefix. self.error_message: str = error_message + + def __str__(self) -> str: + return self.error_message
python/qemu/qmp/events.py +42 -8
104 lines not on this page · raw message
python/qemu/qmp/legacy.py +13 -3
@@ -86,7 +86,14 @@ def __init__(self, "server argument should be False when passing a socket") self._qmp = QMPClient(nickname) - self._aloop = asyncio.get_event_loop() + + try: + self._aloop = asyncio.get_running_loop() + except RuntimeError: + # No running loop; since this is a sync shim likely to be + # used in fully sync programs, create one if neccessary. + self._aloop = asyncio.get_event_loop_policy().get_event_loop() + self._address = address self._timeout: Optional[float] = None
@@ -231,6 +238,9 @@ def pull_event(self, :return: The first available QMP event, or None. """ + # Kick the event loop to allow events to accumulate + self._sync(asyncio.sleep(0)) + if not wait: # wait is False/0: "do not wait, do not except." if self._qmp.events.empty():
@@ -286,8 +296,8 @@ def settimeout(self, timeout: Optional[float]) -> None: """ Set the timeout for QMP RPC execution. - This timeout affects the `cmd`, `cmd_obj`, and `command` methods. - The `accept`, `pull_event` and `get_event` methods have their + This timeout affects the `cmd`, `cmd_obj`, and `cmd_raw` methods. + The `accept`, `pull_event` and `get_events` methods have their own configurable timeouts. :param timeout:
python/qemu/qmp/message.py +15 -7
@@ -28,7 +28,8 @@ class Message(MutableMapping[str, object]): be instantiated from either another mapping (like a `dict`), or from raw `bytes` that still need to be deserialized. - Once instantiated, it may be treated like any other MutableMapping:: + Once instantiated, it may be treated like any other + :py:obj:`~collections.abc.MutableMapping`:: >>> msg = Message(b'{"hello": "world"}') >>> assert msg['hello'] == 'world'
@@ -50,12 +51,19 @@ class Message(MutableMapping[str, object]): >>> dict(msg) {'hello': 'world'} + Or pretty-printed:: + + >>> print(str(msg)) + { + "hello": "world" + } :param value: Initial value, if any. :param eager: When `True`, attempt to serialize or deserialize the initial value immediately, so that conversion exceptions are raised during the call to ``__init__()``. + """ # pylint: disable=too-many-ancestors
@@ -178,15 +186,15 @@ class DeserializationError(ProtocolError): :param raw: The raw `bytes` that prompted the failure. """ def __init__(self, error_message: str, raw: bytes): - super().__init__(error_message) + super().__init__(error_message, raw) #: The raw `bytes` that were not understood as JSON. self.raw: bytes = raw def __str__(self) -> str: - return "\n".join([ + return "\n".join(( super().__str__(), f" raw bytes were: {str(self.raw)}", - ]) + )) class UnexpectedTypeError(ProtocolError):
@@ -197,13 +205,13 @@ class UnexpectedTypeError(ProtocolError): :param value: The deserialized JSON value that wasn't an object. """ def __init__(self, error_message: str, value: object): - super().__init__(error_message) + super().__init__(error_message, value) #: The JSON value that was expected to be an object. self.value: object = value def __str__(self) -> str: strval = json.dumps(self.value, indent=2) - return "\n".join([ + return "\n".join(( super().__str__(), f" json value was: {strval}", - ]) + ))
python/qemu/qmp/models.py +5 -1
@@ -54,7 +54,11 @@ def __repr__(self) -> str: class Greeting(Model): """ - Defined in qmp-spec.rst, section "Server Greeting". + Defined in qmp-spec.rst, "Server Greeting". + + See `2.2 Server Greeting + <https://gitlab.com/qemu-project/qemu/-/blob/master/docs/interop/qmp-spec.txt#L61>`_ + for details. :param raw: The raw Greeting object. :raise KeyError: If any required fields are absent.
python/qemu/qmp/protocol.py +103 -66
336 lines not on this page · raw message
python/qemu/qmp/qmp_client.py +100 -33
247 lines not on this page · raw message
python/qemu/qmp/qmp_shell.py +120 -47
248 lines not on this page · raw message
python/qemu/qmp/qmp_tui.py +25 -9
@@ -21,6 +21,7 @@ import logging from logging import Handler, LogRecord import signal +import sys from typing import ( List, Optional,
@@ -30,17 +31,27 @@ cast, ) -from pygments import lexers -from pygments import token as Token -import urwid -import urwid_readline + +try: + from pygments import lexers + from pygments import token as Token + import urwid + import urwid_readline +except ModuleNotFoundError as exc: + print( + f"Module '{exc.name}' not found.", + "You need the optional 'tui' group: pip install qemu.qmp[tui]", + sep='\n', + file=sys.stderr, + ) + sys.exit(1) from .error import ProtocolError from .legacy import QEMUMonitorProtocol, QMPBadPortError from .message import DeserializationError, Message, UnexpectedTypeError from .protocol import ConnectError, Runstate from .qmp_client import ExecInterruptedError, QMPClient -from .util import create_task, pretty_traceback +from .util import pretty_traceback # The name of the signal that is used to update the history list
@@ -225,7 +236,7 @@ def cb_send_to_server(self, raw_msg: str) -> None: """ try: msg = Message(bytes(raw_msg, encoding='utf-8')) - create_task(self._send_to_server(msg)) + asyncio.create_task(self._send_to_server(msg)) except (DeserializationError, UnexpectedTypeError) as err: raw_msg = format_json(raw_msg) logging.info('Invalid message: %s', err.error_message)
@@ -246,7 +257,7 @@ def kill_app(self) -> None: Initiates killing of app. A bridge between asynchronous and synchronous code. """ - create_task(self._kill_app()) + asyncio.create_task(self._kill_app()) async def _kill_app(self) -> None: """
@@ -377,7 +388,12 @@ def run(self, debug: bool = False) -> None: screen = urwid.raw_display.Screen() screen.set_terminal_properties(256) - self.aloop = asyncio.get_event_loop() + try: + self.aloop = asyncio.get_running_loop() + except RuntimeError: + # No running asyncio event loop. Create one if necessary. + self.aloop = asyncio.get_event_loop_policy().get_event_loop() + self.aloop.set_debug(debug) # Gracefully handle SIGTERM and SIGINT signals
@@ -393,7 +409,7 @@ def run(self, debug: bool = False) -> None: handle_mouse=True, event_loop=event_loop) - create_task(self.manage_connection(), self.aloop) + self.aloop.create_task(self.manage_connection()) try: main_loop.run() except Exception as err:
python/qemu/qmp/util.py +10 -106
152 lines not on this page · raw message
python/tests/protocol.py +5 -5
@@ -8,7 +8,6 @@ from qemu.qmp import ConnectError, Runstate from qemu.qmp.protocol import AsyncProtocol, StateError -from qemu.qmp.util import asyncio_run, create_task class NullProtocol(AsyncProtocol[None]):
@@ -124,7 +123,7 @@ async def _runner(): if allow_cancellation: return raise - return create_task(_runner()) + return asyncio.create_task(_runner()) @contextmanager
@@ -228,7 +227,7 @@ def async_test(async_test_method): Decorator; adds SetUp and TearDown to async tests. """ async def _wrapper(self, *args, **kwargs): - loop = asyncio.get_event_loop() + loop = asyncio.get_running_loop() loop.set_debug(True) await self._asyncSetUp()
@@ -271,7 +270,7 @@ async def _watcher(): msg=f"Expected state '{state.name}'", ) - self.runstate_watcher = create_task(_watcher()) + self.runstate_watcher = asyncio.create_task(_watcher()) # Kick the loop and force the task to block on the event. await asyncio.sleep(0)
@@ -589,7 +588,8 @@ async def _asyncTearDown(self): async def testSmoke(self): with TemporaryDirectory(suffix='.qmp') as tmpdir: sock = os.path.join(tmpdir, type(self.proto).__name__ + ".sock") - server_task = create_task(self.server.start_server_and_accept(sock)) + server_task = asyncio.create_task( + self.server.start_server_and_accept(sock)) # give the server a chance to start listening [...] await asyncio.sleep(0)
Signature · 1 line
2.48.1
Re: [PATCH v2 03/12] python: sync changes from external qemu.qmp package
Quoted 7 lines · John Snow
Synchronize changes from the standalone python-qemu-qmp repository for the primary purpose of removing some pre-3.9 compatibility crud that we no longer need or want.
My plan for this release cycle is to finally eliminate this duplication so this type of patch will no longer be necessary -- but for now, it is.
Quoted 6 lines
NB: A few changes are not synchronized; mostly license and documentation strings. Everything functional is fully copied verbatim. The license strings are not sync'd only because they point to different LICENSE filenames for the different repositories.
Signed-off-by John Snow <[email protected]>
PATCH v2 4/12 python: update shebangs to standard, using /usr/bin/env
In reply to John SnowThis is the standard shebang we should always be using, as it plays nicely with virtual environments and our desire to always be using a specific python interpreter in our environments.
(It also makes sure I can find all of the python scripts in our tree easily.)
--- roms/edk2-build.py | 2 +- scripts/cpu-x86-uarch-abi.py | 2 +- scripts/userfaultfd-wrlat.py | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
roms/edk2-build.py +1 -1
@@ -1,4 +1,4 @@ -#!/usr/bin/python3 +#!/usr/bin/env python3 """ build helper script for edk2, see https://gitlab.com/kraxel/edk2-build-config
scripts/cpu-x86-uarch-abi.py +1 -1
@@ -1,4 +1,4 @@ -#!/usr/bin/python3 +#!/usr/bin/env python3 # # SPDX-License-Identifier: GPL-2.0-or-later #
scripts/userfaultfd-wrlat.py +1 -1
@@ -1,4 +1,4 @@ -#!/usr/bin/python3 +#!/usr/bin/env python3 # # userfaultfd-wrlat Summarize userfaultfd write fault latencies. # Events are continuously accumulated for the
Signature · 1 line
2.48.1
Re: [PATCH v2 04/12] python: update shebangs to standard, using /usr/bin/env
Quoted 7 lines · John Snow
This is the standard shebang we should always be using, as it plays nicely with virtual environments and our desire to always be using a specific python interpreter in our environments.
(It also makes sure I can find all of the python scripts in our tree easily.)
Quoted 6 lines
Signed-off-by John Snow <[email protected]>--- roms/edk2-build.py | 2 +- scripts/cpu-x86-uarch-abi.py | 2 +- scripts/userfaultfd-wrlat.py | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
Re: [PATCH v2 04/12] python: update shebangs to standard, using /usr/bin/env
In reply to John SnowQuoted 8 lines · John Snow
This is the standard shebang we should always be using, as it plays nicely with virtual environments and our desire to always be using a specific python interpreter in our environments.
(It also makes sure I can find all of the python scripts in our tree easily.)
Signed-off-by John Snow <[email protected]>
PATCH v2 5/12 python: fix illegal escape sequences
In reply to John Snow\{ is an illegal escape sequence, to get a literal backslash we need to use \\.
This is being fixed because of an issue spotted when attempting to use the pyupgrade script.
--- scripts/feature_to_c.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/feature_to_c.py +1 -1
@@ -25,7 +25,7 @@ def writeliteral(indent, bytes): elif c >= 32 and c < 127: sys.stdout.write(c.to_bytes(1, 'big').decode()) else: - sys.stdout.write(f'\{c:03o}') + sys.stdout.write(f'\\{c:03o}') if quoted: sys.stdout.write('"')
Signature · 1 line
2.48.1
Re: [PATCH v2 05/12] python: fix illegal escape sequences
Quoted 16 lines · John Snow
\{ is an illegal escape sequence, to get a literal backslash we need to use \\.
This is being fixed because of an issue spotted when attempting to use the pyupgrade script.
Signed-off-by John Snow <[email protected]>--- scripts/feature_to_c.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)scripts/feature_to_c.py +0
-0
scripts/feature_to_c.py def writeliteral(indent, bytes): · line 31
Quoted 8 lines
scripts/feature_to_c.py +1
-1@@ -25,7 +25,7 @@ def writeliteral(indent, bytes): elif c >= 32 and c < 127: sys.stdout.write(c.to_bytes(1, 'big').decode()) else: - sys.stdout.write(f'\{c:03o}') + sys.stdout.write(f'\\{c:03o}') if quoted: sys.stdout.write('"')
Re: [PATCH v2 05/12] python: fix illegal escape sequences
In reply to John SnowQuoted 7 lines · John Snow
\{ is an illegal escape sequence, to get a literal backslash we need to use \\.
This is being fixed because of an issue spotted when attempting to use the pyupgrade script.
Signed-off-by John Snow <[email protected]>
PATCH v2 6/12 python: upgrade to python3.9+ syntax
In reply to John SnowThis patch is fully automated, using pymagic, isort and autoflake.
Create a script named pymagic.sh:
=========================
pyupgrade --exit-zero-even-if-changed --keep-percent-format \
--py39-plus "$@"autoflake -i "$@"
isort --settings-file python/setup.cfg \
-p compat -p qapidoc_legacy -p iotests -o qemu "$@"
=========================
Then, from qemu.git root:
Quoted 2 lines
find . -type f -name '*.py' | xargs pymagic
git grep --name-only "#!/usr/bin/env python" | xargs pymagic
This changes a lot of old Pythonisms, but in particular it upgrades the old Python type hint paradigm to the new 3.9+ paradigm wherein you no longer need to import List, Dict, Tuple, Set, etc from the Typing module and instead directly subscript the built-in types list, dict, tuple, set, etc. The old-style annotations are deprecated as of 3.9 and are eligible for removal starting in Python 3.14, though the exact date of their removal is not yet known.
pyupgrade updates the imports and type hint paradigms (as well as updating other old 'isms, such as removing the unicode string prefix). autoflake in turn then removes any unused import statements, possibly left behind by pyupgrade. Lastly, isort fixes the import order and formatting to the standard we use in qemu.git/python and scripts/qapi in particular.
--- docs/conf.py | 13 +- docs/sphinx/compat.py | 4 +- docs/sphinx/dbusdoc.py | 28 +- docs/sphinx/dbusdomain.py | 39 +-- docs/sphinx/dbusparser.py | 4 +- docs/sphinx/depfile.py | 11 +- docs/sphinx/fakedbusdoc.py | 5 +- docs/sphinx/hxtool.py | 7 +- docs/sphinx/kerneldoc.py | 8 +- docs/sphinx/qapi_domain.py | 81 +++--- docs/sphinx/qapidoc.py | 29 +- docs/sphinx/qapidoc_legacy.py | 1 - docs/sphinx/qmp_lexer.py | 7 +- .gitlab-ci.d/check-dco.py | 9 +- .gitlab-ci.d/check-patch.py | 7 +- .gitlab-ci.d/check-units.py | 8 +- python/qemu/machine/console_socket.py | 2 +- python/qemu/machine/machine.py | 34 +-- python/qemu/machine/qtest.py | 13 +- python/qemu/qmp/events.py | 25 +- python/qemu/qmp/legacy.py | 15 +- python/qemu/qmp/message.py | 16 +- python/qemu/qmp/models.py | 11 +- python/qemu/qmp/protocol.py | 11 +- python/qemu/qmp/qmp_client.py | 14 +- python/qemu/qmp/qmp_shell.py | 15 +- python/qemu/qmp/qmp_tui.py | 21 +- python/qemu/utils/accel.py | 6 +- python/qemu/utils/qemu_ga_client.py | 13 +- python/qemu/utils/qom_common.py | 15 +- python/qemu/utils/qom_fuse.py | 13 +- python/scripts/mkvenv.py | 25 +- python/tests/protocol.py | 1 - roms/edk2-build.py | 17 +- scripts/analyse-9p-simpletrace.py | 2 + scripts/analyse-locks-simpletrace.py | 5 +- scripts/analyze-migration.py | 48 ++-- scripts/block-coroutine-wrapper.py | 4 +- scripts/check_sparse.py | 9 +- scripts/ci/gitlab-pipeline-status | 4 +- .../codeconverter/codeconverter/patching.py | 71 +++-- .../codeconverter/codeconverter/qom_macros.py | 15 +- .../codeconverter/qom_type_info.py | 10 +- .../codeconverter/codeconverter/regexps.py | 5 +- .../codeconverter/test_patching.py | 9 +- .../codeconverter/test_regexps.py | 3 +- scripts/codeconverter/codeconverter/utils.py | 5 +- scripts/codeconverter/converter.py | 16 +- scripts/compare-machine-types.py | 49 ++-- scripts/coverage/compare_gcov_json.py | 7 +- scripts/cpu-x86-uarch-abi.py | 4 +- scripts/decodetree.py | 12 +- scripts/device-crash-test | 18 +- scripts/dump-guest-memory.py | 5 +- scripts/feature_to_c.py | 5 +- scripts/kvm/kvm_flightrecorder | 7 +- scripts/kvm/vmxcap | 6 +- scripts/meson-buildoptions.py | 3 +- scripts/minikconf.py | 13 +- scripts/modinfo-collect.py | 7 +- scripts/modinfo-generate.py | 4 +- scripts/modules/module_block.py | 5 +- scripts/mtest2make.py | 8 +- scripts/oss-fuzz/minimize_qtest_trace.py | 18 +- scripts/oss-fuzz/output_reproducer.py | 18 +- .../oss-fuzz/reorder_fuzzer_qtest_trace.py | 6 +- scripts/performance/dissect.py | 2 +- scripts/performance/topN_callgrind.py | 2 +- scripts/performance/topN_perf.py | 2 +- scripts/probe-gdb-support.py | 4 +- scripts/python_qmp_updater.py | 1 + scripts/qapi-gen.py | 1 + scripts/qapi/commands.py | 13 +- scripts/qapi/common.py | 39 ++- scripts/qapi/error.py | 1 - scripts/qapi/events.py | 8 +- scripts/qapi/expr.py | 21 +- scripts/qapi/features.py | 2 +- scripts/qapi/gen.py | 14 +- scripts/qapi/introspect.py | 42 ++- scripts/qapi/parser.py | 44 ++- scripts/qapi/schema.py | 271 +++++++++--------- scripts/qapi/source.py | 10 +- scripts/qapi/types.py | 20 +- scripts/qapi/visit.py | 14 +- scripts/qcow2-to-stdout.py | 8 +- scripts/qemu-gdb.py | 13 +- scripts/qemu-plugin-symbols.py | 4 +- scripts/qemu-stamp.py | 1 + scripts/qemu-trace-stap | 1 - scripts/qemugdb/aio.py | 1 + scripts/qemugdb/coroutine.py | 1 + scripts/qemugdb/mtree.py | 1 + scripts/qemugdb/tcg.py | 2 +- scripts/qemugdb/timers.py | 2 +- scripts/qmp/qemu-ga-client | 1 + scripts/qmp/qmp | 1 + scripts/qmp/qmp-shell | 1 + scripts/qmp/qmp-shell-wrap | 1 + scripts/qmp/qom-fuse | 1 + scripts/qmp/qom-get | 1 + scripts/qmp/qom-list | 1 + scripts/qmp/qom-set | 1 + scripts/qmp/qom-tree | 1 + scripts/qom-cast-macro-clean-cocci-gen.py | 3 +- scripts/render_block_graph.py | 28 +- scripts/replay-dump.py | 10 +- scripts/rust/rustc_args.py | 10 +- scripts/shaderinclude.py | 4 +- scripts/signrom.py | 3 +- scripts/simplebench/bench-backup.py | 9 +- scripts/simplebench/bench-example.py | 4 +- scripts/simplebench/bench_block_job.py | 9 +- scripts/simplebench/bench_prealloc.py | 10 +- scripts/simplebench/bench_write_req.py | 5 +- scripts/simplebench/img_bench_templater.py | 10 +- scripts/simplebench/results_to_text.py | 4 +- scripts/simplebench/simplebench.py | 2 +- scripts/simplebench/table_templater.py | 4 +- scripts/simpletrace.py | 16 +- scripts/symlink-install-tree.py | 3 +- scripts/tracetool.py | 5 +- scripts/tracetool/__init__.py | 11 +- scripts/tracetool/backend/__init__.py | 2 - scripts/tracetool/backend/dtrace.py | 2 - scripts/tracetool/backend/ftrace.py | 2 - scripts/tracetool/backend/log.py | 2 - scripts/tracetool/backend/simple.py | 2 - scripts/tracetool/backend/syslog.py | 2 - scripts/tracetool/backend/ust.py | 2 - scripts/tracetool/format/__init__.py | 2 - scripts/tracetool/format/c.py | 2 - scripts/tracetool/format/d.py | 5 +- scripts/tracetool/format/h.py | 2 - scripts/tracetool/format/log_stap.py | 3 +- scripts/tracetool/format/simpletrace_stap.py | 3 +- scripts/tracetool/format/stap.py | 2 - scripts/tracetool/format/ust_events_c.py | 2 - scripts/tracetool/format/ust_events_h.py | 2 - scripts/u2f-setup-gen.py | 15 +- scripts/undefsym.py | 3 +- scripts/userfaultfd-wrlat.py | 8 +- scripts/vmstate-static-checker.py | 1 + scripts/xml-preprocess-test.py | 3 +- scripts/xml-preprocess.py | 4 +- target/hexagon/gen_analyze_funcs.py | 3 - target/hexagon/gen_decodetree.py | 9 +- target/hexagon/gen_helper_funcs.py | 3 - target/hexagon/gen_helper_protos.py | 4 +- target/hexagon/gen_idef_parser_funcs.py | 4 - target/hexagon/gen_op_attribs.py | 6 +- target/hexagon/gen_opcodes_def.py | 6 +- target/hexagon/gen_printinsn.py | 5 +- target/hexagon/gen_tcg_func_table.py | 6 +- target/hexagon/gen_tcg_funcs.py | 3 - target/hexagon/gen_trans_funcs.py | 8 +- target/hexagon/hex_common.py | 15 +- tests/docker/docker.py | 37 ++- tests/functional/aspeed.py | 4 +- tests/functional/qemu_test/__init__.py | 30 +- tests/functional/qemu_test/archive.py | 2 +- tests/functional/qemu_test/asset.py | 10 +- tests/functional/qemu_test/decorators.py | 1 + tests/functional/qemu_test/linuxkernel.py | 5 +- tests/functional/qemu_test/ports.py | 3 +- tests/functional/qemu_test/testcase.py | 2 +- tests/functional/qemu_test/tuxruntest.py | 12 +- tests/functional/qemu_test/uncompress.py | 4 +- tests/functional/replay_kernel.py | 7 +- tests/functional/reverse_debugging.py | 5 +- .../functional/test_aarch64_aspeed_ast2700.py | 9 +- .../test_aarch64_aspeed_ast2700fc.py | 9 +- tests/functional/test_aarch64_imx8mp_evk.py | 4 +- tests/functional/test_aarch64_raspi3.py | 2 +- tests/functional/test_aarch64_raspi4.py | 7 +- tests/functional/test_aarch64_replay.py | 4 +- .../functional/test_aarch64_reverse_debug.py | 2 +- tests/functional/test_aarch64_rme_sbsaref.py | 8 +- tests/functional/test_aarch64_rme_virt.py | 11 +- tests/functional/test_aarch64_sbsaref.py | 9 +- .../functional/test_aarch64_sbsaref_alpine.py | 8 +- .../test_aarch64_sbsaref_freebsd.py | 8 +- tests/functional/test_aarch64_smmu.py | 8 +- tests/functional/test_aarch64_tcg_plugins.py | 4 +- tests/functional/test_aarch64_tuxrun.py | 1 + tests/functional/test_aarch64_virt.py | 11 +- tests/functional/test_aarch64_virt_gpu.py | 7 +- tests/functional/test_aarch64_xlnx_versal.py | 3 +- tests/functional/test_acpi_bits.py | 21 +- tests/functional/test_alpha_clipper.py | 2 +- tests/functional/test_arm_aspeed_ast1030.py | 7 +- tests/functional/test_arm_aspeed_ast2500.py | 2 +- tests/functional/test_arm_aspeed_ast2600.py | 11 +- tests/functional/test_arm_aspeed_bletchley.py | 2 +- tests/functional/test_arm_aspeed_palmetto.py | 2 +- tests/functional/test_arm_aspeed_rainier.py | 3 +- tests/functional/test_arm_aspeed_romulus.py | 2 +- .../functional/test_arm_aspeed_witherspoon.py | 2 +- tests/functional/test_arm_bflt.py | 8 +- tests/functional/test_arm_bpim2u.py | 10 +- tests/functional/test_arm_canona1100.py | 3 +- tests/functional/test_arm_collie.py | 2 +- tests/functional/test_arm_cubieboard.py | 10 +- tests/functional/test_arm_emcraft_sf2.py | 7 +- tests/functional/test_arm_integratorcp.py | 12 +- tests/functional/test_arm_microbit.py | 8 +- tests/functional/test_arm_orangepi.py | 11 +- tests/functional/test_arm_quanta_gsj.py | 9 +- tests/functional/test_arm_raspi2.py | 7 +- tests/functional/test_arm_smdkc210.py | 2 +- tests/functional/test_arm_stellaris.py | 8 +- tests/functional/test_arm_sx1.py | 2 +- tests/functional/test_arm_tuxrun.py | 1 + tests/functional/test_arm_vexpress.py | 2 +- tests/functional/test_arm_virt.py | 3 +- tests/functional/test_avr_mega2560.py | 2 +- tests/functional/test_avr_uno.py | 2 +- tests/functional/test_cpu_queries.py | 1 + tests/functional/test_empty_cpu_model.py | 1 + tests/functional/test_hppa_seabios.py | 4 +- tests/functional/test_i386_tuxrun.py | 1 + tests/functional/test_intel_iommu.py | 6 +- tests/functional/test_linux_initrd.py | 2 +- tests/functional/test_loongarch64_virt.py | 10 +- tests/functional/test_m68k_mcf5208evb.py | 2 +- tests/functional/test_m68k_nextcube.py | 8 +- tests/functional/test_m68k_q800.py | 3 +- tests/functional/test_m68k_tuxrun.py | 1 + tests/functional/test_mem_addr_space.py | 4 +- tests/functional/test_memlock.py | 9 +- .../functional/test_microblaze_s3adsp1800.py | 9 +- tests/functional/test_mips64_malta.py | 2 +- tests/functional/test_mips64_tuxrun.py | 1 + tests/functional/test_mips64el_fuloong2e.py | 11 +- tests/functional/test_mips64el_loongson3v.py | 8 +- tests/functional/test_mips64el_malta.py | 17 +- tests/functional/test_mips64el_tuxrun.py | 1 + tests/functional/test_mips_malta.py | 8 +- tests/functional/test_mips_tuxrun.py | 1 + tests/functional/test_mipsel_malta.py | 11 +- tests/functional/test_mipsel_tuxrun.py | 1 + tests/functional/test_multiprocess.py | 10 +- tests/functional/test_netdev_ethtool.py | 5 +- tests/functional/test_or1k_sim.py | 2 +- tests/functional/test_pc_cpu_hotplug_props.py | 1 + tests/functional/test_ppc64_e500.py | 7 +- tests/functional/test_ppc64_hv.py | 16 +- tests/functional/test_ppc64_mac99.py | 8 +- tests/functional/test_ppc64_powernv.py | 4 +- tests/functional/test_ppc64_pseries.py | 4 +- tests/functional/test_ppc64_reverse_debug.py | 2 +- tests/functional/test_ppc64_tuxrun.py | 3 +- tests/functional/test_ppc_40p.py | 10 +- tests/functional/test_ppc_74xx.py | 4 +- tests/functional/test_ppc_amiga.py | 3 +- tests/functional/test_ppc_bamboo.py | 9 +- tests/functional/test_ppc_mac.py | 2 +- tests/functional/test_ppc_mpc8544ds.py | 3 +- tests/functional/test_ppc_sam460ex.py | 7 +- tests/functional/test_ppc_tuxrun.py | 1 + tests/functional/test_ppc_virtex_ml507.py | 3 +- tests/functional/test_riscv32_tuxrun.py | 1 + tests/functional/test_riscv64_tuxrun.py | 1 + tests/functional/test_riscv_opensbi.py | 4 +- tests/functional/test_rx_gdbsim.py | 10 +- tests/functional/test_s390x_ccw_virtio.py | 9 +- tests/functional/test_s390x_topology.py | 11 +- tests/functional/test_s390x_tuxrun.py | 1 + tests/functional/test_sh4_r2d.py | 2 +- tests/functional/test_sh4_tuxrun.py | 1 + tests/functional/test_sh4eb_r2d.py | 7 +- tests/functional/test_sparc64_sun4u.py | 3 +- tests/functional/test_sparc64_tuxrun.py | 1 + tests/functional/test_sparc_sun4m.py | 2 +- tests/functional/test_virtio_balloon.py | 10 +- tests/functional/test_virtio_gpu.py | 14 +- tests/functional/test_virtio_version.py | 1 + tests/functional/test_x86_64_hotplug_blk.py | 6 +- tests/functional/test_x86_64_hotplug_cpu.py | 6 +- tests/functional/test_x86_64_kvm_xen.py | 8 +- tests/functional/test_x86_64_replay.py | 4 +- tests/functional/test_x86_64_reverse_debug.py | 2 +- tests/functional/test_x86_64_tuxrun.py | 1 + .../functional/test_x86_cpu_model_versions.py | 13 +- tests/functional/test_xtensa_lx60.py | 2 +- tests/guest-debug/run-test.py | 9 +- tests/guest-debug/test_gdbstub.py | 15 +- tests/image-fuzzer/qcow2/fuzz.py | 3 +- tests/image-fuzzer/qcow2/layout.py | 32 ++- tests/image-fuzzer/runner.py | 19 +- tests/lcitool/refresh | 6 +- tests/migration-stress/guestperf-batch.py | 1 + tests/migration-stress/guestperf-plot.py | 1 + tests/migration-stress/guestperf.py | 1 + .../migration-stress/guestperf/comparison.py | 3 +- tests/migration-stress/guestperf/engine.py | 8 +- tests/migration-stress/guestperf/hardware.py | 2 +- tests/migration-stress/guestperf/plot.py | 4 +- tests/migration-stress/guestperf/progress.py | 4 +- tests/migration-stress/guestperf/report.py | 9 +- tests/migration-stress/guestperf/scenario.py | 2 +- tests/migration-stress/guestperf/shell.py | 18 +- tests/migration-stress/guestperf/timings.py | 4 +- tests/qapi-schema/test-qapi.py | 2 +- tests/qemu-iotests/030 | 6 +- tests/qemu-iotests/040 | 6 +- tests/qemu-iotests/041 | 7 +- tests/qemu-iotests/044 | 11 +- tests/qemu-iotests/045 | 10 +- tests/qemu-iotests/055 | 4 +- tests/qemu-iotests/056 | 6 +- tests/qemu-iotests/057 | 5 +- tests/qemu-iotests/065 | 10 +- tests/qemu-iotests/093 | 9 +- tests/qemu-iotests/096 | 4 +- tests/qemu-iotests/118 | 4 +- tests/qemu-iotests/124 | 8 +- tests/qemu-iotests/129 | 2 + tests/qemu-iotests/132 | 3 +- tests/qemu-iotests/136 | 4 +- tests/qemu-iotests/139 | 3 +- tests/qemu-iotests/141 | 1 + tests/qemu-iotests/147 | 13 +- tests/qemu-iotests/148 | 2 + tests/qemu-iotests/149 | 7 +- tests/qemu-iotests/151 | 6 +- tests/qemu-iotests/152 | 2 + tests/qemu-iotests/155 | 2 + tests/qemu-iotests/163 | 11 +- tests/qemu-iotests/165 | 3 +- tests/qemu-iotests/194 | 7 +- tests/qemu-iotests/196 | 2 + tests/qemu-iotests/202 | 1 + tests/qemu-iotests/203 | 1 + tests/qemu-iotests/205 | 11 +- tests/qemu-iotests/206 | 1 + tests/qemu-iotests/207 | 6 +- tests/qemu-iotests/208 | 1 + tests/qemu-iotests/209 | 11 +- tests/qemu-iotests/210 | 1 + tests/qemu-iotests/211 | 1 + tests/qemu-iotests/212 | 1 + tests/qemu-iotests/213 | 1 + tests/qemu-iotests/216 | 1 + tests/qemu-iotests/218 | 1 + tests/qemu-iotests/219 | 1 + tests/qemu-iotests/224 | 13 +- tests/qemu-iotests/228 | 11 +- tests/qemu-iotests/234 | 4 +- tests/qemu-iotests/235 | 14 +- tests/qemu-iotests/236 | 1 + tests/qemu-iotests/237 | 2 + tests/qemu-iotests/238 | 3 +- tests/qemu-iotests/240 | 2 +- tests/qemu-iotests/242 | 23 +- tests/qemu-iotests/245 | 1 + tests/qemu-iotests/246 | 1 + tests/qemu-iotests/248 | 12 +- tests/qemu-iotests/254 | 3 +- tests/qemu-iotests/255 | 1 + tests/qemu-iotests/256 | 4 +- tests/qemu-iotests/257 | 28 +- tests/qemu-iotests/258 | 10 +- tests/qemu-iotests/260 | 8 +- tests/qemu-iotests/262 | 4 +- tests/qemu-iotests/264 | 7 +- tests/qemu-iotests/274 | 1 + tests/qemu-iotests/277 | 10 +- tests/qemu-iotests/280 | 2 +- tests/qemu-iotests/281 | 4 +- tests/qemu-iotests/283 | 1 + tests/qemu-iotests/295 | 6 +- tests/qemu-iotests/296 | 7 +- tests/qemu-iotests/297 | 8 +- tests/qemu-iotests/298 | 2 + tests/qemu-iotests/299 | 1 + tests/qemu-iotests/300 | 10 +- tests/qemu-iotests/302 | 4 +- tests/qemu-iotests/303 | 13 +- tests/qemu-iotests/304 | 3 +- tests/qemu-iotests/307 | 2 +- tests/qemu-iotests/310 | 1 + tests/qemu-iotests/check | 7 +- tests/qemu-iotests/fat16.py | 11 +- tests/qemu-iotests/findtests.py | 19 +- tests/qemu-iotests/iotests.py | 66 +++-- tests/qemu-iotests/linters.py | 9 +- tests/qemu-iotests/nbd-fault-injector.py | 13 +- tests/qemu-iotests/qcow2.py | 5 +- tests/qemu-iotests/qcow2_format.py | 14 +- tests/qemu-iotests/qed.py | 10 +- tests/qemu-iotests/testenv.py | 25 +- tests/qemu-iotests/testrunner.py | 32 +-- tests/qemu-iotests/tests/block-status-cache | 1 + .../tests/export-incoming-iothread | 1 + .../qemu-iotests/tests/graph-changes-while-io | 11 +- tests/qemu-iotests/tests/image-fleecing | 1 + tests/qemu-iotests/tests/inactive-node-nbd | 8 +- .../tests/iothreads-commit-active | 2 + tests/qemu-iotests/tests/iothreads-create | 2 + tests/qemu-iotests/tests/iothreads-nbd-export | 3 + tests/qemu-iotests/tests/iothreads-stream | 2 + tests/qemu-iotests/tests/luks-detached-header | 5 +- .../tests/migrate-bitmaps-postcopy-test | 10 +- tests/qemu-iotests/tests/migrate-bitmaps-test | 2 +- .../qemu-iotests/tests/migrate-during-backup | 1 + .../tests/mirror-change-copy-mode | 3 +- .../tests/mirror-ready-cancel-error | 1 + tests/qemu-iotests/tests/nbd-multiconn | 2 +- .../qemu-iotests/tests/nbd-reconnect-on-open | 11 +- .../qemu-iotests/tests/parallels-read-bitmap | 8 +- tests/qemu-iotests/tests/qsd-migrate | 2 +- .../tests/remove-bitmap-from-backing | 8 +- tests/qemu-iotests/tests/reopen-file | 3 +- .../qemu-iotests/tests/stream-error-on-reset | 8 +- .../tests/stream-unaligned-prefetch | 13 +- .../qemu-iotests/tests/stream-under-throttle | 4 +- tests/qemu-iotests/tests/vvfat | 7 +- tests/tcg/aarch64/gdbstub/test-mte.py | 2 +- tests/tcg/aarch64/gdbstub/test-sve-ioctl.py | 4 +- tests/tcg/aarch64/gdbstub/test-sve.py | 2 +- tests/tcg/i386/test-avx.py | 11 +- tests/tcg/i386/test-mmx.py | 9 +- tests/tcg/multiarch/gdbstub/catch-syscalls.py | 6 +- .../gdbstub/follow-fork-mode-child.py | 4 +- .../gdbstub/follow-fork-mode-parent.py | 2 +- tests/tcg/multiarch/gdbstub/interrupt.py | 1 - tests/tcg/multiarch/gdbstub/late-attach.py | 4 +- tests/tcg/multiarch/gdbstub/memory.py | 2 - tests/tcg/multiarch/gdbstub/prot-none.py | 5 +- tests/tcg/multiarch/gdbstub/registers.py | 3 +- tests/tcg/multiarch/gdbstub/sha1.py | 1 - .../multiarch/gdbstub/test-proc-mappings.py | 1 - .../multiarch/gdbstub/test-qxfer-auxv-read.py | 1 - .../gdbstub/test-qxfer-siginfo-read.py | 2 +- .../gdbstub/test-thread-breakpoint.py | 1 - .../system/validate-memory-counts.py | 7 +- tests/tcg/s390x/gdbstub/test-signals-s390x.py | 2 - tests/tcg/s390x/gdbstub/test-svc.py | 1 - tests/vm/aarch64vm.py | 16 +- tests/vm/basevm.py | 37 +-- tests/vm/centos.aarch64 | 11 +- tests/vm/freebsd | 7 +- tests/vm/haiku.x86_64 | 7 +- tests/vm/netbsd | 5 +- tests/vm/openbsd | 5 +- tests/vm/ubuntu.aarch64 | 6 +- tests/vm/ubuntuvm.py | 6 +- 448 files changed, 1959 insertions(+), 1631 deletions(-)
docs/conf.py +7 -6
@@ -1,4 +1,3 @@ -# -*- coding: utf-8 -*- # # QEMU documentation build configuration file, created by # sphinx-quickstart on Thu Jan 31 16:40:14 2019.
@@ -28,9 +27,11 @@ import os import sys + import sphinx from sphinx.errors import ConfigError + # The per-manual conf.py will set qemu_docdir for a single-manual build; # otherwise set it here if this is an entire-manual-set build. # This is always the absolute path of the docs/ directory in the source tree.
@@ -93,9 +94,9 @@ default_role = 'any' # General information about the project. -project = u'QEMU' -copyright = u'2025, The QEMU Project Developers' -author = u'The QEMU Project Developers' +project = 'QEMU' +copyright = '2025, The QEMU Project Developers' +author = 'The QEMU Project Developers' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the
@@ -291,8 +292,8 @@ # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ - (master_doc, 'QEMU.tex', u'QEMU Documentation', - u'The QEMU Project Developers', 'manual'), + (master_doc, 'QEMU.tex', 'QEMU Documentation', + 'The QEMU Project Developers', 'manual'), ]
docs/sphinx/compat.py +1 -3
@@ -8,13 +8,11 @@ Any, Callable, Optional, - Type, ) from docutils import nodes from docutils.nodes import Element, Node, Text from docutils.statemachine import StringList - import sphinx from sphinx import addnodes, util from sphinx.directives import ObjectDescription
@@ -85,7 +83,7 @@ def _compat_make_xref( # pylint: disable=unused-argument rolename: str, domain: str, target: str, - innernode: Type[TextlikeNode] = addnodes.literal_emphasis, + innernode: type[TextlikeNode] = addnodes.literal_emphasis, contnode: Optional[Node] = None, env: Optional[BuildEnvironment] = None, inliner: Any = None,
docs/sphinx/dbusdoc.py +6 -22
@@ -7,30 +7,16 @@ # Author: Marc-André Lureau <[email protected]> """dbus-doc is a Sphinx extension that provides documentation from D-Bus XML.""" -import os -import re -from typing import ( - TYPE_CHECKING, - Any, - Callable, - Dict, - Iterator, - List, - Optional, - Sequence, - Set, - Tuple, - Type, - TypeVar, - Union, -) +from typing import Any -import sphinx +import dbusdomain +from dbusparser import parse_dbus_xml from docutils import nodes from docutils.nodes import Element, Node from docutils.parsers.rst import Directive, directives from docutils.parsers.rst.states import RSTState from docutils.statemachine import StringList, ViewList +import sphinx from sphinx.application import Sphinx from sphinx.errors import ExtensionError from sphinx.util import logging
@@ -38,8 +24,6 @@ from sphinx.util.docutils import SphinxDirective, switch_source_input from sphinx.util.nodes import nested_parse_with_titles -import dbusdomain -from dbusparser import parse_dbus_xml logger = logging.getLogger(__name__)
@@ -116,7 +100,7 @@ def add_interface(self, iface): self.indent = self.indent[:-3] -def parse_generated_content(state: RSTState, content: StringList) -> List[Node]: +def parse_generated_content(state: RSTState, content: StringList) -> list[Node]: """Parse a generated content by Documenter.""" with switch_source_input(state, content): node = nodes.paragraph()
@@ -157,7 +141,7 @@ def run(self): return result -def setup(app: Sphinx) -> Dict[str, Any]: +def setup(app: Sphinx) -> dict[str, Any]: """Register dbus-doc directive with Sphinx""" app.add_config_value("dbusdoc_srctree", None, "env") app.add_directive("dbus-doc", DBusDocDirective)
docs/sphinx/dbusdomain.py +20 -19
@@ -6,15 +6,11 @@ # # Author: Marc-André Lureau <[email protected]> +from collections.abc import Iterable, Iterator from typing import ( Any, - Dict, - Iterable, - Iterator, - List, NamedTuple, Optional, - Tuple, cast, )
@@ -24,7 +20,12 @@ from sphinx import addnodes from sphinx.addnodes import desc_signature, pending_xref from sphinx.directives import ObjectDescription -from sphinx.domains import Domain, Index, IndexEntry, ObjType +from sphinx.domains import ( + Domain, + Index, + IndexEntry, + ObjType, +) from sphinx.locale import _ from sphinx.roles import XRefRole from sphinx.util import nodes as node_utils
@@ -87,7 +88,7 @@ def handle_signature(self, sig: str, signode: desc_signature) -> str: signode += addnodes.desc_name(sig, sig) return sig - def run(self) -> List[Node]: + def run(self) -> list[Node]: _, node = super().run() name = self.arguments[0] section = nodes.section(ids=[name + "-section"])
@@ -113,7 +114,7 @@ class DBusMethod(DBusMember): } ) - doc_field_types: List[Field] = [ + doc_field_types: list[Field] = [ TypedField( "arg", label=_("Arguments"),
@@ -171,7 +172,7 @@ class DBusSignal(DBusMethod): Implementation of ``dbus:signal``. """ - doc_field_types: List[Field] = [ + doc_field_types: list[Field] = [ TypedField( "arg", label=_("Arguments"),
@@ -203,7 +204,7 @@ class DBusProperty(DBusMember): } ) - doc_field_types: List[Field] = [] + doc_field_types: list[Field] = [] def get_index_text(self, ifacename: str, name: str) -> str: return _("%s (%s property)") % (name, ifacename)
@@ -244,7 +245,7 @@ def handle_signature(self, sig: str, signode: desc_signature) -> str: signode += addnodes.desc_sig_keyword_type(ty, ty) return sig - def run(self) -> List[Node]: + def run(self) -> list[Node]: self.name = "dbus:member" return super().run()
@@ -281,10 +282,10 @@ class DBusIndex(Index): def generate( self, docnames: Iterable[str] = None - ) -> Tuple[List[Tuple[str, List[IndexEntry]]], bool]: - content: Dict[str, List[IndexEntry]] = {} + ) -> tuple[list[tuple[str, list[IndexEntry]]], bool]: + content: dict[str, list[IndexEntry]] = {} # list of prefixes to ignore - ignores: List[str] = self.domain.env.config["dbus_index_common_prefix"] + ignores: list[str] = self.domain.env.config["dbus_index_common_prefix"] ignores = sorted(ignores, key=len, reverse=True) ifaces = sorted(
@@ -329,7 +330,7 @@ class DBusDomain(Domain): name = "dbus" label = "D-Bus" - object_types: Dict[str, ObjType] = { + object_types: dict[str, ObjType] = { "interface": ObjType(_("interface"), "iface", "obj"), "method": ObjType(_("method"), "meth", "obj"), "signal": ObjType(_("signal"), "sig", "obj"),
@@ -347,7 +348,7 @@ class DBusDomain(Domain): "sig": DBusXRef(), "prop": DBusXRef(), } - initial_data: Dict[str, Dict[str, Tuple[Any]]] = { + initial_data: dict[str, dict[str, tuple[Any]]] = { "objects": {}, # fullname -> ObjectEntry } indices = [
@@ -355,7 +356,7 @@ class DBusDomain(Domain): ] @property - def objects(self) -> Dict[str, ObjectEntry]: + def objects(self) -> dict[str, ObjectEntry]: return self.data.setdefault("objects", {}) # fullname -> ObjectEntry def note_object(
@@ -368,7 +369,7 @@ def clear_doc(self, docname: str) -> None: if obj.docname == docname: del self.objects[fullname] - def find_obj(self, typ: str, name: str) -> Optional[Tuple[str, ObjectEntry]]: + def find_obj(self, typ: str, name: str) -> Optional[tuple[str, ObjectEntry]]: # skip parens if name[-2:] == "()": name = name[:-2]
@@ -396,7 +397,7 @@ def resolve_xref( builder, fromdocname, objdef.docname, objdef.node_id, contnode ) - def get_objects(self) -> Iterator[Tuple[str, str, str, str, str, int]]: + def get_objects(self) -> Iterator[tuple[str, str, str, str, str, int]]: for refname, obj in self.objects.items(): yield (refname, refname, obj.objtype, obj.docname, obj.node_id, 1)
docs/sphinx/dbusparser.py +2 -2
@@ -78,7 +78,7 @@ def __init__(self, name, signature, access): elif self.access == "write": self.writable = True else: - raise ValueError('Invalid access type "{}"'.format(self.access)) + raise ValueError(f'Invalid access type "{self.access}"') self.doc_string = "" self.since = "" self.deprecated = False
@@ -277,7 +277,7 @@ def handle_start_element(self, name, attrs): elif direction == "out": self._cur_object.out_args.append(arg) else: - raise ValueError('Invalid direction "{}"'.format(direction)) + raise ValueError(f'Invalid direction "{direction}"') self._cur_object = arg elif name == DBusXMLParser.STATE_ANNOTATION: self.state = DBusXMLParser.STATE_ANNOTATION
docs/sphinx/depfile.py +6 -5
@@ -1,4 +1,3 @@ -# coding=utf-8 # # QEMU depfile generation extension #
@@ -11,17 +10,19 @@ an external build system""" import os -import sphinx -import sys from pathlib import Path +import sys + +import sphinx + __version__ = '1.0' def get_infiles(env): for x in env.found_docs: yield str(env.doc2path(x)) - yield from ((os.path.join(env.srcdir, dep) - for dep in env.dependencies[x])) + yield from (os.path.join(env.srcdir, dep) + for dep in env.dependencies[x]) for mod in sys.modules.values(): if hasattr(mod, '__file__'): if mod.__file__:
docs/sphinx/fakedbusdoc.py +3 -2
@@ -7,9 +7,10 @@ # Author: Marc-André Lureau <[email protected]> """dbus-doc is a Sphinx extension that provides documentation from D-Bus XML.""" +from typing import Any + from docutils.parsers.rst import Directive from sphinx.application import Sphinx -from typing import Any, Dict class FakeDBusDocDirective(Directive):
@@ -20,7 +21,7 @@ def run(self): return [] -def setup(app: Sphinx) -> Dict[str, Any]: +def setup(app: Sphinx) -> dict[str, Any]: """Register a fake dbus-doc directive with Sphinx""" app.add_directive("dbus-doc", FakeDBusDocDirective)
docs/sphinx/hxtool.py +3 -4
@@ -1,4 +1,3 @@ -# coding=utf-8 # # QEMU hxtool .hx file parsing extension #
@@ -16,17 +15,17 @@ # Each hxtool-doc:: directive takes one argument which is the # path of the .hx file to process, relative to the source tree. +from enum import Enum import os import re -from enum import Enum from docutils import nodes +from docutils.parsers.rst import Directive, directives from docutils.statemachine import ViewList -from docutils.parsers.rst import directives, Directive +import sphinx from sphinx.errors import ExtensionError from sphinx.util.docutils import switch_source_input from sphinx.util.nodes import nested_parse_with_titles -import sphinx __version__ = '1.0'
docs/sphinx/kerneldoc.py +3 -5
@@ -1,4 +1,3 @@ -# coding=utf-8 # # Copyright © 2016 Intel Corporation #
@@ -28,16 +27,15 @@ # import codecs +import glob import os +import re import subprocess import sys -import re -import glob from docutils import nodes, statemachine +from docutils.parsers.rst import Directive, directives from docutils.statemachine import ViewList -from docutils.parsers.rst import directives, Directive - import sphinx from sphinx.util import logging from sphinx.util.docutils import switch_source_input
docs/sphinx/qapi_domain.py +34 -47
244 lines not on this page · raw message
docs/sphinx/qapidoc.py +11 -18
@@ -1,4 +1,3 @@ -# coding=utf-8 # # QEMU qapidoc QAPI file parsing extension #
@@ -68,14 +67,8 @@ if TYPE_CHECKING: - from typing import ( - Any, - Generator, - List, - Optional, - Sequence, - Union, - ) + from collections.abc import Generator, Sequence + from typing import Any from sphinx.application import Sphinx from sphinx.util.typing import ExtensionMetadata
@@ -98,7 +91,7 @@ class Transmogrifier: } def __init__(self) -> None: - self._curr_ent: Optional[QAPISchemaDefinition] = None + self._curr_ent: QAPISchemaDefinition | None = None self._result = StringList() self.indent = 0
@@ -132,7 +125,7 @@ def add_line_raw(self, line: str, source: str, *lineno: int) -> None: """Append one line of generated reST to the output.""" # NB: Sphinx uses zero-indexed lines; subtract one. - lineno = tuple((n - 1 for n in lineno)) + lineno = tuple(n - 1 for n in lineno) if line.strip(): # not a blank line
@@ -176,7 +169,7 @@ def add_field( name: str, body: str, info: QAPISourceInfo, - typ: Optional[str] = None, + typ: str | None = None, ) -> None: if typ: text = f":{kind} {typ} {name}: {body}"
@@ -185,8 +178,8 @@ def add_field( self.add_lines(text, info) def format_type( - self, ent: Union[QAPISchemaDefinition | QAPISchemaMember] - ) -> Optional[str]: + self, ent: QAPISchemaDefinition | QAPISchemaMember + ) -> str | None: if isinstance(ent, (QAPISchemaEnumMember, QAPISchemaFeature)): return None
@@ -306,7 +299,7 @@ def _insert_member_pointer(self, ent: QAPISchemaDefinition) -> None: def _get_target( ent: QAPISchemaDefinition, - ) -> Optional[QAPISchemaDefinition]: + ) -> QAPISchemaDefinition | None: if isinstance(ent, (QAPISchemaCommand, QAPISchemaEvent)): return ent.arg_type if isinstance(ent, QAPISchemaObjectType):
@@ -674,7 +667,7 @@ def _highlightlang(self) -> addnodes.highlightlang: ) return node - def admonition_wrap(self, *content: nodes.Node) -> List[nodes.Node]: + def admonition_wrap(self, *content: nodes.Node) -> list[nodes.Node]: title = "Example:" if "title" in self.options: title = f"{title} {self.options['title']}"
@@ -687,7 +680,7 @@ def admonition_wrap(self, *content: nodes.Node) -> List[nodes.Node]: ) return [admon] - def run_annotated(self) -> List[nodes.Node]: + def run_annotated(self) -> list[nodes.Node]: lang_node = self._highlightlang() content_node: nodes.Element = nodes.section()
@@ -708,7 +701,7 @@ def run_annotated(self) -> List[nodes.Node]: return content_node.children - def run(self) -> List[nodes.Node]: + def run(self) -> list[nodes.Node]: annotated = "annotated" in self.options if annotated:
docs/sphinx/qapidoc_legacy.py +0 -1
@@ -1,4 +1,3 @@ -# coding=utf-8 # type: ignore # # QEMU qapidoc QAPI file parsing extension
docs/sphinx/qmp_lexer.py +4 -3
@@ -10,11 +10,12 @@ # See the COPYING file in the top-level directory. """qmp_lexer is a Sphinx extension that provides a QMP lexer for code blocks.""" -from pygments.lexer import RegexLexer, DelegatingLexer -from pygments.lexers.data import JsonLexer from pygments import token +from pygments.lexer import DelegatingLexer, RegexLexer +from pygments.lexers.data import JsonLexer from sphinx import errors + class QMPExampleMarkersLexer(RegexLexer): """ QMPExampleMarkersLexer lexes QMP example annotations.
@@ -31,7 +32,7 @@ class QMPExampleMarkersLexer(RegexLexer): class QMPExampleLexer(DelegatingLexer): """QMPExampleLexer lexes annotated QMP examples.""" def __init__(self, **options): - super(QMPExampleLexer, self).__init__(JsonLexer, QMPExampleMarkersLexer, + super().__init__(JsonLexer, QMPExampleMarkersLexer, token.Error, **options) def setup(sphinx):
.gitlab-ci.d/check-dco.py +5 -4
@@ -8,8 +8,9 @@ import os import os.path -import sys import subprocess +import sys + namespace = "qemu-project" if len(sys.argv) >= 2:
@@ -25,7 +26,7 @@ ancestor = subprocess.check_output(["git", "merge-base", "check-dco/master", "HEAD"], - universal_newlines=True) + text=True) ancestor = ancestor.strip()
@@ -38,7 +39,7 @@ log = subprocess.check_output(["git", "log", "--format=%H %s", ancestor + "..."], - universal_newlines=True) + text=True) if log == "": commits = []
@@ -48,7 +49,7 @@ for sha, subject in commits: msg = subprocess.check_output(["git", "show", "-s", sha], - universal_newlines=True) + text=True) lines = msg.strip().split("\n") print("🔍 %s %s" % (sha, subject))
.gitlab-ci.d/check-patch.py +4 -3
@@ -8,8 +8,9 @@ import os import os.path -import sys import subprocess +import sys + namespace = "qemu-project" if len(sys.argv) >= 2:
@@ -28,13 +29,13 @@ ancestor = subprocess.check_output(["git", "merge-base", "check-patch/master", "HEAD"], - universal_newlines=True) + text=True) ancestor = ancestor.strip() log = subprocess.check_output(["git", "log", "--format=%H %s", ancestor + "..."], - universal_newlines=True) + text=True) subprocess.check_call(["git", "remote", "rm", "check-patch"])
.gitlab-ci.d/check-units.py +4 -4
@@ -7,10 +7,10 @@ # # SPDX-License-Identifier: GPL-2.0-or-later -from os import access, R_OK, path -from sys import argv, exit -import json from collections import Counter +import json +from os import R_OK, access, path +from sys import argv, exit def extract_build_units(cc_path):
@@ -21,7 +21,7 @@ def extract_build_units(cc_path): Hash table of ["unit"] = count """ - j = json.load(open(cc_path, 'r')) + j = json.load(open(cc_path)) files = [f['file'] for f in j] build_units = Counter(files)
python/qemu/machine/console_socket.py +1 -1
@@ -123,7 +123,7 @@ def recv(self, bufsize: int = 1, flags: int = 0) -> bytes: elapsed_sec = time.time() - start_time if elapsed_sec > self._recv_timeout_sec: raise socket.timeout - return bytes((self._buffer.popleft() for i in range(bufsize))) + return bytes(self._buffer.popleft() for i in range(bufsize)) def setblocking(self, value: bool) -> None: """When not draining we pass thru to the socket,
python/qemu/machine/machine.py +15 -19
@@ -17,6 +17,7 @@ # Based on qmp.py. # +from collections.abc import Sequence import errno from itertools import chain import locale
@@ -31,12 +32,7 @@ from typing import ( Any, BinaryIO, - Dict, - List, Optional, - Sequence, - Tuple, - Type, TypeVar, )
@@ -156,9 +152,9 @@ def __init__(self, self._qmp_timer = qmp_timer self._name = name or f"{id(self):x}" - self._sock_pair: Optional[Tuple[socket.socket, socket.socket]] = None + self._sock_pair: Optional[tuple[socket.socket, socket.socket]] = None self._cons_sock_pair: Optional[ - Tuple[socket.socket, socket.socket]] = None + tuple[socket.socket, socket.socket]] = None self._temp_dir: Optional[str] = None self._base_temp_dir = base_temp_dir self._log_dir = log_dir
@@ -176,11 +172,11 @@ def __init__(self, self._qemu_log_path: Optional[str] = None self._qemu_log_file: Optional[BinaryIO] = None self._popen: Optional['subprocess.Popen[bytes]'] = None - self._events: List[QMPMessage] = [] + self._events: list[QMPMessage] = [] self._iolog: Optional[str] = None self._qmp_set = True # Enable QMP monitor by default. self._qmp_connection: Optional[QEMUMonitorProtocol] = None - self._qemu_full_args: Tuple[str, ...] = () + self._qemu_full_args: tuple[str, ...] = () self._launched = False self._machine: Optional[str] = None self._console_index = 0
@@ -188,7 +184,7 @@ def __init__(self, self._console_device_type: Optional[str] = None self._console_socket: Optional[socket.socket] = None self._console_file: Optional[socket.SocketIO] = None - self._remove_files: List[str] = [] + self._remove_files: list[str] = [] self._user_killed = False self._quit_issued = False
@@ -196,7 +192,7 @@ def __enter__(self: _T) -> _T: return self def __exit__(self, - exc_type: Optional[Type[BaseException]], + exc_type: Optional[type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType]) -> None: self.shutdown()
@@ -288,11 +284,11 @@ def _load_io_log(self) -> None: # back to the platform default. _, encoding = locale.getlocale() if self._qemu_log_path is not None: - with open(self._qemu_log_path, "r", encoding=encoding) as iolog: + with open(self._qemu_log_path, encoding=encoding) as iolog: self._iolog = iolog.read() @property - def _base_args(self) -> List[str]: + def _base_args(self) -> list[str]: args = ['-display', 'none', '-vga', 'none'] if self._qmp_set:
@@ -324,7 +320,7 @@ def _base_args(self) -> List[str]: return args @property - def args(self) -> List[str]: + def args(self) -> list[str]: """Returns the list of arguments given to the QEMU binary.""" return self._args
@@ -685,14 +681,14 @@ def _qmp(self) -> QEMUMonitorProtocol: @classmethod def _qmp_args(cls, conv_keys: bool, - args: Dict[str, Any]) -> Dict[str, object]: + args: dict[str, Any]) -> dict[str, object]: if conv_keys: return {k.replace('_', '-'): v for k, v in args.items()} return args def qmp(self, cmd: str, - args_dict: Optional[Dict[str, object]] = None, + args_dict: Optional[dict[str, object]] = None, conv_keys: Optional[bool] = None, **args: Any) -> QMPMessage: """
@@ -714,7 +710,7 @@ def qmp(self, cmd: str, return ret def cmd(self, cmd: str, - args_dict: Optional[Dict[str, object]] = None, + args_dict: Optional[dict[str, object]] = None, conv_keys: Optional[bool] = None, **args: Any) -> QMPReturnValue: """
@@ -745,7 +741,7 @@ def get_qmp_event(self, wait: bool = False) -> Optional[QMPMessage]: return self._events.pop(0) return self._qmp.pull_event(wait=wait) - def get_qmp_events(self, wait: bool = False) -> List[QMPMessage]: + def get_qmp_events(self, wait: bool = False) -> list[QMPMessage]: """ Poll for queued QMP events and return a list of dicts """
@@ -800,7 +796,7 @@ def event_wait(self, name: str, return self.events_wait([(name, match)], timeout) def events_wait(self, - events: Sequence[Tuple[str, Any]], + events: Sequence[tuple[str, Any]], timeout: float = 60.0) -> Optional[QMPMessage]: """ events_wait waits for and returns a single named event from QMP.
python/qemu/machine/qtest.py +4 -9
@@ -17,15 +17,10 @@ # Based on qmp.py. # +from collections.abc import Sequence import os import socket -from typing import ( - List, - Optional, - Sequence, - TextIO, - Tuple, -) +from typing import Optional, TextIO from qemu.qmp import SocketAddrT
@@ -145,10 +140,10 @@ def __init__(self, qmp_timer=qmp_timer) self._qtest: Optional[QEMUQtestProtocol] = None self._qtest_sock_pair: Optional[ - Tuple[socket.socket, socket.socket]] = None + tuple[socket.socket, socket.socket]] = None @property - def _base_args(self) -> List[str]: + def _base_args(self) -> list[str]: args = super()._base_args assert self._qtest_sock_pair is not None fd = self._qtest_sock_pair[0].fileno()
python/qemu/qmp/events.py +8 -17
@@ -448,19 +448,10 @@ def accept(self, event) -> bool: """ import asyncio +from collections.abc import AsyncIterator, Iterable, Iterator from contextlib import contextmanager import logging -from typing import ( - AsyncIterator, - Callable, - Iterable, - Iterator, - List, - Optional, - Set, - Tuple, - Union, -) +from typing import Callable, Optional, Union from .error import QMPError from .message import Message
@@ -504,10 +495,10 @@ def __init__( self._queue: 'asyncio.Queue[Message]' = asyncio.Queue() # Intended as a historical record, NOT a processing queue or backlog. - self._history: List[Message] = [] + self._history: list[Message] = [] #: Primary event filter, based on one or more event names. - self.names: Set[str] = set() + self.names: set[str] = set() if isinstance(names, str): self.names.add(names) elif names is not None:
@@ -517,7 +508,7 @@ def __init__( self.event_filter: Optional[EventFilter] = event_filter def __repr__(self) -> str: - args: List[str] = [] + args: list[str] = [] if self.names: args.append(f"names={self.names!r}") if self.event_filter:
@@ -532,7 +523,7 @@ def __repr__(self) -> str: return f"{type(self).__name__}{state}({argstr})" @property - def history(self) -> Tuple[Message, ...]: + def history(self) -> tuple[Message, ...]: """ A read-only history of all events seen so far.
@@ -596,7 +587,7 @@ def empty(self) -> bool: """ return self._queue.empty() - def clear(self) -> List[Message]: + def clear(self) -> list[Message]: """ Clear this listener of all pending events.
@@ -650,7 +641,7 @@ class Events: property. """ def __init__(self) -> None: - self._listeners: List[EventListener] = [] + self._listeners: list[EventListener] = [] #: Default, all-events `EventListener`. See `qmp.events` for more info. self.events: EventListener = EventListener()
python/qemu/qmp/legacy.py +6 -9
@@ -22,15 +22,12 @@ # import asyncio +from collections.abc import Awaitable import socket from types import TracebackType from typing import ( Any, - Awaitable, - Dict, - List, Optional, - Type, TypeVar, Union, )
@@ -41,13 +38,13 @@ #: QMPMessage is an entire QMP message of any kind. -QMPMessage = Dict[str, Any] +QMPMessage = dict[str, Any] #: QMPReturnValue is the 'return' value of a command. QMPReturnValue = object #: QMPObject is any object in a QMP message. -QMPObject = Dict[str, object] +QMPObject = dict[str, object] # QMPMessage can be outgoing commands or incoming events/returns. # QMPReturnValue is usually a dict/json object, but due to QAPI's
@@ -121,7 +118,7 @@ def __enter__(self: _T) -> _T: return self def __exit__(self, - exc_type: Optional[Type[BaseException]], + exc_type: Optional[type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType]) -> None: # Implement context manager exit function.
@@ -202,7 +199,7 @@ def cmd_obj(self, qmp_cmd: QMPMessage) -> QMPMessage: ) def cmd_raw(self, name: str, - args: Optional[Dict[str, object]] = None) -> QMPMessage: + args: Optional[dict[str, object]] = None) -> QMPMessage: """ Build a QMP command and send it to the QMP Monitor.
@@ -260,7 +257,7 @@ def pull_event(self, ) ) - def get_events(self, wait: Union[bool, float] = False) -> List[QMPMessage]: + def get_events(self, wait: Union[bool, float] = False) -> list[QMPMessage]: """ Get a list of QMP events and clear all pending events.
python/qemu/qmp/message.py +5 -11
@@ -5,16 +5,10 @@ message sent to or from the server. """ +from collections.abc import Iterator, Mapping, MutableMapping import json from json import JSONDecodeError -from typing import ( - Dict, - Iterator, - Mapping, - MutableMapping, - Optional, - Union, -) +from typing import Optional, Union from .error import ProtocolError
@@ -71,7 +65,7 @@ def __init__(self, value: Union[bytes, Mapping[str, object]] = b'{}', *, eager: bool = True): self._data: Optional[bytes] = None - self._obj: Optional[Dict[str, object]] = None + self._obj: Optional[dict[str, object]] = None if isinstance(value, bytes): self._data = value
@@ -125,7 +119,7 @@ def __bytes__(self) -> bytes: # Conversion Methods @property - def _object(self) -> Dict[str, object]: + def _object(self) -> dict[str, object]: """ A `dict` representing this QMP message.
@@ -150,7 +144,7 @@ def _serialize(cls, value: object) -> bytes: return json.dumps(value, separators=(',', ':')).encode('utf-8') @classmethod - def _deserialize(cls, data: bytes) -> Dict[str, object]: + def _deserialize(cls, data: bytes) -> dict[str, object]: """ Deserialize JSON `bytes` into a native Python `dict`.
python/qemu/qmp/models.py +3 -8
@@ -8,14 +8,9 @@ # pylint: disable=too-few-public-methods from collections import abc +from collections.abc import Mapping, Sequence import copy -from typing import ( - Any, - Dict, - Mapping, - Optional, - Sequence, -) +from typing import Any, Optional class Model:
@@ -72,7 +67,7 @@ def __init__(self, raw: Mapping[str, Any]): self._check_member('QMP', abc.Mapping, "JSON object") self.QMP = QMPGreeting(self._raw['QMP']) - def _asdict(self) -> Dict[str, object]: + def _asdict(self) -> dict[str, object]: """ For compatibility with the iotests sync QMP wrapper.
python/qemu/qmp/protocol.py +4 -7
@@ -15,6 +15,7 @@ import asyncio from asyncio import StreamReader, StreamWriter +from collections.abc import AsyncGenerator, Awaitable from contextlib import asynccontextmanager from enum import Enum from functools import wraps
@@ -24,13 +25,9 @@ from ssl import SSLContext from typing import ( Any, - AsyncGenerator, - Awaitable, Callable, Generic, - List, Optional, - Tuple, TypeVar, Union, cast,
@@ -50,7 +47,7 @@ _U = TypeVar('_U') _TaskFN = Callable[[], Awaitable[None]] # aka ``async def func() -> None`` -InternetAddrT = Tuple[str, int] +InternetAddrT = tuple[str, int] UnixAddrT = str SocketAddrT = Union[UnixAddrT, InternetAddrT]
@@ -248,7 +245,7 @@ def __init__(self, name: Optional[str] = None) -> None: self._writer_task: Optional[asyncio.Future[None]] = None # Aggregate of the above two tasks, used for Exception management. - self._bh_tasks: Optional[asyncio.Future[Tuple[None, None]]] = None + self._bh_tasks: Optional[asyncio.Future[tuple[None, None]]] = None #: Disconnect task. The disconnect implementation runs in a task #: so that asynchronous disconnects (initiated by the
@@ -744,7 +741,7 @@ async def _wait_disconnect(self) -> None: assert self.runstate == Runstate.DISCONNECTING assert self._dc_task - aws: List[Awaitable[object]] = [self._dc_task] + aws: list[Awaitable[object]] = [self._dc_task] if self._bh_tasks: aws.insert(0, self._bh_tasks) all_defined_tasks = asyncio.gather(*aws)
python/qemu/qmp/qmp_client.py +4 -10
@@ -8,17 +8,11 @@ """ import asyncio +from collections.abc import Mapping import logging import socket import struct -from typing import ( - Dict, - List, - Mapping, - Optional, - Union, - cast, -) +from typing import Optional, Union, cast from .error import ProtocolError, QMPError from .events import Events
@@ -262,7 +256,7 @@ def __init__(self, name: Optional[str] = None) -> None: self._execute_id = 0 # Incoming RPC reply messages. - self._pending: Dict[ + self._pending: dict[ Union[str, None], 'asyncio.Queue[QMPClient._PendingT]' ] = {}
@@ -339,7 +333,7 @@ async def _negotiate(self) -> None: """ self.logger.debug("Negotiating capabilities ...") - arguments: Dict[str, List[str]] = {} + arguments: dict[str, list[str]] = {} if self._greeting and 'oob' in self._greeting.QMP.capabilities: arguments.setdefault('enable', []).append('oob') msg = self.make_execute_msg('qmp_capabilities', arguments=arguments)
python/qemu/qmp/qmp_shell.py +6 -9
@@ -125,6 +125,7 @@ import argparse import ast +from collections.abc import Iterator, Sequence import json import logging import os
@@ -134,12 +135,8 @@ import sys from typing import ( IO, - Dict, - Iterator, - List, NoReturn, Optional, - Sequence, cast, )
@@ -167,7 +164,7 @@ class QMPCompleter: # NB: Python 3.9+ will probably allow us to subclass list[str] directly, # but pylint as of today does not know that List[str] is simply 'list'. def __init__(self) -> None: - self._matches: List[str] = [] + self._matches: list[str] = [] def append(self, value: str) -> None: """Append a new valid completion to the list of possibilities."""
@@ -229,7 +226,7 @@ def __init__(self, address: SocketAddrT, self._greeting: Optional[QMPMessage] = None self._completer = QMPCompleter() self._transmode = False - self._actions: List[QMPMessage] = [] + self._actions: list[QMPMessage] = [] self._histfile = os.path.join(os.path.expanduser('~'), '.qmp-shell_history') self.pretty = pretty
@@ -246,7 +243,7 @@ def close(self) -> None: def _fill_completion(self) -> None: try: - cmds = cast(List[Dict[str, str]], self.cmd('query-commands')) + cmds = cast(list[dict[str, str]], self.cmd('query-commands')) for cmd in cmds: self._completer.append(cmd['name']) except ExecuteError:
@@ -265,14 +262,14 @@ def _completer_setup(self) -> None: readline.read_history_file(self._histfile) except FileNotFoundError: pass - except IOError as err: + except OSError as err: msg = f"Failed to read history '{self._histfile}': {err!s}" LOG.warning(msg) def _save_history(self) -> None: try: readline.write_history_file(self._histfile) - except IOError as err: + except OSError as err: msg = f"Failed to save history file '{self._histfile}': {err!s}" LOG.warning(msg)
python/qemu/qmp/qmp_tui.py +7 -14
@@ -22,14 +22,7 @@ from logging import Handler, LogRecord import signal import sys -from typing import ( - List, - Optional, - Tuple, - Type, - Union, - cast, -) +from typing import Optional, Union, cast try:
@@ -120,7 +113,7 @@ def format_json(msg: str) -> str: def has_handler_type(logger: logging.Logger, - handler_type: Type[Handler]) -> bool: + handler_type: type[Handler]) -> bool: """ The Logger class has no interface to check if a certain type of handler is installed or not. So we provide an interface to do so.
@@ -151,7 +144,7 @@ class App(QMPClient): :param retry_delay: The delay(sec) before each retry """ - def __init__(self, address: Union[str, Tuple[str, int]], num_retries: int, + def __init__(self, address: Union[str, tuple[str, int]], num_retries: int, retry_delay: Optional[int]) -> None: urwid.register_signal(type(self), UPDATE_MSG) self.window = Window(self)
@@ -448,11 +441,11 @@ class Editor(urwid_readline.ReadlineEdit): def __init__(self, parent: App) -> None: super().__init__(caption='> ', multiline=True) self.parent = parent - self.history: List[str] = [] + self.history: list[str] = [] self.last_index: int = -1 self.show_history: bool = False - def keypress(self, size: Tuple[int, int], key: str) -> Optional[str]: + def keypress(self, size: tuple[int, int], key: str) -> Optional[str]: """ Handles the keypress on this widget.
@@ -530,7 +523,7 @@ def __init__(self, parent: App) -> None: super().__init__(self.history) def add_to_history(self, - history: Union[str, List[Tuple[str, str]]]) -> None: + history: Union[str, list[tuple[str, str]]]) -> None: """ Appends a message to the list and set the focus to the last appended message.
@@ -541,7 +534,7 @@ def add_to_history(self, self.history.append(urwid.Text(history)) self.history.set_focus(len(self.history) - 1) - def mouse_event(self, size: Tuple[int, int], _event: str, button: float, + def mouse_event(self, size: tuple[int, int], _event: str, button: float, _x: int, _y: int, focus: bool) -> None: # Unfortunately there are no urwid constants that represent the mouse # events.
python/qemu/utils/accel.py +3 -3
@@ -17,7 +17,7 @@ import logging import os import subprocess -from typing import List, Optional +from typing import Optional LOG = logging.getLogger(__name__)
@@ -31,7 +31,7 @@ } -def list_accel(qemu_bin: str) -> List[str]: +def list_accel(qemu_bin: str) -> list[str]: """ List accelerators enabled in the QEMU binary.
@@ -43,7 +43,7 @@ def list_accel(qemu_bin: str) -> List[str]: return [] try: out = subprocess.check_output([qemu_bin, '-accel', 'help'], - universal_newlines=True) + text=True) except: LOG.debug("Failed to get the list of accelerators in %s", qemu_bin) raise
python/qemu/utils/qemu_ga_client.py +4 -9
@@ -39,16 +39,11 @@ import argparse import asyncio import base64 +from collections.abc import Sequence import os import random import sys -from typing import ( - Any, - Callable, - Dict, - Optional, - Sequence, -) +from typing import Any, Callable, Optional from qemu.qmp import ConnectError, SocketAddrT from qemu.qmp.legacy import QEMUMonitorProtocol
@@ -76,7 +71,7 @@ def __init__(self, address: SocketAddrT): def sync(self, timeout: Optional[float] = 3) -> None: # Avoid being blocked forever if not self.ping(timeout): - raise EnvironmentError('Agent seems not alive') + raise OSError('Agent seems not alive') uid = random.randint(0, (1 << 32) - 1) while True: ret = self.qga.sync(id=uid)
@@ -159,7 +154,7 @@ def fsfreeze(self, cmd: str) -> object: # Can be int (freeze, thaw) or GuestFsfreezeStatus (status) return getattr(self.qga, 'fsfreeze' + '_' + cmd)() - def fstrim(self, minimum: int) -> Dict[str, object]: + def fstrim(self, minimum: int) -> dict[str, object]: # returns GuestFilesystemTrimResponse ret = getattr(self.qga, 'fstrim')(minimum=minimum) assert isinstance(ret, dict)
python/qemu/utils/qom_common.py +4 -11
@@ -18,14 +18,7 @@ import argparse import os import sys -from typing import ( - Any, - Dict, - List, - Optional, - Type, - TypeVar, -) +from typing import Any, Optional, TypeVar from qemu.qmp import QMPError from qemu.qmp.legacy import QEMUMonitorProtocol
@@ -44,7 +37,7 @@ def __init__(self, name: str, type_: str, self.default_value = default_value @classmethod - def make(cls, value: Dict[str, Any]) -> 'ObjectPropertyInfo': + def make(cls, value: dict[str, Any]) -> 'ObjectPropertyInfo': """ Build an ObjectPropertyInfo from a Dict with an unknown shape. """
@@ -136,7 +129,7 @@ def run(self) -> int: """ raise NotImplementedError - def qom_list(self, path: str) -> List[ObjectPropertyInfo]: + def qom_list(self, path: str) -> list[ObjectPropertyInfo]: """ :return: a strongly typed list from the 'qom-list' command. """
@@ -147,7 +140,7 @@ def qom_list(self, path: str) -> List[ObjectPropertyInfo]: @classmethod def command_runner( - cls: Type[CommandT], + cls: type[CommandT], args: argparse.Namespace ) -> int: """
python/qemu/utils/qom_fuse.py +3 -10
@@ -33,21 +33,14 @@ ## import argparse +from collections.abc import Iterator, Mapping from errno import ENOENT, EPERM import stat import sys -from typing import ( - IO, - Dict, - Iterator, - Mapping, - Optional, - Union, -) +from typing import IO, Optional, Union import fuse from fuse import FUSE, FuseOSError, Operations - from qemu.qmp import ExecuteError from .qom_common import QOMCommand
@@ -79,7 +72,7 @@ def configure_parser(cls, parser: argparse.ArgumentParser) -> None: def __init__(self, args: argparse.Namespace): super().__init__(args) self.mount = args.mount - self.ino_map: Dict[str, int] = {} + self.ino_map: dict[str, int] = {} self.ino_count = 1 def run(self) -> int:
python/scripts/mkvenv.py +9 -16
@@ -58,6 +58,7 @@ # later. See the COPYING file in the top-level directory. import argparse +from collections.abc import Iterator, Sequence from importlib.metadata import ( Distribution, EntryPoint,
@@ -76,15 +77,7 @@ import sys import sysconfig from types import SimpleNamespace -from typing import ( - Any, - Dict, - Iterator, - Optional, - Sequence, - Tuple, - Union, -) +from typing import Any, Optional, Union import venv
@@ -510,7 +503,7 @@ def diagnose( online: bool, wheels_dir: Optional[Union[str, Path]], prog: Optional[str], -) -> Tuple[str, bool]: +) -> tuple[str, bool]: """ Offer a summary to the user as to why a package failed to be installed.
@@ -623,7 +616,7 @@ def pip_install( ) -def _make_version_constraint(info: Dict[str, str], install: bool) -> str: +def _make_version_constraint(info: dict[str, str], install: bool) -> str: """ Construct the version constraint part of a PEP 508 dependency specification (for example '>=0.61.5') from the accepted and
@@ -648,10 +641,10 @@ def _make_version_constraint(info: Dict[str, str], install: bool) -> str: def _do_ensure( - group: Dict[str, Dict[str, str]], + group: dict[str, dict[str, str]], online: bool = False, wheels_dir: Optional[Union[str, Path]] = None, -) -> Optional[Tuple[str, bool]]: +) -> Optional[tuple[str, bool]]: """ Use pip to ensure we have the packages specified in @group.
@@ -716,7 +709,7 @@ def _do_ensure( return None -def _parse_groups(file: str) -> Dict[str, Dict[str, Any]]: +def _parse_groups(file: str) -> dict[str, dict[str, Any]]: if not HAVE_TOMLLIB: if sys.version_info < (3, 11): raise Ouch("found no usable tomli, please install it")
@@ -727,7 +720,7 @@ def _parse_groups(file: str) -> Dict[str, Dict[str, Any]]: # Use loads() to support both tomli v1.2.x (Ubuntu 22.04, # Debian bullseye-backports) and v2.0.x - with open(file, "r", encoding="ascii") as depfile: + with open(file, encoding="ascii") as depfile: contents = depfile.read() return tomllib.loads(contents) # type: ignore
@@ -756,7 +749,7 @@ def ensure_group( parsed_deps = _parse_groups(file) - to_install: Dict[str, Dict[str, str]] = {} + to_install: dict[str, dict[str, str]] = {} for group in groups: try: to_install.update(parsed_deps[group])
python/tests/protocol.py +0 -1
@@ -5,7 +5,6 @@ from tempfile import TemporaryDirectory import avocado - from qemu.qmp import ConnectError, Runstate from qemu.qmp.protocol import AsyncProtocol, StateError
roms/edk2-build.py +9 -8
@@ -4,13 +4,14 @@ https://gitlab.com/kraxel/edk2-build-config """ +import argparse +import configparser import os -import sys -import time import shutil -import argparse import subprocess -import configparser +import sys +import time + rebase_prefix = "" version_override = None
@@ -24,16 +25,16 @@ def check_rebase(): gitdir = '.git' if os.path.isfile(gitdir): - with open(gitdir, 'r', encoding = 'utf-8') as f: + with open(gitdir, encoding = 'utf-8') as f: (unused, gitdir) = f.read().split() if not os.path.exists(f'{gitdir}/rebase-merge/msgnum'): return - with open(f'{gitdir}/rebase-merge/msgnum', 'r', encoding = 'utf-8') as f: + with open(f'{gitdir}/rebase-merge/msgnum', encoding = 'utf-8') as f: msgnum = int(f.read()) - with open(f'{gitdir}/rebase-merge/end', 'r', encoding = 'utf-8') as f: + with open(f'{gitdir}/rebase-merge/end', encoding = 'utf-8') as f: end = int(f.read()) - with open(f'{gitdir}/rebase-merge/head-name', 'r', encoding = 'utf-8') as f: + with open(f'{gitdir}/rebase-merge/head-name', encoding = 'utf-8') as f: head = f.read().strip().split('/') rebase_prefix = f'[ {int(msgnum/2)} / {int(end/2)} - {head[-1]} ] '
scripts/analyse-9p-simpletrace.py +2 -0
@@ -4,8 +4,10 @@ # # Author: Harsh Prateek Bora import os + import simpletrace + symbol_9p = { 6 : 'TLERROR', 7 : 'RLERROR',
scripts/analyse-locks-simpletrace.py +3 -2
@@ -1,14 +1,15 @@ #!/usr/bin/env python3 -# -*- coding: utf-8 -*- # # Analyse lock events and compute statistics # # Author: Alex Bennée <[email protected]> # -import simpletrace import argparse + import numpy as np +import simpletrace + class MutexAnalyser(simpletrace.Analyzer): "A simpletrace Analyser for checking locks."
scripts/analyze-migration.py +23 -25
173 lines not on this page · raw message
scripts/block-coroutine-wrapper.py +2 -2
@@ -23,9 +23,9 @@ along with this program. If not, see <http://www.gnu.org/licenses/>. """ -import sys +from collections.abc import Iterator import re -from typing import Iterator +import sys def gen_header():
scripts/check_sparse.py +5 -4
@@ -5,10 +5,11 @@ # parsing import json -import subprocess import os -import sys import shlex +import subprocess +import sys + def cmdline_for_sparse(sparse, cmdline): # Do not include the C compiler executable
@@ -43,7 +44,7 @@ def build_path(s): return s if not root_path else os.path.join(root_path, s) ccjson_path = build_path(sys.argv[1]) -with open(ccjson_path, 'r') as fd: +with open(ccjson_path) as fd: compile_commands = json.load(fd) sparse = sys.argv[2:]
@@ -52,7 +53,7 @@ def build_path(s): cmdline = shlex.split(cmd['command']) cmd = cmdline_for_sparse(sparse, cmdline) print('REAL_CC=%s' % shlex.quote(cmdline[0]), - ' '.join((shlex.quote(x) for x in cmd))) + shlex.join(cmd)) sparse_env['REAL_CC'] = cmdline[0] r = subprocess.run(cmd, env=sparse_env, cwd=root_path) if r.returncode != 0:
scripts/ci/gitlab-pipeline-status +2 -2
@@ -19,8 +19,8 @@ import http.client import json import os import subprocess -import time import sys +import time class CommunicationFailure(Exception):
@@ -40,7 +40,7 @@ def get_local_branch_commit(branch): stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, cwd=os.path.dirname(__file__), - universal_newlines=True).stdout.strip() + text=True).stdout.strip() if result == branch: raise ValueError("There's no local branch named '%s'" % branch) if len(result) != 40:
scripts/codeconverter/codeconverter/patching.py +41 -30
210 lines not on this page · raw message
scripts/codeconverter/codeconverter/qom_macros.py +8 -7
@@ -5,15 +5,16 @@ # # This work is licensed under the terms of the GNU GPL, version 2. See # the COPYING file in the top-level directory. -import re from itertools import chain +import logging +import re from typing import * -from .regexps import * from .patching import * +from .regexps import * from .utils import * -import logging + logger = logging.getLogger(__name__) DBG = logger.debug INFO = logger.info
@@ -590,16 +591,16 @@ def gen_patches(self) -> Iterable[Patch]: self.debug("checker_dict: %r", checker_dict) for uppercase,checkers in checker_dict.items(): fields = ('instancetype', 'classtype', 'uppercase', 'typename') - fvalues = dict((field, set(getattr(m, field) for m in checkers - if getattr(m, field, None) is not None)) - for field in fields) + fvalues = {field: {getattr(m, field) for m in checkers + if getattr(m, field, None) is not None} + for field in fields} for field,values in fvalues.items(): if len(values) > 1: for c in checkers: c.warn("%s mismatch (%s)", field, ' '.join(values)) return - field_dict = dict((f, v.pop() if v else None) for f,v in fvalues.items()) + field_dict = {f: v.pop() if v else None for f,v in fvalues.items()} yield from self.gen_patches_for_type(uppercase, checkers, field_dict) def find_conflicts(self, uppercase: str, checkers: List[TypeDeclaration]) -> bool:
scripts/codeconverter/codeconverter/qom_type_info.py +6 -4
@@ -6,10 +6,12 @@ # This work is licensed under the terms of the GNU GPL, version 2. See # the COPYING file in the top-level directory. import re -from .regexps import * + from .patching import * -from .utils import * from .qom_macros import * +from .regexps import * +from .utils import * + TI_FIELDS = [ 'name', 'parent', 'abstract', 'interfaces', 'instance_size', 'instance_init', 'instance_post_init', 'instance_finalize',
@@ -89,8 +91,8 @@ def initializers(self) -> Optional[TypeInfoInitializers]: fields = self.group('fields') if fields is None: return None - d = dict((fm.group('field'), fm) - for fm in self.group_finditer(FieldInitializer, 'fields')) + d = {fm.group('field'): fm + for fm in self.group_finditer(FieldInitializer, 'fields')} self._initializers = d # type: ignore return self._initializers
scripts/codeconverter/codeconverter/regexps.py +3 -2
@@ -6,9 +6,10 @@ # This work is licensed under the terms of the GNU GPL, version 2. See # the COPYING file in the top-level directory. """Helpers for creation of regular expressions""" -import re - import logging +import re + + logger = logging.getLogger(__name__) DBG = logger.debug INFO = logger.info
scripts/codeconverter/codeconverter/test_patching.py +8 -1
@@ -6,9 +6,16 @@ # This work is licensed under the terms of the GNU GPL, version 2. See # the COPYING file in the top-level directory. from tempfile import NamedTemporaryFile -from .patching import FileInfo, FileMatch, Patch, FileList + +from .patching import ( + FileInfo, + FileList, + FileMatch, + Patch, +) from .regexps import * + class BasicPattern(FileMatch): regexp = '[abc]{3}'
scripts/codeconverter/codeconverter/test_regexps.py +2 -1
@@ -5,9 +5,10 @@ # # This work is licensed under the terms of the GNU GPL, version 2. See # the COPYING file in the top-level directory. -from .regexps import * from .qom_macros import * from .qom_type_info import * +from .regexps import * + def test_res() -> None: def fullmatch(regexp, s):
scripts/codeconverter/codeconverter/utils.py +3 -2
@@ -5,9 +5,10 @@ # # This work is licensed under the terms of the GNU GPL, version 2. See # the COPYING file in the top-level directory. -from typing import * - import logging +from typing import * + + logger = logging.getLogger(__name__) DBG = logger.debug INFO = logger.info
scripts/codeconverter/converter.py +9 -7
@@ -9,18 +9,20 @@ # This work is licensed under the terms of the GNU GPL, version 2. See # the COPYING file in the top-level directory. # -import sys import argparse -import os -import os.path +import logging import re -from typing import * +import sys -from codeconverter.patching import FileInfo, match_class_dict, FileList +from codeconverter.patching import FileInfo, FileList, match_class_dict import codeconverter.qom_macros -from codeconverter.qom_type_info import TI_FIELDS, type_infos, TypeInfoVar +from codeconverter.qom_type_info import ( + TI_FIELDS, + TypeInfoVar, + type_infos, +) + -import logging logger = logging.getLogger(__name__) DBG = logger.debug INFO = logger.info
scripts/compare-machine-types.py +26 -23
155 lines not on this page · raw message
scripts/coverage/compare_gcov_json.py +4 -3
@@ -15,8 +15,9 @@ import argparse import json -import sys from pathlib import Path +import sys + def create_parser(): parser = argparse.ArgumentParser(
@@ -55,11 +56,11 @@ def load_json(json_file_path: Path, verbose = False) -> dict[str, set[int]]: lines = filecov["lines"] - executed_lines = set( + executed_lines = { linecov["line_number"] for linecov in filecov["lines"] if linecov["count"] != 0 and not linecov["gcovr/noncode"] - ) + } # if this file has any coverage add it to the system if len(executed_lines) > 0:
scripts/cpu-x86-uarch-abi.py +3 -1
@@ -6,9 +6,11 @@ # compatibility levels for each CPU model. # -from qemu.qmp.legacy import QEMUMonitorProtocol import sys +from qemu.qmp.legacy import QEMUMonitorProtocol + + if len(sys.argv) != 2: print("syntax: %s QMP-SOCK\n\n" % __file__ + "Where QMP-SOCK points to a QEMU process such as\n\n" +
scripts/decodetree.py +6 -6
@@ -20,11 +20,12 @@ # See the syntax and semantics in docs/devel/decodetree.rst. # +import getopt import io import os import re import sys -import getopt + insnwidth = 32 bitop_width = 32
@@ -87,7 +88,6 @@ class CycleError(ValueError): """Subclass of ValueError raised if cycles exist in the graph""" - pass class TopologicalSorter: """Topologically sort a graph"""
@@ -113,7 +113,7 @@ def static_order(self): # Add empty dependencies where needed data.update({item:{} for item in extra_items_in_deps}) while True: - ordered = set(item for item, dep in data.items() if not dep) + ordered = {item for item, dep in data.items() if not dep} if not ordered: break r.extend(ordered)
@@ -1559,7 +1559,7 @@ def main(): for filename in args: input_file = filename - f = open(filename, 'rt', encoding='utf-8') + f = open(filename, encoding='utf-8') parse_file(f, toppat) f.close()
@@ -1579,9 +1579,9 @@ def main(): prop_size(stree) if output_null: - output_fd = open(os.devnull, 'wt', encoding='utf-8', errors="ignore") + output_fd = open(os.devnull, 'w', encoding='utf-8', errors="ignore") elif output_file: - output_fd = open(output_file, 'wt', encoding='utf-8') + output_fd = open(output_file, 'w', encoding='utf-8') else: output_fd = io.TextIOWrapper(sys.stdout.buffer, encoding=sys.stdout.encoding,
scripts/device-crash-test +9 -9
@@ -24,16 +24,16 @@ Run QEMU with all combinations of -machine and -device types, check for crashes and unexpected errors. """ -import os -import sys -import glob -import logging -import traceback -import re -import random import argparse from itertools import chain +import logging +import os from pathlib import Path +import random +import re +import sys +import traceback + try: from qemu.machine import QEMUMachine
@@ -286,7 +286,7 @@ def infoQDM(vm): yield d -class QemuBinaryInfo(object): +class QemuBinaryInfo: def __init__(self, binary, devtype): if devtype is None: devtype = 'device'
@@ -303,7 +303,7 @@ class QemuBinaryInfo(object): self.alldevs = set(qomListTypeNames(vm, implements=devtype, abstract=False)) # there's no way to query DeviceClass::user_creatable using QMP, # so use 'info qdm': - self.no_user_devs = set([d['name'] for d in infoQDM(vm, ) if d['no-user']]) + self.no_user_devs = {d['name'] for d in infoQDM(vm, ) if d['no-user']} self.machines = list(m['name'] for m in vm.cmd('query-machines')) self.user_devs = self.alldevs.difference(self.no_user_devs) self.kvm_available = vm.cmd('query-kvm')['enabled']
scripts/dump-guest-memory.py +3 -2
@@ -16,6 +16,7 @@ import ctypes import struct + try: UINTPTR_T = gdb.lookup_type("uintptr_t") except Exception as inst:
@@ -62,7 +63,7 @@ def le32_to_cpu(val): def le64_to_cpu(val): return struct.unpack("<Q", struct.pack("=Q", val))[0] -class ELF(object): +class ELF: """Representation of a ELF file.""" def __init__(self, arch):
@@ -501,7 +502,7 @@ class DumpGuestMemory(gdb.Command): shape and this command should mostly work.""" def __init__(self): - super(DumpGuestMemory, self).__init__("dump-guest-memory", + super().__init__("dump-guest-memory", gdb.COMMAND_DATA, gdb.COMPLETE_FILENAME) self.elf = None
scripts/feature_to_c.py +4 -1
@@ -1,7 +1,10 @@ #!/usr/bin/env python3 # SPDX-License-Identifier: GPL-2.0-or-later -import os, sys, xml.etree.ElementTree +import os +import sys +import xml.etree.ElementTree + def writeliteral(indent, bytes): sys.stdout.write(' ' * indent)
scripts/kvm/kvm_flightrecorder +4 -3
@@ -32,8 +32,9 @@ # consuming CPU cycles. No disk I/O is performed since the ring buffer holds a # fixed-size in-memory trace. -import sys import os +import sys + tracing_dir = '/sys/kernel/debug/tracing'
@@ -60,7 +61,7 @@ def stop_tracing(): write_file(trace_path('current_tracer'), 'nop') def dump_trace(): - tracefile = open(trace_path('trace'), 'r') + tracefile = open(trace_path('trace')) try: lines = True while lines:
@@ -71,7 +72,7 @@ def dump_trace(): def tail_trace(): try: - for line in open(trace_path('trace_pipe'), 'r'): + for line in open(trace_path('trace_pipe')): sys.stdout.write(line) except KeyboardInterrupt: pass
scripts/kvm/vmxcap +3 -3
@@ -26,7 +26,7 @@ MSR_IA32_VMX_VMFUNC = 0x491 MSR_IA32_VMX_PROCBASED_CTLS3 = 0x492 MSR_IA32_VMX_EXIT_CTLS2 = 0x493 -class msr(object): +class msr: def __init__(self): try: self.f = open('/dev/cpu/0/msr', 'rb', 0)
@@ -40,7 +40,7 @@ class msr(object): except: return default -class Control(object): +class Control: def __init__(self, name, bits, cap_msr, true_cap_msr = None): self.name = name self.bits = bits
@@ -80,7 +80,7 @@ class Allowed1Control(Control): val = m.read(nr, 0) return (0, val) -class Misc(object): +class Misc: def __init__(self, name, bits, msr): self.name = name self.bits = bits
scripts/meson-buildoptions.py +2 -1
@@ -21,9 +21,10 @@ # along with this program. If not, see <https://www.gnu.org/licenses/>. import json -import textwrap import shlex import sys +import textwrap + # Options with nonstandard names (e.g. --with/--without) or OS-dependent # defaults. Try not to add any.
scripts/minikconf.py +7 -6
@@ -12,9 +12,10 @@ # the top-level directory. import os -import sys -import re import random +import re +import sys + __all__ = [ 'KconfigDataError', 'KconfigParserError', 'KconfigData', 'KconfigParser' ,
@@ -402,8 +403,8 @@ def do_include(self, include): if incl_abs_fname in self.data.previously_included: return try: - fp = open(incl_abs_fname, 'rt', encoding='utf-8') - except IOError as e: + fp = open(incl_abs_fname, encoding='utf-8') + except OSError as e: raise KconfigParserError(self, '%s: %s' % (e.strerror, include))
@@ -696,7 +697,7 @@ def scan_token(self): parser.do_assignment(name, value == 'y') external_vars.add(name[7:]) else: - fp = open(arg, 'rt', encoding='utf-8') + fp = open(arg, encoding='utf-8') parser.parse_file(fp) fp.close()
@@ -705,7 +706,7 @@ def scan_token(self): if key not in external_vars and config[key]: print ('CONFIG_%s=y' % key) - deps = open(argv[2], 'wt', encoding='utf-8') + deps = open(argv[2], 'w', encoding='utf-8') for fname in data.previously_included: print ('%s: %s' % (argv[1], fname), file=deps) deps.close()
scripts/modinfo-collect.py +3 -4
@@ -1,11 +1,10 @@ #!/usr/bin/env python3 -# -*- coding: utf-8 -*- -import os -import sys import json import shlex import subprocess +import sys + def process_command(src, command): skip = False
@@ -53,7 +52,7 @@ def main(args): cmdline = process_command(src, command) print("MODINFO_DEBUG cmd", cmdline) result = subprocess.run(cmdline, stdout = subprocess.PIPE, - universal_newlines = True) + text = True) if result.returncode != 0: sys.exit(result.returncode) for line in result.stdout.split('\n'):
scripts/modinfo-generate.py +2 -2
@@ -1,9 +1,9 @@ #!/usr/bin/env python3 -# -*- coding: utf-8 -*- import os import sys + def print_array(name, values): if len(values) == 0: return
@@ -109,7 +109,7 @@ def main(args): error = False for dep in deps.difference(modules): - print("Dependency {} cannot be satisfied".format(dep), + print(f"Dependency {dep} cannot be satisfied", file=sys.stderr) error = True
scripts/modules/module_block.py +3 -2
@@ -10,8 +10,9 @@ # This work is licensed under the terms of the GNU GPL, version 2. # See the COPYING file in the top-level directory. -import sys import os +import sys + def get_string_struct(line): data = line.split()
@@ -35,7 +36,7 @@ def add_module(fheader, library, format_name, protocol_name): def process_file(fheader, filename): # This parser assumes the coding style rules are being followed - with open(filename, "r") as cfile: + with open(filename) as cfile: found_start = False library, _ = os.path.splitext(os.path.basename(filename)) for line in cfile:
scripts/mtest2make.py +4 -4
@@ -8,10 +8,10 @@ import itertools import json import os -import shlex import sys -class Suite(object): + +class Suite: def __init__(self): self.deps = set() self.speeds = ['quick']
@@ -66,8 +66,8 @@ def process_tests(test, targets, suites): suites[s].deps.update(deps) def emit_prolog(suites, prefix): - all_targets = ' '.join((f'{prefix}-{k}' for k in suites.keys())) - all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml' for k in suites.keys())) + all_targets = ' '.join(f'{prefix}-{k}' for k in suites.keys()) + all_xml = ' '.join(f'{prefix}-report-{k}.junit.xml' for k in suites.keys()) print() print(f'all-{prefix}-targets = {all_targets}') print(f'all-{prefix}-xml = {all_xml}')
scripts/oss-fuzz/minimize_qtest_trace.py +9 -9
@@ -1,15 +1,15 @@ #!/usr/bin/env python3 -# -*- coding: utf-8 -*- """ This takes a crashing qtest trace and tries to remove superfluous operations """ -import sys import os +import struct import subprocess +import sys import time -import struct + QEMU_ARGS = None QEMU_PATH = None
@@ -41,7 +41,7 @@ def usage(): timing dependent instructions. Off by default. -M2: try setting bits in operand of write/out to zero. Off by default. -""".format((sys.argv[0]))) +""".format(sys.argv[0])) deduplication_note = """\n\ Note: While trimming the input, sometimes the mutated trace triggers a different
@@ -71,7 +71,7 @@ def check_if_trace_crashes(trace, path): except subprocess.TimeoutExpired: print("subprocess.TimeoutExpired") return False - print("Identifying Crashes by this string: {}".format(CRASH_TOKEN)) + print(f"Identifying Crashes by this string: {CRASH_TOKEN}") global deduplication_note print(deduplication_note) return True
@@ -136,7 +136,7 @@ def remove_lines(newtrace, outpath): prior = newtrace[i:i+remove_step] for j in range(i, i+remove_step): newtrace[j] = "" - print("Removing {lines} ...\n".format(lines=prior)) + print(f"Removing {prior} ...\n") if check_if_trace_crashes(newtrace, outpath): i += remove_step # Double the number of lines to remove for next round
@@ -247,7 +247,7 @@ def clear_bits(newtrace, outpath): continue # write ADDR SIZE DATA # outx ADDR VALUE - print("\nzero setting bits: {}".format(newtrace[i])) + print(f"\nzero setting bits: {newtrace[i]}") prefix = " ".join(newtrace[i].split()[:-1]) data = newtrace[i].split()[-1]
@@ -281,9 +281,9 @@ def minimize_trace(inpath, outpath): if not check_if_trace_crashes(trace, outpath): sys.exit("The input qtest trace didn't cause a crash...") end = time.time() - print("Crashed in {} seconds".format(end-start)) + print(f"Crashed in {end-start} seconds") TIMEOUT = (end-start)*5 - print("Setting the timeout for {} seconds".format(TIMEOUT)) + print(f"Setting the timeout for {TIMEOUT} seconds") newtrace = trace[:] global M1, M2
scripts/oss-fuzz/output_reproducer.py +9 -9
@@ -1,5 +1,4 @@ #!/usr/bin/env python3 -# -*- coding: utf-8 -*- """ Convert plain qtest traces to C or Bash reproducers
@@ -10,11 +9,12 @@ or similar """ -import sys -import os import argparse -import textwrap from datetime import date +import os +import sys +import textwrap + __author__ = "Alexander Bulekov <[email protected]>" __copyright__ = "Copyright (C) 2021, Red Hat, Inc."
@@ -47,10 +47,10 @@ def c_comment(s): def print_c_function(s): print("/* ") for l in s.splitlines(): - print(" * {}".format(l)) + print(f" * {l}") def bash_reproducer(path, args, trace): - result = '\\\n'.join(textwrap.wrap("cat << EOF | {} {}".format(path, args), + result = '\\\n'.join(textwrap.wrap(f"cat << EOF | {path} {args}", 72, break_on_hyphens=False, drop_whitespace=False)) for l in trace.splitlines():
@@ -60,14 +60,14 @@ def bash_reproducer(path, args, trace): def c_reproducer(name, args, trace): result = [] - result.append("""static void {}(void)\n{{""".format(name)) + result.append(f"""static void {name}(void)\n{{""") # libqtest will add its own qtest args, so get rid of them args = args.replace("-accel qtest","") args = args.replace(",accel=qtest","") args = args.replace("-machine accel=qtest","") args = args.replace("-qtest stdio","") - result.append("""QTestState *s = qtest_init("{}");""".format(args)) + result.append(f"""QTestState *s = qtest_init("{args}");""") for l in trace.splitlines(): param = l.split() cmd = param[0]
@@ -90,7 +90,7 @@ def c_reproducer(name, args, trace): if len(param) ==1: result.append("qtest_clock_step_next(s);") else: - result.append("qtest_clock_step(s, {});".format(param[1])) + result.append(f"qtest_clock_step(s, {param[1]});") result.append("qtest_quit(s);\n}") return "\n".join(result)
scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py +3 -3
@@ -1,5 +1,4 @@ #!/usr/bin/env python3 -# -*- coding: utf-8 -*- """ Use this to convert qtest log info from a generic fuzzer input into a qtest
@@ -64,6 +63,7 @@ import sys + __author__ = "Alexander Bulekov <[email protected]>" __copyright__ = "Copyright (C) 2020, Red Hat, Inc." __license__ = "GPL version 2 or (at your option) any later version"
@@ -73,11 +73,11 @@ def usage(): - sys.exit("Usage: {} /path/to/qtest_log_output".format((sys.argv[0]))) + sys.exit(f"Usage: {(sys.argv[0])} /path/to/qtest_log_output") def main(filename): - with open(filename, "r") as f: + with open(filename) as f: trace = f.readlines() # Leave only lines that look like logged qtest commands
scripts/performance/dissect.py +1 -1
@@ -108,7 +108,7 @@ def main(): # Read the callgrind_annotate output to callgrind_data[] callgrind_data = [] - with open(annotate_out_path, 'r') as data: + with open(annotate_out_path) as data: callgrind_data = data.readlines() # Line number with the total number of instructions
scripts/performance/topN_callgrind.py +1 -1
@@ -83,7 +83,7 @@ # Read the callgrind_annotate output to callgrind_data[] callgrind_data = [] -with open('/tmp/callgrind_annotate.out', 'r') as data: +with open('/tmp/callgrind_annotate.out') as data: callgrind_data = data.readlines() # Line number with the total number of instructions
scripts/performance/topN_perf.py +1 -1
@@ -111,7 +111,7 @@ # Read the reported data to functions[] functions = [] -with open("/tmp/perf_report.out", "r") as data: +with open("/tmp/perf_report.out") as data: # Only read lines that are not comments (comments start with #) # Only read lines that are not empty functions = [line for line in data.readlines() if line and line[0]
scripts/probe-gdb-support.py +2 -2
@@ -1,5 +1,4 @@ #!/usr/bin/env python3 -# coding: utf-8 # # Probe gdb for supported architectures. #
@@ -19,9 +18,10 @@ import argparse import re -from subprocess import check_output, STDOUT, CalledProcessError +from subprocess import STDOUT, CalledProcessError, check_output import sys + # Mappings from gdb arch to QEMU target MAP = { "alpha" : ["alpha"],
scripts/python_qmp_updater.py +1 -0
@@ -9,6 +9,7 @@ import sys from typing import Optional + start_reg = re.compile(r'^(?P<padding> *)(?P<res>\w+) = (?P<vm>.*).qmp\(', flags=re.MULTILINE)
scripts/qapi-gen.py +1 -0
@@ -15,5 +15,6 @@ from qapi import main + if __name__ == '__main__': sys.exit(main.main())
scripts/qapi/commands.py +4 -9
@@ -13,12 +13,7 @@ See the COPYING file in the top-level directory. """ -from typing import ( - Dict, - List, - Optional, - Set, -) +from typing import Optional from .common import c_name, mcgen from .gen import (
@@ -276,7 +271,7 @@ def gen_marshal(name: str, def gen_register_command(name: str, - features: List[QAPISchemaFeature], + features: list[QAPISchemaFeature], success_response: bool, allow_oob: bool, allow_preconfig: bool,
@@ -308,7 +303,7 @@ def __init__(self, prefix: str, gen_tracing: bool): prefix, 'qapi-commands', ' * Schema-defined QAPI/QMP commands', None, __doc__, gen_tracing=gen_tracing) - self._visited_ret_types: Dict[QAPIGenC, Set[QAPISchemaType]] = {} + self._visited_ret_types: dict[QAPIGenC, set[QAPISchemaType]] = {} self._gen_tracing = gen_tracing def _begin_user_module(self, name: str) -> None:
@@ -375,7 +370,7 @@ def visit_command(self, name: str, info: Optional[QAPISourceInfo], ifcond: QAPISchemaIfCond, - features: List[QAPISchemaFeature], + features: list[QAPISchemaFeature], arg_type: Optional[QAPISchemaObjectType], ret_type: Optional[QAPISchemaType], gen: bool,
scripts/qapi/common.py +17 -22
@@ -11,15 +11,10 @@ # This work is licensed under the terms of the GNU GPL, version 2. # See the COPYING file in the top-level directory. +from collections.abc import Sequence import re -from typing import ( - Any, - Dict, - Match, - Optional, - Sequence, - Union, -) +from re import Match +from typing import Any, Optional, Union #: Magic string that gets removed along with all space to its right.
@@ -95,22 +90,22 @@ def c_name(name: str, protect: bool = True) -> str: (like C keywords) by prepending ``q_``. """ # ANSI X3J11/88-090, 3.1.1 - c89_words = set(['auto', 'break', 'case', 'char', 'const', 'continue', + c89_words = {'auto', 'break', 'case', 'char', 'const', 'continue', 'default', 'do', 'double', 'else', 'enum', 'extern', 'float', 'for', 'goto', 'if', 'int', 'long', 'register', 'return', 'short', 'signed', 'sizeof', 'static', 'struct', 'switch', 'typedef', 'union', 'unsigned', - 'void', 'volatile', 'while']) + 'void', 'volatile', 'while'} # ISO/IEC 9899:1999, 6.4.1 - c99_words = set(['inline', 'restrict', '_Bool', '_Complex', '_Imaginary']) + c99_words = {'inline', 'restrict', '_Bool', '_Complex', '_Imaginary'} # ISO/IEC 9899:2011, 6.4.1 - c11_words = set(['_Alignas', '_Alignof', '_Atomic', '_Generic', - '_Noreturn', '_Static_assert', '_Thread_local']) + c11_words = {'_Alignas', '_Alignof', '_Atomic', '_Generic', + '_Noreturn', '_Static_assert', '_Thread_local'} # GCC http://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/C-Extensions.html # excluding _.* - gcc_words = set(['asm', 'typeof']) + gcc_words = {'asm', 'typeof'} # C++ ISO/IEC 14882:2003 2.11 - cpp_words = set(['bool', 'catch', 'class', 'const_cast', 'delete', + cpp_words = {'bool', 'catch', 'class', 'const_cast', 'delete', 'dynamic_cast', 'explicit', 'false', 'friend', 'mutable', 'namespace', 'new', 'operator', 'private', 'protected', 'public', 'reinterpret_cast', 'static_cast', 'template',
@@ -118,9 +113,9 @@ def c_name(name: str, protect: bool = True) -> str: 'using', 'virtual', 'wchar_t', # alternative representations 'and', 'and_eq', 'bitand', 'bitor', 'compl', 'not', - 'not_eq', 'or', 'or_eq', 'xor', 'xor_eq']) + 'not_eq', 'or', 'or_eq', 'xor', 'xor_eq'} # namespace pollution: - polluted_words = set(['unix', 'errno', 'mips', 'sparc', 'i386', 'linux']) + polluted_words = {'unix', 'errno', 'mips', 'sparc', 'i386', 'linux'} name = re.sub(r'[^A-Za-z0-9_]', '_', name) if protect and (name in (c89_words | c99_words | c11_words | gcc_words | cpp_words | polluted_words)
@@ -139,7 +134,7 @@ def __init__(self, initial: int = 0) -> None: self._level = initial def __repr__(self) -> str: - return "{}({:d})".format(type(self).__name__, self._level) + return f"{type(self).__name__}({self._level:d})" def __str__(self) -> str: """Return the current indentation as a string of spaces."""
@@ -199,11 +194,11 @@ def guardend(name: str) -> str: name=c_fname(name).upper()) -def gen_ifcond(ifcond: Optional[Union[str, Dict[str, Any]]], +def gen_ifcond(ifcond: Optional[Union[str, dict[str, Any]]], cond_fmt: str, not_fmt: str, all_operator: str, any_operator: str) -> str: - def do_gen(ifcond: Union[str, Dict[str, Any]], + def do_gen(ifcond: Union[str, dict[str, Any]], need_parens: bool) -> str: if isinstance(ifcond, str): return cond_fmt % ifcond
@@ -226,11 +221,11 @@ def gen_infix(operator: str, operands: Sequence[Any]) -> str: return do_gen(ifcond, False) -def cgen_ifcond(ifcond: Optional[Union[str, Dict[str, Any]]]) -> str: +def cgen_ifcond(ifcond: Optional[Union[str, dict[str, Any]]]) -> str: return gen_ifcond(ifcond, 'defined(%s)', '!%s', ' && ', ' || ') -def docgen_ifcond(ifcond: Optional[Union[str, Dict[str, Any]]]) -> str: +def docgen_ifcond(ifcond: Optional[Union[str, dict[str, Any]]]) -> str: # TODO Doc generated for conditions needs polish return gen_ifcond(ifcond, '%s', 'not %s', ' and ', ' or ')
scripts/qapi/error.py +0 -1
@@ -1,4 +1,3 @@ -# -*- coding: utf-8 -*- # # Copyright (c) 2017-2019 Red Hat Inc. #
scripts/qapi/events.py +4 -4
@@ -12,7 +12,7 @@ See the COPYING file in the top-level directory. """
Lines of this file are not shown. View the raw message.
scripts/qapi/introspect.py +20 -22
42 lines not on this page · raw message
scripts/qapi/parser.py +20 -24
44 lines not on this page · raw message
scripts/qapi/schema.py +131 -140
271 lines not on this page · raw message
scripts/qom-cast-macro-clean-cocci-gen.py +2 -1
Lines of this file are not shown. View the raw message.
scripts/simplebench/img_bench_templater.py +5 -5
Lines of this file are not shown. View the raw message.
scripts/tracetool/format/simpletrace_stap.py +1 -2
Lines of this file are not shown. View the raw message.
scripts/tracetool/format/ust_events_c.py +0 -2
Lines of this file are not shown. View the raw message.
scripts/tracetool/format/ust_events_h.py +0 -2
Lines of this file are not shown. View the raw message.
target/hexagon/gen_idef_parser_funcs.py +0 -4
Lines of this file are not shown. View the raw message.
tests/functional/qemu_test/__init__.py +22 -8
Lines of this file are not shown. View the raw message.
tests/functional/qemu_test/decorators.py +1 -0
Lines of this file are not shown. View the raw message.
tests/functional/qemu_test/linuxkernel.py +4 -1
Lines of this file are not shown. View the raw message.
tests/functional/qemu_test/tuxruntest.py +8 -4
Lines of this file are not shown. View the raw message.
tests/functional/qemu_test/uncompress.py +2 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_aspeed_ast2700.py +6 -3
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_aspeed_ast2700fc.py +6 -3
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_imx8mp_evk.py +2 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_raspi3.py +1 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_raspi4.py +5 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_replay.py +2 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_reverse_debug.py +1 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_rme_sbsaref.py +6 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_rme_virt.py +8 -3
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_sbsaref.py +6 -3
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_sbsaref_alpine.py +6 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_sbsaref_freebsd.py +6 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_tcg_plugins.py +2 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_tuxrun.py +1 -0
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_virt_gpu.py +3 -4
Lines of this file are not shown. View the raw message.
tests/functional/test_aarch64_xlnx_versal.py +2 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_aspeed_ast1030.py +5 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_aspeed_ast2500.py +1 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_aspeed_ast2600.py +7 -4
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_aspeed_bletchley.py +1 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_aspeed_palmetto.py +1 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_aspeed_rainier.py +2 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_aspeed_romulus.py +1 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_aspeed_witherspoon.py +1 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_canona1100.py +1 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_cubieboard.py +7 -3
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_emcraft_sf2.py +6 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_integratorcp.py +8 -4
Lines of this file are not shown. View the raw message.
tests/functional/test_arm_quanta_gsj.py +7 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_empty_cpu_model.py +1 -0
Lines of this file are not shown. View the raw message.
tests/functional/test_loongarch64_virt.py +7 -3
Lines of this file are not shown. View the raw message.
tests/functional/test_m68k_mcf5208evb.py +1 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_mem_addr_space.py +3 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_microblaze_s3adsp1800.py +6 -3
Lines of this file are not shown. View the raw message.
tests/functional/test_mips64el_fuloong2e.py +8 -3
Lines of this file are not shown. View the raw message.
tests/functional/test_mips64el_loongson3v.py +6 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_mips64el_malta.py +11 -6
Lines of this file are not shown. View the raw message.
tests/functional/test_mips64el_tuxrun.py +1 -0
Lines of this file are not shown. View the raw message.
tests/functional/test_netdev_ethtool.py +3 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_pc_cpu_hotplug_props.py +1 -0
Lines of this file are not shown. View the raw message.
tests/functional/test_ppc64_reverse_debug.py +1 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_ppc_virtex_ml507.py +1 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_riscv32_tuxrun.py +1 -0
Lines of this file are not shown. View the raw message.
tests/functional/test_riscv64_tuxrun.py +1 -0
Lines of this file are not shown. View the raw message.
tests/functional/test_s390x_ccw_virtio.py +6 -3
Lines of this file are not shown. View the raw message.
tests/functional/test_s390x_topology.py +7 -4
Lines of this file are not shown. View the raw message.
tests/functional/test_sparc64_tuxrun.py +1 -0
Lines of this file are not shown. View the raw message.
tests/functional/test_virtio_balloon.py +7 -3
Lines of this file are not shown. View the raw message.
tests/functional/test_virtio_version.py +1 -0
Lines of this file are not shown. View the raw message.
tests/functional/test_x86_64_hotplug_blk.py +5 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_x86_64_hotplug_cpu.py +5 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_x86_64_kvm_xen.py +6 -2
Lines of this file are not shown. View the raw message.
tests/functional/test_x86_64_reverse_debug.py +1 -1
Lines of this file are not shown. View the raw message.
tests/functional/test_x86_cpu_model_versions.py +7 -6
Lines of this file are not shown. View the raw message.
tests/migration-stress/guestperf-batch.py +1 -0
Lines of this file are not shown. View the raw message.
tests/migration-stress/guestperf-plot.py +1 -0
Lines of this file are not shown. View the raw message.
tests/migration-stress/guestperf/comparison.py +2 -1
Lines of this file are not shown. View the raw message.
tests/migration-stress/guestperf/engine.py +5 -3
Lines of this file are not shown. View the raw message.
tests/migration-stress/guestperf/hardware.py +1 -1
Lines of this file are not shown. View the raw message.
tests/migration-stress/guestperf/plot.py +2 -2
Lines of this file are not shown. View the raw message.
tests/migration-stress/guestperf/progress.py +2 -2
Lines of this file are not shown. View the raw message.
tests/migration-stress/guestperf/report.py +5 -4
Lines of this file are not shown. View the raw message.
tests/migration-stress/guestperf/scenario.py +1 -1
Lines of this file are not shown. View the raw message.
tests/migration-stress/guestperf/shell.py +9 -9
Lines of this file are not shown. View the raw message.
tests/migration-stress/guestperf/timings.py +2 -2
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/iotests.py +38 -28
66 lines not on this page · raw message
tests/qemu-iotests/nbd-fault-injector.py +7 -6
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/block-status-cache +1 -0
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/export-incoming-iothread +1 -0
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/graph-changes-while-io +9 -2
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/image-fleecing +1 -0
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/inactive-node-nbd +6 -2
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/iothreads-commit-active +2 -0
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/iothreads-create +2 -0
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/iothreads-nbd-export +3 -0
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/iothreads-stream +2 -0
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/luks-detached-header +3 -2
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test +6 -4
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/migrate-bitmaps-test +1 -1
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/migrate-during-backup +1 -0
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/mirror-change-copy-mode +2 -1
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/mirror-ready-cancel-error +1 -0
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/nbd-reconnect-on-open +9 -2
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/parallels-read-bitmap +7 -1
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/remove-bitmap-from-backing +7 -1
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/stream-error-on-reset +7 -1
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/stream-unaligned-prefetch +10 -3
Lines of this file are not shown. View the raw message.
tests/qemu-iotests/tests/stream-under-throttle +2 -2
Lines of this file are not shown. View the raw message.
tests/tcg/aarch64/gdbstub/test-sve-ioctl.py +2 -2
Lines of this file are not shown. View the raw message.
tests/tcg/multiarch/gdbstub/catch-syscalls.py +3 -3
Lines of this file are not shown. View the raw message.
tests/tcg/multiarch/gdbstub/follow-fork-mode-child.py +2 -2
Lines of this file are not shown. View the raw message.
tests/tcg/multiarch/gdbstub/follow-fork-mode-parent.py +1 -1
Lines of this file are not shown. View the raw message.
tests/tcg/multiarch/gdbstub/interrupt.py +0 -1
Lines of this file are not shown. View the raw message.
tests/tcg/multiarch/gdbstub/late-attach.py +2 -2
Lines of this file are not shown. View the raw message.
tests/tcg/multiarch/gdbstub/prot-none.py +3 -2
Lines of this file are not shown. View the raw message.
tests/tcg/multiarch/gdbstub/registers.py +2 -1
Lines of this file are not shown. View the raw message.
tests/tcg/multiarch/gdbstub/test-proc-mappings.py +0 -1
Lines of this file are not shown. View the raw message.
tests/tcg/multiarch/gdbstub/test-qxfer-auxv-read.py +0 -1
Lines of this file are not shown. View the raw message.
tests/tcg/multiarch/gdbstub/test-qxfer-siginfo-read.py +1 -1
Lines of this file are not shown. View the raw message.
tests/tcg/multiarch/gdbstub/test-thread-breakpoint.py +0 -1
Lines of this file are not shown. View the raw message.
tests/tcg/multiarch/system/validate-memory-counts.py +4 -3
Lines of this file are not shown. View the raw message.
tests/tcg/s390x/gdbstub/test-signals-s390x.py +0 -2
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.48.1
Part of the diff is not shown here. View the raw message.
Re: [PATCH v2 06/12] python: upgrade to python3.9+ syntax
Quoted 69 lines
scripts/simpletrace.py +0
-2@@ -9,13 +9,15 @@ # # For help see docs/devel/tracing.rst -import sys -import structimport inspect
scripts/simpletrace.py +2
-0+import struct +import sysimport warnings
scripts/simpletrace.py +2
-1-from tracetool import read_events, Event + +from tracetool import Event, read_eventsfrom tracetool.backend.simple import
is_string+
__all__= ['Analyzer', 'Analyzer2', 'process', 'run']# This is the binary format that the QEMU "simple" trace backend
scripts/simpletrace.py +2
-6@@ -166,11 +168,9 @@ def runstate_set(self, timestamp, pid, new_state): def begin(self): """Called at the start of the trace.""" - pass def catchall(self, event, rec): """Called if no specific method for processing a trace event has been found.""" - pass def _build_fn(self, event): fn = getattr(self, event.name, None)@@ -208,7 +208,6 @@ def _process_event(self, rec_args, *, event, event_id, timestamp_ns, pid, **kwar def end(self): """Called at the end of the trace.""" - pass def __enter__(self): self.begin()@@ -263,7 +262,6 @@ def runstate_set(self, new_state, *, timestamp_ns, pid, **kwargs): def catchall(self, *rec_args, event, timestamp_ns, pid, event_id, **kwargs): """Called if no specific method for processing a trace event has been found.""" - pass def _process_event(self, rec_args, *, event, **kwargs): fn = getattr(self, event.name, self.catchall)@@ -279,7 +277,7 @@ def process(events, log, analyzer, read_header=True): """ if isinstance(events, str): - with open(events, 'r') as f: + with open(events) as f: events_list = read_events(f, events) elif isinstance(events, list): # Treat as a list of events already produced by tracetool.read_events@@ -332,7 +330,7 @@ def run(analyzer): except (AssertionError, ValueError): raise SimpleException(f'usage: {sys.argv[0]} [--no-header] <trace-events> <trace-file>\n') - with open(trace_event_path, 'r') as events_fobj, open(trace_file_path, 'rb') as log_fobj: + with open(trace_event_path) as events_fobj, open(trace_file_path, 'rb') as log_fobj: process(events_fobj, log_fobj, analyzer, read_header=not no_header)
Quoted 1 line
if
__name__== '__main__':
I'm not really a fan of the implicit default arguments, but I guess the rest is fine. If this is the way everyone else wants to go, I won't stand in the way.
—
Mads Ynddal
Re: [PATCH v2 06/12] python: upgrade to python3.9+ syntax
Quoted 76 lines · Mads Ynddal
Quoted 25 lines
scripts/simpletrace.py +0
-2@@ -9,13 +9,15 @@ # # For help see docs/devel/tracing.rst -import sys -import structimport inspect
scripts/simpletrace.py +2
-0+import struct +import sysimport warnings
scripts/simpletrace.py +2
-1-from tracetool import read_events, Event + +from tracetool import Event, read_eventsfrom tracetool.backend.simple import
is_string+
__all__= ['Analyzer', 'Analyzer2', 'process', 'run']# This is the binary format that the QEMU "simple" trace backend
scripts/simpletrace.py def runstate_set(self, timestamp, pid, new_state): · line 173
Quoted 7 lines
scripts/simpletrace.py +0
-1@@ -166,11 +168,9 @@ def runstate_set(self, timestamp, pid, new_state): def begin(self): """Called at the start of the trace.""" - pass def catchall(self, event, rec): """Called if no specific method for processing a trace event hasbeen found."""
scripts/simpletrace.py def _process_event(self, rec_args, *, event,
Quoted 5 lines
scripts/simpletrace.py +0
-1- pass def _build_fn(self, event): fn = getattr(self, event.name, None)@@ -208,7 +208,6 @@ def _process_event(self, rec_args, *, event,
event_id,timestamp_ns, pid, **kwarscripts/simpletrace.py def runstate_set(self, new_state, *, timestamp_ns,
Quoted 8 lines
scripts/simpletrace.py +0
-1def end(self): """Called at the end of the trace.""" - pass def __enter__(self): self.begin()@@ -263,7 +262,6 @@ def runstate_set(self, new_state, *, timestamp_ns,pid, **kwargs):
scripts/simpletrace.py def runstate_set(self, new_state, *, timestamp_ns, · line 262
Quoted 2 lines
scripts/simpletrace.py +0
-0def catchall(self, *rec_args, event, timestamp_ns, pid, event_id,**kwargs):
scripts/simpletrace.py def runstate_set(self, new_state, *, timestamp_ns, · line 263
Quoted 1 line
scripts/simpletrace.py +0
-0"""Called if no specific method for processing a trace event hasbeen found."""
scripts/simpletrace.py def process(events, log, analyzer, read_header=True): · line 283
Quoted 13 lines
scripts/simpletrace.py +1
-2- pass def _process_event(self, rec_args, *, event, **kwargs): fn = getattr(self, event.name, self.catchall)@@ -279,7 +277,7 @@ def process(events, log, analyzer, read_header=True): """ if isinstance(events, str): - with open(events, 'r') as f: + with open(events) as f: events_list = read_events(f, events) elif isinstance(events, list): # Treat as a list of events already produced bytracetool.read_events
scripts/simpletrace.py def run(analyzer): · line 331
Quoted 3 lines
scripts/simpletrace.py +0
-0@@ -332,7 +330,7 @@ def run(analyzer): except (AssertionError, ValueError): raise SimpleException(f'usage: {sys.argv[0]} [--no-header]<trace-events> <trace-file>\n')
scripts/simpletrace.py def run(analyzer): · line 334
Quoted 2 lines
scripts/simpletrace.py +0
-1- with open(trace_event_path, 'r') as events_fobj,
open(trace_file_path, 'rb')aslog_fobj:scripts/simpletrace.py def run(analyzer): · line 332
Quoted 1 line
scripts/simpletrace.py +1
-0+ with open(trace_event_path) as events_fobj, open(trace_file_path,'rb') as
log_fobj:
Quoted 9 lines
scripts/simpletrace.py def run(analyzer): · line 333
Quoted 1 line
scripts/simpletrace.py +0
-0process(events_fobj, log_fobj, analyzer, read_header=not
no_header)Quoted 2 lines
if
__name__== '__main__':I'm not really a fan of the implicit default arguments, but I guess the rest is fine. If this is the way everyone else wants to go, I won't stand in the way.
I can manually restore it. It just happens to be the style the upgrade tool prefers which saves an awful lot of manual work, but I have no prejudice against explicit open parameters.
Thanks for reviewing!
Quoted 4 lines
—
Mads Ynddal
Re: [PATCH v2 06/12] python: upgrade to python3.9+ syntax
In reply to John SnowQuoted 36 lines · John Snow
This patch is fully automated, using pymagic, isort and autoflake.
Create a script named
pymagic.sh:=========================
pyupgrade --exit-zero-even-if-changed --keep-percent-format \ --py39-plus "$@"autoflake -i "$@"
isort --settings-file python/setup.cfg \
-p compat -pqapidoc_legacy-p iotests -o qemu "$@"
=========================Then, from qemu.git root:
Quoted 2 lines
find . -type f -name '*.py' | xargs pymagic
git grep --name-only "#!/usr/bin/env python" | xargs pymagicThis changes a lot of old Pythonisms, but in particular it upgrades the old Python type hint paradigm to the new 3.9+ paradigm wherein you no longer need to import List, Dict, Tuple, Set, etc from the Typing module and instead directly subscript the built-in types list, dict, tuple, set, etc. The old-style annotations are deprecated as of 3.9 and are eligible for removal starting in Python 3.14, though the exact date of their removal is not yet known.
pyupgrade updates the imports and type hint paradigms (as well as updating other old 'isms, such as removing the unicode string prefix). autoflake in turn then removes any unused import statements, possibly left behind by pyupgrade. Lastly, isort fixes the import order and formatting to the standard we use in qemu.git/python and
scripts/qapiin particular.Signed-off-by John Snow <[email protected]>
[...]
Quoted 1 line
448 files changed, 1959 insertions(+), 1631 deletions(-)
*Ächz*
I hate it when people ask me to split up my mechanical patches...
One split is by subsystem / maintainer. I've done this a few times, and it's quite a bother. Questionable use of your time if you ask me.
There's another split here... Your pymagic.sh runs three tools. If you commit after each one, the patch splits into three.
I understand you pass --py39-plus to pyupgrade to get the type hints modernized. If you run it without --py39-plus for all the miscellaneous upgrades, commit, then run it with --py39-plus for just the type hint upgrades, commit, the last patch splits again.
Thoughts?
Re: [PATCH v2 06/12] python: upgrade to python3.9+ syntax
Quoted 37 lines · Markus Armbruster
Quoted 35 lines · John Snow
This patch is fully automated, using pymagic, isort and autoflake.
Create a script named
pymagic.sh:=========================
pyupgrade --exit-zero-even-if-changed --keep-percent-format \ --py39-plus "$@"autoflake -i "$@"
isort --settings-file python/setup.cfg \
-p compat -pqapidoc_legacy-p iotests -o qemu "$@"
=========================Then, from qemu.git root:
Quoted 2 lines
find . -type f -name '*.py' | xargs pymagic
git grep --name-only "#!/usr/bin/env python" | xargs pymagicThis changes a lot of old Pythonisms, but in particular it upgrades the old Python type hint paradigm to the new 3.9+ paradigm wherein you no longer need to import List, Dict, Tuple, Set, etc from the Typing module and instead directly subscript the built-in types list, dict, tuple, set, etc. The old-style annotations are deprecated as of 3.9 and are eligible for removal starting in Python 3.14, though the exact date of their removal is not yet known.
pyupgrade updates the imports and type hint paradigms (as well as updating other old 'isms, such as removing the unicode string prefix). autoflake in turn then removes any unused import statements, possibly left behind by pyupgrade. Lastly, isort fixes the import order and formatting to the standard we use in qemu.git/python and
scripts/qapiin particular.
Quoted 8 lines
Quoted 1 line
Signed-off-by John Snow <[email protected]>[...]
Quoted 1 line
448 files changed, 1959 insertions(+), 1631 deletions(-)*Ächz*
Gesundheit.
Quoted 6 lines
I hate it when people ask me to split up my mechanical patches...
One split is by subsystem / maintainer. I've done this a few times, and it's quite a bother. Questionable use of your time if you ask me.
I'd prefer not to unless it is requested of me specifically. I don't think most maintainers really care about the nuances of Python and as long as their stuff continues to work they're not going to mind much.
Or, to be frank: I don't think this series would ever garner enough review and attention to warrant the labor it'd take to tailor it to such a review. It's mechanical, it's boring, it should be fine.
I switched from a manual patch series to a tool-driven one specifically to make it more mindless and less interesting, and going through and splitting it back out is ... eh. I would prefer not to.
Quoted 4 lines
There's another split here... Your
pymagic.shruns three tools. If you commit after each one, the patch splits into three.
I use all three because each one alone isn't sufficient to then pass the static analysis checks, they each do a little bit of damage that another tool corrects afterwards.
pyupgrade works to modernize syntax, but leaves impotent import statements hanging.
autoflake removes those impotent imports.
isort fixes the import statement ordering and formatting to our standard. (And then I do some manual fixups to fix the linting tests where things were auto-formatted suboptimally.)
I can still split it out for review purposes, like I did here with some manual fixups appended to the end.
Just, for merge, they'll be combined by necessity as a result of our no-regressions-for-bisect rule.
Quoted 6 lines
I understand you pass --py39-plus to pyupgrade to get the type hints modernized. If you run it without --py39-plus for all the miscellaneous upgrades, commit, then run it with --py39-plus for just the type hint upgrades, commit, the last patch splits again.
I can try it! I actually didn't try running it without py39-plus at all, so I don't know what that'll do. but no harm in an experiment.
Quoted 2 lines
Thoughts?
First and foremost I just thought it'd be good to get this mechanical change squared away in one giant patch so we could add this one singular horrible mega-commit into the git blame "ignored commits" list to minimize the impact of the "flag day".
This upgrade will have to happen "eventually" but it needn't be "right now", but I figured it'd be good to get it out of the way... or put another way, "better my mess than someone else's".
--js
Re: [PATCH v2 06/12] python: upgrade to python3.9+ syntax
Quoted 76 lines · John Snow
Quoted 37 lines · Markus Armbruster
Quoted 35 lines · John Snow
This patch is fully automated, using pymagic, isort and autoflake.
Create a script named
pymagic.sh:=========================
pyupgrade --exit-zero-even-if-changed --keep-percent-format \ --py39-plus "$@"autoflake -i "$@"
isort --settings-file python/setup.cfg \
-p compat -pqapidoc_legacy-p iotests -o qemu "$@"
=========================Then, from qemu.git root:
Quoted 2 lines
find . -type f -name '*.py' | xargs pymagic
git grep --name-only "#!/usr/bin/env python" | xargs pymagicThis changes a lot of old Pythonisms, but in particular it upgrades the old Python type hint paradigm to the new 3.9+ paradigm wherein you no longer need to import List, Dict, Tuple, Set, etc from the Typing module and instead directly subscript the built-in types list, dict, tuple, set, etc. The old-style annotations are deprecated as of 3.9 and are eligible for removal starting in Python 3.14, though the exact date of their removal is not yet known.
pyupgrade updates the imports and type hint paradigms (as well as updating other old 'isms, such as removing the unicode string prefix). autoflake in turn then removes any unused import statements, possibly left behind by pyupgrade. Lastly, isort fixes the import order and formatting to the standard we use in qemu.git/python and
scripts/qapiin particular.Quoted 8 lines
Quoted 1 line
Signed-off-by John Snow <[email protected]>[...]
Quoted 1 line
448 files changed, 1959 insertions(+), 1631 deletions(-)*Ächz*
Gesundheit.
Quoted 6 lines
I hate it when people ask me to split up my mechanical patches...
One split is by subsystem / maintainer. I've done this a few times, and it's quite a bother. Questionable use of your time if you ask me.
I'd prefer not to unless it is requested of me specifically. I don't think most maintainers really care about the nuances of Python and as long as their stuff continues to work they're not going to mind much.
Or, to be frank: I don't think this series would ever garner enough review and attention to warrant the labor it'd take to tailor it to such a review. It's mechanical, it's boring, it should be fine.
I switched from a manual patch series to a tool-driven one specifically to make it more mindless and less interesting, and going through and splitting it back out is ... eh. I would prefer not to.
Quoted 4 lines
There's another split here... Your
pymagic.shruns three tools. If you commit after each one, the patch splits into three.
Quoted 6 lines
I use all three because each one alone isn't sufficient to then pass the static analysis checks, they each do a little bit of damage that another tool corrects afterwards.
pyupgrade works to modernize syntax, but leaves impotent import statements hanging.
Import statements it made impotent, I presume.
Quoted 2 lines
autoflake removes those impotent imports.
isort fixes the import statement ordering and formatting to our standard.
Out of curiosity: what messes up ordering and formatting?
Quoted 8 lines
(And then I do some manual fixups to fix the linting tests where things were auto-formatted suboptimally.)
I can still split it out for review purposes, like I did here with some manual fixups appended to the end.
Just, for merge, they'll be combined by necessity as a result of our no-regressions-for-bisect rule.
I see.
Quoted 11 lines
Quoted 5 lines
I understand you pass --py39-plus to pyupgrade to get the type hints modernized. If you run it without --py39-plus for all the miscellaneous upgrades, commit, then run it with --py39-plus for just the type hint upgrades, commit, the last patch splits again.
I can try it! I actually didn't try running it without py39-plus at all, so I don't know what that'll do. but no harm in an experiment.
Quoted 1 line
Quoted 7 lines
Quoted 1 line
Thoughts?
First and foremost I just thought it'd be good to get this mechanical change squared away in one giant patch so we could add this one singular horrible mega-commit into the git blame "ignored commits" list to minimize the impact of the "flag day".
Point.
Still, it's awfully hard to see what the horrible mega-commit does.
A patch that does one thing entirely mechanically is fine even when it's huge. Understanding the one thing is easy. I can usually develop confidence in the patch.
A patch that does many things mechanically can be problematic. If it's small enough, I can just review them like any other patch. If it's way too big for that, we have to rely on appeal to authority, i.e. the tool(s) that generated the patch. Certainly not nothing, but it gives me an uneasy feeling.
This is why I'm keen to see the type hint upgrade split off. I expect the type hint part to do one thing entirely mechanically (fine), and I hope the other part will be small enough to let me build confidence in it.
Quoted 3 lines
This upgrade will have to happen "eventually" but it needn't be "right now", but I figured it'd be good to get it out of the way... or put another way, "better my mess than someone else's".
I'd prefer upgrade now rather than later for the Python code I maintain.
PATCH v2 7/12 fixup
In reply to John SnowThese are manual fixes that ought be merged into the prior patch, but are here separated for clarity to separate the automated portions of this patch from the non-automated portions.
These fixes are only the necessary portions to allow python static checks (and checkpatch.pl) to pass again, and don't perform any "optional" cleanups. (i.e. this patch strictly prevents regressions.)
--- python/qemu/utils/qom_fuse.py | 1 + scripts/qapi/common.py | 28 ++++++++++++++-------------- 2 files changed, 15 insertions(+), 14 deletions(-)
python/qemu/utils/qom_fuse.py +1 -0
@@ -41,6 +41,7 @@ import fuse from fuse import FUSE, FuseOSError, Operations + from qemu.qmp import ExecuteError from .qom_common import QOMCommand
scripts/qapi/common.py +14 -14
@@ -91,29 +91,29 @@ def c_name(name: str, protect: bool = True) -> str: """ # ANSI X3J11/88-090, 3.1.1 c89_words = {'auto', 'break', 'case', 'char', 'const', 'continue', - 'default', 'do', 'double', 'else', 'enum', 'extern', - 'float', 'for', 'goto', 'if', 'int', 'long', 'register', - 'return', 'short', 'signed', 'sizeof', 'static', - 'struct', 'switch', 'typedef', 'union', 'unsigned', - 'void', 'volatile', 'while'} + 'default', 'do', 'double', 'else', 'enum', 'extern', + 'float', 'for', 'goto', 'if', 'int', 'long', 'register', + 'return', 'short', 'signed', 'sizeof', 'static', + 'struct', 'switch', 'typedef', 'union', 'unsigned', + 'void', 'volatile', 'while'} # ISO/IEC 9899:1999, 6.4.1 c99_words = {'inline', 'restrict', '_Bool', '_Complex', '_Imaginary'} # ISO/IEC 9899:2011, 6.4.1 c11_words = {'_Alignas', '_Alignof', '_Atomic', '_Generic', - '_Noreturn', '_Static_assert', '_Thread_local'} + '_Noreturn', '_Static_assert', '_Thread_local'} # GCC http://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/C-Extensions.html # excluding _.* gcc_words = {'asm', 'typeof'} # C++ ISO/IEC 14882:2003 2.11 cpp_words = {'bool', 'catch', 'class', 'const_cast', 'delete', - 'dynamic_cast', 'explicit', 'false', 'friend', 'mutable', - 'namespace', 'new', 'operator', 'private', 'protected', - 'public', 'reinterpret_cast', 'static_cast', 'template', - 'this', 'throw', 'true', 'try', 'typeid', 'typename', - 'using', 'virtual', 'wchar_t', - # alternative representations - 'and', 'and_eq', 'bitand', 'bitor', 'compl', 'not', - 'not_eq', 'or', 'or_eq', 'xor', 'xor_eq'} + 'dynamic_cast', 'explicit', 'false', 'friend', 'mutable', + 'namespace', 'new', 'operator', 'private', 'protected', + 'public', 'reinterpret_cast', 'static_cast', 'template', + 'this', 'throw', 'true', 'try', 'typeid', 'typename', + 'using', 'virtual', 'wchar_t', + # alternative representations + 'and', 'and_eq', 'bitand', 'bitor', 'compl', 'not', + 'not_eq', 'or', 'or_eq', 'xor', 'xor_eq'} # namespace pollution: polluted_words = {'unix', 'errno', 'mips', 'sparc', 'i386', 'linux'} name = re.sub(r'[^A-Za-z0-9_]', '_', name)
Signature · 1 line
2.48.1
PATCH v2 8/12 python: further 3.9+ syntax upgrades
In reply to John Snowpyupgrade does not catch absolutely everything - there are still a few deprecated type aliases we need to shift away from using.
--- docs/sphinx/compat.py | 8 ++------ docs/sphinx/qapi_domain.py | 6 +++--- python/qemu/machine/console_socket.py | 4 ++-- python/qemu/qmp/events.py | 9 +++++++-- python/qemu/qmp/protocol.py | 3 +-- python/qemu/utils/qemu_ga_client.py | 4 ++-- tests/qemu-iotests/fat16.py | 3 ++- tests/qemu-iotests/iotests.py | 8 ++++++-- 8 files changed, 25 insertions(+), 20 deletions(-)
docs/sphinx/compat.py +2 -6
@@ -2,13 +2,9 @@ Sphinx cross-version compatibility goop """ +from collections.abc import Callable import re -from typing import ( - TYPE_CHECKING, - Any, - Callable, - Optional, -) +from typing import TYPE_CHECKING, Any, Optional from docutils import nodes from docutils.nodes import Element, Node, Text
docs/sphinx/qapi_domain.py +3 -3
@@ -39,8 +39,8 @@ if TYPE_CHECKING: - from collections.abc import Iterable - from typing import AbstractSet, Any + from collections.abc import Iterable, Set + from typing import Any from docutils.nodes import Element, Node from sphinx.addnodes import desc_signature, pending_xref
@@ -825,7 +825,7 @@ def clear_doc(self, docname: str) -> None: del self.objects[fullname] def merge_domaindata( - self, docnames: AbstractSet[str], otherdata: dict[str, Any] + self, docnames: Set[str], otherdata: dict[str, Any] ) -> None: for fullname, obj in otherdata["objects"].items(): if obj.docname in docnames:
python/qemu/machine/console_socket.py +2 -2
@@ -17,7 +17,7 @@ import socket import threading import time -from typing import Deque, Optional +from typing import Optional class ConsoleSocket(socket.socket):
@@ -43,7 +43,7 @@ def __init__(self, self._recv_timeout_sec = 300.0 self._sleep_time = 0.5 - self._buffer: Deque[int] = deque() + self._buffer: deque[int] = deque() if address is not None: socket.socket.__init__(self, socket.AF_UNIX, socket.SOCK_STREAM) self.connect(address)
python/qemu/qmp/events.py +7 -2
@@ -448,10 +448,15 @@ def accept(self, event) -> bool: """ import asyncio -from collections.abc import AsyncIterator, Iterable, Iterator +from collections.abc import ( + AsyncIterator, + Callable, + Iterable, + Iterator, +) from contextlib import contextmanager import logging -from typing import Callable, Optional, Union +from typing import Optional, Union from .error import QMPError from .message import Message
python/qemu/qmp/protocol.py +1 -2
@@ -15,7 +15,7 @@ import asyncio from asyncio import StreamReader, StreamWriter -from collections.abc import AsyncGenerator, Awaitable +from collections.abc import AsyncGenerator, Awaitable, Callable from contextlib import asynccontextmanager from enum import Enum from functools import wraps
@@ -25,7 +25,6 @@ from ssl import SSLContext from typing import ( Any, - Callable, Generic, Optional, TypeVar,
python/qemu/utils/qemu_ga_client.py +2 -2
@@ -39,11 +39,11 @@ import argparse import asyncio import base64 -from collections.abc import Sequence +from collections.abc import Callable, Sequence import os import random import sys -from typing import Any, Callable, Optional +from typing import Any, Optional from qemu.qmp import ConnectError, SocketAddrT from qemu.qmp.legacy import QEMUMonitorProtocol
tests/qemu-iotests/fat16.py +2 -1
@@ -15,8 +15,9 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. +from collections.abc import Callable import string -from typing import Callable, Optional, Protocol +from typing import Optional, Protocol SECTOR_SIZE = 512
tests/qemu-iotests/iotests.py +6 -2
@@ -20,7 +20,12 @@ import atexit import bz2 from collections import OrderedDict -from collections.abc import Iterable, Iterator, Sequence +from collections.abc import ( + Callable, + Iterable, + Iterator, + Sequence, +) from contextlib import contextmanager import faulthandler import json
@@ -35,7 +40,6 @@ import time from typing import ( Any, - Callable, Optional, TextIO, TypeVar,
Signature · 1 line
2.48.1
PATCH v2 9/12 python: update mkvenv to type-check under different python versions
In reply to John SnowCurrently, we instruct mypy to pretend it is type checking under a specific version of python. For our case, where our scripts may in fact actually be executing under a number of different environments in the build system, we want mypy to remove this restriction.
This patch adjusts some of the import logic in mkvenv.py to make it type check under a wider range of python versions. It's largely arbitrary, to work around some mypy fussiness.
--- python/scripts/mkvenv.py | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
python/scripts/mkvenv.py +9 -6
@@ -99,16 +99,18 @@ HAVE_DISTLIB = False # Try to load tomllib, with a fallback to tomli. -# HAVE_TOMLLIB is checked below, just-in-time, so that mkvenv does not fail +# TOML is checked below, just-in-time, so that mkvenv does not fail # outside the venv or before a potential call to ensurepip in checkpip(). -HAVE_TOMLLIB = True +TOML = None try: import tomllib + TOML = tomllib except ImportError: try: - import tomli as tomllib + import tomli + TOML = tomli except ImportError: - HAVE_TOMLLIB = False + pass # Do not add any mandatory dependencies from outside the stdlib: # This script *must* be usable standalone!
@@ -194,6 +196,7 @@ def compute_venv_libpath(context: SimpleNamespace) -> str: # Python 3.12+, not strictly necessary because it's documented # to be the same as 3.10 code below: if sys.version_info >= (3, 12): + assert isinstance(context.lib_path, str) return context.lib_path # Python 3.10+
@@ -710,7 +713,7 @@ def _do_ensure( def _parse_groups(file: str) -> dict[str, dict[str, Any]]: - if not HAVE_TOMLLIB: + if not TOML: if sys.version_info < (3, 11): raise Ouch("found no usable tomli, please install it")
@@ -722,7 +725,7 @@ def _parse_groups(file: str) -> dict[str, dict[str, Any]]: # Debian bullseye-backports) and v2.0.x with open(file, encoding="ascii") as depfile: contents = depfile.read() - return tomllib.loads(contents) # type: ignore + return TOML.loads(contents) # type: ignore def ensure_group(
Signature · 1 line
2.48.1
PATCH v2 10/12 python: remove version restriction for mypy
In reply to John SnowRemoving this configuration line allows mypy to type check files from the context of whatever version of Python it happens to be running under, which is actually what we want here, so long as we do actually support running these scripts under multiple versions of Python, which we currently do.
--- python/setup.cfg | 1 - 1 file changed, 1 deletion(-)
python/setup.cfg +0 -1
@@ -78,7 +78,6 @@ exclude = __pycache__, [mypy] strict = True -python_version = 3.9 namespace_packages = True warn_unused_ignores = False
Signature · 1 line
2.48.1
PATCH v2 11/12 scripts/codeconverter: remove unused code
In reply to John SnowRemove (select) bits of unused code, as it is not typed correctly and appears to not be in use. The quickest solution is to remove it entirely.
'Match' is not defined, and neither is 'TypeInfoReference'. What these bits of code were meant to do is unclear.
--- scripts/codeconverter/codeconverter/patching.py | 3 --- scripts/codeconverter/codeconverter/qom_type_info.py | 4 ---- 2 files changed, 7 deletions(-)
scripts/codeconverter/codeconverter/patching.py +0 -3
@@ -418,9 +418,6 @@ def show_diff(self) -> None: f.flush() subprocess.call(['diff', '-u', self.filename, f.name]) - def ref(self): - return TypeInfoReference - class FileList(RegexpScanner): def __init__(self): super().__init__()
scripts/codeconverter/codeconverter/qom_type_info.py +0 -4
@@ -29,10 +29,6 @@ ParsedArray = List[str] ParsedInitializerValue = Union[str, ParsedArray] -class InitializerValue(NamedTuple): - raw: str - parsed: Optional[ParsedInitializerValue] - match: Optional[Match] class ArrayItem(FileMatch): regexp = RE_ARRAY_ITEM
Signature · 1 line
2.48.1
PATCH v2 12/12 scripts/codeconverter: remove * imports
In reply to John SnowPython static analysis tools generally don't like dealing with asterisk imports as it's tough to deduce what's actually getting used and from where. Replace these imports with explicit imports.
This also helps eliminate deprecated imports from the typing module.
These files have more style issues that are not addressed by this patch, so the command-line I am using to ensure I am not creating new identifier problems is a flake8 command set to squelch most style issues:
flake8 --ignore=E111,E114,E117,E127,E128,E129,E201,E221,E222,\
E225,E226,E252,E231,E261,E265,E301,E302,E303,\
E305,E306,E501,E502,E703,E712,E731,E741,F841,\
W292,W391,W503,W504 \
*.pyI am also using pylint's errors-only mode:
pylint -E *.py
--- .../codeconverter/codeconverter/patching.py | 4 +- .../codeconverter/codeconverter/qom_macros.py | 83 +++++++++++++------ .../codeconverter/qom_type_info.py | 76 ++++++++++++++--- .../codeconverter/test_patching.py | 9 +- .../codeconverter/test_regexps.py | 32 ++++++- scripts/codeconverter/codeconverter/utils.py | 7 +- 6 files changed, 161 insertions(+), 50 deletions(-)
scripts/codeconverter/codeconverter/patching.py +2 -2
@@ -25,6 +25,8 @@ Union, ) +from .utils import LineAndColumn, line_col + logger = logging.getLogger(__name__) DBG = logger.debug
@@ -32,8 +34,6 @@ WARN = logger.warning ERROR = logger.error -from .utils import * - T = TypeVar('T')
scripts/codeconverter/codeconverter/qom_macros.py +59 -24
166 lines not on this page · raw message
scripts/codeconverter/codeconverter/qom_type_info.py +63 -13
128 lines not on this page · raw message
scripts/codeconverter/codeconverter/test_patching.py +2 -7
@@ -7,13 +7,8 @@ # the COPYING file in the top-level directory. from tempfile import NamedTemporaryFile -from .patching import ( - FileInfo, - FileList, - FileMatch, - Patch, -) -from .regexps import * +from .patching import FileInfo, FileList, FileMatch +from .regexps import NAMED, RE_IDENTIFIER, S class BasicPattern(FileMatch):
scripts/codeconverter/codeconverter/test_regexps.py +29 -3
@@ -5,9 +5,35 @@ # # This work is licensed under the terms of the GNU GPL, version 2. See # the COPYING file in the top-level directory. -from .qom_macros import * -from .qom_type_info import * -from .regexps import * + +import re + +from .qom_macros import ( + RE_CHECK_MACRO, + RE_MACRO_DEFINE, + RE_STRUCT_TYPEDEF, + InitialIncludes, +) +from .qom_type_info import ( + RE_TI_FIELD_INIT, + RE_TI_FIELDS, + RE_TYPEINFO_START, + TypeInfoVar, +) +from .regexps import ( + CPP_SPACE, + RE_ARRAY, + RE_ARRAY_CAST, + RE_ARRAY_ITEM, + RE_COMMENT, + RE_COMMENTS, + RE_EXPRESSION, + RE_FUN_CALL, + RE_IDENTIFIER, + RE_MACRO_CONCAT, + RE_SIMPLE_VALUE, + SP, +) def test_res() -> None:
scripts/codeconverter/codeconverter/utils.py +6 -1
@@ -6,7 +6,12 @@ # This work is licensed under the terms of the GNU GPL, version 2. See # the COPYING file in the top-level directory. import logging -from typing import * +from typing import ( + NamedTuple, + NewType, + Optional, + TypeVar, +) logger = logging.getLogger(__name__)
Signature · 1 line
2.48.1
Quoted 3 lines · John Snow
This series does a couple things that I'll probably end up splitting out into smaller series if history is any guide, but either way, here it goes:
I'm surprised with the large number of Ccs. I suggest splitting the series with subsystem boundaries to reduce the number of Ccs for each split series.
Quoted 10 lines
A) Convert qemu.git/python/ to a
PEP517/pyproject.tomlpackage. Ultimately this means deletingsetup.pyand fully migrating to newer python infrastructure. I think this should be safe to do by now, but admittedly I am not *confident* as it relies on setuptools versions in the wild, not python versions. My motivation for trying it is to fix "make check-dev", which has been broken for the last two Fedora releases under newer setuptools which have started removing support for the pre-PEP517 packaging formats, which will only continue to get worse from here on out.
I'm afraid this change may not be safe.
The documentation of setuptools says pyproject.toml is supported since 61.0.0:
https://setuptools.pypa.io/en/stable/userguide/pyproject_config.html
Looking at Repology as suggested in docs/about/build-platforms.rst, the following distributions have older setuptools, unfortunately:
CentOS Stream 9: 53.0.0
Debian 11: 44.1.1
Ubuntu 22.04: 44.1.1
docs/about/build-platforms.rst also says:
Quoted 6 lines
Optional build dependencies
Build components whose absence does not affect the ability to build QEMU may not be available in distros, or may be too old for our requirements. Many of these, such as additional modules for the functional testing framework or various linters, are written in Python and therefore can also be installed using ``pip``.
In my understanding, QEMU Python Tooling is an optional component so dropping the support of old Linux distribution is tolerated according to the documentation. But I think we still need to assess the potential impact on the usability to justify that.
A possible option is to fetch a new version of setuptools version when necessary. The same documentation says:
Quoted 7 lines
Python build dependencies
Some of QEMU's build dependencies are written in Python. Usually these are only packaged by distributions for the default Python runtime.
If QEMU bumps its minimum Python version and a non-default runtime is required, it may be necessary to fetch python modules from the Python Package Index (PyPI) via ``pip``, in order to build QEMU.
Automatically fetching a compatible version of setuptools as described here will reduce the adverse effect of the change.
The documentation page of setuptools I mentioned earlier has the following note:
Quoted 8 lines
If compatibility with legacy builds or versions of tools that don’t support certain packaging standards (e.g. PEP 517 or PEP 660), a simple
setup.pyscript can be added to your project [1] (while keeping the configuration in pyproject.toml):from setuptools import setup
setup()
I have no idea what it means though. (Will it automatically fetch new setuptools to parse pyproject.toml or something?)
Quoted 4 lines
C) Move us to 3.9+ style type hints. They are deprecated in 3.9, and *could* be removed at any time. I figured now was a good time as any to get rid of them before they become a problem randomly some day in the future.
I guess you meant that the old style type hints are deprecated.
Regards,
Akihiko Odaki