preallocate filter
Status from this message: Thanks, applied to my block-next branch (for 6.0):
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v7 Latest21 patches 20 repliesApplied
- v615 patches 21 repliesSuperseded
- v510 patches 19 repliesSuperseded
Earlier versions · 4
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v4 | 10 patches | 3 replies | Superseded | |
| v3 | 12 patches | 16 replies | Superseded | |
| v2 | 9 patches | 1 reply | Superseded | |
| v1 | 5 patches | 11 replies | Superseded |
Changes in v7 from the cover letter · 16 lines
01: add Alberto's r-b
07: don't remove sentence from the comment
08: - drop extra "s->file_end = end;" line
- improve check/set perm handlers
09: add Max's r-b
10: add Max's r-b
11: new
12: - skip if preallocate unsupported
- drop auto and quick groups
13: new
14: - improve 'average' field of result spec
- drop extra 'dim = ...' line
15-18: new
19: a lot of changes
20: new
21: add results dump to json
Patches in v7 Applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/21 | block: simplify comment to BDRV_REQ_SERIALISINGinclude/ | +1 | 0 replies | |
| 2/21 | block/io.c: drop assertion on double waiting for request serialisationblock/ | +1 | 0 replies | |
| 3/21 | block/io: split out bdrv_find_conflicting_requestblock/ | +41 | 0 replies | |
| 4/21 | block/io: bdrv_wait_serialising_requests_locked: drop extra bs argblock/ | +5 | 0 replies | |
| 5/21 | block: bdrv_mark_request_serialising: split non-waiting functionblock/ block/ include/ | +26 | 0 replies | |
| 6/21 | block: introduce BDRV_REQ_NO_WAIT flagblock/ include/ | +18 | 0 replies | |
| 7/21 | block: bdrv_check_perm(): process children anywayblock.c | +5 | 1 reply | |
| 8/21 | block: introduce preallocate filterblock/ block/ docs/ +1 more | +605 | 2 replies | |
| 9/21 | qemu-io: add preallocate mode parameter for truncate commandqemu-io-cmds.c | +32 | 0 replies | |
| 10/21 | iotests: qemu_io_silent: support --image-optstests/ | +6 | 0 replies | |
| 11/21 | iotests.py: execute_setup_common(): add required_fmts argumenttests/ | +8 | 1 reply | |
| 12/21 | iotests: add 298 to test new preallocate filter drivertests/ tests/ tests/ | +192 | 1 reply | |
| 13/21 | scripts/simplebench: fix grammar: s/successed/succeeded/scripts/ | +6 | 1 reply | |
| 14/21 | scripts/simplebench: support iopsscripts/ | +28 | 1 reply | |
| 15/21 | scripts/simplebench: use standard deviation for +- errorscripts/ | +6 | 1 reply | |
| 16/21 | simplebench: rename ascii() to results_to_text()scripts/ scripts/ scripts/ | +7 | 1 reply | |
| 17/21 | simplebench: move results_to_text() into separate filescripts/ scripts/ scripts/ +1 more | +52 | 1 reply | |
| 18/21 | simplebench/results_to_text: improve view of the tablescripts/ | +12 | 2 replies | |
| 19/21 | simplebench/results_to_text: add difference line to the tablescripts/ | +60 | 2 replies | |
| 20/21 | simplebench/results_to_text: make executablescripts/ | +14 | 1 reply | |
| 21/21 | scripts/simplebench: add bench_prealloc.pyscripts/ | +132 | 3 replies |
Reviews trailers per patch
All 21 patches have a Reviewed-by or Acked-by.
| Patch | Stefan Hajnoczi | Max Reitz | Alberto Garcia | Paolo Bonzini |
|---|---|---|---|---|
| 1/21 | R-b | R-b | ||
| 2/21 | R-b | |||
| 3/21 | R-b | |||
| 4/21 | R-b | |||
| 5/21 | R-b | |||
| 6/21 | R-b | |||
| 7/21 | R-b | |||
| 8/21 | R-b | |||
| 9/21 | R-b | |||
| 10/21 | R-b | |||
| 11/21 | R-b | |||
| 12/21 | R-b | |||
| 13/21 | R-b | |||
| 14/21 | R-b | |||
| 15/21 | R-b | |||
| 16/21 | R-b | |||
| 17/21 | R-b | |||
| 18/21 | A-b | |||
| 19/21 | R-b | |||
| 20/21 | R-b | |||
| 21/21 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.
Cover letter [PATCH v7 00/21] preallocate filter
Hi all!
Here is a filter, which does preallocation on write.
v7:
01: add Alberto's r-b
07: don't remove sentence from the comment
08: - drop extra "s->file_end = end;" line
- improve check/set perm handlers
09: add Max's r-b
10: add Max's r-b
11: new
12: - skip if preallocate unsupported
- drop auto and quick groups
13: new
14: - improve 'average' field of result spec
- drop extra 'dim = ...' line
15-18: new
19: a lot of changes
20: new
21: add results dump to json
In Virtuozzo we have to deal with some custom distributed storage solution, where allocation is very-very expensive operation. We have to workaround it in Qemu, so here is a new filter.
Still, the filter shows good results for me even for xfs and ext4.
Here are results, produced by new benchmark (last several patches):
All results are in iops (larger means better)
---------------------------------- ------------- ------------
no-prealloc prealloc
ssd-ext4, aligned sequential 16k 2e+04 2.7e+04
+36%
hdd-ext4, aligned sequential 16k 4.8e+03 ± 5% 3.1e+03 ± 4%
-36%
ssd-xfs, aligned sequential 16k 1.6e+04 ± 5% 2.5e+04 ± 3%
+59%
hdd-xfs, aligned sequential 16k 4.7e+03 ± 22% 3.2e+03 ± 3%
-31%
ssd-ext4, unaligned sequential 64k 2e+04 2.7e+04
+36%
hdd-ext4, unaligned sequential 64k 4.9e+03 ± 2% 3.2e+03 ± 8%
-34%
ssd-xfs, unaligned sequential 64k 1.5e+04 2.6e+04 ± 2%
+69%
hdd-xfs, unaligned sequential 64k 5.2e+03 ± 2% 3.3e+03 ± 5%
-35%---------------------------------- ------------- ------------
Note: it's on Fedora 30, kernel 5.6.13-100.fc30.x86_64
The tests are actually qemu-img bench, run like:
./qemu-img create -f qcow2 $img 16G
aligned:
./qemu-img bench -c 10000 -d 64 -f qcow2 -s 16k -t none -n -w $img
unaligned
./qemu-img bench -c 10000 -d 64 -f qcow2 -o 1k -s 64k -t none -n -w $img
and for preallocation, you'll drop -f qcow2, add --image-opts, and instead of just $img use
driver=qcow2,file.driver=preallocate,file.file.driver=file,file.file.filename=$img
Or, use new benchmark like this:
./bench_prealloc.py ../../build/qemu-img \
ssd-ext4:/path/to/mount/point \
ssd-xfs:/path2 hdd-ext4:/path3 hdd-xfs:/path4Vladimir Sementsov-Ogievskiy (21):
block: simplify comment to BDRV_REQ_SERIALISING block/io.c: drop assertion on double waiting for request serialisation block/io: split out bdrv_find_conflicting_request block/io: bdrv_wait_serialising_requests_locked: drop extra bs arg
block: bdrv_mark_request_serialising: split non-waiting function
block: introduce BDRV_REQ_NO_WAIT flag
block: bdrv_check_perm(): process children anyway
block: introduce preallocate filter
qemu-io: add preallocate mode parameter for truncate command
iotests: qemu_io_silent: support --image-opts
iotests.py: execute_setup_common(): add required_fmts argument
iotests: add 298 to test new preallocate filter driver scripts/simplebench: fix grammar: s/successed/succeeded/ scripts/simplebench: support iops
scripts/simplebench: use standard deviation for +- error
simplebench: rename ascii() to results_to_text()
simplebench: move results_to_text() into separate file simplebench/results_to_text: improve view of the table simplebench/results_to_text: add difference line to the table simplebench/results_to_text: make executable scripts/simplebench: add bench_prealloc.py
docs/system/qemu-block-drivers.rst.inc | 26 ++ qapi/block-core.json | 20 +- include/block/block.h | 20 +- include/block/block_int.h | 3 +- block.c | 7 +- block/file-posix.c | 2 +- block/io.c | 130 +++--- block/preallocate.c | 559 +++++++++++++++++++++++++ qemu-io-cmds.c | 46 +- block/meson.build | 1 + scripts/simplebench/bench-example.py | 3 +- scripts/simplebench/bench_prealloc.py | 132 ++++++ scripts/simplebench/bench_write_req.py | 3 +- scripts/simplebench/results_to_text.py | 126 ++++++ scripts/simplebench/simplebench.py | 66 ++- tests/qemu-iotests/298 | 186 ++++++++ tests/qemu-iotests/298.out | 5 + tests/qemu-iotests/group | 1 + tests/qemu-iotests/iotests.py | 16 +- 19 files changed, 1225 insertions(+), 127 deletions(-) create mode 100644 block/preallocate.c create mode 100755 scripts/simplebench/bench_prealloc.py create mode 100755 scripts/simplebench/results_to_text.py create mode 100644 tests/qemu-iotests/298 create mode 100644 tests/qemu-iotests/298.out
Signature · 1 line
2.21.3