fallocate: introduce FALLOC_FL_WRITE_ZEROES flag
Subsystems (8)
v2 cover fallocate: introduce FALLOC_FL_WRITE_ZEROES flag
From: Zhang Yi <[email protected]>
Changes since v1:
- Rebase codes on 6.16-rc2.
- Use max_{hw|user}_wzeroes_unmap_sectors queue limits instead of BLK_FEAT_WRITE_ZEROES_UNMAP feature to represent the status of the unmap write zeroes operation as Christoph and Darrick suggested. This redoes the first 5 patches, so remove all the reviewed-by tags, please review them again.
- Simplify the description of FALLOC_FL_WRITE_ZEROES in patch 06 as Darrick suggested.
- Revise the check order of FALLOC_FL_WRITE_ZEROES in patch 08 as Christoph suggested.
Changes since RFC v4:
- Rebase codes on 6.16-rc1.
- Add a new queue_limit flag, and change the write_zeroes_unmap sysfs interface to RW mode. User can disable the unmap write zeroes operation by writing '0' to it when the operation is slow.
- Modify the documentation of write_zeroes_unmap sysfs interface as Martin suggested.
- Remove the statx interface.
- Make the bdev and ext4 don't allow to submit FALLOC_FL_WRITE_ZEROES if the block device does not enable the unmap write zeroes operation, it should return -EOPNOTSUPP.
Changes sicne RFC v3:
- Rebase codes on 6.15-rc2.
- Add a note in patch 1 to indicate that the unmap write zeros command is not always guaranteed as Christoph suggested.
- Rename bdev_unmap_write_zeroes() helper and move it to patch 1 as Christoph suggested.
- Introduce a new statx attribute flag STATX_ATTR_WRITE_ZEROES_UNMAP as Christoph and Christian suggested.
- Exchange the order of the two patches that modified blkdev_fallocate() as Christoph suggested.
Changes since RFC v2:
- Rebase codes on next-20250314.
- Add support for nvme multipath.
- Add support for NVMeT with block device backing.
- Clear FALLOC_FL_WRITE_ZEROES if dm clear limits->max_write_zeroes_sectors.
- Complement the counterpart userspace tools(util-linux and xfs_io) and tests(blktests and xfstests), please see below for details.
Changes since RFC v1:
- Switch to add a new write zeroes operation, FALLOC_FL_WRITE_ZEROES, in fallocate, instead of just adding a supported flag to FALLOC_FL_ZERO_RANGE.
- Introduce a new flag BLK_FEAT_WRITE_ZEROES_UNMAP to the block device's queue limit features, and implement it on SCSI sd driver, NVMe SSD driver and dm driver.
- Implement FALLOC_FL_WRITE_ZEROES on both the ext4 filesystem and block device (bdev).
v1: https://lore.kernel.org/linux-fsdevel/[email protected]/ RFC v4: https://lore.kernel.org/linux-fsdevel/[email protected]/ RFC v3: https://lore.kernel.org/linux-fsdevel/[email protected]/ RFC v2: https://lore.kernel.org/linux-fsdevel/[email protected]/ RFC v1: https://lore.kernel.org/linux-fsdevel/[email protected]/
The counterpart userspace tools changes and tests are here:
- util-linux: https://lore.kernel.org/linux-fsdevel/[email protected]/
- xfsprogs: https://lore.kernel.org/linux-fsdevel/[email protected]/
- xfstests: https://lore.kernel.org/linux-fsdevel/[email protected]/
(needs to be updated)
- blktests: https://lore.kernel.org/linux-fsdevel/[email protected]/
(needs to be updated)
Original Description:
Currently, we can use the fallocate command to quickly create a pre-allocated file. However, on most filesystems, such as ext4 and XFS, fallocate create pre-allocation blocks in an unwritten state, and the FALLOC_FL_ZERO_RANGE flag also behaves similarly. The extent state must be converted to a written state when the user writes data into this range later, which can trigger numerous metadata changes and consequent journal I/O. This may leads to significant write amplification and performance degradation in synchronous write mode. Therefore, we need a method to create a pre-allocated file with written extents that can be used for pure overwriting. At the monent, the only method available is to create an empty file and write zero data into it (for example, using 'dd' with a large block size). However, this method is slow and consumes a considerable amount of disk bandwidth, we must pre-allocate files in advance but cannot add pre-allocated files while user business services are running.
Fortunately, with the development and more and more widely used of flash-based storage devices, we can efficiently write zeros to SSDs using the unmap write zeroes command if the devices do not write physical zeroes to the media. For example, if SCSI SSDs support the UMMAP bit or NVMe SSDs support the DEAC bit[1], the write zeroes command does not write actual data to the device, instead, NVMe converts the zeroed range to a deallocated state, which works fast and consumes almost no disk write bandwidth. Consequently, this feature can provide us with a faster method for creating pre-allocated files with written extents and zeroed data. However, please note that this may be a best-effort optimization rather than a mandatory requirement, some devices may partially fall back to writing physical zeroes due to factors such as receiving unaligned commands.
This series aims to implement this by:
1. Introduce a new feature BLK_FEAT_WRITE_ZEROES_UNMAP to the block device queue limit features, which indicates whether the storage is device explicitly supports the unmapped write zeroes command. This flag should be set to 1 by the driver if the attached disk supports this command.
2. Introduce a queue limit flag, BLK_FLAG_WRITE_ZEROES_UNMAP_DISABLED, along with a corresponding sysfs entry. Users can query the support status of the unmap write zeroes operation and disable this operation if the write zeroes operation is very slow.
/sys/block/<disk>/queue/write_zeroes_unmap
3. Introduce a new flag, FALLOC_FL_WRITE_ZEROES, into the fallocate. Filesystems that support this operation should allocate written extents and issue zeroes to the specified range of the device. For local block device filesystems, this operation should depend on the write_zeroes_unmap operaion of the underlying block device. It should return -EOPNOTSUPP if the device doesn't enable unmap write zeroes operaion.
This series implements the BLK_FEAT_WRITE_ZEROES_UNMAP feature and BLK_FLAG_WRITE_ZEROES_UNMAP_DISABLED flag for SCSI, NVMe and device-mapper drivers, and add the FALLOC_FL_WRITE_ZEROES and STATX_ATTR_WRITE_ZEROES_UNMAP support for ext4 and raw bdev devices. Any comments are welcome.
I've tested performance with this series on ext4 filesystem on my machine with an Intel Xeon Gold 6248R CPU, a 7TB KCD61LUL7T68 NVMe SSD which supports unmap write zeroes command with the Deallocated state and the DEAC bit. Feel free to give it a try.
0. Ensure the NVMe device supports WRITE_ZERO command.
$ cat /sys/block/nvme5n1/queue/write_zeroes_max_bytes
8388608
$ nvme id-ns -H /dev/nvme5n1 | grep -i -A 3 "dlfeat"
dlfeat : 25
[4:4] : 0x1 Guard Field of Deallocated Logical Blocks is set to CRC
of The Value Read
[3:3] : 0x1 Deallocate Bit in the Write Zeroes Command is Supported
[2:0] : 0x1 Bytes Read From a Deallocated Logical Block and its
Metadata are 0x001. Compare 'dd' and fallocate with unmap write zeroes, the later one is significantly faster than 'dd'.
Create a 1GB and 10GB zeroed file.
$dd if=/dev/zero of=foo bs=2M count=$count oflag=direct
$time fallocate -w -l $size bar
#1G
dd: 0.5s
FALLOC_FL_WRITE_ZEROES: 0.17s
#10G
dd: 5.0s
FALLOC_FL_WRITE_ZEROES: 1.7s2. Run fio overwrite and fallocate with unmap write zeroes simultaneously, fallocate has little impact on write bandwidth and only slightly affects write latency.
a) Test bandwidth costs.
$ fio -directory=/test -direct=1 -iodepth=10 -fsync=0 -rw=write \
-numjobs=10 -bs=2M -ioengine=libaio -size=20G -runtime=20 \
-fallocate=none -overwrite=1 -group_reportin -name=bw_testWithout background zero range:
bw (MiB/s): min= 2068, max= 2280, per=100.00%, avg=2186.40
With background zero range:
bw (MiB/s): min= 2056, max= 2308, per=100.00%, avg=2186.20
b) Test write latency costs.
$ fio -filename=/test/foo -direct=1 -iodepth=1 -fsync=0 -rw=write \
-numjobs=1 -bs=4k -ioengine=psync -size=5G -runtime=20 \
-fallocate=none -overwrite=1 -group_reportin -name=lat_test Without background zero range:
lat (nsec): min=9269, max=71635, avg=9840.65
With a background zero range:
lat (usec): min=9, max=982, avg=11.03
3. Compare overwriting in a pre-allocated unwritten file and a written file in O_DSYNC mode. Write to a file with written extents is much faster.
# First mkfs and create a test file according to below three cases,
# and then run fio.
$ fio -filename=/test/foo -direct=1 -iodepth=1 -fdatasync=1 \
-rw=write -numjobs=1 -bs=4k -ioengine=psync -size=5G \
-runtime=20 -fallocate=none -group_reportin -name=testunwritten file: IOPS=20.1k, BW=78.7MiB/s unwritten file + fast_commit: IOPS=42.9k, BW=167MiB/s written file: IOPS=98.8k, BW=386MiB/s
Thanks,
Yi.
---
[1] https://nvmexpress.org/specifications/
NVM Command Set Specification, Figure 82 and Figure 114.
Zhang Yi (9):
block: introduce max_{hw|user}_wzeroes_unmap_sectors to queue limits
nvme: set max_hw_wzeroes_unmap_sectors if device supports DEAC bit
nvmet: set WZDS and DRB if device enables unmap write zeroes operation
scsi: sd: set max_hw_wzeroes_unmap_sectors if device supports
SD_ZERO_*_UNMAP
dm: clear unmap write zeroes limits when disabling write zeroes
fs: introduce FALLOC_FL_WRITE_ZEROES to fallocate
block: factor out common part in blkdev_fallocate()
block: add FALLOC_FL_WRITE_ZEROES support
ext4: add FALLOC_FL_WRITE_ZEROES support
Documentation/ABI/stable/sysfs-block | 33 ++++++++++++++ block/blk-settings.c | 20 ++++++++- block/blk-sysfs.c | 26 +++++++++++ block/fops.c | 44 +++++++++++-------- drivers/md/dm-table.c | 4 +- drivers/nvme/host/core.c | 20 +++++---- drivers/nvme/target/io-cmd-bdev.c | 4 ++ drivers/scsi/sd.c | 5 +++ fs/ext4/extents.c | 66 +++++++++++++++++++++++----- fs/open.c | 1 + include/linux/blkdev.h | 10 +++++ include/linux/falloc.h | 3 +- include/trace/events/ext4.h | 3 +- include/uapi/linux/falloc.h | 17 +++++++ 14 files changed, 212 insertions(+), 44 deletions(-)
Signature · 1 line
2.46.1
PATCH v2 1/9 block: introduce max_{hw|user}_wzeroes_unmap_sectors to queue limits
From: Zhang Yi <[email protected]>
Currently, disks primarily implement the write zeroes command (aka REQ_OP_WRITE_ZEROES) through two mechanisms: the first involves physically writing zeros to the disk media (e.g., HDDs), while the second performs an unmap operation on the logical blocks, effectively putting them into a deallocated state (e.g., SSDs). The first method is generally slow, while the second method is typically very fast.
For example, on certain NVMe SSDs that support NVME_NS_DEAC, submitting REQ_OP_WRITE_ZEROES requests with the NVME_WZ_DEAC bit can accelerate the write zeros operation by placing disk blocks into a deallocated state, which opportunistically avoids writing zeroes to media while still guaranteeing that subsequent reads from the specified block range will return zeroed data. This is a best-effort optimization, not a mandatory requirement, some devices may partially fall back to writing physical zeroes due to factors such as misalignment or being asked to clear a block range smaller than the device's internal allocation unit. Therefore, the speed of this operation is not guaranteed.
It is difficult to determine whether the storage device supports unmap write zeroes operation. We cannot determine this by only querying bdev_limits(bdev)->max_write_zeroes_sectors. Therefore, first, add a new hardware queue limit parameters, max_hw_wzeroes_unmap_sectors, to indicate whether a device supports this unmap write zeroes operation. Then, add two new counterpart software queue limits, max_wzeroes_unmap_sectors and max_user_wzeroes_unmap_sectors, which allow users to disable this operation if the speed is very slow on some sepcial devices.
Finally, for the stacked devices cases, initialize these two parameters to UINT_MAX. This operation should be enabled by both the stacking driver and all underlying devices.
Thanks to Martin K. Petersen for optimizing the documentation of the write_zeroes_unmap sysfs interface.
--- Documentation/ABI/stable/sysfs-block | 33 ++++++++++++++++++++++++++++ block/blk-settings.c | 20 +++++++++++++++-- block/blk-sysfs.c | 26 ++++++++++++++++++++++ include/linux/blkdev.h | 10 +++++++++ 4 files changed, 87 insertions(+), 2 deletions(-)
Documentation/ABI/stable/sysfs-block +33 -0
@@ -778,6 +778,39 @@ Description: 0, write zeroes is not supported by the device. +What: /sys/block/<disk>/queue/write_zeroes_unmap_max_hw_bytes +Date: January 2025 +Contact: Zhang Yi <[email protected]> +Description: + [RO] This file indicates whether a device supports zeroing data + in a specified block range without incurring the cost of + physically writing zeroes to the media for each individual + block. If this parameter is set to write_zeroes_max_bytes, the + device implements a zeroing operation which opportunistically + avoids writing zeroes to media while still guaranteeing that + subsequent reads from the specified block range will return + zeroed data. This operation is a best-effort optimization, a + device may fall back to physically writing zeroes to the media + due to other factors such as misalignment or being asked to + clear a block range smaller than the device's internal + allocation unit. If this parameter is set to 0, the device may + have to write each logical block media during a zeroing + operation. + + +What: /sys/block/<disk>/queue/write_zeroes_unmap_max_bytes +Date: January 2025 +Contact: Zhang Yi <[email protected]> +Description: + [RW] While write_zeroes_unmap_max_hw_bytes is the hardware limit + for the device, this setting is the software limit. Since the + unmap write zeroes operation is a best-effort optimization, some + devices may still physically writing zeroes to media. So the + speed of this operation is not guaranteed. Writing a value of + '0' to this file disables this operation. Otherwise, this + parameter should be equal to write_zeroes_unmap_max_hw_bytes. + + What: /sys/block/<disk>/queue/zone_append_max_bytes Date: May 2020 Contact: [email protected]
block/blk-settings.c +18 -2
@@ -50,6 +50,8 @@ void blk_set_stacking_limits(struct queue_limits *lim) lim->max_sectors =UINT_MAX ; lim->max_dev_sectors =UINT_MAX ; lim->max_write_zeroes_sectors =UINT_MAX ; + lim->max_hw_wzeroes_unmap_sectors =UINT_MAX ; + lim->max_user_wzeroes_unmap_sectors =UINT_MAX ; lim->max_hw_zone_append_sectors =UINT_MAX ; lim->max_user_discard_sectors =UINT_MAX ; }
@@ -333,6 +335,12 @@ int blk_validate_limits(struct queue_limits *lim)if (!lim->max_segments) lim->max_segments =BLK_MAX_SEGMENTS ; +if (lim->max_hw_wzeroes_unmap_sectors && + lim->max_hw_wzeroes_unmap_sectors != lim->max_write_zeroes_sectors) +return -EINVAL ; + lim->max_wzeroes_unmap_sectors =min (lim->max_hw_wzeroes_unmap_sectors, + lim->max_user_wzeroes_unmap_sectors); + lim->max_discard_sectors =min (lim->max_hw_discard_sectors, lim->max_user_discard_sectors);
@@ -418,10 +426,11 @@ int blk_set_default_limits(struct queue_limits *lim) {/* * Most defaults are set by capping the bounds in blk_validate_limits, -* but max_user_discard_sectors is special and needs an explicit -* initialization to the max value here. +* but these limits are special and need an explicit initialization to +* the max value here. */ lim->max_user_discard_sectors =UINT_MAX ; + lim->max_user_wzeroes_unmap_sectors =UINT_MAX ;return blk_validate_limits (lim); }
@@ -708,6 +717,13 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b, t->max_dev_sectors =min_not_zero (t->max_dev_sectors, b->max_dev_sectors); t->max_write_zeroes_sectors =min (t->max_write_zeroes_sectors, b->max_write_zeroes_sectors); + t->max_user_wzeroes_unmap_sectors = +min (t->max_user_wzeroes_unmap_sectors, + b->max_user_wzeroes_unmap_sectors); + t->max_hw_wzeroes_unmap_sectors = +min (t->max_hw_wzeroes_unmap_sectors, + b->max_hw_wzeroes_unmap_sectors); + t->max_hw_zone_append_sectors =min (t->max_hw_zone_append_sectors, b->max_hw_zone_append_sectors);
block/blk-sysfs.c +26 -0
@@ -161,6 +161,8 @@ static ssize_t queue_##_field##_show(struct gendisk *disk, char *page) \ QUEUE_SYSFS_LIMIT_SHOW_SECTORS_TO_BYTES(max_discard_sectors) QUEUE_SYSFS_LIMIT_SHOW_SECTORS_TO_BYTES(max_hw_discard_sectors) QUEUE_SYSFS_LIMIT_SHOW_SECTORS_TO_BYTES(max_write_zeroes_sectors) +QUEUE_SYSFS_LIMIT_SHOW_SECTORS_TO_BYTES(max_hw_wzeroes_unmap_sectors) +QUEUE_SYSFS_LIMIT_SHOW_SECTORS_TO_BYTES(max_wzeroes_unmap_sectors) QUEUE_SYSFS_LIMIT_SHOW_SECTORS_TO_BYTES(atomic_write_max_sectors) QUEUE_SYSFS_LIMIT_SHOW_SECTORS_TO_BYTES(atomic_write_boundary_sectors) QUEUE_SYSFS_LIMIT_SHOW_SECTORS_TO_BYTES(max_zone_append_sectors)
@@ -205,6 +207,24 @@ static int queue_max_discard_sectors_store(struct gendisk *disk,return 0 ; } +static int queue_max_wzeroes_unmap_sectors_store (struct gendisk *disk, +const char *page,size_t count,struct queue_limits *lim) +{ +unsigned long max_zeroes_bytes, max_hw_zeroes_bytes; +ssize_t ret; + + ret =queue_var_store (&max_zeroes_bytes, page, count); +if (ret <0 ) +return ret; + + max_hw_zeroes_bytes = lim->max_hw_wzeroes_unmap_sectors <<SECTOR_SHIFT ; +if (max_zeroes_bytes !=0 && max_zeroes_bytes != max_hw_zeroes_bytes) +return -EINVAL ; + + lim->max_user_wzeroes_unmap_sectors = max_zeroes_bytes >>SECTOR_SHIFT ; +return 0 ; +} +static int queue_max_sectors_store (struct gendisk *disk,const char *page,size_t count,struct queue_limits *lim)
@@ -514,6 +534,10 @@ QUEUE_LIM_RO_ENTRY(queue_atomic_write_unit_min, "atomic_write_unit_min_bytes");QUEUE_RO_ENTRY (queue_write_same_max,"write_same_max_bytes" );QUEUE_LIM_RO_ENTRY (queue_max_write_zeroes_sectors,"write_zeroes_max_bytes" ); +QUEUE_LIM_RO_ENTRY (queue_max_hw_wzeroes_unmap_sectors, +"write_zeroes_unmap_max_hw_bytes" ); +QUEUE_LIM_RW_ENTRY (queue_max_wzeroes_unmap_sectors, +"write_zeroes_unmap_max_bytes" );QUEUE_LIM_RO_ENTRY (queue_max_zone_append_sectors,"zone_append_max_bytes" );QUEUE_LIM_RO_ENTRY (queue_zone_write_granularity,"zone_write_granularity" );
@@ -662,6 +686,8 @@ static struct attribute *queue_attrs[] = { &queue_atomic_write_unit_min_entry.attr, &queue_atomic_write_unit_max_entry.attr, &queue_max_write_zeroes_sectors_entry.attr, + &queue_max_hw_wzeroes_unmap_sectors_entry.attr, + &queue_max_wzeroes_unmap_sectors_entry.attr, &queue_max_zone_append_sectors_entry.attr, &queue_zone_write_granularity_entry.attr, &queue_rotational_entry.attr,
include/linux/blkdev.h +10 -0
@@ -383,6 +383,9 @@ struct queue_limits {unsigned int max_user_discard_sectors;unsigned int max_secure_erase_sectors;unsigned int max_write_zeroes_sectors; +unsigned int max_wzeroes_unmap_sectors; +unsigned int max_hw_wzeroes_unmap_sectors; +unsigned int max_user_wzeroes_unmap_sectors;unsigned int max_hw_zone_append_sectors;unsigned int max_zone_append_sectors;unsigned int discard_granularity;
@@ -1042,6 +1045,7 @@ static inline void blk_queue_disable_secure_erase(struct request_queue *q)static inline void blk_queue_disable_write_zeroes (struct request_queue *q) { q->limits.max_write_zeroes_sectors =0 ; + q->limits.max_wzeroes_unmap_sectors =0 ; }/*
@@ -1378,6 +1382,12 @@ static inline unsigned int bdev_write_zeroes_sectors(struct block_device *bdev)return bdev_limits (bdev)->max_write_zeroes_sectors; } +static inline unsigned int +bdev_write_zeroes_unmap_sectors (struct block_device *bdev) +{ +return bdev_limits (bdev)->max_wzeroes_unmap_sectors; +} +static inline bool bdev_nonrot (struct block_device *bdev) {return blk_queue_nonrot (bdev_get_queue (bdev));
Signature · 1 line
2.46.1
Re: [PATCH v2 1/9] block: introduce max_{hw|user}_wzeroes_unmap_sectors to queue limits
Looks good:
Re: [PATCH v2 1/9] block: introduce max_{hw|user}_wzeroes_unmap_sectors to queue limits
In reply to Zhang Yiblock/blk-settings.c int blk_validate_limits(struct queue_limits *lim) · line 340
Quoted 8 lines · Zhang Yi
}block/blk-settings.c +3
-0@@ -333,6 +335,12 @@ int blk_validate_limits(struct queue_limits *lim)if (!lim->max_segments) lim->max_segments =BLK_MAX_SEGMENTS ; +if (lim->max_hw_wzeroes_unmap_sectors && + lim->max_hw_wzeroes_unmap_sectors != lim->max_write_zeroes_sectors) +return -EINVAL ;
JFYI, I noticed that I am failing this check in raid0_set_limits() -> queue_limits_set() -> queue_limits_commit_update() -> blk_validate_limits() for v6.17-rc2
The raid0 array consists of NVMe partitions. Here lim->max_hw_wzeroes_unmap_sectors = 4096 and lim->max_write_zeroes_sectors = 0 values for the failure, above.
john@raspberrypi:~ $ cat /sys/block/nvme0n1/queue/write_zeroes_max_bytes 2097152
john@raspberrypi:~ $ cat /sys/block/nvme0n1/queue/write_zeroes_unmap_max_bytes 2097152
john@raspberrypi:~ $ cat /sys/block/nvme0n1/queue/write_zeroes_unmap_max_hw_bytes
2097152
john@raspberrypi:~ $
Quoted 7 lines
block/blk-settings.c +3
-0+ lim->max_wzeroes_unmap_sectors =min (lim->max_hw_wzeroes_unmap_sectors, + lim->max_user_wzeroes_unmap_sectors); + lim->max_discard_sectors =min (lim->max_hw_discard_sectors, lim->max_user_discard_sectors);@@ -418,10 +426,11 @@ int blk_set_default_limits(struct queue_limits *lim)
Re: [PATCH v2 1/9] block: introduce max_{hw|user}_wzeroes_unmap_sectors to queue limits
Quoted 13 lines · John Garry
Quoted 7 lines · Zhang Yi
}
block/blk-settings.c +0
-0@@ -333,6 +335,12 @@ int blk_validate_limits(struct queue_limits *lim) if (!lim->max_segments) lim->max_segments = BLK_MAX_SEGMENTS;+ if (
lim->max_hw_wzeroes_unmap_sectors&&
+lim->max_hw_wzeroes_unmap_sectors!=lim->max_write_zeroes_sectors) + return-EINVAL;JFYI, I noticed that I am failing this check in
raid0_set_limits()->queue_limits_set()->queue_limits_commit_update()->blk_validate_limits()for v6.17-rc2The raid0 array consists of NVMe partitions. Here
lim->max_hw_wzeroes_unmap_sectors= 4096 andlim->max_write_zeroes_sectors= 0 values for the failure, above.
Quoted 10 lines
john@raspberrypi:~ $ cat
/sys/block/nvme0n1/queue/write_zeroes_max_bytes2097152
john@raspberrypi:~ $ cat/sys/block/nvme0n1/queue/write_zeroes_unmap_max_bytes2097152
john@raspberrypi:~ $ cat/sys/block/nvme0n1/queue/write_zeroes_unmap_max_hw_bytes
2097152
john@raspberrypi:~ $
Thank you for checking on this!
The problem is that raid0_set_limits() only sets max_write_zeroes_sectors without synchronously setting max_hw_wzeroes_unmap_sectors. It appears that all stacked drivers that call blk_set_stacking_limits() to initialize stacked limits but independently adjust max_write_zeroes_sectors are problematic, including all md drivers and drbd. These drivers need to update max_hw_wzeroes_unmap_sectors as well, I will send out a fix soon.
Thanks,
Yi.
Quoted 7 lines
Quoted 7 lines
+
lim->max_wzeroes_unmap_sectors=min(lim->max_hw_wzeroes_unmap_sectors, + lim->max_user_wzeroes_unmap_sectors);
+
lim->max_discard_sectors=min(lim->max_hw_discard_sectors, lim->max_user_discard_sectors); @@ -418,10 +426,11 @@ intblk_set_default_limits(struct queue_limits *lim)
PATCH v2 2/9 nvme: set max_hw_wzeroes_unmap_sectors if device supports DEAC bit
In reply to Zhang YiFrom: Zhang Yi <[email protected]>
When the device supports the Write Zeroes command and the DEAC bit, it indicates that the deallocate bit in the Write Zeroes command is supported, and the bytes read from a deallocated logical block are zeroes. This means the device supports unmap Write Zeroes operation, so set the max_hw_wzeroes_unmap_sectors to max_write_zeroes_sectors on the device's queue limit.
--- drivers/nvme/host/core.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-)
drivers/nvme/host/core.c +11 -9
@@ -2420,22 +2420,24 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,else lim.write_stream_granularity =0 ; - ret =queue_limits_commit_update (ns->disk->queue, &lim); -if (ret) { -blk_mq_unfreeze_queue (ns->disk->queue, memflags); - goto out; - } - -set_capacity_and_notify (ns->disk, capacity); -/* * Only set the DEAC bit if the device guarantees that reads from * deallocated data return zeroes. While the DEAC bit does not * require that, it must be a no-op if reads from deallocated data * do not return zeroes. */ -if ((id->dlfeat &0x7 ) ==0x1 && (id->dlfeat & (1 <<3 ))) +if ((id->dlfeat &0x7 ) ==0x1 && (id->dlfeat & (1 <<3 ))) { ns->head->features |=NVME_NS_DEAC ; + lim.max_hw_wzeroes_unmap_sectors = lim.max_write_zeroes_sectors; + } + + ret =queue_limits_commit_update (ns->disk->queue, &lim); +if (ret) { +blk_mq_unfreeze_queue (ns->disk->queue, memflags); + goto out; + } + +set_capacity_and_notify (ns->disk, capacity);set_disk_ro (ns->disk,nvme_ns_is_readonly (ns, info));set_bit (NVME_NS_READY , &ns->flags);blk_mq_unfreeze_queue (ns->disk->queue, memflags);
Signature · 1 line
2.46.1
Re: [PATCH v2 2/9] nvme: set max_hw_wzeroes_unmap_sectors if device supports DEAC bit
Looks good:
Re: [PATCH v2 2/9] nvme: set max_hw_wzeroes_unmap_sectors if device supports DEAC bit
In reply to Zhang YiDear Zhang Yi,
In reviewing your patch series implementing support for the FALLOC_FL_WRITE_ZEROES flag, I noted the logic propagating max_write_zeroes_sectors to max_hw_wzeroes_unmap_sectors in commit 545fb46e5bc6 "nvme: set max_hw_wzeroes_unmap_sectors if device supports DEAC bit" [1]. This appears to be intended for devices that support the Write Zeroes command alongside the DEAC bit to indicate unmap capability.
Furthermore, within core.c, the NVME_QUIRK_DEALLOCATE_ZEROES quirk already identifies devices that deterministically return zeroes after a deallocate command [2]. This quirk currently enables Write Zeroes support via discard in existing implementations [3, 4].
Given this, would it be appropriate to respect NVME_QUIRK_DEALLOCATE_ZEROES also to enable unmap Write Zeroes for these devices, following the prior commit 6e02318eaea5 "nvme: add support for the Write Zeroes command" [5]? I have included a proposed change to nvme_update_ns_info_block() below for your consideration.
Best regards
Robert Pang
drivers/nvme/host/core.c +3 -1
@@ -2422,7 +2422,9 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,* require that, it must be a no-op if reads from deallocated data * do not return zeroes. */ -if ((id->dlfeat &0x7 ) ==0x1 && (id->dlfeat & (1 <<3 ))) { +if ((id->dlfeat &0x7 ) ==0x1 && (id->dlfeat & (1 <<3 )) || + (ns->ctrl->quirks &NVME_QUIRK_DEALLOCATE_ZEROES ) && + (ns->ctrl->oncs &NVME_CTRL_ONCS_DSM )) { ns->head->features |=NVME_NS_DEAC ; lim.max_hw_wzeroes_unmap_sectors = lim.max_write_zeroes_sectors; }
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=545fb46e5bc6
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/nvme.h#n72
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/core.c#n938
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/core.c#n2122
[5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6e02318eaea5
Re: [PATCH v2 2/9] nvme: set max_hw_wzeroes_unmap_sectors if device supports DEAC bit
Hi Robert!
Quoted 14 lines · Robert Pang
Dear Zhang Yi,
In reviewing your patch series implementing support for the
FALLOC_FL_WRITE_ZEROESflag, I noted the logic propagatingmax_write_zeroes_sectorstomax_hw_wzeroes_unmap_sectorsin commit545fb46e5bc6"nvme: setmax_hw_wzeroes_unmap_sectorsif device supports DEAC bit" [1]. This appears to be intended for devices that support the Write Zeroes command alongside the DEAC bit to indicate unmap capability.Furthermore, within
core.c, theNVME_QUIRK_DEALLOCATE_ZEROESquirk already identifies devices that deterministically return zeroes after a deallocate command [2]. This quirk currently enables Write Zeroes support via discard in existing implementations [3, 4].
Quoted 6 lines
Given this, would it be appropriate to respect
NVME_QUIRK_DEALLOCATE_ZEROESalso to enable unmap Write Zeroes for these devices, following the prior commit6e02318eaea5"nvme: add support for the Write Zeroes command" [5]? I have included a proposed change tonvme_update_ns_info_block()below for your consideration.
Thank you for your point. Overall, this makes sense to me, but I have one question below.
Quoted 8 lines
Best regards
Robert Pangdrivers/nvme/host/core.c +0
-0
drivers/nvme/host/core.c static int nvme_update_ns_info_block(struct nvme_ns *ns, · line 2427
Quoted 7 lines
drivers/nvme/host/core.c +3
-1@@ -2422,7 +2422,9 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,* require that, it must be a no-op if reads from deallocated data * do not return zeroes. */ -if ((id->dlfeat &0x7 ) ==0x1 && (id->dlfeat & (1 <<3 ))) { +if ((id->dlfeat &0x7 ) ==0x1 && (id->dlfeat & (1 <<3 )) || + (ns->ctrl->quirks &NVME_QUIRK_DEALLOCATE_ZEROES ) && + (ns->ctrl->oncs &NVME_CTRL_ONCS_DSM )) {
^^^^^^^^^^^^^^^^^^
Why do you want to add a check for NVME_CTRL_ONCS_DSM? In nvme_config_discard(), it appears that we prioritize ctrl->dmrsl, allowing discard to still be supported even on some non-standard devices where NVME_CTRL_ONCS_DSM is not set. In nvme_update_disk_info(), if the device only has NVME_QUIRK_DEALLOCATE_ZEROES, we still populate lim->max_write_zeroes_sectors (which might be non-zero on devices that support NVME_CTRL_ONCS_WRITE_ZEROES). Right? So I'm not sure if we only need to check for NVME_QUIRK_DEALLOCATE_ZEROES here.
drivers/nvme/host/core.c static int nvme_update_ns_info_block(struct nvme_ns *ns, · line 2428
Quoted 1 line
drivers/nvme/host/core.c +0
-0ns->head->features |=NVME_NS_DEAC ;
I think we should not set NVME_NS_DEAC for the quirks case.
Cheers,
Yi.
Quoted 8 lines
drivers/nvme/host/core.c +0
-0lim.max_hw_wzeroes_unmap_sectors = lim.max_write_zeroes_sectors; }[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=545fb46e5bc6
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/nvme.h#n72
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/core.c#n938
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/core.c#n2122
[5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6e02318eaea5
Re: [PATCH v2 2/9] nvme: set max_hw_wzeroes_unmap_sectors if device supports DEAC bit
Dear Zhang Yi
Thank you for your quick response. Please see my comments below:
Quoted 44 lines · Zhang Yi
Hi Robert!
Quoted 14 lines · Robert Pang
Dear Zhang Yi,
In reviewing your patch series implementing support for the
FALLOC_FL_WRITE_ZEROESflag, I noted the logic propagatingmax_write_zeroes_sectorstomax_hw_wzeroes_unmap_sectorsin commit545fb46e5bc6"nvme: setmax_hw_wzeroes_unmap_sectorsif device supports DEAC bit" [1]. This appears to be intended for devices that support the Write Zeroes command alongside the DEAC bit to indicate unmap capability.Furthermore, within
core.c, theNVME_QUIRK_DEALLOCATE_ZEROESquirk already identifies devices that deterministically return zeroes after a deallocate command [2]. This quirk currently enables Write Zeroes support via discard in existing implementations [3, 4].Quoted 6 lines
Given this, would it be appropriate to respect
NVME_QUIRK_DEALLOCATE_ZEROESalso to enable unmap Write Zeroes for these devices, following the prior commit6e02318eaea5"nvme: add support for the Write Zeroes command" [5]? I have included a proposed change tonvme_update_ns_info_block()below for your consideration.Thank you for your point. Overall, this makes sense to me, but I have one question below.
Quoted 8 lines
Best regards
Robert Pangdrivers/nvme/host/core.c +0
-0drivers/nvme/host/core.c static int nvme_update_ns_info_block(struct nvme_ns *ns, · line 2427
Quoted 7 lines
drivers/nvme/host/core.c +3
-1@@ -2422,7 +2422,9 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,* require that, it must be a no-op if reads from deallocated data * do not return zeroes. */ -if ((id->dlfeat &0x7 ) ==0x1 && (id->dlfeat & (1 <<3 ))) { +if ((id->dlfeat &0x7 ) ==0x1 && (id->dlfeat & (1 <<3 )) || + (ns->ctrl->quirks &NVME_QUIRK_DEALLOCATE_ZEROES ) && + (ns->ctrl->oncs &NVME_CTRL_ONCS_DSM )) {^^^^^^^^^^^^^^^^^^
Quoted 8 lines
Why do you want to add a check for
NVME_CTRL_ONCS_DSM? Innvme_config_discard(), it appears that we prioritizectrl->dmrsl, allowing discard to still be supported even on some non-standard devices whereNVME_CTRL_ONCS_DSMis not set. Innvme_update_disk_info(), if the device only hasNVME_QUIRK_DEALLOCATE_ZEROES, we still populatelim->max_write_zeroes_sectors(which might be non-zero on devices that supportNVME_CTRL_ONCS_WRITE_ZEROES). Right? So I'm not sure if we only need to check forNVME_QUIRK_DEALLOCATE_ZEROEShere.
The check for NVME_CTRL_ONCS_DSM is to follow the same check in [3]. There, the check was added by 58a0c875ce02 "nvme: don't apply NVME_QUIRK_DEALLOCATE_ZEROES when DSM is not supported" [6]. The idea is to limit NVME_QUIRK_DEALLOCATE_ZEROES
to those devices that support DSM.
Quoted 4 lines
drivers/nvme/host/core.c static int nvme_update_ns_info_block(struct nvme_ns *ns, · line 2428
Quoted 1 line
drivers/nvme/host/core.c +0
-0ns->head->features |=NVME_NS_DEAC ;I think we should not set
NVME_NS_DEACfor the quirks case.
Make sense. In that case, will it be more appropriate to set max_hw_wzeroes_unmap_sectors in nvme_update_disk_info() where NVME_QUIRK_DEALLOCATE_ZEROES is checked? I.e.
drivers/nvme/host/core.c +0 -0
@@ -2120,9 +2120,10 @@ static bool nvme_update_disk_info(struct
nvme_ns *ns, struct nvme_id_ns *id,
lim->io_min = phys_bs;
lim->io_opt = io_opt;
if ((ns->ctrl->quirks & NVME_QUIRK_DEALLOCATE_ZEROES) &&
- (ns->ctrl->oncs & NVME_CTRL_ONCS_DSM))
+ (ns->ctrl->oncs & NVME_CTRL_ONCS_DSM)) {
lim->max_write_zeroes_sectors = UINT_MAX;
- else
+ lim->max_hw_wzeroes_unmap_sectors = UINT_MAX;
+ } else
lim->max_write_zeroes_sectors = ns->ctrl->max_zeroes_sectors;
return valid;}
Best regards
Robert
Quoted 11 lines
Cheers,
Yi.Quoted 8 lines
drivers/nvme/host/core.c +0
-0lim.max_hw_wzeroes_unmap_sectors = lim.max_write_zeroes_sectors; }[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=545fb46e5bc6
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/nvme.h#n72
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/core.c#n938
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/core.c#n2122
[5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6e02318eaea5
[6] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=58a0c875ce02
Re: [PATCH v2 2/9] nvme: set max_hw_wzeroes_unmap_sectors if device supports DEAC bit
Quoted 57 lines · Robert Pang
Dear Zhang Yi
Thank you for your quick response. Please see my comments below:
Quoted 44 lines · Zhang Yi
Hi Robert!
Quoted 14 lines · Robert Pang
Dear Zhang Yi,
In reviewing your patch series implementing support for the
FALLOC_FL_WRITE_ZEROESflag, I noted the logic propagatingmax_write_zeroes_sectorstomax_hw_wzeroes_unmap_sectorsin commit545fb46e5bc6"nvme: setmax_hw_wzeroes_unmap_sectorsif device supports DEAC bit" [1]. This appears to be intended for devices that support the Write Zeroes command alongside the DEAC bit to indicate unmap capability.Furthermore, within
core.c, theNVME_QUIRK_DEALLOCATE_ZEROESquirk already identifies devices that deterministically return zeroes after a deallocate command [2]. This quirk currently enables Write Zeroes support via discard in existing implementations [3, 4].Quoted 6 lines
Given this, would it be appropriate to respect
NVME_QUIRK_DEALLOCATE_ZEROESalso to enable unmap Write Zeroes for these devices, following the prior commit6e02318eaea5"nvme: add support for the Write Zeroes command" [5]? I have included a proposed change tonvme_update_ns_info_block()below for your consideration.Thank you for your point. Overall, this makes sense to me, but I have one question below.
Quoted 8 lines
Best regards
Robert Pangdrivers/nvme/host/core.c +0
-0drivers/nvme/host/core.c static int nvme_update_ns_info_block(struct nvme_ns *ns, · line 2427
Quoted 7 lines
drivers/nvme/host/core.c +3
-1@@ -2422,7 +2422,9 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,* require that, it must be a no-op if reads from deallocated data * do not return zeroes. */ -if ((id->dlfeat &0x7 ) ==0x1 && (id->dlfeat & (1 <<3 ))) { +if ((id->dlfeat &0x7 ) ==0x1 && (id->dlfeat & (1 <<3 )) || + (ns->ctrl->quirks &NVME_QUIRK_DEALLOCATE_ZEROES ) && + (ns->ctrl->oncs &NVME_CTRL_ONCS_DSM )) {^^^^^^^^^^^^^^^^^^Quoted 8 lines
Why do you want to add a check for
NVME_CTRL_ONCS_DSM? Innvme_config_discard(), it appears that we prioritizectrl->dmrsl, allowing discard to still be supported even on some non-standard devices whereNVME_CTRL_ONCS_DSMis not set. Innvme_update_disk_info(), if the device only hasNVME_QUIRK_DEALLOCATE_ZEROES, we still populatelim->max_write_zeroes_sectors(which might be non-zero on devices that supportNVME_CTRL_ONCS_WRITE_ZEROES). Right? So I'm not sure if we only need to check forNVME_QUIRK_DEALLOCATE_ZEROEShere.
Quoted 6 lines
The check for
NVME_CTRL_ONCS_DSMis to follow the same check in [3]. There, the check was added by58a0c875ce02"nvme: don't applyNVME_QUIRK_DEALLOCATE_ZEROESwhen DSM is not supported" [6]. The idea is to limitNVME_QUIRK_DEALLOCATE_ZEROES
to those devices that support DSM.
OK.
Quoted 25 lines
Quoted 4 lines
drivers/nvme/host/core.c static int nvme_update_ns_info_block(struct nvme_ns *ns, · line 2428
Quoted 1 line
drivers/nvme/host/core.c +0
-0ns->head->features |=NVME_NS_DEAC ;I think we should not set
NVME_NS_DEACfor the quirks case.Make sense. In that case, will it be more appropriate to set
max_hw_wzeroes_unmap_sectorsinnvme_update_disk_info()whereNVME_QUIRK_DEALLOCATE_ZEROESis checked? I.e.drivers/nvme/host/core.c +3
-2@@ -2120,9 +2120,10 @@ static bool nvme_update_disk_info(struct nvme_ns *ns, struct nvme_id_ns *id, lim->io_min = phys_bs; lim->io_opt = io_opt;if ((ns->ctrl->quirks &NVME_QUIRK_DEALLOCATE_ZEROES ) && - (ns->ctrl->oncs &NVME_CTRL_ONCS_DSM )) + (ns->ctrl->oncs &NVME_CTRL_ONCS_DSM )) { lim->max_write_zeroes_sectors =UINT_MAX ; -else + lim->max_hw_wzeroes_unmap_sectors =UINT_MAX ; + }else lim->max_write_zeroes_sectors = ns->ctrl->max_zeroes_sectors;return valid;
Quoted 2 lines
}
Yeah, it looks good to me.
Best regards,
Yi.
Quoted 15 lines
Best regards
RobertQuoted 11 lines
Cheers,
Yi.Quoted 8 lines
drivers/nvme/host/core.c +0
-0lim.max_hw_wzeroes_unmap_sectors = lim.max_write_zeroes_sectors; }[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=545fb46e5bc6
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/nvme.h#n72
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/core.c#n938
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/core.c#n2122
[5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6e02318eaea5[6] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=58a0c875ce02
Re: [PATCH v2 2/9] nvme: set max_hw_wzeroes_unmap_sectors if device supports DEAC bit
Dear Zhang Yi
Quoted 89 lines · Zhang Yi
Quoted 57 lines · Robert Pang
Dear Zhang Yi
Thank you for your quick response. Please see my comments below:
Quoted 44 lines · Zhang Yi
Hi Robert!
Quoted 14 lines · Robert Pang
Dear Zhang Yi,
In reviewing your patch series implementing support for the
FALLOC_FL_WRITE_ZEROESflag, I noted the logic propagatingmax_write_zeroes_sectorstomax_hw_wzeroes_unmap_sectorsin commit545fb46e5bc6"nvme: setmax_hw_wzeroes_unmap_sectorsif device supports DEAC bit" [1]. This appears to be intended for devices that support the Write Zeroes command alongside the DEAC bit to indicate unmap capability.Furthermore, within
core.c, theNVME_QUIRK_DEALLOCATE_ZEROESquirk already identifies devices that deterministically return zeroes after a deallocate command [2]. This quirk currently enables Write Zeroes support via discard in existing implementations [3, 4].Quoted 6 lines
Given this, would it be appropriate to respect
NVME_QUIRK_DEALLOCATE_ZEROESalso to enable unmap Write Zeroes for these devices, following the prior commit6e02318eaea5"nvme: add support for the Write Zeroes command" [5]? I have included a proposed change tonvme_update_ns_info_block()below for your consideration.Thank you for your point. Overall, this makes sense to me, but I have one question below.
Quoted 8 lines
Best regards
Robert Pangdrivers/nvme/host/core.c +0
-0drivers/nvme/host/core.c static int nvme_update_ns_info_block(struct nvme_ns *ns, · line 2427
Quoted 7 lines
drivers/nvme/host/core.c +3
-1@@ -2422,7 +2422,9 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,* require that, it must be a no-op if reads from deallocated data * do not return zeroes. */ -if ((id->dlfeat &0x7 ) ==0x1 && (id->dlfeat & (1 <<3 ))) { +if ((id->dlfeat &0x7 ) ==0x1 && (id->dlfeat & (1 <<3 )) || + (ns->ctrl->quirks &NVME_QUIRK_DEALLOCATE_ZEROES ) && + (ns->ctrl->oncs &NVME_CTRL_ONCS_DSM )) {^^^^^^^^^^^^^^^^^^Quoted 8 lines
Why do you want to add a check for
NVME_CTRL_ONCS_DSM? Innvme_config_discard(), it appears that we prioritizectrl->dmrsl, allowing discard to still be supported even on some non-standard devices whereNVME_CTRL_ONCS_DSMis not set. Innvme_update_disk_info(), if the device only hasNVME_QUIRK_DEALLOCATE_ZEROES, we still populatelim->max_write_zeroes_sectors(which might be non-zero on devices that supportNVME_CTRL_ONCS_WRITE_ZEROES). Right? So I'm not sure if we only need to check forNVME_QUIRK_DEALLOCATE_ZEROEShere.Quoted 6 lines
The check for
NVME_CTRL_ONCS_DSMis to follow the same check in [3]. There, the check was added by58a0c875ce02"nvme: don't applyNVME_QUIRK_DEALLOCATE_ZEROESwhen DSM is not supported" [6]. The idea is to limitNVME_QUIRK_DEALLOCATE_ZEROES
to those devices that support DSM.OK.
Quoted 21 lines
Quoted 4 lines
drivers/nvme/host/core.c static int nvme_update_ns_info_block(struct nvme_ns *ns, · line 2428
Quoted 1 line
drivers/nvme/host/core.c +0
-0ns->head->features |=NVME_NS_DEAC ;I think we should not set
NVME_NS_DEACfor the quirks case.Make sense. In that case, will it be more appropriate to set
max_hw_wzeroes_unmap_sectorsinnvme_update_disk_info()whereNVME_QUIRK_DEALLOCATE_ZEROESis checked? I.e.drivers/nvme/host/core.c +1
-2@@ -2120,9 +2120,10 @@ static bool nvme_update_disk_info(struct nvme_ns *ns, struct nvme_id_ns *id, lim->io_min = phys_bs; lim->io_opt = io_opt;if ((ns->ctrl->quirks &NVME_QUIRK_DEALLOCATE_ZEROES ) && - (ns->ctrl->oncs &NVME_CTRL_ONCS_DSM )) + (ns->ctrl->oncs &NVME_CTRL_ONCS_DSM )) { lim->max_write_zeroes_sectors =UINT_MAX ; -else
Quoted 8 lines
Quoted 6 lines
drivers/nvme/host/core.c +2
-0+ lim->max_hw_wzeroes_unmap_sectors =UINT_MAX ; + }else lim->max_write_zeroes_sectors = ns->ctrl->max_zeroes_sectors;return valid;}
Yeah, it looks good to me.
Thank you for your confirmation. I will follow up and submit the patch to other maintainers for their review.
Best regards,
Robert
Quoted 19 lines
Best regards,
Yi.Quoted 15 lines
Best regards
RobertQuoted 11 lines
Cheers,
Yi.Quoted 8 lines
drivers/nvme/host/core.c +0
-0lim.max_hw_wzeroes_unmap_sectors = lim.max_write_zeroes_sectors; }[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=545fb46e5bc6
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/nvme.h#n72
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/core.c#n938
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/nvme/host/core.c#n2122
[5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6e02318eaea5[6] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=58a0c875ce02
PATCH v2 3/9 nvmet: set WZDS and DRB if device enables unmap write zeroes operation
In reply to Zhang YiFrom: Zhang Yi <[email protected]>
Set the WZDS and DRB bits to the namespace dlfeat if the underlying block device enables the unmap write zeroes operation, make the nvme target device supports the unmap write zeroes command.
--- drivers/nvme/target/io-cmd-bdev.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/nvme/target/io-cmd-bdev.c +4 -0
@@ -46,6 +46,10 @@ void nvmet_bdev_set_limits(struct block_device *bdev, struct nvme_id_ns *id) id->npda = id->npdg;/* NOWS = Namespace Optimal Write Size */ id->nows =to0based (bdev_io_opt (bdev) /bdev_logical_block_size (bdev)); + +/* Set WZDS and DRB if device supports unmapped write zeroes */ +if (bdev_write_zeroes_unmap_sectors (bdev)) + id->dlfeat = (1 <<3 ) |0x1 ; }void nvmet_bdev_ns_disable (struct nvmet_ns *ns)
Signature · 1 line
2.46.1
Re: [PATCH v2 3/9] nvmet: set WZDS and DRB if device enables unmap write zeroes operation
Looks good:
PATCH v2 4/9 scsi: sd: set max_hw_wzeroes_unmap_sectors if device supports SD_ZERO_*_UNMAP
In reply to Zhang YiFrom: Zhang Yi <[email protected]>
When the device supports the Write Zeroes command and the zeroing mode is set to SD_ZERO_WS16_UNMAP or SD_ZERO_WS10_UNMAP, this means that the device supports unmap Write Zeroes, so set the corresponding max_hw_write_zeroes_unmap_sectors to max_write_zeroes_sectors on the device's queue limit.
--- drivers/scsi/sd.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/scsi/sd.c +5 -0
@@ -1141,6 +1141,11 @@ static void sd_config_write_same(struct scsi_disk *sdkp, out: lim->max_write_zeroes_sectors = sdkp->max_ws_blocks * (logical_block_size >>SECTOR_SHIFT ); + +if (sdkp->zeroing_mode ==SD_ZERO_WS16_UNMAP || + sdkp->zeroing_mode ==SD_ZERO_WS10_UNMAP ) + lim->max_hw_wzeroes_unmap_sectors = + lim->max_write_zeroes_sectors; }static blk_status_t sd_setup_flush_cmnd (struct scsi_cmnd *cmd)
Signature · 1 line
2.46.1
PATCH v2 5/9 dm: clear unmap write zeroes limits when disabling write zeroes
In reply to Zhang YiFrom: Zhang Yi <[email protected]>
The unmap write zeroes limits have been set to the stacking queue limits by default in blk_set_stacking_limits() and blk_stack_limits(), but it should be cleared if any underlying device does not support it.
--- drivers/md/dm-table.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/md/dm-table.c +3 -1
@@ -2065,8 +2065,10 @@ int dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, limits->discard_alignment =0 ; } -if (!dm_table_supports_write_zeroes (t)) +if (!dm_table_supports_write_zeroes (t)) { limits->max_write_zeroes_sectors =0 ; + limits->max_hw_wzeroes_unmap_sectors =0 ; + }if (!dm_table_supports_secure_erase (t)) limits->max_secure_erase_sectors =0 ;
Signature · 1 line
2.46.1
PATCH v2 6/9 fs: introduce FALLOC_FL_WRITE_ZEROES to fallocate
In reply to Zhang YiFrom: Zhang Yi <[email protected]>
With the development of flash-based storage devices, we can quickly write zeros to SSDs using the WRITE_ZERO command if the devices do not actually write physical zeroes to the media. Therefore, we can use this command to quickly preallocate a real all-zero file with written extents. This approach should be beneficial for subsequent pure overwriting within this file, as it can save on block allocation and, consequently, significant metadata changes, which should greatly improve overwrite performance on certain filesystems.
Therefore, introduce a new operation FALLOC_FL_WRITE_ZEROES to fallocate. This flag is used to convert a specified range of a file to zeros by issuing a zeroing operation. Blocks should be allocated for the regions that span holes in the file, and the entire range is converted to written extents. If the underlying device supports the actual offload write zeroes command, the process of zeroing out operation can be accelerated. If it does not, we currently don't prevent the file system from writing actual zeros to the device. This provides users with a new method to quickly generate a zeroed file, users no longer need to write zero data to create a file with written extents.
Users can determine whether a disk supports the unmap write zeroes feature through querying this sysfs interface:
/sys/block/<disk>/queue/write_zeroes_unmap_max_hw_bytes
Users can also enable or disable the unmap write zeroes operation through this sysfs interface:
/sys/block/<disk>/queue/write_zeroes_unmap_max_bytes
Finally, this flag cannot be specified in conjunction with the FALLOC_FL_KEEP_SIZE since allocating written extents beyond file EOF is not permitted. In addition, filesystems that always require out-of-place writes should not support this flag since they still need to allocated new blocks during subsequent overwrites.
--- fs/open.c | 1 + include/linux/falloc.h | 3 ++- include/uapi/linux/falloc.h | 17 +++++++++++++++++ 3 files changed, 20 insertions(+), 1 deletion(-)
fs/open.c +1 -0
@@ -281,6 +281,7 @@ int vfs_fallocate(struct file *file, int mode, loff_t offset, loff_t len)break ;case FALLOC_FL_COLLAPSE_RANGE :case FALLOC_FL_INSERT_RANGE : +case FALLOC_FL_WRITE_ZEROES :if (mode &FALLOC_FL_KEEP_SIZE )return -EOPNOTSUPP ;break ;
include/linux/falloc.h +2 -1
@@ -36,7 +36,8 @@ struct space_resv { FALLOC_FL_COLLAPSE_RANGE | \ FALLOC_FL_ZERO_RANGE | \ FALLOC_FL_INSERT_RANGE | \ - FALLOC_FL_UNSHARE_RANGE) + FALLOC_FL_UNSHARE_RANGE | \ + FALLOC_FL_WRITE_ZEROES)/* on ia32 l_start is on a 32-bit boundary */ #if defined(CONFIG_X86_64 )
include/uapi/linux/falloc.h +17 -0
@@ -78,4 +78,21 @@*/ #define FALLOC_FL_UNSHARE_RANGE 0x40 +/* +* FALLOC_FL_WRITE_ZEROES zeroes a specified file range in such a way that +* subsequent writes to that range do not require further changes to the file +* mapping metadata. This flag is beneficial for subsequent pure overwriting +* within this range, as it can save on block allocation and, consequently, +* significant metadata changes. Therefore, filesystems that always require +* out-of-place writes should not support this flag. +* +* Different filesystems may implement different limitations on the +* granularity of the zeroing operation. Most will preferably be accelerated +* by submitting write zeroes command if the backing storage supports, which +* may not physically write zeros to the media. +* +* This flag cannot be specified in conjunction with the FALLOC_FL_KEEP_SIZE. +*/ +#define FALLOC_FL_WRITE_ZEROES 0x80 +#endif /* _UAPI_FALLOC_H_ */
Signature · 1 line
2.46.1
PATCH v2 7/9 block: factor out common part in blkdev_fallocate()
In reply to Zhang YiFrom: Zhang Yi <[email protected]>
Only the flags passed to blkdev_issue_zeroout() differ among the two zeroing branches in blkdev_fallocate(). Therefore, do cleanup by factoring them out.
--- block/fops.c | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-)
block/fops.c +14 -18
@@ -850,6 +850,7 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start,struct block_device *bdev =I_BDEV (inode);loff_t end = start + len -1 ;loff_t isize; +unsigned int flags;int error;/* Fail if we don't recognize the flags. */
@@ -877,34 +878,29 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start,inode_lock (inode);filemap_invalidate_lock (inode->i_mapping); -/* -* Invalidate the page cache, including dirty pages, for valid -* de-allocate mode calls to fallocate(). -*/ switch (mode) {case FALLOC_FL_ZERO_RANGE :case FALLOC_FL_ZERO_RANGE |FALLOC_FL_KEEP_SIZE : - error =truncate_bdev_range (bdev,file_to_blk_mode (file), start, end); -if (error) - goto fail; - - error =blkdev_issue_zeroout (bdev, start >>SECTOR_SHIFT , - len >>SECTOR_SHIFT ,GFP_KERNEL , -BLKDEV_ZERO_NOUNMAP ); + flags =BLKDEV_ZERO_NOUNMAP ;break ;case FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE : - error =truncate_bdev_range (bdev,file_to_blk_mode (file), start, end); -if (error) - goto fail; - - error =blkdev_issue_zeroout (bdev, start >>SECTOR_SHIFT , - len >>SECTOR_SHIFT ,GFP_KERNEL , -BLKDEV_ZERO_NOFALLBACK ); + flags =BLKDEV_ZERO_NOFALLBACK ;break ;default : error = -EOPNOTSUPP ; + goto fail; } +/* +* Invalidate the page cache, including dirty pages, for valid +* de-allocate mode calls to fallocate(). +*/ + error =truncate_bdev_range (bdev,file_to_blk_mode (file), start, end); +if (error) + goto fail; + + error =blkdev_issue_zeroout (bdev, start >>SECTOR_SHIFT , + len >>SECTOR_SHIFT ,GFP_KERNEL , flags); fail:filemap_invalidate_unlock (inode->i_mapping);inode_unlock (inode);
Signature · 1 line
2.46.1
PATCH v2 8/9 block: add FALLOC_FL_WRITE_ZEROES support
In reply to Zhang YiFrom: Zhang Yi <[email protected]>
Add support for FALLOC_FL_WRITE_ZEROES, if the block device enables the unmap write zeroes operation, it will issue a write zeroes command.
--- block/fops.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
block/fops.c +11 -1
@@ -841,7 +841,7 @@ static ssize_t blkdev_read_iter(struct kiocb *iocb, struct iov_iter *to)#define BLKDEV_FALLOC_FL_SUPPORTED \ (FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE | \ - FALLOC_FL_ZERO_RANGE) + FALLOC_FL_ZERO_RANGE | FALLOC_FL_WRITE_ZEROES)static long blkdev_fallocate (struct file *file,int mode,loff_t start,loff_t len)
@@ -856,6 +856,13 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start,/* Fail if we don't recognize the flags. */ if (mode & ~BLKDEV_FALLOC_FL_SUPPORTED )return -EOPNOTSUPP ; +/* +* Don't allow writing zeroes if the device does not enable the +* unmap write zeroes operation. +*/ +if ((mode &FALLOC_FL_WRITE_ZEROES ) && + !bdev_write_zeroes_unmap_sectors (bdev)) +return -EOPNOTSUPP ;/* Don't go off the end of the device. */ isize =bdev_nr_bytes (bdev);
@@ -886,6 +893,9 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start,case FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE : flags =BLKDEV_ZERO_NOFALLBACK ;break ; +case FALLOC_FL_WRITE_ZEROES : + flags =0 ; +break ;default : error = -EOPNOTSUPP ; goto fail;
Signature · 1 line
2.46.1
PATCH v2 9/9 ext4: add FALLOC_FL_WRITE_ZEROES support
In reply to Zhang YiFrom: Zhang Yi <[email protected]>
Add support for FALLOC_FL_WRITE_ZEROES if the underlying device enable the unmap write zeroes operation. This first allocates blocks as unwritten, then issues a zero command outside of the running journal handle, and finally converts them to a written state.
--- fs/ext4/extents.c | 66 ++++++++++++++++++++++++++++++------- include/trace/events/ext4.h | 3 +- 2 files changed, 57 insertions(+), 12 deletions(-)
fs/ext4/extents.c +55 -11
@@ -4501,6 +4501,8 @@ static int ext4_alloc_file_blocks(struct file *file, ext4_lblk_t offset,struct ext4_map_blocks map;unsigned int credits;loff_t epos, old_size =i_size_read (inode); +unsigned int blkbits = inode->i_blkbits; +bool alloc_zero = false;BUG_ON (!ext4_test_inode_flag (inode,EXT4_INODE_EXTENTS )); map.m_lblk = offset;
@@ -4513,6 +4515,17 @@ static int ext4_alloc_file_blocks(struct file *file, ext4_lblk_t offset,if (len <=EXT_UNWRITTEN_MAX_LEN ) flags |=EXT4_GET_BLOCKS_NO_NORMALIZE ; +/* +* Do the actual write zero during a running journal transaction +* costs a lot. First allocate an unwritten extent and then +* convert it to written after zeroing it out. +*/ +if (flags &EXT4_GET_BLOCKS_ZERO ) { + flags &= ~EXT4_GET_BLOCKS_ZERO ; + flags |=EXT4_GET_BLOCKS_UNWRIT_EXT ; + alloc_zero = true; + } +/* * credits to insert 1 extent into extent tree */
@@ -4549,9 +4562,7 @@ static int ext4_alloc_file_blocks(struct file *file, ext4_lblk_t offset,* allow a full retry cycle for any remaining allocations */ retries =0 ; - map.m_lblk += ret; - map.m_len = len = len - ret; - epos = (loff_t )map.m_lblk << inode->i_blkbits; + epos = (loff_t)(map.m_lblk + ret) << blkbits;inode_set_ctime_current (inode);if (new_size) {if (epos > new_size)
@@ -4571,6 +4582,21 @@ static int ext4_alloc_file_blocks(struct file *file, ext4_lblk_t offset, ret2 = ret3 ? ret3 : ret2;if (unlikely (ret2))break ; + +if (alloc_zero && + (map.m_flags & (EXT4_MAP_MAPPED |EXT4_MAP_UNWRITTEN ))) { + ret2 =ext4_issue_zeroout (inode, map.m_lblk, map.m_pblk, + map.m_len); +if (likely (!ret2)) + ret2 =ext4_convert_unwritten_extents (NULL , + inode, (loff_t )map.m_lblk << blkbits, + (loff_t )map.m_len << blkbits); +if (ret2) +break ; + } + + map.m_lblk += ret; + map.m_len = len = len - ret; }if (ret == -ENOSPC &&ext4_should_retry_alloc (inode->i_sb, &retries)) goto retry;
@@ -4636,7 +4662,11 @@ static long ext4_zero_range(struct file *file, loff_t offset,if (end_lblk > start_lblk) {ext4_lblk_t zero_blks = end_lblk - start_lblk; - flags |= (EXT4_GET_BLOCKS_CONVERT_UNWRITTEN |EXT4_EX_NOCACHE ); +if (mode &FALLOC_FL_WRITE_ZEROES ) + flags =EXT4_GET_BLOCKS_CREATE_ZERO |EXT4_EX_NOCACHE ; +else + flags |= (EXT4_GET_BLOCKS_CONVERT_UNWRITTEN | +EXT4_EX_NOCACHE ); ret =ext4_alloc_file_blocks (file, start_lblk, zero_blks, new_size, flags);if (ret)
@@ -4745,11 +4775,18 @@ long ext4_fallocate(struct file *file, int mode, loff_t offset, loff_t len)if (IS_ENCRYPTED (inode) && (mode & (FALLOC_FL_COLLAPSE_RANGE |FALLOC_FL_INSERT_RANGE )))return -EOPNOTSUPP ; +/* +* Don't allow writing zeroes if the underlying device does not +* enable the unmap write zeroes operation. +*/ +if ((mode &FALLOC_FL_WRITE_ZEROES ) && + !bdev_write_zeroes_unmap_sectors (inode->i_sb->s_bdev)) +return -EOPNOTSUPP ;/* Return error if mode is not supported */ if (mode & ~(FALLOC_FL_KEEP_SIZE |FALLOC_FL_PUNCH_HOLE | -FALLOC_FL_COLLAPSE_RANGE |FALLOC_FL_ZERO_RANGE | -FALLOC_FL_INSERT_RANGE )) +FALLOC_FL_ZERO_RANGE |FALLOC_FL_COLLAPSE_RANGE | +FALLOC_FL_INSERT_RANGE |FALLOC_FL_WRITE_ZEROES ))return -EOPNOTSUPP ;inode_lock (inode);
@@ -4780,16 +4817,23 @@ long ext4_fallocate(struct file *file, int mode, loff_t offset, loff_t len)if (ret) goto out_invalidate_lock; -if (mode &FALLOC_FL_PUNCH_HOLE ) +switch (mode &FALLOC_FL_MODE_MASK ) { +case FALLOC_FL_PUNCH_HOLE : ret =ext4_punch_hole (file, offset, len); -else if (mode &FALLOC_FL_COLLAPSE_RANGE ) +break ; +case FALLOC_FL_COLLAPSE_RANGE : ret =ext4_collapse_range (file, offset, len); -else if (mode &FALLOC_FL_INSERT_RANGE ) +break ; +case FALLOC_FL_INSERT_RANGE : ret =ext4_insert_range (file, offset, len); -else if (mode &FALLOC_FL_ZERO_RANGE ) +break ; +case FALLOC_FL_ZERO_RANGE : +case FALLOC_FL_WRITE_ZEROES : ret =ext4_zero_range (file, offset, len, mode); -else +break ; +default : ret = -EOPNOTSUPP ; + } out_invalidate_lock:filemap_invalidate_unlock (mapping);
include/trace/events/ext4.h +2 -1
@@ -92,7 +92,8 @@ TRACE_DEFINE_ENUM(ES_REFERENCED_B); { FALLOC_FL_KEEP_SIZE, "KEEP_SIZE"}, \ { FALLOC_FL_PUNCH_HOLE, "PUNCH_HOLE"}, \ { FALLOC_FL_COLLAPSE_RANGE, "COLLAPSE_RANGE"}, \ - { FALLOC_FL_ZERO_RANGE, "ZERO_RANGE"}) + { FALLOC_FL_ZERO_RANGE, "ZERO_RANGE"}, \ + { FALLOC_FL_WRITE_ZEROES, "WRITE_ZEROES"})TRACE_DEFINE_ENUM (EXT4_FC_REASON_XATTR );TRACE_DEFINE_ENUM (EXT4_FC_REASON_CROSS_RENAME );
Signature · 1 line
2.46.1
Quoted 52 lines · Zhang Yi
From: Zhang Yi <[email protected]>
Changes since v1:
- Rebase codes on 6.16-rc2.
- Use max_{hw|user}_wzeroes_unmap_sectors queue limits instead ofBLK_FEAT_WRITE_ZEROES_UNMAPfeature to represent the status of the unmap write zeroes operation as Christoph and Darrick suggested. This redoes the first 5 patches, so remove all the reviewed-by tags, please review them again.
- Simplify the description ofFALLOC_FL_WRITE_ZEROESin patch 06 as Darrick suggested.
- Revise the check order ofFALLOC_FL_WRITE_ZEROESin patch 08 as Christoph suggested.
Changes since RFC v4:
- Rebase codes on 6.16-rc1.
- Add a newqueue_limitflag, and change thewrite_zeroes_unmapsysfs interface to RW mode. User can disable the unmap write zeroes operation by writing '0' to it when the operation is slow.
- Modify the documentation ofwrite_zeroes_unmapsysfs interface as Martin suggested.
- Remove the statx interface.
- Make the bdev and ext4 don't allow to submitFALLOC_FL_WRITE_ZEROESif the block device does not enable the unmap write zeroes operation, it should return-EOPNOTSUPP.
Changes sicne RFC v3:
- Rebase codes on 6.15-rc2.
- Add a note in patch 1 to indicate that the unmap write zeros command is not always guaranteed as Christoph suggested.
- Renamebdev_unmap_write_zeroes()helper and move it to patch 1 as Christoph suggested.
- Introduce a new statx attribute flagSTATX_ATTR_WRITE_ZEROES_UNMAPas Christoph and Christian suggested.
- Exchange the order of the two patches that modifiedblkdev_fallocate()as Christoph suggested.
Changes since RFC v2:
- Rebase codes on next-20250314.
- Add support for nvme multipath.
- Add support for NVMeT with block device backing.
- ClearFALLOC_FL_WRITE_ZEROESif dm clearlimits->max_write_zeroes_sectors.
- Complement the counterpart userspacetools(util-linux and xfs_io)and tests(blktests and xfstests), please see below for details.
Changes since RFC v1:
- Switch to add a new write zeroes operation,FALLOC_FL_WRITE_ZEROES, in fallocate, instead of just adding a supported flag toFALLOC_FL_ZERO_RANGE.
- Introduce a new flagBLK_FEAT_WRITE_ZEROES_UNMAPto the block device's queue limit features, and implement it on SCSI sd driver, NVMe SSD driver and dm driver.
- ImplementFALLOC_FL_WRITE_ZEROESon both the ext4 filesystem and block device (bdev).
Quoted 1 line
[...]
If needed, the branch can be declared stable and thus be used as base for other work.
---
Applied to the vfs-6.17.fallocate branch of the vfs/vfs.git tree. Patches in the vfs-6.17.fallocate branch should appear in linux-next soon.
Please report any outstanding bugs that were missed during review in a new review to the original patch series allowing us to drop it.
It's encouraged to provide Acked-bys and Reviewed-bys even though the patch has now been applied. If possible patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. If in doubt, please check the listed branch.
tree: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git branch: vfs-6.17.fallocate
[1/9] block: introduce max_{hw|user}_wzeroes_unmap_sectors to queue limits
https://git.kernel.org/vfs/vfs/c/2695a9b086fd [2/9] nvme: set max_hw_wzeroes_unmap_sectors if device supports DEAC bit https://git.kernel.org/vfs/vfs/c/bf07c1180194 [3/9] nvmet: set WZDS and DRB if device enables unmap write zeroes operation https://git.kernel.org/vfs/vfs/c/a6c7ab5adcba [4/9] scsi: sd: set max_hw_wzeroes_unmap_sectors if device supports SD_ZERO_*_UNMAP https://git.kernel.org/vfs/vfs/c/92372ed1cc88 [5/9] dm: clear unmap write zeroes limits when disabling write zeroes https://git.kernel.org/vfs/vfs/c/e383d550e716 [6/9] fs: introduce FALLOC_FL_WRITE_ZEROES to fallocate https://git.kernel.org/vfs/vfs/c/1ed1b5df86ec [7/9] block: factor out common part in blkdev_fallocate() https://git.kernel.org/vfs/vfs/c/96433508c8c0 [8/9] block: add FALLOC_FL_WRITE_ZEROES support https://git.kernel.org/vfs/vfs/c/2b4e5f9b3eb9 [9/9] ext4: add FALLOC_FL_WRITE_ZEROES support https://git.kernel.org/vfs/vfs/c/51954e469396
Quoted 66 lines · Christian Brauner
Quoted 52 lines · Zhang Yi
From: Zhang Yi <[email protected]>
Changes since v1:
- Rebase codes on 6.16-rc2.
- Use max_{hw|user}_wzeroes_unmap_sectors queue limits instead ofBLK_FEAT_WRITE_ZEROES_UNMAPfeature to represent the status of the unmap write zeroes operation as Christoph and Darrick suggested. This redoes the first 5 patches, so remove all the reviewed-by tags, please review them again.
- Simplify the description ofFALLOC_FL_WRITE_ZEROESin patch 06 as Darrick suggested.
- Revise the check order ofFALLOC_FL_WRITE_ZEROESin patch 08 as Christoph suggested.
Changes since RFC v4:
- Rebase codes on 6.16-rc1.
- Add a newqueue_limitflag, and change thewrite_zeroes_unmapsysfs interface to RW mode. User can disable the unmap write zeroes operation by writing '0' to it when the operation is slow.
- Modify the documentation ofwrite_zeroes_unmapsysfs interface as Martin suggested.
- Remove the statx interface.
- Make the bdev and ext4 don't allow to submitFALLOC_FL_WRITE_ZEROESif the block device does not enable the unmap write zeroes operation, it should return-EOPNOTSUPP.
Changes sicne RFC v3:
- Rebase codes on 6.15-rc2.
- Add a note in patch 1 to indicate that the unmap write zeros command is not always guaranteed as Christoph suggested.
- Renamebdev_unmap_write_zeroes()helper and move it to patch 1 as Christoph suggested.
- Introduce a new statx attribute flagSTATX_ATTR_WRITE_ZEROES_UNMAPas Christoph and Christian suggested.
- Exchange the order of the two patches that modifiedblkdev_fallocate()as Christoph suggested.
Changes since RFC v2:
- Rebase codes on next-20250314.
- Add support for nvme multipath.
- Add support for NVMeT with block device backing.
- ClearFALLOC_FL_WRITE_ZEROESif dm clearlimits->max_write_zeroes_sectors.
- Complement the counterpart userspacetools(util-linux and xfs_io)and tests(blktests and xfstests), please see below for details.
Changes since RFC v1:
- Switch to add a new write zeroes operation,FALLOC_FL_WRITE_ZEROES, in fallocate, instead of just adding a supported flag toFALLOC_FL_ZERO_RANGE.
- Introduce a new flagBLK_FEAT_WRITE_ZEROES_UNMAPto the block device's queue limit features, and implement it on SCSI sd driver, NVMe SSD driver and dm driver.
- ImplementFALLOC_FL_WRITE_ZEROESon both the ext4 filesystem and block device (bdev).Quoted 1 line
[...]
If needed, the branch can be declared stable and thus be used as base for other work.
---
Applied to the vfs-6.17.fallocate branch of the vfs/vfs.git tree. Patches in the vfs-6.17.fallocate branch should appear in linux-next soon.
Please report any outstanding bugs that were missed during review in a new review to the original patch series allowing us to drop it.
Quoted 8 lines
It's encouraged to provide Acked-bys and Reviewed-bys even though the patch has now been applied. If possible patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. If in doubt, please check the listed branch.
tree: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git branch: vfs-6.17.fallocate
Hi Christian,
I noticed that this patch series doesn't appear to be merged into this branch. Just wondering if it might have been missed?
Best regards,
Yi.
Quoted 19 lines
[1/9] block: introduce max_{hw|user}_wzeroes_unmap_sectors to queue limits
https://git.kernel.org/vfs/vfs/c/2695a9b086fd [2/9] nvme: set max_hw_wzeroes_unmap_sectors if device supports DEAC bit https://git.kernel.org/vfs/vfs/c/bf07c1180194 [3/9] nvmet: set WZDS and DRB if device enables unmap write zeroes operation https://git.kernel.org/vfs/vfs/c/a6c7ab5adcba [4/9] scsi: sd: set max_hw_wzeroes_unmap_sectors if device supports SD_ZERO_*_UNMAP https://git.kernel.org/vfs/vfs/c/92372ed1cc88 [5/9] dm: clear unmap write zeroes limits when disabling write zeroes https://git.kernel.org/vfs/vfs/c/e383d550e716 [6/9] fs: introduce FALLOC_FL_WRITE_ZEROES to fallocate https://git.kernel.org/vfs/vfs/c/1ed1b5df86ec [7/9] block: factor out common part in blkdev_fallocate() https://git.kernel.org/vfs/vfs/c/96433508c8c0 [8/9] block: add FALLOC_FL_WRITE_ZEROES support https://git.kernel.org/vfs/vfs/c/2b4e5f9b3eb9 [9/9] ext4: add FALLOC_FL_WRITE_ZEROES support https://git.kernel.org/vfs/vfs/c/51954e469396
Christian Brauner
maintainerQuoted 73 lines · Zhang Yi
Quoted 72 lines · Christian Brauner
Quoted 52 lines · Zhang Yi
From: Zhang Yi <[email protected]>
Changes since v1:
- Rebase codes on 6.16-rc2.
- Use max_{hw|user}_wzeroes_unmap_sectors queue limits instead ofBLK_FEAT_WRITE_ZEROES_UNMAPfeature to represent the status of the unmap write zeroes operation as Christoph and Darrick suggested. This redoes the first 5 patches, so remove all the reviewed-by tags, please review them again.
- Simplify the description ofFALLOC_FL_WRITE_ZEROESin patch 06 as Darrick suggested.
- Revise the check order ofFALLOC_FL_WRITE_ZEROESin patch 08 as Christoph suggested.
Changes since RFC v4:
- Rebase codes on 6.16-rc1.
- Add a newqueue_limitflag, and change thewrite_zeroes_unmapsysfs interface to RW mode. User can disable the unmap write zeroes operation by writing '0' to it when the operation is slow.
- Modify the documentation ofwrite_zeroes_unmapsysfs interface as Martin suggested.
- Remove the statx interface.
- Make the bdev and ext4 don't allow to submitFALLOC_FL_WRITE_ZEROESif the block device does not enable the unmap write zeroes operation, it should return-EOPNOTSUPP.
Changes sicne RFC v3:
- Rebase codes on 6.15-rc2.
- Add a note in patch 1 to indicate that the unmap write zeros command is not always guaranteed as Christoph suggested.
- Renamebdev_unmap_write_zeroes()helper and move it to patch 1 as Christoph suggested.
- Introduce a new statx attribute flagSTATX_ATTR_WRITE_ZEROES_UNMAPas Christoph and Christian suggested.
- Exchange the order of the two patches that modifiedblkdev_fallocate()as Christoph suggested.
Changes since RFC v2:
- Rebase codes on next-20250314.
- Add support for nvme multipath.
- Add support for NVMeT with block device backing.
- ClearFALLOC_FL_WRITE_ZEROESif dm clearlimits->max_write_zeroes_sectors.
- Complement the counterpart userspacetools(util-linux and xfs_io)and tests(blktests and xfstests), please see below for details.
Changes since RFC v1:
- Switch to add a new write zeroes operation,FALLOC_FL_WRITE_ZEROES, in fallocate, instead of just adding a supported flag toFALLOC_FL_ZERO_RANGE.
- Introduce a new flagBLK_FEAT_WRITE_ZEROES_UNMAPto the block device's queue limit features, and implement it on SCSI sd driver, NVMe SSD driver and dm driver.
- ImplementFALLOC_FL_WRITE_ZEROESon both the ext4 filesystem and block device (bdev).Quoted 1 line
[...]
If needed, the branch can be declared stable and thus be used as base for other work.
---
Applied to the vfs-6.17.fallocate branch of the vfs/vfs.git tree. Patches in the vfs-6.17.fallocate branch should appear in linux-next soon.
Please report any outstanding bugs that were missed during review in a new review to the original patch series allowing us to drop it.
It's encouraged to provide Acked-bys and Reviewed-bys even though the patch has now been applied. If possible patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. If in doubt, please check the listed branch.
Quoted 7 lines
Quoted 2 lines
tree: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git branch: vfs-6.17.fallocateHi Christian,
I noticed that this patch series doesn't appear to be merged into this branch. Just wondering if it might have been missed?
Dammit, my script missed to push the branch. Fixed now. Thanks for checking!
Zhang,
Quoted 5 lines
This series implements the
BLK_FEAT_WRITE_ZEROES_UNMAPfeature andBLK_FLAG_WRITE_ZEROES_UNMAP_DISABLEDflag for SCSI, NVMe and device-mapper drivers, and add theFALLOC_FL_WRITE_ZEROESandSTATX_ATTR_WRITE_ZEROES_UNMAPsupport for ext4 and raw bdev devices. Any comments are welcome.
This looks OK to me.
Signature · 1 line
Martin K. Petersen
Quoted 9 lines · Martin K. Petersen
Zhang,
Quoted 5 lines
This series implements the
BLK_FEAT_WRITE_ZEROES_UNMAPfeature andBLK_FLAG_WRITE_ZEROES_UNMAP_DISABLEDflag for SCSI, NVMe and device-mapper drivers, and add theFALLOC_FL_WRITE_ZEROESandSTATX_ATTR_WRITE_ZEROES_UNMAPsupport for ext4 and raw bdev devices. Any comments are welcome.
Quoted 4 lines
This looks OK to me.
Reviewed-by Martin K. Petersen <[email protected]>
Thank you, Martin and Christoph, for the patient review. I will update my test patches next.
Best regards,
Yi.