Skip to contentSkip to conversation

xfs: add FALLOC_FL_WRITE_ZEROES to XFS code base

PATCH Superseded Lukas Herbolt 4 participants 5 messages Feb 25 – Feb 27 XFS filesystem

v10 of 12 Superseded Latest is v12 →

lore
Share & tools
lore
#0 lore raw

PATCH v10 1/1 xfs: add FALLOC_FL_WRITE_ZEROES to XFS code base

Add support for FALLOC_FL_WRITE_ZEROES if the underlying device enable the unmap write zeroes operation.

---
 fs/xfs/xfs_bmap_util.c |  5 +++--
 fs/xfs/xfs_bmap_util.h |  2 +-
 fs/xfs/xfs_file.c      | 43 +++++++++++++++++++++++++++++-------------
 3 files changed, 34 insertions(+), 16 deletions(-)
fs/xfs/xfs_bmap_util.c +3 -2
index 2208a720ec3f..0c1b1fa82f8b 100644
@@ -646,7 +646,8 @@ int
 xfs_alloc_file_space(
 	struct xfs_inode	*ip,
 	xfs_off_t		offset,
-	xfs_off_t		len)
+	xfs_off_t		len,
+	uint32_t		bmapi_flags)
 {
 	xfs_mount_t		*mp = ip->i_mount;
 	xfs_off_t		count;
@@ -748,7 +749,7 @@ xfs_alloc_file_space(
 		 * will eventually reach the requested range.
 		 */
 		error = xfs_bmapi_write(tp, ip, startoffset_fsb,
-				allocatesize_fsb, XFS_BMAPI_PREALLOC, 0, imapp,
+				allocatesize_fsb, bmapi_flags, 0, imapp,
 				&nimaps);
 		if (error) {
 			if (error != -ENOSR)
fs/xfs/xfs_bmap_util.h +1 -1
index c477b3361630..2895cc97a572 100644
@@ -56,7 +56,7 @@ int	xfs_bmap_last_extent(struct xfs_trans *tp, struct xfs_inode *ip,
 
 /* preallocation and hole punch interface */
 int	xfs_alloc_file_space(struct xfs_inode *ip, xfs_off_t offset,
-		xfs_off_t len);
+		xfs_off_t len, uint32_t bmapi_flags);
 int	xfs_free_file_space(struct xfs_inode *ip, xfs_off_t offset,
 		xfs_off_t len, struct xfs_zone_alloc_ctx *ac);
 int	xfs_collapse_file_space(struct xfs_inode *, xfs_off_t offset,
fs/xfs/xfs_file.c +30 -13

81 lines not on this page · raw message

Signature · 1 line
2.53.0
#1 lore raw

fs/xfs/xfs_file.c xfs_falloc_zero_range( · line 1315

Quoted 5 lines
fs/xfs/xfs_file.c +5 -0
+	} else {
+		if (xfs_falloc_force_zero(ip, ac)) {
+			error = xfs_zero_range(ip, offset, len, ac, NULL);
+			goto set_filesize;
+		} else {

No need for an else after a goto.

#2 lore raw
In reply to Lukas Herbolt
Quoted 3 lines · Lukas Herbolt

Add support for FALLOC_FL_WRITE_ZEROES if the underlying device enable the unmap write zeroes operation.

Hi Lukas,

I independently started implmenting this feature as well. I ran a test case on your patches and it resulted in a warning in iomap_zero_range. iomap_zero_range has a check for folios outside eof, and it is being called as a part of setsize, i.e, before we change the size of the file.

I think we need to do a PREALLOC and then do a XFS_BMAPI_ZERO with XFS_BMAPI_CONVERT. Or I don't know if we should change the warning in iomap_zero_range.

Doing unwritten extents first and then converting them to written with zeroes is what ext4 does as well. Maybe it is better this way because we can quickly allocate the blocks and return while holding the aglocks and then do the actually write. I guess someone more experienced with XFS can comment on that.

I can send what I have and I will CC you in the series.

This is the warning I get when I test your patch:

WARNING:

[  112.551102] WARNING: fs/iomap/buffered-io.c:1525 at iomap_zero_range+0x42d/0x7b0, CPU#2: write_zeroes/411
[  112.560073] RIP: 0010:iomap_zero_range+0x42d/0x7b0
[  112.593471]  xfs_zero_range+0x86/0xd0 [xfs]
<snip>
[  112.594100]  xfs_setattr_size+0x5c2/0xd90 [xfs]
<snip>
[  112.598895]  xfs_falloc_setsize+0x158/0x200 [xfs]

This is the test case:
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>

#ifndef FALLOC_FL_ZERO_RANGE
#define FALLOC_FL_ZERO_RANGE 0x10
#endif

#ifndef FALLOC_FL_WRITE_ZEROES
#define FALLOC_FL_WRITE_ZEROES 0x80
#endif

#define TEST_SIZE (10 * 1024 * 1024)

void test_fallocate(const char *filename, int mode, const char *mode_name) {

    int fd;

    printf("Testing %s on %s...\n", mode_name, filename);

    unlink(filename);

    fd = open(filename, O_RDWR | O_CREAT, 0666);
    if (fd < 0) {
        perror("open failed");
        return;
    }

    if (fallocate(fd, mode, 0, TEST_SIZE) == 0) {
        printf(" -> fallocate(%s) succeeded!\n", mode_name);
    } else {
        printf(" -> fallocate(%s) failed: %s\n", mode_name, strerror(errno));
    }

    close(fd);

    /* Dump extent info using xfs_io */
    char cmd[256];
    snprintf(cmd, sizeof(cmd), "xfs_io -c 'bmap -vp' %s", filename);
    printf("=== Extents for %s ===\n", filename);
    system(cmd);
    printf("\n");
}
int main() {
    printf("Starting fallocate tests...\n");
    printf("------------------------------------------------\n\n");

    test_fallocate("test_zero_range.bin", FALLOC_FL_ZERO_RANGE, "FALLOC_FL_ZERO_RANGE");
    test_fallocate("test_write_zeroes.bin", FALLOC_FL_WRITE_ZEROES, "FALLOC_FL_WRITE_ZEROES");

    printf("Test complete.\n");
    return 0;
}

--
Pankaj

#3 lore raw
Quoted 6 lines · Pankaj Raghav (Samsung)
Quoted 3 lines · Lukas Herbolt

Add support for FALLOC_FL_WRITE_ZEROES if the underlying device enable the unmap write zeroes operation.

Hi Lukas,

Quoted 9 lines

I independently started implmenting this feature as well. I ran a test case on your patches and it resulted in a warning in iomap_zero_range. iomap_zero_range has a check for folios outside eof, and it is being called as a part of setsize, i.e, before we change the size of the file.

I think we need to do a PREALLOC and then do a XFS_BMAPI_ZERO with XFS_BMAPI_CONVERT. Or I don't know if we should change the warning in iomap_zero_range.

The reason the warning is there is because iomap_zero_range() uses buffered writes but doesn't actually bump i_size for writes beyond eof. Therefore if it ends up zeroing folios that start beyond eof, writeback would potentially toss those folios if i_size wasn't updated somehow or another by the time it occurs..

I'd guess there are two likely scenarios that lead to this warning, but you'd have to confirm. One is that we're unnecessarily zeroing an unwritten range for some reason. That would probably be harmless, but unexpected. The other would be zeroing written blocks beyond eof, which is risky and probably something we want to avoid, but also suspicious in that I don't think we should ever have written extents beyond eof in XFS (but rather either delalloc or written).

Brian

Quoted 83 lines

Doing unwritten extents first and then converting them to written with zeroes is what ext4 does as well. Maybe it is better this way because we can quickly allocate the blocks and return while holding the aglocks and then do the actually write. I guess someone more experienced with XFS can comment on that.

I can send what I have and I will CC you in the series.

This is the warning I get when I test your patch:

WARNING:

[  112.551102] WARNING: fs/iomap/buffered-io.c:1525 at iomap_zero_range+0x42d/0x7b0, CPU#2: write_zeroes/411
[  112.560073] RIP: 0010:iomap_zero_range+0x42d/0x7b0
[  112.593471]  xfs_zero_range+0x86/0xd0 [xfs]
<snip>
[  112.594100]  xfs_setattr_size+0x5c2/0xd90 [xfs]
<snip>
[  112.598895]  xfs_falloc_setsize+0x158/0x200 [xfs]

This is the test case:
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>

#ifndef FALLOC_FL_ZERO_RANGE
#define FALLOC_FL_ZERO_RANGE 0x10
#endif

#ifndef FALLOC_FL_WRITE_ZEROES
#define FALLOC_FL_WRITE_ZEROES 0x80
#endif

#define TEST_SIZE (10 * 1024 * 1024)

void test_fallocate(const char *filename, int mode, const char *mode_name) {

    int fd;

    printf("Testing %s on %s...\n", mode_name, filename);

    unlink(filename);

    fd = open(filename, O_RDWR | O_CREAT, 0666);
    if (fd < 0) {
        perror("open failed");
        return;
    }

    if (fallocate(fd, mode, 0, TEST_SIZE) == 0) {
        printf(" -> fallocate(%s) succeeded!\n", mode_name);
    } else {
        printf(" -> fallocate(%s) failed: %s\n", mode_name, strerror(errno));
    }

    close(fd);

    /* Dump extent info using xfs_io */
    char cmd[256];
    snprintf(cmd, sizeof(cmd), "xfs_io -c 'bmap -vp' %s", filename);
    printf("=== Extents for %s ===\n", filename);
    system(cmd);
    printf("\n");
}
int main() {
    printf("Starting fallocate tests...\n");
    printf("------------------------------------------------\n\n");

    test_fallocate("test_zero_range.bin", FALLOC_FL_ZERO_RANGE, "FALLOC_FL_ZERO_RANGE");
    test_fallocate("test_write_zeroes.bin", FALLOC_FL_WRITE_ZEROES, "FALLOC_FL_WRITE_ZEROES");

    printf("Test complete.\n");
    return 0;
}

--
Pankaj

#4 lore raw
Quoted 18 lines
Quoted 11 lines

Hi Lukas,

I independently started implmenting this feature as well. I ran a test case on your patches and it resulted in a warning in iomap_zero_range. iomap_zero_range has a check for folios outside eof, and it is being called as a part of setsize, i.e, before we change the size of the file.

I think we need to do a PREALLOC and then do a XFS_BMAPI_ZERO with XFS_BMAPI_CONVERT. Or I don't know if we should change the warning in iomap_zero_range.

The reason the warning is there is because iomap_zero_range() uses buffered writes but doesn't actually bump i_size for writes beyond eof. Therefore if it ends up zeroing folios that start beyond eof, writeback would potentially toss those folios if i_size wasn't updated somehow or another by the time it occurs..

Quoted 8 lines

I'd guess there are two likely scenarios that lead to this warning, but you'd have to confirm. One is that we're unnecessarily zeroing an unwritten range for some reason. That would probably be harmless, but unexpected. The other would be zeroing written blocks beyond eof, which is risky and probably something we want to avoid, but also suspicious in that I don't think we should ever have written extents beyond eof in XFS (but rather either delalloc or written).

Thanks for the reply Brian. The issue is the latter.

As a part of the new FALLOC_FL_WRITE_ZEROES flag, we do want to physcally zero beyond eof before we update the size of the file as a part of fallocate. But we will zero them out by directly talking to the block device. As you say, we should do only unwritten extents beyond eof, we can do something similar to what ext4 does: write unwritten extents first, increase the size of the file, then zero out those extents with XFS_BMAPI_CONVERT with XFS_BMAPI_ZERO.

I will send the patches soon.
--
Pankaj