Skip to contentSkip to conversation

syscalls/statx01: loosen the stx_blocks check

PATCH New Steve Muckle 2 participants 7 messages Jul 22, 2019 – Jul 25, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 syscalls/statx01: loosen the stx_blocks check

A filesystem may preallocate blocks (ext4 does this), so even though the contents of the file may fit into a single block, statx may report more than one block. Loosen the test to accommodate this.

---
 testcases/kernel/syscalls/statx/statx01.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/syscalls/statx/statx01.c +1 -1
index 23ea99b4a..5e705dc7d 100644
@@ -92,7 +92,7 @@ static void test_normal_file(void)
 			buff.stx_mode, MODE);
 
 
-	if (buff.stx_blocks <= buff.stx_blksize/512)
+	if (buff.stx_blocks <= 128)
 		tst_res(TPASS, "stx_blocks(%"PRIu64") is valid",
 			buff.stx_blocks);
 	else
Signature · 1 line
2.22.0.657.g960e92d24f-goog
#1 lore raw

PATCH 1/1 syscalls/statx01: loosen the stx_blocks check

Hi!

Quoted 14 lines

A filesystem may preallocate blocks (ext4 does this), so even though the contents of the file may fit into a single block, statx may report more than one block. Loosen the test to accommodate this.

---
 testcases/kernel/syscalls/statx/statx01.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/syscalls/statx/statx01.c +0 -0
index 23ea99b4a..5e705dc7d 100644

testcases/kernel/syscalls/statx/statx01.c static void test_normal_file(void) · line 95

Quoted 5 lines
testcases/kernel/syscalls/statx/statx01.c +1 -1
@@ -92,7 +92,7 @@ static void test_normal_file(void)
 			buff.stx_mode, MODE);
 
 
-	if (buff.stx_blocks <= buff.stx_blksize/512)
+	if (buff.stx_blocks <= 128)

Why 128?

Can't we rather multiply the buff.stx_blksize/512 by 16?

The filesystem block size is not constant the ppc64le has page size and block size 64k,@least for Btrfs, so 128 would be a single block there so if we preallocated blocks there this would stil fail.

Signature · 2 lines
Cyril Hrubis
[email protected]
#2 lore raw

PATCH 1/1 syscalls/statx01: loosen the stx_blocks check

Quoted 18 lines · Cyril Hrubis

Hi!

Quoted 17 lines

A filesystem may preallocate blocks (ext4 does this), so even though the contents of the file may fit into a single block, statx may report more than one block. Loosen the test to accommodate this.

---
  testcases/kernel/syscalls/statx/statx01.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/syscalls/statx/statx01.c +0 -0
index 23ea99b4a..5e705dc7d 100644
@@ -92,7 +92,7 @@ static void test_normal_file(void)
  			buff.stx_mode, MODE);
 
 
Quoted 4 lines

testcases/kernel/syscalls/statx/statx01.c static void test_normal_file(void) · line 95

Quoted 2 lines
testcases/kernel/syscalls/statx/statx01.c +1 -1
-	if (buff.stx_blocks <= buff.stx_blksize/512)
+	if (buff.stx_blocks <= 128)

Why 128?

Was just restoring it to what it was previously.

Quoted 1 line

Can't we rather multiply the buff.stx_blksize/512 by 16?

That would work for ext4, but should this be loosened further to give some headroom in case some FS preallocates more? Or is it preferable to keep the test as strict as possible and loosen it when/if failures happen?

#3 lore raw

PATCH 1/1 syscalls/statx01: loosen the stx_blocks check

Hi!

Quoted 9 lines
Quoted 1 line

Why 128?

Was just restoring it to what it was previously.

Quoted 1 line

Can't we rather multiply the buff.stx_blksize/512 by 16?

That would work for ext4, but should this be loosened further to give some headroom in case some FS preallocates more? Or is it preferable to keep the test as strict as possible and loosen it when/if failures happen?

That is hard to decide. I would go for the lowest limit that works for everybody.

Also for me ext4 allocates only one block for the file no matter what I do, something has to be tuned differently in your setup.

Isn't it just a difference in the block size and inode size?

Signature · 2 lines
Cyril Hrubis
[email protected]
#4 lore raw

PATCH 1/1 syscalls/statx01: loosen the stx_blocks check

Quoted 11 lines · Cyril Hrubis

Hi!

Quoted 9 lines
Quoted 1 line

Why 128?

Was just restoring it to what it was previously.

Quoted 1 line

Can't we rather multiply the buff.stx_blksize/512 by 16?

That would work for ext4, but should this be loosened further to give some headroom in case some FS preallocates more? Or is it preferable to keep the test as strict as possible and loosen it when/if failures happen?

Quoted 7 lines

That is hard to decide. I would go for the lowest limit that works for everybody.

Also for me ext4 allocates only one block for the file no matter what I do, something has to be tuned differently in your setup.

Isn't it just a difference in the block size and inode size?

On my platform stx_blocks = 16 (8kb) and stx_blksize = 4096, so two fs blocks are being allocated.

Interestingly, if I create a 256 byte file on this Android platform in /data, it 4kb:

vsoc_x86:/data # dd if=/dev/zero of=test_file bs=1 count=256 256+0 records in
256+0 records out
256 bytes (256 B) copied, 0.000926 s, 270 K/s vsoc_x86:/data # ls -ls test_file
4 -rw-rw-rw- 1 root root 256 2019-07-24 19:28 test_file

However if I go into the directory specified in TMPDIR, it takes 8kb:

vsoc_x86:/data/local/tmp # dd if=/dev/zero of=test_file bs=1 count=256 256+0 records in
256+0 records out
256 bytes (256 B) copied, 0.000927 s, 270 K/s vsoc_x86:/data/local/tmp # ls -ls
total 8
8 -rw-rw-rw- 1 root root 256 2019-07-24 19:30 test_file

It is also 8kb if I create it in /data/local, but not if I create a directory at /data/testdir and create the file there.

Perhaps the x-attributes are spilling out of the inode in some cases and into a data block? I don't know enough about filesystems to say.

#5 lore raw

PATCH 1/1 syscalls/statx01: loosen the stx_blocks check

Hi!

Quoted 35 lines
Quoted 11 lines
Quoted 3 lines

That would work for ext4, but should this be loosened further to give some headroom in case some FS preallocates more? Or is it preferable to keep the test as strict as possible and loosen it when/if failures happen?

That is hard to decide. I would go for the lowest limit that works for everybody.

Also for me ext4 allocates only one block for the file no matter what I do, something has to be tuned differently in your setup.

Isn't it just a difference in the block size and inode size?

On my platform stx_blocks = 16 (8kb) and stx_blksize = 4096, so two fs blocks are being allocated.

Interestingly, if I create a 256 byte file on this Android platform in /data, it 4kb:

vsoc_x86:/data # dd if=/dev/zero of=test_file bs=1 count=256 256+0 records in
256+0 records out
256 bytes (256 B) copied, 0.000926 s, 270 K/s vsoc_x86:/data # ls -ls test_file
4 -rw-rw-rw- 1 root root 256 2019-07-24 19:28 test_file

However if I go into the directory specified in TMPDIR, it takes 8kb:

vsoc_x86:/data/local/tmp # dd if=/dev/zero of=test_file bs=1 count=256 256+0 records in
256+0 records out
256 bytes (256 B) copied, 0.000927 s, 270 K/s vsoc_x86:/data/local/tmp # ls -ls
total 8
8 -rw-rw-rw- 1 root root 256 2019-07-24 19:30 test_file

Quoted 2 lines

It is also 8kb if I create it in /data/local, but not if I create a directory at /data/testdir and create the file there.

That is a bit strange, unfortunately I do not have a single Android device around, otherwise I would try to reproduce it.

Quoted 2 lines

Perhaps the x-attributes are spilling out of the inode in some cases and into a data block? I don't know enough about filesystems to say.

That may be, you can try to list these with getfattr.

Signature · 2 lines
Cyril Hrubis
[email protected]
#6 lore raw

PATCH 1/1 syscalls/statx01: loosen the stx_blocks check

Quoted 4 lines · Cyril Hrubis
Quoted 2 lines

Perhaps the x-attributes are spilling out of the inode in some cases and into a data block? I don't know enough about filesystems to say.

That may be, you can try to list these with getfattr.

This turned out to be because of ext4 per-file encryption which adds another x-attr, possibly causing the allocation of another block for xattrs. This wasn't apparent initially because the encryption xattr is hidden and not reported by getfattr.

I'll send out an updated patch.