Skip to contentSkip to conversation

btrfs-progs: mkfs: only output the warning if the sectorsize is not supported

PATCH Superseded Qu Wenruo 4 participants 6 messages Apr 15, 2021 – Apr 17, 2021 Btrfs file system

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

PATCH 1/1 btrfs-progs: mkfs: only output the warning if the sectorsize is not supported

Currently mkfs.btrfs will output a warning message if the sectorsize is
not the same as page size:
WARNING: the filesystem may not be mountable, sectorsize 4096 doesn't match page size 65536

But since btrfs subpage support for 64K page size is comming, this output is populating the golden output of fstests, causing tons of false alerts.

This patch will make teach mkfs.btrfs to check /sys/fs/btrfs/features/supported_sectorsizes, and compare if the sector size is supported.

Then only output above warning message if the sector size is not supported.

---
 common/fsfeatures.c | 36 +++++++++++++++++++++++++++++++++++-
 1 file changed, 35 insertions(+), 1 deletion(-)
common/fsfeatures.c +35 -1
index 569208a9e5b1..13b775da9c72 100644
@@ -16,6 +16,8 @@
 
 #include "kerncompat.h"
 #include <sys/utsname.h>
+#include <sys/stat.h>
+#include <fcntl.h>
 #include <linux/version.h>
 #include <unistd.h>
 #include "common/fsfeatures.h"
@@ -327,8 +329,15 @@ u32 get_running_kernel_version(void)
 
 	return version;
 }
+
+/*
+ * The buffer size if strlen("4096 8192 16384 32768 65536"),
+ * which is 28, then round up to 32.
+ */
+#define SUPPORTED_SECTORSIZE_BUF_SIZE	32
 int btrfs_check_sectorsize(u32 sectorsize)
 {
+	bool sectorsize_checked = false;
 	u32 page_size = (u32)sysconf(_SC_PAGESIZE);
 
 	if (!is_power_of_2(sectorsize)) {
@@ -340,7 +349,32 @@ int btrfs_check_sectorsize(u32 sectorsize)
 		      sectorsize);
 		return -EINVAL;
 	}
-	if (page_size != sectorsize)
+	if (page_size == sectorsize) {
+		sectorsize_checked = true;
+	} else {
+		/*
+		 * Check if the sector size is supported
+		 */
+		char supported_buf[SUPPORTED_SECTORSIZE_BUF_SIZE] = { 0 };
+		char sectorsize_buf[SUPPORTED_SECTORSIZE_BUF_SIZE] = { 0 };
+		int fd;
+		int ret;
+
+		fd = open("/sys/fs/btrfs/features/supported_sectorsizes",
+			  O_RDONLY);
+		if (fd < 0)
+			goto out;
+		ret = read(fd, supported_buf, sizeof(supported_buf));
+		close(fd);
+		if (ret < 0)
+			goto out;
+		snprintf(sectorsize_buf, SUPPORTED_SECTORSIZE_BUF_SIZE,
+			 "%u", page_size);
+		if (strstr(supported_buf, sectorsize_buf))
+			sectorsize_checked = true;
+	}
+out:
+	if (!sectorsize_checked)
 		warning(
 "the filesystem may not be mountable, sectorsize %u doesn't match page size %u",
 			sectorsize, page_size);
Signature · 1 line
2.31.1
#1 lore raw
Quoted 61 lines · Qu Wenruo

Currently mkfs.btrfs will output a warning message if the sectorsize is
not the same as page size:
WARNING: the filesystem may not be mountable, sectorsize 4096 doesn't match page size 65536

But since btrfs subpage support for 64K page size is comming, this output is populating the golden output of fstests, causing tons of false alerts.

This patch will make teach mkfs.btrfs to check /sys/fs/btrfs/features/supported_sectorsizes, and compare if the sector size is supported.

Then only output above warning message if the sector size is not supported.

---
 common/fsfeatures.c | 36 +++++++++++++++++++++++++++++++++++-
 1 file changed, 35 insertions(+), 1 deletion(-)
common/fsfeatures.c +16 -1
index 569208a9e5b1..13b775da9c72 100644
@@ -16,6 +16,8 @@
 
 #include "kerncompat.h"
 #include <sys/utsname.h>
+#include <sys/stat.h>
+#include <fcntl.h>
 #include <linux/version.h>
 #include <unistd.h>
 #include "common/fsfeatures.h"
@@ -327,8 +329,15 @@ u32 get_running_kernel_version(void)
 
 	return version;
 }
+
+/*
+ * The buffer size if strlen("4096 8192 16384 32768 65536"),
+ * which is 28, then round up to 32.
+ */
+#define SUPPORTED_SECTORSIZE_BUF_SIZE	32
 int btrfs_check_sectorsize(u32 sectorsize)
 {
+	bool sectorsize_checked = false;
 	u32 page_size = (u32)sysconf(_SC_PAGESIZE);
 
 	if (!is_power_of_2(sectorsize)) {
@@ -340,7 +349,32 @@ int btrfs_check_sectorsize(u32 sectorsize)
 		      sectorsize);
 		return -EINVAL;
 	}
-	if (page_size != sectorsize)
+	if (page_size == sectorsize) {
+		sectorsize_checked = true;
+	} else {
+		/*
+		 * Check if the sector size is supported
+		 */
+		char supported_buf[SUPPORTED_SECTORSIZE_BUF_SIZE] = { 0 };

common/fsfeatures.c int btrfs_check_sectorsize(u32 sectorsize) · line 368

Quoted 10 lines
common/fsfeatures.c +10 -0
+		char sectorsize_buf[SUPPORTED_SECTORSIZE_BUF_SIZE] = { 0 };
+		int fd;
+		int ret;
+
+		fd = open("/sys/fs/btrfs/features/supported_sectorsizes",
+			  O_RDONLY);
+		if (fd < 0)
+			goto out;
+		ret = read(fd, supported_buf, sizeof(supported_buf));
+		close(fd);

There are some sysfs helpers, like sysfs_read_file and sysfs_open_fsid_file that would avoid the boilerplate code. We don't have a helper for the toplevel sysfs directory so that would need to be added first.

Now that I look at it, the sysfs_read_file could actually do that in one go, including open and close. I'll probably do that as a cleanup later and apply your patch as it's a fix. Thanks.

#2 lore raw
In reply to Qu Wenruo
Quoted 34 lines · Qu Wenruo

Currently mkfs.btrfs will output a warning message if the sectorsize is
not the same as page size:
WARNING: the filesystem may not be mountable, sectorsize 4096 doesn't match page size 65536

But since btrfs subpage support for 64K page size is comming, this output is populating the golden output of fstests, causing tons of false alerts.

This patch will make teach mkfs.btrfs to check /sys/fs/btrfs/features/supported_sectorsizes, and compare if the sector size is supported.

Then only output above warning message if the sector size is not supported.

---
 common/fsfeatures.c | 36 +++++++++++++++++++++++++++++++++++-
 1 file changed, 35 insertions(+), 1 deletion(-)
common/fsfeatures.c +2 -0
index 569208a9e5b1..13b775da9c72 100644
@@ -16,6 +16,8 @@
 
 #include "kerncompat.h"
 #include <sys/utsname.h>
+#include <sys/stat.h>
+#include <fcntl.h>
 #include <linux/version.h>
 #include <unistd.h>
 #include "common/fsfeatures.h"

common/fsfeatures.c u32 get_running_kernel_version(void) · line 335

Quoted 7 lines
common/fsfeatures.c +4 -0
@@ -327,8 +329,15 @@ u32 get_running_kernel_version(void)
 
 	return version;
 }
+
+/*
+ * The buffer size if strlen("4096 8192 16384 32768 65536"),
+ * which is 28, then round up to 32.

I think there is a typo in this comment, because it doesn't quite parse.

Quoted 26 lines
common/fsfeatures.c +16 -1
+ */
+#define SUPPORTED_SECTORSIZE_BUF_SIZE	32
 int btrfs_check_sectorsize(u32 sectorsize)
 {
+	bool sectorsize_checked = false;
 	u32 page_size = (u32)sysconf(_SC_PAGESIZE);
 
 	if (!is_power_of_2(sectorsize)) {
@@ -340,7 +349,32 @@ int btrfs_check_sectorsize(u32 sectorsize)
 		      sectorsize);
 		return -EINVAL;
 	}
-	if (page_size != sectorsize)
+	if (page_size == sectorsize) {
+		sectorsize_checked = true;
+	} else {
+		/*
+		 * Check if the sector size is supported
+		 */
+		char supported_buf[SUPPORTED_SECTORSIZE_BUF_SIZE] = { 0 };
+		char sectorsize_buf[SUPPORTED_SECTORSIZE_BUF_SIZE] = { 0 };
+		int fd;
+		int ret;
+
+		fd = open("/sys/fs/btrfs/features/supported_sectorsizes",
+			  O_RDONLY);

common/fsfeatures.c int btrfs_check_sectorsize(u32 sectorsize) · line 374

Quoted 10 lines
common/fsfeatures.c +10 -0
+		if (fd < 0)
+			goto out;
+		ret = read(fd, supported_buf, sizeof(supported_buf));
+		close(fd);
+		if (ret < 0)
+			goto out;
+		snprintf(sectorsize_buf, SUPPORTED_SECTORSIZE_BUF_SIZE,
+			 "%u", page_size);
+		if (strstr(supported_buf, sectorsize_buf))
+			sectorsize_checked = true;

Two comments here.
1: I think we should be checking sectorsize against the file rather than page_size.
2: strstr seems too permissive, since it doesn't have a notion of tokens. If not for the power_of_2 check above, we would admit all kinds of silly things like 409. But even with it, we would permit "4" now and with your example from the comment, "8", "16", and "32".

common/fsfeatures.c int btrfs_check_sectorsize(u32 sectorsize) · line 380

Quoted 6 lines
common/fsfeatures.c +3 -0
+	}
+out:
+	if (!sectorsize_checked)
 		warning(
 "the filesystem may not be mountable, sectorsize %u doesn't match page size %u",
 			sectorsize, page_size);

Do you have plans to change the contents of this string to match the new meaning of the check, or is that too harmful to testing/automation?

Quoted 3 lines
Signature · 1 line
2.31.1
#3 lore raw
Quoted 6 lines · Boris Burkov
Quoted 3 lines · Qu Wenruo

+/*
+ * The buffer size if strlen("4096 8192 16384 32768 65536"),
+ * which is 28, then round up to 32.

I think there is a typo in this comment, because it doesn't quite parse.

Typo fixed.

Quoted 32 lines
Quoted 32 lines

+ */

common/fsfeatures.c +21 -1
+#define SUPPORTED_SECTORSIZE_BUF_SIZE	32
 int btrfs_check_sectorsize(u32 sectorsize)
 {
+	bool sectorsize_checked = false;
 	u32 page_size = (u32)sysconf(_SC_PAGESIZE);
 
 	if (!is_power_of_2(sectorsize)) {
@@ -340,7 +349,32 @@ int btrfs_check_sectorsize(u32 sectorsize)
 		      sectorsize);
 		return -EINVAL;
 	}
-	if (page_size != sectorsize)
+	if (page_size == sectorsize) {
+		sectorsize_checked = true;
+	} else {
+		/*
+		 * Check if the sector size is supported
+		 */
+		char supported_buf[SUPPORTED_SECTORSIZE_BUF_SIZE] = { 0 };
+		char sectorsize_buf[SUPPORTED_SECTORSIZE_BUF_SIZE] = { 0 };
+		int fd;
+		int ret;
+
+		fd = open("/sys/fs/btrfs/features/supported_sectorsizes",
+			  O_RDONLY);
+		if (fd < 0)
+			goto out;
+		ret = read(fd, supported_buf, sizeof(supported_buf));
+		close(fd);
+		if (ret < 0)
+			goto out;
Quoted 8 lines

common/fsfeatures.c int btrfs_check_sectorsize(u32 sectorsize) · line 374

Quoted 4 lines
common/fsfeatures.c +4 -0
+		snprintf(sectorsize_buf, SUPPORTED_SECTORSIZE_BUF_SIZE,
+			 "%u", page_size);
+		if (strstr(supported_buf, sectorsize_buf))
+			sectorsize_checked = true;

Two comments here.
1: I think we should be checking sectorsize against the file rather than page_size.

What do you mean by 'against the file'?

I read it as comparing what system reports as page size, converted to string and looked up in the sysfs file.

Quoted 4 lines

2: strstr seems too permissive, since it doesn't have a notion of tokens. If not for the power_of_2 check above, we would admit all kinds of silly things like 409. But even with it, we would permit "4" now and with your example from the comment, "8", "16", and "32".

In general you're right. Practically speaking, this will work. We know what the kernel module puts to that file and if getconf returns some absurd number for PAGE_SIZE other things will break. The code assumes perfect match, in any other case it prints the warning. So even if there are some funny values either in getconf or the sysfs file, it leads to something noticealbe to the user. A silent error would be worse and worth fixing, but that way around it works.

#4 lore raw
Quoted 51 lines · David Sterba
Quoted 6 lines · Boris Burkov
Quoted 3 lines · Qu Wenruo

+/*
+ * The buffer size if strlen("4096 8192 16384 32768 65536"),
+ * which is 28, then round up to 32.

I think there is a typo in this comment, because it doesn't quite parse.

Typo fixed.

Quoted 32 lines
Quoted 32 lines

+ */

common/fsfeatures.c +21 -1
+#define SUPPORTED_SECTORSIZE_BUF_SIZE	32
 int btrfs_check_sectorsize(u32 sectorsize)
 {
+	bool sectorsize_checked = false;
 	u32 page_size = (u32)sysconf(_SC_PAGESIZE);
 
 	if (!is_power_of_2(sectorsize)) {
@@ -340,7 +349,32 @@ int btrfs_check_sectorsize(u32 sectorsize)
 		      sectorsize);
 		return -EINVAL;
 	}
-	if (page_size != sectorsize)
+	if (page_size == sectorsize) {
+		sectorsize_checked = true;
+	} else {
+		/*
+		 * Check if the sector size is supported
+		 */
+		char supported_buf[SUPPORTED_SECTORSIZE_BUF_SIZE] = { 0 };
+		char sectorsize_buf[SUPPORTED_SECTORSIZE_BUF_SIZE] = { 0 };
+		int fd;
+		int ret;
+
+		fd = open("/sys/fs/btrfs/features/supported_sectorsizes",
+			  O_RDONLY);
+		if (fd < 0)
+			goto out;
+		ret = read(fd, supported_buf, sizeof(supported_buf));
+		close(fd);
+		if (ret < 0)
+			goto out;
Quoted 8 lines

common/fsfeatures.c int btrfs_check_sectorsize(u32 sectorsize) · line 374

Quoted 4 lines
common/fsfeatures.c +4 -0
+		snprintf(sectorsize_buf, SUPPORTED_SECTORSIZE_BUF_SIZE,
+			 "%u", page_size);
+		if (strstr(supported_buf, sectorsize_buf))
+			sectorsize_checked = true;

Two comments here.
1: I think we should be checking sectorsize against the file rather than page_size.

Quoted 2 lines

What do you mean by 'against the file'?

I am saying we should do
snprintf(sectorsize_buf, SUPPORTED_SECTORSIZE_BUF_SIZE, "%u", sectorsize); so that we lookup sectorsize, not page_size, in supported_sectorsizes. Sorry for the confusing wording.

Quoted 2 lines

I read it as comparing what system reports as page size, converted to string and looked up in the sysfs file.

Apologies if I am misunderstanding the purpose of this patch, but the way I understand it is:

Today, on a system with 64K pages, we must use a 64K sectorsize. As part of supporting a 4K sectorsize, we want to relax this check to allow sectorsize=4K.

If the user runs mkfs.btrfs -s 4096, how would checking whether 64K is present in "supported_sectorsizes" help validate the input? As long page_size is in "supported_sectorsizes", any power of 2 between 4k and 64k is legit. I suppose that could be the logic, but it seems kind of bizarre to me. If that is really the intent, I would argue the filename "supported_sectorsizes" doesn't make sense.

Quoted 6 lines
Quoted 5 lines

2: strstr seems too permissive, since it doesn't have a notion of tokens. If not for the power_of_2 check above, we would admit all kinds of silly things like 409. But even with it, we would permit "4" now and with your example from the comment, "8", "16", and "32".

Quoted 7 lines

In general you're right. Practically speaking, this will work. We know what the kernel module puts to that file and if getconf returns some absurd number for PAGE_SIZE other things will break. The code assumes perfect match, in any other case it prints the warning. So even if there are some funny values either in getconf or the sysfs file, it leads to something noticealbe to the user. A silent error would be worse and worth fixing, but that way around it works.

Looking more closely, I think that the [4k,64k] check mostly covers my concern anyway. I also agree that we should be pragmatic and not over-complicate the check. However, I do think if my point above holds, then the strstr input could be different than just the one fixed page size.

#5 lore raw
In reply to Boris Burkov
Quoted 36 lines · Boris Burkov
Quoted 35 lines · Qu Wenruo

Currently mkfs.btrfs will output a warning message if the sectorsize is
not the same as page size:
WARNING: the filesystem may not be mountable, sectorsize 4096 doesn't match page size 65536

But since btrfs subpage support for 64K page size is comming, this output is populating the golden output of fstests, causing tons of false alerts.

This patch will make teach mkfs.btrfs to check /sys/fs/btrfs/features/supported_sectorsizes, and compare if the sector size is supported.

Then only output above warning message if the sector size is not supported.

---
  common/fsfeatures.c | 36 +++++++++++++++++++++++++++++++++++-
  1 file changed, 35 insertions(+), 1 deletion(-)
common/fsfeatures.c +2 -0
index 569208a9e5b1..13b775da9c72 100644
@@ -16,6 +16,8 @@
 
  #include "kerncompat.h"
  #include <sys/utsname.h>
+#include <sys/stat.h>
+#include <fcntl.h>
  #include <linux/version.h>
  #include <unistd.h>
  #include "common/fsfeatures.h"
@@ -327,8 +329,15 @@ u32 get_running_kernel_version(void)
 
Quoted 8 lines

common/fsfeatures.c u32 get_running_kernel_version(void) · line 335

Quoted 6 lines
common/fsfeatures.c +4 -0
  	return version;
  }
+
+/*
+ * The buffer size if strlen("4096 8192 16384 32768 65536"),
+ * which is 28, then round up to 32.

I think there is a typo in this comment, because it doesn't quite parse.

I mean the strlen() == 28, then round the value to 32.

Any better alternative?

Quoted 33 lines
Quoted 33 lines
common/fsfeatures.c +22 -1
+ */
+#define SUPPORTED_SECTORSIZE_BUF_SIZE	32
  int btrfs_check_sectorsize(u32 sectorsize)
  {
+	bool sectorsize_checked = false;
  	u32 page_size = (u32)sysconf(_SC_PAGESIZE);
 
  	if (!is_power_of_2(sectorsize)) {
@@ -340,7 +349,32 @@ int btrfs_check_sectorsize(u32 sectorsize)
  		      sectorsize);
  		return -EINVAL;
  	}
-	if (page_size != sectorsize)
+	if (page_size == sectorsize) {
+		sectorsize_checked = true;
+	} else {
+		/*
+		 * Check if the sector size is supported
+		 */
+		char supported_buf[SUPPORTED_SECTORSIZE_BUF_SIZE] = { 0 };
+		char sectorsize_buf[SUPPORTED_SECTORSIZE_BUF_SIZE] = { 0 };
+		int fd;
+		int ret;
+
+		fd = open("/sys/fs/btrfs/features/supported_sectorsizes",
+			  O_RDONLY);
+		if (fd < 0)
+			goto out;
+		ret = read(fd, supported_buf, sizeof(supported_buf));
+		close(fd);
+		if (ret < 0)
+			goto out;
Quoted 8 lines

common/fsfeatures.c int btrfs_check_sectorsize(u32 sectorsize) · line 374

Quoted 4 lines
common/fsfeatures.c +4 -0
+		snprintf(sectorsize_buf, SUPPORTED_SECTORSIZE_BUF_SIZE,
+			 "%u", page_size);
+		if (strstr(supported_buf, sectorsize_buf))
+			sectorsize_checked = true;

Two comments here.
1: I think we should be checking sectorsize against the file rather than page_size.

Damn it, all my bad.

Quoted 4 lines

2: strstr seems too permissive, since it doesn't have a notion of tokens. If not for the power_of_2 check above, we would admit all kinds of silly things like 409. But even with it, we would permit "4" now and with your example from the comment, "8", "16", and "32".

Indeed I took a shortcut here.

It's indeed not elegant, I'll change it to use " " as token to analyse each value and compare to sector size.

Thanks,
Qu

Quoted 14 lines

common/fsfeatures.c int btrfs_check_sectorsize(u32 sectorsize) · line 380

Quoted 7 lines
common/fsfeatures.c +3 -0
+	}
+out:
+	if (!sectorsize_checked)
  		warning(
  "the filesystem may not be mountable, sectorsize %u doesn't match page size %u",
  			sectorsize, page_size);

Do you have plans to change the contents of this string to match the new meaning of the check, or is that too harmful to testing/automation?

Quoted 3 lines

--
2.31.1