Skip to contentSkip to conversation

nvdimm: Export supported alignments via sysfs

PATCH In review Oliver O'Halloran 2 participants 11 messages Apr 27, 2017 – May 12, 2017 libnvdimm: non-volatile memory device

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvdimm: Export supported alignments via sysfs

Adds two new sysfs attributes for pfn (and dax) devices:
supported_alignements and default_alignment. These advertise to userspace what alignments this kernel supports, and provides a nominal default alignment to use.

---
I'm not sure it makes sense to provide these for pfn devices. In the dax case we have hard restrictions because of how fault handling works, but I'm not convinced this makes sense for the pfn case since it's going to be used with fs-dax.

---
 drivers/nvdimm/pfn_devs.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
drivers/nvdimm/pfn_devs.c +26 -0
index 6c033c9a2f06..5157e7d89f0b 100644
@@ -260,6 +260,30 @@ static ssize_t size_show(struct device *dev,
 }
 static DEVICE_ATTR_RO(size);
 
+static ssize_t supported_alignments_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	/* Fun fact: These aren't always constants! */
+	unsigned long supported_alignments[] = {
+		PAGE_SIZE,
+		HPAGE_PMD_SIZE,
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+		HPAGE_PUD_SIZE,
+#endif
+		0,
+	};
+
+	return nd_sector_size_show(0, supported_alignments, buf);
+}
+DEVICE_ATTR_RO(supported_alignments);
+
+static ssize_t default_alignment_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	return sprintf(buf, "%ld\n", HPAGE_PMD_SIZE);
+}
+DEVICE_ATTR_RO(default_alignment);
+
 static struct attribute *nd_pfn_attributes[] = {
 	&dev_attr_mode.attr,
 	&dev_attr_namespace.attr,
@@ -267,6 +291,8 @@ static struct attribute *nd_pfn_attributes[] = {
 	&dev_attr_align.attr,
 	&dev_attr_resource.attr,
 	&dev_attr_size.attr,
+	&dev_attr_supported_alignments.attr,
+	&dev_attr_default_alignment.attr,
 	NULL,
 };
 
Signature · 6 lines
2.9.3

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
#1 lore raw
Quoted 11 lines · Oliver O'Halloran

Adds two new sysfs attributes for pfn (and dax) devices:
supported_alignements and default_alignment. These advertise to userspace what alignments this kernel supports, and provides a nominal default alignment to use.

---
I'm not sure it makes sense to provide these for pfn devices. In the dax case we have hard restrictions because of how fault handling works, but I'm not convinced this makes sense for the pfn case since it's going to be used with fs-dax.

We still want this for fs-dax so we can make sure that the namespace is aligned to allow for opportunistic large mappings. We have pmd support for fs-dax currently shipping, and looking to expand that to pud support.

Quoted 40 lines
---
 drivers/nvdimm/pfn_devs.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
drivers/nvdimm/pfn_devs.c +24 -0
index 6c033c9a2f06..5157e7d89f0b 100644
@@ -260,6 +260,30 @@ static ssize_t size_show(struct device *dev,
 }
 static DEVICE_ATTR_RO(size);
 
+static ssize_t supported_alignments_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       /* Fun fact: These aren't always constants! */
+       unsigned long supported_alignments[] = {
+               PAGE_SIZE,
+               HPAGE_PMD_SIZE,
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+               HPAGE_PUD_SIZE,
+#endif
+               0,
+       };
+
+       return nd_sector_size_show(0, supported_alignments, buf);
+}
+DEVICE_ATTR_RO(supported_alignments);
+
+static ssize_t default_alignment_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       return sprintf(buf, "%ld\n", HPAGE_PMD_SIZE);
+}
+DEVICE_ATTR_RO(default_alignment);
+
 static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_mode.attr,
        &dev_attr_namespace.attr,

drivers/nvdimm/pfn_devs.c static struct attribute *nd_pfn_attributes[] = { · line 296

Quoted 6 lines
drivers/nvdimm/pfn_devs.c +2 -0
@@ -267,6 +291,8 @@ static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_align.attr,
        &dev_attr_resource.attr,
        &dev_attr_size.attr,
+       &dev_attr_supported_alignments.attr,
+       &dev_attr_default_alignment.attr,
        NULL,

So, we don't need DEVICE_ATTR_RO(default_alignment), that can be reflected by setting nd_pfn->align to HPAGE_PMD_SIZE by default and passing nd_pfn->align to nd_sector_size_show(). Should probably rename nd_sector_size_show() to nd_size_select_show().

The other concern is that the current DEVICE_ATTR_RW(align) can be made redundant by this new interface if you make it writable. I wonder if we can avoid breaking old ndctl versions by making the current align setting the first one in the output? Worse comes to worse we can live with two attributes 'align' and 'aligns', but I'd like to see if can add this to the existing attribute.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#2 lore raw
Quoted 70 lines · Dan Williams
Quoted 11 lines · Oliver O'Halloran

Adds two new sysfs attributes for pfn (and dax) devices:
supported_alignements and default_alignment. These advertise to userspace what alignments this kernel supports, and provides a nominal default alignment to use.

---
I'm not sure it makes sense to provide these for pfn devices. In the dax case we have hard restrictions because of how fault handling works, but I'm not convinced this makes sense for the pfn case since it's going to be used with fs-dax.

We still want this for fs-dax so we can make sure that the namespace is aligned to allow for opportunistic large mappings. We have pmd support for fs-dax currently shipping, and looking to expand that to pud support.

Quoted 40 lines
---
 drivers/nvdimm/pfn_devs.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
drivers/nvdimm/pfn_devs.c +24 -0
index 6c033c9a2f06..5157e7d89f0b 100644
@@ -260,6 +260,30 @@ static ssize_t size_show(struct device *dev,
 }
 static DEVICE_ATTR_RO(size);
 
+static ssize_t supported_alignments_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       /* Fun fact: These aren't always constants! */
+       unsigned long supported_alignments[] = {
+               PAGE_SIZE,
+               HPAGE_PMD_SIZE,
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+               HPAGE_PUD_SIZE,
+#endif
+               0,
+       };
+
+       return nd_sector_size_show(0, supported_alignments, buf);
+}
+DEVICE_ATTR_RO(supported_alignments);
+
+static ssize_t default_alignment_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       return sprintf(buf, "%ld\n", HPAGE_PMD_SIZE);
+}
+DEVICE_ATTR_RO(default_alignment);
+
 static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_mode.attr,
        &dev_attr_namespace.attr,

drivers/nvdimm/pfn_devs.c static struct attribute *nd_pfn_attributes[] = { · line 296

Quoted 6 lines
drivers/nvdimm/pfn_devs.c +2 -0
@@ -267,6 +291,8 @@ static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_align.attr,
        &dev_attr_resource.attr,
        &dev_attr_size.attr,
+       &dev_attr_supported_alignments.attr,
+       &dev_attr_default_alignment.attr,
        NULL,

So, we don't need DEVICE_ATTR_RO(default_alignment), that can be reflected by setting nd_pfn->align to HPAGE_PMD_SIZE by default and passing nd_pfn->align to nd_sector_size_show(). Should probably rename nd_sector_size_show() to nd_size_select_show().

Quoted 6 lines

The other concern is that the current DEVICE_ATTR_RW(align) can be made redundant by this new interface if you make it writable. I wonder if we can avoid breaking old ndctl versions by making the current align setting the first one in the output? Worse comes to worse we can live with two attributes 'align' and 'aligns', but I'd like to see if can add this to the existing attribute.

Ok, so we can make this backward compatible, all that is needed is to list the current setting as the first entry in the list and make it un-decorated. For example a size list like this with 528 selected:

    "512 520 [528] 4096 4104 4160 4224"

...would become this:

    "528 512 520 [528] 4096 4104 4160 4224"

...slightly messy, but it allows us to avoid growing redundant attributes.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#3 lore raw
Quoted 82 lines · Dan Williams
Quoted 70 lines · Dan Williams
Quoted 11 lines · Oliver O'Halloran

Adds two new sysfs attributes for pfn (and dax) devices:
supported_alignements and default_alignment. These advertise to userspace what alignments this kernel supports, and provides a nominal default alignment to use.

---
I'm not sure it makes sense to provide these for pfn devices. In the dax case we have hard restrictions because of how fault handling works, but I'm not convinced this makes sense for the pfn case since it's going to be used with fs-dax.

We still want this for fs-dax so we can make sure that the namespace is aligned to allow for opportunistic large mappings. We have pmd support for fs-dax currently shipping, and looking to expand that to pud support.

Quoted 40 lines
---
 drivers/nvdimm/pfn_devs.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
drivers/nvdimm/pfn_devs.c +24 -0
index 6c033c9a2f06..5157e7d89f0b 100644
@@ -260,6 +260,30 @@ static ssize_t size_show(struct device *dev,
 }
 static DEVICE_ATTR_RO(size);
 
+static ssize_t supported_alignments_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       /* Fun fact: These aren't always constants! */
+       unsigned long supported_alignments[] = {
+               PAGE_SIZE,
+               HPAGE_PMD_SIZE,
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+               HPAGE_PUD_SIZE,
+#endif
+               0,
+       };
+
+       return nd_sector_size_show(0, supported_alignments, buf);
+}
+DEVICE_ATTR_RO(supported_alignments);
+
+static ssize_t default_alignment_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       return sprintf(buf, "%ld\n", HPAGE_PMD_SIZE);
+}
+DEVICE_ATTR_RO(default_alignment);
+
 static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_mode.attr,
        &dev_attr_namespace.attr,

drivers/nvdimm/pfn_devs.c static struct attribute *nd_pfn_attributes[] = { · line 296

Quoted 6 lines
drivers/nvdimm/pfn_devs.c +2 -0
@@ -267,6 +291,8 @@ static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_align.attr,
        &dev_attr_resource.attr,
        &dev_attr_size.attr,
+       &dev_attr_supported_alignments.attr,
+       &dev_attr_default_alignment.attr,
        NULL,

So, we don't need DEVICE_ATTR_RO(default_alignment), that can be reflected by setting nd_pfn->align to HPAGE_PMD_SIZE by default and passing nd_pfn->align to nd_sector_size_show(). Should probably rename nd_sector_size_show() to nd_size_select_show().

Quoted 6 lines

The other concern is that the current DEVICE_ATTR_RW(align) can be made redundant by this new interface if you make it writable. I wonder if we can avoid breaking old ndctl versions by making the current align setting the first one in the output? Worse comes to worse we can live with two attributes 'align' and 'aligns', but I'd like to see if can add this to the existing attribute.

Ok, so we can make this backward compatible, all that is needed is to list the current setting as the first entry in the list and make it un-decorated. For example a size list like this with 528 selected:

Quoted 7 lines
    "512 520 [528] 4096 4104 4160 4224"

...would become this:

    "528 512 520 [528] 4096 4104 4160 4224"

...slightly messy, but it allows us to avoid growing redundant attributes.

This is pretty gross, are you sure you want to do this?
_______________________________________________
Linux-nvdimm mailing list [email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#4 lore raw
In reply to Dan Williams
Quoted 13 lines · Dan Williams
Quoted 11 lines · Oliver O'Halloran

Adds two new sysfs attributes for pfn (and dax) devices:
supported_alignements and default_alignment. These advertise to userspace what alignments this kernel supports, and provides a nominal default alignment to use.

---
I'm not sure it makes sense to provide these for pfn devices. In the dax case we have hard restrictions because of how fault handling works, but I'm not convinced this makes sense for the pfn case since it's going to be used with fs-dax.

Quoted 4 lines

We still want this for fs-dax so we can make sure that the namespace is aligned to allow for opportunistic large mappings. We have pmd support for fs-dax currently shipping, and looking to expand that to pud support.

Sure, but whether we can use a PUD for userspace mappings mostly depends on the allocation decisions of the filesystem rather than the alignment of the namespace. The reservations for the PFN superblock, altmap and dax labels mean the namespace is always going to be unaligned so forcing a PUD alignment will result in a lot of wasted space for dubious benefits. I suppose there's no reason not to provide the functionality, but I don't see it buying us much.

Quoted 41 lines
Quoted 41 lines
---
 drivers/nvdimm/pfn_devs.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
drivers/nvdimm/pfn_devs.c +24 -0
index 6c033c9a2f06..5157e7d89f0b 100644
@@ -260,6 +260,30 @@ static ssize_t size_show(struct device *dev,
 }
 static DEVICE_ATTR_RO(size);
 
+static ssize_t supported_alignments_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       /* Fun fact: These aren't always constants! */
+       unsigned long supported_alignments[] = {
+               PAGE_SIZE,
+               HPAGE_PMD_SIZE,
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+               HPAGE_PUD_SIZE,
+#endif
+               0,
+       };
+
+       return nd_sector_size_show(0, supported_alignments, buf);
+}
+DEVICE_ATTR_RO(supported_alignments);
+
+static ssize_t default_alignment_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       return sprintf(buf, "%ld\n", HPAGE_PMD_SIZE);
+}
+DEVICE_ATTR_RO(default_alignment);
+
 static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_mode.attr,
        &dev_attr_namespace.attr,
@@ -267,6 +291,8 @@ static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_align.attr,
Quoted 8 lines

drivers/nvdimm/pfn_devs.c static struct attribute *nd_pfn_attributes[] = { · line 296

Quoted 5 lines
drivers/nvdimm/pfn_devs.c +2 -0
        &dev_attr_resource.attr,
        &dev_attr_size.attr,
+       &dev_attr_supported_alignments.attr,
+       &dev_attr_default_alignment.attr,
        NULL,

So, we don't need DEVICE_ATTR_RO(default_alignment), that can be reflected by setting nd_pfn->align to HPAGE_PMD_SIZE by default.

Hmm true, if we do this then we can use the alignment of the seed as the default rather than having a separate attribute.

Quoted 2 lines

passing nd_pfn->align to nd_sector_size_show(). Should probably rename nd_sector_size_show() to nd_size_select_show().

I agree. I figured another respin would be required so I kept the changes to a minimum.

Quoted 6 lines

The other concern is that the current DEVICE_ATTR_RW(align) can be made redundant by this new interface if you make it writable. I wonder if we can avoid breaking old ndctl versions by making the current align setting the first one in the output? Worse comes to worse we can live with two attributes 'align' and 'aligns', but I'd like to see if can add this to the existing attribute.

I'd rather have a small amount of redundancy and keep the the attribute consistent with the the btt sector size attribute. We could always remove align some time down the track since I imagine ndctl is the only thing that consumes that part of the interface and ndctl already handles align being missing.

Oliver
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#5 lore raw
Quoted 93 lines · Oliver O'Halloran
Quoted 13 lines · Dan Williams
Quoted 11 lines · Oliver O'Halloran

Adds two new sysfs attributes for pfn (and dax) devices:
supported_alignements and default_alignment. These advertise to userspace what alignments this kernel supports, and provides a nominal default alignment to use.

---
I'm not sure it makes sense to provide these for pfn devices. In the dax case we have hard restrictions because of how fault handling works, but I'm not convinced this makes sense for the pfn case since it's going to be used with fs-dax.

Quoted 4 lines

We still want this for fs-dax so we can make sure that the namespace is aligned to allow for opportunistic large mappings. We have pmd support for fs-dax currently shipping, and looking to expand that to pud support.

Sure, but whether we can use a PUD for userspace mappings mostly depends on the allocation decisions of the filesystem rather than the alignment of the namespace. The reservations for the PFN superblock, altmap and dax labels mean the namespace is always going to be unaligned so forcing a PUD alignment will result in a lot of wasted space for dubious benefits. I suppose there's no reason not to provide the functionality, but I don't see it buying us much.

Quoted 41 lines
Quoted 41 lines
---
 drivers/nvdimm/pfn_devs.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
drivers/nvdimm/pfn_devs.c +24 -0
index 6c033c9a2f06..5157e7d89f0b 100644
@@ -260,6 +260,30 @@ static ssize_t size_show(struct device *dev,
 }
 static DEVICE_ATTR_RO(size);
 
+static ssize_t supported_alignments_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       /* Fun fact: These aren't always constants! */
+       unsigned long supported_alignments[] = {
+               PAGE_SIZE,
+               HPAGE_PMD_SIZE,
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+               HPAGE_PUD_SIZE,
+#endif
+               0,
+       };
+
+       return nd_sector_size_show(0, supported_alignments, buf);
+}
+DEVICE_ATTR_RO(supported_alignments);
+
+static ssize_t default_alignment_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       return sprintf(buf, "%ld\n", HPAGE_PMD_SIZE);
+}
+DEVICE_ATTR_RO(default_alignment);
+
 static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_mode.attr,
        &dev_attr_namespace.attr,
@@ -267,6 +291,8 @@ static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_align.attr,
Quoted 8 lines

drivers/nvdimm/pfn_devs.c static struct attribute *nd_pfn_attributes[] = { · line 296

Quoted 5 lines
drivers/nvdimm/pfn_devs.c +2 -0
        &dev_attr_resource.attr,
        &dev_attr_size.attr,
+       &dev_attr_supported_alignments.attr,
+       &dev_attr_default_alignment.attr,
        NULL,

So, we don't need DEVICE_ATTR_RO(default_alignment), that can be reflected by setting nd_pfn->align to HPAGE_PMD_SIZE by default.

Hmm true, if we do this then we can use the alignment of the seed as the default rather than having a separate attribute.

Quoted 2 lines

passing nd_pfn->align to nd_sector_size_show(). Should probably rename nd_sector_size_show() to nd_size_select_show().

I agree. I figured another respin would be required so I kept the changes to a minimum.

Quoted 6 lines

The other concern is that the current DEVICE_ATTR_RW(align) can be made redundant by this new interface if you make it writable. I wonder if we can avoid breaking old ndctl versions by making the current align setting the first one in the output? Worse comes to worse we can live with two attributes 'align' and 'aligns', but I'd like to see if can add this to the existing attribute.

Quoted 2 lines

I'd rather have a small amount of redundancy and keep the the attribute consistent with the the btt sector size attribute.

I'd rather not, that's expanding the kernel-user ABI for only vanity reasons as far as I can see.

Quoted 4 lines

We could
always remove align some time down the track since I imagine ndctl is the only thing that consumes that part of the interface and ndctl already handles align being missing.

No, that breaks old ndctl binaries that depend on the align attribute to be there if the kernel supports device-dax.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#6 lore raw
Quoted 94 lines · Dan Williams
Quoted 93 lines · Oliver O'Halloran
Quoted 13 lines · Dan Williams
Quoted 11 lines · Oliver O'Halloran

Adds two new sysfs attributes for pfn (and dax) devices:
supported_alignements and default_alignment. These advertise to userspace what alignments this kernel supports, and provides a nominal default alignment to use.

---
I'm not sure it makes sense to provide these for pfn devices. In the dax case we have hard restrictions because of how fault handling works, but I'm not convinced this makes sense for the pfn case since it's going to be used with fs-dax.

Quoted 4 lines

We still want this for fs-dax so we can make sure that the namespace is aligned to allow for opportunistic large mappings. We have pmd support for fs-dax currently shipping, and looking to expand that to pud support.

Sure, but whether we can use a PUD for userspace mappings mostly depends on the allocation decisions of the filesystem rather than the alignment of the namespace. The reservations for the PFN superblock, altmap and dax labels mean the namespace is always going to be unaligned so forcing a PUD alignment will result in a lot of wasted space for dubious benefits. I suppose there's no reason not to provide the functionality, but I don't see it buying us much.

Quoted 41 lines
Quoted 41 lines
---
 drivers/nvdimm/pfn_devs.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
drivers/nvdimm/pfn_devs.c +24 -0
index 6c033c9a2f06..5157e7d89f0b 100644
@@ -260,6 +260,30 @@ static ssize_t size_show(struct device *dev,
 }
 static DEVICE_ATTR_RO(size);
 
+static ssize_t supported_alignments_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       /* Fun fact: These aren't always constants! */
+       unsigned long supported_alignments[] = {
+               PAGE_SIZE,
+               HPAGE_PMD_SIZE,
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+               HPAGE_PUD_SIZE,
+#endif
+               0,
+       };
+
+       return nd_sector_size_show(0, supported_alignments, buf);
+}
+DEVICE_ATTR_RO(supported_alignments);
+
+static ssize_t default_alignment_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       return sprintf(buf, "%ld\n", HPAGE_PMD_SIZE);
+}
+DEVICE_ATTR_RO(default_alignment);
+
 static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_mode.attr,
        &dev_attr_namespace.attr,
@@ -267,6 +291,8 @@ static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_align.attr,
Quoted 8 lines

drivers/nvdimm/pfn_devs.c static struct attribute *nd_pfn_attributes[] = { · line 296

Quoted 5 lines
drivers/nvdimm/pfn_devs.c +2 -0
        &dev_attr_resource.attr,
        &dev_attr_size.attr,
+       &dev_attr_supported_alignments.attr,
+       &dev_attr_default_alignment.attr,
        NULL,

So, we don't need DEVICE_ATTR_RO(default_alignment), that can be reflected by setting nd_pfn->align to HPAGE_PMD_SIZE by default.

Hmm true, if we do this then we can use the alignment of the seed as the default rather than having a separate attribute.

Quoted 2 lines

passing nd_pfn->align to nd_sector_size_show(). Should probably rename nd_sector_size_show() to nd_size_select_show().

I agree. I figured another respin would be required so I kept the changes to a minimum.

Quoted 6 lines

The other concern is that the current DEVICE_ATTR_RW(align) can be made redundant by this new interface if you make it writable. I wonder if we can avoid breaking old ndctl versions by making the current align setting the first one in the output? Worse comes to worse we can live with two attributes 'align' and 'aligns', but I'd like to see if can add this to the existing attribute.

Quoted 5 lines
Quoted 2 lines

I'd rather have a small amount of redundancy and keep the the attribute consistent with the the btt sector size attribute.

I'd rather not, that's expanding the kernel-user ABI for only vanity reasons as far as I can see.

It's an extension of the user-kernel ABI in any case. This is just the most byzantine way to do it.

Quoted 7 lines
Quoted 4 lines

We could
always remove align some time down the track since I imagine ndctl is the only thing that consumes that part of the interface and ndctl already handles align being missing.

No, that breaks old ndctl binaries that depend on the align attribute to be there if the kernel supports device-dax.

Fair enough.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#7 lore raw
Quoted 109 lines · Oliver O'Halloran
Quoted 94 lines · Dan Williams
Quoted 93 lines · Oliver O'Halloran
Quoted 13 lines · Dan Williams
Quoted 11 lines · Oliver O'Halloran

Adds two new sysfs attributes for pfn (and dax) devices:
supported_alignements and default_alignment. These advertise to userspace what alignments this kernel supports, and provides a nominal default alignment to use.

---
I'm not sure it makes sense to provide these for pfn devices. In the dax case we have hard restrictions because of how fault handling works, but I'm not convinced this makes sense for the pfn case since it's going to be used with fs-dax.

Quoted 4 lines

We still want this for fs-dax so we can make sure that the namespace is aligned to allow for opportunistic large mappings. We have pmd support for fs-dax currently shipping, and looking to expand that to pud support.

Sure, but whether we can use a PUD for userspace mappings mostly depends on the allocation decisions of the filesystem rather than the alignment of the namespace. The reservations for the PFN superblock, altmap and dax labels mean the namespace is always going to be unaligned so forcing a PUD alignment will result in a lot of wasted space for dubious benefits. I suppose there's no reason not to provide the functionality, but I don't see it buying us much.

Quoted 41 lines
Quoted 41 lines
---
 drivers/nvdimm/pfn_devs.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
drivers/nvdimm/pfn_devs.c +24 -0
index 6c033c9a2f06..5157e7d89f0b 100644
@@ -260,6 +260,30 @@ static ssize_t size_show(struct device *dev,
 }
 static DEVICE_ATTR_RO(size);
 
+static ssize_t supported_alignments_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       /* Fun fact: These aren't always constants! */
+       unsigned long supported_alignments[] = {
+               PAGE_SIZE,
+               HPAGE_PMD_SIZE,
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+               HPAGE_PUD_SIZE,
+#endif
+               0,
+       };
+
+       return nd_sector_size_show(0, supported_alignments, buf);
+}
+DEVICE_ATTR_RO(supported_alignments);
+
+static ssize_t default_alignment_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       return sprintf(buf, "%ld\n", HPAGE_PMD_SIZE);
+}
+DEVICE_ATTR_RO(default_alignment);
+
 static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_mode.attr,
        &dev_attr_namespace.attr,
@@ -267,6 +291,8 @@ static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_align.attr,
Quoted 8 lines

drivers/nvdimm/pfn_devs.c static struct attribute *nd_pfn_attributes[] = { · line 296

Quoted 5 lines
drivers/nvdimm/pfn_devs.c +2 -0
        &dev_attr_resource.attr,
        &dev_attr_size.attr,
+       &dev_attr_supported_alignments.attr,
+       &dev_attr_default_alignment.attr,
        NULL,

So, we don't need DEVICE_ATTR_RO(default_alignment), that can be reflected by setting nd_pfn->align to HPAGE_PMD_SIZE by default.

Hmm true, if we do this then we can use the alignment of the seed as the default rather than having a separate attribute.

Quoted 2 lines

passing nd_pfn->align to nd_sector_size_show(). Should probably rename nd_sector_size_show() to nd_size_select_show().

I agree. I figured another respin would be required so I kept the changes to a minimum.

Quoted 6 lines

The other concern is that the current DEVICE_ATTR_RW(align) can be made redundant by this new interface if you make it writable. I wonder if we can avoid breaking old ndctl versions by making the current align setting the first one in the output? Worse comes to worse we can live with two attributes 'align' and 'aligns', but I'd like to see if can add this to the existing attribute.

Quoted 5 lines
Quoted 2 lines

I'd rather have a small amount of redundancy and keep the the attribute consistent with the the btt sector size attribute.

I'd rather not, that's expanding the kernel-user ABI for only vanity reasons as far as I can see.

It's an extension of the user-kernel ABI in any case. This is just the most byzantine way to do it.

Quoted 5 lines
Quoted 4 lines

We could
always remove align some time down the track since I imagine ndctl is the only thing that consumes that part of the interface and ndctl already handles align being missing.

Quoted 4 lines
Quoted 2 lines

No, that breaks old ndctl binaries that depend on the align attribute to be there if the kernel supports device-dax.

Fair enough.

All that said, there's nothing stopping us from making 'align' it's own mechanism. Where the first entry in the list is the current setting, in contrast to btt that decorates the current sector-size setting with square brackets.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#8 lore raw
Quoted 113 lines · Dan Williams
Quoted 112 lines · Oliver O'Halloran
Quoted 94 lines · Dan Williams
Quoted 93 lines · Oliver O'Halloran
Quoted 13 lines · Dan Williams
Quoted 11 lines · Oliver O'Halloran

Adds two new sysfs attributes for pfn (and dax) devices:
supported_alignements and default_alignment. These advertise to userspace what alignments this kernel supports, and provides a nominal default alignment to use.

---
I'm not sure it makes sense to provide these for pfn devices. In the dax case we have hard restrictions because of how fault handling works, but I'm not convinced this makes sense for the pfn case since it's going to be used with fs-dax.

Quoted 4 lines

We still want this for fs-dax so we can make sure that the namespace is aligned to allow for opportunistic large mappings. We have pmd support for fs-dax currently shipping, and looking to expand that to pud support.

Sure, but whether we can use a PUD for userspace mappings mostly depends on the allocation decisions of the filesystem rather than the alignment of the namespace. The reservations for the PFN superblock, altmap and dax labels mean the namespace is always going to be unaligned so forcing a PUD alignment will result in a lot of wasted space for dubious benefits. I suppose there's no reason not to provide the functionality, but I don't see it buying us much.

Quoted 41 lines
Quoted 41 lines
---
 drivers/nvdimm/pfn_devs.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
drivers/nvdimm/pfn_devs.c +24 -0
index 6c033c9a2f06..5157e7d89f0b 100644
@@ -260,6 +260,30 @@ static ssize_t size_show(struct device *dev,
 }
 static DEVICE_ATTR_RO(size);
 
+static ssize_t supported_alignments_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       /* Fun fact: These aren't always constants! */
+       unsigned long supported_alignments[] = {
+               PAGE_SIZE,
+               HPAGE_PMD_SIZE,
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+               HPAGE_PUD_SIZE,
+#endif
+               0,
+       };
+
+       return nd_sector_size_show(0, supported_alignments, buf);
+}
+DEVICE_ATTR_RO(supported_alignments);
+
+static ssize_t default_alignment_show(struct device *dev,
+               struct device_attribute *attr, char *buf)
+{
+       return sprintf(buf, "%ld\n", HPAGE_PMD_SIZE);
+}
+DEVICE_ATTR_RO(default_alignment);
+
 static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_mode.attr,
        &dev_attr_namespace.attr,
@@ -267,6 +291,8 @@ static struct attribute *nd_pfn_attributes[] = {
        &dev_attr_align.attr,
Quoted 8 lines

drivers/nvdimm/pfn_devs.c static struct attribute *nd_pfn_attributes[] = { · line 296

Quoted 5 lines
drivers/nvdimm/pfn_devs.c +2 -0
        &dev_attr_resource.attr,
        &dev_attr_size.attr,
+       &dev_attr_supported_alignments.attr,
+       &dev_attr_default_alignment.attr,
        NULL,

So, we don't need DEVICE_ATTR_RO(default_alignment), that can be reflected by setting nd_pfn->align to HPAGE_PMD_SIZE by default.

Hmm true, if we do this then we can use the alignment of the seed as the default rather than having a separate attribute.

Quoted 2 lines

passing nd_pfn->align to nd_sector_size_show(). Should probably rename nd_sector_size_show() to nd_size_select_show().

I agree. I figured another respin would be required so I kept the changes to a minimum.

Quoted 6 lines

The other concern is that the current DEVICE_ATTR_RW(align) can be made redundant by this new interface if you make it writable. I wonder if we can avoid breaking old ndctl versions by making the current align setting the first one in the output? Worse comes to worse we can live with two attributes 'align' and 'aligns', but I'd like to see if can add this to the existing attribute.

Quoted 5 lines
Quoted 2 lines

I'd rather have a small amount of redundancy and keep the the attribute consistent with the the btt sector size attribute.

I'd rather not, that's expanding the kernel-user ABI for only vanity reasons as far as I can see.

It's an extension of the user-kernel ABI in any case. This is just the most byzantine way to do it.

Quoted 7 lines
Quoted 4 lines

We could
always remove align some time down the track since I imagine ndctl is the only thing that consumes that part of the interface and ndctl already handles align being missing.

No, that breaks old ndctl binaries that depend on the align attribute to be there if the kernel supports device-dax.

Quoted 6 lines
Quoted 1 line

Fair enough.

All that said, there's nothing stopping us from making 'align' it's own mechanism. Where the first entry in the list is the current setting, in contrast to btt that decorates the current sector-size setting with square brackets.

I'd be okay with this provided we force the alignment to one of the supported values. Currently the only validation done by the kernel is:

        if (!is_power_of_2(val) || val < PAGE_SIZE || val > SZ_1G)
                return -EINVAL;

So you can set an unsupported value by poking at sysfs directly. This behaviour is useful for testing since you can use it to force an alignment failure in the DAX fault handler. I'm not overly concerned if it goes, but it's something to keep in mind. I still think it would be cleaner if we just added a separate attribute.

Oliver
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#9 lore raw
Quoted 1 line · Oliver O'Halloran

On Wed, May 3, 2017 at 2:17 PM, Dan Williams <[email protected]> wrote:

[..]

Quoted 7 lines
Quoted 6 lines
Quoted 1 line

Fair enough.

All that said, there's nothing stopping us from making 'align' it's own mechanism. Where the first entry in the list is the current setting, in contrast to btt that decorates the current sector-size setting with square brackets.

Quoted 5 lines

I'd be okay with this provided we force the alignment to one of the supported values. Currently the only validation done by the kernel is:

        if (!is_power_of_2(val) || val < PAGE_SIZE || val > SZ_1G)
                return -EINVAL;

Yes, we'd need to validate the input against the supported values. There are no known binaries in the wild that I know of that depend on this looser definition, so we should be ok to change it.

Quoted 3 lines

So you can set an unsupported value by poking at sysfs directly. This behaviour is useful for testing since you can use it to force an alignment failure in the DAX fault handler.

I'd rather move that test support to something like the nfit_test infrastructure.

Quoted 3 lines

I'm not overly concerned
if it goes, but it's something to keep in mind. I still think it would be cleaner if we just added a separate attribute.

I'm still having a hard time seeing how redundant sysfs attributes is "clean".
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#10 lore raw
Quoted 24 lines · Dan Williams
Quoted 1 line · Oliver O'Halloran

On Wed, May 3, 2017 at 2:17 PM, Dan Williams <[email protected]> wrote:

[..]

Quoted 7 lines
Quoted 6 lines
Quoted 1 line

Fair enough.

All that said, there's nothing stopping us from making 'align' it's own mechanism. Where the first entry in the list is the current setting, in contrast to btt that decorates the current sector-size setting with square brackets.

Quoted 5 lines

I'd be okay with this provided we force the alignment to one of the supported values. Currently the only validation done by the kernel is:

        if (!is_power_of_2(val) || val < PAGE_SIZE || val > SZ_1G)
                return -EINVAL;

Yes, we'd need to validate the input against the supported values. There are no known binaries in the wild that I know of that depend on this looser definition, so we should be ok to change it.

Quoted 3 lines

So you can set an unsupported value by poking at sysfs directly. This behaviour is useful for testing since you can use it to force an alignment failure in the DAX fault handler.

Quoted 8 lines

I'd rather move that test support to something like the nfit_test infrastructure.

Quoted 3 lines

I'm not overly concerned
if it goes, but it's something to keep in mind. I still think it would be cleaner if we just added a separate attribute.

I'm still having a hard time seeing how redundant sysfs attributes is "clean".

It turns out the NVML project is also parsing the 'align' attribute outside of ndctl. So, now I'm with you, I think it would better to move the 'possible alignments' to its own read-only attribute ('aligns'?) and leave 'align' as the interface to read/write the current setting.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm