ndctl, create-namespace: Allow 64K and 16M alignments
PATCH 1/3 ndctl, create-namespace: Allow 64K and 16M alignments
These are needed on powerpc since 64K is the default page size and 16MB is the PMD size when using the hash MMU.
--- ndctl/builtin-xaction-namespace.c | 2 ++ util/size.h | 1 + 2 files changed, 3 insertions(+)
ndctl/builtin-xaction-namespace.c +2 -0
@@ -494,7 +494,9 @@ static int validate_namespace_options(struct ndctl_region *region,switch (p->align) {case SZ_4K : +case SZ_64K :case SZ_2M : +case SZ_16M :case SZ_1G :break ;default :
util/size.h +1 -0
@@ -3,6 +3,7 @@#define SZ_1K 0x00000400#define SZ_4K 0x00001000 +#define SZ_64K 0x00010000#define SZ_1M 0x00100000#define SZ_2M 0x00200000#define SZ_4M 0x00400000
Signature · 6 lines
2.9.3 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
PATCH 2/3 ndctl, create-namespace: read default alignment from sysfs
Read the default alignment from the hpage_pmd_size in sysfs. On PPC the PMD size depends on the MMU being used. When the traditional hash MMU is used (P9 and earlier) the PMD size is 16MB while the newer radix MMU uses a 2MB PMD size. The choice of MMU is done at runtime depending on what the hardware supports so we need to detect this at runtime rather than hardcoding it.
--- ndctl/Makefile.am | 3 ++- ndctl/builtin-xaction-namespace.c | 41 +++++++++++++++++++++++++++++---------- 2 files changed, 33 insertions(+), 11 deletions(-)
ndctl/Makefile.am +2 -1
@@ -10,7 +10,8 @@ ndctl_SOURCES = ndctl.c \ ../util/log.c \ builtin-list.c \ builtin-test.c \ - ../util/json.c + ../util/json.c \ + ../util/sysfs.c if ENABLE_SMART ndctl_SOURCES += util/json-smart.c
ndctl/builtin-xaction-namespace.c +31 -10
113 lines not on this page · raw message
Signature · 6 lines
2.9.3 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [PATCH 2/3] ndctl, create-namespace: read default alignment from sysfs
Quoted 58 lines · Oliver O'Halloran
Read the default alignment from the
hpage_pmd_sizein sysfs. On PPC the PMD size depends on the MMU being used. When the traditional hash MMU is used (P9 and earlier) the PMD size is 16MB while the newer radix MMU uses a 2MB PMD size. The choice of MMU is done at runtime depending on what the hardware supports so we need to detect this at runtime rather than hardcoding it.Signed-off-by Oliver O'Halloran <[email protected]>--- ndctl/Makefile.am | 3 ++- ndctl/builtin-xaction-namespace.c | 41 +++++++++++++++++++++++++++++---------- 2 files changed, 33 insertions(+), 11 deletions(-)ndctl/Makefile.am +2
-1@@ -10,7 +10,8 @@ ndctl_SOURCES = ndctl.c \ ../util/log.c \ builtin-list.c \ builtin-test.c \ - ../util/json.c + ../util/json.c \ + ../util/sysfs.c if ENABLE_SMART ndctl_SOURCES += util/json-smart.cndctl/builtin-xaction-namespace.c +8
-1@@ -22,6 +22,7 @@#include <sys/types.h> #include <util/size.h> #include <util/json.h> +#include <util/sysfs.h> #include <json-c/json.h> #include <util/filter.h> #include <ndctl/libndctl.h> @@ -54,6 +55,8 @@ static struct parameters {const char *align; } param; +char default_align_buf[SYSFS_ATTR_SIZE ]; +void builtin_xaction_namespace_reset (void ) {/* @@ -137,7 +140,24 @@ enum namespace_action { ACTION_DESTROY, }; -static int set_defaults (enum namespace_action mode) +const char *sysfs_read_default_align (struct ndctl_ctx *ctx,const char *def, +const char *path) +{ +/* +* HACK: The command handlers aren't supposed to write into
ndctl/builtin-xaction-namespace.c enum namespace_action { · line 157
Quoted 10 lines
ndctl/builtin-xaction-namespace.c +10
-0+* the ndctl command context, but we want the debug +* output to go somewhere sensible. +*/ +if (__sysfs_read_attr ((struct log_ctx *)ctx, path, default_align_buf)) +return strdup (def); + +if (!strlen (default_align_buf)) +return def; + +return default_align_buf;
I chatted with Dave Hansen about this and we're thinking we should go ahead and add a new attribute to the device-dax sysfs with the list of supported alignments, similar to what we have in the btt case for supported sector sizes.
The reason is that the sensitivity to page sizes is a device-dax internal requirement. Theoretically device-dax could support any alignment and handle it with a mix of page sizes. However, since device-dax wants to be strict and predictable about the tlb size backing a given device-dax mapping then it should list the possible options.
Looking at the transparent_hugepage sysfs is a bit of a layering violation. There is no strict guarantee that device-dax is tied to thp in the longterm. The thp sysfs is also awkward because it does not tell us the pud page size.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [PATCH 2/3] ndctl, create-namespace: read default alignment from sysfs
Quoted 75 lines · Dan Williams
Quoted 58 lines · Oliver O'Halloran
Read the default alignment from the
hpage_pmd_sizein sysfs. On PPC the PMD size depends on the MMU being used. When the traditional hash MMU is used (P9 and earlier) the PMD size is 16MB while the newer radix MMU uses a 2MB PMD size. The choice of MMU is done at runtime depending on what the hardware supports so we need to detect this at runtime rather than hardcoding it.Signed-off-by Oliver O'Halloran <[email protected]>--- ndctl/Makefile.am | 3 ++- ndctl/builtin-xaction-namespace.c | 41 +++++++++++++++++++++++++++++---------- 2 files changed, 33 insertions(+), 11 deletions(-)ndctl/Makefile.am +2
-1@@ -10,7 +10,8 @@ ndctl_SOURCES = ndctl.c \ ../util/log.c \ builtin-list.c \ builtin-test.c \ - ../util/json.c + ../util/json.c \ + ../util/sysfs.c if ENABLE_SMART ndctl_SOURCES += util/json-smart.cndctl/builtin-xaction-namespace.c +8
-1@@ -22,6 +22,7 @@#include <sys/types.h> #include <util/size.h> #include <util/json.h> +#include <util/sysfs.h> #include <json-c/json.h> #include <util/filter.h> #include <ndctl/libndctl.h> @@ -54,6 +55,8 @@ static struct parameters {const char *align; } param; +char default_align_buf[SYSFS_ATTR_SIZE ]; +void builtin_xaction_namespace_reset (void ) {/* @@ -137,7 +140,24 @@ enum namespace_action { ACTION_DESTROY, }; -static int set_defaults (enum namespace_action mode) +const char *sysfs_read_default_align (struct ndctl_ctx *ctx,const char *def, +const char *path) +{ +/* +* HACK: The command handlers aren't supposed to write into ndctl/builtin-xaction-namespace.c enum namespace_action { · line 157
Quoted 10 lines
ndctl/builtin-xaction-namespace.c +10
-0+* the ndctl command context, but we want the debug +* output to go somewhere sensible. +*/ +if (__sysfs_read_attr ((struct log_ctx *)ctx, path, default_align_buf)) +return strdup (def); + +if (!strlen (default_align_buf)) +return def; + +return default_align_buf;I chatted with Dave Hansen about this and we're thinking we should go ahead and add a new attribute to the device-dax sysfs with the list of supported alignments, similar to what we have in the btt case for supported sector sizes.
Quoted 6 lines
The reason is that the sensitivity to page sizes is a device-dax internal requirement. Theoretically device-dax could support any alignment and handle it with a mix of page sizes. However, since device-dax wants to be strict and predictable about the tlb size backing a given device-dax mapping then it should list the possible options.
Sounds good to me. Using the thp sysfs entries was always going to be a bit of hack job, but I couldn't find anything better to use as a sane default. I'll post a patch Monday (unless you already wrote one?)
Quoted 4 lines
Looking at the
transparent_hugepagesysfs is a bit of a layering violation. There is no strict guarantee that device-dax is tied to thp in the longterm. The thp sysfs is also awkward because it does not tell us the pud page size.
Are there any concrete plans on moving device-dax away from THP? THP under hash has some horrible quirks and although the hardware supports up to 16G pages they're impossible to use outside of hugetlbfs. It would be nice to make use of the larger page sizes with dax devs, but I'm not sure I want to jump down that rabbit hole...
Oliver
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [PATCH 2/3] ndctl, create-namespace: read default alignment from sysfs
Quoted 92 lines · Oliver O'Halloran
Quoted 75 lines · Dan Williams
Quoted 58 lines · Oliver O'Halloran
Read the default alignment from the
hpage_pmd_sizein sysfs. On PPC the PMD size depends on the MMU being used. When the traditional hash MMU is used (P9 and earlier) the PMD size is 16MB while the newer radix MMU uses a 2MB PMD size. The choice of MMU is done at runtime depending on what the hardware supports so we need to detect this at runtime rather than hardcoding it.Signed-off-by Oliver O'Halloran <[email protected]>--- ndctl/Makefile.am | 3 ++- ndctl/builtin-xaction-namespace.c | 41 +++++++++++++++++++++++++++++---------- 2 files changed, 33 insertions(+), 11 deletions(-)ndctl/Makefile.am +2
-1@@ -10,7 +10,8 @@ ndctl_SOURCES = ndctl.c \ ../util/log.c \ builtin-list.c \ builtin-test.c \ - ../util/json.c + ../util/json.c \ + ../util/sysfs.c if ENABLE_SMART ndctl_SOURCES += util/json-smart.cndctl/builtin-xaction-namespace.c +8
-1@@ -22,6 +22,7 @@#include <sys/types.h> #include <util/size.h> #include <util/json.h> +#include <util/sysfs.h> #include <json-c/json.h> #include <util/filter.h> #include <ndctl/libndctl.h> @@ -54,6 +55,8 @@ static struct parameters {const char *align; } param; +char default_align_buf[SYSFS_ATTR_SIZE ]; +void builtin_xaction_namespace_reset (void ) {/* @@ -137,7 +140,24 @@ enum namespace_action { ACTION_DESTROY, }; -static int set_defaults (enum namespace_action mode) +const char *sysfs_read_default_align (struct ndctl_ctx *ctx,const char *def, +const char *path) +{ +/* +* HACK: The command handlers aren't supposed to write into ndctl/builtin-xaction-namespace.c enum namespace_action { · line 157
Quoted 10 lines
ndctl/builtin-xaction-namespace.c +10
-0+* the ndctl command context, but we want the debug +* output to go somewhere sensible. +*/ +if (__sysfs_read_attr ((struct log_ctx *)ctx, path, default_align_buf)) +return strdup (def); + +if (!strlen (default_align_buf)) +return def; + +return default_align_buf;I chatted with Dave Hansen about this and we're thinking we should go ahead and add a new attribute to the device-dax sysfs with the list of supported alignments, similar to what we have in the btt case for supported sector sizes.
Quoted 6 lines
The reason is that the sensitivity to page sizes is a device-dax internal requirement. Theoretically device-dax could support any alignment and handle it with a mix of page sizes. However, since device-dax wants to be strict and predictable about the tlb size backing a given device-dax mapping then it should list the possible options.
Sounds good to me. Using the thp sysfs entries was always going to be a bit of hack job, but I couldn't find anything better to use as a sane default. I'll post a patch Monday (unless you already wrote one?)
Quoted 4 lines
Looking at the
transparent_hugepagesysfs is a bit of a layering violation. There is no strict guarantee that device-dax is tied to thp in the longterm. The thp sysfs is also awkward because it does not tell us the pud page size.
Quoted 5 lines
Are there any concrete plans on moving device-dax away from THP? THP under hash has some horrible quirks and although the hardware supports up to 16G pages they're impossible to use outside of hugetlbfs. It would be nice to make use of the larger page sizes with dax devs, but I'm not sure I want to jump down that rabbit hole...
Yes, we have done this for the 1GB / pud case on x86_64. We added the base THP primitives for pud support [1] even though the rest of the THP implementation only understands pmds.
[1]: commit a00cc7d9dd93 mm, x86: add support for PUD-sized transparent hugepages
_______________________________________________
Linux-nvdimm mailing list [email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
PATCH 3/3 ndctl: print alignment of dax namespaces
In reply to Oliver O'HalloranPrint the alignment for dax namespaces creation. It's good to be explicit about it since about the selected region alignment dictates the type of faults we allow on the namespace.
--- util/json.c | 5 +++++ 1 file changed, 5 insertions(+)
util/json.c +5 -0
@@ -299,6 +299,11 @@ struct json_object *util_namespace_to_json(struct ndctl_namespace *ndns,if (!jobj) goto err;json_object_object_add (jndns,"uuid" , jobj); + jobj =json_object_new_int64 (ndctl_dax_get_align (dax)); +if (!jobj) + goto err; +json_object_object_add (jndns,"align" , jobj); +if (include_dax) { dax_region =ndctl_dax_get_daxctl_region (dax); jobj = util_daxctl_region_to_json(dax_region,
Signature · 6 lines
2.9.3 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [PATCH 3/3] ndctl: print alignment of dax namespaces
Quoted 14 lines · Oliver O'Halloran
Print the alignment for dax namespaces creation. It's good to be explicit about it since about the selected region alignment dictates the type of faults we allow on the namespace.
Signed-off-by Oliver O'Halloran <[email protected]>--- util/json.c | 5 +++++ 1 file changed, 5 insertions(+)util/json.c +0
-0
util/json.c struct json_object *util_namespace_to_json(struct ndctl_namespace *ndns, · line 306
Quoted 8 lines
util/json.c +5
-0@@ -299,6 +299,11 @@ struct json_object *util_namespace_to_json(struct ndctl_namespace *ndns,if (!jobj) goto err;json_object_object_add (jndns,"uuid" , jobj); + jobj =json_object_new_int64 (ndctl_dax_get_align (dax)); +if (!jobj) + goto err; +json_object_object_add (jndns,"align" , jobj); +
Yes, we need this, but lets hide this attribute when the 'align' is zero. That helps with ndctl running on older kernels that do not have an align attribute available.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Quoted 14 lines · Oliver O'Halloran
These are needed on powerpc since 64K is the default page size and 16MB is the PMD size when using the hash MMU.
Signed-off-by Oliver O'Halloran <[email protected]>--- ndctl/builtin-xaction-namespace.c | 2 ++ util/size.h | 1 + 2 files changed, 3 insertions(+)ndctl/builtin-xaction-namespace.c +0
-0
ndctl/builtin-xaction-namespace.c static int validate_namespace_options(struct ndctl_region *region, · line 502
Quoted 9 lines
ndctl/builtin-xaction-namespace.c +2
-0@@ -494,7 +494,9 @@ static int validate_namespace_options(struct ndctl_region *region,switch (p->align) {case SZ_4K : +case SZ_64K :case SZ_2M : +case SZ_16M :case SZ_1G :break ;default :
Following the comment on patch2, this is another location where it would be useful to just ask the kernel for the supported alignments. See how we do this for btt with the calls to ndctl_btt_get_supported_sector_size() later in the function.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm