libnvdimm: Update the meaning for persistence_domain values
Subsystems (2)
RFC PATCH 1/1 libnvdimm: Update the meaning for persistence_domain values
Currently, kernel shows the below values
"persistence_domain":"cpu_cache"
"persistence_domain":"memory_controller"
"persistence_domain":"unknown"
This patch updates the meaning of these values such that
"cpu_cache" indicates no extra instructions is needed to ensure the persistence of data in the pmem media on power failure.
"memory_controller" indicates platform provided instructions need to be issued as per documented sequence to make sure data flushed is guaranteed to be on pmem media in case of system power loss.
--- arch/powerpc/platforms/pseries/papr_scm.c | 7 ++++++- include/linux/libnvdimm.h | 6 +++--- 2 files changed, 9 insertions(+), 4 deletions(-)
arch/powerpc/platforms/pseries/papr_scm.c +6 -1
@@ -360,8 +360,13 @@ static int papr_scm_nvdimm_init(struct papr_scm_priv *p)if (p->is_volatile) p->region =nvdimm_volatile_region_create (p->bus, &ndr_desc); -else +else { +/* +* We need to flush things correctly to guarantee persistance +*/ +set_bit (ND_REGION_PERSIST_MEMCTRL , &ndr_desc.flags); p->region =nvdimm_pmem_region_create (p->bus, &ndr_desc); + }if (!p->region) {dev_err (dev,"Error registering region %pR from %pOF\n" , ndr_desc.res, p->dn);
include/linux/libnvdimm.h +3 -3
@@ -52,9 +52,9 @@ enum {*/ ND_REGION_PERSIST_CACHE =1 ,/* -* Platform provides mechanisms to automatically flush outstanding -* write data from memory controler to pmem on system power loss. -* (ADR) +* Platform provides instructions to flush data such that on completion +* of the instructions, data flushed is guaranteed to be on pmem even +* in case of a system power loss. */ ND_REGION_PERSIST_MEMCTRL = 2,
Signature · 4 lines
2.24.1 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Quoted 29 lines · Aneesh Kumar K.V
Currently, kernel shows the below values
"persistence_domain":"cpu_cache"
"persistence_domain":"memory_controller"
"persistence_domain":"unknown"This patch updates the meaning of these values such that
"
cpu_cache" indicates no extra instructions is needed to ensure the persistence of data in the pmem media on power failure."
memory_controller" indicates platform provided instructions need to be issued as per documented sequence to make sure data flushed is guaranteed to be on pmem media in case of system power loss.Signed-off-by Aneesh Kumar K.V <[email protected]>--- arch/powerpc/platforms/pseries/papr_scm.c | 7 ++++++- include/linux/libnvdimm.h | 6 +++--- 2 files changed, 9 insertions(+), 4 deletions(-)arch/powerpc/platforms/pseries/papr_scm.c +0
-1@@ -360,8 +360,13 @@ static int papr_scm_nvdimm_init(struct papr_scm_priv *p)if (p->is_volatile) p->region =nvdimm_volatile_region_create (p->bus, &ndr_desc); -else
arch/powerpc/platforms/pseries/papr_scm.c static int papr_scm_nvdimm_init(struct papr_scm_priv *p) · line 372
Quoted 10 lines
arch/powerpc/platforms/pseries/papr_scm.c +6
-0+else { +/* +* We need to flush things correctly to guarantee persistance +*/ +set_bit (ND_REGION_PERSIST_MEMCTRL , &ndr_desc.flags); p->region =nvdimm_pmem_region_create (p->bus, &ndr_desc); + }if (!p->region) {dev_err (dev,"Error registering region %pR from %pOF\n" , ndr_desc.res, p->dn);
Would you also update of_pmem to indicate the persistence domain, please?
Quoted 5 lines
include/linux/libnvdimm.h +0
-0
include/linux/libnvdimm.h enum { · line 57
Quoted 9 lines
include/linux/libnvdimm.h +3
-3@@ -52,9 +52,9 @@ enum {*/ ND_REGION_PERSIST_CACHE = 1,/* -* Platform provides mechanisms to automatically flush outstanding -* write data from memory controler to pmem on system power loss. -* (ADR) +* Platform provides instructions to flush data such that on completion +* of the instructions, data flushed is guaranteed to be on pmem even +* in case of a system power loss.
I find the prior description easier to understand.
-Jeff
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Quoted 28 lines · Jeff Moyer
Quoted 26 lines · Aneesh Kumar K.V
Currently, kernel shows the below values
"persistence_domain":"cpu_cache"
"persistence_domain":"memory_controller"
"persistence_domain":"unknown"This patch updates the meaning of these values such that
"
cpu_cache" indicates no extra instructions is needed to ensure the persistence of data in the pmem media on power failure."
memory_controller" indicates platform provided instructions need to be issued as per documented sequence to make sure data flushed is guaranteed to be on pmem media in case of system power loss.Signed-off-by Aneesh Kumar K.V <[email protected]>--- arch/powerpc/platforms/pseries/papr_scm.c | 7 ++++++- include/linux/libnvdimm.h | 6 +++--- 2 files changed, 9 insertions(+), 4 deletions(-)arch/powerpc/platforms/pseries/papr_scm.c +0
-0@@ -360,8 +360,13 @@ static int papr_scm_nvdimm_init(struct papr_scm_priv *p)
Quoted 17 lines
arch/powerpc/platforms/pseries/papr_scm.c static int papr_scm_nvdimm_init(struct papr_scm_priv *p) · line 372
Quoted 13 lines
arch/powerpc/platforms/pseries/papr_scm.c +6
-1if (p->is_volatile) p->region =nvdimm_volatile_region_create (p->bus, &ndr_desc); -else +else { +/* +* We need to flush things correctly to guarantee persistance +*/ +set_bit (ND_REGION_PERSIST_MEMCTRL , &ndr_desc.flags); p->region =nvdimm_pmem_region_create (p->bus, &ndr_desc); + }if (!p->region) {dev_err (dev,"Error registering region %pR from %pOF\n" , ndr_desc.res, p->dn);Would you also update
of_pmemto indicate the persistence domain, please?
sure.
Quoted 8 lines
Quoted 8 lines
include/linux/libnvdimm.h +0
-0@@ -52,9 +52,9 @@ enum {*/ ND_REGION_PERSIST_CACHE = 1,/*
Quoted 8 lines
include/linux/libnvdimm.h enum { · line 57
Quoted 6 lines
include/linux/libnvdimm.h +3
-3-* Platform provides mechanisms to automatically flush outstanding -* write data from memory controler to pmem on system power loss. -* (ADR) +* Platform provides instructions to flush data such that on completion +* of the instructions, data flushed is guaranteed to be on pmem even +* in case of a system power loss. I find the prior description easier to understand.
I was trying to avoid the term 'automatically, 'memory controler' and ADR. Can I update the above as
/*
* Platform provides mechanisms to flush outstanding write data
* to pmem on system power loss.
*/
-aneesh
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Quoted 70 lines · Aneesh Kumar K.V
Quoted 36 lines · Jeff Moyer
Quoted 34 lines · Aneesh Kumar K.V
Currently, kernel shows the below values
"persistence_domain":"cpu_cache"
"persistence_domain":"memory_controller"
"persistence_domain":"unknown"This patch updates the meaning of these values such that
"
cpu_cache" indicates no extra instructions is needed to ensure the persistence of data in the pmem media on power failure."
memory_controller" indicates platform provided instructions need to be issued as per documented sequence to make sure data flushed is guaranteed to be on pmem media in case of system power loss.Signed-off-by Aneesh Kumar K.V <[email protected]>---
arch/powerpc/platforms/pseries/papr_scm.c| 7 ++++++- include/linux/libnvdimm.h | 6 +++---2 files changed, 9 insertions(+), 4 deletions(-)arch/powerpc/platforms/pseries/papr_scm.c +5
-1@@ -360,8 +360,13 @@ static int papr_scm_nvdimm_init(struct papr_scm_priv *p) if (p->is_volatile) p->region = nvdimm_volatile_region_create(p->bus, &ndr_desc); - else + else { + /* + * We need to flush things correctly to guarantee persistance + */ + set_bit(ND_REGION_PERSIST_MEMCTRL, &ndr_desc.flags);Quoted 9 lines
arch/powerpc/platforms/pseries/papr_scm.c static int papr_scm_nvdimm_init(struct papr_scm_priv *p) · line 372
Quoted 5 lines
arch/powerpc/platforms/pseries/papr_scm.c +1
-0p->region = nvdimm_pmem_region_create(p->bus, &ndr_desc); + } if (!p->region) { dev_err(dev, "Error registering region %pR from %pOF\n", ndr_desc.res, p->dn);Would you also update
of_pmemto indicate the persistence domain, please?sure.
Quoted 10 lines
Quoted 10 lines
include/linux/libnvdimm.h +0
-2@@ -52,9 +52,9 @@ enum {*/ ND_REGION_PERSIST_CACHE = 1, /* -* Platform provides mechanisms to automatically flush outstanding -* write data from memory controler to pmem on system power loss. Quoted 8 lines
Quoted 6 lines
include/linux/libnvdimm.h +1
-1- * (ADR) + * Platform provides instructions to flush data such that oncompletion
+ * of the instructions, data flushed is guaranteed to be on pmem even
+ * in case of a system power loss.I find the prior description easier to understand.
Quoted 7 lines
I was trying to avoid the term 'automatically, 'memory controler' and ADR. Can I update the above as
/*
* Platform provides mechanisms to flush outstanding write data
* to pmem on system power loss.
*/
Wanted to add more details. So with the above interpretation, if the persistence_domain is found to be 'cpu_cache', application can expect a store instruction to guarantee persistence. If it is 'none' there is no persistence ( I am not sure how that is the difference from 'volatile' pmem region). If it is 'memory_controller' ( I am not sure whether that is the right term), application needs to follow the recommended mechanism to flush write data to pmem.
-aneesh
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Quoted 6 lines · Aneesh Kumar K.V
Quoted 5 lines
/*
* Platform provides mechanisms to flush outstanding write data
* to pmem on system power loss.
*/
Quoted 7 lines
Wanted to add more details. So with the above interpretation, if the
persistence_domainis found to be 'cpu_cache', application can expect a store instruction to guarantee persistence. If it is 'none' there is no persistence ( I am not sure how that is the difference from 'volatile' pmem region). If it is 'memory_controller' ( I am not sure whether that is the right term), application needs to follow the recommended mechanism to flush write data to pmem.
There is no enum for "NONE". If there is no persistence domain specified, then it is undefined/unknown, which results in the message:
nd_pmem namespace0.0: unable to guarantee persistence of writes
Other than that, yes, that's right.
-Jeff
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Quoted 85 lines · Aneesh Kumar K.V
Quoted 74 lines · Aneesh Kumar K.V
Quoted 40 lines · Jeff Moyer
Quoted 38 lines · Aneesh Kumar K.V
Currently, kernel shows the below values
"persistence_domain":"cpu_cache"
"persistence_domain":"memory_controller"
"persistence_domain":"unknown"This patch updates the meaning of these values such that
"
cpu_cache" indicates no extra instructions is needed to ensure the persistence
of data in the pmem media on power failure."
memory_controller" indicates platform provided instructions need to be issued
as per documented sequence to make sure data flushed is guaranteed to be on pmem
media in case of system power loss.Signed-off-by Aneesh Kumar K.V <[email protected]>--- arch/powerpc/platforms/pseries/papr_scm.c | 7 ++++++- include/linux/libnvdimm.h | 6 +++--- 2 files changed, 9 insertions(+), 4 deletions(-)arch/powerpc/platforms/pseries/papr_scm.c +5
-1@@ -360,8 +360,13 @@ static int papr_scm_nvdimm_init(struct papr_scm_priv *p)if (p->is_volatile) p->region =nvdimm_volatile_region_create (p->bus, &ndr_desc); -else +else { +/* +* We need to flush things correctly to guarantee persistance +*/ +set_bit (ND_REGION_PERSIST_MEMCTRL , &ndr_desc.flags);Quoted 9 lines
arch/powerpc/platforms/pseries/papr_scm.c static int papr_scm_nvdimm_init(struct papr_scm_priv *p) · line 372
Quoted 5 lines
arch/powerpc/platforms/pseries/papr_scm.c +1
-0p->region =nvdimm_pmem_region_create (p->bus, &ndr_desc); + }if (!p->region) {dev_err (dev,"Error registering region %pR from %pOF\n" , ndr_desc.res, p->dn);Would you also update
of_pmemto indicate the persistence domain, please?sure.
Quoted 10 lines
Quoted 10 lines
include/linux/libnvdimm.h +0
-2@@ -52,9 +52,9 @@ enum {*/ ND_REGION_PERSIST_CACHE = 1,/* -* Platform provides mechanisms to automatically flush outstanding -* write data from memory controler to pmem on system power loss. Quoted 8 lines
Quoted 6 lines
include/linux/libnvdimm.h +1
-1-* (ADR) +* Platform provides instructions to flush data such that on completion
+ * of the instructions, data flushed is guaranteed to be on pmem
even
+ * in case of a system power loss.I find the prior description easier to understand.
Quoted 8 lines
I was trying to avoid the term 'automatically, 'memory controler' and ADR. Can I update the above as
/*
* Platform provides mechanisms to flush outstanding write data
* to pmem on system power loss.
*/
Quoted 3 lines
Wanted to add more details. So with the above interpretation, if the
persistence_domainis found to be 'cpu_cache', application can expect a store instruction to guarantee persistence.
The expectation is globally visible stores are persisted
Quoted 3 lines
If it is 'none' there is no
persistence ( I am not sure how that is the difference from 'volatile' pmem region).
"None" means the "platform does not enumerate a persistence domain". That's not necessarily "volatile" because the user may know that they have battery backup, or some other private/out-of-band capability not exported by the platform. In which case they would manually need to manipulate the pmemX/write_cache property manually.
Quoted 3 lines
If it is '
memory_controller' ( I am not sure whether that is the right term), application needs to follow the recommended mechanism to flush write data to pmem.
If it is memory_controller the expectation is that flushing data out of the cpu caches and making those writebacks to memory globally visible is sufficient for the platform to persist flushed data on a power loss.
_______________________________________________
Linux-nvdimm mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Quoted 5 lines · Aneesh Kumar K.V
Quoted 3 lines
Would you also update
of_pmemto indicate the persistence domain, please?sure.
Thanks!
Quoted 17 lines
Quoted 8 lines
Quoted 8 lines
include/linux/libnvdimm.h +0
-0@@ -52,9 +52,9 @@ enum {*/ ND_REGION_PERSIST_CACHE = 1,/* Quoted 8 lines
include/linux/libnvdimm.h enum { · line 57
Quoted 6 lines
include/linux/libnvdimm.h +3
-3-* Platform provides mechanisms to automatically flush outstanding -* write data from memory controler to pmem on system power loss. -* (ADR) +* Platform provides instructions to flush data such that on completion +* of the instructions, data flushed is guaranteed to be on pmem even +* in case of a system power loss. I find the prior description easier to understand.
Quoted 2 lines
I was trying to avoid the term 'automatically, 'memory controler' and ADR. Can I update the above as
I can understand avoiding the very x86-specific "ADR," but is memory controller not accurate for your platform?
Quoted 4 lines
/*
* Platform provides mechanisms to flush outstanding write data
* to pmem on system power loss.
*/
That's way too broad. :) The comments are describing the persistence domain. i.e. if you get data to $HERE, it is guaranteed to make it out to stable media.
-Jeff
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Quoted 26 lines · Jeff Moyer
Quoted 17 lines · Aneesh Kumar K.V
Quoted 8 lines
Quoted 8 lines
include/linux/libnvdimm.h +0
-0@@ -52,9 +52,9 @@ enum {*/ ND_REGION_PERSIST_CACHE = 1,/* Quoted 8 lines
include/linux/libnvdimm.h enum { · line 57
Quoted 6 lines
include/linux/libnvdimm.h +3
-3-* Platform provides mechanisms to automatically flush outstanding -* write data from memory controler to pmem on system power loss. -* (ADR) +* Platform provides instructions to flush data such that on completion +* of the instructions, data flushed is guaranteed to be on pmem even +* in case of a system power loss. I find the prior description easier to understand.
Quoted 2 lines
I was trying to avoid the term 'automatically, 'memory controler' and ADR. Can I update the above as
I can understand avoiding the very x86-specific "ADR," but is memory controller not accurate for your platform?
Quoted 8 lines
Quoted 4 lines
/*
* Platform provides mechanisms to flush outstanding write data
* to pmem on system power loss.
*/That's way too broad. :) The comments are describing the persistence domain. i.e. if you get data to $HERE, it is guaranteed to make it out to stable media.
With technologies like OpenCAPI, we possibly may not want to call them "memory controller"? In a way platform mechanism will flush them such that on power failure, it is guaranteed to be on the pmem media. But should we call these boundary "memory_controller"? May be we can consider "memory controller" an overloaded term there. Considering we are calling this as memory_controller for application to parse via sysfs, may be the documentation can also carry the same term.
-aneesh
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Quoted 37 lines · Aneesh Kumar K.V
Quoted 26 lines · Jeff Moyer
Quoted 17 lines · Aneesh Kumar K.V
Quoted 8 lines
Quoted 8 lines
include/linux/libnvdimm.h +0
-0@@ -52,9 +52,9 @@ enum {*/ ND_REGION_PERSIST_CACHE = 1,/* Quoted 8 lines
include/linux/libnvdimm.h enum { · line 57
Quoted 6 lines
include/linux/libnvdimm.h +3
-3-* Platform provides mechanisms to automatically flush outstanding -* write data from memory controler to pmem on system power loss. -* (ADR) +* Platform provides instructions to flush data such that on completion +* of the instructions, data flushed is guaranteed to be on pmem even +* in case of a system power loss. I find the prior description easier to understand.
Quoted 2 lines
I was trying to avoid the term 'automatically, 'memory controler' and ADR. Can I update the above as
I can understand avoiding the very x86-specific "ADR," but is memory controller not accurate for your platform?
Quoted 8 lines
Quoted 4 lines
/*
* Platform provides mechanisms to flush outstanding write data
* to pmem on system power loss.
*/That's way too broad. :) The comments are describing the persistence domain. i.e. if you get data to $HERE, it is guaranteed to make it out to stable media.
Quoted 7 lines
With technologies like OpenCAPI, we possibly may not want to call them "memory controller"? In a way platform mechanism will flush them such that on power failure, it is guaranteed to be on the pmem media. But should we call these boundary "
memory_controller"? May be we can consider "memory controller" an overloaded term there. Considering we are calling this asmemory_controllerfor application to parse via sysfs, may be the documentation can also carry the same term.
I don't see how OpenCAPI or any other transport has any bearing on the "memory_controller" term. It's still a controller of persistent memory and it needs to have the write data received at its buffers / queue to ensure that the data gets persisted, or, as in the cpu_cache case, some other agent takes responsibility for shuttling pending writes that have hit the cache out over the transport to be persisted.
_______________________________________________
Linux-nvdimm mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Quoted 47 lines · Dan Williams
Quoted 37 lines · Aneesh Kumar K.V
Quoted 26 lines · Jeff Moyer
Quoted 17 lines · Aneesh Kumar K.V
Quoted 8 lines
Quoted 8 lines
include/linux/libnvdimm.h +0
-0@@ -52,9 +52,9 @@ enum {*/ ND_REGION_PERSIST_CACHE = 1,/* Quoted 8 lines
include/linux/libnvdimm.h enum { · line 57
Quoted 6 lines
include/linux/libnvdimm.h +3
-3-* Platform provides mechanisms to automatically flush outstanding -* write data from memory controler to pmem on system power loss. -* (ADR) +* Platform provides instructions to flush data such that on completion +* of the instructions, data flushed is guaranteed to be on pmem even +* in case of a system power loss. I find the prior description easier to understand.
Quoted 2 lines
I was trying to avoid the term 'automatically, 'memory controler' and ADR. Can I update the above as
I can understand avoiding the very x86-specific "ADR," but is memory controller not accurate for your platform?
Quoted 8 lines
Quoted 4 lines
/*
* Platform provides mechanisms to flush outstanding write data
* to pmem on system power loss.
*/That's way too broad. :) The comments are describing the persistence domain. i.e. if you get data to $HERE, it is guaranteed to make it out to stable media.
Quoted 7 lines
With technologies like OpenCAPI, we possibly may not want to call them "memory controller"? In a way platform mechanism will flush them such that on power failure, it is guaranteed to be on the pmem media. But should we call these boundary "
memory_controller"? May be we can consider "memory controller" an overloaded term there. Considering we are calling this asmemory_controllerfor application to parse via sysfs, may be the documentation can also carry the same term.
Quoted 7 lines
I don't see how OpenCAPI or any other transport has any bearing on the "
memory_controller" term. It's still a controller of persistent memory and it needs to have the write data received at its buffers / queue to ensure that the data gets persisted, or, as in thecpu_cachecase, some other agent takes responsibility for shuttling pending writes that have hit the cache out over the transport to be persisted.
Agreed. I want to make sure we document that details correctly. It is a controller of persistent memory and in some cases, there is no reserve power available to keep things in self-refresh mode and to flush things automatically. The platform provided mechanism will ensure the write data is in the pmem media.
Should the later have a persistence_domain value "pmem media" ? Even then I am not sure how applications are supposed to use this information.
IMHO what is important for application is to differentiate between whether a platform specific flush mechanism is needed or not. Hence was trying to keep this as two value property. IS there any other detail application is supposed to infer from this property?
-aneesh
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]