Skip to contentSkip to conversation

Minor fixups for nd_device_register

PATCH In review Alexander Duyck 2 participants 9 messages Sep 25, 2018 libnvdimm: non-volatile memory device

lore
Share & tools
lore
#0 lore raw

cover Minor fixups for nd_device_register

This patch series addresses a couple minor issues in nd_device_register.

The first patch resolves an issue in which we weren't holding the parent lock so it is possible the parent could have been released prior to us taking care of initializing the device via the asynchronous init.

The second patch moves the initialization of the device node to the earliest point where we can do it. By doing this we can avoid repeatedly updating the node on each uevent.

---

Alexander Duyck (2):
nvdimm: Hold reference on parent while scheduling async init
nvdimm: Set device node in nd_device_register

 drivers/nvdimm/bus.c |   20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

--
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#1 lore raw

PATCH 1/2 nvdimm: Hold reference on parent while scheduling async init

Unlike asynchronous initialization in the core we have not yet associated the device with the parent, and as such the device doesn't hold a reference to the parent.

In order to resolve that we should be holding a reference on the parent until the asynchronous initialization has completed.

---
 drivers/nvdimm/bus.c |    4 ++++
 1 file changed, 4 insertions(+)
drivers/nvdimm/bus.c +4 -0
index 8aae6dcc839f..9148015ed803 100644
@@ -488,6 +488,8 @@ static void nd_async_device_register(void *d, async_cookie_t cookie)
 		put_device(dev);
 	}
 	put_device(dev);
+	if (dev->parent)
+		put_device(dev->parent);
 }
 
 static void nd_async_device_unregister(void *d, async_cookie_t cookie)
@@ -507,6 +509,8 @@ void __nd_device_register(struct device *dev)
 	if (!dev)
 		return;
 	dev->bus = &nvdimm_bus_type;
+	if (dev->parent)
+		get_device(dev->parent);
 	get_device(dev);
 	async_schedule_domain(nd_async_device_register, dev,
 			&nd_async_domain);

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#3 lore raw

PATCH 2/2 nvdimm: Set device node in nd_device_register

In reply to Alexander Duyck

This change makes it so that we don't repeatedly overwrite the device node for nvdimm regions. The earliest we can set the node is immediately after calling device init, so I have moved the code there so we can avoid rewriting the node with each uevent.

---
 drivers/nvdimm/bus.c |   16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)
drivers/nvdimm/bus.c +10 -6
index 9148015ed803..96f4d0e1706a 100644
@@ -54,12 +54,6 @@ static int to_nd_device_type(struct device *dev)
 
 static int nvdimm_bus_uevent(struct device *dev, struct kobj_uevent_env *env)
 {
-	/*
-	 * Ensure that region devices always have their numa node set as
-	 * early as possible.
-	 */
-	if (is_nd_region(dev))
-		set_dev_node(dev, to_nd_region(dev)->numa_node);
 	return add_uevent_var(env, "MODALIAS=" ND_DEVICE_MODALIAS_FMT,
 			to_nd_device_type(dev));
 }
@@ -519,6 +513,16 @@ void __nd_device_register(struct device *dev)
 void nd_device_register(struct device *dev)
 {
 	device_initialize(dev);
+
+	/*
+	 * Ensure that region devices always have their NUMA node set as
+	 * early as possible. This way we are able to make certain that any
+	 * memory associated with the creation and the creation itself of
+	 * the region is associated with the correct node.
+	 */
+	if (is_nd_region(dev))
+		set_dev_node(dev, to_nd_region(dev)->numa_node);
+
 	__nd_device_register(dev);
 }
 EXPORT_SYMBOL(nd_device_register);

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#4 lore raw

Re: [PATCH 2/2] nvdimm: Set device node in nd_device_register

Quoted 33 lines · Alexander Duyck

This change makes it so that we don't repeatedly overwrite the device node for nvdimm regions. The earliest we can set the node is immediately after calling device init, so I have moved the code there so we can avoid rewriting the node with each uevent.

---
 drivers/nvdimm/bus.c |   16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)
drivers/nvdimm/bus.c +1 -6
index 9148015ed803..96f4d0e1706a 100644
@@ -54,12 +54,6 @@ static int to_nd_device_type(struct device *dev)
 
 static int nvdimm_bus_uevent(struct device *dev, struct kobj_uevent_env *env)
 {
-       /*
-        * Ensure that region devices always have their numa node set as
-        * early as possible.
-        */
-       if (is_nd_region(dev))
-               set_dev_node(dev, to_nd_region(dev)->numa_node);
        return add_uevent_var(env, "MODALIAS=" ND_DEVICE_MODALIAS_FMT,
                        to_nd_device_type(dev));
 }
@@ -519,6 +513,16 @@ void __nd_device_register(struct device *dev)
 void nd_device_register(struct device *dev)
 {
        device_initialize(dev);
+

drivers/nvdimm/bus.c void __nd_device_register(struct device *dev) · line 526

Quoted 10 lines
drivers/nvdimm/bus.c +9 -0
+       /*
+        * Ensure that region devices always have their NUMA node set as
+        * early as possible. This way we are able to make certain that any
+        * memory associated with the creation and the creation itself of
+        * the region is associated with the correct node.
+        */
+       if (is_nd_region(dev))
+               set_dev_node(dev, to_nd_region(dev)->numa_node);
+
        __nd_device_register(dev);

Any reason to not put this inside __nd_device_register()? If you're ok with that I can just fix up when applying.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#2 lore raw

Re: [PATCH 1/2] nvdimm: Hold reference on parent while scheduling async init

In reply to Alexander Duyck
Quoted 7 lines · Alexander Duyck

Unlike asynchronous initialization in the core we have not yet associated the device with the parent, and as such the device doesn't hold a reference to the parent.

In order to resolve that we should be holding a reference on the parent until the asynchronous initialization has completed.

Looks good to me. Thanks for splitting this out, applied.
_______________________________________________
Linux-nvdimm mailing list [email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#5 lore raw

Re: [PATCH 2/2] nvdimm: Set device node in nd_device_register

In reply to Dan Williams
Quoted 40 lines · Dan Williams
Quoted 38 lines · Alexander Duyck

This change makes it so that we don't repeatedly overwrite the device node for nvdimm regions. The earliest we can set the node is immediately after calling device init, so I have moved the code there so we can avoid rewriting the node with each uevent.

---
  drivers/nvdimm/bus.c |   16 ++++++++++------
  1 file changed, 10 insertions(+), 6 deletions(-)
drivers/nvdimm/bus.c +6 -6
index 9148015ed803..96f4d0e1706a 100644
@@ -54,12 +54,6 @@ static int to_nd_device_type(struct device *dev)
 
  static int nvdimm_bus_uevent(struct device *dev, struct kobj_uevent_env *env)
  {
-       /*
-        * Ensure that region devices always have their numa node set as
-        * early as possible.
-        */
-       if (is_nd_region(dev))
-               set_dev_node(dev, to_nd_region(dev)->numa_node);
         return add_uevent_var(env, "MODALIAS=" ND_DEVICE_MODALIAS_FMT,
                         to_nd_device_type(dev));
  }
@@ -519,6 +513,16 @@ void __nd_device_register(struct device *dev)
  void nd_device_register(struct device *dev)
  {
         device_initialize(dev);
+
+       /*
+        * Ensure that region devices always have their NUMA node set as
+        * early as possible. This way we are able to make certain that any
+        * memory associated with the creation and the creation itself of
+        * the region is associated with the correct node.
Quoted 8 lines

drivers/nvdimm/bus.c void __nd_device_register(struct device *dev) · line 526

Quoted 5 lines
drivers/nvdimm/bus.c +4 -0
+        */
+       if (is_nd_region(dev))
+               set_dev_node(dev, to_nd_region(dev)->numa_node);
+
         __nd_device_register(dev);

Any reason to not put this inside __nd_device_register()? If you're ok with that I can just fix up when applying.

I put it here since regions never call __nd_device_register directly, they always call nd_device_register. I figured that by placing it here we reduce the impact of this on the other devices that aren't going to need it.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#6 lore raw

Re: [PATCH 2/2] nvdimm: Set device node in nd_device_register

Quoted 53 lines · Alexander Duyck
Quoted 40 lines · Dan Williams
Quoted 38 lines · Alexander Duyck

This change makes it so that we don't repeatedly overwrite the device node for nvdimm regions. The earliest we can set the node is immediately after calling device init, so I have moved the code there so we can avoid rewriting the node with each uevent.

---
  drivers/nvdimm/bus.c |   16 ++++++++++------
  1 file changed, 10 insertions(+), 6 deletions(-)
drivers/nvdimm/bus.c +6 -6
index 9148015ed803..96f4d0e1706a 100644
@@ -54,12 +54,6 @@ static int to_nd_device_type(struct device *dev)
 
  static int nvdimm_bus_uevent(struct device *dev, struct kobj_uevent_env *env)
  {
-       /*
-        * Ensure that region devices always have their numa node set as
-        * early as possible.
-        */
-       if (is_nd_region(dev))
-               set_dev_node(dev, to_nd_region(dev)->numa_node);
         return add_uevent_var(env, "MODALIAS=" ND_DEVICE_MODALIAS_FMT,
                         to_nd_device_type(dev));
  }
@@ -519,6 +513,16 @@ void __nd_device_register(struct device *dev)
  void nd_device_register(struct device *dev)
  {
         device_initialize(dev);
+
+       /*
+        * Ensure that region devices always have their NUMA node set as
+        * early as possible. This way we are able to make certain that any
+        * memory associated with the creation and the creation itself of
+        * the region is associated with the correct node.
Quoted 8 lines

drivers/nvdimm/bus.c void __nd_device_register(struct device *dev) · line 526

Quoted 5 lines
drivers/nvdimm/bus.c +4 -0
+        */
+       if (is_nd_region(dev))
+               set_dev_node(dev, to_nd_region(dev)->numa_node);
+
         __nd_device_register(dev);

Any reason to not put this inside __nd_device_register()? If you're ok with that I can just fix up when applying.

Quoted 4 lines

I put it here since regions never call __nd_device_register directly, they always call nd_device_register. I figured that by placing it here we reduce the impact of this on the other devices that aren't going to need it.

I can't imagine there's much of an impact because we're already reading the cacheline that has dev->parent so dev->type is already there. If we're going to add support for looking up the parent node to schedule namespace registration work to the proper core I would expect that to appear in the __nd_device_register() common code as well.

In general the only consideration for calling nd_device_register() vs __nd_device_register() is whether or not you want a device_initialize() to occur.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#7 lore raw

Re: [PATCH 2/2] nvdimm: Set device node in nd_device_register

Quoted 66 lines · Dan Williams
Quoted 53 lines · Alexander Duyck
Quoted 40 lines · Dan Williams
Quoted 38 lines · Alexander Duyck

This change makes it so that we don't repeatedly overwrite the device node for nvdimm regions. The earliest we can set the node is immediately after calling device init, so I have moved the code there so we can avoid rewriting the node with each uevent.

---
   drivers/nvdimm/bus.c |   16 ++++++++++------
   1 file changed, 10 insertions(+), 6 deletions(-)
drivers/nvdimm/bus.c +6 -6
index 9148015ed803..96f4d0e1706a 100644
@@ -54,12 +54,6 @@ static int to_nd_device_type(struct device *dev)
 
   static int nvdimm_bus_uevent(struct device *dev, struct kobj_uevent_env *env)
   {
-       /*
-        * Ensure that region devices always have their numa node set as
-        * early as possible.
-        */
-       if (is_nd_region(dev))
-               set_dev_node(dev, to_nd_region(dev)->numa_node);
          return add_uevent_var(env, "MODALIAS=" ND_DEVICE_MODALIAS_FMT,
                          to_nd_device_type(dev));
   }
@@ -519,6 +513,16 @@ void __nd_device_register(struct device *dev)
   void nd_device_register(struct device *dev)
   {
          device_initialize(dev);
+
+       /*
+        * Ensure that region devices always have their NUMA node set as
+        * early as possible. This way we are able to make certain that any
+        * memory associated with the creation and the creation itself of
+        * the region is associated with the correct node.
Quoted 8 lines

drivers/nvdimm/bus.c void __nd_device_register(struct device *dev) · line 526

Quoted 5 lines
drivers/nvdimm/bus.c +4 -0
+        */
+       if (is_nd_region(dev))
+               set_dev_node(dev, to_nd_region(dev)->numa_node);
+
          __nd_device_register(dev);

Any reason to not put this inside __nd_device_register()? If you're ok with that I can just fix up when applying.

Quoted 4 lines

I put it here since regions never call __nd_device_register directly, they always call nd_device_register. I figured that by placing it here we reduce the impact of this on the other devices that aren't going to need it.

I can't imagine there's much of an impact because we're already reading the cacheline that has dev->parent so dev->type is already there. If we're going to add support for looking up the parent node to schedule namespace registration work to the proper core I would expect that to appear in the __nd_device_register() common code as well.

Quoted 3 lines

In general the only consideration for calling nd_device_register() vs __nd_device_register() is whether or not you want a device_initialize() to occur.

Yeah, I kind of gathered that. If you want to move it feel free to.

- Alex
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm