fpga: improve protection against low-level control module unloading
Subsystems (2)
v5 cover fpga: improve protection against low-level control module unloading
This RFC proposes a solution to keep protecting the fpga manager against the unloading of the low-level control modules while addressing the limitations of the current implementation. Currently, the code assumes that the low-level module registers a driver for the parent device that is later used to take the module's refcount. This proposal removes this limitation by adding a module owner field to the fpga_manager struct that can be set while registering the manager.
For more context, please refer to these threads:
https://lore.kernel.org/linux-fpga/ZS6hhlvjUcqyv8zL@yilunxu-OptiPlex-7050
https://lore.kernel.org/linux-fpga/ZT9qENE9fE3Z0KCW@yilunxu-OptiPlex-7050
v5:
- Updated the documentation
- Removed kernel-doc comments for helper macros v4:
- Use helper macros to set the owner module v3:
- Improved locking
v2:
- Fixed protection against races during module removal
Marco Pagani (1):
fpga: add an owner and use it to take the low-level module's refcount
Documentation/driver-api/fpga/fpga-mgr.rst | 34 ++++---- drivers/fpga/fpga-mgr.c | 93 ++++++++++++++-------- include/linux/fpga/fpga-mgr.h | 28 +++++-- 3 files changed, 102 insertions(+), 53 deletions(-)
base-commit: c849ecb2ae8413f86c84627cb0af06dffce4e215
Signature · 1 line
2.43.0
RFC PATCH v5 1/1 fpga: add an owner and use it to take the low-level module's refcount
Add a module owner field to the fpga_manager struct to take the low-level control module refcount instead of assuming that the parent device has a driver and using its owner pointer. The owner is now passed as an additional argument at registration time. To this end, the functions for registration have been modified to take an additional owner parameter and renamed to avoid conflicts. The old function names are now used for helper macros that automatically set the module that registers the fpga manager as the owner. This ensures compatibility with existing low-level control modules and reduces the chances of registering a manager without setting the owner.
To detect when the owner module pointer becomes stale, set the mops pointer to null during fpga_mgr_unregister() and test it before taking the module's refcount. Use a mutex to protect against a crash that can happen if __fpga_mgr_get() gets suspended between testing the mops pointer and taking the refcount while the low-level module is being unloaded.
Update the documentation to keep it consistent with the new interface for registering an fpga manager.
Other changes: opportunistically move put_device() from __fpga_mgr_get() to fpga_mgr_get() and of_fpga_mgr_get() to improve code clarity since the device refcount in taken in these functions.
--- Documentation/driver-api/fpga/fpga-mgr.rst | 34 ++++---- drivers/fpga/fpga-mgr.c | 93 ++++++++++++++-------- include/linux/fpga/fpga-mgr.h | 28 +++++-- 3 files changed, 102 insertions(+), 53 deletions(-)
Documentation/driver-api/fpga/fpga-mgr.rst +20 -14
@@ -24,7 +24,8 @@ How to support a new FPGA device -------------------------------- To add another FPGA manager, write a driver that implements a set of ops. The -probe function calls fpga_mgr_register() or fpga_mgr_register_full(), such as:: +probe function calls ``fpga_mgr_register()`` or ``fpga_mgr_register_full()``, +such as:: static const struct fpga_manager_ops socfpga_fpga_ops = { .write_init = socfpga_fpga_ops_configure_init,
@@ -69,10 +70,11 @@ probe function calls fpga_mgr_register() or fpga_mgr_register_full(), such as:: } Alternatively, the probe function could call one of the resource managed -register functions, devm_fpga_mgr_register() or devm_fpga_mgr_register_full(). -When these functions are used, the parameter syntax is the same, but the call -to fpga_mgr_unregister() should be removed. In the above example, the -socfpga_fpga_remove() function would not be required. +register functions, ``devm_fpga_mgr_register()`` or +``devm_fpga_mgr_register_full()``. When these functions are used, the +parameter syntax is the same, but the call to ``fpga_mgr_unregister()`` should be +removed. In the above example, the ``socfpga_fpga_remove()`` function would not be +required. The ops will implement whatever device specific register writes are needed to do the programming sequence for this particular FPGA. These ops return 0 for
@@ -125,15 +127,19 @@ API for implementing a new FPGA Manager driver * struct fpga_manager - the FPGA manager struct * struct fpga_manager_ops - Low level FPGA manager driver ops * struct fpga_manager_info - Parameter structure for fpga_mgr_register_full() -* fpga_mgr_register_full() - Create and register an FPGA manager using the +* __fpga_mgr_register_full() - Create and register an FPGA manager using the fpga_mgr_info structure to provide the full flexibility of options -* fpga_mgr_register() - Create and register an FPGA manager using standard +* __fpga_mgr_register() - Create and register an FPGA manager using standard arguments -* devm_fpga_mgr_register_full() - Resource managed version of - fpga_mgr_register_full() -* devm_fpga_mgr_register() - Resource managed version of fpga_mgr_register() +* __devm_fpga_mgr_register_full() - Resource managed version of + __fpga_mgr_register_full() +* __devm_fpga_mgr_register() - Resource managed version of __fpga_mgr_register() * fpga_mgr_unregister() - Unregister an FPGA manager +Helper macros ``fpga_mgr_register_full()``, ``fpga_mgr_register()``, +``devm_fpga_mgr_register_full()``, and ``devm_fpga_mgr_register()`` are available +to ease the registration. + .. kernel-doc:: include/linux/fpga/fpga-mgr.h :functions: fpga_mgr_states
@@ -147,16 +153,16 @@ API for implementing a new FPGA Manager driver :functions: fpga_manager_info .. kernel-doc:: drivers/fpga/fpga-mgr.c - :functions: fpga_mgr_register_full + :functions: __fpga_mgr_register_full .. kernel-doc:: drivers/fpga/fpga-mgr.c - :functions: fpga_mgr_register + :functions: __fpga_mgr_register .. kernel-doc:: drivers/fpga/fpga-mgr.c - :functions: devm_fpga_mgr_register_full + :functions: __devm_fpga_mgr_register_full .. kernel-doc:: drivers/fpga/fpga-mgr.c - :functions: devm_fpga_mgr_register + :functions: __devm_fpga_mgr_register .. kernel-doc:: drivers/fpga/fpga-mgr.c :functions: fpga_mgr_unregister
drivers/fpga/fpga-mgr.c +60 -33
225 lines not on this page · raw message
include/linux/fpga/fpga-mgr.h +22 -6
@@ -201,6 +201,8 @@ struct fpga_manager_ops {* @state: state of fpga manager * @compat_id: FPGA manager id for compatibility check. * @mops: pointer to struct of fpga manager ops +* @mops_mutex: protects mops from low-level module removal +* @mops_owner: module containing the mops * @priv: low level driver private date */ struct fpga_manager {
@@ -210,6 +212,8 @@ struct fpga_manager {enum fpga_mgr_states state;struct fpga_compat_id *compat_id;const struct fpga_manager_ops *mops; +struct mutex mops_mutex; +struct module *mops_owner;void *priv; };
@@ -230,18 +234,30 @@ struct fpga_manager *fpga_mgr_get(struct device *dev);void fpga_mgr_put (struct fpga_manager *mgr); +#define fpga_mgr_register_full (parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner); +#define fpga_mgr_register (parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE)struct fpga_manager * -fpga_mgr_register (struct device *parent,const char *name, -const struct fpga_manager_ops *mops,void *priv); +__fpga_mgr_register (struct device *parent,const char *name, +const struct fpga_manager_ops *mops,void *priv,struct module *owner); +void fpga_mgr_unregister (struct fpga_manager *mgr); +#define devm_fpga_mgr_register_full (parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner); +#define devm_fpga_mgr_register (parent, name, mops, priv) \ + __devm_fpga_mgr_register(parent, name, mops, priv, THIS_MODULE)struct fpga_manager * -devm_fpga_mgr_register (struct device *parent,const char *name, -const struct fpga_manager_ops *mops,void *priv); +__devm_fpga_mgr_register (struct device *parent,const char *name, +const struct fpga_manager_ops *mops,void *priv, +struct module *owner);#endif /*_LINUX_FPGA_MGR_H */
Signature · 1 line
2.43.0
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 13 lines
include/linux/fpga/fpga-mgr.h +7
-3+#define fpga_mgr_register_full (parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner); +#define fpga_mgr_register (parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE)struct fpga_manager * -fpga_mgr_register (struct device *parent,const char *name, -const struct fpga_manager_ops *mops,void *priv); +__fpga_mgr_register(struct device *parent, const char *name,
include/linux/fpga/fpga-mgr.h struct fpga_manager *fpga_mgr_get(struct device *dev); · line 255
Quoted 10 lines
include/linux/fpga/fpga-mgr.h +6
-1+ const struct fpga_manager_ops *mops, void *priv, struct module *owner); +void fpga_mgr_unregister (struct fpga_manager *mgr); +#define devm_fpga_mgr_register_full (parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner);
Add a line here. I can do it myself if you agree.
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
Next time if you think patches are ready for serious review and merge, drop the RFC prefix. That avoids an extra query.
Thanks,
Yilun
Quoted 14 lines
include/linux/fpga/fpga-mgr.h +5
-2+#define devm_fpga_mgr_register (parent, name, mops, priv) \ + __devm_fpga_mgr_register(parent, name, mops, priv, THIS_MODULE)struct fpga_manager * -devm_fpga_mgr_register (struct device *parent,const char *name, -const struct fpga_manager_ops *mops,void *priv); +__devm_fpga_mgr_register (struct device *parent,const char *name, +const struct fpga_manager_ops *mops,void *priv, +struct module *owner);#endif /*_LINUX_FPGA_MGR_H */ Signature · 1 line
2.43.0
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 17 lines · Xu Yilun
Quoted 17 lines
include/linux/fpga/fpga-mgr.h +9
-3+#define fpga_mgr_register_full (parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner); +#define fpga_mgr_register (parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE)struct fpga_manager * -fpga_mgr_register (struct device *parent,const char *name, -const struct fpga_manager_ops *mops,void *priv); +__fpga_mgr_register (struct device *parent,const char *name, +const struct fpga_manager_ops *mops,void *priv,struct module *owner); +void fpga_mgr_unregister (struct fpga_manager *mgr);
Quoted 8 lines
include/linux/fpga/fpga-mgr.h struct fpga_manager *fpga_mgr_get(struct device *dev); · line 255
Quoted 6 lines
include/linux/fpga/fpga-mgr.h +4
-1+#define devm_fpga_mgr_register_full (parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner);Add a line here. I can do it myself if you agree.
Sure, that is fine by me. I also spotted a typo in the commit log body (in taken -> is taken). Do you want me to send a v6, or do you prefer to fix that in place?
Quoted 3 lines
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
I'm ready for the patch to be merged. However, I recently sent an RFC to propose a safer implementation of try_module_get() that would simplify the code and may also benefit other subsystems. What do you think?
https://lore.kernel.org/linux-modules/[email protected]/
Quoted 2 lines
Next time if you think patches are ready for serious review and merge, drop the RFC prefix. That avoids an extra query.
Okay, I'll do it like that next time.
Thanks,
Marco
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 29 lines · Marco Pagani
Quoted 17 lines · Xu Yilun
Quoted 17 lines
include/linux/fpga/fpga-mgr.h +9
-3+#define fpga_mgr_register_full (parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner); +#define fpga_mgr_register (parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE)struct fpga_manager * -fpga_mgr_register (struct device *parent,const char *name, -const struct fpga_manager_ops *mops,void *priv); +__fpga_mgr_register (struct device *parent,const char *name, +const struct fpga_manager_ops *mops,void *priv,struct module *owner); +void fpga_mgr_unregister (struct fpga_manager *mgr);Quoted 8 lines
include/linux/fpga/fpga-mgr.h struct fpga_manager *fpga_mgr_get(struct device *dev); · line 255
Quoted 6 lines
include/linux/fpga/fpga-mgr.h +4
-1+#define devm_fpga_mgr_register_full (parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner);Add a line here. I can do it myself if you agree.
Quoted 3 lines
Sure, that is fine by me. I also spotted a typo in the commit log body (in taken -> is taken). Do you want me to send a v6, or do you prefer to fix that in place?
No need, I can fix it.
Quoted 11 lines
Quoted 4 lines
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
I'm ready for the patch to be merged. However, I recently sent an RFC to propose a safer implementation of
try_module_get()that would simplify the code and may also benefit other subsystems. What do you think?
I suggest take your fix to linux-fpga/for-next now. If your try_module_get() proposal is applied before the end of this cycle, we could re-evaluate this patch.
Thanks,
Yilun
Quoted 9 lines
Quoted 3 lines
Next time if you think patches are ready for serious review and merge, drop the RFC prefix. That avoids an extra query.
Okay, I'll do it like that next time.
Thanks,
Marco
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 46 lines · Xu Yilun
Quoted 29 lines · Marco Pagani
Quoted 17 lines · Xu Yilun
Quoted 17 lines
include/linux/fpga/fpga-mgr.h +9
-3+#define fpga_mgr_register_full (parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner); +#define fpga_mgr_register (parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE)struct fpga_manager * -fpga_mgr_register (struct device *parent,const char *name, -const struct fpga_manager_ops *mops,void *priv); +__fpga_mgr_register (struct device *parent,const char *name, +const struct fpga_manager_ops *mops,void *priv,struct module *owner); +void fpga_mgr_unregister (struct fpga_manager *mgr);Quoted 8 lines
include/linux/fpga/fpga-mgr.h struct fpga_manager *fpga_mgr_get(struct device *dev); · line 255
Quoted 6 lines
include/linux/fpga/fpga-mgr.h +4
-1+#define devm_fpga_mgr_register_full (parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner);Add a line here. I can do it myself if you agree.
Quoted 3 lines
Sure, that is fine by me. I also spotted a typo in the commit log body (in taken -> is taken). Do you want me to send a v6, or do you prefer to fix that in place?
No need, I can fix it.
Quoted 10 lines
Quoted 4 lines
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
I'm ready for the patch to be merged. However, I recently sent an RFC to propose a safer implementation of
try_module_get()that would simplify the code and may also benefit other subsystems. What do you think?
Quoted 5 lines
I suggest take your fix to linux-fpga/for-next now. If your
try_module_get()proposal is applied before the end of this cycle, we could re-evaluate this patch.
That's fine by me.
Thanks,
Marco
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 49 lines · Marco Pagani
Quoted 46 lines · Xu Yilun
Quoted 29 lines · Marco Pagani
Quoted 17 lines · Xu Yilun
Quoted 17 lines
include/linux/fpga/fpga-mgr.h +9
-3+#define fpga_mgr_register_full (parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner); +#define fpga_mgr_register (parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE)struct fpga_manager * -fpga_mgr_register (struct device *parent,const char *name, -const struct fpga_manager_ops *mops,void *priv); +__fpga_mgr_register (struct device *parent,const char *name, +const struct fpga_manager_ops *mops,void *priv,struct module *owner); +void fpga_mgr_unregister (struct fpga_manager *mgr);Quoted 8 lines
include/linux/fpga/fpga-mgr.h struct fpga_manager *fpga_mgr_get(struct device *dev); · line 255
Quoted 6 lines
include/linux/fpga/fpga-mgr.h +4
-1+#define devm_fpga_mgr_register_full (parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner);Add a line here. I can do it myself if you agree.
Quoted 3 lines
Sure, that is fine by me. I also spotted a typo in the commit log body (in taken -> is taken). Do you want me to send a v6, or do you prefer to fix that in place?
No need, I can fix it.
Quoted 10 lines
Quoted 4 lines
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
I'm ready for the patch to be merged. However, I recently sent an RFC to propose a safer implementation of
try_module_get()that would simplify the code and may also benefit other subsystems. What do you think?
Quoted 7 lines
Quoted 5 lines
I suggest take your fix to linux-fpga/for-next now. If your
try_module_get()proposal is applied before the end of this cycle, we could re-evaluate this patch.That's fine by me.
Sorry, I still found issues about this solution.
void fpga_mgr_unregister(struct fpga_manager *mgr)
{
dev_info(&mgr->dev, "%s %s\n", __func__, mgr->name);
/*
* If the low level driver provides a method for putting fpga into
* a desired state upon unregister, do it.
*/
fpga_mgr_fpga_remove(mgr);
mutex_lock(&mgr->mops_mutex);
mgr->mops = NULL;
mutex_unlock(&mgr->mops_mutex);
device_unregister(&mgr->dev);
}Note that fpga_mgr_unregister() doesn't have to be called in module_exit(). So if we do fpga_mgr_get() then fpga_mgr_unregister(), We finally had a fpga_manager dev without mops, this is not what the user want and cause problem when using this fpga_manager dev for other FPGA APIs.
I have this concern when I was reviewing the same improvement for fpga bridge. The change for fpga bridge seems workable, the mutex keeps hold until fpga_bridge_put(). But I somewhat don't prefer the unregistration been unnecessarily blocked for long term.
I think your try_module_get_safe() patch may finally solve the invalid module owner issue. Some options now, we ignore the invalid module owner issue (it exists before this change) and merge the rest of the improvements, or we wait for your patch accepted then re-evaluate. I prefer the former.
Thanks,
Yilun
Quoted 4 lines
Thanks,
Marco
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 79 lines · Xu Yilun
Quoted 49 lines · Marco Pagani
Quoted 46 lines · Xu Yilun
Quoted 29 lines · Marco Pagani
Quoted 17 lines · Xu Yilun
Quoted 17 lines
include/linux/fpga/fpga-mgr.h +9
-3+#define fpga_mgr_register_full (parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner); +#define fpga_mgr_register (parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE)struct fpga_manager * -fpga_mgr_register (struct device *parent,const char *name, -const struct fpga_manager_ops *mops,void *priv); +__fpga_mgr_register (struct device *parent,const char *name, +const struct fpga_manager_ops *mops,void *priv,struct module *owner); +void fpga_mgr_unregister (struct fpga_manager *mgr);Quoted 8 lines
include/linux/fpga/fpga-mgr.h struct fpga_manager *fpga_mgr_get(struct device *dev); · line 255
Quoted 6 lines
include/linux/fpga/fpga-mgr.h +4
-1+#define devm_fpga_mgr_register_full (parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE)struct fpga_manager * -devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full (struct device *parent,const struct fpga_manager_info *info, +struct module *owner);Add a line here. I can do it myself if you agree.
Quoted 3 lines
Sure, that is fine by me. I also spotted a typo in the commit log body (in taken -> is taken). Do you want me to send a v6, or do you prefer to fix that in place?
No need, I can fix it.
Quoted 10 lines
Quoted 4 lines
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
I'm ready for the patch to be merged. However, I recently sent an RFC to propose a safer implementation of
try_module_get()that would simplify the code and may also benefit other subsystems. What do you think?Quoted 7 lines
Quoted 5 lines
I suggest take your fix to linux-fpga/for-next now. If your
try_module_get()proposal is applied before the end of this cycle, we could re-evaluate this patch.That's fine by me.
Sorry, I still found issues about this solution.
void
fpga_mgr_unregister(struct fpga_manager *mgr){ dev_info(&mgr->dev, "%s %s\n", __func__, mgr->name); /* * If the low level driver provides a method for putting fpga into * a desired state upon unregister, do it. */ fpga_mgr_fpga_remove(mgr); mutex_lock(&mgr->mops_mutex); mgr->mops = NULL; mutex_unlock(&mgr->mops_mutex); device_unregister(&mgr->dev); }
Quoted 4 lines
Note that
fpga_mgr_unregister()doesn't have to be called inmodule_exit(). So if we dofpga_mgr_get()thenfpga_mgr_unregister(), We finally had afpga_managerdev without mops, this is not what the user want and cause problem when using thisfpga_managerdev for other FPGA APIs.
How about moving mgr->mops = NULL from fpga_mgr_unregister() to class->dev_release()? In that way, mops will be set to NULL only when the manager dev refcount reaches 0.
If fpga_mgr_unregister() is called from module_exit(), we are sure that nobody got the manager dev earlier using fpga_mgr_get(), or it would have bumped up the module's refcount, preventing its removal in the first place. In this case, when device_unregister() is called, it will trigger dev_release() since the manager dev refcount has reached 0.
If fpga_mgr_unregister() is called elsewhere in the module that registered the manager (1), we have two subcases:
a) someone got the manager dev earlier and bumped the module's refcount. Hence, the ops are safe since the module cannot be removed until the manager dev is released by calling (the last) put_device(). This, in turn, will trigger class->dev_release().
b) no one got manager dev. In this case, class->dev_release() will be called immediately.
(1) The caller of fpga_mgr_register_*() is responsible for calling fpga_mgr_unregister(), as specified in the docs.
Quoted 4 lines
I have this concern when I was reviewing the same improvement for fpga bridge. The change for fpga bridge seems workable, the mutex keeps hold until
fpga_bridge_put(). But I somewhat don't prefer the unregistration been unnecessarily blocked for long term.
I also don't like the idea of potentially blocking the unregistration, but I could not find a better solution for the bridge at the moment.
Quoted 5 lines
I think your
try_module_get_safe()patch may finally solve the invalid module owner issue. Some options now, we ignore the invalid module owner issue (it exists before this change) and merge the rest of the improvements, or we wait for your patch accepted then re-evaluate. I prefer the former.
Yeah, try_module_get_safe() would make things simpler and easier. I'm currently working on a series of selftests to demonstrate that the function is safe from deadlocks, as requested by the maintainer. I hope I can convince people of the advantages.
Thanks,
Marco
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 82 lines · Marco Pagani
Quoted 79 lines · Xu Yilun
Quoted 49 lines · Marco Pagani
Quoted 46 lines · Xu Yilun
Quoted 29 lines · Marco Pagani
Quoted 17 lines · Xu Yilun
Quoted 17 lines
(quoted diff) +9
-3+#define fpga_mgr_register_full(parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner); +#define fpga_mgr_register(parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE) struct fpga_manager * -fpga_mgr_register(struct device *parent, const char *name, - const struct fpga_manager_ops *mops, void *priv); +__fpga_mgr_register(struct device *parent, const char *name, + const struct fpga_manager_ops *mops, void *priv, struct module *owner); + void fpga_mgr_unregister(struct fpga_manager *mgr);Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +4
-1+#define devm_fpga_mgr_register_full(parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner);Add a line here. I can do it myself if you agree.
Quoted 3 lines
Sure, that is fine by me. I also spotted a typo in the commit log body (in taken -> is taken). Do you want me to send a v6, or do you prefer to fix that in place?
No need, I can fix it.
Quoted 10 lines
Quoted 4 lines
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
I'm ready for the patch to be merged. However, I recently sent an RFC to propose a safer implementation of
try_module_get()that would simplify the code and may also benefit other subsystems. What do you think?Quoted 7 lines
Quoted 5 lines
I suggest take your fix to linux-fpga/for-next now. If your
try_module_get()proposal is applied before the end of this cycle, we could re-evaluate this patch.That's fine by me.
Sorry, I still found issues about this solution.
void
fpga_mgr_unregister(struct fpga_manager *mgr){ dev_info(&mgr->dev, "%s %s\n", __func__, mgr->name); /* * If the low level driver provides a method for putting fpga into * a desired state upon unregister, do it. */ fpga_mgr_fpga_remove(mgr); mutex_lock(&mgr->mops_mutex); mgr->mops = NULL; mutex_unlock(&mgr->mops_mutex); device_unregister(&mgr->dev); }
Quoted 8 lines
Quoted 4 lines
Note that
fpga_mgr_unregister()doesn't have to be called inmodule_exit(). So if we dofpga_mgr_get()thenfpga_mgr_unregister(), We finally had afpga_managerdev without mops, this is not what the user want and cause problem when using thisfpga_managerdev for other FPGA APIs.How about moving
mgr->mops= NULL fromfpga_mgr_unregister()toclass->dev_release()? In that way, mops will be set to NULL only when the manager dev refcount reaches 0.
I'm afraid it doesn't help. The lifecycle of the module and the fpga mgr dev is different.
We use mops = NULL to indicate module has been freed or will be freed in no time. On the other hand mops != NULL means module is still there, so that try_module_get() could be safely called. It is possible someone has got fpga mgr dev but not the module yet, at that time the module is unloaded, then try_module_get() triggers crash.
Quoted 3 lines
If
fpga_mgr_unregister()is called frommodule_exit(), we are sure that nobody got the manager dev earlier usingfpga_mgr_get(), or it would have bumped up
No, someone may get the manager dev but not the module yet, and been scheduled out.
Quoted 40 lines
the module's refcount, preventing its removal in the first place. In this case, when
device_unregister()is called, it will triggerdev_release()since the manager dev refcount has reached 0.If
fpga_mgr_unregister()is called elsewhere in the module that registered the manager (1), we have two subcases:a) someone got the manager dev earlier and bumped the module's refcount. Hence, the ops are safe since the module cannot be removed until the manager dev is released by calling (the last)
put_device(). This, in turn, will triggerclass->dev_release().b) no one got manager dev. In this case,
class->dev_release()will be called immediately.(1) The caller of fpga_mgr_register_*() is responsible for calling
fpga_mgr_unregister(), as specified in the docs.Quoted 4 lines
I have this concern when I was reviewing the same improvement for fpga bridge. The change for fpga bridge seems workable, the mutex keeps hold until
fpga_bridge_put(). But I somewhat don't prefer the unregistration been unnecessarily blocked for long term.I also don't like the idea of potentially blocking the unregistration, but I could not find a better solution for the bridge at the moment.
Quoted 5 lines
I think your
try_module_get_safe()patch may finally solve the invalid module owner issue. Some options now, we ignore the invalid module owner issue (it exists before this change) and merge the rest of the improvements, or we wait for your patch accepted then re-evaluate. I prefer the former.Yeah,
try_module_get_safe()would make things simpler and easier. I'm currently working on a series of selftests to demonstrate that the function is safe from deadlocks, as requested by the maintainer. I hope I can convince people of the advantages.Thanks,
Marco
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 102 lines · Xu Yilun
Quoted 82 lines · Marco Pagani
Quoted 79 lines · Xu Yilun
Quoted 49 lines · Marco Pagani
Quoted 46 lines · Xu Yilun
Quoted 29 lines · Marco Pagani
Quoted 17 lines · Xu Yilun
Quoted 17 lines
(quoted diff) +9
-3+#define fpga_mgr_register_full(parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner); +#define fpga_mgr_register(parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE) struct fpga_manager * -fpga_mgr_register(struct device *parent, const char *name, - const struct fpga_manager_ops *mops, void *priv); +__fpga_mgr_register(struct device *parent, const char *name, + const struct fpga_manager_ops *mops, void *priv, struct module *owner); + void fpga_mgr_unregister(struct fpga_manager *mgr);Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +4
-1+#define devm_fpga_mgr_register_full(parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner);Add a line here. I can do it myself if you agree.
Quoted 3 lines
Sure, that is fine by me. I also spotted a typo in the commit log body (in taken -> is taken). Do you want me to send a v6, or do you prefer to fix that in place?
No need, I can fix it.
Quoted 10 lines
Quoted 4 lines
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
I'm ready for the patch to be merged. However, I recently sent an RFC to propose a safer implementation of
try_module_get()that would simplify the code and may also benefit other subsystems. What do you think?Quoted 7 lines
Quoted 5 lines
I suggest take your fix to linux-fpga/for-next now. If your
try_module_get()proposal is applied before the end of this cycle, we could re-evaluate this patch.That's fine by me.
Sorry, I still found issues about this solution.
void
fpga_mgr_unregister(struct fpga_manager *mgr){ dev_info(&mgr->dev, "%s %s\n", __func__, mgr->name); /* * If the low level driver provides a method for putting fpga into * a desired state upon unregister, do it. */ fpga_mgr_fpga_remove(mgr); mutex_lock(&mgr->mops_mutex); mgr->mops = NULL; mutex_unlock(&mgr->mops_mutex); device_unregister(&mgr->dev); }Quoted 8 lines
Quoted 4 lines
Note that
fpga_mgr_unregister()doesn't have to be called inmodule_exit(). So if we dofpga_mgr_get()thenfpga_mgr_unregister(), We finally had afpga_managerdev without mops, this is not what the user want and cause problem when using thisfpga_managerdev for other FPGA APIs.How about moving
mgr->mops= NULL fromfpga_mgr_unregister()toclass->dev_release()? In that way, mops will be set to NULL only when the manager dev refcount reaches 0.I'm afraid it doesn't help. The lifecycle of the module and the fpga mgr dev is different.
We use mops = NULL to indicate module has been freed or will be freed in no time. On the other hand mops != NULL means module is still there, so that
try_module_get()could be safely called. It is possible someone has got fpga mgr dev but not the module yet, at that time the module is unloaded, thentry_module_get()triggers crash.Quoted 1 line
Quoted 6 lines
Quoted 2 lines
If
fpga_mgr_unregister()is called frommodule_exit(), we are sure that nobody got the manager dev earlier usingfpga_mgr_get(), or it would have bumped upNo, someone may get the manager dev but not the module yet, and been scheduled out.
You are right. Overall, it's a bad idea. How about then using an additional bool flag instead of "overloading" the mops pointer? Something like:
get:
if (!mgr->owner_valid || !try_module_get(mgr->mops_owner))
remove:
mgr->owner_valid = false;
Another possibility that comes to my mind would be to "overload" the owner pointer itself by using the ERR_PTR/IS_ERR macros. However, it looks ugly to me.
Thanks,
Marco
[...]
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 112 lines · Marco Pagani
Quoted 102 lines · Xu Yilun
Quoted 82 lines · Marco Pagani
Quoted 79 lines · Xu Yilun
Quoted 49 lines · Marco Pagani
Quoted 46 lines · Xu Yilun
Quoted 29 lines · Marco Pagani
Quoted 17 lines · Xu Yilun
Quoted 17 lines
(quoted diff) +9
-3+#define fpga_mgr_register_full(parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner); +#define fpga_mgr_register(parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE) struct fpga_manager * -fpga_mgr_register(struct device *parent, const char *name, - const struct fpga_manager_ops *mops, void *priv); +__fpga_mgr_register(struct device *parent, const char *name, + const struct fpga_manager_ops *mops, void *priv, struct module *owner); + void fpga_mgr_unregister(struct fpga_manager *mgr);Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +4
-1+#define devm_fpga_mgr_register_full(parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner);Add a line here. I can do it myself if you agree.
Quoted 3 lines
Sure, that is fine by me. I also spotted a typo in the commit log body (in taken -> is taken). Do you want me to send a v6, or do you prefer to fix that in place?
No need, I can fix it.
Quoted 10 lines
Quoted 4 lines
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
I'm ready for the patch to be merged. However, I recently sent an RFC to propose a safer implementation of
try_module_get()that would simplify the code and may also benefit other subsystems. What do you think?Quoted 7 lines
Quoted 5 lines
I suggest take your fix to linux-fpga/for-next now. If your
try_module_get()proposal is applied before the end of this cycle, we could re-evaluate this patch.That's fine by me.
Sorry, I still found issues about this solution.
void
fpga_mgr_unregister(struct fpga_manager *mgr){ dev_info(&mgr->dev, "%s %s\n", __func__, mgr->name); /* * If the low level driver provides a method for putting fpga into * a desired state upon unregister, do it. */ fpga_mgr_fpga_remove(mgr); mutex_lock(&mgr->mops_mutex); mgr->mops = NULL; mutex_unlock(&mgr->mops_mutex); device_unregister(&mgr->dev); }Quoted 8 lines
Quoted 4 lines
Note that
fpga_mgr_unregister()doesn't have to be called inmodule_exit(). So if we dofpga_mgr_get()thenfpga_mgr_unregister(), We finally had afpga_managerdev without mops, this is not what the user want and cause problem when using thisfpga_managerdev for other FPGA APIs.How about moving
mgr->mops= NULL fromfpga_mgr_unregister()toclass->dev_release()? In that way, mops will be set to NULL only when the manager dev refcount reaches 0.I'm afraid it doesn't help. The lifecycle of the module and the fpga mgr dev is different.
We use mops = NULL to indicate module has been freed or will be freed in no time. On the other hand mops != NULL means module is still there, so that
try_module_get()could be safely called. It is possible someone has got fpga mgr dev but not the module yet, at that time the module is unloaded, thentry_module_get()triggers crash.Quoted 1 line
Quoted 6 lines
Quoted 2 lines
If
fpga_mgr_unregister()is called frommodule_exit(), we are sure that nobody got the manager dev earlier usingfpga_mgr_get(), or it would have bumped upNo, someone may get the manager dev but not the module yet, and been scheduled out.
Quoted 8 lines
You are right. Overall, it's a bad idea. How about then using an additional bool flag instead of "overloading" the mops pointer? Something like:
get:
if (!mgr->owner_valid || !try_module_get(mgr->mops_owner))remove:
mgr->owner_valid = false;
I'm not quite sure which function is actually mentioned by "remove". I assume it should be fpga_mgr_unregister(). IIUC this flag means no more reference to fpga mgr, but existing references are still valid.
It works for me. But the name of this flag could be reconsidered to avoid misunderstanding. The owner is still valid (we still need to put the owner) but allows no more reference. Maybe "owner_inactive"?
I still wanna this owner reference change been splitted, so that we could simply revert it when the try_module_get_safe() got accepted.
Thanks,
Yilun
Quoted 11 lines
Another possibility that comes to my mind would be to "overload" the owner pointer itself by using the ERR_PTR/IS_ERR macros. However, it looks ugly to me.
Thanks,
Marco[...]
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 116 lines · Xu Yilun
Quoted 115 lines · Marco Pagani
Quoted 102 lines · Xu Yilun
Quoted 82 lines · Marco Pagani
Quoted 79 lines · Xu Yilun
Quoted 49 lines · Marco Pagani
Quoted 46 lines · Xu Yilun
Quoted 29 lines · Marco Pagani
Quoted 17 lines · Xu Yilun
Quoted 17 lines
(quoted diff) +9
-3+#define fpga_mgr_register_full(parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner); +#define fpga_mgr_register(parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE) struct fpga_manager * -fpga_mgr_register(struct device *parent, const char *name, - const struct fpga_manager_ops *mops, void *priv); +__fpga_mgr_register(struct device *parent, const char *name, + const struct fpga_manager_ops *mops, void *priv, struct module *owner); + void fpga_mgr_unregister(struct fpga_manager *mgr);Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +4
-1+#define devm_fpga_mgr_register_full(parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner);Add a line here. I can do it myself if you agree.
Quoted 3 lines
Sure, that is fine by me. I also spotted a typo in the commit log body (in taken -> is taken). Do you want me to send a v6, or do you prefer to fix that in place?
No need, I can fix it.
Quoted 10 lines
Quoted 4 lines
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
I'm ready for the patch to be merged. However, I recently sent an RFC to propose a safer implementation of
try_module_get()that would simplify the code and may also benefit other subsystems. What do you think?Quoted 7 lines
Quoted 5 lines
I suggest take your fix to linux-fpga/for-next now. If your
try_module_get()proposal is applied before the end of this cycle, we could re-evaluate this patch.That's fine by me.
Sorry, I still found issues about this solution.
void
fpga_mgr_unregister(struct fpga_manager *mgr){ dev_info(&mgr->dev, "%s %s\n", __func__, mgr->name); /* * If the low level driver provides a method for putting fpga into * a desired state upon unregister, do it. */ fpga_mgr_fpga_remove(mgr); mutex_lock(&mgr->mops_mutex); mgr->mops = NULL; mutex_unlock(&mgr->mops_mutex); device_unregister(&mgr->dev); }Quoted 8 lines
Quoted 4 lines
Note that
fpga_mgr_unregister()doesn't have to be called inmodule_exit(). So if we dofpga_mgr_get()thenfpga_mgr_unregister(), We finally had afpga_managerdev without mops, this is not what the user want and cause problem when using thisfpga_managerdev for other FPGA APIs.How about moving
mgr->mops= NULL fromfpga_mgr_unregister()toclass->dev_release()? In that way, mops will be set to NULL only when the manager dev refcount reaches 0.I'm afraid it doesn't help. The lifecycle of the module and the fpga mgr dev is different.
We use mops = NULL to indicate module has been freed or will be freed in no time. On the other hand mops != NULL means module is still there, so that
try_module_get()could be safely called. It is possible someone has got fpga mgr dev but not the module yet, at that time the module is unloaded, thentry_module_get()triggers crash.Quoted 1 line
Quoted 6 lines
Quoted 2 lines
If
fpga_mgr_unregister()is called frommodule_exit(), we are sure that nobody got the manager dev earlier usingfpga_mgr_get(), or it would have bumped upNo, someone may get the manager dev but not the module yet, and been scheduled out.
You are right. Overall, it's a bad idea. How about then using an additional bool flag instead of "overloading" the mops pointer? Something like:
Quoted 8 lines
Quoted 5 lines
get:
if (!mgr->owner_valid || !try_module_get(mgr->mops_owner))remove:
mgr->owner_valid = false;I'm not quite sure which function is actually mentioned by "remove". I assume it should be
fpga_mgr_unregister().
Yes, I was referring to fpga_mgr_unregister().
Quoted 2 lines
IIUC this flag means no more reference to fpga mgr, but existing references are still valid.
Yes.
Quoted 4 lines
It works for me. But the name of this flag could be reconsidered to avoid misunderstanding. The owner is still valid (we still need to put the owner) but allows no more reference. Maybe "
owner_inactive"?
Right, owner_valid might be misleading. How about removing any reference to the owner module and name the flag unreg?
__fpga_mgr_get:
if (mgr->unreg || !try_module_get(mgr->mops_owner)) mgr = ERR_PTR(-ENODEV);
fpga_mgr_unregister:
mgr->unreg = true;
Quoted 2 lines
I still wanna this owner reference change been splitted, so that we could simply revert it when the
try_module_get_safe()got accepted.
I guess it may take some time to have try_module_get_safe() accepted. What do you prefer to do with the bridge and the region in the meantime?
Thanks,
Marco
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 135 lines · Marco Pagani
Quoted 116 lines · Xu Yilun
Quoted 115 lines · Marco Pagani
Quoted 102 lines · Xu Yilun
Quoted 82 lines · Marco Pagani
Quoted 79 lines · Xu Yilun
Quoted 49 lines · Marco Pagani
Quoted 46 lines · Xu Yilun
Quoted 29 lines · Marco Pagani
Quoted 17 lines · Xu Yilun
Quoted 17 lines
(quoted diff) +9
-3+#define fpga_mgr_register_full(parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner); +#define fpga_mgr_register(parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE) struct fpga_manager * -fpga_mgr_register(struct device *parent, const char *name, - const struct fpga_manager_ops *mops, void *priv); +__fpga_mgr_register(struct device *parent, const char *name, + const struct fpga_manager_ops *mops, void *priv, struct module *owner); + void fpga_mgr_unregister(struct fpga_manager *mgr);Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +4
-1+#define devm_fpga_mgr_register_full(parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner);Add a line here. I can do it myself if you agree.
Quoted 3 lines
Sure, that is fine by me. I also spotted a typo in the commit log body (in taken -> is taken). Do you want me to send a v6, or do you prefer to fix that in place?
No need, I can fix it.
Quoted 10 lines
Quoted 4 lines
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
I'm ready for the patch to be merged. However, I recently sent an RFC to propose a safer implementation of
try_module_get()that would simplify the code and may also benefit other subsystems. What do you think?Quoted 7 lines
Quoted 5 lines
I suggest take your fix to linux-fpga/for-next now. If your
try_module_get()proposal is applied before the end of this cycle, we could re-evaluate this patch.That's fine by me.
Sorry, I still found issues about this solution.
void
fpga_mgr_unregister(struct fpga_manager *mgr){ dev_info(&mgr->dev, "%s %s\n", __func__, mgr->name); /* * If the low level driver provides a method for putting fpga into * a desired state upon unregister, do it. */ fpga_mgr_fpga_remove(mgr); mutex_lock(&mgr->mops_mutex); mgr->mops = NULL; mutex_unlock(&mgr->mops_mutex); device_unregister(&mgr->dev); }Quoted 8 lines
Quoted 4 lines
Note that
fpga_mgr_unregister()doesn't have to be called inmodule_exit(). So if we dofpga_mgr_get()thenfpga_mgr_unregister(), We finally had afpga_managerdev without mops, this is not what the user want and cause problem when using thisfpga_managerdev for other FPGA APIs.How about moving
mgr->mops= NULL fromfpga_mgr_unregister()toclass->dev_release()? In that way, mops will be set to NULL only when the manager dev refcount reaches 0.I'm afraid it doesn't help. The lifecycle of the module and the fpga mgr dev is different.
We use mops = NULL to indicate module has been freed or will be freed in no time. On the other hand mops != NULL means module is still there, so that
try_module_get()could be safely called. It is possible someone has got fpga mgr dev but not the module yet, at that time the module is unloaded, thentry_module_get()triggers crash.Quoted 1 line
Quoted 6 lines
Quoted 2 lines
If
fpga_mgr_unregister()is called frommodule_exit(), we are sure that nobody got the manager dev earlier usingfpga_mgr_get(), or it would have bumped upNo, someone may get the manager dev but not the module yet, and been scheduled out.
You are right. Overall, it's a bad idea. How about then using an additional bool flag instead of "overloading" the mops pointer? Something like:
Quoted 8 lines
Quoted 5 lines
get:
if (!mgr->owner_valid || !try_module_get(mgr->mops_owner))remove:
mgr->owner_valid = false;I'm not quite sure which function is actually mentioned by "remove". I assume it should be
fpga_mgr_unregister().Yes, I was referring to
fpga_mgr_unregister().Quoted 2 lines
IIUC this flag means no more reference to fpga mgr, but existing references are still valid.
Yes.
Quoted 1 line
Quoted 6 lines
Quoted 3 lines
It works for me. But the name of this flag could be reconsidered to avoid misunderstanding. The owner is still valid (we still need to put the owner) but allows no more reference. Maybe "
owner_inactive"?Right,
owner_validmight be misleading. How about removing any reference to the owner module and name the flag unreg?
the full name "unregistered" is better.
Quoted 8 lines
__fpga_mgr_get:if (mgr->unreg || !try_module_get(mgr->mops_owner)) mgr = ERR_PTR(-ENODEV);
fpga_mgr_unregister:
mgr->unreg= true;
Quoted 6 lines
Quoted 2 lines
I still wanna this owner reference change been splitted, so that we could simply revert it when the
try_module_get_safe()got accepted.I guess it may take some time to have
try_module_get_safe()accepted. What do you prefer to do with the bridge and the region in the meantime?
This issue could happen in little chance. I actually don't have much preference, either way is good to me.
Thanks,
Yilun
Quoted 4 lines
Thanks,
Marco
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 136 lines · Xu Yilun
Quoted 135 lines · Marco Pagani
Quoted 116 lines · Xu Yilun
Quoted 115 lines · Marco Pagani
Quoted 102 lines · Xu Yilun
Quoted 82 lines · Marco Pagani
Quoted 79 lines · Xu Yilun
Quoted 49 lines · Marco Pagani
Quoted 46 lines · Xu Yilun
Quoted 29 lines · Marco Pagani
Quoted 17 lines · Xu Yilun
Quoted 17 lines
(quoted diff) +9
-3+#define fpga_mgr_register_full(parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner); +#define fpga_mgr_register(parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE) struct fpga_manager * -fpga_mgr_register(struct device *parent, const char *name, - const struct fpga_manager_ops *mops, void *priv); +__fpga_mgr_register(struct device *parent, const char *name, + const struct fpga_manager_ops *mops, void *priv, struct module *owner); + void fpga_mgr_unregister(struct fpga_manager *mgr);Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +4
-1+#define devm_fpga_mgr_register_full(parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner);Add a line here. I can do it myself if you agree.
Quoted 3 lines
Sure, that is fine by me. I also spotted a typo in the commit log body (in taken -> is taken). Do you want me to send a v6, or do you prefer to fix that in place?
No need, I can fix it.
Quoted 10 lines
Quoted 4 lines
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
I'm ready for the patch to be merged. However, I recently sent an RFC to propose a safer implementation of
try_module_get()that would simplify the code and may also benefit other subsystems. What do you think?Quoted 7 lines
Quoted 5 lines
I suggest take your fix to linux-fpga/for-next now. If your
try_module_get()proposal is applied before the end of this cycle, we could re-evaluate this patch.That's fine by me.
Sorry, I still found issues about this solution.
void
fpga_mgr_unregister(struct fpga_manager *mgr){ dev_info(&mgr->dev, "%s %s\n", __func__, mgr->name); /* * If the low level driver provides a method for putting fpga into * a desired state upon unregister, do it. */ fpga_mgr_fpga_remove(mgr); mutex_lock(&mgr->mops_mutex); mgr->mops = NULL; mutex_unlock(&mgr->mops_mutex); device_unregister(&mgr->dev); }Quoted 8 lines
Quoted 4 lines
Note that
fpga_mgr_unregister()doesn't have to be called inmodule_exit(). So if we dofpga_mgr_get()thenfpga_mgr_unregister(), We finally had afpga_managerdev without mops, this is not what the user want and cause problem when using thisfpga_managerdev for other FPGA APIs.How about moving
mgr->mops= NULL fromfpga_mgr_unregister()toclass->dev_release()? In that way, mops will be set to NULL only when the manager dev refcount reaches 0.I'm afraid it doesn't help. The lifecycle of the module and the fpga mgr dev is different.
We use mops = NULL to indicate module has been freed or will be freed in no time. On the other hand mops != NULL means module is still there, so that
try_module_get()could be safely called. It is possible someone has got fpga mgr dev but not the module yet, at that time the module is unloaded, thentry_module_get()triggers crash.Quoted 1 line
Quoted 6 lines
Quoted 2 lines
If
fpga_mgr_unregister()is called frommodule_exit(), we are sure that nobody got the manager dev earlier usingfpga_mgr_get(), or it would have bumped upNo, someone may get the manager dev but not the module yet, and been scheduled out.
You are right. Overall, it's a bad idea. How about then using an additional bool flag instead of "overloading" the mops pointer? Something like:
Quoted 8 lines
Quoted 5 lines
get:
if (!mgr->owner_valid || !try_module_get(mgr->mops_owner))remove:
mgr->owner_valid = false;I'm not quite sure which function is actually mentioned by "remove". I assume it should be
fpga_mgr_unregister().Yes, I was referring to
fpga_mgr_unregister().Quoted 2 lines
IIUC this flag means no more reference to fpga mgr, but existing references are still valid.
Yes.
Quoted 1 line
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
It works for me. But the name of this flag could be reconsidered to avoid misunderstanding. The owner is still valid (we still need to put the owner) but allows no more reference. Maybe "
owner_inactive"?Right,
owner_validmight be misleading. How about removing any reference to the owner module and name the flag unreg?the full name "unregistered" is better.
That's fine by me.
Quoted 12 lines
Quoted 12 lines
__fpga_mgr_get:if (mgr->unreg || !try_module_get(mgr->mops_owner)) mgr = ERR_PTR(-ENODEV);
fpga_mgr_unregister:
mgr->unreg= true;Quoted 2 lines
I still wanna this owner reference change been splitted, so that we could simply revert it when the
try_module_get_safe()got accepted.
Quoted 7 lines
Quoted 3 lines
I guess it may take some time to have
try_module_get_safe()accepted. What do you prefer to do with the bridge and the region in the meantime?This issue could happen in little chance. I actually don't have much preference, either way is good to me.
Okay, I'll also send the patch for the region then.
Thanks,
Marco
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
In reply to Xu YilunQuoted 126 lines · Xu Yilun
Quoted 112 lines · Marco Pagani
Quoted 102 lines · Xu Yilun
Quoted 82 lines · Marco Pagani
Quoted 79 lines · Xu Yilun
Quoted 49 lines · Marco Pagani
Quoted 46 lines · Xu Yilun
Quoted 29 lines · Marco Pagani
Quoted 17 lines · Xu Yilun
Quoted 17 lines
(quoted diff) +9
-3+#define fpga_mgr_register_full(parent, info) \ + __fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner); +#define fpga_mgr_register(parent, name, mops, priv) \ + __fpga_mgr_register(parent, name, mops, priv, THIS_MODULE) struct fpga_manager * -fpga_mgr_register(struct device *parent, const char *name, - const struct fpga_manager_ops *mops, void *priv); +__fpga_mgr_register(struct device *parent, const char *name, + const struct fpga_manager_ops *mops, void *priv, struct module *owner); + void fpga_mgr_unregister(struct fpga_manager *mgr);Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +4
-1+#define devm_fpga_mgr_register_full(parent, info) \ + __devm_fpga_mgr_register_full(parent, info, THIS_MODULE) struct fpga_manager * -devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info); +__devm_fpga_mgr_register_full(struct device *parent, const struct fpga_manager_info *info, + struct module *owner);Add a line here. I can do it myself if you agree.
Quoted 3 lines
Sure, that is fine by me. I also spotted a typo in the commit log body (in taken -> is taken). Do you want me to send a v6, or do you prefer to fix that in place?
No need, I can fix it.
Quoted 10 lines
Quoted 4 lines
There is still a RFC prefix for this patch. Are you ready to get it merged? If yes, Acked-by: Xu Yilun <[email protected]>
I'm ready for the patch to be merged. However, I recently sent an RFC to propose a safer implementation of
try_module_get()that would simplify the code and may also benefit other subsystems. What do you think?Quoted 7 lines
Quoted 5 lines
I suggest take your fix to linux-fpga/for-next now. If your
try_module_get()proposal is applied before the end of this cycle, we could re-evaluate this patch.That's fine by me.
Sorry, I still found issues about this solution.
void
fpga_mgr_unregister(struct fpga_manager *mgr){ dev_info(&mgr->dev, "%s %s\n", __func__, mgr->name); /* * If the low level driver provides a method for putting fpga into * a desired state upon unregister, do it. */ fpga_mgr_fpga_remove(mgr); mutex_lock(&mgr->mops_mutex); mgr->mops = NULL; mutex_unlock(&mgr->mops_mutex); device_unregister(&mgr->dev); }Quoted 8 lines
Quoted 4 lines
Note that
fpga_mgr_unregister()doesn't have to be called inmodule_exit(). So if we dofpga_mgr_get()thenfpga_mgr_unregister(), We finally had afpga_managerdev without mops, this is not what the user want and cause problem when using thisfpga_managerdev for other FPGA APIs.How about moving
mgr->mops= NULL fromfpga_mgr_unregister()toclass->dev_release()? In that way, mops will be set to NULL only when the manager dev refcount reaches 0.I'm afraid it doesn't help. The lifecycle of the module and the fpga mgr dev is different.
We use mops = NULL to indicate module has been freed or will be freed in no time. On the other hand mops != NULL means module is still there, so that
try_module_get()could be safely called. It is possible someone has got fpga mgr dev but not the module yet, at that time the module is unloaded, thentry_module_get()triggers crash.Quoted 1 line
Quoted 6 lines
Quoted 2 lines
If
fpga_mgr_unregister()is called frommodule_exit(), we are sure that nobody got the manager dev earlier usingfpga_mgr_get(), or it would have bumped upNo, someone may get the manager dev but not the module yet, and been scheduled out.
Quoted 8 lines
You are right. Overall, it's a bad idea. How about then using an additional bool flag instead of "overloading" the mops pointer? Something like:
get:
if (!mgr->owner_valid || !try_module_get(mgr->mops_owner))remove:
mgr->owner_valid = false;I'm not quite sure which function is actually mentioned by "remove". I assume it should be
fpga_mgr_unregister(). IIUC this flag means no more reference to fpga mgr, but existing references are still valid.
Quoted 7 lines
It works for me. But the name of this flag could be reconsidered to avoid misunderstanding. The owner is still valid (we still need to put the owner) but allows no more reference. Maybe "
owner_inactive"?I still wanna this owner reference change been splitted, so that we could simply revert it when the
try_module_get_safe()got accepted.
Just to be sure that I understood correctly, you want to split the changes into two patches, like:
a) add module owner to the manager struct and take it in __fpga_mgr_get(); move put_device() from __fpga_mgr_get() to fpga_mgr_get() and of_fpga_mgr_get().
b) add the mutex and the unregistered flag for protection against races.
So that (b) can be reverted if try_module_get_safe() will be accepted?
Quoted 1 line
[...]
Thanks,
Marco
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 10 lines
Just to be sure that I understood correctly, you want to split the changes into two patches, like:
a) add module owner to the manager struct and take it in
__fpga_mgr_get(); moveput_device()from__fpga_mgr_get()tofpga_mgr_get()andof_fpga_mgr_get().b) add the mutex and the unregistered flag for protection against races.
So that (b) can be reverted if
try_module_get_safe()will be accepted?
Yes, that's what I mean.
Quoted 7 lines
Quoted 2 lines
[...]
Thanks,
Marco
Re: [RFC PATCH v5 1/1] fpga: add an owner and use it to take the low-level module's refcount
Quoted 7 lines · Xu Yilun
Quoted 7 lines
Just to be sure that I understood correctly, you want to split the changes into two patches, like:
a) add module owner to the manager struct and take it in
__fpga_mgr_get(); moveput_device()from__fpga_mgr_get()tofpga_mgr_get()andof_fpga_mgr_get().
Quoted 6 lines
Quoted 3 lines
b) add the mutex and the unregistered flag for protection against races.
So that (b) can be reverted if
try_module_get_safe()will be accepted?Yes, that's what I mean.
On second thought, I would prefer to send only the patches (a) for the manager, bridge, and region for the moment. As you said, the chances of having a race are slim (no crash has ever been reported to date). So, after (a), I think it is worth focusing first on try_module_get_safe() since it could also be useful for other subsystems. The (b) patches could always be applied later if try_module_get_safe() is not accepted. Thanks,
Marco