amba: Remove deferred device addition
PATCH v5 1/1 amba: Remove deferred device addition
The uevents generated for an amba device need PID and CID information that's available only when the amba device is powered on, clocked and out of reset. So, if those resources aren't available, the information can't be read to generate the uevents. To workaround this requirement, if the resources weren't available, the device addition was deferred and retried periodically.
However, this deferred addition retry isn't based on resources becoming available. Instead, it's retried every 5 seconds and causes arbitrary probe delays for amba devices and their consumers.
Also, maintaining a separate deferred-probe like mechanism is maintenance headache.
With this commit, instead of deferring the device addition, we simply defer the generation of uevents for the device and probing of the device (because drivers needs PID and CID to match) until the PID and CID information can be read. This allows us to delete all the amba specific deferring code and also avoid the arbitrary probing delays.
--- drivers/amba/bus.c | 313 +++++++++++++++++++++------------------------ 1 file changed, 145 insertions(+), 168 deletions(-)
v1 -> v2:
- Dropped RFC tag
- Complete rewrite to not use stub devices.
v2 -> v3:
- Flipped the if() condition for hard-coded periphids.
- Added a stub driver to handle the case where all amba drivers are modules loaded by uevents.
- Cc Marek after I realized I forgot to add him.
v3 -> v4:
- Finally figured out and fixed the issue reported by Kefeng (bus match can't return an error other than -EPROBE_DEFER).
- I tested the patch on "V2P-CA15" on qemu
- Marek tested v3, but that was so long ago and the rebase wasn't clean, so I didn't include the tested-by.
v4 -> v5:
- Fixed up amba_match() to always return -EPROBE_DEFER on any failure. This fixes the issue Sudeep reported on Juno.
- Rebased the patch on top of Russell's for-next branch to avoid conflict with linux-next.
Sudeep,
Feel free to add your tested-by.
Russell,
Can you pick up this patch please?
-Saravana
drivers/amba/bus.c +145 -168
@@ -130,11 +130,100 @@ static struct attribute *amba_dev_attrs[] = { };ATTRIBUTE_GROUPS (amba_dev); +static int amba_read_periphid (struct amba_device *dev) +{ +struct reset_control *rstc; +u32 size, pid, cid; +void __iomem *tmp; +int i, ret; + + ret =dev_pm_domain_attach (&dev->dev, true); +if (ret) { +dev_dbg (&dev->dev,"can't get PM domain: %d\n" , ret); + goto err_out; + } + + ret =amba_get_enable_pclk (dev); +if (ret) { +dev_dbg (&dev->dev,"can't get pclk: %d\n" , ret); + goto err_pm; + } + +/* +* Find reset control(s) of the amba bus and de-assert them. +*/ + rstc =of_reset_control_array_get_optional_shared (dev->dev.of_node); +if (IS_ERR (rstc)) { + ret =PTR_ERR (rstc); +if (ret != -EPROBE_DEFER ) +dev_err (&dev->dev,"can't get reset: %d\n" , ret); + goto err_clk; + } +reset_control_deassert (rstc); +reset_control_put (rstc); + + size =resource_size (&dev->res); + tmp =ioremap (dev->res.start, size); +if (!tmp) { + ret = -ENOMEM ; + goto err_clk; + } + +/* +* Read pid and cid based on size of resource +* they are located at end of region +*/ +for (pid =0 , i =0 ; i <4 ; i++) + pid |= (readl (tmp + size -0x20 +4 * i) &255 ) << (i *8 ); +for (cid =0 , i =0 ; i <4 ; i++) + cid |= (readl (tmp + size -0x10 +4 * i) &255 ) << (i *8 ); + +if (cid ==CORESIGHT_CID ) { +/* set the base to the start of the last 4k block */ +void __iomem *csbase = tmp + size -4096 ; + + dev->uci.devarch =readl (csbase +UCI_REG_DEVARCH_OFFSET ); + dev->uci.devtype =readl (csbase +UCI_REG_DEVTYPE_OFFSET ) &0xff ; + } + +if (cid ==AMBA_CID || cid ==CORESIGHT_CID ) { + dev->periphid = pid; + dev->cid = cid; + } + +if (!dev->periphid) + ret = -ENODEV ; + +iounmap (tmp); + +err_clk: +amba_put_disable_pclk (dev); +err_pm: +dev_pm_domain_detach (&dev->dev, true); +err_out: +return ret; +} +static int amba_match (struct device *dev,struct device_driver *drv) {struct amba_device *pcdev =to_amba_device (dev);struct amba_driver *pcdrv =to_amba_driver (drv); +if (!pcdev->periphid) { +int ret =amba_read_periphid (pcdev); + +/* +* Returning any error other than -EPROBE_DEFER from bus match +* can cause driver registration failure. So, if there's a +* permanent failure in reading pid and cid, simply map it to +* -EPROBE_DEFER. +*/ +if (ret) +return -EPROBE_DEFER ; +dev_set_uevent_suppress (dev, false); +kobject_uevent (&dev->kobj,KOBJ_ADD ); + } +/* When driver_override is set, only bind to the matching driver */ if (pcdev->driver_override)return !strcmp (pcdev->driver_override, drv->name);
@@ -368,6 +457,42 @@ static int __init amba_init(void)postcore_initcall (amba_init); +static int amba_proxy_probe (struct amba_device *adev, +const struct amba_id *id) +{ +WARN (1 ,"Stub driver should never match any device.\n" ); +return -ENODEV ; +} + +static const struct amba_id amba_stub_drv_ids[] = { + {0 ,0 }, +}; + +static struct amba_driver amba_proxy_drv = { + .drv = { + .name ="amba-proxy" , + }, + .probe = amba_proxy_probe, + .id_table = amba_stub_drv_ids, +}; + +static int __init amba_stub_drv_init (void ) +{ +if (!IS_ENABLED (CONFIG_MODULES )) +return 0 ; + +/* +* The amba_match() function will get called only if there is at least +* one amba driver registered. If all amba drivers are modules and are +* only loaded based on uevents, then we'll hit a chicken-and-egg +* situation where amba_match() is waiting on drivers and drivers are +* waiting on amba_match(). So, register a stub driver to make sure +* amba_match() is called even if no amba driver has been registered. +*/ +return amba_driver_register (&amba_proxy_drv); +} +late_initcall_sync (amba_stub_drv_init); +/** * amba_driver_register - register an AMBA device driver * @drv: amba device driver structure
@@ -410,156 +535,6 @@ static void amba_device_release(struct device *dev)kfree (d); } -static int amba_read_periphid (struct amba_device *dev) -{ -struct reset_control *rstc; -u32 size, pid, cid; -void __iomem *tmp; -int i, ret; - - ret =dev_pm_domain_attach (&dev->dev, true); -if (ret) - goto err_out; - - ret =amba_get_enable_pclk (dev); -if (ret) - goto err_pm; - -/* -* Find reset control(s) of the amba bus and de-assert them. -*/ - rstc =of_reset_control_array_get_optional_shared (dev->dev.of_node); -if (IS_ERR (rstc)) { - ret =PTR_ERR (rstc); -if (ret != -EPROBE_DEFER ) -dev_err (&dev->dev,"can't get reset: %d\n" , ret); - goto err_clk; - } -reset_control_deassert (rstc); -reset_control_put (rstc); - - size =resource_size (&dev->res); - tmp =ioremap (dev->res.start, size); -if (!tmp) { - ret = -ENOMEM ; - goto err_clk; - } - -/* -* Read pid and cid based on size of resource -* they are located at end of region -*/ -for (pid =0 , i =0 ; i <4 ; i++) - pid |= (readl (tmp + size -0x20 +4 * i) &255 ) << (i *8 ); -for (cid =0 , i =0 ; i <4 ; i++) - cid |= (readl (tmp + size -0x10 +4 * i) &255 ) << (i *8 ); - -if (cid ==CORESIGHT_CID ) { -/* set the base to the start of the last 4k block */ -void __iomem *csbase = tmp + size -4096 ; - - dev->uci.devarch =readl (csbase +UCI_REG_DEVARCH_OFFSET ); - dev->uci.devtype =readl (csbase +UCI_REG_DEVTYPE_OFFSET ) &0xff ; - } - -if (cid ==AMBA_CID || cid ==CORESIGHT_CID ) { - dev->periphid = pid; - dev->cid = cid; - } - -if (!dev->periphid) - ret = -ENODEV ; - -iounmap (tmp); - -err_clk: -amba_put_disable_pclk (dev); -err_pm: -dev_pm_domain_detach (&dev->dev, true); -err_out: -return ret; -} - -static int amba_device_try_add (struct amba_device *dev,struct resource *parent) -{ -int ret; - - ret =request_resource (parent, &dev->res); -if (ret) - goto err_out; - -/* Hard-coded primecell ID instead of plug-n-play */ -if (dev->periphid !=0 ) - goto skip_probe; - - ret =amba_read_periphid (dev); -if (ret) - goto err_release; - -skip_probe: - ret =device_add (&dev->dev); -err_release: -if (ret) -release_resource (&dev->res); -err_out: -return ret; -} - -/* -* Registration of AMBA device require reading its pid and cid registers. -* To do this, the device must be turned on (if it is a part of power domain) -* and have clocks enabled. However in some cases those resources might not be -* yet available. Returning EPROBE_DEFER is not a solution in such case, -* because callers don't handle this special error code. Instead such devices -* are added to the special list and their registration is retried from -* periodic worker, until all resources are available and registration succeeds. -*/ -struct deferred_device { -struct amba_device *dev; -struct resource *parent; -struct list_head node; -}; - -static LIST_HEAD (deferred_devices); -static DEFINE_MUTEX (deferred_devices_lock); - -static void amba_deferred_retry_func (struct work_struct *dummy); -static DECLARE_DELAYED_WORK(deferred_retry_work, amba_deferred_retry_func); - -#define DEFERRED_DEVICE_TIMEOUT (msecs_to_jiffies(5 * 1000)) - -static int amba_deferred_retry (void ) -{ -struct deferred_device *ddev, *tmp; - -mutex_lock (&deferred_devices_lock); - -list_for_each_entry_safe (ddev, tmp, &deferred_devices, node) { -int ret =amba_device_try_add (ddev->dev, ddev->parent); - -if (ret == -EPROBE_DEFER ) -continue ; - -list_del_init (&ddev->node); -amba_device_put (ddev->dev); -kfree (ddev); - } - -mutex_unlock (&deferred_devices_lock); - -return 0 ; -} -late_initcall (amba_deferred_retry); - -static void amba_deferred_retry_func (struct work_struct *dummy) -{ -amba_deferred_retry (); - -if (!list_empty (&deferred_devices)) -schedule_delayed_work (&deferred_retry_work, -DEFERRED_DEVICE_TIMEOUT ); -} -/** * amba_device_add - add a previously allocated AMBA device structure * @dev: AMBA device allocated by amba_device_alloc
@@ -571,28 +546,30 @@ static void amba_deferred_retry_func(struct work_struct *dummy)*/ int amba_device_add (struct amba_device *dev,struct resource *parent) { -int ret =amba_device_try_add (dev, parent); - -if (ret == -EPROBE_DEFER ) { -struct deferred_device *ddev; - - ddev =kmalloc (sizeof (*ddev),GFP_KERNEL ); -if (!ddev) -return -ENOMEM ; +int ret; - ddev->dev = dev; - ddev->parent = parent; - ret =0 ; + ret =request_resource (parent, &dev->res); +if (ret) +return ret; -mutex_lock (&deferred_devices_lock); +/* If primecell ID isn't hard-coded, figure it out */ +if (!dev->periphid) { +/* +* AMBA device uevents require reading its pid and cid +* registers. To do this, the device must be on, clocked and +* out of reset. However in some cases those resources might +* not yet be available. If that's the case, we suppress the +* generation of uevents until we can read the pid and cid +* registers. See also amba_match(). +*/ +if (amba_read_periphid (dev)) +dev_set_uevent_suppress (&dev->dev, true); + } -if (list_empty (&deferred_devices)) -schedule_delayed_work (&deferred_retry_work, -DEFERRED_DEVICE_TIMEOUT ); -list_add_tail (&ddev->node, &deferred_devices); + ret =device_add (&dev->dev); +if (ret) +release_resource (&dev->res); -mutex_unlock (&deferred_devices_lock); - }return ret; }EXPORT_SYMBOL_GPL (amba_device_add);
Signature · 1 line
2.37.0.170.g444d1eabd0-goog
Quoted 20 lines · Saravana Kannan
The uevents generated for an amba device need PID and CID information that's available only when the amba device is powered on, clocked and out of reset. So, if those resources aren't available, the information can't be read to generate the uevents. To workaround this requirement, if the resources weren't available, the device addition was deferred and retried periodically.
However, this deferred addition retry isn't based on resources becoming available. Instead, it's retried every 5 seconds and causes arbitrary probe delays for amba devices and their consumers.
Also, maintaining a separate deferred-probe like mechanism is maintenance headache.
With this commit, instead of deferring the device addition, we simply defer the generation of uevents for the device and probing of the device (because drivers needs PID and CID to match) until the PID and CID information can be read. This allows us to delete all the amba specific deferring code and also avoid the arbitrary probing delays.
Quoted 5 lines
Cc Rob Herring <[email protected]>Cc Ulf Hansson <[email protected]>Cc Saravana Kannan <[email protected]>Cc Linus Walleij <[email protected]>Cc Sudeep Holla <[email protected]>
on Juno with linux-next(which had the reported issue [1]) + this patch(which fixes the issue)
--
Regards,
Sudeep
[1] https://lore.kernel.org/all/20220701150848.75eeprptmb5beip7@bogus/
Russell King (Oracle)
maintainerQuoted 27 lines · Sudeep Holla
Quoted 20 lines · Saravana Kannan
The uevents generated for an amba device need PID and CID information that's available only when the amba device is powered on, clocked and out of reset. So, if those resources aren't available, the information can't be read to generate the uevents. To workaround this requirement, if the resources weren't available, the device addition was deferred and retried periodically.
However, this deferred addition retry isn't based on resources becoming available. Instead, it's retried every 5 seconds and causes arbitrary probe delays for amba devices and their consumers.
Also, maintaining a separate deferred-probe like mechanism is maintenance headache.
With this commit, instead of deferring the device addition, we simply defer the generation of uevents for the device and probing of the device (because drivers needs PID and CID to match) until the PID and CID information can be read. This allows us to delete all the amba specific deferring code and also avoid the arbitrary probing delays.
Quoted 5 lines
Cc Rob Herring <[email protected]>Cc Ulf Hansson <[email protected]>Cc Saravana Kannan <[email protected]>Cc Linus Walleij <[email protected]>Cc Sudeep Holla <[email protected]>
Quoted 4 lines
Tested-by Sudeep Holla <[email protected]>on Juno with linux-next(which had the reported issue [1]) + this patch(which fixes the issue)
Ok, but this patch needs to end up in the patch system for me to apply it. Can someone please add "KernelVersion: 5.19-rc7" or whatever version the patch was generated against (just the tagged version is sufficient) somewhere in the email, and send it to [email protected].
Thanks.
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
Quoted 30 lines · Russell King (Oracle)
Quoted 29 lines · Sudeep Holla
Quoted 20 lines · Saravana Kannan
The uevents generated for an amba device need PID and CID information that's available only when the amba device is powered on, clocked and out of reset. So, if those resources aren't available, the information can't be read to generate the uevents. To workaround this requirement, if the resources weren't available, the device addition was deferred and retried periodically.
However, this deferred addition retry isn't based on resources becoming available. Instead, it's retried every 5 seconds and causes arbitrary probe delays for amba devices and their consumers.
Also, maintaining a separate deferred-probe like mechanism is maintenance headache.
With this commit, instead of deferring the device addition, we simply defer the generation of uevents for the device and probing of the device (because drivers needs PID and CID to match) until the PID and CID information can be read. This allows us to delete all the amba specific deferring code and also avoid the arbitrary probing delays.
Quoted 5 lines
Cc Rob Herring <[email protected]>Cc Ulf Hansson <[email protected]>Cc Saravana Kannan <[email protected]>Cc Linus Walleij <[email protected]>Cc Sudeep Holla <[email protected]>Tested-by Sudeep Holla <[email protected]>
Quoted 7 lines
Quoted 2 lines
on Juno with linux-next(which had the reported issue [1]) + this patch(which fixes the issue)
Ok, but this patch needs to end up in the patch system for me to apply it. Can someone please add "KernelVersion: 5.19-rc7" or whatever version the patch was generated against (just the tagged version is sufficient) somewhere in the email, and send it to [email protected].
If this is a part of the submission process for the subsystems/files you maintain, do you think it would be worth while adding this address to MAINTAINERS to make it easier for future contributors?
Quoted 29 lines · Russell King (Oracle)
Quoted 27 lines · Sudeep Holla
Quoted 25 lines · Saravana Kannan
The uevents generated for an amba device need PID and CID information that's available only when the amba device is powered on, clocked and out of reset. So, if those resources aren't available, the information can't be read to generate the uevents. To workaround this requirement, if the resources weren't available, the device addition was deferred and retried periodically.
However, this deferred addition retry isn't based on resources becoming available. Instead, it's retried every 5 seconds and causes arbitrary probe delays for amba devices and their consumers.
Also, maintaining a separate deferred-probe like mechanism is maintenance headache.
With this commit, instead of deferring the device addition, we simply defer the generation of uevents for the device and probing of the device (because drivers needs PID and CID to match) until the PID and CID information can be read. This allows us to delete all the amba specific deferring code and also avoid the arbitrary probing delays.
Cc Rob Herring <[email protected]>Cc Ulf Hansson <[email protected]>Cc Saravana Kannan <[email protected]>Cc Linus Walleij <[email protected]>Cc Sudeep Holla <[email protected]>
Quoted 7 lines
Quoted 4 lines
Tested-by Sudeep Holla <[email protected]>on Juno with linux-next(which had the reported issue [1]) + this patch(which fixes the issue)
Ok, but this patch needs to end up in the patch system for me to apply it. Can someone please add "KernelVersion: 5.19-rc7" or whatever version
Where am I supposed to add that? Just somewhere in the email body?
The patch you are replying to was based on your linu-arm/for-next the day I sent it. Do you still need me to rebase it on Linus's tree?
Quoted 2 lines
the patch was generated against (just the tagged version is sufficient) somewhere in the email, and send it to [email protected].
I'll send out the same patch as is to that email. Wait, is there a typo in the domain name? Did you leave out the x by accident or is it really armlinu? I'm also getting a DNS failure for either one of those domains.
I'll wait to hear from you before I send another email.
-Saravana
Quoted 6 lines
Thanks.
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
Quoted 47 lines · Saravana Kannan
Quoted 29 lines · Russell King (Oracle)
Quoted 27 lines · Sudeep Holla
Quoted 25 lines · Saravana Kannan
The uevents generated for an amba device need PID and CID information that's available only when the amba device is powered on, clocked and out of reset. So, if those resources aren't available, the information can't be read to generate the uevents. To workaround this requirement, if the resources weren't available, the device addition was deferred and retried periodically.
However, this deferred addition retry isn't based on resources becoming available. Instead, it's retried every 5 seconds and causes arbitrary probe delays for amba devices and their consumers.
Also, maintaining a separate deferred-probe like mechanism is maintenance headache.
With this commit, instead of deferring the device addition, we simply defer the generation of uevents for the device and probing of the device (because drivers needs PID and CID to match) until the PID and CID information can be read. This allows us to delete all the amba specific deferring code and also avoid the arbitrary probing delays.
Cc Rob Herring <[email protected]>Cc Ulf Hansson <[email protected]>Cc Saravana Kannan <[email protected]>Cc Linus Walleij <[email protected]>Cc Sudeep Holla <[email protected]>Quoted 7 lines
Quoted 4 lines
Tested-by Sudeep Holla <[email protected]>on Juno with linux-next(which had the reported issue [1]) + this patch(which fixes the issue)
Ok, but this patch needs to end up in the patch system for me to apply it. Can someone please add "KernelVersion: 5.19-rc7" or whatever version
Where am I supposed to add that? Just somewhere in the email body?
The patch you are replying to was based on your linu-arm/for-next the day I sent it. Do you still need me to rebase it on Linus's tree?
Quoted 2 lines
the patch was generated against (just the tagged version is sufficient) somewhere in the email, and send it to [email protected].
Quoted 6 lines
I'll send out the same patch as is to that email. Wait, is there a typo in the domain name? Did you leave out the x by accident or is it really armlinu? I'm also getting a DNS failure for either one of those domains.
I'll wait to hear from you before I send another email.
Odd, I don't see the requirement for the arm patch-bot for the amba bus code anywhere in the documentation, am I missing it?
Anyway, I can take this now if no one objects, through my driver-core tree so that it can get into 5.20-rc1.
thanks,
greg k-h
Russell King (Oracle)
maintainerQuoted 53 lines · Greg Kroah-Hartman
Quoted 52 lines · Saravana Kannan
Quoted 29 lines · Russell King (Oracle)
Quoted 27 lines · Sudeep Holla
Quoted 25 lines · Saravana Kannan
The uevents generated for an amba device need PID and CID information that's available only when the amba device is powered on, clocked and out of reset. So, if those resources aren't available, the information can't be read to generate the uevents. To workaround this requirement, if the resources weren't available, the device addition was deferred and retried periodically.
However, this deferred addition retry isn't based on resources becoming available. Instead, it's retried every 5 seconds and causes arbitrary probe delays for amba devices and their consumers.
Also, maintaining a separate deferred-probe like mechanism is maintenance headache.
With this commit, instead of deferring the device addition, we simply defer the generation of uevents for the device and probing of the device (because drivers needs PID and CID to match) until the PID and CID information can be read. This allows us to delete all the amba specific deferring code and also avoid the arbitrary probing delays.
Cc Rob Herring <[email protected]>Cc Ulf Hansson <[email protected]>Cc Saravana Kannan <[email protected]>Cc Linus Walleij <[email protected]>Cc Sudeep Holla <[email protected]>Quoted 7 lines
Quoted 4 lines
Tested-by Sudeep Holla <[email protected]>on Juno with linux-next(which had the reported issue [1]) + this patch(which fixes the issue)
Ok, but this patch needs to end up in the patch system for me to apply it. Can someone please add "KernelVersion: 5.19-rc7" or whatever version
Where am I supposed to add that? Just somewhere in the email body?
The patch you are replying to was based on your linu-arm/for-next the day I sent it. Do you still need me to rebase it on Linus's tree?
Quoted 2 lines
the patch was generated against (just the tagged version is sufficient) somewhere in the email, and send it to [email protected].
I'll send out the same patch as is to that email. Wait, is there a typo in the domain name? Did you leave out the x by accident or is it really armlinu? I'm also getting a DNS failure for either one of those domains.
Quoted 4 lines
Quoted 1 line
I'll wait to hear from you before I send another email.
Odd, I don't see the requirement for the arm patch-bot for the amba bus code anywhere in the documentation, am I missing it?
The requirement for the arm patch-bot is "patches that I apply", and since I'm listed as maintainer for that code...
Sorry I missed the original reply highlighting the typo. Yes, it's a typo (my X key was being a bit dodgy last week) and in any case, "armlinu.org.uk" doesn't exist in the DNS, so sending to it would fail. Also, every email I've sent over the last decade or more contains a signature that gives a URL to the patch system, and the email address is mentioned in its "Help" page (although there's other stuff on the help page that needs to be updated to current practices.)
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!