Add post-init-providers binding to improve suspend/resume stability
Subsystems (6)
v4 cover Add post-init-providers binding to improve suspend/resume stability
This patch series adds a "post-init-providers" device tree binding that can be used to break dependency cycles in device tree and enforce a more determinstic probe/suspend/resume order. This will also improve the stability of global async probing and async suspend/resume and allow us to enable them more easily. Yet another step away from playing initcall chicken with probing and step towards fully async probing and suspend/resume.
Patch 3 (the binding documentation) provides a lot more details and examples.
v3->v4:
- Fixed MAINTAINERS file to go with the file rename.
v2->v3:
- Changes doc/code from "post-init-supplier" to "post-init-providers"
- Fixed some wording that was ambiguous for Conor.
- Fixed indentation, additionalProperies and white space issues in the yaml syntax.
- Fixed syntax errors in the example.
v1->v2:
- Addressed Documentation/commit text errors pointed out by Rob
- Reordered MAINTAINERS chunk as pointed out by Krzysztof
Saravana Kannan (4):
driver core: Adds flags param to fwnode_link_add()
driver core: Add FWLINK_FLAG_IGNORE to completely ignore a fwnode link
dt-bindings: Add post-init-providers property
of: property: fw_devlink: Add support for "post-init-providers"
property
.../bindings/post-init-providers.yaml | 105 ++++++++++++++++++ MAINTAINERS | 13 ++- drivers/base/core.c | 14 ++- drivers/firmware/efi/sysfb_efi.c | 2 +- drivers/of/property.c | 17 ++- include/linux/fwnode.h | 5 +- 6 files changed, 142 insertions(+), 14 deletions(-) create mode 100644 Documentation/devicetree/bindings/post-init-providers.yaml
Signature · 1 line
2.44.0.rc0.258.g7320e95886-goog
PATCH v4 1/4 driver core: Adds flags param to fwnode_link_add()
Allow the callers to set fwnode link flags when adding fwnode links.
--- drivers/base/core.c | 5 +++-- drivers/firmware/efi/sysfb_efi.c | 2 +- drivers/of/property.c | 2 +- include/linux/fwnode.h | 3 ++- 4 files changed, 7 insertions(+), 5 deletions(-)
drivers/base/core.c +3 -2
@@ -92,12 +92,13 @@ static int __fwnode_link_add(struct fwnode_handle *con,return 0 ; } -int fwnode_link_add (struct fwnode_handle *con,struct fwnode_handle *sup) +int fwnode_link_add (struct fwnode_handle *con,struct fwnode_handle *sup, +u8 flags) {int ret;mutex_lock (&fwnode_link_lock); - ret =__fwnode_link_add (con, sup,0 ); + ret =__fwnode_link_add (con, sup, flags);mutex_unlock (&fwnode_link_lock);return ret; }
drivers/firmware/efi/sysfb_efi.c +1 -1
@@ -336,7 +336,7 @@ static int efifb_add_links(struct fwnode_handle *fwnode)if (!sup_np)return 0 ; -fwnode_link_add (fwnode,of_fwnode_handle (sup_np)); +fwnode_link_add (fwnode,of_fwnode_handle (sup_np),0 );of_node_put (sup_np);return 0 ;
drivers/of/property.c +1 -1
@@ -1085,7 +1085,7 @@ static void of_link_to_phandle(struct device_node *con_np, tmp_np =of_get_next_parent (tmp_np); } -fwnode_link_add (of_fwnode_handle (con_np),of_fwnode_handle (sup_np)); +fwnode_link_add (of_fwnode_handle (con_np),of_fwnode_handle (sup_np),0 ); }/**
include/linux/fwnode.h +2 -1
@@ -210,7 +210,8 @@ static inline void fwnode_dev_initialized(struct fwnode_handle *fwnode, }extern bool fw_devlink_is_strict (void ); -int fwnode_link_add (struct fwnode_handle *con,struct fwnode_handle *sup); +int fwnode_link_add (struct fwnode_handle *con,struct fwnode_handle *sup, +u8 flags);void fwnode_links_purge (struct fwnode_handle *fwnode);void fw_devlink_purge_absent_suppliers (struct fwnode_handle *fwnode);
Signature · 1 line
2.44.0.rc0.258.g7320e95886-goog
PATCH v4 2/4 driver core: Add FWLINK_FLAG_IGNORE to completely ignore a fwnode link
In reply to Saravana KannanA fwnode link between specific supplier-consumer fwnodes can be added multiple times for multiple reasons. If that dependency doesn't exist, deleting the fwnode link once doesn't guarantee that it won't get created again.
So, add FWLINK_FLAG_IGNORE flag to mark a fwnode link as one that needs to be completely ignored. Since a fwnode link's flags is an OR of all the flags passed to all the fwnode_link_add() calls to create that specific fwnode link, the FWLINK_FLAG_IGNORE flag is preserved and can be used to mark a fwnode link as on that need to be completely ignored until it is deleted.
--- drivers/base/core.c | 9 ++++++++- include/linux/fwnode.h | 2 ++ 2 files changed, 10 insertions(+), 1 deletion(-)
drivers/base/core.c +8 -1
@@ -1012,7 +1012,8 @@ static struct fwnode_handle *fwnode_links_check_suppliers(return NULL ;list_for_each_entry (link, &fwnode->suppliers, c_hook) -if (!(link->flags &FWLINK_FLAG_CYCLE )) +if (!(link->flags & + (FWLINK_FLAG_CYCLE |FWLINK_FLAG_IGNORE )))return link->supplier;return NULL ;
@@ -1963,6 +1964,9 @@ static bool __fw_devlink_relax_cycles(struct device *con, }list_for_each_entry (link, &sup_handle->suppliers, c_hook) { +if (link->flags &FWLINK_FLAG_IGNORE ) +continue ; +if (__fw_devlink_relax_cycles (con, link->supplier)) {__fwnode_link_cycle (link); ret = true;
@@ -2041,6 +2045,9 @@ static int fw_devlink_create_devlink(struct device *con,int ret =0 ;u32 flags; +if (link->flags &FWLINK_FLAG_IGNORE ) +return 0 ; +if (con->fwnode == link->consumer) flags =fw_devlink_get_flags (link->flags);else
include/linux/fwnode.h +2 -0
@@ -53,8 +53,10 @@ struct fwnode_handle {* fwnode link flags * * CYCLE: The fwnode link is part of a cycle. Don't defer probe. +* IGNORE: Completely ignore this link, even during cycle detection. */ #define FWLINK_FLAG_CYCLE BIT(0) +#define FWLINK_FLAG_IGNORE BIT(1)struct fwnode_link {struct fwnode_handle *supplier;
Signature · 1 line
2.44.0.rc0.258.g7320e95886-goog
PATCH v4 3/4 dt-bindings: Add post-init-providers property
In reply to Saravana KannanThe post-init-providers property can be used to break a dependency cycle by marking some provider(s) as a post device initialization provider(s). This allows an OS to do a better job at ordering initialization and suspend/resume of the devices in a dependency cycle.
--- .../bindings/post-init-providers.yaml | 105 ++++++++++++++++++ MAINTAINERS | 13 ++- 2 files changed, 112 insertions(+), 6 deletions(-) create mode 100644 Documentation/devicetree/bindings/post-init-providers.yaml
Documentation/devicetree/bindings/post-init-providers.yaml new file +105 -0
@@ -0,0 +1,105 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +# Copyright (c) 2020, Google LLC. All rights reserved. +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/post-init-providers.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Post device initialization providers + +maintainers: + - Saravana Kannan <[email protected]> + +description: | + This property is used to indicate that the device(s) pointed to by the + property are not needed for the initialization of the device that lists this + property. This property does not make a device (that's previously not a + provider) into a provider. It simply downgrades an existing provider to a + post device initialization provider. + + A device can list its providers in devicetree using one or more of the + standard devicetree bindings. By default, it is assumed that the provider + device can be initialized before the consumer device is initialized. + + However, that assumption cannot be made when there are cyclic dependencies + between devices. Since each device is a provider (directly or indirectly) of + the others in the cycle, there is no guaranteed safe order for initializing + the devices in a cycle. We can try to initialize them in an arbitrary order + and eventually successfully initialize all of them, but that doesn't always + work well. + + For example, say, + * The device tree has the following cyclic dependency X -> Y -> Z -> X (where + -> denotes "depends on"). + * But X is not needed to fully initialize Z (X might be needed only when a + specific functionality is requested post initialization). + + If all the other -> are mandatory initialization dependencies, then trying to + initialize the devices in a loop (or arbitrarily) will always eventually end + up with the devices being initialized in the order Z, Y and X. + + However, if Y is an optional provider for X (where X provides limited + functionality when Y is not initialized and providing its services), then + trying to initialize the devices in a loop (or arbitrarily) could end up with + the devices being initialized in the following order: + + * Z, Y and X - All devices provide full functionality + * Z, X and Y - X provides partial functionality + * X, Z and Y - X provides partial functionality + + However, we always want to initialize the devices in the order Z, Y and X + since that provides the full functionality without interruptions. + + One alternate option that might be suggested is to have the driver for X + notice that Y became available at a later point and adjust the functionality + it provides. However, other userspace applications could have started using X + with the limited functionality before Y was available and it might not be + possible to transparently transition X or the users of X to full + functionality while X is in use. + + Similarly, when it comes to suspend (resume) ordering, it's unclear which + device in a dependency cycle needs to be suspended/resumed first and trying + arbitrary orders can result in system crashes or instability. + + Explicitly calling out which link in a cycle needs to be broken when + determining the order, simplifies things a lot, improves efficiency, makes + the behavior more deterministic and maximizes the functionality that can be + provided without interruption. + + This property is used to provide this additional information between devices + in a cycle by telling which provider(s) is not needed for initializing the + device that lists this property. + + In the example above, Z would list X as a post-init-providers and the + initialization dependency would become X -> Y -> Z -/-> X. So the best order + to initialize them becomes clear: Z, Y and then X. + +select: true + +properties: + post-init-providers: + # One or more providers can be marked as post initialization provider + description: + List of phandles to providers that are not needed for initializing or + resuming this device. + $ref: /schemas/types.yaml#/definitions/phandle-array + items: + maxItems: 1 + +additionalProperties: true + +examples: + - | + gcc: clock-controller@1000 { + compatible = "vendor,soc4-gcc", "vendor,soc1-gcc"; + reg = <0x1000 0x80>; + clocks = <&dispcc 0x1>; + #clock-cells = <1>; + post-init-providers = <&dispcc>; + }; + dispcc: clock-controller@2000 { + compatible = "vendor,soc4-dispcc", "vendor,soc1-dispcc"; + reg = <0x2000 0x80>; + clocks = <&gcc 0xdd>; + #clock-cells = <1>; + };
MAINTAINERS +7 -6
@@ -6060,12 +6060,6 @@ S: Maintained F: drivers/base/devcoredump.c F: include/linux/devcoredump.h -DEVICE DEPENDENCY HELPER SCRIPT -M: Saravana Kannan <[email protected]> -L: [email protected] -S: Maintained -F: scripts/dev-needs.sh - DEVICE DIRECT ACCESS (DAX) M: Dan Williams <[email protected]> M: Vishal Verma <[email protected]>
@@ -8300,6 +8294,13 @@ F: include/linux/firewire.h F: include/uapi/linux/firewire*.h F: tools/firewire/ +FIRMWARE DEVICE LINK (fw_devlink) +M: Saravana Kannan <[email protected]> +L: [email protected] +S: Maintained +F: Documentation/devicetree/bindings/post-init-providers.yaml +F: scripts/dev-needs.sh + FIRMWARE FRAMEWORK FOR ARMV8-A M: Sudeep Holla <[email protected]> L: [email protected] (moderated for non-subscribers)
Signature · 1 line
2.44.0.rc0.258.g7320e95886-goog
Rob Herring
maintainerRe: [PATCH v4 3/4] dt-bindings: Add post-init-providers property
Quoted 5 lines · Saravana Kannan
The post-init-providers property can be used to break a dependency cycle by marking some provider(s) as a post device initialization provider(s). This allows an OS to do a better job at ordering initialization and suspend/resume of the devices in a dependency cycle.
Quoted 7 lines
Signed-off-by Saravana Kannan <[email protected]>--- .../bindings/post-init-providers.yaml | 105 ++++++++++++++++++ MAINTAINERS | 13 ++- 2 files changed, 112 insertions(+), 6 deletions(-) create mode 100644 Documentation/devicetree/bindings/post-init-providers.yaml
My bot found errors running 'make DT_CHECKER_FLAGS=-m dt_binding_check' on your patch (DT_CHECKER_FLAGS is new in v5.13):
yamllint warnings/errors:
dtschema/dtc warnings/errors:Documentation/devicetree/bindings/post-init-providers.example.dtb: /example-0/clock-controller@1000: failed to match any schema with compatible: ['vendor,soc4-gcc', 'vendor,soc1-gcc'] Documentation/devicetree/bindings/post-init-providers.example.dtb: /example-0/clock-controller@1000: failed to match any schema with compatible: ['vendor,soc4-gcc', 'vendor,soc1-gcc'] Documentation/devicetree/bindings/post-init-providers.example.dtb: /example-0/clock-controller@2000: failed to match any schema with compatible: ['vendor,soc4-dispcc', 'vendor,soc1-dispcc'] Documentation/devicetree/bindings/post-init-providers.example.dtb: /example-0/clock-controller@2000: failed to match any schema with compatible: ['vendor,soc4-dispcc', 'vendor,soc1-dispcc']
doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/project/devicetree-bindings/patch/[email protected]
The base for the series is generally the latest rc1. A different dependency should be noted in *this* patch.
If you already ran 'make dt_binding_check' and didn't see the above error(s), then make sure 'yamllint' is installed and dt-schema is up to date:
pip3 install dtschema --upgrade
Please check and re-submit after running the above command yourself. Note that DT_SCHEMA_FILES can be set to your schema file to speed up checking your schema. However, it must be unset to test all examples with your schema.
PATCH v4 4/4 of: property: fw_devlink: Add support for "post-init-providers" property
In reply to Saravana KannanAdd support for this property so that dependency cycles can be broken and fw_devlink can do better probe/suspend/resume ordering between devices in a dependency cycle.
--- drivers/of/property.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-)
drivers/of/property.c +14 -3
@@ -1066,7 +1066,8 @@ of_fwnode_device_get_match_data(const struct fwnode_handle *fwnode, }static void of_link_to_phandle (struct device_node *con_np, -struct device_node *sup_np) +struct device_node *sup_np, +u8 flags) {struct device_node *tmp_np =of_node_get (sup_np);
@@ -1085,7 +1086,8 @@ static void of_link_to_phandle(struct device_node *con_np, tmp_np =of_get_next_parent (tmp_np); } -fwnode_link_add (of_fwnode_handle (con_np),of_fwnode_handle (sup_np),0 ); +fwnode_link_add (of_fwnode_handle (con_np),of_fwnode_handle (sup_np), + flags); }/**
@@ -1198,6 +1200,8 @@ static struct device_node *parse_##fname(struct device_node *np, \* to a struct device, implement this ops so fw_devlink can use it * to find the true consumer. * @optional: Describes whether a supplier is mandatory or not +* @fwlink_flags: Optional fwnode link flags to use when creating a fwnode link +* for this property. * * Returns: * parse_prop() return values are
@@ -1210,6 +1214,7 @@ struct supplier_bindings { const char *prop_name, int index);struct device_node *(*get_con_dev)(struct device_node *np);bool optional; +u8 fwlink_flags; };DEFINE_SIMPLE_PROP (clocks,"clocks" ,"#clock-cells" )
@@ -1240,6 +1245,7 @@ DEFINE_SIMPLE_PROP(leds, "leds", NULL) DEFINE_SIMPLE_PROP(backlight, "backlight", NULL) DEFINE_SIMPLE_PROP(panel, "panel", NULL) DEFINE_SIMPLE_PROP(msi_parent, "msi-parent", "#msi-cells") +DEFINE_SIMPLE_PROP(post_init_providers, "post-init-providers", NULL) DEFINE_SUFFIX_PROP(regulators, "-supply", NULL)DEFINE_SUFFIX_PROP (gpio,"-gpio" ,"#gpio-cells" )
@@ -1349,6 +1355,10 @@ static const struct supplier_bindings of_supplier_bindings[] = { { .parse_prop = parse_regulators, }, { .parse_prop = parse_gpio, }, { .parse_prop = parse_gpios, }, + { + .parse_prop = parse_post_init_providers, + .fwlink_flags = FWLINK_FLAG_IGNORE, + }, {} };
@@ -1393,7 +1403,8 @@ static int of_link_property(struct device_node *con_np, const char *prop_name) : of_node_get(con_np); matched = true; i++; -of_link_to_phandle (con_dev_np, phandle); +of_link_to_phandle (con_dev_np, phandle, + s->fwlink_flags);of_node_put (phandle);of_node_put (con_dev_np); }
Signature · 1 line
2.44.0.rc0.258.g7320e95886-goog
Quoted 42 lines · Saravana Kannan
This patch series adds a "post-init-providers" device tree binding that can be used to break dependency cycles in device tree and enforce a more determinstic probe/suspend/resume order. This will also improve the stability of global async probing and async suspend/resume and allow us to enable them more easily. Yet another step away from playing initcall chicken with probing and step towards fully async probing and suspend/resume.
Patch 3 (the binding documentation) provides a lot more details and examples.
v3->v4:
- FixedMAINTAINERSfile to go with the file rename.
v2->v3:
- Changes doc/code from "post-init-supplier" to "post-init-providers"
- Fixed some wording that was ambiguous for Conor.
- Fixed indentation, additionalProperies and white space issues in the yaml syntax.
- Fixed syntax errors in the example.
v1->v2:
- AddressedDocumentation/committext errors pointed out by Rob
- ReorderedMAINTAINERSchunk as pointed out by KrzysztofSaravana Kannan (4):
driver core: Adds flags param tofwnode_link_add()
driver core: AddFWLINK_FLAG_IGNOREto completely ignore a fwnode link
dt-bindings: Add post-init-providers property
of: property:fw_devlink: Add support for "post-init-providers"
property.../bindings/post-init-providers.yaml | 105 ++++++++++++++++++ MAINTAINERS | 13 ++- drivers/base/core.c | 14 ++- drivers/firmware/efi/sysfb_efi.c | 2 +- drivers/of/property.c | 17 ++- include/linux/fwnode.h | 5 +- 6 files changed, 142 insertions(+), 14 deletions(-) create mode 100644 Documentation/devicetree/bindings/post-init-providers.yaml
Quoted 1 line
--
This looks reasonable to me overall, so
Quoted 25 lines · Saravana Kannan
This patch series adds a "post-init-providers" device tree binding that can be used to break dependency cycles in device tree and enforce a more determinstic probe/suspend/resume order. This will also improve the stability of global async probing and async suspend/resume and allow us to enable them more easily. Yet another step away from playing initcall chicken with probing and step towards fully async probing and suspend/resume.
Patch 3 (the binding documentation) provides a lot more details and examples.
v3->v4:
- FixedMAINTAINERSfile to go with the file rename.
v2->v3:
- Changes doc/code from "post-init-supplier" to "post-init-providers"
- Fixed some wording that was ambiguous for Conor.
- Fixed indentation, additionalProperies and white space issues in the yaml syntax.
- Fixed syntax errors in the example.
v1->v2:
- AddressedDocumentation/committext errors pointed out by Rob
- ReorderedMAINTAINERSchunk as pointed out by Krzysztof
Quoted 6 lines
Saravana Kannan (4):
driver core: Adds flags param tofwnode_link_add()
driver core: AddFWLINK_FLAG_IGNOREto completely ignore a fwnode link
dt-bindings: Add post-init-providers property
of: property:fw_devlink: Add support for "post-init-providers"
property
I'm assuming Greg takes this. You'll probably need to resend without the binding. Submit it to dtschema please.
Rob