Skip to contentSkip to conversation

clk: Fix race condition between clk_set_parent and clk_enable()

PATCH Superseded Saravana Kannan 4 participants 12 messages May 1, 2013 – May 16, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 clk: Fix race condition between clk_set_parent and clk_enable()

Without this patch, the following race conditions are possible.

Race condition 1:
* clk-A has two parents - clk-X and clk-Y.
* All three are disabled and clk-X is current parent.
* Thread A: clk_set_parent(clk-A, clk-Y).
* Thread A: <snip execution flow>
* Thread A: Grabs enable lock.
* Thread A: Sees enable count of clk-A is 0, so doesn't enable clk-Y.
* Thread A: Updates clk-A SW parent to clk-Y
* Thread A: Releases enable lock.
* Thread B: clk_enable(clk-A).
* Thread B: clk_enable() enables clk-Y, then enabled clk-A and returns.

clk-A is now enabled in software, but not clocking in hardware since the hardware parent is still clk-X.

The only way to avoid race conditions between clk_set_parent() and clk_enable/disable() is to ensure that clk_enable/disable() calls don't require changes to hardware enable state between changes to software clock topology and hardware clock topology.

There are options to achieve the above:
1. Grab the enable lock before changing software/hardware topology and release it afterwards.
2. Keep the clock enabled for the duration of software/hardware topology change so that any additional enable/disable calls don't try to change the hardware state. Once the topology change is complete, the clock can be put back in its original enable state.

Option (1) is not an acceptable solution since the set_parent() ops might need to sleep.

Therefore, this patch implements option (2).

This patch doesn't violate any API semantics. clk_disable() doesn't guarantee that the clock is actually disabled. So, no clients of a clock can assume that a clock is disabled after their last call to clk_disable(). So, enabling the clock during a parent change is not a violation of any API semantics.

This also has the nice side effect of simplifying the error handling code.

---
It's been a while since I submitted a patch. So, apologies if I'm cc'ing people who no longer care about the state of the common clock framework.

 drivers/clk/clk.c |   72 +++++++++++++++++++++++-----------------------------
 1 files changed, 32 insertions(+), 40 deletions(-)
drivers/clk/clk.c +32 -40
index 934cfd1..fe4055f 100644
@@ -1377,67 +1377,59 @@ static int __clk_set_parent(struct clk *clk, struct clk *parent, u8 p_index)
 	unsigned long flags;
 	int ret = 0;
 	struct clk *old_parent = clk->parent;
-	bool migrated_enable = false;
 
-	/* migrate prepare */
-	if (clk->prepare_count)
+	/*
+	 * Migrate prepare state between parents and prevent race with
+	 * clk_enable().
+	 *
+	 * If the clock is not prepared, then a race with
+	 * clk_enable/disable() is impossible since we already have the
+	 * prepare lock (future calls to clk_enable() need to be preceded by
+	 * a clk_prepare()).
+	 *
+	 * If the clock is prepared, migrate the prepared state to the new
+	 * parent and also protect against a race with clk_enable() by
+	 * forcing the clock and the new parent on.  This ensures that all
+	 * future calls to clk_enable() are practically NOPs with respect to
+	 * hardware and software states.
+	 */
+	if (clk->prepare_count) {
 		__clk_prepare(parent);
-
-	flags = clk_enable_lock();
-
-	/* migrate enable */
-	if (clk->enable_count) {
-		__clk_enable(parent);
-		migrated_enable = true;
+		clk_enable(parent);
+		clk_enable(clk);
 	}
 
 	/* update the clk tree topology */
+	flags = clk_enable_lock();
 	clk_reparent(clk, parent);
-
 	clk_enable_unlock(flags);
 
 	/* change clock input source */
 	if (parent && clk->ops->set_parent)
 		ret = clk->ops->set_parent(clk->hw, p_index);
-
 	if (ret) {
-		/*
-		 * The error handling is tricky due to that we need to release
-		 * the spinlock while issuing the .set_parent callback. This
-		 * means the new parent might have been enabled/disabled in
-		 * between, which must be considered when doing rollback.
-		 */
-		flags = clk_enable_lock();
 
+		flags = clk_enable_lock();
 		clk_reparent(clk, old_parent);
-
-		if (migrated_enable && clk->enable_count) {
-			__clk_disable(parent);
-		} else if (migrated_enable && (clk->enable_count == 0)) {
-			__clk_disable(old_parent);
-		} else if (!migrated_enable && clk->enable_count) {
-			__clk_disable(parent);
-			__clk_enable(old_parent);
-		}
-
 		clk_enable_unlock(flags);
 
-		if (clk->prepare_count)
+		if (clk->prepare_count) {
+			clk_disable(clk);
+			clk_disable(parent);
 			__clk_unprepare(parent);
-
+		}
 		return ret;
 	}
 
-	/* clean up enable for old parent if migration was done */
-	if (migrated_enable) {
-		flags = clk_enable_lock();
-		__clk_disable(old_parent);
-		clk_enable_unlock(flags);
-	}
-
-	/* clean up prepare for old parent if migration was done */
-	if (clk->prepare_count)
+	/*
+	 * Finish the migration of prepare state and undo the changes done
+	 * for preventing a race with clk_enable().
+	 */
+	if (clk->prepare_count) {
+		clk_disable(clk);
+		clk_disable(old_parent);
 		__clk_unprepare(old_parent);
+	}
 
 	/* update debugfs with new clk tree topology */
 	clk_debug_reparent(clk, parent);
Signature · 4 lines
1.7.8.3

The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation
#1 lore raw

Quoting Saravana Kannan (2013-04-30 21:42:08)

Quoted 41 lines

Without this patch, the following race conditions are possible.

Race condition 1:
* clk-A has two parents - clk-X and clk-Y.
* All three are disabled and clk-X is current parent.
* Thread A: clk_set_parent(clk-A, clk-Y).
* Thread A: <snip execution flow>
* Thread A: Grabs enable lock.
* Thread A: Sees enable count of clk-A is 0, so doesn't enable clk-Y.
* Thread A: Updates clk-A SW parent to clk-Y
* Thread A: Releases enable lock.
* Thread B: clk_enable(clk-A).
* Thread B: clk_enable() enables clk-Y, then enabled clk-A and returns.

clk-A is now enabled in software, but not clocking in hardware since the hardware parent is still clk-X.

The only way to avoid race conditions between clk_set_parent() and clk_enable/disable() is to ensure that clk_enable/disable() calls don't require changes to hardware enable state between changes to software clock topology and hardware clock topology.

There are options to achieve the above:
1. Grab the enable lock before changing software/hardware topology and release it afterwards.
2. Keep the clock enabled for the duration of software/hardware topology change so that any additional enable/disable calls don't try to change the hardware state. Once the topology change is complete, the clock can be put back in its original enable state.

Option (1) is not an acceptable solution since the set_parent() ops might need to sleep.

Therefore, this patch implements option (2).

This patch doesn't violate any API semantics. clk_disable() doesn't guarantee that the clock is actually disabled. So, no clients of a clock can assume that a clock is disabled after their last call to clk_disable(). So, enabling the clock during a parent change is not a violation of any API semantics.

Quoted 3 lines

This also has the nice side effect of simplifying the error handling code.

I've taken this patch into clk-next for testing. The code itself looks fine. The only thing that remains to be seen is if any platforms have a problem with disabled clocks getting turned on during a reparent operation.

On platforms that I have worked on this is OK, but I suppose there could be some platform out there where a clock is prepared and disabled, and briefly enabling the clock during the reparent operation somehow puts the hardware in a bad state.

Anyways that's a long shot and this look OK until somebody screams.

Regards,
Mike

Quoted 116 lines

---
It's been a while since I submitted a patch. So, apologies if I'm cc'ing people who no longer care about the state of the common clock framework.

 drivers/clk/clk.c |   72 +++++++++++++++++++++++-----------------------------
 1 files changed, 32 insertions(+), 40 deletions(-)
drivers/clk/clk.c +32 -40
index 934cfd1..fe4055f 100644
@@ -1377,67 +1377,59 @@ static int __clk_set_parent(struct clk *clk, struct clk *parent, u8 p_index)
        unsigned long flags;
        int ret = 0;
        struct clk *old_parent = clk->parent;
-       bool migrated_enable = false;
 
-       /* migrate prepare */
-       if (clk->prepare_count)
+       /*
+        * Migrate prepare state between parents and prevent race with
+        * clk_enable().
+        *
+        * If the clock is not prepared, then a race with
+        * clk_enable/disable() is impossible since we already have the
+        * prepare lock (future calls to clk_enable() need to be preceded by
+        * a clk_prepare()).
+        *
+        * If the clock is prepared, migrate the prepared state to the new
+        * parent and also protect against a race with clk_enable() by
+        * forcing the clock and the new parent on.  This ensures that all
+        * future calls to clk_enable() are practically NOPs with respect to
+        * hardware and software states.
+        */
+       if (clk->prepare_count) {
                __clk_prepare(parent);
-
-       flags = clk_enable_lock();
-
-       /* migrate enable */
-       if (clk->enable_count) {
-               __clk_enable(parent);
-               migrated_enable = true;
+               clk_enable(parent);
+               clk_enable(clk);
        }
 
        /* update the clk tree topology */
+       flags = clk_enable_lock();
        clk_reparent(clk, parent);
-
        clk_enable_unlock(flags);
 
        /* change clock input source */
        if (parent && clk->ops->set_parent)
                ret = clk->ops->set_parent(clk->hw, p_index);
-
        if (ret) {
-               /*
-                * The error handling is tricky due to that we need to release
-                * the spinlock while issuing the .set_parent callback. This
-                * means the new parent might have been enabled/disabled in
-                * between, which must be considered when doing rollback.
-                */
-               flags = clk_enable_lock();
 
+               flags = clk_enable_lock();
                clk_reparent(clk, old_parent);
-
-               if (migrated_enable && clk->enable_count) {
-                       __clk_disable(parent);
-               } else if (migrated_enable && (clk->enable_count == 0)) {
-                       __clk_disable(old_parent);
-               } else if (!migrated_enable && clk->enable_count) {
-                       __clk_disable(parent);
-                       __clk_enable(old_parent);
-               }
-
                clk_enable_unlock(flags);
 
-               if (clk->prepare_count)
+               if (clk->prepare_count) {
+                       clk_disable(clk);
+                       clk_disable(parent);
                        __clk_unprepare(parent);
-
+               }
                return ret;
        }
 
-       /* clean up enable for old parent if migration was done */
-       if (migrated_enable) {
-               flags = clk_enable_lock();
-               __clk_disable(old_parent);
-               clk_enable_unlock(flags);
-       }
-
-       /* clean up prepare for old parent if migration was done */
-       if (clk->prepare_count)
+       /*
+        * Finish the migration of prepare state and undo the changes done
+        * for preventing a race with clk_enable().
+        */
+       if (clk->prepare_count) {
+               clk_disable(clk);
+               clk_disable(old_parent);
                __clk_unprepare(old_parent);
+       }
 
        /* update debugfs with new clk tree topology */
        clk_debug_reparent(clk, parent);
Signature · 4 lines
1.7.8.3

The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation
#2 lore raw
Quoted 42 lines · Mike Turquette

Quoting Saravana Kannan (2013-04-30 21:42:08)

Quoted 41 lines

Without this patch, the following race conditions are possible.

Race condition 1:
* clk-A has two parents - clk-X and clk-Y.
* All three are disabled and clk-X is current parent.
* Thread A: clk_set_parent(clk-A, clk-Y).
* Thread A: <snip execution flow>
* Thread A: Grabs enable lock.
* Thread A: Sees enable count of clk-A is 0, so doesn't enable clk-Y.
* Thread A: Updates clk-A SW parent to clk-Y
* Thread A: Releases enable lock.
* Thread B: clk_enable(clk-A).
* Thread B: clk_enable() enables clk-Y, then enabled clk-A and returns.

clk-A is now enabled in software, but not clocking in hardware since the hardware parent is still clk-X.

The only way to avoid race conditions between clk_set_parent() and clk_enable/disable() is to ensure that clk_enable/disable() calls don't require changes to hardware enable state between changes to software clock topology and hardware clock topology.

There are options to achieve the above:
1. Grab the enable lock before changing software/hardware topology and release it afterwards.
2. Keep the clock enabled for the duration of software/hardware topology change so that any additional enable/disable calls don't try to change the hardware state. Once the topology change is complete, the clock can be put back in its original enable state.

Option (1) is not an acceptable solution since the set_parent() ops might need to sleep.

Therefore, this patch implements option (2).

This patch doesn't violate any API semantics. clk_disable() doesn't guarantee that the clock is actually disabled. So, no clients of a clock can assume that a clock is disabled after their last call to clk_disable(). So, enabling the clock during a parent change is not a violation of any API semantics.

Quoted 6 lines
Quoted 3 lines

This also has the nice side effect of simplifying the error handling code.

I've taken this patch into clk-next for testing. The code itself looks fine.

Thanks Mike. I'll send it out again with some typo/grammar corrections.

Quoted 3 lines

The only thing that remains to be seen is if any platforms have a problem with disabled clocks getting turned on during a reparent operation.

I would think that would be a general issue with the clock APIs since disable doesn't guarantee a disable (since it's ref counted).

Also, those clocks could be marked as CLK_SET_PARENT_GATE if it's a real issue.

Quoted 4 lines

On platforms that I have worked on this is OK, but I suppose there could be some platform out there where a clock is prepared and disabled, and briefly enabling the clock during the reparent operation somehow puts the hardware in a bad state.

I can't think of any either, but as I mentioned, we have CLK_SET_PARENT_GATE for that.

Thanks,
Saravana

Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation
#3 lore raw
In reply to Mike Turquette

Hi,

Quoted 53 lines · On Tuesday 14 of May 2013 11:54:17 Mike Turquette wrote:

Quoting Saravana Kannan (2013-04-30 21:42:08)

Quoted 51 lines

Without this patch, the following race conditions are possible.

Race condition 1:
* clk-A has two parents - clk-X and clk-Y.
* All three are disabled and clk-X is current parent.
* Thread A: clk_set_parent(clk-A, clk-Y).
* Thread A: <snip execution flow>
* Thread A: Grabs enable lock.
* Thread A: Sees enable count of clk-A is 0, so doesn't enable clk-Y.
* Thread A: Updates clk-A SW parent to clk-Y
* Thread A: Releases enable lock.
* Thread B: clk_enable(clk-A).
* Thread B: clk_enable() enables clk-Y, then enabled clk-A and returns.

clk-A is now enabled in software, but not clocking in hardware since the hardware parent is still clk-X.

The only way to avoid race conditions between clk_set_parent() and clk_enable/disable() is to ensure that clk_enable/disable() calls don't
require changes to hardware enable state between changes to software clock topology and hardware clock topology.

There are options to achieve the above:
1. Grab the enable lock before changing software/hardware topology and

release it afterwards.

2. Keep the clock enabled for the duration of software/hardware topology>
change so that any additional enable/disable calls don't try to change
the hardware state. Once the topology change is complete, the clock can
be put back in its original enable state.

Option (1) is not an acceptable solution since the set_parent() ops might need to sleep.

Therefore, this patch implements option (2).

This patch doesn't violate any API semantics. clk_disable() doesn't guarantee that the clock is actually disabled. So, no clients of a clock can assume that a clock is disabled after their last call to clk_disable(). So, enabling the clock during a parent change is not a violation of any API semantics.

This also has the nice side effect of simplifying the error handling code.

Quoted 6 lines
Quoted 1 line

I've taken this patch into clk-next for testing. The code itself looks fine. The only thing that remains to be seen is if any platforms have a problem with disabled clocks getting turned on during a reparent operation.

IMHO this behavior should be documented somewhere, with a note that the clock must not be prepared to keep it disabled during reparent operation and possibly also pointing to the CLK_SET_PARENT_GATE flag.

Quoted 4 lines

On platforms that I have worked on this is OK, but I suppose there could be some platform out there where a clock is prepared and disabled, and briefly enabling the clock during the reparent operation somehow puts the hardware in a bad state.

Well, on any platform where default clock settings are not completely correct this is likely to cause problems, because some device might get too high frequency for some period of time, which might crash it alone as well as the whole system.

Best regards,
Tomasz

Quoted 146 lines

Anyways that's a long shot and this look OK until somebody screams.

Regards,
Mike

Quoted 126 lines

---
It's been a while since I submitted a patch. So, apologies if I'm cc'ing people who no longer care about the state of the common clock framework.>

 drivers/clk/clk.c |   72

+++++++++++++++++++++++----------------------------- 1 files changed, 32 insertions(+), 40 deletions(-)

drivers/clk/clk.c +0 -0
index 934cfd1..fe4055f 100644
@@ -1377,67 +1377,59 @@ static int __clk_set_parent(struct clk *clk,

struct clk *parent, u8 p_index)>

        unsigned long flags;
        int ret = 0;
        struct clk *old_parent = clk->parent;
drivers/clk/clk.c +6 -3
-       bool migrated_enable = false;
 
-       /* migrate prepare */
-       if (clk->prepare_count)
+       /*
+        * Migrate prepare state between parents and prevent race with
+        * clk_enable().
+        *
+        * If the clock is not prepared, then a race with
+        * clk_enable/disable() is impossible since we already have

the
+ * prepare lock (future calls to clk_enable() need to be
preceded by + * a clk_prepare()).

drivers/clk/clk.c +2 -0
+        *
+        * If the clock is prepared, migrate the prepared state to the

new + * parent and also protect against a race with
clk_enable() by + * forcing the clock and the new parent on.
This ensures that all + * future calls to clk_enable() are
practically NOPs with respect to + * hardware and software
states.

drivers/clk/clk.c +5 -11
+        */
+       if (clk->prepare_count) {
 
                __clk_prepare(parent);
 
-
-       flags = clk_enable_lock();
-
-       /* migrate enable */
-       if (clk->enable_count) {
-               __clk_enable(parent);
-               migrated_enable = true;
+               clk_enable(parent);
+               clk_enable(clk);
 
        }
 
        /* update the clk tree topology */
 
+       flags = clk_enable_lock();
 
        clk_reparent(clk, parent);
 
-
 
        clk_enable_unlock(flags);
 
        /* change clock input source */
        if (parent && clk->ops->set_parent)
 
                ret = clk->ops->set_parent(clk->hw, p_index);
 
-
 
        if (ret) {
 
-               /*
-                * The error handling is tricky due to that we need to

release - * the spinlock while issuing the .set_parent
callback. This - * means the new parent might have
been enabled/disabled in - * between, which must be
considered when doing rollback. - */

drivers/clk/clk.c +1 -5
-               flags = clk_enable_lock();
 
+               flags = clk_enable_lock();
 
                clk_reparent(clk, old_parent);
 
-
-               if (migrated_enable && clk->enable_count) {
-                       __clk_disable(parent);
-               } else if (migrated_enable && (clk->enable_count ==

0)) { - __clk_disable(old_parent);

drivers/clk/clk.c +6 -16
-               } else if (!migrated_enable && clk->enable_count) {
-                       __clk_disable(parent);
-                       __clk_enable(old_parent);
-               }
-
 
                clk_enable_unlock(flags);
 
-               if (clk->prepare_count)
+               if (clk->prepare_count) {
+                       clk_disable(clk);
+                       clk_disable(parent);
 
                        __clk_unprepare(parent);
 
-
+               }
 
                return ret;
 
        }
 
-       /* clean up enable for old parent if migration was done */
-       if (migrated_enable) {
-               flags = clk_enable_lock();
-               __clk_disable(old_parent);
-               clk_enable_unlock(flags);
-       }
-
-       /* clean up prepare for old parent if migration was done */
-       if (clk->prepare_count)
+       /*
+        * Finish the migration of prepare state and undo the changes

done + * for preventing a race with clk_enable().

drivers/clk/clk.c +1 -0
+        */

drivers/clk/clk.c static int __clk_set_parent(struct clk *clk, struct clk *parent, u8 p_index) · line 1435

Quoted 10 lines
drivers/clk/clk.c +4 -0
+       if (clk->prepare_count) {
+               clk_disable(clk);
+               clk_disable(old_parent);
 
                __clk_unprepare(old_parent);
 
+       }
 
        /* update debugfs with new clk tree topology */
        clk_debug_reparent(clk, parent);

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#4 lore raw
Quoted 58 lines · Tomasz Figa

Hi,

Quoted 55 lines · On Tuesday 14 of May 2013 11:54:17 Mike Turquette wrote:

Quoting Saravana Kannan (2013-04-30 21:42:08)

Quoted 52 lines

Without this patch, the following race conditions are possible.

Race condition 1:
* clk-A has two parents - clk-X and clk-Y.
* All three are disabled and clk-X is current parent.
* Thread A: clk_set_parent(clk-A, clk-Y).
* Thread A: <snip execution flow>
* Thread A: Grabs enable lock.
* Thread A: Sees enable count of clk-A is 0, so doesn't enable clk-Y.
* Thread A: Updates clk-A SW parent to clk-Y
* Thread A: Releases enable lock.
* Thread B: clk_enable(clk-A).
* Thread B: clk_enable() enables clk-Y, then enabled clk-A and returns.

clk-A is now enabled in software, but not clocking in hardware since the hardware parent is still clk-X.

The only way to avoid race conditions between clk_set_parent() and clk_enable/disable() is to ensure that clk_enable/disable() calls don't
require changes to hardware enable state between changes to software clock topology and hardware clock topology.

There are options to achieve the above:
1. Grab the enable lock before changing software/hardware topology and

release it afterwards.

2. Keep the clock enabled for the duration of software/hardware topology>
change so that any additional enable/disable calls don't try to change
the hardware state. Once the topology change is complete, the clock can
be put back in its original enable state.

Option (1) is not an acceptable solution since the set_parent() ops might need to sleep.

Therefore, this patch implements option (2).

This patch doesn't violate any API semantics. clk_disable() doesn't guarantee that the clock is actually disabled. So, no clients of a clock can assume that a clock is disabled after their last call to clk_disable(). So, enabling the clock during a parent change is not a violation of any API semantics.

This also has the nice side effect of simplifying the error handling code.

Quoted 8 lines
Quoted 4 lines

I've taken this patch into clk-next for testing. The code itself looks fine. The only thing that remains to be seen is if any platforms have a problem with disabled clocks getting turned on during a reparent operation.

IMHO this behavior should be documented somewhere, with a note that the clock must not be prepared to keep it disabled during reparent operation and possibly also pointing to the CLK_SET_PARENT_GATE flag.

Reasonable request. I can update the documentation of clk_set_parent() to indicate that the clock might get turned on for the duration of the call and if they need a guarantee the GATE flag should be used.

Quoted 11 lines
Quoted 5 lines

On platforms that I have worked on this is OK, but I suppose there could be some platform out there where a clock is prepared and disabled, and briefly enabling the clock during the reparent operation somehow puts the hardware in a bad state.

Well, on any platform where default clock settings are not completely correct this is likely to cause problems, because some device might get too high frequency for some period of time, which might crash it alone as well as the whole system.

I don't think this is really a problem with this patch. It's present even without this patch.

The patch doesn't switch to some other unspecified parent. It only switches between the new/old parent. Even without this patch, if a clock is prepared while you reparent it, clk_enable() could be called at anytime between the parent switch and the future clock API calls to set up the new parent correctly. I think that's just crappy driver code to switch to a new parent before setting it up correctly. There's absolutely no good reason to do it that way.

-Saravana

Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation
#5 lore raw
Quoted 93 lines · On Tuesday 14 of May 2013 15:46:33 Saravana Kannan wrote:
Quoted 69 lines · Tomasz Figa

Hi,

Quoted 58 lines · On Tuesday 14 of May 2013 11:54:17 Mike Turquette wrote:

Quoting Saravana Kannan (2013-04-30 21:42:08)

Quoted 56 lines

Without this patch, the following race conditions are possible.

Race condition 1:
* clk-A has two parents - clk-X and clk-Y.
* All three are disabled and clk-X is current parent.
* Thread A: clk_set_parent(clk-A, clk-Y).
* Thread A: <snip execution flow>
* Thread A: Grabs enable lock.
* Thread A: Sees enable count of clk-A is 0, so doesn't enable clk-Y.
* Thread A: Updates clk-A SW parent to clk-Y
* Thread A: Releases enable lock.
* Thread B: clk_enable(clk-A).
* Thread B: clk_enable() enables clk-Y, then enabled clk-A and returns.

clk-A is now enabled in software, but not clocking in hardware since the hardware parent is still clk-X.

The only way to avoid race conditions between clk_set_parent() and clk_enable/disable() is to ensure that clk_enable/disable() calls don't
require changes to hardware enable state between changes to software clock topology and hardware clock topology.

There are options to achieve the above:
1. Grab the enable lock before changing software/hardware topology and

release it afterwards.

2. Keep the clock enabled for the duration of software/hardware topology>

change so that any additional enable/disable calls don't try to change
the hardware state. Once the topology change is complete, the clock
can
be put back in its original enable state.

Option (1) is not an acceptable solution since the set_parent() ops might need to sleep.

Therefore, this patch implements option (2).

This patch doesn't violate any API semantics. clk_disable() doesn't guarantee that the clock is actually disabled. So, no clients of a clock can assume that a clock is disabled after their last call to clk_disable(). So, enabling the clock during a parent change is not a
violation of any API semantics.

This also has the nice side effect of simplifying the error handling code.

Quoted 7 lines
Quoted 1 line

I've taken this patch into clk-next for testing. The code itself looks
fine. The only thing that remains to be seen is if any platforms have a problem with disabled clocks getting turned on during a reparent operation.

Quoted 4 lines

IMHO this behavior should be documented somewhere, with a note that the
clock must not be prepared to keep it disabled during reparent operation and possibly also pointing to the CLK_SET_PARENT_GATE flag.

Reasonable request. I can update the documentation of clk_set_parent() to indicate that the clock might get turned on for the duration of the call and if they need a guarantee the GATE flag should be used.

Quoted 10 lines
Quoted 4 lines

On platforms that I have worked on this is OK, but I suppose there could be some platform out there where a clock is prepared and disabled, and briefly enabling the clock during the reparent operation somehow puts the hardware in a bad state.

Well, on any platform where default clock settings are not completely correct this is likely to cause problems, because some device might get
too high frequency for some period of time, which might crash it alone as well as the whole system.

I don't think this is really a problem with this patch. It's present even without this patch.

Quoted 7 lines

The patch doesn't switch to some other unspecified parent. It only switches between the new/old parent. Even without this patch, if a clock is prepared while you reparent it, clk_enable() could be called at anytime between the parent switch and the future clock API calls to set up the new parent correctly. I think that's just crappy driver code to switch to a new parent before setting it up correctly. There's absolutely no good reason to do it that way.

This is not exactly what I meant. I was just giving an example problem of turning a clock on, if it's not set up correctly yet.

AFAIK most (if not all) of current code either does necessary reparenting and initial rate setting early, before clk_prepare(), so it is not a problem or already after clk_enable() (in case of reparenting dynamically at runtime), so there shouldn't be any problem.

Best regards,
Tomasz

Acked-by on v1 1/1
#6 lore raw
In reply to Saravana Kannan
Quoted 68 lines · Saravana Kannan

Without this patch, the following race conditions are possible.

Race condition 1:
* clk-A has two parents - clk-X and clk-Y.
* All three are disabled and clk-X is current parent.
* Thread A: clk_set_parent(clk-A, clk-Y).
* Thread A: <snip execution flow>
* Thread A: Grabs enable lock.
* Thread A: Sees enable count of clk-A is 0, so doesn't enable clk-Y.
* Thread A: Updates clk-A SW parent to clk-Y
* Thread A: Releases enable lock.
* Thread B: clk_enable(clk-A).
* Thread B: clk_enable() enables clk-Y, then enabled clk-A and returns.

clk-A is now enabled in software, but not clocking in hardware since the hardware parent is still clk-X.

The only way to avoid race conditions between clk_set_parent() and clk_enable/disable() is to ensure that clk_enable/disable() calls don't require changes to hardware enable state between changes to software clock topology and hardware clock topology.

There are options to achieve the above:
1. Grab the enable lock before changing software/hardware topology and release it afterwards.
2. Keep the clock enabled for the duration of software/hardware topology change so that any additional enable/disable calls don't try to change the hardware state. Once the topology change is complete, the clock can be put back in its original enable state.

Option (1) is not an acceptable solution since the set_parent() ops might need to sleep.

Therefore, this patch implements option (2).

This patch doesn't violate any API semantics. clk_disable() doesn't guarantee that the clock is actually disabled. So, no clients of a clock can assume that a clock is disabled after their last call to clk_disable(). So, enabling the clock during a parent change is not a violation of any API semantics.

This also has the nice side effect of simplifying the error handling code.

---
It's been a while since I submitted a patch. So, apologies if I'm cc'ing people who no longer care about the state of the common clock framework.

 drivers/clk/clk.c |   72 +++++++++++++++++++++++-----------------------------
 1 files changed, 32 insertions(+), 40 deletions(-)
drivers/clk/clk.c +5 -3
index 934cfd1..fe4055f 100644
@@ -1377,67 +1377,59 @@ static int __clk_set_parent(struct clk *clk, struct clk *parent, u8 p_index)
        unsigned long flags;
        int ret = 0;
        struct clk *old_parent = clk->parent;
-       bool migrated_enable = false;
 
-       /* migrate prepare */
-       if (clk->prepare_count)
+       /*
+        * Migrate prepare state between parents and prevent race with
+        * clk_enable().
+        *
+        * If the clock is not prepared, then a race with

drivers/clk/clk.c static int __clk_set_parent(struct clk *clk, struct clk *parent, u8 p_index) · line 1395

Quoted 10 lines
drivers/clk/clk.c +10 -0
+        * clk_enable/disable() is impossible since we already have the
+        * prepare lock (future calls to clk_enable() need to be preceded by
+        * a clk_prepare()).
+        *
+        * If the clock is prepared, migrate the prepared state to the new
+        * parent and also protect against a race with clk_enable() by
+        * forcing the clock and the new parent on.  This ensures that all
+        * future calls to clk_enable() are practically NOPs with respect to
+        * hardware and software states.
+        */

Maybe an additional note about that since CLK_SET_PARENT_GATE is a prerequisite for doing migration of "prepare", we also interpreted this flags as it is acceptable to enable the clock(s) in this context.

Quoted 34 lines
drivers/clk/clk.c +5 -16
+       if (clk->prepare_count) {
                __clk_prepare(parent);
-
-       flags = clk_enable_lock();
-
-       /* migrate enable */
-       if (clk->enable_count) {
-               __clk_enable(parent);
-               migrated_enable = true;
+               clk_enable(parent);
+               clk_enable(clk);
        }
 
        /* update the clk tree topology */
+       flags = clk_enable_lock();
        clk_reparent(clk, parent);
-
        clk_enable_unlock(flags);
 
        /* change clock input source */
        if (parent && clk->ops->set_parent)
                ret = clk->ops->set_parent(clk->hw, p_index);
-
        if (ret) {
-               /*
-                * The error handling is tricky due to that we need to release
-                * the spinlock while issuing the .set_parent callback. This
-                * means the new parent might have been enabled/disabled in
-                * between, which must be considered when doing rollback.
-                */
-               flags = clk_enable_lock();
 
+               flags = clk_enable_lock();
                clk_reparent(clk, old_parent);

drivers/clk/clk.c static int __clk_set_parent(struct clk *clk, struct clk *parent, u8 p_index) · line 1422

Quoted 10 lines
drivers/clk/clk.c +0 -10
-
-               if (migrated_enable && clk->enable_count) {
-                       __clk_disable(parent);
-               } else if (migrated_enable && (clk->enable_count == 0)) {
-                       __clk_disable(old_parent);
-               } else if (!migrated_enable && clk->enable_count) {
-                       __clk_disable(parent);
-                       __clk_enable(old_parent);
-               }
-

Really good, that we can remove this awkward error handling!

Quoted 36 lines
drivers/clk/clk.c +12 -12
                clk_enable_unlock(flags);
 
-               if (clk->prepare_count)
+               if (clk->prepare_count) {
+                       clk_disable(clk);
+                       clk_disable(parent);
                        __clk_unprepare(parent);
-
+               }
                return ret;
        }
 
-       /* clean up enable for old parent if migration was done */
-       if (migrated_enable) {
-               flags = clk_enable_lock();
-               __clk_disable(old_parent);
-               clk_enable_unlock(flags);
-       }
-
-       /* clean up prepare for old parent if migration was done */
-       if (clk->prepare_count)
+       /*
+        * Finish the migration of prepare state and undo the changes done
+        * for preventing a race with clk_enable().
+        */
+       if (clk->prepare_count) {
+               clk_disable(clk);
+               clk_disable(old_parent);
                __clk_unprepare(old_parent);
+       }
 
        /* update debugfs with new clk tree topology */
        clk_debug_reparent(clk, parent);
--

1.7.8.3

Quoted 2 lines

The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation

Looks good! Thanks for having another round to fixup this kind of tricky code. :-)

#7 lore raw
Quoted 1 line · Ulf Hansson

On 1 May 2013 06:42, Saravana Kannan <[email protected]> wrote:

<snip>

Quoted 18 lines
Quoted 7 lines
drivers/clk/clk.c +5 -2
-       /* migrate prepare */
-       if (clk->prepare_count)
+       /*
+        * Migrate prepare state between parents and prevent race with
+        * clk_enable().
+        *
+        * If the clock is not prepared, then a race with

drivers/clk/clk.c static int __clk_set_parent(struct clk *clk, struct clk *parent, u8 p_index) · line 1395

Quoted 10 lines
drivers/clk/clk.c +10 -0
+        * clk_enable/disable() is impossible since we already have the
+        * prepare lock (future calls to clk_enable() need to be preceded by
+        * a clk_prepare()).
+        *
+        * If the clock is prepared, migrate the prepared state to the new
+        * parent and also protect against a race with clk_enable() by
+        * forcing the clock and the new parent on.  This ensures that all
+        * future calls to clk_enable() are practically NOPs with respect to
+        * hardware and software states.
+        */
Quoted 3 lines

Maybe an additional note about that since CLK_SET_PARENT_GATE is a prerequisite for doing migration of "prepare", we also interpreted this flags as it is acceptable to enable the clock(s) in this context.

Done. Sent v2 patch.

<snip>

Quoted 3 lines

Looks good! Thanks for having another round to fixup this kind of tricky code. :-)

Thanks :-)

Quoted 2 lines

Thanks again.

-Saravana

Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation
#8 lore raw

PATCH v2 1/1 clk: Fix race condition between clk_set_parent and clk_enable()

In reply to Saravana Kannan

Without this patch, the following race condition is possible.
* clk-A has two parents - clk-X and clk-Y.
* All three are disabled and clk-X is current parent.
* Thread A: clk_set_parent(clk-A, clk-Y).
* Thread A: <snip execution flow>
* Thread A: Grabs enable lock.
* Thread A: Sees enable count of clk-A is 0, so doesn't enable clk-Y.
* Thread A: Updates clk-A SW parent to clk-Y
* Thread A: Releases enable lock.
* Thread B: clk_enable(clk-A).
* Thread B: clk_enable() enables clk-Y, then enabled clk-A and returns.

clk-A is now enabled in software, but not clocking in hardware since the hardware parent is still clk-X.

The only way to avoid race conditions between clk_set_parent() and clk_enable/disable() is to ensure that clk_enable/disable() calls don't require changes to hardware enable state between changes to software clock topology and hardware clock topology.

The options to achieve the above are:
1. Grab the enable lock before changing software/hardware topology and release it afterwards.
2. Keep the clock enabled for the duration of software/hardware topology change so that any additional enable/disable calls don't try to change the hardware state. Once the topology change is complete, the clock can be put back in its original enable state.

Option (1) is not an acceptable solution since the set_parent() ops might need to sleep.

Therefore, this patch implements option (2).

This patch doesn't violate any API semantics. clk_disable() doesn't guarantee that the clock is actually disabled. So, no clients of a clock can assume that a clock is disabled after their last call to clk_disable(). So, enabling the clock during a parent change is not a violation of any API semantics.

This also has the nice side effect of simplifying the error handling code.

---
 drivers/clk/clk.c |   91 ++++++++++++++++++++++++++---------------------------
 1 files changed, 45 insertions(+), 46 deletions(-)
drivers/clk/clk.c +45 -46
index 934cfd1..b4dbb8c 100644
@@ -1377,67 +1377,61 @@ static int __clk_set_parent(struct clk *clk, struct clk *parent, u8 p_index)
 	unsigned long flags;
 	int ret = 0;
 	struct clk *old_parent = clk->parent;
-	bool migrated_enable = false;
 
-	/* migrate prepare */
-	if (clk->prepare_count)
+	/*
+	 * Migrate prepare state between parents and prevent race with
+	 * clk_enable().
+	 *
+	 * If the clock is not prepared, then a race with
+	 * clk_enable/disable() is impossible since we already have the
+	 * prepare lock (future calls to clk_enable() need to be preceded by
+	 * a clk_prepare()).
+	 *
+	 * If the clock is prepared, migrate the prepared state to the new
+	 * parent and also protect against a race with clk_enable() by
+	 * forcing the clock and the new parent on.  This ensures that all
+	 * future calls to clk_enable() are practically NOPs with respect to
+	 * hardware and software states.
+	 *
+	 * See also: Comment for clk_set_parent() below.
+	 */
+	if (clk->prepare_count) {
 		__clk_prepare(parent);
-
-	flags = clk_enable_lock();
-
-	/* migrate enable */
-	if (clk->enable_count) {
-		__clk_enable(parent);
-		migrated_enable = true;
+		clk_enable(parent);
+		clk_enable(clk);
 	}
 
 	/* update the clk tree topology */
+	flags = clk_enable_lock();
 	clk_reparent(clk, parent);
-
 	clk_enable_unlock(flags);
 
 	/* change clock input source */
 	if (parent && clk->ops->set_parent)
 		ret = clk->ops->set_parent(clk->hw, p_index);
-
 	if (ret) {
-		/*
-		 * The error handling is tricky due to that we need to release
-		 * the spinlock while issuing the .set_parent callback. This
-		 * means the new parent might have been enabled/disabled in
-		 * between, which must be considered when doing rollback.
-		 */
-		flags = clk_enable_lock();
 
+		flags = clk_enable_lock();
 		clk_reparent(clk, old_parent);
-
-		if (migrated_enable && clk->enable_count) {
-			__clk_disable(parent);
-		} else if (migrated_enable && (clk->enable_count == 0)) {
-			__clk_disable(old_parent);
-		} else if (!migrated_enable && clk->enable_count) {
-			__clk_disable(parent);
-			__clk_enable(old_parent);
-		}
-
 		clk_enable_unlock(flags);
 
-		if (clk->prepare_count)
+		if (clk->prepare_count) {
+			clk_disable(clk);
+			clk_disable(parent);
 			__clk_unprepare(parent);
-
+		}
 		return ret;
 	}
 
-	/* clean up enable for old parent if migration was done */
-	if (migrated_enable) {
-		flags = clk_enable_lock();
-		__clk_disable(old_parent);
-		clk_enable_unlock(flags);
-	}
-
-	/* clean up prepare for old parent if migration was done */
-	if (clk->prepare_count)
+	/*
+	 * Finish the migration of prepare state and undo the changes done
+	 * for preventing a race with clk_enable().
+	 */
+	if (clk->prepare_count) {
+		clk_disable(clk);
+		clk_disable(old_parent);
 		__clk_unprepare(old_parent);
+	}
 
 	/* update debugfs with new clk tree topology */
 	clk_debug_reparent(clk, parent);
@@ -1449,12 +1443,17 @@ static int __clk_set_parent(struct clk *clk, struct clk *parent, u8 p_index)
  * @clk: the mux clk whose input we are switching
  * @parent: the new input to clk
  *
- * Re-parent clk to use parent as it's new input source.  If clk has the
- * CLK_SET_PARENT_GATE flag set then clk must be gated for this
- * operation to succeed.  After successfully changing clk's parent
- * clk_set_parent will update the clk topology, sysfs topology and
- * propagate rate recalculation via __clk_recalc_rates.  Returns 0 on
- * success, -EERROR otherwise.
+ * Re-parent clk to use parent as its new input source.  If clk is in
+ * prepared state, the clk will get enabled for the duration of this call. If
+ * that's not acceptable for a specific clk (Eg: the consumer can't handle
+ * that, the reparenting is glitchy in hardware, etc), use the
+ * CLK_SET_PARENT_GATE flag to allow reparenting only when clk is unprepared.
+ *
+ * After successfully changing clk's parent clk_set_parent will update the
+ * clk topology, sysfs topology and propagate rate recalculation via
+ * __clk_recalc_rates.
+ *
+ * Returns 0 on success, -EERROR otherwise.
  */
 int clk_set_parent(struct clk *clk, struct clk *parent)
 {
Signature · 4 lines
1.7.8.3

The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation
#9 lore raw

Quoting Saravana Kannan (2013-05-15 21:07:24)

Quoted 39 lines

Without this patch, the following race condition is possible.
* clk-A has two parents - clk-X and clk-Y.
* All three are disabled and clk-X is current parent.
* Thread A: clk_set_parent(clk-A, clk-Y).
* Thread A: <snip execution flow>
* Thread A: Grabs enable lock.
* Thread A: Sees enable count of clk-A is 0, so doesn't enable clk-Y.
* Thread A: Updates clk-A SW parent to clk-Y
* Thread A: Releases enable lock.
* Thread B: clk_enable(clk-A).
* Thread B: clk_enable() enables clk-Y, then enabled clk-A and returns.

clk-A is now enabled in software, but not clocking in hardware since the hardware parent is still clk-X.

The only way to avoid race conditions between clk_set_parent() and clk_enable/disable() is to ensure that clk_enable/disable() calls don't require changes to hardware enable state between changes to software clock topology and hardware clock topology.

The options to achieve the above are:
1. Grab the enable lock before changing software/hardware topology and release it afterwards.
2. Keep the clock enabled for the duration of software/hardware topology change so that any additional enable/disable calls don't try to change the hardware state. Once the topology change is complete, the clock can be put back in its original enable state.

Option (1) is not an acceptable solution since the set_parent() ops might need to sleep.

Therefore, this patch implements option (2).

This patch doesn't violate any API semantics. clk_disable() doesn't guarantee that the clock is actually disabled. So, no clients of a clock can assume that a clock is disabled after their last call to clk_disable(). So, enabling the clock during a parent change is not a violation of any API semantics.

Quoted 3 lines

This also has the nice side effect of simplifying the error handling code.

Updated to this version in clk-next.

Thanks,
Mike

Quoted 139 lines
---
 drivers/clk/clk.c |   91 ++++++++++++++++++++++++++---------------------------
 1 files changed, 45 insertions(+), 46 deletions(-)
drivers/clk/clk.c +45 -46
index 934cfd1..b4dbb8c 100644
@@ -1377,67 +1377,61 @@ static int __clk_set_parent(struct clk *clk, struct clk *parent, u8 p_index)
        unsigned long flags;
        int ret = 0;
        struct clk *old_parent = clk->parent;
-       bool migrated_enable = false;
 
-       /* migrate prepare */
-       if (clk->prepare_count)
+       /*
+        * Migrate prepare state between parents and prevent race with
+        * clk_enable().
+        *
+        * If the clock is not prepared, then a race with
+        * clk_enable/disable() is impossible since we already have the
+        * prepare lock (future calls to clk_enable() need to be preceded by
+        * a clk_prepare()).
+        *
+        * If the clock is prepared, migrate the prepared state to the new
+        * parent and also protect against a race with clk_enable() by
+        * forcing the clock and the new parent on.  This ensures that all
+        * future calls to clk_enable() are practically NOPs with respect to
+        * hardware and software states.
+        *
+        * See also: Comment for clk_set_parent() below.
+        */
+       if (clk->prepare_count) {
                __clk_prepare(parent);
-
-       flags = clk_enable_lock();
-
-       /* migrate enable */
-       if (clk->enable_count) {
-               __clk_enable(parent);
-               migrated_enable = true;
+               clk_enable(parent);
+               clk_enable(clk);
        }
 
        /* update the clk tree topology */
+       flags = clk_enable_lock();
        clk_reparent(clk, parent);
-
        clk_enable_unlock(flags);
 
        /* change clock input source */
        if (parent && clk->ops->set_parent)
                ret = clk->ops->set_parent(clk->hw, p_index);
-
        if (ret) {
-               /*
-                * The error handling is tricky due to that we need to release
-                * the spinlock while issuing the .set_parent callback. This
-                * means the new parent might have been enabled/disabled in
-                * between, which must be considered when doing rollback.
-                */
-               flags = clk_enable_lock();
 
+               flags = clk_enable_lock();
                clk_reparent(clk, old_parent);
-
-               if (migrated_enable && clk->enable_count) {
-                       __clk_disable(parent);
-               } else if (migrated_enable && (clk->enable_count == 0)) {
-                       __clk_disable(old_parent);
-               } else if (!migrated_enable && clk->enable_count) {
-                       __clk_disable(parent);
-                       __clk_enable(old_parent);
-               }
-
                clk_enable_unlock(flags);
 
-               if (clk->prepare_count)
+               if (clk->prepare_count) {
+                       clk_disable(clk);
+                       clk_disable(parent);
                        __clk_unprepare(parent);
-
+               }
                return ret;
        }
 
-       /* clean up enable for old parent if migration was done */
-       if (migrated_enable) {
-               flags = clk_enable_lock();
-               __clk_disable(old_parent);
-               clk_enable_unlock(flags);
-       }
-
-       /* clean up prepare for old parent if migration was done */
-       if (clk->prepare_count)
+       /*
+        * Finish the migration of prepare state and undo the changes done
+        * for preventing a race with clk_enable().
+        */
+       if (clk->prepare_count) {
+               clk_disable(clk);
+               clk_disable(old_parent);
                __clk_unprepare(old_parent);
+       }
 
        /* update debugfs with new clk tree topology */
        clk_debug_reparent(clk, parent);
@@ -1449,12 +1443,17 @@ static int __clk_set_parent(struct clk *clk, struct clk *parent, u8 p_index)
  * @clk: the mux clk whose input we are switching
  * @parent: the new input to clk
  *
- * Re-parent clk to use parent as it's new input source.  If clk has the
- * CLK_SET_PARENT_GATE flag set then clk must be gated for this
- * operation to succeed.  After successfully changing clk's parent
- * clk_set_parent will update the clk topology, sysfs topology and
- * propagate rate recalculation via __clk_recalc_rates.  Returns 0 on
- * success, -EERROR otherwise.
+ * Re-parent clk to use parent as its new input source.  If clk is in
+ * prepared state, the clk will get enabled for the duration of this call. If
+ * that's not acceptable for a specific clk (Eg: the consumer can't handle
+ * that, the reparenting is glitchy in hardware, etc), use the
+ * CLK_SET_PARENT_GATE flag to allow reparenting only when clk is unprepared.
+ *
+ * After successfully changing clk's parent clk_set_parent will update the
+ * clk topology, sysfs topology and propagate rate recalculation via
+ * __clk_recalc_rates.
+ *
+ * Returns 0 on success, -EERROR otherwise.
  */
 int clk_set_parent(struct clk *clk, struct clk *parent)
 {
Signature · 4 lines
1.7.8.3

The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation
#10 lore raw
Quoted 40 lines · Mike Turquette

Quoting Saravana Kannan (2013-05-15 21:07:24)

Quoted 39 lines

Without this patch, the following race condition is possible.
* clk-A has two parents - clk-X and clk-Y.
* All three are disabled and clk-X is current parent.
* Thread A: clk_set_parent(clk-A, clk-Y).
* Thread A: <snip execution flow>
* Thread A: Grabs enable lock.
* Thread A: Sees enable count of clk-A is 0, so doesn't enable clk-Y.
* Thread A: Updates clk-A SW parent to clk-Y
* Thread A: Releases enable lock.
* Thread B: clk_enable(clk-A).
* Thread B: clk_enable() enables clk-Y, then enabled clk-A and returns.

clk-A is now enabled in software, but not clocking in hardware since the hardware parent is still clk-X.

The only way to avoid race conditions between clk_set_parent() and clk_enable/disable() is to ensure that clk_enable/disable() calls don't require changes to hardware enable state between changes to software clock topology and hardware clock topology.

The options to achieve the above are:
1. Grab the enable lock before changing software/hardware topology and release it afterwards.
2. Keep the clock enabled for the duration of software/hardware topology change so that any additional enable/disable calls don't try to change the hardware state. Once the topology change is complete, the clock can be put back in its original enable state.

Option (1) is not an acceptable solution since the set_parent() ops might need to sleep.

Therefore, this patch implements option (2).

This patch doesn't violate any API semantics. clk_disable() doesn't guarantee that the clock is actually disabled. So, no clients of a clock can assume that a clock is disabled after their last call to clk_disable(). So, enabling the clock during a parent change is not a violation of any API semantics.

Quoted 6 lines
Quoted 3 lines

This also has the nice side effect of simplifying the error handling code.

Updated to this version in clk-next.

Thanks Mike. I forgot to add the Ack by Ulf. Would be nice if you can put that in.

Btw, I did send this email to the list. But looks like this mail is wedged in the series of tubes in the arm mailing list.

-Saravana

Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation
#11 lore raw

PATCH v2 1/1 clk: Fix race condition between clk_set_parent and clk_enable()

Quoting Saravana Kannan (2013-05-16 14:31:23)

Quoted 43 lines · Mike Turquette
Quoted 42 lines

Quoting Saravana Kannan (2013-05-15 21:07:24)

Quoted 41 lines

Without this patch, the following race condition is possible.
* clk-A has two parents - clk-X and clk-Y.
* All three are disabled and clk-X is current parent.
* Thread A: clk_set_parent(clk-A, clk-Y).
* Thread A: <snip execution flow>
* Thread A: Grabs enable lock.
* Thread A: Sees enable count of clk-A is 0, so doesn't enable clk-Y.
* Thread A: Updates clk-A SW parent to clk-Y
* Thread A: Releases enable lock.
* Thread B: clk_enable(clk-A).
* Thread B: clk_enable() enables clk-Y, then enabled clk-A and returns.

clk-A is now enabled in software, but not clocking in hardware since the hardware parent is still clk-X.

The only way to avoid race conditions between clk_set_parent() and clk_enable/disable() is to ensure that clk_enable/disable() calls don't require changes to hardware enable state between changes to software clock topology and hardware clock topology.

The options to achieve the above are:
1. Grab the enable lock before changing software/hardware topology and release it afterwards.
2. Keep the clock enabled for the duration of software/hardware topology change so that any additional enable/disable calls don't try to change the hardware state. Once the topology change is complete, the clock can be put back in its original enable state.

Option (1) is not an acceptable solution since the set_parent() ops might need to sleep.

Therefore, this patch implements option (2).

This patch doesn't violate any API semantics. clk_disable() doesn't guarantee that the clock is actually disabled. So, no clients of a clock can assume that a clock is disabled after their last call to clk_disable(). So, enabling the clock during a parent change is not a violation of any API semantics.

This also has the nice side effect of simplifying the error handling code.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Updated to this version in clk-next.

Thanks Mike. I forgot to add the Ack by Ulf. Would be nice if you can put that in.

Yes I picked up the Ack.

Regards,
Mike

Quoted 9 lines

Btw, I did send this email to the list. But looks like this mail is wedged in the series of tubes in the arm mailing list.

-Saravana

Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation