Skip to contentSkip to conversation

coresight: Serialize enabling/disabling a link device.

PATCH Superseded Yabin Cui 2 participants 3 messages Aug 9, 2019 – Aug 12, 2019

Subsystems (2)

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Yabin Cui

author
#0 lore raw

PATCH v2 1/1 coresight: Serialize enabling/disabling a link device.

When tracing etm data of multiple threads on multiple cpus through perf interface, some link devices are shared between paths of different cpus. It creates race conditions when different cpus wants to enable/disable the same link device at the same time.

Example 1:
Two cpus want to enable different ports of a coresight funnel, thus calling the funnel enable operation at the same time. But the funnel enable operation isn't reentrantable.

Example 2:
For an enabled coresight dynamic replicator with refcnt=1, one cpu wants to disable it, while another cpu wants to enable it. Ideally we still have an enabled replicator with refcnt=1 at the end. But in reality the result is uncertain.

Since coresight devices claim themselves when enabled for self-hosted usage, the race conditions above usually make the link devices not usable after many cycles.

To fix the race conditions, this patch adds a spinlock to serialize enabling/disabling a link device.

---

v1 -> v2: extend lock range to protect read of refcnt in coresight_disable_link().

---
 drivers/hwtracing/coresight/coresight.c | 12 +++++++++++-
 include/linux/coresight.h               |  3 +++
 2 files changed, 14 insertions(+), 1 deletion(-)
drivers/hwtracing/coresight/coresight.c +11 -1
index 55db77f6410b..e526bdeaeb22 100644
@@ -256,6 +256,7 @@ static int coresight_enable_link(struct coresight_device *csdev,
 	int ret;
 	int link_subtype;
 	int refport, inport, outport;
+	unsigned long flags;
 
 	if (!parent || !child)
 		return -EINVAL;
@@ -274,15 +275,18 @@ static int coresight_enable_link(struct coresight_device *csdev,
 	if (refport < 0)
 		return refport;
 
+	spin_lock_irqsave(&csdev->spinlock, flags);
 	if (atomic_inc_return(&csdev->refcnt[refport]) == 1) {
 		if (link_ops(csdev)->enable) {
 			ret = link_ops(csdev)->enable(csdev, inport, outport);
 			if (ret) {
 				atomic_dec(&csdev->refcnt[refport]);
+				spin_unlock_irqrestore(&csdev->spinlock, flags);
 				return ret;
 			}
 		}
 	}
+	spin_unlock_irqrestore(&csdev->spinlock, flags);
 
 	csdev->enable = true;
 
@@ -296,6 +300,7 @@ static void coresight_disable_link(struct coresight_device *csdev,
 	int i, nr_conns;
 	int link_subtype;
 	int refport, inport, outport;
+	unsigned long flags;
 
 	if (!parent || !child)
 		return;
@@ -315,14 +320,18 @@ static void coresight_disable_link(struct coresight_device *csdev,
 		nr_conns = 1;
 	}
 
+	spin_lock_irqsave(&csdev->spinlock, flags);
 	if (atomic_dec_return(&csdev->refcnt[refport]) == 0) {
 		if (link_ops(csdev)->disable)
 			link_ops(csdev)->disable(csdev, inport, outport);
 	}
 
 	for (i = 0; i < nr_conns; i++)
-		if (atomic_read(&csdev->refcnt[i]) != 0)
+		if (atomic_read(&csdev->refcnt[i]) != 0) {
+			spin_unlock_irqrestore(&csdev->spinlock, flags);
 			return;
+		}
+	spin_unlock_irqrestore(&csdev->spinlock, flags);
 
 	csdev->enable = false;
 }
@@ -1225,6 +1234,7 @@ struct coresight_device *coresight_register(struct coresight_desc *desc)
 	csdev->subtype = desc->subtype;
 	csdev->ops = desc->ops;
 	csdev->orphan = false;
+	spin_lock_init(&csdev->spinlock);
 
 	csdev->dev.type = &coresight_dev_type[desc->type];
 	csdev->dev.groups = desc->groups;
include/linux/coresight.h +3 -0
index a2b68823717b..dd28d9ab841d 100644
@@ -9,6 +9,7 @@
 #include <linux/device.h>
 #include <linux/perf_event.h>
 #include <linux/sched.h>
+#include <linux/spinlock.h>
 
 /* Peripheral id registers (0xFD0-0xFEC) */
 #define CORESIGHT_PERIPHIDR4	0xfd0
@@ -153,6 +154,7 @@ struct coresight_connection {
  *		activated but not yet enabled.  Enabling for a _sink_
  *		appens when a source has been selected for that it.
  * @ea:		Device attribute for sink representation under PMU directory.
+ * @spinlock:	Serialize enabling/disabling this device.
  */
 struct coresight_device {
 	struct coresight_platform_data *pdata;
@@ -166,6 +168,7 @@ struct coresight_device {
 	/* sink specific fields */
 	bool activated;	/* true only if a sink is part of a path */
 	struct dev_ext_attribute *ea;
+	spinlock_t spinlock;
 };
 
 /*
Signature · 1 line
2.23.0.rc1.153.gdeed80330f-goog
maintainer Reviewed-by on v2 1/1
#1 lore raw

Hi Yabin,

Quoted 20 lines · Yabin Cui

When tracing etm data of multiple threads on multiple cpus through perf interface, some link devices are shared between paths of different cpus. It creates race conditions when different cpus wants to enable/disable the same link device at the same time.

Example 1:
Two cpus want to enable different ports of a coresight funnel, thus calling the funnel enable operation at the same time. But the funnel enable operation isn't reentrantable.

Example 2:
For an enabled coresight dynamic replicator with refcnt=1, one cpu wants to disable it, while another cpu wants to enable it. Ideally we still have an enabled replicator with refcnt=1 at the end. But in reality the result is uncertain.

Since coresight devices claim themselves when enabled for self-hosted usage, the race conditions above usually make the link devices not usable after many cycles.

Quoted 8 lines

To fix the race conditions, this patch adds a spinlock to serialize enabling/disabling a link device.

---

v1 -> v2: extend lock range to protect read of refcnt in coresight_disable_link().

Thanks for this. Please find my comments below.

Quoted 26 lines
---
  drivers/hwtracing/coresight/coresight.c | 12 +++++++++++-
  include/linux/coresight.h               |  3 +++
  2 files changed, 14 insertions(+), 1 deletion(-)
drivers/hwtracing/coresight/coresight.c +2 -0
index 55db77f6410b..e526bdeaeb22 100644
@@ -256,6 +256,7 @@ static int coresight_enable_link(struct coresight_device *csdev,
  	int ret;
  	int link_subtype;
  	int refport, inport, outport;
+	unsigned long flags;
 
  	if (!parent || !child)
  		return -EINVAL;
@@ -274,15 +275,18 @@ static int coresight_enable_link(struct coresight_device *csdev,
  	if (refport < 0)
  		return refport;
 
+	spin_lock_irqsave(&csdev->spinlock, flags);
  	if (atomic_inc_return(&csdev->refcnt[refport]) == 1) {
  		if (link_ops(csdev)->enable) {
  			ret = link_ops(csdev)->enable(csdev, inport, outport);

drivers/hwtracing/coresight/coresight.c static int coresight_enable_link(struct coresight_device *csdev, · line 291

Quoted 10 lines
drivers/hwtracing/coresight/coresight.c +2 -0
  			if (ret) {
  				atomic_dec(&csdev->refcnt[refport]);
+				spin_unlock_irqrestore(&csdev->spinlock, flags);
  				return ret;
  			}
  		}
  	}
+	spin_unlock_irqrestore(&csdev->spinlock, flags);
 
  	csdev->enable = true;

Please could we move this inside the spin_lock () too ?

Quoted 19 lines
drivers/hwtracing/coresight/coresight.c +2 -0
@@ -296,6 +300,7 @@ static void coresight_disable_link(struct coresight_device *csdev,
  	int i, nr_conns;
  	int link_subtype;
  	int refport, inport, outport;
+	unsigned long flags;
 
  	if (!parent || !child)
  		return;
@@ -315,14 +320,18 @@ static void coresight_disable_link(struct coresight_device *csdev,
  		nr_conns = 1;
  	}
 
+	spin_lock_irqsave(&csdev->spinlock, flags);
  	if (atomic_dec_return(&csdev->refcnt[refport]) == 0) {
  		if (link_ops(csdev)->disable)
  			link_ops(csdev)->disable(csdev, inport, outport);
  	}
 

drivers/hwtracing/coresight/coresight.c static void coresight_disable_link(struct coresight_device *csdev, · line 337

Quoted 10 lines
drivers/hwtracing/coresight/coresight.c +4 -1
  	for (i = 0; i < nr_conns; i++)
-		if (atomic_read(&csdev->refcnt[i]) != 0)
+		if (atomic_read(&csdev->refcnt[i]) != 0) {
+			spin_unlock_irqrestore(&csdev->spinlock, flags);
  			return;
+		}
+	spin_unlock_irqrestore(&csdev->spinlock, flags);
 
  	csdev->enable = false;
  }

And this too ? I understand this may not be used right now, but we can avoid any surprises when we do so.

With the above fixed, :

maintainer
#2 lore raw

Hi Yabin,

Quoted 26 lines · Suzuki K Poulose

Hi Yabin,

Quoted 22 lines · Yabin Cui

When tracing etm data of multiple threads on multiple cpus through perf interface, some link devices are shared between paths of different cpus. It creates race conditions when different cpus wants to enable/disable the same link device at the same time.

Example 1:
Two cpus want to enable different ports of a coresight funnel, thus calling the funnel enable operation at the same time. But the funnel enable operation isn't reentrantable.

Example 2:
For an enabled coresight dynamic replicator with refcnt=1, one cpu wants to disable it, while another cpu wants to enable it. Ideally we still have an enabled replicator with refcnt=1 at the end. But in reality the result is uncertain.

Since coresight devices claim themselves when enabled for self-hosted usage, the race conditions above usually make the link devices not usable after many cycles.

To fix the race conditions, this patch adds a spinlock to serialize enabling/disabling a link device.

Please could you also add :

Fixes a06ae8609b3dd ("coresight: add CoreSight core layer framework")

Suzuki