Skip to contentSkip to conversation

ARC: Force disable IOC if we don't want to use it

PATCH In review Alexey Brodkin 2 participants 3 messages Dec 18, 2017 – Jan 19, 2018 Synopsys ARC architecture

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 ARC: Force disable IOC if we don't want to use it

If software that was executed before Linux kernel [like boot-ROM or bootloader] enabled IOC but we'd like to not use it [mostly for debugging of weird DMA issues] we essentially need to disable IOC. So we do here.

Note we will only disable IOC if "ioc_enable" variable is force set to 0. As of today that's only possible either before building right in arch/arc/mm/cache.c or via debugger on target reght before execution of the kernel starts.

We may make "ioc_enable" a boot-parameter later though.

Signed-off-by Alexey Brodkin <abrodkin at synopsys.com>

---

Chnages in v2:
* Now disabling really works as in v1 we never entered disabling code if ioc_enable=0.
* Do nothing if IOC was not enabled

 arch/arc/mm/cache.c | 42 ++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 40 insertions(+), 2 deletions(-)
arch/arc/mm/cache.c +40 -2
index 69f77c113875..9cadf7b779ed 100644
@@ -1186,6 +1186,40 @@ noinline void __init arc_ioc_setup(void)
 	__dc_enable();
 }
 
+/*
+ * Disabling of IOC is quite a tricky action because
+ * nobody knows what happens if there're IOC-ahndled tarnsactions in flight
+ * when we're disabling IOC.
+ *
+ * And the problem is external DMA masters [that were initialized and set in a
+ * bootlaoder that was executed before we got here] might continue to send data
+ * to memory even at this point and we have no way to prevent that.
+ *
+ * That said it's much safer to not enable IOC at all anywhere before
+ * in boot-ROM, bootloader etc but if we do need to disable it in Linux kernel
+ * it should be done as early as possible and made by master core while all
+ * slaves aren't active.
+ *
+ */
+noinline void __init arc_ioc_disable(void)
+{
+	/* Exit if IOC was never enabled */
+	if (!read_aux_reg(ARC_REG_IO_COH_ENABLE))
+		return;
+
+	/* Flush + invalidate + disable L1 dcache */
+	__dc_disable();
+
+	/* Flush + invalidate SLC */
+	if (read_aux_reg(ARC_REG_SLC_BCR))
+		slc_entire_op(OP_FLUSH_N_INV);
+
+	write_aux_reg(ARC_REG_IO_COH_ENABLE, 0);
+
+	/* Re-enable L1 dcache */
+	__dc_enable();
+}
+
 /*
  * Cache related boot time checks/setups only needed on master CPU:
  *  - Geometry checks (kernel build and hardware agree: e.g. L1_CACHE_BYTES)
@@ -1247,8 +1281,12 @@ void __init arc_cache_init_master(void)
 	if (is_isa_arcv2() && l2_line_sz && !slc_enable)
 		arc_slc_disable();
 
-	if (is_isa_arcv2() && ioc_enable)
-		arc_ioc_setup();
+	if (is_isa_arcv2()) {
+		if (ioc_enable)
+			arc_ioc_setup();
+		else if (ioc_exists)
+			arc_ioc_disable();
+	}
 
 	if (is_isa_arcv2() && ioc_enable) {
 		__dma_cache_wback_inv = __dma_cache_wback_inv_ioc;
Signature · 1 line
2.14.3
#1 lore raw
Quoted 29 lines · Alexey Brodkin

If software that was executed before Linux kernel [like boot-ROM or bootloader] enabled IOC but we'd like to not use it [mostly for debugging of weird DMA issues] we essentially need to disable IOC. So we do here.

Note we will only disable IOC if "ioc_enable" variable is force set to 0. As of today that's only possible either before building right in arch/arc/mm/cache.c or via debugger on target reght before execution of the kernel starts.

We may make "ioc_enable" a boot-parameter later though.

Signed-off-by Alexey Brodkin <abrodkin at synopsys.com>

---

Chnages in v2:
* Now disabling really works as in v1 we never entered disabling code if ioc_enable=0.
* Do nothing if IOC was not enabled

  arch/arc/mm/cache.c | 42 ++++++++++++++++++++++++++++++++++++++++--
  1 file changed, 40 insertions(+), 2 deletions(-)
arch/arc/mm/cache.c +0 -0
index 69f77c113875..9cadf7b779ed 100644
@@ -1186,6 +1186,40 @@ noinline void __init arc_ioc_setup(void)
  	__dc_enable();

arch/arc/mm/cache.c noinline void __init arc_ioc_setup(void) · line 1196

Quoted 10 lines
arch/arc/mm/cache.c +8 -0
  }
 
+/*
+ * Disabling of IOC is quite a tricky action because
+ * nobody knows what happens if there're IOC-ahndled tarnsactions in flight
+ * when we're disabling IOC.
+ *
+ * And the problem is external DMA masters [that were initialized and set in a
+ * bootlaoder that was executed before we got here] might continue to send data
+ * to memory even at this point and we have no way to prevent that.

Isn't this a sufficient red flag to warrant NOT adding this to kernel !

I agree that the use case is bring up of a new RTL version etc with bootloader already in there - but in that case can we NOT bypass those and run the kernel bare metal ?

arch/arc/mm/cache.c noinline void __init arc_ioc_setup(void) · line 1202

Quoted 6 lines
arch/arc/mm/cache.c +6 -0
+ *
+ * That said it's much safer to not enable IOC at all anywhere before
+ * in boot-ROM, bootloader etc but if we do need to disable it in Linux kernel
+ * it should be done as early as possible and made by master core while all
+ * slaves aren't active.
+ *

Again this is like saying - this is Pandora's Box - don't open it - which is more of a invitation for others to actually open it !

Quoted 38 lines
arch/arc/mm/cache.c +26 -2
+ */
+noinline void __init arc_ioc_disable(void)
+{
+	/* Exit if IOC was never enabled */
+	if (!read_aux_reg(ARC_REG_IO_COH_ENABLE))
+		return;
+
+	/* Flush + invalidate + disable L1 dcache */
+	__dc_disable();
+
+	/* Flush + invalidate SLC */
+	if (read_aux_reg(ARC_REG_SLC_BCR))
+		slc_entire_op(OP_FLUSH_N_INV);
+
+	write_aux_reg(ARC_REG_IO_COH_ENABLE, 0);
+
+	/* Re-enable L1 dcache */
+	__dc_enable();
+}
+
  /*
   * Cache related boot time checks/setups only needed on master CPU:
   *  - Geometry checks (kernel build and hardware agree: e.g. L1_CACHE_BYTES)
@@ -1247,8 +1281,12 @@ void __init arc_cache_init_master(void)
  	if (is_isa_arcv2() && l2_line_sz && !slc_enable)
  		arc_slc_disable();
 
-	if (is_isa_arcv2() && ioc_enable)
-		arc_ioc_setup();
+	if (is_isa_arcv2()) {
+		if (ioc_enable)
+			arc_ioc_setup();
+		else if (ioc_exists)
+			arc_ioc_disable();
+	}
 
  	if (is_isa_arcv2() && ioc_enable) {
  		__dma_cache_wback_inv = __dma_cache_wback_inv_ioc;
#2 lore raw

Hi Vineet,

Quoted 54 lines · Vineet Gupta
Quoted 29 lines · Alexey Brodkin

If software that was executed before Linux kernel [like boot-ROM or bootloader] enabled IOC but we'd like to not use it [mostly for debugging of weird DMA issues] we essentially need to disable IOC. So we do here.

Note we will only disable IOC if "ioc_enable" variable is force set to 0. As of today that's only possible either before building right in arch/arc/mm/cache.c or via debugger on target reght before execution of the kernel starts.

We may make "ioc_enable" a boot-parameter later though.

Signed-off-by Alexey Brodkin <abrodkin at synopsys.com>

---

Chnages in v2:
* Now disabling really works as in v1 we never entered disabling code if ioc_enable=0.
* Do nothing if IOC was not enabled

  arch/arc/mm/cache.c | 42 ++++++++++++++++++++++++++++++++++++++++--
  1 file changed, 40 insertions(+), 2 deletions(-)
arch/arc/mm/cache.c +0 -0
index 69f77c113875..9cadf7b779ed 100644
@@ -1186,6 +1186,40 @@ noinline void __init arc_ioc_setup(void)
  	__dc_enable();

arch/arc/mm/cache.c noinline void __init arc_ioc_setup(void) · line 1196

Quoted 10 lines
arch/arc/mm/cache.c +8 -0
  }
 
+/*
+ * Disabling of IOC is quite a tricky action because
+ * nobody knows what happens if there're IOC-ahndled tarnsactions in flight
+ * when we're disabling IOC.
+ *
+ * And the problem is external DMA masters [that were initialized and set in a
+ * bootlaoder that was executed before we got here] might continue to send data
+ * to memory even at this point and we have no way to prevent that.

Isn't this a sufficient red flag to warrant NOT adding this to kernel !

I agree that the use case is bring up of a new RTL version etc with bootloader already in there - but in that case can we NOT bypass those and run the kernel bare metal ?

arch/arc/mm/cache.c noinline void __init arc_ioc_setup(void) · line 1202

Quoted 6 lines
arch/arc/mm/cache.c +6 -0
+ *
+ * That said it's much safer to not enable IOC at all anywhere before
+ * in boot-ROM, bootloader etc but if we do need to disable it in Linux kernel
+ * it should be done as early as possible and made by master core while all
+ * slaves aren't active.
+ *
Quoted 2 lines

Again this is like saying - this is Pandora's Box - don't open it - which is more of a invitation for others to actually open it !

I agree with all that above.

In fact I already sent a pull-request to Tom Rini for a series among other things
disabling IOC in U-Boot, see:
1) Pull-request: https://lists.denx.de/pipermail/u-boot/2018-January/317602.html
2) Actual patch: http://git.denx.de/?p=u-boot/u-boot-arc.git;a=commit;h=b0146f9e29ca2e82262416aca65395c322a618f9

So indeed this patch for Linux could be dropped.

-Alexey