Skip to contentSkip to conversation

MIPS: OCTEON: cvmx-l2c: use the OCTEON II register model on all CN6XXX

PATCH Applied Orgad Shaneh 2 participants 10 messages Sep 15 – Sep 26 MIPS

lore
Share & tools
lore
#0 lore raw

PATCH 1/5 MIPS: OCTEON: cvmx-l2c: use the OCTEON II register model on all CN6XXX

The L2 helpers pick between the OCTEON II and the CN3XXX/CN5XXX register model with OCTEON_IS_MODEL(OCTEON_CN63XX), which matches the CN63XX only. Every other OCTEON II part - CN66XX, CN68XX, CN61XX - takes the CN3XXX/CN5XXX branch and touches registers that do not exist on it: L2C_DBG, L2C_LCKBASE, L2C_LCKOFF and L2C_SPAR4 instead of the LCKL2 cache operation, the L2C_WPAR_{PPX,IOBX} way-partitioning registers and L2C_TADX_TAG.

On a CN6635 board cvmx_l2c_lock_line(), called from prom_init() under CONFIG_CAVIUM_OCTEON_LOCK_L2 to lock the TLB refill and exception vectors, therefore went through L2C_DBG/L2C_LCKBASE. Whatever those writes hit inside the L2 controller left it silently dropping stores to DRAM above 512 MB: a probe that wrote 0x20400000, 0x40000000, 0x80000000 and 0xe0000000 in prom_init read its values back before the lock block and the old contents right after it, on CCA 0, 2 and 3 alike, while writes below 256 MB and every u-boot "mw" stuck. Since the cvmx bootmem free list keeps its nodes inside the free blocks themselves, the kernel's first split above 0x20000000 was lost, the walk of the free list stopped there, and the board came up with 248 MB of its 8 GB.

Match the whole CN6XXX family, which is what these register accesses are keyed to. The CN63XX keeps exactly the same path.

The geometry helpers in this file need the same treatment but not the same condition; they are handled in the next patch.

Assisted-by Claude:claude-opus-5

---

arch/mips/cavium-octeon/executive/cvmx-l2c.c +11 -11
@@ -54,7 +54,7 @@ int cvmx_l2c_get_core_way_partition(uint32_t core)
 	if (core >= cvmx_octeon_num_cores())
 		return -1;
 
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX))
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX))
 		return cvmx_read_csr(CVMX_L2C_WPAR_PPX(core)) & 0xffff;
 
 	/*
@@ -91,14 +91,14 @@ int cvmx_l2c_set_core_way_partition(uint32_t core, uint32_t mask)
 	mask &= valid_mask;
 
 	/* A UMSK setting which blocks all L2C Ways is an error on some chips */
-	if (mask == valid_mask && !OCTEON_IS_MODEL(OCTEON_CN63XX))
+	if (mask == valid_mask && !OCTEON_IS_MODEL(OCTEON_CN6XXX))
 		return -1;
 
 	/* Validate the core number */
 	if (core >= cvmx_octeon_num_cores())
 		return -1;
 
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
 		cvmx_write_csr(CVMX_L2C_WPAR_PPX(core), mask);
 		return 0;
 	}
@@ -147,10 +147,10 @@ int cvmx_l2c_set_hw_way_partition(uint32_t mask)
 	mask &= valid_mask;
 
 	/* A UMSK setting which blocks all L2C Ways is an error on some chips */
-	if (mask == valid_mask	&& !OCTEON_IS_MODEL(OCTEON_CN63XX))
+	if (mask == valid_mask	&& !OCTEON_IS_MODEL(OCTEON_CN6XXX))
 		return -1;
 
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX))
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX))
 		cvmx_write_csr(CVMX_L2C_WPAR_IOBX(0), mask);
 	else
 		cvmx_write_csr(CVMX_L2C_SPAR4,
@@ -160,7 +160,7 @@ int cvmx_l2c_set_hw_way_partition(uint32_t mask)
 
 int cvmx_l2c_get_hw_way_partition(void)
 {
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX))
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX))
 		return cvmx_read_csr(CVMX_L2C_WPAR_IOBX(0)) & 0xffff;
 	else
 		return cvmx_read_csr(CVMX_L2C_SPAR4) & (0xFF);
@@ -313,7 +313,7 @@ static void fault_in(uint64_t addr, int len)
 
 int cvmx_l2c_lock_line(uint64_t addr)
 {
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
 		int shift = CVMX_L2C_TAG_ADDR_ALIAS_SHIFT;
 		uint64_t assoc = cvmx_l2c_get_num_assoc();
 		uint64_t tag = addr >> shift;
@@ -460,7 +460,7 @@ void cvmx_l2c_flush(void)
 int cvmx_l2c_unlock_line(uint64_t address)
 {
 
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
 		int assoc;
 		union cvmx_l2c_tag tag;
 		uint32_t tag_addr;
@@ -661,7 +661,7 @@ union cvmx_l2c_tag cvmx_l2c_get_tag(uint32_t association, uint32_t index)
 			     (int)index, cvmx_l2c_get_num_sets());
 		return tag;
 	}
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
 		union cvmx_l2c_tadx_tag l2c_tadx_tag;
 		uint64_t address = CVMX_ADD_SEG(CVMX_MIPS_SPACE_XKPHYS,
 						(association << CVMX_L2C_TAG_ADDR_ALIAS_SHIFT) |
@@ -745,7 +745,7 @@ uint32_t cvmx_l2c_address_to_index(uint64_t addr)
 	}
 
 	if (indxalias) {
-		if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+		if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
 			uint32_t a_14_12 = (idx / (CVMX_L2C_MEMBANK_SELECT_SIZE/(1<<CVMX_L2C_IDX_ADDR_SHIFT))) & 0x7;
 
 			idx ^= idx / cvmx_l2c_get_num_sets();
@@ -878,7 +878,7 @@ void cvmx_l2c_flush_line(uint32_t assoc, uint32_t index)
 		return;
 	}
 
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
 		uint64_t address;
 		/* Create the address based on index and association.
 		 * Bits<20:17> select the way of the cache block involved in
Signature · 1 line
2.47.0
#1 lore raw

PATCH 2/5 MIPS: OCTEON: cvmx-l2c: add the CN66XX L2 geometry

cvmx_l2c_get_set_bits() and cvmx_l2c_get_num_assoc() know the CN63XX as the only OCTEON II part, so on a CN6635 both fell through to their "Unsupported OCTEON Model" defaults: 2048 sets, and 8 ways which the CN3XXX/CN5XXX fuse path - a read of L2D_FUS3, which is not a register on OCTEON II - then shifted down to 2.

The CN66XX L2 has the CN63XX geometry, 16 ways and 1024 sets, and reports its fused-off ways in MIO_FUS_DAT3[l2c_crip] like the rest of OCTEON II, so name it in those three branches. With this the board reports assoc 16, sets 1024, and cvmx_l2c_lock_line() computes the tag alias from the right associativity.

The CN68XX and CN61XX are deliberately left alone: their L2 geometry differs from the CN63XX, and the CN68XX also uses bit <2> of MIO_FUS_DAT3[l2c_crip], which the decoding here does not handle. I have no such board to test on.

Assisted-by Claude:claude-opus-5

---

arch/mips/cavium-octeon/executive/cvmx-l2c.c +6 -3
@@ -773,7 +773,9 @@ int cvmx_l2c_get_set_bits(void)
 
 	if (OCTEON_IS_MODEL(OCTEON_CN56XX) || OCTEON_IS_MODEL(OCTEON_CN58XX))
 		l2_set_bits = 11;	/* 2048 sets */
-	else if (OCTEON_IS_MODEL(OCTEON_CN38XX) || OCTEON_IS_MODEL(OCTEON_CN63XX))
+	else if (OCTEON_IS_MODEL(OCTEON_CN38XX) ||
+		 OCTEON_IS_MODEL(OCTEON_CN63XX) ||
+		 OCTEON_IS_MODEL(OCTEON_CN66XX))
 		l2_set_bits = 10;	/* 1024 sets */
 	else if (OCTEON_IS_MODEL(OCTEON_CN31XX) || OCTEON_IS_MODEL(OCTEON_CN52XX))
 		l2_set_bits = 9;	/* 512 sets */
@@ -805,7 +807,8 @@ int cvmx_l2c_get_num_assoc(void)
 	    OCTEON_IS_MODEL(OCTEON_CN50XX) ||
 	    OCTEON_IS_MODEL(OCTEON_CN38XX))
 		l2_assoc = 8;
-	else if (OCTEON_IS_MODEL(OCTEON_CN63XX))
+	else if (OCTEON_IS_MODEL(OCTEON_CN63XX) ||
+		 OCTEON_IS_MODEL(OCTEON_CN66XX))
 		l2_assoc = 16;
 	else if (OCTEON_IS_MODEL(OCTEON_CN31XX) ||
 		 OCTEON_IS_MODEL(OCTEON_CN30XX))
@@ -816,7 +819,7 @@ int cvmx_l2c_get_num_assoc(void)
 	}
 
 	/* Check to see if part of the cache is disabled */
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN63XX) || OCTEON_IS_MODEL(OCTEON_CN66XX)) {
 		union cvmx_mio_fus_dat3 mio_fus_dat3;
 
 		mio_fus_dat3.u64 = cvmx_read_csr(CVMX_MIO_FUS_DAT3);
Signature · 1 line
2.47.0
maintainer Applied
#2 lore raw

Re: [PATCH 2/5] MIPS: OCTEON: cvmx-l2c: add the CN66XX L2 geometry

Quoted 55 lines · Orgad Shaneh

cvmx_l2c_get_set_bits() and cvmx_l2c_get_num_assoc() know the CN63XX as the only OCTEON II part, so on a CN6635 both fell through to their "Unsupported OCTEON Model" defaults: 2048 sets, and 8 ways which the CN3XXX/CN5XXX fuse path - a read of L2D_FUS3, which is not a register on OCTEON II - then shifted down to 2.

The CN66XX L2 has the CN63XX geometry, 16 ways and 1024 sets, and reports its fused-off ways in MIO_FUS_DAT3[l2c_crip] like the rest of OCTEON II, so name it in those three branches. With this the board reports assoc 16, sets 1024, and cvmx_l2c_lock_line() computes the tag alias from the right associativity.

The CN68XX and CN61XX are deliberately left alone: their L2 geometry differs from the CN63XX, and the CN68XX also uses bit <2> of MIO_FUS_DAT3[l2c_crip], which the decoding here does not handle. I have no such board to test on.

Assisted-by Claude:claude-opus-5

---

arch/mips/cavium-octeon/executive/cvmx-l2c.c +6 -3
@@ -773,7 +773,9 @@ int cvmx_l2c_get_set_bits(void)
 
 	if (OCTEON_IS_MODEL(OCTEON_CN56XX) || OCTEON_IS_MODEL(OCTEON_CN58XX))
 		l2_set_bits = 11;	/* 2048 sets */
-	else if (OCTEON_IS_MODEL(OCTEON_CN38XX) || OCTEON_IS_MODEL(OCTEON_CN63XX))
+	else if (OCTEON_IS_MODEL(OCTEON_CN38XX) ||
+		 OCTEON_IS_MODEL(OCTEON_CN63XX) ||
+		 OCTEON_IS_MODEL(OCTEON_CN66XX))
 		l2_set_bits = 10;	/* 1024 sets */
 	else if (OCTEON_IS_MODEL(OCTEON_CN31XX) || OCTEON_IS_MODEL(OCTEON_CN52XX))
 		l2_set_bits = 9;	/* 512 sets */
@@ -805,7 +807,8 @@ int cvmx_l2c_get_num_assoc(void)
 	    OCTEON_IS_MODEL(OCTEON_CN50XX) ||
 	    OCTEON_IS_MODEL(OCTEON_CN38XX))
 		l2_assoc = 8;
-	else if (OCTEON_IS_MODEL(OCTEON_CN63XX))
+	else if (OCTEON_IS_MODEL(OCTEON_CN63XX) ||
+		 OCTEON_IS_MODEL(OCTEON_CN66XX))
 		l2_assoc = 16;
 	else if (OCTEON_IS_MODEL(OCTEON_CN31XX) ||
 		 OCTEON_IS_MODEL(OCTEON_CN30XX))
@@ -816,7 +819,7 @@ int cvmx_l2c_get_num_assoc(void)
 	}
 
 	/* Check to see if part of the cache is disabled */
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN63XX) || OCTEON_IS_MODEL(OCTEON_CN66XX)) {
 		union cvmx_mio_fus_dat3 mio_fus_dat3;
 
 		mio_fus_dat3.u64 = cvmx_read_csr(CVMX_MIO_FUS_DAT3);
Signature · 1 line
2.47.0

applied to mips-next

Thomas.

Signature · 2 lines
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]
#3 lore raw

PATCH 3/5 MIPS: OCTEON: read the L2 crippled fuses from MIO_FUS_DAT3 on OCTEON II

In reply to Orgad Shaneh

prom_init() decides whether to lock lines into the L2 by reading L2D_FUS3, a CN3XXX/CN5XXX register. On OCTEON II the fuses that cripple the L2 are MIO_FUS_DAT3[l2c_crip] - which is what cvmx_l2c_get_num_assoc() already reads for those parts - so the test looks at an address that is not a register on the chip and gets a meaningless answer: it can skip L2 locking on a healthy cache, or lock into a reduced one.

Pick the register by family in a small helper.

Assisted-by Claude:claude-opus-5

---

arch/mips/cavium-octeon/setup.c +19 -1
@@ -653,6 +653,24 @@ void octeon_user_io_init(void)
 	write_c0_derraddr1(0);
 }
 
+#ifdef CONFIG_CAVIUM_OCTEON_LOCK_L2
+static bool __init octeon_l2_is_crippled(void)
+{
+	/*
+	 * L2D_FUS3 exists on CN3XXX/CN5XXX only. On OCTEON II the
+	 * crippled-L2 fuses live in MIO_FUS_DAT3[l2c_crip].
+	 */
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
+		union cvmx_mio_fus_dat3 fus_dat3;
+
+		fus_dat3.u64 = cvmx_read_csr(CVMX_MIO_FUS_DAT3);
+		return fus_dat3.s.l2c_crip != 0;
+	}
+
+	return cvmx_read_csr(CVMX_L2D_FUS3) & (3ull << 34);
+}
+#endif
+
 /**
  * prom_init - Early entry point for arch setup
  */
@@ -801,7 +819,7 @@ void __init prom_init(void)
 	}
 
 #ifdef CONFIG_CAVIUM_OCTEON_LOCK_L2
-	if (cvmx_read_csr(CVMX_L2D_FUS3) & (3ull << 34)) {
+	if (octeon_l2_is_crippled()) {
 		pr_info("Skipping L2 locking due to reduced L2 cache size\n");
 	} else {
 		uint32_t __maybe_unused ebase = read_c0_ebase() & 0x3ffff000;
Signature · 1 line
2.47.0
maintainer Applied
#4 lore raw

Re: [PATCH 3/5] MIPS: OCTEON: read the L2 crippled fuses from MIO_FUS_DAT3 on OCTEON II

Quoted 51 lines · Orgad Shaneh

prom_init() decides whether to lock lines into the L2 by reading L2D_FUS3, a CN3XXX/CN5XXX register. On OCTEON II the fuses that cripple the L2 are MIO_FUS_DAT3[l2c_crip] - which is what cvmx_l2c_get_num_assoc() already reads for those parts - so the test looks at an address that is not a register on the chip and gets a meaningless answer: it can skip L2 locking on a healthy cache, or lock into a reduced one.

Pick the register by family in a small helper.

Assisted-by Claude:claude-opus-5

---

arch/mips/cavium-octeon/setup.c +19 -1
@@ -653,6 +653,24 @@ void octeon_user_io_init(void)
 	write_c0_derraddr1(0);
 }
 
+#ifdef CONFIG_CAVIUM_OCTEON_LOCK_L2
+static bool __init octeon_l2_is_crippled(void)
+{
+	/*
+	 * L2D_FUS3 exists on CN3XXX/CN5XXX only. On OCTEON II the
+	 * crippled-L2 fuses live in MIO_FUS_DAT3[l2c_crip].
+	 */
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
+		union cvmx_mio_fus_dat3 fus_dat3;
+
+		fus_dat3.u64 = cvmx_read_csr(CVMX_MIO_FUS_DAT3);
+		return fus_dat3.s.l2c_crip != 0;
+	}
+
+	return cvmx_read_csr(CVMX_L2D_FUS3) & (3ull << 34);
+}
+#endif
+
 /**
  * prom_init - Early entry point for arch setup
  */
@@ -801,7 +819,7 @@ void __init prom_init(void)
 	}
 
 #ifdef CONFIG_CAVIUM_OCTEON_LOCK_L2
-	if (cvmx_read_csr(CVMX_L2D_FUS3) & (3ull << 34)) {
+	if (octeon_l2_is_crippled()) {
 		pr_info("Skipping L2 locking due to reduced L2 cache size\n");
 	} else {
 		uint32_t __maybe_unused ebase = read_c0_ebase() & 0x3ffff000;
Signature · 1 line
2.47.0

applied to mips-next

Thomas.

Signature · 2 lines
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]
#5 lore raw

PATCH 4/5 MIPS: OCTEON: program L2C_CTL, not L2C_CFG, on OCTEON II

In reply to Orgad Shaneh

cvmx_helper_initialize_packet_io_global() sets the L2 arbitration modes by rewriting L2C_CFG. That register (0x1180080000000) does not exist on OCTEON II; there the L2 controller is configured through L2C_CTL (0x1180080800000), whose corresponding fields are rsp_arb_mode and xmc_arb_mode.

On a CN6335 the stray write hung the board hard when this function ran - no exception, no watchdog, just a dead console.

Keep the L2C_CFG write for CN3XXX/CN5XXX and program L2C_CTL on CN6XXX with the values the vendor SDK uses there (rsp_arb_mode 1, xmc_arb_mode 0), which is the same trade the existing code makes for the older parts: keep IO blocks from being starved under high L2 load.

Assisted-by Claude:claude-opus-5

---

arch/mips/cavium-octeon/executive/cvmx-helper.c +15 -5
@@ -1014,7 +1014,6 @@ int cvmx_helper_initialize_packet_io_global(void)
 {
 	int result = 0;
 	int interface;
-	union cvmx_l2c_cfg l2c_cfg;
 	const int num_interfaces = cvmx_helper_get_number_of_interfaces();
 
 	/*
@@ -1029,10 +1028,21 @@ int cvmx_helper_initialize_packet_io_global(void)
 	 * to the cores. This avoids conditions where IO blocks might
 	 * be starved under very high L2 loads.
 	 */
-	l2c_cfg.u64 = cvmx_read_csr(CVMX_L2C_CFG);
-	l2c_cfg.s.lrf_arb_mode = 0;
-	l2c_cfg.s.rfb_arb_mode = 0;
-	cvmx_write_csr(CVMX_L2C_CFG, l2c_cfg.u64);
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
+		union cvmx_l2c_ctl l2c_ctl;
+
+		l2c_ctl.u64 = cvmx_read_csr(CVMX_L2C_CTL);
+		l2c_ctl.s.rsp_arb_mode = 1;
+		l2c_ctl.s.xmc_arb_mode = 0;
+		cvmx_write_csr(CVMX_L2C_CTL, l2c_ctl.u64);
+	} else {
+		union cvmx_l2c_cfg l2c_cfg;
+
+		l2c_cfg.u64 = cvmx_read_csr(CVMX_L2C_CFG);
+		l2c_cfg.s.lrf_arb_mode = 0;
+		l2c_cfg.s.rfb_arb_mode = 0;
+		cvmx_write_csr(CVMX_L2C_CFG, l2c_cfg.u64);
+	}
 
 	cvmx_pko_initialize_global();
 	for (interface = 0; interface < num_interfaces; interface++) {
Signature · 1 line
2.47.0
maintainer Applied
#6 lore raw

Re: [PATCH 4/5] MIPS: OCTEON: program L2C_CTL, not L2C_CFG, on OCTEON II

Quoted 56 lines · Orgad Shaneh

cvmx_helper_initialize_packet_io_global() sets the L2 arbitration modes by rewriting L2C_CFG. That register (0x1180080000000) does not exist on OCTEON II; there the L2 controller is configured through L2C_CTL (0x1180080800000), whose corresponding fields are rsp_arb_mode and xmc_arb_mode.

On a CN6335 the stray write hung the board hard when this function ran - no exception, no watchdog, just a dead console.

Keep the L2C_CFG write for CN3XXX/CN5XXX and program L2C_CTL on CN6XXX with the values the vendor SDK uses there (rsp_arb_mode 1, xmc_arb_mode 0), which is the same trade the existing code makes for the older parts: keep IO blocks from being starved under high L2 load.

Assisted-by Claude:claude-opus-5

---

arch/mips/cavium-octeon/executive/cvmx-helper.c +15 -5
@@ -1014,7 +1014,6 @@ int cvmx_helper_initialize_packet_io_global(void)
 {
 	int result = 0;
 	int interface;
-	union cvmx_l2c_cfg l2c_cfg;
 	const int num_interfaces = cvmx_helper_get_number_of_interfaces();
 
 	/*
@@ -1029,10 +1028,21 @@ int cvmx_helper_initialize_packet_io_global(void)
 	 * to the cores. This avoids conditions where IO blocks might
 	 * be starved under very high L2 loads.
 	 */
-	l2c_cfg.u64 = cvmx_read_csr(CVMX_L2C_CFG);
-	l2c_cfg.s.lrf_arb_mode = 0;
-	l2c_cfg.s.rfb_arb_mode = 0;
-	cvmx_write_csr(CVMX_L2C_CFG, l2c_cfg.u64);
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
+		union cvmx_l2c_ctl l2c_ctl;
+
+		l2c_ctl.u64 = cvmx_read_csr(CVMX_L2C_CTL);
+		l2c_ctl.s.rsp_arb_mode = 1;
+		l2c_ctl.s.xmc_arb_mode = 0;
+		cvmx_write_csr(CVMX_L2C_CTL, l2c_ctl.u64);
+	} else {
+		union cvmx_l2c_cfg l2c_cfg;
+
+		l2c_cfg.u64 = cvmx_read_csr(CVMX_L2C_CFG);
+		l2c_cfg.s.lrf_arb_mode = 0;
+		l2c_cfg.s.rfb_arb_mode = 0;
+		cvmx_write_csr(CVMX_L2C_CFG, l2c_cfg.u64);
+	}
 
 	cvmx_pko_initialize_global();
 	for (interface = 0; interface < num_interfaces; interface++) {
Signature · 1 line
2.47.0

applied to mips-next

Thomas.

Signature · 2 lines
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]
#7 lore raw

PATCH 5/5 MIPS: OCTEON: require the Core-14449 workaround only on CN63XX pass 1

In reply to Orgad Shaneh

prom_free_prom_memory() probes the hint field of a compiler-emitted PREF and panics when it is not 28, which is what -Wa,-mfix-cn63xxp1 (CONFIG_CAVIUM_CN63XXP1) rewrites every prefetch in the kernel to. The test is gated on OCTEON_IS_MODEL(OCTEON_CN6XXX), so it fires on every OCTEON II part, including the pass 2 chips that do not need the workaround:

  Kernel panic - not syncing: OCTEON II DCache prefetch workaround not
  in place (cfa00004).
  Please build kernel with proper options (CONFIG_CAVIUM_CN63XXP1).

on a CN6335-AAP pass 2.2.

Core-14449 is a CN63XX pass 1.x erratum, and the vendor's own runtime check says so: it errors out only for OCTEON_CN63XX_PASS1_X without the fix, and otherwise merely warns about lost performance when fixed code runs on a part that is not a CN63XX. Narrow the check to CN63XX pass 1.x to match.

The alternative - enabling CONFIG_CAVIUM_CN63XXP1 to satisfy the check - is not free: the assembler flag rewrites every prefetch in the kernel (77 of them in this build) to pref 0x1c, which is worth avoiding on an in-order core whose receive path is dominated by memcpy.

uasm_i_pref() keeps its OCTEON_CN6XXX condition: it only touches the handful of runtime-generated TLB handlers, where the cost does not matter.

Assisted-by Claude:claude-opus-5

---

arch/mips/cavium-octeon/setup.c +1 -1
@@ -1127,7 +1127,7 @@ EXPORT_SYMBOL(prom_putchar);
 
 void __init prom_free_prom_memory(void)
 {
-	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN63XX_PASS1_X)) {
 		/* Check for presence of Core-14449 fix.  */
 		u32 insn;
 		u32 *foo;
Signature · 1 line
2.47.0
maintainer Applied
#8 lore raw

Re: [PATCH 5/5] MIPS: OCTEON: require the Core-14449 workaround only on CN63XX pass 1

Quoted 45 lines · Orgad Shaneh

prom_free_prom_memory() probes the hint field of a compiler-emitted PREF and panics when it is not 28, which is what -Wa,-mfix-cn63xxp1 (CONFIG_CAVIUM_CN63XXP1) rewrites every prefetch in the kernel to. The test is gated on OCTEON_IS_MODEL(OCTEON_CN6XXX), so it fires on every OCTEON II part, including the pass 2 chips that do not need the workaround:

  Kernel panic - not syncing: OCTEON II DCache prefetch workaround not
  in place (cfa00004).
  Please build kernel with proper options (CONFIG_CAVIUM_CN63XXP1).

on a CN6335-AAP pass 2.2.

Core-14449 is a CN63XX pass 1.x erratum, and the vendor's own runtime check says so: it errors out only for OCTEON_CN63XX_PASS1_X without the fix, and otherwise merely warns about lost performance when fixed code runs on a part that is not a CN63XX. Narrow the check to CN63XX pass 1.x to match.

The alternative - enabling CONFIG_CAVIUM_CN63XXP1 to satisfy the check - is not free: the assembler flag rewrites every prefetch in the kernel (77 of them in this build) to pref 0x1c, which is worth avoiding on an in-order core whose receive path is dominated by memcpy.

uasm_i_pref() keeps its OCTEON_CN6XXX condition: it only touches the handful of runtime-generated TLB handlers, where the cost does not matter.

Assisted-by Claude:claude-opus-5

---

arch/mips/cavium-octeon/setup.c +1 -1
@@ -1127,7 +1127,7 @@ EXPORT_SYMBOL(prom_putchar);
 
 void __init prom_free_prom_memory(void)
 {
-	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN63XX_PASS1_X)) {
 		/* Check for presence of Core-14449 fix.  */
 		u32 insn;
 		u32 *foo;
Signature · 1 line
2.47.0

applied to mips-next

Thomas.

Signature · 2 lines
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]
maintainer Applied
#9 lore raw
In reply to Orgad Shaneh
Quoted 128 lines · Orgad Shaneh

The L2 helpers pick between the OCTEON II and the CN3XXX/CN5XXX register model with OCTEON_IS_MODEL(OCTEON_CN63XX), which matches the CN63XX only. Every other OCTEON II part - CN66XX, CN68XX, CN61XX - takes the CN3XXX/CN5XXX branch and touches registers that do not exist on it: L2C_DBG, L2C_LCKBASE, L2C_LCKOFF and L2C_SPAR4 instead of the LCKL2 cache operation, the L2C_WPAR_{PPX,IOBX} way-partitioning registers and L2C_TADX_TAG.

On a CN6635 board cvmx_l2c_lock_line(), called from prom_init() under CONFIG_CAVIUM_OCTEON_LOCK_L2 to lock the TLB refill and exception vectors, therefore went through L2C_DBG/L2C_LCKBASE. Whatever those writes hit inside the L2 controller left it silently dropping stores to DRAM above 512 MB: a probe that wrote 0x20400000, 0x40000000, 0x80000000 and 0xe0000000 in prom_init read its values back before the lock block and the old contents right after it, on CCA 0, 2 and 3 alike, while writes below 256 MB and every u-boot "mw" stuck. Since the cvmx bootmem free list keeps its nodes inside the free blocks themselves, the kernel's first split above 0x20000000 was lost, the walk of the free list stopped there, and the board came up with 248 MB of its 8 GB.

Match the whole CN6XXX family, which is what these register accesses are keyed to. The CN63XX keeps exactly the same path.

The geometry helpers in this file need the same treatment but not the same condition; they are handled in the next patch.

Assisted-by Claude:claude-opus-5

---

arch/mips/cavium-octeon/executive/cvmx-l2c.c +11 -11
@@ -54,7 +54,7 @@ int cvmx_l2c_get_core_way_partition(uint32_t core)
 	if (core >= cvmx_octeon_num_cores())
 		return -1;
 
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX))
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX))
 		return cvmx_read_csr(CVMX_L2C_WPAR_PPX(core)) & 0xffff;
 
 	/*
@@ -91,14 +91,14 @@ int cvmx_l2c_set_core_way_partition(uint32_t core, uint32_t mask)
 	mask &= valid_mask;
 
 	/* A UMSK setting which blocks all L2C Ways is an error on some chips */
-	if (mask == valid_mask && !OCTEON_IS_MODEL(OCTEON_CN63XX))
+	if (mask == valid_mask && !OCTEON_IS_MODEL(OCTEON_CN6XXX))
 		return -1;
 
 	/* Validate the core number */
 	if (core >= cvmx_octeon_num_cores())
 		return -1;
 
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
 		cvmx_write_csr(CVMX_L2C_WPAR_PPX(core), mask);
 		return 0;
 	}
@@ -147,10 +147,10 @@ int cvmx_l2c_set_hw_way_partition(uint32_t mask)
 	mask &= valid_mask;
 
 	/* A UMSK setting which blocks all L2C Ways is an error on some chips */
-	if (mask == valid_mask	&& !OCTEON_IS_MODEL(OCTEON_CN63XX))
+	if (mask == valid_mask	&& !OCTEON_IS_MODEL(OCTEON_CN6XXX))
 		return -1;
 
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX))
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX))
 		cvmx_write_csr(CVMX_L2C_WPAR_IOBX(0), mask);
 	else
 		cvmx_write_csr(CVMX_L2C_SPAR4,
@@ -160,7 +160,7 @@ int cvmx_l2c_set_hw_way_partition(uint32_t mask)
 
 int cvmx_l2c_get_hw_way_partition(void)
 {
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX))
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX))
 		return cvmx_read_csr(CVMX_L2C_WPAR_IOBX(0)) & 0xffff;
 	else
 		return cvmx_read_csr(CVMX_L2C_SPAR4) & (0xFF);
@@ -313,7 +313,7 @@ static void fault_in(uint64_t addr, int len)
 
 int cvmx_l2c_lock_line(uint64_t addr)
 {
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
 		int shift = CVMX_L2C_TAG_ADDR_ALIAS_SHIFT;
 		uint64_t assoc = cvmx_l2c_get_num_assoc();
 		uint64_t tag = addr >> shift;
@@ -460,7 +460,7 @@ void cvmx_l2c_flush(void)
 int cvmx_l2c_unlock_line(uint64_t address)
 {
 
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
 		int assoc;
 		union cvmx_l2c_tag tag;
 		uint32_t tag_addr;
@@ -661,7 +661,7 @@ union cvmx_l2c_tag cvmx_l2c_get_tag(uint32_t association, uint32_t index)
 			     (int)index, cvmx_l2c_get_num_sets());
 		return tag;
 	}
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
 		union cvmx_l2c_tadx_tag l2c_tadx_tag;
 		uint64_t address = CVMX_ADD_SEG(CVMX_MIPS_SPACE_XKPHYS,
 						(association << CVMX_L2C_TAG_ADDR_ALIAS_SHIFT) |
@@ -745,7 +745,7 @@ uint32_t cvmx_l2c_address_to_index(uint64_t addr)
 	}
 
 	if (indxalias) {
-		if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+		if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
 			uint32_t a_14_12 = (idx / (CVMX_L2C_MEMBANK_SELECT_SIZE/(1<<CVMX_L2C_IDX_ADDR_SHIFT))) & 0x7;
 
 			idx ^= idx / cvmx_l2c_get_num_sets();
@@ -878,7 +878,7 @@ void cvmx_l2c_flush_line(uint32_t assoc, uint32_t index)
 		return;
 	}
 
-	if (OCTEON_IS_MODEL(OCTEON_CN63XX)) {
+	if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
 		uint64_t address;
 		/* Create the address based on index and association.
 		 * Bits<20:17> select the way of the cache block involved in
Signature · 1 line
2.47.0

applied to mips-next

Thomas.

Signature · 2 lines
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]