arm: mach-k3: am6_init: Prioritize MSMC traffic over DDR in NAVSS Northbridge
Subsystems (2)
PATCH 1/1 arm: mach-k3: am6_init: Prioritize MSMC traffic over DDR in NAVSS Northbridge
NB0 is bridge to SRAM and NB1 is bridge to DDR.
To ensure that SRAM transfers are not stalled due to delays during DDR refreshes, SRAM traffic should be higher priority (threadmap=2) than DDR traffic (threadmap=0).
This patch does just that.
This is required to fix ICSSG TX lock-ups due to delays in MSMC transfers due to incorrect Northbridge configuration.
--- arch/arm/mach-k3/am6_init.c | 14 ++++++++++++++ arch/arm/mach-k3/include/mach/am6_hardware.h | 7 +++++++ 2 files changed, 21 insertions(+)
arch/arm/mach-k3/am6_init.c +14 -0
@@ -86,6 +86,18 @@ static void store_boot_index_from_rom(void) bootindex = *(u32 *)(CONFIG_SYS_K3_BOOT_PARAM_TABLE_INDEX ); } +static void setup_am654_navss_northbridge (void ) +{ +/* +* NB0 is bridge to SRAM and NB1 is bridge to DDR. +* To ensure that SRAM transfers are not stalled due to +* delays during DDR refreshes, SRAM traffic should be higher +* priority (threadmap=2) than DDR traffic (threadmap=0). +*/ +writel (0x2 ,NAVSS0_NBSS_NB0_CFG_BASE +NAVSS_NBSS_THREADMAP ); +writel (0x0 ,NAVSS0_NBSS_NB1_CFG_BASE +NAVSS_NBSS_THREADMAP ); +} +void board_init_f (ulong dummy) {#if defined(CONFIG_K3_LOAD_SYSFW ) || defined(CONFIG_K3_AM654_DDRSS )
@@ -101,6 +113,8 @@ void board_init_f(ulong dummy)/* Make all control module registers accessible */ ctrl_mmr_unlock (); +setup_am654_navss_northbridge (); +#ifdef CONFIG_CPU_V7R disable_linefill_optimization ();setup_k3_mpu_regions ();
arch/arm/mach-k3/include/mach/am6_hardware.h +7 -0
@@ -47,4 +47,11 @@/* MCU SCRATCHPAD usage */ #define TI_SRAM_SCRATCH_BOARD_EEPROM_START CONFIG_SYS_K3_MCU_SCRATCHPAD_BASE +/* NAVSS Northbridge config */ +#define NAVSS0_NBSS_NB0_CFG_BASE 0x03802000 +#define NAVSS0_NBSS_NB1_CFG_BASE 0x03803000 + +#define NAVSS_NBSS_PID 0x0 +#define NAVSS_NBSS_THREADMAP 0x10 +#endif /* __ASM_ARCH_AM6_HARDWARE_H */
Signature · 2 lines
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
Quoted 59 lines · Roger Quadros
NB0 is bridge to SRAM and NB1 is bridge to DDR.
To ensure that SRAM transfers are not stalled due to delays during DDR refreshes, SRAM traffic should be higher priority (threadmap=2) than DDR traffic (threadmap=0).
This patch does just that.
This is required to fix ICSSG TX lock-ups due to delays in MSMC transfers due to incorrect Northbridge configuration.
Signed-off-by Roger Quadros <[email protected]>Acked-by Andrew F. Davis <[email protected]>Acked-by Tomi Valkeinen <[email protected]>Acked-by Benoit Parrot <[email protected]>--- arch/arm/mach-k3/am6_init.c | 14 ++++++++++++++ arch/arm/mach-k3/include/mach/am6_hardware.h | 7 +++++++ 2 files changed, 21 insertions(+)arch/arm/mach-k3/am6_init.c +14
-0@@ -86,6 +86,18 @@ static void store_boot_index_from_rom(void) bootindex = *(u32 *)(CONFIG_SYS_K3_BOOT_PARAM_TABLE_INDEX ); } +static void setup_am654_navss_northbridge (void ) +{ +/* +* NB0 is bridge to SRAM and NB1 is bridge to DDR. +* To ensure that SRAM transfers are not stalled due to +* delays during DDR refreshes, SRAM traffic should be higher +* priority (threadmap=2) than DDR traffic (threadmap=0). +*/ +writel (0x2 ,NAVSS0_NBSS_NB0_CFG_BASE +NAVSS_NBSS_THREADMAP ); +writel (0x0 ,NAVSS0_NBSS_NB1_CFG_BASE +NAVSS_NBSS_THREADMAP ); +} +void board_init_f (ulong dummy) {#if defined(CONFIG_K3_LOAD_SYSFW ) || defined(CONFIG_K3_AM654_DDRSS )@@ -101,6 +113,8 @@ void board_init_f(ulong dummy)/* Make all control module registers accessible */ ctrl_mmr_unlock (); +setup_am654_navss_northbridge (); +#ifdef CONFIG_CPU_V7R disable_linefill_optimization ();setup_k3_mpu_regions ();arch/arm/mach-k3/include/mach/am6_hardware.h +0
-0@@ -47,4 +47,11 @@/* MCU SCRATCHPAD usage */
arch/arm/mach-k3/include/mach/am6_hardware.h line 57
Quoted 10 lines
arch/arm/mach-k3/include/mach/am6_hardware.h +7
-0#define TI_SRAM_SCRATCH_BOARD_EEPROM_START CONFIG_SYS_K3_MCU_SCRATCHPAD_BASE +/* NAVSS Northbridge config */ +#define NAVSS0_NBSS_NB0_CFG_BASE 0x03802000 +#define NAVSS0_NBSS_NB1_CFG_BASE 0x03803000 + +#define NAVSS_NBSS_PID 0x0 +#define NAVSS_NBSS_THREADMAP 0x10 +#endif /* __ASM_ARCH_AM6_HARDWARE_H */
This was never merged, not even commented on (only apparently rejected in patchwork) - but it is crucial as we now found out:
prueth will quickly stall when these priorities are not applied, at least with SR1.0-based AM65x designs. And you probably know what else could go wrong. Please clarify and merge, possibly reducing the scope to SR1.0 if you can confirm that SR2.0 cannot be affected by design (I can only say this based on few practical experiments here).
If it was good for several TI SDK releases by now, at least something similar should be good for upstream as well, I believe.
Thanks,
Jan
Signature · 2 lines
Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux
Quoted 74 lines · Jan Kiszka
Quoted 59 lines · Roger Quadros
NB0 is bridge to SRAM and NB1 is bridge to DDR.
To ensure that SRAM transfers are not stalled due to delays during DDR refreshes, SRAM traffic should be higher priority (threadmap=2) than DDR traffic (threadmap=0).
This patch does just that.
This is required to fix ICSSG TX lock-ups due to delays in MSMC transfers due to incorrect Northbridge configuration.
Signed-off-by Roger Quadros <[email protected]>Acked-by Andrew F. Davis <[email protected]>Acked-by Tomi Valkeinen <[email protected]>Acked-by Benoit Parrot <[email protected]>--- arch/arm/mach-k3/am6_init.c | 14 ++++++++++++++ arch/arm/mach-k3/include/mach/am6_hardware.h | 7 +++++++ 2 files changed, 21 insertions(+)arch/arm/mach-k3/am6_init.c +14
-0@@ -86,6 +86,18 @@ static void store_boot_index_from_rom(void) bootindex = *(u32 *)(CONFIG_SYS_K3_BOOT_PARAM_TABLE_INDEX ); } +static void setup_am654_navss_northbridge (void ) +{ +/* +* NB0 is bridge to SRAM and NB1 is bridge to DDR. +* To ensure that SRAM transfers are not stalled due to +* delays during DDR refreshes, SRAM traffic should be higher +* priority (threadmap=2) than DDR traffic (threadmap=0). +*/ +writel (0x2 ,NAVSS0_NBSS_NB0_CFG_BASE +NAVSS_NBSS_THREADMAP ); +writel (0x0 ,NAVSS0_NBSS_NB1_CFG_BASE +NAVSS_NBSS_THREADMAP ); +} +void board_init_f (ulong dummy) {#if defined(CONFIG_K3_LOAD_SYSFW ) || defined(CONFIG_K3_AM654_DDRSS )@@ -101,6 +113,8 @@ void board_init_f(ulong dummy)/* Make all control module registers accessible */ ctrl_mmr_unlock (); +setup_am654_navss_northbridge (); +#ifdef CONFIG_CPU_V7R disable_linefill_optimization ();setup_k3_mpu_regions ();arch/arm/mach-k3/include/mach/am6_hardware.h +0
-0@@ -47,4 +47,11 @@/* MCU SCRATCHPAD usage */ arch/arm/mach-k3/include/mach/am6_hardware.h line 57
Quoted 10 lines
arch/arm/mach-k3/include/mach/am6_hardware.h +7
-0#define TI_SRAM_SCRATCH_BOARD_EEPROM_START CONFIG_SYS_K3_MCU_SCRATCHPAD_BASE +/* NAVSS Northbridge config */ +#define NAVSS0_NBSS_NB0_CFG_BASE 0x03802000 +#define NAVSS0_NBSS_NB1_CFG_BASE 0x03803000 + +#define NAVSS_NBSS_PID 0x0 +#define NAVSS_NBSS_THREADMAP 0x10 +#endif /* __ASM_ARCH_AM6_HARDWARE_H */ This was never merged, not even commented on (only apparently rejected in patchwork) - but it is crucial as we now found out:
Quoted 9 lines
prueth will quickly stall when these priorities are not applied, at least with SR1.0-based AM65x designs. And you probably know what else could go wrong. Please clarify and merge, possibly reducing the scope to SR1.0 if you can confirm that SR2.0 cannot be affected by design (I can only say this based on few practical experiments here).
If it was good for several TI SDK releases by now, at least something similar should be good for upstream as well, I believe.
Ping. We need at least some confirmation on what is actually needed. Then, if you do not like to add it to the generic path, it would be easy for us to carry it in the IOT2050 board init only - with the appropriate condition check.
Jan
Signature · 2 lines
Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux
Quoted 81 lines · Jan Kiszka
Quoted 80 lines · Jan Kiszka
Quoted 59 lines · Roger Quadros
NB0 is bridge to SRAM and NB1 is bridge to DDR.
To ensure that SRAM transfers are not stalled due to delays during DDR refreshes, SRAM traffic should be higher priority (threadmap=2) than DDR traffic (threadmap=0).
This patch does just that.
This is required to fix ICSSG TX lock-ups due to delays in MSMC transfers due to incorrect Northbridge configuration.
Signed-off-by Roger Quadros <[email protected]>Acked-by Andrew F. Davis <[email protected]>Acked-by Tomi Valkeinen <[email protected]>Acked-by Benoit Parrot <[email protected]>--- arch/arm/mach-k3/am6_init.c | 14 ++++++++++++++ arch/arm/mach-k3/include/mach/am6_hardware.h | 7 +++++++ 2 files changed, 21 insertions(+)arch/arm/mach-k3/am6_init.c +14
-0@@ -86,6 +86,18 @@ static void store_boot_index_from_rom(void) bootindex = *(u32 *)(CONFIG_SYS_K3_BOOT_PARAM_TABLE_INDEX ); } +static void setup_am654_navss_northbridge (void ) +{ +/* +* NB0 is bridge to SRAM and NB1 is bridge to DDR. +* To ensure that SRAM transfers are not stalled due to +* delays during DDR refreshes, SRAM traffic should be higher +* priority (threadmap=2) than DDR traffic (threadmap=0). +*/ +writel (0x2 ,NAVSS0_NBSS_NB0_CFG_BASE +NAVSS_NBSS_THREADMAP ); +writel (0x0 ,NAVSS0_NBSS_NB1_CFG_BASE +NAVSS_NBSS_THREADMAP ); +} +void board_init_f (ulong dummy) {#if defined(CONFIG_K3_LOAD_SYSFW ) || defined(CONFIG_K3_AM654_DDRSS )@@ -101,6 +113,8 @@ void board_init_f(ulong dummy)/* Make all control module registers accessible */ ctrl_mmr_unlock (); +setup_am654_navss_northbridge (); +#ifdef CONFIG_CPU_V7R disable_linefill_optimization ();setup_k3_mpu_regions ();arch/arm/mach-k3/include/mach/am6_hardware.h +0
-0@@ -47,4 +47,11 @@/* MCU SCRATCHPAD usage */ arch/arm/mach-k3/include/mach/am6_hardware.h line 57
Quoted 10 lines
arch/arm/mach-k3/include/mach/am6_hardware.h +7
-0#define TI_SRAM_SCRATCH_BOARD_EEPROM_START CONFIG_SYS_K3_MCU_SCRATCHPAD_BASE +/* NAVSS Northbridge config */ +#define NAVSS0_NBSS_NB0_CFG_BASE 0x03802000 +#define NAVSS0_NBSS_NB1_CFG_BASE 0x03803000 + +#define NAVSS_NBSS_PID 0x0 +#define NAVSS_NBSS_THREADMAP 0x10 +#endif /* __ASM_ARCH_AM6_HARDWARE_H */ This was never merged, not even commented on (only apparently rejected in patchwork) - but it is crucial as we now found out:
prueth will quickly stall when these priorities are not applied, at least with SR1.0-based AM65x designs. And you probably know what else could go wrong. Please clarify and merge, possibly reducing the scope to SR1.0 if you can confirm that SR2.0 cannot be affected by design (I can only say this based on few practical experiments here).
Quoted 9 lines
Quoted 3 lines
If it was good for several TI SDK releases by now, at least something similar should be good for upstream as well, I believe.
Ping. We need at least some confirmation on what is actually needed. Then, if you do not like to add it to the generic path, it would be easy for us to carry it in the IOT2050 board init only - with the appropriate condition check.
Did some more experiments on SR2 silicon, and while I'm not seeing stalls without this patch there, iperf tests with prueth show high retry rates and, thus, about 10% worse throughput. So it seems to be required for newer silicon as well.
Jan
Signature · 2 lines
Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux
Quoted 91 lines · Jan Kiszka
Quoted 81 lines · Jan Kiszka
Quoted 80 lines · Jan Kiszka
Quoted 59 lines · Roger Quadros
NB0 is bridge to SRAM and NB1 is bridge to DDR.
To ensure that SRAM transfers are not stalled due to delays during DDR refreshes, SRAM traffic should be higher priority (threadmap=2) than DDR traffic (threadmap=0).
This patch does just that.
This is required to fix ICSSG TX lock-ups due to delays in MSMC transfers due to incorrect Northbridge configuration.
Signed-off-by Roger Quadros <[email protected]>Acked-by Andrew F. Davis <[email protected]>Acked-by Tomi Valkeinen <[email protected]>Acked-by Benoit Parrot <[email protected]>--- arch/arm/mach-k3/am6_init.c | 14 ++++++++++++++ arch/arm/mach-k3/include/mach/am6_hardware.h | 7 +++++++ 2 files changed, 21 insertions(+)arch/arm/mach-k3/am6_init.c +14
-0@@ -86,6 +86,18 @@ static void store_boot_index_from_rom(void) bootindex = *(u32 *)(CONFIG_SYS_K3_BOOT_PARAM_TABLE_INDEX ); } +static void setup_am654_navss_northbridge (void ) +{ +/* +* NB0 is bridge to SRAM and NB1 is bridge to DDR. +* To ensure that SRAM transfers are not stalled due to +* delays during DDR refreshes, SRAM traffic should be higher +* priority (threadmap=2) than DDR traffic (threadmap=0). +*/ +writel (0x2 ,NAVSS0_NBSS_NB0_CFG_BASE +NAVSS_NBSS_THREADMAP ); +writel (0x0 ,NAVSS0_NBSS_NB1_CFG_BASE +NAVSS_NBSS_THREADMAP ); +} +void board_init_f (ulong dummy) {#if defined(CONFIG_K3_LOAD_SYSFW ) || defined(CONFIG_K3_AM654_DDRSS )@@ -101,6 +113,8 @@ void board_init_f(ulong dummy)/* Make all control module registers accessible */ ctrl_mmr_unlock (); +setup_am654_navss_northbridge (); +#ifdef CONFIG_CPU_V7R disable_linefill_optimization ();setup_k3_mpu_regions ();arch/arm/mach-k3/include/mach/am6_hardware.h +0
-0@@ -47,4 +47,11 @@/* MCU SCRATCHPAD usage */ arch/arm/mach-k3/include/mach/am6_hardware.h line 57
Quoted 10 lines
arch/arm/mach-k3/include/mach/am6_hardware.h +7
-0#define TI_SRAM_SCRATCH_BOARD_EEPROM_START CONFIG_SYS_K3_MCU_SCRATCHPAD_BASE +/* NAVSS Northbridge config */ +#define NAVSS0_NBSS_NB0_CFG_BASE 0x03802000 +#define NAVSS0_NBSS_NB1_CFG_BASE 0x03803000 + +#define NAVSS_NBSS_PID 0x0 +#define NAVSS_NBSS_THREADMAP 0x10 +#endif /* __ASM_ARCH_AM6_HARDWARE_H */ This was never merged, not even commented on (only apparently rejected in patchwork) - but it is crucial as we now found out:
prueth will quickly stall when these priorities are not applied, at least with SR1.0-based AM65x designs. And you probably know what else could go wrong. Please clarify and merge, possibly reducing the scope to SR1.0 if you can confirm that SR2.0 cannot be affected by design (I can only say this based on few practical experiments here).
Quoted 8 lines
Quoted 3 lines
If it was good for several TI SDK releases by now, at least something similar should be good for upstream as well, I believe.
Ping. We need at least some confirmation on what is actually needed. Then, if you do not like to add it to the generic path, it would be easy for us to carry it in the IOT2050 board init only - with the appropriate condition check.
Quoted 4 lines
Did some more experiments on SR2 silicon, and while I'm not seeing stalls without this patch there, iperf tests with prueth show high retry rates and, thus, about 10% worse throughput. So it seems to be required for newer silicon as well.
At this point, I'd really appreciate it if someone from within TI can chime in here as I expect someone must suspect something here, thanks!
Signature · 1 line
Tom
Quoted 4 lines · Tom Rini
Quoted 3 lines · Jan Kiszka
Quoted 2 lines · Jan Kiszka
Quoted 1 line · Jan Kiszka
On 30.01.20 09:05, Roger Quadros wrote:
[...]
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 6 lines
Quoted 2 lines
#endif /*
__ASM_ARCH_AM6_HARDWARE_H*/This was never merged, not even commented on (only apparently rejected in patchwork) - but it is crucial as we now found out:
Quoted 9 lines
prueth will quickly stall when these priorities are not applied, at least with SR1.0-based AM65x designs. And you probably know what else could go wrong. Please clarify and merge, possibly reducing the scope to SR1.0 if you can confirm that SR2.0 cannot be affected by design (I can only say this based on few practical experiments here).
If it was good for several TI SDK releases by now, at least something similar should be good for upstream as well, I believe.
Ping. We need at least some confirmation on what is actually needed. Then, if you do not like to add it to the generic path, it would be easy for us to carry it in the IOT2050 board init only - with the appropriate condition check.
Quoted 7 lines
Quoted 4 lines
Did some more experiments on SR2 silicon, and while I'm not seeing stalls without this patch there, iperf tests with prueth show high retry rates and, thus, about 10% worse throughput. So it seems to be required for newer silicon as well.
At this point, I'd really appreciate it if someone from within TI can chime in here as I expect someone must suspect something here, thanks!
Here are some internal discussion highlights:
This is critical to provide deterministic access latency to MSMC from ICSSG, it applies to all AM65 silicon revisions.
- is this errata or use-case specific settings?
Incorrect reset values, should've been static and setup as in the patch.
- what SoCs are affected (am64, J721e)?
AM65x only
- if this configuration applied will/might it affect on other use-cases (mostly multi-media related)?
No. This will only effect traffic taking advantage of the 2 ports to MSMC and mapping to independent threads inside MSMC, one for SRAM and one for DDR bound traffic.
In short, the patch is valid, we should be able to apply this consistently on AM65x SoCs independent of the platform.
Hope this helps?
Signature · 3 lines
Regards, Nishanth Menon Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3 1A34 DDB5 849D 1736 249D
Quoted 55 lines · Nishanth Menon
Quoted 4 lines · Tom Rini
Quoted 3 lines · Jan Kiszka
Quoted 2 lines · Jan Kiszka
Quoted 1 line · Jan Kiszka
On 30.01.20 09:05, Roger Quadros wrote:
[...]
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 6 lines
Quoted 2 lines
#endif /*
__ASM_ARCH_AM6_HARDWARE_H*/This was never merged, not even commented on (only apparently rejected in patchwork) - but it is crucial as we now found out:
Quoted 9 lines
prueth will quickly stall when these priorities are not applied, at least with SR1.0-based AM65x designs. And you probably know what else could go wrong. Please clarify and merge, possibly reducing the scope to SR1.0 if you can confirm that SR2.0 cannot be affected by design (I can only say this based on few practical experiments here).
If it was good for several TI SDK releases by now, at least something similar should be good for upstream as well, I believe.
Ping. We need at least some confirmation on what is actually needed. Then, if you do not like to add it to the generic path, it would be easy for us to carry it in the IOT2050 board init only - with the appropriate condition check.
Quoted 7 lines
Quoted 4 lines
Did some more experiments on SR2 silicon, and while I'm not seeing stalls without this patch there, iperf tests with prueth show high retry rates and, thus, about 10% worse throughput. So it seems to be required for newer silicon as well.
At this point, I'd really appreciate it if someone from within TI can chime in here as I expect someone must suspect something here, thanks!
Here are some internal discussion highlights:
This is critical to provide deterministic access latency to MSMC from ICSSG, it applies to all AM65 silicon revisions.
- is this errata or use-case specific settings?
Incorrect reset values, should've been static and setup as in the patch.- what SoCs are affected (am64, J721e)?
AM65x only- if this configuration applied will/might it affect on other use-cases (mostly multi-media related)?
No. This will only effect traffic taking advantage of the 2 ports to MSMC and mapping to independent threads inside MSMC, one for SRAM and one for DDR bound traffic.
Quoted 6 lines
In short, the patch is valid, we should be able to apply this consistently on AM65x SoCs independent of the platform.
Hope this helps?
Thanks a lot!
Who will augment its description and push this again? I have a rebased version [1] at least, slightly polished, but my commit log is possibly not optimal yet.
Jan
[1]
https://github.com/siemens/u-boot/commit/2cf7a2fc14b82b12aa06152dbecc694691801fee
Signature · 2 lines
Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux
On 12:24-20210908, Jan Kiszka wrote:
[...]
Quoted 5 lines
Thanks a lot!
Who will augment its description and push this again? I have a rebased version [1] at least, slightly polished, but my commit log is possibly not optimal yet.
Done. https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/
Signature · 3 lines
Regards, Nishanth Menon Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3 1A34 DDB5 849D 1736 249D