bootstage: Fix unstash and sorting of records
cover bootstage: Fix unstash and sorting of records
This series fixes two issues with bootstage observed trying to measure boot times on my Rockchip aarch64 boards.
First patch fixes a regression in v2024.10-rc1 where bootstage records from SPL no longer is unstashed.
Second patch change to not sort bootstage records in order to show correct elapsed time when TF-A reset the timer counter.
Tested on a few Rockchip boards with:
CONFIG_BOOTSTAGE=y
CONFIG_CMD_BOOTSTAGE=y
CONFIG_BOOTSTAGE_RECORD_COUNT=20
CONFIG_SPL_BOOTSTAGE=y
CONFIG_SPL_BOOTSTAGE_RECORD_COUNT=20
CONFIG_BOOTSTAGE_STASH_ADDR=0x3800000
CONFIG_BOOTSTAGE_STASH_SIZE=0x10000
CONFIG_BOOTSTAGE_STASH=y
CONFIG_BOOTSTAGE_REPORT=y
CONFIG_BOOTDELAY=-1
Jonas Karlman (2):
bootstage: Fix unstash of records from SPL
bootstage: Do not sort records
common/board_f.c | 2 +- common/bootstage.c | 12 ++---------- 2 files changed, 3 insertions(+), 11 deletions(-)
Signature · 1 line
2.45.2
PATCH 1/2 bootstage: Fix unstash of records from SPL
The commit b81e31a1e6c5 ("bootstash: Do not provide a default address for all") changed a bootstage unstash call to bootstage stash, this has resulted in bootstage records stashed in SPL no longer get unstaged in U-Boot proper. Fix this by changing back to a unstage call.
--- common/board_f.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
common/board_f.c +1 -1
@@ -804,7 +804,7 @@ static int initf_bootstage(void)if (ret)return ret;if (from_spl) { - ret =bootstage_stash_default (); + ret =bootstage_unstash_default ();if (ret && ret != -ENOENT ) {debug ("Failed to unstash bootstage: err=%d\n" , ret);return ret;
Signature · 1 line
2.45.2
Re: [PATCH 1/2] bootstage: Fix unstash of records from SPL
Quoted 7 lines · Jonas Karlman
The commit
b81e31a1e6c5("bootstash: Do not provide a default address for all") changed a bootstage unstash call to bootstage stash, this has resulted in bootstage records stashed in SPL no longer get unstaged in U-Boot proper. Fix this by changing back to a unstage call.Fixes b81e31a1e6c5 ("bootstash: Do not provide a default address for all")Signed-off-by Jonas Karlman <[email protected]>
Whoops, thanks!
Signature · 1 line
Tom
Simon Glass
Reviewed-byRe: [PATCH 1/2] bootstage: Fix unstash of records from SPL
Quoted 8 lines · Tom Rini
Quoted 5 lines · Jonas Karlman
The commit
b81e31a1e6c5("bootstash: Do not provide a default address for all") changed a bootstage unstash call to bootstage stash, this has resulted in bootstage records stashed in SPL no longer get unstaged in U-Boot proper. Fix this by changing back to a unstage call.
Quoted 6 lines
Quoted 2 lines
Fixes b81e31a1e6c5 ("bootstash: Do not provide a default address for all")Signed-off-by Jonas Karlman <[email protected]>Whoops, thanks!
Reviewed-by Tom Rini <[email protected]>
Thanks.
How about a test for sandbox_spl to check this? I see that CONFIG_SPL_BOOTSTAGE is not enabled there.
There is a vaguely similar test in test/py/tests/test_handoff.py
Regards,
SImon
Re: [PATCH 1/2] bootstage: Fix unstash of records from SPL
In reply to Jonas KarlmanQuoted 6 lines · Jonas Karlman
The commit
b81e31a1e6c5("bootstash: Do not provide a default address for all") changed a bootstage unstash call to bootstage stash, this has resulted in bootstage records stashed in SPL no longer get unstaged in U-Boot proper. Fix this by changing back to a unstage call.
Applied to u-boot/next, thanks!
Signature · 1 line
Tom
Re: [PATCH 1/2] bootstage: Fix unstash of records from SPL
Hi Tom,
Quoted 11 lines · Tom Rini
Quoted 6 lines · Jonas Karlman
The commit
b81e31a1e6c5("bootstash: Do not provide a default address for all") changed a bootstage unstash call to bootstage stash, this has resulted in bootstage records stashed in SPL no longer get unstaged in U-Boot proper. Fix this by changing back to a unstage call.Applied to u-boot/next, thanks!
I expected this patch to go to master as it fixes a regression introduced in v2024.10-rc1.
Regards,
Jonas
Re: [PATCH 1/2] bootstage: Fix unstash of records from SPL
Quoted 12 lines · Jonas Karlman
Hi Tom,
Quoted 9 lines · Tom Rini
Quoted 6 lines · Jonas Karlman
The commit
b81e31a1e6c5("bootstash: Do not provide a default address for all") changed a bootstage unstash call to bootstage stash, this has resulted in bootstage records stashed in SPL no longer get unstaged in U-Boot proper. Fix this by changing back to a unstage call.
Quoted 5 lines
Quoted 2 lines
Applied to u-boot/next, thanks!
I expected this patch to go to master as it fixes a regression introduced in v2024.10-rc1.
OK, good point. I'll push it to master when I have other stuff as well, I've cherry-picked it now.
Signature · 1 line
Tom
PATCH 2/2 bootstage: Do not sort records
In reply to Jonas KarlmanThe timer counter on Rockchip SoCs may be reset in TF-A, this may cause the bootstage records to be printed out of order and with an incorrect elapsed time.
Fix this by not sorting the bootstage records.
Before on a Radxa ZERO 3W (RK3566) board:
=> bootstage report
Timer summary in microseconds (12 records):
Mark Elapsed Stage
0 0 reset
7,436 7,436 board_init_f
164,826 157,390 SPL
375,392 210,566 end phase
423,909 48,517 board_init_r
472,973 49,064 eth_common_init
476,848 3,875 main_loop
477,003 155 cli_loopAccumulated time:
7,181 of_live
14,739 dm_spl
15,029 dm_r
315,150 dm_fWith this the records can be printed in chronological order when the counter is reset and SPL and board_init_r records show correct elapsed time.
=> bootstage report
Timer summary in microseconds (12 records):
Mark Elapsed Stage
0 0 reset
164,437 164,437 SPL
375,023 210,586 end phase
7,437 7,437 board_init_f
424,390 416,953 board_init_r
473,515 49,125 eth_common_init
477,402 3,887 main_loop
477,571 169 cli_loopAccumulated time:
14,734 dm_spl
315,646 dm_f
7,339 of_live
14,977 dm_rFor the tested board external TPL and BROM take ~164 ms to initialize DRAM and load SPL, SPL take ~210ms to load images from FIT and U-Boot proper take ~477ms to reach cli prompt.
--- common/bootstage.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)
common/bootstage.c +2 -10
@@ -249,6 +249,8 @@ static uint32_t print_time_record(struct bootstage_record *rec, uint32_t prev)printf ("%11s" ,"" );print_grouped_ull (rec->time_us,BOOTSTAGE_DIGITS ); }else { +if (prev > rec->time_us) + prev =0 ;print_grouped_ull (rec->time_us,BOOTSTAGE_DIGITS );print_grouped_ull (rec->time_us - prev,BOOTSTAGE_DIGITS ); }
@@ -257,13 +259,6 @@ static uint32_t print_time_record(struct bootstage_record *rec, uint32_t prev)return rec->time_us; } -static int h_compare_record (const void *r1,const void *r2) -{ -const struct bootstage_record *rec1 = r1, *rec2 = r2; - -return rec1->time_us > rec2->time_us ?1 :-1 ; -} -#ifdef CONFIG_OF_LIBFDT /** * Add all bootstage timings to a device tree.
@@ -342,9 +337,6 @@ void bootstage_report(void) prev =print_time_record (rec,0 ); -/* Sort records by increasing time */ -qsort (data->record, data->rec_count,sizeof (*rec), h_compare_record); -for (i =1 , rec++; i < data->rec_count; i++, rec++) {if (rec->id && !rec->start_us) prev =print_time_record (rec, prev);
Signature · 1 line
2.45.2
Re: [PATCH 2/2] bootstage: Do not sort records
Hi Jonas,
Quoted 4 lines · Jonas Karlman
The timer counter on Rockchip SoCs may be reset in TF-A, this may cause the bootstage records to be printed out of order and with an incorrect elapsed time.
Eek can we just fix TF-A? That seems like a bug.
Quoted 92 lines
Fix this by not sorting the bootstage records.
Before on a Radxa ZERO 3W (RK3566) board:
=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 7,436 7,436 board_init_f 164,826 157,390 SPL 375,392 210,566 end phase 423,909 48,517 board_init_r 472,973 49,064 eth_common_init 476,848 3,875 main_loop 477,003 155 cli_loopAccumulated time:
7,181 of_live 14,739 dm_spl 15,029 dm_r 315,150 dm_fWith this the records can be printed in chronological order when the counter is reset and SPL and
board_init_rrecords show correct elapsed time.=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 164,437 164,437 SPL 375,023 210,586 end phase 7,437 7,437 board_init_f 424,390 416,953 board_init_r 473,515 49,125 eth_common_init 477,402 3,887 main_loop 477,571 169 cli_loopAccumulated time:
14,734 dm_spl 315,646 dm_f 7,339 of_live 14,977 dm_rFor the tested board external TPL and BROM take ~164 ms to initialize DRAM and load SPL, SPL take ~210ms to load images from FIT and U-Boot proper take ~477ms to reach cli prompt.
Signed-off-by Jonas Karlman <[email protected]>--- common/bootstage.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)common/bootstage.c +2
-11@@ -249,6 +249,8 @@ static uint32_t print_time_record(struct bootstage_record *rec, uint32_t prev)printf ("%11s" ,"" );print_grouped_ull (rec->time_us,BOOTSTAGE_DIGITS ); }else { +if (prev > rec->time_us) + prev =0 ;print_grouped_ull (rec->time_us,BOOTSTAGE_DIGITS );print_grouped_ull (rec->time_us - prev,BOOTSTAGE_DIGITS ); }@@ -257,13 +259,6 @@ static uint32_t print_time_record(struct bootstage_record *rec, uint32_t prev)return rec->time_us; } -static int h_compare_record (const void *r1,const void *r2) -{ -const struct bootstage_record *rec1 = r1, *rec2 = r2; - -return rec1->time_us > rec2->time_us ?1 :-1 ; -} -#ifdef CONFIG_OF_LIBFDT /** * Add all bootstage timings to a device tree. @@ -342,9 +337,6 @@ void bootstage_report(void) prev =print_time_record (rec,0 ); -/* Sort records by increasing time */ -qsort (data->record, data->rec_count,sizeof (*rec), h_compare_record); -for (i =1 , rec++; i < data->rec_count; i++, rec++) {if (rec->id && !rec->start_us) prev =print_time_record (rec, prev); --
Quoted 2 lines
2.45.2
Regards,
SImon
Re: [PATCH 2/2] bootstage: Do not sort records
Hi Jonas,
Quoted 65 lines · Simon Glass
Hi Jonas,
Quoted 4 lines · Jonas Karlman
The timer counter on Rockchip SoCs may be reset in TF-A, this may cause the bootstage records to be printed out of order and with an incorrect elapsed time.
Eek can we just fix TF-A? That seems like a bug.
Quoted 54 lines
Fix this by not sorting the bootstage records.
Before on a Radxa ZERO 3W (RK3566) board:
=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 7,436 7,436 board_init_f 164,826 157,390 SPL 375,392 210,566 end phase 423,909 48,517 board_init_r 472,973 49,064 eth_common_init 476,848 3,875 main_loop 477,003 155 cli_loopAccumulated time:
7,181 of_live 14,739 dm_spl 15,029 dm_r 315,150 dm_fWith this the records can be printed in chronological order when the counter is reset and SPL and
board_init_rrecords show correct elapsed time.=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 164,437 164,437 SPL 375,023 210,586 end phase 7,437 7,437 board_init_f 424,390 416,953 board_init_r 473,515 49,125 eth_common_init 477,402 3,887 main_loop 477,571 169 cli_loopAccumulated time:
14,734 dm_spl 315,646 dm_f 7,339 of_live 14,977 dm_rFor the tested board external TPL and BROM take ~164 ms to initialize DRAM and load SPL, SPL take ~210ms to load images from FIT and U-Boot proper take ~477ms to reach cli prompt.
Signed-off-by Jonas Karlman <[email protected]>--- common/bootstage.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)
To put this more strongly, this breaks the timing output on other boards.
At the very least, this should be Kconfig option just enabled for boards with a broken ATF.
Regards,
Simon
Re: [PATCH 2/2] bootstage: Do not sort records
Hi Simon,
Quoted 13 lines · Simon Glass
Hi Jonas,
Quoted 10 lines · Simon Glass
Hi Jonas,
Quoted 4 lines · Jonas Karlman
The timer counter on Rockchip SoCs may be reset in TF-A, this may cause the bootstage records to be printed out of order and with an incorrect elapsed time.
Eek can we just fix TF-A? That seems like a bug.
TF-A is vendor blob, so we have no control of it.
Quoted 50 lines
Quoted 50 lines
Quoted 50 lines
Fix this by not sorting the bootstage records.
Before on a Radxa ZERO 3W (RK3566) board:
=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 7,436 7,436 board_init_f 164,826 157,390 SPL 375,392 210,566 end phase 423,909 48,517 board_init_r 472,973 49,064 eth_common_init 476,848 3,875 main_loop 477,003 155 cli_loopAccumulated time:
7,181 of_live 14,739 dm_spl 15,029 dm_r 315,150 dm_fWith this the records can be printed in chronological order when the counter is reset and SPL and
board_init_rrecords show correct elapsed time.=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 164,437 164,437 SPL 375,023 210,586 end phase 7,437 7,437 board_init_f 424,390 416,953 board_init_r 473,515 49,125 eth_common_init 477,402 3,887 main_loop 477,571 169 cli_loopAccumulated time:
14,734 dm_spl 315,646 dm_f 7,339 of_live 14,977 dm_rFor the tested board external TPL and BROM take ~164 ms to initialize DRAM and load SPL, SPL take ~210ms to load images from FIT and U-Boot proper take ~477ms to reach cli prompt.
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Signed-off-by Jonas Karlman <[email protected]>--- common/bootstage.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)To put this more strongly, this breaks the timing output on other boards.
Please elaborate, I cannot understand why not sorting would break output on other boards, are other boards adding records out of order?
Do you have an example of how this change break other boards?
Quoted 3 lines
At the very least, this should be
Kconfigoption just enabled for boards with a broken ATF.
I can add a Kconfig option, but would like to understand why other/any board would need to sort the records in the first place.
Regards,
Jonas
Quoted 3 lines
Regards,
Simon
Re: [PATCH 2/2] bootstage: Do not sort records
Quoted 80 lines · Jonas Karlman
Hi Simon,
Quoted 13 lines · Simon Glass
Hi Jonas,
Quoted 10 lines · Simon Glass
Hi Jonas,
Quoted 4 lines · Jonas Karlman
The timer counter on Rockchip SoCs may be reset in TF-A, this may cause the bootstage records to be printed out of order and with an incorrect elapsed time.
Eek can we just fix TF-A? That seems like a bug.
TF-A is vendor blob, so we have no control of it.
Quoted 50 lines
Quoted 50 lines
Quoted 50 lines
Fix this by not sorting the bootstage records.
Before on a Radxa ZERO 3W (RK3566) board:
=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 7,436 7,436 board_init_f 164,826 157,390 SPL 375,392 210,566 end phase 423,909 48,517 board_init_r 472,973 49,064 eth_common_init 476,848 3,875 main_loop 477,003 155 cli_loopAccumulated time:
7,181 of_live 14,739 dm_spl 15,029 dm_r 315,150 dm_fWith this the records can be printed in chronological order when the counter is reset and SPL and
board_init_rrecords show correct elapsed time.=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 164,437 164,437 SPL 375,023 210,586 end phase 7,437 7,437 board_init_f 424,390 416,953 board_init_r 473,515 49,125 eth_common_init 477,402 3,887 main_loop 477,571 169 cli_loopAccumulated time:
14,734 dm_spl 315,646 dm_f 7,339 of_live 14,977 dm_rFor the tested board external TPL and BROM take ~164 ms to initialize DRAM and load SPL, SPL take ~210ms to load images from FIT and U-Boot proper take ~477ms to reach cli prompt.
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Signed-off-by Jonas Karlman <[email protected]>--- common/bootstage.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)To put this more strongly, this breaks the timing output on other boards.
Please elaborate, I cannot understand why not sorting would break output on other boards, are other boards adding records out of order?
Quoted 8 lines
Do you have an example of how this change break other boards?
Quoted 3 lines
At the very least, this should be
Kconfigoption just enabled for boards with a broken ATF.I can add a
Kconfigoption, but would like to understand why other/any board would need to sort the records in the first place.
This got me to re-check everything and see I hadn't finished configuring bootstage for pytest for Pi in my lab. And when building with:CONFIG_BOOTSTAGE_STASH_ADDR=0x02400000CONFIG_BOOTSTAGE=yCONFIG_BOOTSTAGE_STASH=yCONFIG_CMD_BOOTSTAGE=y
And this in the boardenv py file:
env__bootstage_cmd_file = {
'addr': 0x2400000,
'size': 0x1000,
'bootstage_magic_addr': 0xb00757a3,
}All three tests pass, with both parts of the series applied. Of course Pi is not a great example as there's not TF-A in the loop here and that's part of the problem equation.
And also, JFYI:
=============================== warnings summary ===============================
tests/test_bootstage.py::test_bootstage_stash
/home/trini/labgrid-venv/lib/python3.11/site-packages/_pytest/python.py:163: PytestReturnNotNoneWarning: Expected None, but tests/test_bootstage.py::test_bootstage_stash returned (37748736, 4096), which will be an error in a future version of pytest. Did you mean to use assert instead of return?
warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
Signature · 1 line
Tom
Re: [PATCH 2/2] bootstage: Do not sort records
In reply to Jonas KarlmanHi Jonas,
Quoted 12 lines · Jonas Karlman
Hi Simon,
Quoted 8 lines · Simon Glass
Hi Jonas,
Quoted 5 lines · Simon Glass
Hi Jonas,
Quoted 1 line · Jonas Karlman
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
The timer counter on Rockchip SoCs may be reset in TF-A, this may cause the bootstage records to be printed out of order and with an incorrect elapsed time.
Eek can we just fix TF-A? That seems like a bug.
TF-A is vendor blob, so we have no control of it.
Can we replace it, or is it one of the ones that has not been released?
Can you file a bug with the vendor?
Quoted 57 lines
Quoted 57 lines
Quoted 56 lines
Quoted 56 lines
Fix this by not sorting the bootstage records.
Before on a Radxa ZERO 3W (RK3566) board:
=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 7,436 7,436 board_init_f 164,826 157,390 SPL 375,392 210,566 end phase 423,909 48,517 board_init_r 472,973 49,064 eth_common_init 476,848 3,875 main_loop 477,003 155 cli_loopAccumulated time:
7,181 of_live 14,739 dm_spl 15,029 dm_r 315,150 dm_fWith this the records can be printed in chronological order when the counter is reset and SPL and
board_init_rrecords show correct elapsed time.=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 164,437 164,437 SPL 375,023 210,586 end phase 7,437 7,437 board_init_f 424,390 416,953 board_init_r 473,515 49,125 eth_common_init 477,402 3,887 main_loop 477,571 169 cli_loopAccumulated time:
14,734 dm_spl 315,646 dm_f 7,339 of_live 14,977 dm_rFor the tested board external TPL and BROM take ~164 ms to initialize DRAM and load SPL, SPL take ~210ms to load images from FIT and U-Boot proper take ~477ms to reach cli prompt.
Signed-off-by Jonas Karlman <[email protected]>--- common/bootstage.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)
Quoted 6 lines
Quoted 1 line
To put this more strongly, this breaks the timing output on other boards.
Please elaborate, I cannot understand why not sorting would break output on other boards, are other boards adding records out of order?
Do you have an example of how this change break other boards?
Oh sure. The original reason was that the array was ordered by ID. That was changed in [1], which I forgot about.
The other bit is for coreboot (or potentially other prior phases) we receive timestamps as a block and add them into U-Boot. We want these to appear in the correct order with respect to the ones already adding to U-Boot.
Sadly coreboot uses its own toolchain and I seem to have deleted it, but once it finishes building the world I will give it a try.
Quoted 7 lines
Quoted 4 lines
At the very least, this should be
Kconfigoption just enabled for boards with a broken ATF.I can add a
Kconfigoption, but would like to understand why other/any board would need to sort the records in the first place.
OK.
Regards,
Simon
[1] 03ecac31498 bootstage: Use rec_count as the array index
Re: [PATCH 2/2] bootstage: Do not sort records
Hi again Jonas,
Quoted 95 lines · Simon Glass
Hi Jonas,
Quoted 12 lines · Jonas Karlman
Hi Simon,
Quoted 8 lines · Simon Glass
Hi Jonas,
Quoted 5 lines · Simon Glass
Hi Jonas,
Quoted 1 line · Jonas Karlman
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
The timer counter on Rockchip SoCs may be reset in TF-A, this may cause the bootstage records to be printed out of order and with an incorrect elapsed time.
Eek can we just fix TF-A? That seems like a bug.
TF-A is vendor blob, so we have no control of it.
Can we replace it, or is it one of the ones that has not been released?
Can you file a bug with the vendor?
Quoted 57 lines
Quoted 57 lines
Quoted 56 lines
Quoted 56 lines
Fix this by not sorting the bootstage records.
Before on a Radxa ZERO 3W (RK3566) board:
=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 7,436 7,436 board_init_f 164,826 157,390 SPL 375,392 210,566 end phase 423,909 48,517 board_init_r 472,973 49,064 eth_common_init 476,848 3,875 main_loop 477,003 155 cli_loopAccumulated time:
7,181 of_live 14,739 dm_spl 15,029 dm_r 315,150 dm_fWith this the records can be printed in chronological order when the counter is reset and SPL and
board_init_rrecords show correct elapsed time.=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 164,437 164,437 SPL 375,023 210,586 end phase 7,437 7,437 board_init_f 424,390 416,953 board_init_r 473,515 49,125 eth_common_init 477,402 3,887 main_loop 477,571 169 cli_loopAccumulated time:
14,734 dm_spl 315,646 dm_f 7,339 of_live 14,977 dm_rFor the tested board external TPL and BROM take ~164 ms to initialize DRAM and load SPL, SPL take ~210ms to load images from FIT and U-Boot proper take ~477ms to reach cli prompt.
Signed-off-by Jonas Karlman <[email protected]>--- common/bootstage.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)Quoted 6 lines
Quoted 1 line
To put this more strongly, this breaks the timing output on other boards.
Please elaborate, I cannot understand why not sorting would break output on other boards, are other boards adding records out of order?
Do you have an example of how this change break other boards?
Oh sure. The original reason was that the array was ordered by ID. That was changed in [1], which I forgot about.
Quoted 7 lines
The other bit is for coreboot (or potentially other prior phases) we receive timestamps as a block and add them into U-Boot. We want these to appear in the correct order with respect to the ones already adding to U-Boot.
Sadly coreboot uses its own toolchain and I seem to have deleted it, but once it finishes building the world I will give it a try.
OK I gave up and just built one toolchain instead of every one known to man, which is the default.
The code has seriously rotted and is a somewhat fascinating window into what U-Boot was like 12 years ago...I will send a little series.
Anyway, the start-romstage and device-xxx things below come from coreboot.
=> bootstage repo
Timer summary in microseconds (20 records):
Mark Elapsed Stage
0 0 reset
25,322 25,322 board_init_f
32,175 6,853 start-romstage
117,158 84,983 board_init_r
208,409 91,251 device-initialize
293,286 84,877 device-done
547,606 254,320 selfboot-jump
621,050 73,444 eth_common_init
699,570 78,520 eth_initialize
744,210 44,640 main_loop
761,030 16,820 usb_start
3,988,265 3,227,235 netboot_common
3,988,330 65 eth_start
6,316,294 2,327,964 bootp_start
12,286,385 5,970,091 bootp_stop
12,297,353 10,968 netboot_common
12,300,596 3,243 netboot_common
28,327,263 16,026,667 cli_loopAccumulated time:
8,600 dm_r
8,659 dm_fWithout the sort, I see:
Timer summary in microseconds (18 records):
Mark Elapsed Stage
0 0 reset
36,582 36,582 board_init_f
125,775 89,193 board_init_r
642,577 516,802 eth_common_init
721,321 78,744 eth_initialize
31,357429,427,733,2 start-romstage
247,602 216,245 device-initialize
333,879 86,277 device-done
596,043 262,164 selfboot-jump
765,897 169,854 main_loop
783,397 17,500 usb_start
4,043,423 3,260,026 netboot_common
4,043,489 66 eth_start
6,371,702 2,328,213 bootp_start
11,609,247 5,237,545 netboot_common
15,476,116 3,866,869 cli_loopAccumulated time:
8,639 dm_f
8,923 dm_rQuoted 10 lines
Quoted 8 lines
Quoted 5 lines
At the very least, this should be
Kconfigoption just enabled for boards with a broken ATF.I can add a
Kconfigoption, but would like to understand why other/any board would need to sort the records in the first place.OK.
Regards,
Simon
Quoted 2 lines
[1]
03ecac31498bootstage: Userec_countas the array index
Re: [PATCH 2/2] bootstage: Do not sort records
Hi Simon,
Quoted 106 lines · Simon Glass
Hi again Jonas,
Quoted 95 lines · Simon Glass
Hi Jonas,
Quoted 12 lines · Jonas Karlman
Hi Simon,
Quoted 8 lines · Simon Glass
Hi Jonas,
Quoted 5 lines · Simon Glass
Hi Jonas,
Quoted 1 line · Jonas Karlman
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
The timer counter on Rockchip SoCs may be reset in TF-A, this may cause the bootstage records to be printed out of order and with an incorrect elapsed time.
Eek can we just fix TF-A? That seems like a bug.
TF-A is vendor blob, so we have no control of it.
Can we replace it, or is it one of the ones that has not been released?
Can you file a bug with the vendor?
Quoted 57 lines
Quoted 57 lines
Quoted 56 lines
Quoted 56 lines
Fix this by not sorting the bootstage records.
Before on a Radxa ZERO 3W (RK3566) board:
=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 7,436 7,436 board_init_f 164,826 157,390 SPL 375,392 210,566 end phase 423,909 48,517 board_init_r 472,973 49,064 eth_common_init 476,848 3,875 main_loop 477,003 155 cli_loopAccumulated time:
7,181 of_live 14,739 dm_spl 15,029 dm_r 315,150 dm_fWith this the records can be printed in chronological order when the counter is reset and SPL and
board_init_rrecords show correct elapsed time.=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 164,437 164,437 SPL 375,023 210,586 end phase 7,437 7,437 board_init_f 424,390 416,953 board_init_r 473,515 49,125 eth_common_init 477,402 3,887 main_loop 477,571 169 cli_loopAccumulated time:
14,734 dm_spl 315,646 dm_f 7,339 of_live 14,977 dm_rFor the tested board external TPL and BROM take ~164 ms to initialize DRAM and load SPL, SPL take ~210ms to load images from FIT and U-Boot proper take ~477ms to reach cli prompt.
Signed-off-by Jonas Karlman <[email protected]>--- common/bootstage.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)Quoted 6 lines
Quoted 1 line
To put this more strongly, this breaks the timing output on other boards.
Please elaborate, I cannot understand why not sorting would break output on other boards, are other boards adding records out of order?
Do you have an example of how this change break other boards?
Oh sure. The original reason was that the array was ordered by ID. That was changed in [1], which I forgot about.
Quoted 7 lines
The other bit is for coreboot (or potentially other prior phases) we receive timestamps as a block and add them into U-Boot. We want these to appear in the correct order with respect to the ones already adding to U-Boot.
Sadly coreboot uses its own toolchain and I seem to have deleted it, but once it finishes building the world I will give it a try.
Quoted 7 lines
OK I gave up and just built one toolchain instead of every one known to man, which is the default.
The code has seriously rotted and is a somewhat fascinating window into what U-Boot was like 12 years ago...I will send a little series.
Anyway, the start-romstage and device-xxx things below come from coreboot.
Interesting, so coreboot is not running before U-Boot on this platform, is it running in parallel with U-Boot?
How is timestamp_add_to_bootstage() called in U-Boot, I cannot see any call to it, what am I missing?
Guessing the records need to be sorted after/during that function call?
Also look like the sorting is done in bootstage_report() so the report is possible logged to FDT with an unsorted order.
Let me know what you want me to do with this, I will drop this for now and just keep carry this patch in my local tree.
Regards,
Jonas
Quoted 68 lines
=> bootstage repo
Timer summary in microseconds (20 records):Mark Elapsed Stage 0 0 reset 25,322 25,322 board_init_f 32,175 6,853 start-romstage 117,158 84,983 board_init_r 208,409 91,251 device-initialize 293,286 84,877 device-done 547,606 254,320 selfboot-jump 621,050 73,444 eth_common_init 699,570 78,520 eth_initialize 744,210 44,640 main_loop 761,030 16,820 usb_start 3,988,265 3,227,235 netboot_common 3,988,330 65 eth_start 6,316,294 2,327,964 bootp_start 12,286,385 5,970,091 bootp_stop 12,297,353 10,968 netboot_common 12,300,596 3,243 netboot_common 28,327,263 16,026,667 cli_loopAccumulated time:
8,600 dm_r 8,659 dm_fWithout the sort, I see:
Timer summary in microseconds (18 records):
Mark Elapsed Stage 0 0 reset 36,582 36,582 board_init_f 125,775 89,193 board_init_r 642,577 516,802 eth_common_init 721,321 78,744 eth_initialize 31,357429,427,733,2 start-romstage 247,602 216,245 device-initialize 333,879 86,277 device-done 596,043 262,164 selfboot-jump 765,897 169,854 main_loop 783,397 17,500 usb_start 4,043,423 3,260,026 netboot_common 4,043,489 66 eth_start 6,371,702 2,328,213 bootp_start 11,609,247 5,237,545 netboot_common 15,476,116 3,866,869 cli_loopAccumulated time:
8,639 dm_f 8,923 dm_rQuoted 10 lines
Quoted 8 lines
Quoted 5 lines
At the very least, this should be
Kconfigoption just enabled for boards with a broken ATF.I can add a
Kconfigoption, but would like to understand why other/any board would need to sort the records in the first place.OK.
Regards,
SimonQuoted 2 lines
[1]
03ecac31498bootstage: Userec_countas the array index
Re: [PATCH 2/2] bootstage: Do not sort records
Hi Jonas,
Quoted 113 lines · Jonas Karlman
Hi Simon,
Quoted 109 lines · Simon Glass
Hi again Jonas,
Quoted 95 lines · Simon Glass
Hi Jonas,
Quoted 12 lines · Jonas Karlman
Hi Simon,
Quoted 8 lines · Simon Glass
Hi Jonas,
Quoted 5 lines · Simon Glass
Hi Jonas,
Quoted 1 line · Jonas Karlman
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
The timer counter on Rockchip SoCs may be reset in TF-A, this may cause the bootstage records to be printed out of order and with an incorrect elapsed time.
Eek can we just fix TF-A? That seems like a bug.
TF-A is vendor blob, so we have no control of it.
Can we replace it, or is it one of the ones that has not been released?
Can you file a bug with the vendor?
Quoted 57 lines
Quoted 57 lines
Quoted 56 lines
Quoted 56 lines
Fix this by not sorting the bootstage records.
Before on a Radxa ZERO 3W (RK3566) board:
=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 7,436 7,436 board_init_f 164,826 157,390 SPL 375,392 210,566 end phase 423,909 48,517 board_init_r 472,973 49,064 eth_common_init 476,848 3,875 main_loop 477,003 155 cli_loopAccumulated time:
7,181 of_live 14,739 dm_spl 15,029 dm_r 315,150 dm_fWith this the records can be printed in chronological order when the counter is reset and SPL and
board_init_rrecords show correct elapsed time.=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 164,437 164,437 SPL 375,023 210,586 end phase 7,437 7,437 board_init_f 424,390 416,953 board_init_r 473,515 49,125 eth_common_init 477,402 3,887 main_loop 477,571 169 cli_loopAccumulated time:
14,734 dm_spl 315,646 dm_f 7,339 of_live 14,977 dm_rFor the tested board external TPL and BROM take ~164 ms to initialize DRAM and load SPL, SPL take ~210ms to load images from FIT and U-Boot proper take ~477ms to reach cli prompt.
Signed-off-by Jonas Karlman <[email protected]>--- common/bootstage.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)Quoted 6 lines
Quoted 1 line
To put this more strongly, this breaks the timing output on other boards.
Please elaborate, I cannot understand why not sorting would break output on other boards, are other boards adding records out of order?
Do you have an example of how this change break other boards?
Oh sure. The original reason was that the array was ordered by ID. That was changed in [1], which I forgot about.
Quoted 7 lines
The other bit is for coreboot (or potentially other prior phases) we receive timestamps as a block and add them into U-Boot. We want these to appear in the correct order with respect to the ones already adding to U-Boot.
Sadly coreboot uses its own toolchain and I seem to have deleted it, but once it finishes building the world I will give it a try.
OK I gave up and just built one toolchain instead of every one known to man, which is the default.
Quoted 7 lines
Quoted 4 lines
The code has seriously rotted and is a somewhat fascinating window into what U-Boot was like 12 years ago...I will send a little series.
Anyway, the start-romstage and device-xxx things below come from coreboot.
Interesting, so coreboot is not running before U-Boot on this platform, is it running in parallel with U-Boot?
It runs before, but U-Boot picks up the timestamps once it is ready to do so, i.e. after bootstage is inited.
Quoted 3 lines
How is
timestamp_add_to_bootstage()called in U-Boot, I cannot see any call to it, what am I missing?
Yes, it was deleted. I just sent a series to bring i tback.
Quoted 5 lines
Guessing the records need to be sorted after/during that function call?
Also look like the sorting is done in
bootstage_report()so the report is possible logged to FDT with an unsorted order.
Yes, although that shouldn't really matter. Perhaps it is worth fixing?
Quoted 3 lines
Let me know what you want me to do with this, I will drop this for now and just keep carry this patch in my local tree.
Hmmm step one is to complain about the bug. If it exists in ATF upstream, then it should be fixed. I can have a crack at fixing it if so. If it is a vendor binary, then they should fix it.
Step 2, I suggest a Kconfig option enabled for these boards, so it is clear it is a workaround.
Regards,
Simon
Quoted 69 lines
Quoted 68 lines
=> bootstage repo
Timer summary in microseconds (20 records):Mark Elapsed Stage 0 0 reset 25,322 25,322 board_init_f 32,175 6,853 start-romstage 117,158 84,983 board_init_r 208,409 91,251 device-initialize 293,286 84,877 device-done 547,606 254,320 selfboot-jump 621,050 73,444 eth_common_init 699,570 78,520 eth_initialize 744,210 44,640 main_loop 761,030 16,820 usb_start 3,988,265 3,227,235 netboot_common 3,988,330 65 eth_start 6,316,294 2,327,964 bootp_start 12,286,385 5,970,091 bootp_stop 12,297,353 10,968 netboot_common 12,300,596 3,243 netboot_common 28,327,263 16,026,667 cli_loopAccumulated time:
8,600 dm_r 8,659 dm_fWithout the sort, I see:
Timer summary in microseconds (18 records):
Mark Elapsed Stage 0 0 reset 36,582 36,582 board_init_f 125,775 89,193 board_init_r 642,577 516,802 eth_common_init 721,321 78,744 eth_initialize 31,357429,427,733,2 start-romstage 247,602 216,245 device-initialize 333,879 86,277 device-done 596,043 262,164 selfboot-jump 765,897 169,854 main_loop 783,397 17,500 usb_start 4,043,423 3,260,026 netboot_common 4,043,489 66 eth_start 6,371,702 2,328,213 bootp_start 11,609,247 5,237,545 netboot_common 15,476,116 3,866,869 cli_loopAccumulated time:
8,639 dm_f 8,923 dm_rQuoted 10 lines
Quoted 8 lines
Quoted 5 lines
At the very least, this should be
Kconfigoption just enabled for boards with a broken ATF.I can add a
Kconfigoption, but would like to understand why other/any board would need to sort the records in the first place.OK.
Regards,
SimonQuoted 2 lines
[1]
03ecac31498bootstage: Userec_countas the array index
Re: [PATCH 2/2] bootstage: Do not sort records
Hi Simon,
Quoted 119 lines · Simon Glass
Hi Jonas,
Quoted 116 lines · Jonas Karlman
Hi Simon,
Quoted 112 lines · Simon Glass
Hi again Jonas,
Quoted 95 lines · Simon Glass
Hi Jonas,
Quoted 12 lines · Jonas Karlman
Hi Simon,
Quoted 8 lines · Simon Glass
Hi Jonas,
Quoted 5 lines · Simon Glass
Hi Jonas,
Quoted 1 line · Jonas Karlman
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
The timer counter on Rockchip SoCs may be reset in TF-A, this may cause the bootstage records to be printed out of order and with an incorrect elapsed time.
Eek can we just fix TF-A? That seems like a bug.
TF-A is vendor blob, so we have no control of it.
Can we replace it, or is it one of the ones that has not been released?
Can you file a bug with the vendor?
Quoted 57 lines
Quoted 57 lines
Quoted 56 lines
Quoted 56 lines
Fix this by not sorting the bootstage records.
Before on a Radxa ZERO 3W (RK3566) board:
=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 7,436 7,436 board_init_f 164,826 157,390 SPL 375,392 210,566 end phase 423,909 48,517 board_init_r 472,973 49,064 eth_common_init 476,848 3,875 main_loop 477,003 155 cli_loopAccumulated time:
7,181 of_live 14,739 dm_spl 15,029 dm_r 315,150 dm_fWith this the records can be printed in chronological order when the counter is reset and SPL and
board_init_rrecords show correct elapsed time.=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 164,437 164,437 SPL 375,023 210,586 end phase 7,437 7,437 board_init_f 424,390 416,953 board_init_r 473,515 49,125 eth_common_init 477,402 3,887 main_loop 477,571 169 cli_loopAccumulated time:
14,734 dm_spl 315,646 dm_f 7,339 of_live 14,977 dm_rFor the tested board external TPL and BROM take ~164 ms to initialize DRAM and load SPL, SPL take ~210ms to load images from FIT and U-Boot proper take ~477ms to reach cli prompt.
Signed-off-by Jonas Karlman <[email protected]>--- common/bootstage.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)Quoted 6 lines
Quoted 1 line
To put this more strongly, this breaks the timing output on other boards.
Please elaborate, I cannot understand why not sorting would break output on other boards, are other boards adding records out of order?
Do you have an example of how this change break other boards?
Oh sure. The original reason was that the array was ordered by ID. That was changed in [1], which I forgot about.
Quoted 7 lines
The other bit is for coreboot (or potentially other prior phases) we receive timestamps as a block and add them into U-Boot. We want these to appear in the correct order with respect to the ones already adding to U-Boot.
Sadly coreboot uses its own toolchain and I seem to have deleted it, but once it finishes building the world I will give it a try.
OK I gave up and just built one toolchain instead of every one known to man, which is the default.
The code has seriously rotted and is a somewhat fascinating window into what U-Boot was like 12 years ago...I will send a little series.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Anyway, the start-romstage and device-xxx things below come from coreboot.
Interesting, so coreboot is not running before U-Boot on this platform, is it running in parallel with U-Boot?
It runs before, but U-Boot picks up the timestamps once it is ready to do so, i.e. after bootstage is inited.
I just tested using coreboot.rom picked from the docker image trini/u-boot-gitlab-ci-runner:jammy-20240808-21Aug2024 and using qemu:
make coreboot_defconfig all
cbfstool coreboot.rom remove -n fallback/payload
cbfstool coreboot.rom add-flat-binary -f u-boot.bin -n fallback/payload -c LZMA -l 0x1110000 -e 0x1110000 qemu-system-x86_64 -bios coreboot.rom -serial mon:stdio -display none
And can now see that coreboot is run before U-Boot, it also shows that coreboot or U-Boot also reset or use different counter, so the records added from coreboot should also not be sorted among the U-Boot bootstage records.
[DEBUG] BS: BS_PAYLOAD_LOAD run times (exec / console): 210 / 4 ms [DEBUG] Jumping to boot code at 0x01110000(0x07f96000)
bootstage_add_record: name=reset mark=0
bootstage_add_record: name=board_init_f mark=32182
U-Boot 2024.10-rc3-00061-gc6b667a71fcc-dirty (Aug 29 2024 - 00:11:45 +0000)
CPU: x86_64, vendor AMD, device 663h
DRAM: 127 MiB
bootstage_add_record: name=board_init_r mark=129405
Core: 19 devices, 13 uclasses, devicetree: separate MMC:
Loading Environment from nowhere... OK
Video: No video mode configured in coreboot (err=-6)
Video: No video mode configured in coreboot (err=-6)
Model: Standard PC (i440FX + PIIX, 1996)
Manufacturer: QEMU
Prior-stage version: 4.22.01-2ff2409037 Prior-stage date: 08/21/2024
Net: bootstage_add_record: name=eth_common_init mark=179706
e1000: 52:54:00:12:34:56
bootstage_add_record: name=eth_initialize mark=238014
eth0: e1000#0
bootstage_add_record: name=start-romstage mark=14867
bootstage_add_record: name=device-initialize mark=146126
bootstage_add_record: name=device-done mark=172344
bootstage_add_record: name=selfboot-jump mark=432299
bootstage_add_record: name=u-boot-inited mark=471225 Finalizing coreboot
bootstage_add_record: name=main_loop mark=251887
starting USB...
bootstage_add_record: name=usb_start mark=256733 No USB controllers found
Hit any key to stop autoboot: 0
bootstage_add_record: name=cli_loop mark=260244 => bootstage report
Timer summary in microseconds (15 records):
Mark Elapsed Stage
0 0 reset
32,182 32,182 board_init_f
129,405 97,223 board_init_r
179,706 50,301 eth_common_init
238,014 58,308 eth_initialize
14,867 14,867 start-romstage
146,126 131,259 device-initialize
172,344 26,218 device-done
432,299 259,955 selfboot-jump
471,225 38,926 u-boot-inited
251,887 251,887 main_loop
256,733 4,846 usb_start
260,244 3,511 cli_loopAccumulated time:
9,222 dm_f
9,102 dm_rAbove is without sorting and the records added from coreboot is imported after eth_initialize, however all the coreboot records should have happened before the board_init_f record.
=> bootstage report
Timer summary in microseconds (15 records):
Mark Elapsed Stage
0 0 reset
15,918 15,918 start-romstage
145,553 129,635 device-initialize
172,201 26,648 device-done
423,150 250,949 selfboot-jump
461,836 38,686 u-boot-inited
31,925 31,925 board_init_f
135,370 135,370 board_init_r
181,557 46,187 eth_common_init
235,974 54,417 eth_initialize
247,009 11,035 main_loop
250,549 3,540 usb_start
354,321 103,772 cli_loopAccumulated time:
9,041 dm_f
9,354 dm_rSomething like above would more correctly show all steps taken in chronological order and with correct elapsed time from prior mark.
Quoted 6 lines
Quoted 4 lines
How is
timestamp_add_to_bootstage()called in U-Boot, I cannot see any call to it, what am I missing?Yes, it was deleted. I just sent a series to bring i tback.
Thanks, tested with your series.
Quoted 8 lines
Quoted 6 lines
Guessing the records need to be sorted after/during that function call?
Also look like the sorting is done in
bootstage_report()so the report is possible logged to FDT with an unsorted order.Yes, although that shouldn't really matter. Perhaps it is worth fixing?
After testing with coreboot I still think removing the sorting as done in this patch is still correct. However the coreboot records should probably be imported into correct position of the bootstage records table.
Quoted 8 lines
Quoted 4 lines
Let me know what you want me to do with this, I will drop this for now and just keep carry this patch in my local tree.
Hmmm step one is to complain about the bug. If it exists in ATF upstream, then it should be fixed. I can have a crack at fixing it if so. If it is a vendor binary, then they should fix it.
Not even sure how to go about to report such bug to rockchip ;-)
Have also not tested with the recently merged rk35xx support in upstream TF-A so not sure it is also an issue with upstream.
Quoted 3 lines
Step 2, I suggest a
Kconfigoption enabled for these boards, so it is clear it is a workaround.
I would disagree, seeing how the coreboot records is also affected and incorrectly intermixed with U-Boot records, doing any type of sorting should be avoided, and instead the intermixed importing of the coreboot records should probably be fixed.
Regards,
Jonas
Quoted 74 lines
Regards,
SimonQuoted 69 lines
Quoted 68 lines
=> bootstage repo
Timer summary in microseconds (20 records):Mark Elapsed Stage 0 0 reset 25,322 25,322 board_init_f 32,175 6,853 start-romstage 117,158 84,983 board_init_r 208,409 91,251 device-initialize 293,286 84,877 device-done 547,606 254,320 selfboot-jump 621,050 73,444 eth_common_init 699,570 78,520 eth_initialize 744,210 44,640 main_loop 761,030 16,820 usb_start 3,988,265 3,227,235 netboot_common 3,988,330 65 eth_start 6,316,294 2,327,964 bootp_start 12,286,385 5,970,091 bootp_stop 12,297,353 10,968 netboot_common 12,300,596 3,243 netboot_common 28,327,263 16,026,667 cli_loopAccumulated time:
8,600 dm_r 8,659 dm_fWithout the sort, I see:
Timer summary in microseconds (18 records):
Mark Elapsed Stage 0 0 reset 36,582 36,582 board_init_f 125,775 89,193 board_init_r 642,577 516,802 eth_common_init 721,321 78,744 eth_initialize 31,357429,427,733,2 start-romstage 247,602 216,245 device-initialize 333,879 86,277 device-done 596,043 262,164 selfboot-jump 765,897 169,854 main_loop 783,397 17,500 usb_start 4,043,423 3,260,026 netboot_common 4,043,489 66 eth_start 6,371,702 2,328,213 bootp_start 11,609,247 5,237,545 netboot_common 15,476,116 3,866,869 cli_loopAccumulated time:
8,639 dm_f 8,923 dm_rQuoted 10 lines
Quoted 8 lines
Quoted 5 lines
At the very least, this should be
Kconfigoption just enabled for boards with a broken ATF.I can add a
Kconfigoption, but would like to understand why other/any board would need to sort the records in the first place.OK.
Regards,
SimonQuoted 2 lines
[1]
03ecac31498bootstage: Userec_countas the array index
Re: [PATCH 2/2] bootstage: Do not sort records
Hi Jonas,
Quoted 198 lines · Jonas Karlman
Hi Simon,
Quoted 119 lines · Simon Glass
Hi Jonas,
Quoted 116 lines · Jonas Karlman
Hi Simon,
Quoted 112 lines · Simon Glass
Hi again Jonas,
Quoted 95 lines · Simon Glass
Hi Jonas,
Quoted 12 lines · Jonas Karlman
Hi Simon,
Quoted 8 lines · Simon Glass
Hi Jonas,
Quoted 5 lines · Simon Glass
Hi Jonas,
Quoted 1 line · Jonas Karlman
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
The timer counter on Rockchip SoCs may be reset in TF-A, this may cause the bootstage records to be printed out of order and with an incorrect elapsed time.
Eek can we just fix TF-A? That seems like a bug.
TF-A is vendor blob, so we have no control of it.
Can we replace it, or is it one of the ones that has not been released?
Can you file a bug with the vendor?
Quoted 57 lines
Quoted 57 lines
Quoted 56 lines
Quoted 56 lines
Fix this by not sorting the bootstage records.
Before on a Radxa ZERO 3W (RK3566) board:
=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 7,436 7,436 board_init_f 164,826 157,390 SPL 375,392 210,566 end phase 423,909 48,517 board_init_r 472,973 49,064 eth_common_init 476,848 3,875 main_loop 477,003 155 cli_loopAccumulated time:
7,181 of_live 14,739 dm_spl 15,029 dm_r 315,150 dm_fWith this the records can be printed in chronological order when the counter is reset and SPL and
board_init_rrecords show correct elapsed time.=> bootstage report
Timer summary in microseconds (12 records):Mark Elapsed Stage 0 0 reset 164,437 164,437 SPL 375,023 210,586 end phase 7,437 7,437 board_init_f 424,390 416,953 board_init_r 473,515 49,125 eth_common_init 477,402 3,887 main_loop 477,571 169 cli_loopAccumulated time:
14,734 dm_spl 315,646 dm_f 7,339 of_live 14,977 dm_rFor the tested board external TPL and BROM take ~164 ms to initialize DRAM and load SPL, SPL take ~210ms to load images from FIT and U-Boot proper take ~477ms to reach cli prompt.
Signed-off-by Jonas Karlman <[email protected]>--- common/bootstage.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)Quoted 6 lines
Quoted 1 line
To put this more strongly, this breaks the timing output on other boards.
Please elaborate, I cannot understand why not sorting would break output on other boards, are other boards adding records out of order?
Do you have an example of how this change break other boards?
Oh sure. The original reason was that the array was ordered by ID. That was changed in [1], which I forgot about.
Quoted 7 lines
The other bit is for coreboot (or potentially other prior phases) we receive timestamps as a block and add them into U-Boot. We want these to appear in the correct order with respect to the ones already adding to U-Boot.
Sadly coreboot uses its own toolchain and I seem to have deleted it, but once it finishes building the world I will give it a try.
OK I gave up and just built one toolchain instead of every one known to man, which is the default.
The code has seriously rotted and is a somewhat fascinating window into what U-Boot was like 12 years ago...I will send a little series.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Anyway, the start-romstage and device-xxx things below come from coreboot.
Interesting, so coreboot is not running before U-Boot on this platform, is it running in parallel with U-Boot?
It runs before, but U-Boot picks up the timestamps once it is ready to do so, i.e. after bootstage is inited.
I just tested using coreboot.rom picked from the docker image trini/u-boot-gitlab-ci-runner:jammy-20240808-21Aug2024 and using qemu:
make
coreboot_defconfigall
cbfstool coreboot.rom remove -n fallback/payload
cbfstool coreboot.rom add-flat-binary -f u-boot.bin -n fallback/payload -c LZMA -l0x1110000-e0x1110000qemu-system-x86_64 -bios coreboot.rom -serial mon:stdio -display noneAnd can now see that coreboot is run before U-Boot, it also shows that coreboot or U-Boot also reset or use different counter, so the records added from coreboot should also not be sorted among the U-Boot bootstage records.
[DEBUG] BS:
BS_PAYLOAD_LOADrun times (exec / console): 210 / 4 ms [DEBUG] Jumping to boot code at 0x01110000(0x07f96000)
bootstage_add_record: name=reset mark=0
bootstage_add_record: name=board_init_f mark=32182U-Boot 2024.10-rc3-00061-gc6b667a71fcc-dirty (Aug 29 2024 - 00:11:45 +0000)
CPU:
x86_64, vendor AMD, device 663h
DRAM: 127 MiB
bootstage_add_record: name=board_init_r mark=129405
Core: 19 devices, 13 uclasses, devicetree: separate MMC:
Loading Environment from nowhere... OK
Video: No video mode configured in coreboot (err=-6)
Video: No video mode configured in coreboot (err=-6)
Model: Standard PC (i440FX + PIIX, 1996)
Manufacturer: QEMU
Prior-stage version: 4.22.01-2ff2409037 Prior-stage date: 08/21/2024
Net:bootstage_add_record: name=eth_common_init mark=179706
e1000: 52:54:00:12:34:56
bootstage_add_record: name=eth_initialize mark=238014
eth0: e1000#0
bootstage_add_record: name=start-romstage mark=14867
bootstage_add_record: name=device-initialize mark=146126
bootstage_add_record: name=device-done mark=172344
bootstage_add_record: name=selfboot-jump mark=432299
bootstage_add_record: name=u-boot-inited mark=471225 Finalizing coreboot
bootstage_add_record: name=main_loop mark=251887starting USB...
bootstage_add_record: name=usb_start mark=256733 No USB controllers found
Hit any key to stop autoboot: 0
bootstage_add_record: name=cli_loop mark=260244 => bootstage report
Timer summary in microseconds (15 records):Mark Elapsed Stage 0 0 reset 32,182 32,182 board_init_f 129,405 97,223 board_init_r 179,706 50,301 eth_common_init 238,014 58,308 eth_initialize 14,867 14,867 start-romstage 146,126 131,259 device-initialize 172,344 26,218 device-done 432,299 259,955 selfboot-jump 471,225 38,926 u-boot-inited 251,887 251,887 main_loop 256,733 4,846 usb_start 260,244 3,511 cli_loop
Quoted 8 lines
Accumulated time:
9,222 dm_f 9,102 dm_rAbove is without sorting and the records added from coreboot is imported after
eth_initialize, however all the coreboot records should have happened before theboard_init_frecord.
Yes, indeed. I will need to have another look at this.
Quoted 43 lines
=> bootstage report
Timer summary in microseconds (15 records):Mark Elapsed Stage 0 0 reset 15,918 15,918 start-romstage 145,553 129,635 device-initialize 172,201 26,648 device-done 423,150 250,949 selfboot-jump 461,836 38,686 u-boot-inited 31,925 31,925 board_init_f 135,370 135,370 board_init_r 181,557 46,187 eth_common_init 235,974 54,417 eth_initialize 247,009 11,035 main_loop 250,549 3,540 usb_start 354,321 103,772 cli_loopAccumulated time:
9,041 dm_f 9,354 dm_rSomething like above would more correctly show all steps taken in chronological order and with correct elapsed time from prior mark.
Quoted 6 lines
Quoted 4 lines
How is
timestamp_add_to_bootstage()called in U-Boot, I cannot see any call to it, what am I missing?Yes, it was deleted. I just sent a series to bring i tback.
Thanks, tested with your series.
Quoted 7 lines
Quoted 6 lines
Guessing the records need to be sorted after/during that function call?
Also look like the sorting is done in
bootstage_report()so the report is possible logged to FDT with an unsorted order.
Quoted 6 lines
Quoted 1 line
Yes, although that shouldn't really matter. Perhaps it is worth fixing?
After testing with coreboot I still think removing the sorting as done in this patch is still correct. However the coreboot records should probably be imported into correct position of the bootstage records table.
Yes, that would work.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Let me know what you want me to do with this, I will drop this for now and just keep carry this patch in my local tree.
Quoted 8 lines
Quoted 3 lines
Hmmm step one is to complain about the bug. If it exists in ATF upstream, then it should be fixed. I can have a crack at fixing it if so. If it is a vendor binary, then they should fix it.
Not even sure how to go about to report such bug to rockchip ;-)
Have also not tested with the recently merged rk35xx support in upstream TF-A so not sure it is also an issue with upstream.
How about emailing Kever Yang about it? I am sure they will be concerned about it too. That is my main concern...dealing with bugs in binary blobs.
Quoted 9 lines
Quoted 4 lines
Step 2, I suggest a
Kconfigoption enabled for these boards, so it is clear it is a workaround.I would disagree, seeing how the coreboot records is also affected and incorrectly intermixed with U-Boot records, doing any type of sorting should be avoided, and instead the intermixed importing of the coreboot records should probably be fixed.
I suspect that we are still missing some code and that the timer really should always increment. I will take another look.
As you say, we can add a new bootstage function to insert records into the correct position, instead of sorting. We don't support multithreaded things at present, so don't have to worry about two programs emitting timings.
Quoted 79 lines
Regards,
JonasQuoted 74 lines
Regards,
SimonQuoted 69 lines
Quoted 68 lines
=> bootstage repo
Timer summary in microseconds (20 records):Mark Elapsed Stage 0 0 reset 25,322 25,322 board_init_f 32,175 6,853 start-romstage 117,158 84,983 board_init_r 208,409 91,251 device-initialize 293,286 84,877 device-done 547,606 254,320 selfboot-jump 621,050 73,444 eth_common_init 699,570 78,520 eth_initialize 744,210 44,640 main_loop 761,030 16,820 usb_start 3,988,265 3,227,235 netboot_common 3,988,330 65 eth_start 6,316,294 2,327,964 bootp_start 12,286,385 5,970,091 bootp_stop 12,297,353 10,968 netboot_common 12,300,596 3,243 netboot_common 28,327,263 16,026,667 cli_loopAccumulated time:
8,600 dm_r 8,659 dm_fWithout the sort, I see:
Timer summary in microseconds (18 records):
Mark Elapsed Stage 0 0 reset 36,582 36,582 board_init_f 125,775 89,193 board_init_r 642,577 516,802 eth_common_init 721,321 78,744 eth_initialize 31,357429,427,733,2 start-romstage 247,602 216,245 device-initialize 333,879 86,277 device-done 596,043 262,164 selfboot-jump 765,897 169,854 main_loop 783,397 17,500 usb_start 4,043,423 3,260,026 netboot_common 4,043,489 66 eth_start 6,371,702 2,328,213 bootp_start 11,609,247 5,237,545 netboot_common 15,476,116 3,866,869 cli_loopAccumulated time:
8,639 dm_f 8,923 dm_rQuoted 10 lines
Quoted 8 lines
Quoted 5 lines
At the very least, this should be
Kconfigoption just enabled for boards with a broken ATF.I can add a
Kconfigoption, but would like to understand why other/any board would need to sort the records in the first place.OK.
Regards,
SimonQuoted 2 lines
[1]
03ecac31498bootstage: Userec_countas the array index
Regards,
Simon