Skip to contentSkip to conversation

Jetson TX2 hangs during dhcp boot

U-Boot Mian Yousaf Kaukab 5 participants 13 messages Jun 14, 2019 – Jun 17, 2019

lore
Share & tools
lore
#1 lore raw
In reply to Mian Yousaf Kaukab
Quoted 11 lines · Mian Yousaf Kaukab

Hi,
I am seeing a hang with dhcp boot on Jetson TX2. Its running firmware from L4T 32.1 with U-Boot from master branch (c2ea87883e) with [1] applied.

I use following command sequence at u-boot prompt to boot:

Tegra186 (P2771-0000-500) # setenv ftdfile tegra186-p2771-0000.dtb Tegra186 (P2771-0000-500) # setenv boot_targets dhcp Tegra186 (P2771-0000-500) # boot

Last thing I see is the following error from TF-A and the board hangs:

What is TF-A?

Quoted 1 line

U-Boot 2019.07-rc4-00137-g6fefd9c475 (Jun 14 2019 - 16:27:34 +0200)

...

Quoted 3 lines

TFTP from server xx.xx.xx.xx; our IP address is xx.xx.xx.xx Filename 'aarch64/grub.efi'.
Load address: 0x80280000

...

Quoted 10 lines

MMC: no card present
Scanning disk sdhci at 3400000.blk...
Disk sdhci at 3400000.blk not ready
Scanning disk sdhci at 3460000.blk...
Found 32 disks
copying carveout for /host1x at 13e00000/display-hub at 15200000/display at 15200000... copying carveout for /host1x at 13e00000/display-hub at 15200000/display at 15210000... copying carveout for /host1x at 13e00000/display-hub at 15200000/display at 15220000...
ERROR: ARI request timed out: req 89 on CPU 4
ASSERT: plat/nvidia/tegra/soc/t186/drivers/mce/ari.c <127> : retries != 0U

That file doesn't exist in U-Boot; I guess it's part of grub.efi? So, this isn't a hang in U-Boot at all. You'd have to get support for that from the authors of Grub I guess.

#0 lore raw

Hi,
I am seeing a hang with dhcp boot on Jetson TX2. Its running firmware from L4T 32.1 with U-Boot from master branch (c2ea87883e) with [1] applied.

I use following command sequence at u-boot prompt to boot:

Tegra186 (P2771-0000-500) # setenv ftdfile tegra186-p2771-0000.dtb Tegra186 (P2771-0000-500) # setenv boot_targets dhcp Tegra186 (P2771-0000-500) # boot

Last thing I see is the following error from TF-A and the board hangs:

ERROR: ARI request timed out: req 89 on CPU 4
ASSERT: plat/nvidia/tegra/soc/t186/drivers/mce/ari.c <127> : retries != 0U

Here is the complete log:

U-Boot 2019.07-rc4-00137-g6fefd9c475 (Jun 14 2019 - 16:27:34 +0200)

TEGRA186

Model: NVIDIA P2771-0000-500
Board: NVIDIA P2771-0000
DRAM:  7.8 GiB
MMC:   sdhci at 3400000: 1, sdhci at 3460000: 0
Loading Environment from MMC... OK
In:    serial
Out:   serial
Err:   serial
Net:   eth0: ethernet at 2490000
Hit any key to stop autoboot:  0 

Tegra186 (P2771-0000-500) # setenv boot_targets dhcp
Tegra186 (P2771-0000-500) # boot
starting USB...
No working controllers found
maximum number of regions parsed, aborting
ethernet at 2490000 Waiting for PHY auto negotiation to complete...... done BOOTP broadcast 1
BOOTP broadcast 2
BOOTP broadcast 3
DHCP client bound to address xx.xx.xx.xx (1259 ms)
Using ethernet at 2490000 device
TFTP from server xx.xx.xx.xx; our IP address is xx.xx.xx.xx Filename 'boot.scr.uimg'.
Load address: 0x82600000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
ethernet at 2490000 Waiting for PHY auto negotiation to complete........ done BOOTP broadcast 1
DHCP client bound to address xx.xx.xx.xx (5 ms)
Using ethernet at 2490000 device
TFTP from server xx.xx.xx.xx; our IP address is xx.xx.xx.xx Filename 'aarch64/grub.efi'.
Load address: 0x80280000
Loading: T #######T ##########################################################

	 #################################################################
	 ##
	 181.6 KiB/s

done
Bytes transferred = 1936896 (1d8e00 hex)
ethernet at 2490000 Waiting for PHY auto negotiation to complete........ done Using ethernet at 2490000 device
TFTP from server xx.xx.xx.xx; our IP address is xx.xx.xx.xx Filename 'dtb/tegra186-p2771-0000.dtb'.
Load address: 0x82400000
Loading: ###

	 2.6 MiB/s

done
Bytes transferred = 30196 (75f4 hex)
MMC: no card present
Scanning disk sdhci at 3400000.blk...
Disk sdhci at 3400000.blk not ready
Scanning disk sdhci at 3460000.blk...
Found 32 disks
copying carveout for /host1x at 13e00000/display-hub at 15200000/display at 15200000... copying carveout for /host1x at 13e00000/display-hub at 15200000/display at 15210000... copying carveout for /host1x at 13e00000/display-hub at 15200000/display at 15220000...
ERROR: ARI request timed out: req 89 on CPU 4
ASSERT: plat/nvidia/tegra/soc/t186/drivers/mce/ari.c <127> : retries != 0U

It turned out that the last call to TF-A was from __asm_invalidate_l3_icache(). Following is the call trace:

efi_load_image()

    efi_load_pe()
        invalidate_icache_all()
            __asm_invalidate_l3_icache() 

__asm_invalidate_l3_icache() has been called a couple of times before this point without any issues.

If I boot the same EFI binary manually with bootefi it works fine. I can see that __asm_invalidate_l3_icache() returns without the hang. Following is the working command sequence:

Tegra186 (P2771-0000-500) # dhcp
Tegra186 (P2771-0000-500) # tftpboot $fdt_addr_r dtb/tegra186-p2771-0000.dtb Tegra186 (P2771-0000-500) # tftpboot $kernel_addr_r aarch64/grub.efi Tegra186 (P2771-0000-500) # bootefi $kernel_addr_r $fdt_addr_r

This issue is 100% reproducible.

Any thoughts on what could be wrong or how to debug it further?

Thanks,
Yousaf

[1]: https://patchwork.ozlabs.org/patch/1115052/

#2 lore raw
In reply to Stephen Warren

Am 14.06.19 um 18:41 schrieb Stephen Warren:

Quoted 8 lines · Mian Yousaf Kaukab
Quoted 7 lines

Hi,
I am seeing a hang with dhcp boot on Jetson TX2. Its running firmware from
L4T 32.1 with U-Boot from master branch (c2ea87883e) with [1] applied.

I use following command sequence at u-boot prompt to boot:

Quoted 7 lines
Quoted 5 lines

Tegra186 (P2771-0000-500) # setenv ftdfile tegra186-p2771-0000.dtb Tegra186 (P2771-0000-500) # setenv boot_targets dhcp Tegra186 (P2771-0000-500) # boot

Last thing I see is the following error from TF-A and the board hangs:

What is TF-A?

The artist formerly known as Arm Trusted Firmware.

Quoted 21 lines
Quoted 1 line

U-Boot 2019.07-rc4-00137-g6fefd9c475 (Jun 14 2019 - 16:27:34 +0200)

...

Quoted 3 lines

TFTP from server xx.xx.xx.xx; our IP address is xx.xx.xx.xx Filename 'aarch64/grub.efi'.
Load address: 0x80280000

...

Quoted 6 lines

MMC: no card present
Scanning disk sdhci at 3400000.blk...
Disk sdhci at 3400000.blk not ready
Scanning disk sdhci at 3460000.blk...
Found 32 disks
copying carveout for

Quoted 8 lines

/host1x at 13e00000/display-hub at 15200000/display at 15200000... copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15210000... copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15220000... ERROR:   ARI request timed out: req 89 on CPU 4
ASSERT: plat/nvidia/tegra/soc/t186/drivers/mce/ari.c <127> : retries != 0U

Quoted 1 line

That file doesn't exist in U-Boot; I guess it's part of grub.efi?

Looks rather like a TF-A path to me, CC'ing Varun.

As Yousaf says above, it's the TF-A you guys ship with latest R32.1.0.

Maybe the tftpboot operation is overwriting memory used by your TF-A, or differing memory usage is uncovering a missing U-Boot EFI reservation?

(I recently wondered whether TX1 may be missing some, as I needed multiple tries to boot the kernel successfully there - TF-A 2.1 didn't work there, so some more investigations TBD. R28.3.0 still wasn't booting U-Boot ~2019.04, I needed R24.2.2 still as you once suggested.)

Regards,
Andreas

Signature · 3 lines
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
#3 lore raw
Quoted 3 lines
Quoted 3 lines

ERROR: ARI request timed out: req 89 on CPU 4
ASSERT: plat/nvidia/tegra/soc/t186/drivers/mce/ari.c <127> : retries != 0U

This time out means that TEGRA_ARI_ROC_FLUSH_CACHE_TRBITS ARI call timed out. Looks like the firmware responsible for handling this ARI call is stuck or overloaded and not servicing any calls.

-----Original Message-----
From: Andreas Färber <[email protected]>
Sent: Friday, June 14, 2019 10:10 AM
To: Stephen Warren <[email protected]>; Mian Yousaf Kaukab <[email protected]>
Cc: u-boot at lists.denx.de; Stephen Warren <[email protected]>; Tom Warren <[email protected]>; Varun Wadekar <[email protected]>
Subject: Re: [U-Boot] Jetson TX2 hangs during dhcp boot

Am 14.06.19 um 18:41 schrieb Stephen Warren:

Quoted 8 lines · Mian Yousaf Kaukab
Quoted 7 lines

Hi,
I am seeing a hang with dhcp boot on Jetson TX2. Its running firmware from L4T 32.1 with U-Boot from master branch (c2ea87883e) with [1] applied.

I use following command sequence at u-boot prompt to boot:

Quoted 7 lines
Quoted 5 lines

Tegra186 (P2771-0000-500) # setenv ftdfile tegra186-p2771-0000.dtb Tegra186 (P2771-0000-500) # setenv boot_targets dhcp Tegra186 (P2771-0000-500) # boot

Last thing I see is the following error from TF-A and the board hangs:

What is TF-A?

The artist formerly known as Arm Trusted Firmware.

Quoted 21 lines
Quoted 1 line

U-Boot 2019.07-rc4-00137-g6fefd9c475 (Jun 14 2019 - 16:27:34 +0200)

...

Quoted 3 lines

TFTP from server xx.xx.xx.xx; our IP address is xx.xx.xx.xx Filename 'aarch64/grub.efi'.
Load address: 0x80280000

...

Quoted 6 lines

MMC: no card present
Scanning disk sdhci at 3400000.blk...
Disk sdhci at 3400000.blk not ready
Scanning disk sdhci at 3460000.blk...
Found 32 disks
copying carveout for

Quoted 8 lines

/host1x at 13e00000/display-hub at 15200000/display at 15200000... copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15210000... copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15220000... ERROR:   ARI request timed out: req 89 on CPU 4
ASSERT: plat/nvidia/tegra/soc/t186/drivers/mce/ari.c <127> : retries != 0U

Quoted 1 line

That file doesn't exist in U-Boot; I guess it's part of grub.efi?

Looks rather like a TF-A path to me, CC'ing Varun.

As Yousaf says above, it's the TF-A you guys ship with latest R32.1.0.

Maybe the tftpboot operation is overwriting memory used by your TF-A, or differing memory usage is uncovering a missing U-Boot EFI reservation?

(I recently wondered whether TX1 may be missing some, as I needed multiple tries to boot the kernel successfully there - TF-A 2.1 didn't work there, so some more investigations TBD. R28.3.0 still wasn't booting U-Boot ~2019.04, I needed R24.2.2 still as you once suggested.)

Regards,
Andreas

--
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 21284 (AG Nürnberg)

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------

#7 lore raw
In reply to Andreas Färber
Quoted 13 lines · Andreas Färber

Am 14.06.19 um 18:41 schrieb Stephen Warren:

Quoted 12 lines · Mian Yousaf Kaukab
Quoted 11 lines

Hi,
I am seeing a hang with dhcp boot on Jetson TX2. Its running firmware from
L4T 32.1 with U-Boot from master branch (c2ea87883e) with [1] applied.

I use following command sequence at u-boot prompt to boot:

Tegra186 (P2771-0000-500) # setenv ftdfile tegra186-p2771-0000.dtb Tegra186 (P2771-0000-500) # setenv boot_targets dhcp Tegra186 (P2771-0000-500) # boot

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Last thing I see is the following error from TF-A and the board hangs:

What is TF-A?

The artist formerly known as Arm Trusted Firmware.

Ah OK. I had only ever heard that called ATF before.

Quoted 21 lines
Quoted 21 lines
Quoted 1 line

U-Boot 2019.07-rc4-00137-g6fefd9c475 (Jun 14 2019 - 16:27:34 +0200)

...

Quoted 3 lines

TFTP from server xx.xx.xx.xx; our IP address is xx.xx.xx.xx Filename 'aarch64/grub.efi'.
Load address: 0x80280000

...

Quoted 14 lines

MMC: no card present
Scanning disk sdhci at 3400000.blk...
Disk sdhci at 3400000.blk not ready
Scanning disk sdhci at 3460000.blk...
Found 32 disks
copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15200000... copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15210000... copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15220000... ERROR:   ARI request timed out: req 89 on CPU 4
ASSERT: plat/nvidia/tegra/soc/t186/drivers/mce/ari.c <127> : retries != 0U

Quoted 3 lines
Quoted 1 line

That file doesn't exist in U-Boot; I guess it's part of grub.efi?

Looks rather like a TF-A path to me, CC'ing Varun.

Ah yes, I do see that path in our ATF code.

Quoted 3 lines

As Yousaf says above, it's the TF-A you guys ship with latest R32.1.0.

Maybe the tftpboot operation is overwriting memory used by your TF-A, or

That's pretty unlikely since ATF code/data is in a protected memory region that non-secure software can't access.

Quoted 1 line

differing memory usage is uncovering a missing U-Boot EFI reservation?

Perhaps. I know almost nothing about U-Boot EFI though.

Quoted 4 lines

(I recently wondered whether TX1 may be missing some, as I needed multiple tries to boot the kernel successfully there - TF-A 2.1 didn't work there, so some more investigations TBD. R28.3.0 still wasn't booting U-Boot ~2019.04, I needed R24.2.2 still as you once suggested.)

Hmm. Are you building your own copy of ATF (you mentioned 2.1; I can't recall what version we ship)?

FWIW, my usptream U-Boot test system uses L4T r28.1 to test Jetson TX1 and TX2. It does test DHCP/TFTP, but not EFI or booting a kernel.

Can you provide complete instructions on how to reproduce this issue (e.g. which components in L4T you replaced, how to build those replacements, how to get/build/use EFI, any config/script/... files you used from U-Boot/EFI to trigger the issue above)?

Your best bet is likely to post all this information to the NVIDIA Jetson forum so our support people can track the process. I don't expect to have much time personally to look at EFI stuff on TX2 right now. Sorry.

#4 lore raw
In reply to Varun Wadekar

Am 14.06.19 um 19:26 schrieb Varun Wadekar:

Quoted 5 lines
Quoted 3 lines
Quoted 3 lines

ERROR: ARI request timed out: req 89 on CPU 4
ASSERT: plat/nvidia/tegra/soc/t186/drivers/mce/ari.c <127> : retries != 0U

This time out means that TEGRA_ARI_ROC_FLUSH_CACHE_TRBITS ARI call timed out. Looks like the firmware responsible for handling this ARI call is stuck or overloaded and not servicing any calls.

Does this ARI firmware sit in the Arm cores' DRAM? If so, can you share which memory region it needs?

Same question for BL31 - is that TZDRAM_BASE? If so, where to find the size? The TF-A docs are not so telling of those layout details.
https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/plat/nvidia-tegra.rst

U-Boot only reserves any memrsv entries of the FDT, not other memory nodes such as within your sysram at 30000000. Those may need C code in U-Boot to reserve them from usage by UEFI apps such as GRUB and Linux.

https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/arch/arm64/boot/dts/nvidia/tegra186.dtsi

Regards,
Andreas

Signature · 3 lines
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
#9 lore raw
In reply to Mian Yousaf Kaukab
Quoted 3 lines · Mian Yousaf Kaukab
Quoted 2 lines

Hmm. Are you building your own copy of ATF (you mentioned 2.1; I can't recall what version we ship)?

No, I am using the one shipped with R32.1.

That did not work at all for me with mainline u-boot (u-boot would hang early), so I went back to some older release (28.?) and all was fine again.

Martin

#11 lore raw
In reply to Mian Yousaf Kaukab

Am 16.06.19 um 21:27 schrieb Mian Yousaf Kaukab:

Quoted 16 lines · Stephen Warren
Quoted 15 lines · Andreas Färber
Quoted 14 lines

Am 14.06.19 um 18:41 schrieb Stephen Warren:

Quoted 13 lines · Mian Yousaf Kaukab
Quoted 12 lines

Hi,
I am seeing a hang with dhcp boot on Jetson TX2. Its running firmware from
L4T 32.1 with U-Boot from master branch (c2ea87883e) with [1] applied.

I use following command sequence at u-boot prompt to boot:

Tegra186 (P2771-0000-500) # setenv ftdfile tegra186-p2771-0000.dtb Tegra186 (P2771-0000-500) # setenv boot_targets dhcp Tegra186 (P2771-0000-500) # boot

Last thing I see is the following error from TF-A and the board hangs:

[...]

Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 1 line

U-Boot 2019.07-rc4-00137-g6fefd9c475 (Jun 14 2019 - 16:27:34 +0200)

...

Quoted 3 lines

TFTP from server xx.xx.xx.xx; our IP address is xx.xx.xx.xx Filename 'aarch64/grub.efi'.
Load address: 0x80280000

...

Quoted 14 lines

MMC: no card present
Scanning disk sdhci at 3400000.blk...
Disk sdhci at 3400000.blk not ready
Scanning disk sdhci at 3460000.blk...
Found 32 disks
copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15200000... copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15210000... copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15220000... ERROR:   ARI request timed out: req 89 on CPU 4
ASSERT: plat/nvidia/tegra/soc/t186/drivers/mce/ari.c <127> : retries != 0U

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 1 line

That file doesn't exist in U-Boot; I guess it's part of grub.efi?

Looks rather like a TF-A path to me, CC'ing Varun.

Ah yes, I do see that path in our ATF code.

Quoted 1 line

As Yousaf says above, it's the TF-A you guys ship with latest R32.1.0.

!!!

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

Maybe the tftpboot operation is overwriting memory used by your TF-A, or

That's pretty unlikely since ATF code/data is in a protected memory region that non-secure software can't access.

Good point - I was playing with the rpi3 TF-A target, which does have everything in unprotected DRAM.

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

differing memory usage is uncovering a missing U-Boot EFI reservation?

Perhaps. I know almost nothing about U-Boot EFI though.

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

(I recently wondered whether TX1 may be missing some, as I needed multiple tries to boot the kernel successfully there - TF-A 2.1 didn't work there, so some more investigations TBD. R28.3.0 still wasn't booting U-Boot ~2019.04, I needed R24.2.2 still as you once suggested.)

Hmm. Are you building your own copy of ATF (you mentioned 2.1; I can't recall what version we ship)?

No, I am using the one shipped with R32.1.

Careful: Yousaf is talking about his TX2 setup, whereas I am talking about my TX1 setup. His answer does not apply to my quoted TX1 section.

Yes, for TX1 I am using an OBS-built upstream TF-A v2.1 / v1.5. Will check Yousaf's LINUX_KERNEL_IMAGE_HEADER patch helps on TX1.

Let's split the two if they're not connected and focus on TX2 here.

Regards,
Andreas

Signature · 3 lines
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
#8 lore raw
In reply to Stephen Warren
Quoted 14 lines · Stephen Warren
Quoted 13 lines · Andreas Färber

Am 14.06.19 um 18:41 schrieb Stephen Warren:

Quoted 12 lines · Mian Yousaf Kaukab
Quoted 11 lines

Hi,
I am seeing a hang with dhcp boot on Jetson TX2. Its running firmware from
L4T 32.1 with U-Boot from master branch (c2ea87883e) with [1] applied.

I use following command sequence at u-boot prompt to boot:

Tegra186 (P2771-0000-500) # setenv ftdfile tegra186-p2771-0000.dtb Tegra186 (P2771-0000-500) # setenv boot_targets dhcp Tegra186 (P2771-0000-500) # boot

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Last thing I see is the following error from TF-A and the board hangs:

What is TF-A?

The artist formerly known as Arm Trusted Firmware.

Ah OK. I had only ever heard that called ATF before.

4def07d5 Update Arm TF references to TF-A

Quoted 39 lines
Quoted 22 lines
Quoted 22 lines
Quoted 2 lines

U-Boot 2019.07-rc4-00137-g6fefd9c475 (Jun 14 2019 - 16:27:34 +0200)

...

Quoted 3 lines

TFTP from server xx.xx.xx.xx; our IP address is xx.xx.xx.xx Filename 'aarch64/grub.efi'.
Load address: 0x80280000

...

Quoted 14 lines

MMC: no card present
Scanning disk sdhci at 3400000.blk...
Disk sdhci at 3400000.blk not ready
Scanning disk sdhci at 3460000.blk...
Found 32 disks
copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15200000... copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15210000... copying carveout for
/host1x at 13e00000/display-hub at 15200000/display at 15220000... ERROR:   ARI request timed out: req 89 on CPU 4
ASSERT: plat/nvidia/tegra/soc/t186/drivers/mce/ari.c <127> : retries != 0U

Quoted 3 lines
Quoted 1 line

That file doesn't exist in U-Boot; I guess it's part of grub.efi?

Looks rather like a TF-A path to me, CC'ing Varun.

Ah yes, I do see that path in our ATF code.

Quoted 3 lines

As Yousaf says above, it's the TF-A you guys ship with latest R32.1.0.

Maybe the tftpboot operation is overwriting memory used by your TF-A, or

That's pretty unlikely since ATF code/data is in a protected memory region that non-secure software can't access.

Quoted 1 line

differing memory usage is uncovering a missing U-Boot EFI reservation?

Perhaps. I know almost nothing about U-Boot EFI though.

Quoted 7 lines
Quoted 4 lines

(I recently wondered whether TX1 may be missing some, as I needed multiple tries to boot the kernel successfully there - TF-A 2.1 didn't work there, so some more investigations TBD. R28.3.0 still wasn't booting U-Boot ~2019.04, I needed R24.2.2 still as you once suggested.)

Hmm. Are you building your own copy of ATF (you mentioned 2.1; I can't recall what version we ship)?

No, I am using the one shipped with R32.1.

Quoted 8 lines

FWIW, my usptream U-Boot test system uses L4T r28.1 to test Jetson TX1 and TX2. It does test DHCP/TFTP, but not EFI or booting a kernel.

Can you provide complete instructions on how to reproduce this issue (e.g. which components in L4T you replaced, how to build those replacements, how to get/build/use EFI, any config/script/... files you used from U-Boot/EFI to trigger the issue above)?

As shown in the u-boot log, I am using dhcp boot.
setenv boot_targets dhcp
boot

See if the instructions on the following link to setup the server works for you:
https://www.suse.com/documentation/suse-best-practices/singlehtml/sbp-multi-pxe-install/sbp-multi-pxe-install.html
and then populate it with Tumbleweed installation files from following:
http://download.opensuse.org/ports/aarch64/tumbleweed/iso/
Grub EFI binary can be found in EFI/BOOT/.

I am only replacing u-boot with the upstream u-boot using following command:
./flash.sh -k kernel jetson-tx2 mmcblk0p1

Alternatively, I can do the testing this week if you provide me debug builds.

Quoted 4 lines

Your best bet is likely to post all this information to the NVIDIA Jetson forum so our support people can track the process. I don't expect to have much time personally to look at EFI stuff on TX2 right now. Sorry.

#5 lore raw
In reply to Varun Wadekar
Quoted 5 lines · Varun Wadekar
Quoted 3 lines
Quoted 3 lines

ERROR: ARI request timed out: req 89 on CPU 4
ASSERT: plat/nvidia/tegra/soc/t186/drivers/mce/ari.c <127> : retries != 0U

This time out means that TEGRA_ARI_ROC_FLUSH_CACHE_TRBITS ARI call timed out. Looks like the firmware responsible for handling this ARI call is stuck or overloaded and not servicing any calls.

Any thoughts on how to debug it?

I couldn't get upstream TF-A to work. Here is how I am building it:
make CROSS_COMPILE=aarch64-linux-gnu- PLAT=tegra TARGET_SOC=t186 DEBUG=1 all

then I build tos.img as following:
./gen_tos_part_img.py bl31.bin JetPack_4.2_Linux_P3310/Linux_for_Tegra/bootloader/tos.img

(try to)flash as following:
./flash.sh -k secure-os jetson-tx2 mmcblk0p1

However, I can't get flashing to work as soon as tos.img from R32.1 is replaced with my build. Is upstream TF-A supposed to work with R32.1?

Alternatively, I can do the testing this week if you provide me any TF-A debug builds.

Thanks,
Yousaf

#12 lore raw

[U-Boot] Jetson TX1 boot problems

In reply to Stephen Warren

Am 14.06.19 um 19:40 schrieb Stephen Warren:

Quoted 8 lines · Andreas Färber
Quoted 4 lines

(I recently wondered whether TX1 may be missing some, as I needed multiple tries to boot the kernel successfully there - TF-A 2.1 didn't work there, so some more investigations TBD. R28.3.0 still wasn't booting U-Boot ~2019.04, I needed R24.2.2 still as you once suggested.)

Hmm. Are you building your own copy of ATF (you mentioned 2.1; I can't recall what version we ship)?

v1.2

Quoted 2 lines

FWIW, my usptream U-Boot test system uses L4T r28.1 to test Jetson TX1 and TX2. It does test DHCP/TFTP, but not EFI or booting a kernel.

So it turned out that what I last tried with R28.3.0 was downstream U-Boot plus upstream TF-A v2.1. I've been digging into this and have a draft fix for upstream TF-A, a BL2 parameter layout change:

https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/1317

With this I can boot into -rc4 on R28.3.0. Haven't re-checked R24 yet.

Some U-Boot side EFI and non-EFI problems (as well as kernel deferred probing problems) remain to be investigated.

Cheers,
Andreas

Signature · 3 lines
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
#10 lore raw
In reply to Martin Husemann
Quoted 8 lines · Martin Husemann
Quoted 3 lines · Mian Yousaf Kaukab
Quoted 2 lines

Hmm. Are you building your own copy of ATF (you mentioned 2.1; I can't recall what version we ship)?

No, I am using the one shipped with R32.1.

That did not work at all for me with mainline u-boot (u-boot would hang early), so I went back to some older release (28.?) and all was fine again.

Have you tested with following patch applied to u-boot?
https://patchwork.ozlabs.org/patch/1115052/

Quoted 2 lines

Martin

BR,
Yousaf

#6 lore raw
In reply to Mian Yousaf Kaukab
Quoted 11 lines · Mian Yousaf Kaukab
Quoted 5 lines · Varun Wadekar
Quoted 3 lines
Quoted 3 lines

ERROR: ARI request timed out: req 89 on CPU 4
ASSERT: plat/nvidia/tegra/soc/t186/drivers/mce/ari.c <127> : retries != 0U

This time out means that TEGRA_ARI_ROC_FLUSH_CACHE_TRBITS ARI call timed out. Looks like the firmware responsible for handling this ARI call is stuck or overloaded and not servicing any calls.

Any thoughts on how to debug it?

I couldn't get upstream TF-A to work. Here is how I am building it:
make CROSS_COMPILE=aarch64-linux-gnu- PLAT=tegra TARGET_SOC=t186 DEBUG=1 all

Quoted 8 lines

then I build tos.img as following:
./gen_tos_part_img.py bl31.bin JetPack_4.2_Linux_P3310/Linux_for_Tegra/bootloader/tos.img

(try to)flash as following:
./flash.sh -k secure-os jetson-tx2 mmcblk0p1

However, I can't get flashing to work as soon as tos.img from R32.1 is replaced with my build. Is upstream TF-A supposed to work with R32.1?

I personally wouldn't /expect/ anything except our downstream versions of ATF and Trusty to work, but they might if you're lucky. I believe we're quite close to (some version of) upstream for ATF so upstream ATF /might/ work (but I don't believe we test it, so no guarantees). For Trusty I'm less sure we're close to any particular upstream version these days, so I wouldn't count on much.